scheduler: call Initialize() and UnInitialize() in action lifecycle#5069
scheduler: call Initialize() and UnInitialize() in action lifecycle#5069aadhil2k4 wants to merge 2 commits intovolcano-sh:masterfrom
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request resolves a critical bug in the scheduler where action lifecycle methods, Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request correctly implements the full lifecycle for scheduler actions by ensuring Initialize() and UnInitialize() are called around Execute(). This is a good fix that aligns the implementation with the Action interface definition. I've added one suggestion to make the implementation slightly more concise and readable by grouping the lifecycle calls within a single anonymous function.
There was a problem hiding this comment.
Pull request overview
Ensures scheduler actions follow their full lifecycle during each scheduling cycle by invoking Initialize(), Execute(), and UnInitialize() in production (not only in unit tests), addressing #5052.
Changes:
- Call
Initialize()before executing each scheduler action. - Ensure
UnInitialize()is invoked after each action execution viadefer.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Thanks for the PR!
Can you align with @devzizu in the original issue or here please? You have completely different lifecycle assumptions than him about the introduction of these functions, and I lean towards his way.
I want to mention that the implementation is not easy on this if we move on to the scheduler's lifetime path. Because you need to account for hot-reload as on config reload pc.actions gets replaced with fresh instances, so you need to prevent the old actions' workers to leak.
pkg/scheduler/scheduler.go
Outdated
| // ensuring Initialize() is called before Execute() and UnInitialize() | ||
| // is deferred for guaranteed cleanup after execution. | ||
| func (pc *Scheduler) executeAction(action framework.Action, ssn *framework.Session) { | ||
| func() { |
There was a problem hiding this comment.
Why do we have this unnecessary anonymous function wrapper here? It doesn't do anything.
pkg/scheduler/scheduler.go
Outdated
| // is deferred for guaranteed cleanup after execution. | ||
| func (pc *Scheduler) executeAction(action framework.Action, ssn *framework.Session) { | ||
| func() { | ||
| action.Initialize() |
There was a problem hiding this comment.
We have a conflicting lifecycle assumptions here. The way devzizu imagined this at L827 is that Initialize() and UnInitialize() will be called once for the scheduler's lifetime (start/shutdown), not per-cycle.
I think I agree with him, since session is not an argument of this function the initial design thought should have been to call it only once. We have OpenSession and CloseSession to adhere for a session level lifecycle.
There was a problem hiding this comment.
Indeed, the absence of session as a parameter to Initialize()/UnInitialize() is a good signal that they were designed for scheduler-lifetime scope, not per-cycle. Per-cycle setup already has its natural place, i.e. Execute(ssn) itself, or OnSessionOpen/OnSessionClose on the plugin side (which do receive the session).
As @hajnalmt is stating, calling them per-cycle would also conflict with the async worker design proposed in #5033, since workers are meant to run for the scheduler's lifetime.
@aadhil2k4 @hajnalmt, I suggest we move the logic to https://github.com/volcano-sh/volcano/blob/master/pkg/scheduler/scheduler.go#L91? There, we can iterate over the actions and call Initialize(), while UnInitialize() can be called by consuming stopCh?
There was a problem hiding this comment.
I donot see any action implement Initialize now.
There was a problem hiding this comment.
@hzxuzhonghu
This is needed for devzizu here currently we don't have a way for the actions to gracefully initilize and unitilize workers spawned by any action. We decided to do this separately because we need to support this I think.
There was a problem hiding this comment.
ACK, i agree with only calling init/unInit once liftime rather than per cycle
|
Hi!! I was occupied last week and couldnt check on this. I'll implement the changes and let you know. |
- Calls Initialize() once on scheduler startup and calls UnInitialize() once on scheduler shutdown via stopCh. - Updated tests to match this new lifecycle change. Signed-off-by: Aadhil Ahamed <aadhil2k4@gmail.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
- Unintialize old actions and Initialize newly loaded actions when the scheduler configuration is reloaded. Signed-off-by: Aadhil Ahamed <aadhil2k4@gmail.com>
| } | ||
|
|
||
| // Initialize all configured actions once. | ||
| func (pc *Scheduler) initActions() { |
There was a problem hiding this comment.
It looks like initActions() is defined but only called in the tests, and could be called in Run() as well right?
Additionally, I believe we should separate concerns between loadSchedulerConf and Run(), but let me know what you think here:
- Keep
loadSchedulerConfdoing only what it does today (parse + swap). Run()can callinitActions()explicitly after the firstloadSchedulerConf().watchSchedulerConf(hot-reload path) explicitly callsUnInitialize()on old actions andInitialize()on new ones before swapping them intopc.actionsunder themutex, sorunOnce()never sees uninitialized actions.
| func (pc *Scheduler) initActions() { | ||
| pc.mutex.Lock() | ||
| actions := pc.actions | ||
| pc.mutex.Unlock() | ||
| for _, action := range actions { | ||
| action.Initialize() | ||
| } | ||
| } | ||
|
|
||
| // Uninitialize all configured actions. | ||
| func (pc *Scheduler) cleanupActions() { |
There was a problem hiding this comment.
move to test file if only needed there and no need to make it as method of Scheduler
| var err error | ||
| if !pc.disableDefaultConf { | ||
| pc.once.Do(func() { | ||
| pc.actions, pc.plugins, pc.configurations, pc.metricsConf, err = UnmarshalSchedulerConf(DefaultSchedulerConf) |
There was a problem hiding this comment.
donot we need to call init for these actions too?
What type of PR is this?
bugfix
What this PR does / why we need it:
Which issue(s) this PR fixes:
Fixes #5052
Special notes for your reviewer:
NONE
Does this PR introduce a user-facing change?