Skip to content

Commit

Permalink
sched_ext: Always call put_prev_task() with scx enabled
Browse files Browse the repository at this point in the history
With the consolidation of put_prev_task/set_next_task(), we are now
skipping the sched_ext ops.stopping/running() transitions when the
previous and next tasks are the same, see commit 436f3ee ("sched:
Combine the last put_prev_task() and the first set_next_task()").

While this optimization makes sense in general, it can negatively impact
performance in some user-space schedulers, that expect to handle such
transitions when tasks exhaust their timeslice (see SCX_OPS_ENQ_LAST).

For example, scx_rustland suffers a significant performance regression
(e.g., gaming benchmarks drop from ~60fps to ~10fps).

To fix this, ensure that put_prev_task()/set_next_task() are never
skipped when the scx scheduling class is enabled, allowing the scx class
to handle such transitions.

This change restores the previous behavior, fixing the performance
regression in scx_rustland.

Link: sched-ext/scx#788
Fixes: 7c65ae8 ("sched_ext: Don't call put_prev_task_scx() before picking the next task")
Signed-off-by: Andrea Righi <[email protected]>
  • Loading branch information
arighi authored and intel-lab-lkp committed Oct 13, 2024
1 parent 7266f0a commit c4ad180
Showing 1 changed file with 15 additions and 15 deletions.
30 changes: 15 additions & 15 deletions kernel/sched/sched.h
Original file line number Diff line number Diff line change
Expand Up @@ -2467,21 +2467,6 @@ __put_prev_set_next_dl_server(struct rq *rq,
rq->dl_server = NULL;
}

static inline void put_prev_set_next_task(struct rq *rq,
struct task_struct *prev,
struct task_struct *next)
{
WARN_ON_ONCE(rq->curr != prev);

__put_prev_set_next_dl_server(rq, prev, next);

if (next == prev)
return;

prev->sched_class->put_prev_task(rq, prev, next);
next->sched_class->set_next_task(rq, next, true);
}

/*
* Helper to define a sched_class instance; each one is placed in a separate
* section which is ordered by the linker script:
Expand Down Expand Up @@ -2520,6 +2505,21 @@ DECLARE_STATIC_KEY_FALSE(__scx_switched_all); /* all fair class tasks on SCX */
#define scx_switched_all() false
#endif /* !CONFIG_SCHED_CLASS_EXT */

static inline void put_prev_set_next_task(struct rq *rq,
struct task_struct *prev,
struct task_struct *next)
{
WARN_ON_ONCE(rq->curr != prev);

__put_prev_set_next_dl_server(rq, prev, next);

if (next == prev && !scx_enabled())
return;

prev->sched_class->put_prev_task(rq, prev, next);
next->sched_class->set_next_task(rq, next, true);
}

/*
* Iterate only active classes. SCX can take over all fair tasks or be
* completely disabled. If the former, skip fair. If the latter, skip SCX.
Expand Down

0 comments on commit c4ad180

Please sign in to comment.