Skip to content

Commit ae6c35d

Browse files
Improve how thread priority is applied
Rebased onto main after lock-free scheduler merge (#193). Introduce typed PriorityLevel, ThreadPriority RAII for set/restore, and integrate priority handling with the lock-free scheduler pools. Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent 30ee1cd commit ae6c35d

35 files changed

Lines changed: 1063 additions & 248 deletions

‎.github/workflows/gcc.yaml‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,9 @@ jobs:
4040
continue-on-error: true
4141

4242
# Use the container for this specific version of gcc
43-
container: ${{ matrix.toolchain.container }}
43+
container:
44+
image: ${{ matrix.toolchain.container }}
45+
options: --privileged
4446

4547
steps:
4648
- name: Checkout Code
@@ -49,7 +51,7 @@ jobs:
4951
# Update for all the actions that need to install stuff
5052
- run: |
5153
apt-get update
52-
apt-get install -y software-properties-common unzip
54+
apt-get install -y software-properties-common unzip sudo
5355
5456
- name: Install GCC
5557
run: |
@@ -90,6 +92,9 @@ jobs:
9092
- name: CCache Stats
9193
run: ccache --show-stats
9294

95+
- name: Add capabilities for scheduler
96+
run: find build/tests -type f -executable -exec sudo setcap cap_sys_nice=eip {} \; || true
97+
9398
- name: Test
9499
timeout-minutes: 2
95100
working-directory: build

‎.github/workflows/sonarcloud.yaml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,9 @@ jobs:
6767
timeout-minutes: 30
6868
run: cmake --build build/ --config Release
6969

70+
- name: Add capabilities for scheduler
71+
run: find build/tests -type f -executable -exec sudo setcap cap_sys_nice=eip {} \; || true
72+
7073
- name: Run tests to generate coverage statistics
7174
timeout-minutes: 10
7275
working-directory: build

‎docs/how-to/extending-dsl.md‎

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -178,17 +178,17 @@ The Fusion Engine walks the inheritance tree and collects all extension points f
178178

179179
## Extension Point Summary
180180

181-
| Point | Purpose | Returns | Fusion Strategy |
182-
| -------------- | ------------------------------------ | ---------- | ------------------- |
183-
| `bind` | Register reaction at creation time | `void` | All called |
184-
| `get` | Retrieve data for callback arguments | Any type | Tuple concatenation |
185-
| `precondition` | Gate whether the task should run | `bool` | Logical AND |
186-
| `pre_run` | Hook before callback execution | `void` | All called |
187-
| `post_run` | Hook after callback execution | `void` | All called |
188-
| `scope` | RAII lock held during execution | RAII type | All held |
189-
| `priority` | Task scheduling priority | `int` | Maximum wins |
190-
| `group` | Concurrency group membership | Set | Union |
191-
| `pool` | Which thread pool to run on | Descriptor | (single value) |
181+
| Point | Purpose | Returns | Fusion Strategy |
182+
| -------------- | ------------------------------------ | --------------- | ------------------- |
183+
| `bind` | Register reaction at creation time | `void` | All called |
184+
| `get` | Retrieve data for callback arguments | Any type | Tuple concatenation |
185+
| `precondition` | Gate whether the task should run | `bool` | Logical AND |
186+
| `pre_run` | Hook before callback execution | `void` | All called |
187+
| `post_run` | Hook after callback execution | `void` | All called |
188+
| `scope` | RAII lock held during execution | RAII type | All held |
189+
| `priority` | Task scheduling priority | `PriorityLevel` | Maximum wins |
190+
| `group` | Concurrency group membership | Set | Union |
191+
| `pool` | Which thread pool to run on | Descriptor | (single value) |
192192

193193
See [Extension Points Reference](../reference/extensions/extension-points.md) and [Fusion Engine](../reference/extensions/fusion-engine.md) for full details.
194194

‎docs/reference/dsl/priority.md‎

Lines changed: 18 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -6,33 +6,42 @@ Sets the scheduling priority for a reaction's tasks in the thread pool.
66

77
```cpp
88
on<Trigger<T>, Priority::REALTIME>()
9+
on<Trigger<T>, Priority::HIGHEST>()
910
on<Trigger<T>, Priority::HIGH>()
1011
on<Trigger<T>, Priority::NORMAL>()
1112
on<Trigger<T>, Priority::LOW>()
13+
on<Trigger<T>, Priority::LOWEST>()
1214
on<Trigger<T>, Priority::IDLE>()
1315
```
1416

1517
## Levels
1618

17-
| Level | Value | Use case |
18-
| -------- | ----- | ------------------------------- |
19-
| REALTIME | 1000 | Safety-critical, hard deadlines |
20-
| HIGH | 750 | Important, time-sensitive |
21-
| NORMAL | 500 | Default for all reactions |
22-
| LOW | 250 | Background processing |
23-
| IDLE | 0 | Only when nothing else to do |
19+
| Level | Use case |
20+
| -------- | ------------------------------- |
21+
| REALTIME | Safety-critical, hard deadlines |
22+
| HIGHEST | Highest non-realtime priority |
23+
| HIGH | Important, time-sensitive |
24+
| NORMAL | Default for all reactions |
25+
| LOW | Background processing |
26+
| LOWEST | Lowest non-idle priority |
27+
| IDLE | Only when nothing else to do |
2428

2529
## Behavior
2630

2731
Higher priority tasks are dequeued before lower priority tasks when a thread becomes available.
2832
Priority affects only queuing order — it does not preempt tasks that are already running.
2933
Scheduling is cooperative.
3034

31-
If no priority is specified, `NORMAL` (500) is used.
35+
If no priority is specified, `NORMAL` is used.
36+
37+
The scheduler maps the seven DSL levels onto five internal buckets.
38+
`LOWEST` and `HIGHEST` share adjacent buckets with `LOW` and `HIGH` respectively.
3239

3340
Priority implements the `priority` extension point.
3441
If multiple DSL words in the same binding set a priority, the maximum value wins.
3542

43+
When a task executes, the worker thread's OS priority is set via `ThreadPriority` RAII for the duration of the callback.
44+
3645
## Example
3746

3847
```cpp
@@ -49,7 +58,7 @@ on<Trigger<Background>, Priority::LOW>().then([](const Background& b) {
4958

5059
- Priority does **not** preempt running tasks.
5160
A low-priority task already executing will not be interrupted by a high-priority task entering the queue.
52-
- Default priority is `NORMAL` (500) when unspecified.
61+
- Default priority is `NORMAL` when unspecified.
5362
- Priority applies per-reaction, not per-reactor.
5463

5564
## See Also

‎docs/reference/extensions/extension-points.md‎

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -48,15 +48,15 @@ Each extension point is documented in detail on its own page:
4848

4949
## Quick Reference
5050

51-
| Point | Signature | Returns | Fusion Strategy |
52-
| -------------- | ------------------------------ | ----------- | --------------------------------- |
53-
| `bind` | `bind<DSL>(reaction, args...)` | void | Arg-distributing (FunctionFusion) |
54-
| `get` | `get<DSL>(task)` | data | Tuple concatenation |
55-
| `precondition` | `precondition<DSL>(task)` | bool | AND (short-circuit) |
56-
| `priority` | `priority<DSL>(task)` | int | Maximum |
57-
| `pool` | `pool<DSL>(task)` | descriptor | Exactly one |
58-
| `group` | `group<DSL>(task)` | set | Set union |
59-
| `run_inline` | `run_inline<DSL>(task)` | Inline enum | Must agree |
60-
| `scope` | `scope<DSL>(task)` | RAII lock | All held |
61-
| `pre_run` | `pre_run<DSL>(task)` | void | Sequential |
62-
| `post_run` | `post_run<DSL>(task)` | void | Sequential |
51+
| Point | Signature | Returns | Fusion Strategy |
52+
| -------------- | ------------------------------ | ------------- | --------------------------------- |
53+
| `bind` | `bind<DSL>(reaction, args...)` | void | Arg-distributing (FunctionFusion) |
54+
| `get` | `get<DSL>(task)` | data | Tuple concatenation |
55+
| `precondition` | `precondition<DSL>(task)` | bool | AND (short-circuit) |
56+
| `priority` | `priority<DSL>(task)` | PriorityLevel | Maximum |
57+
| `pool` | `pool<DSL>(task)` | descriptor | Exactly one |
58+
| `group` | `group<DSL>(task)` | set | Set union |
59+
| `run_inline` | `run_inline<DSL>(task)` | Inline enum | Must agree |
60+
| `scope` | `scope<DSL>(task)` | RAII lock | All held |
61+
| `pre_run` | `pre_run<DSL>(task)` | void | Sequential |
62+
| `post_run` | `post_run<DSL>(task)` | void | Sequential |

‎docs/reference/extensions/points/priority.md‎

Lines changed: 15 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ Returns the priority level for this task in the scheduling queue.
66

77
```cpp
88
template <typename DSL>
9-
static int priority(const threading::ReactionTask& task)
9+
static PriorityLevel priority(const threading::ReactionTask& task)
1010
```
1111
1212
## Details
@@ -15,7 +15,7 @@ static int priority(const threading::ReactionTask& task)
1515
| ----------- | ---------------------------------------------------------------------------- |
1616
| **When** | Task creation (determines queue ordering) |
1717
| **Thread** | The emitter's thread (the thread that called `emit()`) |
18-
| **Returns** | `int` — higher values mean higher priority |
18+
| **Returns** | `PriorityLevel` — higher levels are scheduled before lower ones |
1919
| **Fusion** | Maximum value wins. If multiple words provide priority, the highest is used. |
2020
2121
## Context & Arguments
@@ -28,22 +28,28 @@ The returned value determines where the task sits in the scheduler's priority qu
2828
Higher priority tasks are dequeued and executed before lower priority ones.
2929
3030
Priority does **not** preempt running tasks — it only affects queue ordering.
31+
When a task runs, the OS thread priority is set via `ThreadPriority` RAII for the duration of the callback.
3132
3233
## Example
3334
3435
```cpp
3536
struct HighPriority {
3637
template <typename DSL>
37-
static int priority(const threading::ReactionTask& /*task*/) {
38-
return 750; // Same as Priority::HIGH
38+
static PriorityLevel priority(const threading::ReactionTask& /*task*/) {
39+
return PriorityLevel::HIGH;
3940
}
4041
};
4142
```
4243

4344
## Built-in Words Using priority
4445

45-
- `Priority::REALTIME` — value 1000
46-
- `Priority::HIGH` — value 750
47-
- `Priority::NORMAL` — value 500 (default)
48-
- `Priority::LOW` — value 250
49-
- `Priority::IDLE` — value 0
46+
- `Priority::REALTIME`
47+
- `Priority::HIGHEST`
48+
- `Priority::HIGH`
49+
- `Priority::NORMAL` — default
50+
- `Priority::LOW`
51+
- `Priority::LOWEST`
52+
- `Priority::IDLE`
53+
54+
The scheduler maps these DSL levels onto five internal buckets (`REALTIME`, `HIGH`, `NORMAL`, `LOW`, `IDLE`).
55+
`LOWEST` shares the `LOW` bucket; `HIGHEST` shares the `HIGH` bucket.

‎src/LogLevel.hpp‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -110,7 +110,7 @@ class LogLevel {
110110
*
111111
* @param value The value to construct the LogLevel from
112112
*/
113-
constexpr LogLevel(const Value& value = Value::UNKNOWN) : value(value) {};
113+
constexpr LogLevel(const Value& value = Value::UNKNOWN) noexcept : value(value) {}
114114

115115
/**
116116
* Construct a LogLevel from a string
@@ -168,7 +168,7 @@ class LogLevel {
168168
*
169169
* @return The ostream that was passed in
170170
*/
171-
friend std::ostream& operator<<(std::ostream& os, LogLevel level) {
171+
friend std::ostream& operator<<(std::ostream& os, const LogLevel& level) {
172172
return os << static_cast<std::string>(level);
173173
}
174174

@@ -208,11 +208,11 @@ class LogLevel {
208208
friend bool operator!=(const std::string& lhs, const LogLevel& rhs) { return lhs != static_cast<std::string>(rhs); }
209209
// clang-format on
210210

211-
212211
private:
213212
/// The stored enum value
214213
Value value;
215214
};
215+
216216
} // namespace NUClear
217217

218218
#endif // NUCLEAR_LOGLEVEL_HPP

‎src/PriorityLevel.hpp‎

Lines changed: 157 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,157 @@
1+
/*
2+
* MIT License
3+
*
4+
* Copyright (c) 2024 NUClear Contributors
5+
*
6+
* This file is part of the NUClear codebase.
7+
* See https://github.com/Fastcode/NUClear for further info.
8+
*
9+
* Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
10+
* documentation files (the "Software"), to deal in the Software without restriction, including without limitation the
11+
* rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
12+
* permit persons to whom the Software is furnished to do so, subject to the following conditions:
13+
*
14+
* The above copyright notice and this permission notice shall be included in all copies or substantial portions of the
15+
* Software.
16+
*
17+
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
18+
* WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
19+
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
20+
* OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
21+
*/
22+
23+
#ifndef NUCLEAR_PRIORITY_LEVEL_HPP
24+
#define NUCLEAR_PRIORITY_LEVEL_HPP
25+
26+
#include <cstdint>
27+
#include <ostream>
28+
#include <string>
29+
30+
namespace NUClear {
31+
32+
class PriorityLevel {
33+
public:
34+
enum Value : uint8_t {
35+
UNKNOWN = 0,
36+
IDLE = 1,
37+
LOWEST = 2,
38+
LOW = 3,
39+
NORMAL = 4,
40+
HIGH = 5,
41+
HIGHEST = 6,
42+
REALTIME = 7
43+
};
44+
45+
/**
46+
* Construct a PriorityLevel from a Value
47+
*
48+
* @param value The value to construct the PriorityLevel from
49+
*/
50+
constexpr PriorityLevel(const Value& value = Value::NORMAL) noexcept : value(value) {}
51+
52+
/**
53+
* Construct a PriorityLevel from a string
54+
*
55+
* @param level The string to construct the PriorityLevel from
56+
*/
57+
PriorityLevel(const std::string& level)
58+
: value(level == "IDLE" ? Value::IDLE
59+
: level == "LOWEST" ? Value::LOWEST
60+
: level == "LOW" ? Value::LOW
61+
: level == "NORMAL" ? Value::NORMAL
62+
: level == "HIGH" ? Value::HIGH
63+
: level == "HIGHEST" ? Value::HIGHEST
64+
: level == "REALTIME" ? Value::REALTIME
65+
: Value::NORMAL) {}
66+
67+
/**
68+
* A call operator which will return the value of the PriorityLevel
69+
* This can be useful in situations where the implicit conversion operators are ambiguous.
70+
*
71+
* @return The value of the PriorityLevel
72+
*/
73+
constexpr Value operator()() const {
74+
return value;
75+
}
76+
77+
/**
78+
* A conversion operator which will return the value of the PriorityLevel
79+
*
80+
* @return The value of the PriorityLevel
81+
*/
82+
constexpr operator Value() const {
83+
return value;
84+
}
85+
86+
/**
87+
* A conversion operator which will return the string representation of the PriorityLevel
88+
*
89+
* @return The string representation of the PriorityLevel
90+
*/
91+
operator std::string() const {
92+
return value == Value::IDLE ? "IDLE"
93+
: value == Value::LOWEST ? "LOWEST"
94+
: value == Value::LOW ? "LOW"
95+
: value == Value::NORMAL ? "NORMAL"
96+
: value == Value::HIGH ? "HIGH"
97+
: value == Value::HIGHEST ? "HIGHEST"
98+
: value == Value::REALTIME ? "REALTIME"
99+
: "UNKNOWN";
100+
}
101+
102+
/**
103+
* Stream the PriorityLevel to an ostream, it will output the string representation of the PriorityLevel
104+
*
105+
* @param os The ostream to output to
106+
* @param level The PriorityLevel to output
107+
*
108+
* @return The ostream that was passed in
109+
*/
110+
friend std::ostream& operator<<(std::ostream& os, const PriorityLevel& level) {
111+
return os << static_cast<std::string>(level);
112+
}
113+
114+
// Operators to compare PriorityLevel values and PriorityLevel to Value
115+
// clang-format off
116+
friend constexpr bool operator<(const PriorityLevel& lhs, const PriorityLevel& rhs) { return lhs.value < rhs.value; }
117+
friend constexpr bool operator>(const PriorityLevel& lhs, const PriorityLevel& rhs) { return lhs.value > rhs.value; }
118+
friend constexpr bool operator<=(const PriorityLevel& lhs, const PriorityLevel& rhs) { return lhs.value <= rhs.value; }
119+
friend constexpr bool operator>=(const PriorityLevel& lhs, const PriorityLevel& rhs) { return lhs.value >= rhs.value; }
120+
friend constexpr bool operator==(const PriorityLevel& lhs, const PriorityLevel& rhs) { return lhs.value == rhs.value; }
121+
friend constexpr bool operator!=(const PriorityLevel& lhs, const PriorityLevel& rhs) { return lhs.value != rhs.value; }
122+
123+
friend constexpr bool operator<(const PriorityLevel& lhs, const Value& rhs) { return lhs.value < rhs; }
124+
friend constexpr bool operator>(const PriorityLevel& lhs, const Value& rhs) { return lhs.value > rhs; }
125+
friend constexpr bool operator<=(const PriorityLevel& lhs, const Value& rhs) { return lhs.value <= rhs; }
126+
friend constexpr bool operator>=(const PriorityLevel& lhs, const Value& rhs) { return lhs.value >= rhs; }
127+
friend constexpr bool operator==(const PriorityLevel& lhs, const Value& rhs) { return lhs.value == rhs; }
128+
friend constexpr bool operator!=(const PriorityLevel& lhs, const Value& rhs) { return lhs.value != rhs; }
129+
friend constexpr bool operator<(const Value& lhs, const PriorityLevel& rhs) { return lhs < rhs.value; }
130+
friend constexpr bool operator>(const Value& lhs, const PriorityLevel& rhs) { return lhs > rhs.value; }
131+
friend constexpr bool operator<=(const Value& lhs, const PriorityLevel& rhs) { return lhs <= rhs.value; }
132+
friend constexpr bool operator>=(const Value& lhs, const PriorityLevel& rhs) { return lhs >= rhs.value; }
133+
friend constexpr bool operator==(const Value& lhs, const PriorityLevel& rhs) { return lhs == rhs.value; }
134+
friend constexpr bool operator!=(const Value& lhs, const PriorityLevel& rhs) { return lhs != rhs.value; }
135+
136+
friend bool operator<(const PriorityLevel& lhs, const std::string& rhs) { return static_cast<std::string>(lhs) < rhs; }
137+
friend bool operator>(const PriorityLevel& lhs, const std::string& rhs) { return static_cast<std::string>(lhs) > rhs; }
138+
friend bool operator<=(const PriorityLevel& lhs, const std::string& rhs) { return static_cast<std::string>(lhs) <= rhs; }
139+
friend bool operator>=(const PriorityLevel& lhs, const std::string& rhs) { return static_cast<std::string>(lhs) >= rhs; }
140+
friend bool operator==(const PriorityLevel& lhs, const std::string& rhs) { return static_cast<std::string>(lhs) == rhs; }
141+
friend bool operator!=(const PriorityLevel& lhs, const std::string& rhs) { return static_cast<std::string>(lhs) != rhs; }
142+
friend bool operator<(const std::string& lhs, const PriorityLevel& rhs) { return lhs < static_cast<std::string>(rhs); }
143+
friend bool operator>(const std::string& lhs, const PriorityLevel& rhs) { return lhs > static_cast<std::string>(rhs); }
144+
friend bool operator<=(const std::string& lhs, const PriorityLevel& rhs) { return lhs <= static_cast<std::string>(rhs); }
145+
friend bool operator>=(const std::string& lhs, const PriorityLevel& rhs) { return lhs >= static_cast<std::string>(rhs); }
146+
friend bool operator==(const std::string& lhs, const PriorityLevel& rhs) { return lhs == static_cast<std::string>(rhs); }
147+
friend bool operator!=(const std::string& lhs, const PriorityLevel& rhs) { return lhs != static_cast<std::string>(rhs); }
148+
// clang-format on
149+
150+
private:
151+
/// The stored enum value
152+
Value value;
153+
};
154+
155+
} // namespace NUClear
156+
157+
#endif // NUCLEAR_PRIORITY_LEVEL_HPP

0 commit comments

Comments
 (0)