Problem statement
I have a pipeline that's memory bandwidth bound. Parallelizing it beyond 2 threads is not useful. So, I'd like to pass a hint to the parallel_for() runtime routine that this parallel for should not be worked on by more than 2 threads. This will leave all the other cores available for actual useful work instead of hogging the entire CPU while just waiting for memory fetches to come back.
Proposed solution
I propose we pass an Expr of type Int(32) as optional hint to the parallel() scheduling directive.
I realize that this probably means an API breaking change in the runtime, which is something we want to avoid. Can we think of alternatives? A thread_local stack of hints?
Additional context
Note that I do not want to change my schedule to reduce the length of the parallel for to just two. I want to keep my 100+ iterations on the for loop, but now only two threads are going to work on the work items, in order to minimize threads waiting for other threads to finish their work.
Problem statement
I have a pipeline that's memory bandwidth bound. Parallelizing it beyond 2 threads is not useful. So, I'd like to pass a hint to the parallel_for() runtime routine that this parallel for should not be worked on by more than 2 threads. This will leave all the other cores available for actual useful work instead of hogging the entire CPU while just waiting for memory fetches to come back.
Proposed solution
I propose we pass an
Exprof typeInt(32)as optional hint to theparallel()scheduling directive.I realize that this probably means an API breaking change in the runtime, which is something we want to avoid. Can we think of alternatives? A thread_local stack of hints?
Additional context
Note that I do not want to change my schedule to reduce the length of the parallel for to just two. I want to keep my 100+ iterations on the for loop, but now only two threads are going to work on the work items, in order to minimize threads waiting for other threads to finish their work.