Investigate possible build perf improvements in Copy task #11374
Description
Context
Reported by @davkean:
In 40% of builds, I see something that Eric was looking at; synchronous calls to Copy taking out thread pool threads. The CLR will take 500ms before it adds another thread to replace each one of these (so 5 seconds to replace 10 threads), so this is likely causing thread pool starvation and a non-optimal use of CPU usage while that occurs.
Due to the 'multi-process' nature of msbuild the threatpool starvation might not be such a big concern yet, but blocking the ability to run other waiting build work is of a big concern.
Additionaly there might be additional task in a same bucket - e.g. CopyRefAssembly from roslyn
Suggested approach
- Attempt to perform couple experiments with implanting
TaskHost.Yieldinto the Copy task (similarly to what Zip/Unzip does today) and evaluate the impact on the build - Attempt to see if we can improve the current paralelized copy algorithm (possbily in connection with above)
Goal
Copy is notoriously among the top time consumers within the builds
being able to use the consummed time for additional build tasks might help improve the CPU utilization and throughput of the build