Prevent contention between CancelAllSubmissions and EndBuild

.config/git-merge-flow-config.jsonc

@@ -26,8 +26,12 @@
         "vs17.10": {
             "MergeToBranch": "vs17.11"
         },
-        // MSBuild latest release to main
+        // Automate opening PRs to merge msbuild's vs17.11 (SDK 8.0.4xx) into vs17.12 (SDK 9.0.1xx)
         "vs17.11": {
+            "MergeToBranch": "vs17.12"
+        },
+        // MSBuild latest release to main
+        "vs17.12": {
             "MergeToBranch": "main"
         }
     }

.vsts-dotnet.yml

@@ -34,7 +34,7 @@ variables:
     - name: SourceBranch
       value: ''
   - name: EnableReleaseOneLocBuild
-    value: true # Enable loc for vs17.12
+    value: true # Enable loc for vs17.13
   - name: Codeql.Enabled
     value: true
   - group: DotNet-MSBuild-SDLValidation-Params

NuGet.config

@@ -31,4 +31,10 @@
     <!--  End: Package sources from dotnet-runtime -->
     <!--End: Package sources managed by Dependency Flow automation. Do not edit the sources above.-->
   </disabledPackageSources>
+
+  <auditSources>
+    <clear />
+    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
+  </auditSources>
+
 </configuration>

branding/README.md

@@ -9,8 +9,8 @@ This image is the original graphic used for creating the MSBuild icon.  It was p
 This image is an original graphic created by Immo Landwerth.  It can be used for images where the size is required to be larger than the MSBuild-Icon.png.
 
 ## MSBuild-NuGet-Icon.png ![MSBuild-NuGet-Icon.png](MSBuild-NuGet-Icon.png?raw=true)
-This image is used by the NuGet packages as the ``<iconUrl />``.  Its raw location in this repo is used by the forwarding link [https://go.microsoft.com/fwlink/?linkid=825694](https://go.microsoft.com/fwlink/?linkid=825694).  If it is moved, please update the link.
+This image is used by the NuGet packages as the ``<iconUrl />``.  Its raw location in this repo is used by the forwarding link [https://aka.ms/msbuildicon](https://aka.ms/msbuildicon).  If it is moved, please update the link.
 
 If the below image loads, then the forwarding link is working.
 
-![MSBuild-NuGet-Icon from the forwarding link](https://go.microsoft.com/fwlink/?linkid=825694)
+![MSBuild-NuGet-Icon from the forwarding link](https://aka.ms/msbuildicon)

documentation/release-checklist.md

@@ -12,11 +12,23 @@
 - [ ]  Ping internal "First Responders" Teams channel to get the new channel made available as a promotion target (e.g. dotnet/arcade#12150): {{URL_OF_CHANNEL_PROMOTION_PR}}
 
 ## At release time
-
+Before starting the process:
 - [ ] If the release is being cut more than a few days before the VS-side snap, do these two steps. Otherwise check them off.
   - [ ]  Modify the VS insertion so that it flows from MSBuild `vs{{THIS_RELEASE_VERSION}}` to VS `main` [in the MSBuild-release-branch release definition](https://dev.azure.com/devdiv/DevDiv/_release?definitionId=1319&view=mine&_a=releases). Alternatively, if the release being cut no more than couple of weeks, disable the scheduled releases and create releases from `vs{{THIS_RELEASE_VERSION}}` manually until the VS-side snap: Edit -> Schedule set under Artifacts -> disable toggle
 AND
   - [ ]  Disable automated run of [the MSBuild-main-branch release definition](https://dev.azure.com/devdiv/DevDiv/_release?definitionId=2153&view=mine&_a=releases) (because our {{NEXT_VERSION}} builds don't have a place to go in VS yet)
+
+### Branching from main
+- [ ]  If the new version's branch was created before the Visual Studio fork: fast-forward merge the correct commit (the one that is currently inserted to VS main) to the `vs{{THIS_RELEASE_VERSION}}` branch \
+e.g.: `git push upstream 2e6f2ff7ea311214255b6b2ca5cc0554fba1b345:refs/heads/vs17.10` \
+_(This is for the case where we create the branch too early and want it to be based actually on a different commit. If you waited until a good point in time with `main` in a clean state, just branch off and you are done. The branch should point to a good, recent spot, so the final-branding PR goes in on top of the right set of commits.)_
+- [ ]  Update the branch merge flow in `.config/git-merge-flow-config.jsonc` file to have the currently-in-servicing branches.
+- [ ]  Create {{NEXT_VERSION}} branding PR (in main) including public API baseline package version change: {{URL_OF_NEXT_VERSION_BRANDING_PR}}. 
+  - In the file `eng/Versions.props` Update the `VersionPrefix` to `{{NEXT_VERSION}}` and `PackageValidationBaselineVersion` set to a latest internally available {{THIS_RELEASE_VERSION}} preview version in the [internal dnceng dotnet-tools feed](https://dev.azure.com/dnceng/internal/_artifacts/feed/dotnet-tools-internal). It might be needed to update `CompatibilitySuppressions.xml` files. See [this documentation](https://learn.microsoft.com/en-us/dotnet/fundamentals/apicompat/overview) for more details. You can update `CompatibilitySuppressions.xml` files by running
+`dotnet pack MSBuild.Dev.slnf /p:ApiCompatGenerateSuppressionFile=true`. 
+- [ ]  Merge {{NEXT_VERSION}} branding PR
+
+### Adjust DARC channels and subscriptions
 - [ ]  Remove the `main` to old release channel ({{THIS_RELEASE_VERSION}}) default channel \
 `darc delete-default-channel --repo https://github.com/dotnet/msbuild --branch main --channel "VS {{THIS_RELEASE_VERSION}}"`
 - [ ]  Associate the `main` branch with the next release channel \
@@ -28,24 +40,36 @@ AND
 - [ ]  Ensure that the current release channel `VS {{THIS_RELEASE_VERSION}}` is associated with the correct release branch\
 `darc get-default-channels --source-repo https://github.com/dotnet/msbuild --branch vs{{THIS_RELEASE_VERSION}}` \
 if it is not, `darc add-default-channel  --channel "VS {{THIS_RELEASE_VERSION}}" --branch vs{{THIS_RELEASE_VERSION}} --repo https://github.com/dotnet/msbuild`
-- [ ]  If the branch was created before the fork: fast-forward merge the correct commit (the one that is currently inserted to VS main) to the `vs{{THIS_RELEASE_VERSION}}` branch \
-e.g.: `git push upstream 2e6f2ff7ea311214255b6b2ca5cc0554fba1b345:refs/heads/vs17.10` \
-_(This is for the case where we create the branch too early and want it to be based actually on a different commit. If you waited until a good point in time with `main` in a clean state, just branch off and you are done. The branch should point to a good, recent spot, so the final-branding PR goes in on top of the right set of commits.)_
-- [ ]  Update the branch merge flow in `.config/git-merge-flow-config.jsonc` file to have the currently-in-servicing branches.
+- [ ] Double check subscriptions from our repo `darc get-subscriptions --target-repo dotnet/msbuild` and update subscriptions to `VS{{THIS_RELEASE_VERSION}}` and `main` branches according to [supported versions of VS and SDK](https://learn.microsoft.com/en-us/dotnet/core/porting/versioning-sdk-msbuild-vs#supported-net-versions):
+  - [ ] NuGet client
+    - Based on VS version channel
+    - `darc get-subscriptions --exact --target-repo https://github.com/dotnet/msbuild --source-repo https://github.com/nuget/nuget.client`
+  - [ ] Source Build Packages
+    - Based on .NET version channel
+    - `darc get-subscriptions --exact --target-repo https://github.com/dotnet/msbuild --source-repo https://github.com/dotnet/source-build-reference-packages`
+  - [ ] Roslyn:
+    - Based on VS version channel
+    - `darc get-subscriptions --exact --target-repo https://github.com/dotnet/msbuild --source-repo https://github.com/dotnet/roslyn`
+  - [ ] Arcade:
+    - Based on .NET version channel--does not change every MSBuild release
+    - `darc get-subscriptions --exact --target-repo https://github.com/dotnet/msbuild --source-repo https://github.com/dotnet/arcade`
+
+### Adjust pipelines / releases
 - [ ]  Fix OptProf data flow for the new vs{{THIS_RELEASE_VERSION}} branch
   - [ ] Run the [official build](https://devdiv.visualstudio.com/DevDiv/_build?definitionId=9434) for vs{{THIS_RELEASE_VERSION}} without OptProf (set `SkipApplyOptimizationData` variable in 'Advanced options' section of the 'Run pipeline' menu to `true`) or alternatively with the latest Opt-Prof collected for the main branch (set `Optional OptProfDrop Override` to the drop path of the collected data, which could be found in the logs of the pipeline: Windows_NT -> Build -> search for `OptimizationData`). 
   - [ ] Check that the [OptProf data collection](https://devdiv.visualstudio.com/DevDiv/_build?definitionId=17389) pipeline run is triggered for vs{{THIS_RELEASE_VERSION}}. If not, run manually ('Run pipeline' in upper right)
   - [ ] Run the [official build](https://devdiv.visualstudio.com/DevDiv/_build?definitionId=9434) for vs{{THIS_RELEASE_VERSION}} with no extra customization - OptProf should succeed now
-- [ ]  Create {{NEXT_VERSION}} branding PR (in main) including public API baseline package version change: {{URL_OF_NEXT_VERSION_BRANDING_PR}}. In the file `eng/Versions.props` Update the `VersionPrefix` to `{{NEXT_VERSION}}` and `PackageValidationBaselineVersion` set to a latest internally available {{THIS_RELEASE_VERSION}} preview version in the internal dnceng dotnet-tools feed. It might be needed to update `CompatibilitySuppressions.xml` files. See [this documentation](https://learn.microsoft.com/en-us/dotnet/fundamentals/apicompat/overview) for more details. You can update `CompatibilitySuppressions.xml` files by running
-`dotnet pack MSBuild.Dev.slnf /p:ApiCompatGenerateSuppressionFile=true`. 
+- [ ]  When VS main snaps to {{THIS_RELEASE_VERSION}} and updates its version to {{NEXT_VERSION}}, turn on / modify the VS insertion so that it flows from MSBuild main to VS main.
+  - [ ]  Update the [release-branch insertion release definition](https://dev.azure.com/devdiv/DevDiv/_releaseDefinition?definitionId=2153&_a=definition-variables) to have `InsertTargetBranch` `rel/d{{THIS_RELEASE_VERSION}}`.
+- [ ]  Turn [the release pipeline](https://dev.azure.com/devdiv/DevDiv/_release?definitionId=2153&view=mine&_a=releases) back on.
+
+### Configure localization
 - [ ]  Create {{THIS_RELEASE_VERSION}} localization ticket: https://aka.ms/ceChangeLocConfig (requesting to switch localization from {{PREVIOUS_RELEASE_VERSION}} to {{THIS_RELEASE_VERSION}}): {{URL_OF_LOCALIZATION_TICKET}}
 - [ ]  Enable {{THIS_RELEASE_VERSION}} localization - by setting [`EnableReleaseOneLocBuild`](https://github.com/dotnet/msbuild/blob/vs{{THIS_RELEASE_VERSION}}/.vsts-dotnet.yml) to `true`
 - [ ]  Disable {{PREVIOUS_RELEASE_VERSION}} localization -  by setting [`EnableReleaseOneLocBuild`](https://github.com/dotnet/msbuild/blob/vs{{PREVIOUS_RELEASE_VERSION}}/.vsts-dotnet.yml) to `false`. Update the comment on the same line.
-- [ ]  Merge {{NEXT_VERSION}} branding PR
 - [ ]  Create and merge a PR in main to update a localization version comment in setting [`EnableReleaseOneLocBuild`](https://github.com/dotnet/msbuild/blob/main/.vsts-dotnet.yml) to set up the merge conflict when this line will be updated in the release branch.
-- [ ]  When VS main snaps to {{THIS_RELEASE_VERSION}} and updates its version to {{NEXT_VERSION}}, turn on / modify the VS insertion so that it flows from MSBuild main to VS main.
-- [ ]  Update the [release-branch insertion release definition](https://dev.azure.com/devdiv/DevDiv/_releaseDefinition?definitionId=2153&_a=definition-variables) to have `InsertTargetBranch` `rel/d{{THIS_RELEASE_VERSION}}`.
-- [ ]  Turn [the release pipeline](https://dev.azure.com/devdiv/DevDiv/_release?definitionId=2153&view=mine&_a=releases) back on.
+
+### Final branding
 - [ ]  Prepare final branding PR for `vs{{THIS_RELEASE_VERSION}}`: {{URL_OF_FINAL_BRANDING_PR}}
 - [ ]  Merge final branding to `vs{{THIS_RELEASE_VERSION}}` branch
 - [ ]  Update perfstar MSBuild insertions configuration: [example PR](https://dev.azure.com/devdiv/DevDiv/_git/dotnet-perfstar/pullrequest/522843): {{URL_OF_PERFSTAR_PR}}
@@ -60,6 +84,18 @@ _(This is for the case where we create the branch too early and want it to be ba
 Timing based on the [(Microsoft-internal) release schedule](https://dev.azure.com/devdiv/DevDiv/_wiki/wikis/DevDiv.wiki/10097/Dev17-Release).
 
 - [ ]  Push packages to nuget.org (not currently automated, contact dnceng - search "Publish MSBuild 17.6 to NuGet.org" email subject for template).
+
+  Following packages should be published (`THIS_RELEASE_EXACT_VERSION` is equal to `VersionPrefix` that comes form the eng\Version.props, that were part of the build we are trying to get published):
+    - Microsoft.Build.Utilities.Core.{{THIS_RELEASE_EXACT_VERSION}}.nupkg
+    - Microsoft.Build.{{THIS_RELEASE_EXACT_VERSION}}.nupkg
+    - Microsoft.Build.Framework.{{THIS_RELEASE_EXACT_VERSION}}.nupkg
+    - Microsoft.Build.Runtime.{{THIS_RELEASE_EXACT_VERSION}}.nupkg
+    - Microsoft.Build.Tasks.Core.{{THIS_RELEASE_EXACT_VERSION}}.nupkg
+    - Microsoft.NET.StringTools.{{THIS_RELEASE_EXACT_VERSION}}.nupkg
+    - Microsoft.Build.Templates.{{THIS_RELEASE_EXACT_VERSION}}.nupkg
+
+  **Note:** Microsoft.Build.Conversion.Core and Microsoft.Build.Engine are **not** part of the list. Microsoft.Build.Templates **is** part of the list. Those 3 packages are a difference to the historic publishing list.
+
 - [ ]  Publish docs: submit reference request at https://aka.ms/publishondocs
   - Click on the link labeled *Request – Reference Publishing*
   - You can use existing [ticket](https://dev.azure.com/msft-skilling/Content/_workitems/edit/183613) as a reference

documentation/specs/BuildCheck/BuildCheck-Architecture.md

@@ -106,7 +106,7 @@ How we'll internally handle the distributed model:
 Planned model:
 * Checks factories get registered with the BuildCheck infrastructure (`BuildCheckManager`)
     * For inbox checks - this happens on startup.
-    * For custom checks - this happens on connecting `ILogger` instance in scheduler node receives acquistion event (`BuildCheckAcquisitionEventArgs`). This event is being sent by worker node as soon as it hits a special marker (a magic property function call) during early evaluation. Loading is not processed by worker node as currently we want custom checks only in the main node (as they will be only given data proxied from BuildEventArgs).
+    * For custom checks - this happens on connecting `ILogger` instance in scheduler node receives acquistion event (`BuildCheckAcquisitionEventArgs`). This event is being sent by worker node as soon as it hits a special marker (a magic property function call) during early evaluation. Loading is not processed by worker node as currently we want custom checks only in the main node (as they will be only given data proxied from BuildEventArgs). Loading in worker node in Evaluation context would result in double work as the custom Check needs to be loaded in the main node anyways.
     The `BuildCheckAcquisitionEventArgs` should be sent prior `ProjectEvaluationStartedEventArgs` (buffering will need to take place), or main node will need to replay some initial data after custom check is registered.
 * `BuildCheckManager` receives info about new project starting to be build
     * On scheduler node the information is sourced from `ProjectEvaluationStartedEventArgs`
@@ -136,6 +136,8 @@ Since we are unlikely to enable any checks by default in .NET 9, the focus in th
 
 # Acquisition
 
+(For details on internals of processing acquisition by the infrastructure see [Check Lifecycle](#check-lifecycle))
+
 BuildCheck employs two distinct types of checks: inbox and custom. As a result, the acquisition and distribution processes vary.
 Inbox rules are integrated into the MSBuild repository, while custom checks can be packaged as NuGet packages and detected by MSBuild provided they adhere to a specific structure. 
 

documentation/specs/BuildCheck/Codes.md

@@ -91,7 +91,12 @@ If `BC0202` and [BC0201](#BC0201) are both enabled - then `BC0201` reports only
 
 This check indicates that a property was defined in the observed scope (by default it's the project file only) and it was then not used anywhere in the build.
 
-This is a runtime check, not a static analysis check - so it can have false positives (as property not used in particular build might be needed in a build with different conditions). For this reasons it's currently only suggestion.
+This is a runtime check, not a static analysis check - so it can have false positives - for this reasons it's currently only suggestion.
+
+Common cases of false positives:
+ * Property not used in a particular build might be needed in a build with different conditions or a build of a different target (e.g. `dotnet pack /check` or `dotnet build /t:pack /check` accesses some additional properties as compared to ordinary `dotnet build /check`).
+ * Property accessing is tracked for each project build request. There might be multiple distinct build requests for a project in a single build. Specific case of this is a call to the [MSBuild task](https://learn.microsoft.com/en-us/visualstudio/msbuild/msbuild-task) or [CallTarget task](https://learn.microsoft.com/en-us/visualstudio/msbuild/calltarget-task) that can request a result from a project build, while passing additional or different global properties and/or calling specific target. This happens often as part of common targets - e.g. for [multi-targeted project build parallelization](../../High-level-overview.md#parallelism)
+ * Incremental build might skip execution of some targets, that might have been accessing properties of interest.
 
 <BR/>
 <BR/>

eng/Packages.props

@@ -29,6 +29,7 @@
     <PackageVersion Include="System.Security.Principal.Windows" Version="$(SystemSecurityPrincipalWindowsVersion)" />
     <PackageVersion Include="System.Text.Encoding.CodePages" Version="$(SystemTextEncodingCodePagesVersion)" />
     <PackageVersion Include="System.Text.Json" Version="$(SystemTextJsonVersion)" />
+    <PackageVersion Include="System.Text.RegularExpressions" Version="$(SystemTextRegularExpressionsVersion)" />
     <PackageVersion Include="System.Threading.Channels" Version="$(SystemThreadingChannelsVersion)" />
     <PackageVersion Include="System.Threading.Tasks.Dataflow" Version="$(SystemThreadingTasksDataflowVersion)" />
     <PackageVersion Include="xunit.console" Version="$(XUnitVersion)" />