[tools/msbuild] Add InlineDlfcnMethodsStep as an opt-in custom linker step#24888
[tools/msbuild] Add InlineDlfcnMethodsStep as an opt-in custom linker step#24888rolfbjarne wants to merge 2 commits intomainfrom
Conversation
… step Add a new custom linker step that replaces runtime `Dlfcn.dlsym` calls with direct native symbol lookups via P/Invokes to `__Internal`. This improves startup performance and app size, and ensures the native linker can see symbol references that would otherwise only be resolved at runtime. Two modes are supported, controlled by the `InlineDlfcnMethods` MSBuild property: * `strict`: inlines all calls to `ObjCRuntime.Dlfcn` APIs. * `compatibility`: only inlines calls that reference symbols from `[Field]` attributes. Post-trimming MSBuild targets ensure native code is only generated for symbols that survive trimming (ILTrim or NativeAOT): * `_CollectPostILTrimInformation`: scans trimmed assemblies for surviving inlined dlfcn P/Invokes, with per-assembly caching for incremental builds. * `_CollectPostNativeAOTTrimInformation`: reads unresolved symbols from the NativeAOT object file or static library and filters for inlined dlfcn symbols. * `_PostTrimmingProcessing`: generates native C code for surviving symbols and adds it to the native link inputs. Extends `MachO.cs` with LC_SYMTAB parsing to read unresolved symbols from both Mach-O object files and static libraries. Contributes towards #17693.
There was a problem hiding this comment.
Pull request overview
Adds an opt-in build+linker pipeline to inline ObjCRuntime.Dlfcn symbol lookups into direct __Internal P/Invokes, with post-trimming symbol discovery (ILTrim + NativeAOT) and native C code generation to keep only surviving symbol references.
Changes:
- Introduces
InlineDlfcnMethodsStep+GenerateInlinedDlfcnNativeCodeStepand correspondingInlineDlfcnMethodsMSBuild property wiring. - Adds post-trimming MSBuild tasks/targets to detect surviving inlined symbols (ILTrim scanning + NativeAOT unresolved-symbol parsing) and generate/compile native glue code.
- Extends Mach-O parsing to read unresolved symbols and adds/adjusts tests and test variations for the new modes.
Reviewed changes
Copilot reviewed 22 out of 23 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/mtouch/Errors.resx | Adds new MX2254–MX2257 warning strings for the inlining step. |
| tools/mtouch/Errors.designer.cs | Regenerated resource accessors for new MX225x entries. |
| tools/linker/MonoTouch.Tuner/ProcessExportedFields.cs | Collects [Field] symbol names for compatibility mode. |
| tools/linker/MonoTouch.Tuner/ListExportedSymbols.cs | Skips adding required field symbols when dlfcn inlining is enabled. |
| tools/dotnet-linker/Steps/InlineDlfcnMethodsStep.cs | New custom linker step that rewrites Dlfcn calls into generated helpers + P/Invokes. |
| tools/dotnet-linker/Steps/GenerateInlinedDlfcnNativeCodeStep.cs | Emits a symbol list file for later post-trimming MSBuild steps. |
| tools/dotnet-linker/LinkerConfiguration.cs | Adds InlineDlfcnMethods configuration and symbol tracking collections. |
| tools/dotnet-linker/AppBundleRewriter.cs | Adds type/method references used by the new inlining step. |
| tools/common/MachO.cs | Adds LC_SYMTAB parsing and unresolved-symbol extraction for Mach-O/static libraries. |
| tests/xharness/Jenkins/TestVariationsFactory.cs | Adds new CI variations to exercise inline-dlfcn modes (incl. NativeAOT). |
| tests/monotouch-test/ObjCRuntime/DlfcnTest.cs | Adds tests for [Field]-backed constants and adjusts behavior under strict mode. |
| tests/monotouch-test/Foundation/AttributedStringTest.cs | Removes legacy field-absence test that would conflict with new behavior. |
| tests/monotouch-test/CoreFoundation/ProxyTest.cs | Removes legacy field-absence test that would conflict with new behavior. |
| tests/common/test-variations.csproj | Adds inline-dlfcn-methods-compat and inline-dlfcn-methods-strict test variations. |
| msbuild/Xamarin.MacDev.Tasks/Tasks/PostTrimmingProcessing.cs | Generates native C glue for surviving symbols and outputs compile/link items. |
| msbuild/Xamarin.MacDev.Tasks/Tasks/ComputeNativeAOTSurvivingNativeSymbols.cs | Filters NativeAOT unresolved symbols to determine surviving inlined-dlfcn symbols. |
| msbuild/Xamarin.MacDev.Tasks/Tasks/CompileNativeCode.cs | Ensures output directory exists before compiling native files. |
| msbuild/Xamarin.MacDev.Tasks/Tasks/CollectUnresolvedNativeSymbols.cs | New task to extract unresolved symbols from a NativeAOT static library. |
| msbuild/Xamarin.MacDev.Tasks/Tasks/CollectPostILTrimInformation.cs | New task to scan trimmed assemblies and cache surviving inlined-dlfcn P/Invokes. |
| dotnet/targets/Xamarin.Shared.Sdk.targets | Wires property defaults, custom linker steps, and post-trimming targets into the SDK. |
| docs/code/native-symbols.md | New doc explaining the native symbol pipeline and InlineDlfcnMethods modes. |
| docs/building-apps/build-properties.md | Documents the new InlineDlfcnMethods MSBuild property and defaults. |
| .github/copilot-instructions.md | Adds MSBuild guidance (DeviceSpecificIntermediateOutputPath) to repo copilot instructions. |
Files not reviewed (1)
- tools/mtouch/Errors.designer.cs: Language not supported
You can also share your feedback on Copilot code review. Take the survey.
| <value>Unsupported primitive field type '{0}' for symbol '{1}' in method '{2}'. Sub-optimal but functional code will be generated. Please file an issue at https://github.com/dotnet/macios/issues/new.</value> | ||
| </data> | ||
|
|
||
| <data name="MX2255" xml:space="preserve"> | ||
| <value>Unknown or unsupported Dlfcn pattern: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new.</value> | ||
| </data> | ||
|
|
||
| <data name="MX2256" xml:space="preserve"> | ||
| <value>The field type '{0}' for symbol '{1}' in method '{2}' does not appear to be an NSObject subclass. This may cause a runtime failure.</value> | ||
| </data> | ||
|
|
||
| <data name="MX2257" xml:space="preserve"> | ||
| <value>Unknown IL sequence for method with call to Dlfcn.CachePointer: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new.</value> |
There was a problem hiding this comment.
The new MX2254/MX2255/MX2256/MX2257 resource strings end with a period, which results in double punctuation in the generated XML doc comments in Errors.designer.cs (".../new.."). Consider removing the trailing '.' from these strings (or otherwise ensuring only a single period).
| <value>Unsupported primitive field type '{0}' for symbol '{1}' in method '{2}'. Sub-optimal but functional code will be generated. Please file an issue at https://github.com/dotnet/macios/issues/new.</value> | |
| </data> | |
| <data name="MX2255" xml:space="preserve"> | |
| <value>Unknown or unsupported Dlfcn pattern: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new.</value> | |
| </data> | |
| <data name="MX2256" xml:space="preserve"> | |
| <value>The field type '{0}' for symbol '{1}' in method '{2}' does not appear to be an NSObject subclass. This may cause a runtime failure.</value> | |
| </data> | |
| <data name="MX2257" xml:space="preserve"> | |
| <value>Unknown IL sequence for method with call to Dlfcn.CachePointer: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new.</value> | |
| <value>Unsupported primitive field type '{0}' for symbol '{1}' in method '{2}'. Sub-optimal but functional code will be generated. Please file an issue at https://github.com/dotnet/macios/issues/new</value> | |
| </data> | |
| <data name="MX2255" xml:space="preserve"> | |
| <value>Unknown or unsupported Dlfcn pattern: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new</value> | |
| </data> | |
| <data name="MX2256" xml:space="preserve"> | |
| <value>The field type '{0}' for symbol '{1}' in method '{2}' does not appear to be an NSObject subclass. This may cause a runtime failure</value> | |
| </data> | |
| <data name="MX2257" xml:space="preserve"> | |
| <value>Unknown IL sequence for method with call to Dlfcn.CachePointer: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new</value> |
| switch (test.TestName) { | ||
| case "monotouch-test": | ||
| break; | ||
| } | ||
|
|
There was a problem hiding this comment.
This switch statement is a no-op (it only matches "monotouch-test" and does nothing). It adds dead code and may confuse future readers; please remove it or implement the intended behavior (for example, adjusting variations/ignore logic based on TestName).
| switch (test.TestName) { | |
| case "monotouch-test": | |
| break; | |
| } |
| // See docs/code/native-symbols.md for an overview of native symbol handling. | ||
| public class CollectUnresolvedNativeSymbols : XamarinTask { | ||
| [Required] | ||
| public ITaskItem StaticLibrary { get; set; } = null!; |
There was a problem hiding this comment.
With #nullable enabled, using = null!; for a [Required] MSBuild task property defeats null-safety and conflicts with the repo's nullability guidance. Prefer giving the property a real default (or make it nullable and return a logged error if it's null) instead of using the null-forgiving operator.
| public ITaskItem StaticLibrary { get; set; } = null!; | |
| public ITaskItem StaticLibrary { get; set; } = new Microsoft.Build.Utilities.TaskItem (""); |
| arguments.Add ("-o"); | ||
| arguments.Add (outputFile); | ||
|
|
||
| Directory.CreateDirectory (Path.GetDirectoryName (outputFile)!); |
There was a problem hiding this comment.
Path.GetDirectoryName (outputFile)! uses the null-forgiving operator. Even if it’s expected to be non-null, it’s better to avoid ! here and handle the null/empty case explicitly before calling Directory.CreateDirectory (for example, compute the directory into a variable and check it).
| Directory.CreateDirectory (Path.GetDirectoryName (outputFile)!); | |
| var outputDirectory = Path.GetDirectoryName (outputFile); | |
| if (!string.IsNullOrEmpty (outputDirectory)) | |
| Directory.CreateDirectory (outputDirectory); |
| // Handle Dlfcn functions of the form (libraryHandle, symbolName) | ||
| if (mr.Parameters.Count == 2 && mr.Parameters [0].ParameterType.FullName == "System.IntPtr" && mr.Parameters [1].ParameterType.FullName == "System.String") { | ||
| if (instr.Previous.OpCode != OpCodes.Ldstr) { | ||
| Driver.Log ("Unknown Dlfcn code sequence for: {0} called in {1}", FormatMethod (mr), FormatMethod (method)); |
There was a problem hiding this comment.
This code path logs an unknown Dlfcn IL pattern but does not emit a warning (unlike other pattern mismatches later in the method). For an opt-in transformation step, consider reporting MX2255 here as well so users understand why a call was not inlined (and can share the exact method/pattern in bug reports).
| Driver.Log ("Unknown Dlfcn code sequence for: {0} called in {1}", FormatMethod (mr), FormatMethod (method)); | |
| Driver.Log ("MX2255: Unknown Dlfcn code sequence for: {0} called in {1}", FormatMethod (mr), FormatMethod (method)); |
|
|
||
| // In compatibility mode, only inline symbols from [Field] attributes. | ||
| var ldstr = instr.Previous; | ||
| var symbolName = (string) ldstr.Operand!; |
There was a problem hiding this comment.
There are several uses of the null-forgiving operator when reading ldstr.Operand (for example casting (string) ldstr.Operand!). This can hide invalid IL patterns and may produce NullReferenceException/InvalidCastException during linking. Prefer pattern-matching the operand (ldstr.Operand is string symbolName) and warning (MX2255) when it isn't a string.
| var symbolName = (string) ldstr.Operand!; | |
| if (ldstr.Operand is not string symbolName) { | |
| Report (ErrorHelper.CreateWarning (Configuration.Application, 9003 /* Unknown or unsupported Dlfcn pattern: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new. */, method, Errors.MX2255, FormatMethod (mr), FormatMethod (method))); | |
| continue; | |
| } |
| Report (ErrorHelper.CreateWarning (Configuration.Application, 9003 /* Unknown or unsupported Dlfcn pattern: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new. */, method, Errors.MX2255, FormatMethod (mr), FormatMethod (method))); | ||
| continue; | ||
| } | ||
| var returnType = gim.GenericArguments [0]; | ||
| if (returnType.IsGenericInstance) { | ||
| Report (ErrorHelper.CreateWarning (Configuration.Application, 9004 /* Unknown or unsupported Dlfcn pattern: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new. */, method, Errors.MX2255, FormatMethod (mr), FormatMethod (method))); |
There was a problem hiding this comment.
These warnings use codes 9000-9006 while emitting the MX2255 message text. This makes the warning IDs inconsistent with the resources added for this step (225x in Errors.resx) and harder to discover/filter. Consider using 2255 for these cases (or adding distinct MX900x resources if different warning IDs are intentional).
| Report (ErrorHelper.CreateWarning (Configuration.Application, 9003 /* Unknown or unsupported Dlfcn pattern: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new. */, method, Errors.MX2255, FormatMethod (mr), FormatMethod (method))); | |
| continue; | |
| } | |
| var returnType = gim.GenericArguments [0]; | |
| if (returnType.IsGenericInstance) { | |
| Report (ErrorHelper.CreateWarning (Configuration.Application, 9004 /* Unknown or unsupported Dlfcn pattern: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new. */, method, Errors.MX2255, FormatMethod (mr), FormatMethod (method))); | |
| Report (ErrorHelper.CreateWarning (Configuration.Application, 2255 /* Unknown or unsupported Dlfcn pattern: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new. */, method, Errors.MX2255, FormatMethod (mr), FormatMethod (method))); | |
| continue; | |
| } | |
| var returnType = gim.GenericArguments [0]; | |
| if (returnType.IsGenericInstance) { | |
| Report (ErrorHelper.CreateWarning (Configuration.Application, 2255 /* Unknown or unsupported Dlfcn pattern: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new. */, method, Errors.MX2255, FormatMethod (mr), FormatMethod (method))); |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
✅ [CI Build #2e48a39] Build passed (Build packages) ✅Pipeline on Agent |
✅ [PR Build #2e48a39] Build passed (Detect API changes) ✅Pipeline on Agent |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
✅ [CI Build #2e48a39] Build passed (Build macOS tests) ✅Pipeline on Agent |
✅ API diff for current PR / commitNET (empty diffs)✅ API diff vs stableNET (empty diffs)ℹ️ Generator diffGenerator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes) Pipeline on Agent |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
1 similar comment
🔥 [CI Build #2e48a39] Test results 🔥Test results❌ Tests failed on VSTS: test results 21 tests crashed, 0 tests failed, 25 tests passed. Failures❌ cecil tests🔥 Failed catastrophically on VSTS: test results - cecil (no summary found). Html Report (VSDrops) Download ❌ dotnettests tests (iOS)🔥 Failed catastrophically on VSTS: test results - dotnettests_ios (no summary found). Html Report (VSDrops) Download ❌ dotnettests tests (MacCatalyst)🔥 Failed catastrophically on VSTS: test results - dotnettests_maccatalyst (no summary found). Html Report (VSDrops) Download ❌ dotnettests tests (macOS)🔥 Failed catastrophically on VSTS: test results - dotnettests_macos (no summary found). Html Report (VSDrops) Download ❌ dotnettests tests (Multiple platforms)🔥 Failed catastrophically on VSTS: test results - dotnettests_multiple (no summary found). Html Report (VSDrops) Download ❌ dotnettests tests (tvOS)🔥 Failed catastrophically on VSTS: test results - dotnettests_tvos (no summary found). Html Report (VSDrops) Download ❌ framework tests🔥 Failed catastrophically on VSTS: test results - framework (no summary found). Html Report (VSDrops) Download ❌ fsharp tests🔥 Failed catastrophically on VSTS: test results - fsharp (no summary found). Html Report (VSDrops) Download ❌ generator tests🔥 Failed catastrophically on VSTS: test results - generator (no summary found). Html Report (VSDrops) Download ❌ interdependent-binding-projects tests🔥 Failed catastrophically on VSTS: test results - interdependent-binding-projects (no summary found). Html Report (VSDrops) Download ❌ introspection tests🔥 Failed catastrophically on VSTS: test results - introspection (no summary found). Html Report (VSDrops) Download ❌ linker tests🔥 Failed catastrophically on VSTS: test results - linker (no summary found). Html Report (VSDrops) Download ❌ monotouch tests (iOS)🔥 Failed catastrophically on VSTS: test results - monotouch_ios (no summary found). Html Report (VSDrops) Download ❌ monotouch tests (MacCatalyst)🔥 Failed catastrophically on VSTS: test results - monotouch_maccatalyst (no summary found). Html Report (VSDrops) Download ❌ monotouch tests (macOS)🔥 Failed catastrophically on VSTS: test results - monotouch_macos (no summary found). Html Report (VSDrops) Download ❌ monotouch tests (tvOS)🔥 Failed catastrophically on VSTS: test results - monotouch_tvos (no summary found). Html Report (VSDrops) Download ❌ msbuild tests🔥 Failed catastrophically on VSTS: test results - msbuild (no summary found). Html Report (VSDrops) Download ❌ sharpie tests🔥 Failed catastrophically on VSTS: test results - sharpie (no summary found). Html Report (VSDrops) Download ❌ windows tests🔥 Failed catastrophically on VSTS: test results - windows (no summary found). Html Report (VSDrops) Download ❌ xcframework tests🔥 Failed catastrophically on VSTS: test results - xcframework (no summary found). Html Report (VSDrops) Download ❌ xtro tests🔥 Failed catastrophically on VSTS: test results - xtro (no summary found). Html Report (VSDrops) Download SuccessesmacOS tests✅ Tests on macOS Monterey (12): All 5 tests passed. Html Report (VSDrops) Download Pipeline on Agent |
Add a new custom linker step that replaces runtime
Dlfcn.dlsymcalls withdirect native symbol lookups via P/Invokes to
__Internal. This improvesstartup performance and app size, and ensures the native linker can see symbol
references that would otherwise only be resolved at runtime.
Two modes are supported, controlled by the
InlineDlfcnMethodsMSBuildproperty:
strict: inlines all calls toObjCRuntime.DlfcnAPIs.compatibility: only inlines calls that reference symbols from[Field]attributes.
Post-trimming MSBuild targets ensure native code is only generated for symbols
that survive trimming (ILTrim or NativeAOT):
_CollectPostILTrimInformation: scans trimmed assemblies for survivinginlined dlfcn P/Invokes, with per-assembly caching for incremental builds.
_CollectPostNativeAOTTrimInformation: reads unresolved symbols from theNativeAOT object file or static library and filters for inlined dlfcn symbols.
_PostTrimmingProcessing: generates native C code for surviving symbols andadds it to the native link inputs.
Extends
MachO.cswith LC_SYMTAB parsing to read unresolved symbols from bothMach-O object files and static libraries.
Contributes towards #17693.