Skip to content

[tools/msbuild] Add InlineDlfcnMethodsStep as an opt-in custom linker step#24888

Draft
rolfbjarne wants to merge 2 commits intomainfrom
dev/rolf/inline-dlfcn
Draft

[tools/msbuild] Add InlineDlfcnMethodsStep as an opt-in custom linker step#24888
rolfbjarne wants to merge 2 commits intomainfrom
dev/rolf/inline-dlfcn

Conversation

@rolfbjarne
Copy link
Member

Add a new custom linker step that replaces runtime Dlfcn.dlsym calls with
direct native symbol lookups via P/Invokes to __Internal. This improves
startup performance and app size, and ensures the native linker can see symbol
references that would otherwise only be resolved at runtime.

Two modes are supported, controlled by the InlineDlfcnMethods MSBuild
property:

  • strict: inlines all calls to ObjCRuntime.Dlfcn APIs.
  • compatibility: only inlines calls that reference symbols from [Field]
    attributes.

Post-trimming MSBuild targets ensure native code is only generated for symbols
that survive trimming (ILTrim or NativeAOT):

  • _CollectPostILTrimInformation: scans trimmed assemblies for surviving
    inlined dlfcn P/Invokes, with per-assembly caching for incremental builds.
  • _CollectPostNativeAOTTrimInformation: reads unresolved symbols from the
    NativeAOT object file or static library and filters for inlined dlfcn symbols.
  • _PostTrimmingProcessing: generates native C code for surviving symbols and
    adds it to the native link inputs.

Extends MachO.cs with LC_SYMTAB parsing to read unresolved symbols from both
Mach-O object files and static libraries.

Contributes towards #17693.

… step

Add a new custom linker step that replaces runtime `Dlfcn.dlsym` calls with
direct native symbol lookups via P/Invokes to `__Internal`. This improves
startup performance and app size, and ensures the native linker can see symbol
references that would otherwise only be resolved at runtime.

Two modes are supported, controlled by the `InlineDlfcnMethods` MSBuild
property:

* `strict`: inlines all calls to `ObjCRuntime.Dlfcn` APIs.
* `compatibility`: only inlines calls that reference symbols from `[Field]`
  attributes.

Post-trimming MSBuild targets ensure native code is only generated for symbols
that survive trimming (ILTrim or NativeAOT):

* `_CollectPostILTrimInformation`: scans trimmed assemblies for surviving
  inlined dlfcn P/Invokes, with per-assembly caching for incremental builds.
* `_CollectPostNativeAOTTrimInformation`: reads unresolved symbols from the
  NativeAOT object file or static library and filters for inlined dlfcn symbols.
* `_PostTrimmingProcessing`: generates native C code for surviving symbols and
  adds it to the native link inputs.

Extends `MachO.cs` with LC_SYMTAB parsing to read unresolved symbols from both
Mach-O object files and static libraries.

Contributes towards #17693.
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in build+linker pipeline to inline ObjCRuntime.Dlfcn symbol lookups into direct __Internal P/Invokes, with post-trimming symbol discovery (ILTrim + NativeAOT) and native C code generation to keep only surviving symbol references.

Changes:

  • Introduces InlineDlfcnMethodsStep + GenerateInlinedDlfcnNativeCodeStep and corresponding InlineDlfcnMethods MSBuild property wiring.
  • Adds post-trimming MSBuild tasks/targets to detect surviving inlined symbols (ILTrim scanning + NativeAOT unresolved-symbol parsing) and generate/compile native glue code.
  • Extends Mach-O parsing to read unresolved symbols and adds/adjusts tests and test variations for the new modes.

Reviewed changes

Copilot reviewed 22 out of 23 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
tools/mtouch/Errors.resx Adds new MX2254–MX2257 warning strings for the inlining step.
tools/mtouch/Errors.designer.cs Regenerated resource accessors for new MX225x entries.
tools/linker/MonoTouch.Tuner/ProcessExportedFields.cs Collects [Field] symbol names for compatibility mode.
tools/linker/MonoTouch.Tuner/ListExportedSymbols.cs Skips adding required field symbols when dlfcn inlining is enabled.
tools/dotnet-linker/Steps/InlineDlfcnMethodsStep.cs New custom linker step that rewrites Dlfcn calls into generated helpers + P/Invokes.
tools/dotnet-linker/Steps/GenerateInlinedDlfcnNativeCodeStep.cs Emits a symbol list file for later post-trimming MSBuild steps.
tools/dotnet-linker/LinkerConfiguration.cs Adds InlineDlfcnMethods configuration and symbol tracking collections.
tools/dotnet-linker/AppBundleRewriter.cs Adds type/method references used by the new inlining step.
tools/common/MachO.cs Adds LC_SYMTAB parsing and unresolved-symbol extraction for Mach-O/static libraries.
tests/xharness/Jenkins/TestVariationsFactory.cs Adds new CI variations to exercise inline-dlfcn modes (incl. NativeAOT).
tests/monotouch-test/ObjCRuntime/DlfcnTest.cs Adds tests for [Field]-backed constants and adjusts behavior under strict mode.
tests/monotouch-test/Foundation/AttributedStringTest.cs Removes legacy field-absence test that would conflict with new behavior.
tests/monotouch-test/CoreFoundation/ProxyTest.cs Removes legacy field-absence test that would conflict with new behavior.
tests/common/test-variations.csproj Adds inline-dlfcn-methods-compat and inline-dlfcn-methods-strict test variations.
msbuild/Xamarin.MacDev.Tasks/Tasks/PostTrimmingProcessing.cs Generates native C glue for surviving symbols and outputs compile/link items.
msbuild/Xamarin.MacDev.Tasks/Tasks/ComputeNativeAOTSurvivingNativeSymbols.cs Filters NativeAOT unresolved symbols to determine surviving inlined-dlfcn symbols.
msbuild/Xamarin.MacDev.Tasks/Tasks/CompileNativeCode.cs Ensures output directory exists before compiling native files.
msbuild/Xamarin.MacDev.Tasks/Tasks/CollectUnresolvedNativeSymbols.cs New task to extract unresolved symbols from a NativeAOT static library.
msbuild/Xamarin.MacDev.Tasks/Tasks/CollectPostILTrimInformation.cs New task to scan trimmed assemblies and cache surviving inlined-dlfcn P/Invokes.
dotnet/targets/Xamarin.Shared.Sdk.targets Wires property defaults, custom linker steps, and post-trimming targets into the SDK.
docs/code/native-symbols.md New doc explaining the native symbol pipeline and InlineDlfcnMethods modes.
docs/building-apps/build-properties.md Documents the new InlineDlfcnMethods MSBuild property and defaults.
.github/copilot-instructions.md Adds MSBuild guidance (DeviceSpecificIntermediateOutputPath) to repo copilot instructions.
Files not reviewed (1)
  • tools/mtouch/Errors.designer.cs: Language not supported

You can also share your feedback on Copilot code review. Take the survey.

Comment on lines +1099 to +1111
<value>Unsupported primitive field type '{0}' for symbol '{1}' in method '{2}'. Sub-optimal but functional code will be generated. Please file an issue at https://github.com/dotnet/macios/issues/new.</value>
</data>

<data name="MX2255" xml:space="preserve">
<value>Unknown or unsupported Dlfcn pattern: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new.</value>
</data>

<data name="MX2256" xml:space="preserve">
<value>The field type '{0}' for symbol '{1}' in method '{2}' does not appear to be an NSObject subclass. This may cause a runtime failure.</value>
</data>

<data name="MX2257" xml:space="preserve">
<value>Unknown IL sequence for method with call to Dlfcn.CachePointer: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new.</value>
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new MX2254/MX2255/MX2256/MX2257 resource strings end with a period, which results in double punctuation in the generated XML doc comments in Errors.designer.cs (".../new.."). Consider removing the trailing '.' from these strings (or otherwise ensuring only a single period).

Suggested change
<value>Unsupported primitive field type '{0}' for symbol '{1}' in method '{2}'. Sub-optimal but functional code will be generated. Please file an issue at https://github.com/dotnet/macios/issues/new.</value>
</data>
<data name="MX2255" xml:space="preserve">
<value>Unknown or unsupported Dlfcn pattern: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new.</value>
</data>
<data name="MX2256" xml:space="preserve">
<value>The field type '{0}' for symbol '{1}' in method '{2}' does not appear to be an NSObject subclass. This may cause a runtime failure.</value>
</data>
<data name="MX2257" xml:space="preserve">
<value>Unknown IL sequence for method with call to Dlfcn.CachePointer: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new.</value>
<value>Unsupported primitive field type '{0}' for symbol '{1}' in method '{2}'. Sub-optimal but functional code will be generated. Please file an issue at https://github.com/dotnet/macios/issues/new</value>
</data>
<data name="MX2255" xml:space="preserve">
<value>Unknown or unsupported Dlfcn pattern: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new</value>
</data>
<data name="MX2256" xml:space="preserve">
<value>The field type '{0}' for symbol '{1}' in method '{2}' does not appear to be an NSObject subclass. This may cause a runtime failure</value>
</data>
<data name="MX2257" xml:space="preserve">
<value>Unknown IL sequence for method with call to Dlfcn.CachePointer: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new</value>

Copilot uses AI. Check for mistakes.
Comment on lines +74 to +78
switch (test.TestName) {
case "monotouch-test":
break;
}

Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This switch statement is a no-op (it only matches "monotouch-test" and does nothing). It adds dead code and may confuse future readers; please remove it or implement the intended behavior (for example, adjusting variations/ignore logic based on TestName).

Suggested change
switch (test.TestName) {
case "monotouch-test":
break;
}

Copilot uses AI. Check for mistakes.
// See docs/code/native-symbols.md for an overview of native symbol handling.
public class CollectUnresolvedNativeSymbols : XamarinTask {
[Required]
public ITaskItem StaticLibrary { get; set; } = null!;
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With #nullable enabled, using = null!; for a [Required] MSBuild task property defeats null-safety and conflicts with the repo's nullability guidance. Prefer giving the property a real default (or make it nullable and return a logged error if it's null) instead of using the null-forgiving operator.

Suggested change
public ITaskItem StaticLibrary { get; set; } = null!;
public ITaskItem StaticLibrary { get; set; } = new Microsoft.Build.Utilities.TaskItem ("");

Copilot uses AI. Check for mistakes.
arguments.Add ("-o");
arguments.Add (outputFile);

Directory.CreateDirectory (Path.GetDirectoryName (outputFile)!);
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Path.GetDirectoryName (outputFile)! uses the null-forgiving operator. Even if it’s expected to be non-null, it’s better to avoid ! here and handle the null/empty case explicitly before calling Directory.CreateDirectory (for example, compute the directory into a variable and check it).

Suggested change
Directory.CreateDirectory (Path.GetDirectoryName (outputFile)!);
var outputDirectory = Path.GetDirectoryName (outputFile);
if (!string.IsNullOrEmpty (outputDirectory))
Directory.CreateDirectory (outputDirectory);

Copilot uses AI. Check for mistakes.
// Handle Dlfcn functions of the form (libraryHandle, symbolName)
if (mr.Parameters.Count == 2 && mr.Parameters [0].ParameterType.FullName == "System.IntPtr" && mr.Parameters [1].ParameterType.FullName == "System.String") {
if (instr.Previous.OpCode != OpCodes.Ldstr) {
Driver.Log ("Unknown Dlfcn code sequence for: {0} called in {1}", FormatMethod (mr), FormatMethod (method));
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code path logs an unknown Dlfcn IL pattern but does not emit a warning (unlike other pattern mismatches later in the method). For an opt-in transformation step, consider reporting MX2255 here as well so users understand why a call was not inlined (and can share the exact method/pattern in bug reports).

Suggested change
Driver.Log ("Unknown Dlfcn code sequence for: {0} called in {1}", FormatMethod (mr), FormatMethod (method));
Driver.Log ("MX2255: Unknown Dlfcn code sequence for: {0} called in {1}", FormatMethod (mr), FormatMethod (method));

Copilot uses AI. Check for mistakes.

// In compatibility mode, only inline symbols from [Field] attributes.
var ldstr = instr.Previous;
var symbolName = (string) ldstr.Operand!;
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are several uses of the null-forgiving operator when reading ldstr.Operand (for example casting (string) ldstr.Operand!). This can hide invalid IL patterns and may produce NullReferenceException/InvalidCastException during linking. Prefer pattern-matching the operand (ldstr.Operand is string symbolName) and warning (MX2255) when it isn't a string.

Suggested change
var symbolName = (string) ldstr.Operand!;
if (ldstr.Operand is not string symbolName) {
Report (ErrorHelper.CreateWarning (Configuration.Application, 9003 /* Unknown or unsupported Dlfcn pattern: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new. */, method, Errors.MX2255, FormatMethod (mr), FormatMethod (method)));
continue;
}

Copilot uses AI. Check for mistakes.
Comment on lines +528 to +533
Report (ErrorHelper.CreateWarning (Configuration.Application, 9003 /* Unknown or unsupported Dlfcn pattern: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new. */, method, Errors.MX2255, FormatMethod (mr), FormatMethod (method)));
continue;
}
var returnType = gim.GenericArguments [0];
if (returnType.IsGenericInstance) {
Report (ErrorHelper.CreateWarning (Configuration.Application, 9004 /* Unknown or unsupported Dlfcn pattern: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new. */, method, Errors.MX2255, FormatMethod (mr), FormatMethod (method)));
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These warnings use codes 9000-9006 while emitting the MX2255 message text. This makes the warning IDs inconsistent with the resources added for this step (225x in Errors.resx) and harder to discover/filter. Consider using 2255 for these cases (or adding distinct MX900x resources if different warning IDs are intentional).

Suggested change
Report (ErrorHelper.CreateWarning (Configuration.Application, 9003 /* Unknown or unsupported Dlfcn pattern: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new. */, method, Errors.MX2255, FormatMethod (mr), FormatMethod (method)));
continue;
}
var returnType = gim.GenericArguments [0];
if (returnType.IsGenericInstance) {
Report (ErrorHelper.CreateWarning (Configuration.Application, 9004 /* Unknown or unsupported Dlfcn pattern: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new. */, method, Errors.MX2255, FormatMethod (mr), FormatMethod (method)));
Report (ErrorHelper.CreateWarning (Configuration.Application, 2255 /* Unknown or unsupported Dlfcn pattern: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new. */, method, Errors.MX2255, FormatMethod (mr), FormatMethod (method)));
continue;
}
var returnType = gim.GenericArguments [0];
if (returnType.IsGenericInstance) {
Report (ErrorHelper.CreateWarning (Configuration.Application, 2255 /* Unknown or unsupported Dlfcn pattern: '{0}' in method '{1}'. The call will not be inlined. Please file an issue at https://github.com/dotnet/macios/issues/new. */, method, Errors.MX2255, FormatMethod (mr), FormatMethod (method)));

Copilot uses AI. Check for mistakes.
@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2
Copy link
Collaborator

✅ [CI Build #2e48a39] Build passed (Build packages) ✅

Pipeline on Agent
Hash: 2e48a39b26c5b721b7e1b875b58095a72f5f0be5 [PR build]

@vs-mobiletools-engineering-service2
Copy link
Collaborator

✅ [PR Build #2e48a39] Build passed (Detect API changes) ✅

Pipeline on Agent
Hash: 2e48a39b26c5b721b7e1b875b58095a72f5f0be5 [PR build]

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2
Copy link
Collaborator

✅ [CI Build #2e48a39] Build passed (Build macOS tests) ✅

Pipeline on Agent
Hash: 2e48a39b26c5b721b7e1b875b58095a72f5f0be5 [PR build]

@vs-mobiletools-engineering-service2
Copy link
Collaborator

✅ API diff for current PR / commit

NET (empty diffs)

✅ API diff vs stable

NET (empty diffs)

ℹ️ Generator diff

Generator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes)

Pipeline on Agent
Hash: 2e48a39b26c5b721b7e1b875b58095a72f5f0be5 [PR build]

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

1 similar comment
@vs-mobiletools-engineering-service2
Copy link
Collaborator

🔥 [CI Build #2e48a39] Test results 🔥

Test results

❌ Tests failed on VSTS: test results

21 tests crashed, 0 tests failed, 25 tests passed.

Failures

❌ cecil tests

🔥 Failed catastrophically on VSTS: test results - cecil (no summary found).

Html Report (VSDrops) Download

❌ dotnettests tests (iOS)

🔥 Failed catastrophically on VSTS: test results - dotnettests_ios (no summary found).

Html Report (VSDrops) Download

❌ dotnettests tests (MacCatalyst)

🔥 Failed catastrophically on VSTS: test results - dotnettests_maccatalyst (no summary found).

Html Report (VSDrops) Download

❌ dotnettests tests (macOS)

🔥 Failed catastrophically on VSTS: test results - dotnettests_macos (no summary found).

Html Report (VSDrops) Download

❌ dotnettests tests (Multiple platforms)

🔥 Failed catastrophically on VSTS: test results - dotnettests_multiple (no summary found).

Html Report (VSDrops) Download

❌ dotnettests tests (tvOS)

🔥 Failed catastrophically on VSTS: test results - dotnettests_tvos (no summary found).

Html Report (VSDrops) Download

❌ framework tests

🔥 Failed catastrophically on VSTS: test results - framework (no summary found).

Html Report (VSDrops) Download

❌ fsharp tests

🔥 Failed catastrophically on VSTS: test results - fsharp (no summary found).

Html Report (VSDrops) Download

❌ generator tests

🔥 Failed catastrophically on VSTS: test results - generator (no summary found).

Html Report (VSDrops) Download

❌ interdependent-binding-projects tests

🔥 Failed catastrophically on VSTS: test results - interdependent-binding-projects (no summary found).

Html Report (VSDrops) Download

❌ introspection tests

🔥 Failed catastrophically on VSTS: test results - introspection (no summary found).

Html Report (VSDrops) Download

❌ linker tests

🔥 Failed catastrophically on VSTS: test results - linker (no summary found).

Html Report (VSDrops) Download

❌ monotouch tests (iOS)

🔥 Failed catastrophically on VSTS: test results - monotouch_ios (no summary found).

Html Report (VSDrops) Download

❌ monotouch tests (MacCatalyst)

🔥 Failed catastrophically on VSTS: test results - monotouch_maccatalyst (no summary found).

Html Report (VSDrops) Download

❌ monotouch tests (macOS)

🔥 Failed catastrophically on VSTS: test results - monotouch_macos (no summary found).

Html Report (VSDrops) Download

❌ monotouch tests (tvOS)

🔥 Failed catastrophically on VSTS: test results - monotouch_tvos (no summary found).

Html Report (VSDrops) Download

❌ msbuild tests

🔥 Failed catastrophically on VSTS: test results - msbuild (no summary found).

Html Report (VSDrops) Download

❌ sharpie tests

🔥 Failed catastrophically on VSTS: test results - sharpie (no summary found).

Html Report (VSDrops) Download

❌ windows tests

🔥 Failed catastrophically on VSTS: test results - windows (no summary found).

Html Report (VSDrops) Download

❌ xcframework tests

🔥 Failed catastrophically on VSTS: test results - xcframework (no summary found).

Html Report (VSDrops) Download

❌ xtro tests

🔥 Failed catastrophically on VSTS: test results - xtro (no summary found).

Html Report (VSDrops) Download

Successes

macOS tests

✅ Tests on macOS Monterey (12): All 5 tests passed. Html Report (VSDrops) Download
✅ Tests on macOS Ventura (13): All 5 tests passed. Html Report (VSDrops) Download
✅ Tests on macOS Sonoma (14): All 5 tests passed. Html Report (VSDrops) Download
✅ Tests on macOS Sequoia (15): All 5 tests passed. Html Report (VSDrops) Download
✅ Tests on macOS Tahoe (26): All 5 tests passed. [attempt 2] Html Report (VSDrops) Download

Pipeline on Agent
Hash: 2e48a39b26c5b721b7e1b875b58095a72f5f0be5 [PR build]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants