FileSystemWatcher.Linux: use a single inotify instance and refactor watch tracking.

[tmds]

(By default) Linux allows 128 inotify instances per user. By sharing the inotify instance between the FileSystemWatchers we reduce contention with other applications.

Fixes #62869.

[tmds]

@dotnet/area-system-io @stephentoub ptal.

[adamsitnik]

@tmds big thanks for your contribution! For now I've reviewed 25% of this PR (I need to wrap my head around all the locks and it's going to take me a while), PTAL at my comments.

[tmds]

For the locking, this overview may be helpful:

• watcherslock: guards the list of Watcher
• lock on Watcher: guards its tree of WatchedDirectory
• lock on Watch: guards its list of WatchedDirectory
• addLock: this reader writer lock enables concurrent adding of watches ("readers"), but no watches may be added while an inotify watch is removed ("writer")

To prevent deadlocks, the locks (as needed) are taken in this order: watchersLock, addLock, lock on Watcher, lock on Watch.

[tmds]

@adamsitnik this is a challenge to review so feel free to ask any questions you have while looking at the code.

I'm also going to make some time this week to look at the PR with my reviewer's hat on.

[tmds]

@adamsitnik @jozkee can we target .NET 11 preview 1 for this PR?

[tmds]

@jeffhandley @adamsitnik @jozkee it would be nice if we can work towards getting this merged.

[Copilot]

Pull request overview

This PR refactors the Linux implementation of FileSystemWatcher to use a single shared inotify instance across all FileSystemWatcher objects, addressing the low per-user limit of 128 inotify instances on Linux. The refactoring introduces a new architecture with improved watch management and event processing.

Key changes:

• Introduces a shared inotify instance pattern to reduce contention with other applications
• Implements a dedicated thread for reading inotify events with ThreadPool-based event handler invocation to prevent blocking
• Adds new internal classes (INotify, Watcher, WatchedDirectory, Watch) to track watch state and manage the relationship between paths and watch descriptors

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 7 comments.

File	Description
System/IO/FileSystemWatcher.cs	Extracts CreateBufferOverflowException method for reuse in Linux implementation
System/IO/FileSystemWatcher.Linux.cs	Major refactoring: implements shared inotify instance, new watch tracking structures, improved event processing with Channel-based queuing
System.IO.FileSystem.Watcher.csproj	Adds required dependencies for System.Collections.Concurrent and System.Threading.Channels on non-Windows platforms
Interop.INotify.cs	Adds IN_MASK_ADD flag support needed for sharing inotify instances across multiple watchers

[adamsitnik]

I've reviewed 30-40% of the code. Since I am not very familiar with this code and I know that historically we have disabled plenty of FSW tests (they were flaky) my next step will be to implement stress tests (and when the implementation passes them review the rest).

Big thanks for your contribution @tmds !

[adamsitnik]

@tmds Could you please run these stress tests with just .NET 10 and with your new implementation and report back the results?

For the latter you should be able to do that with corerun $pathToCoreRun FswStressTests.dll

[tmds]

@adamsitnik, thanks for looking at this! I'll run the stress tests on Monday and report how it went.

[tmds]

I built vmr's 10.0 branch with this implementation and ran the stress tests:

$ dotnet run -c Release
=== FileSystemWatcher Stress Tests ===
Started at: 2025-12-15 13:48:48

[1/15] Running: Basic File Creation
    ✓ PASSED in 0.02s

[2/15] Running: Multiple File Operations
    ✓ PASSED in 0.31s

[3/15] Running: Directory Creation
    ✓ PASSED in 0.00s

[4/15] Running: Recursive Directory Monitoring
    ✓ PASSED in 0.15s

[5/15] Running: High-Frequency File Operations
    ✓ PASSED in 2.01s

[6/15] Running: Large File Monitoring
    ✓ PASSED in 0.01s

[7/15] Running: Concurrent Watchers
    ✓ PASSED in 1.00s

[8/15] Running: Rename Operations
    ✓ PASSED in 0.41s

[9/15] Running: Rapid Create/Delete Cycles
    ✓ PASSED in 3.02s

[10/15] Running: Nested Directory Operations
    ✓ PASSED in 2.37s

[11/15] Running: Parallel Watchers and File Creation
    ✓ PASSED in 2.01s

[12/15] Running: Parallel Create/Delete Cycles
    ✓ PASSED in 8.01s

[13/15] Running: Rapid Watcher Start/Stop with Producer Thread
    ✓ PASSED in 10.02s

[14/15] Running: Aggressive Watcher Lifecycle Management
    ✓ PASSED in 0.78s

[15/15] Running: Global Watcher Contention Test
    Created 6492 watchers, 4037 received events, total 8336495 events
    ✓ PASSED in 8.55s

=== Summary ===
Total scenarios: 15
Passed: 15
Failed: 0
Total execution time: 38.67s

All tests passed!

To verify I used this implementation for the test run, I checked the inotify_add_watch syscalls, and all of them were using the IN_MASK_ADD flag (which is added in this PR).

[adamsitnik]

I built vmr's 10.0 branch with this implementation and ran the stress tests:

$ dotnet run -c Release
=== FileSystemWatcher Stress Tests ===
Started at: 2025-12-15 13:48:48

[1/15] Running: Basic File Creation
    ✓ PASSED in 0.02s

[2/15] Running: Multiple File Operations
    ✓ PASSED in 0.31s

[3/15] Running: Directory Creation
    ✓ PASSED in 0.00s

[4/15] Running: Recursive Directory Monitoring
    ✓ PASSED in 0.15s

[5/15] Running: High-Frequency File Operations
    ✓ PASSED in 2.01s

[6/15] Running: Large File Monitoring
    ✓ PASSED in 0.01s

[7/15] Running: Concurrent Watchers
    ✓ PASSED in 1.00s

[8/15] Running: Rename Operations
    ✓ PASSED in 0.41s

[9/15] Running: Rapid Create/Delete Cycles
    ✓ PASSED in 3.02s

[10/15] Running: Nested Directory Operations
    ✓ PASSED in 2.37s

[11/15] Running: Parallel Watchers and File Creation
    ✓ PASSED in 2.01s

[12/15] Running: Parallel Create/Delete Cycles
    ✓ PASSED in 8.01s

[13/15] Running: Rapid Watcher Start/Stop with Producer Thread
    ✓ PASSED in 10.02s

[14/15] Running: Aggressive Watcher Lifecycle Management
    ✓ PASSED in 0.78s

[15/15] Running: Global Watcher Contention Test
    Created 6492 watchers, 4037 received events, total 8336495 events
    ✓ PASSED in 8.55s

=== Summary ===
Total scenarios: 15
Passed: 15
Failed: 0
Total execution time: 38.67s

All tests passed!

To verify I used this implementation for the test run, I checked the inotify_add_watch syscalls, and all of them were using the IN_MASK_ADD flag (which is added in this PR).

Excellent! Our plan is that @jozkee is going to review the PR and I am going to merge it in the 2nd week of January.

[tmds]

hi @jozkee, I look forward to get you review on this so we can get this merged.

[jozkee]

I'm halfway through.