Repository navigation
fs: let glob walk a mounted virtual file system - #66719
Open
samuelmbabhazi wants to merge 1 commit into
Open
samuelmbabhazi wants to merge 1 commit into
samuelmbabhazi wants to merge 1 commit into
Conversation
The native glob walks the file system with libuv, which cannot see a mounted virtual file system: fs.glob(), fs.globSync() and fsPromises.glob() find nothing under a mount point, where the JavaScript walk they replaced read through the fs hooks. Give the walk a reader. When its cwd is under a mount, or a pattern names one absolutely, the hooked readdirSync(), lstatSync(), statSync() and realpathSync.native() serve its reads, so a mount and the real file system are walked alike, with the same patterns, exclusions, depth limit and symbolic link rules. A walk with a reader runs on the main thread; the asynchronous APIs advance it one batch per pull, from a later loop turn as a thread-pool slice would settle in. A virtual file system need not have inodes, so such a walk tells two spellings of a directory apart by realpath, as on Windows. The reserved root sits under os.devNull, a character device, and the walk descends an absolute pattern from the root one segment at a time, so to the walk that one path is a directory holding the reserved root. Refs: nodejs#65392 Signed-off-by: Samuel Mbabhazi <samuelmbabhazi@gmail.com>
Contributor
|
Welcome to Node.js, and thank you for your first contribution! Before review, please take a moment to read:
Please make sure every commit is signed off. For a first pull request, GitHub Actions require collaborator approval and Jenkins CI must be started by a collaborator or triager, so an initial wait is normal. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Since the native glob (#65392),
fs.glob(),fs.globSync()andfsPromises.glob()find nothing in a mounted virtual file system: the walk reads the file system with libuv, which cannot see a mount point, where the JavaScript walk it replaced read through the fs hooks. On v26.11.1 the walk below finds the files; onmainit returns[]for every API:This gives the native walk a reader. When the walk's cwd is under a mount, or one of its patterns names a mount absolutely, the hooked
readdirSync(),lstatSync(),statSync()andrealpathSync.native()serve itsscandir,lstat,statandrealpathreads, so a mount and the real file system are walked by the same engine, with the same patterns,exclude(patterns and callback, with and withoutwithFileTypes),maxDepthandfollowSymlinkssemantics. A walk that reads nothing under a mount is unchanged.Details:
src/glob/glob_walker.{h,cc}: anFsReaderinterface onWalkOptions, consulted by the guarded walk in place of libuv. A walk with a reader never runs free (the free walk and the listing prefetcher scan with raw libuv calls from pool threads) and tells two spellings of a directory apart by realpath, as on Windows: a virtual file system need not have inodes (the in-memory one numbers everystat()call afresh).src/glob/node_glob.{h,cc}:JsFsReadercalls a JavaScript function(op, path), likeJsExcludeFiltercalls theexcludecallback. For the asynchronous APIs, a request with a reader runs each slice on the main thread from aSetImmediate, one batch per pull, so a walk still settles from a later loop turn and an iterator left early still cancels it. The pull's settlement is factored intoResolve()/Reject(), shared with the thread-pool path.lib/internal/vfs/setup.js: theglobReader(cwd, patterns)handler, which gives the walk its reader when the walk can reach a mount. A read that fails answers as a missing path, as the walk's own reads do. The reserved root sits underos.devNull, which is a character device, and the walk descends an absolute pattern from the root one segment at a time, so to the walk that one path is a directory holding the reserved root.lib/internal/fs/glob.js: passes the reader to the binding.The test covers the three APIs: relative patterns from a cwd under the mount (
**, dot directories, symbolic links with and withoutfollowSymlinks,maxDepth),excludeas patterns and as a callback with and withoutwithFileTypes, absolute patterns into the mount and into the reserved root from a real cwd, a..pattern within the mount, the reserved root as cwd, the real file system while a VFS is mounted, a throwingexcludecallback, early exit from the iterator, a tree larger than one batch, and the unmounted path.Verified locally on Linux: the new test, every
test/parallel/test-vfs-*test,test-fs-glob*andtest-path-globpass.