diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 6506b62ad..fac8c8b5f 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -1473,7 +1473,7 @@ command's integration tests check its output. | `vendor_would_revert_redirect` / `vendor_takeover_reverted_redirect` | `skipped` (advisory event) | vendor / scan / get `--mode vendored` over a hosted pin (every ecosystem, v5.0): dry run — the upstream restore was resolved (registry lookups included) and would succeed (for bun, only after the Bun vendored preflight accepted the lock; a refused lock is previewed as the wet run's `failed ` instead) / wet run — the pin's lock entries were restored to their upstream registry entry before vendoring (mode takeover; detail ` was hosted; restored its upstream registry entry () before vendoring (mode takeover)`), so `vendor --revert` later returns to upstream. Fires on the run that takes over, not on re-runs, and not for a purl whose takeover was rolled back because the backend refused it (see "Takeover reconciliation"). | | `redirect_revert_failed` | `failed` | vendor / scan / get `--mode vendored` (dry and wet): the upstream restore of a hosted pin was refused (`--offline`, a registry that does not answer, a lock shape the restore refuses — for `bun.lockb`, a record the codec cannot rebuild) — detail `cannot vendor over the live hosted pin: cannot restore to its upstream registry entry: ; restore it from version control instead (`git checkout -- `)` (for `bun.lock` / `bun.lockb` the detail also adds `, then run \`bun install --force\` (a plain \`bun install\` keeps the patched copy)`); nothing vendored for the purl, hosted wiring left in place, exit 1 `partialFailure`. | | `patch_fetch_failed` (eject) | `failed` | vendor eject (v5.0): a hosted pin's patch record could not be fetched from `…/patches/view/`; the whole eject is refused (`eject_refused`), nothing touched, exit 1. | -| `redirect_pnpm_lockfile_elsewhere` / `redirect_workspace_lockfile_elsewhere` / `cargo_manifest_not_workspace_root` (hosted) | top-level `error.code` (`status: "error"`) | scan / get `--mode hosted` (v5.0): the project directory is a workspace member whose lock lives in another directory, so the rewriters, which read only the project directory, would pin nothing (pnpm: no npm-family lock here, and the nearest ancestor `pnpm-workspace.yaml` or the project's `lockfile-dir` (`.npmrc`) / `lockfileDir` (`pnpm-workspace.yaml`) puts `pnpm-lock.yaml` elsewhere; npm / yarn / Bun, `redirect_workspace_lockfile_elsewhere`: no npm-family lock here, and the nearest ancestor `package.json` whose `workspaces` (array, or the object form's `packages`) matches the directory holds `package-lock.json`, `npm-shrinkwrap.json`, `yarn.lock`, `bun.lock` or `bun.lockb`; a matching root with none of them that is itself listed by an outer root's `workspaces` hands the check to that root; vlt, same code: the nearest ancestor `vlt.json` whose `workspaces` (a string, an array, or an object of groups) matches the directory holds `vlt-lock.json`, or, as vlt falls back to it when `vlt.json` has no `workspaces` field, the `package.json` `workspaces` root above holds `vlt-lock.json`, and the nearer of a `vlt.json` and a `package.json` root is named; `workspaces` patterns use the glob grammar the package managers share: `*`, `?`, `**`, brace sets and sequences (`{a,b}`, `{1..3}`) and character classes (`[a-c]`, `[!a]`); when a pnpm workspace also governs the directory, the nearer root is named and a tie goes to `redirect_pnpm_lockfile_elsewhere`; uv (pypi, #1138), `redirect_workspace_lockfile_elsewhere`: the directory holds a `pyproject.toml`, and the nearest ancestor `pyproject.toml` declaring `[tool.uv.workspace]` lists it in `members` (and not in `exclude`), with no standalone project (`[project]`, no workspace) in between, so uv installs it from that root's `uv.lock` (a `uv.lock`, `poetry.lock`, `pdm.lock`, `Pipfile.lock` or pylock left in the member is never read and does not exempt it); the message says uv workspaces are not patched from their root yet either, and vendored refuses the same layout with `pypi_uv_workspace_unsupported` instead of rewriting a member with Hatch configuration as a lockless Hatch project; a directory whose own locks are all ones its manager never reads inside a workspace member is refused the same way, naming the ignored locks: `package-lock.json` / `npm-shrinkwrap.json` when its `package.json` `workspaces` root holds `package-lock.json` or `npm-shrinkwrap.json` (npm, #1094), `bun.lock` / `bun.lockb` when that root holds `bun.lock` or `bun.lockb` (Bun, #1101), and `vlt-lock.json` in a directory with no `vlt.json` of its own when its vlt workspace root (as above) holds `vlt-lock.json` (vlt, #1134); vendored refuses it with `vendor_lockfile_missing`, and `vex` reads the ignored lock as absent, with one `patched_ref_unattributable` warning naming it when it holds Socket references) or rewrite the member as a lockless project (cargo: the vendored workspace-root check). Refused before any takeover or write, `--dry-run` included; the message names the directory to run from; exit 1. Disk runs only (an in-memory project has no ancestors). | +| `redirect_pnpm_lockfile_elsewhere` / `redirect_workspace_lockfile_elsewhere` / `cargo_manifest_not_workspace_root` (hosted) | top-level `error.code` (`status: "error"`) | scan / get `--mode hosted` (v5.0): the project directory is a workspace member whose lock lives in another directory, so the rewriters, which read only the project directory, would pin nothing (pnpm: no npm-family lock here, and the nearest ancestor `pnpm-workspace.yaml` or the project's `lockfile-dir` (`.npmrc`) / `lockfileDir` (`pnpm-workspace.yaml`) puts `pnpm-lock.yaml` elsewhere; npm / yarn / Bun, `redirect_workspace_lockfile_elsewhere`: no npm-family lock here, and the nearest ancestor `package.json` whose `workspaces` (array, or the object form's `packages`) matches the directory holds `package-lock.json`, `npm-shrinkwrap.json`, `yarn.lock`, `bun.lock` or `bun.lockb`; a matching root with none of them that is itself listed by an outer root's `workspaces` hands the check to that root; vlt, same code: the nearest ancestor `vlt.json` whose `workspaces` (a string, an array, or an object of groups) matches the directory holds `vlt-lock.json`, or, as vlt falls back to it when `vlt.json` has no `workspaces` field, the `package.json` `workspaces` root above holds `vlt-lock.json`, and the nearer of a `vlt.json` and a `package.json` root is named; `workspaces` patterns use the glob grammar the package managers share: `*`, `?`, `**`, brace sets and sequences (`{a,b}`, `{1..3}`) and character classes (`[a-c]`, `[!a]`), read the way the root's lock's package manager reads them (#1097): extglobs (`@(a|b)`, `?(…)`, `*(…)`, `+(…)`, `!(…)`) match for npm and yarn but not Bun or vlt; a `!` pattern excludes what earlier patterns matched for npm, Bun 1.3+ (a `bun.lock` with a top-level `configVersion`) and vlt, excludes whatever the order for yarn berry (a `yarn.lock` with `__metadata`, where a leading `!(` is an extglob), and is ignored by yarn classic and older Bun (a `bun.lock` without `configVersion`, or `bun.lockb` alone); only the root's locks whose package manager lists the directory govern it (and are named), and a lockless root lists it when any of them would; when a pnpm workspace also governs the directory, the nearer root is named and a tie goes to `redirect_pnpm_lockfile_elsewhere`; uv (pypi, #1138), `redirect_workspace_lockfile_elsewhere`: the directory holds a `pyproject.toml`, and the nearest ancestor `pyproject.toml` declaring `[tool.uv.workspace]` lists it in `members` (and not in `exclude`), with no standalone project (`[project]`, no workspace) in between, so uv installs it from that root's `uv.lock` (a `uv.lock`, `poetry.lock`, `pdm.lock`, `Pipfile.lock` or pylock left in the member is never read and does not exempt it); the message says uv workspaces are not patched from their root yet either, and vendored refuses the same layout with `pypi_uv_workspace_unsupported` instead of rewriting a member with Hatch configuration as a lockless Hatch project; a directory whose own locks are all ones its manager never reads inside a workspace member is refused the same way, naming the ignored locks: `package-lock.json` / `npm-shrinkwrap.json` when its `package.json` `workspaces` root holds `package-lock.json` or `npm-shrinkwrap.json` (npm, #1094), `bun.lock` / `bun.lockb` when that root holds `bun.lock` or `bun.lockb` (Bun, #1101), and `vlt-lock.json` in a directory with no `vlt.json` of its own when its vlt workspace root (as above) holds `vlt-lock.json` (vlt, #1134); vendored refuses it with `vendor_lockfile_missing`, and `vex` reads the ignored lock as absent, with one `patched_ref_unattributable` warning naming it when it holds Socket references) or rewrite the member as a lockless project (cargo: the vendored workspace-root check). Refused before any takeover or write, `--dry-run` included; the message names the directory to run from; exit 1. Disk runs only (an in-memory project has no ancestors). | | `redirect_pnpm_settings_elsewhere` | top-level `error.code` (`status: "error"`) | scan / get `--mode hosted`: the project directory is a pnpm workspace member (listed by the `packages:` globs of the nearest ancestor `pnpm-workspace.yaml`) with its own v9 `pnpm-lock.yaml` (`sharedWorkspaceLockfile: false`) and no `pnpm-workspace.yaml` of its own, so its pnpm settings come from that ancestor file, which pnpm reads alone (a member's own file is ignored). A directory those globs do not list (no `packages:`, an empty list, a non-matching or `!`-excluded path) is a standalone project on pnpm 11.28+/12 that reads only its own file: it is pinned and gets its own `pnpm-workspace.yaml` like any single project. A root file that does not parse, or whose patterns use braces, classes or extglobs, counts as listing the project. When that file neither carries `trustLockfile: true` nor explicitly sets another value, the trust auto-config has nowhere to go: refused before any takeover or write, `--dry-run` included; the message names the root file to add `trustLockfile: true` to (or `--no-trust-lockfile-config` pins without it); exit 1. Once the root file trusts the lock (or opts out), the member is pinned and no nested `pnpm-workspace.yaml` is created; the `redirect_pnpm_trust_lockfile` warning names the root file. In memory, a member whose lock is demoted into its workspace root (#492) is never refused; one whose lock is not (the workspace root's files do not confirm it pins or ignores that lock, or socket.yml leaves the root out) is refused with this code as its project error, nothing written for it, whenever its lock is v9, the trust auto-config is on and that file may list it (listed, unreadable, or not readable as globs), whatever it says about `trustLockfile`. | | `eject_refused` | top-level `error.code` (`status: "error"`) | vendor eject (v5.0): a record fetch failed or a pin's upstream restore was refused while planning; nothing was changed, exit 1. | | `eject_planned` | `applied` (reason) | vendor eject `--dry-run` (v5.0): the pin would be restored upstream and vendored; nothing written. | diff --git a/crates/socket-patch-cli/tests/in_process_redirect_pnpm.rs b/crates/socket-patch-cli/tests/in_process_redirect_pnpm.rs index f7c136dee..2581b2f26 100644 --- a/crates/socket-patch-cli/tests/in_process_redirect_pnpm.rs +++ b/crates/socket-patch-cli/tests/in_process_redirect_pnpm.rs @@ -1877,6 +1877,84 @@ async fn hosted_scan_from_brace_or_class_glob_workspace_member_refuses() { } } +/// #1097: the member matcher read every root's `workspaces` the npm way. +/// yarn classic ignores `!` patterns (`packages/a` stays a member of +/// `["packages/*", "!packages/a"]`), as does Bun before 1.3 (a `bun.lock` +/// with no `configVersion`, or a binary `bun.lockb`), and npm and yarn +/// match extglobs (`packages/@(a|b)`). The matcher dropped those members, +/// so a hosted `scan` / `get` from one pinned nothing and exited 0. +#[tokio::test] +#[serial] +async fn hosted_run_from_negated_or_extglob_workspace_member_refuses() { + let server = MockServer::start().await; + mock_discovery(&server).await; + mock_reference(&server).await; + mock_view(&server).await; + const CLASSIC: &str = + "# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.\n# yarn lockfile v1\n"; + const BERRY: &str = "__metadata:\n version: 8\n cacheKey: 10c0\n"; + const OLD_BUN: &str = "{\n \"lockfileVersion\": 1,\n \"workspaces\": {}\n}\n"; + let negated = r#"["packages/*", "!packages/a"]"#; + for (workspaces, lock_name, lock_text) in [ + (negated, "yarn.lock", CLASSIC), + (negated, "bun.lock", OLD_BUN), + (negated, "bun.lockb", "bun-lockfile-format-v0\n"), + (r#"["packages/@(a|b)"]"#, "package-lock.json", "{}\n"), + (r#"["packages/+(a|c)"]"#, "npm-shrinkwrap.json", "{}\n"), + (r#"["packages/!(z)"]"#, "package-lock.json", "{}\n"), + (r#"["packages/?(a)"]"#, "yarn.lock", CLASSIC), + (r#"["packages/*(a|b)"]"#, "yarn.lock", BERRY), + ] { + let tmp = tempfile::tempdir().unwrap(); + let member = write_workspace_member( + tmp.path(), + "package.json", + &format!(r#"{{ "name": "root", "private": true, "workspaces": {workspaces} }}"#), + lock_name, + ); + let lock = tmp.path().join(lock_name); + std::fs::write(&lock, lock_text).unwrap(); + let case = format!("{workspaces} with {lock_name}"); + let (code, doc) = run_hosted_json(&member, &server.uri()); + assert_refused_workspace_lock_elsewhere(&case, code, &doc, &lock, lock_text, &member); + + let out = scrubbed_cli() + .args([ + "get", + UUID, + "--mode", + "hosted", + "--json", + "--yes", + "--cwd", + member.to_str().unwrap(), + "--api-url", + &server.uri(), + "--org", + ORG, + "--api-token", + "fake", + ]) + .output() + .expect("run socket-patch"); + let doc: serde_json::Value = serde_json::from_slice(&out.stdout).unwrap_or_else(|e| { + panic!( + "{case}: get --json output is not JSON ({e}):\n{}\n{}", + String::from_utf8_lossy(&out.stdout), + String::from_utf8_lossy(&out.stderr) + ) + }); + assert_refused_workspace_lock_elsewhere( + &format!("get: {case}"), + out.status.code(), + &doc, + &lock, + lock_text, + &member, + ); + } +} + /// #942: vlt reads its workspaces from `vlt.json` and keeps one /// `vlt-lock.json` at that root, so a hosted run from a member found the /// member's copy, read no lock, pinned nothing and exited 0. It now diff --git a/crates/socket-patch-core/src/hosted/governing_root.rs b/crates/socket-patch-core/src/hosted/governing_root.rs index d2c18ec54..8d3ccaf83 100644 --- a/crates/socket-patch-core/src/hosted/governing_root.rs +++ b/crates/socket-patch-core/src/hosted/governing_root.rs @@ -21,13 +21,15 @@ use std::path::{Path, PathBuf}; -use crate::constants::npm_family::VLT_LOCK; +use crate::constants::npm_family::{BUN_LOCK, BUN_LOCKB, VLT_LOCK}; use crate::formats::governing_locks::{npm_lock_files, NpmLockFamily}; use crate::formats::text::strip_bom; +use crate::formats::yarn::berry_gates::YARN_LOCK; +use crate::formats::yarn::is_berry_lock; use crate::patch::redirect::npmrc::npmrc_top_level_value; -use crate::utils::fs::{read_regular_to_string, read_regular_to_string_sync}; +use crate::utils::fs::{open_regular_file, read_regular_to_string, read_regular_to_string_sync}; use crate::utils::pnpm_workspace::governing_workspace_file; -use crate::utils::workspace_globs::workspaces_include; +use crate::utils::workspace_globs::{workspaces_include, workspaces_include_as, WorkspacesDialect}; use crate::vendor::cargo::NOT_WORKSPACE_ROOT; use crate::vendor::cargo_manifest; use crate::vendor::lock_inventory::ProjectView; @@ -494,7 +496,13 @@ async fn package_json_workspace_root(root: &Path) -> Option<(PathBuf, Vec<&'stat .components() .map(|c| c.as_os_str().to_string_lossy().into_owned()) .collect(); - if !workspaces_include(&patterns, &rel) { + // #1097: each package manager reads `workspaces` its own way. The + // widest reading goes first, so a root that cannot list the member + // costs no lock reads. + if !WorkspacesDialect::ALL + .iter() + .any(|&dialect| workspaces_include_as(&patterns, &rel, dialect)) + { continue; } let mut locks: Vec<&'static str> = WORKSPACE_ROOT_FAMILIES @@ -508,6 +516,19 @@ async fn package_json_workspace_root(root: &Path) -> Option<(PathBuf, Vec<&'stat if ancestor.join(VLT_LOCK).is_file() && !vlt_json_declares_workspaces(ancestor).await { locks.push(VLT_LOCK); } + // Only the locks whose package manager lists the member govern it; + // a root none of them lists the member under is not its root. + let found = !locks.is_empty(); + let mut governing = Vec::new(); + for lock in locks { + if workspaces_include_as(&patterns, &rel, lock_dialect(ancestor, lock).await) { + governing.push(lock); + } + } + if found && governing.is_empty() { + continue; + } + let locks = governing; if locks.is_empty() { // Rush keeps its locks under common/config: the rewriters own // a run from the Rush root. @@ -522,6 +543,58 @@ async fn package_json_workspace_root(root: &Path) -> Option<(PathBuf, Vec<&'stat None } +/// The `package.json` `workspaces` dialect of the package manager that +/// reads `lock` at `root` (#1097). yarn classic and berry tell apart by +/// berry's `__metadata` key ([`is_berry_lock`]; an empty or unreadable +/// `yarn.lock` reads as classic, the wider set), and Bun 1.3+ by the +/// top-level `configVersion` it writes into `bun.lock` (a lock without +/// it, or a binary `bun.lockb` with no `bun.lock` beside it, reads as +/// older Bun, the wider set). Only the head of a lock is read: both keys +/// come first. +async fn lock_dialect(root: &Path, lock: &str) -> WorkspacesDialect { + match lock { + YARN_LOCK => { + if is_berry_lock(&read_head(&root.join(lock)).await) { + WorkspacesDialect::YARN_BERRY + } else { + WorkspacesDialect::YARN_CLASSIC + } + } + BUN_LOCK => { + let head = read_head(&root.join(lock)).await; + if strip_bom(&head) + .lines() + .any(|line| line.starts_with(" \"configVersion\"")) + { + WorkspacesDialect::BUN + } else { + WorkspacesDialect::BUN_PRE_1_3 + } + } + // Bun reads the text lock when both exist. + BUN_LOCKB if root.join(BUN_LOCK).is_file() => Box::pin(lock_dialect(root, BUN_LOCK)).await, + BUN_LOCKB => WorkspacesDialect::BUN_PRE_1_3, + VLT_LOCK => WorkspacesDialect::VLT, + _ => WorkspacesDialect::NPM, + } +} + +/// How much of a lock [`lock_dialect`] reads: the yarn berry and Bun keys +/// it looks for sit in the first lines. +const LOCK_HEAD_BYTES: u64 = 4096; + +/// The first [`LOCK_HEAD_BYTES`] of a regular file, lossily decoded; empty +/// when it cannot be read or is not a regular file (a FIFO fails fast +/// instead of blocking in open(2), Bugbot on #1480). +async fn read_head(path: &Path) -> String { + use tokio::io::AsyncReadExt; + let mut head = Vec::new(); + if let Ok((file, _)) = open_regular_file(path).await { + let _ = file.take(LOCK_HEAD_BYTES).read_to_end(&mut head).await; + } + String::from_utf8_lossy(&head).into_owned() +} + /// #942: the project directory is a member of a vlt workspace, whose /// root `vlt.json` lists it under `workspaces` and holds `vlt-lock.json`. /// vlt reads workspaces only from `vlt.json` and keeps one lock at that @@ -1075,7 +1148,9 @@ mod tests { let member = tmp.path().join("packages/a"); // Lockless root. assert_eq!(code(&member, "npm").await, None); - write(tmp.path(), "yarn.lock", ""); + // A yarn berry lock: berry drops `!`-negated directories (yarn + // classic does not, #1097). + write(tmp.path(), "yarn.lock", "__metadata:\n version: 8\n"); assert_eq!( code(&member, "npm").await.as_deref(), Some(WORKSPACE_LOCKFILE_ELSEWHERE) @@ -1101,6 +1176,206 @@ mod tests { assert_eq!(code(&tmp.path().join("sub"), "npm").await, None); } + /// #1097: the root's lock decides how its `workspaces` read. yarn + /// classic, and Bun before 1.3 (a `bun.lock` without `configVersion`, + /// or a binary `bun.lockb`), ignore `!` patterns; npm, yarn berry and + /// Bun 1.3+ honour them. npm and yarn match extglobs; Bun refuses to + /// install with one, so its lock never lists such a member. + #[tokio::test] + async fn package_json_workspaces_read_in_the_root_locks_dialect() { + const CLASSIC: &str = "# yarn lockfile v1\n\n"; + const BERRY: &str = "__metadata:\n version: 8\n"; + const BUN_13: &str = "{\n \"lockfileVersion\": 1,\n \"configVersion\": 0,\n}\n"; + const BUN_12: &str = "{\n \"lockfileVersion\": 1,\n \"workspaces\": {},\n}\n"; + let negated = r#"["packages/*","!packages/b"]"#; + let refused = Some(WORKSPACE_LOCKFILE_ELSEWHERE); + for (workspaces, locks, expected) in [ + (negated, &[("yarn.lock", CLASSIC)][..], refused), + // An empty yarn.lock (never installed) reads as classic: the + // wider member set fails closed. + (negated, &[("yarn.lock", "")][..], refused), + (negated, &[("yarn.lock", BERRY)][..], None), + (negated, &[("package-lock.json", "{}")][..], None), + (negated, &[("npm-shrinkwrap.json", "{}")][..], None), + (negated, &[("bun.lock", BUN_12)][..], refused), + (negated, &[("bun.lock", BUN_13)][..], None), + (negated, &[("bun.lockb", "\0")][..], refused), + // Bun reads the text lock when both exist. + ( + negated, + &[("bun.lock", BUN_13), ("bun.lockb", "\0")][..], + None, + ), + // Any lock whose manager installs the member refuses. + ( + negated, + &[("package-lock.json", "{}"), ("yarn.lock", CLASSIC)][..], + refused, + ), + ( + r#"["packages/@(a|b)"]"#, + &[("package-lock.json", "{}")][..], + refused, + ), + ( + r#"["packages/+(b|c)"]"#, + &[("yarn.lock", CLASSIC)][..], + refused, + ), + (r#"["packages/!(z)"]"#, &[("yarn.lock", BERRY)][..], refused), + ( + r#"["packages/!(b)"]"#, + &[("package-lock.json", "{}")][..], + None, + ), + (r#"["packages/@(a|b)"]"#, &[("bun.lock", BUN_13)][..], None), + ] { + let tmp = tempfile::tempdir().unwrap(); + write( + tmp.path(), + "package.json", + &format!(r#"{{"private":true,"workspaces":{workspaces}}}"#), + ); + for (name, text) in locks { + write(tmp.path(), name, text); + } + write(tmp.path(), "packages/b/package.json", "{}"); + assert_eq!( + code(&tmp.path().join("packages/b"), "npm").await.as_deref(), + expected, + "{workspaces} with {locks:?}" + ); + } + } + + /// #1097 (review on #1480): only the locks whose package manager lists + /// the member govern it. The refusal names those alone, and a member's + /// own lock is stray only when its own manager lists the member. + #[tokio::test] + async fn only_the_locks_whose_manager_lists_the_member_govern_it() { + const CLASSIC: &str = "# yarn lockfile v1\n"; + const BUN_13: &str = "{\n \"lockfileVersion\": 1,\n \"configVersion\": 1,\n}\n"; + let tmp = tempfile::tempdir().unwrap(); + write( + tmp.path(), + "package.json", + r#"{"private":true,"workspaces":["packages/*","!packages/b"]}"#, + ); + write(tmp.path(), "package-lock.json", "{}"); + write(tmp.path(), "yarn.lock", CLASSIC); + write(tmp.path(), "packages/b/package.json", "{}"); + let member = tmp.path().join("packages/b"); + let refusal = refusal(&ProjectView::Disk(&member), &[candidate("npm")], true) + .await + .expect("yarn classic installs packages/b from the root"); + assert_eq!(refusal.code, WORKSPACE_LOCKFILE_ELSEWHERE); + assert!(refusal.message.contains("yarn.lock"), "{}", refusal.message); + assert!( + !refusal.message.contains("package-lock.json"), + "npm excludes packages/b: {}", + refusal.message + ); + + // npm excludes packages/b, so its own npm lock is the one npm + // reads, not a stray. + write(tmp.path(), "packages/b/package-lock.json", "{}"); + assert_eq!(member_stray_lock(&member).await, None); + // Likewise Bun 1.3 and vlt (no `vlt.json` workspaces). + std::fs::remove_file(member.join("package-lock.json")).unwrap(); + std::fs::remove_file(tmp.path().join("package-lock.json")).unwrap(); + write(tmp.path(), "bun.lock", BUN_13); + write(tmp.path(), "packages/b/bun.lock", BUN_13); + assert_eq!(member_stray_lock(&member).await, None); + std::fs::remove_file(member.join("bun.lock")).unwrap(); + std::fs::remove_file(tmp.path().join("bun.lock")).unwrap(); + write(tmp.path(), "vlt-lock.json", "{}"); + write(tmp.path(), "packages/b/vlt-lock.json", "{}"); + assert_eq!(member_stray_lock(&member).await, None); + // Under a classic yarn.lock alone the member is listed, and its own + // npm lock is not npm's to ignore. + std::fs::remove_file(tmp.path().join("vlt-lock.json")).unwrap(); + std::fs::remove_file(member.join("vlt-lock.json")).unwrap(); + write(tmp.path(), "package-lock.json", "{}"); + write(tmp.path(), "packages/a/package.json", "{}"); + write(tmp.path(), "packages/a/package-lock.json", "{}"); + assert!(member_stray_lock(&tmp.path().join("packages/a")) + .await + .is_some()); + } + + /// Bugbot on #1480: the lock head is read through the FIFO-safe + /// opener, so a FIFO reads as empty instead of hanging the run. + #[cfg(unix)] + #[tokio::test] + async fn lock_head_read_does_not_block_on_a_fifo() { + let tmp = tempfile::tempdir().unwrap(); + let fifo = tmp.path().join("yarn.lock"); + let status = std::process::Command::new("mkfifo") + .arg(&fifo) + .status() + .unwrap(); + assert!(status.success()); + let head = tokio::time::timeout(std::time::Duration::from_secs(10), read_head(&fifo)) + .await + .expect("read_head must not block on a FIFO"); + assert_eq!(head, ""); + write(tmp.path(), "bun.lock", "{\n \"configVersion\": 1,\n}\n"); + assert!(read_head(&tmp.path().join("bun.lock")) + .await + .contains("configVersion")); + } + + /// #1097 (review on #1480): only a top-level `configVersion` marks Bun + /// 1.3+; the same key deeper in a pre-1.3 lock does not. + #[tokio::test] + async fn bun_config_version_is_read_at_the_top_level_only() { + let tmp = tempfile::tempdir().unwrap(); + write( + tmp.path(), + "package.json", + r#"{"private":true,"workspaces":["packages/*","!packages/b"]}"#, + ); + write( + tmp.path(), + "bun.lock", + "{\n \"lockfileVersion\": 1,\n \"workspaces\": {\n \"\": {\n \"configVersion\": \"1\",\n },\n },\n}\n", + ); + write(tmp.path(), "packages/b/package.json", "{}"); + assert_eq!( + code(&tmp.path().join("packages/b"), "npm").await.as_deref(), + Some(WORKSPACE_LOCKFILE_ELSEWHERE) + ); + } + + /// #1097: a lockless intermediate root reads its `workspaces` in the + /// widest dialect, so a member it lists only through an extglob, or + /// excludes only with `!`, still reaches the outer root's lock. + #[tokio::test] + async fn lockless_intermediate_root_reads_the_widest_dialect() { + for inner in [r#"["@(a|b)"]"#, r#"["*","!b"]"#] { + let tmp = tempfile::tempdir().unwrap(); + write( + tmp.path(), + "package.json", + r#"{"private":true,"workspaces":["packages/*"]}"#, + ); + write(tmp.path(), "yarn.lock", "# yarn lockfile v1\n"); + write( + tmp.path(), + "packages/inner/package.json", + &format!(r#"{{"private":true,"workspaces":{inner}}}"#), + ); + write(tmp.path(), "packages/inner/b/package.json", "{}"); + assert_eq!( + code(&tmp.path().join("packages/inner/b"), "npm") + .await + .as_deref(), + Some(WORKSPACE_LOCKFILE_ELSEWHERE), + "{inner}" + ); + } + } + /// #1094: npm installs every workspace member from the root's lock and /// never reads a `package-lock.json` / `npm-shrinkwrap.json` inside the /// member, so a stray member npm lock does not make the member its own diff --git a/crates/socket-patch-core/src/utils/workspace_globs.rs b/crates/socket-patch-core/src/utils/workspace_globs.rs index bd16d629a..8636851f3 100644 --- a/crates/socket-patch-core/src/utils/workspace_globs.rs +++ b/crates/socket-patch-core/src/utils/workspace_globs.rs @@ -9,22 +9,125 @@ //! ignored. A leading `!` negates; each caller applies negations the way //! its package manager does. pnpm's globber also never lets a wildcard //! match a component that starts with `.` ([`glob_matches_no_dot`]). +//! npm and yarn also match extglobs (`@(a|b)`, `?(..)`, `*(..)`, `+(..)`, +//! `!(..)`) within one component, and yarn classic ignores `!` patterns; +//! [`WorkspacesDialect`] names what each package manager reads (#1097). + +/// How one package manager reads `package.json` (or `vlt.json`) +/// `workspaces`, probed on npm 10.9, yarn 1.22.22, yarn 4.10, Bun 1.2.23 +/// and Bun 1.4.2 (#1097). +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) struct WorkspacesDialect { + /// What a pattern with a leading `!` does. + pub(crate) negation: Negation, + /// Extglob groups match (npm's minimatch, yarn's node-glob and + /// micromatch). Off, they are literal characters (Bun fails its + /// install on one, so its lock never lists such a member). + pub(crate) extglob: bool, +} + +/// What a `workspaces` pattern with a leading `!` does. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum Negation { + /// It excludes what earlier patterns matched; a later pattern can + /// list the directory again (npm, Bun 1.3+, vlt). + LastMatchWins, + /// It excludes the directory whatever the order (yarn berry's + /// globby), and a leading `!(` is an extglob group, not a negation. + Unordered, + /// It is ignored: it neither excludes nor lists anything (yarn + /// classic, Bun before 1.3). + Ignored, +} + +impl WorkspacesDialect { + /// npm (`package-lock.json`, `npm-shrinkwrap.json`). + pub(crate) const NPM: Self = Self { + negation: Negation::LastMatchWins, + extglob: true, + }; + /// yarn classic (a `yarn.lock` without berry's `__metadata`). + pub(crate) const YARN_CLASSIC: Self = Self { + negation: Negation::Ignored, + extglob: true, + }; + /// yarn berry (a `yarn.lock` with `__metadata`). + pub(crate) const YARN_BERRY: Self = Self { + negation: Negation::Unordered, + extglob: true, + }; + /// Bun 1.3 and later (a `bun.lock` with `configVersion`). + pub(crate) const BUN: Self = Self { + negation: Negation::LastMatchWins, + extglob: false, + }; + /// Bun before 1.3: it lists `!`-negated directories anyway. + pub(crate) const BUN_PRE_1_3: Self = Self { + negation: Negation::Ignored, + extglob: false, + }; + /// vlt, as #942 read it. + pub(crate) const VLT: Self = Self { + negation: Negation::LastMatchWins, + extglob: false, + }; + /// Every dialect, for a root whose package manager is unknown: it lists + /// a directory when any of them does. + pub(crate) const ALL: [Self; 6] = [ + Self::NPM, + Self::YARN_CLASSIC, + Self::YARN_BERRY, + Self::BUN, + Self::BUN_PRE_1_3, + Self::VLT, + ]; +} /// Whether the member path (`rel`, relative to the workspace root, one /// entry per component) matches a `workspaces` pattern and no later -/// `!`-negated one (npm and yarn: the last matching pattern wins). +/// `!`-negated one (npm and yarn: the last matching pattern wins), in vlt's +/// dialect. pub(crate) fn workspaces_include(patterns: &[String], rel: &[String]) -> bool { + workspaces_include_as(patterns, rel, WorkspacesDialect::VLT) +} + +/// [`workspaces_include`] in a given package manager's dialect. +pub(crate) fn workspaces_include_as( + patterns: &[String], + rel: &[String], + dialect: WorkspacesDialect, +) -> bool { if rel.is_empty() { return false; } - let mut included = false; - for pattern in patterns { - let (negated, pattern) = split_negation(pattern); - if glob_matches(pattern, rel) { - included = !negated; + let matches = |pattern: &str| pattern_matches(pattern, rel, true, dialect.extglob); + match dialect.negation { + Negation::LastMatchWins => { + let mut included = false; + for pattern in patterns { + let (negated, pattern) = split_negation(pattern); + if matches(pattern) { + included = !negated; + } + } + included } + Negation::Unordered => { + let (mut listed, mut excluded) = (false, false); + for pattern in patterns { + match split_negation(pattern) { + (true, rest) if !(dialect.extglob && rest.starts_with('(')) => { + excluded |= matches(rest) + } + _ => listed |= matches(pattern), + } + } + listed && !excluded + } + Negation::Ignored => patterns + .iter() + .any(|pattern| !pattern.starts_with('!') && matches(pattern)), } - included } /// `(true, rest)` for a `!`-negated pattern, `(false, pattern)` otherwise. @@ -38,7 +141,7 @@ pub(crate) fn split_negation(pattern: &str) -> (bool, &str) { /// Whether one (un-negated) pattern matches `rel`. A pattern with no /// components left (`""`, `.`, `./`) matches nothing. pub(crate) fn glob_matches(pattern: &str, rel: &[String]) -> bool { - pattern_matches(pattern, rel, true) + pattern_matches(pattern, rel, true, false) } /// [`glob_matches`] the way pnpm's globber reads `packages:` (`dot: false`, @@ -47,10 +150,10 @@ pub(crate) fn glob_matches(pattern: &str, rel: &[String]) -> bool { /// `packages/*` does not list `packages/.hidden`. A pattern component that /// itself starts with `.` (`.github/**`) still matches it. pub(crate) fn glob_matches_no_dot(pattern: &str, rel: &[String]) -> bool { - pattern_matches(pattern, rel, false) + pattern_matches(pattern, rel, false, false) } -fn pattern_matches(pattern: &str, rel: &[String], dot: bool) -> bool { +fn pattern_matches(pattern: &str, rel: &[String], dot: bool, extglob: bool) -> bool { let rel: Vec> = rel.iter().map(|c| c.chars().collect()).collect(); expand_braces(pattern.trim()).iter().any(|alternative| { let segments: Vec> = alternative @@ -58,7 +161,7 @@ fn pattern_matches(pattern: &str, rel: &[String], dot: bool) -> bool { .filter(|s| !s.is_empty() && *s != ".") .map(|s| s.chars().collect()) .collect(); - !segments.is_empty() && path_glob_matches(&segments, &rel, dot) + !segments.is_empty() && path_glob_matches(&segments, &rel, dot, extglob) }) } @@ -197,21 +300,26 @@ fn brace_sequence(body: &str) -> Option> { } /// `dot: false` keeps wildcards off components starting with `.`. -fn path_glob_matches(pattern: &[Vec], path: &[Vec], dot: bool) -> bool { +fn path_glob_matches(pattern: &[Vec], path: &[Vec], dot: bool, ext: bool) -> bool { let hidden = |component: &Vec| !dot && component.first() == Some(&'.'); match pattern.split_first() { None => path.is_empty(), Some((first, rest)) if first.as_slice() == ['*', '*'] => (0..=path.len()) .take_while(|&skip| skip == 0 || !hidden(&path[skip - 1])) - .any(|skip| path_glob_matches(rest, &path[skip..], dot)), + .any(|skip| path_glob_matches(rest, &path[skip..], dot, ext)), Some((first, rest)) => path.split_first().is_some_and(|(head, tail)| { (!hidden(head) || first.first() == Some(&'.')) - && segment_glob_matches(first, head) - && path_glob_matches(rest, tail, dot) + && if ext { + ExtglobMatcher::default().segment(first, head) + } else { + segment_glob_matches(first, head) + } + && path_glob_matches(rest, tail, dot, ext) }), } } +/// Whether one component matches a component pattern, without extglobs. pub(crate) fn segment_glob_matches(pattern: &[char], name: &[char]) -> bool { match pattern.split_first() { None => name.is_empty(), @@ -227,6 +335,153 @@ pub(crate) fn segment_glob_matches(pattern: &[char], name: &[char]) -> bool { } } +/// One component matched against a pattern with extglob groups (`@(a|b)`, +/// `?(..)`, `*(..)`, `+(..)`, `!(..)`), as minimatch reads them. Every +/// (sub-pattern, name suffix) pair is decided once, so an ambiguous group +/// (`*(a|aa)`) costs polynomial time, not exponential. +#[derive(Default)] +struct ExtglobMatcher { + memo: std::collections::HashMap<(usize, usize, usize, usize), bool>, +} + +impl ExtglobMatcher { + fn segment(&mut self, pattern: &[char], name: &[char]) -> bool { + let key = ( + pattern.as_ptr() as usize, + pattern.len(), + name.as_ptr() as usize, + name.len(), + ); + if let Some(&known) = self.memo.get(&key) { + return known; + } + let matched = self.segment_uncached(pattern, name); + self.memo.insert(key, matched); + matched + } + + fn segment_uncached(&mut self, pattern: &[char], name: &[char]) -> bool { + if let Some(group) = extglob_group(pattern) { + return self.group(&group, name); + } + match pattern.split_first() { + None => name.is_empty(), + Some(('*', rest)) => (0..=name.len()).any(|skip| self.segment(rest, &name[skip..])), + Some(('?', rest)) => !name.is_empty() && self.segment(rest, &name[1..]), + Some(('[', rest)) => match char_class(rest) { + Some((class, after)) => name + .split_first() + .is_some_and(|(c, tail)| class.matches(*c) && self.segment(after, tail)), + None => name.first() == Some(&'[') && self.segment(rest, &name[1..]), + }, + Some((c, rest)) => name.first() == Some(c) && self.segment(rest, &name[1..]), + } + } + + fn group(&mut self, group: &ExtglobGroup<'_>, name: &[char]) -> bool { + let rest = group.rest; + match group.kind { + '@' => self.alternative_then(group, name, |m, tail| m.segment(rest, tail)), + '?' => { + self.segment(rest, name) + || self.alternative_then(group, name, |m, tail| m.segment(rest, tail)) + } + '*' => self.repeat(group, name), + '+' => self.alternative_then(group, name, |m, tail| m.repeat(group, tail)), + // minimatch: `!(a|b)rest` matches when no alternative followed + // by `rest` matches the whole name, and some prefix followed + // by `rest` does. + _ => { + !self.alternative_then(group, name, |m, tail| m.segment(rest, tail)) + && (0..=name.len()).any(|skip| self.segment(rest, &name[skip..])) + } + } + } + + /// Some alternative matches a prefix of `name` and `then` the rest. + fn alternative_then( + &mut self, + group: &ExtglobGroup<'_>, + name: &[char], + then: impl Fn(&mut Self, &[char]) -> bool, + ) -> bool { + (0..=name.len()).any(|split| { + group + .alternatives + .iter() + .any(|alt| self.segment(alt, &name[..split])) + && then(self, &name[split..]) + }) + } + + /// Zero or more non-empty alternatives, then the group's `rest`. + fn repeat(&mut self, group: &ExtglobGroup<'_>, name: &[char]) -> bool { + // A group's `rest` starts right after its own `)`, so its address + // names the group within the pattern. + let key = ( + group.rest.as_ptr() as usize, + usize::MAX, + name.as_ptr() as usize, + name.len(), + ); + if let Some(&known) = self.memo.get(&key) { + return known; + } + let matched = self.segment(group.rest, name) + || (1..=name.len()).any(|split| { + group + .alternatives + .iter() + .any(|alt| self.segment(alt, &name[..split])) + && self.repeat(group, &name[split..]) + }); + self.memo.insert(key, matched); + matched + } +} + +/// An extglob group at the start of a component pattern: its kind +/// (`@`, `?`, `*`, `+`, `!`), its `|`-separated alternatives and the +/// pattern after its `)`. +struct ExtglobGroup<'a> { + kind: char, + alternatives: Vec<&'a [char]>, + rest: &'a [char], +} + +/// The group starting `pattern`, or `None` when it starts no group or the +/// group never closes (its characters are then read as usual, as in +/// minimatch). +fn extglob_group(pattern: &[char]) -> Option> { + let kind = *pattern.first()?; + if !matches!(kind, '@' | '?' | '*' | '+' | '!') || pattern.get(1) != Some(&'(') { + return None; + } + let body = &pattern[2..]; + let (mut depth, mut start) = (0usize, 0usize); + let mut alternatives = Vec::new(); + for (i, &c) in body.iter().enumerate() { + match c { + '(' => depth += 1, + ')' if depth == 0 => { + alternatives.push(&body[start..i]); + return Some(ExtglobGroup { + kind, + alternatives, + rest: &body[i + 1..], + }); + } + ')' => depth -= 1, + '|' if depth == 0 => { + alternatives.push(&body[start..i]); + start = i + 1; + } + _ => {} + } + } + None +} + /// A parsed `[...]` character class. struct CharClass { negated: bool, @@ -354,6 +609,164 @@ mod tests { )); } + /// #1097: npm (minimatch) and yarn (classic node-glob, berry + /// micromatch) match extglobs in `workspaces`; probed on npm 10.9, + /// yarn 1.22.22 and yarn 4.10. + #[test] + fn extglob_dialects_match_like_npm_and_yarn() { + let rel = |p: &str| p.split('/').map(str::to_string).collect::>(); + let pats = |p: &[&str]| p.iter().map(|s| s.to_string()).collect::>(); + let npm = WorkspacesDialect::NPM; + let yes = |p: &str, r: &str| { + assert!( + workspaces_include_as(&pats(&[p]), &rel(r), npm), + "{p} ~ {r}" + ) + }; + let no = |p: &str, r: &str| { + assert!( + !workspaces_include_as(&pats(&[p]), &rel(r), npm), + "{p} !~ {r}" + ) + }; + yes("packages/@(a|b)", "packages/a"); + yes("packages/@(a|b)", "packages/b"); + no("packages/@(a|b)", "packages/c"); + no("packages/@(a|b)", "packages/ab"); + yes("packages/+(a|c)", "packages/c"); + yes("packages/+(a|c)", "packages/aca"); + no("packages/+(a|c)", "packages/b"); + yes("packages/*(a|b)", "packages/ab"); + no("packages/*(a|b)", "packages/c"); + yes("packages/?(a)", "packages/a"); + no("packages/?(a)", "packages/aa"); + yes("packages/!(z)", "packages/a"); + yes("packages/!(z)", "packages/zz"); + no("packages/!(z)", "packages/z"); + yes("packages/!(z)-x", "packages/a-x"); + no("packages/!(z)-x", "packages/z-x"); + // Globs, classes and nesting inside the alternatives. + yes("packages/@(pkg-*|[xy])", "packages/pkg-one"); + yes("packages/@(pkg-*|[xy])", "packages/y"); + yes("packages/@(a|@(b|c))d", "packages/cd"); + yes("packages/pre-@(a|b)-post", "packages/pre-b-post"); + // An unclosed group is literal; braces expand around a group. + yes("packages/@(a", "packages/@(a"); + yes("{apps,packages}/@(a|b)", "apps/b"); + // Without extglobs (Bun, vlt, uv, pnpm) the group is literal. + assert!(!workspaces_include( + &pats(&["packages/@(a|b)"]), + &rel("packages/a") + )); + assert!(!glob_matches("packages/@(a|b)", &rel("packages/a"))); + assert!(!workspaces_include_as( + &pats(&["packages/@(a|b)"]), + &rel("packages/a"), + WorkspacesDialect::BUN + )); + } + + /// #1097: yarn classic and Bun before 1.3 ignore `!` patterns, so a + /// negated directory the positive patterns match stays a member; npm + /// and Bun 1.3+ drop it unless a later pattern lists it again, and + /// yarn berry drops it whatever the order. Probed on npm 10.9, yarn + /// 1.22.22, yarn 4.10, Bun 1.2.23 and Bun 1.4.2. + #[test] + fn negation_is_per_dialect() { + use WorkspacesDialect as D; + let rel = |p: &str| p.split('/').map(str::to_string).collect::>(); + let pats = |p: &[&str]| p.iter().map(|s| s.to_string()).collect::>(); + let negated_last = pats(&["packages/*", "!packages/b"]); + let negated_first = pats(&["!packages/b", "packages/*"]); + let relisted = pats(&["packages/*", "!packages/b", "packages/b"]); + for (dialect, last, first, again) in [ + (D::NPM, false, true, true), + (D::BUN, false, true, true), + (D::VLT, false, true, true), + (D::YARN_BERRY, false, false, false), + (D::YARN_CLASSIC, true, true, true), + (D::BUN_PRE_1_3, true, true, true), + ] { + let b = rel("packages/b"); + assert_eq!( + workspaces_include_as(&negated_last, &b, dialect), + last, + "{dialect:?}" + ); + assert_eq!( + workspaces_include_as(&negated_first, &b, dialect), + first, + "{dialect:?}" + ); + assert_eq!( + workspaces_include_as(&relisted, &b, dialect), + again, + "{dialect:?}" + ); + assert!(workspaces_include_as( + &negated_last, + &rel("packages/a"), + dialect + )); + } + // A leading `!(` is an extglob to yarn berry (every top-level + // directory but `internal`), a negation of `(internal)` to npm, + // and lists nothing to yarn classic. + let bang_group = pats(&["!(internal)"]); + assert!(workspaces_include_as( + &bang_group, + &rel("tools"), + D::YARN_BERRY + )); + assert!(!workspaces_include_as( + &bang_group, + &rel("internal"), + D::YARN_BERRY + )); + assert!(!workspaces_include_as(&bang_group, &rel("tools"), D::NPM)); + assert!(!workspaces_include_as( + &bang_group, + &rel("tools"), + D::YARN_CLASSIC + )); + // A lone negation lists nothing in any dialect. + for dialect in D::ALL { + assert!( + !workspaces_include_as(&pats(&["!packages/b"]), &rel("packages/b"), dialect), + "{dialect:?}" + ); + } + } + + /// An ambiguous extglob decides in polynomial time (a review finding + /// on #1480: plain backtracking took ~2^40 steps here). + #[test] + fn ambiguous_extglobs_stay_fast() { + let rel = |p: &str| p.split('/').map(str::to_string).collect::>(); + let long = format!("packages/{}b", "a".repeat(60)); + for pattern in [ + "packages/*(a|aa)c", + "packages/+(*)x", + "packages/!(*(a|aa))b", + ] { + let started = std::time::Instant::now(); + assert!(!workspaces_include_as( + &[pattern.to_string()], + &rel(&long), + WorkspacesDialect::NPM + )); + assert!( + started.elapsed() < std::time::Duration::from_secs(5), + "{pattern}" + ); + } + assert!(workspaces_include_as( + &["packages/*(a|aa)b".to_string()], + &rel(&long), + WorkspacesDialect::NPM + )); + } + #[test] fn pnpm_wildcards_skip_dot_components() { let rel = |p: &str| p.split('/').map(str::to_string).collect::>();