openai-agents-python/tests/sandbox/_apply_patch_test_session.py
WolfGanG 71306bffa7
fix: keep the file when apply_patch does a case-only rename (#4890)
* fix: keep the file when apply_patch does a case-only rename

On a filesystem that folds case, `notes.txt` and `Notes.txt` are the same
file. `_apply_update` wrote the new text to the destination and then removed
the source, and because the path comparison is case-sensitive the removal
deleted the file that had just been written. The user's edit was lost.

For a case-only rename, remove the source before writing instead. That end
state is correct on a folding filesystem and on a case-sensitive one, so the
SDK does not have to know which kind it is talking to. The write is wrapped
so a failure restores the original text at the source path.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: commit the rename before removing the source

The previous commit removed the source first on a case-only rename and
restored the original text from memory if the write then failed. That trades
one way to lose the file for another: if the write and the restore both fail
the file is gone, and a restore that lands late overwrites whatever another
writer put at the source path in the meantime.

Ask the filesystem instead of comparing strings. `same_file` runs
`[ "$1" -ef "$2" ]`, which compares device and inode, so it answers for a
filesystem that folds case and for one that folds Unicode normalisation, which
`str.casefold` cannot. `mv` renames a path and refuses a directory
destination, because `mv` given a directory moves the source inside it and
exits 0, which for a caller that then removes the source is a way to delete a
file while believing it moved.

The rename now writes a staging file, commits it onto the destination with one
move, and removes the source only when the filesystem says it is a different
file. Nothing is removed before the new content is on disk, and nothing is
written back after a failure.

Three of the new tests run on the macOS runner against a real case-folding
volume, so the behaviour is no longer only modelled.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: answer identity without following symlinks, and stage safely

Acts on five of the six findings the Codex reviewer raised on 5b3650ca.

`test -ef` follows symlinks, so a source symlink pointing at the destination
answered "same file" and the removal was skipped, leaving the old name pointing
at the new one. `same_file` takes `follow_symlinks` now, and the editor asks
with it off, because the answer decides whether removing one path destroys the
other.

The staging write moves inside the `try`, so a write that fails after creating
the file no longer orphans it. The staging basename is a fixed length, because
decorating a destination basename near the 255-byte limit overflowed it. A
`move_to` naming the path the file already has short-circuits to an in-place
write, which is what an update without `move_to` does and what this code did
before the rename was staged.

`docs/testing.md` is reverted: AGENTS.md keeps documentation for unreleased
behaviour out of the pull request that introduces it.

The directory check is still not atomic with the rename. Closing that needs a
per-backend rename primitive, which is a question for the maintainer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: say what the removal does not guarantee

The docstring said a file another writer creates at the source path is never
overwritten. That is true and was standing in for a guarantee it does not make:
the identity answer is read before the removal, and the removal names a path
rather than the entry that answer was about, so such a file can still be
removed.

No behaviour change. Closing the window needs a removal that can be told which
entry it may remove, which is the per-backend question already open with the
maintainer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: complete case-only renames on folding filesystems

* fix: skip redundant moves through path aliases

* fix: ask the sandbox, not the host, whether two paths are one file

`Path` equality folds case on a Windows host, so a case-only `move_to` took the
same-path fast path there. The update was written in place, the operation
reported success, and the requested name was never created in a case-sensitive
sandbox. The host's filesystem semantics say nothing about the sandbox's.

Comparing the canonical spellings keeps the fast path for a genuine no-op and
sends a case-only rename down the staging-and-move path, where `same_file` asks
the sandbox whether the two paths are one entry.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: say what committing a new inode costs an existing move target

A staged commit replaces the destination inode, so a rename that lands on an
existing distinct file replaces that file's mode, ownership and extended
attributes, where writing into it in place kept them. The docstring covered
only the case-folding entry. Preserving them would cost an existence probe and
the single-`mv` commit for that case; the content is worth more than the bits.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: rename and compare entries descriptor-relative on UnixLocal

Since #4931, every UnixLocal file operation runs against a directory
descriptor so the path that was validated is the entry acted on. The
rename and identity check this branch added still went through `sh -lc`,
which re-resolves the path and reopens the window #4931 closed.

`_FileOps` gains `rename`, an `os.rename` with `src_dir_fd` and
`dst_dir_fd`, and `same_file`, a descriptor-relative `stat` compared by
device and inode. UnixLocal overrides `mv` and `same_file` with them,
directly for the host user and through the file worker for a bound user.
`os.rename` also refuses an existing directory as the destination, where
`mv` would have moved the source inside it. The shell versions stay on
`BaseSandboxSession` for backends that only offer `exec`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Restrict sandbox moves to files and symlinks

* fix: preserve diverged source after alias patch commit

* ci: allow Windows tests time to finish cleanup

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Justin Beckwith <jbeckwith@openai.com>
2026-09-28 08:08:51 -07:00

527 lines
18 KiB
Python

from __future__ import annotations
import io
import unicodedata
import uuid
from pathlib import Path, PurePath, PurePosixPath
from typing import cast
from agents.sandbox import Manifest
from agents.sandbox.errors import WorkspaceReadNotFoundError
from agents.sandbox.session.base_sandbox_session import BaseSandboxSession
from agents.sandbox.snapshot import NoopSnapshot
from agents.sandbox.types import ExecResult, User
from tests.utils.factories import TestSessionState
class ApplyPatchSession(BaseSandboxSession):
def __init__(self, manifest: Manifest | None = None) -> None:
self.state = TestSessionState(
manifest=manifest or Manifest(root="/workspace"),
snapshot=NoopSnapshot(id=str(uuid.uuid4())),
)
self.files: dict[Path, bytes] = {}
self.mkdir_calls: list[tuple[Path, bool]] = []
self.rm_calls: list[tuple[Path, bool]] = []
self.mv_calls: list[tuple[Path, Path]] = []
# Link path -> target path, for the paths a test declares to be symlinks.
self.symlinks: dict[Path, Path] = {}
self.directories: set[Path] = set()
def _stored_path(self, path: Path | str) -> Path:
"""Return the key this store holds `path` under.
A store that folds names overrides this. Here every distinct spelling is a distinct
file, which is what a case-sensitive filesystem does.
"""
return self.normalize_path(path)
async def start(self) -> None:
return None
async def stop(self) -> None:
return None
async def shutdown(self) -> None:
return None
async def running(self) -> bool:
return True
async def read(self, path: Path, *, user: str | User | None = None) -> io.BytesIO:
_ = user
normalized = self.normalize_path(path)
if normalized not in self.files:
raise FileNotFoundError(normalized)
return io.BytesIO(self.files[normalized])
async def write(
self,
path: Path,
data: io.IOBase,
*,
user: str | User | None = None,
) -> None:
_ = user
normalized = self.normalize_path(path)
if normalized in self.directories:
raise IsADirectoryError(normalized)
payload = data.read()
if isinstance(payload, str):
self.files[normalized] = payload.encode("utf-8")
else:
self.files[normalized] = bytes(payload)
async def _write_new_file(
self,
path: Path,
data: io.IOBase,
*,
user: str | User | None = None,
) -> None:
normalized = self.normalize_path(path)
if normalized in self.files:
raise FileExistsError(str(normalized))
# Real backends create the parents inside the primitive, so record that here too.
await self.mkdir(normalized.parent, parents=True, user=user)
await self.write(path, data, user=user)
async def _exec_internal(
self,
*command: str | Path,
timeout: float | None = None,
) -> ExecResult:
_ = (command, timeout)
raise AssertionError("_exec_internal() should not be called")
async def persist_workspace(self) -> io.IOBase:
return io.BytesIO()
async def hydrate_workspace(self, data: io.IOBase) -> None:
_ = data
async def mkdir(
self,
path: Path | str,
*,
parents: bool = False,
user: str | User | None = None,
) -> None:
_ = user
normalized = self.normalize_path(path)
self.mkdir_calls.append((normalized, parents))
async def rm(
self,
path: Path | str,
*,
recursive: bool = False,
user: str | User | None = None,
) -> None:
_ = user
normalized = self.normalize_path(path)
self.rm_calls.append((normalized, recursive))
self.files.pop(normalized, None)
async def mv(
self,
source: Path | str,
destination: Path | str,
*,
user: str | User | None = None,
) -> None:
_ = user
if self.normalize_path(destination) in self.directories:
# A real `mv` would move the source inside this directory and report success.
raise IsADirectoryError(self.normalize_path(destination))
stored_source = self._stored_path(source)
if stored_source not in self.files:
raise FileNotFoundError(stored_source)
stored_destination = self._stored_path(destination)
destination_exists = stored_destination in self.files
payload = self.files.pop(stored_source)
normalized_destination = self.normalize_path(destination)
if destination_exists and stored_destination != stored_source:
# APFS keeps an existing entry's spelling when a different inode replaces it
# through a case-variant path. A later rename of that same entry changes the case.
self.files[stored_destination] = payload
else:
self.files.pop(stored_destination, None)
self.files[normalized_destination] = payload
self.mv_calls.append((stored_source, normalized_destination))
async def same_file(
self,
left: Path | str,
right: Path | str,
*,
follow_symlinks: bool = True,
user: str | User | None = None,
) -> bool:
_ = user
if not follow_symlinks and (
self._stored_path(left) in self.symlinks or self._stored_path(right) in self.symlinks
):
return False
return self._resolved_path(left) == self._resolved_path(right)
def _resolved_path(self, path: Path | str) -> Path:
stored = self._stored_path(path)
seen: set[Path] = set()
while stored in self.symlinks and stored not in seen:
seen.add(stored)
stored = self._stored_path(self.symlinks[stored])
return stored
class PosixHostApplyPatchSession(ApplyPatchSession):
"""An apply_patch session whose workspace paths compare case-sensitively on every host.
Linux and macOS hosts compare sandbox paths case-sensitively, while a Windows host folds
case in `Path` comparisons. `PurePosixPath` keeps the host comparison case-sensitive
everywhere so case-only rename coverage does not depend on the operating system that runs
the tests.
"""
def normalize_path(self, path: Path | str, *, for_write: bool = False) -> Path:
normalized = super().normalize_path(path, for_write=for_write)
return cast(Path, PurePosixPath(normalized.as_posix()))
class _CaseFoldingHostPath(PurePosixPath):
"""A pure path that compares case-insensitively, as `WindowsPath` does on a Windows host.
`Path("/workspace/notes.txt") == Path("/workspace/Notes.txt")` is `True` on Windows and
`False` everywhere else. Modelling that here rather than reading `sys.platform` keeps the
coverage on every host that runs the suite.
"""
def __eq__(self, other: object) -> bool:
if isinstance(other, PurePath):
return self.as_posix().casefold() == other.as_posix().casefold()
return NotImplemented
def __hash__(self) -> int:
return hash(self.as_posix().casefold())
class CaseFoldingHostApplyPatchSession(PosixHostApplyPatchSession):
"""A host that folds case in path comparisons, over a sandbox store that does not.
This is the SDK running on Windows against a Linux container. The host's own filesystem
says nothing about the sandbox's, so a case-only `move_to` is a real rename that the
sandbox must perform. The store below keeps every spelling apart; only `normalize_path`
hands back paths that fold.
"""
def normalize_path(self, path: Path | str, *, for_write: bool = False) -> Path:
normalized = super().normalize_path(path, for_write=for_write)
return cast(Path, _CaseFoldingHostPath(normalized.as_posix()))
def _stored_path(self, path: Path | str) -> Path:
return cast(Path, PurePosixPath(self.normalize_path(path).as_posix()))
async def read(self, path: Path, *, user: str | User | None = None) -> io.BytesIO:
_ = user
stored = self._stored_path(path)
if stored not in self.files:
raise FileNotFoundError(stored)
return io.BytesIO(self.files[stored])
async def write(
self,
path: Path,
data: io.IOBase,
*,
user: str | User | None = None,
) -> None:
_ = user
payload = data.read()
stored = self._stored_path(path)
if isinstance(payload, str):
self.files[stored] = payload.encode("utf-8")
else:
self.files[stored] = bytes(payload)
async def rm(
self,
path: Path | str,
*,
recursive: bool = False,
user: str | User | None = None,
) -> None:
_ = user
stored = self._stored_path(path)
self.rm_calls.append((stored, recursive))
self.files.pop(stored, None)
async def mv(
self,
source: Path | str,
destination: Path | str,
*,
user: str | User | None = None,
) -> None:
_ = user
stored_source = self._stored_path(source)
if stored_source not in self.files:
raise FileNotFoundError(stored_source)
stored_destination = self._stored_path(destination)
self.files[stored_destination] = self.files.pop(stored_source)
self.mv_calls.append((stored_source, stored_destination))
class CaseFoldingApplyPatchSession(PosixHostApplyPatchSession):
"""A case-sensitive host over a store that models case-folding APFS.
Case-folding APFS stores `notes.txt` and `Notes.txt` as one file. Replacing that entry
through a case-variant path keeps its stored name, while moving the entry itself changes
the spelling. Lookups here fold case so the double follows that measured behavior.
"""
def _stored_path(self, path: Path | str) -> Path:
normalized = self.normalize_path(path)
folded = normalized.as_posix().casefold()
for stored in self.files:
if stored.as_posix().casefold() == folded:
return stored
return normalized
async def read(self, path: Path, *, user: str | User | None = None) -> io.BytesIO:
return await super().read(self._stored_path(path), user=user)
async def write(
self,
path: Path,
data: io.IOBase,
*,
user: str | User | None = None,
) -> None:
await super().write(self._stored_path(path), data, user=user)
async def rm(
self,
path: Path | str,
*,
recursive: bool = False,
user: str | User | None = None,
) -> None:
await super().rm(self._stored_path(path), recursive=recursive, user=user)
class ParentAliasApplyPatchSession(PosixHostApplyPatchSession):
"""A store where `/workspace/alias` resolves to `/workspace/real`."""
def _stored_path(self, path: Path | str) -> Path:
normalized = cast(PurePosixPath, self.normalize_path(path))
alias = PurePosixPath("/workspace/alias")
try:
relative = normalized.relative_to(alias)
except ValueError:
return cast(Path, normalized)
return cast(Path, PurePosixPath("/workspace/real") / relative)
async def read(self, path: Path, *, user: str | User | None = None) -> io.BytesIO:
return await ApplyPatchSession.read(self, self._stored_path(path), user=user)
async def write(
self,
path: Path,
data: io.IOBase,
*,
user: str | User | None = None,
) -> None:
await ApplyPatchSession.write(self, self._stored_path(path), data, user=user)
async def rm(
self,
path: Path | str,
*,
recursive: bool = False,
user: str | User | None = None,
) -> None:
await super().rm(self._stored_path(path), recursive=recursive, user=user)
async def mv(
self,
source: Path | str,
destination: Path | str,
*,
user: str | User | None = None,
) -> None:
normalized_source = self.normalize_path(source)
normalized_destination = self.normalize_path(destination)
if (
normalized_source != normalized_destination
and normalized_source.name == normalized_destination.name
and self._stored_path(source) == self._stored_path(destination)
):
raise RuntimeError("mv: source and destination are the same file")
await super().mv(source, destination, user=user)
class NormalizationFoldingApplyPatchSession(PosixHostApplyPatchSession):
"""A case-sensitive host over a filesystem that folds case and Unicode normalization.
This is APFS. It stores one entry for the decomposed and the composed spelling of the same
accented name, as well as for `notes.txt` and `Notes.txt`. `str.casefold` answers the first
pair wrong, which is one reason the fix may not ask a string whether two paths are one file.
"""
def _stored_path(self, path: Path | str) -> Path:
normalized = self.normalize_path(path)
folded = unicodedata.normalize("NFC", normalized.as_posix()).casefold()
for stored in self.files:
if unicodedata.normalize("NFC", stored.as_posix()).casefold() == folded:
return stored
return normalized
async def read(self, path: Path, *, user: str | User | None = None) -> io.BytesIO:
return await ApplyPatchSession.read(self, self._stored_path(path), user=user)
async def write(
self,
path: Path,
data: io.IOBase,
*,
user: str | User | None = None,
) -> None:
await ApplyPatchSession.write(self, self._stored_path(path), data, user=user)
async def rm(
self,
path: Path | str,
*,
recursive: bool = False,
user: str | User | None = None,
) -> None:
await ApplyPatchSession.rm(self, self._stored_path(path), recursive=recursive, user=user)
class WriteFailureApplyPatchSession(CaseFoldingApplyPatchSession):
"""A case-folding session whose first write fails, as a dropped sandbox connection would."""
def __init__(self, manifest: Manifest | None = None) -> None:
super().__init__(manifest)
self.fail_next_write = True
async def write(
self,
path: Path,
data: io.IOBase,
*,
user: str | User | None = None,
) -> None:
if self.fail_next_write:
self.fail_next_write = False
raise ConnectionError("sandbox write failed")
await super().write(path, data, user=user)
class ConcurrentWriterApplyPatchSession(CaseFoldingApplyPatchSession):
"""A case-sensitive session where another writer takes the source path and the move fails.
The rename is committed by a move. This session lets the staging write land, then has a
second writer put its own file at the source path, then fails the move. The operation
cannot succeed from here. What it must not do is put the original text back over the file
that other writer just created.
"""
def __init__(self, manifest: Manifest | None = None) -> None:
super().__init__(manifest)
self.concurrent_source: Path | None = None
self.concurrent_text = "written by someone else\n"
async def mv(
self,
source: Path | str,
destination: Path | str,
*,
user: str | User | None = None,
) -> None:
if self.concurrent_source is not None:
self.files[self.normalize_path(self.concurrent_source)] = self.concurrent_text.encode(
"utf-8"
)
raise ConnectionError("sandbox move failed")
class ProviderNotFoundApplyPatchSession(ApplyPatchSession):
async def read(self, path: Path, *, user: str | User | None = None) -> io.BytesIO:
try:
return await super().read(path, user=user)
except FileNotFoundError as exc:
workspace_path = self.normalize_path(path).relative_to("/")
raise WorkspaceReadNotFoundError(
path=Path("/provider/private/root") / workspace_path
) from exc
class UserRecordingApplyPatchSession(ApplyPatchSession):
def __init__(self, manifest: Manifest | None = None) -> None:
super().__init__(manifest)
self.read_users: list[str | None] = []
self.write_users: list[str | None] = []
self.mkdir_users: list[str | None] = []
self.rm_users: list[str | None] = []
self.mv_users: list[str | None] = []
self.same_file_users: list[str | None] = []
@staticmethod
def _user_name(user: str | User | None) -> str | None:
return user.name if isinstance(user, User) else user
async def read(self, path: Path, *, user: str | User | None = None) -> io.BytesIO:
self.read_users.append(self._user_name(user))
return await super().read(path)
async def write(
self,
path: Path,
data: io.IOBase,
*,
user: str | User | None = None,
) -> None:
self.write_users.append(self._user_name(user))
await super().write(path, data)
async def mkdir(
self,
path: Path | str,
*,
parents: bool = False,
user: str | User | None = None,
) -> None:
self.mkdir_users.append(self._user_name(user))
await super().mkdir(path, parents=parents)
async def rm(
self,
path: Path | str,
*,
recursive: bool = False,
user: str | User | None = None,
) -> None:
self.rm_users.append(self._user_name(user))
await super().rm(path, recursive=recursive)
async def mv(
self,
source: Path | str,
destination: Path | str,
*,
user: str | User | None = None,
) -> None:
self.mv_users.append(self._user_name(user))
await super().mv(source, destination)
async def same_file(
self,
left: Path | str,
right: Path | str,
*,
follow_symlinks: bool = True,
user: str | User | None = None,
) -> bool:
self.same_file_users.append(self._user_name(user))
return await super().same_file(left, right, follow_symlinks=follow_symlinks)