Concurrent agents on one file system

📝 摘要

文章指出人们对文件系统的理解普遍被 SQL 数据库和 S3 API 带偏：POSIX 文件系统并不提供 readFile/writeFile 那样的整文件原子读写，也没有真正的「路径」概念，真实原语是 lookup/create/rename/unlink 加基于 inode 的 open/read/write。因此当多个 Agent 并发操作同一文件系统时，「谁赢」、崩溃时文件处于什么状态、如何取版本等问题都没有明确答案，而用户却期望类似数据库的 ACID 保证。作者认为关键在于用户感知的原子性单位是整条命令（如 git clone、npm install、sed），而不是单次 WRITE/RENAME；但文件系统无法自行从这些命令中识别原子边界。Archil 的解法是让文件系统接受更高层命令（serverless tool calling），在每次工具调用前 fork 文件系统以实现快照隔离（copy-on-write，常数时间），调用结束后把成功结果原子合并回主线；也可拒绝与主线冲突的合并、要求在新状态上重跑。代价是 merge 为 O(n)，未来可用 MVCC 存储让非冲突写入免费合并。最终目标是既保留 POSIX 兼容（git、Postgres、npm 等现有工具可用），又给 Agent 提供一等事务能力。

核心概念： 文件系统不是对象存储——它没有原生路径和整文件原子语义，需要把「一条高层命令」定义成事务边界，才能为并发 Agent 提供 ACID。

关键技术栈 / 关键要点：

S3 语义的误导 — PutObject/GetObject 让人误以为文件读写是原子的，POSIX 的 read/write 实际上按字节偏移进行且可被并发修改。
真实 POSIX 原语 — 元数据操作 lookup/create/rename/unlink 与数据操作 open/read/write，灵活性更高但更难提供 ACID。
原子性单位错位 — 文件系统实现的是低层原语，用户感知的是 git clone、npm install 等整条命令，二者不在同一层。
serverless tool calling — 让文件系统直接接收并执行 bash 命令，从而推断出用户想要的原子边界。
快照隔离 + 合并 — 调用前 fork 文件系统（CoW，O(1)），成功后原子合并回主线；冲突时可拒绝合并并要求重跑。
Merge 成本 — 合并为 O(n)，可用 MVCC 存储优化为非冲突写入免费合并。

关键结论 / 实践建议： 构建 Agent 文件系统时不应假设 POSIX 已提供事务语义；应在兼容 POSIX 的前提下，把高层命令作为事务单元，用 fork/快照 + 原子合并来隔离并发 Agent 的写入。

I think there's a lot of confusion out there about what a file system really is. There's lot of places where we can assign blame.

People's familiarity with SQL databases (designed to provide transactional semantics on top of file systems) could lead people to incorrectly assume that there's some kind of ACID property on file systems. Most often, though, I think that people are mostly confused by the S3 API -- which has led an entire generation of developers to believe that file systems are made up of atomic operations on top of individual files. Just look at how just-bash describes their file system interface:

typescript

/**
   * Write content to a file, creating it if it doesn't exist
   */
  writeFile(
    path: string,
    content: FileContent,
    options?: WriteFileOptions | BufferEncoding,
  ): Promise<void>;

  /**
   * Read the contents of a file as decoded text. Default encoding is utf8;
   * pass an explicit text encoding (\`"ascii"\`, etc.) to override.
   *
   * For raw bytes (encoding \`"binary"\` / \`"latin1"\`), use {@link readFileBytes}
   * — the opaque return type forces callers to decide whether to forward
   * bytes unchanged or decode as text.
   *
   * @throws Error if file doesn't exist or is a directory
   */
  readFile(
    path: string,
    options?: ReadFileOptions | BufferEncoding,
  ): Promise<string>;

This is very much analogous to the S3 APIs of PutObject and GetObject.

The only problem? File systems don't support these APIs, they aren't real.

For example, the "readFile" API insinuates to the caller that you can read a consistent version of the file -- like you would in S3. If someone tries to change the file part of the way through reading, you'll either get the entire old version or the entire new version. This is not possible via POSIX file systems.

There's a similar situation with the "writeFile" API. It leads you to believe that either the entire contents of the file will be written atomically or it will fail without editing the file system. Though this can be partially emulated (via atomic renames), it's not possible to ensure that the file system (in POSIX) isn't updated at all if the call fails.

Perhaps the most basic and fundamental lie in these APIs: there's no such thing as a path in a POSIX file system. In most cases, a file has no idea where it's stored relative to root because the file system is a tree that the path is used to navigate, rather than being some kind of KV lookup or write.

Instead, the real POSIX API gives us a much "richer" and also much more confusing set of primitives:

plaintext

## Metadata operations
lookup(parentInodeId, entryName) -> Option<InodeId>
create(parentInode, entryName, type) -> InodeId
rename(parentInode, entryName, newParent, newName)
unlink(parentInode, entryName)

## File operations
open(inodeId) -> Option<FileHandle>
close(fileHandle)

## Data operations
read(fileHandle, start, length) -> Bytes
write(fileHandle, start, data)

This API gives us much more flexibility in how the applications can be written (we can update the middle of a file! we can move files around!), but it makes it way more difficult to provide the ACID-guarantees that the S3 API wants to lie to us about.

One simple way to imagine this is that you want to implement file versioning that looks a lot like Google Drive (this is what customers want, right?). At what point do you take a version? Is it when a file is closed? What if there's another process that has that file open? Is it when you quiesce writes?

What happens if you create a file, start writing to it, and then crash? That file could be there totally, it could be missing writes, or it could be missing. There's nothing in this API that says, like "PutObject" that the whole file should be written before it's made visible to other users.

What happens if two people try to edit a file simultaneously? Who wins? Is it "last writer"? That doesn't make sense in the context of file systems since you're always editing at byte-offsets, and there's not a "full file version" that you can present to some kind of conflict engine.

These answers are not really clear, and shows how far apart file systems are from what people imagine them to be, and I expect that many people trying to build agentic systems are sitting around trying to answer these difficult questions right now.

Now, there's a core problem with starting a company and putting a product to market. I can shout from the rooftops as much as I want about the world being wrong about file systems, and how they should acknowledge the fact that file systems are not object stores -- but people want these semantics that go above and beyond what file systems provide today. As a result, we're going to have to deliver it to them, and we might as well talk about what people want, and the right way to provide it.

What do people want

Now, there's one unspoken requirement that's worth calling out explicitly. People want their existing tools to work. Like many things on this Earth, it's relatively simple to implement a file system with all kinds of semantics and APIs you want if you rewrite the totality of software against your new APIs. I've never been about that. We want to support everything on our file system: agents using just-bash, people calling git, running Postgres, and npm. It's foolish to believe that your new APIs are going to result in the entire world rewriting their software stack around it, so we're going to start from the assumption that we want to support POSIX as the end-interface for this file system.

What do people really want from their file system? They want ACID, like a database. They want their operations to be atomic (meaning that they either complete all at or don't happen), they want them to be consistent (some kind of guarantee that conflicts surface to the application), they want them be isolated (so that concurrent agents can run on the file system without the possibility of corruption), and they want them to be durable (once they happen, they don't want to lose that data).

The key realization (for me) is that regular developers don't think about these properties in terms of the POSIX API that we described above. People don't really care if "WRITE" or "RENAME" happens atomically, they care about the totality of their commands, similar to how a Dockerfile splits up work into different atomic layers.

Even though the process of running "git clone" might result in hundreds of thousands of file system operations, the user perceives that command as a single event in the history of the file system and the "git clone" should either succeed fully or fail fully without making side effects to the file system. This makes it simple, from a durable execution perspective, to look around at the file system and pick up where you left off.

Similarly, if an agent is going to make a bunch of changes to a file system, then the user perceives that set of edits to be tied together into some kind of file system transaction which should either totally succeed (and atomically be visible to other clients simultaneously) or totally fail (with no edits from the set persisting).

You get the idea here. So, the problem that we have to grapple with is that the API that the file system actually implements is at a different layer than the one that the user wants to believe the file system implements.

Looking at these two boxes, you should be struck by a simple truth. It's just not possible for a file system which implements the right box to actually identify and discover the unit of atomicity in the right box in the face of concurrent clones, npm calls, and sed calls. And, if you can't tell what makes up an "atomic operation" you also have no hope of providing isolation between those commands.

What should we do?

Hopefully, the first step here is obvious to you all. You first need the file system to start accepting higher level commands like "git", "npm" and "sed". This is exactly what we've done with Archil, and we call this higher-level API serverless tool calling.

Now, you have the ability to submit an individual bash command to your file system so that the file system can run it on your behalf and edit the files for you. If it were 2025, when we built this, you might ask why anyone would submit bash commands to an external service to run them. The good news in 2026 is that this is obviously the shape of how agents interact with computers, which makes this model a natural fit for more agentic workloads.

Okay, so now if the developer is telling us exactly the higher-level commands that they want to run on the box, and we can now infer that this is the unit at which they want either the whole command to succeed or the whole command to fail, then we have the beginnings of being able to build ACID properties.

Because Archil already supports forking the file system, it's relatively simple to provide snapshot-isolation as a property -- we just need to make sure that we fork the file system before we start any individual tool call. Because snapshots and forking are copy-on-write, it's easy for us to do this in constant-time regardless of the actual amount of data inside of the file system.

Then, a simple way that we can support the "consistent transactions" of it all is to have a merge step that happens at the tail-end of the tool call that pushes the successful data [atomically] back into the "linear, main history". Even better, we could potentially just disallow merges in which the files changed on the main branch, requiring that the compute run again on the new state of the file system [depending on the requirements for speed, consistency, contention]. The only downside here is that "merge" is not O(1) but O(n) in the number of changes that were made on the original branch. I'm pretty certain that we could cut that time down with a more traditional MVCC based storage system which would give us "merging" of non-conflicting writes for free, but that's a topic for another time.

This system makes it possible to have, for example, conflicting agents run on the same repository on the same file system without needing to worry at all about conflicting data writes ending up with corruption for the end-user -- making it really easy for users to build systems that they have confidence in while at the same time being able to run traditional, POSIX software on those machines.

I always tell the team that we have two different sets of work ahead of us as a company. On one hand, we need to continue to pursue becoming the fastest file system in the world: "npm install" should run as fast as a local disk while having a storage system that can support 100 GB/s+ for model training runs. On the other hand, agents need a new set of primitives that weren't really part of the file system to begin with that we need to deliver on behalf of all of the agent builders that we work with.

This is one of those critical pieces for agents. We're actively working on branch merging to support these semantics now, and we'll share more as we get closer to release. That said, we're confident that a solution to this problem that allows users to continue to enjoy the same sandbox semantics with all of the POSIX tools that they like will be the right path for building the first file system that has first-class transactions as part of the API.

📝 摘要 #

📝 摘要