---
title: "Personal agent benchmark results, with intervals"
description: "Personal agent benchmark results for Instinct, Muse, Town and Pine: share of errands completed, 95% intervals, sample sizes and every caveat. No ranks."
url: https://personalagentbench.com/results/
last_updated: 2026-11-09
---

# Personal agent benchmark results

These are the personal agent benchmark results from Personal Agent Bench v1: the share of errands each product completed in our first exam, with the 95% interval and the number of scored tasks beside every figure. In October, on the same 20 tasks, Instinct completed 40% (95% interval 20–64%), Muse 58% (36–77%) and Town 15% (5–36%); Pine has no October figure. These rates measure our replicas of each product with a substituted model, not the apps you download. There are two measurements, one from September and a newer one from October, all of them v1, and they are printed side by side, never merged, with no ranks and no combined score. The chart gives the picture; the tables below it hold every figure, the caveats and the data files.

## Share of errands completed

11 October: all four given the same 20 tasks, once each. Not comparable with September.

- Muse · **58%** (36–77) [Muse: 58% of tasks completed, 95% interval 36 to 77%]
- Instinct · **40%** (20–64) [Instinct: 40% of tasks completed, 95% interval 20 to 64%]
- Town · **15%** (5–36) [Town: 15% of tasks completed, 95% interval 5 to 36%]
- Pine · No data, exam fault

The 09-21 to 09-22 records, filtered, 2026-09-22. All four answered the same tasks.

- Instinct · **52%** (45–61) [Instinct: 52% of tasks completed, 95% interval 45 to 61%]
- Muse · **40%** (33–46) [Muse: 40% of tasks completed, 95% interval 33 to 46%]
- Town · **37%** (30–44) [Town: 37% of tasks completed, 95% interval 30 to 44%]
- Pine · **30%** (23–36) [Pine: 30% of tasks completed, 95% interval 23 to 36%]

As of 2026-09-22. Each product answered its own draw, so rates are not directly comparable.

- Instinct · **43%** (36–50) [Instinct: 43% of tasks completed, 95% interval 36 to 50%]
- Pine · **20%** (14–25) [Pine: 20% of tasks completed, 95% interval 14 to 25%]
- Muse · **19%** (13–25) [Muse: 19% of tasks completed, 95% interval 13 to 25%]
- Town · Figure withheld
- Grok · Exam pending

## What these personal agent benchmark results measure

Both months measure our replicas of each product, not the apps you can download. In September, the comparable 114-task batch ran a replica of each product's runtime with the exam's GLM 5.3 flash model swapped in, and the first batch used the exam's model route as well. Each product's interface reaches the exam through an adapter we wrote, and we have not verified that every production feature is reproduced exactly. These rates do not measure the consumer products end to end. The October batch works the same way: the same four replicas, rebuilt since September, on the same substituted model. Because the replicas are ours, a vendor changing its own app does not move these numbers; a change between months comes from our replicas, the tasks, the scoring rules or the exam. See [model routing and its limits](https://personalagentbench.com/how-the-exam-works/changes/#10-october-exam-delivery-events-and-model-routing).

## The v1 limitation, said plainly

Everything in the tables above is v1: the replicas of each product's runtime that we built for the exam did not reproduce every capability of the products themselves, so a rate here describes what our replica did on our substituted model, not what the consumer app would do on yours. v2 is a different exam on the same bench that answers that: its tasks run on real websites from network-level recordings, replayed so no request leaves the machine. One task is published in the open repository and there are no v2 results yet; [how v2 works](https://personalagentbench.com/blog/record-and-replay/) has the method.

Newest

### v1 · October 2026

11 October batch, 20 tasks, plus 5 more batches listed separately. Every figure checked against the attempt records.

The October measurement. Each product answered the same 20 tasks once, under the exam rules that took effect at the end of September, and every attempt behind these rates has a record we checked the figures against.

| Product | Tasks completed | 95% interval | Attempts kept |
| --- | --- | --- | --- |
| Muse | 58% · [95% interval 36 to 77%, measured 58%] | 36 to 77% | 11 completed of 19 kept, over 20 tasks |
| Instinct | 40% · [95% interval 20 to 64%, measured 40%] | 20 to 64% | 6 completed of 15 kept, over 20 tasks |
| Town | 15% · [95% interval 5 to 36%, measured 15%] | 5 to 36% | 3 completed of 20 kept, over 20 tasks |
| Pine | No data: almost none of its submissions reached the exam, so the column was set aside rather than scored as zero. |   |   |

Not comparable with the September tables: the tasks, the scoring rules, our replicas of the products and the exam itself all changed in between, so a different rate is not a product getting better or worse. Each attempt ran once, so one attempt more or less moves a rate by five to seven points. Pine has no figure because its submissions did not reach the exam in this batch; the cause is not yet known and may be on the exam side.

Attempts were kept only when the exam did not break, the task suited the product, the product’s submissions were reaching the exam at all, and the attempt actually entered the exam. Each exclusion is counted in the records.

**The other 5 October batches**

The exam team ran five more batches in October, each with its own tasks, exam code and product builds. They are listed separately and never added together. Several were hit by exam faults found and fixed during the week (a sandbox wrongly judged broken, an expired certificate at the exam entrance), and the fixes have not been rerun yet.

| Batch | Instinct | Muse | Town | Pine |
| --- | --- | --- | --- | --- |
| 14 October, second run · 7 tasks | 67% · [95% interval 21 to 94%, measured 67%] · 21 to 94% · 2 of 3 | 33% · [95% interval 10 to 70%, measured 33%] · 10 to 70% · 2 of 6 | 57% · [95% interval 25 to 84%, measured 57%] · 25 to 84% · 4 of 7 | No data: almost none of its submissions reached the exam, so the column was set aside rather than scored as zero. |
| 14 October, first run · 7 tasks | 0% · [95% interval 0 to 66%, measured 0%] · 0 to 66% · 0 of 2 | 33% · [95% interval 10 to 70%, measured 33%] · 10 to 70% · 2 of 6 | No data: almost none of its submissions reached the exam, so the column was set aside rather than scored as zero. | No data: almost none of its submissions reached the exam, so the column was set aside rather than scored as zero. |
| 12 October, large batch · 72 tasks | 19% · [95% interval 9 to 38%, measured 19%] · 9 to 38% · 5 of 26 | 21% · [95% interval 11 to 36%, measured 21%] · 11 to 36% · 8 of 39 | No data: almost none of its submissions reached the exam, so the column was set aside rather than scored as zero. | No data: almost none of its submissions reached the exam, so the column was set aside rather than scored as zero. |
| 11 October, rerun · 20 tasks | 41% · [95% interval 22 to 64%, measured 41%] · 22 to 64% · 7 of 17 | 33% · [95% interval 14 to 61%, measured 33%] · 14 to 61% · 4 of 12 | 26% · [95% interval 12 to 49%, measured 26%] · 12 to 49% · 5 of 19 | No data: almost none of its submissions reached the exam, so the column was set aside rather than scored as zero. |
| 11 October, pilot · 20 tasks | 50% · [95% interval 30 to 70%, measured 50%] · 30 to 70% · 10 of 20 | 37% · [95% interval 19 to 59%, measured 37%] · 19 to 59% · 7 of 19 | No data: almost none of its submissions reached the exam, so the column was set aside rather than scored as zero. | 25% · [95% interval 11 to 47%, measured 25%] · 11 to 47% · 5 of 20 |

**v1 · September 2026Two measurements from 21 and 22 September: the first batch, and the same 114 tasks for all four. Supplied as totals, without attempt records. Open to read.**

#### First batch, 2026-09-22

| Product | Tasks completed | 95% interval | Scored tasks | Read it how |
| --- | --- | --- | --- | --- |
| Pine | 20% · [95% interval 14 to 25%, measured 20%] | 14–25% | 167 | More tasks scored than any other product. The September report said some tasks were being re-run; October status is unverified. |
| Instinct | 43% · [95% interval 36 to 50%, measured 43%] | 36–50% | 159 | 159 scored tasks; Pine has the largest published sample at 167. Marginal intervals alone do not establish a significant lead. |
| Muse | 19% · [95% interval 13 to 25%, measured 19%] | 13–25% | 90 | This is the figure after we fixed our own adapter. The September report said a further 83 tasks were being re-run; October status is unverified. |
| Town | — | — | — | Withheld. The figure we were given does not reconcile with the only records we can check: five batches for this product survive in the bench repository's history and all five were refused for incomplete domain coverage, giving roughly 13% across 30 scorable tasks. We are not publishing a number while our own evidence disagrees with it. Asked and pending. |
| Grok | — | — | — | In the exam, no published figure yet. We have not taken this one apart at the runtime level either, so it has scores coming but no teardown. |

As of **2026-09-22** · Source **Zentor** · Sorted by **sample size**, not by rate

Every figure is a share of tasks completed, with a 95% interval and the number of scored tasks it rests on. There is no combined score and no ranking, because a rate without its interval and its sample size is not a result. These September rates describe adapted runtimes using the exam model, on differing scored subsets; they are not consumer-product end-to-end performance.

These figures come from the team running the exam. The bench repository deliberately stopped tracking generated batch records in September, on the grounds that it holds the inputs a run needs rather than the outputs a run produces, so there is no published record behind these numbers for us or anyone else to check them against. We are printing them anyway, with that said plainly, because the alternative is printing nothing and pretending we have no numbers. If the underlying batches are ever published we will reconcile against them and say what changed.

#### The 09-21 to 09-22 records, filtered, 2026-09-22

A second measurement, kept beside the first rather than replacing it. Every product answered the same 114 tasks, which the table above does not: those rates rest on different tasks as well as on different sample sizes, so they were never strictly comparable with each other. These are replicas of product runtimes using the same substituted GLM 5.3 flash model, not shipping consumer products measured end to end.

| Product | Tasks completed | 95% interval | Attempts kept |
| --- | --- | --- | --- |
| Instinct | 52% · [95% interval 45 to 61%, measured 52%] | 45 to 61% | 380 over 114 tasks |
| Muse | 40% · [95% interval 33 to 46%, measured 40%] | 33 to 46% | 637 over 114 tasks |
| Town | 37% · [95% interval 30 to 44%, measured 37%] | 30 to 44% | 513 over 114 tasks |
| Pine | 30% · [95% interval 23 to 36%, measured 30%] | 23 to 36% | 546 over 114 tasks |

Not a fresh result. These are older builds judged under older rules, filtered down from roughly 5,600 attempts by eight rules, each with its own count. The filter that drops attempts which never reached the exam removes over 400 from Pine and over 400 from Muse, and if those were the products declining to act rather than a broken entry, Pine lands near 22% and Muse near 24%.

Supplied transcripts have not been republished. The delivery contained 1649 transcript files for 2076 attempts, with 712 attempts sharing files after filename collisions; complete per-attempt provenance is not established.

Both months in one file: download [CSV](https://personalagentbench.com/data/results.csv) · [JSON](https://personalagentbench.com/data/results.json) · also on [Hugging Face](https://huggingface.co/datasets/personalagentbench/personal-agent-bench) (CC BY 4.0)

To cite these figures: Personal Agent Bench (Zentor), “Personal agent exam results”, as of 2026-10-11, https://personalagentbench.com/results/. The CSV and JSON above hold the same figures, one row per product per batch.

A completion rate on its own is not a result; the interval beside it says how much of that number is measurement and how much is luck, and the task count says how much weight the interval can carry.

The errands behind these rates run in stateful worlds that can be replayed, and judging reads the final state rather than what the product said it did. [How the exam works](https://personalagentbench.com/how-the-exam-works/) sets out the worlds, the five ways a run can end, and why the measurements are published side by side instead of added up. The September rates were measured on 22 September; the exam's rules changed a week later, and [what changed](https://personalagentbench.com/how-the-exam-works/changes/) is on the record. The October batch was run under the new rules.

## How the October personal agent results differ

The October figures are the first we can check attempt by attempt. The exam team handed over the record of every attempt, and we recomputed each rate, interval and exclusion count from those records; all of them matched. The intervals are [Wilson score intervals](https://doi.org/10.1080/01621459.1927.10502953), which stay sensible on small samples like these. The September figures never had that: they came to us as totals. The records themselves, grouped by kind of errand with each conversation and grader's note, are on [the recorded AI agent examples](https://personalagentbench.com/runs/). What else changed in October, from product launches to our own corrections, is gathered in the [October 2026 personal agent report](https://personalagentbench.com/reports/2026-10/).

The headline October table is a single batch from 11 October, in which all four products were given the same 20 tasks. Five other October batches are listed under it and never added in, because each used different tasks, exam code and product builds, and the exam team asks that they not be pooled. We picked this batch because it is the largest one in which most of the products have figures and the fewest attempts were thrown out. It is still a trial run, and its intervals are wide.

Pine has no October figure. In every batch after the first pilot, its submissions in the web pages never reached the exam, and the cause has not been found; it may be on the exam side. That is a gap in the measurement, not a score of zero.

## Two personal agent results that need saying out loud

**Town has no figure in the first batch; the separate 114-task batch includes it.** The supplied first-batch figure disagreed with five surviving batches, all refused for incomplete domain coverage, at roughly 13% across 30 scorable tasks. The supplied figure was more than three times that on 21 tasks. We withheld it pending reconciliation.

**Muse's earlier figure was our fault, not the product's.** An earlier batch put it at 12%. Then we found four defects in our own adapter. It knew two of the three words a confirmation arrives as, it merged pending interactions from two sources and could answer one twice, it kept a duplicate answer branch on the event stream, and it had no branch for a handover request, so a product correctly refusing to act unattended was recorded as a failure. The 19% is the figure after those were fixed. We are publishing the broken number as well as the fixed one, because a bench that quietly replaces its own bad measurements is asking you to trust something you cannot check.

## A personal agent in the exam without a figure

Grok is in the exam with no published figure yet, and we have not taken it apart at the runtime level, so it appears in the results and not in the teardowns. What is public about it is on the [Grok Bot page](https://personalagentbench.com/grok-bot/).

## Reading personal agent results next to the teardowns

The results say how often something finished the job. The [teardowns](https://personalagentbench.com/agents/) say what it can reach, what it keeps, and what you can check afterwards. A product can score well and still be one you would not hand your accounts to, and nothing in a completion rate will tell you that. The [scorecard](https://personalagentbench.com/) puts both on one page, and [how we score](https://personalagentbench.com/how-scoring-works/) explains why they are never folded into one number.

## FAQ

**Which personal agent did best in the benchmark?** We do not rank them. In the October batch three products have figures. Instinct's interval (20 to 64%) overlaps both of the others. Muse's (36 to 77%) and Town's (5 to 36%) only just fail to meet, which on 19 and 20 attempts in a single trial batch is thin ground for an order. Where two intervals overlap, the gap between the rates is within measurement noise, and the first batch gave each product its own draw of tasks, so those rates are not directly comparable. The 114-task batch is the comparable one; read its intervals before reading anything into the order.

**Why did Muse go from 19% in September to 58% in October?** It is not the same measurement. The tasks, the scoring rules, our replica of Muse and the exam all changed in between, and the October figure rests on 19 attempts with an interval of 36 to 77%. Read it as a new measurement, not as Muse improving.

**Are these the real consumer apps?** No. They are replicas of each product's runtime driven through adapters, with the exam's model substituted. They tell you how the runtime arrangement copes with the errands, not how the app on your phone performs end to end.

**Can I download the data?** Yes. The CSV and JSON files linked under the first table hold the same figures, one row per product per batch, with the date they were measured. They are published under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/): reuse them freely, with a credit to Personal Agent Bench and a link to this page.

---

Published by Personal Agent Bench (https://personalagentbench.com), run by the Zentor team. Zentor builds agent software of its own, so its products are not on the scorecard and will not be added.
