Personal Agent Bench

Pine AI teardown: what runs when it calls a company

Phones companies on your behalf about bills, refunds and cancellations.

Product 19pine.aiLast verified 2026-09-20Score not yet scored

Pine sells one promise: you hand it a bill dispute, and it picks up the phone so you don't have to. The marketing is about the outcome. This page is about the machine underneath, because the machine explains both why Pine is good at the narrow thing it does and why it stalls everywhere else.

We took the client apart. Everything below is either something the product states outright or something we reconstructed from several pieces of evidence that agree. Where we couldn't get to the bottom of something, we say so instead of guessing.

Nothing runs on your computer

Open the web app or the Mac app and you're looking at a display, not a brain. The model never runs locally. A server builds the whole request — instructions, retrieved facts, tool contracts, conversation history, your latest message — and the client only ever receives the results over a REST endpoint and a socket connection.

That sounds like a boring implementation detail. It isn't, and here's why: you cannot see what Pine was told. The instructions that shape its behaviour never reach your machine, so there's no version of "check what it was asked to do" available to you. We probed for the production instruction text roughly 126 times, statically and at runtime. Empty every time. That's not a gap in our work; it's a design decision on their side, and it's worth knowing about a product you're handing your account details to.

One small thing that trips people up: the task cards on the Pine homepage each carry a field called prompt, and it's tempting to read that as the system instruction. It's your own message.

Twenty-six tools, and a fax machine

The client exposes 26 distinct tool names. Not their schemas — only the names and how they're grouped. Sorted roughly by what they do, you get phone actions, email, web and travel search, knowledge lookup, timers and to-do management, a file parser, and two ways to interrupt you for information.

And a fax tool.

A personal agent shipping fax support in 2026 tells you exactly who Pine is built for. Nobody faxes their friends. You fax an insurance claims department, a medical records office, a utility that still runs on paper. That single tool name says more about the target user than the entire homepage does.

The thing the tool list doesn't cover is computer use. When Pine drives a browser, it doesn't call one of those 26 tools — the Mac app hands the instruction to a separate browser agent running on your machine, and the cloud path leases you a remote desktop you can watch over VNC and take over from. Two different execution paths, neither of them visible in the tool catalogue.

Four products wearing one brand

Pine is sold as one assistant, but it's four different request shapes underneath: the task assistant most people use, a voice copilot built on LiveKit, a REST API that lets other people's agents place calls, and the leased cloud computer. They share a name and a login. They don't share a shape.

The phone calls keep no transcript

This is the finding with actual consequences for users.

When Pine makes a task call on your behalf, the record it keeps afterwards is the merchant, the number, how long it ran, what it cost you in credits, and a summary. No verbatim transcript. So when the interface tells you a call has ended, that means the line dropped, not that anything was resolved, and you have no way to go back and check what was actually said on your behalf.

The live event stream and the saved history don't even agree on granularity. On one three-way identity-verification call we watched 62 events go by in real time; the history afterwards held 30 rows. Watch only the replay and you'll miss things that happened.

Credits drift too. One voice call reported 70 credits in its final state and took 74 off the balance.

Memory: there isn't a file you can edit

Some personal agents keep your profile in plain files you can open and rewrite. Pine doesn't. There's no personality document on the client; the persona — Pine calls itself Mia on the phone — lives server-side, and what it knows about you sits in a structured fact layer that gets queried on demand rather than loaded up front.

Practical version: you can't audit or correct what it believes about you by editing a file. You go through the product or you don't go at all.

What we couldn't establish

[TODO 保留本节,上线前逐条复核]

Pricing

[TODO 数据从 data/pricing.json 读,带 as-of 日期渲染。仓库里那个 $39/6000 credits 已过期,2026-09-20 官网实为 Plus $99.9、Max $174.9、Team $291.6,均为年付月均价,另有免费档。上线前再核一次月付价。]

How Pine scores

[TODO 接 /how-scoring-works 的五条轴,等 data/scorecard.json 定稿后填]

The five axes

Where it runs

Server-side. Neither the web app nor the Mac app runs the model. Browser work is handed to a separate agent on your own machine; the cloud route leases you a remote desktop you watch over VNC.

How tools are exposed

26 tool names reach the client, grouped by job. The schemas never do. Computer use bypasses that list entirely.

Where memory lives

No file you can edit. A server-side persona plus a structured fact layer queried on demand.

How you reach it

Web and Mac, outbound phone calls, a voice copilot, and a REST endpoint other people's agents can call. Four request shapes under one brand.

What we could not establish

Production instruction text (roughly 126 probes, all empty), the calling tool's schema, which carrier places the calls, and whether work is split across a fast and a slow model.

The other three

Compare all four on the scorecard, read how we score, or see why the existing agent benchmarks don't answer this.