We Tested 38 AI Models on 100 Real QuickBooks Close Checks (LevelBench-100)

Every week, a business owner or controller asks me the same question: "Can I just connect ChatGPT, Claude, or Gemini to QuickBooks and let it close my books?"

Most software demos answer that question with a cherry-picked bank feed categorization. They show an AI model reading a coffee receipt and picking "Meals and Entertainment." That is not a monthly close.

A real monthly close is not a vocabulary test. It is a multi-system reconciliation problem full of traps: prior-period checks sitting in a current export, vendor bills that were already paid twice, customer deposits sitting in revenue before the job starts, and owner add-backs disguised as operating software.

To see what actually happens when you hand a real close to modern AI systems, we built LevelBench-100 (v2.0): an open, 100-check benchmark across 12 monthly close and CFO workflows modeled on four anonymized multi-entity businesses running QuickBooks Online. You can explore the full interactive dataset on our Agentic Accounting Benchmark hub.

How we built a benchmark that does not teach to the test

Most AI benchmarks fail finance operators for two reasons: the prompts tell the model the answer, and the test data has already been cleaned up by a human accountant.

In real life, nobody hands a controller a clean table and says, "Please calculate the ASC 330 landed inventory cost excluding outbound shipping." Instead, you get a messy QuickBooks CSV export, a freight forwarder bill, a Shopify payout report, and a frantic Slack note from the founder.

To mirror real operations, LevelBench-100 v2.0 enforces three strict rules:

1. Zero prompt leakage in unassisted runs: In our unassisted zero-shot tier, models receive only the anonymized entity Chart of Accounts, realistic client onboarding memos (sales handover notes and controller notes), and unfiltered ledger and subledger JSON rows. They never receive pre-digested accounting formulas or hints about which rows are traps.
2. Real production distractor rows: Every workflow embeds actual bookkeeping traps we see in client cleanups:
   • August checks and voided transactions mixed into a September bank reconciliation export.
   • Bank withdrawals that are already linked to an existing QuickBooks BillPayment or predate the vendor invoice.
   • Field service jobs completed in October sitting next to deposits collected in September for work not yet started.
   • Outbound customer shipping charges mixed with capitalizable inbound ocean freight under FASB ASC 330 inventory guidance.
   • Recurring QuickBooks and CRM subscriptions buried inside a seller's list of "one-time" EBITDA add-backs.
3. Three distinct operating setups tested side by side: We tested the exact same 100 binary checks across three setups so CEOs and finance leaders can see whether accuracy comes from the AI model, the prompt, or the engineering pipeline around it.

Plain-English guide to the three setups we tested

Software vendors love to throw around terms like "agentic harness" or "deterministic pipeline" without explaining what they mean for your money. Here is what those terms actually mean in plain English:

• Unassisted AI Model (27 models tested): You hand the raw QuickBooks export and client notes directly to an AI model (like GPT-6.1, Claude Opus 5.5, or Gemini 4 Argon) and ask it to produce the journal entries and reconciliation numbers. There are no code guardrails checking its math.
• Rule-Guided CLI Agent (4 harnesses tested): You run the AI inside a command-line workspace agent (like Google Gemini CLI, Antigravity CLI, or OpenAI Codex CLI) with written accounting standard operating procedures. The agent can write scripts to inspect the files, but the AI still decides what to output.
• Level Deterministic Pipeline (7 pipelines tested): Instead of letting an AI model do arithmetic or post directly to the ledger, a hard-coded accounting rules engine (what we call the Level Deterministic Pipeline) enforces double-entry balance checks, date cutoffs, invoice-to-payment matching, and GAAP accrual rules in code. The AI model is used to read unstructured documents, classify ambiguous memos, and draft commentary, while the deterministic code verifies every penny and blocks any unverified write.

The LevelBench-100 leaderboard: representative results across 38 setups

Below is the headline comparison across the three architectures from the 38-entry LevelBench-100 leaderboard.

Four findings every CEO and finance leader should care about

1. A 93% accurate AI bookkeeper is still unusable without a deterministic gate

In software demos, 93% accuracy sounds great. In accounting, a 93% pass rate across 100 checks means 7 broken numbers in your monthly close. Worse, when you run a probabilistic model three times in a row (pass^3 consistency), a 93% single-run model only produces the exact same error-free close 85.2% of the time.

Even more important is the Dangerous Write Rate: how often a system attempts to post an unbalanced journal entry, double-pay a vendor bill, or silently fabricate a missing tax rate instead of pausing for human review. Several frontier models with 94% to 96% overall accuracy still recorded a 10.7% dangerous write rate when unassisted because they tried to "helpfully" complete a journal entry when a vendor W-9 or legal settlement document was missing.

2. Gemini 4 Argon and GPT-6.1 Sol had zero dangerous writes, for different reasons

Among unassisted models, Gemini 4 Argon (Medium Thinking) took first place at 98.0% accuracy and 0.0% dangerous writes in 26.09 seconds ($0.04 per close). GPT-6.1 Sol also achieved a 0.0% dangerous write rate at 93.0% unassisted accuracy (and 97.0% when rule-guided) because its double-entry and epistemic-hold discipline never forced an unbalanced entry.

By contrast, open-weight models like Kimi K2.5 (67.0% accuracy, 39.3% dangerous write rate) and Qwen 3 Next 80B (61.0% accuracy, 46.4% dangerous write rate) routinely swallowed distractor rows: they capitalized outbound customer shipping into inventory and accepted recurring software subscriptions as one-time EBITDA add-backs.

3. The deterministic pipeline turns $0.01 worker models into 100% close engines

Look at Rank 4 and Rank 7 in the table above. Unassisted GPT-5.6 Terra scores 93.0%, and unassisted DeepSeek V3 scores 80.0%. Yet when you wrap those exact same models inside the Level Deterministic Pipeline, both jump to 100.0% accuracy, 100.0% three-run consistency, and 0.0% dangerous writes at $0.008 to $0.025 per close.

Why? Because an AI model should never be asked to do what 50 lines of deterministic accounting code can verify with certainty. In a real finance operating system, the AI extracts context and flags exceptions, while deterministic code checks every transaction ID, date cutoff, and debit-credit tie-out against the QuickBooks Online accounting API and subledger exports.

4. Your software stack determines how hard your monthly close is

LevelBench-100 also exposes why AI finance projects fail in service businesses: the hardest errors happen in the gap between your operational software (field dispatch, POS, EHR, or Shopify) and your general ledger.

That is why we paired LevelBench-100 with three companion benchmark databases so operators can check their entire stack:

• Multi-Industry Finance and Operations Software Benchmark: Compare the actual ledger, subledger, and close bottlenecks across Ecommerce (Shopify + A2X + QBO), Restaurants (Toast + R365 + QBO), Healthcare, Commercial Cleaning, and Staffing.
• 2026 Contractor Software Comparison: See how 12 contractor accounting and field platforms (ServiceTitan, BuildOps, Jobber, Housecall Pro, QuickBooks Online, Sage Intacct, Viewpoint Vista, and Foundation) handle WIP schedules, certified payroll, and job costing, alongside our demonstration guide for WIP and payroll.
• 2026 Contractor Valuation Multiples across 13 Trades: See why private equity buyers discount messy cash-basis books by 1.5x to 3.0x EBITDA, and compare private tuck-in vs. platform brackets with our public contractor valuation analysis and the independent dataset on Trades Economic Index.

What to ask before you let any AI touch your books

Before you buy a tool that claims to automate your bookkeeping or monthly close, ask the vendor three questions:

1. "What happens when a bank feed row matches two open bills or predates the invoice?" If the system relies on a prompt instead of a deterministic date-and-link check, it will eventually double-count an expense.
2. "Show me your epistemic hold log." When a $18,500 wire hits the bank with no invoice and no W-9, does the AI guess an expense account, or does a hard rule quarantine the transaction into Suspense - Ask Client and block the close until a human resolves it?
3. "Can I run the exact same export three times and get the exact same trial balance to the penny?" If the answer is no, you have a chatbot, not an accounting pipeline.

If you want to see where your current QuickBooks setup, job costing, or subledger sync is leaking margin before your next close, run our profit gap diagnostic tool or test your open jobs in the WIP schedule calculator. And if you want a finance team that runs on deterministic controls rather than monthly guesswork, see how our CFO and accounting services and software integrations work in practice.

Source and claim note

Methodology and dataset note: LevelBench-100 (levelbench-100.v2) is a first-party, synthetic-but-realistic multi-entity benchmark created by Level Research modeled on four anonymized SMB QuickBooks Online structures (CompanyCo: HVAC and plumbing contractor, omnichannel ecommerce brand, B2B SaaS company, and multi-clinic healthcare group). The 100 binary checks span 12 workflows (WF01 through WF12). All 38 benchmark rows, workflow definitions, and grading rules are published in full at /data/benchmarks/levelbench-100.json and /data/benchmarks/levelbench-100.csv and archived in our repository under data/levelbench-100/. A 100.0% score on LevelBench-100 proves that a deterministic pipeline satisfies the 100 defined rules on this benchmark cohort; actual production accuracy in a live business depends on source data completeness, chart-of-accounts governance, and human review of quarantined exceptions.

Related Reading

• Agentic Accounting and QuickBooks Monthly Close Benchmark (LevelBench-100)
• Your CFO's Real Job Is Building the AI Operating System for Your Money
• Why AI Finance Projects Fail in Service Businesses
• Contractor Accounting Software: Prove WIP and Payroll
• Public Contractor Valuation Multiples vs. Private 4x-8x Planning