Skill Hub / Showcase

Same task. Less work.

Tokens saved6,185,709Paired medians · 6/6 tasks · N=24
UI actions saved93Paired medians · 6/6 tasks · N=24
Pass · before → with skill19/24 → 23/24N small, not significant

What the skill changes

Before With skill · median per task

Expense: add multiple rows Expense: add multiple rows. Median tokens: before 1,530,202, with skill 60,438. N=3 before / 3 with skill. 1,530,20260,438 −96%N=3/3 N=3 before / 3 with skill
Before 2/3 · Wilson 95% CI 0.208–0.939; with skill 3/3 · Wilson 95% CI 0.439–1.000
Clock: alarm 7:30 "Gym" Clock: alarm 7:30 "Gym". Median tokens: before 1,113,113, with skill 84,163. N=5 before / 5 with skill. 1,113,11384,163 −92%N=5/5 N=5 before / 5 with skill
Before 4/5 · Wilson 95% CI 0.376–0.964; with skill 5/5 · Wilson 95% CI 0.566–1.000
Expense: reimbursable rows from Markor Expense: reimbursable rows from Markor. Median tokens: before 2,283,598, with skill 440,495. N=3 before / 3 with skill. 2,283,598440,495 −81%N=3/3 N=3 before / 3 with skill
Before 0/3 · Wilson 95% CI 0.000–0.561; with skill 3/3 · Wilson 95% CI 0.439–1.000
Calculator "3+3" Calculator "3+3". Median tokens: before 184,681, with skill 61,837. N=5 before / 5 with skill. 184,68161,837 −67%N=5/5 N=5 before / 5 with skill
Before 5/5 · Wilson 95% CI 0.566–1.000; with skill 5/5 · Wilson 95% CI 0.566–1.000
Expense: add one row Expense: add one row. Median tokens: before 1,752,326, with skill 621,898. N=3 before / 3 with skill. 1,752,326621,898 −65%N=3/3 N=3 before / 3 with skill
Before 3/3 · Wilson 95% CI 0.439–1.000; with skill 2/3 · Wilson 95% CI 0.208–0.939
Contacts: create Ada Lovelace Contacts: create Ada Lovelace. Median tokens: before 889,687, with skill 367,882. N=5 before / 5 with skill. 889,687367,882 −59%N=5/5 N=5 before / 5 with skill
Before 5/5 · Wilson 95% CI 0.566–1.000; with skill 5/5 · Wilson 95% CI 0.566–1.000

Try a skill

4 skills · 6 benchmarked actions
calculatorCalculator "3+3"−67%N=5/5Before / with skill median tokens
Demo · HOME → launch → action · no independent credit

Try asking

  • Calculate 1234 times 5
  • calculate 1234 times 5
  • use the calculator to divide 100 by 8
  • what's 2 to the power of 10

calculate · N=5 seeds · 2026-10-22

Before5/5 · Wilson 95% CI 0.566–1.000claude-haiku-5-5 · 2026-10-22
skill version: 1.0.2 · SHA: b37d4c36335dfcf969967c5053762151415d685b
With skill5/5 · Wilson 95% CI 0.566–1.000claude-haiku-5-5 · 2026-10-22
skill version: 1.0.2 · SHA: b37d4c36335dfcf969967c5053762151415d685b
Replay5/5 · Wilson 95% CI 0.566–1.000no model · 2026-10-22
skill version: 1.0.2 · SHA: b37d4c36335dfcf969967c5053762151415d685b
MedianBeforeWith skillReplay
Tokens184,68161,8370
UI actions616
LLM calls630
Wall42 s60 s43 s

Tokens saved: 123,150; median of paired seeds, N=5. Each pair is before minus with skill, including counted failures.

Full evidence, failures and traces →
contactsContacts: create Ada Lovelace−59%N=5/5Before / with skill median tokens

No clean demo yet for this action.

Try asking

  • Add Ada Lovelace to my contacts with number +1 555 123 4567
  • save Grace Hopper's number 555 0100
  • create a contact for Alan Turing with email alan@example.com
  • new contact Linus, phone +44 20 7946 0000

create_contact · N=5 seeds · 2026-10-22

Before5/5 · Wilson 95% CI 0.566–1.000claude-haiku-5-5 · 2026-10-22
skill version: 1.0.0 · SHA: b37d4c36335dfcf969967c5053762151415d685b
With skill5/5 · Wilson 95% CI 0.566–1.000claude-haiku-5-5 · 2026-10-22
skill version: 1.0.0 · SHA: b37d4c36335dfcf969967c5053762151415d685b
Replay5/5 · Wilson 95% CI 0.566–1.000no model · 2026-10-22
skill version: 1.0.0 · SHA: b37d4c36335dfcf969967c5053762151415d685b
MedianBeforeWith skillReplay
Tokens889,687367,8820
UI actions1026
LLM calls1380
Wall75 s84 s33 s

Tokens saved: 343,499; median of paired seeds, N=5. Each pair is before minus with skill, including counted failures.

Full evidence, failures and traces →
clockClock: alarm 7:30 "Gym"−92%N=5/5Before / with skill median tokens
Demo · HOME → launch → action · no independent credit

Try asking

  • Set an alarm for 7:30 tomorrow morning called Gym
  • wake me up at 6:45
  • create an alarm at 22:00 labelled Meds
  • set a 7am alarm

set_alarm · N=5 seeds · 2026-10-22

Before4/5 · Wilson 95% CI 0.376–0.964claude-haiku-5-5 · 2026-10-22
skill version: 1.0.1 · SHA: b37d4c36335dfcf969967c5053762151415d685b
With skill5/5 · Wilson 95% CI 0.566–1.000claude-haiku-5-5 · 2026-10-22
skill version: 1.0.1 · SHA: b37d4c36335dfcf969967c5053762151415d685b
Replay5/5 · Wilson 95% CI 0.566–1.000no model · 2026-10-22
skill version: 1.0.1 · SHA: b37d4c36335dfcf969967c5053762151415d685b
MedianBeforeWith skillReplay
Tokens1,113,11384,1630
UI actions1416
LLM calls2040
Wall1.9 min45 s22 s

Tokens saved: 1,029,086; median of paired seeds, N=5. Each pair is before minus with skill, including counted failures.

Full evidence, failures and traces →
pro-expenseExpense: reimbursable rows from Markor−81%N=3/3Before / with skill median tokens
Demo · HOME → launch → action · no independent credit

Try asking

  • Go through the transactions in my_expenses.txt in Markor. Log the reimbursable transactions in the pro expense.
  • log the reimbursable transactions from my_expenses.txt in pro expense

expense_add_multiple_from_markor · N=3 seeds · 2026-10-22

Before0/3 · Wilson 95% CI 0.000–0.561claude-haiku-5-5 · 2026-10-22
skill version: 0.1.0 · SHA: 38a33c2369123e8512141cd6cf2f573dfc5456d1
With skill3/3 · Wilson 95% CI 0.439–1.000claude-haiku-5-5 · 2026-10-22
skill version: 0.1.0 · SHA: 38a33c2369123e8512141cd6cf2f573dfc5456d1
Replay3/3 · no CIno model · 2026-10-22
skill version: 0.1.0 · SHA: 38a33c2369123e8512141cd6cf2f573dfc5456d1
MedianBeforeWith skillReplay
Tokens2,283,598440,4950
UI actions222not recorded
LLM calls259not recorded
Wall2.4 min2.3 min87 s

Tokens saved: 2,057,001; median of paired seeds, N=3. Each pair is before minus with skill, including counted failures.

Full evidence, failures and traces →
pro-expenseExpense: add multiple rows−96%N=3/3Before / with skill median tokens
Demo · HOME → launch → action · no independent credit

Try asking

  • Add the following expenses into the pro expense: {rows}
  • log these expenses in Pro Expense
  • add the expenses from this list to pro expense

expense_add_multiple · N=3 seeds · 2026-10-22

Before2/3 · Wilson 95% CI 0.208–0.939claude-haiku-5-5 · 2026-10-22
skill version: 0.1.0 · SHA: 38a33c2369123e8512141cd6cf2f573dfc5456d1
With skill3/3 · Wilson 95% CI 0.439–1.000claude-haiku-5-5 · 2026-10-22
skill version: 0.1.0 · SHA: a1980cff (feat/skillhub 1b3f8e66)
Replay3/3 · Wilson 95% CI 0.439–1.000no model · 2026-10-22
skill version: 0.1.0 · SHA: 38a33c2369123e8512141cd6cf2f573dfc5456d1
MedianBeforeWith skillReplay
Tokens1,530,20260,4380
UI actions3402
LLM calls2530
Wall2.1 min1.8 min1.6 min

Tokens saved: 1,408,794; median of paired seeds, N=3. Each pair is before minus with skill, including counted failures.

Full evidence, failures and traces →
pro-expenseExpense: add one row−65%N=3/3Before / with skill median tokens
Demo · HOME → launch → action · no independent credit

Try asking

  • Add the following expense into the pro expense: {rows}
  • add an expense to pro expense

expense_add_single · N=3 seeds · 2026-10-22

Before3/3 · Wilson 95% CI 0.439–1.000claude-haiku-5-5 · 2026-10-22
skill version: 0.1.0 · SHA: 38a33c2369123e8512141cd6cf2f573dfc5456d1
With skill2/3 · Wilson 95% CI 0.208–0.939claude-haiku-5-5 · 2026-10-22
skill version: 0.1.0 · SHA: a1980cff (feat/skillhub 1b3f8e66)
Replay3/3 · Wilson 95% CI 0.439–1.000no model · 2026-10-22
skill version: 0.1.0 · SHA: 38a33c2369123e8512141cd6cf2f573dfc5456d1
MedianBeforeWith skillReplay
Tokens1,752,326621,8980
UI actions1722
LLM calls25120
Wall3.1 min3.0 min40 s

Tokens saved: 1,224,179; median of paired seeds, N=3. Each pair is before minus with skill, including counted failures.

Full evidence, failures and traces →

Replay: 0 tokens · 24/24 passed · no model.

Method & coverage

claude-haiku-5-5 · data date 2026-10-22. All 6 manifest tasks contribute their published current before/after_mcp arm counts. Every current counted failure is included. Historical pre-fix (supply-chain finding) rows are excluded.

19/24 · Wilson 95% CI 0.595–0.908 before; 23/24 · Wilson 95% CI 0.798–0.993 with skill. Paired McNemar: 5 for / 1 against, p=0.22, N=24 pairs.

KPI savings sum the task medians of paired-seed differences. Bars compare arm medians; N is before / with skill. UI actions are the agent’s own interactions. Tasks were chosen ad hoc; missing pairs are not invented.

calculator: 1/4 actions benchmarked · contacts: 1/4 actions benchmarked · clock: 1/7 actions benchmarked · pro-expense: 3/3 actions benchmarked. Two more skills are TBD.

Maintainer-run, not independent · claude-haiku-5-5 · Every failure is counted (intention-to-treat). · p not significant.