This post covers how I built the test, what was wrong, what the new search does, and the four searches it still gets wrong.

Search that worked until you misremembered

The old search was SQLite's FTS5 full-text index with the unicode61 tokenizer. Typed words matched the beginnings of words in your notes, and if that found nothing, a trigram index matched words of three letters or more anywhere inside longer words. Results were ordered by BM25. A first word ending in a colon, like todo:, became a tag and showed only notes starting with it.

That works whenever you search with the words you wrote. Search for dentist and you get the note with "dentist" in it. The trouble starts when you remember a note differently from how you typed it, and with a notes app you mostly do. You wrote "Mum's birthday" on one phone and search mums on another that has no apostrophe on the first keyboard screen. You wrote "Wi-Fi" and search wifi. You wrote "couch" and search sofa. You wrote "Thurs" and search thursday. None of those found anything.

A library built to break search

Testing search needs a library, and I wasn't going to use anyone's real notes. So I wrote a seeded generator that makes one person's library: 40,000 notes from six years of capture across 12 parts of life (work, family, health, money, food, the house, travel, reading, code, ideas, a journal and odds and ends), about 680,000 words. Around 20% of the notes are in 18 other languages, 27% are archived, 2% are in the trash and 6% have been rescued at least once, roughly what a long-running Gravity library looks like.

Then I went looking for failures. I wrote 314 probes aimed at likely weak spots and ran them through the real search code. 246 failed. I grouped the failures by cause, kept 20 distinct causes, and wrote 10 cases for each.

Every case has three parts: a note, a search a person would reasonably type to find it, and a control search using the note's own words. A case passes when its note shows up in the top 10, which is a phone screen and a short scroll. Each case's note is added to the 40,000 through Gravity's own note-saving code.

The controls are there so a zero means something. All 200 controls passed, so every note was indexed and findable. The 200 real searches scored 0 of 200 in the top 10, and only 7 appeared anywhere in the first 200 results, at ranks 57 to 137.

I ran the same cases through the real gravity binary, through the phone app's own search code on the SQLite version the phone uses, and through the plainer trigram index that Macs on macOS 15 fall back to. Every one gave the same ranks.

Fig 1. Each pin is one search from the eval, in 20 columns of ten by the reason the old search failed. A pin rises when its note lands in the top 10. With the old search two pins are up, counting notes that answer the search as well as the one it was written for. With the new one, 198 are. Open the interactive version.

The 20 ways it failed

The first four were plain bugs. Any word ending in a colon became a strict tag, even one copied from the middle of a note, so password: bluebell42 looked for notes tagged password: and found none. A short tag like w: was a prefix query on w, which also matched every work: and wedding: note, and the scan stopped after 1,000 matches before it reached a real w: note. Tags compared exactly, though the website says accents don't matter. And a single search word under three letters, like zu, switched off the substring stage for every other word in the search.

The rest were limits of matching letters. Keyword search can't know that "Thurs" is Thursday or that a couch is a sofa. A question like where is the spare key required every word, including where, is and the. And BM25 alone decided the order, so for a common word like dentist, a long old note full of the word beat the one you wrote yesterday and rescued twice.

The bug the eval didn't catch

Speed was a separate benchmark, and it found the worst bug of the lot. The phone app uses op-sqlite, which bundles SQLite 3.51.3. On the phone's schema, that version's query planner started the search query from the index on deleted_at, then ran the full-text match once per note. In other words, every search on a phone read every note.

I compiled op-sqlite's own sqlite3.c with the app's iOS flags and ran the phone's query on my Mac against 40,000 notes. dentist took 817 ms. A one-letter search, t, took 288 seconds. The CLI and Mac app use newer or different SQLite builds that start from the full-text index, so they never showed it. Writing the condition as +n.deleted_at IS NULL stops SQLite using that index, and dentist dropped to 1.7 ms.

Keeping myself honest

Hill-climbing a test of 200 cases is easy to fake. You can add each failing word to a list until the score is 100% and the search is no better for anyone else. I set four guards before changing any code.

• A held-out set of 200 more cases in the same 20 classes, written by a separate agent that never saw the implementation. I reported it next to the main score and never tuned on it. The old search scored 16 of 200 there, 10 of them on the short-tag bug.
• The 200 controls had to stay at 200 of 200.
• Every word list (abbreviations, irregular plurals, number words) had to be a general list. None could be the eval's own words.
• Speed was measured against a frozen copy of the old search code, run back to back on the same machine.

There was one more problem to sort out. Some searches have more than one right answer. If the library holds three notes about topping up the boiler pressure, boiler pressure can fairly return any of them. So I had an independent model judge every failing case's top 10 without seeing the ranks, and it marked notes that answer the search as well as the target does. It found such notes for 12 cases. I report both numbers: the original target only, and counting judged answers. Counting judged answers moves the old search from 0 to 2 on the main set and from 16 to 17 on the held-out set.

What search v2 does

One index, built in app code

The new index lives next to your notes in the same SQLite file and replaces both old full-text tables.

• notes_search(rowid, body, extra) holds a search document for each note.
• An FTS5 table over body and extra with prefix indexes, and a trigram table over body for matches inside words.
• An fts5vocab table, so search can ask which words exist in your library and how many notes use each.
• A search_dirty queue. Triggers on notes add a note's row to the queue when it's created or its text changes.

SQL triggers can't build the search document, because op-sqlite on the phone has no custom SQL functions, and the folding rules are too involved for SQL anyway. So each app drains the queue in its own code: before each search, and in the background after writes. Draining a note computes its search document and its meaning vector and upserts them into the index.

Folding

body is the note folded into one canonical form. Text is normalised (NFKC), invisible formatting characters are removed, case is lowered and accent marks are stripped. Letters that don't decompose are spelled out (ß becomes ss, ø becomes o, ł becomes l, þ becomes th), Greek final sigma becomes σ, and katakana becomes hiragana. Words broken across lines with a hyphen are joined. Each emoji becomes a token like emo1f382, so 🎂 is a word you can search for.

The same fold runs on the search. That fixed the accent, script, pasted-character and emoji failures in one place.

Alternate forms

extra holds other ways of writing what's in the note:

• Joined forms of words with apostrophes, dots and hyphens: mums, wifi, phd.
• Numbers without thousands commas, 15k as 15000, 5 km joined as 5km and 5km split as 5 km.
• Codes. Runs of letter and digit groups like 07700 900123, SW1A 1AA or +44 7700 900123 are stored joined, and a +44 number also gets its 0 form.
• Number words both ways (six and 6), abbreviation expansions (thurs to thursday, appt to appointment, xmas to christmas), and English stems using Snowball's Porter2, after mapping irregular forms like mice to mouse.
• Dates and times as canonical tokens. 14/03/2027, 14 March 2027 and March 14th, 2027 all produce md0314 and y2027, and 7.30pm produces t1930. Dates are read day first.
• Overlapping two-character pieces for Chinese, Japanese, Korean and Thai, so 咖啡 matches inside 今天下午去咖啡店见客户.

Searches become concepts

The search is folded the same way and split into concepts. A word concept matches the word as a prefix, its stem, its number form and its expansion. Words of three letters or more also match as a substring. A typed date becomes the same tokens the note produced. Question words like where, is and the become concepts that are never required.

Some expansions use the library itself. A word of five letters or more that no word in your notes starts with gets up to three typo variants from the vocabulary table, by edit distance with the first letter fixed, so perscription finds prescription. A closed word like wifi can match a note that says wi fi, and two typed words whose join is a word in your notes, like pass word, become one.

A tag is still a tag when tagged notes exist. When none do, password: becomes the plain word password. Tags compare folded, so cafe: finds Café:, and a short tag like w: asks the index for notes whose first word is exactly w, so it doesn't scan through every work: note first.

Meaning from a 17 MB model

Some searches share no letters with their note: sofa and couch, lawyer and solicitor. For those, search v2 uses a static embedding model. A static model is a table with one vector per word piece. A text's vector is the sum of its pieces' rows, which takes microseconds with no neural network to run, so it works offline on a phone.

I tested the best current static models on the eval by meaning alone, with no keyword matching at all. On the main set, Sentence Transformers' multilingual static-similarity-mrl-multilingual-v1 found 146 of 200, static-retrieval-mrl-en-v1 found 138, and Minish Lab's potion-retrieval-32M found 137. The multilingual model looked best on paper and covers 50 languages in 27 MB. It lost on the searches keyword matching can't rescue: for lawyer, it ranked the solicitor note 1,155th, where potion-retrieval-32M ranked it 2nd. So Gravity ships potion-retrieval-32M (MIT licence), cut to its first 256 dimensions and stored as int8 with one scale per token. The file is 17.3 MB.

Every platform has to produce the same results, so the arithmetic is integers wherever it can be. Each note's vector is scaled so its largest component is ±127, rounded half away from zero and stored as 256 bytes. Similarity is an integer dot product over the product of integer norms. Rust, TypeScript and Swift get the same numbers bit for bit.

Scoring

Every candidate gets one score:

• 14 × coverage, the share of required concepts it matches
• 2 × the share matched as whole words
• 5 if it carries the searched tag
• 3 × its BM25 over the best BM25 in the results
• 3 × 0.5 ^ (age / 30 days), with age counted back from your newest note
• 1.5 × rescues, counting up to 3, over 3
• 6 × meaning similarity
• 2 × proximity, how close together the matched words sit

Notes that match every concept are kept, as are notes that match at least half of them and notes with a similarity of at least 0.40. Coverage dominates, so a note with all your words beats one with some of them. Among notes that match equally, recent and rescued notes rise, which fixed the dentist case. The recency half-life counts from your newest note so a library you haven't touched in a year doesn't rank as if everything had gone stale.

Climbing

I hill-climbed the reference implementation first, in Rust in gravity-core, running the eval after every change and checking the held-out set beside it. The held-out score tracked the main one the whole way, which told me the changes would hold for searches I hadn't written. With the model switched off, keyword matching alone now finds 192 of 200 on the main set and 190 held out. The model adds the rest.

Then I made it fast again, because the first working version wasn't:

• Finding the newest note's sort key scanned the table for 4 to 11 ms. An INDEXED BY hint made it instant.
• The word stage takes the best 250 notes by BM25 plus the newest 50. When more than 2,000 notes match, as with a single letter, it takes the newest 250 and skips BM25.
• The meaning stage runs for searches of two or more words, or when words found nothing. A single word that matched skips it.
• Vectors moved to their own table, so loading them reads 256 bytes a note. Cold loading dropped from 358 ms to 23 ms.
• Folding plain ASCII takes a fast path.
• A setup step that ran on every search, costing 6 ms, now runs once.
• Result excerpts are built in app code. FTS5's snippet() was quadratic on long notes.

Before and after

All on the 40,400-note library, with the note in the top 10:

Speed is every keystroke of 400 typed searches, 2,329 searches in all, on an M3 Pro with a warm cache, as the apps run it:

Medians stay about the same, the slow tail is much shorter, and the short-tag bug went from 36 ms with no results to 6 ms with the right note. I set out to make no search slower, and that didn't fully hold. Fifteen of 196 fixed searches got slower by 0.5 to 3 ms, all of them searches by meaning. All 15 are searches where meaning now runs: rare words, words that match nothing, and searches of several words. The worst is bluebell at 40,000 notes, from 0.44 ms to 3.43 ms. I think 3 ms is a fair price for finding the note.

On the phone, the planner fix matters more than anything else in this post. With the new SQL on 40,400 notes, a one-word search takes 7 ms in Node and 32 ms with the JIT off, which is close to how Hermes runs on a phone. A 20-word paste takes 17 and 174 ms. Indexing a library that size the first time takes about 33 seconds with the JIT off, in the background, a chunk at a time. The Mac app indexes 2,000 notes in 387 ms and then searches in 1 to 4 ms.

The command line tool starts a new process for every search and loads the vectors each time, so it's the slowest path. At 40,000 notes a meaning search like where is the spare key takes 19 ms warm and about 150 ms cold. At 100,000 notes, cold meaning searches take 320 to 380 ms, over my 250 ms target. The apps keep vectors in memory, so they don't pay that.

One contract, three implementations

Gravity's apps don't share code. The CLI is Rust, the phone app is TypeScript in React Native, and the Mac app is Swift. They share a written contract instead: section 6 of the note contract specifies the index schema, the fold character by character, every alternate form, the model's file format and arithmetic, the query rules and the scoring formula. Beside it sit golden files, search documents, vectors and ranked results for fixed inputs, which every app must reproduce exactly.

I wrote the Rust reference first, then ported it to TypeScript and Swift. Both ports reproduce every golden file. The ports also found two bugs in the reference. The queue trigger used INSERT OR IGNORE, and an upsert on a note already in the queue failed, because an upsert's conflict clause overrides OR IGNORE. It's ON CONFLICT DO NOTHING now. And a wide character just before a word broken across lines crashed the indexer. Both fixes went into the contract, so all three apps got them.

Search in Claude and ChatGPT

Gravity has an MCP server that lets Claude, ChatGPT and other AI apps save and find notes in your synced library. Its search and list_notes tools had their own search, on the sync server, and it was the old design again. On the eval it scored 2 of 200 on the main set and 17 of 200 held out, the same as the old app search.

The sync server is a Cloudflare Worker with one SQLite-backed Durable Object per library, which I wrote about in how Gravity Sync keeps both versions. I moved the TypeScript port into a shared package that the phone app and the Worker both import. The Durable Object runs the same search v2 over its own copy of your notes.

The model ships with the Worker as a static asset behind the ASSETS binding, which the Worker reads and never serves over HTTP. Workers have tight CPU limits per request, so the Durable Object indexes in bounded chunks: up to 200 notes before a read, the rest on alarms alongside the nightly backup. If the model ever fails to load, search falls back to words only.

Those times come from running the backend's own code on its schema in Node. In workerd, Cloudflare's runtime, 40,000 notes take about 8 seconds of CPU to index once, and a search takes 7.5 ms. When you ask Claude to find the note about the spare key, it now finds the same note your phone would.

What it still gets wrong

Four searches still fail, two in each set:

The first three need knowledge about the world that a static word model doesn't have. I tried larger static models, combining two models, and expanding searches with the model's nearest words. None fixed them without breaking something else. The fourth is a ranking call between near-duplicates, and the judge didn't count the others as equally good. A small language model could probably fix all four, but it would be hundreds of megabytes and far slower than 3 ms, and that isn't the trade I want for a notes app that opens ready to type.

The eval, the held-out set, the benchmark and the golden files all stay in the repository. A future change to search has to beat 198 without breaking the controls or the speed table, and the four misses are the cases to beat.

Search v2 is in the latest Gravity for iPhone, iPad, Android and Mac, and your library indexes itself in the background the first time you open it. If you search for something you know is there and Gravity doesn't find it, tell me what you typed at the contact page. Each miss becomes a new case in the eval.

Gravity is a notes app for iPhone, iPad, Android and Mac. It's at gravitynotes.app.

The rule I built everything around is that text you wrote never disappears because of sync. A note can change, sink, get archived or get deleted on purpose. It can't vanish because two phones disagreed about what happened while one was in a tunnel.

Why not last writer wins

Most sync engines settle conflicts by timestamp. Each device sends its version of a record, the server keeps the newest, and every device converges on the same answer. Converging is easy. The trouble is what converging throws away.

Say you fix a typo in a note on your Mac while your phone is offline, and on the phone you add a paragraph to the same note. Last writer wins keeps whichever arrives later. If that's the typo fix, your paragraph is gone, and nothing on either screen tells you. Phone clocks drift too, so "later" can mean the device that's set five minutes fast.

Line-level merging does better until both devices touch the same line, and then it picks a winner as well. A notes app gets plenty of those, because most notes are one line.

So Gravity Sync gives up a little tidiness for safety. When two edits to the same text can't both be the current version, it keeps both: one stays as the note and the other sits next to it as a copy marked Conflict. You merge them by hand, which takes ten seconds, and you never have to wonder where a paragraph went.

One database per library

The server is a Cloudflare Worker with one SQLite-backed Durable Object per library. A Durable Object is a small single-threaded server with its own storage, so every change to your library goes through one queue and one database, one at a time. Two phones pushing at the same moment get an order, and each change sees exactly the state the one before it left behind.

The plan was bigger at first. In September I designed sync around PowerSync and a Neon Postgres database. Partway through I replaced both with this, because a single SQLite database per person maps onto the problem directly. Nobody shares a library, there's no cross-library query to make fast, and transactions are local and synchronous. Accounts live in D1, sign-in is Clerk, and each library is backed up to R2 every night.

Operations, not states

Devices don't upload "here's the note now". They upload what the person did. There are nine kinds of operation:

• create, with the note's first text
• text, with the full new text and the revision it was edited from
• bump and rescue, which move a note to the top
• archive and unarchive
• trash, restore and purge (delete for good)

Sending intent instead of state lets the server merge by meaning. If you archive a note on your phone while you edit it on your Mac, those don't conflict, because one changes where the note lives and the other changes what it says, and both apply. If you rescue the same note on two devices, its rescue count goes up by two, because you rescued it twice.

A text operation always carries the whole note and the revision it started from, never a patch. When it arrives, the server compares that base revision with the note's current one:

• If they match, nobody else got there first. The new text becomes the note and the revision moves on.
• If they don't, another device changed the note in the meantime. Its text stays as the note, and the arriving text becomes a copy labelled Conflict.
• If the note was purged, a device that saw the deletion can't bring the text back, and a device that wrote something new before it heard about the deletion keeps that text as a copy marked "deleted on another device".

Fig 2. Edits go through one at a time. Each card's pin marks the revision it was edited from, and the comb has one tooth raised at the note's revision. A pin that fits passes and the card lands on the note. A stale card goes to the copy tray, and a retry of an edit already applied goes on the receipt spike. Open the interactive version.

Saving and syncing are two different promises

Gravity's whole point is that capture is instant, so sync has to stay out of the way of the editor. Every write on the device stores the note and its sync operation in the same SQLite transaction. Either both are on disk or neither is. If the app crashes between "you pressed Return" and "the phone found a network", the operation is still there when it starts again.

The status in Settings keeps the two promises apart. "Changes saved on this device" means your text is safe on this phone. "Up to date" means the server has it too. The app never shows the second while work is still queued.

Retrying without doing things twice

Phones lose responses all the time. The server commits your change, the reply dies in a lift, and the phone has no idea whether it worked. It has to retry, and the retry mustn't rescue the note a second time.

Every operation carries a mutation ID made on the device. The server stores a receipt for each one in the same transaction as the change itself, with a hash of the payload. A retry with the same ID and the same payload gets back the stored result without running again. The same ID with a different payload is refused, because something has gone wrong and guessing would be worse.

Knowing text without keeping it

Joining sync is where many systems duplicate everything. You've got 3,000 notes on your iPhone through iCloud and the same 3,000 on your Mac. Turn on Gravity Sync on the iPhone, then on the Mac, and a naive server ends up with 6,000.

Gravity's server checks whether it has seen a piece of text before, even text it no longer stores. For every revision a note has held, it keeps a keyed hash of the text, using HMAC-SHA256 with keys that never leave the server. When the Mac joins and offers a note, the server checks the hash. If the note has held that exact text at any point, now or earlier, the Mac's copy adds nothing. If the text is new, it becomes a Conflict copy. The server knows "I've had this text" without storing old versions of your notes.

Order comes along too. Each note keeps its creation time and its place in the list when it joins, capped at the server's clock, so the list on the new device looks the way you left it.

Trying to break it

A bug in merge code doesn't crash anything. It loses a sentence somebody wrote in March, and they notice in June. I wanted tests that went looking for that.

The merge core is a pure TypeScript module with no network, no clock and no randomness of its own. The server passes in the time, the ID generator and the hashes. That makes it cheap to run thousands of times:

• It runs against an in-memory store and against the real SQLite schema with its triggers, and both must agree.
• 21 recorded concurrency cases from early prototypes replay on every check.
• A property-based simulator runs three devices doing all nine operations with offline stretches, reordered delivery, lost responses and retries. After every run it checks one rule: every text anyone committed still exists as a note, a copy or a queued operation, unless someone deliberately replaced or deleted it on a device that could see it.
• The same runs with skewed clocks have to reach the same merge results.
• Mutation testing switches off individual merge rules in a scratch copy and requires the tests to fail. A rule no test notices missing isn't really tested. That run takes about half an hour, and I run it after every change to the merge code.

What it costs

Speed mattered as much as safety, because nobody keeps a sync feature that makes the app feel heavy. In my tests with a 10,500-note library, a fresh iPhone downloaded everything in 30 seconds and Android in 44. A new note written on an iPhone reached the server in about two and a half seconds. A single note can hold up to 1 MB of text, which is longer than most novels.

The trade-offs I chose

Gravity Sync isn't end-to-end encrypted. Notes are encrypted in transit and at rest, and the server can read them, which is what lets it merge by meaning and lets you sign in on a new phone with Google or Apple and get everything back with no recovery key. Losing a phone is the disaster most people actually have, so I chose easy recovery.

It also keeps no version history yet. Conflict copies cover the case where two devices disagree, but you can't scroll back through last month's edits. The revision IDs and hashes are already there, so history can come later without changing the protocol.

Gravity Sync is free during the beta. Turn it on in Settings on any device, sign in with Google, Apple or an email code, and your notes follow you, including between an iPhone and an Android phone. If you find a way to make it lose a note, I want to hear about it at the contact page.

Gravity is a notes app for iPhone, iPad, Android and Mac. It's at gravitynotes.app.