Building AI Ex-Partner Chat: OCR, Sender Detection & the 3 AM Code That Shouldn’t Exist

The problem that started at 3 AM

I built AI Ex-Partner Chat because I couldn't sleep. I kept re-reading old WeChat messages, and I thought: what if I could distill her personality from those messages into something I could actually talk to? Not a replacement — just a shadow. A reflection of the way she used to write, the phrases she repeated, the tone she took when she was being tender versus when she was being impatient.

But the moment I started building it, I hit a wall that had nothing to do with AI or language models. It was a much older, much harder problem: who said what?

When you upload a chat export — a WeChat HTML file, a Telegram JSON — the sender is explicitly labeled. Easy. But most people don't have clean exports. They have screenshots. Hundreds of screenshots. And a screenshot is just pixels. There's no metadata, no sender field, no structured data. Just an image with two columns of text bubbles, and you have to figure out which ones are hers and which ones are yours.

Get this wrong, and the AI persona becomes a chimera — half you, half her, completely confused. It would respond to "I miss you" with your own words reflected back at you. That's not closure. That's a mirror maze.

Version 1: Pixel-based avatar detection (and why it failed)

My first approach was the obvious one: look for avatars. In WeChat, QQ, and Douyin, each message has a small avatar next to it — on the left for the other person, on the right for you. If I could detect those avatar regions, I could label every message.

The algorithm worked like this:

1. Scan the left 15% and right 15% of the image for clusters of non-background pixels (avatar blobs).
2. When a cluster is found on the left, mark that Y range as "their message." Right side = "your message."
3. OCR each message zone and assign the detected sender label.

It worked... sometimes. Clean screenshots with visible avatars were easier, but this diary does not preserve a reproducible accuracy benchmark. The approach failed on cases such as:

• Cropped screenshots where avatars were cut off
• Group chats with multiple avatars on both sides
• Douyin where avatars are circular and sometimes overlap with the message bubble
• WhatsApp which doesn't show avatars in the same way
• Any screenshot with stickers, images, or voice messages that confused the pixel scanner

The fallback was bubble color detection — green bubbles mean you in WeChat, white means them. Blue = you in Douyin, white = them. But bubble colors vary by theme, dark mode, and Android version. It was a fragile heuristic stacked on top of another fragile heuristic.

I needed something fundamentally different.

Version 2: Tesseract bounding boxes + position-based detection

The breakthrough came when I realized that Tesseract.js doesn't just return text — it returns bounding boxes for every recognized line. Each line comes with bbox.x0, bbox.x1, bbox.y0, bbox.y1 — the pixel coordinates of where that line sits on the page.

This is a game-changer because chat messages have a spatial structure:

• The other person's messages start from the left side of the screen
• Your messages start from the right side of the screen
• This is true across WeChat, QQ, Douyin, WhatsApp, Telegram — virtually every chat app

So instead of trying to detect avatars or bubble colors, I could just look at where the text starts. Here's the core algorithm:

// Get image midpoint
const MID_X = imageWidth / 2;

for (const line of ocrLines) {
  const bbox = line.bbox;
  const lineLeft = bbox.x0;     // Where the line starts
  const lineCenterX = (bbox.x0 + bbox.x1) / 2;

  let sender;
  if (lineLeft < MID_X * 0.6) {
    // Left portion → ex-partner
    sender = '对方';
  } else if (lineLeft > MID_X * 0.8) {
    // Right portion → you
    sender = '我';
  } else if (lineCenterX < MID_X * 0.75) {
    // Center-left → likely ex-partner
    sender = '对方';
  } else if (lineCenterX > MID_X * 1.1) {
    // Center-right → likely you
    sender = '我';
  } else {
    // Ambiguous → carry previous sender
    sender = currentSender || '';
  }
}

The thresholds (0.6, 0.8, 0.75, 1.1) are heuristic multipliers of the image midpoint, not percentages of the full width or validated accuracy estimates. The main signal is the left edge (x0), with center position as a fallback. Different crops, layouts and languages can still produce incorrect labels.

This approach is:

• Theme-agnostic: Doesn't care about dark mode, bubble colors, or avatar visibility
• App-agnostic: Works across WeChat, QQ, Douyin, WhatsApp, Telegram
• Robust to cropping: As long as the left/right structure is preserved, it works
• Fast: No pixel scanning needed — just math on coordinates Tesseract already provides

Filtering interface text and OCR noise

OCR on a chat screenshot can also recognize interface text unrelated to the conversation, including:

• Timestamps like "16:24" and "06/02 20:35"
• UI labels like "发送消息" (Send Message) and "消息" (Messages)
• Navigation elements, battery indicators, signal bars rendered as text
• Random noise from image compression artifacts

Before position-based detection can work, you have to filter this noise out. The garbage filter uses three rules:

// Rule 1: Skip timestamps
if (/^\d{1,2}[:\/]\d{2}/.test(text) && text.length < 12) continue;

// Rule 2: Skip UI elements
if (/^(发送消息|发送|消息|Message|Send)/i.test(text)) continue;

// Rule 3: Skip lines with >70% non-meaningful characters
const cjk = text.match(/[\u4e00-\u9fff\u3400-\u4dbf]/g) || [];
const letters = text.match(/[a-zA-Z]/g) || [];
const meaningful = cjk.length + letters.length;
if (meaningful / text.length < 0.3 && text.length > 4) continue;

The filter removes some non-message text, but also has a real limitation: the current code skips lines shorter than two characters, so a one-character reply such as "嗯" can be lost. For lines longer than four characters, fewer than 30% CJK characters or Latin letters triggers another rejection. Review the parsed record; filtering is not lossless.

Step 1.5: The sender confirmation step

Position-based detection is good, but it's not perfect. Some screenshots have unusual layouts. Some messages are centered (like system notifications). Some users screenshot in landscape mode, flipping the left/right convention.

Rather than trying to handle every edge case algorithmically, I added Step 1.5: Sender Confirmation. After OCR and initial sender labeling, the tool pauses and shows the user a preview of the first 15 detected messages with their assigned senders:

• Messages labeled [对方] (ex-partner) appear on the left
• Messages labeled [我] (you) appear on the right
• A Swap Senders button flips all labels if the detection got them backwards
• A Confirm button proceeds to distillation

This is a deliberate design choice: let the algorithm do the heavy lifting, but keep the human in the loop for the final call. The alternative — fully automated detection with no review — would produce silently wrong results that corrupt the entire persona. A 10-second confirmation step prevents hours of distorted conversations.

OCR pre-processing: Why "想" and "相" matter

Chinese OCR can confuse similar-looking characters. 想 contains 相 plus the 心 component; they do not differ by one horizontal stroke. In a low-resolution image, losing the lower component can turn "我想你" into "我相你", creating a meaning-changing error. The original screenshot remains the reference for correction.

The fix is aggressive pre-processing before OCR:

1. 2x upscale: Each detected message zone is enlarged to double its original size using bilinear interpolation. This gives Tesseract more pixels to work with for each character stroke.
2. Grayscale conversion: Color information is irrelevant for character recognition and adds noise. Converting to grayscale simplifies the recognition problem.
3. Contrast stretching: The pixel values are remapped so the darkest pixel becomes pure black and the lightest becomes pure white. This sharpens character edges against colored chat bubble backgrounds.

Pre-processing can make some characters easier to recognize, but no reproducible accuracy percentage is established here. A subsequent AI correction request uses conversation context to suggest fixes; it can also invent words or misassign speakers. Check important text against the screenshot instead of assuming the AI catches every error.

The shared Tesseract worker: Minutes to seconds

Here's a performance mistake I made early on: calling Tesseract.recognize() for each chat bubble individually. Every call creates a new WASM worker, loads the language model, recognizes the text, and terminates the worker. For a screenshot with 30+ message zones, this meant 30+ worker startups — each taking 5–10 seconds. Total OCR time: 3–5 minutes per screenshot.

The fix was a shared worker pattern:

// Create ONE worker, reuse for all zones
const worker = await Tesseract.createWorker('chi_sim+eng');
const results = [];

for (const zone of messageZones) {
  const { data } = await worker.recognize(canvasForZone);
  results.push(data);
}

await worker.terminate();

Reusing a worker can avoid repeated initialization when processing many zones. The current implementation recognizes a prepared screenshot in one call and terminates that worker afterward. The sample pattern above illustrates the reuse idea; it is not the current per-image lifecycle or a speed guarantee for every device.

Persona distillation: The 6-step pipeline

After parsing and sender review, the AI stages call the user's configured provider. DeepSeek was the original integration discussed in this diary; the current tool also supports compatible providers. Imported text is subject to request limits, and the provider's fees and retention policy apply:

1. OCR Correction + Sender ID — DeepSeek reviews the raw OCR text, fixes misrecognized characters, and verifies sender labels based on conversation context
2. Intake — the AI summarizes the excerpts included in the request, not necessarily every imported message
3. Memory Extraction — Identifies core memories, shared experiences, and significant moments
4. Persona Distillation — Builds a detailed personality profile: speech patterns, emotional tendencies, recurring phrases, humor style, communication habits
5. System Prompt Generation — Converts the persona into a chat system prompt with identity guard rules
6. Chat Ready — The user can now converse with the distilled persona

The main distinction is between conversation context and messages attributed to the other person. Current distillation inputs cap the context excerpt at 8,000 characters and the other-person excerpt at 4,000, after earlier message-count selection. Prompts instruct the model to extract style from [对方] and use [我] only as context. These are instructions, not a guarantee against blending or hallucination.

Without this separation, the AI would blend both people's speech patterns into a single confused persona. The strict ex-only extraction is what makes the persona feel like them instead of both of you.

The identity guard: Preventing persona drift

One of the creepiest bugs in early testing was persona drift: after a long conversation, the AI would slowly start adopting the user's speech patterns. If you kept saying "haha" and "lol," the persona would start saying it too. If you were being warm and affectionate, the persona would mirror that warmth regardless of the ex's actual personality.

The fix was an identity guard rule added to every system prompt:

You are [name], NOT the user.
Never mirror the user's speech patterns.
Your personality is fixed based on the distilled persona.
Do not adapt to the user's tone, mood, or vocabulary.

The persona description is included in later requests to help maintain the intended style. A model does not remember like a person; context limits and generation variability still allow drift. Repeating the instruction reduces one source of inconsistency without guaranteeing a faithful replica.

IndexedDB: Where personas live (and die)

All persona data is stored in the browser's IndexedDB — two object stores:

• persona — The distilled personality profile, name, avatar, memory summary, corrections
• chats — Conversation history with auto-incrementing IDs

Why IndexedDB instead of localStorage? Two reasons:

1. Capacity: IndexedDB is better suited to larger structured records than localStorage, but it still has browser-specific quotas. Storage can fail or be evicted, and private browsing has different retention behavior.
2. Structured storage: IndexedDB supports binary data (avatar images) and complex objects natively, without JSON serialization overhead.

There is also /let-go. In the current tool it opens a confirmation with a short countdown; deletion happens only after confirmation. It clears this tool's local persona and chat stores, provider settings and active key, with no in-tool undo. Original files, downloaded exports, consent/language settings and provider-side records remain separate. The emotional idea of letting go is personal; the software action has a specific, limited scope.

The consent overlay

Creating an AI replica of a real person is a serious act. I added a 5-second consent countdown before distillation begins. The user sees:

You are about to create an AI persona based on real chat records. This persona will simulate another person's communication style. Please take a moment to consider whether this is something you truly want to do.

The countdown can't be skipped. You can't click "I understand" until the timer reaches zero. This isn't a legal formality — it's a genuine pause. If 5 seconds feels too long, maybe you shouldn't be doing this.

The hardest bug wasn't technical

Technical stack summary

Component	Technology	Why
OCR Engine	Tesseract.js (WASM)	Client-side, supports Chinese + English, returns bounding boxes
AI Model	DeepSeek (user's API key)	Direct browser-to-provider requests; model quality, costs and retention vary
Local Storage	IndexedDB	Local structured storage with browser quotas and eviction limits
Sender Detection	Bbox position + AI verification	Position-based primary, DeepSeek secondary, user confirmation final
Image Pre-processing	Canvas 2D API	2x upscale, grayscale, contrast stretch — all client-side
Data Lifecycle	/let-go → IndexedDB.clear()	Confirmed local deletion, no in-tool undo; external copies remain separate

What I'd do differently

If I were building this again from scratch, here's what I'd change:

• Start with bounding boxes, not pixel scanning. I wasted two days on avatar detection before realizing Tesseract already gives you position data for free.
• Add the confirmation step earlier. The sender confirmation (Step 1.5) was a late addition, but it should have been there from day one. Algorithm-only detection will always have edge cases; letting the user verify is both more reliable and more respectful.
• Use a shared Tesseract worker from the start. The per-zone recognize() approach was a naive performance mistake that cost minutes of user wait time.
• Design for /let-go first. I built the chat functionality before building the deletion mechanism. In retrospect, the ephemeral nature of the tool should have been the first thing I implemented, not the last.

Frequently asked questions

How does the OCR sender detection work?

Tesseract.js supplies line bounding boxes. The current heuristic compares x0 with 0.6 and 0.8 times the image midpoint, equivalent to 30% and 40% of full width. It then uses center-X thresholds of 37.5% and 55%, followed by the previous sender or an unknown label. These geometry rules can be wrong; review the sender preview before continuing.

Why not use AI for sender identification instead of position-based detection?

The tool actually uses both. Position-based detection via Tesseract bounding boxes provides the initial labels, then DeepSeek performs a second pass for OCR error correction and sender verification. The position-based approach is faster and more reliable for well-structured chat screenshots, while the AI pass catches errors and handles edge cases.

How is persona data stored?

The persona and chat records are stored in this browser's IndexedDB. AI requests send selected context to your configured provider, whose retention policy applies; ordinary site analytics and resource requests are separate. The /let-go command requires confirmation and clears this tool's local stores, settings and active key, not every site's IndexedDB or external copies.

What was the biggest technical challenge?

The sender identification problem. Early versions used pixel-based avatar and bubble color detection, which was fragile and failed on many screenshot formats. Switching to Tesseract.js bounding box position detection and adding a user confirmation step (Step 1.5) dramatically improved accuracy while keeping the user in control.