খোঁজ Khoj — Million-Line Text Reader
Open and search huge English and Bengali text files in a browser tab, without the file ever leaving your machine.
- Role
- Designer & engineer
- Team
- Solo
- Timeline
- Sep 2026
- Organisation
- Personal
- Code
- Private repository
- Reading time
- 2 min
One Web Worker per file streams 16 MB slices, indexes line offsets in typed arrays and runs cancellable searches; a custom virtual reader scrolls millions of lines without hitting the browser’s maximum element height.
Overview
Text editors choke on multi-hundred-megabyte files, and most don't render Bengali well. Khoj ("search" in Bengali) opens them instantly, searches as you type, and lets you pick any Bengali font — including fonts installed on your machine.
The problem
- Files far larger than memory-friendly strings.
- Multi-byte Bengali characters split across read boundaries.
- Browsers cap element height, so a naive virtual list breaks past a few million lines.
Context & constraints
Local-only by design: no server, no upload.
Approach
- A worker per file reads 16 MB slices with a streaming
TextDecoder, so a Bengali character split across slices still decodes correctly, and records line offsets in chunkedUint32Arrays. - Cooperative search yields to the event loop, cancels via a generation counter, streams progress and keeps up to 100,000 matches while still counting the total.
- An anchor-based reader scrolls by line + offset instead of a giant spacer element, batches wheel input per animation frame and measures variable row heights when wrapping is on.
- Fonts — six Bengali web fonts on demand, detection of common installed Bangla fonts, and the Local Font Access API to list every installed font.
Key capabilities
- Multiple files via drag-and-drop or ⌘O
- Live search with regex, match case and wrap
- Jump between matches with Enter / Shift+Enter
- Bengali font picker including local fonts
- LRU cache of 256-line blocks
Architecture
Main thread
One worker per file
The main thread only renders; workers own the bytes. Select a component to trace its connections.
Key decisions
ADR-01Anchor-based scrolling
- Context
- Browsers cap element height (around 33 million pixels in Chromium), so spacer-based virtual lists overflow.
- Decision
- Track position as "line N plus pixel offset" and render a window around it.
- Trade-offs
- A custom scrollbar and wheel handling, but no upper limit on lines.
Quality & delivery
Deployment
Static build; runs from any static host or a local file server.
Outcome
- Read slice size
- 16 MB
- Streaming decode
- Stored matches
- 100k
- Total still counted
Opens million-line Bengali and English files smoothly on a laptop.
Lessons
- The main thread is for pixels. Everything else belongs in a worker.
Building something similar?
Tell me what you’re working on — I reply to every serious message.
Related work
Sangsar — Household Finance PWA
Log any money movement in about ten seconds in either language, then see the month at a glance. A repository layer swaps IndexedDB for Firestore, offline writes sync safely, and 760 unit tests plus 11 end-to-end flows keep it honest.
- TypeScript
- React
- Tailwind CSS
- Vite
- PWA & service workers
- Zustand
- +4