I Built My Own Search Engine Because Google Stopped Being One

Searching the Web Stopped Meaning Finding Things

Somewhere in the last few years, searching the web stopped meaning finding things. You get ads, SEO sludge, and the same twelve domains regardless of what you asked. If you work with AI agents, it's worse — they inherit all of that noise as "ground truth."

So I built Odin: a self-hosted search engine that runs entirely on my own hardware.

A Map, Not a Cache

The core idea is different from a traditional crawler. Odin doesn't try to mirror the internet — that race is unwinnable from a home server. Instead it maintains a map of hubs: the pages that reliably point to good answers for a topic.

Ask it about self-hosted photo management and it doesn't return a stale snapshot — it knows which community wikis, forums, and repositories are authoritative for that space, then fetches the current content live.

The index right now: over 250,000 pages across 10,000+ domains, with full embeddings and around 300,000 extracted entities. Small by Google standards. But it's my 250,000 pages — curated toward the topics I actually research, with no advertiser thumb on the scale.

The Unglamorous Hard Part: Actually Fetching Pages

Everyone thinks search is about ranking. Half the real work is that the modern web actively resists being read. Cloudflare walls, JavaScript-only rendering, bot detection that blocks anything without a full browser fingerprint.

Odin's fetcher runs a four-level fallback chain — from a plain HTTP request all the way up to a full browser session — and currently lands about a 95% success rate on pages that a naive fetch fails on. Video extraction has its own stack of 13 strategies for the sites that bury media behind players.

The fetcher alone is arguably the most sophisticated component. Getting content out of the modern web is an arms race, and Odin is winning more engagements than it loses.

Why This Matters for AI

Odin doubles as an MCP tool server — it speaks the protocol Claude and other AI assistants use. When my agent researches something, it doesn't get "what a search engine's ad model wants me to see." It gets live content from sources chosen for authority.

Research quality went up noticeably, especially for developer topics, self-hosting, and anything the big engines quietly bury. In side-by-side tests against Google across categories I care about — AI tooling, self-hosted software, developer documentation, privacy tools — Odin wins more categories than it loses. Not because it's smarter. Because it isn't fighting me.

What You Get

• Self-hosted Odin instance — Docker-ready, runs on your own hardware
• MCP server — plugs directly into Claude Code or QuetzaCodetl for AI agent search
• Hub discovery engine — maps authoritative sources for any topic you research
• Four-level fetch fallback — HTTP → Playwright → Selenium → curl, 95% success rate
• Full embeddings + entities — 250K+ pages with semantic search
• Setup guide + MCP config — get running in under 30 minutes