Search Internals

Odyn has no package registry. odyn search anima still hands you a list of Odin packages, each ready to install. This page is about how that happens without any server Odyn controls.

The one idea

Most package managers own a central index. Someone runs it, packages publish to it, the tool queries it. That index is convenient, and it is also the thing that makes a package manager a gatekeeper.

Odyn keeps a registry of lists instead. A list is a git repository that happens to describe some packages. Odyn hosts none of them and vouches for none of them. It reads what you point it at.

   your git URLs ──► registry of lists ──► ranked matches for a query

So the job of search is narrow: take the handful of URLs you added, and turn them into a flat, ranked answer. Everything else is detail. Let us follow odyn search anima.

Two places, kept apart

Search touches two locations, and confusing them is the only real way to get lost.

Registry

The lists you added. The source of truth. If Odyn cannot read it, it stops.

Cache

Copies of those lists, kept for reading. Throwaway. Delete it and search rebuilds it.

The registry lives with your config; the cache lives with your other caches. The invariant behind everything: the cache is always rebuildable from the registry, and never the reverse.

Naming a list on disk

A list is identified by its URL, but a URL makes a terrible directory name. So the cache names each list’s folder after a hash of that URL.

awesome-odin  ─►  https://github.com/jakubtomsu/awesome-odin
                            │  hash
                            ▼
              ~/.cache/odyn/lists/de425e5483e8547d/

The hash is never written down. It is a plain function of the URL, so search recomputes it every run and lands in the same place. Nothing to store, nothing to keep in sync.

The URL is the right thing to hash, and the alternatives are worse:

1. A list’s name is yours to choose, so two can collide, and names carry characters that break paths.
2. A list’s latest commit changes on every push, so the cache would miss after every update.
3. The URL is stable and unique. Same list, same folder, always.

Since those folder names are unreadable, Odyn leaves a guide next to them, INDEX.txt, mapping each hash back to a name and URL. Odyn never reads it. It is there for a human poking around the cache.

Caution

INDEX.txt is a picture of the registry, so it must be redrawn whenever the registry changes. Odyn does that inside the act of writing the registry itself, so the two cannot drift.

For Nerds

The hash algorithm Odyn uses is XXH3_128. It’s extremely fast, deterministic, and has high collision resistance. It’s much faster than SHA256 but cryptographically insecure, which doesn’t matter for Odyn at all since it doesn’t need to be.

Fetching, cheaply

To read a list, Odyn needs its repository, and it takes the smallest copy that works: shallow, with file contents deferred until something actually reads them. A few forges do not support that, so a plain shallow clone is the fallback.

not cached ──► thin clone ──► fails? ──► plain shallow clone
cached     ──► fetch latest ──► fails? ──► search the copy you already have

That last arrow matters.

Stale beats broken

Offline, or the host is down? Odyn searches the copy it already has instead of failing. An old list beats no results.

Throughout the fetch, search shows a single line for the whole batch, not one per list:

◜ Searching for 'anima' in 2 lists...

This is a deliberate difference from sync, which reports every dependency it restores. In sync, that report is the only place you learn what happened, so it earns its noise. In search, the results are the report, so narrating each fetch would only say the same thing twice.

Choosing what to read

A cloned list is a folder. Which file in it lists the packages?

Odyn_Index.toml present?
   │
  yes ──► read the file it points to, in the format it names
   │
   no ──► read README.md as a markdown table

Odyn_Index.toml holds no packages itself. It is a signpost: this file, in this format. When it is missing, Odyn falls back to the README that already exists, and this is where the design earns its keep.

Zero cooperation

An ordinary “awesome-odin” list becomes a working Odyn source with no new files and no help from its maintainer. Odyn just reads their README.

One shape, three roads

However a list is read, it yields the same thing every time: a package with a name, a repo, a description, a license, and tags. Three parsers get there, and they all end at the same shape.

   TOML  ─┐
   JSON  ─┼─►  a package  { name, repo, description, license, tags }
 Markdown ─┘

TOML

Structured and exact, chosen by an author who wants control. A real TOML parser reads it, so a malformed file fails loudly instead of being quietly misunderstood.

An author writes format = "toml" in the index and points it at a file of [[package]] blocks.

JSON

The same structured road for authors who would rather write JSON. It carries the same fields and the same arrays, so tags stays a real list rather than a string someone has to split by hand.

An author writes format = "json" and points at a file shaped like { "packages": [ ... ] }.

Markdown

Tolerant, for READMEs nobody wrote for Odyn. Odyn scans every table and keeps the ones shaped like package tables: a name column whose cells hold the repo as a link, and a description column. Several such tables in one document collapse into a single flat set.

The TOML and JSON roads are structured formats an author opts into; the Markdown road is the fallback that needs no opt-in at all. What matters is that none of them is special downstream. Search consumes a list of packages and never asks which parser produced it, so a new road is only ever a new producer of the same shape. That is exactly why adding the JSON road cost so little: one parser, and nothing else in search had to know.

Trusting a stranger's README

An “awesome” README mixes real packages with gists and dead links. Odyn accepts a row on a forge it recognizes, and skips a row on one it does not, unless the author added an explicit trust marker to the URL on purpose.

Matching

Now the packages exist, and the query has to find its own among them. Odyn compares anima against two fields: the display name and the repo slug, because people search by both.

"anima"
   │
   ├─ appears inside the target?  ──► match, best score   (substring)
   │
   └─ close enough to it?         ──► match, ranked lower  (typo tolerance)

The second tier only runs for queries of five characters or more. Below that, edit distance is meaningless: almost anything is two edits from a three-letter word, so a short query would drag in noise. Short queries stay exact.

Facets stack on top as filters that narrow, not rank:

odyn search anima --tag 2d --license MIT
   │
   └─ name matches "anima"  AND  tagged 2d  AND  MIT licensed

Only the name produces a relevance score; the facets are gates. A package survives if it clears every active one. When there is no name at all, results fall back to alphabetical order.

The answer

The survivors are sorted, the spinner is wiped, and its line is replaced by the result block. Each line names the source list as a colored prefix, then the package’s install slug.

awesome-odin:asbott/jamgine
mylist:someone/anima (Codeberg)

help: to install a package, "get" without the list prefix and adjust the platform

    odyn get asbott/jamgine
    odyn get someone/anima -p codeberg

note: found 2 results for 'anima' on 2 lists

Two quiet decisions live in that block.

A forge Odyn recognizes gets a label and the matching -p flag in its install hint; one it does not gets its full URL, which is exactly what get accepts anyway. The same knowledge drives the label and the flag, so what search shows and what get wants can never disagree.

And the help block teaches, rather than lists. It shows the top result as the base example, and adds a -p line only if some result actually needs one. Ten results do not print ten commands; they print the one pattern you needed to see.

The whole path, at a glance

1. Read the registry of lists.
2. For each list, find its cache folder by hashing the URL, and fetch a thin copy.
3. Resolve what to read: the indexed file, or the README.
4. Parse it into packages, by whichever of the three roads fits.
5. Match the query and facets, and rank what survives.
6. Replace the spinner with the answer, install hints and all.

None of it touches a server Odyn owns, because there is none. There are only the lists you chose, read fresh each time.