Search Strategies

A detailed comparison of the three search methods available in the wiki, with practical guidance for choosing the right tool and interpreting results.

1. wiki_search — GET /api/search

Mechanism: Substring / token-based matching against page titles and bodies.

How it works: The endpoint scans the wiki's indexed content for occurrences of the search terms. Matches are scored primarily on lexical proximity — how often the terms appear and where (title matches rank higher than body mentions).

Strengths:

• Fast and predictable; deterministic results based on exact terms.
• Excellent when you know a specific word, phrase, or acronym and want to find every page containing it.
• Title matches surface prominently, making it great for name-based lookups (e.g., "MVC", "DTO", "REST").
• No semantic interpretation overhead — no false positives from "similar" concepts.

Limitations:

• Blind to synonyms and related concepts. Searching for "banking" won't find a page titled "financial institutions."
• No context awareness — "Java" matches the island, the coffee, and the language equally.
• Fragile to terminology shifts; if a page uses a different name for a concept you know, it's invisible.

Best used when: You know a specific word or phrase and want to find pages that contain it.

2. wiki_find — GET /api/find

Mechanism: A semantic + lexical blend. The query is embedded into a vector space (semantic) and also matched lexically, with scores fused to produce a ranked result set.

How it works: Your query is processed through a text embedding model to capture meaning, then combined with traditional keyword matching. The endpoint returns results with two additional diagnostic fields:

• understood — the natural-language paraphrase the embedding model derived from your query, showing what concept it tried to match.
• unknown — any terms from your query that the embedding model could not map to a known semantic concept, indicating gaps in its training vocabulary.

Strengths:

• Finds pages by meaning, not just by keywords. "How to connect a database to a web app" will surface pages about ORMs, connection strings, and dependency injection even if none of those exact phrases appear in the query.
• Handles natural-language descriptions well — no need to guess the right search terms.
• The understood field provides transparency into the model's interpretation, letting you adjust your query if it misunderstood you.
• The unknown field helps identify when your query contains jargon or acronyms the model doesn't recognize.

Limitations:

• Slower than wiki_search due to embedding computation.
• Can surface irrelevant results when the semantic model latches onto the wrong concept (hence the understood diagnostic).
• Results are less deterministic — two very similar queries may return different results due to embedding sensitivity.
• Doesn't guarantee exact keyword matches; a page may rank low even if it contains your exact terms.

Best used when: You have a description of a concept but don't know the right keywords. You want to explore by meaning rather than by name.

Reading the diagnostic fields:

• If understood captures your intent → the results are likely on target.
• If understood diverges from what you meant → try rephrasing or adding clarifying terms.
• If unknown contains significant terms → those terms were ignored semantically; add synonyms or spell out acronyms.

3. wiki_related — GET /api/related

Mechanism: Follows explicit links and shared tags from a known starting page to discover connected content.

How it works: Given a page slug, the endpoint scans:

1. Outbound links — other pages linked from the starting page (direct hyperlinks).
2. Inbound links — pages that link to the starting page.
3. Shared tags — pages that share at least one tag with the starting page.

Results are typically ranked by the number of shared connections (pages linked both directly and through tags will rank higher).

Strengths:

• Surface-level and deep connections alike — finds pages you'd never think to search for by following the wiki's own link structure.
• Highly relevant by construction: related pages share context (links) or classification (tags).
• No query construction needed; just provide the seed page.
• Excellent for exploration and discovery — follow the wiki's topology like a recommendation engine.

Limitations:

• Only works from pages that have links or tags. An orphan page (no links, no tags) returns nothing.
• May miss tangential topics not explicitly linked or tagged together.
• The quality of results depends entirely on how well the wiki is linked and tagged — sparse linking means sparse results.
• Doesn't find conceptually related pages that exist in the same knowledge space but aren't connected by links.

Best used when: You're reading a page and want to discover adjacent or supporting topics. Use it as a navigation tool within the wiki's own structure.

4. Decision Tree: Which to Use?

You have a query → What do you know?

│
├─ You know a specific word, name, or acronym
│  └─→ wiki_search (/api/search)
│      Why? Fast, deterministic, exact-match focused.
│      Example: "What pages mention 'dependency injection'?"
│
├─ You have a description or question in natural language
│  └─→ wiki_find (/api/find)
│      Why? Semantic model captures meaning beyond keywords.
│      Example: "How do I persist data between requests?"
│      Check 'understood' to verify intent; check 'unknown'
│      for unrecognized terms.
│
├─ You're already reading a relevant page
│  └─→ wiki_related (/api/related)
│      Why? Follow the wiki's own links and tags.
│      Example: Read "session-management", then call related
│      on that page to find "token-based-auth" and "jwt".
│
└─ You're not sure → Use both
   ├─ Start with wiki_search for exact keywords you can think of.
   └─ Follow with wiki_find to catch related concepts you missed.

Quick reference:

5. Score Interpretation

Lexical Score (wiki_search)

The lexical score is a keyword-matching score. Higher values mean:

• The search terms appear more frequently in the page.
• The terms appear in higher-weight locations (title > heading > body).
• Multiple terms from the query are present (vs. a single-term match).

Interpretation:

• High lexical score, no semantic score → This is a wiki_search result. The page literally contains your terms. Trust exact keyword presence.
• Pages with high lexical scores are almost certainly relevant, but may be irrelevant if the terms are ambiguous (e.g., "Java" the language vs. "Java" the island).

Semantic Score (wiki_find)

The semantic score is a vector-space similarity score (cosine similarity between the query embedding and the page embedding). Higher values mean:

• The page's content is semantically closer to the query's meaning in the embedding space.
• The page covers the same concepts, even if the exact words differ.

Interpretation:

• 0.80–1.00: Very strong semantic match. The page almost certainly covers the same concept as your query, even with different vocabulary.
• 0.60–0.80: Good semantic match. Likely relevant; worth reading to confirm.
• 0.40–0.60: Moderate match. The page touches on related territory but may not be the primary answer.
• 0.20–0.40: Weak semantic match. Probably tangential or a partial match.
• Below 0.20: Very weak. Likely noise; treat as exploratory at best.

Combined Scores (wiki_find)

When both lexical and semantic scores are present:

• High lexical + high semantic: The page is a strong, confident match. It contains the right terms AND covers the right concept.
• High lexical + low semantic: The page mentions your terms but isn't semantically on point. Could be a tangential mention or an ambiguous term. Investigate.
• Low lexical + high semantic: The page covers the concept but uses different vocabulary. This is a discovery hit — read it; it's likely highly relevant despite missing your keywords.
• Low lexical + low semantic: Skip. Neither mechanism found a meaningful connection.

Practical Tips

1. For wiki_search: Trust lexical scores above 0.7 for direct relevance. Below 0.4, review results manually — they may be false positives.

2. For wiki_find: Pay more attention to the understood field than to raw scores. A score of 0.5 with a well-matched understood paraphrase is often more useful than a score of 0.8 with a misinterpreted one.

3. Hybrid strategy: Use wiki_search first to ground your query in known terminology, then wiki_find to discover semantically related pages that use different vocabulary. This two-step approach catches both known terms and unknown synonyms.

4. For wiki_related: There is no score — results are ranked by connection count. The more links and shared tags a page has with the seed, the higher it ranks. This is inherently a relevance ranking, so trust the order.