Moogle!

A search engine over a folder of plain-text documents, built on the vector space model with TF-IDF weighting, cosine ranking, contextual snippets, spelling suggestions, and four query operators.

Programming I course project, Facultad de Matemática y Computación, University of Havana, 2022. Written in C# with a Blazor Server front end — engine and ranking implemented from scratch, no search library.

🇪🇸 Spanish version: README.es.md · Deeper technical write-up (Spanish): ReadmeMoogleInfo.md · Original assignment brief (Spanish): ReadmeOrden.md

---

How it works

Indexing

Runs once at startup, before the app serves anything. Every document in Content/ is normalised — punctuation stripped, stray characters removed from tokens — and reduced to an inverted index recording, per document: each term, its raw frequency (TF), the document's length, and globally the number of documents containing each term (IDF).

$$\text{TF-IDF}_{w,d} = \frac{\text{TF}_{w,d}}{|d|} \cdot \log\frac{N}{\text{DF}_w}$$

Single-document edge case: with $N = 1$, $\log(N/\text{DF}_w) = \log 1 = 0$, which would zero out every weight. The implementation falls back to plain TF in that case rather than returning nothing.

Ranking

Documents and the query both become vectors whose components are terms and whose values are TF-IDF weights. Relevance is cosine similarity — computed as a dot product over the terms the query and document share, rather than materialising and normalising full vectors. Since a query touches a handful of terms out of a vocabulary of thousands, this is the difference between scanning the query's terms and scanning the whole index. Results are sorted by descending cosine, which lands in $[0, 1]$: 1 is a document fully matching the query, 0 an unrelated one.

The geometric construction is sketched in VectorSpaceModel.odg.

Query operators

The query is treated as a pseudo-document — normalised, TF computed — and its operators extracted first. All four are written without spaces around the term:

Operator	Syntax	Effect
Proximity	this~is~an~example	Terms should appear close together. Scores $1/\text{distance}$ between occurrences, so widely separated matches contribute almost nothing. Works across nested groups and across distinct terms in the document.
Importance	*word, **word	Boosts a term. Each * adds 0.25 to its weight; repeat as needed.
Exclusion	!word	Drops every document containing the term.
Requirement	^word	Keeps only documents containing the term.

Snippets

The snippet is chosen as the substring carrying the most query-term weight — solved as a maximum-sum subarray over per-position TF contributions. Three refinements over the naive version:

• It runs over the original text, not the normalised one, so punctuation and capitalisation survive into the result — at no extra cost.
• It counts distinct query terms, so a passage repeating one word doesn't outscore a passage covering several.
• Terms boosted by the importance operator carry that boost into snippet selection, so the snippet reflects what the user emphasised.

Suggestions

When the query yields too few results, Moogle takes the terms that matched nothing and finds the closest term in the corpus — defined as a term differing in at most half its characters. The surviving terms are reassembled into a suggested query; clicking it re-runs the search with the correction.

---

Running it

Requirements: .NET 6.0 SDK.

make dev
# or
dotnet watch run --project MoogleServer

Then open the printed URL (Chrome recommended — the layout has rendering issues in some Firefox versions).

Documents go in the Content/ folder at the repository root, one .txt per document. It must contain at least one file; if it is missing or empty, Moogle logs a warning at startup. The location is configurable in MoogleEngine/Folder.cs.

---

Layout

MoogleEngine/            # search engine class library
├─ Moogle.cs             # entry point: Query(string) → SearchResult
├─ TF_IDF.cs             # weighting
├─ Vector.cs             # vector space model, cosine via dot product
├─ Query.cs              # query parsing and operator extraction
├─ OperatorsMethods.cs   # ~ * ! ^ semantics
├─ StringMethods.cs      # normalisation, tokenisation, snippet helpers
├─ Document.cs           # per-document index
├─ WordInfo.cs           # term statistics
├─ Folder.cs             # corpus discovery
└─ Text_Files.cs         # I/O

MoogleServer/            # Blazor Server front end (renders results, timing, hit count)
Content/                 # corpus (drop .txt files here)

The UI reports how many documents matched and how long the search took.

Author

Osvaldo R. Moreno Prieto (C111) — @Val020213