Memory System

An AI-powered semantic memory system built with Rust and Google's Gemini API. Store memories with semantic embeddings and retrieve them contextually when asking questions.

Features

• Semantic Memory: Store text memories with AI-generated embeddings for semantic search
• Contextual Retrieval: Ask questions and get relevant memories injected into AI responses
• Vector Similarity Search: Uses cosine similarity to find semantically related memories
• Persistent Storage: Memories stored in JSON format on disk
• CLI Interface: Simple command-line interface for storing and querying memories

Quick Start

Prerequisites

• Rust (latest stable version)
• Google Gemini API key (get one free at aistudio.google.com)

Installation

1. Clone this repository:

git clone <your-repo-url>
cd memory-system

2. Create a .env file with your API key:

echo "GEMINI_API_KEY=your_api_key_here" > .env

3. Build and run:

cargo run -- remember "Hello, I'm building a memory system!"
cargo run -- ask "What am I building?"

📖 Usage

Commands

Store a Memory

cargo run -- remember "Your memory text here"

Ask a Question

cargo run -- ask "Your question here"

The system will:

1. Embed your question
2. Find the top 3 most semantically similar memories
3. Inject them into the AI's context
4. Return an answer based on your stored memories

List All Memories

cargo run -- list

Clear All Memories

cargo run -- clear

Example Session

# Store some memories
$ cargo run -- remember "My name is Alice and I love programming"
$ cargo run -- remember "I work as a software engineer at TechCorp"
$ cargo run -- remember "My favorite programming language is Rust"

# Ask questions
$ cargo run -- ask "What's my name and job?"
# Output: Based on your memories, your name is Alice and you work as a software engineer at TechCorp.

$ cargo run -- ask "What do I like?"
# Output: According to your stored memories, you love programming and your favorite language is Rust.

🏗️ How It Works

Architecture

┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│   User Input    │───▶│  Agent Layer     │───▶│  Gemini API     │
│                 │    │                  │    │                 │
│ • Remember text │    │ • Embed text     │    │ • Embeddings    │
│ • Ask questions │    │ • Search memory  │    │ • Chat          │
└─────────────────┘    └──────────────────┘    └─────────────────┘
                                │
                                ▼
                       ┌──────────────────┐
                       │  Memory Store    │
                       │                  │
                       │ • JSON storage   │
                       │ • Vector search  │
                       │ • Cosine sim.    │
                       └──────────────────┘

Memory Storage

Memories are stored in data/memories.json:

{
  "memories": [
    {
      "id": 0,
      "text": "My name is Alice",
      "embedding": [0.123, -0.456, 0.789, ...]
    }
  ]
}

Each memory contains:

• ID: Unique identifier
• Text: The original text
• Embedding: 768-dimensional vector representing semantic meaning

Semantic Search

When you ask a question:

1. Embed Question: Convert your question to a vector using gemini-embedding-2
2. Vector Search: Compare question vector to all stored memory vectors using cosine similarity
3. Retrieve Top-K: Return the 3 most similar memories
4. Context Injection: Include retrieved memories in the AI prompt
5. Generate Response: Use gemini-2.0-flash to answer with memory context

Cosine Similarity

The system uses cosine similarity to measure semantic relatedness:

similarity = (A • B) / (||A|| × ||B||)

Where:

• A • B is the dot product
• ||A|| and ||B|| are vector magnitudes

🔧 Configuration

Environment Variables

• GEMINI_API_KEY: Your Google Gemini API key (required)

Models Used

Purpose	Model	API Endpoint
Embeddings	gemini-embedding-2	embedContent
Chat	gemini-2.0-flash	generateContent

📦 Dependencies

• tokio: Async runtime
• reqwest: HTTP client for API calls
• serde: JSON serialization/deserialization
• clap: Command-line argument parsing
• colored: Terminal output coloring
• dotenv: Environment variable loading
• anyhow: Error handling

🛠️ Development

Building

cargo build

Testing

cargo test

Code Structure

src/
├── main.rs      # CLI entry point and command routing
├── agent.rs     # Core logic for remembering and asking
├── memory.rs    # Memory storage and vector search
├── embeddings.rs # Google Gemini embedding API client
└── llm.rs       # Google Gemini chat API client

🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Test thoroughly
5. Submit a pull request

📄 License

This project is open source. Feel free to use and modify.

🙏 Acknowledgments

• Built with Google Gemini API
• Inspired by semantic memory systems and vector databases
• Thanks to the Rust community for excellent crates