Building a Local AI Second Brain: A Practical Guide to Obsidian and Ollama

[Roopesh21]

As professionals, our digital notes form a vast, yet often disconnected, repository of knowledge. Standard search can find keywords, but what if you could have an intelligent conversation with your entire knowledge base?

This guide provides a detailed, step-by-step walkthrough to build just that: a private, powerful, and completely local AI assistant inside your notes. We'll use the note-taking app Obsidian as our "second brain" and Ollama to run powerful AI models right on our own laptops. No data ever leaves your machine, there are no subscription fees, and it works entirely offline.

Part 1: The Three Core Components

Our system is built on three free and powerful tools working in harmony:

1. Obsidian: A flexible, markdown-based application for building a personal knowledge base. This is the home for your notes.

2. Ollama: A simple, powerful tool for running large language models (LLMs) on your local machine. This is our AI engine.

3. Smart Connections Plugin: An Obsidian community plugin that bridges the gap, connecting your notes to the AI models running in Ollama to enable a RAG (Retrieval-Augmented Generation) experience.

Part 2: Installation and Setup

First, let's get the foundational software installed and running.

Step 1: Install and Run Ollama

1. Go to the Ollama official website and download the installer for your operating system.

2. Run the installer. Ollama will set itself up to run in the background.

3. Verify it's running by opening a terminal (Command Prompt or PowerShell) and typing ollama list. If it's working, it will show a (currently empty) list of models.

4. Important: The gemma3 model requires Ollama version 0.6 or later. Ensure your installation is up to date by running ollama -v

Step 2: Download the Necessary AI Models

We need two different types of models for a RAG system: one for understanding the content of your notes (embedding) and one for generating answers (chat).

In your terminal, run the following commands one by one to download the exact models for this guide:

ollama pull nomic-embed-text:latest

ollama pull gemma3:1b

After the downloads are complete, you can run ollama list again to confirm they are installed.

Step 3: Install the "Smart Connections" Plugin in Obsidian

1. In Obsidian, go to Settings > Community plugins.

2. Click Browse and search for "Smart Connections".

3. Click Install, wait for it to finish, and then click Enable.

Part 3: The Critical Configuration

This is the most important part to get right. We need to tell the "Smart Connections" plugin to use our local Ollama models.

1. In Obsidian's Settings, browse to find Smart Connections and install it to find it in the left sidebar under Community Plugins. This opens its main settings page.

Configure the Embedding Model (The Librarian): This model reads and indexes your vault.

• On the settings page, scroll down and click the Show environment settings button.
• Scroll down to the Smart Sources section.
• Set Embedding model platform to Ollama (Local).
• Set Embedding model to nomic-embed-text:latest.

Part 4: Indexing - Teaching the AI Your Vault

Now, we need to let Smart Connections read every note in your vault and create a searchable index.

1. Close the Settings window.

2. Open the Command Palette using the shortcut Ctrl+P.

3. Type Smart Connections and select the command: Smart Connections: Open Smart Chat.

4. You'll see a small notification in the bottom corner showing the progress. Let this run until it's complete. Your vault is now fully indexed and ready to be queried.

Part 5: Indexing - Teaching the AI Your Vault

Configure the Chat Model (The Brain): This model generates your answers.

• It is crucial to have a dedicated Chat tab for this setting. If you don't see it, you may need to update the plugin.

• Click on the Chat tab by opening the settings window available in the smart connections chat pane.

• Find the Model section within this tab.

• Set Chat Model Platform to Ollama (Local).

• Set Chat Model to gemma3:1b.

Part 6: The Correct Workflow for Vault-Wide Chat

This is the final step to ensure you are talking to your entire vault, not just individual files.

1. Open the Right Tool: Use the Command Palette (Ctrl+P) and select Smart Connections: Open Smart Chat.

2. Use the Main Chat Interface: You will see a welcome screen that includes an "[Add context]" button.

3. Ask Your Question: Type your question about anything in your vault directly into the main chat input box at the bottom of the pane and press Enter.

The plugin will now automatically search your entire indexed vault, find the most relevant notes, and generate a factual answer based on their content.