I Built My Own AI-Powered Knowledge System. Here's What I Found Out.
Note: This case study describes a personal project — built by me, used by me. It is not a client engagement. I’m sharing it to demonstrate what the technology can do and how I approach operational problems.
The Problem
In over 12 years of working in IT, across different companies and teams, the same problem kept coming back: searching for information that definitely exists — somewhere.
A company wiki? Usually there is one. But finding anything in it means getting the keywords exactly right. A slightly different phrasing and the results make no sense. A classic search engine understands words, not intent.
Everything else lives in scattered files across shared drives, servers, and local machines. Everyone knows it’s in there somewhere. Nobody knows exactly where. The result: thirty minutes hunting for a single piece of information, several times a week.
Then there’s the database problem. To answer a straightforward question — “how many orders did we process last quarter, broken down by region?” — you either need to know SQL, or you need to wait for a developer or analyst to have a free slot. Who have their own priorities.
The real cost of information chaos isn’t the time spent searching. It’s decisions made on incomplete data — and knowledge that walks out the door when someone leaves.
The Goal
One place with fast, easy access to all the information needed. No keyword guessing. No digging through dozens of files. A plain-language question — an answer in seconds.
Additional requirements:
- Data must stay local — no documents sent to external servers.
- The system must work without developer involvement in day-to-day use.
- Support for multiple source types: local files, web pages, SQL database.
The Solution
In December 2024, I built the first version of an application based on RAG — Retrieval-Augmented Generation. I wrote more about RAG here.
How it works: instead of answering from its own training data, the language model first searches a defined document base, finds the relevant passage, and builds its answer from that. Every response has a source. The system doesn’t guess — it quotes.
From the user’s perspective, it feels like talking to a very well-informed colleague who has read every document in the company.
Supported Data Sources
Local files — PDFs, Word documents, Excel spreadsheets. Load them once, query them as many times as needed. Useful for SOPs, procedures, proposals, contracts.
Web pages and articles — paste a URL, the system pulls the content and adds it to the knowledge base. Useful for teams tracking technical documentation, supplier materials, or industry publications.
SQL database — the user asks a question in plain English, the system generates the SQL query, retrieves the data, and returns a readable answer. Non-technical team members can access the database independently — no developer, no analyst, no waiting.
Privacy and Control
The system runs entirely on-premise — the language model runs locally via Ollama, data is stored in a local database. Nothing leaves the company’s own infrastructure. For businesses handling sensitive data, this isn’t a nice-to-have. It’s a hard requirement.
For businesses without those constraints, the system also supports API integration with external model providers: OpenAI, Anthropic, and others.
Results
- Answers in seconds — instead of thirty-plus minutes of digging through files and wikis.
- One interface for all sources — local files, web pages, and SQL database in a single chat interface.
- Database access without SQL — plain-language questions to the database, no developer in the loop.
- Full data privacy — nothing leaves the local infrastructure.
- No learning curve — a chat interface any team member can use from day one, no training required.
Takeaways
The knowledge is already in the company. The problem is it’s everywhere — which means, in practice, it’s nowhere.
This project showed me that RAG is mature enough today to deploy in businesses without a dedicated IT department or a data science team. It doesn’t take months to implement. It takes a clear understanding of the problem and a system designed precisely around the specific use case.
The value is highest where:
- knowledge is scattered across multiple files and systems
- the company is actively hiring and onboarding consumes experienced team members’ time
- data lives in a SQL database but accessing it requires going through someone else
- working with sensitive data rules out sending anything to external AI tools
If you’d like to implement something similar in your business — get in touch. I’d be glad to talk through what this could look like for your specific situation.