TopN AnalyticsSelf-Hosted GenAI Assistant for Natural-Language Data Queries
An eMSP platform exploring GenAI-assisted self-service analytics for internal and partner-facing stakeholders.
Where things stood
Getting an answer from a data warehouse usually requires SQL skills or waiting on an analyst — a bottleneck for time-sensitive operational questions.
Off-the-shelf conversational AI tools typically require sending business data to a third-party hosted API, raising real data-governance concerns for a platform handling sensitive partner, transaction, and customer information.
What we built
Table metadata, relationships, and example queries are indexed into a Chroma vector database.
A locally-hosted open-source language model (DeepSeek-R1, served via Ollama) classifies each question as conversational or data-related; for data questions, retrieval-augmented generation (RAG) pulls the top-5 most relevant context chunks — table structure, column descriptions, relationships, and example query pairs.
That context plus explicit business rules is passed to the local model, which generates SQL, executes it against Amazon Redshift, and converts results back into a plain-language answer via a Flask/LangChain application.
How it works
TopN Analytics built and tested a prototype chat assistant that answers natural-language questions by generating and executing SQL against the data warehouse — entirely on infrastructure the platform controls, so no query content or business data needs to leave the platform's own infrastructure.
This is a tested prototype, not a deployed, in-production system — no evidence of production usage exists in the source material.
What we delivered and learned
The self-hosted design is the differentiating decision: running both the embedding model and the language model locally rather than calling a third-party hosted API directly addresses GenAI data-governance concerns.
Demonstrates a credible, self-hosted path to self-service natural-language analytics for any data-sensitive platform business considering GenAI-assisted analytics.
What it's built on
Want the full picture?
We're happy to walk through the details, numbers, and trade-offs directly.