Disee - Distributed Search Engine
A high-performance, containerized, distributed search engine with dynamic API data ingestion and worker partitioning.
Overview & Motivation
To build a scalable and distributed gateway-worker orchestration system that parallelizes search indexes over partitioned data.
Problem Statement
Running queries and building search indexes on a single node causes computing bottlenecks and high search latency as data grows.
Proposed Solution
Designed a multi-phase distributed architecture where a central Gateway service partitions queries, fetches source feeds (Wikipedia/StackOverflow), and fans out search tasks to worker nodes.
Results & Impact
Achieved horizontal scaling with dynamic worker nodes, enabling distributed query processing and index building across multiple containers.
Technologies Used
Related Projects
B2B RFP Agent AI
An AI-powered B2B enterprise platform automating RFP requirements extraction and product quote estimation.