Ramp migrated from a problematic homegrown industry classification system to standardized NAICS codes using a custom Retrieval-Augmented Generation (RAG) system. The old system suffered from four major issues: incorrect categories, overly generic classifications, inconsistent categorization of similar businesses, and lack of auditability. For example, diverse companies like law firms, dating apps, and consulting firms were all lumped into "Professional Services."

The RAG system operates in three stages: embedding generation creates text embeddings for business queries and the NAICS knowledge base, similarity search computes scores to generate recommended NAICS codes, and LLM-based selection makes the final classification decision. The system achieved up to 60% improvement in accuracy at k (acc@k) metrics for the first two stages, while prompt engineering and parameter tuning in the LLM stage delivered 5-15% improvement in fuzzy accuracy.

The technical infrastructure uses ClickHouse for fast embedding storage and retrieval, Kafka for comprehensive logging of intermediate results, and internal services for embeddings and LLM prompts. Guardrails filter out invalid NAICS code outputs to prevent hallucinations. This approach gives Ramp full control over the algorithm, enabling real-time hyperparameter adjustments and rapid iteration compared to third-party solutions.