Procurement Benchmark Webapp
2024 · Sentence Transformers · BERTopic · OpenAI API · ChromaDB · Streamlit
Impact
- Paid proof of concept delivered under an NDA freelance engagement with a procurement officer, now moving into a migration phase onto the client's own systems
- Validated across 3 live pairing sessions with the client, run over Streamlit via ngrok, each round of verbal feedback feeding the next iteration
- Estimated to cut a benchmarking task from around 30 minutes to under 5, an average agreed with the client; the actual saving depends on order size and item complexity and availability
Business Problem
An independent freelance engagement, under NDA, for a procurement officer whose purchase order data is highly specialized: free-text line items across thousands of purchase orders per year, with no shared category structure. The same item can appear under dozens of different descriptions depending on buyer, supplier, and year.
Spend cannot be aggregated, prices cannot be compared, and benchmarking is impossible. The problem is not missing data; it is the absence of a shared semantic structure that makes raw line items comparable.
Solution
I built four components around a three-level taxonomy of item clusters.
Hierarchical clustering builds the taxonomy through a deterministic-first, LLM-assisted pipeline. TF-IDF and token overlap handle the common case; GPT handles naming and edge cases, with names scored on a 5-point scale and regenerated below threshold.
Misfit detection flags leaves in the wrong category using two signals: intra-subcluster vocabulary profiling (leaves matching zero or one core keyword) and supplier domain profiling (items whose supplier exclusively operates in a different product domain).
Hybrid search queries the full PO corpus by free-text description. BM25 catches exact part numbers; dense vector search via ChromaDB catches semantic equivalents across languages. RRF fuses both signals; a cross-encoder reranker rescores the top-100 candidates before returning results with cluster label, price statistics, supplier list, and order history.
Streamlit benchmark UI surfaces results as a procurement decision tool: cluster summary cards, historical PO table, price trend scatter with inflation comparison, and supplier breakdown. The procurement officer searches by description or uploads a supplier invoice for automatic line-by-line price comparison.
Architecture
Technical Challenges
Free-text variance at scale. The same part can appear under 20 or more descriptions. Keyword rules cover 80% of cases; semantic similarity handles the long tail. LLM calls on every item would be non-deterministic and expensive.
Taxonomy correctness under iteration. Fixing one misfit can introduce another. A validation suite runs after every script: no orphaned leaves, no duplicate subcluster names, leaf count invariant across runs.
Leaf cluster immutability. Items cannot be reassigned within a leaf; only whole clusters move. L3 is treated as fixed fact; the L1/L2 hierarchy is built correctly around it.
Search strictness calibration. BM25 and dense retrieval have different precision-recall tradeoffs on procurement text. Too strict drops valid comparables; too loose pollutes the benchmark with irrelevant prices.
No ground truth. Quality is assessed through SME review and semantic coherence metrics. The first real signal comes from officers using the tool against live supplier quotes.
MVP Status
Validated but not yet on the client's own infrastructure: the app currently runs locally and is shared over ngrok for live sessions, which is fine for paired validation but not a real deployment. Quality has been judged qualitatively so far, three rounds of live feedback rather than a measured acceptance rate, which is the main gap before wider rollout within the client's team.
- Started as a side project in November 2024; proposed and presented as a proof of concept at the client's office in December 2025, then turned into a paid, NDA-bound freelance engagement
- 3 live pairing sessions completed, run over Streamlit via ngrok with the client's procurement officer testing live and giving verbal feedback each round
- PoC phase invoiced and paid; considered validated and ready for migration off ngrok onto the client's own systems
Next Steps
- Migrate to the client's infrastructure: containerize with Docker; deployment approach still under discussion with the business
- Scope and invoice the migration phase as a separate engagement from the PoC
- Quantify search quality: move from qualitative pairing-session feedback to a measured acceptance rate before any wider rollout within the client's team