--- title: "AI-Powered HSN Classification Case Study | Agility AI" url: https://agilitytech.ai/case-studies/hsn-classification description: "Case study: AI HS-code (HSN) classification automating customs categorisation for global trade: faster clearance, fewer errors, full audit trail." publisher: Agility (agilitytech.ai) --- Global Trade # AI-Powered HSN Classification & Product Categorization Processing 21 million international trade shipment records in under 60 minutes using a hybrid ML pipeline for HSN code classification and multilingual product attribute extraction at under $50 per million records. 85%+ Extraction Accuracy Improved from a ~40% baseline through hierarchical pattern matching 350K+ Records/Minute Sustained processing throughput at production scale 21M Records Processed In a single batch in under 60 minutes <$50 Per Million Records Fully offline, zero external API dependency ## Project Overview ### Industry Global Trade ### Region Global ### Project Size Enterprise: 21M+ Record Production Dataset ### Time Frame Q4 2024: Completed ### Technology Stack Python 3.11 spaCy NER ONNX Runtime INT8 HuggingFace Transformers scikit-learn / TF-IDF ## The Challenge ### Unstructured Multilingual Trade Data at Enterprise Scale International trade shipment descriptions arrive in multiple languages, inconsistent formats, and with missing qualifiers. Making automated HSN code classification and structured product extraction extremely difficult at scale. The existing system achieved only ~40% product extraction accuracy due to over-segmentation of product names, missing key qualifiers, and training data misalignment with real-world shipment descriptions. Cloud API-based solutions were prohibitively expensive at the volumes required and could not meet offline deployment requirements. ## Transformational Results ### Enterprise-Scale Offline Classification Engine We replaced a failing ~40% accuracy system with a production-grade hybrid ML pipeline processing 21 million trade records in under 60 minutes at 350K+ records per minute. The hierarchical product extraction engine eliminates over-segmentation, the NER-based attribute extractor handles multilingual inputs, and ONNX INT8 quantization delivers all of this fully offline at under $50 per million records, with zero external API dependency and no GPU infrastructure required. #### Classification Excellence Improved product extraction accuracy from ~40% to 85%+ through hierarchical pattern matching and transformer-based classification Achieved ~80% attribute extraction coverage across Brand, Type, Processing, Grade, Form, and Origin fields Maintained classification performance consistently across multilingual inputs without any external API dependencies #### Scale and Cost Efficiency Processed 21 million trade records in under 60 minutes at 350K+ records per minute sustained throughput Deployed fully offline at under $50 per million records eliminating cloud inference costs entirely ONNX INT8 quantization delivered production inference speed on CPU without GPU infrastructure ## Challenges & Solutions ### Product Over-Segmentation Breaking Extraction Accuracy #### Problem Single product descriptions like 'INDIAN GREEN COFFEE' were being incorrectly split into multiple separate entities, causing systematic extraction failures that drove baseline accuracy to ~40%. #### Solution Implemented hierarchical pattern matching that identifies and preserves compound product names as single entities, prioritizing longer, more specific patterns before attempting shorter generic matches. #### Impact Product extraction accuracy improved from ~40% to 85%+ ### Multilingual and Noisy Input Data #### Problem Shipment descriptions contained mixed languages, shipping terminology, container details, unit inconsistencies, and arbitrary formatting. Breaking any language-specific model trained on clean data. #### Solution Built a comprehensive preprocessing pipeline with automatic language detection, noise removal for shipping terms and container markers, unit normalization, and multilingual support feeding into cross-lingual extraction models. #### Impact Consistent extraction performance across multilingual datasets with no manual preprocessing required ### Enterprise-Scale Volume at Prohibitive Cost #### Problem Processing 21 million records through cloud API inference would be cost-prohibitive and violate secure offline deployment requirements for the enterprise environment. #### Solution Applied ONNX INT8 quantization to compress models for CPU inference and joblib multiprocessing for parallelized batch processing, achieving production throughput without cloud dependency. #### Impact 21 million records processed in under 60 minutes at under $50 per million records, fully offline ### Training Data Misalignment Causing Systematic Errors #### Problem Model outputs did not match the format and patterns in training data, creating systematic misclassification that could not be resolved by retraining without fixing the underlying alignment issue. #### Solution Audited and realigned all extraction patterns against actual training data examples, iteratively validating outputs against expected formats before finalising the production pipeline. #### Impact Systematic misclassification eliminated with consistent 85%+ accuracy across production volumes ## Ready to Classify Trade Data at Enterprise Scale? Contact our ML engineering team to discover how hybrid AI pipelines can transform your international trade data operations. [Get Started Today](https://agilitytech.ai/contact)