Product Classification API Part 2: Data Preparation

Eugene Yan · 9 years ago · 11

This article describes data cleaning and preparation techniques for training a product classification machine learning model, including encoding non-ASCII characters, lowercasing, tokenization, and removing stop words and numeric tokens. The author measures data quality using a purity metric defined as products with matching titles and categories divided by total products, removing mismatched entries from the training set. The preparation process enables building a more accurate classifier by excluding ambiguous or corrupted product entries before model training.