Introducing the Data Measurements Tool: an Interactive Tool for Looking at Datasets

Hugging Face Blog · 4 years ago

Hugging Face released the Data Measurements Tool, an interactive online interface and open-source library that lets users measure, analyze, and compare machine learning datasets without coding. The tool currently works with popular English-language datasets like SQuAD, IMDB, and C4, calculating metrics for vocabulary size, label distribution, word patterns following Zipf's law, and potential stereotypes through word association analysis. Future updates will extend support to more languages and datasets, enable analysis of user-provided data, and allow custom terms for bias detection.