Introducing new audio and vision documentation in 🤗 Datasets

Hugging Face built its Datasets library back in 2020 to make text datasets trivially easy to load and process, and the NLP community piled in with hundreds of contributions during the Datasets Sprint. But text was always just the opening act. Machine learning has moved hard into audio, images, and multimodal combinations of the two, and the tooling needed to catch up.

This update is mostly about closing that gap through documentation and a couple of genuinely useful features rather than a flashy new model. The Quickstart guide now walks through loading and processing audio and image datasets end to end, for both PyTorch and TensorFlow. The standout addition is to_tf_dataset, a function that converts a dataset straight into a tf.data.Dataset without forcing anyone to hand-write shuffling or batching logic. It's a small thing, but it removes a genuine point of friction for TensorFlow users who've long had a rougher time than the PyTorch crowd.

The bigger structural change is that audio and image loading now get their own dedicated documentation sections instead of being buried in general usage notes. That matters because these formats behave differently under the hood — audio files, for instance, get decoded and resampled on the fly through the Audio feature, which is nothing like how a plain text file loads. Hugging Face says this reorganization is also meant to make room for future modalities they haven't announced yet.

The most immediately practical piece, though, is ImageFolder, a builder that lets you skip writing a custom loading script entirely. Drop images into folders named after their classes — dog, cat, whatever — and upload it to the Hub, and labels get generated automatically from the directory structure. It also handles more complex metadata, like bounding box coordinates and category IDs for object detection datasets, by reading a simple JSON file alongside the images. That flexibility means ImageFolder isn't limited to classification tasks; it can support captioning and detection work too, provided you supply the right metadata.

Hugging Face is hinting that an AudioFolder equivalent is coming, though there's no date attached to that yet. In the meantime, the team is pointing people toward datasets like GigaSpeech to test the current audio guide, and it's actively inviting outside contributors to upload their own audio and image datasets to the Hub — a strategy that worked well for growing the text-dataset collection and one they're clearly hoping to repeat here.