My Notes From Spark+AI Summit 2020 (Application-Agnostic Talks)

Conferences like Spark+AI Summit dump hundreds of talks on attendees and expect them to figure out which ones are worth their time. Eugene Yan did the sorting for everyone else, and the result is a compact tour through five sessions that all circle one theme: making models and pipelines smaller, faster, and cheaper without giving up much accuracy.

The deep learning efficiency talk from IBM's Nick Pentreath lays out the toolkit plainly. MobileNet V1 gets away with 4 million parameters versus Inception V3's 24 million by swapping standard convolutions for depth-wise ones, trading some accuracy (70.9% versus 78.8% on ImageNet) for a much lighter footprint. Pruning can strip half the weights from these networks with barely a dent in performance, and quantization can shrink 32-bit weights down to 16-bit while keeping the distribution intact. Model distillation ties it together: train a small student network on a big teacher's soft labels, and sometimes the student ends up more accurate than the teacher, which says a lot about how bloated these architectures usually are. Instagram's feed ranking system is a real-world proof point, running a distilled model as the first filtering pass before two heavier stages narrow candidates down.

Databricks' Sean Owen offered a more procedural version of the same idea: six concrete steps to speed up training. Early stopping alone cut a 60-minute job to 18 minutes while nudging accuracy up. Bigger batch sizes squeezed that down to 9 minutes. Uber's Petastorm library, built to feed Parquet data into TensorFlow or PyTorch almost as fast as reading from memory, let Owen use 10 times the data for just 11 times the epoch time, pushing accuracy to 83%. From there it's the usual scaling ladder: multiple GPUs, then Horovod across machines when one box isn't enough.

Facebook's Joe Spisak covered the production side, walking through PyTorch's built-in pruning and quantization, TorchElastic for fault-tolerant distributed training jobs that survive dropped machines or outbid spot instances, and TorchServe, built with AWS, for serving models with versioning and A/B testing baked in.

The two Spark-specific talks were less about deep learning and more about not wasting compute. Adobe's Yeshwanth Vijayakumar explained why Bloom filters, HyperLogLog, and count-min sketch work at scale: they're monoids, meaning you can combine partial results without recomputing from scratch. And Workday's Jianneng Li dismantled the assumption that broadcast joins are always faster in Spark. The math shows broadcast hash joins win only when the bigger table dwarfs the smaller one relative to the number of cores available; otherwise a sort-merge join wins. Workday is pushing for executor-side broadcast joins (tracked as SPARK-17556) to fix inefficiencies in Spark's current driver-side approach.