Joint Keynote Series

This year, for the first time, the SAD workshop will run a joint keynote series in collaboration with the HumBL workshop. The joint series will feature five keynotes from speakers across industry and academia:

Keynotes on May 13 @ HumBL2019:

Keynotes on May 14 @ SAD2019:

Maria Stone

What we talk about when we talk about crowdsourcing

Bio: Maria Stone is a veteran of the search industry, having worked at Alta Vista, Google, Yahoo, and Microsoft. She started the UX Research team at Google, from its inception in 2001 until 2008, and then worked as a Data Scientist at Yahoo, Microsoft and Apple. Prior to her work in industry, she was an academic researcher and lecturer focused on studying human memory and attention. She holds a Ph.D. in Cognitive Psychology from UC Berkeley, where she worked with Daniel Kahneman. She has authored publications in Cognitive Psychology, HCI, and Information Retrieval.

Brad Klingenberg


Humans, machines and disagreement: lessons from production

Bio: Brad Klingenberg is the VP of Algorithms at Stitch Fix, an online personal styling service that commits to its recommendations by physically delivering inventory to clients. Brad and his team use statistics, machine learning and human-in-the-loop algorithms to optimize the Stitch Fix client experience, the management of inventory and the selection of items for clients. Prior to joining Stitch Fix, Brad received his PhD in Statistics from Stanford University and worked as a data scientist in technology and financial services.


Jon Chamberlain

Jon Chamberlain

Are two heads better than one? An exploration of ambiguity in crowd-collected language decisions from the Phrase Detectives game.

The online game-with-a-purpose Phrase Detectives has been collecting decisions about anaphoric coreference in human language for over 10 years (4 million judgements from 40,000 players). Unlike most crowd systems, the game collects multiple valid solutions for a single task, which complicates aggregation through traditional statistical methods. Analysis of the ambiguous decisions that players make highlights the need for understanding and resolving disagreement that is inherent in language interpretation. This talk will present some of the interesting cases of ambiguity found by the players of Phrase Detectives and propose methods for harnessing crowds that disagree with each other.

Bio: Dr Jon Chamberlain is a lecturer and applied research scientist based at the University of Essex, England. He has been the lead developer behind the Phrase Detectives game-with-a-purpose for over 10 years and is co-investigator on the Disagreements and Language Interpretation (DALI) project that builds on early work to understand ambiguity in human language.


Anima Anandkumar


Bio: Animashree (Anima) Anandkumar is a Bren professor at Caltech CMS department and a director of machine learning research at NVIDIA. Her research spans both theoretical and practical aspects of machine learning. In particular, she has spearheaded research in tensor-algebraic methods, large-scale learning, deep learning, probabilistic models, and non-convex optimization. Anima is the recipient of several awards such as the Alfred. P. Sloan Fellowship, NSF Career Award, Young investigator awards from the Air Force and Army research offices, Faculty fellowships from Microsoft, Google and Adobe, and several best paper awards. She is the youngest named professor at Caltech, the highest honor bestowed to an individual faculty. She is part of the World Economic Forum’s Expert Network consisting of leading experts from academia, business, government, and the media. She has been featured in documentaries by PBS, KPCC, wired magazine, and in articles by MIT Technology review, Forbes, Yourstory, O’Reilly media, and so on.


Anurag Batra

Anurag at Costanoa

Share your world: Partnering with global communities to build smarter, more inclusive ML

Building ML-based models that work equally well for diverse global populations may be a daunting challenge. However, global diversity also offers an opportunity. We’ll talk about how crowdsourcing with global communities facilitates the creation of high-quality data sets to solve problems of visual or linguistic understanding, specialized knowledge or subjective opinion. We’ll show examples of how subjectivity plays a role in unpredictable ways, and talk about techniques to counter (or leverage) that.

Bio: Anurag Batra is a Product Manager with Google AI, focusing on partnering with global communities of people to bring diversity and inclusion to ML training data. Most recently, his team used crowdsourcing to release Open Images Extended, a data set that aims to bridge diversity gaps in the larger Open Images data set. When not mulling over data in ML, you can find Anurag biking the great outdoors with his sons aged 9 and 11.