Program

Location: Hyatt Regency San Francisco hotel, Waterfront E on the Lobby level

  • 9:30 – 9:40 Welcome
  • 9:40 – 10:40 Keynote Talk
  • 10:40 – 11:00 Coffee Break
  • 11:00 – 12:00 Paper Presentations
    • 11:00 – 11:20: Lora Aroyo, Lucas Dixon, Nithum Thain, Olivia Redfield and Rachel Rosen. Crowdsourcing Subjective Tasks:The Case Study of Understanding Toxicity in Online Discussions
    • 11:20 – 11:40: Rafael Zequeira Jiménez, Anna Llagostera, Babak Naderi, Sebastian Möller and Jens Berger. Intra- and Inter-rater Agreement in a Subjective Speech Quality Assessment Task in Crowdsourcing
    • 11:40 – 12:00: Alyssa Lees and Chris Welty. Discovering User Bias in Ordinal Voting Systems
  • 12:00 – 13:30 Lunch Break
  • 13:30 – 14:30 Keynote Talk
    • Brad Klingenberg. Humans, machines and disagreement: lessons from production
  • 14:30 – 15:30 Paper Presentations
    • 14:30 – 14:50: Mike Schaekermann, Graeme Beaton, Minahz Habib, Andrew Lim, Kate Larson and Edith Law. Capturing Expert Arguments from Medical Adjudication Discussions in a Machine-readable Format
    • 14:50 – 15:10: V. K. Chaithanya Manam, Dwarakanath Jampani, Mariam Zaim, Meng-Han Wu and Alexander J. Quinn. TaskMate: A mechanism to improve the quality of instructions in crowdsourcing
    • 15:10 – 15:30: Tong Liu, Akash Venkatachalam, Pratik Sanjay Bongale and Christopher Homan. Learning to Predict Population-Level Label Distributions
  • 15:30 – 16:00 Coffee Break
  • 16:00 – 17:30 Final Discussion