skip to main content
10.1145/2660114acmconferencesBook PagePublication PagesmmConference Proceedingsconference-collections
CrowdMM '14: Proceedings of the 2014 International ACM Workshop on Crowdsourcing for Multimedia
ACM2014 Proceeding
  • General Chairs:
  • Judith Redi,
  • Mathias Lux
Publisher:
  • Association for Computing Machinery
  • New York
  • NY
  • United States
Conference:
MM '14: 2014 ACM Multimedia Conference Orlando Florida USA 7 November 2014
ISBN:
978-1-4503-3128-9
Published:
07 November 2014
Sponsors:
Next Conference
October 28 - November 1, 2024
Melbourne , VIC , Australia
Reflects downloads up to 21 Sep 2024Bibliometrics
Skip Abstract Section
Abstract

The power of crowds, leveraging a large number of human contributors and the capabilities of human computation, has enormous potential to address key challenges in the area of multimedia research. This power is, however, of difficult exploitation: challenges arise from the fact that a community of users or workers is a complex and dynamic system highly sensitive to changes in the form and the parameterization of their activities. Since 2012, the International ACM Workshop on Crowdsourcing for Multimedia, CrowdMM, has been the venue for collecting new insights on the effective deployment of crowdsourcing towards boosting Multimedia research.

In its third edition, CrowdMM14 especially focuses on contributions that propose solutions for the key challenges that face widespread adoption of crowdsourcing paradigms in the multimedia research community. These include: identification of optimal crowd members (e.g., user expertise, worker reliability), providing effective explanations (i.e., good task design), controlling noise and quality in the results, designing incentive structures that do not breed cheating, adversarial environments, gathering necessary background information about crowd members without violating privacy, controlling descriptions of task.

The call for papers attracted 26 international submissions (62% increase with respect to the 2013 edition), three of which were short paper submissions. Of these, 8 were accepted as oral presentations and 5 as poster presentations. All papers received at least three double blind reviews, and 3.4 reviews on average.

For a keynote talk, Nhatvi Nguyen (CEO of Microworkers) talks about Crowdsourcing Challenges from Platform Provider's Point of View. In addition to that CrowdMM features this year a crowd-sourced keynote, during which all CrowdMM14 authors give their view on the future and the Challenges that Crowdsourcing has still ahead.

Skip Table Of Content Section
SESSION: Keynote Address
keynote
Microworkers Crowdsourcing Approach, Challenges and Solutions

Founded in May 2009, Microworkers.com is an international Crowdsourcing platform focusing on Microtasks. At present, more than 600,000 users from over 190 countries have already registered to our platform. This extensively diverse workforce is the key ...

SESSION: Affect
research-article
A Protocol for Cross-Validating Large Crowdsourced Data: The Case of the LIRIS-ACCEDE Affective Video Dataset

Recently, we released a large affective video dataset, namely LIRIS-ACCEDE, which was annotated through crowdsourcing along both induced valence and arousal axes using pairwise comparisons. In this paper, we design an annotation protocol which enables ...

research-article
Modeling Image Appeal Based on Crowd Preferences for Automated Person-Centric Collage Creation

This paper attempts to model IA in personal photo collections through a user-centric perspective. To understand how users deemed an image as being more/less appealing, an extensive crowdsourcing experiment was conducted with 350 workers and five ...

research-article
A Multi-task Learning Framework for Time-continuous Emotion Estimation from Crowd Annotations

We propose Multi-task learning (MTL) for time-continuous or dynamic emotion (valence and arousal) estimation in movie scenes. Since compiling annotated training data for dynamic emotion prediction is tedious, we employ crowdsourcing for the same. Even ...

SESSION: Crowdworkers' Motivation
research-article
Crowdsourcing for Rating Image Aesthetic Appeal: Better a Paid or a Volunteer Crowd?

Crowdsourcing has the potential to become a preferred tool to study image aesthetic appeal preferences of users. Nevertheless, some reliability issues still exist, partially due to the sometimes doubtful commitment of paid workers to perform the rating ...

research-article
Development and Validation of Extrinsic Motivation Scale for Crowdsourcing Micro-task Platforms

In this paper, we introduce a scale for measuring the extrinsic motivation of crowd workers. The new questionnaire is strongly based on the Work Extrinsic Intrinsic Motivation Scale (WEIMS) [17] and theoretically follows the Self-Determination Theory (...

POSTER SESSION: Poster Session
poster
Is That a Jaguar?: Segmenting Ancient Maya Glyphs via Crowdsourcing

Crowdsourcing is popular in multimedia research to obtain image annotation and segmentation data at scale. In the context of analysis of cultural heritage materials, we propose a novel crowdsourced task, namely the segmentation of ancient Maya ...

poster
Making use of Semantic Concept Detection for Modelling Human Preferences in Visual Summarization

In this paper we investigate whether and how the human choice of images for summarizing a visual collection is influenced by the semantic concepts depicted in them. More specifically, by analysing a large collection of human-created visual summaries ...

poster
A Crowdsourcing Procedure for the Discovery of Non-Obvious Attributes of Social Images

Research on mid-level image representations has conventionally concentrated relatively obvious attributes and overlooked non-obvious attributes, i.e., characteristics that are not readily observable when images are viewed independently of their context ...

poster
A Crowdsourced Data Set of Edited Images Online

We present a crowdsourcing approach to tackle the challenge of collecting hard-to-find data. Our immediate need for the data arises because we are studying edited images in context online, and the way that this use impacts users' perceptions. Study of ...

poster
Click'n'Cut: Crowdsourced Interactive Segmentation with Object Candidates

This paper introduces Click'n'Cut, a novel web tool for interactive object segmentation designed for crowdsourcing tasks. Click'n'Cut combines bounding boxes and clicks generated by workers to obtain accurate object segmentations. These segmentations ...

SESSION: Annotation
research-article
Users Tagging Visual Moments: Timed Tags in Social Video

A timed tag is a tag that a user has assigned to a specific time point in a video. Although timed tags are supported by an increasing number of social video platforms on the Internet, multimedia research remains focused on conventional tags, here called ...

research-article
Crowd-based Semantic Event Detection and Video Annotation for Sports Videos

Recent developments in sport analytics have heightened the interest in collecting data on the behavior of individuals and of the entire team in sports events. Rather than using dedicated sensors for recording the data, the detection of semantic events ...

research-article
Getting by with a Little Help from the Crowd: Practical Approaches to Social Image Labeling

Validating user tags helps to refine them, making them more useful for finding images. In the case of interpretation-sensitive tags, however, automatic (i.e., pixel-based) approaches cannot be expected to deliver optimal results. Instead, human input is ...

Contributors
  • Delft University of Technology
  • Alpen-Adria-Universität Klagenfurt

Index Terms

  1. Proceedings of the 2014 International ACM Workshop on Crowdsourcing for Multimedia
      Index terms have been assigned to the content through auto-classification.

      Recommendations

      Acceptance Rates

      CrowdMM '14 Paper Acceptance Rate 8 of 26 submissions, 31%;
      Overall Acceptance Rate 16 of 42 submissions, 38%
      YearSubmittedAcceptedRate
      CrowdMM '1426831%
      CrowdMM '1316850%
      Overall421638%