[Amazon Mechanical Turk] Please review your answers before submitting - Quality issue
Message from Victoria Reznyk (
ocmp5@crowdcomputingsystems.com)
---------------------------------
Dear Member,
Please ignore if you received this message earlier.
Recently we ran a batch for 'Which Product type is the most appropriate' and we did an analysis of individual crowd worker answers and we are not happy with the quality of the answers. We are finding far too many cases where the crowd worker has chosen an answer option that is ridiculously wrong. Following are some examples:
1. Product was Sausage & Hotdogs, image and description clearly showed that but crowd worker chose ‘Tofu’
2. Product was a wine color sweatshirt and hoodie, image and description clearly showed that (infact image showed a model wearing the sweatshirt ) but crowd worker chose ‘Wine’
3. Product was Automotive cleaning brush, image and description clearly showed that but crowd worker chose ‘Tire’
Committing ridiculous errors such as above only proves that the crowd worker is not spending enough time on each question to review the product carefully. The sad part is we have had such errors from new members and also from members who have been with us for a long time.
Incase when you are not sure what a product is , please choose N/A. But blindly choosing any option is not the way how we expect these tasks to be done. This will only bring down the trust that we have placed in you.
To mitigate the above problem we are introducing a feature to randomly check worker’s answers against out golden set while a batch is running. If a worker repeatedly falls short of the quality that is expected, we have every right to remove such worker(s) without further notice.
So please mindfully work on the tasks. We value your hard work and we want to continue collaborating with you. Please understand that this is a general message that we are sending out to all workers who were engaged in our last task and who might be working on the current batch.
Thank you and happy working !