Press/Studies The Decline of Amazon Mechanical Turk (Feb 2016)

[electrolyte]

"As you can see, starting May 2016 (that is before the announcement of price increases), we see a decline in the number of active requesters. Interestingly, when the fee increases are announced, on June 23rd 2015, we see a small bump in the number of active requesters, followed by a small "valley" around the period of fee increases. The numbers remain stable until November, but after that we see a steady decline."

Thoughts?

http://www.behind-the-enemy-lines.com/2016/02/the-decline-of-amazon-mechanical-turk.html

 

[clickhappier]

"As you can see, starting May 2016 (that is before the announcement of price increases), we see a decline in the number of active requesters. Interestingly, when the fee increases are announced, on June 23rd 2015, we see a small bump in the number of active requesters, followed by a small "valley" around the period of fee increases. The numbers remain stable until November, but after that we see a steady decline."

Thoughts?

http://www.behind-the-enemy-lines.com/2016/02/the-decline-of-amazon-mechanical-turk.html

Somebody needs to tell Panos that should say May 2015, not May 2016. Interesting chart.

 

[jan]

I don't know if that's accurate or not - we've sure seen the numbers get quite low.

I think mturk could have been a good platform, but amazon has made it a bit of a mess. Frankly, if I were a small business and needed additional research or support - I wouldn't think of using mturk. If I needed transcription, it's much easier to farm out to a small online company that doesn't need me figuring out the mturk platform and dealing with mediocre (sometimes, it is!) work. If I had that audio (for transcription, for instance) - I wouldn't need any technical sophistication to send it to one of those places. And, if I were really incompetent - they'd talk me through the whole process.

They care. Also, if there were some minor problems, they could work with the transcriber (or someone else, if that individual couldn't do the task) to see it met my exact specifications (without the ugly rejection thing). It wouldn't cost that much more.

I've long thought that there needs to be a middle service that can put together the work - in just the way these outside companies do. Imagine you want linkedin addresses, for instance. A middle person could write up the job and send it out to mturk with just the right quals and/or specifications. Then they could review the work - and it would always come back perfect.

I've seen this advertised, before, but it seems like they didn't get enough business themselves (they didn't know how to market this).

As it is now, well, many of us would do a great job. But the work would be uneven and there would be data to sort through and clean. The middle person could 'clean the data' and send it back perfect. Businesses - even small businesses - want their work done correctly, not always looking over their shoulder thinking that they will be scammed or a bot will be completing the assignment.

Maybe the UCLA platform will have more opportunities for scrutiny? (and fairness

 

[clickhappier]

...
I've long thought that there needs to be a middle service that can put together the work - in just the way these outside companies do. Imagine you want linkedin addresses, for instance. A middle person could write up the job and send it out to mturk with just the right quals and/or specifications. Then they could review the work - and it would always come back perfect.

I've seen this advertised, before, but it seems like they didn't get enough business themselves (they didn't know how to market this).
...

Yes, there are several MTurk middlemen companies out there, of varying levels of service / volume / comprehensiveness.

 

[aveline]

I don't think you can look at a single year and claim that it indicates a trend. I looked as far back as the tracker thing would let me, to May 2014, and it looks like the activity in much of 2014 was a bit lower than it is currently. It just makes me wonder about 2013. Was 2013 about the same, or was it lower or higher? Without knowing any of that, I don't think we can really make any assumptions about the data from 2015 alone.