Achilles2357
Active Member
- Joined
- May 24, 2017
- Messages
- 966
- Reaction score
- 1,459
- Points
- 393
- Gender
- Male
A few thoughts for the night crew on topics driving some people wild:
1) Amazon will NEVER ban scripts. Their currently policy is EXACTLY correct. They officially require that all hits are done via "human intelligence", while also "allowing" scripts without explicitly allowing any and every script. They are selling "human intelligence" to requesters, and will rightfully ban anyone who is identified as using mechanical means to fully complete hits. But they perfectly well know that all "worker customers" who are earning Amazon the most money are using "scripts" in multiple ways, and performing real work to the satisfaction of requesters. In effect, they know that workers who write scripts are working on speculation, doing the work that mturk and the requesters don't do. In any case, there is no way even in principle to explicitly state what counts as a "script" or to identify who is using such, unless all hits are basically Inquisit hits. (If requesters want Inquist, they can PAY FOR INQUISIT.)
2) Auto-accepting hits sounds good if you really don't know what you are talking about. In certain very limited cases it can work, especially if others are not doing it. But even a little thought reveals that auto-accepting hits can quickly create new problems of its own. For instance, I am in the SurveyComet Political Prediction study, with a weekly $16 hit that I'll spend maybe an hour on (mostly browsing news) since they suggest that you'd better spend that much time. Do I want that auto-accepted? Absolutely not! I do it when I am confident I have an hour or so and am in the mood for it. Auto-accepting hits can quickly fill your queue with far more than you can do, and you will then NEVER be able to do the hits you fail to do. There is NEVER going to be a KILLER script or tool that allows workers to avoid using their own intelligence in playing the mturk game. There might seem to be in the short term, but that is illusion.
3) Academic requesters are not going to abandon mturk due to fear of bots. Maybe a few will. Most will go to other platforms, that we already know about or will discover! (But bots can join Prolific too...) The current structure of academia rewards publication count in promotion and retention, and this means that academics must continually churn out papers, and for those in the psych and social sciences this means that they must continually accumulate new data sets. By far the easiest way to do this is by online studies, and mturk is the dominant platform for this, and for all online spot work.
In short: many mturkers are clueless, many requesters are clueless, and some experienced mturkers (like on reddit) intentionally spread disinformation.
1) Amazon will NEVER ban scripts. Their currently policy is EXACTLY correct. They officially require that all hits are done via "human intelligence", while also "allowing" scripts without explicitly allowing any and every script. They are selling "human intelligence" to requesters, and will rightfully ban anyone who is identified as using mechanical means to fully complete hits. But they perfectly well know that all "worker customers" who are earning Amazon the most money are using "scripts" in multiple ways, and performing real work to the satisfaction of requesters. In effect, they know that workers who write scripts are working on speculation, doing the work that mturk and the requesters don't do. In any case, there is no way even in principle to explicitly state what counts as a "script" or to identify who is using such, unless all hits are basically Inquisit hits. (If requesters want Inquist, they can PAY FOR INQUISIT.)
2) Auto-accepting hits sounds good if you really don't know what you are talking about. In certain very limited cases it can work, especially if others are not doing it. But even a little thought reveals that auto-accepting hits can quickly create new problems of its own. For instance, I am in the SurveyComet Political Prediction study, with a weekly $16 hit that I'll spend maybe an hour on (mostly browsing news) since they suggest that you'd better spend that much time. Do I want that auto-accepted? Absolutely not! I do it when I am confident I have an hour or so and am in the mood for it. Auto-accepting hits can quickly fill your queue with far more than you can do, and you will then NEVER be able to do the hits you fail to do. There is NEVER going to be a KILLER script or tool that allows workers to avoid using their own intelligence in playing the mturk game. There might seem to be in the short term, but that is illusion.
3) Academic requesters are not going to abandon mturk due to fear of bots. Maybe a few will. Most will go to other platforms, that we already know about or will discover! (But bots can join Prolific too...) The current structure of academia rewards publication count in promotion and retention, and this means that academics must continually churn out papers, and for those in the psych and social sciences this means that they must continually accumulate new data sets. By far the easiest way to do this is by online studies, and mturk is the dominant platform for this, and for all online spot work.
In short: many mturkers are clueless, many requesters are clueless, and some experienced mturkers (like on reddit) intentionally spread disinformation.
