Amazon Updated Amazon Mechanical Turk Terms — What’s Changed

[electrolyte]

I believe mturk has reached out to script creators and database creators like turkopticon so hopefully they still have an open line of communication. That was between actual mturk folks though probably not the legal team.

It's interesting how MTurk reached out to script creators and workers for input on the new worker site but as far as anyone has said, MTurk didn't reach out to anyone for input on these changes, which sound like they're potentially very major.

 

[Drwho10]

It's interesting how MTurk reached out to script creators and workers for input on the new worker site but as far as anyone has said, MTurk didn't reach out to anyone for input on these changes, which sound like they're potentially very major.

Thats what makes me think these changes weren't written by the mturk admins or with any feedback. Just put in place by the legal team who never bothered to check. All speculation but highly likely in my opinion

 

[stuffalleysays]

Well, I guess until it is sorted I'll just have to look people up manually on turkopticon. Scripts have thus far been our only protections, due to the fact Amazon offers us none and can terminate without cause at any moment. Would be nice if they had a presence on the forum and encouraged some open dialogue.

 

[SAJ]

Maybe one day Mturk will actually understand how Mturk works

 

[electrolyte]

Hi. I come with information.

I just talked to a director at MTurk on the phone who wanted to provide some reassurance and answers for us. Here are a few points that should help:

• The new guidelines are not a new direction Amazon's taking. They're meant to be a clarification of the original TOS that was written almost 13 years ago, now with updated language acknowledging how workers work these days.
• There will not be any mass suspension over people using scripts like they do currently. (I asked this specifically.)
• People who will be suspended for script use are the same people who would have been suspended before. He told me directly: "they'll know what they're doing is wrong." It'll be people who are intentionally scamming, cheating, or doing things we all know are wrong.
• I pointed out that some of the language is confusing, such as the parts that say using scripts to sort/search are okay but you can't be hitting the site too frequently. He acknowledged this and will get clarification on what "too frequently" means, but again repeated the above point about people who will be suspended will be those who who would have been before, who know they're doing the wrong thing.
• I'm meeting him in person on Friday afternoon to get more information and ask more questions. After that meeting, I should have more detailed details to share.

 

[catnapped]

Hi. I come with information.

I just talked to a director at MTurk on the phone who wanted to provide some reassurance and answers for us. Here are a few points that should help:

• The new guidelines are not a new direction Amazon's taking. They're meant to be a clarification of the original TOS that was written almost 13 years ago, now with updated language acknowledging how workers work these days.
• There will not be any mass suspension over people using scripts like they do currently. (I asked this specifically.)
• People who will be suspended for script use are the same people who would have been suspended before. He told me directly: "they'll know what they're doing is wrong." It'll be people who are intentionally scamming, cheating, or doing things we all know are wrong.
• I pointed out that some of the language is confusing, such as the parts that say using scripts to sort/search are okay but you can't be hitting the site too frequently. He acknowledged this and will get clarification on what "too frequently" means, but again repeated the above point about people who will be suspended will be those who who would have been before, who know they're doing the wrong thing.
• I'm meeting him in person on Friday afternoon to get more information and ask more questions. After that meeting, I should have more detailed details to share.

Right before you posted I was going to reply with what was said on FB (from TN--Kristy M?) which effectively said the same points--only the bad people really need to look out.

 

[ecko]

Hi. I come with information.

I just talked to a director at MTurk on the phone who wanted to provide some reassurance and answers for us. Here are a few points that should help:

• The new guidelines are not a new direction Amazon's taking. They're meant to be a clarification of the original TOS that was written almost 13 years ago, now with updated language acknowledging how workers work these days.
• There will not be any mass suspension over people using scripts like they do currently. (I asked this specifically.)
• People who will be suspended for script use are the same people who would have been suspended before. He told me directly: "they'll know what they're doing is wrong." It'll be people who are intentionally scamming, cheating, or doing things we all know are wrong.
• I pointed out that some of the language is confusing, such as the parts that say using scripts to sort/search are okay but you can't be hitting the site too frequently. He acknowledged this and will get clarification on what "too frequently" means, but again repeated the above point about people who will be suspended will be those who who would have been before, who know they're doing the wrong thing.
• I'm meeting him in person on Friday afternoon to get more information and ask more questions. After that meeting, I should have more detailed details to share.

Thank you so much for this.

 

[electrolyte]

Right before you posted I was going to reply with what was said on FB (from TN--Kristy M?) which effectively said the same points--only the bad people really need to look out.

Yup, he said he was calling her as well.

 

[Yatagarasu]

Hi. I come with information.

I just talked to a director at MTurk on the phone who wanted to provide some reassurance and answers for us. Here are a few points that should help:

• The new guidelines are not a new direction Amazon's taking. They're meant to be a clarification of the original TOS that was written almost 13 years ago, now with updated language acknowledging how workers work these days.
• There will not be any mass suspension over people using scripts like they do currently. (I asked this specifically.)
• People who will be suspended for script use are the same people who would have been suspended before. He told me directly: "they'll know what they're doing is wrong." It'll be people who are intentionally scamming, cheating, or doing things we all know are wrong.
• I pointed out that some of the language is confusing, such as the parts that say using scripts to sort/search are okay but you can't be hitting the site too frequently. He acknowledged this and will get clarification on what "too frequently" means, but again repeated the above point about people who will be suspended will be those who who would have been before, who know they're doing the wrong thing.
• I'm meeting him in person on Friday afternoon to get more information and ask more questions. After that meeting, I should have more detailed details to share.

Thank you for this update!

 

[YAHU]

Hi. I come with information.

I just talked to a director at MTurk on the phone who wanted to provide some reassurance and answers for us. Here are a few points that should help:

• The new guidelines are not a new direction Amazon's taking. They're meant to be a clarification of the original TOS that was written almost 13 years ago, now with updated language acknowledging how workers work these days.
• There will not be any mass suspension over people using scripts like they do currently. (I asked this specifically.)
• People who will be suspended for script use are the same people who would have been suspended before. He told me directly: "they'll know what they're doing is wrong." It'll be people who are intentionally scamming, cheating, or doing things we all know are wrong.
• I pointed out that some of the language is confusing, such as the parts that say using scripts to sort/search are okay but you can't be hitting the site too frequently. He acknowledged this and will get clarification on what "too frequently" means, but again repeated the above point about people who will be suspended will be those who who would have been before, who know they're doing the wrong thing.
• I'm meeting him in person on Friday afternoon to get more information and ask more questions. After that meeting, I should have more detailed details to share.

tell him I said hi

 

[Tandem In Time]

It's interesting how MTurk reached out to script creators and workers for input on the new worker site but as far as anyone has said, MTurk didn't reach out to anyone for input on these changes, which sound like they're potentially very major.

Here's a scenario that illustrates just the type of worker that these new policy changes apply to:

1. Malicious worker decides they want to easily make $300-$500/day the dishonest way.
2. Worker makes a script to scrape batches of HITs (e.g. Barcodes) shortly after they drop. This is done while logged-out. The script opens each HIT in the batch, extracts the HIT ID and the UPC and saves them, and then skips to the next HIT. This process generates a ton of page calls to Mturk.com (10's-100's per minute) The whole batch can be scraped within minutes of it dropping.
3. All UPCs and corresponding MTurk HIT IDs are stored offline in a custom offline database, and the UPCs are automatically searched via another script against existing online UPC databases. UPCs that result in no products being found are flagged. The UPC search results from the online databases are automatically saved to the workers offline database (this takes <10 minutes)
4. Dishonest worker now logs into MTurk, and runs another script that previews each Barcode HIT in the batch and checks each against their custom database. Once a HITs ID containing a UPC that corresponded to no product is encountered, the script automatically accepts the HIT, checks the box for "No Product Found", and then auto-submits the HIT. Script moves on to find the next HIT, and repeats the process until the batch is exhausted.

Such a malicious worker would violate all three of these 3 key policy points:
1. High frequency page calls
2. Extracting & storing data
3. Substituting their human judgement for letting a script automatically search for and input the information into the HIT.

To reiterate what was recently discussed above, this is VERY different from how the average Turker uses scripts and does not apply to something like HitDB

 

[catnapped]

tell him I said hi

And tell him a certain somebody really needs a rejection.

 

[YAHU]

And tell him a certain somebody really needs a rejection.

:ave:

 

[catnapped]

Here's a scenario that illustrates just the type of worker that these new policy changes apply to:

1. Malicious worker decides they want to easily make $300-$500/day the dishonest way.
2. Worker makes a script to scrape batches of HITs (e.g. Barcodes) shortly after they drop. This is done while logged-out. The script opens each HIT in the batch, extracts the HIT ID and the UPC and saves them, and then skips to the next HIT. This process generates a ton of page calls to Mturk.com (10's-100's per minute) The whole batch can be scraped within minutes of it dropping.
3. All UPCs and corresponding MTurk HIT IDs are stored offline in a custom offline database, and the UPCs are automatically searched via another script against existing online UPC databases. UPCs that result in no products being found are flagged. The UPC search results from the online databases are automatically saved to the workers offline database (this takes <10 minutes)
4. Dishonest worker now logs into MTurk, and runs another script that previews each Barcode HIT in the batch and checks each against their custom database. Once a HITs ID containing a UPC that corresponded to no product is encountered, the script automatically accepts the HIT, checks the box for "No Product Found", and then auto-submits the HIT. Script moves on to find the next HIT, and repeats the process until the batch is exhausted.

Such a malicious worker would violate all three of these 3 key policy points:
1. High frequency page calls
2. Extracting & storing data
3. Substituting their human judgement for letting a script automatically search for and input the information into the HIT.

To reiterate what was recently discussed above, this is VERY different from how the average Turker uses scripts and does not apply to something like HitDB

The one problem with your theory is those batches only last 20-30 minute nowadays. How's he going to make any worthwhile money?

Hard enough doing it the legitimate way nowadays with it being nothing more than a typical overhyped batch

 

[Tandem In Time]

The one problem with your theory is those batches only last 20-30 minute nowadays. How's he going to make any worthwhile money?

Hard enough doing it the legitimate way nowadays with it being nothing more than a typical overhyped batch

My coworker demonstrated a similar script using one of the HITs which require supplying the direct phone, email, and website of academic researchers listed as authors of research papers. His script scraped the whole batch, queried SciFinder and some other sciencey databases, and connected all of the requested info with the info supplied in the HIT.

It took him a whopping 20 minutes to write the script and validate it. He then scraped 100% of the information from the 2500 or so HITs (0.25 or 0.30 each) in the batch in less than 1 hour. He didn't actually submit any of the HITs because he knew it would have been a violation so this was just him proving to me how awesome he is...

Anyway, had he actually pulled the trigger and ran the script he would have made about $600 in about 2 hours. Countless batches that he could have done the same to... Of course he'd eventually get his account suspended.

However, he has a PhD in computer science, so his typical "side jobs" net him 10x-100x what any of us make Turking.

 

[stuffalleysays]

Thank you to all who sought clarification and provided that information for us.

 

[catnapped]

My coworker demonstrated a similar script using one of the HITs which require supplying the direct phone, email, and website of academic researchers listed as authors of research papers. His script scraped the whole batch, queried SciFinder and some other sciencey databases, and connected all of the requested info with the info supplied in the HIT.

It took him a whopping 20 minutes to write the script and validate it. He then scraped 100% of the information from the 2500 or so HITs (0.25 or 0.30 each) in the batch in less than 1 hour. He didn't actually submit any of the HITs because he knew it would have been a violation so this was just him proving to me how awesome he is...

Anyway, had he actually pulled the trigger and ran the script he would have made about $600 in about 2 hours. Countless batches that he could have done the same to... Of course he'd eventually get his account suspended.

However, he has a PhD in computer science, so his typical "side jobs" net him 10x-100x what any of us make Turking.

I totally agree what you said is possible (likely even being done)...more the matter of being able to grab enough hits through the piranhas/sharks and the throttling to make it worth it.

 

[Jharkan]

Hi. I come with information.

I just talked to a director at MTurk on the phone who wanted to provide some reassurance and answers for us. Here are a few points that should help:

• The new guidelines are not a new direction Amazon's taking. They're meant to be a clarification of the original TOS that was written almost 13 years ago, now with updated language acknowledging how workers work these days.
• There will not be any mass suspension over people using scripts like they do currently. (I asked this specifically.)
• People who will be suspended for script use are the same people who would have been suspended before. He told me directly: "they'll know what they're doing is wrong." It'll be people who are intentionally scamming, cheating, or doing things we all know are wrong.
• I pointed out that some of the language is confusing, such as the parts that say using scripts to sort/search are okay but you can't be hitting the site too frequently. He acknowledged this and will get clarification on what "too frequently" means, but again repeated the above point about people who will be suspended will be those who who would have been before, who know they're doing the wrong thing.
• I'm meeting him in person on Friday afternoon to get more information and ask more questions. After that meeting, I should have more detailed details to share.

are you going to take any selfies with him/her?

 

[Achilles2357]

For various reasons I haven't done much turking in the past week, and now I check in here and panic!
But... after a few minutes of thought, I'm thinking this might be much ado about nothing.
I think it is now fairly obvious to me that there is no way in Hell that Amazon would actually systematically punish things like pounding the mturk server or storing data, since I am certain that at least some people who work for mturk know that the very viability of mturk hinges on these things. But for a range of legal and commercial reasons Amazon cannot ever acknowledge what it actually knows about the "mturk worker" world. I suspect this was written by lawyers who don't know much, and may be meant as a way to justify punishment of obvious wrongdoers. I suspect that people who are completing work to the satisfaction of requesters, and thus ensuring that Amazon keeps getting paid, have little to fear. But who knows? Mturk is very, very, very strange....

 

[Drwho10]

Hi. I come with information.

I just talked to a director at MTurk on the phone who wanted to provide some reassurance and answers for us. Here are a few points that should help:

• The new guidelines are not a new direction Amazon's taking. They're meant to be a clarification of the original TOS that was written almost 13 years ago, now with updated language acknowledging how workers work these days.
• There will not be any mass suspension over people using scripts like they do currently. (I asked this specifically.)
• People who will be suspended for script use are the same people who would have been suspended before. He told me directly: "they'll know what they're doing is wrong." It'll be people who are intentionally scamming, cheating, or doing things we all know are wrong.
• I pointed out that some of the language is confusing, such as the parts that say using scripts to sort/search are okay but you can't be hitting the site too frequently. He acknowledged this and will get clarification on what "too frequently" means, but again repeated the above point about people who will be suspended will be those who who would have been before, who know they're doing the wrong thing.
• I'm meeting him in person on Friday afternoon to get more information and ask more questions. After that meeting, I should have more detailed details to share.

Thanks for putting in the leg work as usual :thumbup: