Amazon Updated Amazon Mechanical Turk Terms — What’s Changed

[coffeeowl]

Hello! I have a few more clarifications and other things from my meeting today:

• I'll reiterate what I said earlier about concern about using scripts. You're fine unless you're doing something you know is wrong, like scamming or running scripts that are so powerful that they're damaging the website or causing problems for other workers. Me running HIT Scraper to search for HITs I want to do isn't nearly enough to break MTurk or cause other problems, so things like that are fine.
• I did ask about HIT Database and similar scripts that save/store information on the worker's computer. I would not be worried about that.
• If a worker is doing something that raises a red flag at Amazon, you're not going to be outright suspended immediately. They will send a warning email (the one that says "we've noticed unusual activity with your account"). If you send an email back and ask what the problem was, they will tell you exactly what they noticed and what the concern is. They should not be replying with a generic "you just violated the participation agreement" type of email and should be saying exactly what the problem was that they noticed. If you get an automated or unhelpful response to your inquiry, please tell me and I will follow up because that should not be happening now.
• There was a question about whether the part of the new Participation Agreement that says "you will use your human intelligence and independent judgment to perform Tasks" means you can't talk to other workers. It means you can't share answers, as in you can't share completion codes or tell people how to respond in surveys or what to submit as answers. This is the same as before, and stuff we wouldn't allow on the forum, either.
• Some workers have been mentioning over the last few days that they've been getting signed out of the new site when they sign into the old site, or vice versa. If that's happening, it's a bug. If this is still happening for you, please send me a PM with whatever details you're comfortable with me sharing (worker ID, what you were doing when it happened, screenshots, etc.) and I'll get them to the team.

Two other things:

• The new Participation Agreement says "you will not reject Tasks performed by Workers without good cause" which means Requesters who reject without good cause are in violation of the Participation Agreement.
• The new Acceptable Use policy says requesters cannot "knowingly publish HITs that Workers will be required to return after accepting them". This means requesters that post a batch and tell workers they don't accept N/A or that they should return HITs that are not found are in violation of the Acceptable Use policy.

If you find a requester rejecting without good cause or asking workers to return HITs that can't be found, you can point out to the requester that they are acting in violation of the Participation and/or Acceptable Use policies. This will hopefully have more weight than just telling requesters that doing those things sucks for workers.

I hope that helps!

Thank you for getting some better clarification for us!

 

[jan]

@electrolyte
So, um............did they say they would finally give me masters?

 

[SarahAshlee90]

Hello! I have a few more clarifications and other things from my meeting today:

• I'll reiterate what I said earlier about concern about using scripts. You're fine unless you're doing something you know is wrong, like scamming or running scripts that are so powerful that they're damaging the website or causing problems for other workers. Me running HIT Scraper to search for HITs I want to do isn't nearly enough to break MTurk or cause other problems, so things like that are fine.
• I did ask about HIT Database and similar scripts that save/store information on the worker's computer. I would not be worried about that.
• If a worker is doing something that raises a red flag at Amazon, you're not going to be outright suspended immediately. They will send a warning email (the one that says "we've noticed unusual activity with your account"). If you send an email back and ask what the problem was, they will tell you exactly what they noticed and what the concern is. They should not be replying with a generic "you just violated the participation agreement" type of email and should be saying exactly what the problem was that they noticed. If you get an automated or unhelpful response to your inquiry, please tell me and I will follow up because that should not be happening now.
• There was a question about whether the part of the new Participation Agreement that says "you will use your human intelligence and independent judgment to perform Tasks" means you can't talk to other workers. It means you can't share answers, as in you can't share completion codes or tell people how to respond in surveys or what to submit as answers. This is the same as before, and stuff we wouldn't allow on the forum, either.
• Some workers have been mentioning over the last few days that they've been getting signed out of the new site when they sign into the old site, or vice versa. If that's happening, it's a bug. If this is still happening for you, please send me a PM with whatever details you're comfortable with me sharing (worker ID, what you were doing when it happened, screenshots, etc.) and I'll get them to the team.

Two other things:

• The new Participation Agreement says "you will not reject Tasks performed by Workers without good cause" which means Requesters who reject without good cause are in violation of the Participation Agreement.
• The new Acceptable Use policy says requesters cannot "knowingly publish HITs that Workers will be required to return after accepting them". This means requesters that post a batch and tell workers they don't accept N/A or that they should return HITs that are not found are in violation of the Acceptable Use policy.

If you find a requester rejecting without good cause or asking workers to return HITs that can't be found, you can point out to the requester that they are acting in violation of the Participation and/or Acceptable Use policies. This will hopefully have more weight than just telling requesters that doing those things sucks for workers.

I hope that helps!

Just curious, does the part about "knowingly publish HITs that Workers will be required to return after accepting them" mean unpaid screeners are no longer allowed on surveys?

 

[electrolyte]

Just curious, does the part about "knowingly publish HITs that Workers will be required to return after accepting them" mean unpaid screeners are no longer allowed on surveys?

That's my thinking. You're doing work (answering screening questions) in exchange for no pay and being forced to return the HIT.

 

[SweetiePuff]

Thanks for the update, @electrolyte Sounds like this 'changes' won't affect us lil people so much who are just turking as we have been.

Curious ... are you able to tell us the Amazon Director's name who provided you with this answers? Did you meet at his office?

 

[electrolyte]

Thanks for the update, @electrolyte Sounds like this 'changes' won't affect us lil people so much who are just turking as we have been.

Curious ... are you able to tell us the Amazon Director's name who provided you with this answers? Did you meet at his office?

That's my feeling, too. As long as us normal workers are doing what we normally doing in the honest ways we're already doing it, we're fine.

I don't feel comfortable naming names in this thread since I don't want him piled on by people who are angry about not being accepted to be a worker or workers who get suspended or something like that, you know? :confused2:

I didn't meet him at the office yesterday. We met at a nearby coffee shop and I had a really good chai tea!

 

[SweetiePuff]

I don't feel comfortable naming names in this thread since I don't want him piled on by people who are angry about not being accepted to be a worker or workers who get suspended or something like that, you know? :confused2:

I didn't meet him at the office yesterday. We met at a nearby coffee shop and I had a really good chai tea!

I love how you think of all the angles! Great call

 

[Cara]

amazing news, thanks electrolyte!

 

[electrolyte]

Just curious, does the part about "knowingly publish HITs that Workers will be required to return after accepting them" mean unpaid screeners are no longer allowed on surveys?

That's my thinking. You're doing work (answering screening questions) in exchange for no pay and being forced to return the HIT.

Okay, thinking about this again, I bet what requesters will try to do to get around it is to put a sentence in the HIT preview page telling workers to do the screener before accepting the HIT.

 

[catnapped]

Okay, thinking about this again, I bet what requesters will try to do to get around it is to put a sentence in the HIT preview page telling workers to do the screener before accepting the HIT.

That may be the go-around for the batch requesters too..."please search for the email/website before accepting the hit...thanks!"

Yeah, that'll work out *super* /sarcasm

 

[catnapped]

Okay, thinking about this again, I bet what requesters will try to do to get around it is to put a sentence in the HIT preview page telling workers to do the screener before accepting the HIT.

Wonder how this affects Pirc? What's he gonna say to take the survey before you accept the hit?

 

[electrolyte]

That may be the go-around for the batch requesters too..."please search for the email/website before accepting the hit...thanks!"

Yeah, that'll work out *super* /sarcasm

I could see that. Ugh. I think if it becomes a trend or requesters push back, we could point to the part that says " As a Requester, you agree that... you will interact with Workers in a professional and courteous manner". Making workers do work unpaid is not interacting in a professional or courteous manner. I would also at that point lobby MTurk to ask legal to add something in a future version of the Participation Agreement saying that requesters can't ask workers do unpaid work.

Edit: Okay, I just jumped directly to it and emailed MTurk about this now.

 

[electrolyte]

Wonder how this affects Pirc? What's he gonna say to take the survey before you accept the hit?

It's not going to affect him and/or he won't care unless someone brings it to his attention. This goes for all requesters. They're not notified of these changes so unless someone points it out and shows the requester that there are changes, they're not going to realize it. I foresee a lot of emails from workers to requesters in these upcoming weeks educating the requesters about the changes. And as I said earlier, some will probably care and want to follow the rules and will change their HIT. Others won't, and we'll report them on Turkopticon and maybe report them to Amazon and then ignore their HITs.

 

[Achilles2357]

Okay, thinking about this again, I bet what requesters will try to do to get around it is to put a sentence in the HIT preview page telling workers to do the screener before accepting the HIT.

This wouldn't be bad at all if the screener was a separate lower paying hit and they had a link to it.

 

[electrolyte]

This wouldn't be bad at all if the screener was a separate lower paying hit and they had a link to it.

For sure. A separate paid screener HIT is how it should be done.

 

[Achilles2357]

For sure. A separate paid screener HIT is how it should be done.

I think what could work would be for Amazon to offer requesters a template that asks if they need a screener hit, and takes them directly to the setup for this when they are setting up the main hit. The template would ensure that the link for the screener is included in the instructions for the main hit. There is no need to enforce this as a technical requirement, so far as I grasp things, since I don't think requesters need to use any Amazon templates at all. But if requesters like Mr. Pirc then end up blatantly violating the rules, they are still in violation of the rules and Amazon can enforce them via "human intelligence". Maybe.

EDIT: Maybe Amazon could offer a discount to requesters on any hit that is set up as a screener hit. Amazon still makes more profit this way.

 

[Antamania]

We fly blind without Turkopticon.
I would really love to know how you perceive the new TOS
A @Antamania thanks!

Oh hello there. I've been away, so I haven't had too much of a chance to check in with you folks recently. As far as using scripts and such go, I'm pretty sure the safeguards are in place, so I doubt very much they're going to be causing any trouble for any turkers. You already page request error tremendously if you try to push it passed what they find acceptable.

AFAIK TO hasn't heard anything from an official Amazon person for a few months now, back when they initially launched the new page. That person was operating under anonymity too because they weren't sure if they should be aiding us. There may have been additional contact to Six or Lilly, but normally they'd post in the moderation forums about it, and I have seen nothing.

Edit:
After reading through some more of this thread - it looks like the only thing that may be changing on TO is people will likely be allowed to mark a hit as violating TOS for rejections. I'm going to post a new discussion on the moderator forum tomorrow and I will post back here with the results, if anyone wishes to discuss them, or has any questions. I could also make a separate thread with that, if administration wishes so I do not derail this one.

Thanks for tagging me, and thank you all for this information I certainly didn't have before reading over the thread!

Edit 2:
After reading the full thread I removed some conjecture that was unneeded from my post.

 

[<Gucci>]

Oh hello there. I've been away, so I haven't had too much of a chance to check in with you folks recently. As far as using scripts and such go, I'm pretty sure the safeguards are in place, so I doubt very much they're going to be causing any trouble for any turkers. You already page request error tremendously if you try to push it passed what they find acceptable.

AFAIK TO hasn't heard anything from an official Amazon person for a few months now, back when they initially launched the new page. That person was operating under anonymity too because they weren't sure if they should be aiding us. There may have been additional contact to Six or Lilly, but normally they'd post in the moderation forums about it, and I have seen nothing.

Edit:
After reading through some more of this thread - it looks like the only thing that may be changing on TO is people will likely be allowed to mark a hit as violating TOS for rejections. I'm going to post a new discussion on the moderator forum tomorrow and I will post back here with the results, if anyone wishes to discuss them, or has any questions. I could also make a separate thread with that, if administration wishes so I do not derail this one.

Thanks for tagging me, and thank you all for this information I certainly didn't have before reading over the thread!

Edit 2:
After reading the full thread I removed some conjecture that was unneeded from my post.

Thank you for all of this very valuable information. I’ll be curious to know if we will be allowed to mark rejections as TOS violations on turkopticon. Some developments were made after I tagged you. I’ll be looking out for your posts!

 

[catnapped]

• The new Participation Agreement says "you will not reject Tasks performed by Workers without good cause" which means Requesters who reject without good cause are in violation of the Participation Agreement.
  
  Going to end up testing this out real soon from the looks of it.

 

[kimadagem]

• The new Participation Agreement says "you will not reject Tasks performed by Workers without good cause" which means Requesters who reject without good cause are in violation of the Participation Agreement.

• The new Acceptable Use policy says requesters cannot "knowingly publish HITs that Workers will be required to return after accepting them". This means requesters that post a batch and tell workers they don't accept N/A or that they should return HITs that are not found are in violation of the Acceptable Use policy.

Apologies for the late response, I'm in catch-up mode.

About these - If they're now TOS violations, how are we going to report them to Amazon? Right now there's a way to report currently posted hits (on both the current and new sites), but hits that have already been returned/submitted/rejected are past that point. Yes, we could add that to TO reviews but that's not going to get back to Amazon; contacting the requester is very likely to have no effect; and messaging Amazon about requester issues usually just produces the "we don't get involved with those" response. I like that Amazon is finally trying to address this but they need to come up with a way to do that because, as "catnapped" said, this will be tested.