04/29 - Shadows of a Setting Sunday!

[jan]

Or learn to ask different questions instead of copy and paste. There's a few that have been rather clever.

They can't. They are validating their hypothesis against subscales that are already normed/validated. So, when we get a survey that has those same 3 subscales (let's say for feminism) - they are comparing your responses to those against their 'new' scale. So one set of questions is different. And they are looking to see if there is an association between that 'new' subscale (or variable) and subscales that usually show an association -and have good validity/reliability.

If they had just new questions, they wouldn't have a way to compare their data against a known instrument.

 

[InfiniteChanges]

They can't. They are validating their hypothesis against subscales that are already normed/validated. So, when we get a survey that has those same 3 subscales (let's say for feminism) - they are comparing your responses to those against their 'new' scale. So one set of questions is different. And they are looking to see if there is an association between that 'new' subscale (or variable) and subscales that usually show an association -and have good validity/reliability.

If they had just new questions, they wouldn't have a way to compare their data against a known instrument.

I'm not sure how this relates to the speed at which we can answer the questions. It's not that they are looking for people who have never been tested before, they are just using a false assertion that all people test at the speed that non-professional survey takers do. My data is no less valid that a brand new person, I just do it a lot freaking faster.

 

[Rogue and Ready]

Greetings from Amazon Mechanical Turk,

The following is a summary of activity for your Mechanical Turk account for the week ending Apr 28, 2018.

Your HIT activity for this week:
- Number of HITs submitted: 72

Approvals and payments that occurred this week:
- Number of HITs approved: 74
- Number of HITs rejected: 0
- Bonus reward earned: $2.75
- HIT reward earned: $44.42
- Total Amount earned this week: $47.17

Not bad for me. I took a lot of time off!

Good morning everyone!

 

[Turtlefemm]

Title: Specify 5 actions that you could do on a web page | AcceptA Requester: Percy Liang [A29YQT3EEC0ZEC] Contact TV: [Hrly: $5.02] [Pay: 1.33] [Fast: null] [Comm: null] [Rej: 0] [ToS: 0] [Blk: 0] TO: [Pay: 2.94] [Fast: 4.29] [Comm: 4.04] [Fair: 4.75] [Reviews: 103] [ToS: 1] TO2: [Hrly: $10.37] [Pen: 1.50 days] [Res: ---] [Rec: 67% of 3] [Rej: 0] [ToS: 0] [Brk: 0] Reward: $0.20 Duration: 1 hour Available: 792 Description: For example, on a flight booking site, an action would be "type LAX into the destination field" Requirements: Location In CA, GB, US; HIT approval rate (%) GreaterThanOrEqualTo 95

 

[jan]

I'm not sure how this relates to the speed at which we can answer the questions. It's not that they are looking for people who have never been tested before, they are just using a false assertion that all people test at the speed that non-professional survey takers do. My data is no less valid that a brand new person, I just do it a lot freaking faster.

Well, now, I can't answer for them .
But there are many scales that assume you have never been exposed to the experimental material (those subscales) before. Or, if you have, it's been a long time since you saw them and really don't remember the content. The idea is that everyone (the population that they are testing) is all on equal footing w/respect to that exposure. There are several biases/ threats to validity if the conditions are not equal.
The other potential problem, from a research perspective, might be that they are priming you for something or other (so if they want you to think about a time you had power.......or no power). They don't want a stock answer - they want you to really inhabit that feeling. Really feeling down because that terrible boss belittled you takes time. If you just have an answer at the tip of your fingertips, then the 'feeling' they are trying to get at.....well. It's not there.

IME (and I'm no expert) - from an idea of just time - they are supposed to test out the survey and see how long it takes to read. Obviously, there is variation in reading styles and speeds. But.....well. You know.

Anyway, those are just my thoughts.

 

[InfiniteChanges]

Title: [S1] Short survey on decision-making(~ 5 minutes) | PANDA
Requester: Julia Hur [A3AYYY0T3VKP78]
TurkerView: [ N/A ]
Description: Take a short survey on your decision-making
Duration: 01:30:00
Available: 1
Reward: $1.00
Qualifications:

• Total approved HITs LessThanOrEqualTo 50000
• Total approved HITs GreaterThanOrEqualTo 50
• HIT approval rate (%) GreaterThanOrEqualTo 95
• Location In US
• Exc: [19-85562] DoesNotExist

[3244QHISJ130A0NLDKH7A0U8YK20LI]​

3-4 minutes.

 

[EverfreeTree]

This torture implement keeps popping up, so I'll keep posting it.

Lord Dermott demands a sacrifice.

Title: !!LONG!! Pitch Perception Experiment - Up to $17.50 BONUS possible for a total of $17.75 in 1:45-2 hours | AcceptA Requester: McDermott Experiments [A19YWY6E1CP566] Contact TV: [Hrly: $13.17] [Pay: 3.29] [Fast: 5.00] [Comm: 4.50] [Rej: 0] [ToS: 0] [Blk: 0] TO: [Pay: 3.27] [Fast: 4.33] [Comm: 3.10] [Fair: 4.40] [Reviews: 154] [ToS: 0] TO2: No Reviews Reward: $0.25 Duration: 4 hours Available: 1 Description: Complete a variety of short auditory perception experiments - making judgments about whether one note is higher or lower, louder or softer, etc than another note Requirements: None

 

[InfiniteChanges]

This torture implement keeps popping up, so I'll keep posting it.

Lord Dermott demands a sacrifice.

Title: !!LONG!! Pitch Perception Experiment - Up to $17.50 BONUS possible for a total of $17.75 in 1:45-2 hours | AcceptA Requester: McDermott Experiments [A19YWY6E1CP566] Contact TV: [Hrly: $13.17] [Pay: 3.29] [Fast: 5.00] [Comm: 4.50] [Rej: 0] [ToS: 0] [Blk: 0] TO: [Pay: 3.27] [Fast: 4.33] [Comm: 3.10] [Fair: 4.40] [Reviews: 154] [ToS: 0] TO2: No Reviews Reward: $0.25 Duration: 4 hours Available: 1 Description: Complete a variety of short auditory perception experiments - making judgments about whether one note is higher or lower, louder or softer, etc than another note Requirements: None

Is it the same one from last week, or a new one?

 

[SAJ]

Title: Short survey regarding perceptions of your work environment | Accept Requester: JESSICA CLEMONS [A3EJGTU5B4TDKX] Contact TV: No Reviews TO: No Reviews TO2: No Reviews Reward: $2.00 Duration: 2 hours Available: 1 Description: This is a simple 20 minute survey regarding your current work environment. Questions will ask you about perceptions of yourself and your work environment.. Requirements: HIT approval rate (%) GreaterThanOrEqualTo 95 [tr][td] HIT exported from Mturk Suite v2.2.3​

[/td][/tr]

[tr][td][/td][/tr]

 

[Persephone78]

This torture implement keeps popping up, so I'll keep posting it.

Lord Dermott demands a sacrifice.

Title: !!LONG!! Pitch Perception Experiment - Up to $17.50 BONUS possible for a total of $17.75 in 1:45-2 hours | AcceptA Requester: McDermott Experiments [A19YWY6E1CP566] Contact TV: [Hrly: $13.17] [Pay: 3.29] [Fast: 5.00] [Comm: 4.50] [Rej: 0] [ToS: 0] [Blk: 0] TO: [Pay: 3.27] [Fast: 4.33] [Comm: 3.10] [Fair: 4.40] [Reviews: 154] [ToS: 0] TO2: No Reviews Reward: $0.25 Duration: 4 hours Available: 1 Description: Complete a variety of short auditory perception experiments - making judgments about whether one note is higher or lower, louder or softer, etc than another note Requirements: None

This.
This gives me Nam flashbacks.

 

[EverfreeTree]

Not sure I WANT to do a 30 minute reading and comprehension task, but here it is anyways.

Title: Coffee Shop World - Narrative Experiment | AcceptA Requester: Andre Oliveira Beukers [A2VWN95BKHWBS8] Contact TV: [Hrly: $11.57] [Pay: 4.00] [Fast: null] [Comm: null] [Rej: 0] [ToS: 0] [Blk: 0] TO: [Pay: 4.50] [Fast: 5.00] [Comm: 0.00] [Fair: 5.00] [Reviews: 2] [ToS: 0] TO2: No Reviews Reward: $6.00 Duration: 2 hours Available: 1 Description: You will read stories, a sentence at a time, and be asked to predict what happens next in the story. Requirements: HIT approval rate (%) GreaterThanOrEqualTo 95; Location EqualTo US

Is it the same one from last week, or a new one?

I THINK it's a different one, but I can't be sure. It's the same one I did yesterday, if my script is being accurate.

 

[Justin D]

Dear MTurk Worker #A32QOCAC7XZJCP,

Great news!

We are running a new game today at 4pm (ET., Sun. April 29th).

Based on feedback from mturkers, we've raised our pay.

Average payer now makes 2.50$ in under 5minutes. Some win up to 5$.

We will send login links to your Mturk account shortly before it's time to play.

Please click the link below to reserve your spot.Sunday Apr 29 2018 at 4:00PM ET [MOD EDIT]

Thank you,
Douglas Guilbeault
Social Innovations Lab

 

[Justin D]

Greetings from Amazon Mechanical Turk,

The following is a summary of activity for your Mechanical Turk account for the week ending Apr 28, 2018.

Your HIT activity for this week:
- Number of HITs submitted: 1,279

Approvals and payments that occurred this week:
- Number of HITs approved: 1,403
- Number of HITs rejected: 5
- Bonus reward earned: $7.15
- HIT reward earned: $103.93
- Total Amount earned this week: $111.08

 

[EverfreeTree]

And to keep it going, here is a HIT from the guy I did a HIT for earlier.
Most of the reward is in bonuses. He approved and sent the bonus in like 20 minutes, for the other HIT.

Title: An academic decision-making Game (30min, average 4$ payment) | AcceptA Requester: ofir yakobi [A2PBVTMRTXA98I] Contact TV: [Hrly: $24.19] [Pay: 4.88] [Fast: 5.00] [Comm: null] [Rej: 0] [ToS: 0] [Blk: 0] TO: [Pay: 1.67] [Fast: 5.00] [Comm: 0.00] [Fair: 5.00] [Reviews: 3] [ToS: 0] TO2: No Reviews Reward: $0.01 Duration: 1 hour 5 minutes Available: 1 Description: A short academic experiment, less than 30minutes - average 4$. You must start the HIT immediately or it expires Requirements: praticipated_in_orp DoesNotExist; ORP2_experiment DoesNotExist; Location EqualTo US

*edit*
The TO states this one MAY be unpleasant, unless they've made it easier.

 

[InfiniteChanges]

Well, now, I can't answer for them .
But there are many scales that assume you have never been exposed to the experimental material (those subscales) before. Or, if you have, it's been a long time since you saw them and really don't remember the content. The idea is that everyone (the population that they are testing) is all on equal footing w/respect to that exposure. There are several biases/ threats to validity if the conditions are not equal.
The other potential problem, from a research perspective, might be that they are priming you for something or other (so if they want you to think about a time you had power.......or no power). They don't want a stock answer - they want you to really inhabit that feeling. Really feeling down because that terrible boss belittled you takes time. If you just have an answer at the tip of your fingertips, then the 'feeling' they are trying to get at.....well. It's not there.

IME (and I'm no expert) - from an idea of just time - they are supposed to test out the survey and see how long it takes to read. Obviously, there is variation in reading styles and speeds. But.....well. You know.

Anyway, those are just my thoughts.

I completely understand, but on the flip side there are people (more than just myself I would assume) that saw through those promptings (or the "you're going to work with someone else" style non-sense) gimmicks when I first started. I understand from a requester standpoint you will always have some outliers (and maybe my data should get tossed sometimes), but what baffles me is that requesters come to mTurk which handles this stuff for tens of thousands of people who are doing the same thing and they expect to find workers who do a survey a week or less who will also just happen to stumble onto their HIT and meet the qualificiations.

That's like trying to sell your car to an auto dealership because you know it's a good place to sell a car, and then getting mad that they don't just give you the asking price because they know how to look for the ways of making a bigger profit on their own end.

 

[rsclone]

Greetings from Amazon Mechanical Turk,
You've received a bonus from MTurk Requesters for work related to 3YLTXLH3DFJT0A7I4BW992TUDXJHP7.
The value of your bonus is: $4.00 USD
The Requester included this note:
Thanks for completing the survey!

PE for the day just quadrupled....lol

 

[Manly5000]

And to keep it going, here is a HIT from the guy I did a HIT for earlier.
Most of the reward is in bonuses. He approved and sent the bonus in like 20 minutes, for the other HIT.

Title: An academic decision-making Game (30min, average 4$ payment) | AcceptA Requester: ofir yakobi [A2PBVTMRTXA98I] Contact TV: [Hrly: $24.19] [Pay: 4.88] [Fast: 5.00] [Comm: null] [Rej: 0] [ToS: 0] [Blk: 0] TO: [Pay: 1.67] [Fast: 5.00] [Comm: 0.00] [Fair: 5.00] [Reviews: 3] [ToS: 0] TO2: No Reviews Reward: $0.01 Duration: 1 hour 5 minutes Available: 1 Description: A short academic experiment, less than 30minutes - average 4$. You must start the HIT immediately or it expires Requirements: praticipated_in_orp DoesNotExist; ORP2_experiment DoesNotExist; Location EqualTo US

*edit*
The TO states this one MAY be unpleasant, unless they've made it easier.

This one took me 15 minutes and I have a bonus of $3.55 coming. It was monotonous but not difficult.

 

[jan]

I completely understand, but on the flip side there are people (more than just myself I would assume) that saw through those promptings (or the "you're going to work with someone else" style non-sense) gimmicks when I first started. I understand from a requester standpoint you will always have some outliers (and maybe my data should get tossed sometimes), but what baffles me is that requesters come to mTurk which handles this stuff for tens of thousands of people who are doing the same thing and they expect to find workers who do a survey a week or less who will also just happen to stumble onto their HIT and meet the qualificiations.

That's like trying to sell your car to an auto dealership because you know it's a good place to sell a car, and then getting mad that they don't just give you the asking price because they know how to look for the ways of making a bigger profit on their own end.

It's true. Have you seen these?
https://www.psychologytoday.com/us/...xperience-amazon-mechanical-turk-mturk-worker
https://www.psychologytoday.com/us/...-i-will-make-when-using-amazon-mturk-research
This has been posted before. He's really nice about us (thinks we should be paid much more. But his conclusion is that he'll only pilot studies - or do some exploratory research on mturk in the future because of the whole over exposure thing.

 

[TSolo315]

My data is no less valid that a brand new person, I just do it a lot freaking faster.

When you see the "a ball and a bat" problem for the thousandth time, and give them the answer that you had memorized long ago -- you are skewing their results as they are testing if you answer the question instinctively (which a majority of people do.) Now, if they don't ask you "Have you seen these questions before???!" it's their own fault they are getting skewed data, but the point stands.

I completely understand, but on the flip side there are people (more than just myself I would assume) that saw through those promptings (or the "you're going to work with someone else" style non-sense) gimmicks when I first started. I understand from a requester standpoint you will always have some outliers (and maybe my data should get tossed sometimes), but what baffles me is that requesters come to mTurk which handles this stuff for tens of thousands of people who are doing the same thing and they expect to find workers who do a survey a week or less who will also just happen to stumble onto their HIT and meet the qualificiations.

That's like trying to sell your car to an auto dealership because you know it's a good place to sell a car, and then getting mad that they don't just give you the asking price because they know how to look for the ways of making a bigger profit on their own end.

A lot of researchers assume they can run a study on mturk like they would in person -- and if they ran a study in person there's a good chance participants would only have done one survey a week or less. They just don't have the perspective, which is a probably a good thing for us because it means more work available.

 

[rsclone]

Title: Answer a survey about your experience as a Turker | Accept Requester: Maritza Johnson [A2O8J68AMFBLNR] Contact TO: No Reviews TO2: No Reviews Reward: $1.75 Duration: 30 minutes Available: 1 Description: We're interested in your experience being a Turker. This survey will be used to identify Turkers who are interested in participating in a 30-40m bonus task using a new tool for democratically writing resolutions. Requirements: Total approved HITs GreaterThan 500; HIT approval rate (%) GreaterThan 97 [tr][td] HIT exported from Mturk Suite v2.2.3​

[/td][/tr]

[tr][td][/td][/tr]