New to MTurk Requesting and getting lots of different advice

piperfishpants

New Member
Requester
Joined
Mar 12, 2020
Messages
3
Reaction score
0
Points
1
Age
38
Gender
Male
I'm live-in tech support for my awesome wife as she pursues her PhD and I have been helping her publish her research study on MTurk. I want to make sure I am using this system to the best of its ability, but it is all very new to me! My wife plans to use MTurk for data collection in the future and I don't know if I got things right this first time. I think a lot of the advice she has been given regarding batch setup and then relayed to me has been spotty at best.

The BIG problem:
  • Responses seem to have dropped off fairly quickly for my wife's batch (she has a participant goal of over 900)
What I have done so far that appears to be working:
  • She is hosting a Qualtrics survey via MTurk for data collection for her thesis
  • Links to and from the survey appear to be working correctly and validation does not appear to be an issue
  • All respondents are being approved and she has been attempting to approve them quickly
  • She has responded to messages when appropriate
  • The only restrictions/parameters on participants are:
    • Location (within the US)
    • Sex/Gender (Male)
    • Contains adult content (discusses questions related to sexual behaviors not pornography)
What I am worried about and would love some guidance on:
  • Average time to complete the survey is showing as 33 minutes on MTurk
    • Her survey is VERY long and she can't change that
  • She was advised to set the reimbursement at $ 0.50 per completion
    • This seems low but she was advised this is a reasonable amount (?)
    • She is largely self-funding this study so money is a limiting factor (as usual) ?
    • It is unclear to me if/how she can change reimbursement amounts in the batch now that it is live.
    • She has a very high participant goal
  • The length of her batch collection is 60 days
    • She is 30-ish days into collection and responses have tapered off
TLDR Questions:
  • Does a long batch availability time mean her task is at the bottom of the list of available tasks (can this be fixed without starting a new batch)?
  • Is her reimbursement too low to generate her intended number of responses given survey length?
  • Is there a way to easily change batch settings to improve response rate once a batch is live?
The MTurk community seems relatively small and any way I can improve what I'm doing would be very much appreciated!
 

A6_Foul_Out

I only use Organic TurkerView reviews
Contributor
HIT Poster
Joined
Aug 17, 2016
Messages
11,754
Reaction score
21,657
Points
1,538
Location
CT's Attic
Gender
Male
requester name?
 

A6_Foul_Out

I only use Organic TurkerView reviews
Contributor
HIT Poster
Joined
Aug 17, 2016
Messages
11,754
Reaction score
21,657
Points
1,538
Location
CT's Attic
Gender
Male
  • Like
Reactions: piperfishpants

A6_Foul_Out

I only use Organic TurkerView reviews
Contributor
HIT Poster
Joined
Aug 17, 2016
Messages
11,754
Reaction score
21,657
Points
1,538
Location
CT's Attic
Gender
Male
yeah, this just looks extremely underpaid.

It doesn't help using amazon's premium qualification cutting the wage in half and sending that to amazon instead of the worker.

republish and you might get more responses.
 
  • Like
Reactions: piperfishpants

piperfishpants

New Member
Requester
Joined
Mar 12, 2020
Messages
3
Reaction score
0
Points
1
Age
38
Gender
Male
the name they use on their requester account, not the name of the researcher
Apologies for the confusion on the requester account info.

how long does it actually take to take the survey?
yeah, $1/hour is pretty shit

It doesn't help using amazon's premium qualification cutting the wage in half and sending that to amazon instead of the worker.

as wage lowers, completion rate speed lowers as well, it will get done, eventually, surely. To what quality? Dunno.

Just pull the batch, and re-publish the needed number of responses.
Yeah. Again, I think she has gotten pretty poor guidance from the start

Unfortunately, she is required to use a premium qualification given her study parameters.

Thanks for the feedback! Any other tips to make things better on the respondent end?
 
Last edited:

A6_Foul_Out

I only use Organic TurkerView reviews
Contributor
HIT Poster
Joined
Aug 17, 2016
Messages
11,754
Reaction score
21,657
Points
1,538
Location
CT's Attic
Gender
Male
Does re-publishing push a task "up" the list of available tasks?
yeah, pretty much. The current task is currently just sitting on Mturk not getting done. Partly because it's bad, partly because it's old.

https://requester.mturk.com/pricing

using "Gender is Male" costs .50 to amazon, per respondent.
Any other tips to make things better on the respondent end?
? not really. if the survey is like 400 bubbles for like $1 it's going to be bad no matter what. pulling the original batch and republishing will probably help get more respondents.
 

TSolo315

SnapNCrackle
Administrator
Joined
Jan 12, 2016
Messages
5,000
Reaction score
16,975
Points
2,538
Gender
Male
Does re-publishing push a task "up" the list of available tasks?
Yeah but it won't help much.

You are going to run into a serious issue with selection bias when posting such low paying work. Only the most desperate of people will complete your survey, and they won't be trying particularly hard.

While pay rate and result quality don't necessarily scale equally (higher pay does guarantee faster submissions though,) there is a floor beneath which the data is pretty useless IMO.
 

su2020

New Member
Joined
Mar 19, 2020
Messages
4
Reaction score
3
Points
203
Age
50
Gender
Female
Hello, in the same boat as your wife. My survey is new but I collected the fist 69% really quick and now it is just going very slow. Any advice?
 

jan

Moderator
Moderator
Joined
Jan 12, 2016
Messages
25,796
Reaction score
49,511
Points
1,463
Gender
Female
Hello, in the same boat as your wife. My survey is new but I collected the fist 69% really quick and now it is just going very slow. Any advice?
how large is your n?
tbh, the same advice sort of follows. People will jump on well paying hits.
also, keep in mind we're all a little distracted this weekend. But most of us are home (all across the country!). And if there is a good/well paying hit - people will take it.

What is the hourly wage of the hit? Is it minimum wage??
 

femmebritt

Member
Joined
Aug 18, 2019
Messages
180
Reaction score
283
Points
263
Age
35
Gender
Female
I'm live-in tech support for my awesome wife as she pursues her PhD and I have been helping her publish her research study on MTurk. I want to make sure I am using this system to the best of its ability, but it is all very new to me! My wife plans to use MTurk for data collection in the future and I don't know if I got things right this first time. I think a lot of the advice she has been given regarding batch setup and then relayed to me has been spotty at best.

The BIG problem:
  • Responses seem to have dropped off fairly quickly for my wife's batch (she has a participant goal of over 900)
What I have done so far that appears to be working:
  • She is hosting a Qualtrics survey via MTurk for data collection for her thesis
  • Links to and from the survey appear to be working correctly and validation does not appear to be an issue
  • All respondents are being approved and she has been attempting to approve them quickly
  • She has responded to messages when appropriate
  • The only restrictions/parameters on participants are:
    • Location (within the US)
    • Sex/Gender (Male)
    • Contains adult content (discusses questions related to sexual behaviors not pornography)
What I am worried about and would love some guidance on:
  • Average time to complete the survey is showing as 33 minutes on MTurk
    • Her survey is VERY long and she can't change that
  • She was advised to set the reimbursement at $ 0.50 per completion
    • This seems low but she was advised this is a reasonable amount (?)
    • She is largely self-funding this study so money is a limiting factor (as usual) ?
    • It is unclear to me if/how she can change reimbursement amounts in the batch now that it is live.
    • She has a very high participant goal
  • The length of her batch collection is 60 days
    • She is 30-ish days into collection and responses have tapered off
TLDR Questions:
  • Does a long batch availability time mean her task is at the bottom of the list of available tasks (can this be fixed without starting a new batch)?
  • Is her reimbursement too low to generate her intended number of responses given survey length?
  • Is there a way to easily change batch settings to improve response rate once a batch is live?
The MTurk community seems relatively small and any way I can improve what I'm doing would be very much appreciated!

Average time to complete 33 minutes and compensation is $0.50.... She probably doesn't have a lot of people signing up for that. Most people, myself included, don't do surveys for < $1/10 minutes.