01/22 - Ferocious WARJlords Friday

Which Survivor team should escape going to council this week?

  • 5ALTY CA9TCHA5

  • WARJLords


The results of this poll are hidden until it is manually edited by the user or site admin.

Status
Not open for further replies.

Jaded

The real themildone
Administrator
Joined
Jan 10, 2016
Messages
46,614
Reaction score
123,828
Points
1,414
Age
123
Gender
Female

electrolyte

The Ghost of MTurk Past
Contributor
Joined
Jan 10, 2016
Messages
19,184
Reaction score
45,978
Points
1,313
There is no empirical evidence, only anecdotal blog posts.

There is no empirical evidence for either side of the argument... and masters does have benefits for new requesters. For one, theres a high likelihood that any possible volume scammers have already had their account wrecked before they ever have a chance to get masters. The value of the benefits that come from using it aren't concrete or any higher than creating closed quals... but the other side of the argument is basically "we assume its a lottery because we have no information and its unfair that your work isn't available to all of us" The only benefit the non-masters side can honestly bring to the table is that the worker sample size is larger which means more chances to hit target demographics in a timely manner.
There is an academic research study that showed using Masters was no better than using qualifications like 5K and 98%. But I'm going to have to search for the link.
 

Jaded

The real themildone
Administrator
Joined
Jan 10, 2016
Messages
46,614
Reaction score
123,828
Points
1,414
Age
123
Gender
Female
Title: Survey on work and motivation | PANDA
Requester: Nicolas [A2HYY92NU80YTF] (Contact)
(TO): [Pay: 3.00][Fair: 5.00][Comm: 5.00][Fast: 5.00]
Description:
To participate in this survey, you must currently have a job.
Time: 60 minutes
Hits Available: 1
Reward: $0.20
Qualifications: None
 

Blue

Unlikeable Ass
Contributor
Joined
Jan 11, 2016
Messages
4,183
Reaction score
13,272
Points
838
Gender
Male
There is an academic research study that showed using Masters was no better than using qualifications like 5K and 98%. But I'm going to have to search for the link.
If it exists and I don't recall... I'd like to see it. My memory is shit.

But we are also talking about new requesters... ones who don't know about using 5k and 98% or other metrics that may not have much discernible meaning to them at that stage in their usage of mturk.
 

Jaded

The real themildone
Administrator
Joined
Jan 10, 2016
Messages
46,614
Reaction score
123,828
Points
1,414
Age
123
Gender
Female
Title: Consumer Decisions - Product Survey(~ 2 minutes) | PANDA
Requester: UCLA Anderson Researcher [A3F2ELCFTPI9GB] (Contact)
(TO): [Pay: 4.17][Fair: 4.95][Comm: 5.00][Fast: 4.90]
Description:
Behavioral Survey. Approximately 2 minutes. Compensation: $0.25.
Time: 10 minutes
Hits Available: 1
Reward: $0.25
Qualifications: Exc: [721045962-5584] has not been granted; Total approved HITs is not less than 100; HIT approval rate (%) is not less than 95; Location is one of: US
 

electrolyte

The Ghost of MTurk Past
Contributor
Joined
Jan 10, 2016
Messages
19,184
Reaction score
45,978
Points
1,313
If it exists and I don't recall... I'd like to see it. My memory is shit.

But we are also talking about new requesters... ones who don't know about using 5k and 98% or other metrics that may not have much discernible meaning to them at that stage in their usage of mturk.
There's a couple out there. I can dig more when I have more time. I can see one in my head but can't find the link to it. Here's another that mentions it, though:

"We used our two worker pools as normal AMT workers with high approval rating and AMT Master workers. We were surprised to find that Master workers are not much superior in quality than regular workers."
http://crowdml.cc/icml2015/papers/CrowdML-Paper19.pdf
 
  • Like
Reactions: jan, Blue and Jaded
Status
Not open for further replies.