Rep Present YooLotto

[EricYooLotto]

Hello MTurkers,

**We are new to MTurk and to this forum(so please help us modify our message if we're missing/making mistakes).

Would you be interested in performing the following task?

What: We are paying workers $2.00 to download free apps & provide a report in March.

Who we are: We are YooLotto, a lottery mobile app invested by 7-Eleven, Inc.

Why: We've launched new features(Earn Cash + Fantasy Lottery) this year with YooBux, our free virtual currency. However, we're having issues with our 3rd party advertisers who credit YooBux for downloading/starting free apps. => 20%~30% are not rewarding correctly, and we must send blacklist apps to remove from the list, which is why we need your help. These free apps on our offer list changes every minute, which require manually checking, so we've budgeted over $20,000+ this month for QA.

How: Earn 100 YooBux, provide report on which Apps/offers rewarded vs. didn't reward in 30 min in .csv file.

Important: We are not using this as a user acquisition or new accounts, you may delete our app and other free apps you've downloaded/started after completing the task+providing report. We really need help on QA to enhance our real users' user experience. In addition, to avoid collecting any personal information, please create a dummy e-mail account (ex. gmail.com) to log in to YooLotto. Then, share the e-mail address with us so that we can verify your work.

We currently have about 30 workers helping us, but need about 100 workers/hourly basis.(about 1000 per day in March)

Please let us know if something like this make sense to launch here/HIT.

Thank you,
Eric

 

[electrolyte]

Hi!

I'm a little confused. To clarify your HIT, you're asking workers to download an app, monitor it for 30 minutes, and then report back results to you? The pay per HIT is $2?

 

[EricYooLotto]

Hello Electrolyte,

Yes, workers would download YooLotto, and go through "Get YooBux" to Featured/recommended offers section to earn 100 YooBux by downloading other free apps(about 5-10) => then report which ones rewarded YooBux and which ones didn't. You can delete YooLotto and the free apps you downloaded after reporting. We then send this report to our 3rd party integration team to remove the ones that didn't reward for our user experience.

The pay per HIT is $2 = $1 from Amazon + $1 from YooLotto(paypal or dwolla)

We've tried launching on Fivver and other microworker sites, and trying to see which one would work best.

Any comments or recommendations would help!

Thank you

 

[EricYooLotto]

On Fivver, we've tried had a worker report after reaching 500 YooBux for $10.00, which took a little too long(since we need to update our list constantly to avoid users from seeing ones that do not reward on time)....which is why we're thinking of trying smaller but more frequent payout for work at reaching 100 YooBux(this gives us about 10 data points for QA).

 

[EricYooLotto]

Did I post on the right section to get feedback from workers on this question?

 

[electrolyte]

Did I post on the right section to get feedback from workers on this question?

Yes, thanks!

 

[EricYooLotto]

Okie....any thoughts? Would you perform this task?

 

[Debatesmith]

Okie....any thoughts? Would you perform this task?

Speaking for myself, I think I would be more inclined to do this if both dollars were being paid from Mturk directly. If that were the case, I would see no problem in doing this HIT.

 

[YAHU]

Okie....any thoughts? Would you perform this task?

It depends on how active we need to be on the app. I don't mind the low pay if I just hit download and wait for a while, but if this is one of those apps where I have to go in and use the other app for a minute before the points are awarded then the pay is way too low

 

[electrolyte]

What is the estimated time investment for the entire thing?

 

[EricYooLotto]

Speaking for myself, I think I would be more inclined to do this if both dollars were being paid from Mturk directly. If that were the case, I would see no problem in doing this HIT.

Okie, this feedback is helpful. We can make all the pay from MTurk directly, but the YooBux will not be redeemable in the app.

It depends on how active we need to be on the app. I don't mind the low pay if I just hit download and wait for a while, but if this is one of those apps where I have to go in and use the other app for a minute before the points are awarded then the pay is way too low

No need to use the free apps, you just need to download and then start it, and come back to YooLotto app to see if it rewards.(can start multiple at a time as wel). We're trying to make it easy for our users....so if they don't reward after these simple steps, we're removing it from our list with your help.

What is the estimated time investment for the entire thing?

Anywhere from 1 min to 5 min. Some of the free apps give 50+ YooBux, some less. The key is we also need to know how long it took you to get 100 YooBux + which ones didn't reward.

 

[Debatesmith]

Okie, this feedback is helpful. We can make all the pay from MTurk directly, but the YooBux will not be redeemable in the app.


No need to use the free apps, you just need to download and then start it, and come back to YooLotto app to see if it rewards.(can start multiple at a time as wel). We're trying to make it easy for our users....so if they don't reward after these simple steps, we're removing it from our list with your help.


Anywhere from 1 min to 5 min. Some of the free apps give 50+ YooBux, some less. The key is we also need to know how long it took you to get 100 YooBux + which ones didn't reward.

These were all good answers for me. If you do end up posting the HIT, you should make clear all of these points in the HIT window.

 

[electrolyte]

Okay, clarifying again. The steps would be:

1. Create a fake email address at Gmail or somewhere else.
2. Download the YooLotto app.
3. Register/log in to the YooLotto app using the fake email.
4. Go through the featured section and download 5-10 other apps.
5. See which ones reward and which ones don't.
6. Continue until you reach 100 "bux".
7. Submit this information including the email address.

This sounds, to me, like it would take longer than 1-5 minutes to complete.

 

[EricYooLotto]

These were all good answers for me. If you do end up posting the HIT, you should make clear all of these points in the HIT window.

Thank you for your feedback, we will circle back with our team on how to post this HIT.

Okay, clarifying again. The steps would be:

1. Create a fake email address at Gmail or somewhere else.
2. Download the YooLotto app.
3. Register/log in to the YooLotto app using the fake email.
4. Go through the featured section and download 5-10 other apps.
5. See which ones reward and which ones don't.
6. Continue until you reach 100 "bux".
7. Submit this information including the email address.

This sounds, to me, like it would take longer than 1-5 minutes to complete.

The waiting time could take upto 15-30min to see if you get rewarded; however the actual time spent for work would be in the 1-5min range.(according to one of our QA workers).
How long should we say that it would take then, @electrolyte ?

 

[electrolyte]

Thank you for your feedback, we will circle back with our team on how to post this HIT.


The waiting time could take upto 15-30min to see if you get rewarded; however the actual time spent for work would be in the 1-5min range.(according to one of our QA workers).
How long should we say that it would take then, @electrolyte ?

I would explain in the instructions that you believe it takes 1-5 minutes of active work but there may be waiting of 15-30 minutes.

 

[EricYooLotto]

I would explain in the instructions that you believe it takes 1-5 minutes of active work but there may be waiting of 15-30 minutes.

Okie, thank you for your feedback!

 

[Eris]

If I'm understanding how this task is supposed to work and what you're looking for, I would suggest splitting it up into 2 different HITs.

HIT Type #1: Strictly quality control, yes/no do these YooLotto apps work. Maybe have workers try a specific number of apps (say 5) before submitting. This would have a set payment and a consistent amount of time each HIT would take, so workers would know exactly what to expect.

HIT Type #2: Test to see how long it takes users to reach a certain amount of YooBux (so 200, 300, 500, etc.). Have a nominal base HIT amount and then award YooBux payouts via the mTurk bonus system based on the total number of points someone's able to accumulate in a set amount of time (say 10 mins). This will provide incentive to replicate real user behavior since I'm assuming most of the people who use YooLotto are trying to maximize their payouts by earning the most YooBux in the least amount of time.

I hope this makes sense, but my thinking is that I could see these HITs taking a wide variation of time to complete. If everything loads and you get lucky, maybe you can reach 100 YooBux in a couple mins. If nothing loads, errors are encountered, or the apps are mostly broken, this could take much longer. On your end, that makes it hard to do a set payment amount for each HIT since the time it takes to complete can be random, and on the worker end, that makes it difficult for us to anticipate whether or not the compensation for the HIT is worth the time involved to complete. Also, if people have to reach a certain YooBux threshold before submitting their work, I think you'll see workers focusing all their efforts on the apps that award the most YooBux while ignoring the lesser paying ones, which wouldn't be as helpful to you since (I'm assuming) you would prefer feedback on all the apps available.

 

[EricYooLotto]

If I'm understanding how this task is supposed to work and what you're looking for, I would suggest splitting it up into 2 different HITs.

HIT Type #1: Strictly quality control, yes/no do these YooLotto apps work. Maybe have workers try a specific number of apps (say 5) before submitting. This would have a set payment and a consistent amount of time each HIT would take, so workers would know exactly what to expect.

HIT Type #2: Test to see how long it takes users to reach a certain amount of YooBux (so 200, 300, 500, etc.). Have a nominal base HIT amount and then award YooBux payouts via the mTurk bonus system based on the total number of points someone's able to accumulate in a set amount of time (say 10 mins). This will provide incentive to replicate real user behavior since I'm assuming most of the people who use YooLotto are trying to maximize their payouts by earning the most YooBux in the least amount of time.

I hope this makes sense, but my thinking is that I could see these HITs taking a wide variation of time to complete. If everything loads and you get lucky, maybe you can reach 100 YooBux in a couple mins. If nothing loads, errors are encountered, or the apps are mostly broken, this could take much longer. On your end, that makes it hard to do a set payment amount for each HIT since the time it takes to complete can be random, and on the worker end, that makes it difficult for us to anticipate whether or not the compensation for the HIT is worth the time involved to complete. Also, if people have to reach a certain YooBux threshold before submitting their work, I think you'll see workers focusing all their efforts on the apps that award the most YooBux while ignoring the lesser paying ones, which wouldn't be as helpful to you since (I'm assuming) you would prefer feedback on all the apps available.

Wow, thank you, @Eris , this feedback is very helpful. Let me work with our team to see how to integrate these suggestions.

 

[EricYooLotto]

We've tried testing on MTurk, but they took down our HIT after 12 hours because a worker flagged our HIT as our task involved downloading other apps....they are stating that we are advertising for other company. (we had approved/rewarded over 10 jobs already). ===> to be clear, we are not as we need to black list the apps that do not reward for our actual users....

What do you guys suggest? I've already spoken to customer service and they've mentioned that our task is against the policy. I'm wondering if MTurk is suitable or not for our public QA testing

 

[Eris]

We've tried testing on MTurk, but they took down our HIT after 12 hours because a worker flagged our HIT as our task involved downloading other apps....they are stating that we are advertising for other company. (we had approved/rewarded over 10 jobs already). ===> to be clear, we are not as we need to black list the apps that do not reward for our actual users....

What do you guys suggest? I've already spoken to customer service and they've mentioned that our task is against the policy. I'm wondering if MTurk is suitable or not for our public QA testing

I think it would help to be clear about both what you're asking for and why. Based on the review I'm seeing for your requester account on Turkopticon (https://turkopticon.ucsd.edu/reports?id=A2F9GZCGDIYRA8), it doesn't seem like the instructions do so. Actually, based on that feedback, it sounds like the HIT is trying to get workers to farm YooBux and give over their personal information so you can cash them in for profit as opposed to a legitimate company trying to run QA testing on their own app service. I could be wrong, but I thought downloading 3rd party software was no longer against mTurk TOS? If that's the case, then the issue is probably because you're asking for email login information, which can be circumvented by giving workers' a dummy login/password instead of asking them to make an account and give over the info. You can raise the compensation too--if your HIT pays well, it's a lot less likely to sit around waiting to be flagged, it makes what you're doing seem much less like a scam, and you'll start receiving positive reviews which will enhance your credibility as a requester and attract a better pool of workers.