About the Jake Street blocks:
Jacob Forest <jacob.william.forest@gmail.com>
11:38 PM (21 minutes ago)
to me
Dear Steve,
Thank you for taking the time to reach out to me with your concerns. I am experimenting with survey response rates based on different means of qualifying workers for multiple batches of the same survey (ie qualifications versus temporary blocks). I assure you, just as the note on the block indicated, the block is very much temporary and will be removed tonight.
To give you a slight look behind the curtain, my first batch was initially getting a very solid response rate. This tapered off to a slow trickle of responses (which is not surprising). Initially, I used a qualification so that I could launch new batches without having repeats in my data set, but these later batches never received the initial numbers that I saw on the first batch.
After this, I ran experiments on increasing pay. Rates of $0.50, $0.55, and $0.60 did not have a statistically significant impact on response rate. Today, I wanted to test the effect of using very temporary blocks instead to see if it appreciably increased response levels. After looking into Amazon's policy, my understanding is that as long as the blocks are removed in a short time span then workers would not be harmed, though on further reading it seems that the policy is murky at best. My hypothesis was that workers were not as keen to sort through their qualifications before taking a survey and that using the block method would dramatically increase response rate.
Surprisingly, it actually decreased response rate significantly. Due to this, as well as the negative feedback from a number of workers, I am ending this part of my experiment early and will be reversing all blocks tonight. You have my apologies for any stress this may have caused. I truly value the effort of turkers and my project would not be possible without you!
Regards,
Jake Street
