Rep Present Hummingbird Watch

[electrolyte]

Hello everyone!

We are working hard to have our HITs ready for the week long trial, and I'm happy to have a schedule update for you. The daily HIT schedule will go as follows (Pacific Time):

1. 6:00 am
2. 10:00 am
3. 2:00 pm
4. 4:30 pm
5. 5:00 pm
6. 5:30 pm
7. 9:00 pm

The first day of the 7 day trial is this Friday (2016-08-12).

Each HIT will consist of 3 assignments, and should take less than a minute to complete.

More information to come! Thanks,

HW

Hi! Just to clarify, you will be posting 1 HIT with 3 assignments total at each time? So at 6:00am, 3 workers will be able to do the task, and so on for each time slot?

 

[HummingbirdWatch]

That's correct. For each specified time, three workers will be able to access the task. Thanks for asking!

 

[electrolyte]

Thanks for the response! If a worker accepts a task at 6:00am, can they also do the task at other time slots, or is it a one-per-person thing?

 

[HummingbirdWatch]

Thanks for the response! If a worker accepts a task at 6:00am, can they also do the task at other time slots, or is it a one-per-person thing?

A worker will be able to access a task at each time slot. In fact we're hoping to see a few repeat workers that can provide feedback during this week.

 

[kayagee]

A worker will be able to access a task at each time slot. In fact we're hoping to see a few repeat workers that can provide feedback during this week.

I know this may be a silly question, but will your requester name on MTurk be HummingbirdWatch?

 

[HummingbirdWatch]

I'm looking forward to working with your hits. A requester that is putting in the work to get things right out of the gate and give feedback to develop their worker pool is a breath of fresh air. Good luck with this project.

Thanks Zacharias, we are excited for this trial and hope it leads to a more long term project very soon.

 

[HummingbirdWatch]

I know this may be a silly question, but will your requester name on MTurk be HummingbirdWatch?

Our requester name is Hummingbird Watch on MTurk, thanks for asking.

 

[HummingbirdWatch]

Hello everyone, here is some more information about our trial week HITs.

Recap: Trial starts tomorrow (Friday) and will go until Thursday.

HITs will be posted at the same time everyday, including the weekend at 6:00 am, 10:00 am, 2:00 pm, 4:30 pm, 5:00 pm, 5:30 pm, 9:00 pm.

Each HIT will consist of 3 assignments. This means that three workers will have access to the same HIT at each time slot. Workers can access as many time slots as they like.

Qualifications:

We decided not to create a custom qualification for this week, instead we are using MTurk’s qualifications:

1. HIT Approval Rate (%) for all Requesters' HITs greater than or equal to 95.

2. Number of HITs Approved greater than or equal to 10000.

3. Location is one of AU, CA, NZ, GB, US.

We will be monitoring the performance of repeat workers, and may decide to offer a custom Hummingbird Watch Qualification to those who submit quality work consistently.

Wage/Bonuses:

We are offering $0.14 per HIT, with a bonus of $0.01 to make a total of $0.15 per HIT. This approach allows us to provide feedback to workers. We will be allotting 2 minutes for the HIT, but it really shouldn’t take more than 1 minute.

What exactly will we be doing?

Inspecting 10-12 images in a column, then clicking the image/check-box if it contains the specified object.

Providing Feedback

We will be monitoring our threads at the following links, please feel free to offer feedback on our HITs:

http://turkernation.com/showthread.php?27594-Feedback-for-the-Requester&p=445469#post445469

http://www.mturkgrind.com/threads/hummingbird-watch-week-long-trial.29977/

http://www.mturkcrowd.com/threads/hummingbird-watch.1553/


We are very excited for this trial, and hope that it leads to a long term project! Good luck and have fun with our HITs .


HW

 

[HummingbirdWatch]

Morning everyone, our first HIT has been published! It's called:
Hummingbird Watch Day 1 Batch 1

UPDATE:

Sorry everyone, there was an issue with payment. It is being processed now and will be uploaded ASAP.

UPDATE:

Payment issue resolved, the HIT is published! Thank you for your patience!

 

[HummingbirdWatch]

Hummingbird Watch Day 1 Batch 2 has been published! The HIT format has been updated: 5 assignments, 5 minutes and an example image are provided.

 

[Squatty]

Hello, I just tried one of your Hits and found it quite enjoyable. The payment is very fair. The only complaint I would have is that the pictures are quite large. I have to scroll a lot on the screen in order to see them all.

 

[HummingbirdWatch]

Hello, I just tried one of your Hits and found it quite enjoyable. The payment is very fair. The only complaint I would have is that the pictures are quite large. I have to scroll a lot on the screen in order to see them all.

Glad you enjoyed it @Squatty. Thanks for the feedback, we're collecting all feedback during this trial week to improve our HIT delivery for the next test week.

 

[HummingbirdWatch]

"Hummingbird Watch Day 1 Batch 3" has been published.

 

[humbleturker]

I got to get used to batch in this case being a series of test HITs. I keep trying to grab them but missing.

 

[HummingbirdWatch]

I got to get used to batch in this case being a series of test HITs. I keep trying to grab them but missing.

Thanks for your interest in our HITs @humbleturker. There will be tests all week until Thursday night. After that we're going to do another test week, perhaps with a larger batch of HITs. If you're interested in receiving a Qualification to complete our HITs stay tuned, we want to have one ready for the next test week. We are interested in quality workers who are interested in accepting our hits regularly.

 

[electrolyte]

Hello, I just tried one of your Hits and found it quite enjoyable. The payment is very fair. The only complaint I would have is that the pictures are quite large. I have to scroll a lot on the screen in order to see them all.

I just tried a HIT, too, and had this same response. I also tried to click on some of the images to make them larger as the instructions suggested, but they didn't get larger. The page just jumped up to the top when I clicked on an image.

It would also be helpful to have a few links at the top to examples of what images with fire or smoke looks like so I have a better idea of what I'm looking for.

Otherwise, looks great! I'd enjoy doing a batch of these.

 

[clickhappier]

A worker will be able to access a task at each time slot. In fact we're hoping to see a few repeat workers that can provide feedback during this week.

Good luck with that since it's so hard to get even one of these. Between the unnecessarily tiny timer, different HIT group every time, and so few assignments each time... frustrating.

Finally caught one and the dang thing expired before I could get all the images to load. It doesn't matter how long the task 'should' take, I prefer HITs to have at least an hour available to reduce the chance of losing things to expirations if everything doesn't go perfectly. Never know when there'll be a personal interruption, internet slowdown, power outage, etc.

 

[HummingbirdWatch]

Good luck with that since it's so hard to get even one of these. Between tiny timer, different HIT group every time, and so few assignments each time... frustrating.

Thanks for the feedback @clickhappier. Our project relies on very quick response time, within a few minutes from HIT creation to worker submission. This test week is on a small scale to test a couple of variables (speed, accuracy, getting to know the MTurk service, etc).

If we go forward with a long term project, it will be on a much larger scale with many more assignments than the current test. Also, there would be more HITs throughout the day (perhaps every hour or more).

Early results are promising, but more tweaking needs to be done on our part. If you have any ideas on how we could improve we'd be very thankful. For example, if there's a better delivery method than posting a different HIT group every time.

To recap, here's what we'd need for a longterm project with MTurk to be successful:

1. HITs to be accepted and submitted within a few minutes of being posted.
2. High accuracy for submissions.
3. HITs to be published every hour, sometimes more.

All feedback that has been provided will be included in a report that I present to my employer, and these changes will be applied to the next tests. We need to know how we can provide the mechanical turk workers with the tools to meet our needs and expectations. Keep the feedback coming, and see the future HITs become more worker friendly and enjoyable .

HW

EDIT: I forgot to mention that we are interested in collecting a list of workers who are interested in becoming long term workers for our HITs. We will require a Qualification to work on our HITs in the future, and are interested in developing and training those who would like priority access to our HITs.

 

[HummingbirdWatch]

I just tried a HIT, too, and had this same response. I also tried to click on some of the images to make them larger as the instructions suggested, but they didn't get larger. The page just jumped up to the top when I clicked on an image.

It would also be helpful to have a few links at the top to examples of what images with fire or smoke looks like so I have a better idea of what I'm looking for.

Otherwise, looks great! I'd enjoy doing a batch of these.

Thank you @electrolyte. We overlooked the importance of having examples early on, and this will be one of the major changes we apply to the next test. We decided not to reject any submissions for this test to compensate for this oversight. Instead, we're accepting all submissions in the hopes that workers like yourself will feel encouraged to reach out to us and help us improve. Thanks for taking the time!

HW

 

[Metallica]

never mind, I see my question was somewhat answered already.