Rep Present Endoscopic Surgery (FESS) Skill Assessment

[Squatty]

Do you plan to add more tasks for each Target?

It seems like I only do 2 for each and then it says I have completed all tasks for that Target.

 

[LOS]

Do you plan to add more tasks for each Target?

It seems like I only do 2 for each and then it says I have completed all tasks for that Target.

That is true. For the initial phase there will be only two - one for the absolute and one for the relative.
Early next week we will release a lot more and then you will not see those messages.

 

[LOS]

BTW, some Target 9 qualifications were remaining to be filled - releasing those.

 

[LOS]

I had a question: is there a way to make sure others who want to get a qualification get a chance to do it rather than the automated accepts that some turkers have running and are not actually completing the hit?

 

[mTurk8705]

I had a question: is there a way to make sure others who want to get a qualification get a chance to do it rather than the automated accepts that some turkers have running and are not actually completing the hit?

I just attempted the Target 9 qualification. After the video I clicked for the test, which opened a window that was all white except for: "Your assignment is not in our records." I refreshed, watched the video again, and had the same result.

 

[LOS]

Here is our requester page link: as you can see it has some HITs on it. There are enough turkers with qualifications for these HITs. So you can try to knock these out
https://www.mturk.com/mturk/searchbar?selectedSearchType=hitgroups&requesterId=A2MG9E64Z06RY7

 

[Squatty]

I had a question: is there a way to make sure others who want to get a qualification get a chance to do it rather than the automated accepts that some turkers have running and are not actually completing the hit?

No there isn't. As long as someone is qualified for a hit, they can grab it with an automatic program. But if they don't complete it, it will expire and someone else can get it.

The only thing you could do would be to give special quals to people you specifically want to do the hit.

 

[laby]

You can put up a qual test instead of a qual hit. On the ones I can't do because I don't have the qual there is a link for a qual hit but instead of a test it says.

1. Please complete the HIT hosted at this link:
   QUALIFICATION HIT with the title: Qualification HIT: FESS Target 5
2. Return here and hit the submit button
3. ONLY once you have completed the HIT successfully you will be granted this qualification

Although I don't know if you can control the amount of people that can take the test so you might end up with more workers than you want.

 

[LOS]

You can put up a qual test instead of a qual hit. On the ones I can't do because I don't have the qual there is a link for a qual hit but instead of a test it says.

1. Please complete the HIT hosted at this link:
   QUALIFICATION HIT with the title: Qualification HIT: FESS Target 5
2. Return here and hit the submit button
3. ONLY once you have completed the HIT successfully you will be granted this qualification

Although I don't know if you can control the amount of people that can take the test so you might end up with more workers than you want.

That is quite the correct way to go. But unfortunately, MTurk as of now does not allow hosting external website based surveys on the qualification tests. Thus, I have to go to another HIT for qualification test and so on.

 

[mTurk8705]

How about a password

That is quite the correct way to go. But unfortunately, MTurk as of now does not allow hosting external website based surveys on the qualification tests. Thus, I have to go to another HIT for qualification test and so on.

I've seen passwords used on some HITS. Is that applicable here?

 

[LOS]

How about a password


I've seen passwords used on some HITS. Is that applicable here?

That is something we thought about. And while that is fine in certain cases. We wanted to keep a track of who responds to which question. And currently Turk does not send us information about it, until the survey/HIT is completed by the worker and they enter that password back on the MTurk.

 

[LOS]

I am quite puzzled here, I released 5 more Target 2 qualifications which people grabbed quite instantly and now I don't see anyone attempting the actual HITs - which was the whole purpose!

 

[LOS]

I am quite puzzled here, I released 5 more Target 2 qualifications which people grabbed quite instantly and now I don't see anyone attempting the actual HITs - which was the whole purpose!

Same story with Target 3!

 

[LOS]

When a PANDA is setup to accept HITs automatically as soon as they are published. Is there a way for you all to set up how mnay of such HITs should be accepted?
Example, I publish some HIT with group id: ASDFASDF and you setup the PANDA for it. Now let's say the HIT duration is 30 minutes and it takes 5 minutes to complete it on average. I would think one would want to accept 6 of these may be at one point (or may be 7 or 8) but no more.

So is there an option in the automated version. In the manual process, I assume you just hit the refresh button multiple times and you have accepted a bunch of the HITs.

Curious so that I can setup the new HITs for the whole bunch of videos in a way that qualified turkers will be able to most efficiently accept and finish them.

 

[Squatty]

When a PANDA is setup to accept HITs automatically as soon as they are published. Is there a way for you all to set up how mnay of such HITs should be accepted?
Example, I publish some HIT with group id: ASDFASDF and you setup the PANDA for it. Now let's say the HIT duration is 30 minutes and it takes 5 minutes to complete it on average. I would think one would want to accept 6 of these may be at one point (or may be 7 or 8) but no more.

So is there an option in the automated version. In the manual process, I assume you just hit the refresh button multiple times and you have accepted a bunch of the HITs.

Curious so that I can setup the new HITs for the whole bunch of videos in a way that qualified turkers will be able to most efficiently accept and finish them.

When I set up a Panda I have 2 options. The first is to accept one hit and then stop. I usually do this for surveys, that we can only do one of. The second is to keep accepting hits until I either stop it or the hits run out or my queue is full (25 hits).
Does that answer your question?

 

[LOS]

That helps. Thank you.

 

[sandyriver]

Trying to take a qualification. I see 2 HITS FESS 2 and 3, when I click on the qualification, it goes to the click on the link to take the test page , but when you click on the qualification link, it gives you this -
There are no HITs in this group available to you at the moment. (Why?)

How to take the qualification? I have to cancel 2 qualifications already because of this.

I've been doing CSATS HITS and loving it. would love to do more like this one.
thanks

 

[Squatty]

Trying to take a qualification. I see 2 HITS FESS 2 and 3, when I click on the qualification, it goes to the click on the link to take the test page , but when you click on the qualification link, it gives you this -
There are no HITs in this group available to you at the moment. (Why?)

How to take the qualification? I have to cancel 2 qualifications already because of this.

I've been doing CSATS HITS and loving it. would love to do more like this one.
thanks

They had qualification hits before. You had to pass those to get the qual.

 

[LOS]

They had qualification hits before. You had to pass those to get the qual.

@sandyriver currently we are not opening any more qualifications. We may do so if need be. We will try and make sure to announce it here.
@Squatty Thanks!

 

[sandyriver]

So I will not be able to take a qual test through mturk even though I see a Target 1 HIT with a "take a qualification test" posted right? I have to wait for the qualification test to be posted here? Just making sure as I don't want to miss out on the opportunity.