Rep Present New Requester Veridium ID - HIT Posted

[Veridium ID]

Veridium ID Data Collection - mturk project

Description: Workers need to interact with Veridium ID application by using their mobile phones by doing an authentication, using mock up servers. During the interaction, data is collected using only mobile sensors e.g. accelerometer.
Keywords: data collection, mobile phone
Number of Assignments per HIT: 5
Reward per Assignment: $0.10
Number of HITS: 200

Average Time per Assignment: 35 seconds

 

[electrolyte]

Title: Data Collection using mobile phone and a custom application | AcceptA Requester: Veridium ID [A22LGXWTD0S4ZK] Contact TV: No Reviews TO: No Reviews TO2: No Reviews Reward: $0.10 Duration: 5 minutes Available: 200 Description: Workers need to interact with Veridium ID application by using their mobile phones by doing an authentication, using mock up servers. During the interaction, data is collected using only mobile sensors e.g. accelerometer. Requirements: None

 

[electrolyte]

Reading through your instructions, it says: "If you decide to quit before completing all 200 HITs, you will not receive any partial payments. In this case, the collected data will be discarded."

Does this mean if someone isn't able to get 200 HITs, then you will reject the HITs that they have already completed?

 

[Veridium ID]

Reading through your instructions, it says: "If you decide to quit before completing all 200 HITs, you will not receive any partial payments. In this case, the collected data will be discarded."

Does this mean if someone isn't able to get 200 HITs, then you will reject the HITs that they have already completed?

If workers do more than 180 HITs, then they will get payed. The experiment needs a minimum number of HITs from a worker to be able to do the research project thus we put as mandatory request so that workers try to get to the end of the HITs.

 

[electrolyte]

If workers do more than 180 HITs, then they will get payed. The experiment needs a minimum number of HITs from a worker to be able to do the research project thus we put as mandatory request so that workers try to get to the end of the HITs.

Hmm, that could be a problem. Workers can't "reserve" 180 HITs for themselves, so it's quite likely that by the time one worker gets around to doing a large number, there could just not be that many HITs left to do because they've already been taken by other workers.

 

[Veridium ID]

Hmm, that could be a problem. Workers can't "reserve" 180 HITs for themselves, so it's quite likely that by the time one worker gets around to doing a large number, there could just not be that many HITs left to do because they've already been taken by other workers.

We have developped a mechanism which counts the number of workers and also we track how many sessions are done for each worker. So, by adding their worker id, we reserve the 200 hits for that worker.

 

[largeeyes]

So, someone does 179 and then you give them 179 rejections? That can destroy a worker's account. I'm no engineer(actually, I am), but how can 180-200 be such a specifically significant sample size?

 

[Veridium ID]

So, someone does 179 and then you give them 179 rejections? That can destroy a worker's account. I'm no engineer(actually, I am), but how can 180-200 be such a specifically significant sample size?

If there are 179 HITs we will pay the worker, it is an extreme example. We want to emphasize the fact that a worker should do all 200 to be relevant for us. From what we see as comments, I think the bonus approach is better.

The data sample acquired will be used for a machine learning flow and through the MTurk platform we gather more than the required samples, just to be sure that we reach the required sample size. The extra sample size is used for testing and also to simulate some real life scenarios.

 

[largeeyes]

If there are 179 HITs we will pay the worker, it is an extreme example. We want to emphasize the fact that a worker should do all 200 to be relevant for us. From what we see as comments, I think the bonus approach is better.

The data sample acquired will be used for a machine learning flow and through the MTurk platform we gather more than the required samples, just to be sure that we reach the required sample size. The extra sample size is used for testing and also to simulate some real life scenarios.

The bonus system is absolutely better. This way you can still pay if they don't complete them all, but also provide great pay if they do.

 

[Veridium ID]

Title: Veridium ID Data Collection
Description: Workers need to interact with Veridium ID application by using their mobile phones by doing an authentication, using mock up servers. During the interaction, data is collected using only mobile sensors e.g. accelerometer.
Keywords: data collection, mobile phone
Qualification Requirement:

Number of Assignments per HIT: 1
Reward per Assignment: $15.00
Assignment duration: 2 hours
Approximate duration: 1 and 1/4 hours
Auto Approval Delay: 7 days

OBSERVATIONS:
1. Interface modified for a better user experience.
2. Hit mechanism modified - Submit button is available only after 200 Authentications, thus we reserve the worker the HIT he is doing.
3. Worker duplicate - by design, a worker can do only 200 Authentications. If he accepts another HIT, he will not be able to do another 200 Authentications. This way, we assure the uniqueness of the workers.
4. HIT Time - from our experiments, the HIT which consists in 200 Authentications, lasts approximately 1 hour and 15 minutes. We have set it 2 hours for convenience.

 

[Veridium ID]

Title: Veridium ID Data Collection
Description: Workers need to interact with Veridium ID application by using their mobile phones by doing an authentication, using mock up servers. During the interaction, data is collected using only mobile sensors e.g. accelerometer.
Keywords: data collection, mobile phone
Qualification Requirement:

Number of Assignments per HIT: 1
Reward per Assignment: $18.00
Assignment duration: 3 hours
Approximate duration: 2
Auto Approval Delay: 7 days

 

[A6_Foul_Out]

Title: Data Collection using mobile phone and a custom application | Accept Requester: Veridium ID [A22LGXWTD0S4ZK] Contact TV: No Reviews TO: No Reviews TO2: No Reviews Reward: $18.00 Duration: 3 hours Available: 1 Description: Workers need to interact with Veridium ID application by using their mobile phones by doing an authentication, using mock up servers. During the interaction, data is collected using only mobile sensors e.g. accelerometer. Requirements: None [tr][td] HIT exported from Mturk Suite v2.2.3​

[/td][/tr]

[tr][td][/td][/tr]

 

[DareAngel3]

Title: Veridium ID Data Collection
Description: Workers need to interact with Veridium ID application by using their mobile phones by doing an authentication, using mock up servers. During the interaction, data is collected using only mobile sensors e.g. accelerometer.
Keywords: data collection, mobile phone
Qualification Requirement:

Number of Assignments per HIT: 1
Reward per Assignment: $18.00
Assignment duration: 3 hours
Approximate duration: 2
Auto Approval Delay: 7 days

V @Veridium ID Hello! I just completed this HIT and had some polite constructive criticism. I tried contacting you via mturk, a response to the registration email, and via the email you provided in the HIT for support, but I never got a response.
My HIT started glitching about halfway through and would not progress. I finally figured out that I could restart the iframe/Mturk window to make it continue, but it continued to glitch throughout the experiment. I struggled to finish within the 3 hours. I would have been really frustrated if I could not submit after putting 2+ hours in, due to a glitch. The way the HIT is designed, I couldn't submit even if my timer ran out at 249/250 completed.
Also, the camera seemed to be completely random as to when it would fail or complete. Multiple times I took seemingly identical images, and some were accepted and others were not. I'm assuming this is what you're testing. However, you may consider increasing the timer so that it allots for both technical errors and "failed" captures, and gives enough time for workers to contact you and for you to respond.
Thanks, and I hope you get some really useful data!

 

[Veridium ID]

V @Veridium ID Hello! I just completed this HIT and had some polite constructive criticism. I tried contacting you via mturk, a response to the registration email, and via the email you provided in the HIT for support, but I never got a response.
My HIT started glitching about halfway through and would not progress. I finally figured out that I could restart the iframe/Mturk window to make it continue, but it continued to glitch throughout the experiment. I struggled to finish within the 3 hours. I would have been really frustrated if I could not submit after putting 2+ hours in, due to a glitch. The way the HIT is designed, I couldn't submit even if my timer ran out at 249/250 completed.
Also, the camera seemed to be completely random as to when it would fail or complete. Multiple times I took seemingly identical images, and some were accepted and others were not. I'm assuming this is what you're testing. However, you may consider increasing the timer so that it allots for both technical errors and "failed" captures, and gives enough time for workers to contact you and for you to respond.
Thanks, and I hope you get some really useful data!

Hi DareAngel3,

Thank you for your feedback regarding the application and also for your effort for finishing the HIT. We have received other complaints for workers in this trial and we will start investigating the issues for the next run. This is the reason for the delay of a reply.

As a information for all workers, even if you finish only 150 Auth. from 250 Auth., you can mail us and we can reach an agreement of how to proceed further. You can get payed as per number of authentications done, until then, or receive a bonus compensation if you do the rest of the authentatications. In this scenario, we create a dedicated HIT using your WorkerId. We consider that every worker should be paid, especially when they really try to finish the HIT posted by us.

As per feedback, we will discuss with the development team and see what went wrong. From our validation process, we did not have a lot of problems, but it is a controlled environment versus real life environment. We hope that for our next trial all workers will have an improved user experience and a more stable application release.

The last request that we have, can you give us a private e-mail with the model of the phone?

 

[DareAngel3]

Hi DareAngel3,

Thank you for your feedback regarding the application and also for your effort for finishing the HIT. We have received other complaints for workers in this trial and we will start investigating the issues for the next run. This is the reason for the delay of a reply.

As a information for all workers, even if you finish only 150 Auth. from 250 Auth., you can mail us and we can reach an agreement of how to proceed further. You can get payed as per number of authentications done, until then, or receive a bonus compensation if you do the rest of the authentatications. In this scenario, we create a dedicated HIT using your WorkerId. We consider that every worker should be paid, especially when they really try to finish the HIT posted by us.

As per feedback, we will discuss with the development team and see what went wrong. From our validation process, we did not have a lot of problems, but it is a controlled environment versus real life environment. We hope that for our next trial all workers will have an improved user experience and a more stable application release.

The last request that we have, can you give us a private e-mail with the model of the phone?

I sent a message to your account here on the forum with my model number. Thanks for your response!

 

[Drex]

EDIT: for some reason it did not quote him

 

[Drex]

Hi DareAngel3,

Thank you for your feedback regarding the application and also for your effort for finishing the HIT. We have received other complaints for workers in this trial and we will start investigating the issues for the next run. This is the reason for the delay of a reply.

As a information for all workers, even if you finish only 150 Auth. from 250 Auth., you can mail us and we can reach an agreement of how to proceed further. You can get payed as per number of authentications done, until then, or receive a bonus compensation if you do the rest of the authentatications. In this scenario, we create a dedicated HIT using your WorkerId. We consider that every worker should be paid, especially when they really try to finish the HIT posted by us.

As per feedback, we will discuss with the development team and see what went wrong. From our validation process, we did not have a lot of problems, but it is a controlled environment versus real life environment. We hope that for our next trial all workers will have an improved user experience and a more stable application release.

The last request that we have, can you give us a private e-mail with the model of the phone?

Where are we supposed to send the email to, because I sent a few yesterday and have yet to receive a response. I used the email that was listed in the hit. My problem was that it got to 202 sessions and would not advance past 202 no matter how many additional sessions that I completed. I sent an email as soon as I encountered the problem which occurred at about 2 hours and 10 minutes into the hit. I sent another a few minutes later with a screenshot of the problem and then sent one about 40 minutes later when the hit was about to expire. I had enough time to finish the hit, but it would not let me advance. I have no problem completing the additional sessions, but you have to understand how frustrating working on a hit for 2+ hours and it all of the sudden just stops working is for a worker and then to not get any responses to your emails and having no idea if you are going to be paid for your work is.

 

[InfiniteChanges]

Just wondering, will you be opening up HITs to workers who have already completed on of these tasks for you?

 

[Drex]

I have currently sent 7 emails and have yet to receive a response. I sent three the day it happened including a screen shot, I sent one last Friday and another this past Monday all to the email listed in the hit. I have sent one today to a company email that another Turker who did get paid said they replied from and I sent another to the Turk address associated with the account. I did not expect replies to the ones that I sent today yet, but in case the requester posts tomorrow I want them to see this. I know that people got emails and hits setup yesterday for compensation, I was not one of them, I cant even get them to respond to an email.

 

[electrolyte]

Tagging V @Veridium ID to come hopefully respond to the questions in this thread. (Perhaps they're not receiving notifications about thread replies.)