To assign this qualification, we came up with a rough heuristic that we felt ensured that we chose annotators who producing the work most closely aligned with what we need based on their performance on previous batches. It was not easy to do with our data, so it’s entirely possible we missed some good annotators in the process – the score can confirm that someone is a good annotator, but doesn’t necessarily predict poor performance.
Unfortunately, your score isn’t high enough for us to confirm that the work you’re producing is what we’re looking for, which is what we really need for those particular HITs. Please feel free to continue working on our other HITs. In general, what we’re looking for are annotators who record every time the subject leaves and enters the video; please be sure to fully watch every video.
We’re actively working on better ways of evaluating annotator performance and providing feedback. We hope to have this implemented soon, at which point we should be able to provide more specific feedback to those interested.