Turk Opticon

[Deleted member 481]

One of the owners of TO asked that I try to get community feedback about the Turk Opticon websites. Do you think they should force everyone to use TO2 by either closing TO1 or archiving it? The co-owner will hopefully be reading the feedback left here.

 

[InfiniteChanges]

Honestly, TV is much easier to us than TO, and more accurate as it's easier for workers to update. The only thing that TO is really useful for anymore is doing a quick check to see if the requester has recently been mass rejecting people.

 

[Serial Mom]

@<Gucci> ?

 

[A6_Foul_Out]

Spoiler: do this

Log in or register now. to view Spoiler content!

 

[electrolyte]

I don't like Turkerview. :/

For TO, I'd like site #1 to be archived but still readable. But you can't add new reviews to it. If you want to review, you have to use TO #2.

 

[ChrisTurk]

I don't like Turkerview. :/

Honest question, since between the two TV is more likely to be updated for criticisms/concerns/feedback. I don't see much that TO2 offers that improves upon the model vs TV. I could at least understand preferring TO1 to it since, y'know, nostalgia and the charm of the 1 bomb. But TO2 is just less accurate across the board and takes in 1/10th the amount of data

Is it aesthetics? You can tell me I'm ugly, I can handle it, I wont cry.. a lot.

 

[electrolyte]

Honest question, since between the two TV is more likely to be updated for criticisms/concerns/feedback. I don't see much that TO2 offers that improves upon the model vs TV. I could at least understand preferring TO1 to it since, y'know, nostalgia and the charm of the 1 bomb. But TO2 is just less accurate across the board and takes in 1/10th the amount of data

Is it aesthetics? You can tell me I'm ugly, I can handle it, I wont cry.. a lot.

It's completely aesthetics. The information, accuracy, etc. all is fine for me. I don't have any issues with that. I just don't like the way it looks. When I see the numbers hovering over a HIT on MTurkSuite, it's fine. But going to the website and looking at a page, it's confusing, takes a lot of time to absorb the information, requires a lot of scrolling to see it all, has too much going on. I don't know. It's just overwhelming to look at a requester's page. But again, I rarely do that since most of the time when I'm checking out reviews, I'm hovering over a HIT with MTS.

 

[InfiniteChanges]

Honest question, since between the two TV is more likely to be updated for criticisms/concerns/feedback. I don't see much that TO2 offers that improves upon the model vs TV. I could at least understand preferring TO1 to it since, y'know, nostalgia and the charm of the 1 bomb. But TO2 is just less accurate across the board and takes in 1/10th the amount of data

Is it aesthetics? You can tell me I'm ugly, I can handle it, I wont cry.. a lot.

Chris, you should add a rejection count to TV, up at the top so anyone coming from something like HIT Forker would see if the requester has a history of rejections.

 

[InfiniteChanges]

I don't like Turkerview. :/

For TO, I'd like site #1 to be archived but still readable. But you can't add new reviews to it. If you want to review, you have to use TO #2.

My problem with both is that they are such a pain to add reviews to. I've found, more often than not, most of the reviews are either old or only happening because people are complaining. This is on both TO1 an TO2.

 

[electrolyte]

My problem with both is that they are such a pain to add reviews to. I've found, more often than not, most of the reviews are either old or only happening because people are complaining. This is on both TO1 an TO2.

A lot are going to be old because TO #1 has been around for years. And I think ANY review site is going to have a lot of people complaining. That's just the nature of people who leave reviews on any kind of review site.

 

[InfiniteChanges]

A lot are going to be old because TO #1 has been around for years. And I think ANY review site is going to have a lot of people complaining. That's just the nature of people who leave reviews on any kind of review site.

That's kind of my problem though. I'm able to fill out a TV review on every survey I complete, and I usually do, because it takes about 20 seconds (more if I'm writing something positive or negative about the survey). I feel there are many like me that only post on TO1 or TO2 when either the requester has specifically asked and they paid well, or I'm mad because I got rejected unfairly. I don't go post on there when the pay was just not good, it takes too long. Coping and pasting this and that. That's assuming that TO is actually functioning and I don't just get loading errors.

 

[ChrisTurk]

It's completely aesthetics. The information, accuracy, etc. all is fine for me. I don't have any issues with that. I just don't like the way it looks. When I see the numbers hovering over a HIT on MTurkSuite, it's fine. But going to the website and looking at a page, it's confusing, takes a lot of time to absorb the information, requires a lot of scrolling to see it all, has too much going on. I don't know. It's just overwhelming to look at a requester's page. But again, I rarely do that since most of the time when I'm checking out reviews, I'm hovering over a HIT with MTS.

JK haha. Cool, thanks! I appreciate it, I've had a few people mention the site can be overwhelming & especially some comments on how data is filtered not being preferable for everyone on initial load from scrapers.

@Fester sorry don't mean to hijack your thread. FWIW I agree w/ @electrolyte on archiving TO1 & going "all in" on TO2.

Chris, you should add a rejection count to TV, up at the top so anyone coming from something like HIT Forker would see if the requester has a history of rejections.

Tbh there are probably less than 50 rejections reported on TV if I had to wild shot in the dark guess. Its just too difficult to report them since it only really takes "live" data and the kind of people who like to report rejections only really like reporting rejections, lol. Kinda as y'all noted above, TO is filled with that good goodness.

Yeah I went to go verify, out of 18,000+ reviews there is only 45 reported rejections on TV. Some of that is just the difference in userbase between TV & TO, but a good bit of it is the inability to retroactively / selectively report negative experiences.

 

[InfiniteChanges]

JK haha. Cool, thanks! I appreciate it, I've had a few people mention the site can be overwhelming & especially some comments on how data is filtered not being preferable for everyone on initial load from scrapers.

@Fester sorry don't mean to hijack your thread. FWIW I agree w/ @electrolyte on archiving TO1 & going "all in" on TO2.


Tbh there are probably less than 50 rejections reported on TV if I had to wild shot in the dark guess. Its just too difficult to report them since it only really takes "live" data and the kind of people who like to report rejections only really like reporting rejections, lol. Kinda as y'all noted above, TO is filled with that good goodness.

Yeah I went to go verify, out of 18,000+ reviews there is only 45 reported rejections on TV. Some of that is just the difference in userbase between TV & TO, but a good bit of it is the inability to retroactively / selectively report negative experiences.

Have you thought about adding back in the feature to update previous postings? I think that the ability to see rejections keeps people doing work for scammers, it also alerts workers to when their rejection wasn't a one off. I think I've personally had about 15 rejections (I don't get many so that's a lot for me) overturned because I wasn't alone. When 25, 50, 100+ people are writing emails to requesters, their supervising professors, or even the IRB, shit gets done. It especially really works when we get rejections because we are too good at what we do and finish 20 minute surveys in 4 minutes. Just saying.

 

[ChrisTurk]

Have you thought about adding back in the feature to update previous postings? I think that the ability to see rejections keeps people doing work for scammers, it also alerts workers to when their rejection wasn't a one off. I think I've personally had about 15 rejections (I don't get many so that's a lot for me) overturned because I wasn't alone. When 25, 50, 100+ people are writing emails to requesters, their supervising professors, or even the IRB, shit gets done. It especially really works when we get rejections because we are too good at what we do and finish 20 minute surveys in 4 minutes. Just saying.

Users can absolutely go back in and edit their review to indicate the HIT was rejected (in fact the majority of the ones reported on TV are exactly that). The thing is most of TV's users simply aren't eating a lot of survey rejections to begin with & surveys make up probably 98% of the incoming data. Past a point workers tend to begin identifying the "red flags" of a rejection prior to even submitting the HIT & bail out. TV's userbase is heavily skewed towards experienced workers, so the rejection chances are already pretty slim.

Last week for people who dropped a weekly report there was 70,000+ submitted HITs and... 14 rejections (and IIRC 8 of them are from one person). There's just not a lot of data there to report to begin with lol.

I totally agree in the power in #s thing, but TV tends to not lose that leverage because its an extension of a massive group. I've leveraged TH/TV multiple times to reach out & work w/ Requesters who have rejected work & gotten it overturned, most of the time on work I didn't even do because I don't chase rejections lol. If you ask in the daily thread any given day you'll find plenty of folks who can vouch for me being pretty good at getting rejections overturned for community members haha.

The nice thing about having TV built off the larger mturk forum communities (MTC, TH, TN, there are major TV contributors whose name I recognize from each site) is that all of that power in #s thing happens in real time on the various daily threads BUT the work isn't also forever hidden from scrapers in a sea of one-bombs after it has become safe to revisit. Most rejections seem to come from ignorance/miscommunication, not malice, but its just human nature to forget about the problem (not update a review) once its been solved.

So I think TV finds a happy balance there. && I think TO being so good at collecting large #s of negative data points is actually a good thing, as much as I crack jokes about it, for all of the reasons you mentioned above ++ offering protection to user accounts who actually need it. But for TV, by far my #1 goal is to do everything possible to not hide good work. I think the worst thing in the world is if a Requester gets an unfair negative rating because it hurts BOTH parties (Requesters, & Workers trying to maximize their hourly). If a Requester gets an undeserved positive rating, that sucks but it'll correct itself very quickly IME since - as mentioned above - people tend to report negative experiences more often than positive. But if a Requester starts off with a 1-bomb for something unfair both sides get unfairly damaged & that stinks for everyone. Finding good work on mTurk is super difficult, so I never want to contribute to that & prefer to make it easier where I can impact it.

 

[SAJ]

My problem with both is that they are such a pain to add reviews to. I've found, more often than not, most of the reviews are either old or only happening because people are complaining. This is on both TO1 an TO2.

100% agree and is reason I never reviewed on either TO it is a hassle to do. TV is a simple click and done and I can go in later if I feel the need to write something.