Post by High Priestess on Sept 20, 2015 8:35:01 GMT
Ainhoa shared on New HOsts forum 6 months ago
My first bad experience with the star system
I have been following all your posts about the star system and how frustrating it can be. I have experienced it now, although I'm not discouraged by it but it does not truly reflect the guest experience. My last guest has given me publicly a positive review and in his private message has said that the pictures in the listing do not reflect the reality....because the apartment is even better than what is reflected there!. So in the star system he has rated the accuracy category with 2 starts which brought my total rating to 4,5 stars. With his rating he meant that the apartment was even better than pictured!! Ugh. Well in the meantime I have had the professional photographer taken photos of the apartment and they will be live in a couple of weeks...
9 comments•3 likes
Following
Like
Denise
Raymond & Elaine
Julie and Eric
Delete
Hide
Rebecca
Rebecca6 months ago
That's crazy, Ainhoa. He can't be very smart to have done that, though. Surely people get the notion of rating positively for things that are actually good??
I just had a guy give me 4 for Location when he specifically booked with me because my place is so close to the venue where he was attending a gig. Sigh.
Reply Like 1 reply Delete
Rebecca
Rebecca6 months ago
Having said that, I kind of get it. Some people just view things very literally.
Nathalie et Claude
Nathalie et Claude6 months ago
That's the best of the day. In the name of the communauty, could you mention it to airbnb to show them the star system has some glitches.
Reply Like 1 like Delete
Deborah
Deborah6 months ago
This is a great example of the serious flaws in the star rating system.
Reply Like Delete
Rebecca
Rebecca6 months ago
Much as I appreciate Ainhoa's frustration over this, I do think it is a rare anomaly and shouldn't be cited as an example of a flawed system. Presumably this guest would have done the same on any other customer-based review site. You really do need to be quite obtuse, aka thick as sh*t, to give a low star rating for something that is good. The real problem is the faulty review guidelines, the ridiculous expectation that everyone should always have 5 stars, and the way that the overall average is calculated. That last point is the crucial one, I think. Random outliers should be discarded from the calculation but they're clearly not.
Reply Like Delete
Deborah
Deborah6 months ago
THe problem with the star ratings system as it is though, is that outliers and anomalies are not being discarded, they are being treated by Airbnb as if they are just as believable as all the other ratings. And also one has to consider -- what happens if you have two "outlier" ratings in a row? Or three?
So the flaw in the system is not that someone rates you low because you exceeded their expectations, but rather that there is no way on the face of it for Airbnb or anyone to look at the star ratings they gave you, and know the backward logic they used to arrive at those, or to have any way of separating out 'Outliers" from more accurate ratings.
Reply Like Delete
Rebecca
Rebecca6 months ago
The thing is though that you need high numbers to start discarding outliers. Three reviews is WAY too low a number to start posting ratings publicly, imo. I've seen new hosts completely destroyed by one asshole guest because of this policy. I'm not a statistician but I reckon you'd need well over 100 reviews before outliers can be discarded safely. So if you get under 3* a few times in a row, it's legitimate time to be concerned. No matter how many reviews got, actually!
Reply Like 2 likes Delete
Deborah
Deborah6 months ago
See my comment that I wrote in the above post, -- I'll repost part of that comment here:
So one of my gripes with the star system is that I don't feel anyone should get scolding messages over delivering an average or better experience to a guest. And for those cases which the guest deems "below average" -- Airbnb should require a guest to provide an explanation of why they have 1 or 2 stars, and if they don't have a very good explanation, this low rating should be scrapped.
If you have ever tried to rate someone 1 or 2 stars on Amazon, from what I recall, the system actually rejects your attempt to do this unless you first have contacted the seller you bought the product from. Which also makes more sense than just taking 1 or 2 star ratings at face value.
So I say, Airbnb should require all guests who try to give a 1 or 2 star rating, to provide a valid explanation for that. THis should be read along with the review they wrote. IF something doesn't add up, the low star ratings should be removed.
Essentially, if Airbnb required guests who were leaving 1 or 2 star ratings to justify those ratings with a verbal explanation, then the method of statistical analysis and removal of outliers via statistics would be moot. One could simply remove or not post low ratings that weren't accompanied by a logical and believable explanation. In other words, if the guest gave a host 2 stars, saying, "accuracy was 2 stars because the place was better than I expected" , the Airbnb staffer reading this would not post the 2 stars. Or if the guest wrote, "value was 2 stars because I got a parking ticket that cost me more than one day's stay at the place", ditto, this rating would be not posted because the explanation didn't justify the rating. A whole variety of vindictive star ratings could be avoided this way, though baldfaced lying wouldn't be eliminated, I suspect that guests would be less likely to tell baldfaced lies than to simply do vindictive star ratings using irrational justifications.
Reply Like 1 like Delete
C C
C C6 months ago
Funny, for the stars we give them, even though Andrew says they don't matter, there have only been 2 or 3 times I've given anyone below 5 stars on anything. The narrative is much more important, and just rating adults with stars is kind of silly to me. I miss the days when you could just live your life & you didn't have to fill out a survey every time you walk in a store or talk to your banker, etc. I do think its interesting how when Airbnb asks at the end of the review prompts how we much we would recommend them, the scale is 1 to 10. Guess they don't want to get a 4.
Reply Like Delete
Nathalie et Claude
Nathalie et Claude6 months ago
The stars system do not bother me as i think, globally, it reflects the reality. What bother me are the comments written by airbnb on our board. They do not know the guest, how were things and do not take into consideration the seriousness of the host.
Reply Like Delete
Deborah
--------------------------------------------------------------------------------------------------------------------------------------
older version w/o commenter names
My first bad experience with the star system
I have been following all your posts about the star system and how frustrating it can be. I have experienced it now, although I'm not discouraged by it but it does not truly reflect the guest experience. My last guest has given me publicly a positive review and in his private message has said that the pictures in the listing do not reflect the reality....because the apartment is even better than what is reflected there!. So in the star system he has rated the accuracy category with 2 starts which brought my total rating to 4,5 stars. With his rating he meant that the apartment was even better than pictured!! Ugh. Well in the meantime I have had the professional photographer taken photos of the apartment and they will be live in a couple of weeks...
Reply 1:
That's crazy. He can't be very smart to have done that, though. Surely people get the notion of rating positively for things that are actually good??
I just had a guy give me 4 for Location when he specifically booked with me because my place is so close to the venue where he was attending a gig. Sigh.
Reply 2:
Having said that, I kind of get it. Some people just view things very literally.
Reply 3:
That's the best of the day. In the name of the communauty, could you mention it to airbnb to show them the star system has some glitches.
Reply 4:
This is a great example of the serious flaws in the star rating system.
Reply 5:
Much as I appreciate the frustration over this, I do think it is a rare anomaly and shouldn't be cited as an example of a flawed system. Presumably this guest would have done the same on any other customer-based review site. You really do need to be quite obtuse, aka thick as sh*t, to give a low star rating for something that is good. The real problem is the faulty review guidelines, the ridiculous expectation that everyone should always have 5 stars, and the way that the overall average is calculated. That last point is the crucial one, I think. Random outliers should be discarded from the calculation but they're clearly not.
Reply 6:
THe problem with the star ratings system as it is though, is that outliers and anomalies are not being discarded, they are being treated by Airbnb as if they are just as believable as all the other ratings. And also one has to consider -- what happens if you have two "outlier" ratings in a row? Or three?
So the flaw in the system is not that someone rates you low because you exceeded their expectations, but rather that there is no way on the face of it for Airbnb or anyone to look at the star ratings they gave you, and know the backward logic they used to arrive at those, or to have any way of separating out 'Outliers" from more accurate ratings.
Reply 7:
The thing is though that you need high numbers to start discarding outliers. Three reviews is WAY too low a number to start posting ratings publicly, imo. I've seen new hosts completely destroyed by one asshole guest because of this policy. I'm not a statistician but I reckon you'd need well over 100 reviews before outliers can be discarded safely. So if you get under 3* a few times in a row, it's legitimate time to be concerned. No matter how many reviews got, actually!
Reply 8:
See my comment that I wrote in the above post, -- I'll repost part of that comment here:
So one of my gripes with the star system is that I don't feel anyone should get scolding messages over delivering an average or better experience to a guest. And for those cases which the guest deems "below average" -- Airbnb should require a guest to provide an explanation of why they have 1 or 2 stars, and if they don't have a very good explanation, this low rating should be scrapped.
If you have ever tried to rate someone 1 or 2 stars on Amazon, from what I recall, the system actually rejects your attempt to do this unless you first have contacted the seller you bought the product from. Which also makes more sense than just taking 1 or 2 star ratings at face value.
So I say, Airbnb should require all guests who try to give a 1 or 2 star rating, to provide a valid explanation for that. THis should be read along with the review they wrote. IF something doesn't add up, the low star ratings should be removed.
Essentially, if Airbnb required guests who were leaving 1 or 2 star ratings to justify those ratings with a verbal explanation, then the method of statistical analysis and removal of outliers via statistics would be moot. One could simply remove or not post low ratings that weren't accompanied by a logical and believable explanation. In other words, if the guest gave a host 2 stars, saying, "accuracy was 2 stars because the place was better than I expected" , the Airbnb staffer reading this would not post the 2 stars. Or if the guest wrote, "value was 2 stars because I got a parking ticket that cost me more than one day's stay at the place", ditto, this rating would be not posted because the explanation didn't justify the rating. A whole variety of vindictive star ratings could be avoided this way, though baldfaced lying wouldn't be eliminated, I suspect that guests would be less likely to tell baldfaced lies than to simply do vindictive star ratings using irrational justifications.
Reply 9:
Funny, for the stars we give them, even though Andrew says they don't matter, there have only been 2 or 3 times I've given anyone below 5 stars on anything. The narrative is much more important, and just rating adults with stars is kind of silly to me. I miss the days when you could just live your life & you didn't have to fill out a survey every time you walk in a store or talk to your banker, etc. I do think its interesting how when Airbnb asks at the end of the review prompts how we much we would recommend them, the scale is 1 to 10. Guess they don't want to get a 4.
Reply 10:
The stars system do not bother me as i think, globally, it reflects the reality. What bother me are the comments written by airbnb on our board. They do not know the guest, how were things and do not take into consideration the seriousness of the host.