口碑经济的罪与罚
|
教师 然后是无所不在的教师评分，尤其是对大学教授的评分。几十年来，高等教育机构一直把对学生的问卷调查成绩作为衡量教职员工工作表现的一部分，现在大多数教育机构都要求课程结束后收集学生的评价。如果你也像我一样，相信教师的本职工作是教书，好让学生能学到并记住知识，而不是给学生讲段子、称兄道弟，那么你在衡量学生的评分时就要睁大眼睛，要根据客观衡量标准看看学生究竟学到了什么，然后再看看它与教师评分之间的关系，你才知道学生的打分是否准确。 好在给教师打分的做法已经实施了很多年，对于学生评分与学生成绩之间的关系也有大量的研究。不过坏消息是，学生的课程评价，几乎与客观标准反映的学生学习成果没有任何关系——人们了解这个事实已经40多年了。比如，一篇发表于1972年的论文研究了293名正在学习微积分课程的在校大学生，结果发现：“学生主观评分最低的教员，获得了最高的客观得分。”不过，尽管学生评分对他们的学习成果没有任何有价值的参考意义，但这还是没有影响评分在教育机构中的普遍使用。 餐馆 和找医生找老师相比，对餐馆质量和用餐体验的打分往往会客观一些，引起的后果也更少。米其林公司从1926年起，便聘请知识渊博、经验丰富的匿名专家，到全球各大城市寻找最好的用餐地点。我们可以在TripAdvisor等网站上，将米其林的评级方式与一般老百姓的评分方式进行对比。 我选择了两座城市，一座是离我住的地方很近的旧金山，另一座是我和我妻子最近刚去过的巴塞罗那。我在2015年的米其林榜单上找到了这两座城市所有获得米其林星的餐厅，然后看了它们在TripAdvisor上的评分。以下是我的发现： 巴塞罗那有21家一星或两星的米其林餐厅。按理说，这些应该是这座城市里最好的餐厅，但其中却只有一家登上了TripAdvisor的前10名，只有两家进入了前50名，只有7家进入了前100名。其中一家名叫Nectari的餐厅拥有一颗米其林星，但它在TripAdvisor上仅排在第2262名；另一家名叫Enoteca的米其林星级餐厅仅排在第1333名。 旧金山的食客兼打分员们对米其林的尊重也只是多了一点点。在旧金山的24家米其林星级餐厅中，只有一家名叫Gary Danko的餐厅在TripAdvisor上排名前10，但有6家米其林星级餐厅挤进了TripAdvisor的前50名。但作为整个旧金湾区仅有的4家二星级米其林餐厅之一的Coi，在TripAdvisor上的排名却仅为第562名。 至少对于这三个领域，可能还有许多其它领域来说，消费者的评分往往与专家意见或客观的衡量标准是不相关的。当然，正因如此，许多声誉管理公司才会做得这样成功，因为声誉是可以“管理”的（你可以从最好和最差的意义上理解这个词汇），而与实际的质量无关。 为什么评分会鼓励错误的行为 由于评分及其产生的口碑会产生经济后果，人们自然就会获得足够大的激励去操控这套体系。一种越来越普遍的方式就是雇佣“水军”来发布虚假评价（或是开发软件，不过软件是比较容易探测和预防的）。据一份研究预测，在Yelp上，有16%餐厅评价是虚假的。虚假评论往往更趋于极端，口碑较差的餐厅通常更倾向于请“水军”。IT研究机构高德纳公司2012年的一份研究估算称，有15%的网络评价都是虚假的。2013年，纽约州检察长“与19家企业达成一致，后者同意停止撰写不实评价。” 数不清的网站时不时都会冒出招聘水军的帖子（然后这些帖子又消失了），让他们写自己的好话，或是写竞争对手的坏话。消费者网购前应该充分获得各方面的评价信息，以做出明智的购买决定。保障这些评价的真实性在经济上具有重要意义。因此，亚马逊和Yelp都在积极构建各种算法以筛除不实评价，同时也在针对虚假评价的发布者采取法律行动。 |
Teachers Then there are those ubiquitous teacher ratings, particularly of college professors. For decades, higher education institutions have used student surveys as part of the faculty evaluation process, and now most places mandate end-of-course student evaluations. If, like me, you believe that the fundamental job of a teacher is to teach—to impart knowledge that students learn and retain—as contrasted, for instance, with providing entertainment or becoming students’ best friends, then it seems reasonable to measure accuracy by examining the relationship between teacher ratings and what students learn through an objective measurement. The good news is that teacher ratings have been done for a long time and there are numerous studies of the relationship between student evaluations and learning. The bad news is that student course evaluations do not have any relationship with objective measures of what students have learned—a fact that has been known for more than four decades. For instance, one paper, published in 1972, studied 293 undergraduates in a calculus course and found that, “Instructors with the lowest subjective ratings received the highest objective scores.” The fact that student ratings do not offer any valuable insight on how well students learn has not affected the prevalence and use of the ratings. Restaurants Restaurant quality and the dining experience are both more subjective and also have fewer consequences than choosing the right doctor or getting a good teacher. Michelin has, since 1926, employed anonymous, knowledgeable, experienced experts to go to cities all over the world and find the very best places to eat. We can compare how Michelin rates restaurants with the same restaurants’ ratings made by the general public on sites such as TripAdvisor. I selected two cities, San Francisco, near where I live, and Barcelona, a place my wife and I recently visited. I looked at the 2015 Michelin lists of the places that earned stars (in San Francisco, I considered only establishments located in the city itself) and also ratings on TripAdvisor. Here’s what I found. Barcelona has 21 one- or two-star Michelin restaurants. Of the Michelin-rated establishments, presumably the very best in the city, only one is in TripAdvisor’s top 10, only 2 are in the top 50, and only 7 of the 21 ranked in TripAdvisor’s top 100. Nectari, with 1 Michelin star, ranks 2,262 on TripAdvisor, and Enoteca ranks 1,333. Diners/raters in San Francisco agree with Michelin only slightly more. Of San Francisco’s 24 Michelin-starred restaurants, one, Gary Danko, is in TripAdvisor’s top 10, but 6 are in the top 50. However, Coi, one of four places in the entire Bay Area that earned two Michelin stars, ranks just 562 on TripAdvisor. At least for these three domains, and quite possibly many others, ratings by consumers—of restaurants, academic instruction, or medical services—are quite uncorrelated with either expert opinion or objective measures of performance. This fact, of course, is precisely why companies in the reputation management space can be successful—reputations can be “managed” in the best and worst sense of that term, regardless of actual quality. Why ratings encourage the wrong behaviors Because ratings, and the reputations those ratings create, have economic consequences, there are, unsurprisingly, substantial incentives to game the system. One increasingly common way of gaming the system entails hiring people (or developing software, which is fortunately easier to detect and prevent) to post inauthentic reviews. One study estimated that 16% of the restaurant reviews on Yelp were fraudulent, that fraudulent reviews were more extreme, and that restaurants with weak reputations were more likely to commit review fraud. A 2012 study by IT research firm Gartner estimated that 15% of online reviews were fake. In 2013, New York State’s attorney general “announced a deal with 19 businesses that agreed to stop writing fake reviews.” Numerous websites pop up (and then disappear) offering to hire people to write positive reviews about you and negative reviews about your competitors. Online purchasing is supposed to give customers access to informative reviews before they make a purchase decision. Maintaining the integrity of these reviews is economically important. Not surprisingly, then, both Amazon.com and Yelp have been increasingly aggressive in their attempts to build algorithms that weed out fake reviews and also to initiate legal action against their perpetrators. |
