Does Anyone Win the Ratings Game?

The Problem With Averaging Star RatingsIt seems everyone has an opinion about agent ratings.There are those that advocate for agent ratings, those that fear agent ratings, and those that see them as inevitable. Everyone that is aware of the phenomenon has an opinion or finds one very quickly.

Agents that don’t have experience with them are concerned about what it might mean to their public image.

Agents that have good experiences with them like them and think everyone else should like them too.Some places, like the Houston Association of REALTORS have actually institutionalized them and made them part of their business culture. Others try to encourage their clients to recommend them on sites like YELP, so that they can be easily found by consumers .

Basically advocates seem to fall into four major categories;

  1. Consumers that think that ratings will be an easy way to find quality representation for their real estate needs (it’s really not for a variety of reasons)
  2. Non-agents in the real estate space who think that the use of online ratings is ties somehow to customer service, creating raving fans, inbound marketing, trends of the future, and transparency
  3. Agents who encourage reviews as part of their culture (HAR again)
  4. Agents that want to establish a differentiation, be found on the web in a (hopefully) positive manner and believe that they can encourage the majority of people to write positive things about them through their own good efforts or through the use of a survey platform like RealSatisfied.

I know that this is not a popular position to take online, where the wisdom of the crowd is that ratings are wonderful, and all things sparkly and new should be embraced, so before we move on, I want to make it plain that I don’t dislike agent ratings platforms, I believe in inbound marketing and creating raving fans, and I do think that ratings systems will continue to evolve and be part of online culture as it evolves. But I think that agent rating systems as they currently exist are substantially flawed, and that we are headed towards even bigger problems as entities outside the industry attempt to quantify what makes an agent suitable for a consumer to work with.

Averaging Star Ratings Works Better for Books and Movies Than for Real Estate

The cartoon from XKCD pretty much illustrates the point. A digest of ratings quantified by stars might be indicative of the way people feel about the product or service while not providing any indication of how well that product or service actually serves the potential consumer’s needs (more on that later when we talk about Homelight) . With all of the moving parts in a real estate transaction, getting valid ratings is a little more complex than the  “I liked it” or “I didn’t like it” types of ratings that are obviously subjective – there are people who probably didn’t “Citizen Kane” arguably one of the best movies ever made, but its the quality of the movie that makes it a topic for film students to this day. Still an “I didn’t like it” rating from an audience member is very valid – a rating that somehow indicated that it was not well written or well made would be just wrong.

Agents Are Often Blamed for Things Outside Their Control

I don’t want to make excuses for poor or incompetent agents, but frequently, the agent ends up with blame for the actions of others, or for events that were beyond their control or their ability to foretell. The old adage about not shooting the messenger should probably be part of the instructions buyers and sellers receive before they enter a real estate transaction   If the buyer and seller are contentious,or the mortgage company declines a loan, or asks for unreasonable actions on the part of the buyer, or there are any of a dozen other stress points, the consumer is likely to blame the “messenger” creating a rating which doesn’t reflect the facts of the situation.

There Should be Some Criteria for making a Rating

I think that rating systems are also flawed when the ability to rate real estate professionals is not tied to actually having used their service. While there is certainly a strong argument for a consumer rating the services of a real estate professional after using their services, that argument fails when an individual rates the services of someone they have not used, making their decision suspect since they really have less basis for making judgement.

The Rush to Ratings Has Created a Problem

Google Ventures’ portfolio company HomeLight, is a startup that’s all about buying and selling real estate. The business model is to make recommendations to consumers based on the data they’ve found on local real estate agents. An article on Venturebeat  explains it this way ” t… it helps you find the real estate agent — the best one for your price point and neighborhood and experience level — and agents only pay for the service if they end up with a new client out of the process.” In the terms of service (if you read them) agents and their firms  agree to pay 25% of the commission earned  for a referral when sold. (Is it just me, or is it odd that the beneficiary of the referral is purportedly the consumer, yet the cost is borne by the agent? – anyway I digress..)

An accuracy problem arises because data, by itself  is often misleading, and in automated rating systems, the error is even greater. I used Homelight in my market to see who was recommended to work with Buyers or Sellers based on the questions asked by the site. The questions, seem to walk you through a process of of elimination to find the agent who would best suit your needs. But the results wouldn’t benefit the average consumer.

Three of the four choices were REO brokers who do not list properties for individual sellers, and if they did have marketing programs designed more for their specialty than for the average home. The fourth choice was a broker who specializes in a smaller market in the center of the city, and while he might list any property brought to him from this site, . the consumer could not be sure that they would be working with him or one of his team members. Indeed,  if the property were a less expensive property, they might well receive less rather than more service from him specifically due to the nature of his  team organization.

The Danger of Apparent Rationality

The choices the consumer makes would seem to be winnowing down the choices of agents, finding agents who will “spend more time” with them, or with greater “experience” or who “sell homes faster” . Though the last is statistical, the other two are subjective, and might be weighted by the agent claims in their profile on Homelight, but certainly are less reliable than some objective data (though what that data might be is beyond me). SO it seems to me that the site seems to be doing more qualifying of the agents than it actually is.

Whatever the site is not, it seems clear that it is another site competing with real estate professionals for the eyes of home buyers and sellers. This new site is positioning itself as a beneficiary to the consumer and the real estate professional alike. Like Zillow and Trulia, the site is positioning itself as a trusted source for real estate information, appearing to be tool for the consumer while it is actually yet another intermediary between the consumer and the real estate professional adding a layer of cost to the real estate transaction. And, like Zillow and Trulia, the site will represent itself as a benefit to the real estate agent, providing them with consumer contacts for a fee, consumer contacts that the real estate industry would be able to obtain for themselves if the site didn’t exist.

Will This be the Birth of a New Frenemy ?

As the conversation about syndication has heated up over the past year or two, agents and firms have engaged in some passionate conversation and taken some strong steps regarding the utilization of  or withdrawal from listing syndication in their business.  But no matter what their position or opinion of syndication, it is obvious that the industry missed the opportunity to avoid the issue when listing aggregation was new to real estate. We just didn’t clearly comprehend the implications, good or bad of their growth. Now we have another tech company that wants a share of an already shrinking pie.

As Homelight solicits the participation of agents in their program, will short term avarice on the part of real estate agents and firms create another stake holder who profits from real estate transactions without participating in them? Or is it possible that this lead generation site (for that’s what it is) will gain traction from real estate agents seeking short cuts for their business?

Can We Create Our Own Solution?

Maybe HAR’s solution points the way we need to go. If we provide the consumer with a place to create and review ratings , they have the opportunity to be heard and fully express themselves. We as agent’s have the opportunity to share the consumer’s opinion of our professional expertise and earn the trust of the public, or to be criticized for our errors. And more importantly, we might avoid having of yet another new industry grow up around a need that we can perceive and satisfy ourselves.

Should we bother? Real estate firms run on notoriously small profit margins. Companies and agents are constantly faced with downward pressure on commissions, and another layer of “referral” expense, generated by us and utilized to the benefit of others is the last thing we need. If reviews are needed or wanted by the consumer, perhaps they might best be provided by the industry. As you can see, I think so. How about you?

Enhanced by Zemanta