Zillow & Trulia Still Apathetic About Data Quality

Zillow & Trulia Still Apathetic About Data Quality

As regular readers here may know, after the world’s longest home search, I finally bought a house in May 2011. Specifically, we closed on May 20, 2011 and paid $224,950.

Last week I happened to pull up my home on Zillow, and was greeted with an interesting surprise. According to Zillow, I purchased my home on May 17, 2011 for $364,000.

Zillow Price History for 3601 Wetmore Ave

The especially strange part about this is that it was new information that had been added to Zillow somewhat recently. I only check my home’s page on Zillow a few times a year but this erroneous information definitely did not exist as recently as a few months ago. Up until this bogus data was added, Zillow’s price history for my home looked similar to my home on Trulia, where the 2011 listing and sale are entirely missing.

Trulia Price History for 3601 Wetmore Ave

Of course, if you pull up my home on Redfin, you get information that is both complete and accurate:

Redfin Price History for 3601 Wetmore Ave

If you spend much time comparing Zillow and Trulia to the underlying county records and MLS data you will run into these kinds of blatant errors with regularity. It is worth noting that incorrect sales data like this was not even covered in the real estate listings data quality study that Redfin commissioned last year, which showed that Zillow and Trulia are also missing approximately 20% of the active listings and that 36% of what they show as homes for sale are no longer actually on the market.

To their credit, for one-off issues like my home’s incorrect data above, Zillow does have a “Flag transaction” button, and support staff there were quick to respond last week when I reported the incorrect data. By the end of the day they reported that they had resolved the issue, and about a day later the site reverted to the version that is merely missing my 2011 purchase. So that’s a win, sort of. There are certainly some people at Zillow and Trulia that care about data quality, but their current approach to improving the quality of their data is like trying to improve your Hummer’s gas mileage by making sure there is enough air pressure in the tires—you might get a few more miles per gallon, but you’re still driving a vehicle that is ridiculously inefficient by design.

These types of systemic data quality issues have been known to Zillow and Trulia for years, and yet they have made only modest infrastructure improvements to address them. This makes sense when you realize that buyers searching for a home on Zillow or Trulia are not really their customers, they’re the product that is being sold. Zillow and Trulia’s customers are real estate agents who buy advertising, for the explicit purpose of turning buyers on those sites into “leads.”

But don’t take my word for it. I’ll let Zillow CEO Spencer Rascoff tell you in his own words how bad data is a good thing for Zillow’s lead-hungry real estate agent customers:

I honestly have no idea why any serious home buyer would use Zillow or Trulia to search for homes for sale. I suppose if you live in a rural area that is not serviced by Redfin or any other MLS-powered site, using Zillow or Trulia is better than having nothing at all, but only slightly.

If only Zillow would put as many resources into data quality as they put into sappy TV commercials.

Full disclosure: The Tim is no longer employed by Redfin, and holds no positions in Zillow, Trulia, or Redfin at the time of this post.

[Update]
Wow that’s rich. After this post went live my home’s page on Zillow magically updated with the correct 2011 sale date and price. Obviously I have some readers with data edit access over at Zillow HQ.

  

About The Tim

Tim Ellis is the founder of Seattle Bubble. His background in engineering and computer / internet technology, a fondness of data-based analysis of problems, and an addiction to spreadsheets all influence his perspective on the Seattle-area real estate market.

25 comments:

  1. 1
    Saffy says:

    I disagree, The Tim. Oftentimes, bad data is worse than none at all.

    Rate this comment: Thumb up 0

  2. 2
    David Losh says:

    Lead generation is what it’s all about. The data is the porn of these sites, so I don’t think it’s significant if it’s photo shopped a little.

    The ultimate purpose is to get some one to hire a professional.

    What you are pointing out is that the consumer may search all they want on these Real Estate sites, but if they want the real deal they will still have to work with a Real Estate agent.

    Rate this comment: Thumb up 0

  3. 3

    Tim, Excellent Report

    It shines a flashlight on a few houses, but this snapshot in time makes all the YOY home price increase data using your related data source appear as falsified “cooked books”…..until someone can prove your house examples were just an innocent oversight.

    Rate this comment: Thumb up 0

  4. 4

    RE: softwarengineer @ 3

    You Got a Monetary Tip From SWE on This Article

    Keep up the Great Work Tim!

    Rate this comment: Thumb up 0

  5. 5
    redmondjp says:

    What????? Bad data on the internet?

    I agree with Saffy – there is so much incorrect and outdated data out there that it requires a good detective to sift through all of it. I’ve tried to track down old friends, and I end up finding multiple addresses and phone numbers (mostly for former residences), with no easy way to tell which information is current. Most amusing to me are the countless data aggregator sites that still show contact information for businesses and other service providers that have been closed for years.

    Rate this comment: Thumb up 0

  6. 6
    David Losh says:

    RE: David Losh @ 2

    Uh Oh, don’t like the idea of hiring a Real Estate professional?

    It could be that the Real Estate Industry is over run with want to be experts who rely solely on the data.

    redfin is the best example that followed the lead of Zillow.

    Of course Zillow is driving up Real Estate prices, that’s how they get the sales. No one wants lower Real Estate prices, because that doesn’t generate excitement.

    Some buyers may want lower prices for housing, but the owners certainly don’t. It’s a game, a shell game of who will pay the most.

    How many buyers are bragging right now that they got a steal of a deal because the price of Real Estate has gone up so much?

    Of course the data is fudged, what would be the incentive to give you straight information on a web site?

    Then on top of that, more than the price that is paid, how many people cost out what they need to do to make a house a home for them?

    I just don’t see a lot of real information about housing on the internet. It’s mostly hype, with a few chosen sales terms thrown in.

    Rate this comment: Thumb up 0

  7. 7
    Saulac says:

    “Bad data is a good thing”. Excellent, the Tim.
    You pointed out how out of date data (i.e., stale listings) serves Zillow’s customers (i.e., agents, as you pointed out).
    If I could, I would like to say that inaccurate data (i.e., inflated estimates; or fictitious prices, as in your example) is another way Zillow serves it customers. There is nothing simplier and more effective for agents to do than using Zillow estimate to tell buyers that they are getting good deals. Allow me to play Mr. Rascoff for a sec. “Out of this world estimate is still good estimate. “The sale price of this home is lower than it market value …”

    Rate this comment: Thumb up 0

  8. 8

    Let me get this straight:
    You’re looking online for a house to buy, and you don’t have an agent. You find the perfect house on Zillow, and whaddya know, there are pictures of three smiling doofuses offering their services to you as Zillow Premier Agents, even though it turns out that the house sold months ago.
    Zillow originally was mostly noted for their automated valuations of properties. It was something of an addiction, this parlor game, this “Zillowing” one’s home, especially as prices were rising. Unfortunately, while their valuations are sometimes right, they are also sometimes way, way off. But that doesn’t seem to matter to them. It’s data, and it doesn’t matter if it’s accurate, current or whatever.
    So what makes a Zillow Premier Agent Premier? Is it that Zillow recognizes that they bend over backwards to help clients? Is it that they’re noted for their honesty and ethics?
    Heckums no!
    What makes a Zillow Premier Agent premier is that they’re paying Zillow premium money.
    If Bernie Madoff or Lucky Luciano had real estate licenses, they too could be Zillow Premier Agents.

    Rate this comment: Thumb up 0

  9. 9
    Roman, REALTOR says:

    Why would you want your sold info to be correct??
    It’s not anyone’s business but your own and when you go to sell a buyer will see what you paid for it. No matter what the market says the house is valued at, a buyer will know roughly what you owe and will not want to pay much more than what you paid when you bought the house.

    Rate this comment: Thumb up 0

  10. 10
    David B. says:

    RE: Roman, REALTOR @ 9 – Like it or not, it’s a matter of public record what a property sold for. If Zillow’s data are incorrect, it’s pure incompetence on their part. Moreover, Zillow tries to pass their site of as being of value to buyers, and such information is useful to buyers.

    Rate this comment: Thumb up 0

  11. 11
    ray pepper says:

    I think Tim’s peeved he didn’t scoop up any (Z) stock or take a job there. They are firing on ALL cylinders and longs are making a KILLING!

    I didn’t think anyone uses Z to search for homes for sale. I thought we all use it to find out the value of our home! Its fun! Sheer titillation!

    What happened to that poll guess the share price of (Z). Wonder who won that contest. NOT ME!! Good Job Spencer: http://finance.yahoo.com/echarts?s=Z+Interactive#symbol=z;range=2y;compare=;indicator=volume;charttype=area;crosshair=on;ohlcvalues=0;logscale=off;source=undefined;

    Rate this comment: Thumb up 0

  12. 12
    mike says:

    RE: Roman, REALTOR @ 9 – Yes, I suppose if everyone was responsible for paying their own debts it wouldn’t matter. As long as the taxpayers get to pick up the tab for everyone and anyone’s mortgage what you paid and how much you owe should be public.

    Rate this comment: Thumb up 0

  13. 13
    Victor Lund says:

    I am a real estate industry technology consultant, so take what I say with a grain of salt. You are 100% accurate in your post. Real Estate professionals complain about the accuracy of listings on these sites all of the time, but seller still insist that agents market their homes for sale there. Its baffling.

    If you want to monitor your home’s value, there is a product offered from CoreLogic called epropertywatch.com – They will show you the automated valuation (AVM) that banks purchase from them to watch home values on loans they finance for consumers. The service is free, and you can watch your debt to equity ratio. They also show you the information from public records like Redfin. They send you an email every month, which is kind of a drag, but it beats looking it up on your own. Kinda like a stock report.

    I find the data to be accurate on high volume areas of the market, but it gets a little dicy when you start looking at luxury homes. I have an expensive home in California which ePropertywatch shows more accurately than a Zestimate, but still not quite right. Not enough comparable data for AVM accuracy. Our home in Buffalo is spot on.

    Rate this comment: Thumb up 0

  14. 14
    The Tim says:

    By ray pepper @ 11:

    I think Tim’s peeved he didn’t scoop up any (Z) stock or take a job there.

    While Stan and his data analysts are doing interesting work there, I’ve never really been interested in working for a company whose entire business model is basically to be a no-value-added middleman that sells advertising to the highest bidder.

    As for their stock, my main reaction is complete perplexion. The current price makes zero sense whatsoever, even if you pretend they don’t have persistent data quality issues.

    Rate this comment: Thumb up 0

  15. 15
    Haybaler says:

    Four or Five years ago I made use of the “Claim this Home” feature offered by Zillow for the purpose of adding interior photos and descriptions of a house that I had remodeled and listed for sale.

    I still receive email updates about that house from Zillow. The info and pics that I posted on the listing are still there…probably to the chagrin of the new occupants.

    Haven’t found a way for me to Un-claim the house.

    Rate this comment: Thumb up 0

  16. 16
    The Tim says:

    RE: Haybaler @ 15 – You should be able to log into Zillow, go to the home, then float the mouse over the “Edit” button below the photos and click “Release ownership claim.”

    Rate this comment: Thumb up 0

  17. 17
    The Tim says:

    Hah! Sometime after this post went live my home’s page on Zillow magically updated with the correct 2011 sale date and price. Nice band-aid, Zillow, but your underlying system is still busted.

    Rate this comment: Thumb up 0

  18. 18
    ARDELL says:

    RE: Haybaler @ 15

    You can remove the photos at any time but maybe NOT after you follow Tim’s advice as to how to release the claim. Once you release the claim you may have no further access to remove the photos, so remove the photos before you do. Releasing the claim does not necessarily change all of the information you added while you were the owner.

    Rate this comment: Thumb up 0

  19. 19
    Marco says:

    RE: The Tim @ 17

    I looked up my own address and found that redfin did not have my 2003 purchase, but listed the prior, in 1998, as the most recent.

    Rate this comment: Thumb up 0

  20. 20
    Kelley says:

    I was at my local dive bar one night when I met a guy who works for Trulia selling products to “all these dumb real estate agents that are actually stupid enough to buy them and I make a fortune doing it,” he told me. I’m a successful real estate agent. He was too drunk to verify my occupation before he blurted those words out of his naive and not-so-business-savvy mouth. I won’t touch Trulia with a 10 foot pole now.

    Rate this comment: Thumb up 0

  21. 21
    Haybaler says:

    RE: ARDELL @ 18RE: The Tim @ 16

    Of Course! Sign in! (Hitting self on forehead).. It never occurred to me that functionality would be different signed in.

    Mission accomplished easily.

    I was so focused on getting myself off I didn’t give a thought to deleting the photos. I hope the new owners are able to edit the descriptions and photos, if they want to, when they choose to Claim the home for themselves.

    Rate this comment: Thumb up 0

  22. 22
    Alexandria says:

    Many online have complained over the last few years that Zillow Zestimates are too high and also that data lag time caused out-of-whack values – both true. However, I am finding puzzling errors unrelated to market fluctuations. I asked for corrections on our property twice; no changes were made but the Zestimate went down twice. (Eg., Zillow 2013 tax appraisal figure was 50% off, yes you read that right.) It also appears that they are connecting property value to a nearby distress listing, where the price is less than the tax appraisal for the land alone at $128,000 (& has not sold). The Zestimate for land & improvements is $108,000. Should make for an interesting 2014 tax appraisal. It’s obvious that Zillow can’t be trusted for a variety of reasons, and can come in both wildly low and high. I never asked Zillow to post anything at all – would be nice if an owner could require them to take their property down entirely. A computer program never substitutes for a brain! Best thing to do is continue to pass the word as you have done here. Thanks!

    Rate this comment: Thumb up 0

  23. 23
    ChrisM says:

    I realize the previous poster had nothing to do with this, and Tim’s auto-generated stuff had nothing to do with this, but I still feel sad when I realize David Losh initiated a lot of these posts.

    Rate this comment: Thumb up 0

  24. 24
    Alexandria says:

    RE: ChrisM @ 23 – Don’t know who David Losh is, but seems to me Zillow speaks for itself. BTW, I checked out ePropertyWatch, and it looks really good as an information source, except that it does not cover many rural properties (~20%) due to unavailability of data.

    Rate this comment: Thumb up 0

  25. 25
    arunray says:

    I am surprised. I found that “zestimate” is always 10 to 35% below what the properties are sold or listed for. I do not trust their “zestimate”.
    But thanks The Tim for valuable information. Keep up the vigilance.

    Rate this comment: Thumb up 0

Leave a Reply

Use your email address to sign up with Gravatar for a custom avatar.
Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Please read the rules before posting a comment.