Good decisions in the real estate industry are based on facts. Facts come from many sources (such as government, consultants, proprietary, …) but not all facts are equally important. Some facts would have a big effect if they changed while a change in other types of facts have little to no effect. This posting notes that some facts are unimportant because, even if they were to change, it is not clear whether the change is real or random. Randomness introduces another reason to attach weights to facts.
In part, the importance of a fact depends on how much you can rely on it. Facts as numbers are more reliable since (mostly) they are governed by the rules of statistics. Facts as words are more slippery since the reliability of the fact depends on who is saying it and on whether the words can mislead. Weighting facts is a way to recognize these considerations.
Some kind of simple or fancy statistical analysis is important because most numerical facts are not reported with perfect precision. A reported fact can be more or less than its true value of what is being measured, due to the use of a random sample in a survey or randomization by a statistical agency intended to maintain personal privacy. Data reported by government statistical agencies are often revised over time as new information becomes available.
Regardless of what the data say, more precise data should be given more weight. If you were told that the average price of a home rose by 10 percent this month then you might be tempted to act on that fact. But, the likely actions would change depending on which of the following has been historically true during the last 12 months:
- during six of the months, price rose by 10 percent and during the other six, the price fell by 10 percent, or
- during six of the months, price rose by 1 percent and during the other six, the price fell by 1 percent.
(Awareness of how much variability is “normal”, and how normal variability varies across data sources, is one of the things which gives experts their advantage.)
Precision measures something about a fact in addition to the number of decimal places reported in a number. I find it interesting that different brokerages can estimate the vacancy rate for the same type of property at the same time in the same and disagree. CMHC and Statistics Canada sometimes report on the quality of their estimates. No brokerage in Canada reports on the accuracy of their estimates or on the track record of their past forecasts. At best, they might deflect curiosity by saying “Well, we did well overall but we really blew it last year on … Ha Ha Ha.”
People who have great confidence in a project, without understanding the precision of the facts, give too much weight to the facts as reported. They tend to end with a spectacular failure. Understanding the role of precision and different ways of weighting a set of facts is one of the insights from the Good Judgment project and is the competitive advantage of the “superforecasters” who beat the professionals.
P.S.: A warning on using “facts” which might be true or could be true: You can hear people say “We should do X because Y might be true” but, without a measure of precision, little weight should be attached to this “fact”. A currently popular example of this logic is that “We should ban (all) Muslim immigrants because it might be true that one of them is a terrorist”. A simpler version in real estate is “prices might go up because government tax policy might change”. There is a theoretical validity to the argument but no weight should be given to the premise of the argument because it is always true that “something could happen”.
Further, if that logic were valid, it would be equally logical that We should not do X because Y might not be true: “We should not ban Muslim immigrants because it might be true that none of them are a terrorist”. And, logic implies that “prices might not go up because government tax policy might not change.” And, since uncertainty makes certain kinds of logic slippery and deceptive, the following conclusion also seems to make sense: “We should ban non-Muslim immigrants because it might be true that one of them is a terrorist”.
The example of banning Muslims is chosen because it is topical and because it is a sign of how the relevant facts seem to change if you are careless. Before Trump’s Executive Order was signed, the story was that potential immigrants might be terrorists. After Trump’s Executive Order, the story became that these people are people with mothers and grandfathers and sick children. And, they have neighbours and American friends. In other words, the falsity of a lie or weightless fact can be revealed by extra dimensions. Investigating context finds these extra dimensions.
If you want to treat the quasi-fact of “something might be true” seriously, then do not think of it as a puzzle in forecasting but as a puzzle in risk management. As is always true with such puzzles, the answer depends on whether probability of it being true is high or low; not whether it could be true.