When we say that a neighborhood ranks a 3 or a 47 or a 91 on a scale of 1 to 100, we tend to treat that as an exact quantity. There are no exact quantities in science, of course, and a client challenged me to figure out what the uncertainty was around a socioeconomic index.
It proved surprisingly difficult, but I think I have finally cracked it. Along with its American Community Survey data, the Census Bureau publishes what it calls Variance Replicate Estimates, which are 80 alternative versions of the data produced for the sole purpose of allowing the calculation of margins of error. The nice thing about the VREs is that they work not just on individual variables, but on any and all combinations of variables. So if I want to find the margin of error around the mean years of education completed (one of the inputs to the Yost Index, which requires looking at a few dozen census variables), I just calculate this measure 80 times using the VREs, compare their spread to the measure calculated using the official data, and I have the margin of error. That was a rather simplistic summary; for a proper technical explanation you can go here.
Overall, at the national level, a Yost Index score can be thought of as certain to an average of plus or minus 8 positions. But this figure varies by the position itself – values at the extremes are much more certain, and values in the middle are less so. This makes sense when you think about it: the wealthiest and poorest areas are unambiguously so, but the difference between the 40th and 60th percentiles is much less well-defined. This is seen in the graph accompanying this post, which shows uncertainty as a function of the Yost Index.
My conclusion is that the margins of error are too large to ignore completely (even though that is what has always been done), but small enough that the index still works as intended. No highly affluent area is in any danger of being misclassified as somewhere in the middle, and so on. The next version of the Yost Index to be published here will include these margins of error.
Pumphandle LLC 