How We Know Anything

2012-Jan-30 Gaining Reliable Knowledge

So how do we know anything? Really KNOW?

Basically, we measure it. We have to constantly work at reducing bias which is always present, especially when trying to show support for some closely held theory. The more a person or organization is tied to a belief/understanding, the harder it is to eliminate bias and the more we, as information consumers, should demand it. Humans are not good at data gathering yet this is the crux of science. And science is our most successful tool, by a long shot, for gaining reliable knowledge. But it may require enormous efforts to reduce bias, especially when dealing with behaviors, but even others areas are difficult.

Another major tenet of reliable knowledge is verification. Can someone else readily get the same result?

This is why on my reviews I measure everything that I can so it's less subjective, especially speed. I LOVE it when others either validate my findings or point out errors. For example, I have a formula for converting speeds listed by my measurement to pilots of different weights. That makes a prediction that can be tested and, if the results aren't accurate, then the formula should be tossed. So far, it has worked quite well.

Looking around at what works and how that's come to be is quite telling. In short, we know by measuring. Our most reliable knowledge comes from evidence. But what constitutes good evidence? Something that can be measured! If you can't measure it, then reliability decreases.

Stories and experiences are great beginnings but to really know anything, we have to measure it.

Knowledge Comes In Degrees. The worst type of knowledge, the least reliable, and the most likely to be uncovered by experiment. Further, we need verification to improve certainty. Evidence is king and, until there's good evidence for any assertion, it should be considered suspect. Intuition is terrible, too. When I first got into paramotoring it only made sense that having the gas tank on top would be more dangerous--an explosion waiting to happen. Guess what? When we measure the accidents where fire was involved, it's just the opposite. That's why we must measure!

Another example is saying that Avgas runs cooler. OK, maybe, but did you control for other possibilities? When you tried the Avgas how long did you run it before measuring? What was the elevation? What was the outside temperature, etc. Same with thrust tests. These are quite valid for comparison between motors but not for absolute data. I've actually run this experiment and found it pretty inconclusive, with results not far off of the error bars of the test. Which brings us to another important aspect of reliable knowledge: replication. Others should be able to replicate the experiment with similar results. Otherwise the conclusion is questionable. Of course not all experiments are created equal. A well designed and implemented experiment should hold lots more sway than a poorly implemented experiment. You have to read how they did it.

There's a lot more to gaining reliable knowledge and I've been inspired by the following article.

So lets be careful about our claims and try to always rely on a sound, rational appraisal of available evidence. We know in degrees but we should always be striving for the highest degree.


The 15 minute "Baloney Detection Kit"

Here is a great little video that talks about how science ferrets out reliable truth from our observed world.

Which oil is best? Start with something that makes logical sense and go from there. But without testing, the knowledge is less certain. You may be right that brand X is better for our application but there are others who say brand Y is better. Well controlled testing is the best way to find out.

© 2016 Jeff Goin & Tim Kaiser   Remember: If there's air there, it should be flown in!