Again, Yudkowsky writes:
The second virtue is relinquishment. P. C. Hodgell said: “That which can be destroyed by the truth should be.” Do not flinch from experiences that might destroy your beliefs. The thought you cannot think controls you more than thoughts you speak aloud. Submit yourself to ordeals and test yourself in fire. Relinquish the emotion which rests upon a mistaken belief, and seek to feel fully that emotion which fits the facts. If the iron approaches your face, and you believe it is hot, and it is cool, the Way opposes your fear. If the iron approaches your face, and you believe it is cool, and it is hot, the Way opposes your calm. Evaluate your beliefs first and then arrive at your emotions. Let yourself say: “If the iron is hot, I desire to believe it is hot, and if it is cool, I desire to believe it is cool.” Beware lest you become attached to beliefs you may not want.
I like that P.C. Hodgell quote: “That which can be destroyed by the truth should be.” Your beliefs should hang upon the truth, truth doesn’t care about your beliefs. Of course, this isn’t the way the human brain works. When we’re comfortable with an idea, or if something we believe engenders positive emotions, we’re more likely to avoid facts that contradict our beliefs.
If you want to have an accurate view of the world around you, you’ve got to cultivate a willingness to give up things you believe, no matter how painful.
I know many who will find this idea foreign. Others will play lip service to the idea. Few will understand just how deep a change we have to make to implement the idea of being willing to relinquish our beliefs. It’s not comfortable. It hurts. Relinquishing cherished beliefs is anathema to the soul if you haven’t made it into something you enjoy. The default human position is to cherish beliefs, not to cherish truth. It require effort to reverse that.
One of the worst methods of practicing non-relinquishment is cherry-picking of facts to support a belief. It’s easy to “prove” anything you desire if you only accept facts in support of your belief. An important thing to remember in this circumstance is that most of the time, when cherry-picking of the facts is going on, the picker doesn’t think they’re doing it. It’s so easy for your brain to utterly dismiss things that don’t fit in to your worldview, that it doesn’t even seem like you’re making a mistake.
Unfortunately, you are.