Evaluating Rationales I

Truth vs. Validity

A carefully, charitably constructed rationale is a tool for evaluating the reasoning it represents.  Because a rationale, by definition, always represents a person's reasoning as deductively valid, we will rarely find ourselves in a position of criticizing a person's reasoning as invalid.  This is actually a departure from the way logical analysis is often taught, and it is worth taking a moment to understand it's significance.

Here is an example of reasoning that might normally be criticized as logically invalid. 

• Kimble:  Boy, Stinson sure is in a good mood today.  I wonder why.
• Jo:  Well, I'm sure you've noticed that whenever Stinson spends the night at Laurie's house he tends to come back in a pretty good mood.  So you tell me.

Here is a typical reconstruction using the more traditional format for representing reasoning.

1. If Stinson spends the night at Laurie's, Stinson comes back in a good mood.

2. Stinson is in a good mood.

3. Therefore Stinson spent the night at Laurie's.

Now, reconstructed in this way, the reasoning clearly is invalid.  To see this, recall the definition of deductive validity. 

Deductive Validity

• Definition:  A rationale is deductively valid if and only if it is impossible for the premises (i.e., reasons and principles)  to be true, and the conclusions to be false.  (Alternatively, if the premises are presumed true, then the conclusion has to be true.)
• Identification:  To determine whether reasoning is deductively valid, assume that the premises are true and see if it is still possible for the conclusion to be false.  If it is impossible for the premises to be true and the conclusion false, then the reasoning is valid. If it is possible for the premises to be true and the conclusions false, then it is invalid.

It is not difficult to see that the reconstruction above is invalid. Assume that premises 1 and 2 are true.  Is it possible that the conclusion (3) is false?  Well, yes it is.  For even if we accept 1 and 2, Stinson still might not have spent the night at Laurie's.  For all we know Stinson might be in a good mood because he just aced his calculus exam. So the most common criticism of this reasoning is that it's invalid. In fact, it corresponds to a formal fallacy called Affirming the Consequent.

But is this is the most charitable reconstruction of the reasoning Jo provided?   The reasoning has been reconstructed as an argument for the conclusion that Stinson spent the night at Laurie's.  But really it's more charitably construed as an explanation of the fact that Stinson is in a good mood.  In other words, again sticking with the conventional format, premises 3 and 2 need to exchange places.

1. If Stinson spends the night at Laurie's, Stinson comes back in a good mood.

2. Stinson spent the night at Laurie's.

3. Stinson is in a good mood.

Now the reasoning is deductively valid.  (Actually, it's not perfectly deductively valid because premise 1 refers to Stinson "coming back in a good mood" not simply being in a good mood, but we will ignore this niggling detail.)  This doesn't mean the reasoning is now perfectly acceptable, of course.  If we assume that 1 and 2 are true, then 3 follows logically.  But we have no evidence that 2 is true.  We may know that Stinson is happy whenever he gets a good grade on a test, and whenever his mom sends him a check, and whenever the Yankees lose, so it could be any of these.

Now, at this point it may strike you that these two ways of analyzing the situation are just two different ways of saying the same thing. And there is something to that.  Although philosophers and logicians correctly insist that there is an important difference between the validity of a person's reasoning and the truth of the premises upon which the reasoning is based, the fact is that any time we make a statement about the validity of a person's reasoning, we can easily express the same point as a statement about the truth of what they are saying.  To see this, consider the the following.

1. Bob kissed Sarah.
2. Sarah kissed Tom.
3. Therefore Bob kissed Tom.

Now here are two different sounding remarks that we might make about this.

• Premises 1 and 2 do not deductively imply the conclusion 3.  If we assume that Bob kissed Sarah and Sarah kissed Tom, it is possible that Bob did not kiss Tom.

Alternatively,

• It is not necessarily true that if Bob kissed Sarah and Sarah kissed Tom, then Bob kissed Tom. Put differently:  It may be false that if Bob kissed Sarah and Sarah kissed Tom, then Bob kissed Tom.

Clearly, these seem to making the same point, even though one is about validity and the other is about truth.  In general, any piece of reasoning that we might assess as valid or invalid can be reconstructed as a conditional statement that can be assessed as true or false.  If the reasoning is valid, the corresponding conditional will be necessarily true.  If the reasoning is invalid, the corresponding conditional will be necessarily false.

The point to appreciate here is that our method of reconstruction will very rarely allow people to be interpreted as reasoning invalidly.  This is a little weird because it seems that people often do reason quite poorly, and at least sometimes this must be because their reasoning is invalid.  On the other hand, it is also in keeping with the traditional philosophical assumption that people tend to be, or at least try to be rational.  And if there were ever a time when one ought to assume that people are trying to  be rational, it would be when they are actually producing reasoning.

Quantifying Beliefs

One of the truisms of logic is that every statement is either true or false.  Truth and falsity is not the sort of thing that comes in  degrees.  One statement can not be truer than another in the way that, say, one person can be heavier or taller than another. You might find this counterintuitive.  After all, we do sometimes say things like "That is so true,"  or "That is not true at all."  These are expressions that certainly suggest that truth comes in degrees.  But if you actually look at examples of statements that seem like they could be true to different degrees, it's easy to see that that these modes of expression shouldn't be taken literally.  Consider:

• Michael Jordan was a talented basketball player.

• Michael Jordan was one of the best basketball players in the history of the game.

or

• The average distance between the sun and the earth is more than 5 miles.

• The average distance between the sun and the earth is about 93,000,000 miles.

You might initially want to say of both of these examples that the second statement is truer than the first one. But really all four statements are true. Whatever we might be trying to express by saying that one statement is truer than another is more precisely expressed by saying that one is more informative than the other.  Or consider two guesses regarding the number of jelly beans in a jar where the actual number is 613.

• The number of jelly beans in the jar is 610.
• The number of jelly beans in the jar is 1018.

Here again you might be tempted to say that the first guess is truer (or less false) than the second. But all you would mean by that is that the first guess is closer to the right number than the second.  Literally, both guesses are false.

Truth, then, is an all-or-nothing affair.  Belief, on the other hand, is something very different.  It makes perfectly good sense to say of  a certain statement that you believe it to a certain degree.  All of us know what it is to be extremely doubtful, mildly skeptical, uncertain, pretty sure, or almost positive that something is true.  (This basic intuition is formalized by statisticians as the degree of confidence that it is appropriate to have in a certain statement.  Statisticians typically stipulate that a 95% degree of confidence is sufficient for more or less unqualified belief.)  In other words, while it makes no literal sense to say that you believe a statement to be true to some degree, it is perfectly sensible to say that you believe to some degree that a statement is true.

Ideally, any conclusion you draw should be a function of the strength of the evidence you have for that conclusion. The stronger the evidence for a conclusion, the stronger our belief in that conclusion should become; the weaker the evidence, the weaker our belief should become.  In statistics we quantify our degree of belief in terms of the concept of probability.  For example, if Jed were to say that he is 99% sure that his mother loves him, this could be reconstructed as the statement that  there is a .99 probability that Jed's mother loves Jed.    

Interestingly, the degree of confidence we have in the truth of a certain statement is partly a function of the amount of information the statement contains.  For example, you may know that scientists estimate the sun's mean distance from the earth to be 93,000,000 miles. But, if you asked yourself which of the above statements about the distance between the sun and the earth is more likely to be true, you should conclude that it is the first one.  Of course, it  is far less informative to say that the sun is more than 5 miles from the earth than to say that it is about 93,000,000 miles from earth.  (After all, "more than 5 miles from earth" is just about anywhere in the universe.)  But you can be more confident that the sun is over 5 miles away than that a precise calculation of the sun's distance from the earth is correct.  Similarly, the relatively uninformative guess that there are between 100 and 1000 jelly beans in the jar is far more likely to be true than the precise guess that there are 610 jelly beans in the jar. 

Deductive Validity vs. Inductive Strength

The simplest model for evaluating a rationale is to determine whether the premises (reasons and principles) are true.  If the rationale is valid and the premises are true, then the truth of the conclusion is guaranteed.  This is a dandy, highly objective characterization of a perfect rationale, but it takes no account of the fact that in the real world the truth and falsity of premises is not something that is simply revealed to us.  We are almost always in a position of believing the premises to some degree or another, and this logically has to affect the degree to which we believe the conclusion. Consider a a simple illustration of this.

Suppose we have an opaque and absolutely enormous cauldron of Jelly Belly^© jelly beans.  We're not allowed to look into the cauldron, but we're allowed to reach into it and eat them at will.  After sampling thousands of beans from this jar we've only seen two flavors. About 90% of them have been red (cinnamon flavored) and 10% of them have been white (French vanilla), and they seem to be pretty well mixed up.  Now, suppose at some point you just reach into the jar blindly and pull out a jelly bean.  Is the following statement true or false?

• I now have a red jelly bean in my hand.

Of course, if you were just asked to guess, you would probably guess that this is true.  But if you were permitted to quantify your belief a little, you would probably say that you are pretty sure, about 90% sure to be more precise, that you are holding a red jelly bean in your hand. 

What sort of rationale would you produce for this conclusion?  Perhaps something like this:

Notice that in order to reconstruct this as a deductively valid argument we're forced to attribute a principle that is not perfectly reliable.  In fact, we know that it will misfire about 10% of the time. Knowing this we might regard this principle as a more charitable attribution.

It is now a much more reliable principle, but if we substitute this principle into the rationale it will no longer be deductively valid; the consequent of the principle no longer matches the conclusion of the argument. 

The concept of inductive strength was invented to describe precisely this situation.

Inductive Strength

• Definition: A rationale is inductively strong if and only if it is very unlikely for the premises to be true and the conclusion to be false. (Alternatively, if the premises are true, the conclusion is very likely to be true.)
• Identification:  To determine whether reasoning is inductively valid, assume that the premises are true and see if it is possible that the conclusion is unlikely.  If it is possible for the conclusion to be unlikely even when the premises are presumed true, then the rationale is not inductively strong.

Inductive strength is a legitimate concept, but it is notoriously vague.  After all, what does "very likely" (or unlikely) mean, exactly? We could just accept a stipulation, like the one used in statistics, that a statement is very likely if the probability of it's being true is 95% or more. But that puts us in the awkward position of completely rejecting a conclusion whose probability is 94.5%.  This seems very illogical. 

The concept of inductive strength is required if you think of accepting a conclusion as an all-or-nothing affair.  But if you allow belief to be quantified, as we do, it is possible to acknowledge the goodness of the sort of reasoning considered above without abandoning the concept of deductive validity.  One way to model the reasoning while preserving the deductive structure is to attach a probability to the conclusion itself.  So, rather than conclude that Jed has a red jelly bean, the actual conclusion could be "The probability that Jed has a red jelly bean is .90"  The resulting rationale would look like this.

This mode of reconstruction would be true to the language of probability theory, and it also accurately represents some forms of ordinary reasoning.  For example if Jed were to reason less formally about the above situation he might say:

Jed:   I probly got me a red jelly bean here since I jes  picked 'it outta that jar wut mostly got red ones in it.  Gawd I do love the red ones.

and we might want to represent his reasoning as follows:

This rationale, while quite a bit less precise than the mathematically quantified example is still deductively valid, and apparently in accord with our ideal of charitable interpretation. But there is a problem, and that is that it's really not clear what conclusions of the sort represented in these two rationales actually mean.   After all, Jed is talking about a jelly bean has already been drawn.  It's flavor has already been determined.  So, in what sense could it's color still  be a matter of probability at all?  It seems like it might actually be more charitable to interpret statements like "This is probably a red jelly bean," as meaning "I am pretty sure that this is a red jelly bean." In other words, it may be more reasonable to think of the conclusion itself simply as "Jed's jelly bean is red," a conclusion which Jed just happens to believe to a high degree.

Evaluating Principles:  Reliability vs. Truth

One fairly clean way to deal with the above concerns is to retain the deductive structure and simply recognize that most of the principles we use are not completely reliable. Until now, largely for the sake of simplicity, we have treated principles as if they are perfect connecting devices:  if the antecedent connection is satisfied, then the consequent condition is automatically activated.  But in reality the vast majority of principles we use are not perfectly reliable.  For example:

• Uncle Jed died because he choked on a handful of cinnamon jelly beans.

Although choking is the fourth leading cause of accidental death in the United States (there were 4300 deaths attributed to accidental choking in the U.S. in 2003) it is actually a very poor predictor of death. But we will not completely reject this explanation just because the principle is weak, for it does afford us some understanding of Uncle Jed's death.  The same could be said of the following argument.

• I don't think Aunt Myrtle is going to miss Uncle Jed much.  He was getting to be a cantankerous old goat.

This, too, is not a particularly strong principle, since whether Aunt Myrtle will miss Uncle Jed depends a lot on what sort of person Aunt Myrtle is. Still, the argument appears to have some merit.

We do not have to compromise the deductive validity of these rationales, nor do we have to express the reasons or conclusions in probabilistic terms in order to understand construct charitable rationales.  All we need to do is operate with a critical awareness of just how reliable the principles are.   

You will notice that we have been speaking of principles in terms of their strength or reliability rather than their truth or falsity.  This is important. We can speak of principles as being true or false, and their are times when it is valuable to do so.  For example, the most useful thing you might be able to say about the following reasoning is that it is committed to a false principle.

• Uncle Jed eats the cinnamon jelly beans because he hates them.

The principle here is "If person x dislikes food y, then x eats food y."  When a principle is so unreliable that we can make it more reliable by actually negating the antecedent or consequent condition (i.e., If person x likes food y, then x eats food y.), then it can be useful to speak of the principle as being just plain false. But we discover a commitment to principles like this only rarely. The problem with using truth and falsity as a criterion for evaluating principles, generally, however, is that almost all of the principles that inform our everyday reasoning turn out to be false.  This is because just about all the principles that inform our everyday reasoning have exceptions, and technically just one exception is enough to make an unqualified principle false.  Even highly reliable principles like:  All jelly beans are sweet, hence:

• If x is a jelly bean, then x is sweet.

will be false the first time some enterprising fool decides to make a jelly bean that isn't sweet.  Obviously,  the fact that a principle has exceptions doesn't mean that we shouldn't use it in our reasoning. The usefulness of a principle is a function of its reliability, not it's truth and falsity.

Abusing Principles

A preference for evaluating principles in terms of their truth value rather than their reliability is at the heart of a common error in reasoning. We call it Exceptional Refutation. Exceptional refutation is attempting to refute a principle by focusing on its known exceptions.

Exceptional Refutation

Definition: Attempting to criticize the use of a principle by showing that it is has exceptions, but giving no reason to believe that the principle is generally unreliable or that the exceptions apply to the particular case at hand.
Identification: Identify the principle and the alleged exceptions in the absence of reasons for thinking the exceptions cast doubt on the reliability of the principle as stated.

Exceptional Refutation is a mistake because showing that a principle has exceptions (and is therefore literally false) is not the same as showing that it is unreliable.  As we learned from the above discussion, we could try to fend off this kind of mistake to some extent  by explicitly formulating our conclusions and/or principles in probabilistic terms.  Sometimes we actually do this, but it makes for some very complicated, and sometimes confusing reasoning.  It is simpler, and just as effective, to operate with the understanding that virtually all the principles we use on a daily basis have exceptions.  It is, of course, important for us to know what those exceptions are.  It is also important for us to know when principles are so weak, and have so many exceptions, that they should not be used at all.  But with rare exceptions, no principle is required to be exceptionless.

Here are some examples of Exceptional Refutation.

Example 1

• Phyllis:  It says here that college athletes are admitted with much lower average high school GPA's and SAT scores than other students.  To me that isn't fair.
• Rachel: That's bull.  My roommate Delilah is on a volleyball scholarship. I don't know what her SAT scores were, but she told me she was almost the valedictorian at her high school.

Identification:  The principle here is "If x is a college athlete, then x was subject to lower academic admissions criteria than other students."  Rachel's reaction actually demonstrates a failure to grasp the principle, since it doesn't imply anything about the actual qualifications of particular students. However, it constitutes an exceptional refutation in that Delilah is offered as an apparent exception to the rule.

Example 2

Gizmo:  Look, I've clicked on all those automotive consumer links you sent me and I know they say that Hyundai's have one of the highest customer satisfaction ratings.  But I notice you still aren't driving one, and I personally know two people who hate their Hyundai's: my dad and my brother.  You should hear them commiserating about the problems they have with those heaps.  

Identification:  In this example the relevant principle, apparently based on market research,  is:  If x is a Hyundai and y is an owner of x, then y is satisfied with x.  The speaker identifies two exceptions to this principle apparently believing that this constitutes a basis for rejecting the principle itself. The fact that the speaker personally knows two people who are unhappy with the cars is not a rational basis for believing the principle is unreliable.

Example 3

Cole:  Quit giving me that "Airplanes are safer than cars," crap!  People have been saying that to me all my life.  Well, I'm sorry, but I was in a friggin' 747 when it crash landed!  So unless you've been through something like that, you can keep your travel advice to yourself.

Identification:  In this example, the relevant principle is "If x is a form of airline travel and y is a form of automobile travel, then x is safer than y."  Cole cites an example of unsafe airline travel as an exception as an attempt to refute the principle.  However, the original principle does not imply that there are no airline crashes.  The fact that Cole was personally involved in such an accident does nothing to elevate the likelihood of an airplane accident relative to a car accident.

Example 4

Angus:  Americans participate in the illusion that citizens of the United States enjoy a Constitutional right to free speech. But there is no such right and never has been.  If you think otherwise, just try yelling "I've got a bomb!" on your next  vacation.  If you don't have the stomach for that, try something less exciting, like visiting your kids' elementary school class and doing a show and tell on your favorite pornographic websites. Either way you'll find out how free this country really is.

Identification:  Here, Angus assumes that the relevant principle is something like:  If x is speech, then x is protected by the U.S. Constitution. Although many people do seem to think the right to free speech is absolute, in reality it, like all legal principles,  has exceptions. Speech intended to incite violence, or which is judged to pose a clear and present danger to others is not protected.  This does not mean there is no right to free speech, it just means the right is not absolute.

Example 5  

  Barb:  You know, in the United States you don't have a chance of being elected president if people think you don't believe in God.   I heard on NPR this morning that 7 out of 10 voters says they won't vote for a candidate who doesn't.
 Butch:  Jeezus, I really wish you wouldn't repeat that kind of crap.  I mean do you, yourself, know lots of people who say that?  I sure don't.  A person's religious views are a purely private matter.  They have nothing to do with whether someone is competent to be elected president.

Identification:  Here the generalization in question is:  If  X is a U.S. presidential candidate who does not believe in God and Y is a voter, then there is a 70% chance that Y will not vote for X.  Butch questions this generalization on the basis that neither he, nor anyone he knows holds that view.  But Butch may simply be one of the 30% of the voting public who does not hold that view, and it may be that he is acquainted mainly with people who share his views.

Example 6

Dalton: You all say that my ideas are crazy, but you forget that most truly brilliant and revolutionary ideas seem crazy to the average person.

Identification:  It seems here that Dalton is challenging the principle:  "If x seems like a crazy idea, then x is a crazy idea," by claiming that some brilliant ideas often seem crazy at first.  But the fact that there are exceptions doesn't defeat the rule, and Dalton has given no reason  to think that his idea constitutes an exception, anyway.