Evaluating arguments
Andy J. Wills
This lecture contained two sections, each aimed at improving your ability to evaluate arguments - both your own, and other people’s. In the first part of the lecture, I considered weak arguments - how to identify them and hence avoid being taken in by them. In the second part, I demonstrated a systematic method for evaluating arguments.
Weak arguments
Weak arguments are those that do not support the arguer’s conclusion (or do so only very weakly). Weak arguments are prevalent in general society, and even in science, because they have the appearance of being good arguments. I’m telling you about weak arguments so you can detect and reject them. Do not be tempted by the dark side of weak arguments! They are very powerful, as they can often be used to win arguments without having to go to the bother of considering the evidence.
Across several slides, I summarise some of the more common weak arguments. The examples on the slides start light hearted and clear, but the later examples indicate that serious issues are at stake here.
As an exercise, see if you can come up with your own examples of the weak arguments discussed in the lectures.
How to evaluate an argument
Evaluating arguments is at the heart of all good scientific writing and is central to doing well in all essays and reports throughout your course. In the second section of this lecture, I talked you through a step-by-step method for evaluating other peoples’ arguments:
Infallible Flowchart
I call the method the Infallible Flowchart because it always works. When reading a paper (or textbook) have this flowchart beside you and go through its steps for each key claim.
You start by identifying the conclusion of the argument. If you don’t know what it is that is being claimed, you cannot evaluate that claim. So, start with the conclusion and work backwards.
Next, you identify the premise or premises. In other words, identify the statements that are used to support the conclusion. These are called premises.
Next, you work out the relationship between the premise(s) and conclusion. For example, if there are two premises, are they independent? In other words do they each provide separate arguments for the conclusion? Or perhaps they are conjoint? In other words, the argument assumes that both premises have to be correct for the conclusion to follow.
Next, you ask whether the premises support the conclusion. In other words, does the conclusion follow from the reasons given? Is it a logical deduction? A logical deduction is where the conclusion necessarily follows, given the truth of the premise(s). Logical deduction is quite rare. Often, an inference is being made. An inference is a conclusion that is likely to be true given the truth of the premises, but not inevitably so.
Finally, you ask whether the premises are true. Even if the conclusion logically follows from the premises, it is not a valid conclusion unless the premises are true. In science, previous or current research is generally used to evaluate whether the premises are true.
AIDS example
We worked through an example of argument evaluation using an argument about AIDS. We identified the conclusion and premises, and then decided that the relationship between them was independent. The two premises provide two different reasons why the authors wants us to believe his conclusion. If one reason fails, the other may still support the conclusion.
We then came to the less formulaic part - do the premises support the conclusion? This is quite difficult to say, and one of the reasons it’s difficult is because the conclusion is somewhat vague. What does the phrase “not necessarily a death sentence” mean? Life is a death sentence - we all die - and in the legally-related sense of the phrase, there is often a delay of years or even decades between the announcement of a death sentence and it being carried out. This makes it hard to see the first premise as supporting the conclusion - the author would need to make a more specific conclusion in order to use this premise appropriately.
The relationship of the second premise to the conclusion is somewhat clearer. AIDS is a cause of death, being HIV+ is a symptom. If being HIV+ does not necessarily lead to AIDS, it therefore does not necessarily lead to death. This is fairly close to what the author seems to wish to conclude, although again the conclusion could be clearer.
Finally, and crucially, we asked whether the premises were true. This is the legwork of science, and its interconnectedness. Evaluation of the truth of these premises depends on other arguments, in papers, and on the soundness of those arguments.
The first premise is relatively straight forward and testable. One well-cited study estimates mean incubation time of HIV to be 10 years in young adults (Bacchetti & Moss, 1989) so, on first glance, the premise seems not unreasonable.
In the second premise - “suggesting”, “may” and “significant” are hedges. In other words, they are terms which have the effect of reducing the clarity of the premise without changing its essential character. If the second premise is to support the conclusion, we have to consider it as the statement, “some people who test positive for HIV never develop AIDS”. Put this way, the premise is hard to evaluate. There is one sense in which it is true - no test is perfect, so some people who test positive for HIV are not, in fact, HIV+. But this is unlikely to be what the author intended. If so, the statement is “some people who are HIV+ never develop AIDS”. Again, there is a fairly trivial sense in which this is true. For example, someone who commits suicide on the news they are HIV+ is never going to develop AIDS. As a matter of principle, then, it is going to be very difficult to discount the possibility that AIDS would have developed if the patient had not died of some other cause first.
What the author may be referring to is the presence of what the field describes as “long-term non-progressors” - people who are still asymptomatic after 12 years. Such cases do exist, and their prevalence is hard to estimate, but they are certainly very rare, one estimate is 1 in 500 cases of HIV+.
In summary, the first premise seems to be supported by scientific evidence, but the author’s conclusion is too vague to be supported by the first premise. The second premise supports the conclusion, but it is too vague to be clearly evaluated. If the author refers to long-term non-progressors, then these do exist, but it seems too much of a leap from an approximately 1 in 500 chance of not developing symptoms for more than 12 years to “a significant number of people who test positive never developing AIDS”.
Fox hunting example
A further example of argument evaluation was set as homework, one possible answer can be read here.
Enthymemes (not in lecture)
The term enthymeme refers to a situation, undesirable but quite common even in scientific writing, where one or more of the premises are missing and have to be inferred by the reader. For example, “It is biologically natural for humans to eat animal flesh. Therefore, it is morally permissible for humans to eat animal flesh”. There’s a missing premise here, “Whatever is biologically natural for humans is morally permissible”. It’s important to spot the missing premise (the enthymeme) because often it’s quite weak. For example, in this case there are some behaviours, like killing each other, that we share with our oldest ancestors and closest primate relatives (yes, despite the common misperception, we are not the only species that kills members of our own species). Such behaviours have a reasonable claim to therefore be “biologically natural” but presumably few would argue that they are “morally permissible” (at least in most cases).
Incidentally, note the connection between enthymemes and weak arguments. Use of enthymemes is sometimes a form of weak argumentation, and one that could be used to make an argument look stronger than it actually is.
Using the flowchart in your writing
In the end, it all comes back to the flowchart. You should also see this as the recipe for constructing good arguments. One of the most common problems in student essays and reports is that one or more of these steps are missing in the arguments presented. Most commonly, the conclusion is given, but the premises are not. Or, if they are, the relationship between premise and conclusion is not clarified. Or the truth of the premises is not examined.
My-side bias (not in lecture)
It’s also really important to be aware of the a common bias in argumentation - my-side bias. This is a bias that people often do not realise they have, but which reliably occurs unless they deliberately take steps to avoid it. I give advice on how to avoid this bias in your own thinking.
One clear illustration of my-side bias comes from a study by Stanovich and West (2006). The study shows that peoples’ assessment of the strength of claims is affected by the relation of those claims to their personal circumstances. Thus, for example, non-drinkers are more likely than drinkers to think that drinking at university leads to alcoholism later in life.
Another example comes from a study by Vallone, Ross & Lepper (1985), concerning the interpretation of news reports about the Lebanon War. Pro-Israeli students considered the reports to be anti-Israel, but pro-Arab students considered the reports to be pro-Israel. So, in this case, both sides considered the reports to be biased towards the opposing side in the conflict. This is of course impossible, and it’s another example of my-side bias - the assessment of evidence is affected by what you already believe.
A third study by Perkins et al. (1991) allows us to give some practical advice on how to avoid my-side bias. Their study asked participants about issues of social and political significance on which people typical already have a view, although those views differ between individuals e.g. “Would providing more money for state schools significantly improve the quality of teaching and learning?”. People were asked to give their considered opinion, along with their reasons for that opinion. On average, people listed three my-side reasons and one other-side reason.
So, again, we see my-side bias here - people seem to bias their argumentation towards the side they already believe. The bad news from Perkin et al.’s study was that my-side bias is not reduced by age, experience, or education level, of the people evaluating the arguments. So, it doesn’t matter how old or wise or well educated you are, or how much practical experience you have of the topic – you still exhibit my-side bias. However, the good news is that my-side bias is reduced if you are specifically prompted to consider all aspects, to look for arguments on both sides, and to evaluate objectively. This approach to evaluating arguments is sometimes referred to as ‘scaffolding’, because it involves putting in place a supporting structure that reminds you to look at both sides of the argument fully. When scaffolded, people produced on average six arguments on each side, eliminating my-side bias in their argumentation.
So, to avoid my-side bias, scaffold yourself!
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International Licence.