Is the media biased? Can the study of language tell us? (Part 2)

In my last post, I started thinking about media bias by looking very closely at how the choices we make when we talk or write about events can lead to diverse interpretations of the same event.  It turns out that we can choose to say different things about events and still all be accurate in our descriptions (of course, that’s not to say we can say anything and still be accurate).  Importantly, the choices we make stem from our biases, and it is unavoidable that all discourse (that is speech, writing, and other forms of communication) is in some sense biased.  Therefore, the more interesting question is not whether the media is biased but how it is biased.

I said before that I will address the issue of media bias through looking at different media outlets’ coverage of the recent fiscal cliff negotiations.  If you need a quick summary of these events, I’ll provide one (keep in mind my re-telling of the events will be my own biased interpretation of them; you can always check out Wikipedia’s description for a different interpretation).

To make a long story short, due to difficulties negotiating the federal government’s budget, the US Congress and the President established a set of punishments meant to incentivize final agreement on the budget.  Those punishments included higher tax rates and spending cuts and were called the “fiscal cliff”,  which was set to take effect in January 2013 if no resolution could be reached.  After the elections in November, Republicans (who control the House of Representatives and were led in the negotiations by Speaker of the House John Boenher) and Democrats (largely led by President Obama) began the process of sending each other offers and counter-offers.  Eventually, a compromise was reached and Congress passed a bill, which President Obama signed.

The negotiations and the final compromise were well-covered by many US news outlets.  This allowed me to collect many articles from the three most widely read online news outlets: CNN, Fox News, and MSNBC.  Although each news outlet also published editorials about the issues, I’m interested only in their “news” coverage.  Hence, I’m looking at articles that at least purport to be giving just “the facts”.  In total, I collected between 15 and 30 articles for each news outlet, totaling around 16,000 to 18,000 words for each collection.  I’ll be calling these collections of articles “corpora” (or the singular form, “a corpus”).

In my previous post, I looked at just two sentences and was able to do a close analysis.  The problem with this approach was that it was difficult to ascertain how well the ideas I presented were characteristic of media bias in general.  However, now I have approximately 50,000 words of text.  I can make a stronger claim about how general the biases I detect are, but, at the same time, I can’t possibly work with all of the articles in the same in-depth way.  Therefore, I have used a statistical technique known as keywords analysis to find the words in each corpus that occur more frequently than they do in other articles about politics from the same time period and from the same news sites (a fourth corpus with 300 articles and approximately 200,000 words).  Doing so, has given me a list of keywords (words that occur in one news outlet’s corpus more frequently than they do in news coverage in general), which I’ve presented in Table 1.

Table 1 may look a little daunting, so I’ll talk briefly about how to read it.  First, each row lists a word that was found to be key in at least one of the three news outlets.  You can figure out which one or ones by looking across the row and finding the yellow cells.  The numbers in each of the cells represent the number of times each word appeared in each of the three corpora per 1000 words.  For example, “bill” was key in Fox News.  It occurred 3.69 times per 1000 words in Fox News, 2.39 times in CNN, and 2.25 times in MSNBC.

These numbers are somewhat interesting on their own, but they don’t really tell us the whole story.  They give us a place to start looking at the articles more closely, because they tell us where the statistical differences are.  However, in order to determine what they mean, I as the analyst have to look more closely at the data.  I’ll be taking a closer look at the data in a series of posts to this blog, so be sure to subscribe if you want to be alerted as I post the rest of the findings of my analysis.

I’m going to start by taking a look at how each of the news outlets talked about budget issues in the fiscal cliff articles.

I’ll start with the MSNBC corpus.  The word “income” was key in the MSNBC corpus.  As I looked more closely at this, I discovered that this was caused by an interesting pattern of bias between the three different news outlets.  First, let’s take a look at the data in the MSNBC corpus, which I’ve presented below.  What I’m presenting below is a screenshot from the software I’ve used to analyze the articles.  You’ll notice the word “income” is in blue in the center.  We’re only seeing instances where “income” occurs in MSNBC’s articles.  There’s no reason to read all of the lines; just take a look at the words around “income”.

What I see here is that MSNBC tends to talk frequently about the richest sectors of society: the “upper”, “higher”, and “top” income earners.  In addition, it also tends to refer to these income earners simply as “people”.  Hence, we see “upper-income people” or “higher-income people” frequently.  In addition, it also mentions lower incomes quite a bit.  We can compare that to how Fox News uses the word “income” below.

Although Fox News also mentions low and high incomes, I actually see slightly more discussion of people as a whole – “all income” and “every income”.  I also see a number of mentions of “middle income”, which was never mentioned by MSNBC.  In addition, while MSNBC called the people in these brackets simply “people”, Fox News chose words like “families” and “earners” far more frequently.  Thus, we see phrases like “all income earners” and “middle-income families”.  There is a sense that Fox News coverage emphasizes the effects of the fiscal policies, especially tax increases, indiscriminately on all tax brackets or on the middle class, whereas MSNBC emphasizes their differential effects on different classes, especially those at the extremes.  In addition, Fox News chooses to talk about “families” or “earners”, words that seem to carry with them more positive evaluation than MSNBC’s “people”.  Now compare this to CNN’s coverage below.

Looking at CNN’s coverage, it seems to refer to “income” in a much more abstract way.   CNN writers more frequently refer to classifications of income for tax purposes: “household income”, “individual income”, “personal income”, and “business income”.   They also frequently talked about “income brackets”.  This choice is in sharp contrast to the more human elements presented in the coverage from MSNBC and Fox News.

So what does all this mean in terms of media bias?  Well, I think what we see here is pretty clear evidence that the fiscal cliff and its effects were presented very differently in the three news outlets.  In particular, we see that different socioeconomic classes were highlighted by the different news outlets.

MSNBC tended to talk more about the extremes of the socioeconomic spectrum, presenting the tax increases as primarily an effect that higher income earners would experience.

Fox News, on the other hand, presented the tax increases as something that would affect all classes.  Fox News also tended to use the words “families” and “earners” with the word “income”.  The choice of these words seems to suggest a greater ability to argue that the tax increases are illegitimate because they affect good people, people who have earned their money or people who are raising families.

Interestingly, CNN’s coverage was more abstract.  While CNN talked both about the different ends of the socioeconomic spectrum (as MSNBC did) as well as all income levels (as Fox News did), CNN writers also tended to keep their discussion of these issues at the level of tax code, specifying things like “income brackets”.

Overall, by just looking at this one word we start to get a picture of how a reader might be influenced to think about the fiscal cliff negotiations.  These differences are especially clear in the contrast between Fox News and MSNBC.  Given the way Fox News portrayed the effects of the fiscal cliff, as something that would affect us all and especially “families” and “earners”, I would suspect that a reader exposed to their articles would be more likely to view tax increases associated with the fiscal cliff negatively.  On the other hand, MSNBC chose to talk more about upper-income people, and I suspect, therefore, that many readers likely would not view these same tax increases as a major problem, given that they’re intended for those who can afford it.  Not surprisingly, these tendencies are consistent with the popular idea that Fox News is a more conservative media outlet, while MSNBC is a more liberal or progressive one.

This analysis begins to show us how different choices that media outlets make in how they talk about the topic can be caused by their biases but also can work to propagate their biases in very subtle ways.  In the next post, I’ll continue this analysis looking at how words like “hikes” and “revenue” present the fiscal issues  involved in different ways.  Please be sure to subscribe to my blog if you’d like to be alerted to the next post.

Update (Feb 3, 2013): I’ve posted Part 3 of this discussion of media bias.  I looked at how Fox News used “tax hikes” to portray tax increases negatively, whereas CNN used “tax revenue” to portray them more positively.