Hi everyone! This article is a companion to the Instagram video series I am currently doing about research. To watch, click on the Instagram icon at the top of the page or find me @laurenbrockmeierwriter.
In this series of videos, I broke down the ins and outs of doing research into bite-sized pieces. I’ve tried to make this article scannable so you can jump to the section you need. I’ll continue this series in later videos and add content to this post, so check back periodically. If there is something specific you’d like me to cover, please let me know in the comments.
Part 1: Finding Appropriate Sources for Your Research
Primary and Secondary Sources
There are so many types of sources available to you as a writer. Knowing what kind you need and how to find them is vital. Let’s start with some basics.
When you are doing research on an academic subject like history, biology or anthropology you will find two types of sources– primary and secondary. A primary historical source is a piece of written work from the period being looked at, for instance, a diary entry from 1902. Other fields have different types of primary sources. Research data, artifacts, scientific reports, ethnographies, and first-hand accounts are primary sources.
Secondary sources analyze primary sources. They may use primary sources to support a theory, interpret an event or raise additional questions on a subject. One example is an article in a research journal that uses primary sources to prove that King Tut was murdered. In the sciences, secondary sources are typically literature reviews. A literature review is an article that looks at the results of several studies on a topic to summarize the current state of knowledge and what still needs to be studied. In some cases, an article that summarizes scientific knowledge for public consumption, such as an article on NASA.gov, could be considered a reliable secondary source.
In the world of internet research, there are many places to find sources. It’s good to understand the difference between primary and secondary sources, so you know which one you’ve found. Here’s a less academic example of the difference: A blog that gives a first-hand account of a contemporary event is a primary source. In contrast, an article that summarizes or interprets first-hand accounts of that same event is a secondary source.
Which Source is Right for You?
A scholarly journal, a blog, a Wikipedia article, and government websites are all legitimate sources for different types of information. You might be thinking, “Wait a second, isn’t Wikipedia terrible?” The answer is that it’s all about how you use it. That goes for many sources on the internet. Just because you shouldn’t use WebMD to diagnose yourself doesn’t mean it’s not a great source when you are trying to describe disease symptoms in a fictional story. When you embark on your research journey, you need to know what sources are appropriate for your needs.
Here’s an example:
If you want to know about the symptoms of the black plague, a website like WebMD or Healthline are decent sources. If you want to know how many people in the US or the world contract the black plague every year, you should look at cdc.gov or who.int. Now let’s say you are interested in the circumstances of the plague epidemic in San Francisco in 1900-1904. You’ll need to find scholarly articles to learn more about this. Wikipedia could be a good starting place if you need an overview of the information and some ideas about other sources to look at. Find the bibliography section at the bottom and make a list for your next trip to the library. You can also find scholarly sources on sites like JSTOR, but you will need to pay for access if you aren’t affiliated with a University. Let’s go a step further. You want to know what life was like for prostitutes during this same epidemic. Secondary sources can help you with this, but if you can find a primary source like a diary entry or a newspaper article from the time period, that’s even better. (JSTOR is also great for primary sources– more on that below.)
Of course, if you are doing research for a fictional book, you may be more interested in how something feels than in hard facts. If your character is about to go rock climbing and you’ve never been, you could read a blog online written by someone who climbs regularly. You could also arrange to interview a rock climber. A rock climber’s blog could help you with other important details like what equipment your character should be taking and what the culture of rock climbing enthusiasts is like.
How to Use Wikipedia
Wikipedia allows anyone to edit their articles. I’ve heard they’ve gotten better at vetting their contributors, but there are still some pages with poorly documented information. The good news is that these pages will include notes stating they need more sources. You should look elsewhere for information when you come across a page like this. That said, a well-written Wikipedia page with a robust bibliography at the bottom is a great jumping-off point for your research. If you are researching something scholarly, make sure the sources are by authors with PhDs or other appropriate qualifications. Then find those sources and take a look at their bibliographies. You can snowball your sources this way to speed up your research process.
Wikipedia is also a legitimate source for information on pop-culture subjects like movies, music, and television. I recommend staying away from pages on hot-button topics, as these are likely to be written from a biased perspective. (The global warming page changed every other week as people from different sides of the debate edited it.)
Example of a well-written Wikipedia page: https://en.wikipedia.org/wiki/San_Francisco_plague_of_1900%E2%80%931904
What is JSTOR?
I mentioned JSTOR above as a source for primary and secondary scholarly sources. It’s a database where you can search for various peer-reviewed secondary sources as well as all kinds of primary sources (from voice recordings to newspaper articles). If you are affiliated with a university (or some high schools), you should have free access to all that JSTOR offers. If not, you will need to get a paid subscription. This is well worth it if you need to do a considerable amount of research on a topic. They offer sources for various subjects, including medicine, history, physics, and anthropology.
Take a look: https://www.jstor.org/
Part 2: Vetting Your Sources
We all know that misinformation is a huge problem. You may be able to easily spot misinformation on social media, but it can be more difficult to notice incomplete information and subtle bias on blogs, webpages, or scholarly articles. Bias comes in many forms and flavors and is found in sources on all subjects, even fairly dry academic fields. For instance, one geologist might have a pet theory that they cling to and bolster, while another takes a more objective view of the evidence. Here are five signs of bias to watch out for:
1. Loaded or Overly Emotional Language
Loaded language refers to terms that have meaning beyond their literal meaning and a positive or negative emotional connotation. Some examples include “terrorist,” “bureaucrat,” and “primitive.” These sorts of words inject emotion and opinion into a story and can influence a reader’s (or watcher’s) opinions without them realizing it. Imagine how you might feel differently about a news story if people are referred to as “illegal immigrants” as opposed to “asylum seekers.” Media outlets and scholars should strive to use more neutral, less charged language.
Emotional language includes unneeded adjectives or words designed to create a specific emotion in the reader. These are inappropriate in news sources or scholarly articles that should be laying out facts and allowing the reader to decide what they feel and believe. For instance, if I say, “The greedy company bilked its customers,” I’m leading you to several conclusions that may or may not be true without giving you any real facts to back up these conclusions. It would be better to say, “Customers allege that the company overcharged them.”
Be on the lookout for these two tip-offs of bias. Ask yourself how you feel after reading something and why you feel that way. What facts have you been given to justify your feelings? Remember that accusations and feelings are not facts. For instance, to know whether or not the company in the example above was, in fact, greedy or had bilked its customers, you’d have to hear many details about what customers claimed, what the company said, and what its financial records showed. If the article doesn’t provide these facts (along with a list of sources for their information), there is no basis for their claims (as far as readers can tell).
2. Unqualified Sources (Who Wrote This?)
One of the most important things about a source is who wrote it and what qualifies them to give you this information. For instance, I’m qualified to talk about research because I have a degree in history– a subject that deals heavily with research.
Peer-reviewed journals like “The Lancet” or “American Ethnologist” subject every article they publish to a rigorous review for research standards and quality. They look into whether the research was conducted according to best practices, whether proper ethical oversight was given, if applicable, and whether the researchers or funders had any conflicts of interest. You should still use the skills this article is teaching you even when you approach work that’s been peer reviewed, but you can expect peer-reviewed work to be high quality. Peer-reviewed work will also be conducted by people who are qualified.
Stuff you find on the internet outside of peer-reviewed journals, or research databases can be written by anyone. Just because a person makes a charismatic, emotionally persuasive argument doesn’t mean they know what they are talking about. If they are being emotionally persuasive, they are most likely a biased source. You should vet the people who are providing you with information. Do they hold a relevant degree or have validated life experience in the subject they are talking about? For instance, someone with a degree in business is not qualified to speak about public health. Of course, if someone is trained in conducting research or has relevant life experience, they may be able to provide good facts on a subject outside their degree. Regardless of whether a person holds a degree, they should always provide legitimate sources and tell their readers or viewers how they got their information.
Keep in mind that even professionals can be biased. Just because someone quotes a doctor or a historian to back up their claims doesn’t mean that their information is true. They could be finding people with the same biases, or they could be misquoting or misrepresenting their statements. This is why it’s vital to look up your own sources using your new research and bias detection skills.
3. Are There Financial Conflicts of Interest? (or Why Was this Written?)
If you are looking at a scholarly article, it will tell whether or not the people involved in the study or article have any conflicts of interest to declare. These sorts of academic sources should also tell you the name of the study’s sponsor. One of the easier-to-spot examples of possible bias in academia is the bias associated with sponsorship. This is especially common in areas where scholarship intersects with religion, public policy, or corporate interests.
For instance, a study on a skincare ingredient conducted by a company that sells skincare products should be viewed skeptically. As should a study aimed at proving a particular religious event that a religious university sponsors. Sometimes this type of bias is less obvious. A study on food insecurity commissioned by a government and conducted by a non-profit that receives hefty government subsidies may or may not produce accurate, unbiased information. Any of these studies might produce some true conclusions, but comparing their findings with those conducted by more neutral parties would be best.
Statistics, in particular, should be taken with a grain of salt. It’s extremely easy to manipulate statistics to say whatever you want them to say. If a scholarly source includes statistics, it should talk about group size, margins of error, how they accounted for various confounding factors, who funded the analysis, and what incentives were offered to participants. Unfortunately, we’ve normalized quick, uncontextualized statistics in news media and other sources. Look for conflicting statistics on whatever subject you are researching and read about why the researchers arrived at different results. Statistics are often used to “prove” something, but they are seldom as straightforward as they seem. They are also often utilized by people and organizations who benefit from them and therefore don’t stand up to the “who wrote this and why?” test.
Outside of academic sources, financial bias can take other forms. Beware of people trying to sell you things on the internet. There are many blogs out there that provide information designed to lead you to buy a product or service. If you are looking for information on the benefits of massage, it can be easy to stumble onto a massage parlor’s blog. Sometimes these articles are well disguised as neutral information sources. Pay attention to what links are contained in the article, the web address, and whether there are mentions of specific products or promotions. The massage parlor might provide some accurate information. Still, if you look at a medical website, you’ll probably find that the massage parlor overstated some things and deemphasized a lack of evidence for their claims.
4. Are Both Sides Treated Equally?
This is a tricky one regarding scholarly sources but a bit more straightforward when looking at journalism. Let’s deal with the straightforward end first. If a news article spends most of its length talking about one side of an issue and only a short time talking about the other side, it might be biased.
Research methods can also introduce this type of bias. If I interview 12 cat lovers and only three dog lovers for an article about which pet is best for you, I’m writing a biased article. These differences can sway a reader’s opinion, even if the writer didn’t intend this. Sometimes it comes comes down to language. If I say dogs are loving animals but are difficult to care for while cats can also be cuddly and are easy to care for, I’m not making a 1:1 comparison. I’m highlighting two benefits for cats and only one for dogs. It would be better to list what kinds of care cats and dogs need and allow the reader to come to their own conclusions about what suits them best.
In scholarly sources, it’s normal for an article to strive to prove a particular theory or hypothesis and to spend only a paragraph or two acknowledging other arguments. There is more than one interpretation of most historical events. One scholar may think that the facts surrounding the injuries on a mummified body lead to the conclusion that the person was sacrificed. Another scholar may argue that the person was murdered. So how should you know who’s right?
( If you are researching a fictional book, you may not need to know, but for the sake of argument, let’s say you want to include some factual information that will stand the test of time.)
Read articles on both (or all) sides and look at the rebuttals. The rebuttal is a section of the article usually found toward the end where the scholar acknowledges other interpretations and tells the reader why they think those other interpretations are wrong (or at least not complete). If you read the rebuttals on both sides, you should get a better idea of which side you are more inclined to agree with. It’s also possible that more information is needed to prove either side correct. An article that acknowledges other theories or lack of information should be viewed as more reliable than an article that doesn’t. (A google search for “controversy over X subject” can give you a good idea of the different views involved.) Note that some topics are well proven and not controversial and therefore don’t require a rebuttal.
5. Look at Several Sources
This last rule can help you identify all the signs mentioned above. It’s also the only way to answer another vital question– is information left out? Unless you are an expert in the subject, there is no way you can know if a source is leaving information out unless you find another source that includes it.
Looking at sources from different sides of a debate or scholarly schools of thought will give you a more well-rounded view of the subject and help you compensate in your writing for the small biases that are present even in well-written articles. Try to be aware of all the individuals or groups who have a stake in the subject you are researching and try to find information from all sides, especially those that may be underrepresented in the literature.
Reading different sides can make you aware of your own bias. Take note if you find yourself leaning in one direction for personal reasons. When this happens, read more about the side you are biased against so you can use that information to mitigate your bias.
Here’s where to go for more information:
For more guidance on evaluating scholarly and popular sources: https://libguides.bridgewater.edu/c.php?g=944802&p=6811022
For help identifying your own implicit bias:
Part 3: Understanding The Information You Find
Once you find sources and vet them for bias, you must understand what you are reading. This can be difficult if you aren’t familiar with certain terms, ideas, or norms in a particular field. While I can’t include this information for all possible fields in this article, I’m going to list some common ones. I’m going to begin by talking about correlation and causation because these phenomena are often seen in academic studies and everyday life. Understanding these concepts is vital for understanding the veracity of the information you are looking at.
Correlation, Causation, and Cause and Effect
Many academic quandaries come down to one basic question: Did A cause B? It’s an easy question to get wrong, especially if you don’t understand the difference between correlation and causation. Biased sources or people purposely spreading misinformation depend on the audience not knowing much about correlation and causation.
Correlation means that two or more things are associated with each other. Correlations can be completely coincidental or affected by other unknown variables. Just because there is a correlation between variables doesn’t mean that one directly and reliably causes the other. For instance, someone selling ice cream may notice they sell more on hot days. There’s a positive correlation between ice cream sales and the temperature. This means they both go up together. This correlation may not be reliable enough to be an objective truth. The ice cream dealer might have cooler days where they sell more than usual. It’s also possible that a third variable is affecting ice cream sales. For instance, there may be more tourists or more activities that bring people together near the ice cream shop during the summer months. A random hot day that didn’t happen during the tourist season or the fair might not raise ice cream sales.
So when two things are correlated, there’s a loose but not always reliable connection between them. Sometimes correlations are completely random.
For there to be causation, the relationship between two variables must be consistent and provable. A change in one variable reliably causes a change in the other. Three conditions must be met for causation to be proven, and a well-designed study or analysis will take all of them into account. The conditions are covariance, which means there’s a relationship between the two variables where one affects the other, temporal precedence– establishing that the cause occurs before the effect, and control for third variables. In our ice cream example, we’d have to prove that temperatures affect ice cream sales, that the temperature goes up before sales increase, and that a third variable like the tourist season or local events wasn’t affecting ice cream sales. To control for the third variable, we could look at hot days during a month that doesn’t have other things going on that might attract customers.
Misleading statistics, unscientific surveys, and fake news often present coincidental correlations as proven causations. Badly designed scientific studies will fail to control for third variables.
Medical trials take the concept of causation and expand on it. The following is a simplified example of how this works:
A well-designed study for a new medication will need to factor in the normal incidence of certain health conditions among the general population when determining whether the medication causes a side effect. Suppose one-quarter of the general population typically experiences a mild rash in any given year (this is called background incidence), and your research subjects represent the population accurately. In that case, if more than a quarter of them develop a rash, there’s a chance the new medication caused the rash. If a quarter or less of them develop a rash, it probably would have happened anyway, and causation is not proven.
To sum up, correlation vs. causation is a way of thinking and questioning that can help you avoid erroneous or biased conclusions. It’s used in many academic fields and understanding it can give you a better idea of how the researchers were analyzing things.
For more information on these concepts and their applications in science check out this source: https://www.understandinghealthresearch.org/useful-information/correlation-and-causation-15
Phew! That was a lot of info. If you’ve stuck with me this far you deserve to treat yourself to a snack or something. Hopefully, this article has given you some tools you can use to improve your research. Let me know what else you’d like me to cover in the comments.