Research Planning Prompts

2.01 Reviewing the Literature

Reviewing the work of other researchers can provide many examples for how to organize and write research reports. By paying careful attention to the style and organization used by previous researchers, researchers can get ideas for how to best structure their own writing to meet the expectations of academic readers. Have you begun reading literature related to your research question? If so, it may provide breadcrumbs you can follow. Look closely at the references listed in any articles that seem connected to your topic to see which may be worth reading and including in your own review. Make note of them in your research journal.

Is your research purpose or hypothesis explicitly suggested In the literature you have read? If yes, note in your journal which description best fits your research approach at present (strict replication, modified replication, newly combined area, resolving a conflict). How might this affect your strategy to review the literature?

2.02 The Structure of Research Articles

As you consider how to structure your literature review, examine an empirical research article, either quantitative or qualitative, related to the topic of your research. Does it follow the exact structure described in this topic? Make notes in your research journal about how it may differ in heading labels or other ways. Do other articles on your topic adhere to a similar structure? Make note of what proportion of the article is spent on each section. Is it similar to other research articles in your field?

2.03 Locating Literature in Databases

Does it seem like there are many recent, relevant studies on your topic, or are you having to search topics adjacent to your research? If you’re finding too much or too little, think about how you may narrow or widen what you have defined as your area of interest and make some notes on your ideas in your research journal.

Name some keywords you might use in an electronic search of the literature, and list them in your research journal. Many articles list keywords at the beginning or in the abstract, so look at relevant articles you’ve already found and see which they use. You can also try referring to a thesaurus, specialized encyclopedia, or the standardized database terms for your keywords.

Now locate at least three scholarly databases related to your topic and try what you think are your best keywords in each database. Note how many results you get for each term. How may were published in the last 10 years? How many in the decade before that? Seeing when topics were producing more, or fewer studies can give you some sense of the trends in research interest. Is your topic so current there were no related studies 20 years ago and few today? Was there a surge in studies decades ago that waned recently? Make notes in your journal about what those trends tell you about your topic and how you might argue for the relevance of your research.

Lastly, find the contact info of the research librarian you can call or email with questions. Larger libraries will have more than one research librarian, and each will often specialize in an area of study, such as social sciences or business. What can they do to help you find the most relevant studies? Make notes in your research journal about what you might ask them. Make a list of the key terms you think will help you find the most relevant articles. What synonyms can you think of for those key terms?

2.04 Connecting the Literature to Your Study

As you develop your literature review and research question, make notes in your journal about how each study you intend to include specifically relates to your study. Do the researchers use the same measure you will use? Are they studying the same population? How similar or different are the research questions in each study to yours? Where is your research plan most like other studies? How does it differ? Describing how your study is similar to or different from others will help your reader understand what you are doing and why.

2.05 Keeping Notes on Literature

In your research journal, include a list of the key information elements that should be included in a Literature Notes table. Use a spreadsheet program to make a table based on the example provided in this topic. Make a note of the file name, where you made it, and where you stored it.

Review and select a citation manager to use. Citation managers save time and help organize your notes on individual articles. As you develop your collection of sources and can better compare them, it may become evident that you have some gaps to fill. Consulting your database search notes on what databases you searched, the parameters used, and the number of items returned can help decide where and how to look further. Those research notes will also help you consult with a research librarian.

2.06 Reading and Assessing Quantitative Literature

Once you identify a quantitative article on your research topic, use the information in this topic and the list of questions and suggestions below to assess the quality of the article. Write down in your journal any terms or concepts that you are unsure about, then fill in the definitions. Use the Topic 2.05 notes layout to capture the important details from this study. Note in your research journal what seems strong about the article and the study methods and what seems potentially weak. Did you identify any weaknesses not identified by the authors? In what ways might you use this article’s findings or other aspects of the article to advance your own ideas about your study?

2.07 Reading and Assessing Qualitative Literature

As you identify qualitative literature on your research topic, use the information here to assess the quality of the articles and keep notes in your journal. Again, the Topic 2.05 notes layout will help capture the important details from each study. How well might the methods of the study apply to your sample? What might limit their application? If you have made notes for both a qualitative and a quantitative article, consider the ways each type of study builds evidence and presents findings. Where are they different? Where are they similar? Think about which method speaks to you more and why. Is the appeal personal, or does it reflect something about what kind of research question you want to ask?

2.08 Using Literature to Explore Explanatory Theories

In your research journal, note the theories that have been applied to your topic. Many of these will be what we have called second-order theories, meaning they provide a focused explanation of a specific behavior, event, or phenomenon. Consider, too, which of the major theories they fit within. You may find that many fit within one, or that researchers have used many different major theories to explain their findings. Make note of any patterns that can help structure how you present them or that suggest gaps or opportunities for applying alternative theories.

2.09 Synthesis versus Summaries

In your research journal, collect or paraphrase the points you may like to include in your own work from the findings of the article, along with the page number. What central themes or findings are emerging from your reading of the literature on your topic? Make notes in your journal about any studies you intend to include that may differ enough from others to need more extensive discussion. As described in Topic 2.05, it can be helpful to keep a spreadsheet of studies you intend to use, with columns for main topics, sample sizes, methodological details, and main findings. That will help you capture key details consistently and compare studies in the literature. A brief summary at the end of a review can help readers to grasp the whole of the literature review, so make notes about how your points of interest fit together.

2.10 Why Academics Rely on Citation

Have you identified the citation and reference style you will use in writing your literature review? If you are using a citation manager (and you should be), it will format both citations and reference lists for you in the style you specify. However, it is still important that you know how citations and references should be presented. Recording in your research notes some key examples will help familiarize you with the style you will be using. You can look up examples in the style guide for your discipline. You will also see many examples in the articles you are reading for your review. In addition, many online journals have citation tools that will show you the way the reference to the article you are reading would look in the most common publication styles.

3.01 Concepts and Variables

Name the major variables you will be studying. In your notebook, define each variable, trying to make them as operational as possible. Are your operationalized definitions measurable? How so? Are your they concrete and specific enough that someone else would know exactly what they are and be able to measure them as you intend? (Note: After you have read published research on your topic, you may want to come back here and redefine some of your definitions in light of how other researchers have defined them.)

3.02 Variables in Quantitative Studies

List in your notebook the major variables you will be studying. Note the ways you might measure each variable in terms of the categories used, the number of categories, and any options on which type of variable it is. Keep in mind that variables can often be measured in several ways. Review the literature that uses variables that are like yours. How have they treated each variable? Are there advantages or disadvantages to adopting a similar way to measure the variable?

3.03 Variables in Experimental Studies

If you will be conducting an experiment, name the independent and dependent variables you will be studying. Is there an opportunity to look at more than one independent or dependent variable? Evaluate the reasons to pursue a study with additional variables, or the reasons to maintain single variables.

3.04 Confounding Variables in Experimental Research

If you are planning an experiment, what steps might you take to minimize potential sources of confounding? How will you document them in your research report?

3.05 Constructing Composite Measures

Which complex characteristics are you interested in studying that could require a composite measure to capture? Do any composite measures already exist in the literature? What elements comprise them? How will you make a case for the characteristics in your composite measure? List any measurable components you plan to include and how they relate to the characteristic. Would any of these be subject to social desirability effects? How might you minimize these effects? List the steps you might take to improve the validity of your composite measure(s).

3.06 Using Performance Measures

Will you be using an achievement test, aptitude test, or intelligence test in your research? Have you selected it yet? What is it designed to predict? Is it designed to be culture-free? If yes, do you believe it is fully culture-free?

3.07 Using Norm and Criterion Tests

If you might use tests as measures in your research, would they be norm-referenced tests or criterion-referenced tests? How does this choice influence the report you can make on the results? How would this change if you were to change test types? Which type fits best your hypothesis or research question?

3.08 Falsification and the Null Hypotheses

Will you be testing the null hypothesis in your research? Whether you are or not, how would you state the null hypothesis for your main research question?

3.09 Formulating Quantitative Questions

In your journal, record a research hypothesis, purpose, or question for the study you are planning. (Note: You may have more than one of each.) Try making both a directional and a non-directional hypothesis. Try stating your question conceptually and then in terms of variables. Where does your question need more focus? Where is it already well focused?

Brainstorm what explanations you can anticipate about your alternative hypothesis. Begin to think about how you can limit the feasibility of these alternatives. Could you use literature, sampling choices, control variables, or additional tests to limit this explanation, thereby strengthening the test of your proposed explanation?

4.01 Getting Started with Quantitative Research Design

Consider your own research interest. Begin the process of narrowing the target population, defining the unit of analysis, and thinking through the concepts that you will need to define. Record these so you can refine them as you work through topics that cover each area in more detail.

4.02 Types of Nonexperimental Quantitative Research

Consider the differences in the types of nonexperimental research discussed in this topic. Which most fits with the study you would like to pursue and why? Does your research question clearly fit into one type? Explain the basis for your choice. If you are considering a causal-comparative study, briefly explain why you chose this method. What are you comparing? What are the relevant characteristics that you would want to match in your two groups? What are some potential pitfalls that readers may want you to account for? How can you address these in your research plan?

4.03 Cross-sectional and Longitudinal Designs

Thinking about your research question, what might you find out if you could conduct a longitudinal study that you could not with a cross-sectional, moment-in-time design? Longitudinal studies are logistically impossible for many researchers to conduct but making note of what might be discovered from participants over time may highlight limitations in a cross-sectional approach. Even if you plan to conduct a qualitative study, make a list of the types of questions about your topic that might be asked using a survey. Surveys are useful tools that can be used in many ways, including gathering demographic information about a sample.

4.04 Survey Research

If you are planning to use a survey in your study, even just to collect demographic data from participants in your qualitative interviews, consider the information in this topic and identify where you may encounter problems in the collection or analysis of your survey. What are the biggest threats, and how might you take steps to overcome them? Make notes on both in your research journal.

4.05 Writing Survey Questions

Considering your research question, identify the key concept or concepts that you intend to investigate. Think about how each concept can be translated into questions about your respondents’ lives. Make a list of the behaviors, experiences, opinions, or attitudes that may be connected to each concept. Once you have that, generate one or more question for each. Then check each question against this topic’s guidelines for good questions.

4.06 Using Existing Data

Considering your research topic, do you think there are existing data sets that could be useful? Census or educational data? Health data? Social impact data? Make a plan to investigate, even if you intend to gather your own data. Identify which data sets might be useful by using search tools and by contacting a research librarian for help. Get the codebook and take a look at how they present their data. Make note of the data structure and the ways variables and other elements are described.

4.07 Experimental Research

Does your discipline and research interest lend itself to an experimental study design? What might an experiment reveal about your interest? Consider how your research would be different if experimental approaches are used.

4.08 Causal Claims: Experimental Study Design

Could your research interest be investigated with a true experimental design? If so, how could you use the designs proposed in this topic to explore your research question?

4.09 Quasi-Experimental Designs

If you are planning to conduct a quasi-experimental design, which one will you use? Why are you using a quasi-experimental design instead of a true experimental one? What might your most critical reviewer ask you to justify about this study design? Do you have an argument to justify it?

5.01 Measures of Quality: Validity & Reliability

How will you evaluate the validity and reliability of any measures you will use in your research? Write down some preliminary thoughts that you can add to as you work through more specifics in Part 5.

5.02 Judgmental Validity

Consider any questions or measures you are planning to use in your research. If you had to explain the items’ validity to a critic, how would you justify your questions or measures in terms of judgmental validity? Do you anticipate that your measure(s) will have high or low face validity? Explain the reason.

5.03 Empirical Validity

How will you evaluate the empirical validity of any measures you will be using in your research? Explain. (Note that if you use a published measure, information on empirical validity may already be available in the manual for the measure or in a research report that used the measure.)

5.04 Construct Validity

Does your study work with any constructs? What components make up your construct or are related to or affected by it? What are some of the measures you can use as indicators of your construct? How would you argue for this item’s construct validity in your research report?

5.05 Evaluating Reliability

What items in your study may need to be tested for reliability? What methods will you use to examine reliability in your study? What evidence would you offer to a skeptic reading your work that shows the measure is reliable?

5.06 Visualizing Reliability and Validity

Consider the measures you will be using in your research. Can you visualize how the data you collect might be distributed in terms of the dartboard? Are measures more likely to “miss” in a particular direction?

5.07 Principles of Reliability and Validity

Will you use or create any measures in your research (e.g., tests, questionnaires, interview schedules, or personality scales)? Have you considered how to translate the quality you wish to capture into measurements? If you will be using measures, do a little research by looking up some measures that have already been validated. How did they capture concepts? Can you use any existing measures, or can you use anything you have learned to improve the validity of your own measure?

5.08 Threats to Internal Validity

If you are planning to conduct an experiment, consider which threats, if any, it will be subject to. Are there any steps you can take to help control or learn about these threats?

5.09 Threats to External Validity

If you are planning to conduct an experiment, which threats to external validity, if any, will your experiment be most vulnerable to? Are there any steps you can take to document these effects or design your study to reduce them?

6.01 Populations and Samples

Do you anticipate that you will be drawing a sample for your study, or will you be conducting a census? Considering your research question and population of study, what is the appropriate unit of analysis? Are there alternatives? In your notebook, identify your population and justify your unit of analysis in terms of your research question and study design.

6.02 Simple Random and Systematic Sampling

If you intend to use sampling to obtain participants for your research, do you plan to use random sampling, systematic sampling, or some other method of sampling? In your notebook, write down what you intend to use and why it is appropriate to your study.

6.03 Stratified Random Sampling

Consider your sampling plan thus far. Could you use a stratified random sampling approach instead? What would your strata be? How might this make your sampling easier or less biased? Are there any downsides to this approach for your research project?

6.04 Cluster Sampling

In light of the topics covered so far on sampling, make notes on which approach might best suit your study. Would cluster sampling work in your project? How might the clusters create a sample that is more alike than other methods? Based on what you know so far, can you describe and defend the logic of your sampling choices? Can you also provide the downsides or limitations of your sampling choice and any ways you might try to counteract them? These notes can help you when it is time to write up your results.

6.05 Challenges in Probability Sampling

Start to sketch out your plan for sampling. First, define the characteristics of the population you will draw from. Be as specific as possible. Consider where you may be able to continue to become more specific—for instance, are there particular criteria you are considering around age, region, or other elements that might make your study more specific?

Think about ways that you can define your sampling frame for selection. What are the options for you to draw a sample and where could this selection become biased relative to your main topics of interest? Is there anything you can do to reduce the bias? Take some notes in your journal as you start to plan your sampling strategy. Write down some initial thoughts on how you might describe your sampling plan and justify your choices.

6.06 Nonprobability Sampling in Quantitative Research

Define the characteristics of the population you will draw from for a sample. Be as specific as possible. Consider where you may be able to continue to become more specific—for instance, are there particular criteria you are considering for age, region, or other elements that might make your study more specific?

Think about ways that your sample could become biased relative to your main topics of interest. Can you describe the source of the bias? Is there anything you can do to reduce the bias? Take some notes in your journal as you start to plan your sampling strategy.

6.07 Error and Bias in Samples

Thinking about your sampling plan, identify as many sources of possible bias as you can think of. Once you’ve recorded those, write down ideas for ways you might decrease bias in each of those areas.

6.08 Modern Sampling Techniques

Will your sampling strategy involve cell phones? How will these barriers to sampling affect your approach? Are there any steps you can take that improve your sample quality?

6.09 Sampling and Internet Surveys

If you are considering a survey that uses digital methods and requires random sampling techniques, what are some ways that you might ensure your sample meets the standards of random sampling? Are there ways that you can stratify your sample to improve results? Will you offer any alternative forms of data collection to resolve potential biases in the sample?

6.10 Sampling and Demographics

In your research journal, write down a list of demographic information you may want to collect from your participants. What characteristics are relevant to your topic or research question?

If you have defined the population from which you are drawing your sample, can you find information about the proportion of people with those demographic characteristics within that population? If you are researching a school or organization, they may have some demographic information available; if it is a region, you may be able to reference census or other data that describes the demographics of that area. Consider this information in your sampling strategy. Which of the demographic dimensions you identified matter to your study? What sampling strategy might allow you to reach enough people in each group so statistical weighting is unnecessary? If you not all subgroups are reachable, which demographic categories might need weighting if underrepresented?

6.11 Sample Composition and Bias

In your research journal, free write about ways that your data collection choices could lead to biases. What options might you have for varying your data collection or correcting for the potential bias?

6.12 Sample Size in Quantitative Studies

If you are planning a quantitative study, do you know the size of the target population? How might you determine it? What is your anticipated sample size based on the information in this topic and the topics in this part thus far? Explain the basis for your decision. What factors affect your sample size considerations? How will you justify your sample size when you describe it in reporting on your project (or proposal)?

7.01 Descriptive Statistics

If you are using statistics in your research, how will you use descriptive and inferential statistics relative to your research questions? Most studies include some univariate statistics about the sample and the variables of interest. What might you include in your report? What associations might you want to analyze in your sample? What will you want to test about the sample data compared with the likely values for the population? Takes notes on these aspects of your study.

7.02 Mean and Standard Deviation

Means are important for variables with real numeric values. For which variables will you be reporting means and standard deviations? Conceptually, how will they be relevant to your study and its outcomes?

7.03 Median and Range

Try describing the median and interquartile range of your variables. How do the mean and median compare? Do you anticipate reporting medians and interquartile ranges? Are you using any variables that would best use a median?

7.04 Mode and Frequency

Are there values in your data where exploring the frequency of responses can help you understand its characteristics? Consider what frequencies you might represent from your sample. Where might it be appropriate to show visualization of the variable in your final research reporting?

7.05 Describing Bivariate Data

Consider your variables. Would you describe them using a cross tabulation, conditional mean, or correlation? What can you get from looking at the values by comparison? Will you use this to report any of your sample data?

7.06 Inferential Statistics

Thinking about your research plan, what relationships between variables might be appropriate for investigation with inferential statistics? Are you prepared to use them, or do you have a plan for getting computational help? If you are at a college or university, note in your journal any information about where you can get such help including how to contact this help resource.

7.07 Statistical Tests for Comparing Groups

If you are doing quantitative research involving more than two variables, will you conduct a chi-square, t-test, or ANOVA? Will it be one-way or two-way? In plain English, write in your journal what are you comparing when you do this test. If you are new to using these tests, this is a great time to check in with someone more knowledgeable on what you’ve written to ensure you have it right.

7.08 Regression Basics

Will you be performing regression in your study? What is the dependent variable? What independent variables might be used? In plain English, write in your journal what are you testing and what variables (if any) will serve as controls. If you are new to using regression, this is a great time to check in with someone more knowledgeable on what you’ve written to ensure you have it right.

7.09 Normal Distribution

How can you verify that an assumption of normal distribution is appropriate to your data? Do you anticipate that your data’s distribution will be skewed?

7.10 Statistical Significance

If you are using statistics, do you expect your data to have a normal distribution? Consider how you might articulate why the data will or will not be normally distributed. Look up quantitative papers in your field, and flip to the visual presentation of the study’s statistics. Where do you find alpha, p-value, confidence intervals, and confidence levels? How do the authors discuss them?

7.11 Statistics for Effect Size

Do you plan to report value(s) of d in the Results section of your research report? How will you use it? Will you be comparing the difference between two means in your analysis? If so, do you expect to find a statistically significant difference? Do you expect to find a large value of d?

7.12 Practical Significance of Results

Can you anticipate any considerations that might limit the practical significance of the results you hope to obtain in your study? How might you argue for the practical significance of your study? Think carefully about who may be affected by decisions made based on any results you obtain, as well as any costs or barriers that might be associated with implementing any changes that might stem from those results. Make careful notes in your research journal about any people that may have a stake in your research results. Who might be most affected? Who are the decisionmakers? What will it take to justify altering processes, places, behaviors, or anything else? Your notes on practical significance may be helpful as you draft your introduction and conclusion.

8.01 Decisions in Qualitative Research

If you are considering a qualitative research study, what are you interested in as a general topic area? How might you use the methods described here to carry out this research? What aspect of qualitative work are you most concerned about – and what steps might you take to help you with this concern?

8.02 Generating Qualitative Questions

Which “small worlds” intrigue you? Make a list of specific, narrowly defined social subgroups and note the reasons why each may be interesting. Who is in that world and where is it? What about the small sub-group is unique? What about them do you find mysterious? How can you reach members? Note the ways you might contact and find out more about each. What types of questions might you ask?

8.03 Collecting Qualitative Data

Are you using interviews, focus groups, and/or observations in your qualitative study? What are your strategies for recruitment and/or gaining access to a site? How will you prepare to make your pitch in order to get access or get people to participate in your study?

Consider that qualitative researchers often analyze their data simultaneously with data collection. What strategies do you have in place to help you organize your data and annotate it? What tools will you use and how much time will you allocate time for the work to prepare and analyze the data you collect?

8.04 Mixing Collection and Analysis: Grounded Theory

If you plan to conduct qualitative research, how would you design your research to use the grounded theory approach? Investigate resources to help you learn more about how to code qualitative data, including the approach to coding as well as the practical tools you can use to code the data and analyze it. This may include how to code in programs you already own, or you may want to find out what qualitative data analysis (QDA) software is available to you or is most used in your discipline. 

8.05 Qualitative Coding Processes and Techniques

Assuming you will use coding for your qualitative analysis approach, what seem like workable ways to approach the mark up? If you think you might use a word processing program or something similar, try using it to tag some text with codes. Are they easily distinguishable? Would using a variety of colors help visualize patterns?

8.06 Assessing Quality in Qualitative Research 

If you are planning to conduct qualitative research, which of the techniques described in this topic, if any, do you plan to use to ensure that your data has integrity? Explain your choice(s). 

8.07 Recruiting Participants for Qualitative Studies

If you are planning to conduct qualitative research, consider how you would approach your research question using purposive sampling or theoretical sampling strategies. What are some of the ways you could sample the group that interests you? Identify the characteristics that are important for your study. What types of people might represent the most extreme examples of those characteristics? Would a homogeneous sample suit your research question?

8.08 Sample Size and Diversity in Qualitative Research

If you are planning to conduct qualitative research, which approach do you think you will take to planning your sampling strategy? Even if you plan to collect data based on a model of reaching saturation, do you have an estimated range for the sample size your research project will require? Consider whether you will collect from more than one site or from people in more than one role.

8.09 Designing Interviews

If you are planning to conduct qualitative research using interviews, how do you plan to gain experience or test your interview guide before conducting your research? How much time do you expect to spend on each interview? How does that affect how many people you might interview? Thinking about your topic, what types of questions or phrases risk leading your respondents or otherwise biasing answers?

8.10 Conducting Interviews

Interviewing requires careful planning and practice, so take a few minutes to sketch out how you will conduct them. Will they be in person? If so, where? How will you record interviews? Is video an option? Practice with recording devices will make for a smoother experience, and your test recordings can be used to try an automated transcription option. Even if you are not intending to videotape interviews, try videotaping yourself conducting a practice interview with someone who is not part of your study. Seeing yourself in action can give you valuable information for adjusting your technique.

8.11 Planning Focus Group Facilitation

If you are planning to conduct qualitative research, how might focus groups provide you with the information you need to answer your research question? Will you use more than one qualitative method in your study? Is your decision about which to pursue limited by practical considerations related to time or access? Examine the relationship between your chosen design and your research question.

8.12 Observation and Ethnography

If you are planning to conduct qualitative research, how might focus groups provide you with the information you need to answer your research question? Will you use more than one qualitative method in your study? Is your decision about which to pursue limited by practical considerations related to time or access? Examine the relationship between your chosen design and your research question.

8.13 Designing Case Study Research

If you considered a case study approach to a research interest or question you have, what would constitute a case? (Remember that a case does not have to be an individual; it might be a group, event, or period of time.) What would be the boundaries around the case? What types of data collection methods and time frame would you need to do a case study project? Would your project be idiographic or an exploration to set up a nomothetic research project?

8.14 Mixed Methods Designs

If you considered expanding your current study to use a mixed methods design, what would you want to explore in the new strand of your research? Which design would make the most sense to answer your question?