Class 3 - Research III: Practices
Agenda
Skills corner: Note taking (10 minutes)
Conceptual grounding (10 minutes)
Core paper discussion (40 minutes)
Break (5 minutes)
Compare-contrast presentation (30 minutes)
Summative lecture on concepts (25 minutes)
Skills corner
Strategic note taking for scholarly articles
Note taking and reading are two sides to the same coin in scholarly work
Typically note taking / highlighting happens in the ‘later’ passes of reading, once a applicability threshold is reached
Pro tip 1: As soon as you think a paper has any relevance, download it in annotable format and start taking notes!
Pro tip 2: Develop a system for organizing your notes and stick with it! An example
OK - what notes should I take?
Narrowly
Your interpretation of what the paper is about
Key claims of the paper and their support (or not)
Connections you noticed to other papers and ideas
The 5Cs of the paper, the parts you could follow, and what you agree with (or not)
As I get on in this business, I tend to highlight and notate less
OK - what notes should I take?
Broadly:
Citations related to key ideas and passages to chase down later
Methods and measures used that could be useful in future
Theories and constructs that you want to learn more about
Synonyms for terms that are important to you
The logic and supporting argumentation for key claims raised
Stylistic elements and ways of organizing a paper that you like
Grounding
Building a toolbox of practices
The building blocks of an article - The 5Cs
The building blocks of a design - The MIR Framework
The building blocks of a study - What to do before collecting data
The building blocks of a contribution - Two complementary perspectives
Readings for Today
Common Readings
Lange, D., & Pfarrer, M. D. (2017). Editors’ Comments: Sense and Structure—The Core Building Blocks of an AMR Article. Academy of Management Review, 42(3), 407-416. https://doi.org/10.5465/amr.2016.0225
Tobi, H., & Kampen, J. K. 2018. Research design: the methodology for interdisciplinary research framework. Qual Quant, 52(3), 1209-1225.
Aguinis, H. & Vandenberg, R. J. (2014). An ounce of prevention is worth a pound of cure: improving research quality before data collection. Annual Review of Organizational Psychology and Organizational Behavior, 1: 569-595.
Lange and Pfarrer (2017)
Lange and Pfarrer (2017)
Discussion Questions
Why might this method be effective? When might this approach hinder, rather than help, exposition?
How does this align with the lessons of the Pyramid Principle?
Is this the way the research process must unfold?
Tobi and Kampen (2018)
Research design: the methodology for interdisciplinary research framework.
The MIR framework also introduces an agenda: the research team needs to carefully think through different parts of the design of their study before starting its execution (Fig. 1). First, the team discusses the conceptual design of their study which contains the ‘why’ and ‘what’ of the research. Second, the team discusses the technical design of the study which contains the ‘how’ of the research. Only after the team agrees that the complete research design is sufficiently crystalized, the execution of the work (including fieldwork) starts. (p. 1211)
Tobi and Kampen (2018)
Discussion Questions
Does the figure help to put the various elements of research design in context?
Do you need to perform all of these steps? Which are essential (if any)?
Aguinis and Vandenberg (2014)
An ounce of prevention is worth a pound of cure: improving research quality before data collection.
The suggestions offered here are motivated by our combined experience as past journal editors, associate editors, and editorial board members. We noted, sadly, that the majority of manuscripts not accepted for publication after traversing the peer-review process could have actually been rejected before the data were collected. The problems with the majority of rejected manuscripts are related to theory, research design, and/or measurement. Rarely are data analyses grounds for rejection because weak analyses can often be fixed if all of the other components are strong. (p 1.2)
Aguinis and Vandenberg (2014)
Discussion Questions
Tell me one recommendation that resonated with you and why.
When might the data collection be sufficiently compelling that it would stand in contraposition to the quote on the previous slide? Are such situations rare?
Break
Compare / Contrast Presentations
Corley, K. G., & Gioia, D. A. (2011). Building Theory about Theory Building: What Constitutes a Theoretical Contribution. Academy of Management Review, 36(1), 12-32. https://doi.org/10.5465/amr.2009.0486
David A. Whetten, 1989. What constitutes a theoretical contribution? Academy of Management Review, 14: 490-495.
Lecture - Practices
Practices you can use
The MIR framework - The space of design choices and how they interrelate
An ounce of prevention - Or how to avoid a wasted data collection effort
The 5Cs - The introduction as a basis for paper development
The contribution - And its dependence upon the other practices
The fountainhead - Research questions
What is the “starting point” of the MIR framework?
How can increase the confidence that we are tackling a theoretical or practical problem?
How might the 5Cs help to iteratively identify or articulate a research question?
Consensus creating or shifting research
Crafting research questions using the 5Cs
What are our desiderata for “good” research questions?
Let’s go back through our last three classes!
Principles:
Knowledge-accruing: Falsifiable / has falsifiable implications
Useful: Applicable / advances theoretical understanding
Generalizable: Replicable / not an ad hoc, one off, “just so” story
Answerable: Conclusion reachable through defensible logical pathways
What are our desiderata for “good” research questions?
Let’s go back through our last three classes!
Positions:
To what end? Theoretical v. practical knowledge
By doing what? How much work required to obtain precision / realism / generality?
With whom? Who cares about the answer?
Means of advance? Does the answer give rise to a novel knowledge claim?
What makes a good research question for today’s purposes?
Feasibility: Can it be answered with a series of 1-3 papers?
Personal interest: Is it a question that you want to know the answer to?
Next class
I will have the first concept check posted to Brightspace before next class.
It is super simple, about 10 questions based on our discussions so far.
You will have one hour to complete the assessment once started (plenty of time)!
Next class
Elements I: Theory and phenomena
Bacharach, S. B. (1989). Organizational Theories: Some Criteria for Evaluation. The Academy of Management Review, 14(4), 496-515. https://doi.org/10.2307/258555
Makadok, R., Burton, R., & Barney, J. (2018). A practical guide for making theory contributions in strategic management. Strategic Management Journal, 39(6), 1530-1545. https://doi.org/10.1002/smj.2789
Johns, G. 2006. The Essential Impact of Context on Organizational Behavior. The Academy of Management Review, 31, No. 2, 386-408.
Next class
Elements I: Theory and phenomena
- Compare / Contrast
Sutton, R. I., & Staw, B. M. (1995). What Theory is Not. Administrative Science Quarterly, 40(3), 371. https://doi.org/10.2307/2393788
Hambrick, D. C. 2007. The field of management’s devotion to theory: Too much of a good thing? Academy of Management Journal, 50, 1346-1352.
Workshop
Workshop Agenda
Pulse Check
A roadmap for the class (11a - 12p)
Research Question Presentations (1 - 3p)
Self-organized group discussions (3 - 4p)
Reflections
Class Pulse Check
Roadmap - Where are we going?
On the following slide, I illustrate how the class fits together
Note that it assumes that the intent is to complete a quantitative, empirical project, but I have indicated “offramps” to other types of contributions in red
Class content is indicated in blue
Information relied upon from ‘outside the system’ is shown in green
A pictoral representation of the research process
The paradigm of our class
This class is rooted in a paradigm
We have a well-articulated research question (Classes 1-3)
We will build a model with hypotheses to answer this question (Classes 4-7)
That model is nested within a causal system and reality
Our hope is to recover a causal effect through an estimation process (Classes 8-12)
For this to be effective, we want this estimate to be:
Accurate: hopefully unbiased, but at least consistent
Precise: “efficient”, using sample information well
How your model fits within this paradigm
On the following slide, I illustrate how your focal model compares to the wider system of relationships
Note that you may need to include elements outside of your model of interest to specify the causal system (as we will discuss in Class 5: Models + Hypotheses)
Also note that our model may be influenced by observable and unobservable factors (as we will discuss in Class 6: Constructs + Variables)
This presumes you have a story you want to understand rather than trying to maximize the predictability of the DV (a different question to answer)
How does this process help us build knowledge?
How does this process help us build knowledge?
Research Presentations
