Class 5 - Elements II: Models and hypotheses
Agenda
Skills corner (5 minutes)
Conceptual grounding (10 minutes)
Core paper discussion (40 minutes)
Break
Compare-contrast presentation (30 minutes)
Summative lecture on concepts (30 minutes)
Skills corner
Developing discernment
What makes an article credible? How can we assess its quality? Should we trust the findings?
Internal consistency - does the logic of the argument hold?
External consistency - are the arguments and data consistent with what is already known?
Craft - is the article nuanced and layered theoretically and empirically, paying attention to multiple factors?
Clarity - does the paper have a point of view and makes clear what it is adding to the conversation?
Fit - is there a reasonable connection between the argument raised and the methods used to test it?
Developing discernment
Potential correlates of quality
Journal of publication (?) - good articles may be too revolutionary; reviewers are an imperfect screen for errors
High citation counts (?) - citations for interest, rather than correctness
Predictive validity (?) - later work that replicates and extend findings; what about Newtonian mechanics?
Your judgment (?) - improves with experience; tempered by biases you may accumulate
Finding articles to consider
In rough order of “rigor” and “reach”
targeted “mini-search” (e.g., quick Google Scholar Search)
literature “scan” (quick set of keyword searches on Web of Science)
“[Insert type here] Literature Review” (capital letters) (follow a protocol)
Grounding
Making things concrete
Model-building: Drawing in theory and phenomenological features
Hypotheses: Making model-based predictions and providing a basis to accumulate knowledge
Some thoughts on models and knowledge
All models are wrong, but some are useful. - George Box
Knowledge is knowing that a tomato is a fruit. Wisdom is knowing not to put it in a fruit salad. - Brian O’Driscoll
Readings for Today
Common Readings
Harris, J. D., Johnson, S. G., & Souder, D. 2013. Model-Theoretic Knowledge Accumulation: The Case of Agency Theory and Incentive Alignment. Academy of Management Review, 38(3), 442-454.
Chamberlin, T. C. (1965). The Method of Multiple Working Hypotheses. Science, 148(3671), 754-759. https://doi.org/10.1126/science.148.3671.754
Pearl, J. 2010. 3. The Foundations of Causal Inference. Sociological Methodology, 40(1), 75-149.
Harris Johnson and Souder (2013)
Harris Johnson and Souder (2013)
Discussion Questions
Can you see yourself employing a model-theoretic lens in your work?
Is this view consistent with our research “principles”?
Harris Johnson and Souder (2013)
Key Points
Empirical research can build theory without changing theoretical claims by building models that graft, contextualize, and repurpose representational models: synthesizing general models / theories with aspects of the real world from which hypotheses can be derived (Harris, Johnson, and Souder 2013, 445)
Figure 1 provides a nice worked example of how we can traverse from a particular phenomenon to a theory and back
Chamberlin (1965)
The effort is to bring up into view every rational explanation of new phenomena, and to develop every tenable hypothesis respecting their cause and history. The investigator thus becomes the parent of a family of hypotheses: and, by his parental relation to all, he is forbidden to fasten his affections unduly upon any one. (Chamberlin 1965, 756)
Chamberlin (1965)
Discussion Questions
How would you actually implement the method of multiple working hypotheses in your work?
Where might our discussion of “thinking about thinking” (Bayesian and sketpical thinking) help or hurt us in its application?
Chamberlin (1965)
Fertility in processes is also the natural outcome of the method. Each hypothesis suggests its own criteria, its own means of proof, its own methods of developing the truth; and if a group of hypotheses encompass the subject on all sides, the total outcome of means and of methods is full and rich. (Chamberlin 1965, 765)
Chamberlin (1965)
Key Points
While a particular theory focuses our attention when designing a study, we should not be blinded to alternative explanations
Alternative explanations should be “baked into” a study and given due consideration
Discriminating evidence is needed to distinguish between multiple plausible theories
Pearl (2010)
Pearl (2010)
Discussion questions
I know this was dense, what did you get out of this paper?
Based on this paper, what is necessary to make a causal claim?
Pearl (2010)
Pearl (2010)
Key points
Models can be represented using directed acyclic graphs
The effects do not need to be linear in nature (but it is often nicer to talk about the linear case and generalize)
To determine if an estimated effect is causal, the graph must meet certain criteria such as the back-door criteria (e.g., {W1, Z3})
Experimental designs allow for a “do X” operation which make identification easier
Break
Compare / Contrast Presentations
Adner, R., Pólos, L., Ryall, M., & Sorenson, O. 2009. The Case for Formal Theory. Academy of Management Review, 34(2), 201-208.
Miller, K. D. & Tsang, E. W. K. 2011. Testing management theories: Critical realist philosophy and research methods. Strategic Management Journal, 32(2): 139-158.
Lecture - Hypotheses and models
Hypotheses: A method for accumulating knowledge
Knowledge accumulates through refining the theoretical statements at the top of the diagram, but it also accumulates as scholars cobble to- gether a better understanding of reality (Cart- wright, 1999) by improving their ability to de- velop and use representational models. Empirical support for hypotheses derived from a representational model demonstrates the use- fulness of that model in the proscribed context. (Harris, Johnson, and Souder 2013, 447)
Hypotheses: A foundational element
Given that hypotheses are so fundamental to model building, you would expect that I would have given you a reading about how to write them, right?
Despite trying, I could not find a great one - the best I could find (drawn upon below is Neuman (2014) in concert with Pearl (2001) - both books)
Hypotheses: Some desiderata
Hypotheses: Some anti-criteria
Models: Styles of construction
Verbal theorizing: the standard method employed by most papers
Directed acyclic graphs (DAGs): becoming more common, even if the logic is operating in the background
Formal theory: use varies significantly by discipline
Subtle point: Each relationship in a model is technically a hypothesis; you elect to focus on certain relationships of interest (e.g., the novel ones)
Progressing your final project: Building a model
This week’s readings should help start launching you on a trajectory to convert your research question into a model with associated hypotheses
These hypotheses are often one of three flavors:
Tests of relationships between constructs that you intend to perform (the standard empirical paper strategy)
Relationships that are implied by a theoretical system that can be tested in the future (these are often called propositions)
Assertions of the existance of a particular construct or mechanism (this is often the point of a qualitative piece)
Given the nature of this class, my preference is (1)
Some example models
Some example models
A counter-example
Preparation for next workshop
During our second in-person meeting, we will have another workshop session. Students will make a five-minute presentation about the draft model that they will examine for their dissertation work that builds upon their research topic/idea. Again, this can involve a few PowerPoint slides to help guide the discussion. This will be graded by the presence or absence of your preparation for this session. The balance of the workshop will be group work to discuss and advance those research questions, where I will move from group to group to facilitate discussions.
Preparation for next class
Next class
Elements III: Constructs and variables
Suddaby, R. (2010). Editor’s Comments: Construct Clarity in Theories of Management and Organization. Academy of Management Review, 35(3), 346-357. https://doi.org/10.5465/amr.35.3.zok346
Larsen, K, Bong, CH. 2016. A tool for addressing construct identity in literature reviews and meta-analyses. MIS Quarterly 40, No. 3: 529-552.
Bagozzi, R. P., Yi, Y., & Phillips, L. W. (1991). Assessing Construct Validity in Organizational Research. Administrative Science Quarterly, 36, No. 3, 421-458. https://doi.org/10.2307/2393203
Next class
Elements III: Constructs and variables
- Compare / Contrast
Schaffer, J. A., DeGeest, D., & Li, A. 2016. Tackling the problem of construct proliferation: A guide to assessing discriminant validity of conceptually related constructs. Organizational Research Methods, 19: 80- 110.
Law, K. S., Wong, C.-S., & Mobley, W. H. (1998). Toward a Taxonomy of Multidimensional Constructs. The Academy of Management Review, 23(4), 741. https://doi.org/10.2307/259060