| Sampling Design in Strategic Management Research on Firm Performance: An Analysis and Critique |
| | Short, Jeremy C. | Louisiana State U. | jshort@tiger.lsu.edu | (225)-388-6114 |
| | Ketchen, Jr., David J. | Louisiana State U. | mgketch@lsuvm.sncc.lsu.edu | 504-388-6140 |
| | This paper chronicles the use of sampling designs in the strategy literature for studies investigating
the determinants of firm performance, and explains basic sampling issues for researchers in strategic
management. A content analysis of 243 studies over a 19-year period reveals that strategy research
generally suffers from poor sampling decisions, most notably the use of nonrandom samples. Such
decisions may unnecessarily limit the generalizability of results, restricting the amount and quality of
evidence that could explain the determinants of firm performance. |
| | Keywords: sampling,; strategic management research,; firm performance studies |
| Increasing Replication in Strategy Research |
| | Singh, Kulwant | National U. of Singapore | fbaks@nus.edu.sg | (65)874-3134 |
| | Leong, Siew Meng | National U. of Singapore | fbalsm@nus.edu.sg | (65) 874-3172 |
| | Ang, Siah Hwee | National U. of Singapore | fbap6220@nus.edu.sg | (65)874-3143 |
| | The proposition that replication is essential to ensure the reliability and validity of research is well established. For strategy, a
relatively new and pre-paradigmatic social science dealing with complex phenomena, extensive replication is essential for
rigorous theory development. Yet, few replication studies in strategy have been published, a finding we establish empirically
through a two-stage evaluation of on-line bibliographic records, and in-depth analysis of leading strategy journals.
In this study, we focus on explaining this trend and on recommendations for correcting it. We attribute limited replication
research to explanations common to other social sciences, and to strategy specific factors. Strategy specific deterrents
include: (1) strategy's focus on firm level data, which is difficult to obtain; (2) the fact that many databases used in strategy
are private, which limits access; (3) strategy's focus on complex variables and relationships, which are difficult to isolate
and measure, making replication difficult; and (4) the youth and immaturity of strategy as a theoretical field suggests
greater need for original discovery rather than replication. Importantly, we argue that the explanations for limited replication
are precisely the same factors that dictate the need for more replication research.
We focus on three categories of recommendations for increasing replication research. First, we suggest adopting the concept
of the good-enough replication for strategy studies, which will address several of the field-specific constraints to replication
research. Second, we propose means for promoting replication research, and third we recommend ways for increasing the
publication of such research.
|
| | Keywords: Strategy; Replication; Research Methods |
| Using meta-analysis to detect interaction effects |
| | Griffin, Mark A. | Queensland U. of Technology | m.griffin@qut.edu.au | 61-7-3864-4264 |
| | Hart, Peter M. | U. of Melbourne, Australia | srchart@ozemail.com.au | 61-7-9344-6353 |
| | Shaw, Craig | U. of Queensland | craig@psy.uq.edu.au | 61 7 3365 6372 |
| | Interactions between variables are important in many areas of management research. However, the most common method
for detecting interactions has a number of limitations that mean the results from individual studies may not reflect the true
value of the interaction in the population. The example of interaction between stressors and social supports is used to illustrate
the wide variation in results that have been obtained when testing theoretical propositions about interactions. The example also
illustrates that most studies do not include sufficient information to conduct a meta-analysis of interaction effect sizes. In
particular the covariation between interaction terms and other measures is necessary to estimate the effect size of interactions
across studies but is typically not included in published research. This paper uses data from a large organizational study to
simulate the results of multiple studies. Meta-analysis procedures were used to estimate the effect size of the interaction term
across the simulated studies. The results of the meta-analysis closely mirrored the results obtained using the overall data from
the organizational study. The paper concludes that meta-analysis is a useful tool for testing interactions and that researchers
should include greater information about interaction terms when reporting individual studies. |
| | Keywords: meta-analysis,; moderated regression,; stress |
| The Control Group and Meta-Analysis |
| | Hunter, John E. | Michigan State U. | hunterj@PILOT.MSU.EDU | (517) 323-7927 |
| | Rodgers, Robert | U. of Kentucky | pub708@ukcc.uky.edu | (606)-427-4026 |
| | Many pretest-posttest studies that evaluate the effectiveness of organizational interventions do not use control groups. What are the implications of this problem when studies are evaluated using meta-analysis? Standard meta-analytic practice is to adjust the findings of all studies that use control group designs. What about the many studies - we estimate 89% - that do not use control groups? Standard practice is to enter the findings of such studies into a study data base without any adjustment for the effects of extraneous factors. If extraneous factors are systematically magnifying or diminishing the effects of an intervention, a terribly misleading result will be produced.
Our paper develops a novel meta-analytic procedure for estimating the effect of an organizational intervention which makes appropriate adjustments for the influence of extraneous factors. This procedure combines evidence on control group change with evidence on change from the intervention. The analysis makes it possible to adjust for the effects of extraneous factors in all studies that evaluate the effectivenss of an organizational intervention, including a vast majority of studies that do not have control groups. |
| | Keywords: Meta-Analysis,; Control Groups,; Organization Interventions |