Second Edition of the International Summer School on Research Methods in Business

08.07.2019

A scientific revolution is changing the way research is conducted in Social Sciences. This revolution is driven by an enhanced emphasis on methodological rigor and transparency, a pattern mimicking Hard Sciences.

In order to tackle this challenge, the Department of Economics and Management hosted the second edition of the International Summer School on Research Methods in Business, in collaboration with the Consortium for Advancement of Research Methods and Analysis CARMA. The summer school offers four intense (20-hours) courses on a variety of topics ranging from questionnaire design to structural equation modeling. The cohort of 40 attendees is quite heterogeneous ranging from Master students to PhD to junior faculties.

‘We launched this initiative in 2018 with a strong support from the members of Executive Committee of the PhD in Economics and Management – says Prof. Federica Ricceri, former deputyhead of the PhD programme – as we envision that gaining state of the art knowledge in research methods and data analysis is crucial to producing high quality research. The Summer School has benefited from generous financial support from the University of Padova and the Department of Economics and Management.

‘Indeed, funding is crucial to ensure our instructors are worldwide leading experts in the various domains of research methods – claims Prof. Amedeo Pugliese, scientific director of the Summer School. For example, one of the instructors in the 2019 edition is Prof. Paul Bliese (University of South Carolina, US) who serves as the current editor-in-chief of Organizational Research Methods, the leading scientific outlet in terms of research methods in business'. In addition to having Paul Bliese, the summer school features current and former editors of ORM: this is a sign of the high quality of the program, that should be a useful resource to all researchers in the field of social sciences.

We have asked Prof. Bliese a few questions in relation to the relevance of research methods.

What is changing in the research methods domain: why do many talk about a ‘credibility revolution’?
Many factors underly concerns about credibility of scientific research. On the extreme are those rare but well-known cases where researchers fabricate data or results. I think the more common problem, however, is that statistics can be somewhat non-intuitive when it comes to expectations about replication. For instance, in statistics we often ask: “how likely is it that I would get this observed value simply by chance?”. If the answer is low (e.g., “I would only have a 5% chance of observing this value simply by chance”) we say we have a statistically significant result.
Interestingly, however, if we have a study that just meets this 5% rule, we have about a 50% chance that someone who tries to replicate our finding and uses our same sample size and procedures will fail. On the one hand, this may seem like a credibility problem.
In my mind, however, it represents the fact that we often have unrealistic expectations about replication in part because it is non-intuitive that a finding that would have occurred by chance 5% of the time only has a 50% chance of being replicated.

Which are the benefits of the Summer Schools or courses like this one?
Short courses such as those provided through CARMA play a key role in helping advance the scientific rigor of organizational research. Learning to effectively use statistics is an iterative process. That is, it is very hard to master statistical techniques with just a single exposure from a statistics class, so short courses provide two important things for students and faculty. First, they provide participants with exposure to new areas of statistics, and second they provide an opportunity for participants to hear statistics explained from different instructors using different examples. This process of exposure and re-exposure is key to helping advance rigor within the field.

Which are the main risks researchers in the business domain are facing? We are in an era in which the standards set by the hard sciences are extremely high in terms of rigor, transparency and data quality: should we apply them in the social sciences, in spite of the underlying differences in the object/subject of study? From your privileged position as Editor at ORM, would you share with us your view?
I don’t know if I have any particular privileged view as Editor – from my vantage point I see researchers and authors who are strongly committed to advancing best practices in organizational research (both for quantitative and qualitative research) and ORM plays a key role in making this research accessible to a broad audience. With respect to transparency, organizational researchers may face some unique challenges with making data widely available. For instance, firms often partner with researcher to collectively tackle problems, and many firms are open to publishing results in ways that protect the identity of the firm. At the same time, the firms are understandably reluctant to make their data open. I also see, in terms of rigor, that organizational researchers may have fewer opportunities to implement randomized trials and randomly assign firms or other entities to condition. Therefore, it can be more difficult for organizational researchers to make strong causal claims. Despite these issues, I think the rigor is strong and researchers are making important advances by applying analytics to organizational data.