C ommunications of the

A ssociation for
I nformation
S ystems

Essay DOI: 10.17705/1CAIS.044XX ISSN: 1529-3181

Don’t Throw the Baby Out With the Bathwater:

Comments on “Recent Developments in PLS”
Daniel Russo
Department of Computer Science, Aalborg University, Copenhagen, Denmark
[email protected]

Klaas-Jan Stol
School of Computer Science and Information Technology, University College Cork & Lero, Cork, Ireland
[email protected]

Abstract:

Evermann and Rönkkö aim to present an overview of recent advances in PLS, and while some advances are
described with several useful recommendations, we argue that their article does not fully deliver on its promise. In this
response, we argue that their position presents an unbalanced view, ignores several methodological advances by IS
scholars. We note that several recommendations are so stringent that implementing that there are philosophical and
practical differences that are insufficiently taken into account. Further, several studies that highlight the shortcomings
of PLS seem to be based on specially designed cases that are not necessarily representative of typical use of PLS. In
our response, we call for a more balanced debate that takes into consideration different perspectives and that studies
of the performance of PLS are conducted fairly. While we do not disagree with E&R’s recommendations, the
implementation of those is challenged by a lack of tool support, and we observe that besides scholars using PLS,
editors and reviewers also have a responsibility to be cognizant of methodological advances. We commend E&R for
their efforts in studying the limitations of PLS which have spurred several methodological advances, but also caution
that we should not ‘throw the baby out with the bathwater,” by discarding PLS for its known limitations.

Keywords: Structural Equation Modeling, Partial Least Squares.

[Department statements, if appropriate, will be added by the editors. Teaching cases and panel reports will have a
statement, which is also added by the editors.]
[Note: this page has no footnotes.]
This manuscript underwent [editorial/peer] review. It was received xx/xx/20xx and was with the authors for XX months for XX
revisions. [firstname lastname] served as Associate Editor. or The Associate Editor chose to remain anonymous.]

Volume 44 Paper XX pp. 123 – 133 January 2022

124 Error! Reference source not found.

1 Introduction

When [Copernican astronomy’s] predecessor, the Ptolemaic system, was first developed during
the last two centuries before Christ and the first two after, it was admirably successful in
predicting the changing positions of both stars and planets. No other ancient system had
performed so well; for the stars, Ptolemaic astronomy is still widely used today as an
engineering approximation; for the planets, Ptolemy’s predictions were as good as Copernicus’.
—Thomas Kuhn, The Structure of Scientific Revolutions

Throughout history, scholars have developed theories and models to explain and predict the world around
them. Claudius Ptolemy (c. 100 a.d. — c. 175 a.d.) developed theories to predict the position of heavenly
bodies and tide tables, which were based on the false premise that Earth was at the center of the
universe. Copernicus, on the other hand, correctly placed the Sun at the center, and thus his theories
should have performed better… except that his theory still made the wrong assumption that planetary
orbits were circular and thus did not necessarily reflect reality more accurately. In fact, in certain cases,
Ptolemy’s predictions were more accurate! Other examples of competing and incompatible theories in the
history of science include quantum mechanics vs. the theory of relativity, the wave theory of light vs. the
particle (corpuscular) theory of light.
We are reminded of these competing theories in the debates on the use of PLS, because PLS is typically
used as an alternative to the common factor based approach to structural equation modeling (SEM),
commonly referred to as covariance-based SEM (CB-SEM). PLS-SEM and CB-SEM employ different
representations of theoretical constructs; both can be used to evaluate structural equation models, but the
way they work is different.
In this commentary we respond to Evermann and Rönkkö’s (2021) article “Recent Developments in PLS.”
We recently completed a review of the use of PLS in the Software Engineering (SE) literature (Russo and
Stol, 2021); many of the studies that we reviewed, though published in quality SE journals and
conferences, seem to have been authored by IS researchers, and thus our observations would be directly
relevant to IS researchers also.
We applaud Evermann and Rönkkö (E&R) for taking on the major task of studying PLS as an approach to
evaluate structural equation models (SEM). It is essential that scholars stay informed about new
methodological advances, and that methods are used and reported appropriately. The mis-use and
misrepresentation of methods happens not only for quantitative methods; we also observed this problem
in the use of qualitative methods such as Grounded Theory (Stol et al., 2016). Methodological reviews
help to offer a mirror to the community to assess the state of practice of reporting our research and
identify areas for improvement.
E&R correctly observed that PLS was ascribed properties and benefits that had not been empirically
demonstrated. Several misconceptions exist around the PLS approach, many of which have been
debated in other critical articles (Aguirre-Urreta and Marakas, 2014; Henseler et al., 2014; Petter, 2018;
Rigdon et al., 2014). E&R’s concerns about the mis-use, reporting, and, evidently, reviewers’ lack of
knowledge of PLS are warranted. As our understanding of PLS has advanced over the years, it is
imperative that these are clearly communicated to the wider IS community who, as E&R pointed out, were
among the early adopters, and who have relied on its use to develop some well-known theories within the
IS literature.
And so, the tone of E&R’s abstract sounds very promising indeed:
“This paper comments on recent developments in PLS to ensure that IS researchers have up-
to-date methodological knowledge and best practices if they decide to use PLS.”
Except that the article does not deliver on this promise, in our view, for two reasons. First, discussion of
several recent developments that are relevant to IS scholars are notably lacking. Relevant work by Kock
and Lynn (2012) and Kock (2015a, 2015b 2018, 2019a, 2019b) offers several approaches to address
some issues in PLS-SEM. For example, Kock (2015a, 2019a) offers a new method to conduct factor-
based PLS-SEM analyses. A key issue underpinning the debate between proponents and detractors of

Volume 44 10.17705/1CAIS.044XX Paper XX

Communications of the Association for Information Systems 125

PLS-SEM is the representation of latent variables as composites, rather than as common factors as is the
case in CB-SEM. These two representations are incompatible. As Kock pointed out, the proposed factor-
based PLS-SEM, which is labeled PLSF-SEM, “could be a solid step in the resolution of this debate.”
A second reason why we argue that E&R’s article does not deliver is, what we perceive to be an overly
critical and polarizing position towards PLS, which has been extensively documented in their previous
articles (Rönkkö and Evermann, 2013; Rönkkö et al., 2015; Rönkkö et al., 2016; Rönkkö et al., 2021).
While we don’t disagree with the recommendations that E&R propose, we would argue that to truly
achieve their stated goal, namely to “ensure that IS researchers have up-to-date methodological
knowledge,” a more inclusive review of recent developments is needed, but also a less polarizing
presentation of the shortcomings of PLS.
In this commentary we seek to add some nuance to E&R’s points of critique. We do not share their critical
view on PLS; as scholars, we have used both PLS-SEM (Russo, 2021; Sharma and Stol, 2020) and CB-
SEM (Russo et al., 2021; Schaarschmidt and Stol, 2018), and we have no bias towards one or the other.
A central argument is that both PLS-SEM and CB-SEM are useful tools to IS and SE scholars, but that
both have strengths and weaknesses, much like both the Ptolemaic and Copernican systems. Despite the
weakness (indeed, incorrectness) of the Ptolemaic system, it remains to be useful. Hence, we argue,
“don’t throw the baby out with the bathwater!” Which ‘system’ to use (PLS-SEM, CB-SEM) depends on the
situation, context, and focus of the research (Evermann and Tate, 2016; Petter, 2018), although as we
point out later in our response, current guidelines are somewhat limited.
In what follows, we present a number of comments and observations (Sec. 2), and we conclude with a
number of recommendations to help the debate forward (Sec. 3).

2 Comments and Observations

We appreciate the detailed presentation that E&R offer on the technical foundation of PLS. They perceive
a number of threats to the validity of PLS and based on these they make a number of recommendations.
While we do not disagree with these recommendations, we do present four comments and observations
that, we believe, add nuance to their critique of PLS.

2.1 Toward a More Balanced Debate of PLS

We argue that E&R present a somewhat, might we say, “confusing” claim that they “do not intend to argue
for or against the use of PLS compared or other methods.” We believe that this is in effect what E&R’s
article does. The authors are openly critical towards the use of PLS:
“One has to wonder what, despite all these improvements, can be gained by using PLS over
well-established, better understood, and conceptually simpler alternatives? Again, this is best
left as a rhetorical question for the reader to ponder.”
First, we noticed a number of inconsistent statements. E&R have argued in their previous work against the
use of PLS (Rönkkö and Evermann, 2013; Rönkkö et al., 2015; Rönkkö et al., 2016; Rönkkö et al., 2021).
Similarly, E&R write in their “recent developments in PLS” paper that “Our critique has been
misinterpreted as denying that PLS is a SEM technique. However, we make no such claim [...]” Yet, one
of the “highlights” of Rönkkö et al.’s (2015) article states: “Partial least squares (PLS) is simply an
indicator weighting system and not SEM.” This is why we are somewhat left confused with E&R’s
statements. It would be curious if their earlier standpoint now changed dramatically—though we
acknowledge that everybody is entitled to revisit their opinions!
Second, while E&R state that “it is left to readers to come to their own conclusions” regarding the question
of whether or not to use PLS, we argue that their presentation is one-sided on the one-hand, and possibly
perceived as “overwhelming” on the other. We believe it is one-sided in that the reader is not presented all
information; certain assumptions underpinning E&R’s arguments have been left implicit. Further, as E&R
point out, the debate about PLS is not new, and several scholars have presented PLS in a more positive
light. However, counterarguments in support of PLS, or alternative viewpoints on PLS don’t seem to be
reflected in E&R’s article (cf. Rigdon, 2016; Rigdon et al., 2014; 2017). It might have been prudent to
include a summary of the debate thus far to give readers a full account of the issues and perspectives.
The argument is “overwhelming,” because we believe it is fair to say that the average IS or SE researcher
is not intimately familiar with many of the technical intricacies of the formulas underpinning PLS. By

Volume 44 10.17705/1CAIS.044XX Paper XX

126 Error! Reference source not found.

presenting a case that is quite technical in nature, supported by formulas, deductions, and results of
simulation studies (which many applied scholars using PLS-SEM, we reckon, would have difficulty
performing; we do appreciate Aguirre-Urreta & Rönkkö’s (2015) tutorial), E&R’s arguments seem quite
reasonable; the reader is “overwhelmed” by the arguments, and the formulaic presentation would suggest
that “it must be true because the math says so,” despite the fact that there have been reasonable counter-
arguments that question the assumptions that those formulas are based on (cf. Rigdon 2016).
Third, while E&R do not explicitly compare PLS-SEM to the best-known alternative, namely CB-SEM,
scholars interested in researching latent variables have few other options than CB-SEM, and sometimes
CB-SEM analyses do not work (e.g. models that don’t converge). We appreciate that E&R discuss
generalized structure component analysis (GSCA) as an alternative; perhaps a more in-depth comparison
between PLS-SEM and GSCA would have made a better topic, to inform IS researchers of viable
alternatives to PLS. That would have equally supported E&R’s intent of their article as stated in their
conclusion: “to help IS researchers remain current with important methodological developments and to
provide a sound foundation upon which to base their choice of statistical methods.”
Fourth, E&R take a rather hardline stance in their treatment of PLS, which we would argue contributes to
a polarization of the debate. We perceive E&R’s stance as negative (despite them denying it), and others
taking an opposite stance that we would characterize as, perhaps, too optimistic—as exemplified by Hair
et al.’s (2011) article entitled “PLS-SEM: Indeed A Silver Bullet.” We do not believe in silver bullets; in
statistical analysis, every methodological choice comes with affordances, but also at a cost: one has “to
pay the reaper” (Curran, 2003). We believe that taking hardline positions on these matters preempts a
more fruitful conversation. E&R were right to ask critical and important questions about a decade ago; it is
somewhat ironic that their initial critical perspective, which seemed to focus on discouraging scholars to
use PLS, has led to several improvements of PLS—clearly, their points of critique have been taken
seriously! However, we argue that a more nuanced conversational tone might be more useful to the
overall goal of understanding the limitations of PLS and offering solutions where possible.

2.2 Know Your Audience

E&R are clearly experts in the technical intricacies of PLS and CB-SEM, and as such they are able to
highlight the various differences to a great level of detail. Perhaps “blinded” by their own expertise, in their
collection of issues, they may have lost sight of their audiences. For example, we observed that E&R
wrote:
• “we, as a discipline, have staked our collective reputation on PLS and ‘bet the farm’ with very little
understanding.”
• “Ironically, these tests [e.g., SRMR] are largely ignored by IS researchers who tend to favor the
CR and AVE statistics despite their demonstrated incapability to differentiate good models from
bad ones.”
In these quotes, “we” and “IS researchers” point to the IS discipline as a whole. Our argument here is that
“we,” “we as a discipline,” and “researchers” are widely varying groups of people with different agendas
and expertise. We agree with the general sentiment that all scholars have a responsibility to study and
familiarize themselves with the methods they use—this is, after all, the core business of scholars in search
for some truth and understanding of the world. We would readily agree (from our experience reviewing
papers) that certainly not all scholars do this. However, we argue that those scholars who did do due
diligence do not believe they are “betting the farm.” PLS is but one method of many used in IS research,
and so claiming that the reputation of the IS discipline as a whole is at stake, is hyperbolic.
In a similar manner, E&R make several points and recommendations that seem to overestimate the
expertise of the average applied IS scholar using PLS.1 For example:
• “Given that simulation studies for PLS are easy to perform with modern PLS software tools...”
• “If a researcher uses PLS, adequate model fit should be established using dg prior to interpreting
estimates or assessing factorial structure” (Recommendation 6)

1
To be clear, we do not suggest that IS researchers using PLS are not experts at what they do, but based on our review (Russo and
Stol, 2021) we did find variation in expertise that was exhibited in the various papers.

Volume 44 10.17705/1CAIS.044XX Paper XX

Communications of the Association for Information Systems 127

• “If a researcher uses PLS for prediction, the study should adequately justify and reflect this
motivation and assess out-of-sample prediction at the individual level” (Recommendation 8)
• “If a researcher uses PLS with dependent formative constructs, researchers should verify and
publish the statistical model (i.e., the equations) that is assumed and estimated to avoid any
confusion” (Recommendation 10)
• “PLS composites should be compared to unweighted composites to demonstrate any possible
advantage that the PLS composites might have” (Recommendation 13)
All of these suggested procedures are rather advanced steps and not straightforward to perform—this is
what we mean that E&R may be ‘blinded’ by their own expertise. These procedures go well beyond the
typical introductory textbooks on PLS, though we agree that some tools may provide better support for
some of these than others. For example, Recommendation 10 (listed above) suggests that the equations
of the statistical model be published; while this may be easier to do in an open source package that
requires explicit specification of a model, other popular tools that offer a graphical interface do not readily
generate these equations.
Then there is the issue of reviewers, who have the same responsibility of familiarizing themselves with
recent methodological advances. For example, CR and AVE statistics, as well as an exploratory factor
analysis (EFA) to inspect crossloadings, were used for decades for establishing discriminant validity
(Bagozzi and Yi, 1988; Fornell and Larcker, 1981). Indeed, E&R rightly point out that these measures
have weaknesses, and that the HTMT criterion would be better. Yet, reviewers who do not keep up to
date may simply request that authors report CR, AVE, and results of an EFA out of habit, unaware of their
shortcomings. Authors who no longer report these criteria, may simply throw their hands up and comply
with reviewers’ requests, no matter how unreasonable or inappropriate, simply to progress their paper in
the review process. Thus, we cannot ‘blame’ researchers for using outdated measures without also
requiring due diligence from editors handling such papers.
For many researchers, the recommendations and advice by E&R are unrealistic and hard to follow.2 The
use of HTMT is not unproblematic because, as E&R point out, “its use with PLSc presents two
challenges,” all the while E&R’s Recommendation 1 was to use PLSc. Other recommendations are so
stringent that, in effect, E&R discourage the use of PLS altogether.

2.3 Philosophical and Practical Differences

There are clearly philosophical differences in how scholars perceive PLS (cf. Rigdon, 2016). Many
scholars who simply use data analysis methods neglect the differences between PLS-SEM and CB-SEM.
This is not dissimilar to how Principal Component Analysis (PCA) is often (mistakenly) mixed up with
factor analysis methods—even one of the most popular statistics packages, IBM SPSS, groups PCA with
“real” factor analysis methods in its user interface. For the outsider, the difference is trivial; whether a
theoretical construct is measured using a common factor model or as a composite, is then perceived as
just a minor issue without clear implications. While several scholars have written extensively on this
(Rigdon et al., 2017; Sarstedt et al., 2016), one argument that has been put forth is that a common factor
and a composite are two different ways to represent a theoretical construct (Rigdon, 2016). One could
even argue that the common factor model might not be an optimal or correct representation, either. For
example, latent variables are most commonly modeled as “reflective,” which implies that the unobserved
latent variable that the researcher believes to exist, causes the observed variables. This assumption does
not hold when dealing with certain concepts, such as socio-economic status (SES) which is commonly
measured by income, education, and occupational status (Bollen and Bauldry, 2011). It cannot be true
that the “unmeasured” concept SES causes income, education, and occupational status. A person does
not get higher wages, a better education, or a higher-status job because their SES rises; it is the other
way around. And thus, SES must be measured as a formative variable, something that leads to both
practical (Bollen and Davis, 2009) and theoretical challenges (Rigdon, 2016). For example, Rigdon
(2016), taking a realist perspective, argues that conceptual (latent) variables “transcend data” and “it is
impossible to ‘form’ conceptual variables out of data.” We only can construct proxy variables that we

2
In this respect, proper tool support is very important. While there are several commercial packages such as SmartPLS and
WarpPLS, we would argue that open source solutions may offer the best option to support transparency. One potential issue with
open source solutions is, however, the risk that packages disappear due to a lack of maintainers. This is what recently happened to
the plspm package for the R language.

Volume 44 10.17705/1CAIS.044XX Paper XX

128 Error! Reference source not found.

argue represent these conceptual variables. Who is to say, then, that the common factor based proxy is
better than a composite-based one?
E&R also question whether the weighted indicator representation used in PLS is appropriate as there has
been no demonstration of its benefits over simple unweighted summed scales. In effect, what it comes
down to is how latent variables are represented. There are several ways to generate factor scores
(DiStefano et al., 2009); PLS weighted indicators model is another. Unweighted scores assume that all
indicators (items) contribute equally, or are equally important, which may be an unreasonable assumption.
It is very common in certain types of models within the common factor SEM framework to use average or
sum scores of a set of indicators. For example, latent growth models (Bollen and Curran, 2006), which are
implemented within the CB-SEM framework, require simple scores (as opposed to multiple indicator
common factors), as does moderation analysis. With several ways to represent a latent variable with a
single score (DiStefano et al., 2009; Skondral and Laake, 2001), one should wonder how much it matters
at all? E&R wonder why the generalized structure component analysis (GSCA) approach has not seen
more widespread adoption; we do not have the answer to that question. Henseler (2012), who noted that
GSCA is a reinvention of Glang’s (1988) work, pointed out that all software implementations of the GSCA
algorithm suffered from an error (Henseler, 2012, pp. 403); clearly, this would not help instill any
confidence in this approach.
It is in this context that we believe it is worth emphasizing that no singular study is sufficient to build
rigorous theory or provide evidence for proposed theories. Much research within the IS discipline, but also
in many other disciplines that use SEM, focuses on latent, unmeasurable variables. Clear examples within
the IS literature are trust and job satisfaction; these constitute feelings, perceptions, and opinions of
people, and are inherently hard to measure. As a scholarly community we should never rely on a single
study that investigates important relationships between such variables. Instead, we should focus on
conducting families of studies, ideally using a variety of methods. The issue of how precise or reliable
some of the parameter estimates really are loses importance because other studies may either confirm or
disconfirm such findings.

2.4 Exceptional Cases and Flawed Evidence

Various articles that demonstrate the shortcomings of PLS use specific examples, either previous models
that were published, or simulations that highlight a poor performance of PLS (cf. Rönkkö, 2014). Clearly,
papers that seek to inform the research community about potential issues with methods are very important
and welcome. Specific models are used because they can be carefully studied and simulated, thus
pinpointing the exact problems.
However, there are two problems with using such studies as a foundation to ‘discard’ PLS. First, some of
these studies that seek to highlight issues such as the known bias in parameter estimates suffer from
design flaws (Henseler et al., 2014; Rigdon et al., 2014). For example, simulation studies rely on the
generation of some known truth, a known population from which samples are drawn to investigate
potential issues. Rigdon (2016) pointed out that these simulations defined their “truth” based on a
common factor model to evaluate PLS which uses composites to represent constructs. As others have
pointed out (Kock 2015a; 2019a; Rigdon, 2016), and as we briefly mentioned above as well, the
composite and factor-based representations are not compatible. This raises serious questions about the
validity of such simulation studies.
Second, demonstrating a potential issue does not mean that these issues are prevalent or of concern in
all studies using PLS-SEM. Thus, E&R’s suggestion that because “many of our main theories are built on
empirical research that uses PLS,” these theories are ‘invalid’ in some way is an overreaction in our view.
One fruitful avenue for further research could be to conduct several replications to investigate the validity
of these “main theories” using different analytical strategies. First, replications could use PLS-SEM while
following updated guidelines that include recent methodological advances in recent years, including the
recommendations proffered by E&R. Second, replications using CB-SEM, which models latent variables
differently, which would provide a realistic and interesting case to better understand differences between
PLS-SEM and CB-SEM. Third, replications following E&R’s recommendation to use generalized structure
component analysis (GSCA), which also would offer an interesting case study to increase our
understanding of benefits and drawbacks of both PLS-SEM and GSCA using realistic examples. Well-
designed replications can either give us more confidence in our theories if they remain supported, suggest
adjustments if they are not, and offer opportunities to investigate how results vary between different

Volume 44 10.17705/1CAIS.044XX Paper XX

Communications of the Association for Information Systems 129

analysis methods using realistic research models, rather than models that were designed to highlight
potential issues.

3 Conclusion and Takeaways

We wish to conclude our response, not by discarding all concerns that E&R have highlighted, but by
making a number of suggestions as a means to advance the debate on validity issues around PLS.
First, we believe the scholarly debate on the use and misuse of PLS has been a fruitful one (given the
advances in PLS in recent years); we recognize and appreciate E&R’s contributions in this matter.
However, the rhetoric of the debate and the arguments put forth are, at times, perhaps somewhat
“overheated” and hyperbolic. Petter (2018) observed that anti-PLS rhetoric has increased in recent years.
The seemingly extreme stances that some scholars take may not be beneficial to the overall debate; this
polarization will scare away some, while others will simply ignore the issues, hoping that reviewers aren’t
too bothered by it either. None of those positions are useful to the scientific endeavor.
Second, E&R have made several useful recommendations. We don’t argue with these, although as we
pointed out, some recommendations may be beyond the average IS scholar’s skillset. That is no excuse
of course, but it could very well be a barrier to the “proper” use of PLS, as advocated by E&R. Such
practical matters, which also include user-friendly and correct software implementations are clearly
important for the uptake of new advances (cf. Henseler, 2012). Several open source packages are freely
available such as the various libraries for the R language (including “seminr,” “semPLS,” and “matrixpls”)
and also for the Python language (e.g., “plspm”). Other users may prefer a graphical user interface offered
in commercial packages such as SmartPLS and WarpPLS. We note that using script-based analyses
(using R, Python or other languages) allows for better replication packages as every operation is explicitly
coded, whereas GUI-based tools do not allow for a “replay” of the analysis. While this may seem a minor
point, it does contribute to the transparency of analysis and inspection by readers, which in turn may help
give confidence in research findings.
Third, based on our arguments, we make a number of suggestions, summarized in Table 1 below. Central
to our suggestions is that a more balanced discussion is necessary. One-sided stances only lead to
polarization, which is not helpful to applied IS scholars because this forces them to “take sides,” fervently
defending their position with references to scientific work that offers support for one claim or another, in
order to get their work past reviewers. We hope that, with all the expertise that is clearly present within the
IS community, we can come together and make progress on this issue.

Table 1. Summary of comments and suggestions to advance the debate

Theme Summary Our Suggestions

Balancing the E&R’s aim to inform the IS community A more balanced and nuanced debate of evidence in
debate on PLS with updated guidelines is favor and against PLS-SEM would be welcome based on
commendable. However, in our view a fair discussion that includes all work that makes
treatment of recent methodological methodological advances. Extreme positions should be
advances is incomplete and somewhat prevented, or else a debate will not progress.
unbalanced.

Know Your E&R make important observations, but Some recommendations clearly target PLS-SEM users,
Audience many recommendations are hard to and offering further guidance to them is necessary,
implement for the average applied through editorials, tutorials, and textbooks for example.
scholar. Scholars, supervisors, editors Other recommendations propose rather technical
and reviewers all share a responsibility procedures, and we argue that the average applied
to remain informed of recent researcher is not readily equipped to perform these
methodological advances. Stating that without sufficient guidance. Other observations suggest a
the IS community has ‘bet the farm’ on “research roadmap” (see also our observations under the
using PLS-SEM to develop key theme “exceptional cases and flawed evidence”).
theories is hyperbolic.

Philosophical The way latent variables (theoretical Despite several debates around the use of PLS, the

Volume 44 10.17705/1CAIS.044XX Paper XX

130 Error! Reference source not found.

and practical constructs) are represented in different various epistemological and ontological perspectives are
differences approaches (PLS-SEM, CB-SEM) is perhaps not appreciated by different scholars. Debates
different, and it is important to such as these certainly help to highlight and explicate the
recognize the underpinning different stances, but a more elaborate in-depth
epistemological and ontological discussion that presents an “objective” comparison is
foundations. Practical issues involve needed, illustrated with concrete examples. Practical
the correct modeling of theoretical guidelines that address various issues such as modeling
constructs that are not consistent with ‘formative’ constructs are needed. A more clear
the standard reflectively modeled understanding of when to use PLS-SEM or CB-SEM is
common factor approach, such as needed; current guidelines that are typically offered by
socioeconomic status. advocates of PLS-SEM do not offer insight as to why
PLS-SEM would be more appropriate under certain
circumstances.

Exceptional E&R claim that their critiques are Comparisons must be valid and fair. Studies should
cases and based on deductions and simulations. consider not only extreme or exceptional cases, but also
flawed Exceptional cases that highlight include analyses of ‘normal’ studies. Study designs
evidence problems with PLS-SEM are not demonstrating problems should be sound. Potential
necessarily representative of general barriers to this are the fact that this work may be tedious,
use of PLS-SEM. As some scholars boring, and ‘high risk’, meaning that this type of work can
have shown, some simulation studies be very hard and may not result in publications in the top
highlighting ‘problem’ were flawed. venues; this is an inhibitor to junior scholars who do not
see this as a “career building” activity. Conducting several
types of replication studies to investigate foundational
studies is a potential avenue for future work.

Acknowledgments
We thank the Editor Dr John Venable for the invitation and constructive feedback that has led to a more
precise articulation of our points. This work was partially funded by the Science Foundation Ireland grants
15/SIRG/3293 and 13/RC/2094_P2.

References

Aguirre-Urreta, M.I., and Marakas, G.M. (2014) Research Note — Partial Least Squares and Models with
Formatively Specified Endogenous Constructs: A Cautionary Note. Information Systems Research, 25 (4),
761–78.
Aguirre-Urreta, M. & Rönkkö, M. (2015) Sample Size Determination and Statistical Power Analysis in PLS
Using R: An Annotated Tutorial. Communications of the Association for Information Systems, 36(3), article
3.
Bagozzi, R.P. and Yi, Y. (1988) On the evaluation of structural equation models. Journal of the Academy
of Marketing Science, 16(1), 74-94.
Bollen, K. & Bauldry, S. (2011) Three Cs in Measurement Models: Causal Indicators, Composite
Indicators, and Covariates. Psychol Methods, 16(3), 265-284
Bollen, K.A. & Davis, W.R. (2009) Causal indicator models: identification, estimation, and testing.
Structural Equation Modeling: A Multidisciplinary Journal, 16(3), 498-522.
Curran, P.J. (2003) Have multilevel models been structural equation models all along? Multivariate
Behavioral Research, 38(4), 529-569.
DiStefano, C., Zhu, M. & Mîndrilã, D. (2009) Understanding and Using Factor Scores: Considerations for
the Applied Researcher. Practical Assessment, Research, and Evaluation, 14, Article 20
Evermann, J., & Rönkkö, M. (2021) Recent Developments in PLS. Communications of the Association for
Information Systems, 50.

Volume 44 10.17705/1CAIS.044XX Paper XX

Communications of the Association for Information Systems 131

Evermann, J. & Tate, M. (2016) Assessing the predictive performance of structural equation model
estimators. Journal of Business Research, 69, 4565-4582
Fornell, C., & Larcker, D.F. (1981) Evaluating structural equation models with unobservable variables and
measurement error. Journal of Marketing Research, 18(1), 39-50.
Glang, M. (1988) Maximierung der Summe erklärter Varianzen in linear-rekursiven
Strukturgleichungsmodellen mit multiplen Indikatoren: Eine Alternative zum Schätzmodus B des Partial-
Least-Squares-Verfahrens (Engl.: Maximization of the Sum of Explained Variances in Linear-recursive
Structural Equation Models with Multiple Indicators: An Alternative to Mode B of the Partial Least Squares
Approach). PhD Thesis. University of Hamburg.
Hair, Joe F, Christian M Ringle, and Marko Sarstedt. (2011) PLS-SEM: Indeed a Silver Bullet. Journal of
Marketing Theory and Practice 19(2), 139–52.
Henseler, J. (2012) Why generalized structured component analysis is not universally preferable to
structural equation modeling. J. of the Acad. Mark. Sci., 40, 402–413
Henseler, J., Dijkstra, T.K., Sarstedt, M., Ringle, C.M., Diamantopoulos, A., Straub, D.W., Ketchen, D.J.,
Hair, J.F., Tomas, G., Hult, M. & Calantone, R.J. (2014) Common Beliefs and Reality About PLS:
Comments on Rönkkö and Evermann (2013). Organizational Research Methods, 17(2), 182–209.
Kock, N. (2015a) A note on how to conduct a factor-based PLS-SEM analysis. International Journal of e-
Collaboration, 11(3), 1-9.
Kock, N. (2015b) Common method bias in PLS ‐ SEM: A full collinearity assessment approach.
International Journal of e-Collaboration, 11(4), 1–10.
Kock, N. (2018) Should bootstrapping be used in PLS-SEM? Toward stable p-value calculation methods.
Journal of Applied Structural Equation Modeling, 2(1), 1-12.
Kock, N. (2019a). From composites to factors: Bridging the gap between PLS and covariance-based
structural equation modeling. Information Systems Journal, 29(3), 674-706.
Kock, N. (2019b) Factor-based structural equation modelling: going beyond PLS and composites.
International Journal of Data Analysis Techniques and Strategies, 11(1), 1-28
Kock, N. & Lynn, G.S. (2012) Lateral Collinearity and Misleading Results in Variance-Based SEM: An
Illustration and Recommendations. Journal of the Association for Information Systems, 13(7), 546-580.
Petter, S. (2018) “Haters gonna hate”: PLS and Information Systems Research. The DATA BASE for
Advances in Information Systems, 49(2), 10-13
Rigdon, Edward E, Jan-Michael Becker, Arun Rai, Christian M Ringle, Adamantios Diamantopoulos,
Elena Karahanna, Detmar W Straub, and Theo K Dijkstra. (2014) Conflating Antecedents and Formative
Indicators: A Comment on Aguirre-Urreta and Marakas. Information Systems Research 25(4), 780–84.
Rigdon, E.E. (2016) Choosing PLS path modeling as analytical method in European management
research: A realist perspective. European Management Journal, 34(6), 598-605
Rigdon, E.E, Sarstedt, M. & Ringle, C.M. (2017) On Comparing Results from CB-SEM and PLS-SEM:
Five Perspectives and Five Recommendations. Marketing Zfp 39 (3), 4–16.
Rönkkö, M. (2014) The effects of chance correlations on partial least squares path modeling.
Organizational Research Methods, 17(2), 164-181
Rönkkö, M., McIntosh, C.N., Antonakis, J. & Edwards, J.R. (2016) Partial Least Squares Path Modeling:
Time for Some Serious Second Thoughts. Journal of Operations Management 47–48 (1), 9–27.
Rönkkö, M., McIntosh, C.N., & Antonakis, J. (2015) On the Adoption of Partial Least Squares in
Psychological Research: Caveat Emptor. Personality and Individual Differences 87 (December), 76–84.
Rönkkö, M., and Evermann, J. (2013) A Critical Examination of Common Beliefs About Partial Least
Squares Path Modeling. Organizational Research Methods 16(3), 425-48.
Russo, D. & Stol, K. (2021) PLS-SEM in Software Engineering Research: A Survey and Introduction, ACM
Computing Surveys, 54(4).

Volume 44 10.17705/1CAIS.044XX Paper XX

132 Error! Reference source not found.

Russo, D, Hanel, H.P.P., Altnickel, S., van Berkel, N. (2021) Predictors of well-being and productivity
among software professionals during the COVID-19 pandemic–a longitudinal study. Empirical Software
Engineering, 26(4), 1-63.
Russo, D. (2021) The Agile Success Model: A Mixed Methods Study of a Large-Scale Agile
Transformation. ACM Transactions on Software Engineering and Methodology, 30(4).
Sarstedt, M., Hair, J.F., Ringle, C.M., Thiele, K.O., & Gudergan, S.P. (2016) Estimation Issues with PLS
and CBSEM: Where the Bias Lies! Journal of Business Research 69(10), 3998–4010.
Schaarschmidt, M. and Stol, K. (2018) Company soldiers and gone-natives: role conflict and career
ambition among firm-employed open-source developers. International Conference on Information
Systems.
Sharma, G.G. & Stol, K. (2020) Exploring Onboarding Success, Organizational Fit, and Turnover Intention
of Software Professionals. Journal of Systems and Software, 159, article no. 110442.
Skondral, A. & Laake, P. (2001) Regression among factor scores. Psychometrika, 66, 563-575.
Stol, K., Ralph, P. & Fitzgerald, B. (2016) Grounded Theory in Software Engineering Research: A Critical
Review and Guidelines. International Conference on Software Engineering.

Volume 44 10.17705/1CAIS.044XX Paper XX

Communications of the Association for Information Systems 133

About the Authors

Daniel Russo is an Associate Professor of Computer Science at Aalborg University in Copenhagen,
Denmark. His research and teaching interests are in the broad area of Empirical Software Engineering.
His research has been published in ACM Computing Surveys, ACM Transactions on Software
Engineering and Methodology, IEEE Transactions on Software Engineering, Empirical Software
Engineering, ACM Transactions on Management Information Systems, Semantic Web Journal, Journal of
Systems and Software, and in the Proceedings of the International Conference on Software Engineering
(ACM/IEEE ICSE), as also in the Proceedings of the International Conference on Human Factors in
Computing Systems (ACM CHI). His research specializes in studying the effects of the COVID-19
pandemic on IT professionals, Agile software development, and research methodology. He teaches
software engineering and systems development at both Bachelor's and Masters, and Structural Equation
Modelling at the Ph.D. level. He is also PI or co-PI of several research projects and collaborates with large
professional organizations, defense agencies, and multinational banking institutions. Corresponding autor.
Klaas-Jan Stol is a lecturer with the School of Computer Science and Information Technology at
University College Cork, Ireland. He received his PhD from the University of Limerick, Ireland. He is a
Funded Investigator with Lero, the Science Foundation Ireland Research Centre for Software. Stol’s
research interests include contemporary software development methods, as well as research
methodology. His work has appeared in several journals, including IEEE Transactions on Software
Engineering, ACM Transactions on Software Engineering and Methodology, ACM Computing Surveys,
Journal of Systems and Software, and IEEE Software. He has also presented his work at several
international flagship conferences such as the International Conference on Information Systems and the
International Conference on Software Engineering. He serves on several editorial boards.

Copyright © 2019 by the Association for Information Systems. Permission to make digital or hard copies of
all or part of this work for personal or classroom use is granted without fee provided that copies are not
made or distributed for profit or commercial advantage and that copies bear this notice and full citation on
the first page. Copyright for components of this work owned by others than the Association for Information
Systems must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on
servers, or to redistribute to lists requires prior specific permission and/or fee. Request permission to
publish from: AIS Administrative Office, P.O. Box 2712 Atlanta, GA, 30301-2712 Attn: Reprints or via e-
mail from [email protected].

Volume 44 10.17705/1CAIS.044XX Paper XX