Exploratory Factor Analysis vs Principal Components: from concept to application | by Rafael Valdece Sousa Bastos | Jan, 2021


How to scale back parameters with Exploratory Factor Analysis

Rafael Valdece Sousa Bastos
Photo by on

In knowledge science, we frequently need to measure variables comparable to social-economic standing (SES). Some variables have numerous parameters (or pieces), for instance, SES can also be measured according to source of revenue, schooling, and so on. Then, to continue with the research, it is not uncommon to scale back the selection of parameters to fewer elements via Principal Components Analysis (PCA). However, we will be able to see why some variables can’t be lowered by PCA and we will be able to learn the way to use Exploratory Factor Analysis in our want.

Model distinction

Both of them are used to scale back the selection of parameters to fewer variables. Also, each strategies think that the variance of a parameter is split into explicit variance, commonplace variance, and blunder variance.

In PCA, after we retain an element, we consider each explicit variance and commonplace variance. While in EFA we best consider commonplace variance. Seeing the following determine, we will be able to suppose that A’s are explicit variances, B is the typical variance, and C’s are error variances. In PCA we use A’s + B whilst in EFA we best use B.

Example of the variance of 3 pieces and its members of the family with a hypothetical ingredient. Image by creator.

PCA is according to the formative style, the place the adaptation within the element is according to the adaptation in merchandise responses (i.e. degree of source of revenue will impact the social-economic standing). While EFA is according to the reflective style, the place the adaptation of the pieces is according to the adaptation of a assemble (i.e. an individual’s happiness will alternate their reaction to the pieces, now not the opposite). We can see this illustration with the next determine.

Difference between PCA and EFA. Image by creator.

With that being mentioned, the typical application of EFA is to measure mental variables. For instance, if you need to measure an individual’s degree of happiness we will be able to use best the typical variance for the reason that pieces of the tool are attempting to measure what they have got in commonplace (i.e. the extent of happiness).

PCA has most commonly 3 major steps.

  1. Compute the covariance matrix
  2. Compute eigenvalues and eigenvectors
  3. Rotation of elements

While in EFA we’ve got:

  1. Verification of information adequacy
  2. Computation of covariance/correlation matrix (Factor Extraction)
  3. Selection of things to retain
  4. Rotation of things

Since there are numerous posts right here on Towards Data Science referring to PCA, I will be able to center of attention on EFA from now on. In the following segment, I will be able to describe each and every step from EFA.

Data adequacy

We most often use two checks to measure if our knowledge is good enough to continue with EFA.

Bartlett’s take a look at of sphericity

This take a look at verifies the speculation that variables aren’t correlated within the inhabitants. Therefore, the null speculation is that the correlation matrix is equivalent to an id matrix. If the correlation matrix is equivalent to an id matrix, we can not continue with EFA, since there’s no correlation between variables. The statistical research at the back of this take a look at is going as follows:

χ² =- [(n-1)-(2v+5)/6]ln|R|

n is the pattern measurement
v is the selection of variables
|R| is the determinant of the correlation matrix

In the literature, we will be able to see that if the extent of importance equals p < 0.05 that implies we will be able to continue with EFA.

Kaiser-Meyer-Olkin (KMO)

Verify the share of variance of things that may be led to by elements. The take a look at verifies if the inverse correlation matrix is shut to a diagonal matrix, evaluating the values of linear correlations with values of partial correlations.


rjk = correlation coefficient between Xj and Xk

pjk = correlation coefficient between Xj and Xk, controlling for different Xs.

Values below 0.Five are thought to be unacceptable, between 0.50 and nil.70 are mediocre, between 0.70 and nil.80 are just right, above 0.80 are thought to be nice and above 0.90 are very good.

Factor Extraction

In EFA we’ve got numerous strategies of extraction to make a choice from. If knowledge are typically allotted, it is strongly recommended to use most chance, because it allows a number of goodness of have compatibility indices, importance take a look at of ingredient loadings, calculation of self assurance durations, and so on. However, if the knowledge doesn’t observe an ordinary distribution, it is strongly recommended to use major axis factoring.

Retaining Factors

There are numerous strategies we will be able to use to choose the selection of pieces, I will be able to center of attention basically on 3 of them.

a) Kaiser criterion: it proposes if an element’s eigenvalue is above 1.0, we must retain that ingredient. The common sense at the back of it’s: if an element has an eigenvalue = 3.0, that implies that the ingredient explains an identical quantity of variance as Three pieces.

Watch out, this criterion is understood to over and underestimate the selection of elements. I might now not counsel the use of it by myself.

b) Scree plot: the place we overview when there’s a considerable decline within the magnitude of the eigenvalues. This way additionally has some boundaries, as a result of it could generate ambiguous effects and are open to subjective interpretation.

c) Parallel research: the eigenvalue of the pattern and eigenvalue of random knowledge are calculated. The selection of elements is chosen when the selection of eigenvalues of actual knowledge is greater than from simulated knowledge. This way most often works neatly.

Now we will be able to see an academic for computing an EFA in R. We will probably be the use of the package deal to make our computations. Also, we will be able to be the use of the , which is composed of things referring to unfavourable and sure feelings.

Reading knowledge

We will learn the dataset the use of the R serve as learn.delim

Affects <- learn.delim(“")

The title we make a choice for our knowledge body was once Affets. Using View(Affects) we will be able to see our dataset:

Loading psych package deal

To get started manipulating knowledge, we will be able to want to obtain the psych package deal. Just run the next code.

set up.applications(“psych”)

Ok. Now the package deal is in your laptop. Now, we want to make it get started running with library(psych) .

Data adequacy

To see if it’s adequate to do EFA with our knowledge, we will be able to first calculate Bartlett’s take a look at of sphericity.

First, we calculate the correlation matrix:

correlation <- cor(Affects)

Then, we will be able to calculate Bartlett’s take a look at with cortest.bartlett(correlation, n=1033) , the place the second one argument is the pattern measurement. We can have the next output:

The importance degree was once smaller than 0.05, this means that we will be able to continue with EFA (if we think values below 0.05 point out the adequacy of our knowledge).

Now we will be able to compute KMO. Use the next code KMO(Affects) .

We can see that the full Measure of Sample Adequacy (MSA) was once 0.94, this means that that it’s very good. We too can see the MSA of each and every merchandise under the full MSA. Based on those effects, we will be able to continue with EFA.

Extracting and maintaining elements

Using just one line of code, we will be able to be ready to extract the selection of elements and choose which elements we’re going to retain.

fa.parallel(Affects,fm=”pa”, fa=”fa”,  major = “Parallel Analysis Scree Plot”, n.iter=500)


  1. the primary argument is our knowledge body
  2. fm is the extraction way; we’re the use of major axis factoring (“pa”)
  3. fa = “fa” ; if we write “fa” we will be able to be doing an EFA. If we write “laptop” we will be able to be doing a PCA. Since we’re the use of the reflective style, we will be able to be doing an EFA.
  4. major = identify of our symbol.
  5. n.iter = selection of interactions we want to do.

We can have the next output:

It tells us that we want to retain Three elements according to Parallel Analysis. Another output from the similar line of code is the following determine:

We can see our knowledge eigenvalues (in blue) and the simulated knowledge eigenvalues (in purple). Look on the intersection between the purple and blue eigenvalues and spot that the fourth eigenvalue of a few random knowledge explains extra variance than the fourth eigenvalue from our knowledge. However, if we believe the Kaiser criterion (black line crossing the determine), we will be able to be extracting simply two elements. Since we would like to retain as few elements as imaginable, we will be able to continue with the research with best two elements.

Before we bounce to the following segment, I need to display you the variation between PCA and EFA when extracting the eigenvalues.

fa.parallel(Affects, fm = "pa", cor = "cor", fa= "each", major = "Parallel Analysis Scree Plot", n.iter=500)

In the determine above, PC represents major elements and FA represents ingredient research. If we use the Kaiser criterion in PCA, we will be able to retain 3 elements, whilst if we use Parallel Analysis criterion, we will be able to retain 2 elements. We too can realize that eigenvalues of PCA are upper than from the EFA, that’s as a result of that factor I mentioned sooner than, PCA takes into consideration commonplace and explicit variance, whilst EFA best takes commonplace variance.

Item knowledge

We can see which pieces give an explanation for extra of the ingredient according to ingredient loadings, commonalities. Also, with the next code we will be able to see defined variance and goodness of have compatibility indices of our two-factor style:

have compatibility <- fa(Affects, nfactors = 2, n.obs = 1033 , rotate = “oblimin”, residuals = TRUE, fm = “pa”)print(have compatibility, kind = TRUE)

That prints out:

In the primary desk, we will be able to see that Positive Affects Items (AP’s) load strongly at the first ingredient (PA1), whilst Negative Affects Items (AN’s) load strongly on the second one ingredient (PA2). The commonality is represented by h2.

Bellow the primary desk, we’ve got the share of defined variance for each and every ingredient. The ingredient of Positive Affects (PA1) defined 25% of the knowledge variance, whilst the Negative Affects (PA2) defined 24%.

The goodness of have compatibility indices could also be calculated. Although they’re broadly utilized in Confirmatory Factor Analysis, their use in EFA is somewhat of a thriller. There are few research that overview the usage of goodness of have compatibility indices in EFA, subsequently it can be tricky to interpret this a part of the knowledge.

To download a visible illustration of the ingredient loadings, we will be able to use the serve as fa.diagram(have compatibility).

We too can see a geometrical visualization of things of their axis with plot(have compatibility) .

In the graphic above, we would like our Positive Affects pieces to be nearer in combination whilst staying a ways from the cluster of Negative Affects pieces.

Given all steps thus far, we will be able to see that our measure introduced a two-factor construction, which is equal to was once theorized by earlier authors. In addition, we discovered that pieces loaded of their given elements, and the criteria defined a significant portion of the variance.

I should spotlight that, even supposing we’ve got statistical strategies to choose the selection of elements to retain, best the researcher can make a choice which way would be the perfect one. That approach the criterion to choose the most efficient way to retain elements is open to subjective interpretation. Also, I introduced a statistical software to analyze mental knowledge the place the reflective style is extra good enough.

Feel unfastened to touch me by

Gmail: rafavsbastos@gmail.com
Website for consulting and partnerships:




Please enter your comment!
Please enter your name here