Navigate
Courses Tutorials Practice Pricing Tools Upgrade
Lesson 2 of 8

Factor Analysis

In Lesson 1, PCA took Rosa's four flower measurements and compressed them into a couple of readable directions. It answered one question: which directions capture the most spread? Factor analysis answers a different, deeper one: what hidden things could have produced these measurements in the first place?

Meet Priya, a school counselor. At the end of term she has percentage scores for 250 students in six subjects: reading, vocabulary and essay writing (English), and arithmetic, algebra and geometry (maths). When she lines the columns up and looks at how they move together, a pattern jumps out, shown in the grid below. The three English subjects rise and fall together. The three maths subjects rise and fall together. But an English score tells you almost nothing about a maths score.

Priya never gave a test called "language ability" or "number ability". Yet it is as if two such hidden abilities are quietly steering all six columns. Factor analysis is the tool that finds them.

By the end of this lesson you will be able to:

  • Say what a latent (hidden) factor is, and how the common-factor model explains a set of correlations
  • Run factor analysis in R, then read the loadings, communalities and uniquenesses
  • Decide how many factors to keep, rotate them so you can name them, and say how all this differs from PCA

Prerequisites: Lesson 1, PCA in R (correlation, loadings, standardizing, variance explained). You can run R and read its output. No linear algebra is assumed; every term is defined as it appears.

Two green blocks, and cold blue-white everywhere else. That two-block shape is the fingerprint we are about to explain.

The clue

The clue is in the correlations

Let us make Priya's data concrete so you can compute on it. Each lesson runs in a fresh R session, so we build the marks right here (run this once). Two hidden abilities, one number per student, are turned into six subject scores, each with its own random noise. You never see the two abilities; you only see the six columns they produce.

RInteractive R
set.seed(1) n <- 250 language <- rnorm(n) # a hidden "language ability", one value per student (unseen) number <- rnorm(n) # a hidden "number ability" (unseen) # Each ability drives some subjects strongly, plus that subject's own noise: make <- function(load, ability) round(pmin(100, pmax(0, 62 + 15 * (load * ability + rnorm(n, 0, 0.62))))) scores <- data.frame( reading = make(0.80, language), vocabulary = make(0.75, language), essay = make(0.70, language), arithmetic = make(0.78, number), algebra = make(0.82, number), geometry = make(0.72, number) ) head(scores, 3) #> reading vocabulary essay arithmetic algebra geometry #> 1 55 41 66 64 72 66 #> 2 61 56 74 76 58 67 #> 3 41 41 45 82 69 43

  

  





Now look at how the six columns move together:




  

    
RInteractive R

    
round(cor(scores), 2)
#>            reading vocabulary essay arithmetic algebra geometry
#> reading       1.00       0.57  0.51       0.13    0.04     0.09
#> vocabulary    0.57       1.00  0.51       0.03    0.01     0.04
#> essay         0.51       0.51  1.00       0.13    0.04     0.09
#> arithmetic    0.13       0.03  0.13       1.00    0.64     0.63
#> algebra       0.04       0.01  0.04       0.64    1.00     0.62
#> geometry      0.09       0.04  0.09       0.63    0.62     1.00

    

      
      
    

    

  

  





Inside the English block, correlations sit around 0.5. Inside the maths block, around 0.6. Across the two blocks, they collapse to near zero. Six columns, but really only two "clumps" of shared movement.



Key Insight
Correlation that clusters into blocks is the tell-tale sign of hidden structure. Something the three English subjects share, and something different the three maths subjects share, is making each block move as one. Naming those two somethings is the whole job of factor analysis.