Sums of Squares and Cross Products Matrix

This lesson introduces the sums of squares and cross products matrix (aka, SSCP matrix). We show how to use matrix methods to compute the SSCP matrix, using both raw scores and deviation scores.

Sum of Squares: Vectors

In statistics, many formulas require the calculation of sums of squares; i.e., squaring all of the elements in a set and then taking the sum of those squares.

Using matrix algebra, the sum of squares for all the elements of a vector is calculated according to the following formula:

Σ xi2 = x'x

where

x is an n x 1 column vector of scores: x1, x2, . . . , xn
Σ xi2 is the sum of the squared values from vector x

To illustrate, let's find the sum of squares for the elements of vector x, where x' = [ 1 2 3 ].

Σ xi2   =    [ 1 2 3 ]   
1
2
3
      =    ( 1 * 1 ) + ( 2 * 2 ) + ( 3 * 3 )    =    1 + 4 + 9   =   14
x' x

Thus, the sum of the squared elements from vector x is 14.

Sums of Squares and Cross Products: Matrices

With matrices, we can compute not only sums of squares but also sums of cross products. For an r x c matrix, an individual cross product is represented by XrjXrk. The sum of cross products between all the elements of columns j and k is represented by Σ XrjXrk, summed over r. A matrix of sums of squares and sums of cross products is represented by XX, as shown below.

XX    =   
Σ X12   Σ X1 X2   . . .   Σ X1 Xc
Σ X2 X1   Σ X22   . . .   Σ X2 Xc
. . .   . . .   . . .   . . .
Σ Xc X1   Σ Xc X2   . . .   Σ Xc2

where

X is an r x c matrix of raw scores: X11, X12, . . . , Xrc
XX is a c x c matrix of sums of squares and sums of cross products
Σ Xi2 is the sum of the square of all elements in column i of matrix X
Σ Xi Xj is the sum of cross products produced by multiplying each element in column i of matrix X with the corresponding element from column j and summing the result

Thus, the diagonal elements of XX are sums of squares, and the off-diagonal elements are cross products. Note that the cross product matrix XX is a symmetric matrix.

See problem 1 for an example showing how to create a cross product matrix.

Sums of Squares and Deviation Scores

In the previous lesson, we showed how to transform a matrix of raw scores into a matrix of deviation scores. There are advantages to working with deviation scores.

  • Computations can be easier.
  • Equations are often more comprehensible.

As a result, researchers often transform their raw data into deviation scores before they calculate sums of squares and cross products. This is such a common practice that the term "sums of squares" has two meanings. It can refer to raw score sums of squares or to deviation score sums of squares.

See problem 2 for an example showing how to create a cross product matrix, using deviation scores.

Test Your Understanding of This Lesson

Problem 1

Using matrix A below, find the cross products matrix defined by matrix AA.

A   =   
90 60 90
90 90 30
60 60 60
60 60 90
30 30 30

Solution 1

AA   =   
90 90 60 60 30
60 90 60 60 30
90 30 60 90 30
   
90 60 90
90 90 30
60 60 60
60 60 90
30 30 30
   =   
24,300 21,600 20,700
21,600 19,800 18,000
18,000 15,600 21,600
 

Problem 2

Matrix A, shown below, is a matrix of raw scores. Find the deviation score sums of squares matrix produced from matrix A; that is, find matrix aa.

A   =   
90 60 90
90 90 30
60 60 60
60 60 90
30 30 30

Solution 2

First, we transform raw score matrix A into deviation score matrix a, as shown below. Previously, we described how to transform raw scores to deviation scores. We repeat the transformation formula below, and then we make the transformation.

a = A - 11'A ( 1 / r )

where

1 is an 5 x 1 column vector of ones
a is an 5 x 3 matrix of deviation scores: a11, a12, . . . , a53
A is an 5 x 3 matrix of raw scores: A11, A12, . . . , A53
r is the number of rows in matrix A

a   =   
90 60 90
90 90 30
60 60 60
60 60 90
30 30 30
   -   
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
   
90 60 90
90 90 30
60 60 60
60 60 90
30 30 30
    ( 1/5 )

a   =   
90 60 90
90 90 30
60 60 60
60 60 90
30 30 30
   -   
66 60 60
66 60 60
66 60 60
66 60 60
66 60 60
   =   
24 0 30
24 30 -30
-6 0 0
-6 0 30
-36 -30 -30

Then, to find the deviation score sums of square matrix, we simply compute a'a, as shown below.

aa   =   
24 24 -6 -6 36
0 30 0 0 -30
30 -30 0 30 -30
   
24 0 30
24 30 -30
-6 0 0
-6 0 30
-36 -30 -30
   =   
2520 1800 900
1800 1800 0
900 0 3600