class: center, middle, inverse, title-slide # Factor analysis ## Research methods ### Jüri Lillemets ### 2021-12-11 --- class: center middle clean # What are the underlying factors behind variables? --- class: center middle inverse # Intuition --- ## Factor analysis (FA) and principal component analysis (PCA) In PCA we attempt to maximize the **total variation in data**. In factor analysis we instead try to maximize some **underlying shared variation between variables**. -- PCs are uniquely defined while **factors depend on estimation and rotation method**. -- In PCA each PC is a linear combination of variables but **in FA the variables are expressed as linear combinations of factors**. --- What loads what? .pull-left[ <div id="htmlwidget-1c8b82f5895e78145fcf" style="width:400px;height:400px;" class="grViz html-widget"></div> <script type="application/json" data-for="htmlwidget-1c8b82f5895e78145fcf">{"x":{"diagram":"digraph pc {\n rankdir=LR\n graph [fontname = RobotoCondensed, fontsize = 8]\n node [shape = box] x_1; x_2; x_3; x_4; x_5\n node [shape = circle] PC_1; PC_2\n \n {x_1 x_2 x_3 x_4 x_5} -> PC_1\n {x_1 x_2 x_3 x_4 x_5} -> PC_2\n }","config":{"engine":"dot","options":null}},"evals":[],"jsHooks":[]}</script> ] .pull-right[ <div id="htmlwidget-c18bad36630c5558782c" style="width:400px;height:400px;" class="grViz html-widget"></div> <script type="application/json" data-for="htmlwidget-c18bad36630c5558782c">{"x":{"diagram":"digraph {\n rankdir = LR\n graph [fontname = RobotoCondensed, fontsize = 8]\n node [shape = circle] x_1; x_2; x_3; x_4; x_5\n node [shape = box] F_1; F_2; u_1; u_2; u_3; u_4; u_5\n \n {rank = same x_1; x_2; x_3; x_4; x_5}\n {rank = same u_1; u_2; u_3; u_4; u_5}\n \n F_1 -> {x_1 x_2 x_3 x_4 x_5}\n F_2 -> {x_1 x_2 x_3 x_4 x_5}\n \n x_1 -> u_1 [dir=back]\n u_2 -> x_2\n u_3 -> x_3\n u_4 -> x_4\n u_5 -> x_5\n }","config":{"engine":"dot","options":null}},"evals":[],"jsHooks":[]}</script> ] --- ### Should we use PCA or FA? If our goal would be to **summarize variables** into a smaller set of variables, then we should use PCA. If the goal is to **determine some underlying variables**, then we should use FA. --- ## Orthogonal factor model Variables are loaded by factors. Each variable can thus be represented as a linear combination of factor loadings as follows: `$$x_j = q_{jl} F_l + U_j +\mu_j,$$` where - `\(x_j\)` is a variable `\(j\)`, - `\(q_{jl}\)` the loading of the `\(j\)`th variable on the `\(l\)`th common factor, - `\(F_l\)` the `\(l\)`th common factor, - `\(U_j\)` the `\(j\)`th specific factor and - `\(\mu_j\)` the mean of `\(x_j\)`. --- class: center middle inverse # Estimation --- ## Estimating the model The idea behind FA is to **explain the common variance of varibles by extracting factors from the correlations between variables**. -- There are three methods for extracting factors: - minimum residuals, - maximum likelihood and - principal axis method. --- A common example of factors are **personality traits**: we can not measure personality directly but we can ask subjects various questions and then use factor analysis to summarize the responses into factors that represent these traits. -- Let's use data on 25 personality items of 2800 people. .compact[ | | A1| A2| A3| A4| A5| C1| C2| C3| C4| C5| E1| E2| E3| E4| E5| N1| N2| N3| N4| N5| O1| O2| O3| O4| O5| gender| education| age| |:-----|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|------:|---------:|---:| |61617 | 2| 4| 3| 4| 4| 2| 3| 3| 4| 4| 3| 3| 3| 4| 4| 3| 4| 2| 2| 3| 3| 6| 3| 4| 3| 1| NA| 16| |61618 | 2| 4| 5| 2| 5| 5| 4| 4| 3| 4| 1| 1| 6| 4| 3| 3| 3| 3| 5| 5| 4| 2| 4| 3| 3| 2| NA| 18| |61620 | 5| 4| 5| 4| 4| 4| 5| 4| 2| 5| 2| 4| 4| 4| 5| 4| 5| 4| 2| 3| 4| 2| 5| 5| 2| 2| NA| 17| |61621 | 4| 4| 6| 5| 5| 4| 4| 3| 5| 5| 5| 3| 4| 4| 4| 2| 5| 2| 4| 1| 3| 3| 4| 3| 5| 2| NA| 17| |61622 | 2| 3| 3| 4| 5| 4| 4| 5| 3| 2| 2| 2| 5| 4| 5| 2| 3| 4| 4| 3| 3| 3| 4| 3| 3| 1| NA| 17| |61623 | 6| 6| 5| 6| 5| 6| 6| 6| 1| 3| 2| 1| 6| 5| 6| 3| 5| 2| 2| 3| 4| 3| 5| 6| 1| 2| 3| 21| ] - ... - `gender` Males = 1, Females = 2 - `education` 1 = HS, 2 = finished HS, 3 = some college, 4 = college graduate 5 = graduate degree - `age` age in years --- The item data were collected using a 6 point response scale where 1 is very inaccurate and 6 is very accurate. .smaller[ - `A1` Am indifferent to the feelings of others. (q_146) - `A2` Inquire about others' well-being. (q_1162) - `A3` Know how to comfort others. (q_1206) - `A4` Love children. (q_1364) - `A5` Make people feel at ease. (q_1419) - `C1` Am exacting in my work. (q_124) - `C2` Continue until everything is perfect. (q_530) - `C3` Do things according to a plan. (q_619) - `C4` Do things in a half-way manner. (q_626) - `C5` Waste my time. (q_1949) - `E1` Don't talk a lot. (q_712) - `E2` Find it difficult to approach others. (q_901) - `E3` Know how to captivate people. (q_1205) - `E4` Make friends easily. (q_1410) - `E5` Take charge. (q_1768) - `N1` Get angry easily. (q_952) - `N2` Get irritated easily. (q_974) - `N3` Have frequent mood swings. (q_1099) - `N4` Often feel blue. (q_1479) - `N5` Panic easily. (q_1505) - `O1` Am full of ideas. (q_128) - `O2` Avoid difficult reading material.(q_316) - `O3` Carry the conversation to a higher level. (q_492) - `O4` Spend time reflecting on things. (q_1738) - `O5` Will not probe deeply into a subject. (q_1964) ] --- In psychology *Big Five personality traits* is a suggested taxonomy of psychological attributes that includes the following traits: - agreeableness (friendly/compassionate vs. critical/rational) - conscientiousness(efficient/organized vs. extravagant/careless) - extraversion(outgoing/energetic vs. solitary/reserved) - neuroticism (sensitive/nervous vs. resilient/confident) - openness (inventive/curious vs. consistent/cautious) -- So we assume that there are 5 latent factors hidden behind the accuracy of statements. --- Let's see what are the loadings if we estimate the factor model without rotation. .compact[ | | MR1| MR2| MR3| MR4| MR5| |:----------------------------------------|-----:|----:|-----:|-----:|-----:| |Am indifferent to the feelings of others | NA| NA| NA| NA| -0.36| |Inquire about others' well-being | 0.46| NA| NA| NA| 0.35| |Know how to comfort others | 0.53| 0.30| NA| NA| 0.30| |Love children | 0.40| NA| NA| NA| NA| |Make people feel at ease | 0.57| NA| NA| NA| NA| |Am exacting in my work | 0.32| NA| 0.45| NA| NA| |Continue until everything is perfect | 0.33| NA| 0.48| NA| NA| |Do things according to a plan | 0.31| NA| 0.36| 0.30| NA| |Do things in a half-way manner | -0.46| NA| -0.44| NA| NA| |Waste my time | -0.49| NA| NA| NA| NA| |Don't talk a lot | -0.40| NA| NA| NA| NA| |Find it difficult to approach others | -0.61| NA| NA| NA| 0.32| |Know how to captivate people | 0.53| 0.33| NA| NA| NA| |Make friends easily | 0.58| NA| -0.33| NA| NA| |Take charge | 0.51| NA| NA| NA| NA| |Get angry easily | -0.44| 0.62| NA| NA| NA| |Get irritated easily | -0.42| 0.61| NA| NA| NA| |Have frequent mood swings | -0.41| 0.61| NA| NA| NA| |Often feel blue | -0.52| 0.40| NA| NA| NA| |Panic easily | -0.34| 0.42| NA| NA| NA| |Am full of ideas | 0.33| NA| NA| -0.36| NA| |Avoid difficult reading material | NA| NA| NA| 0.37| NA| |Carry the conversation to a higher level | 0.41| NA| NA| -0.44| NA| |Spend time reflecting on things | NA| NA| NA| NA| NA| |Will not probe deeply into a subject | NA| NA| NA| 0.42| NA| ] --- ## Rotations We can apply a rotation on factors **to better distinguish factors**. The result of rotation is an **adjusted loadings matrix, where factors are more distinct and thus more intuitive to interpret**. While rotation introduces some subjectivity as there are no rules about the choice of rotation, it often **eases description of factors**. There are many rotations to choose from but we look at three: varimax, oblimin and quartimax. --- ### Varimax This rotation maximizes variances of squared loadings **within factors**. **Each factor has few variables with high loadings**. Factors are **uncorrelated**. .compact[ | | MR2| MR1| MR3| MR5| MR4| |:----------------------------------------|----:|-----:|-----:|-----:|-----:| |Am indifferent to the feelings of others | NA| NA| NA| -0.41| NA| |Inquire about others' well-being | NA| NA| NA| 0.62| NA| |Know how to comfort others | NA| 0.32| NA| 0.64| NA| |Love children | NA| NA| NA| 0.42| NA| |Make people feel at ease | NA| 0.39| NA| 0.53| NA| |Am exacting in my work | NA| NA| 0.54| NA| NA| |Continue until everything is perfect | NA| NA| 0.65| NA| NA| |Do things according to a plan | NA| NA| 0.55| NA| NA| |Do things in a half-way manner | NA| NA| -0.61| NA| NA| |Waste my time | 0.30| NA| -0.55| NA| NA| |Don't talk a lot | NA| -0.57| NA| NA| NA| |Find it difficult to approach others | NA| -0.68| NA| NA| NA| |Know how to captivate people | NA| 0.54| NA| NA| NA| |Make friends easily | NA| 0.65| NA| 0.30| NA| |Take charge | NA| 0.50| 0.32| NA| NA| |Get angry easily | 0.77| NA| NA| NA| NA| |Get irritated easily | 0.75| NA| NA| NA| NA| |Have frequent mood swings | 0.73| NA| NA| NA| NA| |Often feel blue | 0.59| -0.33| NA| NA| NA| |Panic easily | 0.54| NA| NA| NA| NA| |Am full of ideas | NA| NA| NA| NA| 0.50| |Avoid difficult reading material | NA| NA| NA| NA| -0.45| |Carry the conversation to a higher level | NA| 0.30| NA| NA| 0.59| |Spend time reflecting on things | NA| NA| NA| NA| 0.37| |Will not probe deeply into a subject | NA| NA| NA| NA| -0.54| ] --- ### Oblimin This rotation also involves maximization of squared loading variances **within factors** but with possibly **correlated** factors. .compact[ | | MR2| MR1| MR3| MR5| MR4| |:----------------------------------------|----:|-----:|-----:|-----:|-----:| |Am indifferent to the feelings of others | NA| NA| NA| -0.41| NA| |Inquire about others' well-being | NA| NA| NA| 0.64| NA| |Know how to comfort others | NA| NA| NA| 0.66| NA| |Love children | NA| NA| NA| 0.43| NA| |Make people feel at ease | NA| NA| NA| 0.53| NA| |Am exacting in my work | NA| NA| 0.55| NA| NA| |Continue until everything is perfect | NA| NA| 0.67| NA| NA| |Do things according to a plan | NA| NA| 0.57| NA| NA| |Do things in a half-way manner | NA| NA| -0.61| NA| NA| |Waste my time | NA| NA| -0.55| NA| NA| |Don't talk a lot | NA| -0.56| NA| NA| NA| |Find it difficult to approach others | NA| -0.68| NA| NA| NA| |Know how to captivate people | NA| 0.42| NA| NA| NA| |Make friends easily | NA| 0.59| NA| NA| NA| |Take charge | NA| 0.42| NA| NA| NA| |Get angry easily | 0.81| NA| NA| NA| NA| |Get irritated easily | 0.78| NA| NA| NA| NA| |Have frequent mood swings | 0.71| NA| NA| NA| NA| |Often feel blue | 0.47| -0.39| NA| NA| NA| |Panic easily | 0.49| NA| NA| NA| NA| |Am full of ideas | NA| NA| NA| NA| 0.51| |Avoid difficult reading material | NA| NA| NA| NA| -0.46| |Carry the conversation to a higher level | NA| NA| NA| NA| 0.61| |Spend time reflecting on things | NA| -0.32| NA| NA| 0.37| |Will not probe deeply into a subject | NA| NA| NA| NA| -0.54| ] --- ### Quartimax This rotation aims to maximize variances of factor loadings **within variables**. The result is a factor structure in which **each variable is loaded by only few factors**. .compact[ | | MR1| MR2| MR3| MR5| MR4| |:----------------------------------------|-----:|----:|-----:|-----:|-----:| |Am indifferent to the feelings of others | NA| NA| NA| -0.39| NA| |Inquire about others' well-being | NA| NA| NA| 0.59| NA| |Know how to comfort others | 0.40| NA| NA| 0.59| NA| |Love children | NA| NA| NA| 0.39| NA| |Make people feel at ease | 0.46| NA| NA| 0.47| NA| |Am exacting in my work | NA| NA| 0.54| NA| NA| |Continue until everything is perfect | NA| NA| 0.65| NA| NA| |Do things according to a plan | NA| NA| 0.55| NA| NA| |Do things in a half-way manner | NA| NA| -0.60| NA| NA| |Waste my time | NA| 0.30| -0.54| NA| NA| |Don't talk a lot | -0.58| NA| NA| NA| NA| |Find it difficult to approach others | -0.70| NA| NA| NA| NA| |Know how to captivate people | 0.57| NA| NA| NA| NA| |Make friends easily | 0.68| NA| NA| NA| NA| |Take charge | 0.52| NA| 0.30| NA| NA| |Get angry easily | NA| 0.79| NA| NA| NA| |Get irritated easily | NA| 0.77| NA| NA| NA| |Have frequent mood swings | NA| 0.73| NA| NA| NA| |Often feel blue | -0.37| 0.56| NA| NA| NA| |Panic easily | NA| 0.52| NA| NA| NA| |Am full of ideas | NA| NA| NA| NA| 0.49| |Avoid difficult reading material | NA| NA| NA| NA| -0.44| |Carry the conversation to a higher level | 0.32| NA| NA| NA| 0.59| |Spend time reflecting on things | NA| NA| NA| NA| 0.38| |Will not probe deeply into a subject | NA| NA| NA| NA| -0.53| ] ??? The difference from Varimax rotation is that factors are not necessarily orthogonal but oblique, thus can be correlated. The resulting factors may be more similar to each other than in case of Varimax but the variances of loadings within factors are likely to be higher. --- How to think about rotations? <!-- --> --- class: center middle inverse # Assumptions --- ### Sphericity **Variables need to be correleated** to a certain extent for us to able to summarize them into a smaller set of factors. If data cloud on scatter plot is spherical, the correlations in data are random and factor analysis is thus not accurate. Whether or not correlation matrix is statistically significantly different from zero (identity matrix) can be tested using *Bartlett's test*. `\(H_0:\)` Correlation matrix is zero `\(H_1:\)` Correlation matrix is different from zero If the test returns `\(p \ge 0.05\)`, then we should not trust the results of factor analysis. --- .compact[ | | A1| A2| A3| A4| A5| C1| C2| C3| C4| C5| E1| E2| E3| E4| E5| N1| N2| N3| N4| N5| O1| O2| O3| O4| O5| |:--|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:| |A1 | 1.0| -0.3| -0.3| -0.1| -0.2| 0.0| 0.0| 0.0| 0.1| 0.1| 0.1| 0.1| 0.0| -0.1| 0.0| 0.2| 0.1| 0.1| 0.1| 0.0| 0.0| 0.1| -0.1| -0.1| 0.1| |A2 | -0.3| 1.0| 0.5| 0.3| 0.4| 0.1| 0.1| 0.2| -0.1| -0.1| -0.2| -0.2| 0.3| 0.3| 0.3| -0.1| -0.1| 0.0| -0.1| 0.0| 0.1| 0.0| 0.2| 0.1| -0.1| |A3 | -0.3| 0.5| 1.0| 0.4| 0.5| 0.1| 0.1| 0.1| -0.1| -0.2| -0.2| -0.3| 0.4| 0.4| 0.3| -0.1| -0.1| 0.0| -0.1| 0.0| 0.1| 0.0| 0.2| 0.0| 0.0| |A4 | -0.1| 0.3| 0.4| 1.0| 0.3| 0.1| 0.2| 0.1| -0.2| -0.2| -0.1| -0.2| 0.2| 0.3| 0.2| -0.1| -0.1| -0.1| -0.2| 0.0| 0.1| 0.0| 0.1| 0.0| 0.0| |A5 | -0.2| 0.4| 0.5| 0.3| 1.0| 0.1| 0.1| 0.1| -0.1| -0.2| -0.2| -0.3| 0.4| 0.5| 0.3| -0.2| -0.2| -0.1| -0.2| -0.1| 0.2| 0.0| 0.2| 0.0| 0.0| |C1 | 0.0| 0.1| 0.1| 0.1| 0.1| 1.0| 0.4| 0.3| -0.3| -0.2| 0.0| -0.1| 0.1| 0.1| 0.2| -0.1| 0.0| 0.0| -0.1| -0.1| 0.2| -0.1| 0.2| 0.1| -0.1| |C2 | 0.0| 0.1| 0.1| 0.2| 0.1| 0.4| 1.0| 0.4| -0.4| -0.3| 0.0| -0.1| 0.2| 0.1| 0.2| 0.0| 0.0| 0.0| 0.0| 0.1| 0.2| 0.0| 0.2| 0.1| -0.1| |C3 | 0.0| 0.2| 0.1| 0.1| 0.1| 0.3| 0.4| 1.0| -0.3| -0.3| 0.0| -0.1| 0.1| 0.1| 0.2| -0.1| -0.1| -0.1| -0.1| 0.0| 0.1| 0.0| 0.1| 0.0| 0.0| |C4 | 0.1| -0.1| -0.1| -0.2| -0.1| -0.3| -0.4| -0.3| 1.0| 0.5| 0.1| 0.2| -0.1| -0.1| -0.2| 0.2| 0.2| 0.2| 0.3| 0.2| -0.1| 0.2| -0.1| 0.1| 0.2| |C5 | 0.1| -0.1| -0.2| -0.2| -0.2| -0.2| -0.3| -0.3| 0.5| 1.0| 0.1| 0.3| -0.2| -0.2| -0.2| 0.2| 0.2| 0.2| 0.3| 0.2| -0.1| 0.1| -0.1| 0.1| 0.1| |E1 | 0.1| -0.2| -0.2| -0.1| -0.2| 0.0| 0.0| 0.0| 0.1| 0.1| 1.0| 0.5| -0.3| -0.4| -0.3| 0.0| 0.0| 0.1| 0.2| 0.1| -0.1| 0.0| -0.2| 0.1| 0.1| |E2 | 0.1| -0.2| -0.3| -0.2| -0.3| -0.1| -0.1| -0.1| 0.2| 0.3| 0.5| 1.0| -0.4| -0.5| -0.4| 0.2| 0.2| 0.2| 0.3| 0.3| -0.2| 0.1| -0.2| 0.2| 0.1| |E3 | 0.0| 0.3| 0.4| 0.2| 0.4| 0.1| 0.2| 0.1| -0.1| -0.2| -0.3| -0.4| 1.0| 0.4| 0.4| -0.1| -0.1| 0.0| -0.1| -0.1| 0.3| -0.1| 0.4| 0.1| -0.1| |E4 | -0.1| 0.3| 0.4| 0.3| 0.5| 0.1| 0.1| 0.1| -0.1| -0.2| -0.4| -0.5| 0.4| 1.0| 0.3| -0.1| -0.1| -0.1| -0.3| -0.1| 0.1| 0.1| 0.2| -0.1| 0.1| |E5 | 0.0| 0.3| 0.3| 0.2| 0.3| 0.2| 0.2| 0.2| -0.2| -0.2| -0.3| -0.4| 0.4| 0.3| 1.0| 0.0| 0.0| -0.1| -0.2| -0.1| 0.3| -0.1| 0.3| 0.0| -0.1| |N1 | 0.2| -0.1| -0.1| -0.1| -0.2| -0.1| 0.0| -0.1| 0.2| 0.2| 0.0| 0.2| -0.1| -0.1| 0.0| 1.0| 0.7| 0.6| 0.4| 0.4| -0.1| 0.1| -0.1| 0.1| 0.1| |N2 | 0.1| -0.1| -0.1| -0.1| -0.2| 0.0| 0.0| -0.1| 0.2| 0.2| 0.0| 0.2| -0.1| -0.1| 0.0| 0.7| 1.0| 0.5| 0.4| 0.4| 0.0| 0.1| 0.0| 0.1| 0.0| |N3 | 0.1| 0.0| 0.0| -0.1| -0.1| 0.0| 0.0| -0.1| 0.2| 0.2| 0.1| 0.2| 0.0| -0.1| -0.1| 0.6| 0.5| 1.0| 0.5| 0.4| 0.0| 0.1| 0.0| 0.2| 0.1| |N4 | 0.1| -0.1| -0.1| -0.2| -0.2| -0.1| 0.0| -0.1| 0.3| 0.3| 0.2| 0.3| -0.1| -0.3| -0.2| 0.4| 0.4| 0.5| 1.0| 0.4| 0.0| 0.1| -0.1| 0.2| 0.0| |N5 | 0.0| 0.0| 0.0| 0.0| -0.1| -0.1| 0.1| 0.0| 0.2| 0.2| 0.1| 0.3| -0.1| -0.1| -0.1| 0.4| 0.4| 0.4| 0.4| 1.0| -0.1| 0.2| -0.1| 0.1| 0.1| |O1 | 0.0| 0.1| 0.1| 0.1| 0.2| 0.2| 0.2| 0.1| -0.1| -0.1| -0.1| -0.2| 0.3| 0.1| 0.3| -0.1| 0.0| 0.0| 0.0| -0.1| 1.0| -0.2| 0.4| 0.2| -0.2| |O2 | 0.1| 0.0| 0.0| 0.0| 0.0| -0.1| 0.0| 0.0| 0.2| 0.1| 0.0| 0.1| -0.1| 0.1| -0.1| 0.1| 0.1| 0.1| 0.1| 0.2| -0.2| 1.0| -0.3| -0.1| 0.3| |O3 | -0.1| 0.2| 0.2| 0.1| 0.2| 0.2| 0.2| 0.1| -0.1| -0.1| -0.2| -0.2| 0.4| 0.2| 0.3| -0.1| 0.0| 0.0| -0.1| -0.1| 0.4| -0.3| 1.0| 0.2| -0.3| |O4 | -0.1| 0.1| 0.0| 0.0| 0.0| 0.1| 0.1| 0.0| 0.1| 0.1| 0.1| 0.2| 0.1| -0.1| 0.0| 0.1| 0.1| 0.2| 0.2| 0.1| 0.2| -0.1| 0.2| 1.0| -0.2| |O5 | 0.1| -0.1| 0.0| 0.0| 0.0| -0.1| -0.1| 0.0| 0.2| 0.1| 0.1| 0.1| -0.1| 0.1| -0.1| 0.1| 0.0| 0.1| 0.0| 0.1| -0.2| 0.3| -0.3| -0.2| 1.0| ] P-value here is 0. > Are the correlations significantly different from zero? --- ### Sampling adequacy In addition to correlations it is necessary that **each factor is not related to only few variables**. We can use Kaiser-Meyer-Olkin (KMO) measure to determine if sum of partial correlations is higher than sum of correlations or not. The value of the measure is interpreted as follows regarding the suitability of FA. - 0 ... 0.6 - Not suitable - 6 ... 0.7 - Suitable - 0.7 ... 0.9 - Adequate - 0.9 ... 1 - Excellent -- .compact[ | A1| A2| A3| A4| A5| C1| C2| C3| C4| C5| E1| E2| E3| E4| E5| N1| N2| N3| N4| N5| O1| O2| O3| O4| O5| |----:|----:|----:|----:|---:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:| | 0.74| 0.84| 0.87| 0.87| 0.9| 0.83| 0.79| 0.85| 0.82| 0.86| 0.83| 0.88| 0.89| 0.87| 0.89| 0.78| 0.78| 0.86| 0.88| 0.86| 0.85| 0.78| 0.84| 0.76| 0.76| ] > Is a factor analysis with these varables adequate? --- class: center middle inverse # Interpretation --- ## Loadings Loadings represent the **influence factors have on variables.** .compact[ | | MR2| MR1| MR3| MR5| MR4| |:----------------------------------------|----:|-----:|-----:|-----:|-----:| |Am indifferent to the feelings of others | NA| NA| NA| -0.41| NA| |Inquire about others' well-being | NA| NA| NA| 0.62| NA| |Know how to comfort others | NA| 0.32| NA| 0.64| NA| |Love children | NA| NA| NA| 0.42| NA| |Make people feel at ease | NA| 0.39| NA| 0.53| NA| |Am exacting in my work | NA| NA| 0.54| NA| NA| |Continue until everything is perfect | NA| NA| 0.65| NA| NA| |Do things according to a plan | NA| NA| 0.55| NA| NA| |Do things in a half-way manner | NA| NA| -0.61| NA| NA| |Waste my time | 0.30| NA| -0.55| NA| NA| |Don't talk a lot | NA| -0.57| NA| NA| NA| |Find it difficult to approach others | NA| -0.68| NA| NA| NA| |Know how to captivate people | NA| 0.54| NA| NA| NA| |Make friends easily | NA| 0.65| NA| 0.30| NA| |Take charge | NA| 0.50| 0.32| NA| NA| |Get angry easily | 0.77| NA| NA| NA| NA| |Get irritated easily | 0.75| NA| NA| NA| NA| |Have frequent mood swings | 0.73| NA| NA| NA| NA| |Often feel blue | 0.59| -0.33| NA| NA| NA| |Panic easily | 0.54| NA| NA| NA| NA| |Am full of ideas | NA| NA| NA| NA| 0.50| |Avoid difficult reading material | NA| NA| NA| NA| -0.45| |Carry the conversation to a higher level | NA| 0.30| NA| NA| 0.59| |Spend time reflecting on things | NA| NA| NA| NA| 0.37| |Will not probe deeply into a subject | NA| NA| NA| NA| -0.54| ] --- Let's now see the result with interpretations for each factor. .compact[ | | Neuroticism| Extraversion| Conscientiousness| Agreeableness| Opennness| |:----------------------------------------|-----------:|------------:|-----------------:|-------------:|---------:| |Am indifferent to the feelings of others | NA| NA| NA| -0.41| NA| |Inquire about others' well-being | NA| NA| NA| 0.62| NA| |Know how to comfort others | NA| 0.32| NA| 0.64| NA| |Love children | NA| NA| NA| 0.42| NA| |Make people feel at ease | NA| 0.39| NA| 0.53| NA| |Am exacting in my work | NA| NA| 0.54| NA| NA| |Continue until everything is perfect | NA| NA| 0.65| NA| NA| |Do things according to a plan | NA| NA| 0.55| NA| NA| |Do things in a half-way manner | NA| NA| -0.61| NA| NA| |Waste my time | 0.30| NA| -0.55| NA| NA| |Don't talk a lot | NA| -0.57| NA| NA| NA| |Find it difficult to approach others | NA| -0.68| NA| NA| NA| |Know how to captivate people | NA| 0.54| NA| NA| NA| |Make friends easily | NA| 0.65| NA| 0.30| NA| |Take charge | NA| 0.50| 0.32| NA| NA| |Get angry easily | 0.77| NA| NA| NA| NA| |Get irritated easily | 0.75| NA| NA| NA| NA| |Have frequent mood swings | 0.73| NA| NA| NA| NA| |Often feel blue | 0.59| -0.33| NA| NA| NA| |Panic easily | 0.54| NA| NA| NA| NA| |Am full of ideas | NA| NA| NA| NA| 0.50| |Avoid difficult reading material | NA| NA| NA| NA| -0.45| |Carry the conversation to a higher level | NA| 0.30| NA| NA| 0.59| |Spend time reflecting on things | NA| NA| NA| NA| 0.37| |Will not probe deeply into a subject | NA| NA| NA| NA| -0.54| ] --- We can visualize the loadings as well. <!-- --> --- And we can add some variables to the visualization. <!-- --> --- ## Uniqueness and communality These measures characterize **each variable** in FA. *Uniqueness* is a measure of proportion of variation **explained by a particular variable and not by factors**. *Communality* is the inverse of this, i.e. the **variation of a variable that is shared with other variables**. ??? Communality is the variation of a variable that is shared with other variables. --- Recall the factor model `\(x_j = q_{jl} F_l + U_j +\mu_j\)`. .compact[ | | Neuroticism| Extraversion| Conscientiousness| Agreeableness| Opennness| Uniqueness| Mean| |:----------------------------------------|-----------:|------------:|-----------------:|-------------:|---------:|----------:|----:| |Am indifferent to the feelings of others | NA| NA| NA| -0.41| NA| 0.81| 2.41| |Inquire about others' well-being | NA| NA| NA| 0.62| NA| 0.55| 4.80| |Know how to comfort others | NA| 0.32| NA| 0.64| NA| 0.48| 4.60| |Love children | NA| NA| NA| 0.42| NA| 0.72| 4.70| |Make people feel at ease | NA| 0.39| NA| 0.53| NA| 0.54| 4.56| |Am exacting in my work | NA| NA| 0.54| NA| NA| 0.67| 4.50| |Continue until everything is perfect | NA| NA| 0.65| NA| NA| 0.55| 4.37| |Do things according to a plan | NA| NA| 0.55| NA| NA| 0.68| 4.30| |Do things in a half-way manner | NA| NA| -0.61| NA| NA| 0.55| 2.55| |Waste my time | 0.30| NA| -0.55| NA| NA| 0.57| 3.30| |Don't talk a lot | NA| -0.57| NA| NA| NA| 0.65| 2.97| |Find it difficult to approach others | NA| -0.68| NA| NA| NA| 0.46| 3.14| |Know how to captivate people | NA| 0.54| NA| NA| NA| 0.56| 4.00| |Make friends easily | NA| 0.65| NA| 0.30| NA| 0.47| 4.42| |Take charge | NA| 0.50| 0.32| NA| NA| 0.60| 4.42| |Get angry easily | 0.77| NA| NA| NA| NA| 0.35| 2.93| |Get irritated easily | 0.75| NA| NA| NA| NA| 0.40| 3.51| |Have frequent mood swings | 0.73| NA| NA| NA| NA| 0.45| 3.22| |Often feel blue | 0.59| -0.33| NA| NA| NA| 0.51| 3.19| |Panic easily | 0.54| NA| NA| NA| NA| 0.65| 2.97| |Am full of ideas | NA| NA| NA| NA| 0.50| 0.69| 4.82| |Avoid difficult reading material | NA| NA| NA| NA| -0.45| 0.74| 2.71| |Carry the conversation to a higher level | NA| 0.30| NA| NA| 0.59| 0.54| 4.44| |Spend time reflecting on things | NA| NA| NA| NA| 0.37| 0.75| 4.89| |Will not probe deeply into a subject | NA| NA| NA| NA| -0.54| 0.70| 2.49| ] --- ## Complexity This can be calculated for **each variable** and it expresses the **number of factors which load the variable**. .compact[ | | Neuroticism| Extraversion| Conscientiousness| Agreeableness| Opennness| Complexity| |:----------------------------------------|-----------:|------------:|-----------------:|-------------:|---------:|----------:| |Am indifferent to the feelings of others | NA| NA| NA| -0.41| NA| 1.29| |Inquire about others' well-being | NA| NA| NA| 0.62| NA| 1.38| |Know how to comfort others | NA| 0.32| NA| 0.64| NA| 1.56| |Love children | NA| NA| NA| 0.42| NA| 2.17| |Make people feel at ease | NA| 0.39| NA| 0.53| NA| 2.04| |Am exacting in my work | NA| NA| 0.54| NA| NA| 1.29| |Continue until everything is perfect | NA| NA| 0.65| NA| NA| 1.15| |Do things according to a plan | NA| NA| 0.55| NA| NA| 1.10| |Do things in a half-way manner | NA| NA| -0.61| NA| NA| 1.45| |Waste my time | 0.30| NA| -0.55| NA| NA| 1.79| |Don't talk a lot | NA| -0.57| NA| NA| NA| 1.11| |Find it difficult to approach others | NA| -0.68| NA| NA| NA| 1.35| |Know how to captivate people | NA| 0.54| NA| NA| NA| 2.03| |Make friends easily | NA| 0.65| NA| 0.30| NA| 1.56| |Take charge | NA| 0.50| 0.32| NA| NA| 2.17| |Get angry easily | 0.77| NA| NA| NA| NA| 1.21| |Get irritated easily | 0.75| NA| NA| NA| NA| 1.14| |Have frequent mood swings | 0.73| NA| NA| NA| NA| 1.03| |Often feel blue | 0.59| -0.33| NA| NA| NA| 1.82| |Panic easily | 0.54| NA| NA| NA| NA| 1.41| |Am full of ideas | NA| NA| NA| NA| 0.50| 1.55| |Avoid difficult reading material | NA| NA| NA| NA| -0.45| 1.53| |Carry the conversation to a higher level | NA| 0.30| NA| NA| 0.59| 1.66| |Spend time reflecting on things | NA| NA| NA| NA| 0.37| 2.61| |Will not probe deeply into a subject | NA| NA| NA| NA| -0.54| 1.09| ] --- ## Scores As for PCA, we can calculate a score of each factor for each observation. .compact[ | | gender| education| age| MR2| MR1| MR3| MR5| MR4| |:-----|------:|---------:|---:|------:|------:|------:|------:|------:| |61617 | 1| NA| 16| -0.428| 0.062| -1.223| -0.679| -1.477| |61618 | 2| NA| 18| 0.071| 0.574| -0.629| -0.200| -0.256| |61620 | 2| NA| 17| 0.533| 0.288| 0.027| -0.751| 0.235| |61621 | 2| NA| 17| -0.176| 0.045| -1.002| 0.053| -1.064| |61622 | 1| NA| 17| -0.320| 0.406| -0.061| -0.821| -0.685| |61623 | 2| 3| 21| 0.226| 1.069| 1.323| -0.107| 0.309| |61624 | 1| NA| 18| -1.211| -0.056| -0.054| 0.108| 0.828| |61629 | 1| 2| 19| 0.309| -1.391| -0.771| -1.838| -0.331| |61630 | 1| 1| 19| NA| NA| NA| NA| NA| |61633 | 2| NA| 17| 0.909| 0.748| 1.202| 0.130| 0.114| |61634 | 1| 1| 21| -0.145| 0.280| -0.181| -0.077| -0.282| |61636 | 1| NA| 16| NA| NA| NA| NA| NA| |61637 | 2| NA| 16| -1.196| -0.706| 0.117| -0.121| 0.001| |61639 | 1| NA| 16| -1.497| 0.767| 0.155| 0.176| -0.823| |61640 | 1| 1| 17| -0.422| -0.009| 0.777| -1.647| -0.243| |61643 | 1| NA| 17| 1.116| 1.403| -0.108| -0.448| -0.053| |61650 | 2| NA| 17| 0.820| 0.839| -0.713| 0.352| 0.469| |61651 | 1| NA| 17| 1.445| 1.212| 0.341| -0.473| -0.424| |61653 | 2| NA| 16| 1.192| 0.775| -0.430| -0.946| -0.625| |61654 | 2| NA| 17| 0.270| 2.046| -2.577| -0.667| -0.130| ] -- We can use these scores, for example, to see how social status is related to personality. --- Gender <!-- --> --- Education <!-- --> --- Age <!-- --> --- ## FA is a controversial method We can never be sure if the factors actually exist and were correctly estimated. This is due to the different options a researcher to choose - the number of factors, - method of estimating the model, - rotation technique and - interpreting factors. --- --- class: center middle inverse # Practical application --- Use the data set `gss_spending`. >Find the hidden factors behind respondents' attitudes toward public spending. Save the factor scores as new variables. -- >Explain the factors by respondents' background. -- > If you find a strong association test its statistical significance. --- class: inverse