Submission status
Reviewing
Submission Editor
Noah Carl
Title
Psychometric Analysis of the Multifactor General Knowledge Test
Abstract
The Multifactor general knowledge test for the openpsychometrics website was evaluated on multiple dimensions, including its reliability, ability to generate differences in areas where it is known that groups differ, how it should be scored, whether older individuals scored higher, and its dimensionality. The best method to generate the scores was to treat every checkbox as an item and add up the correct and incorrect scores. This generated a highly reliable (ω = 0.93) test, with a low median completion time (577 seconds), and a high ceiling (IQ = 149). One set of items (internet abbreviations) were found to have very low g-loadings, so we recommend removing them. The test also had age, national, and gender differences which replicate previous literature.
The test was clearly biased against non-Anglos, especially in the sections of aesthetic knowledge, cultural knowledge, literary knowledge, and technical knowledge. DIF testing suggested that the test was not biased in favor of Anglo countries, calling into question its usefulness in identifying highly biased tests. Between sexes, DIF found that many items were biased against both genders, but the magnitude of the bias did not vary by either sex. We highly recommend using this test to examine the general knowledge of native English speakers, and the use of a cultural and linguistic translation for non-English speakers.
Keywords
intelligence,
sex differences,
statistics,
knowledge,
methods
Supplemental materials link
https://osf.io/erx6q/
Reviewer 1: Considering / Revise
Reviewer 2: Accept
Reviewer 3: Considering / Revise
Reviewer 4: Accept