Submission status
Reviewing
Submission Editor
Noah Carl
Title
Psychometric Analysis of the Multifactor General Knowledge Test
Abstract
Different methods of scoring the multifactor general knowledge test were evaluated to test what the best method of calculating scores is. The best methods of calculating scores were treating every checkbox as an item and adding up all of the scores of the items or creating IRT models of the answers and distractors separately. The test was highly reliable (ω = 0.93), had a low median completion time (571 seconds), and a high ceiling (IQ = 149). One set of items (internet abbreviations) were found to have very low g-loadings, so we recommend removing them. The test also had age, national, and gender differences which replicate previous literature.
The test was clearly biased against non-Anglos, especially in the sections of aesthetic knowledge, cultural knowledge, literary knowledge, and technical knowledge. DIF testing could not detect this bias, calling into question its usefulness in identifying highly biased tests. Between sexes, DIF found that many items were biased against both genders, but the magnitude of the bias did not vary by either sex. We recommend scoring this test by adding up all of the distractors and answers, and the use of a cultural and linguistic translation on an international scale.
Keywords
intelligence,
sex differences,
statistics,
knowledge,
methods
Supplemental materials link
https://osf.io/erx6q/
Reviewer 1: Considering / Revise
Reviewer 2: Accept
Reviewer 3: Considering / Revise
Reviewer 4: Accept