Since in colloquial language they have very similar meanings, it is easy to confuse the terms reliability and validity when we talk about science and, more specifically, psychometrics.
With this text, we intend to elucidate the main differences between reliability and validity. We hope you find it helpful to clarify this common question.
What is reliability?
In psychometry, the concept of “reliability” refers to the precision of an instrument; more precisely, the reliability coefficients inform us of the consistency and stability of the measurements taken with this tool.
The higher the reliability of an instrument, the lower the number of random and unpredictable errors that will appear when using it to measure certain attributes. Reliability excludes predictable errors, that is, those which are subject to experimental control.
According to classical testing theory, reliability is the proportion of variance explained by true scores. Thus, the direct score of a test would consist of the sum of the random error and the actual score.
The two main components of reliability these are temporal stability and internal consistency. The first concept indicates that scores change little when measured on different occasions, while internal consistency refers to the extent to which the items that make up the test measure the same psychological construct.
Therefore, a high reliability coefficient indicates that scores on a test fluctuate little internally and over time and, in short, that the instrument is free from measurement errors.
Definition of validity
When we talk about validity, we are referring to whether the test correctly measures the construct it intends to measure. This concept is defined as the relationship between the score obtained on a test and another related measure; the degree of linear correlation between the two elements determines the coefficient of validity.
Likewise, in scientific research, a high validity indicates the extent to which the results obtained with a given instrument or in a study can be generalized.
There are different types of validity, which depend on how it is calculated; this makes it a term with very different meanings. We can basically distinguish between content validity, criteria (or empirical) validity and concept validity.
Content validity defines the extent to which the items in a psychometric test are a representative sample of the items that make up the concept to be assessed. The instrument must include all the fundamental aspects of construction; for example, if we are to do an appropriate test to measure depression, we will necessarily have to include items that assess mood and decrease in pleasure.
Criterion validity measures the ability of the instrument to predict aspects related to the plan or area of interest. Finally, concept validity is intended determine if the test measures what it intends to measure, For example of convergence with scores obtained in similar tests.
Differences between reliability and validity
Although these two psychometric properties are closely related, the truth is that they refer to clearly differentiated aspects. Let’s see what these differences are.
1. The object of the analysis
Reliability is a characteristic of the instrument, in that it measures the properties of the elements that compose it. On the other hand, the validity does not refer exactly to the instrument but to generalizations made from the results obtained through this.
2. The information they provide
Although this is a somewhat simplistic way of approaching it, it is generally claimed that validity indicates that a psychometric tool actually measures the construct it seeks to measure, while reliability refers to know if it is measured correctly, without errors.
3. The way they are calculated
In order to measure reliability, three procedures are used basically: the method of two halves, that of parallel shapes and the test-retest. The most commonly used procedure is the two-half procedure, in which the items are divided into two groups after the test is answered; then the correlation between the two halves is analyzed.
The method of parallel or alternative forms consists in creating two equivalent tests to measure the degree of correlation between the items. The test-retest is simply based on passing the test twice, under as similar conditions as possible. The two procedures can be combined, resulting in the test-retest with parallel forms, which consists in leaving a time interval between the first form of the test and the second.
For its part, the validity it is calculated in different ways depending on the typeBut in general, all methods are based on comparing the score of the objective test with other data from the same subjects with respect to similar traits; the aim is for the test to act as a predictor of the shot.
Among the methods used to assess validity are factor analysis and the multi-method-multirash feature matrix technique. Likewise, the validity of content is often determined by rational analysis, not statistical analysis; for example, it includes face validity, which refers to the subjective judgment of experts on the validity of the test.
4. The relationship between the two concepts
The reliability of a psychometric instrument influences its validity: the more reliable it is, the greater its validity will also be. Therefore, the validity coefficients of a tool are always lower than those of reliability, and validity indirectly tells us about reliability.