1. Can a test be reliable and not valid or vice versa? Why or why not?
The quality of any statistical analysis can be only as good as is the quality of the data upon which it is based; thus, the reliability and validity of data collected for use in statistical analysis is of paramount importance. Validity refers to the extent to which data a data collection instrument, or an analytical procedure (such as a test) measures what it is actually desired to measure. Reliability refers to the accuracy and precision of a data collection procedure or the outcome of an analytical procedure (such as a test). Validity, thus, is the extent to which differences found through a particular data collection procedure or analytical procedure reflects true differences among those variables being measured, while reliability refers to the capacity of an instrument or an analytical procedure to yield similar measurements under similar conditions.
To accomplish the objective of validity in measurement, it is necessary to have some standard that is external to the measurement procedure, in order to evaluate the validity of the procedure. Thus, if it were desired to measure the height of a number of individuals in terms of feet and inches, it would be necessary to have a measuring instrument that was calibrated in feet and inches. The validity of this measuring instrument calibrated in feet and inches would be determined based on its ability to accurately measure feet and inches in accordance with an external standard, such as a master measurement instrument maintained by the National Bureau of Standards. An instrument or analytical procedure designed to measure some psychological characteristic or factor would have to be evaluated in the context of validity based on its ability to yield results consistent with those of another instrument or analytical procedure whose validity has previously been established. Thus, if an instrument were designed ...