office (03) 9820 3006 or 920 3008
fax (03) 9820 3007

Step 4 - Validation of research instruments

Validation of research instruments

Cultural competence in:

Cross-cultural validation of research instruments

Which diversity variable?

The researcher in defining variables should be aware of the need for circumspection in use of contentious terms such as ‘race’ (see Terminology section) and to recognize the dynamic nature of ethnicity-related concepts, as well as the socio-political contexts in which they are developed.

“This will ensure the scientific validity of the research question and will allow research findings to be evaluated within the context of continuously changing scientific and societal conceptions of these definitions” (Fisher, Hoagwood, Boyce, Duster, Frank, Grisso, Levine, Macklin, Spencer, Takanishi, Trimble, & Zayas 2002;Fisher & Wallace 2000).

Most published North American studies base their measures of diversity on categories of race but, as Fuller (Fuller 2003) notes, lumping together into a 'racial' group large numbers of individuals who share little in terms of phenotype, culture, and/or behaviour inhibits reaching appropriate solutions, with progress being made when the issue of health disparities is reframed as one of phenotype/environmental mismatch.

A more fundamental deficit is that although place of birth is supposedly routinely collected by population-based cancer registries (at least in the United States) the quality of the data is unclear because registry birthplace information is incomplete. Gomez et al. (Gomez et al. 2004) quantified misclassification of birthplace data for Asian cancer patients in the Greater Bay Area Cancer Registry in northern California by comparing registry birthplace information with self-reported birthplace from interview, and then identified sociodemographic and hospital characteristics associated with birthplace completeness and misclassification. 35% had unrecorded registry birthplace. Among US-born Asians, those misclassified as foreign-born were more likely than those correctly classified to prefer a non-English primary language.

Validating the instruments

Cultural validity for all instruments is essential principle but unfortunately one that is seldom achieved. In the case of assessment, for example, “In the case of non-English-speaking or multilingual participants, investigators should use translated versions of standardized instruments that have been subjected to rigorous standards of measurement equivalence to ensure that the desired psychological constructs are properly measured” (Fisher, Hoagwood, Boyce, Duster, Frank, Grisso, Levine, Macklin, Spencer, Takanishi, Trimble, & Zayas 2002;Fisher & Wallace 2000). Similar issues are picked by Gil and Bob (Gil & Bob 1999).

Translating instruments

See issues in translating instruments, for example, on cancer (Hilton & Skrutkowski 2002) which describe conducting translation for equivalence, types of equivalence, and strategies to translate instruments that promote equivalence and how to test the translated version for equivalence.

Step-wise Validation for Cross-Cultural Equivalence

Content Equivalence. The content of each item of the instrument is relevant to the phenomena within each culture studied.

Semantic Equivalence.  The meaning of each item is the same in each culture after translation into the language and idiom (written or oral) of each culture. 

Technical Equivalence.  The method of assessment (e.g., pencil and paper, interview) is comparable in each culture with respect to the data it yields.

Criterion Equivalence.  The interpretation of the measured variable remains the same with respect to the norm within each culture.

Conceptual Equivalence.  The instrument is measuring the same theoretical construct in each culture.

Statistical Method for Evaluating Cross-Cultural Equivalence

Validation step             Statistical method

Content Equivalence    Item-matching task

Semantic Equivalence  Correlation between alternate Forms administered to bilinguals

Technical Equivalence Differential Item Functioning

Criterion Equivalence  Predictive - Regression, ROC

Conceptual Equivalence          Correlation methods - Correlation, Factor Analysis