NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Myrick KL, Salvaggio M, Ejike-King L, et al. Planning, Development, Design, and Operation of the 2016 National Culturally and Linguistically Appropriate Services Survey for Office-based Physicians [Internet]. Atlanta (GA): National Center for Health Statistics (NCHS); 2025 Jan.
Planning, Development, Design, and Operation of the 2016 National Culturally and Linguistically Appropriate Services Survey for Office-based Physicians [Internet].
Show detailsCreation of Draft Questionnaire
A review of the scientific literature was conducted to identify relevant studies on the topics of culturally and linguistically appropriate health services, cultural competency training, and awareness and adoption of National CLAS Standards. Different combinations of the following keywords were used as search terms in Google Scholar and PubMed: “culturally and linguistically appropriate services,” “cultural competence,” “National CLAS Standards,” “physician survey,” and “health care provider survey.” Only peer-reviewed articles published in English were included. No publication date restrictions were imposed.
The literature review identified five relevant survey and assessment instruments published in peer-reviewed scientific journals. The five survey and assessment instruments were the Cultural Competence Self-Assessment Questionnaire (CCSAQ) (16,17), Tool for Assessing Cultural Competence Training (TACCT) (18), Cultural Competence Assessment (CCA) (19), Cultural Competence Health Practitioner Assessment (CCHPA) (20), and Cultural Assessment Survey (CAS) (21,22) (Table B).
Table B
Key surveys and instruments from literature review to identify relevant studies on topics of culturally and linguistically appropriate health services, cultural competency training, and awareness and adoption of National CLAS Standards
None of these survey and assessment instruments comprehensively incorporated the multiple domains of the National CLAS Standards (Table A). CCSAQ contained the most relevant items consistent with the key objectives of the National CLAS Physician Survey; however, CCSAQ and its modified version focus narrowly on race and ethnicity rather than broadly on diverse cultural characteristics, health beliefs, literacy, and language preferences as defined by the National CLAS Standards. TAACT was designed to assess educational curricula rather than providers’ experiences and practices in cultural and linguistic competency. CCA was designed to assess components of cultural competence, including cultural awareness, cultural sensitivity, and cultural competence behaviors in hospice workers. CCA lacked questions about cultural competence: workforce training, workforce education, organizational governance, leadership, and National CLAS Standards awareness. CCHPA focused on self-assessment of health professionals regarding their cultural competence, and CAS was designed to evaluate attitudinal differences among medical students. Neither CCHPA nor CAS were designed for survey data collection.
As a result, a new survey needed to be developed. A draft version of the National CLAS Physician Survey was developed based on the content of the five existing instruments, the three key survey objectives, and the 15 domains of the National CLAS Standards (Table A). Through a repetitive (iterative) process, the Office of Minority Health and the National Center for Health Statistics (NCHS) decided on the content of the draft survey, the ordering and phrasing of the items, and the appropriateness of the response categories. This draft version was then refined through cognitive interview testing.
Cognitive Interview Testing
Comprehension of questionnaire items for the National CLAS Physician Survey was assessed using cognitive interview testing, which allowed for understanding burden and potential response errors. The testing was approved by the NCHS Ethics Review Board (23). The methodology used was based on a qualitative question-evaluation method designed to identify question-response problems and examine the construct validities of survey questions (24–26). A separate Q-Bank report, from the NCHS Collaborating Center for Questionnaire Design and Evaluation Research, provides details on the cognitive testing of the content of the National CLAS Physician Survey (instructions, sections, questions, and response categories) using a paper mode of data collection (23). A separate assessment tested the usability of the web format of the questionnaire. Both the paper and web formats of the questionnaire had the same content, question phrasing, and response categories.
Results and Recommendations From Cognitive Interview Testing
Cognitive interview testing identified the following three emergent themes about participant difficulties answering the draft National CLAS Physician Survey: 1) understanding the definition of culture and cultural competency training; 2) differentiating formal from informal training and policies; and 3) interpreting the questions universally for all patients or situationally for specific groups of patients (23).
The National CLAS Physician Survey had a series of questions about training in cultural competency; these are related to the first theme. These questions ask whether the respondent has ever received training in cultural competency, whether they participated in training in cultural competency in the past 12 months, the population groups that the training included, the specific topics covered in the training, and whether the training was related to credentialing. The challenges with these questions were related to the setting in which the training occurred (23). Respondents younger than age 50 typically considered training in cultural competency as part of medical school or residency education (23). However, respondents age 50 and older typically thought of training in cultural competency as continuing medical education. Not providing a definition for the term “training in cultural competency” led to respondent uncertainty about what activities qualified for the series of questions that asked about it. Consequently, a definition was included in the final questionnaire to reduce respondent uncertainty: Training in cultural competency includes educational opportunities that address topics of culture in settings such as employee orientation, continuing medical education, conferences, or webinars.
The National CLAS Physician Survey included several questions about training and policies related to cultural competency and CLAS that did not specify the formality (or informality) of the training or policy, representing the second theme. Differentiating formal and informal training was a challenge for respondents, specifically, uncertainty about whether the questions referred to formal mandated training or informal voluntary training. Also, incomplete understanding of what training in cultural competency was added to question complexity. Providing a definition that described both formal and informal training in cultural competency resolved this confusion. The formal and informal nature of training as it related to the types of translators used also caused difficulty for respondents. This issue was resolved by specifying translator training in the three response options, "staff/contractor trained as a medical interpreter," "bilingual staff," or "patient's relative or friend."
The questions on policies related to training for new hires and the existence and awareness of policies about the provision of CLAS also posed challenges. Respondents had difficulty determining whether survey questions that referenced policy were referring to formal or informal policies. Many respondents defined formal policies as being written, while informal policies were not. Several respondents assumed the questions referred to formal policies and were uncertain whether the questions were to be answered regarding the practice’s policies or the policies of the larger organization. As a result, the question about having “at least one policy” related to CLAS was revised to ask about having “at least one formal written policy.” For the question about whether factors affected the provision of CLAS, policy (informal or formal) was changed to “formal written policy.”
The final emergent theme was related to respondent challenges differentiating universal and situational interpretation of questions. The questions about whether the respondent considered race, ethnicity, or other cultural factors during assessment of patient needs, diagnosis, and treatment or when conducting health education (Appendix I, Questions 19–22) posed challenges as respondents expressed that there was not a single response for all situations. For example, certain conditions have no known racial, ethnic, or cultural differences in diagnosis or treatment. Thus, a physician may diagnose and treat all patients with that condition the same, regardless of racial, ethnic, or cultural differences. However, for conditions that do differ in diagnosis or treatment by racial, ethnic, or cultural groups, a physician may consider race, ethnicity, or cultural factors on a situational basis when diagnosing or treating patients. These situational differences made it difficult to decide the appropriate responses.
Finalization of National CLAS Physician Survey Questionnaire
The key changes made between the original draft and the final questionnaire are shown in Table C. The changes from the original to revised questionnaire and then to the final version were either related to the themes described previously or to improve clarity, reduce burden, or meet reporting standards. The questions for the final version were reordered for various reasons, including improvement of questionnaire flow, sequential placement of eligibility questions early in the questionnaire, and placement of complex questions at the end of the questionnaire.
Table C
Key modifications to National CLAS Physician Survey questionnaire from original and revised versions to final version: National Culturally and Linguistically Appropriate Services Physician Survey, 2016
After these revisions were made, the final version of the questionnaire had 42 items, capturing information on the respondent’s demographic and practice characteristics, cultural competency training, provision of CLAS, and awareness of the National CLAS Standards (Appendix I). Each item on the questionnaire corresponds to a specific domain of the enhanced National CLAS Standards and to a key objective of the survey (Appendix II), except for items related to eligibility, weighting, and physician demographics. The final questionnaire was estimated to take about 9.5 minutes to complete.
The questions used to determine eligibility (questions 2, 3, and 4) and weight the data (questions 2 and 5) were placed at the beginning of the questionnaire to ensure that this required information was available for all respondents. This placement also reduced the burden on ineligible respondents. Survey questions 6–36 address the enhanced National CLAS Standards. The goal was to not bias the respondents’ subsequent responses to questions by asking about the National CLAS Standards early in the questionnaire. As a result, those questions were placed near the end of the questionnaire, contrary to the survey methodology recommendation to place the most important questions first (27).
Representation of Enhanced National CLAS Standards in Items of National CLAS Physician Survey
Appendix II shows how the enhanced National CLAS Standards are represented within the content of the National CLAS Physician Survey. Standard 1 of the enhanced National CLAS Standards is the Principal Standard. Some of the questions in the National CLAS Physician Survey, especially the attitudinal ones, fit best with this overarching standard. Specifically, questions 19–22 ask about the consideration of cultural factors when assessing, diagnosing, and treating patients and when conducting health education. In addition, attitudinal questions about both the expected outcomes of providing CLAS (questions 24–29) and the effect of factors that help or hinder the provision of CLAS (questions 30–34) are included.
Standards 2–4 address governance, leadership, and workforce. Standard 2 is addressed in questions 11 and 36: question 11 is about written CLAS policies at the practice and the physician’s knowledge of the policy (question 11a). Question 36 asks whether the physician’s practice has adopted the National CLAS Standards. Standard 3 is associated with questions 9–11. Standard 4 is captured in questions 6–10. These questions ask about training in cultural competency. One question specifically asks whether training is required for physician new hires (question 9).
Standards 5–8 address communication and language assistance. Each of the standards in this domain are addressed in the National CLAS Physician Survey. Standard 5 is captured in questions 12–14. Question 12 asks about printed materials available for patients with limited English literacy. Question 13 asks about the free language-assistance services that are available at the physician’s practice. Question 14 asks whether interpreters are used for patients who have limited English proficiency. Standards 6 and 7 are assessed in questions 14, 16, and 17. Question 14 asks whether interpreters are used and, if so, the types of interpreters used (question 14a). Question 14a asks whether staff and contractors trained as medical interpreters, bilingual staff, or the patient’s relative or friend are used. Question 16 asks about the physician’s fluency in languages other than English. Question 17 asks the number of languages the physician feels comfortable using to provide healthcare services. The physician fluency questions (questions 16 and 17) were moved from the beginning of the questionnaire to the middle of the questionnaire with the other language questions to improve flow and cohesiveness. The final standard of this domain, Standard 8, is addressed in questions 12, 13, and 15. Questions 12 and 13 ask about items for patients with limited English literacy, which include universal symbols and infographics (question 12) and multimedia materials, such as recorded messages and translated signs (question 13). Question 15 asks about the types of materials available for patients in languages other than English.
Standards 9–15 address engagement, continuous improvement, and accountability. Standards 9–11 are captured in the survey. Standard 9 is captured in questions 11 and 36 on policy implementation. Standard 10 is addressed in questions 18, 23, 37, and 38. Question 18 is about the physician’s knowledge of patient health beliefs, customs, and values. The frequency at which the practice is assessed for the cultural and linguistic appropriateness of services is addressed in question 23. Recording of patient characteristics related to culture is asked in questions 37 and 38. Standard 11 is addressed in questions 37 and 38. Standards 12–15 are not represented in the National CLAS Physician Survey.
The remaining survey questions collect demographic information. The demographic questions (37–42) were placed at the end of the questionnaire, which is a common survey methodology practice (27). Question 37a asks for the percentage of the physician’s patient population by race and ethnicity. Initially, this question was considered too burdensome to ask physicians who may not know their patient population’s racial and ethnic composition. Because of this, it was recommended that a skip pattern be added so that those who indicated that they did not collect information on their patient population’s racial and ethnic composition would not be asked the more detailed question about race and ethnicity (question 37a). Physician demographics (sex, race, and ethnicity) are asked in questions 39–41. The race and ethnicity categories were revised to align with HHS implementation guidance on data collection standards for race, ethnicity, sex, primary language, and disability status (28). The final item on the questionnaire was a checkbox asking the respondent to “verify that this questionnaire was completed by the physician to whom it was addressed.” This statement and instructions that the questionnaire should be completed only by the physician to whom it was addressed were important factors considered during the development process. Using proxy respondents was inappropriate due to the nature of the questions and the inability of anyone other than the physician to respond accurately to the survey questions.
- Survey Planning and Development - Planning, Development, Design, and Operation o...Survey Planning and Development - Planning, Development, Design, and Operation of the 2016 National Culturally and Linguistically Appropriate Services Survey for Office-based Physicians
Your browsing activity is empty.
Activity recording is turned off.
See more...