Standard Errors and Variance Estimation

Standard error is primarily a measure of the sampling variability that occurs by chance because only a sample is surveyed, rather than the entire universe.

The sampling methodology used in the 2016 National CLAS Physician Survey uses a list sample. The design variables reflect this sampling methodology. Examples of SUDAAN, SAS, Stata, and SPSS statements that incorporate these design variables for variance estimation follow. All examples use a data set named “CLASdata” that represents the National CLAS Physician Survey public-use micro-data file.

Variance Estimation Examples in SUDAAN

The linearized Taylor series procedure in SUDAAN software is used to estimate variances for the National CLAS Physician Survey estimates. SUDAAN’s 1-stage WR (with replacement) option is used. This code provides a without replacement ultimate cluster (1-stage) estimate of standard errors for a cross-tabulation with a data set called CLASdata. SAS-callable SUDAAN software requires that the data set be sorted by the NEST variable before analysis. In this case, SAS’s PROC SORT procedure was used to sort the data set:

PROC SORT DATA=CLASdata; by CSTRATM; run;

The required SUDAAN statements for estimating variances in the sorted data set are:

PROC (procedure) DATA = CLASdata DESIGN = (statistic type);

NEST CSTRATM / MISSUNIT;

WEIGHT CLASWEIGHT;

The items in parentheses are replaced with the information required by the specific SUDAAN procedure used, and design type specified. The variance variables required by the software are included in a VAR, CLASS, LEVEL, and TABLE statements (see below).

For example, to produce frequency tables using the CROSSTAB procedure in SAS-callable SUDAAN, the following statements are used:

PROC CROSSTAB DATA=CLASdata filetype=SAS Design=WR;

NEST CSTRATM / MISSUNIT;

WEIGHT CLASWEIGHT;

CLASS REGION CLASTRAIN;

TABLE REGION*CLASTRAIN;

Variance Estimation Examples in SAS

The following is an example of the PROC CROSSTAB SUDAAN analysis (shown previously) using the SAS SURVEYFREQ procedure:

PROC SURVEYFREQ DATA=CLASdata;

STRATA CSTRATM;

WEIGHT CLASWEIGHT;

TABLE REGION*CLASTRAIN;

run;

Variance Estimation Examples in STATA

The pweight (CLASWEIGHT), stratum (CSTRATM), and psu (PHYCODE) are set with the svyset command as follows:

Stata 8:

svyset [pweight=CLASWEIGHT], psu (PHYCODE) strata(CSTRATM)

Stata 9 and later:

svyset PHYCODE [pweight=CLASWEIGHT], strata(CSTRATM)

Variance Estimation Examples in SPSS

To obtain variance estimates, which account for the sample design, IBM SPSS Statistics's "Complex Samples" module can be used. This description applies to version 24.0. From the main menu, click on "Analyze," "Complex Samples," then "Prepare for Analysis." The "Analysis Preparation Wizard" can be used to set CSTRATM as the stratum variable, PHYCODE as the cluster variable, and CLASWEIGHT as the weighting variable. The WR design option may be chosen. This will create the PLAN FILE syntax, which should resemble the following code, where PLAN FILE reflects the location selected to store the file on the computer:

CSPLAN ANALYSIS

/PLAN FILE=’DIRECTORY\PLANNAME.CSAPLAN’

/PLAN VARS ANALYSISWEIGHT=CLASWEIGHT

/PRINT PLAN

/DESIGN STAGELABEL= ‘ANY LABEL’ STRATA=CSTRATM CLUSTER=PHYCODE

/ESTIMATOR TYPE=WR.

For more information, see the National CLAS Physician Survey public-use file documentation (15).