Understanding Survey Results

Understanding Survey Results

Online Reporting System Tutorials
Excluded Respondents
Respondents Included in Raw Data File
Benchmark Reports
Means and Frequency Reports
Standards for Interpreting Mean Differences
Weights and Local Student Characteristics
Student Level Breakout Definitions
Student Identifier Data

College leaders will wish to familiarize themselves with Community College Survey of Student Engagement (CCSSE) findings before communicating about the results. The following notes should be considered.

Online Reporting System Tutorials

The Center offers four tutorials to help member colleges navigate and understand the various features of the CCSSE online reporting system.

Excluded Respondents

The total counts of respondents in an institution’s raw data file will differ from the numbers reported in the institutional reports due to intentional exclusion of certain surveys. Respondents are excluded from the institutional reports for the following reasons:

  • The respondent did not indicate whether he or she was enrolled full-time or part-time at the institution.
  • The survey is invalid. A survey is invalid if a student does not answer any of the 19 sub-items in item 4, answers very often to all 19 sub-items, or answers never to all 19 sub-items.
  • The respondent reported his or her age as under 18.
  • The respondent indicated that he or she had taken the survey in a previous class or did not respond to item 3.
  • Oversampled respondents are not included because they are selected outside of CCSSE’s primary sampling procedures.

Back to top

 

Respondents Included in Raw Data File

Raw data files contain responses from all students who completed CCSSE, with the exception of invalid surveys and those completed by students under the age of 18. For the purposes of working with your data file, excluded respondents do not have a weight listed for the IWEIGHT variable. Therefore, to run analysis without excluded respondents, simply remove any observations where IWEIGHT is missing. This will ensure that the analysis only includes primary sample respondents who do not meet any of the exclusionary criteria. 

Back to top

 

Benchmark Reports

To assist colleges in their efforts to reach for excellence, CCSSE introduced national benchmarks. Research shows that the more actively engaged students are — with college faculty and staff, with other students, and with the subject matter — the more likely they are to learn and to achieve their academic goals.

CCSSE benchmarks are groups of conceptually related survey items that focus on institutional practices and student behaviors that promote student engagement—and that are positively related to student learning and persistence. Benchmarks are used to compare each institution’s performance to that of similar institutions and with the CCSSE Cohort. The five benchmarks of effective educational practice in community colleges are: active and collaborative learning, student effort, academic challenge, student-faculty interaction, and support for learners.

Benchmark reports consist of tables showing the college's scores on each benchmark, followed by means and frequency tables of items in each benchmark. While the benchmark scores provide an overview of how the college is doing in particular areas, colleges must be mindful that the results from the individual survey items composing each benchmark deserve examination.

How Benchmarks are Calculated

Back to top

 

Means and Frequency Reports

Responses to individual CCSSE survey items are summarized in two formats–means and frequencies. 

Means reports present an average for each survey item that has scaled responses (e.g., strongly agree to strongly disagree) and compare average item responses between member colleges and various groups (e.g., similarly sized colleges), or between subgroups within a college (e.g., men & women).  Means are not run on dichotomous items: those with only two response options (e.g., yes/no; enrolled/not enrolled). These items are summarized in the frequency reports.

Frequency reports present the observed frequencies of occurrence (counts and percentages) of the values for each survey item, excluding demographic survey items. These reports are useful for understanding how data are distributed across response categories. Please note that counts and percentages on frequency reports are subject to rounding.

Back to top

 

Standards for Interpreting Mean Differences

When interpreting mean differences across comparison groups, the Center uses a combination of two measures: (1) a t-test with a very conservative alpha level of .001 or less is used to determine if the difference between two means is significant and not likely due to chance, and (2) an effect size of .20 (absolute value) or more using Cohen’s d is used to show the magnitude of difference between the two means. If a comparison is significant at an alpha level of .001 or less and has an effect size of .20 or greater, then it is considered to be a statistically significant difference worthy of further investigation. Comparisons that meet these criteria are marked with a double-asterisk (**). For internal analysis of small groups, it may make sense for colleges to use a larger alpha level but typically not a larger effect size.

Back to top

 

Weights and Local Student Characteristics

Full-time students, who by definition are enrolled in more classes than part-time students, are more likely to be sampled. To adjust for this sampling bias, a statistical weighting procedure is applied to CCSSE results when an analysis contains both full-time and part-time students. Weighting is uniquely calculated for each institution and is based on the most recent publicly available IPEDS enrollment figures.

Under certain circumstances, deactivating weights may be a more informative way to examine institutional CCSSE data. Even the most recent IPEDS data are approximately two years old and may not always accurately represent a college’s current student population. For example, in the case that a college has experienced a significant change in enrollment characteristics during the two years prior to administering CCSSE, the college’s institutional research department may want to consider whether the weights based on the IPEDS numbers are completely appropriate.

For example, a college where the vast majority of students are either full-time or part-time (e.g., 92% full-time) may want to look at the unweighted results for the majority group of students to guide campus discussions.

Back to top

 

Student Level Breakout Definitions

Breakout reports including benchmarks, means, and frequencies are available in each of the areas below. Each category is based on student responses to specific survey items.

Full-Time & Part-Time (Enrollment Status)
Item 2: “Thinking about this current academic term, how would you characterize your enrollment at this college?” 

Developmental & Non-Developmental
Three sub-items in Item 8: “Which of the following have you done, or are you currently doing at this college?

8c. Developmental remedial/reading course (also referred to as Basic Skills, College Prep, etc.)
8d. Developmental remedial/writing course (also referred to as Basic Skills, College Prep, etc.)
8e. Developmental remedial/math course (also referred to as Basic Skills, College Prep, etc.)

If a student responded that he or she has taken or is currently taking any one or more of these three types of courses, he or she is classified as Developmental; if a student responded that he or she has not taken nor is currently taking any of these three types of courses, he or she is classified as Non-Developmental. In addition, to be classified as Developmental or Non-Developmental, a student must have responded to all three items.

Traditional & Nontraditional-Age
Item 38: “Mark your age group.”

Respondents under age 18 are excluded from all data sets. Respondents marking age groups 18-19, 20-21, and 22-24 are classified as Traditional-Age and those marking age groups 25-29, 30-39, 40-49, 50-64, or 65+ are classified as Nontraditional-Age.

First-Generation & Not First-Generation
Item 47:“Who in your family has attended at least some college?” (Mark all that apply)

If the respondent indicated that his or her mother or father had attended at least some college, then the student is classified as Not First-Generation; otherwise, he or she is classified as First-Generation.

Gender Identity
Item 39: “Your gender identity.” 

Race/Ethnicity
Item 45: “What is your racial or ethnic identification? (Mark all that apply)”  

Race/Ethnicity is coded as "I prefer not to respond" if a student selected this response option, regardless of whether or not he or she also marked another response.

If a student selected more than one race/ethnicity response option, the race/ethnicity variable is set to "Two or More Races."

0 to 29 Credits & 30+ Credits
Item 23: “How many total credit hours have you earned at this college, not counting the courses you are currently taking this academic term?”

Back to top

 

Student Identifier Data

In accordance with Texas state law and The University of Texas at Austin’s policies, the Center does not provide student-identifier data in the institution's raw data file available for download via the CCSSE online reporting system. To request a securely transmitted data file with student identifiers, please contact your CCSSE liaison or surveyops@cccse.org.

Back to top

Center for Community College Student Engagement
—a Research and Service Initiative—
Program in Higher Education Leadership | Department of Educational Administration  | College of Education
The University of Texas at Austin
Comments to: webmaster@cccse.org