Classworks Universal Screener

Reading

Rating Summary

Classification Accuracyhalf bubble
GeneralizabilityModerate High
Reliabilityfull bubble
Validityempty bubble
Disaggregated Reliability and Validity DataN A
Efficiency
AdministrationGroup
Administration & Scoring Time30 Minutes
Scoring KeyComputer Scored
Benchmarks / NormsYes
Cost Technology, Human Resources, and Accommodations for Special Needs Service and Support       Purpose and Other Implementation Information Usage and Reporting

Classworks Universal Screeners cost $4.00 per student per year which includes test forms.

Testers will require less than 1 hour of training.

Paraprofessionals can administer the test.

Curriculum Advantage, Inc.
1735 N. Brown Rd., Ste. 400
Lawrenceville, GA 300043

Phone: 888-841-4790
Web Site: www.classworks.com

Field-tested training manuals are included and should provide all implementation information.

Classworks implementation managers provide ongoing professional development, implementation planning, coaching and mentoring, and program evaluation support. In addition, Classworks offers an online help center, webinars, Classworks YouTube Channel videos, phone and web ticket support, knowledge base and technical support website, and GoToAssist capabilities.  

Classworks Universal Screeners identify student needs for additional specialized instruction in reading/language arts and mathematics. Universal Screeners provide information on overall student performance and specific student strengths and weaknesses in each of the 3-6 major strand measures. Strands assessed vary by grade level. Each screener includes items on grade level, below grade level, and two grades below so assessments have a sufficient reach and enough content coverage for students who may be struggling.

Classworks Universal Screeners are group administered in 30 minutes and scored automatically.

Scores include: raw, IRT-based, developmental benchmarks, equated, and strand level proficiency feedback.

Raw scores are calculated as the total number of items answered correctly on the screener. Performance on the screeners is reported as a scaled score on a vertical scale ranging from 200 to 800 spanning across grades K to 10.

 

Classification Accuracy

Classification Accuracy in Predicting Proficiency Level on the Georgia Criterion-Referenced Competency Test
  Overall Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
False Positive Rate 0.47 0.45 0.39 0.43 0.51 0.66 0.65
False Negative Rate 0.08 0.34 0.12 0.08 0.07 0.01 0.04
Sensitivity 0.92 0.66 0.88 0.92 0.93 0.99 0.96
Specificity 0.53 0.55 0.61 0.57 0.49 0.34 0.35
Positive Predictive Power 0.32 0.86 0.31 0.24 0.44 0.34 0.35
Negative Predictive Power 0.97 0.27 0.96 0.98 0.94 0.99 0.96
Overall Classification Rate 0.61 0.64 0.65 0.62 0.62 0.51 0.51
Observed 0.61 0.64 0.65 0.62 0.62 0.51 0.51
Chance 0.47 0.57 0.52 0.51 0.45 0.38 0.39
Area Under the Curve 0.78 0.62 0.84 0.80 0.77 0.74 0.76
Base Rate 0.20 0.81 0.17 0.13 0.31 0.26 0.27
Cut Points*: * * * * * * *
At 90% Sensitivity, Specificity equals 0.53 Not reported Not reported Not reported Not reported Not reported Not reported
At 80% Sensitivity, Specificity equals 0.50 Not reported Not reported Not reported Not reported Not reported Not reported
At 70% Sensitivity, Specificity equals 0.48 Not reported Not reported Not reported Not reported Not reported Not reported

*The outcome measure is the state high-stakes test for Georgia, the Criterion-Referenced Competency Test (CRCT). Risk and no-risk categories are defined by proficiency levels on the CRCT based upon scaled scores, with cut scores based on Annual Measureable Objectives (AMOs) as designated in the state assessment plan. The cut points for the Reading Universal Screeners were the scaled scores established through Book Marking Procedure to identify the performance level of students who are potentially at risk in grades 3 through 8.

Generalizability

Description of study sample:

  • Number of States: 1
  • Size: 11,365
  • Gender:
    • 51.5% Male
    • 48.5% Female 
  • SES: 60% Eligible for free or reduced-price lunch
  • Race/Ethnicity:
    • 65.4% White, Non-Hispanic
    • 28.7% Black, Non-Hispanic
    • 4.8% Hispanic
    • 0.1% American Indian/Alaska Native
    • 1.0% Asian/Pacific Islander
  • Language proficiency status: 2.9% Limited English Proficient

 

Cross Validation Study Description of Sample:

Cross Validation Study 1:

  • Number of States: 1
  • Size: 361
  • Gender:
    • 51% Male
    • 49% Female 
  • SES: 55% Eligible for free or reduced-price lunch
  • Race/Ethnicity:
    • 37% White, Non-Hispanic
    • 28% Black, Non-Hispanic
    • 33% Hispanic
    • 0% American Indian/Alaska Native
    • 2% Asian/Pacific Islander
  • Language proficiency status: 2.9% Limited English Proficient

Cross Validation Study 2:

  • Number of States: 1
  • Size: 1,147
  • Gender:
    • 51.6% Male
    • 48.4% Female 
  • SES: 66.2% Eligible for free or reduced-price lunch
  • Race/Ethnicity:
    • 71.6% White, Non-Hispanic
    • 26.8% Black, Non-Hispanic
    • 1.4% Hispanic
    • 0% American Indian/Alaska Native
    • 0.2% Asian/Pacific Islander
  • Language proficiency status: 2.9% Limited English Proficient

Reliability

Type of Reliability Age or Grade n Coefficient SEM Information (including normative data)/Subjects
KR-20, internal consistency Grades 3-8 7,614 0.93 0.27 Data collected in 2010 across multiple states as part of the item calibration process.  After the item calibration and analysis, final forms were built and internal consistency measures were calculated.
KR-20, internal consistency Grade 3 475 0.95 0.32 Data collected in 2010 across multiple states as part of the item calibration process.  After the item calibration and analysis, final forms were built and internal consistency measures were calculated.
KR-20, internal consistency Grade 4 611 0.94 0.28 Data collected in 2010 across multiple states as part of the item calibration process.  After the item calibration and analysis, final forms were built and internal consistency measures were calculated.
KR-20, internal consistency Grade 5 919 0.95 0.23 Data collected in 2010 across multiple states as part of the item calibration process.  After the item calibration and analysis, final forms were built and internal consistency measures were calculated.
KR-20, internal consistency Grade 6 1,809 0.95 0.25 Data collected in 2010 across multiple states as part of the item calibration process.  After the item calibration and analysis, final forms were built and internal consistency measures were calculated.
KR-20, internal consistency Grade 7 2,521 0.89 0.24 Data collected in 2010 across multiple states as part of the item calibration process.  After the item calibration and analysis, final forms were built and internal consistency measures were calculated.
KR-20, internal consistency Grade 8 1,279 0.92 0.28 Data collected in 2010 across multiple states as part of the item calibration process.  After the item calibration and analysis, final forms were built and internal consistency measures were calculated.
Test-retest Grades 3-8 343 0.83   Students taking the Reading Universal Screener two times in the Fall of 2011 with no more than 1 week between administrations. 
Test-retest Grade 3 55 0.87   Students taking the Reading Universal Screener two times in the Fall of 2011 with no more than 1 week between administrations. 
Test-retest Grade 4 47 0.81   Students taking the Reading Universal Screener two times in the Fall of 2011 with no more than 1 week between administrations. 
Test-retest Grade 5 30 0.75   Students taking the Reading Universal Screener two times in the Fall of 2011 with no more than 1 week between administrations. 
Test-retest Grade 6 83 0.89   Students taking the Reading Universal Screener two times in the Fall of 2011 with no more than 1 week between administrations. 
Test-retest Grade 7 90 0.84   Students taking the Reading Universal Screener two times in the Fall of 2011 with no more than 1 week between administrations. 
Test-retest Grade 8 38 0.80   Students taking the Reading Universal Screener two times in the Fall of 2011 with no more than 1 week between administrations. 

 

Validity

Type of Validity Age or Grade Test or Criterion n (range) Coefficient Information (including normative data)/Subjects
Content Grades 3 -8       An item alignment review was conducted to ensure that the Universal Screener items align to the Classworks objectives and individual state standards for reading.
Content Grades 3 - 8       Item content and bias reviews were conducted to ensure that the items selected for the Reading Universal Screeners were appropriate and reasonable for the purpose of screening students.
Content Grades 3 – 8       Test specifications include a range of coverage including items at grade level, one grade below, and two grades below.  This test design has been found to be effective for the purpose of screening.
Content Grades 3 – 8       Field test was conducted using a national sampling of students.  The item level data was used to calibrate the items using the Rasch model.  Items that did not fit the model or showed differential performance were edited or removed from the final forms.
Construct Grade 3 GA CRCT Grade 3 Reading 58 0.75 Correlation between performance on the Reading Universal Screeners and the Georgia CRCT Test.
Construct Grade 4 GA CRCT Grade 4 Reading 3,579 0.70 Correlation between performance on the Reading Universal Screeners and the Georgia CRCT Test.
Construct Grade 5 GA CRCT Grade 5 Reading 3,934 0.55 Correlation between performance on the Reading Universal Screeners and the Georgia CRCT Test.
Construct Grade 6 GA CRCT Grade 6 Reading 1,445 0.50 Correlation between performance on the Reading Universal Screeners and the Georgia CRCT Test.
Construct Grade 7 GA CRCT Grade 7 Reading 1,084 0.59 Correlation between performance on the Reading Universal Screeners and the Georgia CRCT Test.
Construct Grade 8 GA CRCT Grade 8 Reading 1,265 0.66 Correlation between performance on the Reading Universal Screeners and the Georgia CRCT Test.