mCLASS:Reading 3D

Text Reading and Comprehension

Rating Summary

Classification Accuracyhalf bubble
GeneralizabilityModerate High
Reliabilityhalf bubble
Validityfull bubble
Disaggregated Reliability and Validity Datafull bubble
Efficiency
AdministrationIndividual
Administration & Scoring Time5 - 8 Minutes
Scoring KeyComputer Scored
Benchmarks / NormsYes
Cost Technology, Human Resources, and Accommodations for Special Needs Service and Support Purpose and Other Implementation Information Usage and Reporting

Cost per student per year:
$20.90 per student (includes $8.90 platform subscription)

$300 for complete Amplify Atlas Edition Administration kit, which includes Teacher’s Guide, and Student benchmark books.

Each teacher administering mCLASS: Reading 3D needs a needs a tablet computer or smartphone which typically costs $200-$300 from national resellers.

$400 remote installation fee per campus. Includes telephone guidance through the installation of the software on teacher handhelds and desktop computers, troubleshooting, and verification of installation process.

Computer and Internet access are required for full use of product services.

Testers will require 4-8 hours of training.

Paraprofessionals can administer the test.

Amplify                                                              
55 Washington St. Suite 900
Brooklyn, NY 11201
Phone #: 1-800-823-1969, option 1

Website: http://www.amplify.com/

Amplify’s Customer Care Center offers complete user-level support from 7:00 a.m. to 7:00 p.m. EST, Monday through Friday. Customers may contact a customer support representative via telephone, e-mail, or electronically through the mCLASS Website. Calls to the Customer Care Center’s toll-free number are answered immediately by an automated attendant and routed to customer support agents according to regional expertise.  Additionally, customers have self-service access to instructions, documents, and frequently asked questions on our Website.

mCLASS:3D - TRC is a set of screening and progress monitoring measures for grades K-3. Text Reading and Comprehension (TRC) is an individually administered assessment using leveled readers from a book set to determine a student’s instructional reading level. During this measure, students are asked to read a benchmark book and complete a number of follow-up tasks, which may include Oral Comprehension, Retelling, and/or Written Comprehension. Assessors observe and record the student’s oral reading behaviors through the administration of TRC to determine reading accuracy, fluency and various comprehension components. The comprehension components help assessors determine whether the student understands the meaning of the text and the student’s instructional reading level.

While the student reads from the set of leveled readers, the teacher follows along on the handheld, tapping with the stylus to record the student’s performance. The handheld software offers a pre-loaded class list indicating required assessment tasks, provides the teacher with directions and prompts to ensure standardized, accurate administration, and automates the precise timing requirements.  Upon completion of each task, the handheld automatically calculates the student’s score and provides a risk evaluation.

mCLASS:3D - TRC takes 5 to 8 minutes to administer.  Scoring time is 0 minutes.

1 extra form is available per test level.

Raw, and developmental benchmarks scores are available.

The raw score provided is the reading level of the student. Cut Points for each proficiency level is provided.

Developmental benchmarks for each measure, grade, and time of year (beginning, middle, end) report each score as Above Proficient, Proficient, Below Proficient, or Far Below Proficient.

 

Classification Accuracy

Classification Accuracy in Predicting Proficiency on the Grade 3 Indiana Statewide Testing for Educational Progress Plus (ISTEP+)
  n = 21,066
False Positive Rate 0.11
False Negative Rate 0.45
Sensitivity 0.55
Specificity 0.89
Positive Predictive Power 0.62
Negative Predictive Power 0.86
Overall Classification Rate 0.81
AUC (ROC) 0.85
Base Rate 0.24
Cut Points: Reading Level = “L”
At 90% Sensitivity, Specificity equals 0.57
At 80% Sensitivity, Specificity equals 0.78
At 70% Sensitivity, Specificity equals 0.78

Cross Validation Study

Classification Accuracy in Predicting Proficiency on the TRC at MOY to predict DIBELS Next Composite score at EOY for Kindergarten

 

Overall

White

Black

Hispanic

False Positive Rate

0.04

0.03

0.06

0.06

False Negative Rate

0.63

0.68

0.61

0.58

Sensitivity

0.37

0.32

0.39

0.42

Specificity

0.96

0.97

0.94

0.94

Positive Predictive Power

0.95

0.94

0.93

0.96

Negative Predictive Power

0.43

0.49

0.43

0.32

Overall Classification Rate

0.56

0.58

0.57

0.54

AUC (ROC)

0.69

0.71

0.68

0.64

Base Rate

0.67

0.59

0.67

0.77

Cut Points:

C

C

C

C

At 90% Sensitivity, Specificity equals

2.03

1.82

1.98

2.59

At 80% Sensitivity, Specificity equals

1.83

1.67

1.78

2.25

At 70% Sensitivity, Specificity equals

1.63

1.57

1.57

1.91

 

Classification Accuracy in Predicting Proficiency on the TRC at MOY to predict DIBELS Next Composite score at EOY for 1st Grade

 

Overall

White

Black

Hispanic

False Positive Rate

0.09

0.08

0.11

0.10

False Negative Rate

0.34

0.36

0.32

0.33

Sensitivity

0.66

0.64

0.68

0.67

Specificity

0.91

0.92

0.89

0.90

Positive Predictive Power

0.85

0.80

0.85

0.90

Negative Predictive Power

0.78

0.83

0.75

0.67

Overall Classification Rate

0.80

0.82

0.79

0.77

AUC (ROC)

0.81

0.82

0.80

0.78

Base Rate

0.43

0.33

0.48

0.57

Cut Points:

G

G

G

G

At 90% Sensitivity, Specificity equals

1.09

1.05

1.09

1.21

At 80% Sensitivity, Specificity equals

1.02

1.00

1.00

1.07

At 70% Sensitivity, Specificity equals

0.94

0.95

0.91

0.94

 

Classification Accuracy in Predicting Proficiency on the TRC at MOY to predict DIBELS Next Composite score at EOY for 2nd Grade

 

Overall

White

Black

Hispanic

False Positive Rate

0.07

0.06

0.09

0.10

False Negative Rate

0.46

0.49

0.43

0.38

Sensitivity

0.54

0.51

0.57

0.62

Specificity

0.93

0.94

0.91

0.90

Positive Predictive Power

0.87

0.83

0.88

0.89

Negative Predictive Power

0.71

0.78

0.64

0.64

Overall Classification Rate

0.76

0.79

0.73

0.74

AUC (ROC)

0.79

0.81

0.76

0.77

Base Rate

0.45

0.35

0.54

0.57

Cut Points:

L

L

L

L

At 90% Sensitivity, Specificity equals

1.22

1.15

1.29

1.27

At 80% Sensitivity, Specificity equals

1.14

1.10

1.18

1.14

At 70% Sensitivity, Specificity equals

1.06

1.05

1.06

1.00

 

Classification Accuracy in Predicting Proficiency on the TRC at MOY to predict DIBELS Next Composite score at EOY for 3rd Grade

 

Overall

White

Black

Hispanic

False Positive Rate

0.10

0.07

0.15

0.14

False Negative Rate

0.42

0.47

0.38

0.36

Sensitivity

0.58

0.53

0.62

0.64

Specificity

0.90

0.93

0.85

0.86

Positive Predictive Power

0.85

0.84

0.87

0.89

Negative Predictive Power

0.68

0.75

0.57

0.58

Overall Classification Rate

0.74

0.77

0.71

0.72

AUC (ROC)

0.77

0.79

0.72

0.73

Base Rate

0.50

0.40

0.62

0.64

Cut Points:

O

O

O

O

At 90% Sensitivity, Specificity equals

1.22

1.18

1.31

1.32

At 80% Sensitivity, Specificity equals

1.12

1.12

1.15

1.15

At 70% Sensitivity, Specificity equals

1.02

1.05

0.98

0.97

 

Classification Accuracy in Predicting Proficiency on the TRC at MOY to predict DIBELS Next Composite score at EOY for 4th Grade

 

Overall

White

Black

Hispanic

False Positive Rate

0.27

0.23

0.32

0.30

False Negative Rate

0.26

0.28

0.25

0.22

Sensitivity

0.74

0.72

0.75

0.78

Specificity

0.73

0.77

0.68

0.70

Positive Predictive Power

0.81

0.79

0.83

0.86

Negative Predictive Power

0.64

0.69

0.57

0.57

Overall Classification Rate

0.74

0.74

0.73

0.76

AUC (ROC)

0.73

0.74

0.70

0.72

Base Rate

0.61

0.55

0.67

0.70

Cut Points:

R

R

R

R

At 90% Sensitivity, Specificity equals

0.98

0.98

0.99

0.98

At 80% Sensitivity, Specificity equals

0.82

0.86

0.78

0.75

At 70% Sensitivity, Specificity equals

0.66

0.74

0.57

0.51

 

Classification Accuracy in Predicting Proficiency on the TRC at MOY to predict DIBELS Next Composite score at EOY for 5th Grade

 

Overall

White

Black

Hispanic

False Positive Rate

0.29

0.26

0.35

0.32

False Negative Rate

0.24

0.25

0.23

0.21

Sensitivity

0.76

0.75

0.77

0.79

Specificity

0.71

0.74

0.65

0.68

Positive Predictive Power

0.80

0.76

0.81

0.85

Negative Predictive Power

0.66

0.73

0.60

0.58

Overall Classification Rate

0.74

0.75

0.73

0.76

AUC (ROC)

0.73

0.74

0.71

0.72

Base Rate

0.60

0.52

0.66

0.70

Cut Points:

T

T

T

T

At 90% Sensitivity, Specificity equals

0.93

0.90

0.90

0.94

At 80% Sensitivity, Specificity equals

0.78

0.80

0.70

0.71

At 70% Sensitivity, Specificity equals

0.63

0.69

0.51

0.47

 

 

Generalizability

Description of study sample:

  • Number of States: 1
  • Size: 21,066
  • Gender:
    • 46% Male
    • 44% Female 
    • 10% Unknown
  • SES: 40% Eligible for free or reduced-price lunch
  • Race/Ethnicity:
    • 57.0% White, Non-Hispanic
    • 15.6% Black, Non-Hispanic
    • 9.5% Hispanic
    • 0.2% American Indian/Alaska Native
    • 1.2% Asian/Pacific Islander
    • 4.9% Other
    • 11.6% Unknown

Cross Validation Study Description of study sample:

  • Number of States: 1
  • Size: 1,046
  • Gender:
    • 50% Male
    • 49% Female 
    • 1% Unknown
  • SES: 35% Eligible for free or reduced-price lunch
  • Race/Ethnicity:
    • 64.4% White, Non-Hispanic
    • 20.4% Black, Non-Hispanic
    • 7.4% Hispanic
    • 0.5% American Indian/Alaska Native
    • 1.8% Asian/Pacific Islander
    • 4.7% Other
    • 0.8% Unknown

Description of cross-validation study sample:

  • Number of States: 1 (North Carolina)
  • Gender
    • 43% Male
    • 41% Female
    • 15% Unknown
  • Size: 279,176
  • Race/Ethnicity:
    • 34% White, Non-Hispanic
    • 2% American Indian/Alaska Native
    • 19% Black, Non-Hispanic
    • 23% Asian, Pacific Islander
    • 9% Hispanic
    • 32% Unknown
    • 3% Other

Reliability

Type of Reliability Age or Grade n Coefficient SEM Information (including normative data)/Subjects
Range Median
Marginal Reliability* (Primary Sample) K-3 1,774-60,134 0.79-0.88 0.86 0.024 Students in this sample were administered the assessment during the 2009-2010 school year and were comprised of 41% male and 39% female; 31% were eligible for free/reduced lunch, 49%  were white, 12% were African American, and 8% were Hispanic-Latino.
K 50,822   0.79  
1 60,134   0.86  
2 56,301   0.88  
3 1,774   0.86  
Marginal Reliability* K-6 230,103
(5-71,996)
0.91-1.00 0.963 0.05-0.06 (median = 0.05)

Students in this sample were administered the assessment during the 2014-2015 school year and were comprised of:

* 43% female, 45% male.

* 27% white, 23% African American, and 17% Hispanic-Latino.

* 38% eligible for free/reduced lunch.

Marginal Reliability* K 66,781 0.92   0.05  
Marginal Reliability* 1 71,966 0.91   0.05  
Marginal Reliability* 2 68,572 0.95   0.05  
Marginal Reliability* 3 12,592 0.96   0.05  
Marginal Reliability* 4 5,549 0.96   0.05  
Marginal Reliability* 5 4,638 0.97   0.05  
Marginal Reliability* 6 5 1.00   0.05  
Marginal Reliability (Cross- Validation) K-2 2,262-2,403 0.84-0.87 0.86 0.008 Students in this sample were administered the assessment during the 2009-2010 school year. The sample is comprised of 43% female and 48% male students; students were identified as 58% white, 15% Black or African-American, and 9% Hispanic-Latino.
K 2,358   0.84  
1 2,403   0.86  
2 2,262   0.87  
Inter-Rater (Cross-Validation) K-3 27 Total   0.73   Calculated as the average Spearman’s rank order correlation between final results recorded by all pairs of raters.

* See Sireci, Thissen, and Wainer (1991). On the reliability of testlet-based tests. Journal of Educational Measurement, 28(3), 237-247.

 

Validity

Type of Validity Age or Grade Test or Criterion n Coefficient Information (including normative data)/Subjects
Content K-3       Alignment of mClass:3D - TRC to the Common Core State Standards in English Language Arts
Predictive (Primary Sample) 2 Grade 3 ISTEP+ ELA 21,066 0.70  
Predictive (Cross-Validation) 2 Grade 3 EOG Reading Comprehensive 1,046 0.76  
Concurrent (Primary Sample) 3 Grade 3 ISTEP+ ELA 1,641 0.72

 

 

 

Type of Validity

Age or
Grade

 

Test or
Criterion

N
(range)

Coefficient (if applicable)

 

Information (including normative data)/Subjects

range

Median

Content

 

 

 

 

 

Evidence for the content validity of TRC is demonstrated through the alignment of the test content to the Common Core State Standards (CCSS) for ELA.

 

Reference: Amplify (2013). How mClass:Reading 3D Text Reading and Comprehension Aligns with the Common Core State Standards in English Language Arts. Brooklyn, NY: Author.

Construct

 

 

 

 

 

Evidence for the construct validity of TRC is demonstrated through the procedures used to establish performance levels in reference to the CCSS for ELA.

 

Reference: Amplify (2013). mCLASS Reading3D TRC - Using the Common Core State Standards for English Language Arts to Revise Performance Standards. Brooklyn, NY: Author.

Concurrent

Grade K

DIBELS Next Composite Score

 

2,652

 

0.68

Validity was computed using data from end of year at school year 2011-2012 in North Carolina. 7% African American, 3% Hispanic.

Grade 1

2,678

 

0.81

Validity was computed using data from end of year at school year 2011-2012 in North Carolina. 8% African American, 3% Hispanic.

Grade 2

2,619

 

0.74

Validity was computed using data from end of year at school year 2011-2012 in North Carolina. 8% African American, 3% Hispanic.

Grade 3

1,156

 

0.77

Validity was computed using data from end of year at school year 2011-2012 in North Carolina. 16% African American, 8% Hispanic.

Grade 4

412

 

0.66

Validity was computed using data from end of year at school year 2011-2012 in North Carolina. 14% African American, 3% Hispanic.

Grade 5

409

 

0.72

Validity was computed using data from end of year at school year 2011-2012 in North Carolina. 15% African American, 6% Hispanic.

Predictive

Grade K

DIBELS Next Composite Score

1,799

 

0.58

Validity was computed using TRC data at middle of year to predict DIBELS end of year at school year 2011-2012 in North Carolina. 9% African American, 5% Hispanic.

Grade 1

1,826

 

0.76

Validity was computed using TRC data at middle of year to predict DIBELS end of year at school year 2011-2012 in North Carolina. 10% African American, 4% Hispanic.

Grade 2

1,709

 

0.73

Validity was computed using TRC data at middle of year to predict DIBELS end of year at school year 2011-2012 in North Carolina. 11% African American, 5% Hispanic.

Grade 3

1,162

 

0.74

Validity was computed using TRC data at middle of year to predict DIBELS end of year at school year 2011-2012 in North Carolina. 16% African American, 7% Hispanic.

Grade 4

409

 

0.59

Validity was computed using TRC data at middle of year to predict DIBELS end of year at school year 2011-2012 in North Carolina. 14% African American, 3% Hispanic.

Grade 5

408

 

0.72

Validity was computed using TRC data at middle of year to predict DIBELS end of year at school year 2011-2012 in North Carolina. 15% African American, 6% Hispanic.

 

Disaggregated Reliability, Validity, and Classification Data for Diverse Populations

Disaggregated Reliability, Validity, and Classification Data for Diverse Populations

Disaggregated Classification Accuracy

Classification Accuracy in Predicting Proficiency on the Grade 3 Indiana Statewide Testing for Educational Progress Plus (ISTEP+)
  White
n = 12,010
Black or African-American (Non-Hispanic)
n = 3,280
Hispanic or Latino
n = 2,007
False Positive Rate 0.08 0.20 0.14
False Negative Rate 0.51 0.38 0.45
Sensitivity 0.49 0.62 0.55
Specificity 0.92 0.80 0.86
Positive Predictive Power 0.58 0.66 0.70
Negative Predictive Power 0.89 0.77 0.76
Overall Classification Rate 0.84 0.73 0.74
AUC (ROC) 0.85 0.80 0.81
Base Rate 0.18 0.38 0.38
Cut Points: TRC Instructional Reading Level: “L” “L” “L”
At 90% Sensitivity, Specificity equals 63% 43% 41%
At 80% Sensitivity, Specificity equals 83% 67% 63%
At 70% Sensitivity, Specificity equals 83% 77% 76%

Disaggregated Reliability

Type of Reliability Grade n

Coefficient

(Median)

SEM Information/Subjects
Marginal Reliability (Primary Sample) K-3 1,076-30,057 0.93   For students in the sample indicated as ethnicity = “White”
K-3 269-7,945 0.92   For students in the sample indicated as ethnicity = “Black or African American”
K-3 63-5,157 0.92   For students in the sample indicated as ethnicity = “Hispanic-Latino”
Marginal Reliability (Cross-Validation) K-2 1,335-1,429 0.92   For students in the sample indicated as ethnicity = “White”
K-2 327-401 0.92   For students in the sample indicated as ethnicity = “Black or African American”
K-2 185-231 0.91   For students in the sample indicated as ethnicity = “Hispanic-Latino”
Marginal Reliability K-6 62,626 (951-20,388) 0.93 0.05 For students in the sample indicated as ethnicity = “White”
Marginal Reliability K-6 52,636 (1997-16,0023) 0.933 0.05 For students in the sample indicated as ethnicity = “Black or African American”
Marginal Reliability K-6 38,291 (749-12,528) 0.92 0.05 For students in the sample indicated as ethnicity = “Hispanic-Latino”

Disaggregated Validity

Type of Validity Grade Test or Criterion n (range) Coefficient Information/Subjects
median
Predictive (Primary Sample) 2 Grade 3 ISTEP+ ELA 12,010 0.68 For students in the sample indicated as ethnicity = “White”
2 Grade 3 ISTEP+ ELA 3,280 0.65 For students in the sample indicated as ethnicity = “Black or African American”
2 Grade 3 ISTEP+ ELA 2,007 0.69 For students in the sample indicated as ethnicity = “Hispanic-Latino”
Predictive (Cross-Validation) 2 Grade 3 EOG Reading Comprehension 674 0.73 For students in the sample indicated as ethnicity = “White”
2 Grade 3 EOG Reading Comprehension 213 0.74 For students in the sample indicated as ethnicity = “Black or African American”
2 Grade 3 EOG Reading Comprehension 77 0.69 For students in the sample indicated as ethnicity = “Hispanic-Latino”
Concurrent
(Primary Sample)
3 Grade 3 ISTEP+ ELA 1,007 0.73 For students in the sample indicated as ethnicity = “White”
3 Grade 3 ISTEP+ ELA 261 0.78 For students in the sample indicated as ethnicity = “Black or African American”
3 Grade 3 ISTEP+ ELA 59 0.42 For students in the sample indicated as ethnicity = “Hispanic-Latino”

Disaggregated Validity

Type of Validity Age or Grade Test or Criterion n (range) Coefficient (if applicable) Information (including normative data)/Subjects
range median
Concurrent (White) Grade K DIBELS Next Composite Score 967   0.70 Validity was computed using data from end of year at school year 2011-2012 in North Carolina. 
Concurrent (Black) 176   0.70
Concurrent (Hispanic) 85   0.78
Concurrent (White) Grade 1 DIBELS Next Composite Score 838   0.81
Concurrent (Black) 215   0.86
Concurrent (Hispanic) 83   0.77
Concurrent (White) Grade 2 DIBELS Next Composite Score 802   0.75
Concurrent (Black) 220   0.74
Concurrent (Hispanic) 91   0.73
Concurrent (White) Grade 3 DIBELS Next Composite Score 660   0.78
Concurrent (Black) 180   0.77
Concurrent (Hispanic) 88   0.73
Concurrent (White) Grade 4 DIBELS Next Composite Score 246   0.66
Concurrent (Black) 57   0.65
Concurrent (Hispanic) 13   0.46
Concurrent (White) Grade 5 DIBELS Next Composite Score 256   0.74
Concurrent (Black) 63   0.74
Concurrent (Hispanic) 26   0.69
Predictive (White) Grade K DIBELS Next Composite Score 951   0.59 Validity was computed using TRC data at middle of year to predict DIBELS end of year at school year 2011-2012 in North Carolina.
Predictive (Black) 166   0.65
Predictive (Hispanic) 83   0.48
Predictive (White) Grade 1 DIBELS Next Composite Score 811   0.78
Predictive (Black) 185   0.78
Predictive (Hispanic) 77   0.66
Predictive (White) Grade 2 DIBELS Next Composite Score 782   0.73
Predictive (Black) 182   0.75
Predictive (Hispanic) 92   0.69
Predictive (White) Grade 3 DIBELS Next Composite Score 658   0.76
Predictive (Black) 186   0.75
Predictive (Hispanic) 87   0.70
Predictive (White) Grade 4 DIBELS Next Composite Score 246   0.62
Predictive (Black) 56   0.60
Predictive (Hispanic) 13   0.35
Predictive (White) Grade 5 DIBELS Next Composite Score 257   0.75
Predictive (Black) 62   0.70
Predictive (Hispanic) 26   0.70