i-Ready

Diagnostic for Reading/English Language Arts

Rating Summary

Classification Accuracyfull bubble
GeneralizabilityBroad
Reliabilityfull bubble
Validityfull bubble
Disaggregated Reliability and Validity Datafull bubble
Efficiency
AdministrationIndividual
Administration & Scoring Time30-60 Minutes
Scoring KeyComputer Scored
Benchmarks / NormsYes

Cost

Technology, Human Resources, and Accommodations for Special Needs

Service and Support

Purpose and Other Implementation Information

Usage and Reporting

i-Ready Diagnostic for Mathematics costs $6.00 per student for annual license which includes online student access to assessment, plus staff access to management and reporting suite, downloadable lesson plans, and user resources; account set-up and secure hosting; all program maintenance/ updates/enhancements during the active license term; unlimited user access to U.S.-based service and support via toll-free phone and email during business hours.

Onsite professional development sessions (each lasting up to 3-4 hours for up to 30 participants) cost an additional $1,400/session. Live online webinars are available as a lower-cost option for $500/session.

Computer and internet access is required. 

Testers will require less than 4-8 hours of training.

Paraprofessionals can administer the test. 

Students with Disabilities

Curriculum Associates developed all items in i-Ready to be as accessible as possible for all students. In most cases, students who require accommodations will not require additional help during administration. The thoughtful planning that went into the general design of the assessment ensures that a large percentage of students requiring accommodations will have the necessary adjustments without compromising the interpretation or purpose of the test. i-Ready works with third-party (external) assistive technologies, such as screen readers and screen magnifiers.

Curriculum Associates, LLC
153 Rangeway Road

North Billerica, MA 01862

Phone: (800) 225-0248

www.curriculumassociates.com

www.i-ready.com/support

User Guide, video tutorials, and additional training and implementation resources are provided online to supplement onsite professional development.

Ongoing support is available from dedicated account manager plus unlimited access to in-house technical support during business hours.

Offering a continuum of scale scores from kindergarten through high school, i‑Ready Diagnostic is a web-based, K-12 screening assessment for reading/English language arts. i-Ready was designed and developed specifically to assess student mastery of the Common Core State Standards (CCSS). The assessment meets the expected rigor in each of the covered CCSS domains—Phonological Awareness, Phonics, High-Frequency Words, Vocabulary, Comprehension of Informational Text, and Comprehension of Literature—providing real-time, actionable information for each domain.

Screening is administered up to three times per academic year, with 12-18 weeks of instruction between assessments. Each screening takes 30-60 minutes—which may be broken into multiple administrations, if desired—and may be conducted with all students or with specific groups of students who have been identified as at risk of academic failure. i-Ready’s sophisticated adaptive algorithm automatically selects from thousands of multiple-choice and technology-enhanced items to get to the core of each student's strengths and challenges, regardless of the grade level at which he or she is performing. The depth of the item bank enables the assessment to truly pinpoint each student’s ability and ensures the accuracy of results.

The system automatically analyzes, scores, and reports student responses and results. Available as soon as a student completes the assessment, i‑Ready’s intuitive reports provide comprehensive information (including developmental analyses) about student performance, group students who struggle with the same concepts, make instructional recommendations to target skill deficiencies, and monitor progress and growth as students follow their individualized instructional paths.

Reports include suggested next steps for instruction and PDF Tools for Instruction lesson plans for the teacher to use during individual, small-group, or whole-class instruction. In addition, should educators also purchase the optional i‑Ready Instruction, the system automatically prescribes online lessons that address each student’s identified academic needs.

 

Classification Accuracy

Primary Sample

Classification Accuracy in Predicting Proficiency on Smarter Balanced Assessment Consortium (SBAC)

False Positive Rate

0.12

False Negative Rate

0.25

Sensitivity

0.75

Specificity

0.88

Positive Predictive Power

0.54

Negative Predictive Power

0.95

Overall Classification Rate

0.86

AUC (ROC)

0.88-0.90 across grades 3-8; median AUC=0.90

Base Rate

0.16

Cut Points:

Grade 3: 465

Grade 4: 488

Grade 5: 512

Grade 6: 523

Grade 7: 539

Grade 8: 553

At 90% Sensitivity, Specificity equals

0.69

At 80% Sensitivity, Specificity equals

0.83

At 70% Sensitivity, Specificity equals

0.91

 

Cross-Validation Sample

Classification Accuracy in Predicting Proficiency on New York State (NYS) assessments

False Positive Rate

0.23

False Negative Rate

0.14

Sensitivity

0.86

Specificity

0.77

Positive Predictive Power

0.56

Negative Predictive Power

0.94

Overall Classification Rate

0.79

AUC (ROC)

0.86-0.91 across grades 3-8 with median AUC=0.89

Base Rate

0.25

Cut Points:

Grade 3: 465

Grade 4: 488

Grade 5: 512

Grade 6: 523

Grade 7: 539

Grade 8: 553

At 90% Sensitivity, Specificity equals

0.66

At 80% Sensitivity, Specificity equals

0.81

At 70% Sensitivity, Specificity equals

0.89

 

Generalizability

Description of study sample:

  • Number of States: 4
  • Regions: West and Midwest
  • Gender
    • 47% Male
    • 44% Female
    • 9% Unknown
  • SES: 20% eligible for free or reduced-price lunch
  • Race/Ethnicity:
    • 30% White, Non-Hispanic
    • 0.3% American Indian/Alaska Native
    • 4% Black, Non-Hispanic
    • 13% Asian, Pacific Islander
    • 23% Hispanic
    • 3% Other
    • 26% Unknown
  • Language proficiency status: 12% English language learners

Description of cross-validation study sample:

  • Number of States: 1
  • Regions: Northeast
  • Gender
    • 45% Male
    • 44% Female
    • 11% Unknown
  • SES: 26% eligible for free or reduced-price lunch
  • Race/Ethnicity:
    • 27% White, Non-Hispanic
    • 0.2% American Indian/Alaska Native
    • 26% Black, Non-Hispanic
    • 6% Asian, Pacific Islander
    • 14% Hispanic
    • 2% Other
    • 26% Unknown
  • Language proficiency status: 20% English language learners

Reliability

Type of Reliability

Age or Grade

n (range)

Coefficient

SEM

Information (including normative data)/Subjects

range

median

Rasch person reliability

Grades K-12

2.8 million

 

0.97

 

Person reliability estimate from Rasch item calibration conducted on the entire i-Ready item bank for reading/ELA in spring 2014.

 

SEM in logit unit

Grades K-12

13,100 to 665,100

 

 

Range 0.24-0.28, median 0.28

In an IRT model, SEMs are affected by factors such as how well the data fit the underlying model, student response consistency, student location on the ability continuum, match of items to student ability, and test length.  Given the adaptive nature of i-Ready and the wide difficulty range in the item bank, standard errors are expected to be low and very close to the theoretical minimum for the test of the given length.

 

The theoretical minimum would be reached if each interim estimate of student ability is assessed by an item with difficulty matching perfectly to the student’s ability estimated from previous items. Theoretical minimums are restricted by the number of items served in the assessment—the more items that are served up, the lower the SEM could potentially be.

 

The ranges of SEMs are based on all i-Ready Diagnostic assessments taken during the 2014–2015 school year. The minimally obtainable SEM for i-Ready Reading is 0.23 to 0.27, depending on the grade and test flow.

 

Test-retest

Grades K-12

921-93,195

0.69-0.89

0.84

 

Evidence of test-retest stability was assessed based on a subsample of students who, during the 2014–2015 school year, took i-Ready Diagnostic twice within the recommended 12-18 week testing window. The average testing interval is 106 days (15 weeks). Correlations between the two tests were calculated. In lower grades where growth and variability are expected to be higher, test-retest correlations are expected to be relatively lower.

 

Test-retest correlations for grades K-12 reading are 0.69, 0.80, 0.82, 0.83, 0.85, 0.84, 0.85, 0.84, 0.84, 0.85, 0.86, 0.89, and 0.85, respectively.

 

 

Validity

Type of Validity

Age or Grade

Test or Criterion

n (range)

Coefficient (if applicable)

Information (including normative data)/Subjects

range

Median

Concurrent/

Construct

Grades 3-5

2013 Illinois Standards Achievement Test

301-353

0.78-0.81

0.81

Correlations between state assessments and the i-Ready spring screener. Testing interval between the two assessments is within six weeks.

Concurrent/

Construct

Grades 3-8

2014 New York State Assessments

374-948

0.81-0.83

0.82

Correlations between state assessments and the i-Ready spring screener. Testing interval between the two assessments is within six weeks.

Concurrent/

Construct

Grades 3-8

2014 Virginia Standards of Learning Assessment

387-514

0.76-0.83

0.81

Correlations between state assessments and the i-Ready spring screener. Testing interval between the two assessments is within six weeks.

Concurrent/

Construct

Grades 3-6

2014 Utah Student Assessment of Growth and Excellence

236-418

0.80-0.84

0.82

Correlations between state assessments and the i-Ready spring screener. Testing interval between the two assessments is within six weeks.

Concurrent/

Construct

Grades 3-4

2014 Florida Comprehensive Assessment Test

100-130

0.73-0.89

0.81

Correlations between state assessments and the i-Ready spring screener. Testing interval between the two assessments is within six weeks.

Concurrent/

Construct

Grades 3-8

2015 SBAC Assessments

4,767-5,994

0.81-0.83

0.81

Correlations between the spring i‑Ready test and spring SBAC assessments. Testing interval is between 1–3 months. Sample is from four states, 17 districts.

Predictive

Grade 2

2014 New York State Assessments

818

 

0.79

Correlations between the spring i‑Ready screening tests taken at 2nd grade and students’ state assessment scores one year later at 3rd grade. 

Predictive

Grade 2

2014 Utah Student Assessment of Growth and Excellence

98

 

0.72

Correlations between the spring i‑Ready screening tests taken at 2nd grade and students’ state assessment scores one year later at 3rd grade. 

Predictive

 

Grades 3-8

2015 SBAC Assessments

5,910-7,546

0.77-0.79

0.79

Correlations between the fall i‑Ready test and spring SBAC assessments. Testing interval is between 5-8 months.

Content Validity: As part of the accumulation of evidence for the valid use of an instrument, it is important that the test is measuring the intended construct. Much of this evidence is obtained through the four building-block process. As Mark Wilson explains, “…documentation of the steps taken… constitute a thorough representation of the content validity evidence for the instrument. It also lays the foundation for the remaining aspects of validity evidence—in a sense, the evidence related to content is the target of the remaining evidence”. A detailed discussion of the development process as it relates to test content is provided in Chapter 2 of the i-Ready Technical Manual.

Disaggregated Reliability, Validity, and Classification Data for Diverse Populations

Disaggregated Classification analysis:

Classification analyses were performed for three sub groups of students who had taken both the i-Ready Diagnostic in the Fall and the New York State (NYS) Assessment in the following Spring. The three groups are: English Language Learners (ELL), African American, and Hispanic. Sample sizes range from 264-327 across grades 3 to 8 for the ELL sample; from 172-179 across grades 3-6 for the African American sample; and from 63 to 112 across grades 3 to 8 for the Hispanic sample.

Students included in the study:

Students in the disaggregated classification analysis were recruited from 18 general education school districts.

External criterion measure for the disaggregated classification study and how it is broken down into risk and no-risk categories:

The proficiency levels defined by 2014 NYS Assessments are used to classify students. Students who were below Level II on the NYS test were classified as risk and students who were at or above Level II were classified as no-risk.                                                                                                                                                      

How the disaggregated classification analyses were performed and cut-points determined:

The i-Ready cut scores were determined at the 20th percentile on the i-Ready National Norms. Using these cut scores, students were identified as at-risk if they were below the 20th percentile in the fall i‑Ready Diagnostic test or no-risk if they scored at or above the cut. Classification indices between the at-risk/no-risk on i-Ready and at-risk/no-risk on the NYS assessment are calculated per the classification worksheet.

AUC values are calculated using the Risk/No-Risk categories on the criterion (outcome) measure as the dependent variable and the i-Ready score as the independent variable.

Results: Classification data are presented in tables 1 and 2 below.

Table 1. AUC by grade and group

Grade African American Hispanic ELL

3

0.90

0.87

0.94

4

0.91

0.92

0.95

5

0.90

0.91

0.95

6

0.85

0.93

0.95

7

--*

0.89

0.87

8

--*

0.85

0.90

*Data omitted due to small sample size
 
Table 2. Additional classification analyses data by group
Group False Positive Rate False Negative Rate Sensitivity Specificity Positive Predictive Power

Negative Predictive Power

Overall Classification Rate Sample Base Rate

African American

0.41

0.07

0.93

0.59

0.62

0.92

0.73

0.42

ELL

0.22

0.08

0.92

0.78

0.59

0.97

0.81

0.26

Hispanic

0.30

0.04

0.96

0.70

0.55

0.98

0.77

0.28

 
Table 2 (continued). Specificity for Various Sensitivities
Group Sensitivity 70% Sensitivity 80% Sensitivity 90%

African American

0.88

0.82

0.66

ELL

0.94

0.88

0.76

Hispanic

0.89

0.85

0.70

 
Disaggregated Reliability

Type of Reliability

Age or Grade

n (range)

Coefficient

SEM

Information (including normative data)/Subjects

range

median

SEM in logit unit (Economically disadvantaged)

Grades K-12

629,876

 

 

Range 0.244-0.280, median 0.274

Data are based on the i-Ready 2014-2015 operational pool of students who were identified with each special group membership. The minimally obtainable SEM for i‑Ready Reading is 0.23 to 0.27, depending on the grade and test flow.

SEM in logit unit (ELL)

Grades K-12

610,786

 

 

Range 0.242-0.278, median 0.270

See above

SEM in logit unit (Special Education)

Grades  K-12

310,435

 

 

Range 0.244-0.278, median 0.268

See above

Test-retest reliability (Economically Disadvantaged)

Grade K-10

Total 78,105; range by grade (K-8):

3,590 to 12,093; range by grade (9-10): 137-689

0.68 to 0.85

0.82

 

Evidence of disaggregated test-retest stability was assessed based on a subsample of students in the respective demographic groups who, during the 2014–2015 school year, took i-Ready Diagnostic twice within the recommended 12-18 week testing window. The average testing interval is 106 days (15 weeks).

Correlations between the two tests were calculated. In lower grades where growth and variability are expected to be higher, test-retest correlations are expected to be relatively lower. (grades with sample size less than 100 are excluded)

Test-retest reliability (ELL)

Grade K-10

Total 72,282; range by grade (K-8): 2,606 to 13,656; range by grade (9-10): 123-169

0.63 to 0.85

0.81

 

See above

Test-retest reliability (Special Education)

Grade K-12

Total 59,255; range by grade (K-8): 1,409  to 9,763; range by grade (9-12): 138 to 321

0.68 to 0.87

0.85

 

See above

 

Disaggregated Validity

 

Type of Validity

Age or Grade

 

Test or Criterion

n (range)

Coefficient (if applicable)

 

Information (including normative data)/Subjects

range

Median

Concurrent (ELL)

Grades 3-8

2014 New York State Assessments

275-331

0.82-0.89

0.87

Students who took both NYS assessments and the spring i-Ready Diagnostic. The testing interval is six weeks.

Concurrent (African American)

Grades 3-6

2014 New York State Assessments

184-212

0.68-0.76

0.74

Students who took both NYS assessments and the spring i-Ready Diagnostic. The testing interval is six weeks.

Concurrent (Hispanic)

Grades 3-8

2014 New York State Assessments

63-137

0.73-0.84

0.80

Students who took both NYS assessments and the spring i-Ready Diagnostic. The testing interval is six weeks.