Acuity

Mathematics

Rating Summary

Classification Accuracyhalf bubble
GeneralizabilityModerate High
Reliabilityhalf bubble
Validityfull bubble
Disaggregated Reliability and Validity DataN A
Efficiency
AdministrationGroup
Administration & Scoring Time50 Minutes
Scoring KeyComputer Scored
Benchmarks / NormsYes
Cost Technology, Human Resources, and Accommodations for Special Needs Service and Support Purpose and Other Implementation Information Usage and Reporting

Annual cost per student:
Acuity is licensed using an annual subscription, by student and content area. Acuity costs $9.50 for one subject (Math or Reading); $11.90 for both subjects per student per year.

In additional to the subscription price, a onetime set up fee and one day of professional development are required in the first year only, for a total of $5,200.

 

 

Internet access is required for full use of product services.

Testers will require less than one hour of training.

Paraprofessionals can administer the test.

Accommodations:
Students are restricted to a 45 minute testing window without the use of a calculator. However for students with disabilities, Acuity enables tests to be un-timed and to be completed using an online calculator. In addition, assessments may be administered with accommodations as defined by students’ IEPs and as provided in regular classroom assessments.

Tom Moellering
1333 Burr Ridge Parkway
Burr Ridge, IL 60527

Phone: 630-789-4574
Website: http://www.mheducation.com/prek-12/platforms/acuity.html

A local point of contact, high quality professional development staff, and tiered technical support for all system users is included in the subscription price.

Ongoing technical support is provided though a call center, email, a 24x7 online knowledge database, and a 24x7 online community of other users to share tips and best practices.

Acuity is an online assessment solution designed to improve student achievement and learning. Available in Math and ELA for students in grades 3-8, Acuity provides three predictive screening assessments within each student and grade.

Acuity assessments can be administered within a single 45 minute classroom period in a variety of modes, including online, paper/pencil, or using clicker devices.

Scoring is done automatically when scores are submitted online through scan and score or online administration.

The following scores are available: raw, standard, percentile, IRT-based, equated, developmental benchmarks, subscale/subtest, error analysis, and predicted performance level on state summative test.

 

Classification Accuracy

Classification Accuracy in Predicting Proficiency Level on State Summative NCLB Assessment
MO, Form A Grade 3
n = 3,576
Grade 4
n = 3,958
Grade 5
n = 3,510
Grade 6
n = 3,295
Grade 7
n = 2,481
Grade 8
n = 2,439
False Positive Rate 0.07 0.05 0.07 0.06 0.08 0.14
False Negative Rate 0.42 0.38 0.44 0.38 0.36 0.35
Sensitivity 0.58 0.62 0.57 0.62 0.64 0.65
Specificity 0.93 0.95 0.93 0.94 0.92 0.86
Positive Predictive Power 0.17 0.44 0.10 0.48 0.53 0.45
Negative Predictive Power 0.99 0.98 0.99 0.97 0.95 0.93
Overall Classification Rate 0.92 0.93 0.92 0.92 0.88 0.83
AUC (ROC) 0.91 0.94 0.90 0.92 0.91 0.87
Base Rate 0.02 0.06 0.01 0.08 0.13 0.15
Cut Points: 293* 315* 327* 390* 455* 492*
At 90% Sensitivity, Specificity equals 0.80 0.87 0.76 0.76 0.77 0.70
At 80% Sensitivity, Specificity equals 0.86 0.92 0.76 0.88 0.86 0.76
At 70% Sensitivity, Specificity equals 0.90 0.94 0.88 0.90 0.90 0.85

*The Acuity cut point is determined through a prediction study of matched student data for Acuity and the associated state summative NCLB test. The cut score is the Acuity score that best predicted the cut score for the second lowest performance level on the state summative. Students with scores below that Acuity cut score were identified as at risk of being in the lowest performance level on the state summative.

Generalizability

Description of study sample:

  • Number of States: 1 (Missouri)
  • Size: Approximately 2,500 – 4,000 per grade and content area
  • Gender:
    • 50.9% Male
    • 49.1% Female
  • SES: 24.4% Eligible for free or reduced-price lunch
  • Race/Ethnicity:
    • 53.1% White, Non-Hispanic
    • 30.3% Black, Non-Hispanic
    • 14.1% Hispanic
    • 0.7% American Indian/Alaska Native
    • 1.8% Asian
    • 0.0% Native Hawaiian/Pacific Islander

Note: Acuity is developed specifically for each state and therefore it is not appropriate to have a representative national sample but rather a representative state-specific sample for each state-specific version of Acuity. That is, in Colorado there is a representative sample of Colorado students. In Missouri there is a representative sample of Missouri students. This provides generalizability to students in each state. Each state has its own state content standards, and each state-specific Acuity form measures the content standard for that state. The Missouri Acuity form data is provided as an example that is typical of Acuity in different states.

Reliability

Type of Reliability Age or Grade n Coefficient
median
Information (including normative data)/Subjects
Feldt- Raju Mathematics
Grade 3
7,144 0.89 State representative sample of more than 2500 students.
Feldt- Raju Mathematics
Grade 4
7,366 0.88 State representative sample of more than 2500 students.
Feldt- Raju Mathematics
Grade 5
7,201 0.82 State representative sample of more than 2500 students.
Feldt- Raju Mathematics
Grade 6
7,264 0.86 State representative sample of more than 2500 students.
Feldt- Raju Mathematics
Grade 7
5,918 0.86 State representative sample of more than 2500 students.
Feldt- Raju Mathematics
Grade 8
5,607 0.82 State representative sample of more than 2500 students.

Validity

Type of Validity Age or Grade Test or Criterion n (range) Coefficient (if applicable) Information (including normative data)/Subjects
Range Median
Construct 3 Criterion: Grade 3 MAP (Math) 6,863 NA 0.75 Construct validity is supported by the correlation of the Acuity form C test and the state NCLB test, which are typically administered within six to eight weeks of each other.
Content 3 Criterion: Grade 3 MAP (Math) NA (supported by test design) NA NA Content validity is supported by (a) the Acuity test design which reflects the content structure of the associated and subsequent state NCLB assessment in terms of the percent of items measuring each standard, (b) the alignment of the items with the state content standards, and (c) the alignment of the Acuity item formats (selected-response and constructed-response) with that of the criterion assessment—the state summative assessments. Thus, the content structure of Acuity and the associated and subsequent state summative assessments are congruent, as described in the Acuity technical report.
Predictive 3 Criterion: Grade 3 MAP (Math) 2,004 0.67-0.72 Acuity/MAP Correlation: 0.70

Classification Consistency: 93.72%

The correlation is the Pearson correlation for matched data between Acuity and the criterion MAP test.

The classification accuracy is the percent of students whose predicted and observed performance levels matched exactly when predicting to “Level 1: Below Basic” versus “Level 2 or Higher: Basic, Proficient, or Advanced.” This information is provided in Tables 3 and 18 of the Acuity Prediction Study Technical Report provided.

Construct 4 Criterion: Grade 4 MAP (Math) 6,861 NA 0.80 Construct validity is supported by the correlation of the Acuity form C test and the state NCLB test, which are typically administered within six to eight weeks of each other.
Content 4 Criterion: Grade 4 MAP (Math) NA (supported by test design) NA NA Content validity is supported by (a) the Acuity test design which reflects the content structure of the associated and subsequent state NCLB assessment in terms of the percent of items measuring each standard, (b) the alignment of the items with the state content standards, and (c) the alignment of the Acuity item formats (selected-response and constructed-response) with that of the criterion assessment—the state summative assessments. Thus, the content structure of Acuity and the associated and subsequent state summative assessments are congruent, as described in the Acuity technical report.
Predictive 4 Criterion: Grade 4 MAP (Math) 2,313 0.76-0.78 Acuity/MAP Correlation: 0.77

Classification Consistency: 93.77%

The correlation is the Pearson correlation for matched data between Acuity and the criterion MAP test.

The classification accuracy is the percent of students whose predicted and observed performance levels matched exactly when predicting to “Level 1: Below Basic” versus “Level 2 or Higher: Basic, Proficient, or Advanced.” This information is provided in Tables 3 and 18 of the Acuity Prediction Study Technical Report provided.

Construct 5 Criterion: Grade 5 MAP (Math) 6,741 NA 0.77 Construct validity is supported by the correlation of the Acuity form C test and the state NCLB test, which are typically administered within six to eight weeks of each other.
Content 5 Criterion: Grade 5 MAP (Math) NA (supported by test design) NA NA Content validity is supported by (a) the Acuity test design which reflects the content structure of the associated and subsequent state NCLB assessment in terms of the percent of items measuring each standard, (b) the alignment of the items with the state content standards, and (c) the alignment of the Acuity item formats (selected-response and constructed-response) with that of the criterion assessment—the state summative assessments. Thus, the content structure of Acuity and the associated and subsequent state summative assessments are congruent, as described in the Acuity technical report.
Predictive 5 Criterion: Grade 5 MAP (Math) 1,787 0.70-0.74 Acuity/MAP Correlation: 0.72 Classification Consistency: 92.84% The correlation is the Pearson correlation for matched data between Acuity and the criterion MAP test.

The classification accuracy is the percent of students whose predicted and observed performance levels matched exactly when predicting to “Level 1: Below Basic” versus “Level 2 or Higher: Basic, Proficient, or Advanced.” This information is provided in Tables 3 and 18 of the Acuity Prediction Study Technical Report provided.

Construct 6 Criterion: Grade 6 MAP (Math) 5,826 NA 0.76 Construct validity is supported by the correlation of the Acuity form C test and the state NCLB test, which are typically administered within six to eight weeks of each other.
Content 6 Criterion: Grade 6 MAP (Math) NA (supported by test design) NA NA Content validity is supported by (a) the Acuity test design which reflects the content structure of the associated and subsequent state NCLB assessment in terms of the percent of items measuring each standard, (b) the alignment of the items with the state content standards, and (c) the alignment of the Acuity item formats (selected-response and constructed-response) with that of the criterion assessment—the state summative assessments. Thus, the content structure of Acuity and the associated and subsequent state summative assessments are congruent, as described in the Acuity technical report.
Predictive 6 Test: Acuity Grade 6 Math. Criterion: Grade 6 MAP (Math) 1,552 0.73-0.77 Acuity/MAP Correlation: 0.75

Classification Consistency: 93.45%

The correlation is the Pearson correlation for matched data between Acuity and the criterion MAP test.

The classification accuracy is the percent of students whose predicted and observed performance levels matched exactly when predicting to “Level 1: Below Basic” versus “Level 2 or Higher: Basic, Proficient, or Advanced.” This information is provided in Tables 3 and 18 of the Acuity Prediction Study Technical Report provided.

Construct 7 Criterion: Grade 7 MAP (Math) 4,493 NA 0.73 Construct validity is supported by the correlation of the Acuity form C test and the state NCLB test, which are typically administered within six to eight weeks of each other.
Content 7 Criterion: Grade 7 MAP (Math) NA (supported by test design) NA NA Content validity is supported by (a) the Acuity test design which reflects the content structure of the associated and subsequent state NCLB assessment in terms of the percent of items measuring each standard, (b) the alignment of the items with the state content standards, and (c) the alignment of the Acuity item formats (selected-response and constructed-response) with that of the criterion assessment—the state summative assessments. Thus, the content structure of Acuity and the associated and subsequent state summative assessments are congruent, as described in the Acuity technical report.
Predictive 7 Criterion: Grade 7 MAP (Math) 1,101 0.75-0.77 Acuity/MAP Correlation: 0.76

Classification Consistency: 90.19%

The correlation is the Pearson correlation for matched data between Acuity and the criterion MAP test.

The classification accuracy is the percent of students whose predicted and observed performance levels matched exactly when predicting to “Level 1: Below Basic” versus “Level 2 or Higher: Basic, Proficient, or Advanced.” This information is provided in Tables 3 and 18 of the Acuity Prediction Study Technical Report provided.

Construct 8 Criterion: Grade 8 MAP (Math) 4,282 NA 0.73 Construct validity is supported by the correlation of the Acuity form C test and the state NCLB test, which are typically administered within six to eight weeks of each other.
Content 8 Criterion: Grade 8 MAP (Math) NA (supported by test design) NA NA Content validity is supported by (a) the Acuity test design which reflects the content structure of the associated and subsequent state NCLB assessment in terms of the percent of items measuring each standard, (b) the alignment of the items with the state content standards, and (c) the alignment of the Acuity item formats (selected-response and constructed-response) with that of the criterion assessment—the state summative assessments. Thus, the content structure of Acuity and the associated and subsequent state summative assessments are congruent, as described in the Acuity technical report.
Predictive 8 Criterion: Grade 8 MAP (Math) 1,033 0.70-0.73 Acuity/MAP Correlation: 0.72

Classification Consistency: 85.69%

The correlation is the Pearson correlation for matched data between Acuity and the criterion MAP test.

The classification accuracy is the percent of students whose predicted and observed performance levels matched exactly when predicting to “Level 1: Below Basic” versus “Level 2 or Higher: Basic, Proficient, or Advanced.” This information is provided in Tables 3 and 18 of the Acuity Prediction Study Technical Report provided.