AIMSweb

Mathematics Concepts and Applications

Rating Summary

Classification Accuracyhalf bubble
GeneralizabilityModerate Low
Reliabilityfull bubble
Validityempty bubble
Disaggregated Reliability and Validity Dataempty bubble
Efficiency
AdministrationIndividual Group
Administration & Scoring Time11-13 Minutes
Scoring KeyComputer Scored
Benchmarks / NormsYes
Cost Technology, Human Resources, and Accommodations for Special Needs Service and Support Purpose and Other Implementation Information Usage and Reporting

Annual cost per student:
AIMSweb M-CAP costs $4.00 per student per year.

Training manuals, administration and scoring manuals, and materials for benchmarking and progress monitoring are available for download* on www.aimsweb.com. Also included in the subscription is ongoing technical support available online and by phone.

*All materials are provided via download in PDF format.

Internet access is required for full use of product services.

Testers will require 1 – 4 hours of training.

Paraprofessionals can administer the test.

M-CAP underwent an intensive data collection and rigorous analysis during its standardization to avoid cultural and linguistic bias; this included cultural and sensitivity review and revision of all probes.

Pearson Assessment Inc.
19500 Bulverde Road
San Antonio, TX 78259

Phone: 866-313-6194

Website: AIMSweb.com

Access to field tested training manuals is included with AIMSweb subscriptions which provide administration, scoring and implementation information.

Ongoing technical support is provided.

M-CAP is a test that assesses the general mathematics problem-solving skills expected in grades 2-8. The mathematics domains assessed include number sense, operations, patterns and relationships, data and probability, measurement, data and statistics, geometry, and algebra.

Administration time is 8 minutes for grades 2-6, and 10 minutes for grades 7 and 8. Administration time is the same for individuals or groups. Scoring time is 3 minutes per student.

Scoring is done by computing for the total point value on the Answer Key. The rest may be administered in a large or small group setting or to individual students.

Raw score, percentile score, and developmental benchmark scores are available.

Raw scores are determined by computing for the sum of the item points in the Correct column of the Answer Key. The sum is the student’s total score for the probe. These data can be interpreted in a norm-referenced way via grade-based percentiles, or categorically in a standard interpretive format.

 

Classification Accuracy

Classification Accuracy in Predicting Proficiency on North Carolina End of Grade Test
  Grade 3
Fall Winter
False Positive Rate 0.32 0.28
False Negative Rate 0.20 0.18
Sensitivity 0.80 0.82
Specificity 0.68 0.72
Positive Predictive Power 0.30 0.42
Negative Predictive Power 0.95 0.94
Overall Classification Rate 0.70 0.74
AUC (ROC) 0.81 0.84
Base Rate 0.15 0.19
Cut Points: 6 8
At 90% Sensitivity, Specificity equals 0.58 0.65
At 80% Sensitivity, Specificity equals 0.68 0.72
At 70% Sensitivity, Specificity equals 0.79 0.80

 

Classification Accuracy in Predicting Proficiency on Illinois Standards Achievement Test (ISAT)
  Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
Fall Winter Fall Winter Fall Winter Fall Winter Fall Winter
False Positive Rate 0.25 0.26 0.30 0.25 0.18 0.24 0.22 0.23 0.21 0.24
False Negative Rate 0.23 0.25 0.22 0.24 0.15 0.20 0.18 0.17 0.17 0.17
Sensitivity 0.77 0.75 0.78 0.76 0.85 0.80 0.82 0.83 0.83 0.83
Specificity 0.75 0.74 0.70 0.75 0.82 0.76 0.78 0.77 0.79 0.76
Positive Predictive Power 0.26 0.27 0.21 0.26 0.56 0.51 0.53 0.53 0.53 0.50
Negative Predictive Power 0.97 0.96 0.97 0.96 0.95 0.93 0.93 0.93 0.94 0.94
Overall Classification Rate 0.75 0.74 0.70 0.75 0.82 0.77 0.79 0.79 0.80 0.77
AUC (ROC) 0.84 0.83 0.82 0.83 0.91 0.85 0.88 0.89 0.88 0.87
Base Rate 0.10 0.11 0.10 0.10 0.22 0.24 0.24 0.24 0.23 0.23
Cut Points: 8 12 6 8 7 11 8 12 6 8
At 90% Sensitivity, Specificity equals 0.58 0.60 0.61 0.64 0.82 0.71 0.72 0.72 0.71 0.70
At 80% Sensitivity, Specificity equals 0.75 0.74 0.70 0.75 0.87 0.80 0.83 0.82 0.79 0.82
At 70% Sensitivity, Specificity equals 0.83 Not reported 0.79 Not reported 0.90 Not reported 0.89 0.91 0.89 0.85

 

Generalizability

Description of study sample:

  Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
Fall Winter Fall Winter Fall Winter Fall Winter Fall Winter Fall Winter
Number of States: 1 1 1 1 1 1 1 1 1 1 1 1
Size: 553 731 700 700 752 736 631 855 723 942 640 783
Gender Male 37% 40% 46% 46% 50% 50% 53% 52% 46% 47% 48% 50%
Female 39% 41% 45% 45% 45% 45% 43% 46% 49% 50% 43% 44%
Unknown 24% 19% 9% 9% 5% 5% 4% 3% 4% 3% 9% 6%
SES: Eligible for free or reduced-price lunch <1% <1% 30% 31% 22% 19% 24% 26% 15% 16% 19% 21%
Race/ Ethnicity White, Non-Hispanic 39% 35% 62% 61% 75% 77% 58% 53% 58% 54% 59% 58%
Black, Non-Hispanic 18% 25% 6% 8% 7% 7% 6% 10% 4% 8% 4% 7%
Hispanic 9% 12% 7% 7% 6% 4% 11% 15% 7% 11% 6% 10%
American Indian/Alaska Native 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%
Asian/Pacific Islander 4% 4% 1% 1% 1% 1% 2% 3% 2% 3% 4% 3%
Other 3% 4% 2% 2% 2% 2% 1% 2% 1% 3% 1% 3%
Unknown 26% 21% 21% 21% 9% 9% 22% 16% 28% 22% 25% 18%
Disability classification: % with disability classification 4% 4% 10% 10% 7% 7% 10% 9% 13% 11% 13% 10%
Language proficiency status: % ELL 6% 4% 3% 3% 2% 1% 4% 4% 1% 1% 1% 1%

Reliability

Type of Reliability Age or Grade n (range) Coefficient
median
SEM Information (including normative data)/Subjects
Alternate form 2 1,064 0.86 4.6 Average inter-probe correlation in the standardization sample

Gender: F 50%, M 50%
Ethnicity:
African American 9%
American Indian 2%
Asian 3%
Hispanic 26%
White, non-Hispanic 60%
Other 1%
Household income:
Low 50%
Middle 23%
High 27%
(national distribution is 33% per level)
Region:
Northeast 7%
Midwest 31%
South 52%
West 11%

Alternate form 3 965 0.81 4.5
Alternate form 4 1,026 0.80 5.1
Alternate form 5 867 0.84 4.3
Alternate form 6 858 0.86 4.4
Alternate form 7 912 0.88 4.2
Alternate form 8 858 0.86 4.0
Inter-rater 2 60 0.99   Cases pulled at random from the standardization sample.

Sex: Male 50%, Female 50%
Ethnicity:
African American 12%
Asian 4%
Hispanic 25%
White 58%
Other 2%
Household income:
Low 51%
Middle 27%
High 21%
Region:
Northeast 10%
Midwest 28%
South 52%
West 10%

Inter-rater 3 60 0.99  
Inter-rater 4 60 0.99  
Inter-rater 5 60 0.99  
Inter-rater 6 60 0.99  
Inter-rater 7 59 0.99  
Inter-rater 8 60 0.97  

 

Validity

Type of Validity Age or Grade Test or Criterion n (range) Coefficient Information (including normative data)/Subjects
Median
Predictive 3 (fall) NCEGT 553 0.63 (0.60) NCGET: North Carolina End of Grade Test
ISAT: Illinois Standards Achievement Test
4 (fall) ISAT 700 0.67 (0.60)
5 (fall) ISAT 752 0.57 (0.60)
6 (fall) ISAT 631 0.76 (0.78)
7 (fall) ISAT 723 0.61 (0.74)
8 (fall) ISAT 640 0.69 (0.74)
3 (winter) NCEGT 733 0.67 (0.64)
4 (winter) ISAT 699 0.60 (0.56)
5 (winter) ISAT 736 0.60 (0.63)
6 (winter) ISAT 855 0.75 (0.74)
7 (winter) ISAT 942 0.66 (0.76)
8 (winter) ISAT 783 0.71 (0.73)
Construct 3 (spring) NCEGT 736 0.64 (0.64) (largely the same samples as for the Classification Accuracy analyses)
4 (spring) ISAT 665 0.64 (0.58)
5 (spring) ISAT 746 0.60 (0.65)
6 (spring) ISAT 959 0.78 (0.78)
7 (spring) ISAT 930 0.71 (0.80)
8 (spring) ISAT 784 0.73 (0.76)

Content Validity:
The M–CAP content design was based on the National Council of Teachers of Mathematics (NCTM) standards, as well as on the principles set forth in the National Resource Council (NRC) report Adding it Up, which indicates that the curricula for grades K–8 should comprise a number of domains, of which an understanding of number concepts and operations are deemed critical. The M–CAP domains (depending on the grade) are as follows: Numbers Sense; Operations; Patterns & Relationships; Measurement; Geometry; Data & Probability; Algebra; Probability; and Data & Statistics. The content coverage of the Stanford Achievement Test, Tenth Edition (Stanford 10) served as a general guideline for determining the proportion of items by learning domains at each grade level. The Stanford 10 was chosen because it is one of the most widely used norm-referenced assessments of mathematics achievement in the United States. See table below for the breakdown of content by grade:

  Grade
2 3 4 5 6 7 8
Number Sense
Operations
Patterns & Relationships
Measurement
Geometry
Data & Probability      
Algebra      
Probability        
Data & Statistics        

Disaggregated Reliability, Validity, and Classification Data for Diverse Populations

Disaggregated Reliability, Validity, and Classification Data for Diverse Populations

Disaggregated Classification Accuracy

Classification Accuracy in Predicting Proficiency on North Carolina End of Grade Test
  Grade 3
Fall Winter
African American White, non-Hispanic African American White, non-Hispanic
False Positive Rate 0.51 0.16 0.40 0.20
False Negative Rate 0.29 0.14 0.21 0.00
Sensitivity 0.71 0.86 0.79 1.00
Specificity 0.49 0.84 0.60 0.80
Positive Predictive Power 0.43 0.15 0.56 0.16
Negative Predictive Power 0.76 0.99 0.82 1.00
Overall Classification Rate 0.57 0.84 0.68 0.81
AUC (ROC) 0.63 0.91 0.76 0.93
Base Rate 0.35 0.03 0.39 0.04
Cut Points: 6 5 8 8
At 90% Sensitivity, Specificity equals 0.22 0.84 0.49  
At 80% Sensitivity, Specificity equals 0.32   0.60 0.85
At 70% Sensitivity, Specificity equals 0.49   0.70 0.89

 

Classification Accuracy in Predicting Proficiency on Illinois Standards Achievement Test (ISAT)
  Grade 4 Grade 5
Fall Winter Fall Winter
Hispanic White, non-Hispanic African American White, non-Hispanic Hispanic White, non-Hispanic African American White, non-Hispanic
False Positive Rate 0.21 0.32 0.43 0.23 0.34 0.31 0.53 0.25
False Negative Rate 0.18 0.18 0.21 0.15 0.33 0.14 0.35 0.21
Sensitivity 0.82 0.82 0.79 0.85 0.67 0.86 0.65 0.79
Specificity 0.79 0.68 0.58 0.77 0.66 0.69 0.47 0.75
Positive Predictive Power 0.53 0.18 0.39 0.27 0.42 0.16 0.37 0.19
Negative Predictive Power 0.94 0.98 0.88 0.98 0.84 0.99 0.74 0.98
Overall Classification Rate 0.80 0.69 0.63 0.78 0.66 0.70 0.53 0.75
AUC (ROC) 0.84 0.85 0.75 0.88 0.66 0.85 0.69 0.84
Base Rate 0.22 0.08 0.26 0.09 0.27 0.06 0.32 0.07
Cut Points: 7 9 12 12 4 7 7 9
At 90% Sensitivity, Specificity equals 0.23 0.61 0.40 0.77 0.13 0.69 0.36 0.69
At 80% Sensitivity, Specificity equals 0.79 0.78 0.58 0.84 0.34   0.39 0.75
At 70% Sensitivity, Specificity equals 0.87 0.85   0.88 0.66 0.86 0.47 0.81

 

Classification Accuracy in Predicting Proficiency on Illinois Standards Achievement Test (ISAT)
  Grade 6 Grade 7 Grade 8
Fall Winter Fall Winter Fall Winter
Hispanic White, non-Hispanic Hispanic White, non-Hispanic Hispanic White, non-Hispanic Hispanic White, non-Hispanic Hispanic White, non-Hispanic Hispanic White, non-Hispanic
False Positive Rate 0.21 0.20 0.23 0.25 0.29 0.25 0.28 0.23 0.25 0.26 0.35 0.21
False Negative Rate 0.09 0.13 0.16 0.19 0.11 0.17 0.23 0.20 0.20 0.12 0.21 0.13
Sensitivity 0.91 0.88 0.84 0.81 0.89 0.83 0.77 0.80 0.80 0.88 0.79 0.87
Specificity 0.79 0.80 0.77 0.75 0.71 0.75 0.72 0.77 0.75 0.74 0.65 0.79
Positive Predictive Power 0.82 0.40 0.74 0.37 0.77 0.30 0.70 0.31 0.80 0.31 0.63 0.36
Negative Predictive Power 0.90 0.98 0.86 0.96 0.85 0.97 0.79 0.97 0.75 0.98 0.81 0.98
Overall Classification Rate 0.86 0.81 0.80 0.76 0.80 0.76 0.74 0.78 0.78 0.76 0.71 0.80
AUC (ROC) 0.93 0.91 0.84 0.85 0.82 0.86 0.80 0.88 0.83 0.89 0.81 0.90
Base Rate 0.51 0.13 0.44 0.15 0.53 0.12 0.46 0.11 0.56 0.11 0.43 0.12
Cut Points: 6 8 10 12 6 10 10 13 6 8 7 8
At 90% Sensitivity, Specificity equals 0.79 0.80 0.70 0.68 0.71 0.68 0.65 0.73   0.74 0.57 0.79
At 80% Sensitivity, Specificity equals 0.88 0.87 0.77 0.80   0.82 0.72 0.84 0.75 0.90 0.65 0.85
At 70% Sensitivity, Specificity equals   0.92   0.84 0.79 0.86 0.77 0.87 0.94 0.94 0.70 0.89

Disaggregated Validity

Type of Validity Age or Grade Test or Criterion n Coefficient Disaggregation
median
Predictive 3 (fall) NCEGT 100 0.66 (0.49) African American students in the Classification Accuracy samples
4 (fall) ISAT 44 0.78 (0.58)
5 (fall) ISAT 51 0.62 (0.34)
6 (fall) ISAT 38 0.81 (0.50)
7 (fall) ISAT 29 0.60 (0.34)
8 (fall) ISAT 28 0.63 (0.43)
3 (winter) NCEGT 185 0.68 (0.56)
4 (winter) ISAT 54 0.31 (0.25)
5 (winter) ISAT 53 0.54 (0.34)
6 (winter) ISAT 82 0.76 (0.55)
7 (winter) ISAT 73 0.84 (0.71)
8 (winter) ISAT 57 0.61 (0.50)
3 (fall) NCEGT 50 0.55 (0.40) Hispanic students in the Classification Accuracy samples
4 (fall) ISAT 50 0.64 (0.55)
5 (fall) ISAT 44 0.68 (0.55)
6 (fall) ISAT 69 0.88 (0.76)
7 (fall) ISAT 51 0.80 (0.65)
8 (fall) ISAT 36 0.83 (0.65)
3 (winter) NCEGT 86 0.63 (0.45)
4 (winter) ISAT 50 0.69 (0.60)
5 (winter) ISAT 26 0.66 (0.55)
6 (winter) ISAT 131 0.87 (0.73)
7 (winter) ISAT 105 0.76 (0.68)
8 (winter) ISAT 80 0.77 (0.68)
Construct 3 (spring) NCEGT 191 0.64 (0.54) African American students
4 (spring) ISAT 53 0.41 (0.34)
5 (spring) ISAT 59 0.42 (0.30)
6 (spring) ISAT 83 0.84 (0.68)
7 (spring) ISAT 67 0.89 (0.74)
8 (spring) ISAT 51 0.72 (0.55)
3 (spring) NCEGT 93 0.64 (0.50) Hispanic students
4 (spring) ISAT 45 0.69 (0.64)
5 (spring) ISAT 41 0.73 (0.69)
6 (spring) ISAT 128 0.83 (0.72)
7 (spring) ISAT 104 0.79 (0.70)
8 (spring) ISAT 77 0.81 (0.73)