PALS

Early Literacy (Kindergarten)

Rating Summary

Classification Accuracyfull bubble
GeneralizabilityModerate High
Reliabilityfull bubble
Validityhalf bubble
Disaggregated Reliability and Validity Datafull bubble
Efficiency
AdministrationIndividual Group
Administration & Scoring Time23 - 43 Minutes
Scoring KeyComputer Scored
Benchmarks / NormsYes
Cost Technology, Human Resources, and Accommodations for Special Needs Service and Support Purpose and Other Implementation Information Usage and Reporting

Cost per student for K: $5 first year, $1.60 subsequent years (materials only); $7 first year, $3.60 subsequent years (materials plus Online Score Entry & Reporting System ; based on a class of 25 students).

PALS-K Teacher Set includes everything a new teacher needs to screen 25 students in fall and spring: an Administration and Scoring Guide, Teacher Packet, Class Summary Sheets, Student Packets, and class Student Spelling Sheets.

The PALS Online Score Entry and Reporting System costs $50 per one-year subscription, per teacher and classroom of up to 30 students, and includes school and district administrator access.

Screening tool may be administered completely using paper version; however, full use of product services requires internet access.

Testers will require less than 1 hour of training.

Developer strongly recommends that classroom teachers screen their own students, though others (e.g., reading specialists, special education teachers) can administer the test.

Allowable accommodations include multiple testing sessions, breaks between tasks, scheduling assessment for optimal times, one-on-one administration, small-group administration, un-timed administration (with the exception of Word Recognition in Isolation), assessing in an alternative location, repeating directions, repeating practice items, students repeat directions, checking for understanding, use of visual aid, and/or use of auditory aid.

To order PALS materials or to receive customer support:

PALS Marketplace
PO BOX 2616
Charlottesville, VA 22902

Phone: 1-888-924-7257
Fax: 1-888-924-7258

E-mail:
info@palsmarketplace.com

Website :
http://www.palsmarketplace.com/

Training manuals should provide all implementation information.

For questions concerning PALS research and development:
Phone: 1- 888-882-7257

E-mail:pals@virginia.edu

The Phonological Awareness Literacy Screening (PALS) is a criterion-referenced screening, diagnostic, and progress monitoring tool.  PALS consists of three instruments :  PALS-PreK, PALS-K, and PALS 1-3 . PALS measures young children’s knowledge of important literacy fundamentals , including phonological awareness, alphabet awareness, letter sound knowledge, spelling, concept of word, word recognition in isolation, and oral passage reading.  The major purpose of PALS is to identify students who may be at risk for reading difficulties and who need additional reading instruction beyond what is provided to typically developing readers.  The second purpose of PALS is to provide teachers with explicit diagnostic information about what students know and need to know about the fundamental components of literacy that may be used to target instruction to meet students’ needs.  The third purpose of PALS is to monitor students’ progress and determine the effectiveness of instruction or intervention.

PALS-K measures kindergarten students’ knowledge of phonological awareness and early literacy skills.  The phonological awareness component of PALS–K instrument is a measure of a young child’s ability to identify rhyme units and isolate beginning sounds.  The literacy skills component of PALS-K measures a young child’s knowledge of: alphabet, letter sounds, phoneme-grapheme correspondences, concept of word, and word recognition.

Raw scores from each PALS-K task are summed to produce a Summed Score; tasks and maximum points for each include: Rhyme Awareness (Max 10); Beginning Sound Awareness (10); Alphabet Recognition (26); Letter Sounds (26); Spelling (20); and Concept of Word (word list only) (10).  Thus, Summed Scores range from 0 – 102.

Group/Individually administered.

 

Classification Accuracy

Classification Accuracy in predicting the Stanford Achievement Test, Stanford Reading First
n=3,506
False  Positive Rate 0.039
False Negative Rate 0.537
Sensitivity 0.463
Specificity 0.961
Positive Predictive Power 0.455
Negative Predictive Power 0.962
Overall Classification Rate 0.928
AUC (ROC) 0.910
Base Rate 0.066
Cut Points:
     Spring PALS-K
     Stanford Reading First
 
81
488
At 90% Sensitivity, Specificity equals 0.729
At 80% Sensitivity, Specificity equals 0.850
At 70% Sensitivity, Specificity equals 0.900

 

Generalizability

Description of study sample:

  • State: Virginia
  • Size: 3,506
  • Gender:
    • 50.2% Male
    • 49.4% Female 
    • 0.3% Unknown
  • SES: 58% Eligible for free or reduced-price lunch
  • Race/Ethnicity:
    • 45.4% White, Non-Hispanic
    • 48.7% Black, Non-Hispanic
    • 3.5% Hispanic
    • 0.3% American Indian/Alaska Native
    • 0.6% Asian/Pacific Islander
  • Disability classification:
    • 1.4% Developmental Delay
    • 3% Learning Disability
  • Language proficiency status: 0.9% received ESL services

Cross Validation Study Description of study sample:

  • Number of States: 1
  • Size: 65,605
  • Gender:
    • 51.4%  Male
    • 48.6%  Female
  • SES: 37.4% median school-level free and reduced lunch
  • Race/Ethnicity:
    • 56.8% White, Non-Hispanic
    • 26.8% Black, Non-Hispanic
    • 8.9% Hispanic
    • 0.3% American Indian/Alaska Native
    • 3.6% Asian/Pacific Islander
    • 3.3% Unknown
  • Disability classification:
    • 5.8% speech/language impairment
    • 2.9% developmental delay
    • 0.5% learning disability
  • Language proficiency status: 6.1% receiving ESL services

Reliability

Type of Reliability Age or Grade n (range) Coefficient
Range Median
Inter-rater K 45 – 154 0.96 – 0.99 0.99
Test-retest K 467 – 472 0.78 – 0.95 0.89
Subtask reliability K 50,949 – 85,723 0.79 – 0.88  

 

Validity

Type of Validity Grade Test or Criterion n (range) Coefficient
Predictive K PALS 1-3 Fall 1st
Entry Level Summed Score
52,585 0.75
Predictive K PALS 1-3 Fall 1st 51,106 0.78
Predictive K Entry Level Summed Score 41,846 0.77
Criterion-related: predictive K Stanford Achievement Test 74 0.70
Criterion-related: predictive K PALS 1-3 Fall 1st and Spring 2nd Summed Scores Statewide samples; approx 50,000 0.67, 0.53
Criterion-related: predictive K Virginia Standards of Learning (SOL) in Reading Approx 27,000 0.29 - 0.47
Construct K   As one piece of evidence in support of the construct validity of PALS-K, we conduct Principal Component Analyses on the tasks that make up the PALS-K Summed Score each year, based on statewide data.  These analyses consistently yield a single factor (eigenvalue greater than 1.0) that accounts for approximately two-thirds of the variance (0.65 in Spring 2008) in these data.
Content K   The content validity is supported by the process of item development (described in the Technical Reference, Section III, pp. 11 – 19).  The guiding principles underlying this process were that (a) tasks and items were a representative sample of tasks from other early literacy screening instruments, (b) items had a history of use in phonological awareness and early literacy research, and (c) items were aligned with Virginia’s Standards of Learning for English (Reading).  Each of these principles was further supported by the process of review by an Advisory Panel of literacy professionals from across Virginia, and by an External Review Panel, which consisted of nationally recognized experts in the field of reading, communication sciences, or psychology.

 

Disaggregated Reliability, Validity, and Classification Data for Diverse Populations

Classification Accuracy in Predicting the Stanford Achievement Test, Stanford Reading First: By Race/Ethnicity
  Black
n=1,706
White
n=1,593
Hispanic
n=122
False Positive Rate 0.051 0.025 0.036
False Negative Rate 0.534 0.545 0.462
Sensitivity 0.465 0.455 0.538
Specificity 0.948 0.974 0.963
Positive Predictive Power 0.477 0.391 0.636
Negative Predictive Power 0.946 0.980 0.946
Overall Classification Rate 0.904 0.957 0.912
AUC (ROC) 0.898 0.912 0.924
Base Rate 0.092 0.034 0.107
Cut Points:
Spring PALS-K
Stanford Reading First
 
81
488
 
81
488
 
81
488
At 90% Sensitivity, Specificity equals 0.723 0.726 0.780
At 80% Sensitivity, Specificity equals 0.820 0.800 0.850
At 70% Sensitivity, Specificity equals 0.877 0.930 0.900

Disaggregated Reliability

Type of Reliability   Grade n (range) Coefficient Range
Subtask reliability Across 4 SES quartiles; SES determined based on school-level free and reduced price lunch percentages K 50,949 – 85,723 0.80 – 0.87
Subtask reliability Across race/ethnicity groups K 50,949 – 85,723 0.80 – 0.89