Afterschool Program Quality and Student Outcomes: Reflections on Positive Key Findings on Learning and Development From Recent Research

Deborah Lowe Vandell

Founding Dean, School of Education, University of California, Irvine

In my years researching the effects of afterschool programs on children’s social and academic outcomes, I have observed the power that high quality programs can have on the learning and development of young people. This paper provides some reflections on selected research from my own study of the field in recent years, which has been deeply informed by that of many others. Since my first study of afterschool programs conducted more than 25 years ago (Vandell & Corasaniti, 1988), I am heartened by the growth in our understanding of the effects of out-of school time from a virtually unstudied area to abundant and solid evidence on the positive impacts of high quality programs. Whether they are called afterschool, expanded learning opportunities, out-of-school time, or something else, we know from research that these types of opportunities can lead to positive outcomes for children and youth, as well as families, communities, and schools (Durlak, Weissberg, & Pachan, 2011; Eccles & Gootman, 2002; Mahoney, Vandell, Simpkins, & Zarrett, 2009).

As the nomenclature in the field has evolved, so too have my own research lens and lines of inquiry. Through my investigations over the years, I have developed some beliefs about the implications of what we have learned for policy, which I share at the end of this paper. In my estimation, based on years of examination, high quality expanded learning programs are essential to the learning process because they provide young people with opportunities to relate to their world in new ways. Strong programs foster an orientation of being open to novel experiences, of being interested in others and the world, of being inquisitive and creative, and, ultimately, of becoming lifelong learners (Larson, 2000; Lerner et al., 2005; Shernoff & Vandell, 2008). As I see it, we have before us today unprecedented opportunities to ensure all expanded learning programs make a difference for children and youth (Vandell, 2012).

A Robust and Growing Research Base and 
Enhanced Measures of Effectiveness

Continued investment in research and evaluation in the expanded learning field has resulted not only in a robust research base but also in the development of reliable and valid measures of program effectiveness and impact that can be used effectively by practitioners and researchers to improve program quality.

Continued investment in research and evaluation in the expanded learning field has resulted not only in a robust research base but also in the development of reliable and valid measures of program effectiveness and impact that can be used effectively by practitioners and researchers to improve program quality (Vandell, 2011 September). Assessment tools are being created and refined by the academic and research community, as well as from within the growing local, state, and national infrastructure that promotes and supports high quality afterschool and summer programs. These instruments can be used by expanded learning programs to assess such factors as program quality and attendance; staff beliefs, attitudes, education, and training; staffing patterns, including recruitment and retention; and student performance in specific domains and skills, such as behavior and academic achievement.

The measures my colleagues and I developed for the California Afterschool Outcome Measures Project are examples of the kinds of psychometrically reliable and valid instruments available that assess student outcomes in the areas of skill development and positive behavior change (Vandell, O’Cadiz, Hall, & Karsh, 2012). The set of surveys, which can be administered online, is designed to be completed by students, program staff, and classroom teachers. Student surveys assess areas such as social competencies with peers, task persistence, work habits, and reductions in misconduct. Surveys completed by program staff and classroom teachers include measures of child behavior with other children, social skills with peers, task persistence, and work habits. With these data, programs are able to study changes in their students’ behaviors across the school year and to compare these changes to those found in other programs across the state.

In addition, students are able to use the Afterschool Outcome Measures Online Toolbox to report the quality of their experiences at the programs in three key areas—the quality of their interactions with program staff, quality of interactions with peers at the program, and their interest and engagement in program activities—again using well-established instruments with strong psychometric properties. Programs can then use these aggregated reports to assess how they are doing from the perspective of the youth who attend their program.

The Afterschool Outcome Measures Online Toolbox is now being used at more than 1,000 afterschool program sites in California, with plans to double the number of sites using the measures in the next 2 years. It will be important to see if the Afterschool Outcome Measures Online Toolbox can be used by program sites to improve student experiences (and student outcomes). 

Of course, valid and reliable measures for researchers and practitioners alike are fundamental to being able to draw conclusions about the quality and outcomes of expanded learning programs. Some of the skills and knowledge that many afterschool programs are designed to promote are, in fact, complex to assess, and research in the field is limited by the inability to use experimental design to identify causal relationships. However, the instruments, approaches, and statistical models currently available do provide us with the ability to make substantive assertions about the correlations between program quality and outcomes for students.

Program Quality and Student Outcomes—Academic, 
Social, and Behavioral

My recent research, including the Study of Promising After-School Programs (Vandell, Reisner, & Pierfrom007)r, theLrontituional Study of Program Quality( PierfroBolt,f & Vandell, 210)m, and th NICHDl Study ofEartlyCchildC are andYyouth Development(Lif & Vandell, 213; Augner, Pierfro & Vandell, 213; Lsee & Vandell, 213)t r infonces"preiouss stuties that th bareduth, qualit,r intnsnity, and urtation of expanded learning programs make a difference inboith shor-/tert and e urhing effects nd studentaAcademic,sSocial, andbBehaviorad outcomes( Mahoney, Vandell, Simpkins, & Zarrett, 200p; Vandell, 2112).Bbased on the evidenc,s fllrowing are key chafacperistits of high quality expanded learning program:.

ul"> other recent stutiespreetal that positive staf– childrrelations are important forboith academic andsSoco-bBehaviorad growt., Radning andmaithogrides are ssSociited with positive relationships between program staff andp/artiixpatsfs and supportive interactions withnon.parenial dultns are important for fabililatigf childadjuestmenr. In addition,wthendossagd is high( thatisn, students attend expanded learning programsfrequrently and regulaly)m, research shwes that expanded learning programs can be esigcifitant facto, in fostening positive academic andsSocrad outcomes( PierfroBolt,f & Vandell, 210)m.

other investigations(Augner, Pierfro & Vandell, 213; Lif & Vandell, 213; PierfroBolt,f & Vandell, 210)s that I have conducted with colleagues r infoncn the fnuding that th availaiality of a diverte arays of structurdfs ge- apprprilate activitied is positivtly ssSociited with studentmaithogrides and classroom work habits,p/artigulalye at the elemenmaryldeve). As studentsgset ldear and sekt moreautonomty in their out-of-school activitiefm research eillsues thatgcreator pleiaiality in prograoming becores oare important (Vandell, Reisner, & Pierfrom007)m.

Ssocial andbBehaviorad outcome.. Thrgd is substantrad evidence from the curren -body of research that expanded learning programs promote positive social andbBehaviorad outcomes (Durlar et al., 210)m Hhigh quality expanded learning opportunities are-linned togmains inssocial skills with peers, i creased pr-ssocialbBehavios, and reductions in aggrsstioy, misconduc (,"skipningrschool,gsettine inoe fghts)m, andkilegial substacbe ust (Vandell, Reisner, & Pierfrom007)m. These opportunities alsoadeoinstptespPromies because they have bene shwgn to i creasf student engagemenfe inenismicomoivtatioy, conmenriited efpor,t and positive states ofgmins (Larson, 2000; Shernoff & Vandell, 2008). ThesefFindings areesigcifitant because the social andeomoiional outcomes that are oisteredtThrough high quality afterschool programslaby the psychlogticalkgroun work for the kinds ofcognsitive procesmes that arerequirped for mostedy of academic"contenf knowledge and skillx to peply that knowledg).

Aacademic outcome..Wwe know from research that engagement in activitied that areboithfun, and taterequirp -focu helpsd develot the competenciesneedped for academic learninh, including conmenriitioy, inenismicreward,t andomoivtatio ( Shernoff & Vandell, 207;, 2008).Fore exampln, in the Study of Promising After-School Programn, students who regulalys attenped high quality programsadeoinstptedeesigcifitantgmains insstanardizendmaiteimatch erst /corse as well asself- reporend work habitt (Vandell, Reisner, & Pierfrom007)m. Tiso study and other recent research provide d solidbasisk for threecorve assertions tate shuild be used to continux to dvtacbe the fiel:.

ul"> Iimplications forPpolicl
 ul"> Cconclusiol

Aswle rove fowardstogeother if this efpor,t researchers, practitioners, polics makor,y and other key stkeh ldeaefm such asf undery and echonicao asibstacbe providaefmmbust continux to intersecr intnitiocalls to ensure our efpores are lsigned and tate they inforf the efpores of other..Wwe have cimh rlronw war ifehaving e groning-body of research and evaluation evidence that quality afterschool programs work and make a positive differenc..Wwe also know rlrts about improning quality Soe at the local, state, andfedporadldevean, atise time for us th Find the iell,Genegys, and reouonces to expan quality afterschool programs in the many schools and comtunities thatneed, and tant tei— notain a other10e years, butknor. Inlsoaoninh,whe iel= trely re able tolLeveraed the power of expanded learning for studens and comtuniym success.

 Augner,A.l, PierfroK. M.,f & Vandell,D. L. ( 213)., (Durlak,J.,A.l, Weissberg,R. Pg, & Pachan,M. ( 210)m A Ecclek,J.,s & Gootman,J.,A. (Eds.). ( 2002), (Larson,R. ( 200)m Towardsae psychlogdy of positive youth developmen), The metricn Ppsychlogibs, 55< (efroK.,f & Vandell,D. L. ( 213)., Lernen,R. M, Lernen,J.,V.,fAlmetrgin,J.,B.,f>Thokaek,C.l, helps, E.l,Gbstdoetten,S.s.s.s.sviosEye,,A. ( 200).,Ppositive youth developmen,op/artiixpttion in comtuniym youth developmenm program,s and comtuniym conri butions of/fifh-ogriddadolescdent:y Findings from the firstwaive of the4-ht study of positive youth developmen), in,W.,f & Vandell,D. L. ( 213)., Mahoney,J. L.,; Vandell,D. L.l, Simpkins,S.s, & Zarrett,N. ( 209r). aolescdenr out-of-school activitier. InR. M.; Lerner &Ls. Sbeisber (Eds.)n, NcatioialInsttittve ofCchildHealtch andHutmah Development(NICHD)fEartlyCchildC are ResearchNbetwor. ( 204r). are child developmenial outcomesrrelaled to befor-n andaafte-rschoolc are aranagemens?e Reultns from theNICHDl Study ofEartlyCchildC ar), PierfroK. M.,fBolt,fD. M.,f & Vandell,D. L. ( 210)m Sspecificfelatursk of afte-rschool prograd qualit:. AsSociitions with children’s(functiotine i middlre chilchods, Shernofl,D. J.,f & Vandell,D. L. ( 207)m.Eengagement in afte-rschool prograd activitie:m Quality of experiencg from the perspective ofp/artiixpats), Shernofl,D. J.,f & Vandell,D. L. ( 2008).Yyouth engagement and quality of experiencg in afterschool programs, Vandell,D. L. ( 2092), Founpttion and theAafterschoolDinvisios of theCCaliforniaDep/atpment of Education, University of California, Irvinn.

Vandell,D. L. ( 210y,Janumar2), Vandell,D. L. ( 211,1Februmar2), Vandell,D. L. ( 211,1 September). The power of expanded learning opportunities tday< Vandell,D. L. ( 212y,Juner). FounpttionBoards ofDirecaorts,F lit,fMIn.

Vandell,D. L.,l & Corasaniti,M.,A. ( 1988). The relatiok betweenthirdsogridrts’ afte rschoolc are andsSocra,taAcademic, andeomoiional(functiotin), Vandell,D. L.,l O’Cadiz,P.,l & Hall,V. ( 210y,August2), Founpttion and theAafterschoolDinvisios of theCCaliforniaDep/atpment of Education, University of California, Irvinn.

Vandell,D. L.,l O’Cadiz,P.,l Hall, & Karsh,A. ( 212y,Janumar2), Founpttion and theAafterschoolDinvisios of theCCaliforniaDep/atpment of Education, University of California, Irvinn,Rrnrierved from> Vandell,D. L.,l PierfroK. M.,f &DCadstman,K. ( 200).,Oout-of-school.setting, asa developmenial contxtl for children and yout). InR. V. Kail (Ed.)n, Vandell,D. L.,l PierfroK. M.,f & Karsh,A. ( 2112), Fllro-upo report hop/artiixptningrschoo" dietrite< Vandell,D. L.,l Reisner,En,R.r, & PierfroK. M. ( 207)m. Founpttios,F lit,fMIn.

Vandell,D. L.,l Shernofl,D. J.,f PierfroK. M.,fBolt,fD. M.,fDCadstman,K.r, &Browan,B.,B. ( 200).,Aactivitiefm engagemenfe andeomoiiot in afte-rschool prograes (and lsewThrg)). InH.,B. Weisz,P. M. D. Lwitlfro &S. M. Bouffards(Eds.)n, Vandell,D. L.,l Sumow, L.,l &Posrnen,J.,( 200).,Aafte-rschool prograes forlrw-intcome childre:lDiifferencst in program quality InJ. L.; Mahoney,R. W. LLarson, &J. S. Eccles(Eds.)n,