A Literature Review Evaluating Early Childhood Programs
Research over the past decade has identified that what happens to children during the early years has a significant impact on learning and development (Doherty, 1995; McCain and Mustard, 1999). It has demonstrated that poor early childhood education programs have a negative impact on the development of young children while quality programs have a positive impact. Children cared for in poor quality programs have more frequent behaviour problems in elementary school than do their peers. They have poorer social skills, poorer academic progress and show less independence than their peers in elementary school. Children who received quality early childhood education programs had higher levels of language development than their peers, demonstrated greater social competency, were better able to regulate their own behaviour and performed better in all school subjects. (Doherty, G. & Derkowski, 1995: Browne, Bulletin of the Sparrow Lake Alliance, April, 1995)
This knowledge has led to an increased interest in ensuring that children have access to quality early learning and care programs. Parents want to know if a program is providing quality early learning and development experiences, and the Canadian Federal Government has made a commitment to promote quality in early learning programs through the Multilateral Framework on Early Learning and Child Care (Government of Canada, 2003). This Framework states that its objective is to "further promote early childhood development by improving access to affordable, quality early learning and childcare programs and services". (Government of Canada, 2003) This knowledge also means that evaluation processes can be put in place that will let individuals and governments know if various programs are providing quality programs.
An accreditation process for licensed early learning and care centres is one way to help ensure quality. In such a process a licensed early learning and care program has its curriculum, administration, staff qualifications, etc. measured against proven quality indicators to determine the program's standards. In some countries (e.g. Australia) the federal government has been involved in developing and maintaining an accreditation process for early learning and care programs. In other countries (e.g. U.S.) a national association for early childhood educators (NAEYC) has taken on this task. Currently, Ontario has no accreditation process although some agencies and communities use tools developed by Harms and Clifford (e.g. ITERS, ECERS, SACERS) to help assess the quality of programs. The purpose of this paper is to review what the literature says concerning: a) key indicators of quality outcomes for children attending licensed centre based programs and b) evaluation being used to assess quality in these programs.
In the fall of 2001, the Ottawa Valley Branch (OVB) of the Association of Early Childhood Educators of Ontario (AECEO) approached the provincial board of the AECEO to ask if it would take on the task of developing an accreditation process that would incorporate all of the key indicators of quality child care (as identified in the current research) and that could be adapted to address Ontario conditions. Although the provincial board initially agreed to take on the accreditation project at the provincial level it became apparent that, because of manpower constraints at the provincial level, the OVB would be in a better position to proceed with the project. In early 2003 the OVB took over the project with the stipulation that they report back to the provincial board.
This literature review is one of the components of the accreditation project. This paper is a compilation of literature reviewed by members of the Accreditation Committee of the Ottawa Child Care Council and additional literature reviewed by members once it became an ad hoc committee for the OVB. The review includes information on centre characteristics that are indicators of quality outcomes for children as well as looking at evaluation tools that use these indicators to assess centre-based programs
Literature Review Process
The current review looked at research and previous literature reviews that have focused on the issue of quality outcomes for children in centre-based programs and on the tools used to evaluate the quality of programs. The authors attempted to limit the research to the past ten years, although older studies were occasionally included to illustrate how findings have not changed over time. The literature was not limited to Canadian content but included literature from other countries that are similar to Canada in terms of: licensed early learning and child care programs, historical perspectives, attendance policies for early learning programs, and demographics (rural, urban mix). The review of evaluation tools was limited to tools whose effectiveness had been validated.
In Canada there are several known experts involved in research on quality issues in early learning program (e.g. G. Doherty, M. Friendly, F. Mustard). This is true of other countries as well (e.g. Linda Mitchell, NZ, Peter Moss, UK, T. Harms, U.S.). The reviewers used the authors' expertise to help identify the relevance of the research to narrow the field of the literature identified for review.
Literature Review Results
In general the review findings indicated that the majority of studies evaluating quality have been done on preschool programs, rather than on programs for infant, toddler or school-age children, and therefore the results reflect this limitation.
To facilitate the reading of the results the review findings have been organized into three categories:
- quality indicators and outcomes
- evaluation tools
- feasibility of evaluating programs
Quality Indicators and Outcomes
The literature consistently identified indicators within programs that have certain characteristics and that produce desired learning and social outcomes. In the context of this review the desired outcomes being highlighted are related to improved cognitive performance and social competency (both short-term and long-term). Since these are the outcomes researchers have linked to quality.
Reviews and studies in Canada and other countries have identified similar indicators of program quality (Doherty, 2000; Baden, 1982; Bryant, 2003; Chandler, 1995; Mitchell, 2002; Government of New Zealand, 2003).
Researchers may use different terminology when describing indicators of high quality childcare; however, all research reflects similar characteristics. The following characteristics of childcare programs are predictive of quality outcomes for children. These characteristics are often referred to as "key indicators of quality" or "quality indicators".
Key Indictors of Quality:
- physical setting that protects children health and safety
- staff with two or more years of training in early childhood education
- a centre director who has formal training in early childhood education and in administration
- higher remuneration (leads to less turn over and more stability)
- staff with high moral and high job satisfaction
- in-service education related to child development and child care
- a group size that does not overwhelm the children (varies with the ages of the children)
- a responsible adult/child ratio that allows the educator to provide individual attention to the children
Doherty, G. (2000). Issues on Canadian Child Care: What does the Research Tell Us? (In Research Connections Canada, 5). Ottawa, ON: Canadian Child Care Federation
Quality indicators have been used by researchers when developing tools to assess quality within programs. The following section of the review will discuss the effectiveness of evaluation tools of measuring program quality and how quality impacts children's learning and development. (SmartStart, 2003; Mitchell, 2002; David, 2000; NAEYC, 1998; Harms, T. & Clifford, R., 1998; National Childcare Accreditation Council (Australia), 2001).
The ECERS (Early Childhood Environmental Rating Scale) appears to be more widely used than any of the other tools encountered in the review. It is being used in several of the subject countries including Canada, U.S. and the U.K. (Smart Start, 2003; Neil, 2002; Doherty, 2000; Surestart, 2003). The authors of the ECERS tool have also designed and validated tools for assessing infant/toddler programs (ITERS) and school-age programs (SACERS). Organizations using the ECERS, ITERS or SACERS often combine it with other tools to assure assessment of all quality indicators since its focus is on the environment rather than teacher/child interactions.
Other organizations and countries have designed and validated their own accreditation process. The research indicates that quality indicators have been incorporated in their accreditation tools. NAEYC (U.S.) and the Preschool Learning Alliance (U.K.) are examples of organizations that have developed accreditation processes. A study by Neil (2002) indicated that completion of the NAEYC resulted in improved outcomes for children, and the Pre-school Learning Alliance (1992) found that programs involved in the accreditation process demonstrated greater gains in cognitive and social skills than children from centres not involved in the process.
Australia has developed and implemented a national accreditation system based on the NAEYC process known as Quality Improvement and Accreditation System (QIAS). Ireland has also developed an assessment tool based on the NAEYC accreditation tool.
The Alberta government has initiated the development of an accreditation process based on Gillian Doherty's research on quality indicators (You Bet I Care, 2000). Manitoba is also looking at the feasibility of developing an accreditation process. Both these provinces are working with the Canadian Child Care Federation to develop the projects.
Feasibility of Evaluating Programs
The literature indicates that it is possible to measure quality within licensed centre-based programs, be it through an accreditation process like NAEYCs or by developing a process that includes a tool like ECERS. Does this mean; however, that it is actually feasible to evaluate programs?
The government of Australia has been able to implement a national accreditation process for early learning and care centres (QIAS), and New Zealand and the U.K. have launched major initiatives as well. The New Zealand government has begun a ten year initiative (Pathways to the Future: Nga Huarahi Arataki) to raise the standards of centres and the quality of training for early childhood educators. In the U.K. several initiatives are being introduced through Surestart. The government has published a series of quality guidelines aimed at increasing standards (The National Standards for under Eight). An initiative by the Department of Education and Skills (U.K.) is also aimed at improving quality in early learning and care programs (The Effective Provision of Preschool Education Project, 2003). Part of this initiative uses the ECERS-Revised and the ECERS-Extension to evaluate programs for quality.
In the U.S. an initiative to address quality in early learning programs nation wide has been developed by the National Association of Education for Young Children (NAEYC). Since the 1980s this validated accreditation process has been available to all early learning programs. The Association is presently in the final stages of revising the current accreditation tools. At the state level there are initiatives such as Smart Start (North Carolina) that also address quality assurance.
In conclusion, the literature indicated that it is possible to evaluate licensed centre-based early learning programs. The findings also indicate that tools are available which are recognized in several countries as being valid predictors of quality outcomes for children in centre based program. Some tools have been used, as published, in more than one country (e.g. Harms and Clifford rating scales) while others have been modified to ensure that the needs of a particular country are being met (e.g. Australia's adaptation of the NAEYC's accreditation process). The research indicates that Ontario could successfully develop an accreditation process that meets the needs of province. However, such a commitment will require ongoing funding in order for it to succeed.