Untitled Document

Return

A Call to Action: Family and Consumer Sciences Program Evaluation
Lori Myers
Instructor, Family and Consumer Sciences Education
School of Human Ecology
Louisiana Tech University
Ruston, LA
and
Doctoral Candidate
Iowa State University
Family and Consumer Sciences Leadership Academy

Introduction
The mission of family and consumer sciences (FCS) programs is to empower “individuals and families across the life span to manage the challenges of living and working in a diverse global society” (National Association of State Administrators of Family and Consumer Sciences, 1998, p. 2). Ultimately, FCS programs strive to make a difference in the lives of students, families, schools, and communities. The 2005 eYearbook focuses on FCS programs throughout the nation that have been nominated as exemplary programs that are achieving the profession’s mission.

Exemplary is defined as “worthy of imitation; commendable” (The American Heritage Dictionary of the English Language, 2000). But how does one determine whether a program is exemplary or worthy? How do they respond to daunting questions, such as: Is the program making a difference? How can the program’s impact on students’ lives and achievement be demonstrated? The answer to these questions can be found in program evaluation.

Ley (1998) stressed, “No matter how strong or good the program may be, without actual data to support such claims, other efforts may be in vain” (p. 23). Program evaluation can provide data necessary for assessing the quality or effectiveness of FCS programs. It is critical that efforts continue to develop strong FCS programs in middle and secondary schools across the nation. Therefore, this article is designed to explore the purpose of program evaluation, to discuss the intensified need for evaluating FCS programs, and to provide suggestions regarding appropriate evaluation of those programs.

The Purpose of Evaluation
Determining merit or worth is the sole purpose of evaluation (Fitzpatrick, Sanders, & Worthen, 2004). Weiss indicated that program evaluation could be used “to understand how a program works and what the impacts or consequences are for learners” (as cited in Hitch & Youatt, 2002, p. 272). Gajda and Jewiss (2004) echoed the importance of program evaluation, “given all the hard work and planning that went into the development of the program, it may very well experience success – but you need the tools to document that your program is making a difference” (p. 1). White and Loyd (2000) emphasized that programs are evaluated to “provide valid findings about the effectiveness of programming for those responsible for creation, improvement, and continuation” (p. 245). But, this raises another question of how one defines effectiveness. Simply stated, effectiveness can be defined as “having an intended or expected effect” (The American Heritage Dictionary of the English Language, 2000).

In general, evaluation includes “(1) determining standards for judging quality, (2) collecting relevant information, and (3) applying the standards to determine value, quality, utility, effectiveness, or significance” (Fitzpatrick, Sanders, & Worthen, 2004, p. 5). Evaluations can be either formative or summative. Although the purpose of each is the same, they differ with regard to use, audience, design, and data collection.

Whereas program improvement is the main use of a formative evaluation, a summative evaluation provides data to help decision makers and stakeholders judge whether to continue, expand, or eliminate the program. Formative evaluations provide feedback to the program and monitor how the program is doing; summative evaluations provide feedback to administrators and teachers and assess the effectiveness or impact of the program. Data collected are used for diagnostic purposes in a formative evaluation and for judgmental purposes in a summative evaluation. Teachers can use formative and summative evaluations to gather meaningful data about their students, the curriculum, and the instructional processes and resources used within the program. “Wise use of these results will ensure a relevant, continually changing program” (Chamberlain & Cummings, 2003, p. 109). With an understanding of the purpose of the evaluation, one can determine the evaluation design and methods for data collection and data analysis.

Jacobs (2003) proposed five tiers of program evaluation, which differ in complexity. These include: “needs assessment, monitoring and accountability, quality review and program clarification, achieving outcomes, and establishing impact” (p. 70). Table 1 identifies each of these tiers and its primary focus. One should choose the approach that best suits their purpose, audience, and tasks.

Table 1: Focus of the Five-Tiered Approach to Evaluation
Evaluation Tiers	Primary Focus
Needs Assessment Monitoring and Accountability Quality Review and Program Clarification Achieving Outcomes Establishing Impact	Establish need for the program Accurately describe program elements Assess quality and consistency of program Determine program effects or outcomes Study longer-term evaluation period

The Need for Evaluation
How have FCS programs fared in recent years? Ley (1998) indicated that requests for AAFCS to assist in saving secondary programs are not uncommon. In fact, AAFCS (2003) reported various efforts to save secondary FCS programs in four states. Thus, the requests should be a call to action for the FCS profession. AAFCS recognized the need for action when it called for an extensive literature review to document the effectiveness of secondary FCS programs RFPLitRevFinal.pdf Vincenti (2004) further justified this need by indicating that “AAFCS is initiating a study of the impact of FCS in public school programs because we need credible data for a proactive information campaign to focus on critical audiences who make decisions about our programs.” (p. 1).

Throughout the profession’s history, several evaluation studies have been documented (Wallace & Hall, 1984; Gritzmacher & Tooke, 1996; Gritzmacher, Tooke, Pestle, Clawson, Johnson & Baum, 1988; Griggs & McFadden, 1980b). The Vocational Education Act of 1976 seemed to be one catalyst in a nationwide call for research to document the effectiveness of home economics programs (Hendrickson & David, 1980). One of the primary purposes of this national evaluation was to determine “the effectiveness of the programs as measured by acquisition of consumer and homemaking knowledge and skills” (p. 15). In response, Griggs and McFadden (1980a) assessed program effectiveness by reviewing “research that describes the outcomes of programs in an attempt to determine impact of the programs on learners” (p. 17).

In the 1980s and early 1990s, states conducted and published results of their own evaluation studies (see for example, Bell & Glosson, 1983; Sutter, 1985; Miller, 1988; Montana State Office of Public Instruction, 1989). Couch and Felstehausen (2001) reported that nearly half of the research articles published between 1985 and 2000 in major FCS publication outlets related to FCS programs. Of those articles, “a majority of [the] studies focuses on the evaluation of secondary programs and/or courses” (p. 262). Yet, there is a call for further research to demonstrate effectiveness of FCS programs and their impact on students’ knowledge, attitudes, and behaviors (Gentzler, Hausafus, Browne, Kruempel, & Myers, 2004).

Erwin, Moran, and McInnis (1996) stated, “It is obvious. . . that some [home economics programs] flourish and seem to be an essential and integral part of their school. Others, however, have very low enrollment and in fact may end up being phased out” (p. 18). Thus, the future of these programs is uncertain. Ley (1998) recognized that “the critical issue of program accountability – how these programs affect people’s lives – is one that is not adequately addressed by our profession” (p. 25). As legislative mandates, budget cuts, and other external factors threaten the existence of these programs, data documenting the effectiveness and impact of FCS programs are crucial. With increasing accountability and diminishing resources, FCS programs will be faced with demands to do more with less. If programs are going to survive and thrive in the coming years, they must be equipped to deal with change effectively as they review curricula and evaluate programs.

Appropriate Evaluation of FCS Programs
Because of the need for FCS program evaluation, it is important to determine the factors or components that should be included to assess whether a program is considered exemplary. Some criteria, such as the use of state and national standards and legislative mandates, have been suggested in previous research. Individual states may have established their own criteria for program evaluation. White and Loyd (2000) suggested that “national standards are fundamental to effective program evaluation” (p. 246). Other suggestions will be presented in this paper and, perhaps, could serve as talking points for future discussion.
As an administrator, Madsen (2004) believed that FCS programs must focus on “(1) providing quality programs and (2) reinforcing and/or teaching mathematics, science, and reading” (p. 37). One of the first steps then in determining a quality program is to identify and document the program’s activities, indicators, and outcomes that are to be evaluated (Gajda and Jewiss, 2004). Stated simply, “think of the desired outcomes as what you ultimately want the program to accomplish, the activities as what you will do to get there, and the indicators as the gauge of whether, and to what degree, you are making progress” (p. 2). The progress of the program is determined by examining the process and outcome indicators. Process indicators, the quantity and quality of the program activities, “help track the progress that your program is making as you work toward achieving the desired outcomes” (p. 3). Outcome indicators, the quality and quality of the outcomes that the program is achieving, “provide the most compelling evidence that the program is making a difference in the lives of program participants” (p. 3).

With state and federal legislation and reduced funding, secondary school administrators will be required to make tough decisions regarding programs. The No Child Left Behind Act (2001) federal legislation called for increased accountability, outcomes-based evaluation, and the integration of academic core content, such as reading, writing, mathematics, and science. If FCS programs want to maintain their presence and status in secondary schools, these programs must be ready to justify the impact and/or effectiveness of their programs. Thus, the question must be asked: How do secondary FCS programs address these issues and assure stakeholders that they play an essential role in students’ lives?

Because the No Child Left Behind Act emphasized the importance of academic core content (reading, math, and science), the integration of academic content with FCS content could be one criteria used in FCS program evaluation. Card (2004) provided practical examples of how FCS programs address academic content, life skills, and leadership development. She notes, “Throughout its nearly 100-year history, FCS courses have integrated reading and writing strategies, critical thinking, math and science applications, problem solving, as well as technology, research, and practical life skills” (p. 10). Various articles (Newell, 2004; Kennedy, 2004; Sanden, 2004; and Ward, 2004) again provided practical examples and best practices of integrating academic content. Newell (2004) acknowledged “every aspect of the FCS curriculum is tied to state educational standards for core subjects” (p. 13).

Furthermore, Riley (2004) presented a series of questions that could be answered in evaluating FCS programs. These include: “(1) Is the content current and relevant? (2) Does the program serve a wide variety of students? (3) Is service learning evident and are there opportunities for leadership development? (4) What are the student-teacher interactions? [and] (5) How is information shared with administrators?” (p. 35). With these questions in mind, teachers and administrators could determine appropriate outcomes, activities, and indicators for use in program evaluation.

Goetting (1993) identified various methods for evaluating program impact and communicating those results to stakeholders. Data from program evaluation can come from observations, checklists, case studies, knowledge tests, and questionnaires (Ley, 1998). The key is to collect quantitative and qualitative data documenting that the program is impacting lives.

Not only is it important to evaluate FCS programs, but FCS professionals need to be prepared to communicate the evaluation results data to stakeholders. MacColl and White (1998) reminded professionals that “parents, educators, school board members, and legislators all want to know what works and what doesn’t’’ (p. 1). According to Ley (1998), “letting others outside of the classroom know what is going on is probably one of the most important things a teacher can do to safeguard [FCS] programs in the secondary schools” (p. 22).

CONCLUSION
If secondary FCS programs want to remain viable in U.S. public schools, in light of federal legislative mandates and budget cuts, program evaluation and assessment could be the key to enhanced and sustained programs. Hopefully, FCS educators will be able to document, with both qualitative and quantitative data, that exemplary programs exist and that those programs have a positive impact on student achievement and students’ lives. This article was written to provide a general overview of program evaluation and to possibly serve as an impetus for future discussion regarding the evaluation of FCS programs.

The exemplary programs featured in this eYearbook provide an opportunity for professionals to review the components or factors that make these programs noteworthy and to consider how they can be integrated into programs throughout the United States. In reading about these programs, consider the following:

• What components do these programs have in common?
• What measures have been taken in these programs to document success and achievement?
• How have these programs used data, feedback, and results?
• How have data been communicated to program stakeholders?

Evaluating FCS programs and communicating the results of the evaluation are critical to sustaining the profession. If the FCS profession doesn’t take action to save our secondary FCS programs, who will?

REFERENCES
American Association of Family and Consumer Sciences. (2003). Annual activities and accomplishments report: May 1, 2002 through April 20, 2003. Report presented at the Annual Meeting of the American Association of Family and Consumer Sciences, Washington, D. C.

Bell, C. G. & Glosson, L. R. (1983). Effectiveness of consumer and homemaking education in Texas. Final report. Austin, TX: Texas Education Agency. (ERIC Document Reproduction No. ED237750).

Card, B. (2004). No child is left behind in the FCS classroom. Journal of Family and Consumer Sciences, 96(1), 8-12.

Chamberlain, V. M. & Cummings, M. N. (2003). Creative Instructional Methods for Family and Consumer Sciences, Nutrition and Wellness. New York: Glencoe-McGraw Hill.

Couch, S. & Felstehausen, G. (2001). Research in family and consumer sciences education, 1985-2000. Family and Consumer Sciences Research Journal, 30(2), 256-270.

Erwin, W. T., Moran, J. D., & McInnis, J. (1996). Role definition and perceptions of home economics secondary programs. Journal of Family and Consumer Sciences, 88 (4) 17-24.

Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2004). Program evaluation: Alternative approaches and practical guidelines (Third Edition). Boston: Pearson Education, Inc.

Gajda, R. & Jewiss, J. (2004). Thinking about how to evaluate your program? These strategies will get you started. Practical Assessment, Research and Evaluation, 9(8). Retrieved October 10, 2004 from http://PAREonline.net.

Gentzler, Y. S., Hausafus, C. O., Browne, L. B., Kruempel, B., & Myers, L. A. (2004). A study of the effectiveness of public school family and consumer sciences programs (2004): Final report. Ames, IA: Iowa State University, College of Family and Consumer Sciences.

Goetting, M. A. (1993). Marketing the accountability of one’s program. In E. P. Anderson & V. L. Clark (Eds.), Marketing home economics: Issues and practices. Yearbook 13 (pp. 246-255). Alexandria, VA: American Association of Family and Consumer Sciences.

Griggs, M. & McFadden, J. (1980a, Spring). Effectiveness of consumer and homemaking education programs. Journal of Home Economics, 17-18.

Griggs, M. & McFadden, J. (1980b, December). The effectiveness of consumer and homemaking education: A review and synthesis of extant data. Washington, DC: National Institutes of Education. (ERIC Document Reproduction No. ED211721).

Gritzmacher, J. E. & Tooke, F. (1996). Evaluation. In S. Redick (Ed.), Review and synthesis of family and consumer sciences education research 1985-1995. (pp. 131-137). Peoria, IL: American Association of Family and Consumer Sciences and Glencoe/McGraw-Hill.

Gritzmacher, J., Tooke, F., Pestle, R., Clawson, B., Johnson, M. & Baum, R. (1988). Consumer and homemaking program effectiveness in seven states. Journal of Vocational Home Economics Education, 6(5), 31-43.

Hendrickson, G. & David, H. (1980). A policy inquiry. Journal of Home Economics, 14-15.

Hitch, E. J. & Youatt, J. P. (2002). Communicating family and consumer sciences: A guidebook for professionals. Goodheart-Willcox: Tinley Park, IL.

Jacobs, F. H. (2003). Child and family program evaluation: Learning to enjoy complexity. Applied Developmental Science, 7(2), 62-75.

Kennedy, M. J. (2004). Today’s FCS classroom prepares youth for life. Journal of Family and Consumer Sciences, 96(1), 92.

Ley, C. J. (1998). Influencing policy makers about secondary family and consumer sciences programs. Journal of Family and Consumer Sciences, 90 (4), 21-25.

MacColl, G. S. & White, K. D. (1998). Communicating educational research data to general, nonresearcher audiences. Practical Assessment, Research and Evaluation, 6(7). Retrieved October 10, 2004 from http://PAREonline.net.

Madsen, L. (2004). FCS must take part in the political dialogue. Journal of Family and Consumer Sciences, 96(1), 37.

Miller, S. W. (1988). Impact of Kentucky consumer and homemaking programs on students. Kentucky State Department of Education: Frankfort, KY. (ERIC Document Reproduction Services No. ED333213).

Montana State Office of Public Instruction. (1989). Impact of consumer and homemaking education of 1983 and 1988 graduates in Montana. Final report. Author: Helena, MT. (ERIC Document Reproduction Services No. ED 336597).

National Association of State Administrators of Family and Consumer Sciences. (1998). National standards for family and consumer sciences education. Decatur, GA: Vocational-Technical Education Consortium of States.

Newell, B. A. (2004). An overview: How FCS integrates the core academics. Journal of Family and Consumer Sciences, 96(1), 13-14.

No Child Left Behind Act of 2001. Pub L. No. 107-110.

Riley, V. M. (2004). What it takes to support FCS in a cost-cutting era. Journal of Family and Consumer Sciences, 96(1), 35-36.

Sanden, J. (2004). Math/FCS class boosts test scores. Journal of Family and Consumer Sciences, 96(1), 18.

Sutter, S. A. (1985). Statewide evaluation of vocational education consumer and homemaking programs. Final report. Pennsylvania Research Coordinating Unit for Vocational Education: Harrisburg, PA. (ERIC Document Reproduction Services No. ED252680).

The American Heritage Dictionary of the English Language. 4th edition. (2000). Houghton Mifflin Company.

Vincenti, V. (2004). Recent media attention provides golden opportunities. Journal of Family and Consumer Sciences, 96(1), 1.

Wallace, S. & Hall, H. (1984, March). Research in home economics education: Past achievements, present accomplishments and future needs. Home Economics Research Journal, 12(3), 403-419.

Ward, J. D. (2004). Principles of food science class sheds light on chemistry. Journal of Family and Consumer Sciences, 96(1), 23-24.

White, R. A. & Loyd, C. M. (2000). Implications for evaluation. In A. Vail, W. S. Fox, and P. Wild (Eds.), Family and consumer sciences teacher education: Yearbook 20. Leadership for change: National standards for family and consumer sciences education. (pp. 241-250). Peoria, IL: American Association of Family and Consumer Sciences and Glencoe/McGraw-Hill.

Top