        
|
Assessing Your School’s Conflict Management Program
-What is Program Assessment?
Simply stated program assessment is a way to determine whether a comprehensive conflict management program is meeting its goals and objectives. In other words, assessments help us to grade out programs — how are our programs doing — excellent?, need improvement?, poorly? Assigning out programs a grade can help us in making the determination whether to continue, stop, or start specific initiatives and can provide data and analysis to answer questions such as:
-Is the program, as implemented, consistent with its design?
-How well is the program achieving its goals and objectives (e.g., are administrators, teachers, students and staff more effectively managing conflict, are teachers spending less time dealing with student conflicts and more time teaching?)
-Are participants satisfied with program implementation and outcomes?
-What changes may be needed to improve the program?
-Why Assess?
Assessment data are useful in finding out what works and what does not work in a program and may be a critical factor in decisions to modify or expand a program.
Officials and funders are looking for assessment data as they make decisions about future funding and support for programs. Schools that are able to describe what they are producing in the way of positive outcomes will be better positioned to receive continued program support.
Planning for assessment at the time that you are setting up your program allows for appropriate data collection mechanisms to be established early on and avoids the potentially difficult and time-consuming process of trying to reconstruct useful data.
-What to Assess?
Although the terminology may differ, assessments are commonly characterized as either (1) program effectiveness or outcome or (2) program design and implementation or process. Assessing program outcomes may be useful in determining whether a program is meeting its goals and should be continued or expanded; assessing program design and implementation often help in determining how program implementation can be improved. Decisions on the future of programs or how they can be improved are usually not made solely on the basis of program assessment results. Other institutional concerns, priorities, budget limitations, etc., will also affect program decisions.
-How to Assess?
What are the program’s goals and objectives?
As part of your conflict management program, you will most likely be developing an action plan for implementing you school’s comprehensive conflict management program. This plan should set out your school’s goals and objectives and should describe your general approach to achieving these goals and objectives. A goal is an expected or desired result — a broad aim to be achieved. An example of a program goal is, to decrease the amount of time teachers spend dealing with student conflicts in the classroom. An objective is a method to attain a goal. An example of a program objective is, to integrate conflict management skills into all Social Studies classes by January of 1998.
As a starting point for program assessment you should think about how you will know whether or not you have achieved these goals and objectives. How will your school measure whether or not it has reached its goal(s)? For example, if you determine that one of your program goals is to change students’ attitudes toward conflict, how will you measure whether or not your program has been able to bring about this change? If you find that you cannot effectively measure the goals and objectives that you have developed, you may have to think about modifying them so that they can be measured.
Measuring Success
Collecting Baseline Data: Baseline data is an important element to effective program evaluation because it allows you to look at what the situation or environment would have been without the program. If available, you may use baseline data to compare your school’s environment before and after program implementation. Baseline data could include statistics on the number of office referrals, in-school suspensions, expulsions, attendance, etc., collected during the year prior to program implementation. By comparing these figures to those collected at the end of the first year of program implementation, you may be able to see changes indicating that your conflict management program is having a positive impact on your school’s environment. However, it is important to not that other events occurring at your schools could also affect a change in these statistics. Statistics alone may not be an accurate measure of your program’s success.
Some schools already collect statistical data on the number of office referrals, in-school suspensions, expulsions, attendance, etc. If your school already keeps these kinds of data, this would be an excellent starting point for determining what kind of information your school can use to measure program success.
Tools: Written questionnaires and interviews can be used to collect baseline data to measure program success. Questionnaires or interviews can be used before and after program implementation to survey students, teachers, administrators, parents, counselors and support staff regarding their attitudes about conflict, the school environment, how they feel about interpersonal relationships, etc. Survey responses, pre and post-implementation, can be compared to see if there are any changes that can be attributed to the implementation of a school conflict management program. Again, other events at your school could affect responses and should be taken into account in any assessment.
In addition to statistics, written questionnaires and interviews there are other tools that can be used to assess program outcomes. Focus groups (collecting information from a group of persons simultaneously) and blind observation (where observers watch students’ behavior in a classroom pre and post-implementation) are two other tools useful in collecting assessment data.
Action Research: Action research is another technique for conducting program assessment. This approach involves systematic problem identification and solution where we learn from our own situation as we try to modify and improve it. Steps in action research may include:
-Data collection (facts, opinions, etc.)
-Diagnosis (identifying ‘gaps’ between "what is" and "what ought to be" as
supported by the data)
-Action (planning and carrying out steps that we predict will improve our
program)
-Evaluation (re-diagnosis: what is the gap now?)
Although the above steps may look neat and sequential, rarely will it work so cleanly!
Program Assessment without Baseline Data
If it is not possible to administer written questionnaires or conduct interviews prior to implementation, a post-implementation questionnaire or interview can still be used to collect valuable information to assess program outcomes. One approach would be to ask program participants a series of short open-ended questions regarding conflict, relationships, and the school/classroom environment. From the responses one could distill themes that could be used in measuring program outcomes.
Statements or testimony from key program participants can be quite effective in providing interested persons with information about your program. Although testimonials do not provide quantitative data, they can give people a strong indication as to how participants feel about the program and the impact that the program is having on the school environment.
Things to be Aware of in Collecting Data
Confidentiality of Data: Often records contain names of individuals who are entitled to privacy and should not be named. Similarly, when collecting information through interviews and surveys, evaluators may want to keep confidential the names of people supplying testimonial information. Offering confidentiality may increase willingness to cooperate in assessment. Although anonymous surveys should protect identity of respondents, absolute confidentiality should not be promised unless it can be delivered.
Uniformity: It is important that data be collected in a uniform manner. For example, it more than one person is conducting interviews, they all need to be asking the same questions in the same order. Also, some terms or categories may need to be defined so that all respondents have a common understanding. Precautions need to be taken to ensure that data is collected in a reliable manner that permits comparison of data.
Pretesting: It is helpful to pretest data collection instruments to ensure that questions are clear and can be understood by those who will answer them, and that they provide the information needed for effective assessment.
Who are the Audiences for the Assessment and What Do They Want to Know?
Identify Your Audience: There are usually a variety of people who have an interest in the results of a program assessment. These audiences may be interested in different issues and seek different types of information. For example, potential audiences could include school administrators, teachers, parents, PTO, state legislators, community groups, etc. In planning an assessment these audiences should be kept in mind so that their questions can effectively be addressed.
While it is not possible to answer all potential questions, it is useful to figure out what the possible questions are and then focus on the most important. Talking to members of various audiences may help to identify the issues they are most interested in, improving the likelihood that assessment results will be useful and meaningful in future decision making.
Who Will Conduct the Assessment?
Assessments can be conducted by persons outside a school or within a school. There are advantages and disadvantages to each option. An outside person has the potential for the greatest impartiality and depending on the expertise available within the school, an outside person may have more technical knowledge and experience. Outside assessments, however, can be relatively expensive. Using someone involved in the program implementation may be the least expensive and could offer the best understanding of the program context, but it also carries with it potential perceptions and a lack of impartiality.
One example of a school conducting their own program assessment is Kenwood Heights Elementary School in Springfield, Ohio. Kenwood implemented a conflict management program in 1992. Students and staff committed to making Kenwood a more "peaceful and respectful" school. By conducting an internal program assessment, Kenwood has been able to demonstrate a reduction in incidents of violence and fighting, and an increase in student test scores, student grades, and student attendance. Over the course of three years, Kenwood’s Principal internally tracked the progress that the school was making in reaching its goal of a "peaceful and respectful" school. Statistics were collected for the following categories:
-Office referrals
-Suspensions
-Emergency removals
-Student fights
-Police involvement
-Number of "peaceful" days
-Number of students on the Honor Roll
-Number of students with perfect attendance
-Number of students who received awards for bringing up their grades
-Student attendance
Administrators kept track of these data on computers and are now able to use the data to demonstrate program effectiveness or outcomes. This assessment was relatively inexpensive and has provided the school with valuable data to demonstrate positive outcomes.
Regardless or who does the evaluation, it is useful to have someone in the program who can serve as the liaison with whoever is conducting the assessment.
It may be useful to set up a group of stakeholders as an informal "advisory committee" which can be used as a sounding board on assessment design and implementation issues, to help refine the focus of the evaluation, and to review and comment on data collection methodology.
Back to Schools Main Page
|
State Home Page | OCDRCM Home |
The Commission | Updates | Useful Resources