Comprehensive Systems of Care for Children and Youth in the North Caucasus (Health and Social Care for Children in the North Caucasus) Cooperative Agreement No. 118-A-00-06-00050-00 And Subcontract with Keystone Service Systems Inc. Final Performance Monitoring and Evaluation Report Submitted to: . Olga Kulikova Cognizant Technical Officer (CTO) USAID/Russia 19/23 Novinsky Bulvar Moscow 121099, Russia and Charles J. Hooker, III Maria Dolbunova – Chief of Party Keystone Services Systems, Inc. “Keystone” Social-Educational Foundation for Social 124 Pine Street Assistance to Children and Families Harrisburg, PA 17101 60/2 Novocheremushkinskaya Street Moscow 117420, Russia Submitted by: Leo T. Surla, Jr. Monitoring and Evaluation Specialist 3218 O Street NW, Suite 3 Washington, DC 20007 11 February 2009 ii TABLE OF CONTENTS LIST OF ACRONYMS ............................................................................................................... iv EXECUTIVE SUMMARY ......................................................................................................... vi PROJECT BACKGROUND........................................................................................................ 1 PERFORMANCE MONITORING AND EVALUATION....................................................... 2 PROJECT BASELINE................................................................................................................. 3 PROJECT PROGRESS ............................................................................................................... 3 NORTH OSSETIA ..................................................................................................... 3 KABARDINO-BALKARIA......................................................................................... 4 TRAINING .................................................................................................................................... 6 STRATEGIC FRAMEWORK, INDICATORS, TARGETS, AND PERFORMANCE ........ 7 FINDINGS AND CONCLUSIONS............................................................................................. 9 PROVISION OF CHILDREN AND FAMILY SERVICES .............................................. 9 SUSTAINABILITY.................................................................................................. 11 SUCCESS STORIES ................................................................................................ 12 PERFORMANCE MONITORING AND EVALUATION ............................................. 13 PROGRAM ISSUES................................................................................................. 13 RECOMMENDATIONS............................................................................................................ 14 ANNEXES A. SUMMARY OF CSOC TRAINING PROGRAM .................................................................. 16 B. CSOC PROJECT SUMMARY................................................................................................ 33 C. LIST OF CSOC PROJECT ORGANIZATIONS .................................................................... 40 iii D. REVIEW OF MATERIALS ON HIPPOTHERAPY .............................................................. 44 E. BESLAN COUNSELING DATA............................................................................................ 48 F. NALCHIK COUNSELING DATA.......................................................................................... 56 G. PERFORMANCE MONITORING AND EVALUATION PLAN ......................................... 60 iv LIST OF ACRONYMS ADS Automated Directive System CAF Charity Aid Foundation CSOC Comprehensive Systems of Care CTO Cognizant Technical Officer IR Intermediate Result KBR Kabardino-Balkaria Republic KFCF Keystone Foundation for Children and Families KHS Keystone Human Services KHSI Keystone Human Services International M&E Monitoring and Evaluation NGO Non-governmental Organization NO North Ossetia RCPMSS Republic Center of Psychological, Medical and Social Support SO Strategic Objective TACIS Technical Aid to the Commonwealth of Independent States UNICEF United Nations Children's Fund USAID United States Agency for International Development v QuickTime™ and a decompressor are needed to see this picture. vi EXECUTIVE SUMMARY The USAID-funded Comprehensive Systems of Care (CSOC) Project was implemented in the Russian North Caucasus by Keystone Service Systems, Inc. through a donor agreement between Keystone Human Services International (KHSI of Harrisburg, Pennsylvania) and the Keystone Foundation for Children and Families (KFCF of Moscow, Russia). The CSOC Project began on June 29, 2006 and ended on December 31, 2008. The CSOC Project provided targeted facilities and community-based programs for services to address the immediate psychological, educational, and health needs of vulnerable children and families. Project activities were developed and implemented around the principles of being child-centered, family-driven, strength-based, culturally competent and sensitive, and involving interagency collaboration. The project reached, but was not limited to, the following at risk groups: children, youth and families affected by violent trauma, children and youth with disabilities, children at risk of institutionalization, and families at risk of dissolution. Performance monitoring and evaluation (M&E) was an integral part of the Keystone methodology for achieving the intended project results. Project outputs and results through Center-collected statistics were monitored and contrasted against performance-based indicators for the CSOC Project activities. Performance monitoring and evaluation incorporated a participatory methodology to engage CSOC Project staff, Center staff in Nalchik and Beslan, and senior Keystone consultants in the implementation of the M&E Plan. MetaMetrics Inc., the subcontractor to Keystone, provided the basic technical assistance for the development and implementation of the Performance Monitoring and Evaluation Plan. CSOC Project activities were directed from the Moscow-based Keystone Foundation for Children and Families (KFCF) by the CSOC Chief of Party, Maria Dolbunova. The Project Manager, Vasanta Romanova, in addition to her management responsibilities, provided technical support for the collecting of performance monitoring and evaluation information from the Beslan and Nalchik CSOC Project Centers. Technical and administrative support was provided by the founding organization, Keystone Human Services International, based in Harrisburg, Pennsylvania, as coordinated by Charles Hooker. The CSOC Project focused on four main objectives, and activities were organized and implemented under the objectives as inter-related components. o Objective 1: Address the immediate psychological, educational, and health needs of the most vulnerable children and their families (technical assistance, equipment, grants). o Objective 2: Increase the professional capacity of the individuals serving these families. o Objective 3: Foster community development and citizen participation through involvement in program governance and oversight. o Objective 4: Initiate and maintain a comprehensive project monitoring and performance measurement strategy. TRAINING Training was a key activity for the CSOC Project and was the basic activity to assure sustainability. A major training topic, delivered by the CSOC Project, that directly related to sustainability was fundraising. Fundraising and Development Seminars were delivered in North Ossetia and in KBR. Community officials, citizens, service providers, youth, and families and vii children as recipients of CSOC services were the training participants. Training events included conferences, roundtables, seminars, workshops, and group outings. Over the two and a half year Project period a total of 670 persons participated in 24 training events. Approximately 90% of the training participants were women. Eight trainings were conducted in the first year during the eight-month period January 2007 through August 2007. Six events were held in the Republic of North Ossetia–Alania and two events were held in the Kabardino-Balkaria Republic. In Year 2, September 2007 through August 2008, there were a total of 434 participants for the sixteen training events. PROVISION OF CHILDREN AND FAMILY SERVICES Positive outcomes for participating families resulting from long-term (up to two and a half years of consultations) and short-term (visits to the Center over a two month period) services demonstrated the effectiveness of rehabilitation programs. Along with providing services for severe trauma, other issues affecting the well being of families in the North Caucasus were addressed by the Keystone Centers. Services at the Beslan Polylinic Center were expanded to include residents of surrounding areas and persons with disabilities. As a result of the demonstrated improvements in family well-being and child behaviors, there was an increased demand for family and children services in both Beslan and Nalchik. There was also an increase in adults seeking counseling services at both Centers. With increased demand for services, several Center counselors had issues characterized as “burnout.” The CSOC Project actively sought to engage volunteers through youth programs and hippotherapy1 events. Keystone established a working relationship with numerous agencies engaged in family and children services. Keystone supported interagency communications and was instrumental in increasing contact between the Ministry of Social Development and the Ministry of Health. SUSTAINABILITY Emphasis on program areas relating to sustainability was a Keystone priority. Funding to continue new services for families and children is only one area that deserves future support and continuing emphasis. Community support for and the identification of the value of family and children services are other areas to be considered for continuation in future program endeavors. CSOC efforts of training and technical assistance for fundraising and grant proposal preparation supported the development and continuation of organizations to provide services to families, children, and youth. Training and technical assistance were provided to service personnel at the Nalchik and Belsan Centers to develop their counseling capabilities. This CSOC effort enabled Center personnel to carry on into the future with a family-oriented and diversified perspective for providing services to children and families. 1 Hippotherapy is a treatment approach that uses the movement of a horse in addition to treatment principles that apply to the particular profession of the therapist providing the service. The client passively interacts with, and responds to, the horse's movement, which affects the physical, psychological, cognitive, social, behavioral, and communication systems of the body. See Annex D for additional information. viii SUCCESS STORIES The CSOC Project had numerous success stories from individuals and families receiving services from the Beslan and Nalchik Centers. Children who had exhibited anti-social behavior had dramatic shifts in their attitudes and behavior. The hippotherapy program at Vladikavkaz (Algus) is credited with being successful in reuniting children with their parents. Some local customs regarding families and children were overcome (a unmarried Muslim woman decided to have and keep her baby and more men are participating in CSOC programs). There was an increase in adult participation in CSOC programs and greater recognition by parents of the value of family counseling. The hippotherapy programs at Vladikavkaz and Nalchik provided opportunities to engage fathers in participating at the events. Both CSOC Centers expanded their services beyond the immediate urban area and incorporated family counseling in providing services to children. Services expanded to include persons with disabilities. PERFORMANCE MONITORING AND EVALUATION Senior Keystone staff, program consultants and Center staff expressed satisfaction in the M&E workshops and saw the value of reviewing and analyzing the newly organized Center information for understanding the process of providing family and children services and improving management practices. Information collection by Center staff provided valuable input for assessing Keystone program effectiveness. M&E activities by Center staff also supported improvements in managing caseloads and the well-being of counselors. Performance monitoring and evaluation information and analysis was incorporated in Keystone reports to USAID/Russia. PROGRAM ISSUES Keystone recognized the importance of and was committed to engaging the community in the objective of improving and expanding family and children services. Personnel engaged in the Beslan Center experienced several serious challenges in having sufficient resources to conduct the Program. Few professionals in the North Caucasus have appropriate technical backgrounds to address psycho-psychiatric issues of children and the needs of children with disabilities and other problems. Keystone lacked the full support of the Beslan Polyclinic doctor and Polyclinic Director to fund the Center service providers. Regional government authorities, at the end of the CSOC Project, still questioned the ulterior motives of Keystone program of services to families and children. While there was CSOC success in working with local authorities, much work remains to obtain continuing governmental support for the provision of children and family services. RECOMMENDATIONS A major factor in successfully implementing a family-oriented counseling effort is to incorporate parents into the process. More work is needed to overcome the reluctance of men to acknowledge their roles and importance to the family. An issue is to respect and compensate, if possible, for their sense and need for privacy. Continued support for and increased emphasis on successful program methods may assure future programmatic success in providing services to children and families in the North Caucasus. The support of local and regional organizations and governments is critical for successful program implementation. Future related projects should actively engage community organizations. The addition of a focused public information component could increase ix community awareness of program efforts and the opportunities to provide citizen support. Future related program efforts by USAID would also be aided by the implementation of performance monitoring and evaluation components that are sufficiently funded and supported. 1 COMPREHENSIVE SYSTEMS OF CARE FOR CHILDREN AND YOUTH IN THE NORTH CAUCASUS (CSOC) FINAL PERFORMANCE MONITORING AND EVALUATION REPORT The USAID-funded Comprehensive Systems of Care (CSOC) Project was implemented in the Russian North Caucasus by Keystone Service Systems, Inc. through a donor agreement between Keystone Human Services International (KHSI of Harrisburg, Pennsylvania) and the Keystone Foundation for Children and Families (KFCF of Moscow, Russia). The CSOC Project began on June 29, 2006 and ended on December 31, 2008. Through the provision of targeted facilities and community-based programs, services were provided to address the immediate psychological, educational, and health needs of vulnerable children and families. A Keystone Center has been operational in the community of Beslan in North Ossetia since January of 2007. A Keystone Center began operations in August 2007 in the town of Nalchik, the capital of Kabardino-Balkaria, the Republic adjacent in the west to that of North Ossetia. Project activities were developed and implemented around the principles of being child￾centered, family-driven, strength-based, culturally competent and sensitive, and involving interagency collaboration. With the input from local North Caucasus Program staff and regional partners, Keystone was committed that provided services follow each of these principles, as well as focus on the family's capacity to remain intact. The project reached, but was not limited to, the following at risk groups: children, youth and families affected by violent trauma, children and youth with disabilities, children at risk of institutionalization, and families at risk of dissolution. Monitoring and evaluation (M&E) was an integral part of the Keystone methodology for achieving the intended project results. Project outputs and results through Center-collected statistics, were monitored and contrasted against performance-based indicators for the CSOC Project activities. Performance monitoring and evaluation incorporated a participatory methodology to engage CSOC Project staff, Center staff in Nalchik and Beslan, and senior consultants in the implementation of the M&E Plan. The Performance Monitoring and Evaluation Plan (M&E Plan) included as Annex G provided the recommended general procedures for information collection and maintenance, analysis, and generating of M&E reports. A First Annual CSOC Performance Monitoring and Evaluation Report covered the six-month performance period of January 2007 through June 2007. This Final Performance Monitoring and Evaluation Report covers the project period through December 2008. PROJECT BACKGROUND The North Caucasus is an ethnically, historically, economically, and politically distinct region within the territory of Russia, and certain areas within this region are increasingly isolated from the rest of the Federation. The official statistics of the Ministry of Labor and Social Development of North Ossetia show that of the approximate 640,000 inhabitants in the Republic that 165,000 children aged 17 and younger live in the region, with close to 3,000 of these officially identified as disabled. With a slightly larger population of 780,000 persons in Kabardino-Balkaria, the number of children including those that are officially identified as disabled would be in the same magnitude. Basic health indicators in the North Caucasus remain among the worst in the country. 2 Political instability, and especially the terrorist attack on School No. 1 in Beslan of September 2004, had devastating consequences on the development of the youth of the region and the stability of their family units. Their physical and emotional health was impacted and, consequently, their ability to contribute to the future stability of the North Caucasus. Children who grow up in trauma are at great risk of engaging in violent behavior as adults and are very vulnerable to being recruited into radical ideologies. Similarly, the 2005 incident in Nalchik has potentially resulted in similar traumatic consequences in a Republic where less external rehabilitation support was provided in the aftermath. Since the 2004 school siege at Beslan in neighbouring North Ossetia, Mosques in Nalchik have been closed and armed operations were undertaken against suspected Islamic militants. In October 2005, Islamic militants, purportedly including many students, staged a large-scale, armed assault on government buildings in Nalchik. The swift government response deployed hundreds of troops and special forces. Many dozens were reported dead, including militants, civilians and Russian forces. Although there are a number of rehabilitation and social service programs for children currently operating in the North Caucasus, these are often limited and not adequately funded and coordinated. There are few government providers who have the appropriate education and training to provide psycho-psychological services. Many programs are in jeopardy regarding their ability to survive over the long-term. There has been little emphasis in the region on the importance of the family in social service programs that address trauma, disability, and behavior issues facing children. In January of 2007, the CSOC Project leadership finalized staff and consultant agreements, developed organizational relationships, and secured partnerships and support from Governmental and Non-Governmental entities in North Ossetia. A five-pronged approach to the enhancement of services was the foundation of a comprehensive system of care for children and families in the region. Beginning in February of 2007, with authorization and encouragement from USAID/Russia, exploratory visits to develop CSOC Project activities in the neighboring Kabardino-Balkaria Republic were undertaken. In June of 2007, employment agreements were finalized with three personnel of the Republic Center of Psychological, Medical and Social Support (RCPMSS). CSOC Project activities were directed from the Moscow-based Keystone Foundation for Children and Families (KFCF) by the CSOC Chief of Party, Maria Dolbunova. The Project Manager, Vasanta Romanova, also provided technical support for the collecting of performance monitoring and evaluation information from the Beslan and Nalchik CSOC Project Centers. Technical and administrative support were provided by the founding organization, Keystone Human Services International, based in Harrisburg, Pennsylvania, as coordinated by Charles Hooker. PERFORMANCE MONITORING AND EVALUATION The M&E Plan was developed in February 2007 according to the guidelines of the USAID Automated Directives System (ADS) 201-203 to support the management, monitoring, assessment, evaluation, and reporting requirements of USAID/Russia. The implementation of the M&E Plan was a process of ongoing review and improvement. As indicated in the USAID 3 Performance Management Toolkit, the M&E Plan served as a living document to manage the CSOC Project for results. In contrast to the traditional methodology of third-party monitoring and evaluation, Keystone implemented a participatory approach to take advantage of CSOC Project staff and consultants who were familiar with project objectives, community considerations, and local constraints to improving services to children and families. Staff and consultants cooperated to document project information for M&E analysis and to uncover success stories related to the project. The participatory approach to performance monitoring and evaluation required briefings to the staff and stakeholders by the subcontracted MetaMetrics M&E Specialist in Beslan and Nalchik on evaluation methods and the elicitation of a commitment by Center staff members to collect M&E information and document activities that contribute to project results. CSOC Project staff and consultants collected performance information with the guidance of the full￾time Russia-based Project Manager, Vasanta Romanova who served as the M&E Specialist providing technical support under the guidance of the CSOC Project Chief of Party. MetaMetrics Inc., the subcontractor to Keystone, provided the basic technical assistance for the development and implementation of the Performance Monitoring and Evaluation Plan. Mr. Leo T. Surla, Jr., an M&E Specialist, worked with CSOC Project leadership and selected staff and consultants during visits in February and July 2007 to develop the organizational structure and procedures to support the collection and analysis of results and program descriptive information as related to Work Plan activities. In September of 2008, the MetaMetrics M&E Specialist visited and interviewed with CSOC Project staff and consultants to collect statistics and other information related to CSOC Project performance. PROJECT BASELINE CSOC Center operations in Beslan began in January 2007. As of December 31, 2006 (serving as a baseline period), the Performance Indicator SO 3(6): Number of orphans and vulnerable children receiving child welfare services was 72 as reported by the CSOC Project consultant Alexander Venger on the services previously provided at the Beslan Polyclinic. This figure is for the number of individuals (children) that received services. On average, the six service providers at the Polyclinic each had a caseload of 11 individual children. The caseloads also involved other members of the families of these children. PROJECT PROGRESS The CSOC Project focused on four main objectives and activities were organized and implemented under the objectives as inter-related components. Selected highlights of CSOC Project activities are listed below according to the four objectives. More detailed descriptions of CSOC Project activities are presented in Annex B. Objective 1: Address the immediate psychological, educational, and health needs of the most vulnerable children and their families (technical assistance, equipment, grants). NORTH OSSETIA ο Expansion of the services provided by the Children's Rehabilitation Room (Center) at the Polyclinic Hospital in Beslan to also include psychiatric disability 4 and severe health issues in addition to treatment for victims of Beslan School No. 1 (employment agreements with the Project Director for the Medical Psychological Services and 5 Psychologists) o Support for Algus (Hippotherapy Program2 ) in Vladikavkaz including Family Retreats ο Support for Youth Forums (Youth from a North Ossetia Youth Center attended a Youth Forum Film Festival held in KBR and the Children’s Day Activity, Flowers of Life held in Beslan) o Two grants awarded by the Project to appropriate objectives-oriented efforts (Mozdok Project on developing services to families in areas near to Beslan and the Youth Project on Trainings for Teenagers) o Equipment provided to Rehab Center at the Polyclinic KABARDINO-BALKARIA o Expansion of the services provided to children with special needs by the Nalchik Republic Center for Psychological, Medical and Social Support (Agreements signed with 3 staff) o Support provided to six persons at the Nalchik Horse Sports School to achieve Hippotherapy Certification at the Moscow training program o Four grants to as coordinated with the Community Council (Horse Sports School, Ratmir Soccer Competition project, Zayukovo Boarding School project, and Kindergarten #25 Project Center) o Planning to provide equipment as appropriate to the project and requested by the Community Council o Employment agreements (7 persons) for achieving project objectives: Program Director, Community Development Coordinator, 3 staff at Republic Center, 2 Youth Program Coordinators o Support for Youth Forums (e.g. Youth Forum Film Festival and Youth Training Program in Vladikavkaz) 2 Hippotherapy is a treatment approach that uses the movement of a horse in addition to treatment principles that apply to the particular profession of the therapist providing the service. The client passively interacts with, and responds to, the horse's movement, which affects the physical, psychological, cognitive, social, behavioral, and communication systems of the body. See Annex D for additional information. 5 Objective 2: Increase the professional capacity of the individuals serving these families. ο Delivered Intensive In-home Family Therapy Training Programs and Supervision of Center personnel that included participants from all aspects of the service delivery system in North Ossetia for specialists involved in multidisciplinary services to children and families o Delivered Fundraising and Development Seminars in North Ossetia in March 2007 and in KBR in December 2007 and January 2008 o Provision of technical training for Keystone Center personnel through Irina Bgazhnokova, Developmental Problems Specialist of the Moscow Institute of Open Education from November 2007 through May 2008 o Training for Ministry of Labor and Social Protection in Nalchik through the Social Rehabilitation Centre for Minors (Otradnoye in Moscow) o Visits to KHS programs by professionals in social services arena (Open World Leadership Center/Library of Congress) o North Caucasus Roundtable overview of programs in the region in Vladikavkaz as preparation for a key international conference on child welfare systems Objective 3: Foster community development and citizen participation through involvement in program governance and oversight. ο Development of community resources including coordination of activities with UNICEF, CAF and Ministry of Labor and Social Development in North Ossetia through the September 2007 Roundtable in Vladikavkaz o Orientation meeting for establishing a Community Council held with key government and NGO providers of services to children and families in Kabardino-Balkaria, held in January/February 2008 o Completion of initial stage of Fund Raising Training Program and Advanced Training in Grant Writing for participants in the North Caucasus o Participation of 17 children from Social Rehabilitation Center “Kind Heart” in “Hearts of Young Muscovites are Open for Everybody” festival in Moscow on September 6, 2008 Objective 4: Initiate and maintain a comprehensive project monitoring and performance measurement strategy. ο Completion of the Performance Monitoring and Evaluation Plan o Full-time Russia-based Project Manager for monitoring and evaluation 6 o Conduct of North Caucasus Keystone staff training in M&E o Conduct of M&E briefing with providers of family and children services in Kabardino-Balkaria o Completion of the 13 month Performance Monitoring and Evaluation Report o Completion of the Final Performance Monitoring and Evaluation Report TRAINING Training was a key activity for the CSOC Project and was the basic activity to assure sustainability. A major training topic, delivered by the CSOC Project, that directly related to sustainability was fundraising. Fundraising and Development Seminars were delivered in North Ossetia and in KBR. Community officials, citizens, service providers, youth, and families and children as recipients of CSOC services were the training participants. Training events included conferences, roundtables, seminars, workshops, and group outings. CSOC Project staff and consultants planned training events and defined the size and nature of the target populations and topics. Over the two and a half year Project period a total of 670 persons participated in 24 training events. Approximately 90% of the training participants were women. Summaries of the training events are presented in Annex A. Eight trainings were conducted in the first year during the eight-month period January 2007 through August 2007. Six events were held in the Republic of North Ossetia–Alania and two events were held in the Kabardino-Balkaria Republic. There were a total of 236 participants during the first year. Since some persons participated in more than one training event, the total of separate individuals that participated was estimated at 137. Training on the “Basis of Fundraising: Attracting Funds for Social Organizations” was the largest training program and had 72 participants of which 61 were women. “Fostering Community Development and Citizen Participation Through Involvement in Program Governance and Oversight” was conducted as sessions with a total of 84 participants. “Increase Professional Capacity for Serving Vulnerable Children and Their Families” consisted of three sessions with 45 individual participants. “Project Performance Monitoring and Evaluation” consisted of three training events with 17 individual participants. Some events were conducted in one day. Others, primarily the education courses, were conducted over several days. The 237 participants were engaged in training for 567 training days. In Year 2, September 2007 through August 2008, there were a total of 434 participants for the sixteen training events. Since some persons participated in more than one training event, the total of separate individuals that participated was estimated at 225. “Improving the system of educational and social services for children and adolescents with special educational needs and for their families” was the largest training program, with 72 participants. “Basics of diagnostic and correction of disorders of Autism spectrum” was a four-day training program with 52 total participants. “Otradnoye program for specialists working with multi-crisis families and their children” was conducted in five sessions throughout the winter and spring of 2008. Each session lasted three days and a total of 42 specialists received training. “Supervision support of specialists providing psychological, social support for the population of North Ossetia” was 7 conducted in five separate sessions (three in North Ossetia, two in KBR), each with 20-25 participants. Others, primarily the education courses, were conducted over several days. The 434 participants were engaged in training for approximately 1,600 training days. STRATEGIC FRAMEWORK, INDICATORS, TARGETS, AND PERFORMANCE The CSOC Project addressed the USAID/Russia Strategic Objective (SO) 3: Use of Improved Health and Child Welfare Practices Increased. The Performance Indicator (6) to reflect progress against the Strategic Objective is “Annual number of orphans and vulnerable children receiving child welfare services.” In the Beslan Center for the calendar year 2007, a total of 150 individual orphans and vulnerable children received counseling services. In a twelve-month annual period based on available information through June 2008, an estimated 800 orphans and vulnerable children received counseling services at the Nalchik Center. Many of these children at both Centers participated in CSOC supported hippotherapy and other youth programs. Corresponding to SO 3 is the USAID/Russia Intermediate Result (IR) 3.4(1): Cumulative number of organizations involved in the development, reform and/or implementation of child welfare systems. According to the Keystone Quarterly Reports on the Performance Indicator IR 3.4(1), the number of Russia and North Caucasus-based organizations that had been involved during the CSOC Project period in the development, reform, and/or implementation of child welfare systems was 129 (Annex C). The total of North Ossetia-based organizations was 59 and the number of Kabardino-Balkaria-based organizations was 48. The remaining 22 organizations were primarily located in Moscow. Additional CSOC Project performance information as requested by USAID/Russia was collected as follows: o Number of children and families served (both direct and indirect beneficiaries) Over the 24-month period January 2007 through December 2008, the number of individual children served was an estimated 389 at the Beslan Center. The number of families served in Beslan was estimated at 434. In Nalchik, individual children served over the 12-month period October 2007 through September 2008 was 730 with an estimated similar number of families. The total served by the combined Centers over the CSOC Project through December 2008 was 1119 children and approximately the same number of families. Over the CSOC Project period through December 2008, direct beneficiaries also included adults. The total number of individual adults served by the Beslan Center was 359 and at the Nalchik Center was an estimated 259. Total direct 8 beneficiaries from the two Center programs, including children, was over 1,700. Using a factor of three to one3 results in an estimated 5,100 indirect beneficiaries. o Number of new services developed In addition to funding the operations of the Beslan and Nalchik Centers, the CSOC Project supported the expansion of services to disabled children and supported hippotherapy in the North Caucasus. Services to adults and families was expanded. o Number of seminars/workshops/trainings conducted A total of 670 persons participated in 24 training events. Approximately 90% of the training participants were women. Training topics and number of training events: Family Therapy Training: 5 Development/Disability: 5 Community Development: 3 M&E Workshops: 3 Psychological/Social Support: 2 Youth Programs: 2 Supervision Support: 2 Fund Raising Training: 2 o Number of specialists (and community leaders) trained Three service providers at the Beslan Center and six at the Nalchik Center received intensive training through CSOC consultants and through participation in the CSOC region-wide training program. An estimated 200 additional specialists providing services to children, families, and youth were participants in CSOC training programs in new treatment methodologies. An estimated 100 community leaders including government officials and NGO personnel were trained. o Number of community-level events conducted A total of five community-level events were conducted over the CSOC Project period. Keystone in the Cooperative Agreement with USAID/Russia proposed that CSOC technical assistance would result in the following selected outputs: o Establishment of two Comprehensive Systems of Care (CSOCs) Systems were established in North Ossetia and in Kabardino-Balkaria. 3 The factor of 300% or 3 to 1 to estimate the number of indirect beneficiaries to direct beneficiaries was developed and derived from discussions with Beslan Center personnel and CSOC Specialist Consultants. 9 o Development of model services for no less than 450 children and their families through the Center-based and home-based programs and other CSOC activities Over 1119 children and their families were served by the CSOC Center-based programs that utilized model services as instituted by Keystone technical assistance and training. o Assistance to up to 300 young people in a youth club format and other CSOC activities Over 1,000 young people participated in youth club events sponsored by CSOC. o Development of the fund-raising capacity of both Centers, NGOs and community organizations involved in CSOCs, to generate financial support and achieve long-term sustainability Training programs in fund raising were delivered to participants in North Ossetia and KBR. Grant proposals were submitted to Keystone as a result and other grant proposals have been developed for eventual submission to potential funding organizations. FINDINGS AND CONCLUSIONS Beslan and Nalchik Center statistics, CSOC Quarterly Reports, interviews with direct beneficiaries, and discussions with Project staff, Center personnel, and CSOC consultants provided the basis for the findings below. PROVISION OF CHILDREN AND FAMILY SERVICES o Rehabilitation Programs Are Effective: Direct beneficiaries, both children and adult family members, provided testimonials on the value of the family services received from the Beslan Center. There were positive results from long-term (two to three years of consultations) and short-term (visits to the Center over a two month period) services. Hippotherapy was cited as effective in establishing communications with children and enhancing their self-awareness. o Adults Need Family Services: Adult family members are essential to the health of the family and often benefit personally from Center visits. At times, adults of the family are directly responsible for family issues and may need special attention from counselors. o Beslan Center Trends: A review of the monthly statistics from the Beslan Center (see Annex E) provided the following: - Adults accounted for a slightly larger number of initial visits in 2007 (12 per month) and remained the same in 2008 (12 per month). Initial visits by families 10 rose dramatically in 2008 (20 per month) over the family initial visits in 2007 (12 per month). Childrens’ initial visits also increased dramatically in 2008 (17 per month) as compared to 2007 (11 per month). - Repeated visits by adults, families, and children increased sharply from 2007 to 2008. - There was little month to month or seasonal changes in initial and repeated visits other than the trend changes from 2007 to 2008. - Outside Center work focused heavily on children in both years. o Nalchik Center Trends: A review of quarterly statistics from the Nalchik Center (see Annex F) provided the following: - Speech disorders and mental retardation constituted the overwhelming majority of the reasons for visits to the Center - The gender split of children visiting the center reversed quite dramatically after the initial quarterly report with more boys constituting the caseload - Visitors tended to be more from urban areas - There was a much greater probability of a school initiating a child's visit than his/her family doing so o Severe Trauma is not the Only Issue for Providing Family and Children Services: While there has been a focus on providing services to the community of individuals affected by the incident at Beslan School No. 1 over the past four years, other issues affect the well being of families in the North Caucasus that were addressed by the Keystone Centers. At the Nalchik Center, children with special needs were the target beneficiaries. o Burnout Can be a Problem for Counselors: With increased demand for services, several Center counselors had issues characterized as “burnout.” A counselor at the Beslan Center stated that she provided consultations to seventeen individuals in one day and that she relied on her own paid therapist to support her well-being. CSOC Project consultants addressed the issues stemming from increasing caseloads through training and support. o Adults and Men: More adults over time were seeking counseling services at both Centers. Potentially more men were participating in Center programs. Hippotherapy programs in Nalchik and Beslan provided settings for adult involvement. 11 o Volunteers: The CSOC Project actively sought to engage volunteers. Approximately 30 persons volunteered at the youth program “Flowers for Life.” An average of 4 persons participated at each hippotherapy event. Two CSOC training programs were delivered on volunteerism. o Demand for Services: As a result of the demonstrated improvements in family well-being and child behaviors, there was an increased demand for family and children services in both Beslan and Nalchik. o Expansion of Services: Previous to CSOC engagement, the Beslan Polyclinic Center only provided services to Beslan residents as was generally advocated by the community organization, “The Mothers of Beslan.” Services are now being provided to residents of surrounding areas. Center staff now deal with the family as an entire unit in addressing issues of children. The type of services expanded at Beslan with persons with disabilities now being accepted as clients. o Rapport with Local Authorities: In spite of lingering suspicion by local government authorities regarding CSOC program motives, Keystone established a working relationship with numerous agencies engaged in family and children services. Keystone supported interagency communications and was instrumental in increasing contact between the Ministry of Social Development and the Ministry of Health. o Emergency Consultant Services: The Senior CSOC Consultants, Venger and Morozova, were present in North Ossetia to provide psychological services to refugees from the Georgia/South Ossetia conflict of August 2008. SUSTAINABILITY o Sustainability is a Priority for the Keystone effort: Program areas relating to sustainability was a Keystone priority. Funding support to continue new services for families and children is only one area that deserves future support and continuing emphasis. Community support for and the identification of the value of family and children services are other areas to be considered for continuation in future program endeavors. o Fund Raising: Through CSOC efforts of training and support for fund raising and grant proposal preparation, five grant applications from North Caucasus organizations were in process of being submitted for potential funding through a pending program by the Open Society, an active partner with Keystone. o Grants: CSOC awarded five mini-grants to related children, family, and youth organizations in the North Caucasus, a direct result of CSOC fund raising training. The skills in fundraising as provided through two CSOC training events supported the development and continuation of organizations to provide services to families, children, and youth. 12 o Continuation of Financial Support to the Nalchik Center: Through the efforts of CSOC Consultant Marina Sokolova of Handicap International, three of the seven persons who are providing children and family services at the Nalchik Center will be funded through TACIS. o Training of Service Providers: The service personnel at the Nalchik and Beslan Centers received intensive training and technical assistance to develop their counseling capabilities. An estimated additional 200 service providers in the North Caucasus received training in counseling. These persons can be expected to carry on into the future with a family-oriented and diversified perspective for providing services to children and families. The potential for sustainability is demonstrated by the commitment of the two married staff members at the Beslan Center to develop a Civil Society Organization to continue providing more and improved services to children and families in the North Caucasus. o Keystone Russia: Keystone Human Services International (KHSI of Harrisburg, Pennsylvania) founded the Keystone Foundation for Children and Families (KFCF of Moscow, Russia) which supports the CSOC Project. KFCF may continue to operate and support family and children services after completion of the CSOC Project in December 2008. SUCCESS STORIES o Results from Receiving Services: The CSOC Quarterly Reports presented numerous success stories from individuals and families receiving services from the Beslan and Nalchik Centers. Children who had exhibited anti-social behavior had dramatic shifts in their attitudes and behavior. The hippotherapy program at Vladikavkaz (Algus) was credited with being successful in uniting children with their parents. Some local customs regarding families and children were overcome (a unmarried Muslim woman decided to have and keep her baby and more men are participating in CSOC programs). In many cases, family members expressed appreciation for the changes that had come about from Center counseling. These results are the primary outcomes of the continuation of counseling services and the improvements in counseling methods as a result of CSOC consultant technical assistance and training. o Engaging Adults: There was an increase in adult participation in CSOC programs and greater recognition by parents of the value of family counseling. The hippotherapy programs at Vladikavkaz and Nalchik were opportunities to engage fathers in participating at the events. o Expansion of Services: Both CSOC Centers provided services beyond the immediate urban area. Both Centers incorporated family counseling in providing services to children. Services expanded to include persons with disabilities. o Community Organizations: The Beslan Center integrated people with similar problems which supported the organization of these clients to demand services. 13 o Program Coordination: The U.S. government strategy of partnering with the Russian government to provide mutually acceptable development programs was successfully implemented through the CSOC Project. Key to this achievement was the oversight, monitoring, and management of personnel at USAID/Moscow. PERFORMANCE MONITORING AND EVALUATION o M&E is Effective in Educating Staff: Senior Keystone staff, program consultants and Center staff expressed satisfaction in the M&E workshops conducted by the MetaMetrics M&E Specialist and the Keystone Project Manager for M&E. They also saw the value of reviewing and analyzing the newly organized Center information for understanding the process of providing family and children services and improving management practices. o M&E Information Collection on Caseloads: Information collection by Center staff on caseflow, estimating indirect beneficiaries, and success stories provided valuable input for assessing Keystone program effectiveness. M&E activities by Center staff also supported improvements in managing caseloads and the well￾being of counselors. o Documenting CSOC Project Progress: The efforts of the Keystone Project Manager and the Center staff to collect performance information, success stories, and program descriptions resulted in more detailed Quarterly Reports to USAID/Moscow. The USAID CTO acknowledged the value of the enhanced Quarterly Reports. PROGRAM ISSUES o Engagement of the Community is Important: Keystone was committed to engaging the community in the objective of improving and expanding family and children services. The community, through local organizations and efforts of citizens, can encourage the local and regional governments to provide necessary services. o Lack of Resources: Personnel engaged in the Beslan Center experienced several serious challenges in having sufficient resources. As examples, transportation was needed for children served by the Center in Beslan to Algus (hippotherapy) located near Vladikavkaz. The Beslan Center staff members lived in Vladikavkaz and used large portions of their salaries for taxi transportation to Beslan. o Technical Backgrounds: Few professionals in the North Caucasus have appropriate technical backgrounds to address psycho-psychiatric issues of children and the needs of children with disabilities and other problems. o Government Support: At the outset of the CSOC program the Polyclinic doctor was reluctant allow Keystone to fund service providers. Polyclinic Director was not very helpful. Regional government authorities at the end of the CSOC 14 Project, still questioned the ulterior motives of Keystone. While there was CSOC success in working with local authorities, much work remains to obtain continuing governmental support for the provision of children and family services. RECOMMENDATIONS The foregoing findings and conclusions, in summary, indicate that the two-year USAID￾funded Comprehensive Systems of Care (CSOC) Project as implemented in the Russian North Caucasus by Keystone Service Systems, Inc. achieved its many program objectives. The experience of the overall program can guide future related efforts of USAID and other interested organizations. o Engaging Adult Male Family Members: A major factor in successfully implementing a family-oriented counseling effort is to incorporate parents into the process. The engagement of fathers is essential for supporting family well-being. More work is needed to overcome the reluctance of men to acknowledge their roles and importance to the family. An issue is to respect and compensate, if possible, for their sense and need for privacy. A telling statistic is that almost 90% of the participants in CSOC training programs were women. o Continue Effective Program Efforts: While an obvious recommendation, the continuing support for and possibly increased emphasis on successful program methods may assure future programmatic success in providing services to children and families in the North Caucasus. For example, the instituting of hippotherapy in Nalchik, proven early in the implementation of the CSOC Project to be a successful methodology in Vladikavkaz, was also promoted to support the improvement of family and children services in the Kabardino-Balkaria Republic. o Coordinate with Local and Government Agencies: The support of local and regional organizations and governments is critical for successful program implementation. Keystone demonstrated that this lesson was learned as the Project was expanded to the Kabardino-Balkaria Republic. o Community Organization: Civil society organizations and engaged citizens can advocate for changes and resources to support the provision of children and family services. The CSOC Project supported the engagement of community organizations through the technical assistance, training, and small grants programming. Future related projects should actively engage community organizations. The addition of a focused public information component could increase community awareness of program efforts and the opportunities to provide citizen support. o Performance Monitoring and Evaluation: The CSOC Project demonstrated the value of the implementation of a participatory methodology for collecting and analyzing monitoring and evaluation information. Additional documentation was 15 included in the quarterly reports to USAID and the collected information proved valuable to Center management. Future related program efforts by USAID would also be aided by the implementation of performance monitoring and evaluation components that are sufficiently funded and supported. 16 ANNEX A SUMMARY OF CSOC TRAINING PROGRAM February – July 2007 Training Event Name: 1. Keystone M&E Workshop for Beslan Polyclinic Staff Presenter(s): Leo T. Surla, Jr. M&E Specialist, MetaMetrics Inc., Subcontractor to Keystone Dates: Tuesday Afternoon, February 20, 2007 Number of Sessions: One Session, Two Hours Number of Participants: 10 Characteristics: One Psychiatrist, 5 Psychologists, one Director of Keystone, two Expert Psychologists, one Social Worker Organization(s): Children’s Rehabilitation Room (Center) at the Polyclinic Hospital in Beslan Gender: Two male participants, 8 female participants Location(s): Beslan, North Ossetia Summary: A presentation was made on basic principles of monitoring and evaluation. The objective was to secure Polyclinic (Center) staff support for the identification of program indicators and initiate the procedures for collecting, maintaining, and reporting program descriptions, outputs, results and impact of Center services to children and families. Expected Results: Center staff members become familiar with M&E concepts Realized Results: Agreement was secured to expand the number of indicators and collect the necessary M&E information from direct and indirect beneficiaries. Alexander Venger developed and refined an Excel program to collect program information from each staff member. Participant Comments: The following day, the staff responded positively on the Workshop in the presence of the Keystone Chief of Party. They expressed interest in maintaining and analyzing the information to be collected from program clients. They also expressed interest in participating in a Focus Group Workshop that is planned to be available when the M&E Specialist returns to Beslan. 17 Training Event Name: 2. 3. Basis of Fundraising. Attracting funds for social organizations CSOC Component: Objective 3: Foster community development and citizen participation through involvement in program governance and oversight. Presenter(s): Ann Moffitt, CFRE, Vice President for Development for Keystone Service Systems and a Certified Fundraising Executive by the International Fundraising Professionals Association Dates: March, 2007 Advanced Fundraising Training took place in February 2008 with 20 participants Number of Sessions: 2 days (5 hours each session) Number of Participants: 72 Characteristics: Organization(s): 42 organizations from different parts of North Ossetia Republic – Alania and Republic of Kabardino-Balkaria. Among them 3 institutions belong to the authority of the Ministry of Health: 1) Republic Centre of Medical, Social and Psychological Help, Vladikavkaz; 2) Rehabilitation Centre, Beslan; 3) Republic Psychiatric Clinic of North Ossetia Republic – Alania. 5 institutions belong to the authority of the Ministry of Education: 1) Secondary school N8, Beslan; 2) Centre “Rosinka”; 3) Kindergarden “Raduga”; 4) Department of Child Social Support under the Ministry of Education (correctional pedagogue); 5) Republic Centre of Science and Technique Creativity of Pupils. 5 NGOs: 1) Family Centre “Binonte”; 2) Fund “Uspenie”; 3) North Ossetian Republic Branch of Russian Red Cross; 4) Kabardino- Balkaria humanitarian Fund named of Zaramuk Kardagushev; 5) Charity Fund “Centre of Peace Making and Public Development”. 21 institutions belong to the authority of the Ministry of Labour and Social Development: 1) Republic Centre for Rehabilitation Disabled Children “Feniks”, Vladikavkaz; 2) Republic Social Rehabilitation Centre for Minors “Namis”, Nalchik; 3) Republic Centre of Social Rehabilitation of Minors “Dobroe Serdze”, Vladikavkaz; 4) Republic orphanage for mentally retarded children “Laska”; 5) Republic Therapy and Rehabilitation Centre for Children, village Tamisk; 6) Comprehensive Centre of Public Social Services of Iristonskiy Municipal Okrug, Vladikavkaz; 7) Comprehensive Centre of Public Social Services of Zaterechniy Municipal Okrug, Vladikavkaz; 8) Comprehensive Centre of Public Social Services of Promyshlenniy Municipal Okrug; 9) Comprehensive Centre of Public Social Services of North-West Municipal Okrug; 10) Department of Public Social Protection of Pravoberezhniy Region; 11) Department of Public Social Protection of Alagirskiy Region, Alagir; 12) Department of Public Social Protection of Dig. Region; 13) Department of Public Social 18 Protection of Mozdok Region, Mozdok; 14) Department of Public Social Protection of Iraphskiy Region, village Chicola; 15) Department of Public Social Protection of Kirovskiy Region, village Elkhotovo; 16) Department of Public Social Protection of Prigorodniy Region, village Oktyabrskoe; 17) Department of Public Social Protection of Ardonskiy Region, Ardon; 18) Centre of Psychological, Pedagogical, Medical and Social Help; 19) Diagnostic and Consulting Centre “Doverie”; 20) Alagir Centre of Health for Families and Children, Alagir; 21) Republic Centre of Psychological, Medical and Social Support. 3 Ministries: 1) Ministry of Labour and Social Development of the Republic of Kabardino-Balkaria; 2) Ministry of Labour and Social Development of the Republic of North-Ossetia – Alania, Vladikavkaz; 3) Ministry of Education and Science of the Republic of North-Ossetia – Alania. 5 other institutions: 1) North Ossetian Republic Committee on Public Employment Service; 2) Committee on Minors’ Affairs and of Protection of their Rights of Iraphskiy Region, village Chikola; 3) Boarder of Representative of Pravoberezhniy Region; 4) Institute of Youth Problems “Open Hearts”; 5) North Caucasus Youth Union, Republic of Kabardino-Balkaria. Gender: 11 males, 61 females. Location(s): Vladikavkaz, North Ossetia - Alania Summary: Discussion of issues related to Basis of Fundraising, Attracting funds for social organizations. Expected Results: Direct communication and discussions between organizations participated; stimulation of collaboration and fundraising activities and grant application writing skills of the specialists work in Child Care System. Realized Results: 4 mini-grant projects worked out and presented to CSOC project director (Maria Kalitina). Training Event Name: 4. Intensive In-Home Family Therapy. CSOC Component: Objective 2: Increase the professional capacity of the individuals serving the most vulnerable children and their families. Presenter(s): Natalia Snurnikova and Kirill Sokolov, experts from the NGO “Otradnoya” from Moscow. Dates: February 20-22, 2007 Number of Sessions: 3 days (5 hours each session) Number of Participants: 40 Characteristics: 19 Organization(s): 15 organizations from North Ossetia: 11 from public social protection system (4 comprehensive centres of public social services, 1 centre of social help for families and children, 1 social rehabilitation centre for minors, 1 Guidance of public social protection, 2 Committees on minors affairs and of their rights protection, 1 rehabilitation centre for disabled children, 1 social service centre for lonely elderly people and disabled); 2 organizations from the State Education System (1 State education school-internat, 1 State university); 1 institution presented the Health System (Central local clinic); 1 centre “Doverie” presented by NGO. Gender: 3 males, 37 females. Location(s): Vladikavkaz, Republic of North Ossetia - Alania Summary: Discussion of issues related to different forms of social rehabilitation oriented on teenagers and their families with serious behavioural problems. (Peculiarities of team work, possibilities of the participants to organize a team work at their working places. Theoretical basis of IFT. Theory of social learning. Salutogenic processes, theory of A. Antonovsky. Family behavioral stereotypes, learned helplessness. 9 principles of multi-system rehabilitation. Multi-system theory. Phase of Involvement. Home visits. Method of Positive Reformulations. Phase of inquiry. Formulation of goals. Multi-system analysis. Motivating interview. Genogram. Map of social contacts. Case analyses). Expected Results: Increased use of comprehensive approach in serving children and families. Comprehensive team work competence raised. Realized Results: 13 organizations formed multidisciplinary teams of working with families with multiple crises. New tools of comprehensive diagnostic of families with multiple crises were practiced at the workshops. Training Event Name: 5. Intensive In-Home Family Therapy. CSOC Component: Objective 2: Increase the professional capacity of the individuals serving the most vulnerable children and their families. Presenter(s): Natalia Snurnikova and Yulia Somova, experts from the NGO “Otradnoya” from Moscow. Dates: May 14-16, 2007 Number of Sessions: 3 days (5 hours each session) Number of Participants: 44 Characteristics: Organization(s): 20 organizations from all over the Republic: 20 (6 Comprehensive Centres of Social Services; 3 Departments of Social Protection; 3 Committees on minor’s affairs; 1 Centre on Prevention of Drug addiction, alcohol addiction and delinquency among youth; 1 Clinic hospital; 1 Centre of Social Help for Families and Children; 1 Centre of Social Rehabilitation for Minors; 1 Centre of Rehabilitation of Disabled children; 1 Centre (not determined); 1 State University; 1 Boarding School) Gender: 3 males, 41 females. Location(s): Vladikavkaz, Republic of North Ossetia - Alania Summary: Discussion of issues related to different forms of social rehabilitation oriented on teenagers and their families with serious behavioural problems. (Working out a Working Plan; Network meeting. Practical tools of Family Therapy). Expected Results: Increased use of comprehensive approach in serving children and families. Comprehensive team work competence raised. Raised skills in Family Therapy. Realized Results: 13 organizations formed multidisciplinary teams of working with families with multiple crises. New tools of Family Therapy of families with multiple crises were practiced at the workshops. Training Event Name: 6. Intensive In-Home Family Therapy. CSOC Component: Objective 2: Increase the professional capacity of the individuals serving the most vulnerable children and their families. Presenter(s): Natalia Snurnikova and Kirill Sokolov, experts from the NGO “Otradnoya” from Moscow. Dates: June 18-20, 2007 Number of Sessions: 3 days (5 hours each session) Number of Participants: 45 Characteristics: Organization(s): 20 organizations from all over the Republic: (6 Comprehensive Centres of Social Services; 3 Departments of Social Protection; 3 Committees on minor’s affairs; 1 Centre on Prevention of Drug addiction, alcohol addiction and delinquency among youth; 1 Clinic hospital; 1 Centre of Social Help for Families and Children; 1 Centre of Social Rehabilitation for Minors; 1 Centre of Rehabilitation of Disabled children; 1 Centre (not determined); 1 State University; 1 Boarding School) Gender: 3 males, 42 females. Location(s): Vladikavkaz, Republic of North Ossetia - Alania 21 Summary: Discussion of issues related to different forms of social rehabilitation oriented on teenagers and their families with serious behavioural problems. (Young people Therapy; Practical tools of Young people Therapy. Completion of the multidisciplinary therapy course). Expected Results: Increased use of comprehensive approach in serving children and families. Comprehensive team work competence raised. Raised skills in Young People Therapy. Realized Results: New tools of Young People Therapy of families with multiple crises were practiced at the workshops. Multidisciplinary teams from 13 organizations started at their places practical work of using Intensive In-home Family Therapy. Training Event Name: 7. Keystone M&E Workshop for Beslan Center Staff Presenter(s): Leo T. Surla, Jr. M&E Specialist, MetaMetrics Inc., Subcontractor to Keystone; Vasanta Romanova, Project Manager; Maria Kalitina, Keystone Director Dates: Monday, 16 July 2007 Number of Sessions: One Session, Four Hours Number of Participants: 8 Characteristics: One Psychiatrist, 4 Psychologists, One Social Worker, Two Expert Psychologists Organization(s): Children’s Rehabilitation Room (Center) at the Polyclinic Hospital in Beslan Gender: Two male participants, 6 female participants Location(s): Beslan, North Ossetia Summary: A presentation was made on information collected by Center staff relative to monitoring and evaluation. The objective was to improve the quality and relevance of program indicators and encourage improved procedures for collecting, maintaining, and reporting program descriptions, outputs, results and impact of Center services to children and families. Expected Results: Center staff members become familiar with advanced M&E data collection concepts Realized Results: Agreement was secured to systematize the given indicators and collect the necessary M&E information. The estimating of the number of indirect beneficiaries was discussed. Alexander Venger and Elena Morozova promised to refine the service definitions and obtain missing information. 22 Participant Comments: The monthly review information was considered helpful to assess progress of staff activities. New questions were discussed for collection of information on clients and families. Training Event Name: 8. Keystone M&E Workshop for Nalchik Rehabilitation Center for Psychological, Pedagogical, and Medical Support Staff Presenter(s): Leo T. Surla, Jr. M&E Specialist, MetaMetrics Inc., Subcontractor to Keystone; Vasanta Romanova, Project Manager; Maria Kalitina, Keystone Director Dates: Thursday, 19 July 2007 Number of Sessions: One Session, Two Hours Number of Participants: 6 Characteristics: Three Center Staff, One Keystone Project Director, One Project Consultant, One Keystone Youth Coordinator. Gender: Two male participants, 4 female participants Location(s): Nalchik, Kabardino-Balkaria Summary: A presentation was made on basic principles of monitoring and evaluation. The objective was to secure Center staff support for the identification of program indicators and initiate the procedures for collecting, maintaining, and reporting program descriptions, outputs, results and impact of Center services to children with disabilities and families. Expected Results: Center staff members become familiar with M&E concepts Realized Results: Agreement was secured to expand the number of indicators and collect the necessary M&E information from direct and indirect beneficiaries with support and guidance from Vasanta Romanova. Participant Comments: Irina Kishukova, Keystone Consultant, stated that the indicators can be very useful for the work of the Center. Training Event Name: 9. Community Council Meeting. CSOC Component: Objective 3: Foster community development and citizen participation through involvement in program governance and oversight. Presenter(s): Maria Kalitina, Keystone Director; Leo T. Surla, Jr., Director of MetaMetrics Inc., Vasanta Romanova, Project Manager; Irina Kishukova, Project Consultant. Dates: 20 July 2007 Number of Sessions: 1 day, 3 hours Number of Participants: 11 23 Characteristics: Organization(s): Kabardino-Balkarian Humanitarian Fund named by Zaramuk Kardangush; NO Charity Organization “Center of Peace building and Society Development”; Charity Organization «Development»; Interregional Community Movement “Pure hearts”; Republic State Education Institution Boarding School; Municipal Child Preschool Education of Compensatory Type N25; Republic Culture-Health Club of People with disabilities “Impulse”; Children Club “AYAZ”; Republic Social Rehabilitation Centre for Minors “Namys”; State Educational Institute “Specialized (correction) Secondary Education School-Boarding School N1 of the VIII type”, village Zayukovo. Gender: 5 males, 7 females. Location(s): Summary: Discussion of issues related to the role of the Community Council in working out the Project Strategy, the Project Activities and sharing and make transparent the project activities Expected Results: Sharing and gathering the information concerning the needs assessment of the children and families at risk in KBR. Strengthened links between the organizations involved in the Community Council Realized Results: List of needs and resources of each organization participated in the Community Council Meeting. August 2007 – June 2008 Training Event Name: 1. Intensive In-Home Family Therapy CSOC Component: Objective 2: Increase the professional capacity of the individuals serving the most vulnerable children and their families. Presenter(s): Natalia Snurnikova and Kirill Sokolov, experts from the NGO “Otradnoya” from Moscow. Dates: September 23-25, 2007 Number of Sessions: 3 days (5 hours each session) Number of Participants: 40 Location(s): Vladikavkaz, Republic of North Ossetia – Alania 24 Summary: Discussion of issues related to different forms of social rehabilitation oriented on teenagers and their families with serious behavioural problems. Expected Results: Increased use of comprehensive approach in serving children and families. Comprehensive team work competence raised. Raised skills in Young People Therapy. Training Event Name: 2. “Psychology of children and teenagers with developmental problems” and “Child Cerebral Paralysis” CSOC Component: Objective 2: Increase the professional capacity of the individuals serving the most vulnerable children and their families. Presenter(s): Irina Bgazhnokova and Nina Simonova, Moscow State Open Institute Dates: November 13-19 and December 8-16, 2007 Number of Sessions: Two course sessions, 16 total days Characteristics: Specialists from four child disability organizations Location(s): Kabardino-Balkaria Summary: Training in Special Education (correction pedagogics) and Special Psychology, Diagnostics / Evaluation: Differential Diagnostics, Early Childhood Evaluation, Diagnostics of Developmental Problems, Family involvement in educational support to children with developmental problems, Development of children with Cerebral Paralysis, Correction of Speech disorders Training Event Name: 3. “Supervision support of specialists providing psychological, social support for the population of North Ossetia” CSOC Component: Objective 2: Increase the professional capacity of the individuals serving the most vulnerable children and their families. Presenter(s): Institute of Psychotherapy and Consulting “Harmony” Dates: December 19-22, 2007 Number of Sessions: 1 Number of Participants: 22 Characteristics: Participants from North Ossetian System of Health (9 participants), the System of Education (7 specialists) and the System of Social Protection (6 specialists). Location(s): Vladikavkaz and Nalchik, Republic of North Ossetia Summary: Supervision support services for the professionals working with vulnerable children and families in North Ossetia 25 Expected Results: The training was aimed at 1) providing a professional support in working with difficult cases; 2) developing skills of self- and mutual-support; 3) developing the skills of providing supervision support and of building of a professional community Realized Results: The colleagues from different organizations and departments learned how wide and vast is the variety of psychological services they provide. The reviewed evidence showing that the network of psychological help provides a good basis for interagency collaboration in providing services for different categories of need in the population. Training Event Name: 4. “Making training programs for teenagers and young people” CSOC Component: Objective 2: Increase the professional capacity of the individuals serving the most vulnerable children and their families. Presenter(s): Amir Tagiev, an expert in youth engagement programs Number of Sessions: 2 Number of Participants: 22 Characteristics: Trainees were psychologists, social workers and youth leaders from different organizations of NO & KBR Summary: Training in developing and conducting trainings for youth and exchanging experience among trainees. Realized Results: The trainees learned how to set targets; make the trainings interesting for the group; being confident in conducting trainings; organizing the speech in a good order; develop mechanisms of guiding the group. The groups of trainees developed training projects. Training Event Name: 5. “Supervision support of specialists providing psychological, social support for the population of North Ossetia” CSOC Component: Objective 2: Increase the professional capacity of the individuals serving the most vulnerable children and their families. Presenter(s): Irina Ruklinskaya and Maria Soloveychik, Institute of Psychotherapy and Consulting “Harmony” Dates: January 30-February 1 and March 17-19, 2008 Number of Sessions: 2 26 Location(s): Republic of North Ossetia Summary: Supervision support services for the professionals working with vulnerable children and families in North Ossetia Expected Results: The general tasks of the two seminars were: creating and supporting the group atmosphere that encourages participants’ emotional involvement and friendly attitude towards each other; professional burn out prevention methods; further development of professionals skills in analyzing the accomplished work; discovering the areas of professional growth and the actual professional tasks. Realized Results: The participants had an opportunity to analyze cases from their practice and to identify the area of actual professional difficulties and problems. At the third seminar the priority task for the Ossetian group was working with symptoms of professional burn out. Therefore, special exercises were used for addressing professionals’ own resources, learning about themselves, and using the therapeutic resource of the group. The case analyses enabled the participants to address to their actual professional tasks. Training Event Name: 6. Community Council Meeting CSOC Component: Objective 3: Foster community development and citizen participation through involvement in program governance and oversight. Presenter(s): Irina Kishukova and Vasanta Romanova Dates: February 1, 2008 Number of Sessions: 1 Number of Participants: 22 Characteristics: Representatives from 12 organizations attended: 2 secondary schools; 1 Horse Sports school; 1 Child Settlement; 4 Social Rehabilitation Centers; 1 Psychological, Medical and Social Support Center; 1 NGO; 2 Charitable Foundations Location(s): Nalchik, KBR Summary: Presentations made on possibilities for mini-grant applications and on the results of the project activities in KBR for the year 2007. Expected Results: Discussion of issues related to the role of the Community Council in working out the Project Strategy, the Project Activities and sharing and make transparent the project activities Realized Results: Discussion included: the framework of project implementation, which raised interest in opportunities of joining Keystone’s 27 project; Amount of vocational training for young people remains to be a serious impediment for socializing of vulnerable children in KBR; Within the new IREX project, new possibilities appear for including children from vulnerable groups into vocational education classes; Serious educational problems (vocational and of individual programs; Children from vulnerable groups need a special meeting and a focused discussion with further appeal to the KBR government and KBR President; Parents of children with disabilities need more support. Training Event Name: 7. “Supervision support of specialists providing psychological, social support for the population of North Ossetia” CSOC Component: Objective 2: Increase the professional capacity of the individuals serving the most vulnerable children and their families. Presenter(s): Irina Ruklinskaya and Maria Soloveychik, Institute of Psychotherapy and Consulting “Harmony” Dates: February 3-6 and March 11-14, 2008 Number of Sessions: 2 Number of Participants: 23 Characteristics: 8 participants represented state educational organizations, 8 trainees were from rehabilitation centers for children, 5 psychologists – from Penal Executive Inspection System of KBR, 1 representative – from a charitable organization, and 2 psychologists – from Internal Affairs authorities of KBR. All participants were women. Location(s): KBR Summary: Supervision support services for the professionals working with vulnerable children and families in North Ossetia Expected Results: The general tasks of the two seminars were: creating and supporting the group atmosphere that encourages participants’ emotional involvement and friendly attitude towards each other; professional burn out prevention methods; further development of professionals skills in analyzing the accomplished work; discovering the areas of professional growth and the actual professional tasks. Realized Results: The trainers focused their attention on methods of self-learning, on training the skills of monitoring and understanding one’s own reactions important both for burn out prevention as well as for professional growth of the specialists. 28 Training Event Name: 8. “How to build an effective interaction between governmental and nongovernmental organizations in order to solve the problems of children and adults with developmental disorders” CSOC Component: Objective 2: Increase the professional capacity of the individuals serving the most vulnerable children and their families. Presenter(s): Mariana Sokolova, Handicap International; Juriy Katz, ARDI “Svet”; and Juriy Kuznezov, “We together” Dates: March 2008 Characteristics: Among participants of the seminar there were representatives of governmental organizations and NGOs, as well as parents of children with disabilities. Location(s): North Ossetia and KBR Realized Results: Through the discussion and videos used at the seminar the participants learned the mechanisms of attracting attention of different levels of society to the problems of children with disabilities and families. Training Event Name: 9. “Psychological and pedagogical assistance to preschool age children with developmental problems”; “Correction of speech disorders and of other superior mental functions of children” CSOC Component: Objective 2: Increase the professional capacity of the individuals serving the most vulnerable children and their families. Presenter(s): Natalia Sokolova and Elena Kutepova, Moscow Institute of Open Education Dates: March 2-8 and March 22-29, 2008 Number of Sessions: 2 Location(s): KBR Summary: Natalia Sokolova’s program: 1) Main criteria of diagnostics of the developmental delinquency, making psychological and pedagogical professional conclusions; 2) Content and methods of correctional and educational work at preschools of compensatory type. Characteristic of the program for children of preschool age with intellectual deficiency; 3) Organization and content of individual pedagogical itineraries aimed at correctional and developmental support provided for children with severe structural disorders; 4) Organizing of a stimulating intellectual development child activity for the preschool age children with intellectual deficiency; 5) Role of the family in pedagogical support of a child with developmental problems at his/her different stages of the 29 childhood. Consulting the parents regarding psychological and pedagogical diagnostic aspects of their children. Elena Kutepova’s program focused on improving professional competency of specialists of KBR on the following aspects: 1) Principles of the contemporary system of special education; 2) Psychological and pedagogical characteristics of children with diverse speech disorders; 3) General aspects of the theory and practice of speech therapy and psychology; 4) Methods of comprehensive diagnostics; 5) Methods and techniques of psycho-correctional speech therapy. Training Event Name: 10. “Otradnoye” program for specialists working with multi￾crisis families and their children CSOC Component: Objective 2: Increase the professional capacity of the individuals serving the most vulnerable children and their families. Presenter(s): Vasanta Romanova and Madina Sonova at the first seminar and Vasanta Romanova and Mikhail Ponomarev at the second and third seminars Dates: January-March 2008 Number of Sessions: 3 Number of Participants: 42 Characteristics: Psychologists, specialists on social work, educators, pedagogues from 14 different governmental organizations (under different executive powers of KBR) and non-governmental organizations, and from different regions of KBR Location(s): KBR Realized Results: The seminar gave tools for involving the nearest surroundings of a disadvantaged child and his family into the process of overcoming difficulties, tools for increasing the existing resources of the surroundings that are needed to solve the problems of a family, and tools for the prevention of crisis in a family. Training Event Name: 11. “Supervision support of specialists providing psychological, social support for the population of North Ossetia” CSOC Component: Objective 2: Increase the professional capacity of the individuals serving the most vulnerable children and their families. Presenter(s): Irina Ruklinskaya and Maria Soloveychik, Institute of Psychotherapy and Consulting “Harmony” Dates: April 9-11, 2008 Number of Sessions: 1 Location(s): Republic of North Ossetia 30 Summary: The general tasks were: summarizing and systematization of the knowledge and experience learned during the whole course about supervision and professional burn-out prevention. Realized Results: The students could demonstrate and receive feedback from trainers on their own supervision skills. The trainees also analyzed their own maps of self-professional and individual support and made the further steps of their improvement. For the majority of trainees the concept of supervision was new, getting the practical skills of applying it in practice. Training Event Name: 12. “Supervision support of specialists providing psychological, social support for the population of North Ossetia” CSOC Component: Objective 2: Increase the professional capacity of the individuals serving the most vulnerable children and their families. Presenter(s): Irina Ruklinskaya and Maria Soloveychik, Institute of Psychotherapy and Consulting “Harmony” Dates: April 14-17, 2008 Number of Sessions: 1 Location(s): KBR Summary: The general tasks were: summarizing and systematization of the knowledge and experience learned during the whole course about supervision and professional burn-out prevention. Realized Results: The students could demonstrate and receive feedback from trainers on their own supervision skills. There was a decision made that the supervision support program will be included into the work of resource center of Institute of Professional Growth of the System of Education and that a supervision group will be created at the level of City Department. Training Event Name: 13. “Basics of diagnostic and correction of disorders of Autism spectrum” CSOC Component: Objective 2: Increase the professional capacity of the individuals serving the most vulnerable children and their families. Presenter(s): Sergey Morozov Dates: May 16-20, 2008 Number of Sessions: 1 (32 total hours) Number of Participants: 52 31 Location(s): KBR Realized Results: During the course, 17 consultations for 12 families were conducted with recommendations for parents following. Training Event Name: 14. “Otradnoye” program for specialists working with multi￾crisis families and their children CSOC Component: Objective 2: Increase the professional capacity of the individuals serving the most vulnerable children and their families. Presenter(s): Vasanta Romanova and Madina Sonova at the first seminar and Vasanta Romanova and Mikhail Ponomarev at the second and third seminars Dates: May and June 2008 Number of Sessions: 2 (3 days each) Number of Participants: 42 Characteristics: Psychologists, specialists on social work, educators, pedagogues from 14 different governmental organizations (under different executive powers of KBR) and non-governmental organizations, and from different regions of KBR Location(s): KBR Summary: Network therapy training Realized Results: Thirty-eight (38) trainees (8 teams) among 42 made successful presentations of their case work in using network therapy at their organizations, demonstrated knowledge at examination tests, and received diplomas. Training Event Name: 15. Conference in Nalchik: “Improving the system of educational and social services for children and adolescents with special educational needs and for their families” CSOC Component: Objective 3: Foster community development and citizen participation through involvement in program governance and oversight. Presenter(s): Professors and PhDs in Psychology and Pedagogy, Republic Center of Rehabilitation of Children with special needs “Phoenix” and Orphanage for children with severe developmental disorders “Laska” Dates: May 22-23, 2008 Number of Sessions: 1 Number of Participants: 72 Characteristics: Specialized educational organizations, rehabilitation centers for children and orphanages from Kabardino-Balkaria and North 32 Ossetia, as well as from Center of Therapeutic Pedagogy of Pskov, Moscow Institute of Open Education, International Association “Handicap International”, Kabardino-Balkarian Charitable Foundation of Zaramuk Kardangushev, Youth Non-profit organization “Pure Hearts,” and non-profit organization of parents “Nadezhda” Location(s): Nalchik, KBR Summary: A scientific and practical conference was held over a two-day period to discuss current issues with educational and social services for disabled children and their families and improvements Expected Results: To mobilize the efforts, summarize the best experience and knowledge, and to generate new proposals to be realized in building a comprehensive system of care for children and youth with special needs and their families in Kabardino-Balkaria. Realized Results: Thirty-eight (38) trainees (8 teams) among 42 made successful presentations of their case work in using network therapy at their organizations, demonstrated knowledge at examination tests, and received diplomas. The conference’s outcome was the Resolution prepared together with the conference participants and moderators and the CSOC project experts Irina Bgazhnokova and Mariana Sokolova (See Attachment #5). The Resolution was sent to the the Ministries of Education and Science, of Labor and Social Development and of Health of the Republic of Kabardino￾Balkaria. 33 ANNEX B CSOC PROJECT SUMMARY The USAID-funded Comprehensive Systems of Care (CSOC) Project was implemented in the Russian North Caucasus by Keystone Service Systems, Inc. through a donor agreement between Keystone Human Services International (KHSI of Harrisburg, Pennsylvania) and the Keystone Foundation for Children and Families (KFCF of Moscow, Russia). The CSOC Project began on June 29, 2006 and ended on December 31, 2008. Through the provision of targeted facilities and community-based programs, services were provided to address the immediate psychological, educational, and health needs of vulnerable children and families. A Keystone Center has been operational in the community of Beslan in North Ossetia since January of 2007. A Keystone Center began operations in August 2007 in the town of Nalchik, the capital of Kabardino-Balkaria, the Republic adjacent in the west to that of North Ossetia. The CSOC Project focused on four main objectives and activities: Objective 1: Address the immediate psychological, educational, and health needs of the most vulnerable children and their families. ο Expansion of the services provided by the Children's Rehabilitation Room (Center) at the Polyclinic Hospital in Beslan (employment agreements in with the Project Director for the Medical Psychological Services and 5 Psychologists) ο Support for a “Mountain House” (Family Retreat) Rehabilitation Center ο Support for Youth Forums Objective 2: Increase the professional capacity of the individuals serving these families. ο Agreement to develop and provide Intensive In-home Family Therapy workshop to include participants from all aspects of the service delivery system in North Ossetia, including Social Workers, Psychologists, Psychiatrists, Educational Specialists, and other disciplines involved in multidisciplinary services to children and families ο Planning for Fundraising and Development Seminar in April 2007 provided by Ann Moffitt, Vice President for Development for Keystone and a Certified Fundraising Executive by the International Fundraising Professionals Association. Objective 3: Foster community development and citizen participation through involvement in program governance and oversight. ο Continued Development of community resources including coordination of actives with UNICEF and CAF 34 Objective 4: Initiate and maintain a comprehensive project monitoring and performance measurement strategy. ο Completion of the Performance Monitoring and Evaluation Plan SUMMARY OF ACTIVITIES Objective 1 Chief of Party and Keystone leadership established and expanded a community needs assessment. Based on initial feedback received from Russian (national and local) experts in the fields of mental health, psychology, psychiatry, social issues, and education, during the first series of organizing meetings in July 2006, specific and targeted subcontracts were solicited to help fill the service gaps identified. Various Governmental, Non-governmental, and Community organizations were engaged in the process of identifying and assessing the immediate needs of the community’s most vulnerable children. One of the major project deliverables was a set of resources in the form of a CSOC Implementation Toolkit to be available by the end of the project’s second year. This manual included a full set of policies, procedures, budgets, and training materials in Russian for use in program replication. The Chief of Party worked each month on the collection and recording of relevant information to contribute to the Toolkit. In addition to the materials listed above, she tracked information for a section on lessons learned and challenges overcome. A “five-pronged” approach was adopted to support the program, three of which relate to Objective 1: Expansion of the services provided by the Children’s Rehabilitation Room (Center) at the Polyclinic Hospital in Beslan: The CSOC program’s medical/psychological services were initiated at the Polyclinic in Beslan in the first quarter of the project. Several services at the Center were then expanded, including initial diagnostic consultation and psychological counseling. Other services introduced during CSOC implementation include: educational courses for parents, early intervention and working with children of early age (0-3 years), and working with pregnant women (prevention). Support for a Family Retreat Rehabilitation Center: Meetings were initially held to determine operational costs at the “Algus” and “Uspenie” Foundations. Retreats began in October 2006, with ten families participating through December. In 2007, trips to the Algus Center were conducted with three families in January and four families on in February. The purpose of these retreat outings was to provide a safe, comfortable, and relaxing setting for families and their children to participate in rehabilitation and parent education activities under the direction of our project specialists. Retreats continued on a schedule of one every two weeks for the remained of the Project. The use of Hippotherapy was introduced early in the Project and accelerated to 60-70 patients per month in 2007 as the method proved to be successful in helping rehabilitate disabled children. Collaboration efforts with local schools expanded treatment to include a wider range of children. 35 CSOC initiated a number of other activities for children during the two-year implementation period. Ten children from Beslan, together with 3 psychologists (2 from Beslan Policlinic, 1 from Beslan school) visited Moscow to attend the Christmas fir-tree performance at the Cathedral of Christ the Saviour. A cultural program was also developed for the group with the active participation of volunteers (students of MSPPU – Moscow State Psychology and Pedagogy University). Support for Youth Forums: Encouraging more youth participation in the community began with an assessment of local Ministries and their level of coordination. The assessment found a very low level of collaboration, creating an initial obstacle for this sub-component of Objective 1. The CSOC team worked with the FICE International Youth Representatives from Bosnia, the Committee for Youth of the Ministry of Nationalities, and others in the development and implementation of Youth Forums throughout the duration of the project. In November 2007 in KBR a Youth forum was held under the slogan “Peace in the North Caucasus through creative collaboration of young people”. Fifteen teenagers from “Erasika” Youth Center from North Ossetia attended the Youth Forum in KBR, which was organized with support of the Committee on Youth Affairs of KBR. Teenagers from 10 schools, members of different clubs and TV-journalism studios, gathered together to talk about problems and show some of their artistic works. The Youth Forum gathered about 250 participants many of whom gave interviews to the regional radio and TV. In June 2008, Children’s Day Action (“Flowers of Life”) was held on in Beslan’s Park. About 30 volunteers (students of schools and higher education institutions) participated in the holiday for more than 650 children of Beslan. Letters to the city Mayor to coordinate the event were sent and agreed in advance. Many concourses and attractions for children, including flower plantings were arranged by the initiative group of Keystone staff at Beslan Rehab Center. Objective 2 Development of an Education/Training Program for Social Work Professionals in North Ossetia: This constitutes the fourth item of the five-pronged approach. Professional training and technical assistance supporting rehabilitation professionals in the North Caucasus is a key component of this project. CSOC designed and implemented a series of training and educational events, such as Intensive In-Home Family Therapy Workshops, to improve gaps in services and training. These gaps were identified in the initial activities of the “Social Work with Children and Families in a Network Perspective” project already underway in North Ossetia through the Swedish International Development Cooperation Agency (SIDA) and with direct support from Keystone. Social workers, psychologists, psychiatrists, educational professionals and other disciplines were trained in best practices for Intensive In-home Family Therapy. The last training workshops took place in September 2007. Thirty-one individuals were certified in the field as a result of the training. In the spring of 2007, Keystone received formal written notification that its grant proposal to the Open World Leadership Center (OWLC) was successful. In October 2007, the grant sent eight delegates from the region to participate in a US exchange training program. 36 Through this initiative, Keystone leveraged United States Government (Library of Congress) dollars to provide professional development for individuals from North Ossetia and KBR with the goal of providing a foundation of training and development for several of the partners and collaborators in Russia. A burn-out prevention retreat trip for a group of specialists was conducted on the 28th of September 2007 at the Tsey Mountain Resort. The goal was to provide necessary therapy for the psychologists at the policlinic and, by including specialists working on the bases of other institutions, to provide a comfortable setting for establishing closer professional relationships, friendly affiliations, and building an extended but cohesive professional team. In North Ossetia, a high need for supervision support services for the professionals working with vulnerable children and families in North Ossetia was identified. Keystone used the expertise of the Nongovernmental Educational Organization of Supplementary Education “Institute of Psychotherapy and Consulting “Harmony” to conduct the training. In late 2007 and early 2008, specialists participated in training on the Harmony program in North Ossetia and KBR. The general tasks of the two seminars were: creating and supporting the group atmosphere that encourages participants’ emotional involvement and friendly attitude towards each other; professional burn out prevention methods; further development of professionals skills in analyzing the accomplished work; discovering the areas of professional growth and the actual professional tasks. The training continued through April 2008. To support the specialists working with teenagers, a workshop on “Making training programs for teenagers and young people” was conducted by an expert in youth engagement programs. The 22 trainees were psychologists, social workers and youth leaders from different organizations of NO & KBR. The two-day workshop was targeted at: 1) training in developing and conducting trainings for youth and 2) exchanging experience among trainees. In KBR, courses were conducted by the Moscow Open State Institute and designed by Irina Bgazhnokova. Themes included: special education (correction pedagogics) and special psychology, differential diagnostics, early childhood evaluation, diagnostics of developmental problems, family involvement in educational support to children with developmental problems, development of children with Cerebral Paralysis, correction of speech disorders. Training was completed in May 2008. The “Otradnoye” program for specialists working with multi-crisis families and their children began in earl 2007. After the orientation meeting conducted by Tatiana Barsukova with the heads of state organizations working with disadvantaged children and families, the professional competency improvement course on Network Therapy began in Kabardino-Balkaria by the certified network therapy trainers from Otradnoye Center (Moscow). Five seminars were completed by April 2008. Thirty-eight trainees (eight teams) among 42 made successful presentations of their case work in using network therapy at their organizations. They demonstrated competent knowledge at examination tests, and received diplomas. In April 2008, a two-day trip to the Association of Parents of Children with Disabilities “Svet” in Vladimir was organized for 4 parents of children with special needs and representatives 37 of local NGOs from NC (2 from North Ossetia + 2 from KBR). This trip for the delegation from NC was a good experience of learning a variety of programs for children and young people with special needs built on the parents’ initiatives. Objective 3 Implementation of a Fundraisers Program: The fifth element of the five-pronged approach falls under Objective 3. During the first quarter of the USAID project CSOC laid groundwork for the establishment of program relationships across multiple government and non￾government service systems. To that end, Keystone advocated successfully to have North Ossetia added as a 12th region to a SIDA project that supports development of social services to socially vulnerable children and their families. In April 2007, Keystone conducted a fundraising and development seminar aimed at providing tools for the various NGO’s and Governmental entities for improving the sustainability of their projects. A total of 72 participants attended the event. Project meetings took place with CAF, UNICEF, & SIDA representatives with the aim of continuing to build relationships and coordinate implementation activities. A related goal, which resulted from a meeting with UNICEF, was the formation of a coordinating body. From this, plans for the Beslan Collaborative began. The Collaboration acted as an additional way to meet the program objective of fostering community development and citizen participation through involvement in programmatic governance and oversight. A Community Council Meeting (CCM) held on July 20, 2007, gathered 13 participants from KBR. Participants were represented from 12 Governmental and non-Governmental organizations. The starting goals for the first CCM were achieved: o Getting acquaintance, discussion and concordance of mutual expectations among the representatives of Kabardino-Balkarian Community Council, representatives of “Keystone” Foundation (Moscow), and the representative of USAID (an independent project expert on North Caucasus project) o Discussion of the needs and resources of each Community Council participant o Strengthening the links between the representatives of different organizations presented A second CCM was conducted in February 2008 that included 22 participants. This CCM focused on the amount of vocational training for young people in KBR, new possibilities for including children from vulnerable groups into vocational education classes, and serious educational problems (vocational and of individual programs) for children from vulnerable groups. In September 2007, a CSOC Project Request for Proposals (RFP) was issued and delivered electronically with the intent of soliciting applications for mini-grants from training participants of the first fundraising seminar. Proposals were solicited from a number of these partners to establish services not currently provided but which have been identified as important components of a comprehensive system of care through the needs assessment/gap analysis process. 38 In the spring of 2008, the following mini-grant projects were initiated: 1) Trainings for teenagers at Algus Retreat Center (NO). The mini-grant project of group trainings for teenagers was started in June. 2) Mozdok project (NO). The rehabilitation room for psychological services in Mozdok Agency of Social Protection has been equipped. The Mozdok region of North Ossetia is at the border of the Republic of Chechnya, and terrorist acts in the area make the territory vulnerable. For the whole Mozdok region, this is only place created to address the needs in psychological services of the population of Mozdok region (91,500 inhabitants, 25,000 of which are children). 3) Project prepared by Ratmir Karov (KBR) “Let’s meet summer happily!” - The republican soccer and badminton tournament among all boarding schools of KBR. Children from one Social Rehabilitation Center and eight boarding schools (in total, 225 children) participated in the tournament, which took place in May. To mobilize the efforts, summarize the best experience and knowledge, and to generate new proposals to be realized in building a comprehensive system of care for children and youth with special needs and their families in KBR, a scientific and practical conference “Improving the system of educational and social services for children and adolescents with special educational needs and for their families” was held on May 22-23, 2008, in Nalchik, KBR. Representatives from specialized educational organizations, rehabilitation centers for children and orphanages from KBR and North Ossetia, as well as from Center of Therapeutic Pedagogy of Pskov, Moscow Institute of Open Education, International Association “Handicap International”, Kabardino-Balkarian Charitable Foundation of Zaramuk Kardangushev, Youth Non-profit organization “Pure Hearts,” and non-profit organization of parents “Nadezhda” attended the conference. At the end of the conference, 52 specialists received diplomas from MIOE in a formal atmosphere. The conference’s outcome was a Resolution prepared by conference participants, moderators, and the CSOC project experts Irina Bgazhnokova and Mariana Sokolova. The Resolution was sent to the Legislative Assembly, the Ministries of Education and Science, of Labor and Social Development and of Health of the Republic of Kabardino-Balkaria. The festival “Hearts of Young Muscovites are Open for Everybody” was held in Moscow on September 6, 2008. Keystone partnership organization – NGO “Charitable Foundation on Social Rehabilitation Assistance for Children and their Families “Otradnoye” organized the festival together with Department of Social Protection of Population of the North East Administrative District of Moscow and Ministry of Labor and Social Development of NO￾Alania. Since 2006 it has been a tradition to invite children from North Ossetia to a Moscow child festival dedicated to creativity, strengthening of spiritual and cultural traditions of Russian and Ossetian people, and promotion of tolerance and solidarity between nations. A group of 17 children from Social Rehabilitation Center “Kind Heart” from Vladikavkaz (NO) participated. The festival program included a concert and creativity workshops. Objective 4 39 An M&E plan was refined and implemented by third-party subcontractor, MetaMetrics, who provided technical assistance in the development and implementation of the project’s formal M&E Plan. Leo Surla of MetaMetrics traveled to Russia to conduct a program evaluation visit in North Ossetia from February 17-24, 2007 where Mr. Surla provided training to the CSOC team on the collection of required performance and evaluation data. Mr. Surla also made an M&E presentation to the Board of KFCF on February 25. Mr. Surla’s trip concluded with a meeting with USAID CTO, Olga Kulikova (along with Roger Burns, Keystone’s CFO, and Maria Kalitina) to assist MetaMetrics in the clarity of the USAID deliverables in the Independent M&E Activities. Mr. Surla returned to the region in July of 2007 to develop the organizational structure and procedures to support the collection and analysis of results and program descriptive information as related to Work Plan activities. Information was collected from the two Centers and the First Annual M&E Report was drafted in August of 2007. In September of 2008, the MetaMetrics M&E Specialist visited and interviewed with CSOC Project staff and consultants to collect statistics and other information related to CSOC Project performance. A evaluation report was completed in Washington in October 2008 and was subsequently revised upon receipt of the final quarterly reports. 40 ANNEX C LIST OF CSOC PROJECT ORGANIZATIONS Non-North Caucasus Organizations 1. "Otradnoye" NGO 2. "The Way" NGO 3. “Maria’s Children” center 4. “Our Life” Center for social-psychological rehabilitation, Moscow 5. American Chamber of Commerce in Russia 6. Center for Ecological Education, Obninsk 7. Embassy of the United States of America, Moscow, Russia 8. FICE/Russia 9. HARMONIA Institute of Psychotherapy and Counseling in St.Petersburg 10. Moscow Institute of Open Education 11. Province Di Vercelli, Italy 12. Russian State Duma 13. Science center of cardiology of Russian Academy of Medical Sciences, Moscow 14. SIDA Program 15. Social Rehabilitation Center for Minors “Otradnoye”, Moscow 16. U.S. Ambassador to Russia, U.S. Embassy 17. UNICEF, Russia 18. Union of Social Workers of the Russian Federation 19. Vallenberg Institute of special pedagogies and psychology, St.Petersburg 20. Interregional assotiation of parents of children with disabilities "ARDISVET", Vladimir 21. Handicap International, Moscow 22. USAID North Ossetia 23. “Caritas” program 24. “Doverie” Rehabilitation Center 25. “Thanks to everyone…” emotional rehabilitation program 26. Administration of Beslan 27. Administration of Iraf Region of the Republic of North Ossetia / Alania 28. Binonta Center Beslan UNICEF/Ministry of Education 29. Centre of Social Services of Lonely and Elderly People and Disabled People of Mozdok Region 30. Charity Aid Foundation Beslan 31. Children's house "Khurytyn", Ministry of Education 32. Children's house "Victoria", Ministry of Education 33. Committee on Minors Affairs and Protection of their Rights of Digorskiy region 41 34. Committee on Minors Affairs and Protection of their Rights of Iraphskiy region 35. Committee on Minors Affairs and Protection of their Rights of Mozdok region 36. Committee on Minors Affairs and Protection of their Rights of Prigorodniy region 37. Comprehensive Centre of Public Social Services of Iristonskiy Municipality District 38. Comprehensive Centre of Public Social Services of North-West Municipality District 39. Comprehensive Centre of Public Social Services of Promyshlenniy Municipality District 40. Comprehensive Centre of Public Social Services of Zaterechniy Municipality District 41. Department of Public Social Protection of Alagir region 42. Department of Public Social Protection of Digorskiy region 43. Department of Public Social Protection of Pravoberezhniy region 44. Department of Public Social Protection of Prigorodniy region 45. Department of Work with Minors of Didorskiy region 46. Education administration of Pravoberezhniy region of NO-Alania 47. Foundation “USPENIE”, North Ossetia (NGO) 48. Physical Training Center, Vladikavkaz 49. House of Children's Creativity, Beslan 50. Minister of Labor and Social Development of Ossetia 51. Minister of Youth, Physical Development and Sports 52. Ministry of Education and Science in North Ossetia/Alania 53. Ministry of Public Health on Child Care 54. NGO Elementary Orthodox School “Pokrov” 55. Orphanage / Internat for Children with Mental Retardation "Laska" 56. Ossetian Union of Youth 57. Pravoberezhniy District Central Clinical Hospital in Beslan 58. Principal, School Number 1 Vladikavkaz, Russia 59. Red Cross Republican Department 60. Rehabilitation Center Aglus (NGO) /”Verblyozonok” (“Little Camel”) 61. Rehabilitation Center at Beslan Clinical Hospital 62. Republic Centre of Social Rehabilitation for Minors "Dobroye Serdze" (“Kind Heart”) 63. Republic Rehabilitation Centre for Disabled Children “Phoenix” 64. Republican Center for Prevention of Alcohol and Drug Abuse and Delinquencies Among Young People 65. Republican Center of Psychological, Educational, Medical and Social Support, Vladikavkaz 66. Republican Psychiatric Hospital Dispensary Department 67. Rosinka preschool, NO-Alania 68. School N2 of Ministry of Education, Beslan 69. School N8, Beslan, principal 70. School of Pravoberezhniy District, v. Brut, Ministry of Education 71. School on Komintern street, Beslan, principal 72. St. Mary’s Orthodox Church 73. State Boarding School, NO-Alania 74. State University of North Ossetia named of K.L. Khetagurov 75. Style Art Studio 76. Territorial Centre of Social Help for Families and Children of Alagir 77. Unicef Social-Psychological Rehabilitation Program in the North Caucasus 78. Vladikavkaz Civilization Institute 42 79. Preschool N10 "RADUGA", Beslan 80. Regional organization of disabled people "Independent life", Vladikavkaz Kabardino-Balkaria 81. Charitable Foundation "Center of Peace Building and Social Development", KBR 82. Charitable organization “Solidarity”, KBR 83. Children Club “AYAZ”. The Teenagers Historic and Patriotic Club at the Secondary School N19, KBR 84. Horse Sport School of Nalchik 85. Institute of the Problems of Youth, KBR 86. Interregional Community Movement “Pure hearts”, KBR 87. Kabardino-Balkar Zaramuk Kardangushev Humanitarian Foundation 88. Kabardino-Balkarian Regional Section of the All-Russian Volunteer Organization “Sport Russia” 89. Medical faculty, University of KBR 90. Ministry of Education and Science, KBR 91. Ministry of Health, KBR 92. Ministry of Labor and Social Development, KBR 93. Municipal Child Preschool Education of Compensatory Type N#25, KBR 94. National museum of the Republic of Kabardino-Balkaria 95. Newspaper “Goryanka” 96. Organization “Bivitex” (a preparation from curable dirt of Tambukan Lake) 97. Preschool #18, KBR 98. Regional Health Center for Children “RADUGA”, KBR 99. Republic Culture-Health Club of People with disabilities “Impulse”, KBR 100. Republic State Education Institution Boarding School, Nalchik, KBR 101. Republican Center for the Underaged "NAMYS", KBR 102. Republican Center of Psycho-Medical-Social Support, KBR 103. Republican Center of Student Scientific-technical Creativity, KBR 104. School #4, v. Prokhladniy, KBR 105. State Committee of the Republic of Kabardino-Balkaria on Physical Education and Sport 106. State Educational Institute “Specialized Secondary Education Boarding School N1 of the VIII type”, v. Zayukovo, KBR 107. Tecvando Federation, KBR 108. Youth Tolerance Programs 109. Agriculture Academy 110. Bording school #7, Tyrnauz 111. Department of Education, KBR 112. Department of Federal Service of Executive Punishment of Russia in KBR 113. Inter-district Penal executive inspection #1, KBR 114. Gymnasium #29, Nalchik 115. Ministry of Internal Affairs, KBR 116. Newspaper "KB Truth" 117. Organization of parents of children with disabilities "Nadezhda", Nalchik 43 118. Penal colony 3 119. Penal colony 5 120. Republic Center of Social Help for Families and Children, Nartkala 121. Sanatorium-forest school #1, Nalchik 122. Sanatorium-forest school #2 123. School #19, correction classes, KBR 124. School #4, correction classes, Nalchik 125. School #50 126. Social Rehabilitation Center for Minors, Cherekskiy district 127. Social Rehabilitation Center for Minors, v. Kuba, Baksanskiy District 128. State Traffic Safety Inspectorate 129. TV channel "Edelveys" 44 ANNEX D REVIEW OF MATERIALS ON HIPPOTHERAPY Denise Oki, MetaMetrics Research Associate conducted a web search for information on Hippotherapy. The following is drawn from that research. WebMD provides a short introduction of Hippotherapy by showing a video clip of a boy learning to use gain control of his legs from the natural pelvic movement from riding a highly trained horse. http://www.webmd.com/video/hippotherapy-special-needs NARHA Strides magazine, April 1997 (Vol. 3, No. 2) http://www.cpparent.org/hippotherapy/articles/introduction.htm In classic hippotherapy, it is purely the horse's movement that influences the client. The client passively interacts with, and responds to, the horse's movement. The primary focus of classic hippotherapy is the rider's posture and movement responses. However, other effects may occur in respiration, cognition and speech production. For example, if the treating therapist is a PT whose goal is to strengthen the trunk muscles and positively affect the client's posture, respiration and speech will improve due to the increased trunk strength. Hippotherapy, on the other hand, is a treatment approach that uses the movement of the horse with the addition of the treatment principles that apply to the particular profession of the therapist providing the service. The unique combination of the horse, the horse's movement and a non-clinical environment produces an extraordinary effort on all the systems of the body. Therefore, although hippotherapy is frequently used to achieve physical goals, it also affects psychological, cognitive, social, behavioral and communication outcomes. Hippotherapy is truly a multidisciplinary form of treatment and can be applied by a PT, OT, SLP/CCC, psychologist or psychotherapist. It is a treatment approach that uses activities on the horse that are meaningful to the client and specifically address the individual's goals. It does not teach specific skills associated with being on a horse -- rather, it provides a foundation of improved neuromotor function and sensory processing that can be generalized to a wide variety of activities outside treatment. forward to facing backward and then to quadruped (on all fours). In addition to the facilitation of automatic postural responses and stimulation of trunk muscles, there are increases in sensory input to the following systems of the body: * Vestibular -- because the client is facing backward while the horse is moving forward. * Proprioceptive -- heavy touch pressure through the hip, knee, wrist, elbow and shoulder joints in the quadruped position. * Tactile -- touching the soft warm coat of the horse. 45 * Cognitive -- higher level motor planning skills required to execute the transition. * Motor (physical) -- stability of hips and pelvis required to maintain position while reaching forward with one hand. This article provides an explanation of the reasons and effects of hippotherapy but does not establish concrete, scientific proof of the effects. STUDIES: Aetna provided a complete list and description of tests performed to evaluate the effects of hippotherapy. http://www.aetna.com/cpb/medical/data/100_199/0151.html Using a repeated-measured design, Bertoti (1988) assessed postural changes in 11 children (4 girls and 7 boys, aged 28 to 114 months) with spastic CP after participation in a 10- week hippotherapy program (1-hour session, 2 times per week). Evaluation of posture was carried out 3 times by three pediatric physical therapists -- (i) pretest-1 followed by a 10-week period of no riding, (ii) pretest-2 followed by a 10-week therapeutic riding program, and (iii) post-test. A composite score for each test period was computed for each patient, and a median score was calculated for the entire group at each test period. A statistical difference was observed among the 3 test periods with significant improvement occurring during the period of hippotherapy. Subjective clinical improvements such as reduced hypertonicity, as well as improved weight-bearing and functional balance skills were reported by parents and referring physical therapists. These findings represented the first objective report that hippotherapy may have beneficial effect on the posture of children with spastic CP. However, the author concluded that further investigation is needed to isolate additional variables such as range of motion, balance, weight shift, and strength, and to evaluate the effects of hippotherapy on different disabilities. Bertoti DB. Effect of therapeutic horseback riding on posture in children with cerebral palsy. Phys Ther. 1988;68(10):1505-1512. In an article on hippotherapy, Tuttle (1987) stated that research on the effect and application of the various forms of therapeutic horseback riding is needed to refine program planning, and to support funding and third party reimbursement. Furthermore, a workshop on “The Health Benefits of Pets” sponsored by the National Institutes of Health concluded that “solid data on the success of therapeutic riding is limited. ... Future research is indicated to compare the efficacy of therapeutic riding with other clinical treatment procedures that do not involve the horse and to validate dramatic clinical observations” (NIH, 1983). Additionally, in an article published in the Journal of American Veterinarian Medicine Association, Potter and colleagues (1994) stated that “Lack of scientific documentation of the benefits of therapeutic riding is a major obstacle that must be overcome. ... Research is critically needed in all aspects of therapeutic riding”. Tuttle JI. The horse as member of a therapeutic team. Rehab Nursing. 1987;12 (60):334-335. 46 In a single-subject experimental design study (n = 11), Hammer and associates (2005) examined whether therapeutic riding (TR, Sweden), also known as hippotherapy (HT, United States) may affect balance, gait, spasticity, functional strength, coordination, pain, self-rated level of muscle tension (SRLMT), activities of daily living (ADL), and health-related quality of life in patients with multiple sclerosis (MS). The intervention comprised 10 weekly TR/HT sessions of 30 minutes each. The subjects were measured a maximum of 13 times. Physical tests were: the Berg balance scale, talking a figure of eight, the timed up and go test, 10-m walking, the modified Ashworth scale, the Index of Muscle Function, the Birgitta Lindmark motor assessment, part B, and individual measurements. Self-rated measures were: the visual analog scale for pain, a scale for SRLMT, the Patient-Specific Functional Scale for ADL, and the SF-36. Data were analyzed visually, semi-statistically and considering clinical significance. Results showed improvement for 10 subjects in one or more of the variables, particularly balance, and some improvements were also seen in pain, muscle tension, and ADL. Changes in SF￾36 were mostly positive, with an improvement in Role-Emotional seen in 8 patients. These investigators concluded that balance and Role-Emotional were the variables most often improved, but TR/HT appeared to benefit the subjects differently. Hammer A, Nilsagard Y, Forsberg A, et al. Evaluation of therapeutic riding (Sweden)/hippotherapy (United States). A single-subject experimental design study replicated in eleven patients with multiple sclerosis. Physiother Theory Pract. 2005;21(1):51- 77. Debuse et al (2005) noted that despite a substantial body of anecdotal and clinical evidence for its benefits, research evidence for hippotherapy is sparse. In a questionnaire survey, these researchers explored the views of physiotherapists and people with CP who use hippotherapy. This study was aimed to: (i) establish the pattern of hippotherapy practice in Germany and the U.K.; (ii) examine the perceived main effects of hippotherapy on people with CP in Germany and the U.K.; and (iii) investigate how these effects are being measured in both countries. The results highlighted considerable differences in how hippotherapy is practiced in the U.K. compared with in Germany. In spite of this, the study revealed agreement among respondents on the overall perceived effects of hippotherapy on individuals with CP, namely, the regulation of muscle tone, improvement of postural control and psychological benefits. The results also indicated scant use of outcome measures to evaluate these effects. Debuse D, Chandler C, Gibb C. An exploration of German and British physiotherapists' views on the effects of hippotherapy and their measurement. Physiother Theory Pract. 2005;21(4):219-242. Casady and Nichols-Larsen (2004) examined if hippotherapy has an effect on the general functional development of children with CP. The study employed a repeated-measures design with two pre-tests and two post-tests conducted 10 weeks apart using the Pediatric Evaluation of Disability Inventory (PEDI) and the Gross Motor Function Measure (GMFM) as outcome measures. A convenience sample of 10 children with CP participated whose ages were 2.3 to 6.8 years at baseline (mean +/- SD 4.1 +/- 1.7). Subjects received hippotherapy once weekly for 10 47 weeks between pre-test 2 and post-test 1. Test scores on the GMFM and PEDI were compared before and after hippotherapy. The authors concluded that results of this study suggest that hippotherapy has a positive effect on the functional motor performance of children with CP. Hippotherapy appears to be a viable treatment strategy for therapists with experience and training in this form of treatment and a means of improving functional outcomes in children with CP, although specific functional skills were not investigated. There are two main drawbacks with this study: (i) the GMFM scorers were not blinded to the order of test date and they were allowed to keep the scores sheets, which may have biased the scorers, and (ii) with the individualized approach to treatment, there is no protocol that would allow replication of this study. The authors stated that hippotherapy has the potential to be a valuable treatment strategy in treating children with CP. Future studies should use more homogeneous patient populations in terms of age and type of CP to ascertain precise areas of function affected most by hippotherapy. Casady RL, Nichols-Larsen DS. The effect of hippotherapy on ten children with cerebral palsy. Pediatr Phys Ther. 2004;16(3):165-172. In a review on the use of complementary and alternative therapies for the treatment of children with CP, Liptak (2005) noted that although studies of hippotherapy have shown beneficial effects on body structures and functioning, unanswered questions remain. For example, it is unclear which subgroups of children with CP would benefit the most, what "dose" or frequency of intervention is optimal. Liptak GS. Complementary and alternative therapies for cerebral palsy. Ment Retard Dev Disabil Res Rev. 2005;11(2):156-163. An assessment of the evidence for hippotherapy by the Institute for Clinical Effectiveness and Health Policy (Pichon Riviere, et al., 2006) concluded: "The efficacy of this therapy does not seem to have been sufficiently proven for any specific indication. Its recreational role and impact on the quality of life of these patients have not been sufficiently analyzed." Pichon Riviere A, Augustovski F, Alcaraz A, et al. Usefulness of hippotherapy [summary]. Report ITB No. 28. Buenos Aires, Argentina: Institute for Clinical Effectiveness and Health Policy (IECS); 2006. 48 ANNEX E BESLAN COUNSELING DATA 2007 Beslan Bagaeva Dzansolov Kallagova Kanukova Naskidaeva Plieva TOTAL Month Work type child adults fam child adults fam child adults fam child adults fam child adults fam child adults fam child adults Initial Visit 5 4 8 0 0 0 0 0 0 3 3 3 2 1 3 3 4 4 13 12 18 Repeated Visit 11 8 19 0 0 0 0 0 0 9 17 8 8 0 12 12 23 17 40 48 56 Outside Center Work 4 7 4 0 0 0 0 0 0 0 0 0 0 0 0 3 8 5 7 15 9 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Group Work 0 0 0 0 0 0 0 0 0 1 2 1 0 0 0 0 0 0 1 2 1 Meetings 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Jan Other work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Initial Visit 3 3 7 0 0 0 0 0 0 3 3 3 2 1 3 6 8 7 14 15 20 Repeated Visit 11 7 19 0 0 0 0 0 0 16 22 18 2 4 4 15 33 23 44 66 64 Outside Center Work 5 7 5 0 0 0 0 0 0 2 4 1 0 0 0 0 0 0 7 11 6 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Group Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Meetings 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Feb Other work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Initial Visit 5 3 10 0 0 0 0 0 0 5 3 4 1 0 1 4 3 4 15 9 19 Repeated Visit 8 9 19 0 0 0 0 0 0 19 30 20 7 3 10 20 33 19 54 75 68 Outside Center Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Group Work 9 4 5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 9 4 5 Meetings 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 Mar Other work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Initial Visit 3 1 4 0 0 0 0 0 0 1 1 3 4 2 5 0 0 0 8 4 12 Repeated Visit 9 8 20 0 0 0 0 0 0 11 20 17 5 8 8 16 27 13 41 63 58 Apr Outside Center Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 4 7 4 4 7 49 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Group Work 20 0 20 0 0 0 0 0 0 5 5 0 0 0 0 4 4 0 29 9 20 Meetings 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 Other work 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 Initial Visit 2 2 2 0 0 0 0 0 0 2 2 2 2 4 0 1 1 1 7 9 5 Repeated Visit 8 6 17 0 0 0 0 0 0 13 16 10 8 6 10 9 13 6 38 41 43 Outside Center Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Group Work 12 0 12 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 12 0 12 Meetings 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 May Other work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Initial Visit 5 4 6 0 0 0 2 3 2 5 5 5 0 0 0 6 10 11 18 22 24 Repeated Visit 9 3 15 0 0 0 11 18 8 16 26 19 2 1 4 8 12 6 46 60 52 Outside Center Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Group Work 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 1 0 1 Meetings 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 Jun Other work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Initial Visit 6 6 11 2 0 4 8 15 3 9 16 7 0 0 0 0 0 0 25 37 25 Repeated Visit 16 24 32 9 10 12 21 32 9 22 30 20 0 0 0 0 0 0 68 96 73 Outside Center Work 15 26 12 16 26 11 3 6 3 4 6 4 0 0 0 0 0 0 38 64 30 Preparation 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 2 0 Group Work 0 0 0 0 5 0 0 12 0 0 12 0 0 0 0 0 0 0 0 29 0 Meetings 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 Jul Other work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Initial Visit 0 0 0 0 0 0 0 0 0 1 0 1 1,5 0 1 0 0 0 2,5 0 2 Repeated Visit 5 2 6 0 0 0 8 9 4 11 8 9 26 19 20 0 0 0 50 38 39 Outside Center Work 0 0 0 6 9 3 0 0 0 0 0 0 0 0 0 0 0 0 6 9 3 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Group Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Meetings 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Aug Other work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Initial Visit 0 0 0 0 0 0 6 7 6 5 4 5 1 1 1 0 0 0 12 12 12 Sep Repeated 0 0 0 0 0 0 22 31 16 33 34 28 0 0 0 0 0 0 55 65 42 50 Visit Outside Center Work 0 0 0 0 0 0 7 13 7 6 13 6 3 13 6 0 0 0 16 39 19 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Group Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Meetings 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 1 1 1 Other work 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 2 2 2 Initial Visit 0 0 0 0 0 0 2 2 3 1 1 2 1 5 8 0 0 0 4 8 13 Repeated Visit 0 0 0 0 0 0 20 24 10 24 23 22 0 15 26 0 0 0 44 62 58 Outside Center Work 0 0 0 0 0 0 3 7 3 3 7 3 0 7 3 0 0 0 6 21 9 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Group Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Meetings 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Oct Other work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Initial Visit 0 0 0 0 0 0 11 0 0 0 0 0 0 0 0 0 0 0 11 0 0 Repeated Visit 0 0 0 0 0 0 44 0 0 0 0 0 0 0 0 0 0 0 44 0 0 Outside Center Work 0 0 0 9 14 8 0 0 0 0 0 0 0 0 0 0 0 0 9 14 8 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Group Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Meetings 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Nov Other work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Initial Visit 0 0 0 0 0 0 2 2 1 4,5 0 0 7,5 0 0 0 0 0 14 2 1 Repeated Visit 0 0 0 0 0 0 8 10 3 17 0 0 14 0 0 0 0 0 39 10 3 Outside Center Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Group Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Meetings 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Dec Other work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Initial Visit 29 23 48 2 0 4 31 29 15 40 38 35 22 14 22 20 26 27 144 130 151 Repeated Visit 77 67 147 9 10 12 134 124 50 191 226 171 72 56 94 80 141 84 563 624 558 Outside Center Work 24 40 21 31 49 22 13 26 13 15 30 14 3 20 9 7 12 12 93 177 91 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 TOTAL PER YEAR Group Work 41 4 37 0 5 0 0 12 0 6 19 1 1 0 1 4 4 0 52 44 39 51 Meetings 0 0 10 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 2 1 11 Other Work 0 0 3 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 2 2 5 52 2008 Beslan Tuaeva Dzansolov Kallagova Kanukova Naskidaeva Plieva TOTAL Month Work type fam child adults fam child adults fam child adults fam child adults fam child adults fam child adults fam child adults Initial VIsit 2 4 2 2 3 2 2 5 2 2 5 2 0 0 2 2 4 2 10 21 12 Repeated Visit 4 9 4 0 0 0 14 18 9 23 19 20 0 0 7 11 10 11 52 56 51 Outside Center Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Group Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Meetings 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Jan Other Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Initial Visit 1 1 2 0 0 0 4 5 2 5 4 5 2 3 2 1 1 2 13 14 13 Repeated Visit 25 26 16 6 14 8 21 21 9 31 25 21 2 2 16 25 26 16 110 114 86 Outside Center Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Group Work 2 8 3 0 4 1 0 0 0 0 0 0 0 0 0 2 8 3 4 20 7 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Meetings 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Feb Other Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Initial Visit 0 0 0 1 1 1 5 3 5 2 1 2 1 1 1 0 0 0 9 6 9 Repeated Visit 16 18 15 9 12 10 25 34 14 27 26 26 2 8 17 16 18 15 95 116 97 Outside Center Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Group Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Meetings 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Mar Other Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Initial Visit 0 2 0 1 1 1 8 6 2 8 6 4 1 2 2 7 7 0 25 24 9 Repeated Visit 9 25 8 5 11 8 31 35 9 38 34 19 0 2 21 15 18 6 98 125 71 Outside Center Work 0 0 0 4 16 8 0 0 0 0 0 0 0 0 0 0 0 0 4 16 8 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Apr Group Work 0 5 0 0 0 0 0 0 0 0 16 0 0 0 0 0 10 0 0 31 0 53 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Meetings 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Other Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Initial Visit 0 1 0 1 1 1 2 4 3 4 6 5 0 0 0 0 0 0 7 12 9 Repeated Visit 8 16 12 3 4 3 18 24 6 16 16 4 8 8 22 2 3 1 55 71 48 Outside Center Work 0 0 0 3 9 5 0 0 0 0 0 0 0 0 0 0 0 0 3 9 5 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Group Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Meetings 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 0 0 0 10 May Other Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Initial Visit 0 0 0 1 1 2 8 12 6 5 6 5 1 0 1 1 1 1 16 20 15 Repeated Visit 0 0 0 6 12 7 11 11 8 14 14 11 8 15 24 3 2 5 42 54 55 Outside Center Work 0 0 0 10 21 12 0 0 0 0 0 0 0 0 0 0 0 0 10 21 12 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Group Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Meetings 0 0 0 0 0 0 4 10 4 0 0 0 0 0 0 0 0 0 4 10 4 Jun Other Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Initial VIsit 0 0 0 2 3 0 10 12 5 17 17 9 1 2 1 0 0 0 30 34 15 Repeated Visit 0 0 0 6 6 1 42 52 9 47 48 17 7 22 18 0 0 0 102 128 45 Outside Center Work 0 0 0 1 4 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Group Work 0 0 0 6 7 0 0 0 0 0 0 0 0 0 0 0 0 0 6 7 0 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Meetings 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Jul Other Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Initial Visit 0 0 0 1 1 0 7 7 5 5 6 5 17 10 10 0 0 0 30 24 20 Repeated Visit 0 0 0 6 7 1 13 12 4 22 17 10 39 5 10 0 0 0 80 41 25 Outside Center Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Group Work 0 0 0 5 5 0 0 0 0 0 0 0 0 0 0 0 0 0 5 5 0 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Aug Meetings 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 54 Other Work 0 0 0 0 0 0 0 0 0 0 0 0 0 11 74 0 0 0 0 11 74 Initial Visit 0 0 0 0 0 0 6 7 4 5 6 3 5 5 8 0 0 0 16 18 15 Repeated Visit 0 0 0 0 0 0 31 32 13 31 23 13 24 21 10- 0 0 0 86 76 36 Outside Center Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Group Work 0 0 0 0 0 0 0 5 1 0 0 0 0 0 0 0 0 0 0 5 1 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Meetings 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Sep Other work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Initial Visit 0 0 0 4 3 2 4 5 4 4 4 5 11 5 7 2 2 1 25 19 19 Repeated Visit 0 0 0 12 14 0 16 22 8 19 18 11 51 8 43 12 14 0 110 76 62 Outside Center Work 0 0 0 18 27 0 0 0 0 0 0 0 0 0 0 18 27 0 36 54 0 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Group Work 0 0 0 7 8 0 0 0 0 0 0 0 0 0 0 7 8 0 14 16 0 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Meetings 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Oct Other work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Initial Visit 0 0 0 4 6 4 15 0 0 0 0 0 0 0 0 0 0 0 19 6 4 Repeated Visit 0 0 0 5 6 8 45 0 0 0 0 0 0 0 0 0 0 0 50 6 8 Outside Center Work 0 0 0 14 17 0 0 0 0 0 0 0 0 0 0 0 0 0 14 17 0 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Group Work 0 0 0 5 6 0 0 0 0 0 0 0 0 0 0 0 0 0 5 6 0 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Meetings 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Nov Other work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Initial Visit 0 0 0 7 6 7 5 7 5 23 0 0 3 0 0 0 0 0 38 13 12 Repeated Visit 0 0 0 3 3 4 32 44 10 51 0 0 30 0 0 0 0 0 116 47 14 Outside Center Work 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Group Work 0 0 0 9 11 0 0 0 0 2 0 0 0 0 0 0 0 0 11 11 0 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Meetings 0 0 0 0 0 0 0 0 0 0 0 5 0 0 0 0 0 0 0 0 5 Dec Other work 0 0 0 0 0 0 0 0 0 0 0 5 0 0 0 3 0 5 3 0 5 55 Initial Visit 3 8 4 24 26 20 76 73 43 80 61 45 42 28 34 13 15 6 238 211 152 Repeated Visit 62 94 55 61 89 50 299 305 99 319 240 152 171 91 168 84 91 54 996 910 598 Outside Center Work 0 0 0 50 94 26 0 0 0 0 0 0 0 0 0 18 27 0 68 117 26 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Group Work 2 13 3 32 41 1 0 5 1 2 16 0 0 0 0 9 26 3 45 101 8 Preparation 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Meetings 0 0 0 0 0 0 4 10 14 0 0 5 0 0 0 0 0 0 4 10 19 TOTAL PER YEAR Other Work 0 0 0 0 0 0 0 0 0 0 0 5 0 11 74 3 0 5 3 11 79 56 ANNEX F NALCHIK COUNSELING DATA Background Data Disabled children in KBR 2,761 Disabled children in Nalchik 558 Families at Social Risk in KBR 2,079 Children at Social Risk in KBR 3,071 Nalchik Center Totals: October 2007 - September 2008 Orphans and vulnerable children treated 730 Hippotherapy recipients 50 Parents participating 259 Indirect beneficiaries 2,190 Specialists Involved 183 Organizations directly involved from KBR 35 Quarterly Report #6: (10/07-12/07) New orphans and vulnerable children receiving direct care 120 New Hippotherapy recipients 0 New parents participating 66 New indirect beneficiaries 360 New organizations directly involved from KBR 17 Gender Boys 29% Girls 71% Demographic Urban 47% Rural 53% Age Early Age 3% Preschool 29% School 68% Visit initiated by: Preschool Education Organization 24% Secondary School 53% Ministry of Health 15% Family Itself 8% Reason for visit Speech disorder 20.0% 57 Deaf and weak hearing 2.9% Mental retardation 34.7% Down Syndrome 0.6% Child Cerebral Paralysis 2.9% Disadaptation 20.6% Multiple disorders 5.9% Autism 1.8% Mental disability 10.6% Quarterly Report #7: (01/08-03/08) New orphans and vulnerable children receiving direct care 236 New Hippotherapy recipients 30 New parents participating 44 New indirect beneficiaries 708 New organizations directly involved from KBR 21 New specialists involved 24 Gender Boys 64% Girls 36% Demographic Urban 85% Rural 15% Visit initiated by: Preschool Education Organization 77% Secondary School 20% Ministry of Health 1% Family Itself 2% Reason for visit Speech disorder 77.7% Deaf and weak hearing 0.4% Mental retardation 12.2% Down Syndrome 0.0% Child Cerebral Paralysis 0.4% Disadaptation 4.5% Multiple disorders 0.8% Autism 1.6% Mental disability 2.4% 58 Quarterly Report #8: (04/08-06/08) New orphans and vulnerable children receiving direct care 243 New Hippotherapy recipients 10 New parents participating 56 New indirect beneficiaries 729 New organizations directly involved from KBR 0 New specialists involved 8 Gender Boys 73% Girls 27% Demographic Urban 69% Rural 31% Visit initiated by: Preschool Education Organization 52% Secondary School 43% Ministry of Health 1% Family Itself 4% Reason for visit Speech disorder 61.4% Deaf and weak hearing 1.3% Mental retardation 15.9% Down Syndrome 0.4% Child Cerebral Paralysis 1.3% Disadaptation 6.4% Multiple disorders 1.3% Autism 0.9% Mental disability 11.1% Quarterly Report #9: (07/08-09/08) New orphans and vulnerable children receiving direct care 131 New Hippotherapy recipients 10 New parents participating 93 New indirect beneficiaries 423 New organizations directly involved from KBR 0 New specialists involved 0 Gender Boys 68% Girls 32% Demographic Urban 50% Rural 50% 59 Visit initiated by: Preschool Education Organization 39% Secondary School 42% Ministry of Health 14% Family Itself 5% Reason for visit Speech disorder 32.9% Deaf and weak hearing 2.4% Mental retardation 17.5% Down Syndrome 0.7% Child Cerebral Paralysis 5.6% Disadaptation 6.4% Multiple disorders 5.9% Autism 1.4% Mental disability 25.2% 60 ANNEX G PERFORMANCE MONITORING AND EVALUATION PLAN CONTENTS Project Background and Progress Performance Monitoring and Evaluation Approach Strategic Framework and Indicators Performance Monitoring and Evaluation Purpose M&E Plan and Procedures Development M&E Resources, Organization, and Schedule Appendix A Methodology for Evaluation of Training Programs Appendix B A Guide to Performance M&E Activities Appendix C Participatory M&E with Partner Organizations Appendix D Request for Additional Beslan Polyclinic Information (Developed during the First Annual Performance M&E Report 61 COMPREHENSIVE SYSTEMS OF CARE FOR CHILDREN AND YOUTH IN THE NORTH CAUCASUS (CSOC) PERFORMANCE MONITORING AND EVALUATION PLAN The USAID-funded Comprehensive Systems of Care (CSOC) Project began on June 29, 2006. The two year CSOC Project is being implemented in the Russian North Caucasus by Keystone Service Systems, Inc. through the provision of a series of targeted facility-, home-, and community-based services to address the immediate psychological, educational, and health needs of vulnerable children and families. These systems, currently in progress in North Ossetia and with Project activities to be initiated in Kabardino-Balkaria, are being developed around the principles of being child-centered, family-driven, strength-based, culturally competent, and involving interagency collaboration. With the input from local North Caucasus Program staff and regional partners, Keystone is assuring that services provided follow each of these principles, as well as focus on the family's capacity to remain intact. The project will reach, but not be limited to, the following at risk groups: children, youth and families affected by violent trauma, children and youth with disabilities, children at risk of institutionalization, and families at risk of dissolution. Monitoring and evaluation (M&E) is an integral part of the Keystone methodology for achieving the intended project results. Project outputs and results will be monitored and contrasted against performance-based indicators for the CSOC Project activities. This Performance Monitoring and Evaluation Plan (M&E Plan) will guide the procedures, information collection and maintenance, analysis, and generating of M&E reports. Performance monitoring and evaluation will utilize a participatory methodology to engage CSOC Project staff and senior consultants in the implementation of the M&E Plan. Consistent with the overall participatory approach of the CSOC Project, government and non-government service systems as partners, international donors, and communities and beneficiaries as stakeholders may also be engaged in performance monitoring and evaluation. PROJECT BACKGROUND AND PROGRESS The North Caucasus is an ethnically, historically, economically, and politically distinct region within the territory of Russia, and is increasingly isolated from the rest of the Federation. The official statistics of the Ministry of Labor and Social Development of North Ossetia show that 165,176 children aged 17 and younger live in the region, with close to 3,000 of these officially identified as disabled. Basic health indicators in this region remain among the worst in the country. Political instability, and especially the terrorist attack on School No. 1 in Beslan of September 2004, has had devastating consequences on the development of the youth of the region and the stability of their family units, impacting their physical and emotional health, and consequently, their ability to contribute to the future stability of the North Caucasus. Children who grow up in trauma are at great risk of engaging in violent behavior as adults and are very vulnerable to being recruited into radical ideologies. Although there are a number of rehabilitation and social service programs for children currently operating in the North Caucasus, these are often limited and not adequately coordinated, and many are in jeopardy regarding their ability to survive over the long-term. 62 At the time of the development of the M&E Plan, the CSOC Project leadership was finalizing staff and consultant agreements, developing organizational relationships, and securing partnerships and support from Governmental and Non-Governmental entities in North Ossetia. A five-pronged approach to the enhancement of services was developed to serve as the foundation of a comprehensive system of care for children and families in the region. Having reviewed and assessed community needs and resources, the CSOC Project is providing direct services to children and families in North Ossetia, most notably in Beslan. With encouragement from USAID/Russia, exploratory visits to develop CSOC Project activities in neighboring Kabardino-Balkaria were underway. CSOC Project activities are directed from the Moscow-based Keystone Foundation for Children and Families (KFCF) by the CSOC Chief of Party, Maria Kalitina. Technical support is provided by the founding organization, Keystone Human Services International, based in Harrisburg, Pennsylvania, as coordinated by Charles Hooker. The CSOC Project focuses on four main objectives and activities are organized and implemented under the objectives as inter-related components. Highlights of CSOC Project activities over the past eight months, July 2006 through February 2007, are listed below according to the four objectives. Objective 1: Address the immediate psychological, educational, and health needs of the most vulnerable children and their families. ο Expansion of the services provided by the Children's Rehabilitation Room (Center) at the Polyclinic Hospital in Beslan (employment agreements in with the Project Director for the Medical Psychological Services and 5 Psychologists) ο Support for a “Mountain House” (Family Retreat) Rehabilitation Center ο Support for Youth Forums Objective 2: Increase the professional capacity of the individuals serving these families. ο Agreement to develop and provide Intensive In-home Family Therapy workshop to include participants from all aspects of the service delivery system in North Ossetia, including Social Workers, Psychologists, Psychiatrists, Educational Specialists, and other disciplines involved in multidisciplinary services to children and families ο Planning for Fundraising and Development Seminar in April 2007 provided by Ann Moffitt, Vice President for Development for Keystone and a Certified Fundraising Executive by the International Fundraising Professionals Association. Objective 3: Foster community development and citizen participation through involvement in program governance and oversight. ο Continued Development of community resources including coordination of actives with UNICEF and CAF 63 Objective 4: Initiate and maintain a comprehensive project monitoring and performance measurement strategy. ο Completion of the Performance Monitoring and Evaluation Plan PERFORMANCE MONITORING AND EVALUATION APPROACH In contrast to the traditional methodology of third-party monitoring and evaluation, Keystone will implement a participatory approach that takes advantage of staff, community members, and civil society familiarity with local objectives and constraints to improving services to children and families. Staff and consultants are well-situated to create and find sources of project information for M&E analysis and to uncover success stories related to the project. The participatory approach to performance monitoring and evaluation requires briefings to the staff and stakeholders by the M&E Specialist on evaluation methods and a commitment by staff members to collect M&E information and document activities that contribute to project results. STRATEGIC FRAMEWORK AND INDICATORS The CSOC Project addresses the USAID/Russia Strategic Objective (SO) 3: Use of Improved Health and Child Welfare Practices Increased. The Performance Indicator (6) to reflect progress against the Strategic Objective is “Annual number of orphans and vulnerable children receiving child welfare services.” Corresponding to the SO is the USAID/Russia Intermediate Result (IR) 3.4(1): Cumulative number of organizations involved in the development, reform and/or implementation of child welfare systems. As of December 31, 2006, the Performance Indicator SO 3(6): Number of orphans and vulnerable children receiving child welfare services was 72 as reported by Alexander Venger on the services provided at the Beslan Polyclinic. This figure is for the number of individuals (children) that received services. On average, the six psychologists at the Polyclinic each had a caseload of 11 individual children. The caseloads also involved other members of the families of these children. The Performance Indicator IR 3.4(1): The number of Russia-based organizations involved in the development, reform, and/or implementation of child welfare systems that were contacted by the CSOC Project was 38. The number of individuals related to the organizations that were contacted by the CSOC Project was 67. Additional CSOC Project performance information will be collected. USAID/Russia requested information includes: o Number of children and families served (both direct and indirect beneficiaries o Number of new services developed o Number of seminars/workshops/trainings conducted o Number of specialists (and community leaders) trained o Number of community-level organizations involved o Number of community-level events conducted 64 PERFORMANCE MONITORING AND EVALUATION PURPOSE The M&E Plan was developed according to the guidelines of the Automated Directives System (ADS) 201-203 to support the management, monitoring, assessment, evaluation, and reporting requirements of USAID/Russia. The implementation of this M&E Plan will be a process of ongoing review and improvement. As indicated in the USAID Performance Management Toolkit, this M&E Plan serves as a living document to manage the CSOC Project for results. Updating of the M&E Plan will be conducted at the end of the first year of CSOC Project implementation. Other M&E Plan adjustments will be made as required. CSOC Project staff and consultants will collect performance information with a part-time Russia-based M&E Specialist providing technical support under the guidance of the CSOC Project Chief of Party. Implementation of performance information collection as outlined in the M&E Plan by Project staff and consultants is intended to support informed management decisions, improved organizational processes, identification of performance gaps, and the setting of goals for improvements. Performance monitoring and evaluation should serve to track the ongoing CSOC Project outcomes and support the constructive review of Work Plan activities for the adoption of feasible and effective approaches to implementation. M&E PLAN AND PROCEDURES DEVELOPMENT MetaMetrics Inc., the subcontractor to Keystone, provides the basic technical assistance for the development and implementation of the Performance Monitoring and Evaluation Plan. Mr. Leo T. Surla, Jr., an M&E Specialist, worked with CSOC Project leadership and selected staff and consultants to develop the organizational structure and procedures to support the collection and analysis of results and program descriptive information as related to Work Plan activities. MetaMetrics technical support will be provided over the remaining 16 month CSOC Project period for the implementation of the M&E Plan and the writing of M&E reports. The Performance Monitoring and Evaluation Plan was developed with the full participation of the CSOC Project leadership. A presentation on M&E concepts and performance indicators was held with the consultants and staff of a key CSOC Project component, the Beslan Polyclinic. Elements of the M&E Plan were discussed with CSOC Project leadership resulting in amendments in order to be responsive to the evolving CSOC program. The Beslan Polyclinic M&E presentation was conducted in a participatory manner to engage the staff providers of services in the identification of key performance information. The staff and consultants agreed to collect and maintain key M&E information. In addition to the USAID requested information, the Polyclinic staff agreed to collect additional information as presented below. The staff and consultants defined direct beneficiary as the individual receiving Polyclinic treatment services and indirect beneficiaries as those family members relating to the direct beneficiary. They agreed to collect that information, according to the table that was constructed by Alexander Venger, CSOC Project Consultant, along with: o Number of counseling sessions o Total counseling hours o Number of different children and families served 65 The Polyclinic staff agreed to collect information on the results of counseling and services provided to beneficiaries. “Success stories” are to be noted and documented. The MetaMetrics M&E Specialist, at the visit for the end of the first year, may conduct a focus group with selected beneficiaries to document Polyclinic performance and results. Training will be a key activity for the CSOC Project and two major events are scheduled for the near future. A methodology and presentation materials were provided to CSOC Project leadership on methods for managing and evaluating training events. Training events can include conferences, roundtables, seminars, workshops, study tours, and group outings. CSOC Project staff and consultants will plan training events and define size and nature of the target populations, prospective topics, and training methodologies. This information will support the assignment of resources to collect M&E information on training events. The measures of size of training populations and satisfaction of participants regarding the training events will be collected as appropriate. Other more intensive follow-up information collection and evaluation analysis can address participant application of learned skills and new attitudes on the achievement of overall CSOC Project goals. The general methodology for evaluating training events is included as Annex A. The monitoring and evaluation of technical assistance requires a different approach from delivery of services to beneficiaries and training programs for indicators and information collection. CSOC technical assistance will generate the following selected outputs according to the Keystone Cooperative Agreement with USAID/Russia: o Establishment of two Comprehensive Systems of Care (CSOCs) o Development of model services for no less than 450 children and their families through the Center-based and home-based programs and other CSOC activities o Assistance to up to 300 young people in a youth club format and other CSOC activities o Establishment of an Internet-based dual-language communication system to strengthen professional capacity of child care workers from the region to collaborate with colleagues in other parts of Russia and world-wide o Development of the fund-raising capacity of both Centers, NGOs and community organizations involved in CSOCs, to generate financial support and achieve long￾term sustainability The basic approach to measure technical assistance performance and progress, in addition to the achievement of targets and the generating of ouputs, is the use of milestones or stages of development. For example, in the development of fund-raising capacity, the status of fund￾raising capacity of selected organizations at the beginning of the provision of the technical assistance would constitute the starting baseline. Milestones in the process of developing fund￾raising capacity can include initial meetings, a conference or workshop on fund-raising, formation of a working group within organizations, the writing of fund-raising proposals, and the award of grants or funds to the organization. The CSOC Project may include a grants program for the provision of services to beneficiaries and other activities. Such grants and subcontracts can be managed and evaluated within the CSOC Project M&E effort and provide the performance M&E information. Similarly, 66 the provision of commodities (equipment and computers) can be treated within a similar M&E framework. M&E RESOURCES, ORGANIZATION, AND SCHEDULE The management and evaluation of the components and activities of the Work Plan as well as the collection of M&E information are to be the responsibility of the primary staff and consultants assigned to those tasks. A Russia-based Monitoring and Evaluation (M&E) Specialist, to be identified, will support the CSOC Project staff and consultants in the accomplishment of their M&E activities under the overall supervision of Maria Kalitina, the Chief of Party. M&E presentations, similar to the one conducted by Mr. Surla with the Beslan Polyclinic, will be presented by the Russia-based M&E Specialist to other groups of staff and consultants that provide beneficiary services such as the Uspenie Foundation in Vladikavkaz and the emerging CSOC effort in Kabardino-Balkaria. Similar presentations on the collection of M&E information on training events will be conducted. The local M&E Specialist will obtain and organize M&E information from staff and consultants on a quarterly basis and support the writing of CSOC Project quarterly reports. Baseline data collection, project activities description, and ongoing compilation of outcomes information on CSOC Project progress, and monthly reporting will be the responsibility of the assigned CSOC Project staff, consultants, and partners. The MetaMetrics M&E Specialist, Leo T. Surla, Jr., will provide technical assistance to CSOC Project staff with two additional assignments to Russia over the next sixteen-month period. From July 15 through July 30, 2007 with an estimated level of effort of 13 person-days, he will provide technical assistance to review progress on collecting and reporting M&E information. He will work with CSOC Project leadership, staff, and consultants to document the First Annual Performance Monitoring and Evaluation Report. M&E presentations may be delivered to a Partners Review Advisory Committee, as discussed below. The M&E Report will provide input for the subsequent semi-annual Work Plans. From May 15 through May 31, the M&E Specialist will provide technical assistance for an estimated level of effort of 12 days. He will review M&E information and analysis as input for the Second Annual Performance Monitoring and Evaluation Report and the Final Report on the CSOC Project to be provided by Keystone to USAID/Russia. 67 APPENDIX A METHODOLOGY FOR EVALUATION OF TRAINING PROGRAMS Training events, including conferences and workshops, will be key activities of the CSOC Project over the remaining sixteen-month implementation period. Monitoring and evaluation of training events can go beyond the definition of size and nature of the target populations, prospective topics, and training. During the planning phase prior to the conduct of the training event, resources can be assigned to assure an appropriate evaluation is conducted. A basic M&E element is the collecting of information on training participant characteristics, size of training populations, and satisfaction of participants regarding the training events, workshops or conferences. This annex outlines methods to assist in the design of specific and targeted methodologies to facilitate the evaluation of discussion groups, roundtables, workshops, conferences, and other training events. Recommendations for evaluation procedures are intended only to guide the development of evaluation methodologies for each training event. Evaluation can be conducted on seminar sessions of a half-day to more extensive training of a week or more. The example presented here is for a three-day conference of presentations and workshops. TRAINING DESIGN The design and preparation phase for a conference can take up to several weeks and involve a team of trainers, technical resource individuals, and logistics personnel. A three-day conference will usually consist of an introductory morning session (to include registration, welcome presentations, and discussion of the intent and format of the conference) and the series of presentations, workshops, and general sessions. A definition of the objectives of the conference will assist in the evaluation of the various sessions, the assessment of effectiveness of presenters and resource specialists, and enhancing the overall value of the conference to the participants. Objectives can be established for the conference as a whole as well as for selected topics and sessions. These objectives will ordinarily be consistent with overall CSOC Project objectives and tailored to the needs of the participants. Discussions with USAID/Russia can serve to focus the conference intent and identify priority items to be addressed. In some cases, participants can contribute to the definition of objectives through discussions prior to the conference and/or through a questionnaire sent to participants for pre-registration. The set of conference objectives will support the design of the conference sessions, selection of presenters, and the conference procedures (number of general sessions, use of projectors and power point presentations, simultaneous discussion groups for different interests and topics, workshops, and smaller breakout sessions). CONFERENCE REGISTRATION At registration, participants are requested to complete one or more forms. Information will include, at a minimum, personal and identification data such as agency/organization affiliation, title, address, phone numbers, and email address. Educational background 68 information can be requested or, as appropriate, experience in the general conference theme (e.g. Please briefly describe your interest and experience in fund-raising). The objectives information requested in the pre-registration questionnaire can be included on the conference registration questionnaire. The key information will be the participants' intentions or objectives to be achieved through the conference (e.g. Please state what you would like to see achieved to support your ability to raise funds). This information will be useful for constructing the final conference evaluation forms. The registration forms can include a separate set of questions (pre-conference test) that reflect the attitudes, knowledge, and/or abilities of the participants at the beginning of the conference which will be addressed in the presentations and workshops. These questions will be included again on the evaluation form to be completed by participants at the end of the conference (post-conference test). These conference test questions can be framed as non￾threatening and respectful multiple choice and short essay questions. The number of questions can be as few as three and as many as ten, depending upon the nature of the conference. The participant can choose whether to sign the questionnaire. INTERNAL EVALUATION Presenters at the conference can contribute to the ongoing assessment of the value of the conference. Evening sessions with the conference organizers and resource personnel can serve to adjust the program to be conducted during the next day of forums and workshops. Presenters can also provide their own final assessment of the conference and its effectiveness in achieving conference objectives. If the conference will be presented again to a different audience or regional location, presenters may have information to improve the subsequent conference or workshop design. END OF CONFERENCE EVALUATION The conference will be evaluated on the basis of the conduct of the conference and the achievement of conference objectives. Accordingly, two instruments can be completed by the participants. The conduct of the conference form or Participant Assessment Form can include the following or similar items: ο Overall Conference Effectiveness: An overall rating of the conference for key objectives (e.g. To what extent do you feel the conference will affect your future activities in supporting passage of anti-corruption legislation; scale of one to five: Very Low, Low, Average, High, Very High). ο Conference Format: Rating (very low to very high) of conference components such as registration, opening sessions, presentations, workshops, final conference session. Effectiveness of individual presenters can also be rated. The balance of the conference design (workshops compared to presentations) can also be rated. ο Conference Logistics: Participants can rate conference setting (hotel or conference center), accommodations, meals, and conference coordination. 69 ο Overall Rating: Participants can rate the conference in comparison to other similar conferences. Specific questions can be included (e.g. Did you find the conference to be worthwhile, enjoyable; reports and materials useful?). ο Comments: Space can be included after each question for additional participant comments. A final question can be added for general comments of the participant. ο Participant Identification: The participant can choose to sign the form and/or list the agency or organization affiliation. The pre-conference test, if previously administered, will be completed again by participants. Changes from the pre-conference test may be added, depending upon the conference objectives as expressed by the participants at registration. CONFERENCE EVALUATION ANALYSIS The responses of the final conference evaluation forms can be tabulated. Percentages of responses can be calculated and reviewed. Participants may have repeated certain comments (e.g. hotel personnel were not helpful) that are relevant for conference evaluation and future logistics. Analysis can be conducted on each of the categories of the Participant Assessment Form. Selected comments can be included for each of the categories in the conference evaluation report. IMPACT/FOLLOW-UP EVALUATION Since participants will have provided contact information at the conference registration, a sample or all participants, depending upon the number of participants, can be contacted at some period following the conference. On the registration form, participants can be asked if they would permit a follow-up interview in the future. In addition to conference assessment questions and conference test questions asked at the end of the conference, additional follow-up information plus anecdotal information can be collected. Questions can include “Have you utilized any of the information or materials from the conference?” and “What examples of lessons of the conference have been useful to you?” Follow-up interviews can be conducted by phone, by email questionnaire, by mailed questionnaire forms, and by personal face-to-face interview. It is recommended that a follow-up analysis be conducted and a report generated from the follow-up information. MINIMUM TRAINING EVENT INFORMATION The following sheet is the minimum information requested for all CSOC Project training events. 70 TRAINING EVENT SUMMARY Training Event Name: Name of the conference or workshop Co-Sponsors/Partners: Organizations participating in providing support for the event Presenter(s): Names and titles of individuals presenting sessions of the event Dates: Actual dates of the training event Number of Sessions: Number of days for the event Number of Participants: Characteristics: May include participant information specific to the event Organization(s): Can include participant title within their organization Gender: USAID/Russia requirement Location(s): City where event takes place Summary: Four or five sentence description of the event Expected Results: Purpose and specific objectives of the event Realized Results: May include unintended positive results Participant Comments: Notice should be made of negative and positive comments that can be used to improve future training events 71 APPENDIX B A GUIDE TO PERFORMANCE MONITORING AND EVALUATION ACTIVITIES M&E activities link to the Work Plan which describes and schedules Project activities according to the operational tasks on a semi-annual basis. Performance M&E information will be conducted as a phased process. Initial emphasis is on definition of baselines, where appropriate, to be able to contrast the targeted changes over the CSOC Project implementation. Program descriptive information (activities and outputs) will be provided by CSOC Project staff to the Chief of Party for incorporation into the monthly reports. This programmatic information will be compiled for incorporation into the First Annual Performance M&E Report. Output information such as number of training events and participants will be collected. Subsequent monitoring and evaluation activities will focus more on results and impact of CSOC Project activities. CSOC Project staff have the primary responsibility to define baselines and indicators for targeted results; collect and maintain M&E information; utilize indicators, as appropriate, to prepare monthly reports; and assess and measure activity progress. In discussing M&E procedures, information collection, and related M&E coordination activities with CSOC Project staff and consultants, care was taken to ascertain whether the M&E level of effort would detract from the implementation of their assigned activities. The staff energy devoted to M&E is intended to be rewarded by an enhanced awareness of Project progress and the identification of potential issues and constraints. Identification of PM&E Indicators and Measures Essential to the eventual analysis of CSOC Project performance is the database of information that reflects the changes that can be attributed to Project activities. Key elements of this M&E database are baseline, outputs, results, and impact. The following definitions were discussed with CSOC Project leadership to support efforts to identify M&E information, data, indicators and measures for these elements: Baseline: Information that reflects the existing situation of the provision of services to children and families in June 2006 as appropriate to CSOC Project activities and milestones. Ideally, this would include information that is quantifiable and can be collected and compared periodically during Project implementation in order to measure progress. Quantifiable example: Number of organizations engaged in providing services in 12 month period prior to July, 2006. Some measures will be binary, that is, either exist or don't exist. Examples: A center dedicated to providing services was not in existance. Quality is an important element to reflect a baseline (training programs provided to psychologists and other service providers were considered inadequate or inappropriate). Outputs: The outcomes of Project activities. This information shows the immediate outcome of CSOC Project activities and includes items such as: Quantifiable: number of training programs delivered and number of trained service providers; and Binary: technical assistance (assessment of needs) provided to guide services development, the establishment of a youth program, and the establishment of a website. Detail such as characteristics of trained personnel can be included which add richness to the assessment of the training program (e.g. among the 72 training participants were the Deputy Minister of Health and the Chief Psychiatrist of the Regional Hospital. Similarly, the completion of a needs analysis can include summary information such as the types of recommendations. The quality of the output (the training program was considered excellent by participants) is also important. Results: The consequence of activity outputs. Examples include changes in knowledge and attitudes of trainees/workshop participants, the changes in understanding of services to families and children as a result of a CSOC Project publication, increased access of information through a website, and changes in awareness of services issues as a result of an assessment/study. Impact: The short-term and long-term effects of Project activities. Impact generally relates to programmatic objectives such as institutional strengthening (improved capacity to provide services and improved program efficiency). In the final analysis, impact will relate to the USAID/Russia Strategic Objective and Intermediate Result. In training evaluation, the relative short-term impact can be the workshop participant's application of knowledge and attitudinal shifts following workshop completion. A longer-term impact of the application of such knowledge would be the related change to goals such as institutional strengthening in the area of services to families and children. Sources are to be identified for the key information elements. A major source of performance management and evaluation information will be the involved counterpart and stakeholder personnel. Feedback from these individuals can also provide information on unexpected results and impacts, both positive and negative, of CSOC Project activities. Critical assumptions for achievement of Project objectives such as community support of establishment or increasing of services can also be noted. PM&E Information Collection Information collection, accomplished on a monthly or quarterly basis, will provide performance monitoring and evaluation information for the First Annual Performance M&E Report. In addition to documenting outputs, results and potential impact, performance management and evaluation data may support the identification of issues for review and potential adjustments to Project activities. Information for the four elements (baseline, outputs, results, and impact) will be qualitative and quantitative. Relevant anecdotal information will be noted and retained for M&E analysis (e.g. comments from key informants and decision makers). The engagement of stakeholders and community leaders will be noted for assessment of the effect of CSOC Project activities on sustainability. Information on issues such as gender and discrimination will be collected as appropriate. External factors and their appropriate indicators will be considered and monitored to ensure that CSOC Project implementation responds to the existing or changing situational dynamics in order to achieve USAID goals in services to families and children. The socioeconomic context, accordingly, will be viewed in the M&E process as a spectrum of potential constraints and opportunities. Emerging situational factors will require CSOC Project flexibility and the appropriate response and reconfiguration of targets, plans, and activities. 73 APPENDIX C PARTICIPATORY M&E WITH PARTNER ORGANIZATIONS The emphasis of CSOC Project activities is on working with community and stakeholder organizations as Project partners to achieve the USAID Strategic Objective and Intermediate Result. The Project could expand this participatory approach by actively engaging Project partners in performance monitoring and evaluation. The participatory approach supports transparent, accountable, and responsive engagement and is intended to directly affect community and local organization action on the provision of services to families and children. The participation of personnel from counterpart service agencies and local organizations could increase the analytic capacity of these units and promote the sustainability of services provision. When the key stakeholders participate actively in CSOC Project evaluation, they may discern programmatic requirements that are important to achieving their own goals. They can take responsibility for their individual and organizational decisions that contribute to program action and effectiveness. Participation can also support capacity building that can continue following CSOC Project completion. The impact of the CSOC Project implemented in this fashion has the possibility of extending beyond the results of immediate activities and into the very framework of the community and its institutions. A CSOC Partners Review Advisory Committee of individuals from approximately seven to ten partner organizations could be formed in each community of CSOC activity. At a organizational meeting of invited partner organizations, a briefing on the Performance Monitoring and Evaluation Plan could be given by the Chief of Party and the Russia-based Monitoring and Evaluation Specialist using a M&E Plan powerpoint presentation. In a collaborative approach following the M&E Plan presentation, agreement could be reached on the overall objectives of the Committee (e.g. support for the identification of M&E information sources, insights on services to families and children issues) and future meetings. The recommended chair for such a committee would be the CSOC Project Chief of Party. Meetings could be held periodically to review CSOC Project progress. At a minimum, in July 2007, the MetaMetrics M&E Specialist could present the First Annual Performance Monitoring and Evaluation Report. The possibility of an open public meeting (including the media) in a panel setting following the completion of the First Annual Performance Monitoring and Evaluation Report can be considered. The participatory M&E effort can incorporate consideration of elements reflecting the community context and other project-external factors affecting CSOC Project implementation and success. The reality of activity implementation at the ground-level requires an approach that will be responsive and able to support on-going decision-making as dictated by changing social and cultural developments in the community. The participatory approach can support the generating of information to address aspects of uncertainty with respect to the community environment in which the CSOC Project operates. 74 APPENDIX REQUEST FOR ADDITIONAL BESLAN POLYCLINIC INFORMATION 1. Reasons for Center Visits, Monthly: A reduced set of reasons or issues and each visit given a priority reason and a secondary reason from the same set of reasons. 2. Monthly Visits for Each Staff Person: Number of consultations for the month, number of families that visited, number of persons that visited. 3. Total Center Visits: Total Center visits, number of initial visits, number of repeat visits. 4. Characteristics of Clients: Number of children, by male/female; number of adults (18 and over years), by male/female. Most of the above information is already being collected by the Center. 5. New information requested to be collected, Monthly Caseloads for Each Staff Person: o Of the initial visits, the number of new cases that are accepted by the staff person and agreed to by the family or individual o Classification of current cases being served by the staff person into active or inactive with the definition of active/inactive to be determined by Center staff in consultation with support technical consultants o Number of cases completed (could include inactive cases where no visits to Center have occurred for a time period such as eighteen months) classified, as suggested or determined by Center staff with consultation with technical specialists into categories as suggested below: a) Case completed with satisfactory progress, b) Case completed with major program goals achieved c) Case referred to another service program d) Case completed because sessions ended by client(s) or inactive for a long period with the staff recommendation that further services be provided or obtained from another program by the family/individual.