LIVE, LEARN AND PLAY FINAL PERFORMANCE EVALUATION REPORT JULY 2016 This publication was produced for review by the United States Agency for International Development. It was prepared by USAID/Senegal Mission-Wide Monitoring and Evaluation Project. LIVE, LEARN AND PLAY FINAL PERFORMANCE EVALUATION USAID/SENEGAL Contracted under AID-685-C-15-00003 USAID Senegal Monitoring and Evaluation Project DISCLAIMER This evaluation is made possible by the support of the American people through the United States Agency for International Development (USAID). The contents are the sole responsibility of Management Systems International and do not necessarily reflect the views of USAID or the United States Government. The report was prepared by Sue Upton with support from Souleymane Barry and Aissatou Mbaye. Live, Learn and Play Final Performance Evaluation Report i CONTENTS Acknowledgments ..........................................................................................................................ii Cover Photo ................................................................................................................................................................................... ii Acronyms / Lexicon ......................................................................................................................iii Executive Summary ...................................................................................................................... 1 Evaluation Purpose and Questions ........................................................................................................................................... 1 Project Background ...................................................................................................................................................................... 1 Evaluation Design, Methods and Limitations .......................................................................................................................... 2 Findings and Conclusions ............................................................................................................................................................ 3 Recommendations ........................................................................................................................................................................ 4 Evaluation Purpose & Evaluation Questions .............................................................................. 6 Evaluation Purpose ....................................................................................................................................................................... 6 Evaluation Questions.................................................................................................................................................................... 6 Project Background ...................................................................................................................... 7 Development Problem ................................................................................................................................................................. 7 Evaluation Methods & Limitations .............................................................................................. 9 Data Collection Methods and Sources .................................................................................................................................... 9 Data Analysis Methods ..............................................................................................................................................................10 Limitations .....................................................................................................................................................................................11 Findings, Conclusions and Recommendations ......................................................................... 11 Question 1 ....................................................................................................................................................................................11 Question 2 ....................................................................................................................................................................................14 Question 3A .................................................................................................................................................................................17 Question 3B .................................................................................................................................................................................19 Question 4 ....................................................................................................................................................................................22 Question 5 ....................................................................................................................................................................................23 Question 6 ....................................................................................................................................................................................26 Recommendations ...................................................................................................................... 27 Annex I: Evaluation Statement of Work .................................................................................. 29 Annex II: Plan de Travail ........................................................................................................... 42 Annex III: Evaluation Calendar .................................................................................................. 62 Annex IV: Data Collection Instruments.................................................................................... 63 Annex V: Sources of Information .............................................................................................. 72 Annex VI: Activity Performance Monitoring Plan ................................................................... 75 Annex VII: Disclosure of Any Conflicts of Interest .................................................................. 78 Live, Learn and Play Final Performance Evaluation Report ii ACKNOWLEDGMENTS The authors wish to thank all those in Senegal, South Africa and the U.S. who generously contributed their time and thoughts to the evaluation. This includes USAID and the National Basketball Association (NBA), who provided funding and technical support to the project, as well as John Snow Inc., the Sports for Education and Economic Development (SEED) Academy, the National Ministries of Education and Sport and the National Federation of Basketball. Particular thanks go to the young people, parents, teachers and coaches who met with the evaluation team and shared their experiences of the Live, Learn and Play pilot project. Cover Photo An engaged audience of National Education and Sports Institute (INSEPS) students (physical education teacher trainees) discusses the experience of INSEPS interns working on the Live, Learn and Play (LLP) project. Photo by SEED Live, Learn and Play Final Performance Evaluation Report iii ACRONYMS / LEXICON Activity An activity is a sub-component of a project that contributes to a project purpose. It typically refers to an award (such as a contract or cooperative agreement), or a component of a project such as policy dialogue that may be undertaken directly by Mission staff. (ADS 200.6) DEMSG Department of Middle and Secondary Schools (Direction de l’Enseignement Moyen et Secondaire Général) INSEPS National Senior Institute of Physical Education and Sport (Institut National Supérieur de l’Education Populaire et du Sport) JSI John Snow Inc. LLP Live, Learn and Play MEN Ministry of Education (Ministère de l'Education National) MEP Monitoring and Evaluation Project MOU Memorandum of Understanding NBA National Basketball Association OCA Organizational Capacity Assessment PE Physical Education PPP Public-Private Partnership SEED Sports for Education and Economic Development SOW Statement of Work TAG Technical Assistance Group USAID U.S. Agency for International Development YALI Young African Leaders Initiative Live, Learn and Play Final Performance Evaluation Report 1 EXECUTIVE SUMMARY Evaluation Purpose and Questions This final evaluation aims to inform USAID and other stakeholders about the performance of the Live, Learn and Play (LLP) pilot activity after two years of implementation. This unique public-private partnership (PPP) between the U.S. National Basketball Association (NBA) and USAID/Washington offers substantial opportunities for examination in terms of lessons learned — not only for “sport for development” programs, but also for PPPs and issues of sustainability. The evaluation seeks to answer the following six questions: 1. To what degree and how has LLP developed young people’s competence in terms of leadership and citizenship? 2. How do local communities feel about LLP? 3A. What effect has LLP had on coaches’ capacity to work effectively? 3B. How has LLP been experienced differently by girls and boys and male and female coaches? Does LLP have other gender dimensions? 4. To what extent has JSI contributed to building the institutional and organizational capacity of Sports for Education and Economic Development (SEED)? 5. To what extent are LLP activities linked to “sport for development” sustainable? 6. What lessons can be learned from strategies to develop public-private partnerships (PPPs) within the LLP framework? Project Background LLP is a three-year pilot activity funded through a PPP between USAID/Washington and the NBA. The activity was signed February 5, 2014, and runs through February 2, 2017; it received its $1,150,000 funds through the Young African Leadership Initiative (YALI). The project works with youth from 13 to 18 years old through basketball and physical education (PE) coaches in middle and high schools and basketball clubs to train youth in a leadership curriculum. Senegal was chosen due to the presence of the elite Sports for Education and Economic Development (SEED) basketball academy based in Thiès that was launched in 2002 by a former NBA player originally from Senegal. SEED is the national implementing partner, supported by international implementing partner, John Snow, Inc (JSI), which manages the overall LLP activities and is working to build the institutional and organizational capacity of SEED. As of April 2016, SEED reported a total of 1,595 youth involved in the LLP project, comprising 583 students in participating schools and 1,012 youth in basketball clubs. A total of 241 coaches have received LLP training in five regions of Senegal (Thiès, Dakar, Louga, Kaffrine and Kolda). This innovative project falls under USAID/Senegal Country Development Cooperation Strategy (CDCS) Development Objective (DO) #3: More effective citizen participation in the management of public affairs at the national and local levels. Within DO3, its objectives fall under Intermediary Result #1: Better educated youth. Live, Learn and Play Final Performance Evaluation Report 2 Evaluation Design, Methods and Limitations This evaluation uses a mix of quantitative and qualitative data collection tools to address the six evaluation questions. Following an initial desk review of project documents, fieldwork spanned three weeks, April 21 through May 6, 2016. As most LLP activities occur in schools and clubs in Dakar and Thiès, the evaluation team worked in these two regions, visiting 16 schools and four basketball clubs in the areas. A quantitative survey gathered responses from 499 youths (86.6 percent) in LLP-affiliated schools. Given the limited amount of information available on the literacy levels of youth involved in the LLP clubs, the evaluation team did not conduct the survey among club members. In place of the survey, focus groups were held with LLP club members. Focus groups were also conducted with youth parents. In addition to the large number of youth surveyed to determine their impressions of change over the course of LLP, the evaluation team also asked parents and school staff about changes they have seen in youth who participated in the project. Focus groups and group interviews included parents, LLP coaches and school directors and staff. Individual interviews took place with JSI and SEED staff, as well as key Government of Senegal stakeholders, such as the Ministry of National Education (MEN), the Ministry of Sport and the National Senior Institute for Physical Education and Sport. The matrix design for this evaluation appears in the evaluation work plan of this report (annex II). Informaton sources and data collection methods for each evaluation question are summarized in the following matrix: Evaluation Question Topic Data Source, People Consulted, Data Collection Method LLP youth (557) Parents (28) Coaches (21) School staff (10) SEED (7) INSEPS (1) Other TAG (4) JSI (3) USAID (3) NBA (1) Questionnaire Focus Focus & 1to1 Group & 1to1 Group KII KII KII KII KII 1. Youth competence in leadership and citizenship X X X X X X X 2. Community appreciation X X X X 3a. Capacity of coaches X X X X X X 3b. Gender dimension X X X X X X X X 4. SEED capacity X X X X X 5. LLP Sustainability X X X X X X X 6. Learning from PPP strategies in LLP context X X X X X X KII= Key Informant Interview; Focus = Focus group; 1to1 = one-to-one interviews; Group=group interviews The main limitation of this evaluation is that the survey focused on project schools, resulting in a lack of quantitative data on club-based youth. In 2015 and 2016 there was some limited expansion of activities into basketball clubs in the regions of Louga, Kaffrine and Kolda but due to the shorter perod of Live, Learn and Play Final Performance Evaluation Report 3 intervention in the new clubs, the evaluation focused on LLP-affiliated schools and clubs in Thiès and Dakar only. Findings and Conclusions The majority of LLP youth responding to the survey consistently described positive changes in their behavior and their motivation to maintain good grades in school. Negative behavior — such as being late, arguing with peers, parents, teachers and siblings or acting from a self-centered perspective — has given way to greater control of emotions, less aggressive reactions, greater consideration for others and taking more personal responsibility. Young people (13-18 years) surveyed described feeling more self￾confident and more able to work toward and achieve personal objectives. Several respondents cited individual and collective actions to help less fortunate community groups, including donations to a school for disabled students, visits to orphanages and environmental protection measures at home and at school. Parents, other family members, school staff and coaches said that they noticed LLP participants being healthier, calmer, more helpful, more self-confident and better able to express themselves. Some drew attention to LLP youth having a positive influence on their peers and siblings through setting an example or diffusing conflict. Others mentioned improvements in the school environment due to LLP students’ efforts to provide wastebins and collect litter. Some young people reported changes in the way their families spoke about them; some also felt that they were treated differently at home as a result of LLP. While parents and teachers said they were pleased with the results of the project, many would welcome more direct involvement and information. Several coaches conceived of creative opportunities for learning about citizenship and one group of LLP youth from two neighbouring schools created an association to promote community action. SEED has trained three categories of LLP coaches (SEED professional coaches, schools physical education teachers and teachers in training at INSEPS). All of them report benefits from participating in the project. They now structure and organize their training sessions more effectively and some club coaches and school PE teachers said they altered their sometimes-aggressive approach to coaching for a more supportive mentorship and child-centred pedagogy. One coach said he had become a better citizen in his effort to be a good role model. PE teacher trainees from National Institute for Physical Eduction (INSEPS) had the opportunity to be LLP interns and became enthusiastic advocates of the model and its inclusion in wider INSEPS curricula. LLP works with equal numbers of boys and girls. Many of the youth enrolled had little if any prior experience of basketball. The majority of coaches are men, reflecting the existing imbalance of male and female PE teachers and coaches in Senegal as a whole. However, four of the seven SEED lead coaches are women and INSEPS actively encourages gender balance by promoting women students. LLP youth were asked their opinion about statements concerning rights, roles and responsibilities of men and women or boys and girls in terms of education, work and sports, and the majority demonstrated an understanding of equal rights. The Ministry of Education has offered to provide trainers to enable LLP coaches to be trained in gender issues. The LLP public-private partnership has been particularly complex, bringing together the two very different agencies of USAID and the NBA around a vision, without any accepted procedures and practices to work with. However, JSI has worked hard to facilitate the process and has highlighted key lessons as the importance of identifying the added value of each partner and the development of a common language and shared values to underpin the partnership. Clear identification of roles and responsibilities is not enough; it needs to be backed up by commitment of all parties to make the relationship work and overcome misunderstandings and any frustrations. While the initial Memorandum Live, Learn and Play Final Performance Evaluation Report 4 of Understanding (MOU) was between USAID in Washington and the NBA in New York, the active partners are now USAID/Senegal and the NBA in Africa. Strong demand exists for LLP activities to continue in schools and basketball clubs and various potential opportunities for sustainability are present. NBA Africa is committed to ongoing financial support. JSI and SEED have worked together extremely effectively. SEED has significantly improved its organizational and institutional capacity, to the extent that, based on discussions with USAID/Senegal’s Office of Finance and Management, it could potentially qualify for direct funding from USAID1. USAID’s regular use of its Organizational Capacity Assessment (OCA) tool provided a strong basis to develop a work plan through which SEED, with support from JSI, has moved closer to best practice status. Some uncovered ground remains, but the SEED team is dynamic and has a clear vison with each member aware of his or her role and responsibilities, and there is every indication that the team’s capacity will continue to develop. USAID’s committed funds run out in November 2016, but the agency has not ruled out providing continued support. However, this presents a challenge due to limited resources and the cross-sectoral nature of LLP. Senegal’s Education and Sports ministries have both indicated openness to supporting future activities and INSEPS is committed to teaching the LLP model as part of the PE curricula in Senegalese universities. There is also potential for expanding the LLP model to other African countries through SEED, which receives invitations to train coaches across the continent, and INSEPS, which has direct links with other francophone countries. Recommendations These recommendations concern the implementation of the LLP model and are presented for the consideration of key stakeholders. They aim to root LLP activities more firmly in communities and encourage community-level ownership and visibility to complement SEED’s overall coordination of activities. SEED 1. Create a detailed transition plan and budget for SEED to coordinate LLP in partnership with members of the Technical Support Group, particularly the Government of Senegal. 2. Provide a regular opportunity for coaches to meet. 3. Encourage schools and clubs to provide opportunities for parents to be more involved in LLP activities. 4. Widen the circle of core partners with Senegalese private sector companies that would have interest in underwriting LLP activities. 5. Encourage schools and clubs to involve local government in LLP activities. 6. Organize the coaches’ gender training with the Ministry of Education/Direction of Middle and Secondary Schools (MEN/DEMSG) before the end of the pilot project. INSEPS 7. Integrate the LLP model into the PE teacher training curriculum. 8. Serve as an advocate. Work with SEED and government stakeholders to further the progress of “sport for development” in Senegal and other African countries 1 Improvement in SEED’s scores for the Organisational Capacity Assessment (OCA) carried out by USAID can be seen in Figure 5 on page 23. Live, Learn and Play Final Performance Evaluation Report 5 Ministries of Education, Youth and Sport 9. Take a greater degree of ownership of LLP activities through pursuing a more active partnership role in the future of LLP initiatives in schools and clubs, including 9. Supporting LLP model integration into the physical education teachers training curriculum 10. Support the funding of schools, clubs and others activities based on the model to promote citizenship, healthy behavior, gender equality and any other issues 11. Encourage school principals to establish LLP parents’ groups LLP TAG Members 13. Develop a transition plan that clarifies roles, responsibilities and commitments of all parties and establishes a specific timeline. NBA 14. Continue to support SEED in implementing “sport for development” in Senegal. 15. Continue to collaborate with USAID and other development partners as the opportunity arises. USAID 16. Help SEED develop capacity to reach a best-practice level of financial management (an OCA score of 4). 17. Provide a small development grant to SEED to support its transition plan to manage LLP. 18. Arrange briefings for SEED with technical teams to explore the other potential uses of the LLP model. 19. Review branding regulations and allow SEED to use its logo on LLP materials to promote ownership and sustainability of the model post USAID-funding Live, Learn and Play Final Performance Evaluation Report 6 EVALUATION PURPOSE & EVALUATION QUESTIONS Evaluation Purpose This evaluation aims to inform USAID/Senegal and Washington about the performance of the LLP activity following its first two years of implementation.2 The immediate purpose of the evaluation is to assess the progress, successes and shortcomings of the LLP project in the eyes of its various beneficiaries, both direct and indirect: youth, coaches, parents, school directors, national ministries and teacher training institutions. The Senegal project, through the LLP, served as experimental pilot for the “sports-for-development” approach. As a pilot, this evaluation was a condition of the original MOU signed between USAID/Washington and the NBA to learn lessons from the application of the model. The findings, conclusions and recommendations from this performance evaluation will facilitate the process of Agency decision-making for any similar future interventions in Senegal or other African countries. This performance evaluation seeks to inform at three strategic levels of decision-making through providing information and analysis concerning:  the success and limits of the implemented activities;  the relevance of continuing support for LLP; and  existing sustainability strategies for the ongoing implementation of LLP and “sport for development” 3 as a model. The evaluation will also provide feedback to USAID/Senegal and Washington, the NBA and international implementer JSI, as well as the Senegalese Ministry of Education (MEN), Ministry of Sport, Ministry of Youth, Work and Citizen Development (MJECC) and National Institute for Sports Education (INSEPS), concerning the effectiveness of their role in and support to the project. The evaluation also intends to enable the team from SEED to reflect on its role and performance as the central national implementing partner. USAID/Senegal will share this report with NBA Africa, the MEN, Ministry of Sport, MJECC, INSEPS and other relevant stakeholders. Evaluation Questions The evaluation seeks to answer the following questions: Question 1: To what degree and how has LLP developed young people’s competence in terms of leadership and citizenship? Question 2: How do local communities feel about LLP? Question 3A: What effect has LLP had on coaches’ capacity to work effectively? Question 3B: How has LLP been experienced differently by girls and boys and male and female coaches? Does LLP have other gender dimensions? 2 While LLP is technically an “activity” under USAID parlance, it will be also be referred to as a project in this report to assist the understanding of the wider stakeholder audience. 3 “Sport for development” is the model at the heart of the LLP activity/project. Live, Learn and Play Final Performance Evaluation Report 7 Question 4: To what extent has JSI contributed to building the institutional and organizational capacity of SEED? Question 5: To what extent are LLP activities linked to “sport for development” sustainable? Question 6: What lessons can be learned from strategies to develop public-private partnerships (PPPs) within the LLP framework? PROJECT BACKGROUND Development Problem The 2013 Live, Learn and Play (LLP) project description stated: “About 65 percent of the total population of Africa is below the age of 35, and over 35 percent are between the ages of 15 and 35 – making Africa the youngest continent. By 2020, it is projected that out of four people, three will be on average 20 years old. About 10 million young African youth arrive each year on the labor market. Thus, there is a growing urgency in finding positive opportunities for youth to become engaged in the development of their country by engaging in the political, economic and social development issues that lay before them.” In 2014, an estimated 63 percent of Senegal’s population was under 25 years old. 4 Today, Senegalese and youth from across the Sahel continue to migrate in significant numbers. The risk of unemployed youth with few prospects being attracted by criminal or extremist groups is evident. This is the context in which LLP aims to develop strategies to attract and influence young people to harness their creativity, energy and commitment to become responsible leaders and citizens acting for a positive future. Public-Private Partnership The LLP public-private partnership grew out of discussions between the NBA in New York and USAID/Washington. The idea was to promote social change and youth development using the power of sport through LLP. The model would use basketball to foster skills among African youth so they might become positive change agents within their communities. A four-year MOU was signed in November 2012. Senegal was chosen to pilot LLP because of the presence of SEED, an elite Senegal-based basketball academy started by Amadou Fall, an ex-NBA player from Senegal (and currently managing director of the NBA in Africa). SEED believes that “all youth have the right to a quality education, access to elite sports programs, and the leadership tools to make big dreams come true.” Funds from The Young African Leaders Initiative (YALI) were used to fund the Senegal pilot. The original idea was that coaches would become leadership role models who would train youth on leadership skills and materials from the NBA which were adapated for SEED. Implementing Partners and Stakeholders The JSI Training and Research Institute was selected to implement the LLP project in Senegal and tasked with developing the capacity of SEED, the national implementer. This was detailed in a three-year agreement projected to run from February 5, 2014, to February 2, 2017. An LLP technical assistance group (TAG) was established to ensure national ownership and included government and non- 4 http://www.indexmundi.com/senegal/demographics_profile.html Live, Learn and Play Final Performance Evaluation Report 8 governmental actors from the Senegalese ministries of Education, Youth and Sport, INSEPS, the Senegalese National Basketball Association, the Executive Office of the President, as well as SEED, JSI and USAID. Beneficiaries The project began in 2014 working only with middle and high schools in the Thiès and Dakar regions. Currently, the project operates in eight schools inThiès and nine schools in Dakar for students aged 13 to 18 years. Schools were selected based on their access to a basketball court in reasonable condition (the project also refurbished six courts in strategic positions), the willingness of the school to host the project and the availability of a physical education (PE) teacher as the primary contact point. Schools were mainly mixed-sex, urban, state establishments, some in very poor areas, but two private schools, one rural school and two single-sex schools were also included. A total of 36 students were selected in each school: 18 girls and 18 boys, in mixed schools. The most important criterion for participation is that they should have and maintain good grades. Many of those selected had never played basketball before and selection was competitive, since demand exceeded the number of available spots. The project aimed to reach vulnerable children; while this is not its primary target group, it has done so some extent. In 2015 the project expanded to begin working with basketball clubs. The rationale for working with clubs was that the program would be able to reach a greater number of children using little financial investment, given the project’s limited ressources. Similarly to the schools, each club works with 36 youth per session, inclidng girls and boys in roughly equal numbers, but he number of youth involved often fluctuates. All basket ball club members can be considered as beneficiaiires of the LLP model to some extent since club coaches have been trained to use the approach in their training sessions. Currently, LLP has evolved to include seven basketball clubs in the regions of Thiès, Dakar, Louga, Kaffrine and Kolda, as well as one recently added school in Thiès. The SEED weekly bulletin of April 25, 2016, reports that 1,595 youths were directly involved in the LLP project during that week, including 583 students in 17 schools (nine in Dakar and eight in Thiès) and 1,012 youth in basketball clubs. In addition, 241 coaches, comprising 159 trained coaches and 82 trainers of coaches, have taken part in LLP training. Approach Live, Learn and Play uses basketball to promote young people’s personal development and build their understanding and capacity as responsible leaders and citizens. NBA coaches, in collaboration with SEED, have run two LLP training-of-trainers courses for professional basketball coaches. Trained LLP coaches have then trained other coaches, including school-based PE teachers and PE teachers in training from the national training institute INSEPS in the LLP model, which includes good practice for basketball coaching, mentorship skills, personal development, leadership and citizenship modules. Every week, LLP students attend two out-of-school training sessions, each lasting two hours. A SEED coach, assisted by a PE teacher from the school and an INSEPS intern, leads the sessions, which follow the leadership curriculum adapted by JSI from NBA materials, and links basketball training with leadership and citizenship teaching. Students are encouraged to make a connection between basketball skills and personal development. LINKING BASKETBALL SKILLS WITH SELF-DEVELOPMENT  Control of the ball is linked to self￾control and capacity to manage one’s emotions;  The confidence needed to shoot the ball and score is linked to self￾confidence to achieve goals in everyday life; and  Respect for the rules of basketball is compared to respect for social norms and good citizenship. Live, Learn and Play Final Performance Evaluation Report 9 Students are prompted to set specific personal objectives and report back on progress. Each training session begins and ends with a group solidarity chant: “I act for myself, for my team and for my community.” Schools compete in LLP tournaments on a regular basis and summer camps also take place. In addition to refurbishing six basketball courts, the project has provided shirts, shoes, balls and pumps, as well as stipends and travel expenses for coaches. Project Results The LLP project fits under the third development objective of USAID’s Country Development Cooperation Strategy (CDCS) General Policy, which states: “More effective citizen participation in the management of public affairs at national and local levels.” The project’s overall goal, as defined in its monitoring and evaluation plan was to enable young people to become positively engaged in their communities. Because this was a new public-private partnership developed in the U.S., JSI was conservative in setting targets for the project’s intended outcomes. The project aimed to reach 1,215 young people with the expectation that 20 percent (243) would emerge as youth leaders in participating schools and communities and another 15 percent (182) would demonstrate improved skills and leadership. Of a target 120 coaches trained, the aim was for 80 percent (96) to achieve a “competent to expert” rating. Based on project reporting, all targets have been met and, in some cases, exceeded by more than 100 percent. In addition, the project has carried out baseline and midline assessments; the endline study will take place in June 2016. The M&E plan with targets and actual results is in Annex VI. EVALUATION METHODS & LIMITATIONS The evaluation team comprised an international lead evaluator and two members of the permanent monitoring and evaluation project (MEP) team. The evaluation fieldwork spanned three weeks, April 21 to May 6, 2016. To conduct surveys, focus groups and individual interviews in the limited amount of time, two additional researchers and data entry personnel were recruited and trained as part of the data collection process. The majority of LLP’s activities are centered in Thiès, where SEED is located, and Dakar. In 2015, basketball clubs were included from three other regions, but only one club per region. For this reason, the evaluation team focused on LLP-affiliated schools and clubs in Thiès and Dakar only. This is a limitation but it does not affect the accuracy of the results. Over the three-week period, the evaluation team visited all nine LLP schools and all three basketball clubs in Dakar. In Thiès, the team met with seven LLP-affiliated schools and the one basketball club. The evaluation calendar and a detailed list of respondents can be found in annexes III and V. Data Collection Methods and Sources The evaluation used a mixed-methods approach, gathering quantitative and qualitative data from a range of stakeholders, including beneficiaries and their communities and key informants from private and public implementing agencies and national institutions. This approach aimed to access information concerning the performance levels and changes in attitudes, behavior and practices of beneficiary groups, specifically LLP youth and coaches. The team also sought to understand the degree of project ownership at national and local levels and any potential for sustainability. The evaluation team used the following data collection methods: Project document desk review to provide project progress and level of achievement in terms of set targets and performance measurement. It was assumed that all available quantitative data provided by USAID/Senegal, JSI, SEED, and other key stakeholders was reliable and valid. (Annex V contains a list of the project documents consulted). Live, Learn and Play Final Performance Evaluation Report 10 Youth survey of students in LLP-affiliated schools. The questionnaire probed changes in attitude and behavior in the different areas of LLP learning, such as leadership, citizenship, personal development, gender roles and community participation. Students also related their opinions of their coaches and any changes they observed in teaching approach after coaches and teachers received LLP training. A total of 499 young people completed the survey, which represents 86.6 percent of the total number of LLP￾affiliated school students. The survey is in Annex IV. Focus groups with youth involved in the LLP basketball clubs, as well as their parents and coaches. Three basketball clubs that have been involved with LLP for at least a year took part in focus group discussions, with 58 youth basketball club members participating in six focus groups. The questions asked, per the moderator guide, followed the survey questions and focused on leadership skills learned, changes in behavior, and perceptions of coaches and of the project. Group interviews with parents and coaches. The evaluation team held group interviews at five LLP￾affiliated schools, covering public and private and rural and urban establishments in Dakar and Thiès; these included PE teachers who were also LLP coaches. The team held three group interviews with parents of youth involved in clubs and two group interviews with parents of children in LLP-affiliated schools. A total of 27 coaches took part in interviews across Dakar and Thiès. Training observation using a checklist for sessions with John F. Kennedy and Blaise Diagne schools in Dakar. The checklist focused on the training protocols and subject matter to be taught at each training session. Key informant interviews with school directors and personnel in each LLP-affiliated school the team visited. In addition, interviews took place with the SEED team, USAID/Washington and USAID/Senegal, JSI, NBA Africa and members of the LLP TAG, which included representatives of the MEN, Ministry of Sport, INSEPS and the National Basketball Association. A table summarizing the sources and methods used to gather information to answer each of the evaluation questions is in Annex IV. Data Analysis Methods Quantitative Data Analysis Some statistics were collected from project documents, including a database of school-based participants, and these complemented the evaluation survey results from the questionnaire that LLP students completed. Data entry utilized an input mask based on the survey questionnaire and the database was then exported for a series of uni-, bi- and multi-variate analyses using Sphinx data analysis software. For the analysis of the survey data, in addition to frequency analysis, results were disagregated by region, age, sex and education level in order to verify possible differences in the changes observed Qualitative Data Analysis Qualitative information from focus groups and semi-structured interviews with club-based LLP participants, coaches, teachers and parents complemented, confirmed and was a basis for exploring the data from project documentation. Common themes were identified across the individual and group interviews as well as focus groups. Interviews with those providing project funding, technical support and implementation enabled the evaluation team to understand and assess the potential for sustainability, any difficulties encountered during implementation and lessons to be learned. Live, Learn and Play Final Performance Evaluation Report 11 Limitations Early in the methodology development, the evaluation team decided that conducting youth surveys was critical in allowing a maximum number of youth to provide input to the evaluation. The previous internal evaluation conducted by JSI included only 86 youth beneficiaries. The use of the survey across a larger number of youth provides more insights on the demographics and perceptions of youth toward the project. The introduction of basketball clubs that were not affiliated with schools is a more recent endeavor for LLP and limited information has been kept on the youth involved in these clubs. For this reason, it was not clear whether the youth in LLP clubs would be able to complete the handwritten survey. The evaluation team decided to conduct only focus groups with LLP club youth using a focus group moderator guide concentrated on similar questions from the survey. While qualitative data on the youth involved in clubs was collected to complement the data collected from youth in LLP-affiliated schools, quantitative analysis of club member demographics and responses is not possible. Ultimately, the evaluation team learned that all LLP club youth are enrolled in middle and high schools and would have been able to participate in the survey. The LLP approach is applied to the two broadly similar population groups but in different settings with different degrees of support. The youth in both settings reported similar experiences with similar results. The survey enabled the youth to self-report on changes they had seen in themselves, as well as make observations about their coaches and the way their families treat them. This active participation by the youth was triangulated with information gathered from their families and schools. The evaluation team collected information from parents, teachers and school principals to determine their impressions of changes observed at home and in school with LLP youth and coaches. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS This main section of the report provides a synthesis of the evaluation learnings. It presents findings and conclusions for each of the evaluation questions followed by recommendations. Question 1 To what degree and how has LLP developed young people’s competence in terms of leadership and citizenship? Findings Through the survey and the focus groups, LLP youth consistently described positive changes in their behavior as a result of their involvement in LLP. They also spoke of their motivation to maintain good grades in school, since this was a prerequisite for attending basketball training sessions. Group interviews with parents and school directors reiterated these changes in youth behavior. Negative behaviors such as being late, arguing with peers, parents, teachers or siblings, shouting and being self-centered gave way to greater emotional control, less aggressive reactions, greater consideration for others and taking more personal responsibility. Respondents also cited some examples of individual and collective actions to help less fortunate members of their communities including donations to a school for disabled students, visits to orphanages and environmental protection measures. One school group had visited the local town hall and talked with the mayor about his role and the role of a good citizen. Live, Learn and Play Final Performance Evaluation Report 12 In terms of learning new skills, 98 percent of LLP school students reported having learned new skills as a part of LLP. Only 2 percent reported having learned no new skills. When asked to identify one of the skills acquired, the most frequent response was basketball (18.4 percent), followed by acquiring confidence/self-control (14.8 percent), study and sports (13.6 percent) developing respect for family members and friends (11.5 percent) and etiquette and talk. Youth probably learned more than one new skill but the questionnaire focused on asking them to identify one, assuming that it would be the most important. This was not a multiple choice selection, but required students to complete a phrase in their own words describing how their attitudes or behavior had changed as a result of particpation in LLP. FIGURE 1: STUDENTS’ SKILLS LEARNED THROUGH LLP PARTICIPATION Focus groups with LLP club members also showed that youth felt they were learning new skills. Focus group participants most commonly cited an increase in self-control since joining the LLP club. In the BOPP Club of Dakar, youth noted an increase in their skills since taking part in LLP. This suggests that LLP has encouraged the development of a broad range of desirable behavioural characteristics and personal development attributes. In terms of LLP’s effect in building confidence, the majority of youth (90.2 percent) responded in the survey that LLP had helped build their confidence. The figure below shows that 38.6 percent of these respondents identified having general confidence in themselves, while others specified improved confidence in their basketball playing (22.3 percent), studying (9.7 percent), behaving well (9 percent), being a leader (8.8 percent) and speaking in public (7.9 percent). While most LLP participants reported increased self-confidence, it was more difficult for many youths to identify specific areas. Those who did once again showed an increase in a range of desirable personal characteristics, many linked to leadership and citizenship. 18.4% 2.1% 3.3% 5.4% 14.8% 9.4% 5.4% 11.5% 4.6% 11.5% 13.6% 0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0 18.0 20.0 basketball Lead / Manager Respect the environment Right and civic duty Confidence / Self-Control collective commitment Personal commitment Respect the family members / friends etc. Friendship / Opening Etiquette / talk Study and sports Before, we came to play or to train only and then go home; with LLP, aside from the basketball, we learn things like how we should act, respect for others, citizenship. — LLP Club Focus Group Participant BOPP, Dakar Live, Learn and Play Final Performance Evaluation Report 13 FIGURE 2: CLUB MEMBERS’ SKILLS LEARNED THROUGH LLP PARTICIPATION To teach these new skills and build confidence, the LLP coaches follow a training approach developed from NBA materials that compares aspects of basketball to daily life; in the same way that a person can learn to control a ball, he or she can learn to control themselves, their emotions, and the way they act. The evaluation team observed that at each break, coaches called players together to discuss issues related to leadership and citizenship, including self control, consideration for others and for the environment. In all of the focus group discussions with LLP clubs, participants reiterated this increased self-awareness and increase in confidence. The part of the story that the survey and focus groups results do not tell is the enthusiasm and energy with which LLP participants talk about basketball and the changes they have experienced during their time with the project. They clearly feel pride in their achievements and have a positive perspective on life. It should also be noted that the survey responses were not tick-box responses, but open-ended questions where respondents were required to formulate their own answers. Comparing the ages of LLP youth and the changes they reported experiencing revealed that 94.2 percent of the younger age range (those attending middle school) reported a change in attitude or behavior, whereas only 88.4 percent of the older age group (those attending high school) reported such a change, a difference of nearly 5 percentage points, but the difference is not significant5. In other words, theself-perceived change in attitude or behavior is not dependant on the school level. While we did not see a significant difference between these age ranges in their self-reporting, there was an idea put forward by SEED staff and some other coaches that it is more effective to intervene to support children during their most formative years (in primary and middle school) rather than waiting until high school. Conclusions LLP enabled the vast majority of participants to develop a range of positive behaviors and attitudes that are the building blocks for leadership and citizenship. Their self-confidence, capacity to take personal responsibility and exercise self-control increased alongside the ability to interact more calmly with peers, parents and teachers. Most have been able to set and achieve personal objectives and make 5 The confidence level of this difference (5%) has 95% of chance to fall in the confidence interval which is between [1% - 11%]. 38.6% 7.9% 22.3% 8.8% 3.6% 9.7% 9% 0.0 10.0 20.0 30.0 40.0 50.0 More confidence in myself Good public speaking While playing basketball Be leader (to manage) To trust others While studying at school To have good behavior LLP lets us better understand ourselves and the way that we should act in public and at home and how to better manage our emotions in specific situations — Focus Group Participant, LLP CFAK Club in Thiés Live, Learn and Play Final Performance Evaluation Report 14 positive contributions to their communities. They also learned to be members of a basketball team and to improve their basketball skills through commitment and practice. This pilot project has shown LLP to be a valuable and worthwhile model that brings about behavior change in an enjoyable and effective manner. It influences young people at a time in their lives when they are susceptible and can benefit from mentorship and guidance. LLP contributes to the personal and community development that is particularly important in view of the development challenges currently facing African youth. While this project has focused on personal development, citizenship and leadership the model could also serve as a platform for other important areas of learning for young people. These might include health, business skills, conflict resolution, savings and credit, rural /urban economies - there are many possibilities. Collective action, as seen in some LLP groups, could be encouraged and facilitated so that young people become of aware of and develop their potential as change agents. Question 2 How do local communities feel about LLP? Findings LLP defines “community” as the immediate environment surrounding a young person. The project’s ultimate goal relies on an approach that grants a crucial role to attaining the theoretical achievements of young people in leadership, especially in the area of citizenship in their community (neighborhood, school or other daily life settings). For this evaluation question, the team conducted focus groups and interviews with members of families (parents), school directors, teachers and coaches. This section also includes young people’s survey responses concerning their families’ perceptions of them. Parents and Families Family members of LLP youths described the project participants’ improved behavior at home, saying they exhibited a greater willingness to help with household chores and had more harmonious interactions and shared with siblings. LLP students were consistently described as highly motivated to achieve at school. One mother said: “My daughter has no time to get into any trouble, as when she’s not at school, she is playing basketball.” Some parents also spoke positively about aspects of personal development that they had observed in their children. During focus groups or interviews, many parents said they recognize that their children’s behavior has improved at home and in their communities. They are also more respectful and responsible with their school assignments. “I enrolled my daughter because she was fragile and always falling ill. She used to complain a lot at home, but with the positive influence of the coaches, this has really changed. I sense that she is really a leader among her friends, her school marks are good, her health is stable and she is like a different girl. I don’t know if it’s just her growing up or the club, but I am very satisfied.” — Father of LLP Basketball Club Participant BOPP, Dakar “My daughter shows respect for everyone, which is really different from my other children who don’t go to basketball. She has become really polite.” “LLP has brought joy to the hearts of children. They can look after themselves now, and when they go to practice, they are clean and polite.” “My children are well organized when they get back from training. They wash their kit, take a bath and do their homework.” — Parents of Students at LLP-Affiliated Schools in Dakar and Thiés Live, Learn and Play Final Performance Evaluation Report 15 Parents expressed an interest in being more involved in LLP and a willingness to provide support to the activities. In one discussion, parents agreed that they might organize to ensure that LLP players get home safely after a match. Some said they might provide food for the teams during tournaments. The youth were asked if they saw a change in the way their families spoke about them and treated them since participating in LLP. The majority of respondents (77.4 percent) said that LLP has changed the way their families talk about them. Figures 3 and 4 allow a better understanding of how youth feel their parents talk about them differently since LLP. FIGURE 3: BEFORE LLP, YOUTH SAID THEIR FAMILIES SPOKE ABOUT THEM AS... Source: MEP Senegal, April-May 2016 FIGURE 4: AFTER LLP, YOUTH SAID THEIR FAMILIES SPOKE ABOUT THEM AS … Source: MEP Senegal, April-May 2016 Figure 3 lists characteristics LLP youth said that their families attributed to them before they participated in the project, most commonly talking about them as inactive (32.7 percent) and violent and stubborn (27.8 percent), followed by other undesirable attributes such as timid and neurotic (14 percent) and disruptive (10.4 percent). Figure 4 details characteristics that youth reported their families using to refer to them after LLP: respectful (29.4 percent), very active (20.2 percent), responsible and brave (17.8 percent), kind, wise and gentle (11.7 percent) and open and supportive (10.9 percent). These findings suggest that LLP is the impetus for a virtuous circle, where young people hear good things about themselves, which reinforces their new behaviors. One group of parents from a rural LLP-affiliated school said they would like LLP to raise awareness about the effects of early marriage to counteract pressure from families and help young people, particularly girls, think about the issue. Coaches Like the parents, coaches expressed positive opinions regarding youth and changes in their behaviors and attitudes. Interviewed coaches said that LLP youth were aware of many things in relation to their 0.0% 10.0% 20.0% 30.0% 40.0% 32.7% 4.9% 14.0% 5.2% 27.8% 10.4% 0.5% 1.6% 2.9% 0.0% 10.0% 20.0% 30.0% 20.2% 3.7% 10.9% 29.4% 17.8% 11.7% 6.4% “Another positive aspect in terms of the youth is that we often organize a community service clean-up day, even though we don’t have an environmental club at the school. None of this is my idea, just the ideas of the youth. When I train them on the themes, they run with them, commit to them and take responsibility for them. So I feel that they understand what we are teaching and that's very important. One Saturday, I came as usual to the training practice and when I entered the school I saw that they had brought trash cans and placed them around the courts and I was very happy. I called them together and told them that they were heading in the right direction.” — Female LLP Coach. Malick Sy Middle School, Thiès Live, Learn and Play Final Performance Evaluation Report 16 behavior. They developed a good team on the field, exhibiting solidarity, and above all they learned respect for their studies. During the group interviews, several coaches noted that young people were making more of an effort in their studies due to LLP’s requirement that they maintain good grades as a condition of remaining in the project. However, coaches and parents alike also mentioned concerns about LLP. They noted the lack of medical insurance for the students and said the disclaimer that parents signed raised some fears. The evaluators heard of a few cases where LLP school students did not receive basketball shoes since none were available in their size. Some coaches and parents expressed that some youths do not always have time or resources to eat before practice or tournaments. In some rural areas, youth were potentially at risk when returning to isolated areas after late afternoon matches. Coaches recommended expanding LLP trainings through the school holidays to maintain continuity and support for the youth. Despite these concerns, coaches continually noted that more youth want to join the clubs than they can currently accommodate. Although LLP mainly attracts high achievers, several coaches mentioned support for particularly vulnerable children in their LLP groups. One coach described how she noticed one boy was always late for practice and found out that he was taking food to his father in the nearby prison every evening. She was able to help him talk about his situation and work out a different program so that he could arrive on time and deliver the food. A wheelchair-bound grandmother, who looked after her young grandson, described how he had been expelled from school and the coach had explained to him the relevance of education, found him a new school and encouraged him to work hard in order to continue as an LLP group member. Headmasters and Teachers The LLP community includes school staff. School directors and staff spoke positively about the effects of LLP on their students. Some school personnel said they would have liked more information and involvement, while recognizing that they were often too busy to take an active role. One school director mentioned seeing possibilities for approaching local businesses for sponsorship to help maintain the basketball court and provide equipment. In one case, a school director was waiting to hear when his school basketball court would be refurbished after being told that a contractor had been identified for the job. School staff noted that LLP has a positive influence on participants and their grades. School directors and coaches alike carefully monitored students’ grades and turned some youths away from practices when they received low grades. Conclusions The evaluation team heard numerous examples of children being transformed through their participation in LLP. The project is universally perceived at home and at school as having a positive effect on youth participants in terms of their performance in school, self-confidence, taking personal “I love basketball and I got my colleague to show me the LLP manual, which I found very interesting, including tips and suggestions that I can use in my English classes to motivate students and to encourage selflessness and the determination that a student needs to achieve results. Yes, I really like it.” — English Teacher, Montroll Middle School, Thiés “I’ve watched my daughter and the other LLP girls make extraordinary progress. Those who used to be shy can now speak in public and many no longer have weight problems. They are also doing well in class and are much better at basketball! LLP students from two schools formed an association called ‘Never Give Up’ and collected money from us all to donate to a local disabled center.” — JFK High School Director and LLP Parent, Dakar Live, Learn and Play Final Performance Evaluation Report 17 responsibility, assisting others, team solidarity and collective action. Many of these qualities are characteristics of effective leadership and good citizenship. Parents said they were pleased to be consulted during the evaluation and many expressed a wish to be better informed and more closely involved in the project. This suggests one untapped resource that could be accessed to support sustainability. In terms of effective project implementation, potential exists for improved communication between different actors, due to the number of organizations and people involved and the geographical spread. This is one principal future challenge, since after the pilot finishes, JSI will no longer have any role in coordinating activities in Dakar. Although the evaluation team did not hear about many examples of collective action to benefit the community, those that exist — such as the “Never Give Up” association formed by LLP youth — show that the potential is there. Several coaches talked about working with vulnerable children in their LLP group and some schools are in economically deprived districts. This suggests that while LLP students were often described as “the best students,” among them are vulnerable youth who have benefitted from mentoring from their coaches and support from the project. Question 3A What effect has LLP had on coaches’ capacity to work effectively? Findings As previously described, the LLP project works with three categories of coaches: SEED basketball coaches, PE teachers in LLP schools and trainee PE teachers from INSEPS. Evaluators defined two broad areas of effectiveness across all three categories. The first covers organisation, coordination and session planning so that an effective coach is one who arrives in good time, is well prepared with equipment and lesson plan and fulfils his/her role in the coaching team. The second area covers the capacity and skills to build effective relationships with all types of LLP youth, takng into account their particular and specific needs, strengths and weaknesses. SEED basketball coaches explained that even before the arrival of LLP, they practiced many of its principles, aiming to produce good leaders and citizens as well as good basketball players. However, LLP has enabled them to greatly improve their capacity for good organization and session planning, hence improving their efficiency and effectiveness in directly developing basketball, leadership and citizenship skills among LLP youth, training other coaches to do the same and coordinating project events. The LLP basketball manual that each coach receives is highly valued and coveted by non-LLP coaches. It provides access to not only the LLP curriculum but also to the rules of the game and descriptions of different ball control techniques. “During each LLP training session, we ask the players to cite three examples of the point we are teaching: one example from basketball, one from the street and one from school. If we are teaching personal commitment, someone might say ‘I commit to controlling the ball well and sharing it with the team’ or ‘I commit to doing my homework on time and as well as I can’ or ‘I commit to picking up litter and trying to keep my environment clean.’ We’ve seen enormous improvement in behavior with these types of commitments, which is very encouraging for us as coaches.” “I’ve learned to be a better citizen myself because of LLP. You can’t ask young people to behave in a way that you are not doing yourself, so I’ve changed my behavior to be a better model for them.” — SEED Coaches Live, Learn and Play Final Performance Evaluation Report 18 School PE teachers have expanded their coaching capacity, since they were not accustomed to integrating lessons on leadership and citizenship into their work and many were not basketball specialists. Some initial problems arose in establishing an effective working partnership between SEED basketball coaches and the school PE teachers, so that each had a clearly defined role. This has been largely resolved, often by PE teachers coaching one group of students while the head coach works with another; both coaches participate in the “take charge” sections of the session, which concentrate on personal development, leadership and citizenship issues. Some PE teachers said the travel stipend they received was not enough to compensate them for the time and effort they spent on LLP coaching, but did not indicate how much would be sufficient. SEED and JSI both described the efforts they made to explain the monetary incentives offered to full-time coaches compared to PE teachers, who receive a government salary as their main income. Some PE teachers also said that they felt isolated from the hub of the project and said that no regular forum exists for coaches to meet and share ideas. INSEPS trainee PE teachers have benefited not only from putting their LLP training into practice, but also from a refurbished basketball court. They shared their LLP experience with their peers during a presentation at INSEPS and the university director said he has been so impressed that, with SEED’s support, he intends to integrate LLP practice into the curriculum at this and other universities. Conclusions The LLP project has had a positive effect on coaches’ capacity to organize their work and integrate personal development teaching into basketball training. It has also counteracted a more “traditional” approach to coaching that could be verbally and even physically violent, in exchange for a more learner￾centered pedagogy. The project has largely succeeded in developing a working model to combine the strengths of SEED coaches and school PE teachers during training sessions. While some teachers unfortunately feel under￾rewarded for their efforts, the current arrangement seems reasonable since teachers receive a government salary in addition to their LLP stipend for working after school hours whereas professional SEED coaches receive more because this is their fulltime employment and main source of income. A forum for sharing ideas and difficulties might help wider understanding of the issues around financial motivation, counteract feelings of isolation and spread good practice and innovative ideas in terms of leadership and citizenship learning. Giving trainee teachers from INSEPS a role in the training not only contributed to their individual learning, but also ensured that INSEPS was involved in and aware of the LLP approach. This may prove to be the most effective long-term strategy for influencing future approaches to sport and coaching, “LLP has taught me a lot. I’ve been able to work with the coaches that trained us and gain knowledge about basketball, which is my passion. I’ve learned to be a better coach. The old methods we knew were quite violent toward the youth. To tell a young person that he is worthless and good for nothing can block his development. Since I’ve changed my language, I’ve found that my players do everything not to deceive me or each other and to see them work so hard to achieve their dreams is an extraordinary experience.” — School PE Teacher “This placement taught us about a coach’s attitude toward the learner and the importance of commitment, punctuality and tolerance. It helped us to recognize when a child is happy, when it’s not his day, how to listen when he needs to talk and how to help him develop himself and to aim high, to encourage him when he makes a mistake both on the court and off it. We’d like one more year with the same youth to know them even better and improve our know-how even further. Long live LLP!” — Excpert, INSEPS Trainee’s Placement Report Live, Learn and Play Final Performance Evaluation Report 19 particularly if the LLP “sport for development” model is more widely integrated into curricula for PE teachers across Senegal. Question 3B How has LLP been experienced differently by girls and boys and male and female coaches? Does LLP have other gender dimensions? Findings Equitable treatment of all children is a consistent focus in the implementation of LLP activities. All levels of implementation use this model, as identified in the coaches’ manual. Gender in the Enrollment of LLP Youth Beneficiaries in Schools The project chose equal enrollment numbers for girls and boys. The application of this enrollment approach ensured that each basketball team for each school comprised 18 girls and 18 boys, in mixed gender schools. The socio-demographic data collected during the school survey reflects this, as Table 1 shows. Table 1: Respondent analysis by sex Sex Number % Boys 237 47.5 Girls 262 52.5 Total 499 100.0 Source: MEP–Senegal evaluation, April–May 2016 During evaluation interviews with LLP youth, the team did not detect many significant differences in the LLP experience based on gender, which the survey analysis reflected. Boys and girls were equally willing and able to express their opinions in single- and in mixed-sex groups. One female coach remarked that, on enrollment, many girls did not have the same degree of ball sense as the boys did, and they took some time to develop this, which is probably a reflection of girls spending less leisure time playing ball games. Once girls developed their initial ball-handling skills, gender played an insignificant role in the skill level of participants. Several parents and teachers specifically mentioned girls’ improved health and fitness as a result of LLP. Changes in Attitude and Behavior As a result of participating in LLP, 94.7 percent of girls reported a change in their attitudes and behaviors, compared to 89 percent of boys — a difference of nearly 5 percentage points. However, in terms of achievement of their personal objectives, boys and girls exhibited no difference; 87.8 percent of each sex reported achieving of a personal objective. This refers to an aspect of the LLP approach which encourages participants to set personal objectives in relation to the personal development skills they are learning, and report back to their coaches. Live, Learn and Play Final Performance Evaluation Report 20 Changes in Family Perception A second dimension of changes included youths’ perception of the way their families see them. While a majority of the youths said their families speak about them more positively since they joined LLP, a significant difference emerged between girls and boys in answering yes to this question: 83 percent of girls compared to 71 percent of boys. Areas of change brought by participation in LLP varied by sex, with girls reporting an improvement in their energy levels and contribution to household chores and boys reporting an improvement in their attitude and level of respect for their parents, friends and their community. Youth Opinions and Attitudes on Gender Equality and Gender-Based Violence The survey does not attempt to measure any changes in opinions and attitudes related to gender. It asks a number of questions to broadly assess the perceptions, attitudes and beliefs about gender of the boys and girls concerned because this is an important aspect of leadership and citizenship. LLP treats girls and boys as individuals, according to their needs and models and encourages mutual respect and equal rights between girls and boys. The survey of youth involved in LLP in affiliated schools shows an understanding of the concept of equal rights for girls and boys across the majority of youth participants. The majority of the youths surveyed (97.6 percent) said that both women and men can be good leaders (which is not the same as saying that women can lead as well as men, because there was no idea of comparison in the questions that was asked, or any assumption that men are good leaders). Similar results were found in Boys Girls 89% 95% 11% 5% Has participation in LLP changed your behavior? Yes No Boys Girls 71% 83% 29% 17% Has participation in LLP changed the way your family members speak about you? No Yes Live, Learn and Play Final Performance Evaluation Report 21 terms of equal rights to accessing education, although 3% of girls as opposed to1% of boys thought that this was false. This may be due to misunderstanding the question or genuine belief, but we have no way of knowing since questionnaires were completed anonymously. The survey results demonstrate a majority belief among the youth to respect a girl’s decision when she says “no” to a sexual advance. For both girls and boys, more than nine of ten respondents said they believe that a boy should respect a girl’s decision when she says “no” to any type of sexual advance. This belief, if it is reinforced and established among youth, has the potential to prevent various types of daily violence, among which, sexual based violence constitutes a significant portion. Male and Female Coaches Only 16.6 percent of LLP-trained coaches are women. This reflects the predominance of male PE teachers and sports coaches in general in Senegal; however, INSEPS is promoting more gender balance among trainee PE teachers by encouraging and accepting female trainees. As LLP training sees gender equity as important, the Ministry of Education has offered to work with the program to offer a more in￾depth gender training for LLP coaches before the end of the project. SEED places importance on gender equity among its coaches and more than half of their lead coaches (four of seven) are women, but this Boys Girls 99% 97% 1% 3% Girls and boys have the same right to go to school True False 93% 95% 7% 5% Boys Girls If a girl says NO to a sexual advance, the boy should respect the decision. False True Live, Learn and Play Final Performance Evaluation Report 22 may be a reflection of the fact that basketball is the most popular sport for women in Senegal and the national women’s team has a strong history of success at the international level. Conclusions Gender is integrated across LLP activities and is seen as an importan aspect of the project, The strategies that LLP has put in place ensure that girls and boys have equal access to LLP activities. The survey tells us that the majority of youths have a basic understanding of gender related issues, a positive sign, since this is an essential component of good leadership and effective citizenship. LLP stresses the importance of mutual respect and models gender equity and equal access for girls and boys to education and sports. Question 4 To what extent has JSI contributed to building the institutional and organizational capacity of SEED? Findings USAID has used its standard Organizational Capacity Assessment (OCA) tool to assess SEED on three occasions. On the basis of each assessment, a work plan was developed to enable SEED to continuously build its capacity across different areas, with assistance by JSI. The OCA is a three-day process that examines eight areas of institutional or organizational capacity. It is scored on a scale from 1 to 4, with 4 representing best practice. A summary of SEED’s first and most recent OCA scores is in Figure 5. FIGURE 5: SEED’S OCA SCORES FROM AUGUST 2013 AND MARCH 2016 In all areas of institutional capacity, SEED has improved over the period between August 2013 and March 2016, with the greatest increase in its human resources management capacity. The SEED team recognizes its enormous progress since the start of the process, when it experienced some initial resistance and did not fully understand the need to do the work. In fact, the team has grown and changed; each team member now reports having a clear perception of the roles and responsibilities they fulfill. The entire team comes across as a dynamic, enthusiastic and committed group with a clear vision of where it wants to go. JSI expressed satisfaction with the work SEED has done and recognizes the improvement in SEED’s organizational and institutional capacity, although they both are aware that work 2 1.2 1.7 1.4 2.3 3.5 2.1 3.4 3.4 3.5 3.7 2.4 3 3.8 2.8 4 August 2013 March 2016 Live, Learn and Play Final Performance Evaluation Report 23 remains and further improvements can be made. SEED is committed to continuing to work through the end of the program towards getting closer to a score of “4” for each of the capacity areas. USAID is also pleased with the progress made and now considers that SEED could potentially qualify for direct funding from USAID provided that it keeps improving its scores in the lowest ramking categories. Currently, the lowest organizational capacity score relates to SEED’s financial management capacity. SEED has yet to undergo an external audit and USAID is committed to assisting them get prepared for it by conducting a financial review (similar to an audit) before the end of the project. Another area mentioned as benefiting from further work is procurement and the management of contractors. SEED plays a central role in the LLP project, which is only one part of its work; it also manages residential basketball training academies for girls and boys in Thiès and offers training for basketball coaches across the sub region. In addition to supporting capacity building, JSI has played a significant role in coordinating different project actors and activities, particularly those based in Dakar. Once the project ends, the coordination of project activities will fall primarily on SEED to continue. Conclusions JSI has made an enormous contribution to building SEED’s institutional and organizational capacity and the SEED team has worked hard and made impressive progress. Without doubt, SEED is continually increasing its capacity and its members have a lot of energy and great potential. The concern is that the considerable demands on this relatively young team will create enormous pressure and, without some ongoing support, the quality of the team’s work may suffer. Question 5 To what extent are LLP activities linked to “sport for development” sustainable? Findings A strong demand exists for LLP’s “sport for development” activities to continue in schools and basketball clubs. Basketball, particularly with the allure of Senegalese coaches having been part of the NBA, is highly motivational for youth in Senegal. The idea behind the pilot was based on combined public and private sector support for the activities. Initially, this PPP came from the U.S., but as the project developed in Senegal, interest from the U.S. decreased. At the outset of the project, the NBA was highly protective of its branding and image and insisted on taking the lead in additional private sector underwriting and support for the Senegal pilot. An ongoing issue is the fact that SEED has requested and been refused permission to use the SEED logo alondside those of the NBA, USAID and JSI on LLP products, apparently because of USAID’s branding policies. This makes it difficult for SEED to feel ownership of LLP and may affect the sustainability of the LLP brand once the project finishes. Commitments from the GOS and stakeholders are evident and they are keen for LLP to continue and expand within the middle schools and high schools across Senegal. Based on discussions with stakeholders, Table 2 outlines their commitments and interests in continuing LLP. TABLE 2: LLP STAKEHOLDER RESPONSIBILITIES AND COMMITMENTS Stakeholder Responsibility Commitment USAID/ Washington Signed initial LLP MOU with NBA LLP central funding mechanism ended in 2015. No commitment for future funding Live, Learn and Play Final Performance Evaluation Report 24 Stakeholder Responsibility Commitment JSI Lead international implementing partner for LLP Will no longer work on LLP beyond September, 2016 NBA Signed initial LLP MOU with USAID/Washington Developed and implemented youth coaches leadership training programs via basketball in Africa, including LLP Provide on-going in-kind and financial support to SEED USAID/Senegal Fund and oversee LLP pilot project Funding managed by Education Team will end in November, 2016. No commitment for future funding MEN/DEMSG Oversee management of middle schools and high schools; LLP TAG member Offered to provide trainers to support LLP coaches Ministry of Sports Responsible for overall national sports events and policies; member of LLP TAG Interested in supporting LLP in 2017 budget MJECC/DEP Oversee youth and general education policies and initiatives; LLP TAG member Interest in future support is unclear INSEPS Professional training of all physical education teachers in Senegal and surrounding countries; include PE teacher trainees in LLP school sites Interested in using the LLP coach training model as a standard part of PE teacher training. Interesteding in disseminating model to neighboring countries: Mali, Mauritania, the Gambia and Guinea Bissau SEED Implementer of basketball training programs for development and lead national implementer for LLP Interested in continuing LLP; will continue to receive in-kind equipment from NBA to support activities; interested in using current funding support to leverage additional funds for LLP National Basketball Federation Responsible for national basketball activities and coordination; TAG participant Strong advocate for LLP Parents Limited current role in LLP Some have expressed interested in a larger role Coaches Leaders in implementation of LLP leadership training approach Some expressed interest in further developing the coaches network to sport LLP activities School directors Oversee schools, staff including PE teachers and LLP activities Interest to continue LLP and potential to coordinate fundraising and local government and community support LLP was able to form partnerships with Orange and AMSA Assurance to organize summer basktetball camps, a model that could prove to be sustainable and continued to be implemented by SEED. NBA Africa is currently fundraising to providing ongoing financial support to SEED. With the departure of JSI, a shift must occur, redistributing responsibilities among Senegalese partners. A concern remains over how much financial support GOS entities can provide and whether SEED will require funding to cover its operational costs related to the coordination and management of LLP activities. Live, Learn and Play Final Performance Evaluation Report 25 LLP suffers from being a cross-sectoral project targeting youth and citizenship development. It does not fit neatly into a technical health, vocational training or basic education program. For this reason, USAID has a difficult time placing it within their existing funding streams. The model clearly is attractive to youth, and the significant personal development outcomes have the potential to enable youth to take on leadership and community engagement roles. Within its portfolio, SEED is exploring options to develop vocational training programs for youth and is in discussion with Orange to develop a training program on coding. Outside of vocational training, the lessons taught in LLP around responsibility and conscious control of oneself and one’s actions could easily translate into youth behavior change programs in reproductive health and family planning. The NBA in Africa see “sport for development” is an integral part of its work but believe that they can make a bigger impact by partnering with NGOs, governments and other corporations. NBA Africa’s Associate Vice President explained that a robust corporate environment is a necessary ingredient for funding sustainability and Senegal has this potential, but since the corporate base is less robust than other countries where the NBA attracts corporate sponsorship, such as Nigeria or Kenya, it takes longer to gain traction. The NBA is positive about finding ongoing funding to support SEED and its “sport for development” activities, through tapping both the marketing and corporate social responsibility funds of local and international companies. They believe this will be successful since corporates are interested in supporting successful initiatives that are up and running and have proved their worth. Potential partners include AMSA Assurance, Microsoft, Contour Global Power Company and Orange. The NBA in Africa is positive about its relationship with USAID Senegal and hopeful that ongoing support for SEED will prove possible. Live, Learn and Play Final Performance Evaluation Report 26 Conclusions The ongoing spread of “sport for development” activities is clearly a desirable outcome from this highly successful pilot project. Considering that it has been running for just over two years, the achievements are considerable. However, the current challenge is for the different stakeholder members of the technical assistance group to identify viable roles and responsibilities and for these to be successfully coordinated so that activities in schools and clubs can continue and develop new aspects. The most viable coordinating organization is SEED, as it has played this role during the pilot period, with support from JSI. However, in the absence of JSI, even with the possibility of more active buy-in from schools, SEED will have an increased coordination workload. Although SEED has radically increased its organizational and institutional capacity, it is still a young organization and it would certainly benefit from ongoing support to ensure a gradual transition to overall coordination of ongoing LLP activities. The team will need to assess any need for additional staff in the absence of JSI. With assured ongoing financial support, SEED is more likely to be able to meet its potential with regard to effective coordination of expanding LLP activities. Support from a major donor would also signal SEED’s credibility to other potential partners. Question 6 What lessons can be learned from strategies to develop public-private partnerships within the LLP framework? Findings LLP evolved from an idea discussed between personal contacts in USAID/Washington and the NBA in New York. It is one of a kind. JSI found itself charting new territory in facilitating a partnership where no accepted procedures or practices existed and the public and private partners had different values and different ways of working. Initially, USAID/Washington managed the U.S. components of the partnership while the NBA in New York and USAID/Senegal coordinated the African side along with the NBA Africa office, based in Johannesburg. As the pilot developed, the U.S. offices became progressively less involved and aspects of the original MOU never materialized, nor did any other pilots in Africa. The evaluation phone discussion with USAID’s Africa Bureau revealed that in Washington, with the LLP funding mechanism ending last year, the evaluation is seen as largely superfluous. USAID/Senegal noted that there was no clear PPP action plan developed with clearly defined roles and responsibilities for supporting LLP activities inSenegal. JSI helped the public and private partners find a common language and understanding and, to some extent, clarify their respective roles and responsibilities. They tackled this by looking for shared values and identifying the added value that each partner brought. An initial point of contention was USAID’s insistence on passing funding to SEED through JSI, and paying JSI to work with SEED to build its organizational and institutional capacity. The NBA was unfamiliar with the concept of organizational capacity building and thought the money would be better spent on direct implementation. However, with time, all partners have acknowledged and understood the value of the work with SEED. USAID has been somewhat frustrated by the NBA’s insistence that approaching private funders is solely the NBA’s role, particularly when the predicted levels of private funding did not materialize. As funding for the LLP pilot comes to an end, SEED must take on a larger level of coordination for the activity and review its ability to effectively manage LLP without JSI’s assistance. Live, Learn and Play Final Performance Evaluation Report 27 Conclusions PPPs typically include a level of marketing interest on the part of the private corporation and a level of brand visibility they want to see as a result of their contribution. However, for development purposes, PPPs need to make certain that local ownership is allowed and even in the forefront of project implementation. Branding needs to be clarified from the outset, making sure that local partner logos are included on all project materials. This will support a higher likelihood of long-term sustainability of activities. Funds leverage additional funds; this is a well-known fundraising approach. Fundraising and PPPs can occur on all levels — community, national and international. To have an effective PPP, fundraising at all levels is important, with implementing partners taking this on as part of their role in the PPP. Local partners have a greater chance of effectively fundraising locally if they can leverage the fact that they have international funds as part of building credibility for their activities. PPPs are often fairly broad in agreement, but need to be fine-tuned to clarify roles, responsibilities, expectations and values of each partner. JSI took a broad MOU and scope and effectively turned it into a productive partnership. This level of management, as well as the amount of time it takes in brokering between public and private parties, is important to consider in developing and managing future PPPs. RECOMMENDATIONS These recommendations are based on the findings and conclusions in this report and are targeted at the various key LLP stakeholders. SEED 1. Create a detailed, realistic transition plan and budget for SEED to take over coordination of LLP, with assumed participation and support from the Government of Senegal. 2. Provide opportunities for all coaches to meet regularly, share experiences and learn from each other. Quarterly meetings for coaches should enable them to share their experiences and ideas. This would help spread good initiatives, such as the redistribution of basketball shoes that participants have outgrown and group visits to the town hall to meet the mayor. Giving school PE teachers a project forum would show that the project values their contributions and would strengthen the link between LLP schools. 3. Provide opportunities for parents to be more involved in LLP activities. Schools should involve parents by asking school principals, PE teachers and LLP youth to call a parents’ meeting, share information about LLP and discuss experiences and perceptions of the project and ideas to improve it. This can help identify the role parents might play in implementing these ideas and possibly lead to an LLP parent support group in each school or club. 4. Consider how to share the initiative of LLP students forming an association and carrying out collective action. 5. Widen the circle of core partners with Senegalese private sector companies (Orange, Tigo, Expresso, Sococim) that would have interest in underwriting LLP activities. 6. Involve local governments in LLP activities, a possible avenue for political and financial support that school staff and parents can facilitate. 7. Organize the coaches’ gender training with the Ministry of Education (MEN) and the DEMSG before the end of the pilot project and use the collaboration with the DEMSG as an opportunity to investigate future possibilities for collaboration with the MEN. Live, Learn and Play Final Performance Evaluation Report 28 INSEPS 8. Integrate the LLP model into the PE teacher training curriculum. 9. Serve as an advocate. Work with SEED and government stakeholders to further the progress of “sport for development” in Senegal and other African countries Ministries of Education, Youth and Sport 10. Encourage greater involvement of and ownership by these ministries, with specific responsibilities of integrating the LLP model into public systems and programs, particularly across public schools and sports programs. LLP TAG Members 11. Work together to develop a transition plan. Clarify roles, responsibilities and commitments of all parties and establish a specific timeline. NBA Africa 12. Continue to support SEED in implementing “sport for development” in Senegal. 13. Continue to collaborate with USAID and other development partners as the opportunity arises. USAID 14. Support SEED’s ongoing capacity development, particularly in the area of accounting and management until it reaches a best-practice level of financial management (an OCA score of 4). 15. Provide a small development grant to SEED to support its transition plan to manage LLP. 16. Arrange briefings between SEED and other technical teams to explore the potential use of the LLP model in public health behavior change; vocation training/small business development; conflict resolution and school retention programs. 17. Explore making accomodation when possible to allow the co-branding of projects with local orgnaizations and seek a branding waiver to enable SEED to use its logo on LLP materials. 18. Review the branding regulations that prevent national implementing organizations from using their logo on project documentation and materials, since it is in direct contradiction with principles of local ownership. Live, Learn and Play Final Performance Evaluation Report 29 ANNEX I: EVALUATION STATEMENT OF WORK EVALUATION DES PERFORMANCES DU “LIVE, LEARN AND PLAY” CADRE DE TRAVAIL APRIL 12, 2016 This publication was produced for review by the United States Agency for International Development. It was prepared by Management Systems International, a Tetra Tech Company, for the USAID/Senegal Mission-Wide Monitoring and Evaluation Project. Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 30 EVALUATION DES PERFORMANCES DU “LIVE, LEARN AND PLAY” CADRE DE TRAVAIL Contracted under AID-685-C-15-00003 USAID/Senegal Mission-Wide Monitoring and Evaluation Project DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 31 ACRONYMES/LEXIQUE Activité Une activité est une sous-composante d’un projet et qui contribue à l’objectif de ce projet. Elle se réfère typiquement à un accord (tel un contrat ou un accord de coopération) ou une composante d’un projet telle que le dialogue politique qui peut être entrepris directement par le personnel de la mission (ADS 200.6). CDCS Country Development Cooperation Strategy CDT Cadre De Travail COR Contracting Officer’s Representative CV Curriculum Vitae DEC Development Experience Clearinghouse DEMSG Direction de l’Enseignement Moyen et Secondaire Général DPRE Direction de la Planification et de la Réforme de l’Education EPS Education Physique et Sportive INSEPS Institut National Supérieur de l’Education Populaire et du Sport IP Implementation Partners JSI John Snow, Inc. LLP Live, Learn and Play LOE Level of Effort M&E Monitoring and Evaluation MEN Ministère de l’Education Nationale MEP Monitoring and Evaluation Project MJECC Ministère de la Jeunesse, de l’Emploi et de la Construction Citoyenne NBA National Basket-ball Association OCA Organizational Capacity Assessment OCB Organisation Communautaire de Base ONG Organisation Non Gouvernementale PMP Performance Monitoring Plan PPP Public Private Partnership Projet Le projet est un ensemble d’activités exécuté durant un calendrier établi et un budget destiné à obtenir un résultat de développement (Ex : l’objectif du projet) par la résolution d’un problème associé. Il est explicitement lié au cadre des résultats du CDCS (Glossaire de l’ADS). S&E Suivi & Evaluation SEED Sports for Education and Economic Development SOW Statement of Work TPM Technical Planning Meeting USA United States of America USAID United States Agency International for Development YALI Young African Leaders Initiative Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 32 INFORMATIONS DE BASE Manager du CDT à l’USAID Sala Ba, COR, EDU COR de l’Activité Sala Ba Manager du CDT à MEP Moussa Sy, Spécialiste Senior en Evaluation Intitulé de l’Activité Live, Learn and Play Date de début et de fin de l’Activité 5 février 2014- 2 Février 2017 # Subvention/Contrat AID-685-A-14-00001 Montant du Financement $1,150,000 Partenaires d’exécution JSI Research and Training Institute, Inc. et Sports for Education et Economic Development (SEED) Zones d’intervention Dakar et Thiès Titre de l’évaluation Evaluation des performances du Live, Learn and Play Période à évaluer 5 février 2014 au 31 mars 2016 Date de début et fin de l’évaluation 05 avril 2016 – 31 mai 2016 CONTEXTE DE L’ACTIVITE L’activité Live, Learn and Play (LLP) a pour ambition de promouvoir le développement personnel des jeunes et de renforcer leurs compétences en leadership et citoyenneté, à travers le basketball. Active dans les régions de Dakar et de Thiès, le LLP est financé à hauteur de $1,150,000 par l’USAID et la National Basketball Association (NBA) et a une durée de 3 ans. Il est exécuté par John Snow Inc.(JSI), comme contactant principal et Sports for Education and Economic Development in Senegal (SEED), comme sous-contractant. Il s’inscrit en droite ligne avec l’une des trois composantes du Young African Leaders Initiative (YALI6 ) qui est axé sur le développement du leadership. L’activité LLP est une innovation en matière de promotion de la jeunesse à travers l’utilisation du basketball pour leur développement. A travers les coaches, LLP met le focus sur la promotion du leadership chez les jeunes filles et garçons, âgés entre 13-18 ans, scolarisés (collèges) ou déscolarisés. Il est attendu du LLP, au terme de 3 ans d’intervention, la formation de 720 jeunes et de 120 coaches et le renforcement des capacités institutionnelles de l’organisation Sénégalaise Sports for Education and Economic Development in Senegal (SEED). A terme, le LLP vise à faire émerger une nouvelle génération de jeunes leaders, acteurs du développement et capables de s’engager dans la vie citoyenne. Dans sa conception et sa mise en œuvre, le LLP a mobilisé des structures publiques et privées. A ce titre, la participation des ministères de la jeunesse et des sports, de l’éducation, des directions telles que la Direction de l’Enseignement Moyen Secondaire Général –(DEMSG)-, des structures comme l’Institut National Supérieur de l'Education Populaire et du Sport (INSEPS) a été notée. De même, des acteurs 6 Les trois composantes du YALI sont : i) Développement du leadership, ii) Dialogue et partenariat et iii) Développement de l’entreprenariat. Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 33 privés, comme Orange Sénégal, ont accompagné les activités du LLP, notamment dans l’organisation des camps d’entrainement de vacance qui ont touché un large public. L’évaluation des performances du LLP, après deux ans d’exécution, permettra de disposer d’un ensemble d’informations sur les progrès de l’activité et le niveau d’atteinte de ses objectifs. Dans la mesure où le LLP est une nouvelle activité pour l’USAID, son évaluation permettra d’identifier les forces et faiblesses, de formuler des recommandations et de mettre en exergue les leçons apprises. HYPOTHESE DE DEVELOPPEMENT ET SA MISE EN OEUVRE Pour l’USAID et la NBA, favoriser un esprit de leadership et de citoyenneté chez les jeunes à travers les coaches pourrait constituer un important facteur de développement personnel pour eux et pour leurs communautés d’appartenance. C’est à partir de ce postulat que le LLP a été élaboré pour utiliser le basketball comme source d’épanouissement physique et de divertissement mais aussi comme canal de transmission de valeurs et de compétences aux jeunes. Ce projet innovant intègre bien l’Objectif de Développement n°3 de la politique générale de l’USAID dans le Country Development Cooperation Strategy (CDCS) qui prône « Une participation citoyenne plus efficace dans la gestion des affaires publiques, aux niveaux national et local ». Dans la même perspective, le LLP contribue également à l’atteinte du résultat intermédiaire n°1 à savoir « une jeunesse mieux éduquée ». SOURCES D’INFORMATIONS EXISTANTES La liste (non exhaustive) des sources d’informations est la suivante : N° Type/Titre du document Date 1 LLP Coaching Manual 19 janvier 2013 2 Monitoring and Evaluation Framework and Performance Monitoring Plan Non précisé 3 Final LLP Work Plan Le 1 Mai 2014 4 LLP Implementation Plan Mars 2015 5 LLP Senegal Work Plan Septembre 2015 6 Revised LLP Senegal Work Plan Mars 2015 7 Revised LLP Implementation Plan Mars 2015 9 LLP Senegal Midline Assessment Report Janvier 2016 10 LLP Senegal Annual Report and July-Sept 2015 QR Octobre 2014-Septembre 2015 11 LLP Senegal Annual Report and Jan-March 2015 QR Février 2014- Mars 2015 12 LLP Senegal QR 3 2015 Final Report 1er avril au 30 juin 2015 13 LLP Senegal Quarterly 1 Report Oct - Dec 2015 Octobre à décembre 2015 14 LLP Program Description Non précisé Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 34 N° Type/Titre du document Date 15 Liste des Contacts de LLP (lycées, collèges avec les proviseurs, les principaux et les professeurs d’EPS) pour Dakar et Thiès. Non précisé 16 OCA Score Card Septembre 2015 17 OCA Evaluation Form 19 aout 2013 18 OCA Score Report 25 mars 2015 19 Live Learn and Play PMP 1er juin 2014 BUT, AUDIENCE ET UTILISATIONS PREVUES DES RESULTATS DE L’EVALUATION Le but de cette évaluation est de fournir à l’USAID/Sénégal et à l’USAID/Washington des informations sur les performances du LLP durant ses deux premières années d’exécution. Dans l’immédiat, les résultats de l’évaluation renseigneront surtout les progrès, les succès et les insuffisances du LLP auprès de ses différents cibles directs et/ou indirects (jeunes recrus, coaches, parents des inscrits au programme organisation locale). Le Sénégal, à travers le LLP, a servi de cadre d’expérimentation du modèle « Sports pour le développement ». Les conclusions et recommandations de l’évaluation de ses performances faciliteront le processus de décisions de l’agence sur la suite de l’Activité et sur des interventions futures similaires. De ce fait, cette évaluation des performances du LLP se situe à trois niveaux stratégiques de prise de décision. Son exécution est requise pour : Disposer d’informations sur les succès et limites des activités mises en œuvre ; Apprécier la pertinence de poursuivre l’appui à l’activité ; Renseigner sur les stratégies de pérennisation des acquis et du modèle LLP mis en place par le projet. Pour une bonne participation de toutes les parties prenantes (USAID, NBA, JSI, SEED, MEN, MJECC, ministère des sports, le corps de la paix des Etats-Unis, FSB, INSEPS et le conseiller spécial en sports de la présidence de la République du Sénégal), un atelier de présentation des résultats, conclusions et recommandations sera organisé après les enquêtes de terrain. Cet atelier permettra de recueillir les inputs sur les données de terrain en vue de les amender ou les renforcer. QUESTIONS DE L’EVALUATION Cette évaluation vise à répondre aux questions suivantes : Dans quelle mesure et comment l’activité LLP a développé les compétences des jeunes en leadership et citoyenneté ? Quelle est l’appréciation des communautés sur le LLP ? Quels sont les effets du LLP sur les capacités managériales des coaches ? Quelles sont les incidences du LLP sur la dimension genre (différence d’expérience entre jeunes filles/jeunes garçons, femmes/hommes coaches)? Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 35 Dans quelle mesure John Snow Inc (JSI) a contribué à renforcer les capacités institutionnelles et organisationnelles de SEED ? Dans quelle mesure la formulation et la mise en œuvre des activités liées au «Sport pour le développement » à travers le LLP sont-elles durables ? Quelles sont les leçons à tirer des stratégies de partenariat public privé (PPP) dans le cadre de LLP ? DESAGREGATION SELON LE GENRE ET EFFETS DIFFERENTIELS La prise en compte de la dimension genre dans cette évaluation s’inscrit en droite ligne avec les objectifs du projet. Dans sa conception, le LLP avait manifesté un grand intérêt pour l’adhésion des garçons mais également des filles dans les différents niveaux d’activités. Dès lors, les aspects genre seront intégrés dans cette évaluation à deux niveaux : i) une appréciation de la participation des jeunes filles dans le programme et ii) le niveau d’adhésion des coaches-femmes dans la formation. Sur cette base, l’élaboration des outils de collecte et d’analyse de cette évaluation mettra en avant une désagrégation entre genre (filles/garçons et hommes/femmes). L’analyse des résultats mettra aussi l’accent sur une comparaison de la perception et du vécu des interventions du LLP chez les jeunes garçons et chez les hommes coaches. La dimension genre est également prise en compte dans la formulation des questions d’évaluation, en l’occurrence les questions n°1 et n°3 où l’analyse comparera les expériences, les différences et les sensibilités selon le genre. DESIGN DE L’EVALUATION ET METHODOLOGIES La mise en œuvre de cette évaluation se fera en combinant les approches quantitatives et qualitatives pour saisir à la fois les niveaux de performance et les changements induits par le projet sur les attitudes, comportements et pratiques des cibles. Les opinions et appréciations des différentes parties prenantes seront également recueillies. La collecte des données et informations de l’évaluation se fera à travers trois étapes clés et complémentaires : revue et analyse documentaire, enquête quantitative (par questionnaire auprès des bénéficiaires du projet) et une enquête qualitative (entretiens, focus-group et observation) pour saisir les apports qualitatifs du projet. La revue/analyse documentaire portera sur les documents du projet (rapports mensuels/trimestriels/annuels du projet, etc.) pour apprécier et comparer le niveau d’exécution des activités et d’atteinte des cibles par rapport aux prévisions initiales. En complément de l’analyse de cette documentation, une collecte de données sur le terrain sera réalisée par le biais d’une enquête par questionnaire et la conduite d’entretiens individuels semi-structurés et de focus-groups auprès des jeunes filles et garçons, des enseignants et formateurs en contact avec ces jeunes et/ou impliqués dans le projet, de leurs parents et de leurs communautés. Pour les enquêtes qualitatives : A titre illustratif, pour combler les gaps identifiés lors de la première lecture des activités du LLP, des enquêtes qualitatives seront menées auprès des jeunes pour apprécier la portée et l’assimilation des apprentissages (leadership, citoyenneté, égalité des genres, participation communautaire). La pertinence de l’approche du modèle LLP, la qualité des performances Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 36 des jeunes et des coaches, la mise en pratique des acquis, la perception des parents, des communautés et des coaches sont autant d’indicateurs que les enquêtes vont interroger. Les données qualitatives se focaliseront également sur les perceptions et les avis des bénéficiaires à propos des activités menées par le LLP. Il s’agira de voir auprès des différents cibles (directs/indirects) les leçons tirées de l’expérience LLP, dans ses points forts et ses points faibles. Pour les enquêtes quantitatives : Les données quantitatives à recueillir comprendront, entre autres, l’âge des recrus, le nombre de séances d’entrainement aux quatre dernières semaines précédant l’enquête, le taux de participation des jeunes aux séances d’entrainement (pour mesurer l’assiduité des jeunes et la régularité des entrainements). Dans cette perspective, des outils spécifiques et adaptés seront élaborés pour chaque méthode de collecte et en fonction de chaque groupe-cible : quartiers et corps enseignant/administratif des écoles, jeunes, autorités de l’organisation SEED et d’autres partenaires stratégiques du LLP. Pour l’enquête quantitative auprès des jeunes, un échantillon représentatif de 40% des effectifs de chaque type de bénéficiaires enregistrés à ce jour, sera déterminé avec une marge d’erreur 5%. L’échantillon sera déterminé de manière aléatoire parmi les listes exhaustives de chaque type de bénéficiaires. L’ensemble de ces enquêtes de terrains seront effectuées par les consultants sélectionnés qui seront appuyés par deux enquêteurs qui vont les assister dans la collecte des données. Globalement, l’évaluation comprendra les étapes suivantes : Finalisation de la revue documentaire du projet (rapports de suivi trimestriels, état des lieux à mi￾parcours, plans annuels de travail, rapports annuels d’activités) ; Etablissement d’un échantillon représentatif des groupes cibles ; Identification des personnes-ressources pour les entretiens qualitatifs notamment JSI, NBA, USAID/Washington, SEED, INSEPS, les ministères et directions concernés et les bénéficiaires. (Interviews aux USA et au Sénégal) ; Elaboration des outils de collecte des données ; Organisation d’un Technical Planning Meeting (TPM) avec l’USAID, MEP/Sénégal et les consultants ; Conduite d’entretiens individuels et de focus groups avec les différentes parties prenantes ; Participation et observation de séances d’entrainements des jeunes et des coaches ; Analyse des données et interprétation des résultats (synthèse et organisation des résultats avec une ventilation par sexe) ; Rédaction du rapport préliminaire (à soumettre aux différents niveaux de validation) ; Atelier de validation des résultats, conclusions et recommandations de l’évaluation à l’USAID qui se chargera de sa communication aux différentes parties prenantes ; Finalisation du rapport de l’évaluation sur la base des observations de MEP/Sénégal, de l’USAID/Sénégal, JSI, SEED, NBA, du ministère de l’Education nationale (DEMSG) et de l’atelier de partage des résultats. ANALYSE DES DONNEES La première étape de l’analyse des données consistera à remplir la matrice des réponses, qui est un outil qui permettra d’identifier et d’associer à chaque type de données spécifiques collectées une méthode et des techniques d’analyse. Le contenu des données qualitatives transcrites et codifiées vont être analysées à l’aide de Sphinx, STATA ou SPSS, afin de dégager les grandes tendances d’appréciation des effets du programme sur les Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 37 bénéficiaires. Les données qualitatives issues des entrevues et des groupes de discussion appuieront les résultats provenant des enquêtes quantitatives. Pour trianguler les différentes informations, on procédera à une analyse comparative entre les données de base et les indicateurs de performances issues de l'évaluation. En plus de cela, une comparaison sera effectuée entre les réponses collectées auprès des différents groupes. Il s’agira spécifiquement de comparer et d’apprécier les différents types de données (quantitatives/qualitatives). Cette approche permettra à la fois la vérification et la validation de la cohérence des informations mais aussi l’appréciation des avis et perspectives de chaque groupe ciblé. Ce procédé va permettre de minimiser les biais et d’augmenter la validité des résultats pour chaque question d'évaluation. MEP/Sénégal travaillera avec l’équipe d’évaluation pour déterminer le type de graphiques et de supports à utiliser pour une meilleure visualisation des résultats de cette évaluation. FORCES ET LIMITES Une documentation assez fournie a permis d’assoir de bonnes bases préalables à l’évaluation. L’existence de documents du projet ont permis de mieux cerner ses contours, sa démarche et ses objectifs (rapports de suivi, baseline, midline, OCA, manuels des coaches, etc.). Une des forces de cette évaluation pourrait être la diversité des acteurs impliqués dans la planification et l’exécution du LLP. En effet, le LLP pour anticiper sur les perspectives d’appropriation et d’institutionnalisation du modèle « Sport pour le développement », a élargi sa vision du partenariat en direction du secteur privé et d’autres acteurs institutionnels. Ce qui va permettre de diversifier les sources de réponses aux questions d’évaluation et faciliter la triangulation des informations et leur validation. La composition des focus group par catégorie (groupe de jeunes ventilés par sexe et groupe spécifique aux enseignants, séparément des jeunes ou encore des coaches pour mettre à l’aise les participants dans la formulation de leurs réponses) constituera une force pour la justesse des données. Pour réduire les biais au niveau des outils (prise en charge de la dimension genre et perceptions des communautés), l’équipe d’évaluation va élaborer et adapter chaque type d’outil à un groupe cible. De plus la composition de l’équipe d’enquêteurs sera basée sur leur niveau de connaissances des programmes sport-développement et jeunesse et prendra en compte la dimension genre. Ceci sera ponctué par une approche de la collecte orientée vers le ciblage des communautés (parents des jeunes recrus, écoles, camps d’entrainements). LIVRABLES DE L’EVALUATION Les livrables attendus sont : Le cadre méthodologique (démarche, outils de collecte, méthodes de traitement et d’analyse) ; Briefing et exposé des aspects méthodologiques à l’équipe Education de l’USAID ; La base des données collectées auprès des jeunes recrus, des coaches et des communautés – y compris la base des données brutes avec le dictionnaire des données ; Le rapport préliminaire ; Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 38 Débriefing et présentation des résultats, conclusions et recommandations à l’intention de chaque partie prenante (USAID, partenaires publics et privés du projet, parents des jeunes recrus du projet, écoles et clubs) ; Le rapport final de l’évaluation et Le rapport de la rencontre de partage qui sera annexé au rapport final de l’évaluation. Il faut noter à ce niveau que les rapports, préliminaire et final, seront disponibles en version française et anglaise. COMPOSITION DE L’EQUIPE L’équipe d’évaluation sera composée d’un spécialiste en renforcement des capacités institutionnelles et des programmes éducatifs des jeunes, un Docteur en sociologie du développement et d’un associé en S&E du MEP qui va appuyer leur travail. L’équipe sera dirigée par Mme Susan Grace Upton, experte en évaluation participative et qualitative avec plus de vingt (20) années d’expériences dans le domaine des enfants, jeunes et populations vulnérables à travers l’Afrique de l’Ouest. Elle est spécialisée en évaluations et renforcement des capacités des ONG, OCB, communautés locales, régionales et internationales dans des pays de l’Afrique, de l’Amérique du Sud et de l’Asie. Susan Upton a travaillé dans une ONG malienne s’occupant des droits des jeunes et qui met l'accent sur le leadership des jeunes et de l'éducation par les pairs qui utilise le sport comme moyen de développement de la jeunesse. Le Dr Djibril Fall, appuiera la mission d’évaluation en tant qu’expert en développement de la jeunesse et en évaluation de programmes de développement pour les jeunes. Il totalise plus de vingt-deux (22) d’années d’expérience de travail dans le secteur de la recherche et de l’évaluation. M. Fall ainsi que Mme Upton ont une grande expérience nationale et internationale dans la conduite d’évaluation de programme de jeunesse et développement. L’équipe d’évaluation, en dehors de ces deux consultants précitées, sera accompagnée par deux enquêteurs qui vont les appuyer dans la collecte et la saisie des données qualitatives et quantitatives auprès de différentes cibles du programme LLP. Mr Moussa Sy, le Chargé de l’Evaluation du MEP, contrôlera la fiabilité et la validité des données et du rapport final. Mame Aïssatou Mbaye, associée en S&E du MEP, appuiera l’équipe dans la prise de notes, l’enregistrement audio et la documentation des travaux de terrain. Tous les membres de l’équipe fourniront une déclaration signée attestant qu’il n’y a aucun conflit d’intérêt ou décrieront tout conflit d’intérêt existant. La durée de la mission d’évaluation couvrira une période de 2 mois, à compter du 05 Avril, jusqu’au 31 Mai 2016. Dates Tâches/Livrables LOE Estimé(en jour) Consultant Sr International Consultant National Spécialiste Sr en Evaluation M&E Associé Chef du Projet 5-8 Avril, 2016 Revue documentaire 1 1 1 4 16 Avril 2016 Voyage international 1 0 0 0 18 Avril 2016 Réunion de planification Technique 1 1 1 1 19-21 Avril 2016 Développement des outils, des itinéraires, de la matrice des Réponses 3 3 2 2 1 Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 39 Dates Tâches/Livrables LOE Estimé(en jour) Consultant Sr International Consultant National Spécialiste Sr en Evaluation M&E Associé Chef du Projet 22 Avril 2016 Briefing initial avec l’équipe d’Education de l’USAID et LLP 1 1 0 1 23-30 Avril 2016 Interviews à Dakar et Thiès avec les parties prenantes 7 7 0 4 2-3 Mai 2016 Préparer les résultats, conclusions et recommandations 2 2 2 1 4 Mai 2016 Atelier de Présentation du cadre de résultats, conclusions et recommandations 1 1 1 1 5 Mai 2016 Voyage international 1 0 0 0 05-13 Mai 2016 Analyse des données et rédaction du rapport 7 6 0 4 16 Mai 2016 Revue des données et du draft du rapport 0 0 4 0 1 18 Mai 2016 Soumission du draft du rapport à l’USAID 1-9 Juin 2016 Finalisation du rapport 7 3 2 1 1 10 Juin 2016 Soumission du rapport final à l’USAID Total LOE Estimé 32 25 13 19 3 PARTICIPATION DE L’USAID ET PARTENAIRES Durant tout le processus d’évaluation, le bureau Education de l’USAID et l’équipe MEP/Sénégal, respectivement Mme Sala Ba et Mr Moussa Sy seront les points focaux. Ils travailleront en étroite collaboration avec les consultants en charge de l’exécution de la mission. Ce staff (USAID MEP/Sénégal) sera présent à chaque étape de la mission pour apprécier, orienter, valider et adopter les résultats soumis à leur expertise technique par l’équipe d’évaluation. D’autres partenaires, notamment la NBA, le JSI, le SEED, le MEN, le ministère de la jeunesse, de l’emploi et de la construction citoyenne, le ministère des sports, le corps de la paix des Etats-Unis, la Fédération Sénégalaise de Basket-ball (FSB), l’INSEPS ainsi que le conseiller spécial en sports de la présidence de la République du Sénégal seront également interviewés en tant que parties prenantes du LLP. De façon indicative, il est prévu une réunion entre l’équipe d’évaluation et l’équipe Education de l’USAID vers le 15 avril pour clarifier les objectifs de l’évaluation et sa démarche. Des interviews également auront lieu avec les parties prenantes dans la seconde quinzaine du mois d’Avril. MEP/Sénégal travaillera en étroite collaboration avec JSI et SEED pour la planification des activités. Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 40 PLANIFICATION ET LOGISTIQUES Cette évaluation des performances du LLP mobilisera, durant tout le processus, les équipes du bureau Education de l’USAID et de MEP/Sénégal. La programmation et la logistique (appui technique et managérial) seront gérées par l’équipe du MEP/Sénégal. MEP travaillera directement avec les IPs pour planifier toutes les réunions et activités de terrain. En collaboration avec MEP/Sénégal, l’USAID se chargera de planifier les réunions avec les Officiels. EXIGENCES POUR LE REPORTING Le canevas de l’USAID pour les évaluations sera utilisé pour développer le rapport (voir le modèle en annexe VI). Un rapport final d’un maximum de trente-cinq pages en anglais et français, les annexes non￾inclus, sera soumis à l’USAID/Sénégal. Les annexes comprendront entre autres, le Cadre de Travail de l’évaluation, un narratif complet de la méthodologie incluant les outils de collecte, le rapport de l’atelier de présentation des résultats, conclusions et recommandations ainsi que les instruments de collecte développés. Il faut préciser que le chef de l’équipe est tenu de rédiger les rapports (préliminaire et final) en anglais. Cependant, pour les besoins de partage avec les partenaires, ces rapports seront à leur tour traduits en français. Tous les livrables produits par l’équipe d’évaluation notamment les rapports seront soumis à l’appréciation de l’USAID/Sénégal, et partagés avec les autres partenaires, en consultation avec USAID/Sénégal DOCUMENTS DE REFERENCE ANNEXES X Calendrier d’exécution X Matrice de la revue documentaire X Matrice des résultats, conclusions et recommandations X Matrice des réponses X Budget X CVs Déclaration de conflit d’intérêt X Politique d’évaluation de l’USAID X Structure du rapport d’évaluation de l’USAID Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 41 AUTORISATIONS The undersigned hereby authorizes the following items (checked below) for the Statement of Work (SOW) described above: X Completion of the SOW, as described above; X SOW staffing, as described above; X Concurrence with Contracting Officer’s Travel Approval for the Consultant(s), requested above (if received prior to review). [COR to either sign below or indicate approval in a return email] __________________________________________ _____________________ Contracting Officer’s Representative (COR) Date Roy Geiser, or designate ______________________________________ ____________________ GDO Office Director, or designate Date 42 ANNEX II: PLAN DE TRAVAIL PLAN DE TRAVAIL ET METHODOLOGIE DE L’EVALUATION FINALE DES PERFORMANCE DU « LIVE, LEARN, PLAY » AVRIL 2016 Cette publication a été produit pour revue de l’USAID. Il a été préparé par Susan Upton, Souleymane Barry et Aïssatou Mbaye de Management Systems International. 43 PLAN DE TRAVAIL ET METHODOLOGIE DE L’EVALUATION DES PERFORMANCES DU « LIVE, LEARN AND PLAY » Contracted under AID-685-C-15-00003 USAID/Senegal Mission-Wide Monitoring and Evaluation Project PLAN DE TRAVAIL ET METHODOLOGIE DE L’EVALUATION FINALE DU « DISCLAIMER Live, Learn and Play » The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. 44 A.INTRODUCTION L’activité Live, Learn and Play (LLP) a pour ambition de promouvoir le développement personnel des jeunes et de renforcer leurs compétences en leadership et citoyenneté, à travers le basketball. Active dans les régions de Dakar et de Thiès, (avec des activités restreintes dans les régions de Kolda, Kafrine et Louga) le LLP est financé à hauteur de $1,150,000 par l’USAID et la National Basketball Association (NBA) et a une durée de 3 ans. Il est exécuté par John Snow Inc.(JSI), comme contractant principal et Sports for Education and Economic Development au Sénégal (SEED), comme sous-contractant. Il s’inscrit en droite ligne avec l’une des trois composantes du Young African Leaders Initiative (YALI7 ) qui est axé sur le développement du leadership. L’activité LLP est une innovation en matière de promotion de la jeunesse à travers l’utilisation du basketball pour leur développement. A travers les coaches, LLP met le focus sur la promotion du leadership chez les jeunes filles et garçons, âgés entre 13-18 ans, scolarisés (collèges) ou déscolarisés. Il est attendu du LLP, au terme de 3 ans d’intervention, la formation de 720 jeunes et de 120 coaches et le renforcement des capacités institutionnelles de l’organisation Sénégalaise Sports for Education and Economic Development in Senegal (SEED). A terme, le LLP vise à faire émerger une nouvelle génération de jeunes leaders, acteurs du développement et capables de s’engager dans la vie citoyenne. Dans sa conception et sa mise en œuvre, le LLP a mobilisé des structures publiques et privées. A ce titre, la participation des ministères de la jeunesse et des sports, de l’éducation, des directions telles que la Direction de l’Enseignement Moyen Secondaire Général –(DEMSG)-, des structures comme l’Institut National Supérieur de l'Education Populaire et du Sport (INSEPS) a été notée. De même, des acteurs privés, comme Orange Sénégal, ont accompagné les activités du LLP, notamment dans l’organisation des camps d’entrainement de vacance qui ont touché un large public. L’évaluation des performances du LLP, après deux ans d’exécution, permettra de disposer d’un ensemble d’informations sur les progrès de l’activité et le niveau d’atteinte de ses objectifs. Dans la mesure où le LLP est une nouvelle activité pour l’USAID, son évaluation permettra d’identifier les forces et faiblesses, de formuler des recommandations et de mettre en exergue les leçons apprises. Pour l’USAID et la NBA, favoriser un esprit de leadership et de citoyenneté chez les jeunes à travers les coaches pourrait constituer un important facteur de développement personnel pour eux et pour leurs communautés d’appartenance. C’est à partir de ce postulat que le LLP a été élaboré pour utiliser le basketball comme source d’épanouissement physique et de divertissement mais aussi comme canal de transmission de valeurs et de compétences aux jeunes. Ce projet innovant intègre bien l’Objectif de Développement n°3 de la politique générale de l’USAID dans le Country Development Cooperation Strategy (CDCS) qui prône « Une participation citoyenne plus efficace dans la gestion des affaires 7 Les trois composantes du YALI sont : i) Développement du leadership, ii) Dialogue et partenariat et iii) Développement de l’entreprenariat. 45 publiques, aux niveaux national et local ». Dans la même perspective, le LLP contribue également à l’atteinte du résultat intermédiaire n°1 à savoir « une jeunesse mieux éduquée ». Questions de l’évaluation Cette évaluation vise à répondre aux questions suivantes et nous expliquons ici comment nous comptons répondre à ces questions : 1. Dans quelle mesure et comment l’activité LLP a développé les compétences des jeunes en leadership et citoyenneté ? L’évaluation mènera une mini-enquête auprès des jeunes scolarisés du programme (288 filles et 288 garçons). L’enquête aura lieu dans les 9 écoles à Dakar et les 7 écoles à Thiès. Elle portera sur des changements aux niveaux des idées, des comportements et la confiance en soi des élèves concernés avec un accent sur la citoyenneté et le leadership. Pour accéder aux expériences des jeunes dans les 4 clubs de LLP (205 filles, 209 garçons) nous mènerons des focus groupes à Thiès et à Dakar. Pour chaque club nous comptons parler de 16 jeunes (8 garçons et 8 filles), donc un total de 64 jeunes. Les focus groupes aura lieux lors des séances d’entrainement de Basketball LLP, donc nous profiterons de l’occasion pour améliorer la base de données des participants LLP dans les clubs. Nous comptons savoir le nombre de jeunes non scolarisés ou déscolarisés et le nombre de jeunes travailleurs. Nous ferons des entretiens et/ou des focus groupes auprès des parents, des coaches, des enseignants et des principaux des écoles, selon leur disponibilité, afin d’écouter leurs observations par rapport à l’effet de LLP sur les jeunes participants. 2. Quelle est l’appréciation des communautés sur le LLP ? LLP a mis l’accent sur certaines personnes au sein des communautés des jeunes, surtout leurs parents, leurs enseignants et leurs coaches. Nous rencontrons un certain nombre de parents au niveau des écoles et des clubs afin de savoir leur appréciation de LLP. Selon le nombre de parents disponible nous utiliserons les entretiens individuels et les focus groupes pour faire ces échanges. Dans le même but, nous ferons des entretiens aussi avec des enseignants et des principaux de certaines écoles Si, en cours de nos échanges nous entendons parler des autres éléments des communautés des jeunes qui pourraient avoir une appréciation du projet LLP, nous ferons des efforts pour les rencontrer. 1. a) Quels sont les effets du LLP sur les capacités managériales des coaches ? Nous croyons que ce n’est pas seulement les capacités managériales des coaches qui sont importants, mais aussi leurs capacités d’établir des bon relations avec les joueurs et les joueuses, en terme de faciliter l’apprentissage et appuyer la confiance en soi, en plus de faire passer efficacement les messages de citoyenneté et de compétences de vie. Pour ce raison nous comptons aussi évaluer les effets du LLP sur ces capacités en plus de sur les capacités managériales des coaches. Nous le ferons à travers plusieurs outils : 46 1. Des entretiens et des focus groupes avec des coaches 2. L’enquête auprès des élèves et les focus groupes des jeunes dans les clubs 3. Nos échanges avec des parents et des personnels des écoles 4. Un entretien avec les personnel de SEED, la structure responsable pour la formation et gestion des coaches b) Quelles sont les incidences du LLP sur la dimension genre (différence d’expérience entre jeunes filles/jeunes garçons, femmes/hommes coaches)? La revue documentaire nous a permis de constater l’accent mis sur le genre par les différents éléments du projet, entre autres la formation et le manuel des coaches, la participation des filles et des garçons dans le projet et le recrutement des coaches hommes et des coaches femmes. Lors de l’évaluation nous demanderons 1. aux jeunes (à travers l’enquête et les focus groupes) de s’exprimer par rapport à leurs expériences et croyances et compréhension du concept genre. Des 8 focus groupes des jeunes, 2 seront uniquement les filles ; 2 seront uniquement des garçons ; et 4 seront des groupes mixtes. Cela va nous permettre à constater le niveau de confiance des jeunes dans les groupes unisexe et dans les groupes mixtes 2. aux coaches femmes et aux coaches hommes de parler de leurs expériences et de leur compréhension des aspects genre soulignés par le projet. Nous mettrons un accent sur les leçons qu’ils font passer lors des séances d’entrainement en termes de citoyenneté et en termes de leur comportement sur le terrain. Les données de l’évaluation seront désagrégés et rapportés par rapport au sexe des répondants (jeunes, parents, coaches…) 3. Dans quelle mesure John Snow Inc (JSI) a contribué à renforcer les capacités institutionnelles et organisationnelles de SEED ? Pour répondre à cette question nous se baserons sur les documents de l’OCA et des entretiens avec les personnels de JSI, SEED et l’USAID, les parties prenants clés dans ce travail. Nous demanderons à JSI de nous expliquer le travail qu’ils ont fait avec SEED, les résultats atteints, les défis, ce qui reste à faire et les propositions pour l’avenir. Nous demanderons également à SEED de nous expliquer les mêmes éléments et aussi à l’USAID de nous donner leur point de vue, comme c’est eux qui ont administré l’OCA. Cs échanges nous permettre de trianguler les informations reçues et faire un analyse de l’efficacité du travail de renforcement de capacité de SEED. 4. Dans quelle mesure la formulation et la mise en œuvre des activités liées au «Sport pour le développement » à travers le LLP sont-elles durables ? Nous répondrons à cette question à deux niveaux :  Sur la base des points de vues des parties prenants clés (JSI, SEED, NBA, les coaches, les écoles et le Groupe d’Appui Technique (TAG) du projet…) ramassés lors des entretiens et échanges de l’évaluation 47  Sur la base de notre analyse de l’approche LLP après avoir compris l’expérience du projet LLP au Sénégal 5. Quelles sont les leçons à tirer des stratégies de partenariat public privé (PPP) dans le cadre de LLP ? Les leçons à tirer porteront sur les stratégies de partenariat public privé proposés dans la description originale du projet LLP, le degré de réussite de ces stratégies, d’autres stratégies introduites en cours du projet et des propositions venant des parties prenantes pendant l’évaluation. Nous baserons nos analyses sur la description du projet et des entretiens avec des parties prenantes clés y compris JSI, NBA, USAID, SEED, le Groupe d’Appui Technique (TAG), Orange Sénégal. B. DESIGNE DE L’EVALUATION ET LA METHODOLOGIE DE COLLECTE DES DONNEES La mise en œuvre de cette évaluation se fera en combinant les approches quantitatives et qualitatives pour saisir à la fois les niveaux de performance et les changements induits par le projet sur les attitudes, comportements et pratiques des cibles. Les opinions et appréciations des différentes parties prenantes seront également recueillies. La collecte des données et informations de l’évaluation se fera à travers trois étapes complémentaires :  Le revue et analyse des documents du projet pour apprécier et comparer le niveau d’exécution des activités et d’atteinte des cibles par rapport aux prévisions initiales ;  Une enquête quantitative, par questionnaire auprès des jeunes élèves du projet ; Deux enquêteurs vont passer à tous les 9 écoles LLP. Dans chacun des écoles les 36 élèves LLP auront l’occasion à participer à la passation de l’outil qui sera fait de façon anonyme. L’enquête servira à apprécier la portée et l’assimilation des apprentissages (leadership, citoyenneté, égalité des genres, participation communautaire) et l’appréciation des jeunes de leurs coaches. L’enquête sera complétée par la base de données, qui nous permettre à savoir l’âge et le taux de participation aux séances d’entrainement des élèves.  Des enquêtes qualitatives, y compris des entretiens, des focus-group et l’observation structurée des séances d’entrainement de basketball pour saisir les apports qualitatifs du projet. Les expériences et points de vue des jeunes des clubs de basketball seront recueillis à travers la participation de 15.5% de ces jeunes (64 personnes) dans les focus groupes. Ces focus-groups serviront à apprécier la portée et l’assimilation des apprentissages (leadership, citoyenneté, égalité des genres, participation communautaire) et l’appréciation des jeunes de leurs coaches. Nous espérons également avoir l’occasion de déterminer les pourcentages des jeunes des clubs qui sont scolarisés ou non scolarisés. Nous avons décidé de ne pas faire l’enquête quantitative 48 auprès des jeunes des clubs parce que nous n’avons pas des informations sur leur niveau d’alphabétisation. En complément de l’enquête quantitative et des focus groupes des jeunes, il sera organisé des entretiens semi-structurés individuels et des focus-groups auprès des coaches et des parents, et des entretiens avec des informateurs clefs qui comprennent :  Des enseignants et principaux des écoles,  JSI et SEED, les structures principales de la mise en œuvre du projet  Le Groupe d’Appui Technique et l’INSEPS  Les partenaires privés telles que le NBA des États Unis, le NBA de Sénégal, Orange Sénégal… La pertinence de l’approche du modèle LLP, la qualité des performances des jeunes et des coaches, la mise en pratique des acquis, la perception des parents, des communautés et des coaches sont autant d’indicateurs que les enquêtes vont interroger. Les données qualitatives se focaliseront également sur les perceptions et les avis des bénéficiaires à propos des activités menées par le LLP. Il s’agira de voir auprès des différents cibles (directs/indirects) les leçons tirées de l’expérience LLP, dans ses points forts et ses points faibles. Les enquêtes qualitatives seront effectuées par les trois membres principaux de l’équipe de l’évaluation. Tableau des sources principales d’information pour répondre aux questions de l’évaluation Question d’évaluation Source des informations Jeunes Coaches Parents Ecoles SEED JSI TAG /INSEPS8 Privés Compétences des jeunes en leadership et citoyenneté X X X X X X Appréciation de LLP des communautés X X Capacités des coaches X X X X X X Dimension genre X X X X X X Renforcement de capacité de SEED X X X Durabilité X X X X 8Institut National Supérieur de l'Education Populaire et du Sport 49 Question d’évaluation Source des informations Jeunes Coaches Parents Ecoles SEED JSI TAG /INSEPS8 Privés Stratégies PPP X X X X Les différentes étapes de la méthodologie de l’évaluation sont les suivantes :  Rencontre de planification technique au niveau de MSI pour la mise au point des rôles et responsabilités  Remplissage initial de la Matrice des Réponses aux 6 questions de l’évaluation  Identification des sources des informations  Elaboration des outils de collecte des données  Programmation des visites terrain  Développement du plan de travail détaillé  Rencontre de planification technique avec l’USAID et MEP/Sénégal  Conduite de l’enquête quantitative auprès des jeunes élèves  Conduite d’entretiens individuels et de focus groups avec les différentes parties prenantes ;  Participation et observation de séances d’entrainements des jeunes et des coaches ;  Analyse des données et interprétation des résultats (synthèse et organisation des résultats avec une ventilation par sexe) ;  Préparation des résultats, conclusions et recommandations de l’évaluation  Présentation des résultats, conclusions et recommandations de l’évaluation à l’USAID ;  Rédaction du rapport préliminaire (à soumettre aux différents niveaux de validation) ;  Finalisation du rapport de l’évaluation sur la base des observations de MEP/Sénégal, de l’USAID/Sénégal, JSI, SEED, NBA, du ministère de l’Education nationale (DEMSG)  L’atelier de partage des résultats. D.METHODOLOGIE D’ANALYSE DES DONNEES Les données qui seront collectées à travers les trois (3) techniques (revue documentaire, enquête quantitative et enquête qualitative/observation structurée) sont de nature quantitatives et qualitatives. Analyse des données quantitatives Une partie des données quantitatives/statistiques seront collectées à partir des documents du projet (rapports, Compte-rendus, bases de données) et seront soumis à un traitement statistique qui permettra d’évaluer le niveau de couverture des cibles par le projet ainsi que les caractéristiques des bénéficiaires du projet. Les données de l’enquête quantitative seront saisies à l’aide du logiciel Sphinx. Un masque de saisie sera conçu à cette fin sur la base du questionnaire de l’étude validé. A la suite de la saisie, la base de données 50 sera apurée puis exportée vers SPSS pour les traitements statistiques. Les réalisations quantitatives du projet seront également considérées dans ce traitement statistique secondaire. Il s’agira principalement d’analyses univariées et bi-variées des données et statistiques du projet qui permettront également d’apprécier et de comparer le niveau d’exécution des activités et d’atteinte des cibles par rapport aux prévisions initiales. La seconde partie des données quantitatives sera fournie par l’enquête quantitative qui sera menée auprès des bénéficiaires dans l’ensemble des écoles couvertes à Dakar et Thiès. Les données issues de l’enquête auprès des élèves seront traitées à deux niveaux d’analyse : une analyse descriptive partant de la distribution des fréquences sur les caractéristiques des jeunes ayant bénéficié des interventions du projet. A partir d’une analyse des fréquences (basée sur des tris-à-plats des variables), les caractéristiques sociales et démographiques des bénéficiaires (âge, sexe, année d’enrôlement) seront explorées, permettant de fournir un tableau complet des bénéficiaires touchées par le projet. Cette analyse des fréquences permettra de présenter l’effectif des bénéficiaires touchés par région (Dakar, Thiès) ainsi que leurs répartitions selon l’âge, le sexe et le type d’établissement, éventuellement. L’analyse descriptive sera également utilisée par documenter les changements apportés par le projet chez les bénéficiaires à travers les réponses narratives et, au besoin, dans leurs familles à partir des produits des entretiens et focus-groups. Elle permettra ainsi de présenter les changements majeurs rapportés par les jeunes en comparant les périodes avant/après de leur participation au programme LLP. Un second niveau de traitement des données quantitatives utilisera les traitements bi-variées et/ou multivariées qui procéderont par un croisement des variables indépendantes (région, sexe, âge) aux différents changements rapportés par les jeunes à travers l’enquête par questionnaire. Analyse des données qualitatives Les données qualitatives tirées des entretiens semi-structurés et des focus-groups compléteront les données de l’enquête par questionnaire ainsi que les statistiques collectées des rapports du projet. La maitrise des valeurs et apprentissages reçus à travers les séances d’entrainement sur la citoyenneté, le leadership, l’égalité des genres et sur la participation communautaire seront évalués à travers les données tirées des entretiens et focus réalisés auprès des jeunes dans les clubs, des parents des bénéficiaires, des coaches et de toutes les autres parties prenantes du projet. L’analyse thématique sera utilisée pour le traitement initial des produits des entretiens et focus-groups. Elle permettra de compléter et d’enrichir sur les différents thèmes, les données descriptives issues de l’enquête par questionnaire. L’analyse de contenu complètera l’analyse thématique exploratoire et permettra d’aller au- delà de la simple description des pratiques et changements pour identifier les aspects les plus saillants ou illustratifs de ces changements. 51 Analyse des données d’observation Les données d’observation seront utilisées pour renforcer et compléter la description des pratiques aussi bien des élèves et des coaches, ainsi que leurs interactions in situ. Questions d’évaluation Techniques d’enquêtes Outils Types de données Plan d’analyse des données Commentaires Dans quelle mesure et comment l’activité LLP a développé les compétences des jeunes en leadership et citoyenneté ? Enquête par quest Entretiens SS Focus-groups Observation Questionnaire Guides ESS et focus Fiche d’observation Quanti / Statistiques Quali / Corpus Descriptives Analyses descriptives Univariées / bi-variées Analyse thématique et analyse de contenu Analyse des données quantitatives Analyse des fréquences qui permettra de décrire les caractéristiques sociales et démographiques des bénéficiaires (âge, sexe, année d’enrôlement). L’analyse des fréquences permettra également de présenter les changements majeurs rapportés par les jeunes en comparant les périodes avant/après de leur participation au programme LLP. En outre, une analyse bi-variée sera faite par le croisement de variables lourdes (âge, sexe, région) aux différents changements rapportés par les jeunes dans cette enquête. Quelle est l’appréciation des communautés sur le LLP ? Entretiens SS Focus-groups Guides ESS et focus Qualitatives Analyse thématique et analyse de contenu Quels sont les effets du LLP sur les capacités managériales des coaches ? Entretiens SS Focus-groups Observation Guides ESS et focus Fiche d’observation Qualitatives Descriptives Analyse thématique et analyse de contenu Analyses descriptives Quelles sont les incidences du LLP sur la dimension genre ? Entretiens SS Focus-groups Observation Guides ESS et focus Fiche d’observation Quantitatives Qualitatives Analyse thématique et analyse de contenu Analyses descriptives Analyse des données qualitatives L’analyse thématique sera utilisée pour le traitement initial des produits des entretiens et focus-groups. Elle permettra de compléter et d’enrichir sur les différents thèmes, les données descriptives issues de l’enquête par questionnaire. L’analyse de contenu complètera l’analyse thématique exploratoire et permettra d’aller au- delà de la simple description des pratiques et changements pour identifier les aspects les plus saillants ou illustratifs de ces changements. Dans quelle mesure JSI a-t-il contribué à renforcer les capacités institutionnelles et organisationnelles de SEED ? Entretiens SS Focus-group Guides ESS et focus Quantitatives Qualitatives Analyse thématique et analyse de contenu Analyses descriptives Dans quelle mesure la formulation et la mise en œuvre des activités liées au « Sport pour le développement » à travers le LLP sont-elles durables ? Entretiens SS Focus-groups Guides ESS et focus Qualitatives Analyse thématique et analyse de contenu Analyses descriptives 52 Questions d’évaluation Techniques d’enquêtes Outils Types de données Plan d’analyse des données Commentaires Quelles sont les leçons à tirer des stratégies de partenariat public privé (PPP) dans le cadre de LLP ? Entretiens SS Focus-groups Guides ESS et focus Qualitatives Analyse des données d’observation Les données d’observation seront utilisées pour renforcer et compléter la description des pratiques aussi bien des éléves et des coachs, ainsi que leurs interactions in situ. 53 E. PLAN DE TRAVAIL Le plan de travail a été conçu afin d’assurer que l’équipe puise fournira le rapport final le 2016. Cela ne permet aucun retard à différentes étapes de l’évaluation. Le programme pour la collecte des données sur le terrain : Jour Activités Lundi 18 avril Planification technique MSI Mardi 19 Développement des outils et la matrice des réponses Visite à JSI Dakar Mercredi 20 Développement des questions et logistiques pour l’enquête Observation d’une séance d’entrainement LLP Jeudi 21 Développement des guides d’entretien et les itinéraires pour les focus groupes et entretiens Production du plan de travail Vendredi 22 Briefing initial de l’équipe d’éducation de l’USAID Préparation du travail terrain – rencontre des enquêteurs Samedi 23 Observation de tournoi LLP Lundi 25 Préparation des enquêteurs Test de l’outil de l’enquête Mardi 26 Finalisation de l’outil et préparation des fiches de l’enquête Préparation des focus groupes Mercredi 27 Enquête Dakar 10h00 Entertain SEED Thiès 16h00 Club de Thiès Jeudi 28 Enquête Thiès 10h00 Coaches de Thiès Vendredi 29 Enquête Thiès 16h00 call JSI Washington Samedi 30 Enquête Dakar 10h00 Club ASFA 16h30 Club de Grand Yoff Dimanche 01 10h00 BOPP Dakar Lundi 02 mai Enregistrement et analyse des données Club de Dakar 4 ? Mardi 03 Enregistrement et analyse des données Entretiens à programmer : voie en bas Mercredi 04 Entretiens à programmer : Voir en bas 54 Jour Activités Jeudi 05 Entretiens à Programmer : Personnel des écoles, parents d’élèves ; coaches des écoles TAG, Orange, INSEPS. Fédération de Basketball Vendredi 06 mai Préparation des Résultats, Conclusions et Recommandations Samedi 07 Lundi 09 Atelier de présentation du cadre de Résultats, Conclusions et Recommandations Aux dates clés suivantes, l’équipe de l’évaluation fournira :  Le plan de travail, la méthodologie de l’évaluation et les outils le 22 avril;  Un briefing préliminaire le 09 mai 2016  Le projet du rapport le 23 mai 2016;  Le rapport final le 13 juin 2016 USAID fournira leur feedback et commentaires sur le projet du rapport le 06 juin 2016. Un chronogramme global des activités de l’évaluation est joint en Annexe A. 55 ANNEXE A: Chronogramme des activités de l’évaluation Etape du travail Période 17 - 23 avril 24-30 avril 01 -07 mai 08-14 mai 15-21 mai 22-28 mai 29 mai – 04 juin 05-11 juin 12-18 juin Réunion de planification Technique x Développement des outils, des itinéraires, de la matrice des Réponses x Briefing initial avec l’équipe d’Education de l’USAID et LLP x Interviews à Dakar et Thiès avec les parties prenantes x x Préparer les résultats, conclusions et recommandations x Atelier de Présentation du cadre de résultats, conclusions et recommandations x Analyse des données et rédaction du rapport x x Revue des données et du projet du rapport x Soumission du projet du rapport à l’USAID x Finalisation du rapport x Soumission du rapport final à l’USAID x 56 ANNEXE B: Matrice des Réponses Questions d’Evaluation Type de réponse/ Evidences nécessaire (Cochez un ou plusieurs, si approprié) Méthodes pour la Collecte des données, Ex. Enregistrement, Observation Structurée, Entrevues avec des informateurs clés, Mini-enquête9 Echantillonnage ou approche de la sélection, (si nécessaire) Méthode d’analyse des données. Ex. Méthodes d’analyse des données Fréquence des Distributions, Analyses des tendances, Cross￾Tabulations, Analyse des Contenus Source(s) de données Méthode 1) Dans quelle mesure et comment l’activité LLP a développé les compétences des jeunes en leadership et citoyenneté ? (Autres effets ?) Oui/Non Documents du projet : Revue documentaire des principaux rapports Utilisation des résultats des enquêtes du projet (comparaison) Revue des Manuels de formation Analyse des base de données Mini enquête dans tous les 9 écoles, avec ceux des 36 jeunes de LLP qui sont à l’école le jour de l’enquête Des focus groupes dans les 4 clubs de LLP Tableaux descriptifs : pour présenter les premiers résultats Analyse uni variée Analyse bi-variée Analyse multi-variée Analyse corrélative des variables : apprécier le lien entre les séances d’entrainement (contenue et régularité) avec les capacités des jeunes (leadership et citoyenneté). Analyse de contenu : les informations issues des entretiens individuels et focus-group. Présentation des tendances des performances des jeunes et des appréciations des X Description X Comparaison 10 X Explication11 Séances d’entrainement de BB coaching Observation structurée Jeunes à l’école : Questionnaire Mini-enquête Personnelle d’écoles : Guide d’entretien Interview Semi-Structuré (ISS) Jeunes aux clubs de basket￾ball : Guide d’entretien Focus groupe Parents : Guide d’entretien ISS / Focus groupe Coaches : Guide d’entretien ISS / Focus groupe Communautés : Guide d’entretien ISS / Focus groupe 9 Les données issues des évaluations sont des livrables et les méthodes doivent indiquées comment les données seront collectées, ex., pour les focus groups USAID requiers une transcription. 10 Comparison – to baselines, plans/targets, or to other standards or norms 11 Explanation – for questions that ask “why” or about the attribution of an effect to a specific intervention (causality) 57 Questions d’Evaluation Type de réponse/ Evidences nécessaire (Cochez un ou plusieurs, si approprié) Méthodes pour la Collecte des données, Ex. Enregistrement, Observation Structurée, Entrevues avec des informateurs clés, Mini-enquête9 Echantillonnage ou approche de la sélection, (si nécessaire) Méthode d’analyse des données. Ex. Méthodes d’analyse des données Fréquence des Distributions, Analyses des tendances, Cross￾Tabulations, Analyse des Contenus Source(s) de données Méthode SEED et JSI: Guide d’entretien Entrevues informateurs clés communautés (parents, coaches, communautés et écoles). 2) Quelle est l’appréciation des communautés sur le LLP ? Oui/Non Documents du projet : Revue documentaire des principaux rapports Soulignés des points significatifs X Description Comparaison X Explication Communautés (Parents et personnels des écoles): Guide d’entretien, ISS /Focus groupe des parents et personnels des écoles (autres s’il y en a) Tirage aléatoire ou choix raisonné Analyse des contenus /présentation des tendances 3a) Quels sont les effets du LLP sur les capacités managériales des coaches ? (Seulement managériales ??) Oui/Non Documents du projet : Revue documentaire des principaux rapports X Description X Comparaison X Explication Coaches : Guide d’entretien ISS / Focus groupe Jeunes à l’école : Questionnaire: Mini-enquête Jeunes aux clubs Guide d’entretien Focus groupe SEED : Guide d’entretien Entrevues informateurs clés Enseignants : Guide d’entretien ISS 58 Questions d’Evaluation Type de réponse/ Evidences nécessaire (Cochez un ou plusieurs, si approprié) Méthodes pour la Collecte des données, Ex. Enregistrement, Observation Structurée, Entrevues avec des informateurs clés, Mini-enquête9 Echantillonnage ou approche de la sélection, (si nécessaire) Méthode d’analyse des données. Ex. Méthodes d’analyse des données Fréquence des Distributions, Analyses des tendances, Cross￾Tabulations, Analyse des Contenus Source(s) de données Méthode Observation séance de BB Observation structuré 3b) Quelles sont les incidences du LLP sur la dimension genre (différence d’expérience entre jeunes filles/jeunes garçons, femmes/hommes coaches)? Oui/Non Documents du projet : Revue documentaire des principaux rapports X Description Comparaison X Explication Séances d’entrainement de BB coaching Observation structurée Jeunes à l’école : Questionnaire Mini-enquête Enseignants : Guide d’entretien Interview Semi-Structuré (ISS) Jeunes aux clubs de basket￾ball : Guide d’entretien Focus groupe Parents : Guide d’entretien ISS / Focus groupe Coaches : Guide d’entretien ISS / Focus groupe Communautés : Guide d’entretien ISS / Focus groupe SEED : Guide d’entretien Entrevues informateurs clés 59 Questions d’Evaluation Type de réponse/ Evidences nécessaire (Cochez un ou plusieurs, si approprié) Méthodes pour la Collecte des données, Ex. Enregistrement, Observation Structurée, Entrevues avec des informateurs clés, Mini-enquête9 Echantillonnage ou approche de la sélection, (si nécessaire) Méthode d’analyse des données. Ex. Méthodes d’analyse des données Fréquence des Distributions, Analyses des tendances, Cross￾Tabulations, Analyse des Contenus Source(s) de données Méthode JSI : Personnelle de la structure Guide d’entretien Entretiens informateurs clés, 4) Dans quelle mesure John Snow Inc (JSI) a contribué à renforcer les capacités institutionnelles et organisationnelles de SEED ? Oui/Non Documents du projet : Revue documentaire des principaux rapports Analyse des données de référence sur l’organisation, des résultats et scores enregistrés par l’OCA (entre le baseline et le dernier rapport d’activités). Analyse de contenus, grille d’analyse des capacités institutionnelles, organisationnelles, managériales, etc. Analyse comparative des performances de SEED avant et après l’intervention de JSI : X Description X Comparaison X Explication SEED : Personnelle de la structure. Guide d’entretien Entretiens informateurs clés, Observation Evaluation des performances institutionnelles et managériales de SEED JSI : Personnelle de la structure Guide d’entretien Entretiens informateurs clés, Observation Les stratégies de renforcement des capacités de SEED et les conclusions tirées de cette activité. 5) Dans quelle mesure la formulation et la mise en œuvre des activités liées au «Sport pour le développement » à travers le LLP sont-elles durables ? Oui/Non Documents du projet : Revue documentaire Analyse des principaux rapports Analyse des tendances Analyses des contenus X Description Comparaison X Explication Groupe d’Appui Technique (TAG) Guide d’entretien Entretien des informateurs clé 60 Questions d’Evaluation Type de réponse/ Evidences nécessaire (Cochez un ou plusieurs, si approprié) Méthodes pour la Collecte des données, Ex. Enregistrement, Observation Structurée, Entrevues avec des informateurs clés, Mini-enquête9 Echantillonnage ou approche de la sélection, (si nécessaire) Méthode d’analyse des données. Ex. Méthodes d’analyse des données Fréquence des Distributions, Analyses des tendances, Cross￾Tabulations, Analyse des Contenus Source(s) de données Méthode SEED : Personnelle de la structure. Guide d’entretien Entretiens informateurs clés, Observation JSI : Personnelle de la structure Guide d’entretien Coaches, clubs, ecoles, Entretiens informateurs clés, Observation 6) Quelles sont les leçons à tirer des stratégies de partenariat public privé (PPP) dans le cadre de LLP Oui/Non Documents du projet : Revue documentaire des principaux rapports PPP Coordination Template Analyses des réponses de différentes parties prenantes Description Comparaison Explication JSI : Personnelle de la structure Guide d’entretien Entretiens informateurs clés, SEED : Personnelle de la structure. Guide d’entretien Entretiens informateurs clés NBA/NY Association of Mayors of Senegal? Orange? AMSA Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 61 ANNEXE C: Rôles et Responsabilités des membres de l’équipe Personne / Role Responsabilités Sue Upton Lead evaluator (Consultant) Chef d’équipe pour 1. La conception des outils 2. Les travaux terrain 3. La préparation et présentation des résultats, conclusions et recommandations 4. La rédaction du rapport Aïssatou Mbaye MEP Data analyst and evaluation assistant Synthèse des documents Programmation des rencontres Appui dans la conception des outils Appui travaux terrain (focus groupes / entretien des informateurs clés) Analyse des données Souleymane Barry MEP Research Officer Appui dans la conception des outils Appui travaux terrain (focus groupes / entretien des informateurs clés) Analyse des données, particulièrement l’analyse quantitative Enquêteurs Deguene Diouf Oumar ANNE Passation de l’enquête auprès des élèves Agents de saisie Aichatou Laye Diouf Abdoulaye CISSE Penda Fall Mingue Diouf Saisie des données Statisticien Ibrahima MBALLO Apurement et traitements statistiques Analyse des données Moussa SY MEP Senior Evaluation Specialist Appui à la conception des outils et l’analyse des données Lisa Slifer-Mbacke MEP COP Communication USAID Facilitation et gestion du processus Assurance de qualité Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 62 ANNEX III: EVALUATION CALENDAR Day Activities Monday, April 18 Technical Planning Meeting MEP/MSI Tuesday, April 19 Development of the work plan and “Getting to Answers” matrix Discussion with JSI – Dakar Wednesday, April 20 Development of data collection tools Organization of interview schedules and survey dates Observation of basketball coaching session at Blaise Diagne school Thursday, April 21 Development of data collection tools Organization of interview schedules and survey dates Friday, April 22 Initial briefing with USAID Finalization of work plan, tools and schedules Monday, April 25 Preparation of survey researchers to use the survey questionnaire Test of the questionnaire at Yavuz Selim school Tuesday, April 26 Finalization of the questionnaire in the light of the test, Preparation of survey forms and interview guides Wednesday, April 27 Main evaluation team travels to Thiès Meeting with SEED team Interviews with youth, coach and parents at Thiès basketball club Questionnaire researchers visit schools in Dakar : Ahmet Sy Malick, David Diop, Talibou Dabo, Scat Urban Thursday, April 28 Group interview with LLP coaches in Thiès Interviews with parents, and school personnel at Montrolland middle school Questionnaire applied in schools in Dakar and Thies: ISM, Montrolland, Limo Rufisque, Malick Sy Friday, April 29 Writing weekly update report for USAID Questionnaire applied in schools in Thies: Diakhao Base, Ballabey, Yavuz Selim, RIAOM Saturday, April 30 Visits to Dakar basketball clubs, interviews with youth, parents, coaches Club ASFA Club de Grand Yoff Questionnaire applied in schools in Dakar: Kennedy, Blaise Diagne, Halwar Sunday, May 1 Visits to BOPP basketball club in Dakar, interviews with youth, parents, coaches Monday, May 2 Interviews with: Mrs Faye Beye Kassé DEMSG (Ministry of Education) Mamadou Fall, National Federation of Basketball; Phone call to JSI Washington Tuesday, May 3 Interviews with: Mr Haidera, Ministry of Education Mrs Diallo, Ministry of Sport Principal of Talibou Dabo school, other staff, parents, coach Sala Ba and Tidiane Wone USAID Wednesday, May 4 Interviews with: Djibril Seck, Director of INSEPS + INSEPS LLP interns Principal and coach at Kennedy school Thursday, May 5 Public holiday. Phone interview with ex COP for JSI Friday, May 6 Preparation of Results, Conclusions and Recommendations and Saturday, May 7 Preparation of presentation of initial findings to USAID and other stakeholders Monday, May 9 Presentation of initial findings to USAID and other stakeholders Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 63 ANNEX IV: DATA COLLECTION INSTRUMENTS SURVEY QUESTIONNAIRE Evaluation des performances du Projet Live, Learn & Play dans les régions de Dakar et Thiès QUESTIONNAIRE ELEVES Identification : Ecole : ……………………………………………………...Région …………………….. Date d'enquête |__|__|/|__|__|/|__|__|__|__| Nom de l'enquêteur/enquêtrice : ……………………………………………………………………… Passation du questionnaire : Heure début : |___|___| H |___|___| mn / Heure fin : |___|___|H |___|___| mn Demande de consentement éclairé Bonjour, je m’appelle…………………...……….., je travaille pour le compte de MEP-Sénégal basé à Dakar. Nous menons actuellement une étude pour évaluer les réalisations et résultats du projet Live, Learn &Play (LLP) auprès des jeunes ayant participé à ce projet. Ce projet avait mené diverses activités au profit des jeunes dans des écoles et clubs de basketball à Dakar et à Thiès. Nous sommes là pour rencontrer des jeunes et certains parents de ceux qui ont pu bénéficier des activités du projet dans le but de recueillir leurs opinions et appréciations sur les réalisations. Aussi, sommes-nous intéressés à recueillir vos avis sur ces questions afin de voir ce qu’il faudrait faire pour améliorer les choses dans le futur pour des projets similaires. Votre participation à cette étude est importante ; elle permettra à MEP-Sénégal et à l’USAID, dans le cadre de ses activités, de prendre en compte vos opinions et besoins pour pouvoir améliorer davantage les projets futurs. Cet entretien sera strictement anonyme et confidentiel. Votre nom ne sera mentionné nulle part. Vous êtes libres de participer ou de ne pas participer. Acceptez-vous de participer ? OUI  NON  Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 64  CARACTERISTIQUES SOCIODEMOGRAPHIQUES Q101. Sexe : Garçon  Fille  Q102. Age : |___|___| Q103. Classe : ……………………………………………….. Q104. Situation matrimoniale : Etes-vous marié (e) : Oui  Non  Q105. Année d’enrôlement dans le programme LLP 2014  2015  2016  Q106. Combien de séances d’entrainement avez-vous raté sur les quatre dernières séances d’entrainement ? 0  1  2  3  4  Si vous avez raté des séances pouvez-vous nous dire les raisons ……………………………………. I. CHANGEMENTS D’IDEES ET DE COMPORTMENTS Q201. Votre participation au projet LLP vous-a-t-elle permit d’apprendre quelque chose que vous ne saviez pas avant ? Oui  Non  Q201a. Si oui, veuillez donner un exemple concret en complétant la phrase suivant : Avant ma participation au programme LLP je ne savais pas l’importance de…………………………… ………………mais maintenant je crois que………………………………………. Q202.Votre participation au projet LLP a-t-elle changé quelque chose dans votre confiance en vous-même à faire des choses ? Oui  Non  Q202a. Si oui, veuillez donner un exemple concret en complétant la phrase suivante : Avant LLP je n’avais pas confiance en moi pour ………………………………………………….. mais depuis ma participation à LLP je ……………………………………………………………… ….. Q203. Est-ce que votre participation au projet LLP a changé quelque chose dans vos habitudes, même sur une seule chose ? Oui  Non  Q203a. Si oui, veuillez donner un exemple de chose en complétant la phrase suivante : Avant de participer à LLP j’avais l’habitude de………………………………………………………… mais maintenant je …………………………………………………………………………………………. Q204. Y’a-t-il un objectif personnel que vous avez atteint par votre participation au projet LLP ? Oui  Non  Q204a. Si oui, complétez la phrase suivante : Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 65 L’objectif personnel que j’ai atteint : …………………………………………………………………. Q204b. Complétez la phrase suivante : L’objectif personnel atteint dans le cadre de LLP dont je suis le plus fier :……………………………….. Q205. Y’a-t- il un objectif personnel fixé que vous n’avez pas atteint dans le cadre de votre participation à LLP ? Oui  Non  Q205a. Si oui, complétez la phrase suivante : Un objectif personnel que je n’ai pas atteint………………………………………………………….. …. Q206. Parmi les objectifs personnels (atteints ou pas) dans le cadre de LLP, y’a-t-il eu un objectif qui a été particulièrement difficile pour vous ? Oui  Non  Q206a. Si oui, complétez la phrase suivante : Mon objectif personnel qui a été le plus difficile ……….………………………………………. Q207 Votre participation au projet LLP a-t-elle changé quelque chose dans la façon dont des membres de votre famille parlent de vous ? Oui  Non  Q207a. Si oui, complétez la phrase suivante : Avant ma participation à LLP, des membres de ma famille disaient que j’étais …………………………. …………………………, mais maintenant ils disent que je suis……………………… Q208 Votre participation au projet LLP a-t-elle changé quelque chose dans la façon dont des membres de votre famille vous traitent ? Oui  Non  Q208a. Si oui, complétez la phrase suivante : Avant ma participation à LLP, des membres de ma famille ……………………………………………, mais maintenant ils ……………………………………………… Q209. Depuis votre intégration au projet LLP, vous est-il arrivé d’appuyer votre communauté (quartier, village, etc.) d’une manière quelconque ? Oui  Non  Q209a. Décrire selon votre cas personnel : i) Oui, j’ai appuyé ma communauté de la façon suivante …………………… ii) Non, je n’ai pas encore eu l’occasion d’appuyer ma communauté parce que …..………………………….…………….…………………………………………………….. Q210. Un bon citoyen aide les autres. Donner 3 autres caractéristiques de participation citoyenne : i) ………………………………..………….. ii) ……………………………………………. iii) …………………………………………………………………………………………… III. A PROPOS DE VOS COACHES LLP Q301. Mes coaches sont au nombre de : ………………………………. Répartis ainsi : Hommes…. …….. Femmes…………. Q302. Ce que j’aime le plus chez mes coaches …………………………………………………………… Q303. Ce que je n’aime pas chez mes coaches ……………………………………………………… Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 66 Q304. Un conseil que j’aimerai donner à mes coaches pour les aider à améliorer leur travail : …………………………………………………………………………………………………………….… Q305. Je note le coaching que j’ai reçu dans le cadre de LLP ……… / 10 (de 1 à 10 ou 1 est le plus faible et 10 et le meilleur) IV. INDIQUEZ SI VOUS TROUVEZ LES PHRASES SUIVANTES VRAIES OU FAUSSES Q401. La plupart des filles ont la même force physique que la plupart des garçons : Vrai  Faux  Q402. Les filles et les garçons ont le même droit à étudier : Vrai  Faux  Q403. Les hommes, les garçons, les femmes et les filles peuvent être de bons leaders : Vrai  Faux  Q404. La violence peut résoudre efficacement des problèmes au sein de la famille : Vrai  Faux  Q405. Si une fille dit « non » à une approche sexuelle, le garçon doit respecter son choix : Vrai  Faux  Q406. A travers mes expériences, je trouve que les filles sont en général favorisées par rapport aux garçons : Vrai  Faux  V. APPRECIATIONS GENERALES Q501. Qu’avez-vous le plus apprécié dans le programme LLP ? ……………………… Q502. Y’a-t-il d’autres changements importants pour vous que votre participation au programme LLP a apporté dans votre vie ? 1. 1 Oui  2 Non  Q502a. Si oui, lesquels …………………………………………………………………………………….. Q503. Avez-vous d’autres choses à ajouter Nous vous remercions beaucoup de nous avoir accordé de votre temps pour participer à cette discussion ! Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 67 QUALITATIVE INTERVIEWS TOOLS Comment avez-vous trouvé l’expérience de LLP ? Qu’est-ce que vous a plu ? Qu’est-ce que vous avez appris ? Est-ce que LLP vous a donné des nouvelles idées ? Lesquelles ? Est-ce que votre comportement à changer à cause de LLP ? Comment ? Croyez –vous que vos familles ou vos enseignements ont constatés des changements dans vos comportements ? Comment vous le connaissez ? Demander des questions d’approfondissement par rapport à la citoyenneté et le leadership, par exemple : Est-ce que vous pouvez nous donner des exemples du comportement d’un bon citoyen ou d’une bonne citoyenne ? Qui peut nous dire des exemples de ce qu’il ou elle a fait pour être un bon citoyen / une bonne citoyenne ? Est-ce que dans le cadre de LLP vous avez atteint des objectifs personnels ? Lesquelles ? Avez-vous eu des occasions d’appuyer votre communauté ? Comment ? Avez-vous plus de confiance en soi maintenant que vous aviez avant de participer à LLP ? Pourriez-vous nous donner des exemples ? Nous aimerions parler de vos coaches : Vous avez travaillé avec combien de caches dans le cadre de LLP ? Des hommes ou des femmes ? Qu’est-ce que vous avez aimé le plus de leur façon de travailler ? Est-ce que vous avez des conseils pour les coaches pour les aider à perfectionner leur travail ? Est-ce que vous aimeriez être comme ton coach ? Est-ce qu’il ou elle est un bon model ? Genre – Je vais vous dire des phrases et il faut dire si vous croyez que le phrase et vraie ou faux : La plupart des femmes ont la même force physique que la plupart des hommes Les filles et les garçons ont le même droit à étudier Les hommes, les garçons, les femmes et les filles peuvent être de bons leaders La violence peut résoudre efficacement des problèmes au sein de la famille Si une fille dit « non » à une approche sexuelle, le garçon doit respecter son choix Si une élève fait une approche sexuelle à un enseignant, l’enseignant doit dire « non » A travers mes expériences, je trouve que les filles sont en général défavorisées par rapport aux garçons Qu’avez-vous le plus apprécié dans le programme LLP ? Y’a-t-il d’autres changements importants pour vous que votre participation au programme LLP a apporté dans votre vie ? Avez-vous des questions pour nous ? Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 68 Guide d’entretien (individuel ou focus) avec parents d’enfants participants à LLP Comment comprenez-vous les objectifs du projet LLP ? Votre enfant participe à quelle régularité ? Est-ce que vous avez constaté des effets du projet sur lui ? Lesquelles (négatifs ou positifs)? S’il y en a, indiquez des exemples de votre enfant dans le rôle de leader et /ou de bon citoyen – peut-être des occasions ou il/elle a aidé sa communauté ? Est-ce que vous êtes en contact avec des coach de LLP ? Comment trouvez-vous leur travail ? Est-ce que vous avez des conseils pour eux/elles? Quelle est votre appréciation du projet LLP dans son ensemble ? Avez-vous des recommandations d’amélioration pour des futurs projets comme LLP ? Avez- vous d’autres choses à ajouter ? Avez-vous des questions pour nous ? Merci pour d’avoir pris le temps d’échanger avec nous Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 69 Guide d’entretien avec les coaches Comment voyez-vous l’idée de LLP à utiliser le basketball pour promouvoir la citoyenneté et le leadership ? A quel niveau LLP à marcher dans la pratique ? Comment avez-vous apprécié 1. la formation que vous avez reçu (combien de jours de formation, combien de fois ??) 2. le manuel de coach ? 3. les matérielles fourni par le projet : (ballons, maillot, …) 4. l’organisation du projet (SEED, écoles, club, JSI…) A quel niveau vos capacités de bien faire le travail du coach se sont améliorées à travers LLP ? Donnez des exemples concrets : Quelles activités avez-vous mené pour favoriser le développement de la citoyenneté au sein des jeunes ? Avez-vous vu des résultats de ces activités ? Lesquelles ?? Est-ce qu’il y a des différences entres les réactions des files et les réactions des garçons en termes de citoyenneté et leadership ? Comment avez-vous trouvez le travail avec les groupes mixtes ? Est-ce que votre participation à LLP a eu des effets sur vous-mêmes ? (renforcement de capacités, changement d’idées, changement de comportements…. ) Quel a été le plus grand défis dans votre participation de LLP ? Qu’est-ce qui t’a rendu le plus fier ? Comment les communautés plus larges ont vu le projet LLP ? Est-ce que vous avez entendu les populations parler de ce projet et si oui, qu’est-ce qu’elles ont dit ? Bientôt le financement de l’USAID prendra fin – comment voyez-vous les perspectifs et la durabilité des activités mises en place ? Avez-vous des suggestions pour améliorer n’importe quel aspect du LLP ou des futurs projets ? Avez- vous d’autres choses à ajouter ? Avez-vous des questions pour nous ? Composition du groupe Homme ou Femme Age Période avec LLP Formation LLP (Oui ou Non) Statut (Coach SEED, Prof Ed physique ou INSEPS) Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 70 Guide d’entretien avec personnels des établissements scolaires Comment voyez-vous l’idée de LLP à utiliser le basketball pour promouvoir la citoyenneté et le leadership ? Avez –vous constatez des changements de comportement des enfants qui participent dans le projet LLP à l’école ? Lesquels ? S’il y en a pourriez-vous indiquez des exemples des enfants dans le rôle de leader et /ou de bon citoyen A votre avis, quels aspects du projet LLP ont été les plus /les moins bénéfiques ? Pourquoi ? Avez-vous constaté des différences entres les réactions des files et les réactions de garçons au projet LLP? Comment les communautés plus larges ont vu le projet LLP ? Est-ce que vous avez entendu les populations parler de ce projet et si oui, qu’est-ce qu’elles ont dit ? Bientôt le financement de l’USAID prendra fin – comment voyez-vous les perspectifs et la durabilité des activités mises en place ? Avez-vous des suggestions pour améliorer n’importe quel aspect du LLP ou des futurs projets ? Avez- vous d’autres choses à ajouter ? Avez-vous des questions pour nous ? Merci d’avoir pris le temps d’échanger avec nous Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 71 GUIDE D’OBSERVATION STRUCTUREE DES SEANCES D’ENTRAINEMENT Observation des séances d’entrainement basketball Date: Ecole / Club: Coach: (H/F) Entrainment Type: Observateur: Element à observer Vu ou non-vu Commentaires Respect des étapes clés dans le manuel de coach : Cercle d’équipe initiale Exposé, démonstration d’un aspect de la citoyenneté Discussion en groupe des objectifs liés à la citoyenneté Exercices pratiques Cercle d’équipe finale Ambiance / relations entre les joueurs et le coach Gestions des situations conflictuelles Encouragements des joueurs en groupe Encouragement des joueurs individuels Plan de séance Registre de présence Nombre de coaches présents (hommes / femmes) (INSEPS / SEED/ école) Nombre de joueurs / joueuses Période de la séance Présence des parents / d’autres spectateurs Contrats des joueurs Fiches d’objectifs des joueurs Etat du terrain basketball Disponibilité d’eau à boire Répertoire de services sociaux et médicaux Coach bien organisé, bien équipé (ballons, un sifflet, pompe, accès à une trousse de réanimation cardiorespiratoire …) et ponctuel Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 72 ANNEX V: SOURCES OF INFORMATION Documents Reviewed MOU between NBA and USAID LLP program description and annual reports Project work plans Project Performance Monitoring Plan Project Midline Assessment Project beneficiaries’ data base LLP Coaches Manual LLP Training of Trainers Manual LLP mentorship strategy for coaches SEED OCA review documents and capacity development plan and work plans SEED weekly LLP updates Reports from 2 INSEPS LLP interns Key Informants USAID Sandy Oleksy-Ojikutu, Senior Education Advisor, Bureau for Africa, USAID, Washington Sala Ba, Deputy Education Team Leader Tidiane Wone, Financial Analyst NBA Africa John Manyo-Plange, Associate Vice President - Strategy & Operations JSI Abdoulaye Badiane, Admin and Finance Officer, Dakar Walsas Coulibaly, ex COP, Dakar Melissa Sharer, Program Director, Washington SEED Mohamed Niang, Executive Director Moussa Toure, Athletic Director Pape Abdou Karim Gueye, Project Manager Baye Maguette Seye, M&E officer Khadidiatou Natick Cisse Dieye, Administrative Assistant MamadouThia Diop, Youth program officer Ministry of Education M. Haidara Sherif, Director of Youth activities, Ministry of Education Mme. Diene Rokhaya Diouf, Office of Sport and Youth activities Mme. Faye Beye Kasse, Direction de l’Enseignement Moyen et Secondaire Générale Prof. Djibril Seck Ditrector INSEPS, University of Cheikh Anta Diop Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 73 Ministry of Sport Mme Diallo Marieme Kane, Director of training and sports development National Federation of Basketball Mamadou Fall, General Secretary, Senegal Basketball Federation The evaluation team has also consulted Youth: 499 school-based youth through the survey questionnaire 58 basketball club-based youth through focus group discussions: Basketball club Boys Girls Centre de Formation Abdou Konaré, Thiès 15-18 yrs 5 6 Centre de Formation Abdou Konaré, Thiès 12-14 yrs 5 3 Grand Yoff Club 13-18yrs 11 Grand Yoff Club 13-18yrs (4 over 18) 12 BOPP Club 9-13yrs 4 4 BOPP Club 14-18yrs 4 4 TOTAL 30 28 Coaches: Coaches Male Female 1 Idrissa Cissoko SEED/LLP, Thiès 2 Malick Sy SEED/LLP, Thiès 3 Ibrahima N’Diaye SEED/LLP, Thiès 4 Papa Diop Camera CEM Ballabey, Thiès 5 Papa Sanguéoul Pene Yavuz Selim, Thiès 6 Ndeye Marieme BA CEM Malik Sy, Thiès 7 Cherif Elvalid DIALLO CEM Montrolland, Thiès 8 Mohamed Cherif Cissé ASC BOPP, Dakar 9 Khadim Fam ASC BOPP, Dakar 10 Paul Diatta SEED/LLP, Dakar 11 Malick Gning SEED/LLP, Dakar 12 Ousseynou Kobe Coundoul SEED/LLP, Dakar 13 Ndeye Nancy Cisse SEED/ LLP, Dakar 14 Leopold F Xavier Medang CEM David Diop, Dakar 15 Bamba Cisse LycéeTalibou Dabo 16 Moussa Diame FAYE Blaise Diagne 17 Aboyse Lamine Traore INSEPS, Dakar 18 Aminata Mbengue INSEPS, Daker 19 Aimée Sogue INSEPS, Dakar 20 Anne Marie Dioh Centre de Formation AMD 21 Diane Stanishas Centre de Formation AMD Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 74 School Personnel M. Mbaye, Head teacher + 2 other staff, Lycée Talbou Dabou, Dakar Mme. Sarr, Head teacher Lycée Kennedy, Dakar M. Badji, Headteacher + 3 other staff, CEM Montrolland, Thies M. Diallo, Head teacher CEM Malick Sy, Thies M. Sylla, Head teacher CEM Ballebey, Thies Parents and Family School or Club Male Female Basketball Club, Thiès 2 fathers 2 mothers 1 grandmother CEM Montrolland 1 father 6 mothers 1 grandmother CEM Ballabey, Thiès 1 father 1 older brother 5 mothers 2 older sisters Basketball Club Granf Yoff, Daker 1 father, 1 older brother Basketball Club BOPP Dakar, 2 fathers 1 uncle Lycée Talibou Dabo, Dakar 1 mother TOTAL 10 18 Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 75 ANNEX VI: ACTIVITY PERFORMANCE MONITORING PLAN Narrative Summary Performance Indicator Disaggregat e by Means of verification Frequenc y Target Y1/FY1 4 Target Y2/FY15 Target Y3/FY1 6 LOP Total Goal: Young people become positively engaged in their communities Number of youth participating in community activities Sex, age - Youth baseline/ endline Year 1 and Year 3 240 480 720 720 Project Purpose: Senegalese youth demonstrate leadership, teamwork, and problem-solving capabilities Youth leaders emerge in participating schools and communities Sex, age - Youth baseline/ endline - Teacher endline - Coach endline Year 1 and Year 3 48 20% 96 20% 144 20% 144 20% Demonstration of youth skills and improved leadership Sex, age - Most significant change - Youth baseline/ endline - Teacher endline - Coach endline Annually 36 15% 72 15% 108 15% 108 15% Output 1: Coaches assume community leadership role and provide quality mentoring and leadership/life skills to youth (YALI goal) 1.1 Change in knowledge/skills among coaches Sex, age - Coach baseline/endline Year 1 and Year 3 31 80% 64 80% 96 80% 96 80% 1.2 Change in coaches’ ability to inspire, teach, and motivate youth to positively engage in their communities Sex, age - Coach baseline/endline - Youth endline - SEED Project observation Year 1 and Year 3 31 80% 64 80% 96 80% 96 80% 1.3: % and # of coaches’ who report capacity to organize, manage, and lead youth Sports for Development Program Sex, age - Coach endline Year 1 and Year 3 31 80% 64 80% 96 80% 96 80% 1.4: # of women who participate as coaches, trainers or mentors Sex, age - Training attendance sheets Quarterly 8 16 24 24 1.5: # of coaches who successfully complete training Sex, age - Post test results Annually 39 80 120 120 Output 2: Target youth acquire skills in leadership, teamwork and problem solving 2.1: Change in knowledge/skills/attitude among youth Sex, age - Most significant change - Youth baseline/ endline - Teacher endline - Coach endline - Parent endline Annually 192 80% 384 80% 576 80% 576 80% 2.2 Participation/continuation of youth in program Sex, age - Practice attendance sheets Quarterly 192 80% 384 80% 576 80% 578 Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 76 2.3: % and # of girls participating and number of hours/days attending LLP training programs Sex, age - Practice attendance sheets Quarterly 120 50% 240 50% 360 50% 360 50% 2.4: Youth recruitment and program participation process established Sex, age - Standard Operating Procedure developed Once Yes Yes Yes Yes 2.5: # of youth involved in the program Sex, age - Practice attendance sheets Quarterly 240 480 720 720 Output 3: NGOs viable/mature with improved institutional capacity 3.1: SEED Project applied new management practices - Endline OCA Annually 2 on OCA 2.5 on OCA 3 on OCA 3 on OCA 3.2: Amount of other donors financing mobilized by SEED Project - Endline OCA Annually 2 4 6 6 3.3: Value of incremental own-source financing each year - SEED Project Finance records Annually 20% 20% 20% 20% 3.4: Establishment of an internal governance (board of directors, SOPs, internal guidance, fundraising systems, etc.) by SEED Project - Endline OCA Annually 2 on OCA 2.5 on OCA 3 on OCA 3 on OCA 3.5: SEED Project passes USAID pre￾award survey - USAID pre-award survey results Year 3 Yes Yes 3.6: Increased capacity of schools/clubs/regional training centers to actively engage youth through sports - Most significant change (staff, inseps interviews) Year 3 Yes (Qual.) Yes (Qual.) Yes (Qual.) Yes (Qual.) 3.7: # of partners (private and public) who contribute money or other resources to the LLP partnership - Endline OCA Annually 2 4 6 6 3.8: Partners complete and regularly update PPP coordination template which details common goals/objectives and share decision-making and management responsibilities - PPP Coordination Template Annually Yes Yes Yes Yes Link with DG IRs Transparency DG1: Number of mechanisms/processes developed to promote or strengthen transparency and accountability in recruiting - Endline OCA HR of OCA 3, Management 3) - Standard Operating Procedure Annually 2 on OCA 2.5 on OCA 3 on OCA 3 on OCA Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 77 program beneficiaries and managing the program Civic Education DG2: Number of project beneficiaries engaged in civic education and community sensitization on key project thematic focus areas - Youth Endline Annually 180 360 540 540 Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 78 ANNEX VII: DISCLOSURE OF ANY CONFLICTS OF INTEREST [The Evaluation Policy requires that evaluation reports include a signed statement by each evaluation team member regarding any conflicts of interest. A suggested format is provided below.] Name Susan Upton Title Ms Organization MEP Evaluation Position? X Team Leader Team member Evaluation Award Number (contract or other instrument) USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) I have real or potential conflicts of interest to disclose. Yes X No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1.Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2.Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3.Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4.Current or previous work experience or seeking employment with the USAID operating unit Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 79 managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5.Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6.Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date 13th May 2016 Evaluation des Performances du “Live, Learn And Play”: Cadre de Travail 80 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 Tel: (202) 712-0000 Fax: (202) 216-3524 www.usaid.gov