|General Assembly||Distr.: General|
|4 April 2000|
Item 120 of the preliminary list*
Strengthening the role of evaluation findings in programme design, delivery and policy directives
Note by the Secretary-General
In conformity with paragraph 5 (e) (i) of General Assembly resolution 48/218 B of 29 July 1994, and with paragraph 7 of General Assembly resolution 54/244 of 23 December 1999, the Secretary-General has the honour to transmit herewith the report of the Office of Internal Oversight Services on strengthening the role of evaluation findings in programme design, delivery and policy directives. The Secretary-General concurs with the approach to programme monitoring and evaluation described in the report.
Report of the Office of Internal Oversight Services on strengthening the role
of evaluation findings in programme
design, delivery and policy directives
The present report reviews in-depth evaluation and self-evaluation activities in the United Nations during the period 1998-1999. The report highlights several developments in United Nations evaluation practices and concludes that the situation has improved significantly compared to that described two years ago. It includes background information needed by the Committee for Programme and Coordination and the General Assembly to assist them in the selection of topics for in-depth evaluation in 2002 and 2003.
|II. Central evaluation activities||
|III. Evaluation activities within departments and offices||
|IV. Conclusions and recommendations||
|Annex Evaluation reports completed or scheduled, 1992-2002||
1. The present report is the most recent in a series of reports submitted biennially to the General Assembly through the Committee for Programme and Coordination (CPC) on the strengthening of programme evaluation in the United Nations. Since late 1994, the central evaluation function has operated as part of the Office of Internal Oversight Services.
II. Central evaluation activities
A. In-depth evaluations
2. Reports on in-depth evaluations and related activities undertaken during the period 1992-2000 and those currently scheduled for 2001 and 2002 are listed in the annex to the present report.
B. Topics for future in-depth evaluation
3. The second topic for in-depth evaluation to be reviewed by the CPC in 2002 has yet to be determined. The following substantive activities have not been subject to or scheduled for in-depth evaluation during the period 1992-2001:
(a) Outer space affairs;
(b) Legal affairs.
The human settlements programme has also not been subject to evaluation in the period 1992-2000. However, the Office of Internal Oversight Services conducted an inspection review of the programme and administrative practices of the United Nations Centre for Human Settlements (UNCHS) in 1997 (see A/51/884), and in October 1999 the Office of Internal Oversight Services carried out a follow-up inspection. It assessed the progress made in implementing the previous recommendations, and whether the desired improvement in the financial and administrative management of UNCHS had begun to materialize. In this regard, particular attention was paid to the revitalization efforts begun in September 1998. This allowed the inspection to maintain a forward-looking profile and to provide a constructive input into making the revitalization more effective. A report on this follow-up inspection is before the General Assembly as document A/54/764.
C. Support to evaluation activities by departments
4. During the 1998-1999 biennium, the Central Evaluation Unit conducted one-week training workshops in the Economic and Social Commission for Asia and the Pacific, the Department of Economic and Social Affairs, the United Nations Environment Programme, the Economic and Social Commission for Western Asia and the Economic Commission for Africa Assessments by participants were solicited by questionnaire at the end of each workshop, covering such issues as timing, duration, group size, contents, material distributed and method of instruction. The assessments were positive.
III. Evaluation activities within departments and offices
A. Reviews conducted under the responsibility of the Office of the Secretary-General
5. During 1999 three high-level reviews of an evaluative nature were conducted under the responsibility of the Office of the Secretary-General, two at the request of the General Assembly and one at the initiative of the Secretary-General, with the endorsement of the Security Council. Reports on these reviews were issued in the last quarter of 1999:
(a) Report of the Secretary-General pursuant to General Assembly resolution 53/35. The fall of Srebrenica (A/54/549). The terms of reference of the review requested "a comprehensive report, including an assessment, on the events dating from the establishment of the safe area of Srebrenica until the endorsement of the Peace Agreement by the Security Council" (ibid., para. 1). The report concludes with a section on "Lessons for the future".
(b) Report of the Expert Group to Conduct a Review of the Effective Operation and Functioning of the International Tribunal for the Former Yugoslavia and the International Criminal Tribunal for Rwanda (A/54/634). The terms of reference of the review in General Assembly resolution 53/212, paragraph 5, request the Secretary-General to evaluate "the effective operation and functioning" of the two Tribunals "with the objective of ensuring the efficient use of the resources of the Tribunals". The Expert Group was chaired by the former President of the United Nations Administrative Tribunal and consisted of three other persons with high-level judicial experience and a former Under-Secretary-General for Administration and Management of the United Nations. The Group formulated conclusions on the effectiveness with which the Tribunals were "carrying out the missions mandated by the Security Council" (ibid., para. 260) and made 46 recommendations (ibid.).
(c) Report of the Independent Inquiry into the Actions of the United Nations during the 1994 Genocide in Rwanda. In a letter dated 18 March 1999 (S/1999/339), the Secretary-General informed the Security Council of his intention to appoint an independent inquiry into the actions of the United Nations during the 1994 genocide in Rwanda. In their reply (S/1999/340), the members of the Security Council expressed their support for the initiative. The Secretary-General appointed a former Prime Minister, a former Foreign Minister and a retired Lieutenant General to conduct the inquiry. The Independent Inquiry was given unrestricted access to all United Nations documentation and persons involved, and the mandate of "establishing the facts related to the response of the United Nations to the genocide in Rwanda" (see S/1999/1257, annex, p. 4). The Inquiry was to "establish a chronology of key events" and to "evaluate the mandate and resources of UNAMIR and how they affected the response of the United Nations to the events" and "to draw relevant conclusions and identify the lessons to be learned from the tragedy". The report was also issued on the United Nations web site at www.un.org/News/ossg/rwanda_report.htm and is dated 15 December 1999. It concludes with a section of "final observations" (ibid., p. 50) and 14 recommendations (ibid., p. 53).
B. Large-scale field operations
6. The Department of Peacekeeping Operations. A Lessons Learned Unit was established in April 1995 in response to recommendations included in the in-depth evaluations of the start-up phase of peacekeeping operations (E/AC.51/1994/3 and Corr.1, paras. 8-10, and E/AC.51/1995/2 and Corr.1, paras. 16-18). A review by the Office of Internal Oversight Services of the implementation of those recommendations is contained in documents E/AC.51/1998/4 and Corr.1 and E/AC.51/1999/5.
7. Major lessons learned studies in 1998 and 1999 covered the following: the United Nations Mission in Haiti (UNMIH/UNSMIH/UNTMIH); the United Nations Angola Verification Mission (UNAVEM); the United Nations Transitional Administration in Eastern Slavonia, Baranja and Eastern Sirmium (UNTAES); cooperation between the United Nations and regional organizations in a peacekeeping environment, focusing on United Nations cooperation with the Organization of African Unity/Economic Community of West African States in Liberia, the Organization of American States in Haiti, the Organization for Security and Cooperation in Europe in Eastern Slavonia, the North Atlantic Treaty Organization in Bosnia, and the Commonwealth of Independent States in Georgia and Tajikistan; and disarmament, demobilization and reintegration of ex-combatants in a peacekeeping environment, leading to the establishment of principles and guidelines for future disarmament, demobilization and reintegration operations. In addition, the Lessons Learned Unit reviewed the implementation of lessons learned from peacekeeping operations; this study considered the extent to which lessons already derived from previous operations were being implemented in the field; case studies included Angola, Georgia and Sierra Leone.
8. United Nations High Commissioner for Refugees (UNHCR). In the second half of 1998, a review of all aspects of UNHCRs evaluation system led to a number of significant organizational changes. The evaluation function was separated from the inspection function and transferred in September 1999 to a new Evaluation and Policy Analysis Unit, reporting to the Assistant High Commissioner for Refugees, who also heads the organizations Department of Operations. In the context of this reorganization, two additional Professional posts were assigned to the evaluation and policy analysis function. The consultancy budget available to the evaluation function was also augmented by means of a financial transfer from the former Policy Research Unit. UNHCRs evaluation efforts continued throughout the reorganization process. Evaluations carried out over the past two years have included a focus on major activities in large operations, the analysis of regional strategies and thematic evaluations: review of UNHCRs housing programme in Bosnia and Herzegovina; review of problems of access to land and its ownership in repatriation operations; review of UNHCRs strategy in the European Union; review of the Mali/Niger repatriation operation; review of rehabilitation activities in the Great Lakes region of Africa; a survey of compliance with policies on refugee women, children and the environment, and a review of staff training. At the close of the reporting period, the following projects were in progress: an independent evaluation of emergency preparedness and response in the Kosovo refugee crisis; an interim assessment of the implementation of the programme of action of the Commonwealth of Independent States conference process; an interim assessment of the implementation of policy in relation to refugees in urban areas; a review of UNHCRs role in situations of internal displacement; and a review of policy in relation to the security and neutrality of refugee and returnee-populated areas.
9. United Nations Relief and Works Agency for Palestine Refugees in the Near East (UNRWA). As an operational agency in the field, UNRWA carries out technical evaluations within its programmes and subprogrammes in education, health and relief and social services. Evaluations for the period 1998-1999 included assessments of the quality of child healthcare services; assessment of the implementation of the directly observed short-course treatment strategy for the control of tuberculosis; the quality of non-communicable disease care in Lebanon; the quality of diabetes care in the Gaza Strip; workload measurement and cost-benefit analysis of laboratory services; a nutrition and anaemia survey in Gaza; assessment of the Agencys hospitalization services; and of maternal health-care services. Assessments in progress include: the current prescribing practices of anti-bacterial drugs; the Agencys antenatal risk-scoring system; the prevalence of diabetes mellitus and hypertension; and the computer skills of health personnel; assessment of the implementation of the youth-centred health education programme on prevention of tobacco use and review of the health information system. In the field of education, UNRWA conducted several technical evaluations during 1998-1999 related to: monitoring learning achievement, drop-out rates, developing quality indicators and criteria for standards of formal education, integration between pre-service and in-service teacher education and the education biennial work plan.
C. Other substantive activities
10. Department of Political Affairs. The Department carried out a comprehensive self-evaluation study during the second half of 1999. For that purpose, it engaged the services of the Management Audit and Consulting Section of the Audit and Management Consulting Division of the Office of Internal Oversight Services. The core issues identified related to the Departments organizational structure, job satisfaction and feedback, sharing of information, and information technology integration. Recommendations were made on how to address those issues, the selective implementation of which is under internal consideration.
11. Office for the Coordination of Humanitarian Affairs. The Office for the Coordination of Humanitarian Affairs twice a year holds a Senior Management Retreat, where progress made against the work plan is evaluated. The results are taken into account in the preparation of the work plan and the identification of priorities for the following semester. At the Headquarters level, the Evaluation and Studies Unit undertakes in-depth reviews and evaluations of specific issues, situations or programmes. At the field level, an evaluation of the Office for the Coordination of Humanitarian Affairs Nairobi office, including the Integrated Regional Information Network (IRIN) programme, was conducted in 1999. In the coming year, the Office for the Coordination of Humanitarian Affairs plans to carry out evaluations of its field operations and headquarters activities in a more systematic manner. For this purpose, a Funding and Project Review Group was established in September 1999, which will commission reviews and evaluations of the activities of the Office for the Coordination of Humanitarian Affairs.
12. United Nations Drug Control Programme (UNDCP). A total of 35 evaluations were undertaken during the 1998-1999 biennium of technical cooperation activities supported by the Fund of UNDCP. This included two thematic evaluations as well as 33 project evaluations covering 37 individual projects. There were 2 thematic and 2 project evaluations that were global in scope, while 5 project evaluations were conducted in Africa, 7 in Asia, 4 in Central and Eastern Europe (covering 8 projects) and 15 in Latin America and the Caribbean. In terms of thematic areas, 17 evaluations were done of projects designed to prevent and reduce drug abuse, nearly one half of the total. Nine project evaluations (plus one thematic evaluation) were conducted on cooperation in support of institutions involved in the fight against illicit drug trafficking. Four project evaluations (and one thematic evaluation) were done in the drug control policy support, legislation and advocacy thematic area. Three evaluations of projects to eliminate illicit drug crops through alternative development were completed.
13. United Nations High Commissioner for Human Rights (OHCHR). During the 1998-1999 biennium, OHCHR went through an extended period of restructuring and reform. Critical areas of operations came under intense scrutiny. In October 1999, the Office of Internal Oversight Services undertook an evaluation of OHCHRs management and financial controls and related procedures and the Department of Management undertook a management review of the administrative sector. An independent review of the processing of voluntary contributions was also commissioned. OHCHR is now in the process of implementing the recommendations emanating from these reviews. Reviews on arrangements and policies to support field presence included: a review by an outside consultant on the role, functioning and support needs of the field operations of the OHCHR; an internal review of the Memorandum of Understanding the Office has with partners; and an Office of Internal Oversight Services audit review. A study, supported by the United Nations Fund for International Partnerships (UNFIP) was commissioned to review the functioning of the system of special procedures. A study was also prepared on the staffing needs associated with servicing the human rights treaty bodies; the results were submitted to the annual meeting of chairpersons of treaty bodies. Sectoral reviews were undertaken of the evolving practice of establishing field offices, and policies and practices of technical cooperation in the field of human rights.
14. The United Nations Environment Programme (UNEP). UNEP issues a detailed annual report on its evaluation activities. The annual evaluation report is a requirement of the UNEP Governing Council and as such it is submitted to the Governing Council as an inter-sessional Governing Council document. The 1998 report (dated April 1999) summarized the results of 13 in-depth project evaluations and 77 self-evaluation fact sheets submitted to the Evaluation and Oversight Unit for the 1998 exercise. The 13 projects subjected to in-depth evaluations were: the Global Environment Facility (GEF)/UNEP project on national communication on climate change; biodiversity data management capacitation in developing countries, with 9 of its associated projects; and the country case studies on climate change impacts and adaptations assessment. The average time taken for in-depth evaluations was four weeks, and they were carried out by consultants at an average cost of US$ 8,275 each. In-depth evaluations are carried out by UNEP on projects that have a project funding value of over US$ 500,000, and on all pilot projects, at the end of the project, at the end of a phase or when substantial structural changes are to be made to the project. Each annual evaluation sought to establish whether the projects relevance, effectiveness, efficiency and impact justified the investment by UNEP. The studies also examined whether the projects were sustainable and whether they still provided a catalytic path for UNEP to further its environment mission. For UNEPs strategic management, the evaluations sought to establish the key issues emerging from the projects that would enable management to redesign its strategies in project design, implementation, monitoring and evaluation. In 1999 UNEP carried out 27 in-depth evaluations, one management study of trust funds, one annual evaluation report and more than 70 self-evaluations. The 1999 annual evaluation report will be issued shortly.
15. United Nations Centre for Human Settlements (Habitat). A revitalization process was undertaken during the biennium to redirect Habitats activities and mobilize its limited resources towards well-defined and strategic objectives. Self-evaluation of the work programme is undertaken at the subprogramme level at the end of each biennium; during 1998-1999, 16 operational activities were subject to final evaluations. The community development programme held two global meetings during the biennium to review its programme policies and activities and the urban management programme was subject to an external evaluation in 1999. Lessons learned from reviews and evaluations are then incorporated into the biennial work programme and into the formulation of new activities at the design stage through the internal Programme Review Committee. Additional evaluative activity included a review of the concepts of "community participation" and "community management" and the accompanying concept of "enabling environment" as these have been developed and applied by Habitats community development programme in its own local projects, and the development of a conceptual framework for the evaluation of UNCHS post-conflict rehabilitation projects.
16. United Nations Conference on Trade and Development (UNCTAD). The Trade and Development Board endorsed the decision of its Working Party on the Medium-Term Plan and the Programme Budget to reinforce evaluation processes within the organization. An important feature of these processes stressed by the Working Party is a logical framework approach to planning and assessment. Comprehensive assessments of UNCTADs activities were undertaken during the high-level mid-term review of the implementation of the agreed commitments reached at the ninth session of the Conference (see TD/B/45/8) as well as in the context of preparations for the tenth session. In addition, all UNCTAD subprogrammes were requested to undertake self-assessments covering the biennium 1998-1999. The logical framework approach applied to these processes and the outcome of and lessons learned from them are used within UNCTAD for management review and for reporting to Member States. Surveys are conducted annually covering all publications included in the UNCTAD Guide to Publications. The results of these surveys are reviewed by the UNCTAD secretariats Publications Committee and also by the Working Party. Following intergovernmental requests as well as those of donors, UNCTAD undertook during the biennium four in-depth evaluations of technical cooperation programmes: trade points (TD/B/WP/110 and Add.1), competition law and policy (TD/B/WP/119 and Add.1), possible multilateral framework for investment, and investment policy reviews (see TD/B/WP/109 and Corr.1 and Add.1 and 2 and Add.2/Corr.1).
17. International Trade Centre UNCTAD/WTO (ITC). Self-evaluation studies are usually undertaken once a year, on a rolling basis and are submitted to and discussed in the Joint Advisory Group on the ITC. Each evaluation is followed by a strategy proposal which is also submitted to the Advisory Group for approval. An evaluation and strategy for the product and market research, development and promotion programme was carried out in 1997 (ITC/AG/XXXI/165) but discussed together with the corresponding strategy in 1998. An evaluation and strategy for the trade information programme was carried out in 1998 (ITC/AG(XXXIII)/177) but discussed together with the corresponding strategy in 1999.
18. Department of Economic and Social Affairs. During the 1998-1999 biennium, the Department, in cooperation with the Office of Human Resources Management, organized 10 retreats attended by 522 staff members. These exercises enabled staff who had been exposed to different work experiences and methods of operation in the three predecessor departments to learn about the accumulated knowledge and experience in other parts of the new department, and to assess together the contribution that these different experiences could make to advancing departmental objectives. New methods were devised to coordinate substantive support to the follow-up to United Nations conferences for which the Department is responsible, including the establishment of cross-divisional networks on themes, such as poverty and financing for development, that cut across the functions of different parts of the Department. The Departments role in technical cooperation was reviewed through periodic Directors Technical Cooperation Meetings. The Division for the Advancement of Women undertakes self-evaluations after each session of the Commission on the Status of Women and the Committee on the Elimination of Discrimination against Women.
19. In response to the decision in paragraph 56 of General Assembly resolution 50/120, six evaluations of the impact of the United Nations system support to capacity-building were undertaken by the Department of Economic and Social Affairs between October 1997 and March 1998 in Brazil, El Salvador, Mali, Pakistan, Uganda and Zimbabwe. Each evaluation was carried out by a team of two independent, experienced consultants, one from a recipient and one from a donor country. In addition to these teams, a group of "wise persons" was formed, at the outset, to oversee the independence and technical quality of the whole undertaking. Those parts of the United Nations system that are involved with operational activities were consulted and kept informed of the process at all stages. As an input to the triennial comprehensive policy review of operational activities, these initial evaluations were presented at panel discussions to the Economic and Social Council and the General Assembly in 1998 and issued as a United Nations publication, Capacity-building supported by United Nations: Some evaluations and some lessons (Sales No. E.1999.II.A.4). In resolution 53/192, paragraph 53, the General Assembly took note of the need to continue such evaluations. A new series will be presented to the General Assembly as part of the triennial comprehensive policy review of 2001.
20. As indicated in the annex to the present report, the work of the Department of Economic and Social Affairs is being covered by recent and forthcoming in-depth evaluations of the programmes on statistics, global development trends, issues and policies, the advancement of women, population, and sustainable development. These reports refer to relevant self-evaluation work in the Department, and also to corresponding activities in the regional commissions.
21. Economic Commission for Africa (ECA). During the period, a number of self-assessments were undertaken by ECA as part of the renewal process (reforms). Also as part of the reforms, a number of management-related activities were undertaken and most of their recommendations implemented during the 1998-1999 biennium. A Change Management Team was established in the Commission in order to implement some of the self-assessment findings. ECAs Programme Management Guidelines are designed to facilitate the transition to a renewed ECA. To facilitate the monitoring and evaluation of ECAs programme and subprogrammes, the guidelines have established a number of monitoring tools, including quarterly programme performance reviews. The guidelines outline responsibilities for periodic internal evaluation at ECA within the context of the medium-term plan.
22. Economic Commission for Europe (ECE). During the biennium 1996-1997, ECE and its subsidiary bodies conducted a review of the objectives and elements of all programmes (see A/53/90, para. 22). For the 1998-2001 medium-term plan cycle, self-evaluations are planned for the 2000-2001 biennium. During 1998-1999, an external evaluation was carried out on the ECE subprogramme on timber, in the form of a survey on the users attitudes towards the products and services of the ECE Timber Section.
23. Economic Commission for Latin America and the Caribbean (ECLAC). During the 1998-1999 biennium, the following subprogrammes conducted their self-evaluations:
Strategic administration and state reform;
Environmental and land resource sustainability;
Population and development;
Subregional activities in Mexico and Central America;
Subregional activities in the Caribbean.
The structure of the ECLAC secretariat has been modified by the establishment of an independent subprogramme to address gender mainstreaming. To an important extent this is the result of the self-evaluation exercise by the Social Development Division in the 1996-1997 biennium. Evaluation recommendations have also contributed to other changes in programme design, improvement in delivery and/or savings of resources. For example, two recurrent publications within subprogramme 4, that is, the Economic Survey and the Economic Panorama, were merged into one publication covering the subject matters of both. Similarly, after a thorough review, the Information Services bulletins, three separate issues of which had varying frequencies, formats and contents, were merged into one information newsletter.
24. Economic and Social Commission for Asia and the Pacific (ESCAP). In accordance with the established programme monitoring mechanism, the Economic and Social Commission for Asia and the Pacific, at its annual session in 1998 and 1999, examined the secretariats report on implementation of the programme of work, its main achievements and use of both regular budget and extrabudgetary resources. Based on the recommendations arising from ESCAPs deliberations, efforts were being continued to shift the emphasis in the programme of work from the production of publications to the provision of group training, to increase the dissemination of data and information through ESCAP web sites, and to promote technical cooperation among developing countries in carrying out operational activities. In pursuing the recommendations of the Advisory Committee on Administrative and Budgetary Questions, the Office of Internal Oversight Services and the Joint Inspection Unit on the need to streamline the publications programme of regional commissions, the secretariat produced in 1999 a "Guide to ESCAP publication activities". The benefits derived from the implementation of the Guide will be monitored and assessed during the biennium 2000-2001. As part of the programmed activities, ESCAP facilitated intergovernmental reviews and evaluations of the regional implementation of regional and global mandates relating to the Fourth World Conference on Women, the World Summit for Social Development and the International Conference on Population and Development. Findings and recommendations arising from those meetings will be fully taken into account in the development of future regional programmes and projects in the related fields.
25. Initiatives taken by ESCAP to elicit feedback from target beneficiaries and users of services included evaluation of meetings, training courses and publications, feedback from the users of the web sites as well as consultations at meetings and workshops with country representatives, and stakeholders of technical assistance projects. During the biennium, eight projects funded by the United Nations Population Fund (UNFPA) and executed by ESCAP from 1996-1998 were evaluated by independent population specialists. A joint evaluation exercise was undertaken by the secretariat and the Government of Japan to assess the impact of selected technical assistance projects implemented under the Japan-ESCAP Cooperation Fund.
26. Economic and Social Commission for Western Asia (ESCWA). During the biennium 1998-1999 ESCWA conducted the following evaluation activities:
An evaluation of two key publications from each of ESCWAs five substantive divisions, in addition to one project. Each publication was evaluated by two independent consultants; one from the academic sector and a governmental focal point;
An evaluation of 34 meetings and workshops, at least one from each division, based on questionnaires circulated to the participants at the end of each meeting. These meetings and workshops amounted to one third of total meetings/workshops undertaken during the 1998-1999 biennium;
A self-evaluation exercise of thematic subprogrammes by each of the corresponding substantive divisions of their activities and outputs as a whole relative to the objectives set out for each division;
A self-evaluation exercise by regional advisers, including a key task performed by each, through a standardized questionnaire;
An overall evaluation of regional advisory services by focal points in member countries, through a standardized questionnaire;
An overall evaluation of ESCWAs work since its inception 25 years ago, during a meeting of eminent persons in December 1999.
These evaluation exercises were intended to be indicative, not exhaustive. A comprehensive system for evaluation is to be put in place during the biennium 2000-2001.
D. Public information
27. The evaluation studies undertaken by the Department of Public Information during the second half of 1997 and in 1998-1999 were as follows: a readership survey on the following Department of Public Information publications (June-December 1999): Basic Facts About the United Nations, Notes for Speakers, Image and Reality, A Guide to Information at the United Nations, United Nations in Brief, Annual Report of the Secretary-General on the Work of the Organization, United Nations Chronicle, Development Update, and Africa Recovery; a survey questionnaire among recipients of the Daily Press Clippings (May-June 1998); an evaluation of the 1998 edition of Basic Facts About the United Nations (DPI/1920) (November 1999); a survey among representatives of non-governmental organizations (NGOs) participating in the Department of Public Information/NGO Conference held from 15 to 17 September 1999 (November 1999). Additionally, in response to requests made by the Committee on Information during its twenty-first session, two surveys are currently in progress. The first is part of a case-by-case review of the United Nations information centres (UNICs) which have been integrated with UNDP field offices, and consists of two separate questionnaires addressed, respectively, to the host Governments and to the Directors of 14 integrated UNICs. The second survey relates to the pilot project for the development of an international radio broadcasting capacity for the United Nations. The results of both surveys will be included in forthcoming reports submitted by the Secretary-General at the twenty-second session of the Committee on Information, in May 2000.
E. General Assembly Affairs and Conference Services
28. Department of General Assembly Affairs and Conference Services. During 1998-1999, the Department of General Assembly Affairs and Conference Services engaged in a number of activities that can be described as self-evaluation efforts. Some of these were one-time events, others are periodic or continuing exercises. Among these activities, the following involved attempts to obtain the views of users or to utilize independent benchmarks or judgements to assess performance or improve efficiency:
Unit reviews. Following the establishment of the Department of General Assembly Affairs and Conference Services in June 1997 and its budgetary consolidation on 1 January 1998, internal reviews were carried out in several units of the Departments, covering their structure, staffing, physical facilities, working methods, outputs and efficiency. The reviews were conducted by experienced staff members in the Department, knowledgeable of but not connected with the areas being examined. In the Verbatim Reporting Service, the review eased the adjustment to a technological breakthrough: the use of off-site reporting involving the transmission of compressed and digitized sound recordings to verbatim reporters working at other locations, in order to reduce expenditures on temporary assistance without impairing the quality and timeliness of the outputs. A review of the Copy Preparation and Proofreading Section led to improved workflow control and clearer priority setting, resulting in more timely delivery of outputs. In the Reproduction Section, a number of suggestions were made to improve the physical plant, contributing to better working conditions and enhanced overall efficiency. The review of the Distribution Section, because it focused on staffing issues, was conducted by a task force set up with the participation of the Department of Management. Recommendations on the review of the structure of the Section and updating of job descriptions are being implemented by a joint staff-management working group.
Functional reviews. Responding to comments made by delegations in the Committee on Conferences and the Fifth Committee of the General Assembly, the Translation Services engaged in an extensive review exercise that has led to a reduction of the rate of self-revision and other improvements. The Official Records Editing Section conducted an analysis of its outputs. One of the results has been that the issuance of the annual volume of resolutions of the General Assembly in six languages has been advanced from September to April.
Technological innovations. The Department has embarked on a number of projects aimed at making use of advanced technological applications to improve the efficiency of its operations; two of these have been designed with specific quality considerations in mind. Following extensive testing, selection and adaptations, the installation phase of a computer-assisted translation project was completed in 1999. Through the use of translation memory technology, it is expected to ensure greater terminological and stylistic consistency and reference accuracy in the outputs of the translation services. The acquisition of a radio-controlled warehousing system for the Distribution Section, also installed in 1999, should improve the delivery of documents and publications stocked for requests received after initial circulation.
Ongoing managerial evaluations. In addition to internal managerial reviews, the annual Inter-Agency Meeting on Language Arrangements, Documentation and Publications, which brings together conference services managers from all agencies in the United Nations system and many other international organizations, provides an opportunity for the Department to compare the performance of its conference-servicing units with those of other entities with similar functions. Discussions on quality and productivity indicators, and on managerial and technological solutions for shared concerns, allow participants to identify and emulate best practices.
Ongoing intergovernmental evaluations. The Committee on Conferences devotes a considerable part of its annual substantive session to review aspects of the work of the conference services of the Department. Particular focus of attention in the Committee over the past two years has been the enhancement of the quality of translation outputs, the provision of interpretation services beyond the entitlements of established bodies, and an increased utilization of conference facilities at the United Nations Office at Nairobi. The annual General Assembly resolution on pattern of conferences usually contains many provisions which clarify for the Department the evolving needs and preferences of the users of its outputs and services. Comments from the users of the Departments services are received during the meetings of the Fifth Committee of the General Assembly and at the sessions of the Committee on Conferences. In addition, numerous remarks in praise or criticism of the work of the Department are made throughout the year by delegations and other parties. All complaints are traced to the units and staff members concerned, and corrective action is taken when necessary. Expressions of approval are similarly conveyed to the relevant units to encourage continued quality performance.
F. Support services
29. Department of Management. Among the many internal reviews of work processes conducted during the biennium, the following involved attempts to obtain the views of users or to utilize independent benchmarks to assess performance.
Staff development and learning programmes. In order to assess the impact and cost-effectiveness of Secretariat-wide development activities, the Office of Human Resources Management commissioned a comprehensive external evaluation of training programmes. This evaluation complements the evaluation mechanisms that are integrated into every staff development programme and include participant questionnaires and follow-up instruments. The findings from these evaluations are factored in to improve the design and delivery of the programmes. For competitive examinations: (a) as a result of an evaluation of procedures, electronic communications with candidates over the Internet were further expanded; and (b) board members of the Examinations and Tests Section evaluated the oral interview format and developed a new competency-based interview format, which allows a more valid and reliable assessment of candidates skills and competencies as they relate to the requirements of the position. For career support, the Office of Human Resources Management included feedback from participants in its ongoing self-evaluation of career support programmes, as a result of which improvements were made to the series of information sessions, videos and panel presentations related to career support and development for all staff, and to the orientation and development programmes for new Professional staff members.
Internship programme. Intern feedback sessions were started in the autumn of 1997. These sessions take place on an informal basis in groups of 20 to 25 interns two weeks after the commencement of an internship session. The interns submit internship questionnaires at the end of their internship session during the course of exit interviews.
Evaluation of the status of performance appraisal system (PAS) implementation. The Office of Human Resources Management sent questionnaires to the Joint Monitoring Committees, requesting them to provide information on the status of PAS implementation in their respective departments and offices.
Other reviews undertaken by the Office of Human Resources Management. Reviews undertaken by the Office of Human Resources Management in 1998/1999 in conjunction with other offices/bodies concerned have resulted in the issuance of administrative instructions and their guidelines on family leave, sick leave and maternity leave and on medical standards and clearances. In addition, the following ongoing activities in the Office of Human Resources Management have been reviewed: medical standards for pre-employment and reassignment clearance, forms for pre-placement medical examination and for periodic medical examination and procedures for medical evaluation.
Travel and Transportation Service. The Travel and Transportation Service benchmarks its travel operations, policies and spending with 14 major multinational companies and international organizations, including the World Bank, AT&T, BP Amoco, Citibank, Price Waterhouse Coopers, and Xerox. This exercise allows continuous evaluation of the United Nations processes and operations and the gaining of experience in best business practices. The Service also organizes travel seminars with individual departments to elicit feedback on its performance and that of the in-house travel agency. As part of its internal processes, it also performs a complete audit of all air tickets issued by the travel agency to ensure that the lowest available air fares are obtained. This practice received very favourable review in professional travel industry publications. In its transportation activities, the Service monitors its performance in the area of removal of household goods through a routine survey questionnaire, which is offered to every staff member on finalization of their move. The questionnaire allows immediate remedial action in the case of deficiencies both on the part of the contractor and the United Nations staff.
Other reviews included:
Programme Planning and Budget Division. Following the preparation of the proposed programme budget for the biennium 2000-2001, an internal review of the preparation process was initiated involving all staff of the Division; the results of this review will be used to streamline future budget preparations.
Peacekeeping Financing Division. At the end of 1997, a self-evaluation mechanism was established for the Peacekeeping Financing Division concerning the peacekeeping budget process, in particular, but also including the evaluation of the content, format and presentation of performance reports and proposed budgets for peacekeeping missions. Self-evaluation proposals, as well as proposals for other improvements, were made by working groups made up of all Peacekeeping Financing Division staff. These were examined by the Controller and the Director of the Peacekeeping Financing Division; agreed recommendations were documented and acted upon.
United Nations Medical Service. In reviewing the services provided by the United Nations-sponsored dispensaries, the Medical Services Division prepared and distributed questionnaires to all 43 dispensaries in January 1999. In its March 1999 meeting, the Standing Committee on Medical Directors of the United Nations system selected, based on the responses received, 10 dispensaries to be reviewed on a priority basis. The review of the first 10 dispensaries is scheduled to start in January 2000 after the finalized terms of reference have been submitted and agreed upon by all agencies contributing to the budget of those dispensaries. In view of the growing number of United Nations examining physicians appointed worldwide (720 as of September 1999), a questionnaire was prepared and distributed by the Medical Services Division to all currently appointed examining physicians. The Division will review them and inform the examining physicians of the continuation/ termination of their appointments.
30. Office of Internal Oversight Services reviews of support services. The General Assembly, in its resolution 51/241 of 31 July 1997, identified several areas that warranted more intensive review by the United Nations oversight bodies. Since many of these were in the field of human resources management, the Office of Internal Oversight Services added this area to its oversight priorities in 1998. Descriptions of the management audits of the recruitment process and of personnel issues in the field service are to be found in, respectively, the 1998 and 1999 annual reports of the Office of Internal Oversight Services (A/53/428, paras. 77-78, and A/54/393, paras. 77-78). A report on the Office of Internal Oversight Services inspection review of common services in the United Nations (A/54/157) was submitted to the General Assembly in 1999.
IV. Conclusions and recommendations
31. The institutional arrangements for conducting evaluations in United Nations departments and offices range from units devoted to evaluation through evaluation focal points within monitoring and programme coordination divisions to ad hoc arrangements. The Office of Internal Oversight Services also provides management consulting services to departments and offices at their request.
32. In general, the situation has improved significantly compared to that described two years ago. Of particular note during the 1998-1999 biennium was the conduct of high-level ad hoc examinations of the fall of Srebrenica, the operation and functioning of the International Tribunal for the Former Yugoslavia and the International Criminal Tribunal for Rwanda, and the actions of the United Nations during the 1994 genocide in Rwanda. These reviews represent a deepening of the commitment at the highest levels of the Organization to learning lessons of experience by means of objective and authoritative scrutiny of the manner in which the United Nations operates. For large-scale field operations (peacekeeping, refugees) the evaluation, or lessons learned, function is well established. In other substantive activities in the political, economic and social and public information fields, solid evaluation work is being conducted on a wider scale than in previous bienniums. Conference and support services also have instituted more critical reviews and assessments than in previous bienniums. Reinforcing this work, the Office of Internal Oversight Services conducted management audits and inspections of support service activities during the biennium 1998-1999.
33. The CPC may wish to review the information contained in the annex to the present report and in paragraphs 4 and 5 above, and recommend a second topic for in-depth evaluation by CPC in 2002, and two topics for in-depth evaluation in 2003.
(Signed) Hans Corell
Overseer, Office of Internal Oversight Services
Evaluation reports completed or scheduled, 1992-2002
|Year||Progress reportsa||In-depth evaluation reports||Triennial reviews/follow-up reports||Reports to General Assembly on evaluation|
|1992||Social development||-||Human rights||A/47/116|
Human rights (follow-up)
|1994||Peacekeeping: start-up phasec||Social development||ECA: development issues policies||A/49/99 and Add.1|
|Africa: critical economic situation, recovery and developmentc d||-|
|1995||-||Peacekeeping: start-up phase||-||-|
|1996||-||Peacekeeping: termination phase||UNHCR||A/51/88|
|Department of Public Information|
|Department of Humanitarian Affairs|
|1998||-||Crime prevention and criminal justice||Peacekeeping: start-up phase||A/53/90|
|International drug control||Environment|
|1999||-||Disarmament||Department of Public Information||-|
|Electoral assistance||Peacekeeping: termination phase|
|2000||-||Global development trends, issues and
policies, and global approaches to social and microeconomic issues and policies, and the
corresponding subprogrammes in the regional commissions (programmes 7.3 and 7.4 of the
Advancement of women
|Present report (A/55/63)|
|2001||-||Population||International drug control||-|
|Policy coordination and sustainable development||Crime prevention and
|2002||-||General Assembly and Economic and Social Council affairse||Report due|
a As decided by the Committee for Programme and Coordination in 1994, the
evaluation cycle was shortened by the
elimination of the progress report stage.
b As decided by the Committee for Programme and Coordination, no in-depth evaluation of UNRWA was required.
c These were transitional reports that were described as progress reports but that contained substantive recommendations.
d A follow-up study was conducted by the Joint Inspection Unit (A/50/885, annex).
e Official Records of the General Assembly, Fifty-third Session, Supplement No. 16 (A/53/16), para. 156.