Monitoring and evaluation of universal access impact

A key consideration for the design and implementation of policies aimed at promoting access for all is ensuring ongoing monitoring and evaluation of whether a policy or individual project is meeting its intended goals. This consideration of accountability should be a foundational design component of universal access (UA) approaches, and relies both on clear, measurable objectives and on the ability to measure progress against them. In a sense, this equates UA policies and plans with many other government policies or programmes, for which policy-makers need to design and implement mechanisms for monitoring effects. In addition to transparently disbursing funds in support of universal access service fund (UASF) targeted projects, it is also particularly important to evaluate whether such spending is an effective and efficient use of collected funds.

As such, two approaches to monitoring and evaluating the impact of UA policies may be considered: (i) evaluation of the overall policy, and (ii) evaluation of individual UASF-supported projects. In both cases, the establishment of clear goals and/or milestones will lay the groundwork for later impact evaluation.

For UA policies, governments should set specific, attainable goals for the key aspects of the policy. This could include, for example, ensuring Internet connectivity in a minimum number of locations or to a minimum percentage of the population, ensuring access to a certain level of connectivity without exceeding a certain proportion of per capita national income, and ensuring a minimum level of service quality. The inclusion of specific goals or milestones allows a review of the efforts undertaken as a result of the policy. For example, if a UA policy includes a goal of increasing the percentage of the population with access to a 10 Mbps Internet connection to at least 98 per cent within five years, a subsequent review should be able to evaluate whether that goal was met. If resources permit, an interim or mid-term assessment of the policy’s impact is a particularly useful tool, allowing for course corrections before the target date is reached.

Similarly, UASF-funded projects should be designed to have specific implementation milestones and goals that must be met, and clear criteria against which success can be measured. Traditional voice service-focused UASF-supported projects have often been structured such that payment is disbursed upon successful, timely completion of project milestones, providing recipients with an incentive to meet the stated implementation timeline and goals. This approach is equally applicable to UASF-supported projects for expanding access to the Internet and digital services more broadly. Funding recipients should be able to substantiate that they have met goals that may include not only connectivity, but adoption, price levels, skills and training, variety of services available, or services available to disadvantaged populations.

In line with meeting specific milestones and timelines, UASF-funded projects should include reporting requirements that may incorporate a progress assessment, analysis of any unexpected circumstances, financial statements, and any other relevant analysis, particularly in cases of deviation from initial project plans. As above, such requirements may not markedly differ from reporting requirements for a telephony-focused project but should be tailored to the particular project and its goals. Thus, additional reporting requirements could include, for example, average available broadband speeds, access to particular digital services, or measures to ensure access for people with disabilities. The goals of reporting requirements should be to enable all stakeholders to assess project progress or success, and also to serve as a motivation for the funding recipient to commit appropriate resources to meet the project goals.

For example, India’s Universal Service Obligation Fund (USOF) notes that it includes multiple provisions in its universal service funding agreements in order to guarantee adherence to quality of service obligations (USOF, Monitoring Mechanism). Chile’s telecommunications development fund also incorporates provisions into project contracts that link subsidy disbursal to project completion, while also publishing information regarding the projects supported by the fund.

UA policy and project monitoring is a key policy element for increasing the likelihood of success. While the concept dates back to the earliest UA policy approaches, it can and should be adapted to fit modern UA and digital service needs.