The work the ITU Standardization Sector (ITU-T) on artificial intelligence

Artificial intelligence (AI)

ITU works on the development and use of AI to ensure a sustainable future. To that end, it convenes intergovernmental and multi-stakeholder dialogues, develops international standards and frameworks, and helps in capacity building for the use of AI.

AI and Machine Learning are gaining a larger share of the ITU standardization work programme in fields such as network orchestration and management, multimedia coding, service quality assessment, operational aspects of service provision and telecom management, cable networks, digital health, environmental efficiency, and autonomous driving.

ITU organises the annual AI for Good Global Summit, which aims to connect innovators in the field of AI with public and private sector decision-makers to develop AI solutions that could help in achieving the sustainable development goals (SDGs).

ITU has launched a global AI repository to identify AI related projects, research initiatives, think-tanks and organisations that can accelerate progress towards achieving the SDGs.

Open ITU platforms advancing various aspects of AI and Machine Learning include:

• The ITU Focus Group on ‘Machine Learning for Future Networks including 5G’  is defining the requirements of machine learning as they relate to interfaces, protocols, algorithms, data formats and network architectures.
• The ITU Focus Group on ‘AI for Health’, driven in close collaboration by ITU and WHO, is working towards the establishment of a framework and associated processes for the performance benchmarking of ‘AI for Health’ solutions.
• The ITU Focus Group on ‘Environmental Efficiency for AI and other Emerging Technologies’ will benchmark best practices and describe pathways towards a standardized framework to assess environmental aspects of the adoption of emerging technologies.
• The ITU Focus Group on ‘AI for Autonomous and Assisted Driving’ is working towards the establishment of international standards to monitor and assess the behavioural performance of the AI ‘drivers’ in control of automated vehicles.
• The new Global Initiative on ‘AI and Data Commons’, established in January 2020, aims to support AI for Good projects in achieving global scale. The Initiative will offer assemblies of resources to launch new AI projects aligned with the SDGs, and scale them up fast.

AI/Machine Learning in Communication Networks

AI/ML will also shape how communication networks, a lifeline of our society, will be run. Many companies in the Information and Communication Sector are exploring how to make best use of AI/ML in communication networks such as 5G, but this task is difficult. ITU has been at the forefront of this endeavour and is exploring how to best apply AI/ML in communication networks. The pioneering group of examining how to apply AI/Machine Learning in communication networks is the ITU Focus Group on Machine Learning for Future Networks including 5G. The group has produced ten technical specifications, four of which have already been turned into approved specifications by the ITU membership, and one is in the approval process:

• “Architectural framework for machine learning in future networks including IMT-2020” (ITU-T Y.3172, January 2020)
• “Machine learning in future networks including IMT-2020: use cases” (Supplement 55 to Y.3170 Series, October 2019)
• “Framework for evaluating intelligence level of future networks including IMT-2020: use cases” (ITU-T Y.3173, February 2020)
• “Framework for data handling to enable machine learning in future networks including IMT-2020: use cases” ( ITU-T Y.3174, February 2020)
• Draft ITU-T Recommendation (in the approval process): “ML marketplace integration in future networks including IMT-2020”

The following five FG ML5G specifications are also in the process of being turned into ITU standards:

• FG ML5G specification: “Requirements, architecture and design for machine learning function orchestrator“
• FG ML5G specification: “Serving framework for ML models in future networks including IMT-2020​“
• FG ML5G specification: “Machine Learning Sandbox for future networks including IMT-2020: requirements and architecture framework“
• FG ML5G specification: “Machine learning based end-to-end network slice management and orchestration​“
• FG ML5G specification: “Vertical-assisted Network Slicing Based on a Cognitive Framework​“

In the context of its Machine Learning work in communication networks, ITU has launched the ITU AI/ML in 5G Challenge, a competition for which hundreds of professionals and students from over 50 countries have signed up. The goal of the Challenge is to see how well ITU’s toolset of Machine Learning specifications work in communications networks in practice and where there are gaps. The Challenge will culminate with the Grand Challenge Finale in December 2020, an online conference that will award prizes to the best teams. The Challenge is accompanied by weekly webinars to provide tips on how to tackle the problems.

AI for Health

Considerable resources have been allocated over the past decade to explore the use of AI for health. The potential is immense. But to guarantee safe and ethical implementation of AI in health care, many issues such as regulation, potential for bias, and adequate evaluation of efficacy must first be addressed.

The performance of AI algorithms depends on the quality of the training data and the learning mechanism. Poorly design AI algorithms or biased or imcomplete lead to errors. For new drugs or surgical interventions, agreed frameworks are in place to ensure they can be applied safely. However, as of today, there is no agreed framework for assessing or reporting the results of health AI models before deciding that they pass the test to be applied in a population The lack of confidence or quality control is a major barrier to the uptake of AI in health care.

The ITU-WHO Focus Group on Artificial Intelligence for Health (FG-AI4H) is an inter-agency collaboration between the World Health Organization and the ITU to create a rigorous, standardised benchmarking framework to assess the accuracy of AI in health diagnostic aids, such as the computation of sensitivity and specificity scores in a binary classification problem.

The outline of the benchmarking framework was published in a commentary in The Lancet as: “WHO and ITU establish benchmarking process for artificial intelligence in health”.

Figure 1 provides an overview of the benchmarking process. At [1] any person or organisation can use the data made public by FG-AI4H, possibly in combination with their own data to train an AI solution [2]. The model will be submitted to the FG-AI4H secretariat [3]. The AI solution is tested on verified and representative undisclosed test data [4]. The performance of the AI solution is evaluated against the ground truth [5], out of which scores are derived. The scores are added to a central leaderboard [6].

Thematically, the Focus Group organises its work in – so far – 20 “Topic Groups” (TG), such asTG-Dermatology: images of skin lesions; TG-Ophthalmology: diabetic retinopathy and other diseases; TG-Outbreaks: detecting major infectious disease outbreaks; TG-DiagnosticCT: Volumetric chest computed tomography; TG-Malaria: detecting malaria infections; TG-MCH: Maternal and Child Health.

Participation in the Focus Group is free and open to anyone, in order to access all the documentation on the collaboration site, a free ITU account is needed (instructions provided on the website). The Focus Group has a general mailing list to which you can subscribe, as well as mailing lists for the different Topic Groups, all are open.

https://www.itu.int/go/fgai4h/

Figure 1: Overview of the FG-AI4H Benchmarking Process

______________________