Backround material: AI and ethics in education

International ethical principles for artificial intelligence in education are based on generally accepted values such as safety, justice, responsibility, transparency and human-centred design.  These principles are not precise rules. Instead, they provide guidelines that help various parties, such as governments, education providers and technology companies, to develop their own practices. The principles are based on an international consensus promoted in the field of education by UNESCO, the EU and the OECD. The same consensus is widely conveyed in the policies of other organisations and different countries as well.

The ethical principles stem from the idea that AI can be an integral part of education and improve its quality. Understanding risks and being critical are a part of ethics, as even the most positive applications of AI always include ethical risks that should be recognised. This approach will provide an open, comprehensible and democratic basis for the development of pedagogy in relation to AI, for example. International ethical principles for AI in education highlight the following themes [3]:  

1. Governance and stewardship: The multidisciplinary perspective and the consideration of all ethical considerations are emphasised. AI systems should be deployed and applied reliably and openly.
2. Transparency and accountability: Data processing and the operation of algorithms should be transparent and explainable. Demanding accountability for all the actors involved in the design and use of AI systems, including data minimisation and reporting of possible negative side effects.
3. Sustainability and proportionality: The design and use of AI systems must be sustainable and proportionate, taking into account the well-being of the users.
4. Privacy: The core of privacy is the protection of personal data, which is concretised through trust. Users must be able to trust that their data are utilised responsibly and in accordance with their interests when it comes to competence development, for example. From the perspective of ethics, it is important to examine how this trust can be strengthened in the use of systems and in the management of personal data, such as in terms of consent.
5. Security and safety: AI systems should be designed and implemented in a manner that protects data against cyber crimes and data breaches. Implementation must include monitoring and evaluation mechanisms and security guidelines.
6. Inclusiveness: The design, development and deployment of AI systems must allow equitable access and use of AI. Non-discrimination and neutrality ensure equitable use.
7. Human-centred AI: The goal of using AI systems should be to complement and enhance human cognitive, social, and cultural capabilities while protecting human autonomy without risk of manipulation. The freedom of choice of AI users must be preserved, and human control over AI-based work processes must be secured.

When it comes to ethical guidelines, it is important to identify AI stakeholders and explain their responsibilities in AI activities in an understandable manner. For example, in terms of human-centred AI, this can involve considering whose activities, how and in what circumstances could promote the realisation of human-centred AI. Linking ethical guidelines to the level of everyday issues makes the questions of ethical frameworks more recognisable for various actors. The realisation of ethical principles requires cooperation throughout the chain, from the abstract to the concrete level.

Generative AI is associated with particularly critical considerations, the awareness of which supports the ethical evaluation of its use in educational contexts. Their purpose is not to prevent generative AI from being applied, but to raise awareness and ethical discussion. The aim of ethics review is always to make the application of generative AI more effective by ensuring that the associated challenges and risks are taken into account. UNESCO [13] has identified eight key controversies around generative AI in education and research. 

• Worsening digital poverty: The possibility to create and control generative AI is out of reach of most companies and most countries, which deepens the digital divide and inequality globally.
• Outpacing national regulatory adaptation: The rapid development of generative AI challenges legislation, which does not always keep pace with technology, potentially creating regulatory gaps.
• Use of content without consent: The training data of generative AI models may contain material collected without the consent of copyright holders.
• Unexplainable models: Generative AI models are ‘black boxes,’ making it difficult to understand their operation. This may lead to prejudices and manipulation.
• AI-generated content polluting the internet: Generative AI models can spread discriminatory biases or errors, undermining the reliability of information in the long term.
• Generative AI’s lack of understanding of the real world: Generative AI does not actually understand humans and the world, as it only repeats the language included in its training data using mathematical models.
• Reducing diversity: The responses of generative AI are based on probabilities, which can lead to one-sidedness and averages and thus reduced diversity.
• Generating deeper deepfakes: The development of deepfake technology enables the generation of realistic videos and images, which may lead to spreading disinformation and invasions of privacy.