This page was translated automatically using artificial intelligence (DeepL). The German version is binding. More information about automatic translation
News Intranet
This page was translated automatically using artificial intelligence (DeepL). The German version is binding. More information about automatic translation
Artificial intelligence (AI) is transforming education – from digital learning assistants to chatbots that answer questions round the clock. For these systems to provide genuine support, they must not only recognise human language but also understand it meaningfully [1]. The focus is on two approaches: ‘prompt engineering’ and traditional techniques of natural language processing (NLP).
Prompt engineering involves using targeted instructions to enable more precise control of large language models (LLMs). This involves developing specific prompts that are fed directly into an LLM. Three LLMs were tested in each case – Llama3.1:70b, Mixtral:8x7b and Gemma2:27b – to ensure more robust results and to take into account the different perspectives of the models. The approach taken by NLP techniques often involves fine-tuning, whereby pre-trained models, such as BERT, are adapted to specific tasks. However, traditional algorithms, such as logistic regression, continue to be used to tackle certain tasks. NLP techniques thus focus on training and adapting AI models from the field of machine learning to process data effectively. Prompt engineering, on the other hand, is based on the Transformer architecture, which was introduced in the study ‘Attention is All You Need’ [2] and has since formed the basis of modern LLMs. Whilst prompt engineering is regarded as faster, more flexible and more cost-effective, the question remains as to whether it represents a genuine alternative to the resource-intensive process of model training. This question was examined in greater detail as part of this study.
The study examined the use of both methods in an educational chatbot developed specifically to convey information about blockchain technology. Five typical tasks performed by such a chatbot were analysed: named entity recognition (NER), sentiment analysis (SA), answering questions (Question-Answering, QA), classifying texts (Text Classification, TC) and summarising texts (Text Summarisation, TS). For each of these tasks, both traditional NLP techniques and prompt engineering were employed. In addition, a so-called RAG system (Retrieval-Augmented Generation) was used in the prompt engineering process; this retrieves relevant documents from a specialised database and provides the language model with more context.
Standard quantitative metrics such as accuracy and F1-score were used to evaluate the methods; these were recorded automatically using the Python programming language. In addition, qualitative metrics such as relevance and completeness were also included in the evaluation to provide a more comprehensive picture of the results.
The results (Figure 1) show clear trends: for interpretative tasks such as sentiment analysis (scores: 9.51, 10, 10) and text summarisation (6.48, 6.17, 6.41), prompt engineering proved particularly effective. In contrast, for structured tasks such as named entity recognition (9.89) and text classification (9.17), traditional NLP techniques proved superior.
These differences are also reflected in the respective application potentials. Prompt engineering is particularly well-suited to dialogue-oriented scenarios, as it supports the flow of conversation and enables context-sensitive responses – a finding that confirms earlier studies. Furthermore, it is flexible, adaptable and resource-efficient, making it ideal for dynamic learning environments. Traditional NLP techniques, on the other hand, play a central role in structured analysis, system security (e.g. by filtering harmful content) and precise information retrieval (e.g. through entity recognition).
These clear findings underline the relevance of a targeted, combined use of both methods. A hybrid approach brings together the strengths of both sides and enables the development of chatbots that are conversational, robust and adaptable – a particularly promising path in the field of education.
About the person
Emma-Luisa Richter studied IT security at Mittweida University of Applied Sciences and has been working as a research assistant on the ‘BOLT’ project at the BCCM at Mittweida University of Applied Sciences since December 2024.
Bibliography
[1] Bekkar, Hibat-Allah; Chtouki, Yousra (2024): Chatbots in Education: A Systematic Literature Review, [online] ieeexplore.ieee.org/document/10690334 [10 January 2025]
[2] Vaswani, Ashish; Shazeer, Noam; Parmar, Niki; Uszkoreit, Jakob; Jones, Llion; Gomez, Aidan N.; Kaiser, Łukasz; Polosukhin, Illia (2017): Attention Is All You Need, [online] arxiv.org/abs/1706.03762 [12 January 2025]