The advantages of user-centred prompt engineering for companies

Prompt engineering plays a vital role in the quality of large language models’ output. The Human AI Learning Systems (HAIS) Lab at BFH has created a structured process that brings user-centred design principles to AI application development. This article presents the method, its application, and the potential of large language models for use that is both needs-oriented and effective.

Generative AI and prompt engineering

 In generative AI that uses language models, humans interact with the system by giving it text input and getting an output from the AI. This output is designed using instructions for large language models, which are referred to as “prompts”. The prompts not only define the structure of input and output but also specify which points the model should emphasise or which steps should be carried out to process the input. The optimisation of these prompts to improve the quality of interaction is known as “prompt engineering”. The aim is to improve the AI’s responses and user-friendliness without changing the underlying model parameters.

Figure 1: Illustration of the interaction between humans and AI in prompt engineering. The system message defines the context in which the application is located. The user formulates their input or question, which is then processed. The resulting AI output is then optimised to improve comprehensibility and quality

Why a Human-Centered Approach Is Important

The needs and expectations of users are at the centre of the development of business innovation. AI systems are not the goal themselves, but a means of creating practical, usable and target-oriented solutions. The method of human-centred prompt engineering is based on the principles of design thinking and user-centred design. Through iterative processes, user-centred innovations are developed, and a deep understanding of users and solutions is established. This enables companies to directly address system requirements and existing process difficulties This approach benefits companies by speeding up solution development, reducing the risk of inaccurate results, and fostering more effective development of AI-driven solutions.

The process of human-centred prompt engineering

The Human AI Learning Systems (HAIS) Lab has developed a structured, six-step process for human-centred prompt engineering. This ensures AI-driven business innovations align with user needs and are informed by expert knowledge. For instance, AI can automatically analyse, prioritise, and forward service requests, allowing for more efficient work processes and significantly reduced response times. The focus lies in developing prompts that enhance the comprehensibility and reproducibility of AI output.

Figure 2: The six-stage process of human-centred prompt engineering. It shows how users and experts jointly develop, test and iteratively refine prompts in order to specifically improve AI outputs and make them usable in a prototype.

1. Understanding the needs and problems of users

At the beginning, the focus is on understanding the users and their expectations of the AI system. This is typically achieved through interviews and direct interaction. Sample texts can also be collected for the AI input to contribute to finding solutions at a later stage. Key questions are: Who belongs to the user group? What goals are the users pursuing? What information do they provide to the system?

2. Defining the ideal solution with experts

This is followed by the joint development of an ideal AI output together with experts. This is based on the sample inputs from step 1, which are used to outline an optimal solution. The drafts of ideal outputs form the so-called gold standard, which serves as a benchmark for later comparisons. The key considerations here are: Who are the relevant experts for the problem? How would they support users in achieving their goals? And what does the ideal result of the AI output look like?

3. Drafting prompts

Various prompts are then created. These include prompts with multiple output examples and decomposition into individual steps. These techniques are then tested, and the output is documented. The focus is on analysing the effect of these techniques and changes on the AI output, and on identifying the most promising prompts and techniques. This phase allows to evaluate the various prompt techniques and see how they change the output, as well as how effective the prompts are.

4. Evaluation of prompts

Next, the AI outputs are compared with the previously defined gold standard. User involvement enables the quality of the results and prompts to be evaluated. Adopting a user-centred approach to evaluation allows users to collaborate with the AI to perform tasks. The output of the various prompts is then evaluated. A data-centred approach focuses on factors such as the consistency, response time and accuracy of the output. The gold standard is used as a reference to identify weaknesses and implement targeted improvements.

5. Iterative refinement

The prompts are refined iteratively based on evaluations until the results closely match the desired solutions from the gold standard. This step ensures continuous improvement and that the users’ goals and needs are met. The key questions are: What changes were made? What impact did these changes have on the output? To what extent does the current solution match the defined gold standard?

6. Implementation in a prototype

Finally, the optimised prompts are integrated into a prototype with a large language model. This demonstrates the effectiveness of the solutions developed and enables interaction with users. The prototype is evaluated in terms of how it can be developed, the extent to which it solves the original problem, and whether it fulfils the users’ expectations and needs.

Conclusion: Benefits of the human-centred approach in prompt engineering

When developing AI systems, it’s crucial to focus on the needs and expectations of the users. The human-centred approach in prompt engineering ensures that AI outputs are aligned with specific goals, ensuring that the technical solution meets the user’s expectations. The iterative approach minimises risk and increases the quality of the prompts and AI outputs by involving users and experts. This method provides concrete guidelines for a targeted implementation of generative AI and helps to translate the potential of AI into measurable results.