iGGi PG Researcher Dominik Jeurissen held a talk on "LLM Agents For QA - Potential & Limitations" at this year's Develop:Brighton conference.

Abstract:

With tight deadlines and a constantly evolving game, properly testing a game is challenging. Using AI agents to simplify this work sounds promising, but machine learning is often too slow, and manually implementing the agents takes time. As such, one particularly exciting application for QA is to use Large Language Models (LLMs) as zero-shot game-playing agents. LLM-based agents can play games without pre-training, making them a valuable asset to test a constantly changing game. But how well do they play games? What are their strengths, and what do they struggle with?

In this session, we will review how to implement zero-shot agents with LLMs and show examples of existing LLM-based game-playing agents. We will also show that although these agents have many limitations, they have the potential to be a valuable tool for QA to automate many repetitive tasks.

The objectives of Dominik's talk were to

• provide the audience with an overview of the cutting-edge research on LLM-based zero-shot game-playing agents

• show what these agents can do well and what their limitations are

• give practical tips on how to utilize LLM agents as QA tools

Dominik's talk has been recorded and will be made available to Develop ticket holders. Please contact Dominik directly if you have any queries regarding the presentation.