Professor Yeon Myeong-heum of the Department of Smart Experience Design at the Graduate School of Techno-Design at our university received the Best Paper Award from the Korean Society of Design Science at the 2024 Fall Conference held at KAIST on Saturday, November 2. The paper “Exploratory Experiment Using ChatGPT in the Process of Generating Ideas for Product-Service Systems” by Lee Young-hyun and Yeon Myeong-heum was selected as the best paper after being reviewed by the editorial board among a total of 84 ADR papers (Nos. 147-150) published over the past year. The Archives of Design Research (ADR) is a top-tier international journal in the field of design and the only SCOPUS-level journal in the field of design in Korea, and has been selected as an excellent academic journal by the National Research Foundation of Korea.

This study is a study that experiments with the possibility and limitations of design experiments using LLM and generative AI, which many researchers have been interested in recently, and deals with a timely research topic. On the day of the conference, Professor Yeon Myeong-heum presented this research orally at the session for the presentation of outstanding papers.

The abstract of this paper is as follows.

Artificial intelligence technology is being used more widely across industries, and AI-based tools and development research are actively being conducted in the design field. Among them, generative AI (ChatGPT-4, Bard, etc.) has the potential to be used in the idea generation stage of the design process, and is expected to help overcome the limitations of existing methods and generate high-quality ideas quickly. Therefore, this study aims to systematically verify and explore the ideation effect of generative AI.

Two teams of four designers each were divided to conduct a comparative experiment between the traditional ideation method and the ideation method using generative AI. The teams were asked to come up with high-quality ideas for four hours under the theme of “Healthcare wearable devices for a new generation of Z-generation,” and the process was observed without intervention.

The results of the experiment were as follows.

The first team, which used the traditional method, generated 1,000 ideas, while the second team, which used the generative AI method, generated 1,500 ideas. The team using the generative AI method generated more ideas than the team using the traditional method. After the experiment, the participants' FGI and IDI were conducted to check the possibility of using AI, and the creativity of the ideas derived through expert evaluation was assessed and discussed to gain insights.

As a result of the experiment, the number of ideas using generative AI was about 1.67 times higher than that of the traditional method. The quality assessment also produced results that were comparable to those of traditional ideation methods. Generative AI is effective in expanding the thinking of entrenched designers, and it was clearly efficient in terms of time savings. However, the lack of context consistency and the lack of structural completeness made expert verification and convergence essential.

After evaluating the creativity of ideas derived through expert evaluation, insights were gained through discussion.

The experimental results showed that the number of ideas using generative AI was about 1.67 times higher than the traditional method.

Generative AI is effective in expanding the thinking of entrenched designers, and there was a clear efficiency in terms of time reduction.

In order to achieve better results using generative AI, it is important to clearly convey prior information and use specific and structured questions and prompts. In addition, it requires technology that effectively communicates with AI, the discernment and insight of designers, and high-level decision-making.