The competition used 12,457 training examples translated into Korean from DialogueSum and evaluated summaries for 249 hidden test examples with ROUGE. The three-person team aligned on the core NLP concepts and baseline code before dividing the experiments. I was responsible for data augmentation, summarization models, and LLM fine-tuning. Because many reference summaries retained characteristics of translated prose, I focused the experiments on reproducing the evaluation set's style rather than simply generating the most natural Korean sentences.
Experiment direction
ROUGE is sensitive to wording shared with the reference summary, so I compared the effects of data volume, input structure, and reference-summary style on the actual score separately. Within the team, training was split into two tracks: lightweight open-source NLP models and LLM fine-tuning.
Comparing summarization models and LLMs
While fine-tuning KoBART- and KoGPT-family models, I compared augmentation volume, input token length, and special
tokens that marked speakers and turns. Expanding the data fivefold with GPT-4o-mini produced only a small gain,
and the special tokens had little effect. Adjusting input length helped slightly; EbanLee/kobart-summary-v3
performed best in this track.
In a separate LLM track, I fine-tuned GPT-4o-mini and tuned its few-shot examples and prompt. I also tried QLoRA training on LLaMA 3.2 3B and EXAONE 7.8B.
The best result came from fine-tuning GPT-4o-mini for two epochs on 500 examples. It achieved a ROUGE score of 50.10, and the team finished in 1st place.