Improving Text Classification with Large Language Model-Based Data Augmentation

Zhao, Huanhuan; Chen, Haihua; Ruggles, Thomas A.; Feng, Yunhe; Singh, Debjani; Yoon, Hong-Jun

doi:10.3390/electronics13132535

Improving Text Classification with Large Language Model-Based Data Augmentation

Journal Article · Fri Jun 28 00:00:00 EDT 2024 · Electronics

DOI:https://doi.org/10.3390/electronics13132535· OSTI ID:2394731

Zhao, Huanhuan ^[1]; ^[2]; ^[3]; Feng, Yunhe ^[2]; Singh, Debjani ^[3]; ^[3]

Univ. of Tennessee, Knoxville, TN (United States)
Univ. of North Texas, Denton, TX (United States)
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Large Language Models (LLMs) such as ChatGPT possess advanced capabilities in understanding and generating text. These capabilities enable ChatGPT to create text based on specific instructions, which can serve as augmented data for text classification tasks. Previous studies have approached data augmentation (DA) by either rewriting the existing dataset with ChatGPT or generating entirely new data from scratch. However, it is unclear which method is better without comparing their effectiveness. This study investigates the application of both methods to two datasets: a general-topic dataset (Reuters news data) and a domain-specific dataset (Mitigation dataset). Our findings indicate that: 1. ChatGPT generated new data consistently enhanced model’s classification results for both datasets. 2. Generating new data generally outperforms rewriting existing data, though crafting the prompts carefully is crucial to extract the most valuable information from ChatGPT, particularly for domain-specific data. 3. The augmentation data size affects the effectiveness of DA; however, we observed a plateau after incorporating 10 samples. 4. Combining the rewritten sample with new generated sample can potentially further improve the model’s performance.

View Accepted Manuscript (DOE)

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE

Grant/Contract Number:: AC05-00OR22725

OSTI ID:: 2394731

Journal Information:: Electronics, Journal Name: Electronics Journal Issue: 13 Vol. 13; ISSN 2079-9292

Publisher:: MDPICopyright Statement

Country of Publication:: United States

Language:: English

References (15)

Influence of project characteristics, regulatory pathways, and environmental complexity on hydropower licensing timelines in the US Pracheil, Brenda M.; Levine, Aaron L.; Curtis, Taylor L. Energy Policy, Vol. 162 https://doi.org/10.1016/j.enpol.2022.112801	journal	March 2022
A synthesis of environmental and recreational mitigation requirements at hydropower projects in the United States Schramm, Michael P.; Bevelhimer, Mark S.; DeRolph, Chris R. Environmental Science & Policy, Vol. 61 https://doi.org/10.1016/j.envsci.2016.03.019	journal	July 2016
Enhancing social network hate detection using back translation and GPT-3 augmentations during training and test-time Cohen, Seffi; Presil, Dan; Katz, Or Information Fusion, Vol. 99 https://doi.org/10.1016/j.inffus.2023.101887	journal	November 2023
Enhancing Text Classification Models with Generative AI-aided Data Augmentation Zhao, Huanhuan; Chen, Haihua; Yoon, Hong-Jun 2023 IEEE International Conference On Artificial Intelligence Testing (AITest) https://doi.org/10.1109/AITest58265.2023.00030	conference	July 2023
Text Generation for Imbalanced Text Classification Akkaradamrongrat, Suphamongkol; Kachamas, Pornpimon; Sinthupinyo, Sukree 2019 16th International Joint Conference on Computer Science and Software Engineering (JCSSE) https://doi.org/10.1109/JCSSE.2019.8864181	conference	July 2019
Construction and Evaluation of a High-Quality Corpus for Legal Intelligence Using Semiautomated Approaches Chen, Haihua; Pieptea, Lavinia F.; Ding, Junhua IEEE Transactions on Reliability, Vol. 71, Issue 2 https://doi.org/10.1109/TR.2022.3156126	journal	June 2022
Data Augmentation for Multiclass Utterance Classification – A Systematic Study Xu, Binxia; Qiu, Siyuan; Zhang, Jie Proceedings of the 28th International Conference on Computational Linguistics https://doi.org/10.18653/v1/2020.coling-main.479	conference	January 2020
Transformers: State-of-the-Art Natural Language Processing Wolf, Thomas; Debut, Lysandre; Sanh, Victor Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations https://doi.org/10.18653/v1/2020.emnlp-demos.6	conference	January 2020
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation Yoo, Kang Min; Park, Dongju; Kang, Jaewook Findings of the Association for Computational Linguistics: EMNLP 2021 https://doi.org/10.18653/v1/2021.findings-emnlp.192	conference	January 2021
AEDA: An Easier Data Augmentation Technique for Text Classification Karimi, Akbar; Rossi, Leonardo; Prati, Andrea Findings of the Association for Computational Linguistics: EMNLP 2021 https://doi.org/10.18653/v1/2021.findings-emnlp.234	conference	January 2021
ZeroGen: Efficient Zero-shot Learning via Dataset Generation Ye, Jiacheng; Gao, Jiahui; Li, Qintong Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing https://doi.org/10.18653/v1/2022.emnlp-main.801	conference	January 2022
EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks Wei, Jason; Zou, Kai Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) https://doi.org/10.18653/v1/D19-1670	conference	January 2019
Robust Training under Linguistic Adversity Li, Yitong; Cohn, Trevor; Baldwin, Timothy Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers https://doi.org/10.18653/v1/E17-2004	conference	January 2017
Improving Neural Machine Translation Models with Monolingual Data Sennrich, Rico; Haddow, Barry; Birch, Alexandra Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) https://doi.org/10.18653/v1/P16-1009	conference	January 2016
NLTK: the Natural Language Toolkit Loper, Edward; Bird, Steven Proceedings of the ACL-02 Workshop on Effective tools and methodologies for teaching natural language processing and computational linguistics - https://doi.org/10.3115/1118108.1118117	conference	January 2002

Similar Records

LLMs for Mfg.—On the State of Large Language Models and Applications to Manufacturing

Technical Report · Mon Sep 01 00:00:00 EDT 2025 · OSTI ID:3002123

Multi-sensor text classification experiments -- a comparison

Technical Report · Tue Dec 31 23:00:00 EST 1996 · OSTI ID:638201

Enhancing Electron Microscopy Image Classification Using Data Augmentation

Conference · Sat Nov 16 23:00:00 EST 2024 · OSTI ID:2525894

Related Subjects

97 MATHEMATICS AND COMPUTING
ChatGPT
artificial intelligence
data augmentation
imbalanced data
large language model
machine learning
natural language processing
text classification

Improving Text Classification with Large Language Model-Based Data Augmentation

Citation Formats

References (15)

Similar Records

Related Subjects