As machine learning (ML) systems are increasingly used to make high-stakes decisions in healthcare, hiring, finance, and criminal justice, ensuring their fairness has become critical. However, these models have proven to be susceptible to bias against protected attributes such as race, gender, age, and others when trained on biased data. Our work investigates whether large language models (LLMs), such as OpenAI's ChatGPT, can generate fair and realistic tabular data to make ML models less biased while maintaining predictive accuracy.
This work examines LLMs' synthetic data generation capabilities on two benchmark datasets in ML fairness: Income, derived from U.S. census data to predict income classification, and COMPAS, a criminal justice risk assessment dataset used to predict recidivism. We evaluate model performance using multiple fairness metrics, comparing classification models trained on the original imbalanced datasets to those trained on LLM-generated synthetic data. Our research extends existing studies by using an active learning-inspired approach to select the most informative data points from the original data to provide to the LLM as in-context examples for synthetic data generation. We evaluate different prompting strategies, number of in-context examples, and batch sizes of generated samples to determine how these factors influence fairness and accuracy.
Our results demonstrate that LLMs can be utilized to generate realistic and fair synthetic data. However, limitations such as the inconsistencies in LLM outputs, closed-source decision-making of LLMs, and the computational costs to generate data using LLMs present challenges to fully leveraging their potential for fair data generation.
This work examines LLMs' synthetic data generation capabilities on two benchmark datasets in ML fairness: Income, derived from U.S. census data to predict income classification, and COMPAS, a criminal justice risk assessment dataset used to predict recidivism. We evaluate model performance using multiple fairness metrics, comparing classification models trained on the original imbalanced datasets to those trained on LLM-generated synthetic data. Our research extends existing studies by using an active learning-inspired approach to select the most informative data points from the original data to provide to the LLM as in-context examples for synthetic data generation. We evaluate different prompting strategies, number of in-context examples, and batch sizes of generated samples to determine how these factors influence fairness and accuracy.
Our results demonstrate that LLMs can be utilized to generate realistic and fair synthetic data. However, limitations such as the inconsistencies in LLM outputs, closed-source decision-making of LLMs, and the computational costs to generate data using LLMs present challenges to fully leveraging their potential for fair data generation.