Text Summarization Using Large Language Models: A Comparative Study of MPT-7b-instruct, Falcon-7b-instruct, and OpenAI Chat-GPT Models
Project Overview
The document explores the application of Large Language Models (LLMs) in education, particularly focusing on their effectiveness for text summarization within Natural Language Processing (NLP). Through a comparative analysis of models such as MPT-7b-instruct, Falcon-7b-instruct, and OpenAI's ChatGPT (text-davinci-003) across datasets like CNN/Daily Mail and XSum, it evaluates their performance using metrics such as BLEU, ROUGE, and BERT scores. The results reveal that ChatGPT (text-davinci-003) excels in generating high-quality summaries, highlighting the significant potential of generative AI tools in enhancing educational outcomes. These findings suggest that LLMs can effectively assist in various educational applications, including improving reading comprehension and facilitating efficient information processing for students and educators alike. By leveraging the strengths of these advanced AI models, educational stakeholders can harness technology to foster better learning experiences and outcomes.
Key Applications
Text summarization using Large Language Models (LLMs) like text-davinci-003, MPT-7b-instruct, and Falcon-7b-instruct.
Context: Research on text summarization methods in NLP, targeting researchers and practitioners in the field.
Implementation: Conducted experiments using different hyperparameters and evaluated summaries generated by LLMs on specific datasets (CNN/Daily Mail and XSum).
Outcomes: Demonstrated varying performance of LLMs with text-davinci-003 achieving the highest quality scores in summarization tasks.
Challenges: Limited availability of high-quality, domain-specific summarization datasets.
Implementation Barriers
Data availability
High-quality, domain-specific summarization datasets are often scarce or costly to obtain.
Proposed Solutions: Utilize unsupervised summarization methods or leverage existing datasets for training LLMs.
Project Team
Lochan Basyal
Researcher
Mihir Sanghvi
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Lochan Basyal, Mihir Sanghvi
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai