Reviewriter: AI-Generated Instructions For Peer Review Writing
Project Overview
The document examines the creation and assessment of Reviewriter, an AI-driven tool aimed at facilitating the peer review process for students by offering AI-generated guidance. It addresses common obstacles like writer's block and seeks to improve the quality of peer reviews in educational contexts. The study reveals that students generally responded favorably to the tool, noting enhanced usability and motivation in their writing tasks. However, it also identifies ongoing issues regarding the relevance and accuracy of the AI-generated instructions, indicating that while generative AI can significantly support educational activities, there are still areas that require refinement to maximize its effectiveness. Overall, the findings suggest that with further development, generative AI tools like Reviewriter could play a transformative role in enhancing student engagement and writing skills in peer review scenarios.
Key Applications
Reviewriter - AI-generated instructions for peer review writing
Context: Higher education, specifically for graduate students writing peer reviews in German.
Implementation: Developed and fine-tuned language models on a corpus of student-written peer reviews. Conducted user interviews to gather requirements and designed an intuitive web application based on these insights.
Outcomes: Students reported high ease of use, enjoyment, and intention to use the tool for writing. The tool helped mitigate writer's block and provided diverse ideas for writing.
Challenges: The application can produce factually incorrect or irrelevant outputs, known as hallucinations. There are also concerns about the delay in generating suggestions, which some students found disruptive.
Implementation Barriers
Technical Barrier
The generated content sometimes includes factual inaccuracies and irrelevant phrases, affecting the quality of assistance provided. Additionally, there are issues with the speed of generating suggestions, which can disrupt the writing flow for students.
Proposed Solutions: Further research on improving the accuracy and relevance of AI-generated instructions is necessary. Enhancements to the algorithms and models based on user feedback can mitigate these issues. Implementing a strategy to better time the delivery of suggestions could also reduce disruption, allowing students more control over when they receive assistance.
Project Team
Xiaotian Su
Researcher
Thiemo Wambsganss
Researcher
Roman Rietsche
Researcher
Seyed Parsa Neshaei
Researcher
Tanja Käser
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Xiaotian Su, Thiemo Wambsganss, Roman Rietsche, Seyed Parsa Neshaei, Tanja Käser
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai