Welcome to the WJETT blog
What is WJETT?
The WJETT blog or Warwick Journal of Education - Transforming Teaching blog is designed to encourage staff and students to disseminate good practice and to engage with their peers on academic cultural critique or areas of research that they find interesting. With the increased focus on ‘teachers as researchers’ in the sector, many qualified teachers are expected to publish the outcomes of any action research projects they undertake. The WJETT blog can be the first step on your journey towards publishing and enables you to experience publishing and reviewing in a friendly and supportive environment.
You will need to scroll down the page below the tags to see the posts.
Using AI for Formative Feedback: Current Challenges, Reflections, and Future Investigation
One strand of the WIHEA’s working group for AI in education has focused on the role of AI in formative feedback. As part of this strand, I have been experimenting with feeding my own writing to a range of generative AI (ChatGPT, Google Bard, and Microsoft Bing), to learn more about the sorts of feedback they provide.
The accompanying presentationLink opens in a new window documents my observations during this process. Some issues, such as the propensity of AI to ‘hallucinate’ sources, are well-documented concerns with current models. As discourse on student use of AI begins to make its way into the classroom, these challenges might provide a basis for critical discussion around the accuracy and quality of the feedback produced by language models, and the need for student to review any outputs produced by LLMs.
Other common issues present different challenges for students using LLMs to elicit formative feedback. For instance, the prompt protocol in the presentation revealed a tendency for AI to provide contradictory advice when its suggestions are queried, leading to a confusing stance on whether or not an issue raised actually constitutes a point for improvement within the source text. When tasked with rewriting prompt material for improvement, LLMs consistently misconstrued (and therefore left absent) some of the nuances of my original review, in a fashion which changed key elements of the original argumentation without acknowledgement. The potential challenges for student users which arise from these tendencies is discussed in more detail in the presentation’s notes.
In addition to giving some indication of the potential role of LLMs in formative feedback, this task has also prompted me to reflect on the way I approach and understand generative AI as an educator. Going forward, I want to suggest two points of reflection for future tasks used to generate and model LLM output in pedagogical contexts. Firstly: is the task a reasonable one? Using LLMs ethically requires using my own writing as a basis for prompt material, but my choice to use published work means that the text in question had already been re-drafted and edited to a publishable standard. What improvements were the LLMs supposed to find, at this point? In future, I would be interested to try eliciting LLM feedback on work in progress as a point of comparison.
Secondly, is the task realistic, i.e. does it accurately reflect the way students use and engage with AI independently? The review in my presentation, for example, presupposes that the process of prompting an LLM for improvements to pre-written text is comparable to student use of these programmes. But how accurate is this assumption? In the Department of Applied Linguistics, our in-progress Univoice project sees student researchers interviewing their peers about their academic process. Data from this project might provide clearer insight into the ways students employ AI in their learning and writing, providing a stronger basis for future critical investigation of the strengths and limitations in AI’s capacity as a tool for feedback.
Please see the diverse assessment series for further posts you may find interesting:
Writing guidance
Can I write about anything in my blog post?
Yes pretty much. Academic cultural critique (Thomson and Mewburn, 2013) is always a good source of content for academic blogs. This can include (but is not limited to) comments and reflections on funding; higher education policy or academic life. You might also want to consider blogging about:
- Academic practice (Saper, 2006)
- Information and/or self-help advice
- Technical, teaching and careers advice
- Your research or practice
- How you’ve undertaken research
- The impact of research on your practice
- An area of research/practice that interests you
- Your teaching experiences/reflections
How long can my blog post be?
Each individual blog post should be no longer than 500 words. Long blocks of text are sometimes hard for readers to digest. Break up your content into shorter paragraphs, bullet points and lists whenever possible. Also include a list of keywords or tags as this makes it easier for Google to find your work.
Do I need to use citations?
No, this is a reflective piece so it does not need to include citations (but you obviously can include them if they are relevant).
Can I include links or images?
We would encourage you to include links to any articles that you have considered whilst writing your blog post. We also welcome the use of images (as long as you have permission to use them) as they can often help to illustrate a point and obviously will not be included in the word limit. Please remember this is a public site so if you want to include images of your students in your classes then you will need permission to do this.
What is the process for submitting a piece of work?
Your blog post should be emailed to A.Ball.1@warwick.ac.uk. Once the submission has been reviewed it will either be uploaded at the beginning of the next available week or sent back to you for editing if it requires amendments. You should then send the amended work to me once again and I will then upload it.