How generative AI like ChatGPT is pushing assessment reform

With the advent of ChatGPT, the impact on the tertiary education sector from generative artificial intelligence (GAI) has been brought to the fore and, along with it, considerable uncertainty about how GAI will affect academic assessment and integrity.

In a recent interview, Microsoft cofounder and philanthropist Bill Gates referred to AI advancements as the foremost innovation of our era, one that “will change our world”.

Universities seem likely to benefit from GAI in ways such as assistance with marking, commentary on assignments, customised learning experiences and automation of administrative tasks such as answering frequently asked questions.

On the other hand, GAI also presents challenges to universities because students might misuse it to complete their assessments. GAI, such as AI writer ChatGPT, which was launched in November 2022, can easily provide answers to assessment types such as multiple-choice questions, short-answer questions, true/false questions, open-ended questions, mathematics problems, programming assignments, essay writing and even content creation for presentation slides.

Some academics think that if they use tables or figures in their assignments or exams, students won’t be able to use ChatGPT to produce answers.

So, in order to understand ChatGPT’s functionalities, I fed it a table containing raw data. ChatGPT was able to respond to questions about it as well as draw flowcharts and mind maps. The AI writer also calculated simple parameters such as mean and variance and was able to run complicated analytics techniques on the data.

ChatGPT can research a specific topic and show the results in a table. For example, you can ask ChatGPT to find three projects similar to a project you are working on and draw a table to show year completed, private or government status, link to the website and project location.

I used an example to see if ChatGPT could interpret a figure. I fed ChatGPT a line graph (below) and included the prompt: In one paragraph, write a thorough interpretation of the figure below in terms of the fluctuation of the activity level and project time.

Here is the response ChatGPT provided, which doesn’t seem bad at all (ChatGPT by Open AI accessed on 8 March 2023).

The figure represents the five project management process groups, including initiating, planning, executing, monitoring and controlling, and closing, in terms of their respective activity levels and project time. The initiating process group is shown as having a relatively low activity level and short durations, as it involves the initial definition of the project and its objectives. The planning process group has a higher activity level and longer durations, as it involves the development of a comprehensive project plan. The executing process group is represented as having the highest activity level and longest durations, as it involves the actual implementation of the project plan. The monitoring and controlling process group has a fluctuating activity level, as it involves continuous monitoring and adjustment of project activities as required. Finally the closing process group has a relatively low activity level and short duration, as it involves the wrap-up and closure of the project. Overall, the figure demonstrates how the activity level and project duration fluctuate over the course of the project life cycle, with the highest activity level and longest duration occurring during the executing process group.

Students’ access to GAI will make their responses to many types of assessments highly questionable. This is especially the case when students complete these assessments outside class time or have access to computers during the assessment.

The question here is whether universities should ban GAI. And, if so, are they able to do this?

The answer to both questions is no. Even if universities banned GAI on their networks, students could still use GAI platforms at home or on other networks.

What universities need to do urgently is review the GAI technology and its implications for their institutes, formulate relevant policies for its use, train staff and students and, more importantly, reform the assessment system used in their subjects.

Recommended assessment types to mitigate AI use

These assessment types can help universities to minimise the adverse effects of GAI:

1. Staged assignments: Break the assignment into stages and ask students to submit a report for each stage; these reports are reviewed and commented on by the instructor. Students work on later stages as per comments received on earlier stages.
2. In-class presentations followed by questions: Ask students to give presentations in front of the class; the lecturer then asks students questions about different parts of their presentations. Note that students could still use GAI to generate slide content for them, but by requiring students to present in front of the class and asking random questions, the instructor can genuinely evaluate students’ learning.
3. Group projects: Students are less likely to cheat if they work in groups. Task students with a project that interests them in a group format. They write a project report as well as present the project outcome. Note that individual students might be able to use GAI to find answers to parts of the project allocated to them, but through asking follow-up questions during the project presentation the instructor can evaluate students’ learning.
4. Personal reflection essays: Students write about what they have learned about a specific topic and how they could apply this knowledge in their future careers. This will make it difficult to use GAI to cheat because it needs comprehension of a large volume of subject material.
5. Class discussion: Provide a topic to the class and have each student present their opinion about the topic or comment on each other’s comments. In face-to-face classes where the use of laptops or mobile phones is not allowed during the discussion, this will make it almost impossible for students to use GAI to produce responses.
6. In-class handwritten exams: This type of assessment should be done in class without access to computers. By cutting students’ access to computers, laptops and mobile phones during the exam, students are not able to use GAI to cheat.
7. Performance-based assessments: Disciplines such as music, dance, visual arts and architecture would be more suited to this type of assessment.

The arrival of GAI calls for innovative assessment approaches. The need for such a reform is urgent, and failure to act on it could pose a major threat to any university subject that has been run since 30 November 2022.

So, while platforms using GAI such as ChatGPT hold great potential to transform higher education, universities must address several challenges posed by this technology with care in order to optimise its benefits and minimise associated risks such as threats to academic integrity.