The pain points of teaching computer science

This post is an informal summary of our SIGCSE'23 paper, "What Is Your Biggest Pain Point? An Investigation of CS Instructor Obstacles, Workarounds, and Desires". See the preprint for more details. Thanks to Samim Mirhosseini and Chris Parnin for the great collaboration!

See the discussion of this post on Hacker News.

Teaching is hard.

But exactly what about it is hard? How could technology help?

To investigate the pain points that CS instructors face and their workarounds, we conducted semi-structured interviews with 32 computer science instructors. Their institutions range from large research universities to small liberal arts colleges and community colleges across 7 different countries.

We asked open-ended questions about lecture structure, availability of resources such as TAs and software, time spent on various activities, techniques used to improve learning experiences, the most painful aspects of running courses, and potential solutions to their pain points.

Pain points

We identified 6 categories of pain points:

• Where are students struggling? It is a huge challenge for instructors to know when students are struggling. There is often not an effective feedback loop since instructors often rely on students to ask questions. However, two instructors expressed why this doesn't work: "I can't see where they get stuck and many don't ask questions" and "students are doing lots and lots of things but I can't process all they are doing".
• Answering student questions. In contrast, instructors and TAs are overwhelmed with responding to questions and problems. These questions are often last minute, repetitive in nature, and require technical troubleshooting. For example, "Why does Python no longer exist?".
• Limited TA support. You might be thinking that teaching assistants will just solve all these challenges! Sadly, most departments are understaffed with TAs, if they have any at all, despite growing enrollments. Instructors said that even when they have TAs, they sometimes require considerable time to manage and may not be reliable.
• Grading and feedback. Nearly every instructor brought up the time and tedium of grading. In fact, one instructor told us "grading is probably the biggest burden of the courses". Instructors also described how critical it is to design a rubric and feedback mechanism that is transparent and minimizes lawyering.
• Course material preparation. Instructors lamented about the high cost of creating and maintain course materials, such as lectures, assignments, and quizzes. They often don't update them over the years because they "just don't have the resources". The shift to online courses because of COVID-19 required significant upfront investment, though instructors complained they saw little engagement from students.
• Administrative tasks. The grunt work of running a course includes managing social dynamics, accreditation tasks, enforcing academic honest policies, dealing with LMS issues, etc. Several instructors said they wished they could spend this time improving the course instead. Dealing with cheating was probably the least favorite task of all!

Workarounds

We also identified 5 categories of how instructors attempt to overcome these pain points:

• Interactive textbooks and exercises. To minimize developing their own course materials and the grading of assignments, a few instructors rely on paid interactive textbook services that are regularly updated. This also encouraged a more active learning experience, since the alternative for many instructors involves static notes or a physical textbook.
• Online IDEs and visualizers. To eliminate environment issues, increase engagement, and provide faster feedback, instructors leveraged online coding tools. Supporting students' technical issues with their local development environments could be a full-time job in itself, so online IDEs completely eliminate this pain point. Instructors were particularly impressed by educational tools that show visualizations of the students' code.
• Automated grading. A long time dream of CS instructors has been to completely automate grading. Many mentioned the use of test cases and naive grading scripts, and some instructors mentioned that AI graders may soon be viable. However, there seems to be a tension here: instructors also pointed out the need for the "human touch".
• Flipped classroom. To better understand where students are struggling, instructors have switched to in-class activities while having recorded lectures for homework. This has the added benefit that students can rewatch or rewind the lecture. Several instructors switched to this model because of COVID-19 but stuck with it after seeing the benefits.
• Peer instruction. To provide better feedback and overcome the lack of TAs, instructors have experimented with peer instruction. For example, one instructor has multiple office hour sessions with small groups. Others require student-led study groups. They've also found undergraduate TAs to be effective and cheap to scale core courses.

There are many opportunities to improve the lives of instructors and students in computer science courses. As class sizes increase, how is an instructor to know when a student struggles with the same concept across multiple assignments? How can they provide fair mechanisms for students of various backgrounds to seek help? How do instructors keep their courses modern while sufficiently attending to all the administrivia? What is the right balance of automation and human touch?

What role does AI play, especially those powered by large language models like ChatGPT and Copilot?

Read more in our SIGCSE'23 paper: What Is Your Biggest Pain Point? An Investigation of CS Instructor Obstacles, Workarounds, and Desires.