The Ohio State University – The Renegade Coder

It's that time of the year again where we get feedback from our students. This semester, the format of the reports were a bit shocking, so I wanted to rant about that for a moment. Autumn 2025 Student Comments Today (12/23/2025), I opened up my student reviews from the semester, and I found that not a single comment from the open-ended portion of the survey is listed. Instead, the university has fed the student comments into some AI service and provided a summary of the feedback. Unlike previous semesters, there are now two open-ended questions, where the responses are summarized without student consent. The first of which is for a question which reads: "what specific aspects of this course were effective in promoting your learning (for example, teaching practices, assignments, class material, class structure)?" Here's what the results look like on my end: Sentiments[Polarity]Categorized topics[No. of comments]Overall[28]PositiveTeaching Effectiveness > Teaching Delivery > Class Discussions > Helpfulness14% (4)PositivePersons > Professor > Helpfulness14% (4)PositiveCourse Component > Course Material & Structure > Course in General > Structuredness11% (3) As you can see, the summary is unintelligible. You get broad categories of nothingness, with an extremely small sample fitting the category. Of course, I don't even think this table is as bad as the other one. The other open-ended question reads: "what changes, if any, could be made to improve the learning experience for future students (for example teaching practices, assignments, class material, class structure)?" Here's what that table looks like: Categorizedrecommendations[Type]Categorized topics[No. of comments]Do lessCourse Component > Course Material & Structure > Workload > Quantity8% (2)Do moreCourse Component > Assessment & Evaluation > Assignments > Quantity4% (1)Do moreEngagement4% (1) What could any of this possibly mean? Do more what? Engagement? Assignment quantity? What am I even reading? What am I supposed to do with this information? MLY Analysis? If you read the fine print around the table, you'll see the following: A MLY analysis of student comments identified the following topics and recommendations. The three most common topics or recommendations from each question are listed, along with the percent and total number of comments that were relevant. Instructors can access their MLY dashboard for more details and to view all comments. If this section appears empty, no comments were submitted or comments did not have enough information for MLY to identify insights. Now, I could not tell you what an MLY analysis is. The acronym is not defined in the report, nor does it seem to be defined on their website. All I can find is that it's pronounced "mi-lee", which is a horrific phonetic spelling. Is that a short or long "i"? The ML and Y are then subsequently separated into machine learning (ML) and "the pursuit of answers" (Y). What? Anyway, the MLY dashboard is equally as frustrating. I'm greeted with a broad set of analytics that load quite slowly, but they tell me the following: 127 comments with 70% positive, 6% negative, 17% not explicit, and 8% mixed sentiment 45 total recommendations with 17 "do more", 3 "do less", 6 "start", 6 "continue", and 13 "change" Aside from these general statistics, there are a few tabs at the top of the screen that read "overview," "widgets," "topic explorer," and "comment explorer." The widgets tab just shows a few graphs that take a hot minute to load. The graphs are virtually identical to the tables I shared above, and I found nothing worth reviewing. The topic explorer tab is similar. It's another tab that takes a solid two minutes to produce anything worth browsing, and the interface is atrocious. Imagine walking down a tree of nested topics (like the ones in the table above). It just doesn't seem all that useful to me. I could gather more out of skimming the actual comments. Of course, if I try to go to the comments directly, they're provided in a list in no particular order with sentiment labels. The labels are obviously annoying because they don't give me a chance to make my own judgment on the feedback. I go in already expecting the comment to be "negative," for example. I'm already having some sort of dystopic vision of a future where emails, messages, and posts are all tagged with a sentiment (that can easily be controlled by a third-party for profit), so I can know how to interpret them ahead of time. Also, because the comments are in no order, I have no way of telling which comment maps to which open-ended question. I can, of course, filter by question, but the filtering is comically slow. I also cannot seem to figure out how to export the comments that used to just be in the PDF report. It looks like something I should be able to do according to the MLY docs, but I assume my university has barred me from doing so. What a nightmare. I Hate This I can't articulate how much I hate this. MLY has no context about my course. It doesn't know what I teach, how my class is structured, or who my students are. Yet, we're going to rely on it to provide an accurate portrayal of the overall feedback. Keep in mind that student evaluations are borderline useless in that the research demonstrates high levels of gender- and race-based bias. Now, we're going to rely on machine learning models which likely reproduce those same biases to "objectively" analyze the feedback. It's absurd. I am also really uncomfortable with the idea of handing off supposedly anonymous comments to a service which relies on data to profit. Students never had a chance to consent to this kind of service, and I doubt many, if any, even know their feedback is being processed in this way. I almost feel bad now because I incentivize students to complete the survey because it's now legally a part of my annual review. Also, I just want my data. When students write those comments, they're writing them for me (or maybe to the department to get me fired). Without the ability to export the comments, I have to copy each one by hand before they delete them from the system (which used to happen annually for the old comments). Speaking of not having the data, I remember being told on my annual review about how great some of the comments were from students, with those comments often quoted. How are faculty meant to see what students are saying now? Do they have access to the same interface? Are they really going to check, or are they just going to review the PDF? Lastly, I assume students want us to read their feedback. That's one of the main reasons we collect feedback in the first place. We want students to feel heard. How do you think students will feel when I tell them that I don't have to read their comments anymore? I can't imagine they're going to be too happy. Also, I want to mention that I don't think there is any generative AI at play with this use case. Usually, I'm fairly okay with classification-style machine learning algorithms. For instance, I really like having people labeled in my photos, so I can find all of the photos that feature me, my wife, and my kid. In this case, I think classification is the wrong choice because qualitative data is inherently subjective. Relying on a classification model to categorize text assumes there is some objective way to do so. There's a reason why most qualitative researchers analyze interviews through an interpretative and/or critical lens. They're SEIs But Worse While this article largely covers the topic of machine learning usage on student evaluations, I would be remiss if I didn't briefly rant about the new evaluation format. As you may know, since SB1 in Ohio, what used to be called Student Evaluations of Instruction (SEIs) are now called Survey of Student Learning Experience (SSLE). Now, SSLEs are largely the same as SEIs, but the core questions have shifted from 10 questions to 7. The open-ended section has moved from a generic comment box to two open-ended questions. In addition, there are now three state-mandated questions, which cover ideas like intellectual diversity. I am actually a big fan of the new open-ended questions. Students seem to have written much better feedback than in the past, and I no longer have to prompt them ahead of time for it. On the flip side, a lot was lost in transition. For example, I no longer can see student comments. Instead, we're stuck with these bogus machine learning categories that strip all the feedback of its nuance. I have to log into a separate tool just to see the comments. Not to mention that the statistics no longer compare me with my peers in the department, college, and university, so I have no idea how I'm performing relative to everyone else. The SEIs used to list the mean score against those three cohorts, which I used to like to show off. Of course, I find the state-mandated questions a bit silly. What does it mean when 94% of my students answer "yes" to "are students encouraged to discuss varying opinions and viewpoints in class"? What does it mean when 100% of my students answer "yes" to "does the faculty member create a classroom atmosphere free of political, racial, gender, and religious bias"? Ultimately, I'm hoping that enough folks complain about the new format that we at least get all of the comments back in the PDF. I can live without the score comparison between cohorts. I'm also hoping that maybe students can opt out of their comments being reviewed by a machine learning model. Though, I'm not sure any of this will meaningfully change. Anyway, that's enough ranting for the day. I have so many articles I'm working on right now, that I just have to cut some of them off. This seems like as good a place as any. Of course, you're welcome to continue reading: Higher Education Should Not Be a Job Training Program Generative AI Makes It Feel Bad to Be an Educator It’s Time to Collect Mid-Semester Feedback If you want to show a bit more support, especially since I've been paywalling a lot of these pieces lately, you might check out my list of ways to grow the site. There, you'll find a link to my Patreon, so you can read my spicier takes. Otherwise, take care!