The problem with generalizability is huge. Gains from using Khan Academy tutorials are often seen among a small subset of students—typically those who are already high-performing, motivated, or from higher-income backgrounds. I’ve seen this phenomenon referred to as the “5 Percent Problem.”
This info comes from the Harvard study. “The present study took place in the Fall 2023 semester in Physical Sciences 2 (PS2), which is an introductory physics class for the life sciences and is Harvard’s largest physics class (N=233).”
Harvards acceptance rate is, what, 4%? Do we have a “four percent problem”?
I definitely agree. If students are already high-performing and motivated, then they tend to use any tools that are given to them. Still, the study is still insightful - more research needs to be done!
Hallo, I came across this article as I was looking for more info about this research. I found it very helpful, so thank you for sharing it.
While I'm impressed, there are a few things I might need to understand better. Some people, and a significant number at that, used the AI tutor for much longer than the active session. Surely this must be factored in, when comparing the results between AI tutors and active classroom sessions. One could say that this is an intrinsic advantage of AI tutors, but still, it is no insignificant point.
On the other hand, a significant number of students used the AI tutor quite sparingly, to say so. In the graph about time on task, more than 10% of the students used it for a very short time (when adding the columns in the graph, I count 140 student -I hope I wasn't wrong counting, since N=194). This might mean the AI tutors do wonders (could really be), but also that these students already had a good grasp of the topic at hand, or at least prior knowledge, although this shouldn't be the case, according to the authors.
Thanks for your thoughtful comments. I agree with both of your points and more research is definitely warranted. We're doing some ourselves - watch this space!
Great post! I want to write to you, Dr Oakley and others involved with her Dec 13 Deep Learning/Coursera Cheery Friday email about how to organize a pilot project to do some research related to active SDGs learning using AI in classrooms. Can you suggest links to research already done or underway on that topic?
Well done supporting propaganda research designed to replace human teachers and homegenize education to the point where the controlling parties can brainwash entire generations into believing anything programmed in by whomever owns the tech. Maybe wait for peer review?
The problem with generalizability is huge. Gains from using Khan Academy tutorials are often seen among a small subset of students—typically those who are already high-performing, motivated, or from higher-income backgrounds. I’ve seen this phenomenon referred to as the “5 Percent Problem.”
This info comes from the Harvard study. “The present study took place in the Fall 2023 semester in Physical Sciences 2 (PS2), which is an introductory physics class for the life sciences and is Harvard’s largest physics class (N=233).”
Harvards acceptance rate is, what, 4%? Do we have a “four percent problem”?
I definitely agree. If students are already high-performing and motivated, then they tend to use any tools that are given to them. Still, the study is still insightful - more research needs to be done!
Hallo, I came across this article as I was looking for more info about this research. I found it very helpful, so thank you for sharing it.
While I'm impressed, there are a few things I might need to understand better. Some people, and a significant number at that, used the AI tutor for much longer than the active session. Surely this must be factored in, when comparing the results between AI tutors and active classroom sessions. One could say that this is an intrinsic advantage of AI tutors, but still, it is no insignificant point.
On the other hand, a significant number of students used the AI tutor quite sparingly, to say so. In the graph about time on task, more than 10% of the students used it for a very short time (when adding the columns in the graph, I count 140 student -I hope I wasn't wrong counting, since N=194). This might mean the AI tutors do wonders (could really be), but also that these students already had a good grasp of the topic at hand, or at least prior knowledge, although this shouldn't be the case, according to the authors.
Could it be that more research is warranted?
Thanks for your thoughtful comments. I agree with both of your points and more research is definitely warranted. We're doing some ourselves - watch this space!
Great post! I want to write to you, Dr Oakley and others involved with her Dec 13 Deep Learning/Coursera Cheery Friday email about how to organize a pilot project to do some research related to active SDGs learning using AI in classrooms. Can you suggest links to research already done or underway on that topic?
Well done supporting propaganda research designed to replace human teachers and homegenize education to the point where the controlling parties can brainwash entire generations into believing anything programmed in by whomever owns the tech. Maybe wait for peer review?