Taking a Hard Look at Khan Academy

The College Board is excited. For the FIRST TIME EVER the College Board is offering FREE practice for the SAT. At its annual forum, College Board widely and enthusiastically touted its new partnership with Khan Academy because it brings free SAT practice to all students, thereby finally leveling the playing field. Now any student willing to put in the work will be able to improve his or her scores through easily accessible example videos, practice drills, tests, and study plans. Its not test prep; its productive practice. So goes the pitch to educators. Students, however, arent just getting this pitch; theyre getting the hard sell. When students go to the College Boards website to register for a test, they are directed to create an account at Khan and start practicing. When students receive their PSAT reports, their results will be ported over to the Khan site and analyzed to form a personalized study plan. That is, the data from the PSAT will be used to identify each students current competency level across an array of math and verbal topics. Those competencies inform the order in which topics are presented to each student. Its simple and elegant. Who wouldnt get excited about free, personalized practice for all students? I agree that Khan Academys SAT practice section is a valuable resource for students who would otherwise have no access to test prep. Khan Academy is not, however, a good place to take practice tests. Claims that it will finally level the playing field are overstated, for it is not a substitute for the kind of thoughtful coaching that has historically helped students improve their performance. At best, and with the following caveats, its a good place to do drills. Khan Academys structural weaknesses undermine productive practice. There are two fundamental issues with prepping through Khan Academy: (1) Screens are not paper. This statement is obvious in the abstract but tends to get left behind when digital proponents enthusiastically speak of increasing accessibility. Its important to remember that answering questions on a computer is a very different kinesthetic experience from answering questions on paper with a pencil, and both modes require different strategies. You probably wouldnt practice tennis on a clay court in order to compete on a grass one just as you wouldnt type an essay to practice penmanship. Its practically axiomatic that students should practice the way theyll play. (2) Competency levels can be deceiving. Because Khan Academy divides skills into four levels, it risks giving students a false sense of mastery of the exam. A student could, for instance, understand a comma splice and even identify and fix it during a drill section on combining sentences, thereby earning the highest level of competency. Having mastered that rule, the student sets it aside. In a live test, however, the student could very well miss it because the greater context or time pressure proves distracting. It happens all the time. Focusing on passage content oversimplifies the reading task. You may remember that Khan Academy began in 2006 as a series of YouTube videos in which Sal Khan tutored his cousin in math. It grew into a site with mini-lectures on mostly math and then some science topics; only more recently has it begun to tackle other fields. I would not be the first to offer a critique of how Khan teaches math, but I would suggest that Khan Academys limited experience with the humanities becomes apparent when we analyze the organization of SAT reading and writing practice. For the sake of brevity, Ill ignore the fact that as of this writing nearly one-third of the writing question types are missing from the drill sets (all of the development and organization questions), and will instead focus on the reading section. The Khan site organizes reading skills by passage genres: science, literature, history, and social science. The assumption is that if a student has a weakness, it will be with a kind of passage rather than a kind of question or kind of comprehension task. In my ten years of working with students, Ive come to believe that helping a student learn how to employ comprehension skills across passage types – e.g. defining the authors argument, decoding the meaning of words in context, identifying the implication of a claim – is more important than exposing a student to particular genres. Appreciating a passages content matters less than thinking critically about what a multiple-choice question is asking and how the answers are worded to distract or confuse a reader. Most students understand and can summarize a passage in their own words better than they think they can; its the questions and answer choices that lead to their confusion. Upon answering a question incorrectly, a student should either be given an explanation, or preferably be asked questions that allow him to figure out where he went wrong. But the explanations offered on Khans site are just as general and unhelpful as the generic classification system. They fail to diagnose where a student has made a comprehension error. They assert rather than illuminate. The kind of granular analysis and personal feedback that could help a student recognize actual weaknesses is currently missing from the reading practice. By structuring its reading practice by genre and offering little ancillary support, Khan Academy reinforces the notion that some students are simply bad at comprehending certain subjects. Khans videos model inconsistency. For the sake of argument, lets accept Khans approach and say that a student consistently struggles to understand science passages. He or she could turn to the How-to videos (one passage per genre) presented alongside the practice sets. Unfortunately, these videos fail to offer actionable strategies for breaking down the passage and answering the questions; students simply observe Sal Khan reading the passage and then answering the questions out loud. Everything about his approach is unconscious and inconsistent – the two qualities any qualified tutor would attempt to eliminate from a students process. As one of many possible examples, Khan underlines portions of the science passage but never talks about when or why a student might want to underline or how that underlining might be used later. He simply repeats phrases like this is really interesting. By the time he reaches later passages, hes dropped underlining altogether, implying that its not an important strategy after all . The videos in which he answers questions arent much better. Once again, his approach is inconsistent: sometimes he uses process of elimination and sometimes he jumps to the right answer. He frequently says things like well, no, this doesnt make sense without further explanation. He rarely returns to the passage to justify an answer (which is ironic given the College Boards emphasis on evidence-based reading skills), relying instead on his memory of the passage. Indeed, he exhibits many of the habits of smart students whose scores hit a ceiling because of careless errors. As models, Khans videos are not only problematic, but are also based on an assumption that viewing a display of skill is equivalent to gaining that skill. Khan Academy is like learning to play golf by watching Tiger Woods and then going to a driving range. Most people will get frustrated when their swings are ineffective. Most people need an experienced golfer to watch their swing closely and offer some tips and tweaks. The personalization offered by Khan Academy is less like a golfing lesson and more like a list that tells you to practice ten swings with your 5-iron, ten with your 9-iron, and ten with your driver. Its a drill, sure, but its just as likely to entrench bad habits as it is to help improve your game. Learning skills requires more than technology. The College Boards current slogan for its partnership with Khan – Skills Arent Bought. Theyre learned. – pushes back against the longstanding (albeit mistaken) perception that students can beat the test by spending lots of money to learn a few tricks. But as weve written about before, effective test prep is not about tricks. High quality tutoring is about identifying gaps in knowledge and helping students not only review what theyve learned in school, but also develop test-taking skills. While Khan Academys free practice is better than no practice, its unlikely that even extensive technology-guided practice will prove more beneficial than the focused guidance of a high-quality teacher or tutor assigning regular timed drills and practice tests. Most students succeed when an expert asks the right questions, listens to their responses, and adjusts lessons accordingly. Education technology is improving rapidly, but computers are still a long way from replacing educators.