Tomorrow’s New York Times carries the second installment in the paper’s series “Grading the Digital School.” Like the first installment, this one finds little solid evidence that popular, expensive computer-aided instruction programs actually benefit students. The focus of the new article, written by Trip Gabriel and Matt Richtel, is Cognitive Tutor, a widely esteemed and much coveted software program for teaching math in high schools. The software was developed by Carnegie Learning, a company founded by Carnegie Mellon professors and now owned by Apollo Group, the same company that owns the University of Phoenix.
Carnegie Learning promotes its software as producing “revolutionary results.” It is widely used, and has been applauded by respected thinkers like the Harvard Business School’s Clayton Christensen, who in an article published by the Atlantic two weeks ago used Carnegie Learning as the poster child for the power of software-based education:
Carnegie Learning is the creation of computer and cognitive scientists from Carnegie Mellon University. Their math tutorials draw from cutting-edge research about the way students learn and what motivates them to succeed academically. These scientists have created adaptive computer tutorials that meet students at their individual level of understanding and help them advance via the kinds of exercises they personally find most engaging and effective. The personalization and sophistication is hard for even an expert human tutor to match. It is a powerful, affordable adjunct to classroom instruction, as manifest by Carnegie Learner’s [sic] user base of more than 600,000 secondary students in over 3,000 schools nationwide.
Sounds terrific. But, as the Times story documents, the evidence for Cognitive Tutor’s benefits is weak. In 2010, the U.S. Department of Education analyzed two dozen studies of the software program, and found that it “had no discernible effects” on math test scores for high school students. Another federal study, conducted a year earlier, examined ten leading software programs for teaching math, including Cognitive Tutor, and concluded that they had no “statistically significant effects on test scores.”
Test scores aren’t everything, of course, but it’s fair to say that one of the main reasons cash-strapped schools invest in the computer-aided programs, which can cost three times as much as traditional textbooks, is to boost students’ test scores. And companies like Carnegie Learning use the promise of improved test scores as a prime marketing pitch, sometimes backing it up with cherry-picked case studies or skewed research reports. Certainly, computer-aided instruction programs have a place in schools, but it’s increasingly clear that the benefits of the software have been oversold, and the faith that many educators place in the programs is often unwarranted. As Gabriel and Richtel report: “School officials, confronted with a morass of complicated and sometimes conflicting research, often buy products based on personal impressions, marketing hype or faith in technology for its own sake.”
Back in the early days of the personal computer, the late Steve Jobs, during his original stint at the helm of Apple Computer, played a major role in promoting the use of computers in education. But in a 1996 Wired interview, conducted after he’d left Apple and before he was rehired, he expressed a very different and much more wary view of the role of computer technology in schools:
I used to think that technology could help education. I’ve probably spearheaded giving away more computer equipment to schools than anybody else on the planet. But I’ve had to come to the inevitable conclusion that the problem is not one that technology can hope to solve. What’s wrong with education cannot be fixed with technology. No amount of technology will make a dent … Lincoln did not have a Web site at the log cabin where his parents home-schooled him, and he turned out pretty interesting. Historical precedent shows that we can turn out amazing human beings without technology. Precedent also shows that we can turn out very uninteresting human beings with technology. It’s not as simple as you think when you’re in your 20s – that technology’s going to change the world. In some ways it will, in some ways it won’t.
His view still seems pretty much on the mark.
A topic eloquently covered in Todd Oppenheimer’s 2003 book, The Flickering Mind: The False Promise of Technology in the Classroom and How Learning Can Be Saved.
The study makes sense because the same issue exists regardless if the delivery method is a piece of paper or a SW app; the student is still required to focus for a period of time with no distractions to be successful. They’re marketing from the angle that their app will help the student stay focused longer because the apps are customizable to the students areas of interest, but in reality it’s putting the student on a computer where there are a lot of distractions. So the solution needs to include a way to keep them focused on just that one app and remove access to other distractions while they’re on the computer.
I haven’t used the programs in question, so I don’t know how they function from a users’ end. Of course, the owners of the software product are going to try to fill their pockets as much as they can. I can just say that when I was in high-school, and even to this day, I harbored so much resentment for textbooks.
The textbook industry is no different from any other. Updating textbooks every year usually only serves the interests of the publisher. The size of textbooks was always impossible to get through. I had a friend in college who would spend the first couple of weeks of the session reading the textbooks in the bookstore. He saved a great deal of money and surged ahead of his classmates.
It seems to me that technological software is just more convenient than textbooks. I remember asking my chemistry teacher why not have a ringed binder with removable sections for each chapter. That way we wouldn’t have to carry around enormous texts and flip through them. We could support them while lying at the bed, before sleep. Of course, the teacher had me know that this was problem.
Buying a dedicated eReader was the great decision. I can carry multiple books at once, easily highlight and take notes in them. The biggest problem now is the accessibility of many books and the price. Of course, eBooks are cheaper than physical books, but the lending systems are virtually non-existent. Copyright laws also hinder the selection. The licensing problems holding down Pandora’s stock value are also evident within the eBook industry.
I would agree with Eileen that the method of delivery is, by in large, irrelevant. Steve Jobs was right–technology isn’t going to fix our education system. As you point out in your book, every technological innovation–print, radio, television, Internet–there are those that recognize the educative potential inherent within those media.
What separates classes and performance is the discipline–the discipline to solve a problem from start to finish, the discipline to dismiss shortcuts, the discipline to put in due diligence, the discipline to review. In many ways, it is now harder than ever to remain focused on the task-at-hand, to practice discipline.
Nick – That 1996 Wired interview of SJ is such a breath of fresh air. Thanks for the pointer!