The Science Fair and The Troubleshooting Game

It is early February.  E-Rate applications (to help our client schools obtain Federal funding for technology) are due any day now, but nevertheless I am volunteering a bit of time to serve as a judge at a  Science Fair.  After all, everyone wants to encourage students’ interests in Science, Technology, Engineering and Mathematics [STEM].

The project I am discussing with the other judges is going to end up being awarded an Honorable Mention.  The student looked at factors influencing plant growth.  Some plants were given water, Miracle Grow, and sunlight.  Others were given water and sunlight, but no Miracle Grow.  And so on.  Some were just given water, and the last group was given Clorox Bleach instead of water.  The student’s hypothesis was that the plants given water, Miracle grow, and sunlight would thrive and that the Clorox plants would die.  There were weekly measurements of plant height and leaf size over several months.  The display was neat and it was apparent that a computer had been used to prepare a spreadsheet showing all the data including graphical representations of the effect of various factors.  The key elements of “the scientific method,” as prescribed by the Fair rules, were clearly present.  So why was I troubled by the award of an Honorable Mention?  To me, an “interesting” science experiment is one where you offer a hypothesis that stretches known science — at least a little bit — and where your predicted outcome flies in the face of conventional wisdom.  For example, if the Clorox plants had flourished, that would be interesting!  The scoring rules limited our options, but my position was that this project showed no imagination, no questioning of textbook knowledge, no risk-taking, no innovation, no improvisation.  Yes, it did show ability to follow directions, ability to recite the steps of the scientific method, and an all-too-rare work ethic.  I found it troubling that, from the moment this project was conceived, it was a slam dunk; student, parents and teacher could be confident it would earn a good score.  I worry that, too often, such well-intended activities merely enable students to learn “about the scientific method;” they do not necessarily enable students to experience what “being a scientist” is all about.

In our summer camps and after school programs, we play a different game, one that involves troubleshooting with technology.  Teams of students are each allowed to insert one or more faults into a computer system or network.  (We start with one hardware problem and later progress to software problems, network problems, and situations were there are multiple faults.)  Before they are allowed to insert the fault, they must make a prediction as to what the effect will be on system operation.  Then they must prepare a “Trouble Ticket” that makes a clear distinction between observed symptoms and underlying causes.  Another team must then diagnose and repair the fault.  There are many variations of The Troubleshooting Game.  Sometimes, Instructors set up the faults, to illustrate particular issues.  Sometimes, it is a Help Desk scenario, where students must troubleshoot over the telephone, without being able to see the computer screen or other faulty device.  Often it is a timed race.  The grand finale might involve students inserting a complex cluster of faults, after which it becomes the task of the Instructor to face such a challenge with an honest risk of failure.  It isn’t “rigged.”

The Troubleshooting Game of course provides practice for skills ranging from, “make sure everything is plugged in,” to “turn it off, count to ten, and turn it back on” to “listen for the secret phrase.”  However, when these simpler, “triage” tricks do not solve the problem, a more careful reasoning process must then be used.  For example, suppose there are two identical computer systems.  One is working normally, but in this case the screen on the second system is dark.  What might be wrong?  To formulate possible hypotheses about the fault, we must reason backwards from symptoms to possible underlying causes.  Perhaps the brightness was turned down to zero?  Perhaps the monitor is defective?  Perhaps the video card has failed (or is just not seated correctly in the slot)?  Is there an experiment we could devise that would provide evidence supporting some hypotheses but not others?  What about swapping the monitor from the failing computer with the “known good” monitor from the working computer?  If the symptom “follows” the monitor, what should we conclude?  These student-devised experiments must be improvised on the spot, in collaboration with a team and under time pressure.  Again, the activity is not “rigged” because the “right answer” is not known in advance.

Now let’s suppose the “known good” monitor works correctly on the formerly dark computer.  Does this prove that the problem was a bad monitor?  Let’s check!  Let’s try putting the allegedly bad monitor on the known good cpu.  In this case, strangely, it works!  OK, then, let’s try putting the allegedly bad monitor (which has just been shown to work correctly on the good cpu) back on the originally dark computer.  Wow!  Now both computers are working fine!  How can this possibly be?  In this case, the cable connection from the monitor to the computer might have been just loose enough to cause the dark screen symptom.  The very process of doing our fault isolation experiments resulted in correcting the problem.  (This sort of “observer effect” is reminiscent of Heisenberg’s Uncertainty Principle.)

In my view, the experience of students playing our little game feels more like “being a scientist” than at least some of the Science Fair projects I have been asked to judge.  Except for the most trivial or contrived examples, the reasoning involved in troubleshooting corresponds precisely to the steps of the scientific method, as applied to human-designed (rather than nature-designed) artifacts.  Experiments tend to be easily replicated and data tends to be somewhat less ambiguous, which seems fine for students lacking the resources or statistical training to require p < .05 before an “Aha!” moment is allowed.   Students love “The Game” and beg for additional opportunities to play it.

Once you hear the secret phrase, you know you are getting warm.

A teacher presents with this problem: many parents say they aren’t getting her electronic newsletters.

After some investigation, I notice that some of her messages are “stuck” in the email queues on the server.  Since I deal with these sorts of issues often, I immediately suggest double-checking whether the addresses on a few of the stuck messages are incorrect.  Although it is not unusual for people to say that they double-checked, often there is a typographical error.  “No, that’s not it.”  “Well, OK, at least the syntax of the email addresses seems to be correct.  Let’s explore some other hypotheses.”
Next, I notice that the date on the stuck messages is 12/31/1969; but this is 2008.  “Aha!” say I. Perhaps some of the servers at the parental receiving end are not accepting the messages based on the absurd date.  And, for those cases where the messages did not appear stuck in the queue, yet the intended recipients still claim that they never got them, very likely they are buried in an automated Junk folder or just at the wrong end of a very large inbox sorted by date. “Obviously, the clock battery is dead on your computer.  If you replace the battery and then set the date correctly, this will stop happening.”  I provide detailed instructions on how to remove the old battery and take it to Radio Shack.  “It will cost about $12.  Do not tell the person at the store that it happens to be for a Macintosh computer.”  (Store personnel will invariably say that they do not carry parts for Macintosh computers.)  Great!  Problem solved … or so I thought.
A few weeks later, I get another call from the same teacher.  “It is still happening.”  After some back and forth to verify that the battery was indeed replaced and that the computer is now showing the correct date and time, I have to admit that I am stumped.  Perhaps there is a wrong date on the school firewall or a proxy server? Perhaps the teacher uses one computer to send the newsletters but a different one for daily work such as individual email?  The conversation goes on, via electronic mail, for several more weeks.  I am truly baffled.  Then one day, it happens, the email equivalent of a casual comment in the hallway: “well, actually, the newsletters are being sent out by the FileMaker server; but that shouldn’t matter, should it?”  The secret phrase, at last!
Troubleshooting and its software equivalent, Debugging — whether it is for an electronic mail delivery problem or an overly enthusiastic toaster or a sound system where one channel is silent — these are the topics and thinking skills that have always fascinated me.  Sadly, this topic is not even mentioned in the “core content standards” taught by our schools.  Yet, people who are crack troubleshooters become far more valuable employees in practice (whether in IT or in countless other fields) than those who lack this skill but who list A+, MCSE, CCNA, Ph.D. or whatever other forms of certification on their résumés.  Often the clues to what is going wrong come from observations of the user, the environment, the organization or the situation, rather than the faulty device itself.

The above anecdote illustrates just one example of a powerful troubleshooting technique.  Time and time again, I have noticed that, the moment the user tells you, “bla bla bla, but that shouldn’t matter!” you are finally getting very close to the crux of the matter.  Invariably, a closer look at whatever it is that shouldn’t matter will lead you straight away to what is wrong.  It is like that other simple trick — turning it off, counting to ten, and turning it back on — which also fixes so many problems.  The trick of asking what might have changed that “shouldn’t matter” is among the most powerful tools in the kit used by skillful troubleshooters.  As easy as this is to understand and remember, it seems almost criminal that we not only do not consider it part of the “core content curriculum” for all students, we do not even bother to share such powerful little tricks and habits of mind with our educators.  Anyone who has ever helped another person “fix their computer” has heard this phrase countless times.  Why are we keeping this phrase and other tricks of the troubleshooting trade a secret?  And where, in our broken education system, are the pundits saying, “bla, bla, bla, but that shouldn’t matter?”  If we dig deeper around those areas, we are apt to get a clue how to fix this thing.  Surely, making Troubleshooting / Debugging a required subject for both students and teachers is going to be part of the solution.  But wait, didn’t Seymour Papert tell us this, years ago?

Does school technology even matter?

In a little town, not so long ago, but certainly far far away, concerned citizens openly wondered whether technology mattered much in their schools.  In this town, after a good deal of money had been spent to purchase computers about five years ago, kids were being sent down the hallway for 45 minutes, once per week, to the computer lab, thereby allowing the regular classroom teacher to have a prep period.  In the computer lab, there was a “computer teacher,” who introduced keyboarding skills and demonstrated the proper procedures for setting the margins in a popular word processing application, using a version of that software that was no longer being sold or supported by the manufacturer.  The activities used for word processing practice had little to do with any topics being discussed in the regular classroom (such as, say, how to properly address a business letter to be printed on paper and delivered by the U.S. Postal Service).  A few months later, informal analysis of recent standardized test scores revealed that those classrooms visiting the computer lab once per week for three months did, in fact, score slightly higher than those that did not; but the difference was not statistically significant.  Since funding for the town’s schools had dwindled, year after year, and other programs such as sports, music, and art were facing budget cuts, this disappointing data was cited by several outspoken taxpayers during a heated School Board discussion.  The meeting concluded by cutting the budget line items, not only for the upgraded word processing software, but also the salary for the computer teacher.  The technology program wasn’t improving test scores much, anyway.  So, reducing funding for it really shouldn’t matter.  Right?

Like those taxpayers, my parents never were very excited about computers, perhaps partly because of early assumptions about what benefits might or might not derive from using them.  So many things — such as the emergence of commercial air travel — changed in their lifetimes.  (To this day, more than a few people from my parents’ generation actually take pride in their inability to use those silly, time-wasting contraptions; some are still employed as teachers.)  Their son, however, was born into that first generation of students fortunate enough to so much as lay hands on a computer before graduating from high school.  Even to do that much, I first had to be selected for a special summer program at a nearby University.  Touching that first computer changed my life.  From that moment forward, the potential of technology to change how I learned anything — how people could learn anything — became my life obsession.  Several years later, when I heard Seymour Papert at MIT talk about how every school — indeed every child — should have a computer, as their own personal power tool for learning — while more sensible folks were laughing at the economic infeasibility of this notion — I did not laugh; I was captivated.    When Alan Kay envisioned the Dynabook, in Scientific American — the conceptual forerunner to Apple’s new iPad — folks more worldly-wise than I will ever be explained that such a thing could never be made sufficiently cost-effective for public education; but I remained transfixed.

The children who come into our schools today — the first generation for whom computers have always been part of their lives — part of their furniture and a permanent attachment to their ears — often carry more computer power in their back pockets than the million dollar computers I was allowed to use in grad school.  Possibly they are learning more on the school bus, using these devices, than they learn in class during an entire school day.  Even parents are surprised and sometimes shocked to find that all such devices must be powered down at the school entrance, on penalty of confiscation.  For our children, going to school is a lot like boarding a commercial airliner: after clearing security, we are strapped in our seats, the pilot decides where we are going, how we will get there, and when we will arrive; and — even though most of us may have difficulty believing it —  we are told that the tiny signals from our small devices might somehow interfere with much more important communications and cause the airplane of public education to fall right out of the sky.

For a very long time, educational technologists, myself included, have been utterly convinced that there is something almost magical about these tools for helping people learn.  Technology in schools is no longer a “yes or no?” question, but more of a “how much?” and “of what kind?” question.  Yet the reality of what happens in schools almost universally falls far short of the tremendous potential we envision.  Why is that?  Why aren’t these incredible devices being used in more empowering ways?  Why does school technology so often look like that little town, far, far away, where “computer” is a “subject” you spend 45 minutes a week learning about?  What are the barriers preventing a more enlightened approach from taking hold and changing everything?  Obviously cost has historically been a major obstacle; but Moore’s Law has dramatically altered the math here; and if money were the only problem, then why is it that even economically disadvantaged children, nowadays, seem to have better technology at home than at school?  What are the barriers preventing powerful networks of knowledge devices from utterly transforming our schools? The answers to these questions matter a great deal for our children and for their future.  Through anecdotes and years of hard knocks, I hope I might make a small contribution here toward understanding how we can make technology matter more in our schools.

Originally, this was intended to be a book.

For years, I have been threatening to write a book with this title.  Alas, I never seem to find the quiet, quality time required for such an endeavor.  The tyranny of the urgent can be a powerful tool for procrastination.  Perhaps more importantly, it may be that a traditional book is no longer the best medium for these sorts of ideas, any more than a student of the Renaissance would have written about Gutenberg’s amazing invention on a cuneiform tablet.  Nowadays, a blog does seem more appropriate to my topic, and not just to my human shortcomings as an author.  Sometimes sooner is better than better.  The incorporation of comments from readers is also apt to add value well beyond my own musings.  I suppose that the resulting document could always be printed on paper, someday, if there are still any trees left on this planet by the time I finish.  In any event, for those who care about whether — and in what ways — technology might be used to improve learning in our schools, then my fond hope is that you may find these postings to be helpful and relevant, albeit perhaps less well-organized or polished than had they been held in abeyance for eventual release in printed form.