Computer Science Learning Bottlenecks

During a workshop I was asked to identify a learning bottleneck.  Here’s my first:

Students have difficulty visualizing that a pointer variable and the variable it points to are (usually) different variables, realizing the values in one can be changed independently of the other, and choosing the correct variable to access or assign when figuring how to program the variety of tasks that they need to learn about.

Added a second added later:

When asked to report or draw conclusions from structures, students confuse static structure (such as structure of code, and relationships between classes or other types) and dynamic structure (such as quantities and references between instantiated objects, contents of an activation stack, and history of activations).

And later:

Juha Sorva’s proposed threshold concepts for programming, beginning with the dynamics of program execution, seem on the mark.  Distinguishing between a variable storing a pointer and a variable storing a  non-pointer makes sense only after the concept of variable and its role in dynamics is understood.

Advertisements

Debugging: Meta-cognition and explicit teaching of strategies

The 2007 SIGCSE paper “Debugging: The Good, the Bad, and the Quirky – a Qualitative Analysis of Novices’ Strategies” by Murphy, Lewandowski, McCauley, Simen, Thomas and Zander lists debugging strategies and reports a study of 21 students to observe and categorize their debugging behavior.  To quote from the authors’ conclusion:

Many unproductive activities appeared to stem from insufficient meta-cognition. Some students did not recognize when they were stuck, thus they did not know to try a different approach. Those who stubbornly traced in their heads often did not realize they were suffering from cognitive overload and might be more productive if they tracked on paper or via print statements. Others were blind to alternative bug sources. Debugging instruction should incorporate these meta-cognitive factors, perhaps taking the form of self-questions: “What else could I try?’, “Is this too much to keep track of in my head?”, and “What are other possible sources of the bug?”

Explicit teaching of processes for students to reflect on their own learning and disciplinary experiences is suggested, as it is from a lot of other research literature. We add here that we should experiment with exposing students to research literature itself such as the above cited paper.

  • Students will relate to experiences of people just like themselves in current and immediately anticipated situations.  The students in the study were taking a CS2 course.
  • Particular components of debugging, as itself a component of programming, are identified:

    “Debugging is difficult for novice programmers. Similar to new drivers who must learn to steer, accelerate, brake, etc. all at once, novice debuggers must apply many new skills simultaneously. They must understand the operation of the intended program and the execution of the actual (buggy) program; have general programming expertise and an understanding of the programming  language; comprehend the application domain; and have knowledge of bugs and debugging methods [2]. Unfortunately, most novices’ knowledge of these skills is fragile at best [12], causing many to find debugging difficult and frustrating.”

  • Particular practices and tools are introduced: “Gain domain knowledge, Isolating the problem. Tracing, including: Mental, Print & Debugger, Pattern matching, Consider alternatives, Testing, Environmental, Understanding code, Work around problem, Using resources, Just in Case, Using Tools, Tinkering”
  • More importantly, results and examples of critical evaluation are demonstrated.
  • Such material will foster respect and interest in attitudes and interests of academic research.

Learning GIT

Spent a few days learning GIT.

Some links I found:

  1. The GIT user manual from kernel.org  I cloned the git repository for git itself.  Like the description: “Information Manager from Hell”!
  2. The GIT Parable Starts with a good promise, lets see..  It is really clear and easy to read, although I already understood much of it from other sources.  It ends with3 links to good-looking websites with more detail.
  3. Computer scientist’s intro
  4. (update from csi500 web site’s list).

GIT is based on objects located by references. The references are visible as the hashes.

Key concepts: Working tree. Index. Commits. HEAD pointer.

Concepts: Branch (for clarity, named variable pointer to a commit rather than a line of deveopment).
Tag.

Computer Science Education Journals

 

Transactions on Computing Education http://toce.acm.org/index.html, formerly Journal on Educational Resources in Computing http://jeric.acm.org/

Computer Science Education http://www.tandfonline.com/loi/ncse20

Abstractions feel concrete

Things that people say are abstract such as mathematics feel to me, having training and inclinations towards that subject, very concrete, particular and real.  They are not things for which the rules and nature are not clearly understandable, like justice and freedom. Perhaps the ability to deal with abstract things is the ability to think about them in the same way as concrete things.  Of course, this way of dealing with things may be much more successful in areas such as programming and mathematics than it is in areas like ethics and esthetics.

The following paper probes the trouble with seeking correlations between programming ability and “abstraction” ability. It classifies various meanings of abstraction and reports research into correlating measurements of kinds of abstraction ability with performance in computer science courses: “Abstraction Ability as an Indicator of Success for Learning Computing Science?” by  Bennedsen and Caspersen in ICER’08, September 6–7, 2008, Sydney, Australia( http://www.daimi.au.dk/~mec/publications/conference/32–icer2008.pdf )

 

Shortcomings of Alice, Scratch, Myro, etc.

Having taught a few CS0 courses, I got the impression that programming learning environments including those above enable beginners to do easy attractive and satisfying things, but do not automatically (at least) lead them to do the harder things.  I was particularly disappointed that the old version of Alice did not provide easy to use arrays that could be employed in teaching basic search and sorting algorithms.

Misconceptions and Teaching Them

Collecting and Categorizing Misconceptions

What to pay attention to.

I read the paper Habits of Programming in Scratch by Orni Meerbaum-Salant mentioned in Mark Guzdial’s excellent Computing Education Blog.  It is not at all surprising after trying Scratch.

Guzdial has an interesting earlier blog entry My Students Know Far Less Than I Ever Expected about being surprised at students beginning data structures knowing far less than he expected.  The misconception is basically about the practice of detailed analysis of questions and problems that CS courses are expected to teach.  Respondents remark that the subject was a “trick question” that even some of them got wrong when they first saw it.  It also suggests misconceptions about the meanings of code.

Here is a copy of a reply I added:

Having taught data structures many times, I am not at all surprised by the outcome of the problem, even after the class discussion.

The main misconception that the students have is about what they must pay attention to when they face examples of programs and questions about them in order effectively learn elements of the discipline.  They must often attend to the same things of course when they write or debug their own or other peoples’ code. Even the professionals, who rightly observed that the problem is a “trick question”, fell for the trick, I guess, because they did not realize that they needed to turn on the mode of detailed code and question analysis rather than recognize a familiar pattern.

How can we teach “what it is you must pay attention to”, teach the habit to pay attention to that when it is appropriate, and teach how to recognize the situations where particular things to pay attention to are appropriate?

Should we tell the students that certain in-class questions are the kinds of questions that they must ask themselves while they are engaging in disciplinary work, necessary for that work to be effective?

Research on Misconceptions.

Yesterday, I found some research particularly germane to classifying and addressing misconceptions in elementary programming:

UUhistle: misconceptions about variables surveyed and addressed.

UUhistle, A Program Visualization Tool for Introductory Programming Education distributes a Java applet that animates at a fine grain level a simple model for Python execution that includes the creation and assignments to variables.  It is backed up by research: The project is part of Juha Sorva‘s doctoral dissertation.

From the same academic program, the master’s dissertation of Juha Helminen addresses both the issues of problem solving skills and of misconceptions.  Here’s a page of many links to its publications.

Sorva’s paper “The Same But Different Students’ Understandings of Primitive and Object Variables” in SIGCSE’08, March 12–15, 2008, Portland, Oregon, USA describes different ways of understanding primitive and object variables associated with the corresponding misunderstandings.  From the abstract: “The results show that learners create various kinds of mental models of programming concepts, and that the concept of variable, which is fundamental to most types of programming, is understood in various non-viable ways.”  The author cites two ideas from education theories: First, “constructivist” theory holds that student form and modify mental models, and  “Phenomenography” described by Sorva as an “approach to research that investigates phenomena and people’s relationships to those phenomena”.  Further:

In a research project of this kind, interviews are used as a data collection method. Data collection is followed by or intertwines with qualitative data analysis. During analysis, the researcher, in dialogue with the data, delimits the phenomenon of interest. Different ways of understanding or experiencing the phenomenon are enumerated as an outcome space consisting of a (smallish) number of categories of description. Outcome spaces often take the form of a hierarchy or tree of categories related to each other. The intention in phenomenographic research is not to point out which specific kinds of understanding each individual has, but to identify different ways in which a phenomenon can be understood, or experienced, on a collective level.

And, in contrast to other research (which excludes incorrect understandings from the “outcome space” used by the experimenter to classify response.) he cites, “both correct understandings (as deemed by the researcher) and partially incorrect understandings are included in the outcome space. Partially incorrect understandings are understandings that extend correct understandings in incorrect ways. ”

Cognitive conflict, variable and reference assignment, approach combining constructive and visualization supported teaching.

“Using Cognitive Conflict and Visualisation to Improve Mental Models Held by Novice Programmers” by Linxiao Ma, John Ferguson, Marc Roper, Isla Ross and Murray Wood in SIGCSE’08, March 12–15, 2008, Portland, Oregon, USA.

More recent publication: “Investigating and Improving the Models of Programming Concepts Held by Novice Programmers.” L. Ma, J. Ferguson, M. Roper, and M. Wood. Computer Science Education. 21 (1). March 2011. pp. 57-80 http://www.tandfonline.com/doi/pdf/10.1080/08993408.2011.554722

The Two Humps and a test alleged to predict programming ability from consistency of ones assumptions.

Dehnadi, S., and Bornat, R. 2006. The camel has two humps.
Middlesex University Working Paper.
http://www.cs.mdx.ac.uk/research/PhDArea/saeed

This research was said to attract some attention a few years ago.  It was a major citation in the Ma et. al.  paper.  However, the authors unsuccessfully attempted to reproduce the main results and reported this in http://www.crpit.com/confpapers/CRPITV78Bornat.pdf Another such failure was reported in http://www.daimi.au.dk/~mec/publications/conference/23–iticse2007.pdf and http://www.cs.toronto.edu/~sme/papers/2008/Lung-ICSE2008.pdf

Research about the tight connectedness among the critical concepts in programming

I am extremely impressed by the recent paper by Anthony Robbins: “Learning edge momentum: A new account of outcomes in CS1” He proposes that the bimodal grade distributions in beginning programming courses is the result (in my words) of the central limit theorem failing to apply to such grades because those grades are sums on non-independent random variables. He calls this the “momentum effect.” The paper surveys and criticizes the many proposed reasons for the bimodal phenomenon. Robbins’ formulates and simulates a probabilistic model for concepts learned sequentially and where the effective learning of each depends on effective learning of the previous.  I conjecture that the effect of dependence among the scores does not require the this sequentiality and more flexible models of dependencies may be theoretically interesting.

Some critical responses:  Lister’s column in Sept. 2010 Inroads http://portal.acm.org/citation.cfm?id=1835428.1835434 I do what Lister criticizes by giving partial exam credit on programming problems for each statement or operation the student writes that should appear in a correct solution.  I do this because if I did not do it, the scores would be extremely bimodal. I wonder if bi-modality is not apparent simply because the exams do not test the full measure of what giving a correct solution to a programming problem means! Ultimately, I agree with Lister in that better validated assessments in programming courses are needed if hypotheses like those of Robbins are to be scientifically tested.

Here is another:  http://66.31.107.147/~heines/academic/papers/2011sigcse/CD/docs/p631.pdf  is a paper by Petersen, Craig and Zingaro in the 2011 SIGCSE. The point seems to be summarized by their saying: “In Robins’ model, a misunderstanding of conditionals may result in a decreased probability of understanding loops. Instead, we argue that her misunderstanding of conditionals decreases the probability that she can demonstrate understanding of loops.”  In my view, understanding of loops necessarily includes the ability to demonstrate that whatever is coded in the loop body or in the continuation condition is repeatedly executed.

My blog page on archival journal in computing education.

Mind/Brain more like muscles than a computer

People sometimes write about the brain being like a muscle that gets stronger with exercise.  Here’s a more specific thought:

Both the mind and a computer acquire new capabilities after receiving certain inputs: Teaching in the case of the mind and software (i.e., programs) in the case of the computer.  (This is not to say that teaching is the only or even the most effective way.) However, there is a difference which is critical to teachers and students.  The computer can use its new capabilities immediately after new software is installed, and, for the most part, the quality of the new capabilities, being determined by the software contents, remains fixed at its original level. (An upgrade really is adding new software or replacing portions of old by new.)  On the other hand, the usability of new mental capabilities grows with practice, usually only develops incrementally, and decays when unused.  The brain is more like a system of muscles that together grow in strength in the course of repeating and perfecting new physical tasks than a computer’s hard drive that perfectly stores the entire capability ready for the computer to perform when the program is launched.

Hence the lesson for teachers and students is that mere presentation and memorization of content is utterly insufficient for effective learning.

Hackers and Painters

See

Hackers and Painters

Essay on how “hacking” i.e. building excellent software differs from science in that it is a kind of “making” like painting.

Way Out Web Postings

Here I accumulate some perhaps shocking, unbelievable, etc. stuff that might have some truth to it:

  1. from http://stackoverflow.com/questions/304399/do-caffeinated-drinks-improve-your-programming-productivity-or-cause-more-proble

    “I occasionally need to read mathematics journal articles for programming work that I do, and I find there are often things I simply cannot understand without enough caffeine in my system. The level of abstraction required to understand a mathematical proof is sometimes just too high for me to cope with otherwise.

    Sidebar: I think everyone has a natural “resting” level of abstraction that they are comfortable with; a programmer’s resting abstraction level is on average above that of a non-programmer’s, but professional mathematicians are another level above that. (The worst part is that some of them are additionally normal, friendly people who can socialise and play sports… Not that I’m jealous, of course.)”