Category Archives: Systems

How to read specifications.

How to read the HTML5 and other Specifications…

Early on, Dean Arden declared that one important goal of courses like computer architecture is to enable people to understand specifications.  I’ve tried to weave using specifications, either written by professors like me into assignments, or, more rarely, for required references included in assignments.

The HTML5 spec quote is so cute, I must directly quote it:

1.7.1 How to read this specification

This specification should be read like all other specifications. First, it should be read cover-to-cover, multiple times. Then, it should be read backwards at least once. Then it should be read by picking random sections from the contents list and following all the cross-references.


Free/open source journal

I will try to write about experience with technicalities I encounter in free/open source projects.  This might help academics and other beginners to educate so they can join free or open source communities.

For now, I started working the training materials for Linux on the Beagleboard posted by free-electrons. Read and made notes on the .pdf files with Adobe’s pdf viewer on an Android (Samsung Note 10.1) tablet.  (DISCOVERED LATER: These materials are obsolete.  They predated the Beagleboard xM rev. c1 which I have, and web links in the lab materials are broken.  Free Electrons posted in 2013 that all their Linux-arm training is on the … which is like the Beagleboard.)

The instructions worked ok until the build of the ct-ng toolkit 1.8 failed because of broken links.  I made a note to use current versions, and proceeded with 1.18.

Some step failed because of missing software.  That was easily remedied by installing the (Ubuntu) package.  However, the ct-ng build wasn’t restartable.  Investigation (..fill in ) showed a configuration setting (I knew that systems typically do or should have checkpoint/restart features)

Navigating around the tools built from ct-ng

Installed under x-tools.  Observed one dir. for the built tools  arm-unknown-linux-uclibcgnueabi written about in lab instructions.  Observed a subdirectory arm-unknown-linux-uclibcgnueabi/arm-unknown-linux-uclibcgnueabi having entries: bin  debug-root  include  lib  lib32  lib64  sysroot

CURVE:  hello C program built with tool built from ct-ng failed on Beagleboard’s out-of-the box system.

-sh: ./hello not found

Knew this is likely a missing shared library (from past experience, seeing this non-informative error message) (The lab book contents in a later chapter from testing your cross toolkit discusses the shared library issue, and quotes the “not found” message.)

Tried ldd hello: Failed since the out-of-the-box system didn’t have ldd. (past experience: ldd, stands for list dynamic dependencies. Linux dynamic link libraries have names like lib*.so and many are installed under /usr/lib)

Looked for ldd in x-tools built by ct-ng:  Found it!  Tried arm-linux-ldd ./hello and it failed with a message suggesting the –root option.  Used –help option and read about –root option.  Knew (from this project’s reading) about sysroot.  Observed sysroot has lib subdirectory containing familiar lib*.so and lib*.a files.  So I tried arm-linux-ldd –root that sysroot dir hello and found 2 dependencies:  I then looked for them on the Beagleboard system.  Saw but NO and ld-??? !

With tools, compiled hello.c with –static (past experience, training content about dynamic libraries).  Built.  SUCCESS running the static version on the Beagleboard!

C compiler command line arguments: –static  -o used to name the result hello.static (good idea to use multiple names)

What should I do next?

A root file system (observed out-of-the-box system did not use initramfs or initrd) has a kernel, so to build a root file system, I need a kernel.  So we should work on building a kernel first.


Training explains using an nfs root file system.  The PC requires shorter (don’t know what to call that size) cards, which I didn’t see in the depts. old card box.  Emailed for sysadm for spare nic.

Try instructions from Embedded Linux Primer for kgdb:

How can I get new kernel parameters (kernel command line) in the out-of-the-box Beagleboard?

Ran u-boot (booted Beagle and pressed a key).  Explored environment with commands, beginning with printenv (had read about uboot before).

Printing: Copy from terminal screen, paste into screen running lpr .  (Had to configure our network printer into the PC, basic Ubuntu adm.)

Finding the .config file for a working kernel

Past observations/readings: it’s installed in the /boot subdirectory alongside the kernels and initrds.  NOT found under /boot in the Beagleboard out-of-the-box system!

Noticed config.gz in the /proc directory of the Beagleboard.  (Unix cp command; ok to copy /proc files; know /proc files won’t exist on the disk image!)  cp to my user home dir, so I can get the copy outside the Beagleboard.

Kernel building subproject

I’m familiar with Linux kernel configuration.  Recent reading reminded me that one can start with a given configuration, copy it to .config and then run make oldconfig.

Note kernel version on the out-of-the-box system:  Use uname -a (previous experience)  Also, noted in notebook from boot messages TI DM3730 (model of OMAP–had read about lots of new OMAPs from TI’s recent OMAP tech reference manual; I read the initialization chapter after reading web pages about xloader uboot OMAP booting)

BeagleStudy/kernels must download 2.6.32 kernel src.  Hey, let’s use GIT!

wget -nd

Went to  Saw was a “long term” so I decided on that. Saw URL when hovering.  Decided to use wget.  (Learned of wget long ago. Remembered -nd option.)

tar xzf linux- (long ago; basic kernel building instructions appearing many places.)  FAILED:  Encountered .xz yesterday!!  Did google xz and man -k xz to find out about .xz compression and tool xzcat.

xzcat lin… | tar xvf –  (long ago–l like seeing it happen so I use v flag.  x is extract, f is file, – use stdin (piped from xzcat command)  ls to confirm I have a linux- dir.

cd linux-, git (have to find out again how to init a git repo!) (git not installed!, sudo apt-get install git) (Wow, for Linux kernel)

git init; git add .; (took a long time); git commit -m “pristine…” (knew message was required)

Used USB micro/etc flash card interface to cp config.gz (above)

Know: Automounter make flash cards available under /media  Tab completion to remind names under which partitions are mounted..  is Narcissus… (observed before).. know users are under /home, so use /home/seth

gunzip config.gz ; ls to confirm config.gz replaced by config; rename mv config BeagleOutOfBoxConfig; cp Beag.. .config

Tried git add Beagle-Out-Of-Box-Config .config;   add .config refused because of .gitignore (So stuff built won’t be saved by git…  Decided not to use -f to really want to add them!)

Look at 2009 cross compile material in free-electrons lab.  (BEFORE dispatching make oldconfig)

Forgot install uboot-mkimage!  Try that.  Installed uboot-tools package too. (Free Electrons lab explains getting and using Ubuntu, installing linux packages, and use of apt-get install on Ubuntu Linux. )

lab says: TI OMAP Beagleboard (reminded it’s OMAP based and from TI)

lab says: put bin dir of our xtools in front of PATH (know about unix PATH, etc)

Observed: x-tools/arm-unknown-linux-uclibcgnueabi/bin has arm-linux named tools, but

x-tools/arm-unknown-linux-uclibcgnueabi/arm-unknown-linux-uclibcgnueabi/bin has the tools without the arm–… qualified names.  Did ls -l to see they are not symbolic links.  Are the binaries equal?? YES, at least the gcc one! I’ll follow the lab’s instrs.

I wrote question in lab: Makefile setup Modify the top level makefile… REALLY??? Lets look in the top level makefile.  Also do a make help (know about that).

Reading the makefile and google search found   Have to set ARCH set to an arch in the arch subdir.  CROSS_COMPILE is the prefix appended to commands like gcc for finding them under a search using PATH.  gentoo also explains an INSTALL_MOD_PATH Apparently gentoo explains a way to get the kernel build tree in the embedded system.  gentoo explains two ways of setting environment (What’s that?) for kernel building: Put them in the make command within a shell script, or modify Makefile.

I see Makefile uses uname to set SUBARCH and uses that as default for ARCH.  This is how it knows the arch for a native build.

I think I’ll use a script setting the env.  What’s “$@” after the make for?  command line args..  follow gentoo: make it executable.  Named it mymake (I like my).  Checked: PATH does not contain .; so I’ll do that in the shell.  Checked: mymake help printed help!

Maybe … I’ll make a sysroot dir alongside the linux tree!!

Look for INSTALL_MOD_PATH in Makefile.

# INSTALL_MOD_PATH specifies a prefix to MODLIB for module directory
# relocations required by build roots.  This is not defined in the
# makefile but the argument can be passed to make if needed.

SCROLLING:  Ctrl-midde and Ctrl-right in xterm give option setting window.  Handy to turn on scrollbars.


did mymake oldconfig.  Noticed an extra new line was entered each time I pressed enter!!

So, I viewed xterm’s settings with Ctrl-right and turned OFF auto linefeed. (Probably set it accidentally)

Forgot to write a custom subversion in the makefile. Do that now… (edit the makefile!! .61-sdc1)


mymake oldconfig… something wrong.. looks like it thinks I’m doing an x86, not ARM!! Instead of putting the enviroment settings on the make command line, I put them on separate lines in the script!!!  So they were not exported.  Will fix it:  use shell command export on each one..

…. built 2 kernels and got uBoot to load them; got uncompress messages and starting kernel messges.. then nothing!!

The next day.

Viewed some Google search results.  Worked from Looked at the referenced git repository via the web interface: found a commit to support Beagleboard xM rev. C.  Compared file to file in my last kernel version build…new one has much more.

Tried to use the published command (suspicious of the ;):

git clone git://;branch=linux-3.2.y;protocol=git

It looks like the author took the entry from a bitbake recipe and reprinted it in a git command!  But I tried it anyway.  As I expected, it got the whole linux-stable repository.  A git checkout linux-3.2.y worked of course.  I listed all branches for fun git branch -a.

I see the author says to checkout a particular commit (identified by hash)  must try that…

FAILS!  Check out the version given by above recipe (SRCREV). E.g. git checkout 58cf2f1425abfd3a449f9fe985e48be2d2555022 failed with fatal: reference is not a tree: 58cf2f1425abfd3a449f9fe985e48be2d2555022 Also git show on this reports bad object


Networking with u-Boot.

The current FE lab instr. didn’t work.

Google search: David Neiss’ Blog (Jan 9,2012) said his “stock 05.02 precompiled u-boot image didn’t seem to support tftp.” He got a source tree from u-boot.not.mainline, built it and reported success.  His steps:

  1. setenv serverip  (sdc’s IP here)
  2. setenv ipaddr (sdc’s Beagleboards IP here)
  3. usb start
  4. see output
  5. try ping reports he’s an active TI arm hacker; but no u-boot.not-mainline branch.  I found a 2yr old u-boot repository commit that edited doc/README.usb adding information about usb networking.  Quote of current version of Networking section:

USB Host Networking
If you have a supported USB Ethernet adapter you can use it in U-Boot
to obtain an IP address and load a kernel from a network server.
Note: USB Host Networking is not the same as making your board act as a USB
client. In that case your board is pretending to be an Ethernet adapter
and will appear as a network interface to an attached computer. In that
case the connection is via a USB cable with the computer acting as the host.
With USB Host Networking, your board is the USB host. It controls the
Ethernet adapter to which it is directly connected and the connection to
the outside world is your adapter’s Ethernet cable. Your board becomes an
independent network device, able to connect and perform network operations
independently of your computer.
Device support
Currently supported devices are listed in the drivers according to
their vendor and product IDs. You can check your device by connecting it
to a Linux machine and typing ‘lsusb’. The drivers are in
For example this lsusb output line shows a device with Vendor ID 0x0x95
and product ID 0x7720:
Bus 002 Device 010: ID 0b95:7720 ASIX Electronics Corp. AX88772
If you look at drivers/usb/eth/asix.c you will see this line within the
supported device list, so we know this adapter is supported.
        { 0x0b95, 0x7720 }, /* Trendnet TU2-ET100 V3.0R */
If your adapter is not listed there is a still a chance that it will
work. Try looking up the manufacturer of the chip inside your adapter.
or take the adapter apart and look for chip markings. Then add a line
for your vendor/product ID into the table of the appropriate driver,
build U-Boot and see if it works. If not then there might be differences
between the chip in your adapter and the driver. You could try to get a
datasheet for your device and add support for it to U-Boot. This is not
particularly difficult – you only need to provide support for four basic
functions: init, halt, send and recv.
Enabling USB Host Networking
The normal U-Boot commands are used with USB networking, but you must
start USB first. For example:
usb start
setenv bootfile /tftpboot/uImage
To enable USB Host Ethernet in U-Boot, your platform must of course
support USB with CONFIG_CMD_USB enabled and working. You will need to
add some config settings to your board header file:
#define CONFIG_USB_HOST_ETHER /* Enable USB Ethernet adapters */
#define CONFIG_USB_ETHER_ASIX /* Asix, or whatever driver(s) you want */
As with built-in networking, you will also want to enable some network
commands, for example:
and some bootp options, which tell your board to obtain its subnet,
gateway IP, host name and boot path from the bootp/dhcp server. These
settings should start you off:
You can also set the default IP address of your board and the server
as well as the default file to load when a ‘bootp’ command is issued.
All of these can be obtained from the bootp server if not set.
#define CONFIG_IPADDR (replace with your value)
#define CONFIG_SERVERIP (replace with your value)
#define CONFIG_BOOTFILE uImage
The ‘usb start’ command should identify the adapter something like this:
CrOS> usb start
(Re)start USB…
scanning bus for devices… 3 USB Device(s) found
       scanning bus for storage devices… 0 Storage Device(s) found
       scanning bus for ethernet devices… 1 Ethernet Device(s) found
CrOS> print ethact
You can see that it found an ethernet device and we can print out the
device name (asx0 in this case).
Then ‘bootp’ or ‘dhcp’ should use it to obtain an IP address from DHCP,
perhaps something like this:
CrOS> bootp
Waiting for Ethernet connection… done.
BOOTP broadcast 1
BOOTP broadcast 2
DHCP client bound to address
Using asx0 device
TFTP from server; our IP address is
Filename ‘/tftpboot/uImage-sjg-seaboard-261347’.
Load address: 0x40c000
Loading: #################################################################
Bytes transferred = 3557464 (364858 hex)
Another way of doing this is to issue a tftp command, which will cause the
bootp to happen automatically.
MAC Addresses
Most Ethernet dongles have a built-in MAC address which is unique in the
world. This is important so that devices on the network can be
distinguised from each other. MAC address conflicts are evil and
generally result in strange and eratic behaviour.
Some boards have USB Ethernet chips on-board, and these sometimes do not
have an assigned MAC address. In this case it is up to you to assign
one which is unique. You should obtain a valid MAC address from a range
assigned to you before you ship the product.
Built-in Ethernet adapters support setting the MAC address by means of
an ethaddr environment variable for each interface (ethaddr, eth1addr,
eth2addr). There is similar support on the USB network side, using the
names usbethaddr, usbeth1addr, etc. They are kept separate since we
don’t want a USB device taking the MAC address of a built-in device or
vice versa.
So if your USB Ethernet chip doesn’t have a MAC address available then
you must set usbethaddr to a suitable MAC address. At the time of
writing this functionality is only supported by the SMSC driver.
I compared the include/configs/omap3_beagle.h with the config.h file in my u-boot build tree [How did I know to look there? I had read u-boot documentation about how to configure and build it.]  I observed that usb networking seemed to be configured fine.  I checked include/config_cmd_all.h [How did I know? omap3_beagle.h has an include targeting default commands, so I looked where that would be and noticed other *cmd*.h files in the include directory].  There was no CONFIG_CMD for the tftp command!  But in the latest config_cmd_all.h I saw:
#define CONFIG_CMD_NET /* bootp, tftpboot, rarpboot */
#define CONFIG_CMD_NFS /* NFS support */
#define CONFIG_CMD_PING /* ping support */

That means the NET option implies all 3 commands, including tftp (which is a prefix of tftboot)  The FE lab indicates that tftpboot merely transfers a file, whilst (other sources say) bootp does everything, including getting the board’s IP addr from a bootp server.


Here is the sequence that worked (the first time)

# usb start
(Re)start USB…
USB0:   USB EHCI 1.00
scanning bus 0 for devices… 3 USB Device(s) found
scanning usb for storage devices… 0 Storage Device(s) found
scanning usb for ethernet devices… Warning: failed to set MAC address
1 Ethernet Device(s) found

# print ethact

# setenv usbethaddr 00:11:22:33:44:55
OMAP3 # usb stop
stopping USB..
OMAP3 # usb start
(Re)start USB…
USB0:   USB EHCI 1.00
scanning bus 0 for devices… 1 USB Device(s) found
scanning usb for storage devices… 0 Storage Device(s) found
scanning usb for ethernet devices… 0 Ethernet Device(s) found
OMAP3 # ping
ERROR: Need valid ‘usbnet_devaddr’ to be set
at ether.c:2369/usb_eth_init()
ping failed; host is not alive
OMAP3 # usb tree
USB device tree:
1  Hub (480 Mb/s, 0mA)
u-boot EHCI Host Controller
  [LOOKS LIKE ethernet device is off now!]

OMAP3 # usb reset
(Re)start USB…
USB0:   USB EHCI 1.00
scanning bus 0 for devices… 3 USB Device(s) found
scanning usb for storage devices… 0 Storage Device(s) found
scanning usb for ethernet devices… 1 Ethernet Device(s) found
OMAP3 # usb info
1: Hub,  USB Revision 2.0

2: Hub,  USB Revision 2.0
3: Vendor specific,  USB Revision 2.0
– Class: Vendor specific [THAT’S MY ethernet device!]
– PacketSize: 64  Configurations: 1
– Vendor: 0x0424  Product 0xec00 Version 1.0
Configuration: 1
– Interfaces: 1 Self Powered Remote Wakeup 2mA
Interface: 0
– Alternate Setting 0, Endpoints: 3
– Class Vendor specific
– Endpoint 1 In Bulk MaxPacket 512
– Endpoint 2 Out Bulk MaxPacket 512
– Endpoint 3 In Interrupt MaxPacket 16 Interval 4ms

OMAP3 # ping
ERROR: Need valid ‘usbnet_devaddr’ to be set
at ether.c:2369/usb_eth_init()
Waiting for Ethernet connection… done.
*** ERROR: `ipaddr’ not set
ping failed; host is not alive

[Noticed: 2 Error messages! Note usbnet_devaddr is spelled differently from usbethaddr which is cited README.usb]

OMAP3 # printenv usbethaddr [I knew this from the README.usb and Neiss’ blog]
OMAP3 # setenv ipaddr
OMAP3 # ping

Waiting for Ethernet connection… done.
Using sms0 device
host is alive
OMAP3 # ping
Waiting for Ethernet connection… done.
Using sms0 device
host is alive


Also observed: I had the server ping repeatedly: Network not reachable messages.  Then, when I commanded u-boot to ping, then 3 (and then 2) ping responses appeared and the not reachable messages ceased.  Account for this: u-boot did not run an arp service until the ping command was given.  After the first ping, the server knew the mac addr.  I verified it (00:11:22:33:44:55) was in the server’s arp table with the arp command.


Lightweight Java Visualizer Tool

Lightweight Java Visualizer Tool

Cool tool for teaching data structures…

Computer Science Learning Bottlenecks

During a workshop I was asked to identify a learning bottleneck.  Here’s my first:

Students have difficulty visualizing that a pointer variable and the variable it points to are (usually) different variables, realizing the values in one can be changed independently of the other, and choosing the correct variable to access or assign when figuring how to program the variety of tasks that they need to learn about.

Added a second added later:

When asked to report or draw conclusions from structures, students confuse static structure (such as structure of code, and relationships between classes or other types) and dynamic structure (such as quantities and references between instantiated objects, contents of an activation stack, and history of activations).

And later:

Juha Sorva’s proposed threshold concepts for programming, beginning with the dynamics of program execution, seem on the mark.  Distinguishing between a variable storing a pointer and a variable storing a  non-pointer makes sense only after the concept of variable and its role in dynamics is understood.

Learning GIT

Spent a few days learning GIT.

Some links I found:

  1. The GIT user manual from  I cloned the git repository for git itself.  Like the description: “Information Manager from Hell”!
  2. The GIT Parable Starts with a good promise, lets see..  It is really clear and easy to read, although I already understood much of it from other sources.  It ends with3 links to good-looking websites with more detail.
  3. Computer scientist’s intro
  4. (update from csi500 web site’s list).

GIT is based on objects located by references. The references are visible as the hashes.

Key concepts: Working tree. Index. Commits. HEAD pointer.

Concepts: Branch (for clarity, named variable pointer to a commit rather than a line of deveopment).

Hackers and Painters


Hackers and Painters

Essay on how “hacking” i.e. building excellent software differs from science in that it is a kind of “making” like painting.

Into Systems

I’m into reading Linux kernel code and discussions (such as IBM Developer Works postings).

Assembly language code often does not document what it calculates with mathematical expressions. The similarity of higher level language code to mathematical expressions is of course designed to be much stronger. I find the need to consciously translate what is calculated to a mathematical expression before I understand it. This step is probably bypassed by more practiced programmers.

Systems have features required so different applications with conflicting properties can be supported by the same system. Example: GNU ld has the PROVIDE(SYMBOL = DEFINITION); operation. The documentation gives an example for its need:

For example, traditional linkers defined the symbol `etext’.
However, ANSI C requires that the user be able to use `etext’ as a function name without encountering an error. The `PROVIDE’ keyword may be used to define a symbol, such as `etext’, only if it is referenced but not defined.

Some interpretive languages have a conditional assignment operator that assigns a given value to a target variable when the the target variable is undefined. This is the same idea as “weakly defined” symbols supplied by input files in a linking.

Seeking generalizations: (a) Defaults in systems. (b) Perhaps closed world assumptions  (what is not specified is assumed to be false) in logic.


How many people who use advanced features of linkers have written an elementary linker for educational purposes?  Our curriculum currently required of BS majors includes that (in a systems programming course.) How can we effectively balance education by using complex systems to practice building systems and the building of simpler systems?