My teen years: The transputer operating system

Ray traced robot made in transputer using my C compiler

In my previous article I talked about how I managed to create a self-contained operating system for transputer. This included the basic operating system, a text editor, a Small-C compiler, and an assembler.

The year was 1995, I was age 16, Lemon Tree and Zombie were being played in the radio, ARPANET closed around 1990, and it started to be known as Internet, and in Mexico only a handful of people used it through Compuserve Mexico. I wasn't so lucky to have this service.

I was evolving my operating system, modifying it and improving it, recompiling and rebooting. I was pretty interested in compiling any C language source code I could get my hands on, and I got hold of several CD-ROM discs, but I couldn't compile most things because I had a Small-C compiler, not a full C compiler.

This prompted me to extend my C language compiler to support more C features. Because youth is bold, I kept doing comparisons with text pieces from the input just like Small C did. I was working in a platform with only 128 KB. of RAM, and I could manage that feat because the transputer instruction set generated very small executables. But of course, a proper lexer would have made faster compilations.

As I progressed in my implementation of the C language features, following the K&R book's appendix A, I discovered that developing most of these was pretty direct, like struct, and union. Typedef was the most complicated to understand. The pointer arithmetic syntax precedence was very difficult, in particular for getting an array of function pointers. I'm still proud of how I made byte-code descriptions of C types, the only difficult thing was getting right the recursive calls to handle precedence. Also getting initializers to work was hard to do as these had very loose syntax in real compilers.

The C compiler took me a lot more time than the Pascal compiler, and from my revision notes I used almost a full year to get an almost full K&R C compiler (K&R means Kernighan & Ritchie, or a compiler as described in their original book). My information was outdated, because I had the 1978 book, but the book was updated in 1988, and the ANSI C standard was approved in 1989.

I tuned the preprocessor with each new source code I found for testing. I was amazingly happy when my compiler finally managed to run a chess program by Vern Paxson from an obscure contest ran in USENET, the IOCCC (International Obfuscated C Code Contest). I found this program in the book Expert C Programming by Peter Van Der Linden (1994).

Once the C compiler was supporting floating point, I was able to port back the Ray Tracer I made in Pascal, and I developed a 3D poligonal model program following the course and exercises in the book 3D Computer Graphics by Alan Watt (1993).

The last additions to the host Z280 computer were a SCSI card, and all the peripherals were basically recycled from tech trash, for example, a SCSI hard disk drive (powerful 40 MB when the common ones were 500 MB), a SCSI DAT tape drive, and a SCSI CD-ROM drive Toshiba XM-4101B (at least this was new!).

Compact Disc The Cranberries: Everybody Else Is Doing It, So Why Can't We?

I added to my transputer operating system a way to read High-Sierra and ISO-9660 format structures to access CD-ROM data, and a program to unzip files from these discs. I was pretty proud of how I managed to compile the source code of a public domain version of Inflate for ZIP files. Another thing is that these CD-ROM drives could play audio CD automatically, you just put a CD (mine was Everybody Else Is Doing It, So Why Can't We?) and hear music while working.

The tape drive was DAT format, and I made some backups using my TAR program. I have forgotten completely where are these tape stored, now that's the most secure backup storage! The one no one can find.

I reached the high-point of my development for transputer around the summer of 1996, but the processor started to show its age. 128K RAM of memory was barely enough for working. As I bloated my programs with more features, the transputer started to look more and more slow. And unfortunately, another board was never made. I would have been very interested in handling multiprocessing.

Restoring the operating system

The work for restoring my operating system wasn't so easy as I thought. I had to expand the buildboot.c program to create floppy disk images in the latest file system format, and also hard disk images. Later as I put together the image file, I needed support for growing disk directories.

The main difference is the expansion of directory entries to 64 bytes. This was a by-product of discovering that I was free to specify my operating system as I liked, so the first thing for me was "the great escape" from the 8.3 filename format common in MS-DOS at the time (Windows 95 was barely released a few months ago). It wasn't done in a single step, my first expansion of the file system was fifteen letter names using the 32 bytes directory entries (Jan/01/1996), and the next expansion was thirty-one letter names using 64 byte entries (Feb/06/1996).

The 64-byte entries of my file system. Notice flags for the C subdirectory.

Once the disk image builder was ready, I put the SOM.32.bin file along Interfaz.p inside the floppy image, and tried to boot it up after adding the required services in the transputer emulator (including three drive units: floppy disk, RAM disk, and hard drive). It got stuck after reading the boot sector, and the cause was the byte order for the sector number was big-endian. After doing the change in the emulator, it was able to read the whole file inside the memory and it got stuck.

Apparently the timers didn't work as the screen wasn't updated, so I added timer handling for low-priority threads, and the screen become visible. Still, it got stuck. I made a brief modification to the interface code with the host (a semaphor), but it still didn't operate. The only reason for this is that the MAESTRO.CMG file used a different protocol to interface with the operating system. I searched in my files until I found the correct file and source code, I used this file to boot up the operating system and it worked on the first try, and it advanced enough to complain of a missing C:/Interfaz.p

I had forgotten how my operating system booted up!

Very worn floppy disk from 1993 with MAESTRO assembler code for transputer

Very worn 3.5" floppy disk from 1993 with MAESTRO assembler code for transputer.
Notice it is double density (720 kb).

The minimum required files

It turns out that the floppy disk was used to boot the operating system (the file SOM.32.bin), and in turn it would access a hard disk through drive C: to load the command-line processor.

Once this file was loaded, I needed a few more files to be able to compile programs inside the operating system: editor.p, cc.p, ens.p, and ejecutable.p.

I started by testing my text editor, and it was pretty pleased watching how it colorized the preprocessor directives and the C language elements. All the source code is written with Spanish comments, and it is just too much code to translate it to English.

The visual text editor displaying C source code with syntax coloring.

I never implemented a current directory feature, so you need to type the full path for each file. A feature of my operating system is that the A: drive is assumed, so you need to write c:cc to execute the C compiler from the hard disk drive. Also this is why all the paths have short names like C (for my source code) and lib (for the libraries). And of course, you need to type the path for each DIR command.

However, I was mystified when I tried to run the C compiler. After typing cc, nothing appeared, it kept saying "Archivo inexistente" (file doesn't exist) I was pretty sure it worked the last time. Well, almost 30 years ago.

I tried the MEM command, and it said 99240 bytes free. And running the Info.p utility with the C compiler executable gave me the answer: program size = 38112 bytes, stack size = 8192 bytes, global data = 56816 bytes, after adding all this it required 103120 bytes. I was baffled! For sure I compiled some programs without any extra memory.

It took me a full hour to remember the transputer processor has 4 kilobytes of internal memory, and if you map the 128K RAM in a classical mirrored way, the processor never uses the bus when accessing the lowest memory area $80000000 - $80000fff, but it uses the bus for any address outside like $80020000 - $80020fff, and this mirrors in the lower 4K of the external RAM. So a simple enhancement to get extra 4K was expanding the workspace to start at $80021000 (it moves toward lower addresses).

I modified the boot up code, and the C compiler was up and running!

How to assemble your own files

Given the different size instructions in the transputer, it was too difficult to write a linker. So instead I resorted to a mixed approach, my assembler allowed to process concatenated transputer assembler files, and in turn generate a preliminary executable. This file should be processed by ejecutable.p and this way the user enters the stack size, and the program adds a header information to indicate how much stack space it would use.

Executable header
Offset	Field
+0	OTBE (0x4542544f) Oscar Toledo Binario Ejecutable
+4	0x08031996 (Mar/08/1996)
+8	Code size
+12	Extra data size
+16	Stack size
+20	Data size
+24	Unused
...	...
+60	Unused

I never implemented proper command-line processing in my operating system, so every program had to start with an entry sequence to provide the options and filenames. This in turn made harder to implement batch command files, because all the programs required user input.

I started to compile each program in the source tree I had, and when I executed the raytracing program and the 3D modeler program, I couldn't get any image in the output.

I also had forgotten how to link my own programs! Turns out I made a document with a description of how to rebuild each program, sitting nicely as Documentos/Programas.doc and the Mat.len library was meant to be put at the start of the assembler program, so it could load properly its constants. Without the startup code every mathematical function returned zero.

By the way, I rediscovered why I made a small 512 KB RAM disk in the B drive. First, it was the memory available in the host Z280 system, second, it helped me to assemble faster the programs as it didn't have to access the SCSI hard drive.

So I only had to compile the C language program (read from the hard disk drive) to the RAM disk, and assemble over the RAM disk. Finally, the completed executable could be written back to the hard disk. This was way faster than any assembling I had done before over floppy disk, and it was pretty much required as my programs had grown.

Floating-point errors

Now I got an image in the 3D modeler but it had obvious mathematical miscalculations. I could revise routine by routine, and try to get an idea of what was wrong, or I could go over the small bunch of floating-point instructions and check for correct emulation. I was lucky to find almost immediately that fpldnldbi only removed items in a strange way from the processor stack, and that fpldnlsni had a doubled index (probably I cut and pasted the code from fpldnldbi). And the Utah teapot got displayed in all its 3D glory for the first time since 1996. Good memories!

Utah teapot with rendering bugs in transputer emulation.

Utah teapot rendered in 3D after the corrections.

I got the Utah teapot vertex data from the Appendix D of the book "3D Computer Graphics" which included a course on 3D computer graphics, and the project for the reader (student) was writing a 3D modeler based on the calculations presented by the book. It learned a lot with this book, and I was able to code the modeler in a few weeks.

Interestingly, I thought the ray tracer worked, but it had a very particular failure while handling cylinders, creating random black pixels on the image. Along the path I corrected the cloudy sky function, it passed col[2] instead of col2 causing a crash because it wrote on random memory addresses.

I reminded this was a known bug, I never could make the cylinder code to work at the time. For some reason, I couldn't find the bug. Now with a full emulation, I was able to pinpoint the bug to the quadratic intersection function:


Iptr=8000a0d3 A=8000a0ae B=8000a098 C=8000a098 Wptr=8000f4e8
FA=00000000 FB=4132972c687bac40 FC=4132972c687bac40  ldlp 3
Iptr=8000a0d5 A=8000f4f4 B=8000a0ae C=8000a098 Wptr=8000f4e8
FA=00000000 FB=4132972c687bac40 FC=4132972c687bac40  fpldnlmulsn
Iptr=8000a0d7 A=8000a0ae B=8000a098 C=8000a098 Wptr=8000f4e8
FA=80000000 FB=4132972c687bac40 FC=4132972c687bac40  ajw -2
Iptr=8000a0d8 A=8000a0ae B=8000a098 C=8000a098 Wptr=8000f4e0
FA=80000000 FB=4132972c687bac40 FC=4132972c687bac40  ldlp 0
Iptr=8000a0da A=8000f4e0 B=8000a0ae C=8000a098 Wptr=8000f4e0
FA=80000000 FB=4132972c687bac40 FC=4132972c687bac40  fpstnldb

It did a floating-point load and multiply in single precision, and then immediately saves the value as double precision. So the bug wasn't in the ported code, but in my C compiler.

It happens my C compiler tries to optimize code using 32-bit floating point if all values are 32-bit, but if the operation is too complex, it saves the current value in the workspace as a temporary. There is just a small problem: It always saves the temporary as a 64-bit floating point value.

The change was "relatively" easy. The original code in my compiler ccexpr.c file reads as:


      if(haz_compatible(&izq, info, &der, info2)) {
        crea_nodo(N_MULPF, izq, der, 0);
      } else {

The function haz_compatible() makes compatibles the types for the left and right nodes of the expression tree, and returns a non-zero value for floating-point operations. The transputer assumes we are passing the right values to the instruction, so we can use the fpmul instruction for both float and double.

However, we need to add information for the exception of both being float type, for the purposes of saving the temporary value if the expression tree is too complex:


      if(haz_compatible(&izq, info, &der, info2)) {
        crea_nodo(N_MULPF, izq, der, *((char *) info[0]) == FLOAT);
      } else {

This way the esp[] array (written by crea_nodo) will contain one if the operands are float instead of double. As the code evolved from a C compiler that didn't support struct, the pointers are saved as integers in arrays (yes, very unportable!)

Now we should modify the code for saving temporary values in ccgencod.c. This is the original code in the function gen_nodo():


      } else {
        gen_nodo(nodo_der[nodo]);
        salva(1);
        gen_nodo(nodo_izq[nodo]);
        if(op == N_SUMAPF) {
          recupera(2);
          return;
        } else if(op == N_MULPF) {
          recupera(3);
          return;
        }
        recupera(1);
      }

/*
** Salva el registro A en la pila.
*/
salva(flotante)
  int flotante;
{
  if(flotante) {
    emite_texto("ajw -2\r\nldlp 0\r\nfpstnldb\r\n");
    pila -= 2;
  } else {
    emite_texto("ajw -1\r\nstl 0\r\n");
    --pila;
  }
}

/*
** Recupera el registro A desde la pila.
*/
recupera(flotante)
  int flotante;
{
  if(flotante) {
    emite_linea("ldlp 0");
    if(flotante == 1)
      emite_linea("fpldnldb");
    else if(flotante == 2)
      emite_linea("fpldnladddb");
    else if(flotante == 3)
      emite_linea("fpldnlmuldb");
    emite_linea("ajw 2");
    pila += 2;
  } else {
    emite_texto("ldl 0\r\najw 1\r\n");
    ++pila;
  }
}

It saves the temporary value using the function salva(), and restores it using recupera(). We add the information for float values:


          } else {
        gen_nodo(nodo_der[nodo]);
        salva(esp[nodo] ? 2 : 1);
        gen_nodo(nodo_izq[nodo]);
        if(op == N_SUMAPF) {
          recupera(esp[nodo] ? 5 : 2);
          return;
        } else if(op == N_MULPF) {
          recupera(esp[nodo] ? 6 : 3);
          return;
        }
        recupera(esp[nodo] ? 4 : 1);
      }


/*
** Salva el registro A en la pila.
*/
salva(flotante)
  int flotante;
{
  if(flotante == 1) {
    emite_texto("ajw -2\r\nldlp 0\r\nfpstnldb\r\n");
    pila -= 2;
  } else if(flotante == 2) {
    emite_texto("ajw -1\r\nldlp 0\r\nfpstnlsn\r\n");
    --pila;
  } else {
    emite_texto("ajw -1\r\nstl 0\r\n");
    --pila;
  }
}

/*
** Recupera el registro A desde la pila.
*/
recupera(flotante)
  int flotante;
{
  if(flotante) {
    emite_linea("ldlp 0");
    if(flotante == 1)
      emite_linea("fpldnldb");
    else if(flotante == 2)
      emite_linea("fpldnladddb");
    else if(flotante == 3)
      emite_linea("fpldnlmuldb");
    else if(flotante == 4)
      emite_linea("fpldnlsn");
    else if(flotante == 5)
      emite_linea("fpldnladdsn");
    else if(flotante == 6)
      emite_linea("fpldnlmulsn");
    if(flotante < 4) {
      emite_linea("ajw 2");
      pila += 2;
    } else {
      emite_linea("ajw 1");
      ++pila;
    }
  } else {
    emite_texto("ldl 0\r\najw 1\r\n");
    ++pila;
  }
}

It only remained compiling again the C compiler, and the ray tracer program. After the correction it could render the test cylinder correctly, and finally I could see correctly the ray traced robot I made in 1996. It only took me 29 years to correct the bug! Now it looks so simple, but at the time I couldn't find it.

Ray traced robot with rendering bugs in transputer emulation

Ray traced robot with rendering bugs because a bug of the C compiler for transputer.

Ray traced robot in transputer emulation

Ray traced robot in the transputer after the C compiler corrections.

Installing it (just like new!)

Make sure your Terminal is setup as 80x25 characters, and using ISO Latin 1 (ISO-8859-1) as character set. Did I told you I discovered I could innovate? I used the $80-$9f characters (not used by ISO-8859-1) to put box graphics like in PC as I could define whatever I want in the VGA graphics card, but currently these will appear as blank in the terminal. Maybe I could make a converter to UTF-8.

The os_final directory was added to the git, and inside the tree directory contains the backup of my operating system. There is already a prebuilt image of the floppy disk, and the hard disk, so you only need to run the emulator with the proper arguments. However, for the sake of completeness, here is the sequence of steps to rebuild the drive image files.

The following command lines are provided into the build_f1.sh, build_f2.sh, and build_hd.sh shell files.

With the command line at the os_final directory, the 40 mb. hard disk drive image file is created using the buildboot utility, it includes only the file Interfaz.p (the command-line processor):


    ../transputer/os/buildboot -hd -v2 harddisk.img . tree/Interfaz.p

Now the first floppy disk image is created with the basic files required:


    ../transputer/os/buildboot -fd -v2 floppy.img . tree/SOM.32.bin tree/Interfaz.p tree/Editor.p tree/CC.p tree/Ens.p tree/Ejecutable.p tree/Info.p tree/Halt.p tree/C/CC.c tree/C/CCanasin.c tree/C/CCexpr.c tree/C/CCgencod.c tree/C/CCinter.c tree/C/CCvarios.c tree/C/CCvars.c tree/C/Checar.c tree/C/Compara.c tree/C/Concat.c tree/C/Conjunto.c tree/C/Copiador.c tree/C/CPC.c tree/C/DARC.c tree/C/Divide.c tree/C/DZIP.c tree/C/Editor.c tree/C/Efem.c tree/C/Ejecutable.c tree/C/Ens.c tree/C/Explode.c tree/C/Filtro.c tree/C/GZIP.C tree/C/Hora.c tree/C/Inflate.c tree/C/Info.c tree/C/Interfaz.c tree/C/Libro.c tree/C/Monitor.c tree/C/Prepara.c tree/C/Semana_santa.c tree/C/Som32.c tree/C/TAR.c tree/C/Unreduce.c tree/C/Unshrink.c tree/C/Vaciado.c tree/C/Halt.c tree/lib/E.len tree/lib/Graf.len tree/lib/MAT.len tree/lib/Mensajes.len tree/lib/Stdio.len

Now the operating system should be run using my transputer emulator (a further argument with an ISO image can be added to get a CD-ROM in drive D):


    ../transputer/tem -os2 maestro.cmg floppy.img harddisk.img

Now the files can be copied to the hard disk drive using this command line (CD creates a directory, and COPIA copies files):


    CD C:/C
    CD C:/Lib
    COPIA *.P C:
    COPIA *.C C:/C
    COPIA *.LEN C:/C
    DIR C:
    HALT

The second floppy disk image is now created:


    ../transputer/os/buildboot -fd -v2 floppy.img . tree/SOM.32.bin tree/Halt.p tree/Desarrollo/Ajedrez.c tree/Desarrollo/Reloj.c tree/Desarrollo/Tetris.c tree/C/Modelado.c tree/C/M3D.c tree/Dibujo_3D/* tree/Documentos/Archivo.doc tree/Documentos/Detalles.doc tree/Documentos/Ens.doc tree/Documentos/Peticiones.doc tree/Documentos/Programas.doc tree/Documentos/Servicios.doc tree/Documentos/Trabajo.doc tree/Documentos/Transputer.doc

Now the files can be copied to the hard disk drive using this command line (CD creates a directory, and COPIA copies files):


    CD C:/Desarrollo
    COPIA Ajedrez.c C:/Desarrollo
    COPIA Reloj.c C:/Desarrollo
    COPIA Tetris.c C:/Desarrollo
    COPIA Modelado.c C:/C
    COPIA M3D.c C:/C
    CD C:/Documentos
    COPIA *.doc c:/Documentos
    HALT

You can now reset the floppy disk image (just make sure it contains the SOM.32.bin file), or delete manually the files using the BORRA command (delete). The whole operating system (including source code) fits in under 3 megabytes.

Using the operating system

You can see a list of the available commands typing AYUDA (if you read my previous article, this is way more complex than the few commands of my early operating system).

AYUDA, shows a list of the available commands.
DIR [path] -A -P, displays the directory for the path, the -A option shows names in two columns, and the -P option show the directory in pages.
COPIA [wildcard] [path], copies files from the source to the target path.
REN path new_name, renames a file to the new name.
ILUSTRA path, displays the content of a file.
BORRA wildcard, deletes a file. It can use wildcards.
VER, shows the command-line processor version.
MEM, displays the available memory bytes.
LINEA text, displays the text in the screen (useful for batch commands)
BLP, clears the screen.
FIN, exits the command-line processor.
BD path, deletes a directory.
CD path, creates a directory.

To run several commmands in sequence you can use the semicolon, for example:


    C:EDITOR ; C:CC ; C:ENS

Or you can see an small example of multitasking typing this command to get a clock in the upper-right corner of the screen:


    @C:/DESARROLLO/RELOJ

To exit the operating system use C:HALT as this will close correctly the drive image files.

Compiling programs

To compile programs just type C:CC and press Enter. Type N for the first two questions, then type the path to the source code file, for example, c:/C/Hora.c and then the target assembler file, for example, b:hora.len

You should assemble the program, type C:ENS and press Enter. Type b:hora.len and press Enter, type c:/lib/stdio.len and press Enter, now press Enter alone, and type b:hora.e as output file.

Now we need to convert it into an executable file, type C:EJECUTABLE and press Enter, type b:hora.e and press Enter, type 8192 and press Enter (stack size), type 0 (zero) and press Enter (extra data size), and type C:hora.p as output file.

To run the program type C:Hora and press Enter.

Documentation

I was starting to get a grasp of how documentation was going to be fully electronic, so I wrote some incomplete descriptions of the operating system, file system, programs, and some further notes. All these files are available at the Documentos directory. It was a very different time.

If you read my previous article, you can access the help window of the visual editor using Fn+F1 in macOS. You can read text files using Fn+F4 and navigate files with the arrow keys.

Other programs

Other programs in this operating system are:

Checar.p, it verifies the integrity of a drive.
Compara.p, it compares two text files for differences.
Concat.p, it concatenates several files.
Conjunto.p, it shows the charset.
Copiador.p, it serves to copy a floppy disk (not necessary, as you can copy image files manually)
CPC.p, it serves to copy files from disks with PC format.
DARC.p, it can decompress ARC files.
Divide.p, it divides a file in several parts.
DZIP.p, it can decompress ZIP files.
Efem.p, a program to show ephemerals.
Filtro.p, converter from PC-8 charset to ISO-8859-1. Developed Sep/29/1995, that day all my source code got translated from PC-8 to ISO-8859-1 encoding.
GZIP.p, a program to decompress gzip files.
Hora.p, shows the current time and date.
Info.p, shows info about the executable file.
Libro.p, formats a text file as a foldable book for HP LaserJet IIP
Monitor.p, debugging monitor for low-level development.
Prepara.p, formats a disk (not necessary, buildboot does this)
Semana_santa.p, calculates the easter week for any year.
TAR.p, creates/test/extracts TAR files (used this for DAT tapes).
Vaciado.p, dumps a file to ASCII hexadecimal.
Modela.p, 3D polygonal modeler, you can feed it the escena* files from the C:/Dibujo_3D directory. For example, C:/Dibujo_3D/escena_tetera
M3D.p, ray tracer, port of the Pascal ray tracer. You can feed it M3D files from the c:/Dibujo_3D directory without the M3D extension. For example, C:/Dibujo_3D/P

As you can see I was pretty busy coding all these!

There are three animations of the ray traced robot walking (I used these with the Pascal version of the Ray Tracer), however, I don't have tested these, and the process utility to create a FLI animation was written in Z280 machine code.

What happened later?

I like how this operating system shows original thinking for solving problems, like working with only 128 kb. of RAM, having 31 letter filenames, using full pathnames for referring to files, adapting the free space of the character set for graphical characters, managing a RAM disk to accelerate development, and of course getting to develop a K&R C compiler over it.

I could have made more things, however, the host computer was showing its slowness in particular with graphics, and I reached the limit of the RAM memory. This prevented me of expanding the C compiler to ANSI C, I could compile very few programs from external sources, the linker was never finished so I couldn't compile programs in modules (and in turn the C compiler didn't implement extern or static). I was stuck.

My experiment with transputer was technically finished, although I did a few more programs thinking in the future (like a program to create CD-ROM file systems expecting a CD recorder I never had).

My first visit to an Internet coffee shop was around the summer of 1996, I discovered sadly the Inmos demise, and it was a total disappointment as their promised T9000 with a ten times speed fold never materialized.

Also a newer 32-bit processor was coming and my father was building a new computer where I built new software, and later I would make my windowed operating system, and all my tools from transputer would be ported to this bigger machine.

A by-product of my C compiler, was porting it back to the Z280 computer to compile an educative integrated Z80 development tool (editor/assembler/uploader) I made already using Mark DeSmet PCC for MS-DOS, I cheered myself when I compiled the same program for my Z280 computer without any changes. That was the last gasp of the Z280 computer.

The source code is released at https://github.com/nanochess/transputer

Enjoy it!