172: Tell Forth You Me Please
Transcript for Embedded 172: Tell Forth You Me Please with James Cameron, Christopher White, and Elecia White.
EW (00:00:06):
Welcome to Embedded. I'm Elecia White, here with Christopher White. Our guest this week is James Cameron. We're going to talk about the Forth programming language and One Laptop Per Child. These are related, I promise. Before we get started, don't forget that mbed Connect is coming up probably the Monday after this drops. So if you want tickets, still email me, and I can get you in. Also, finally, I am handing out stickers both at that conference and in mail. So if you want a sticker, send me your mailing address, and I will mail you a sticker. Okay. That's all.
CW (00:00:46):
Hi, James. Good morning, I guess?
JC (00:00:49):
Hi, g'day.
EW (00:00:51):
That's nice and generic. Can you tell us a bit about yourself?
JC (00:00:56):
Well, let's see tall, bearded, gray, um, from the point of view of languages, because you asked me about Forth, the first language was English and then SODA, uh, then BASIC and after that was machine code and then Forth. I had a TRS-80 when I was a child and...
CW (00:01:19):
Which one?
JC (00:01:22):
Well, young, male, human, actually, but... I was gonna say, yes, um, a Model 1 Level 2.
CW (00:01:31):
Alright. I remember this.
JC (00:01:32):
With the reset button at the back, that was always fun. I had weeks and weeks of learning, how that thing worked, reverse engineering it, peaking and poking. And my father used to bring home some articles on things and he brought home an article from Byte on Forth and I was hooked. Yeah, so that's how I started in computing. Um, then later on I got a degree in business, and uh, didn't really go back to electronics and embedded systems except as a hobby, but these days I'm more into it than before. So now I'm a developer at the nonprofit One Laptop Per Child, uh, and I'm focused on sustaining engineering to make sure production will continue. Um, I live with my wife in the Australian Outback miles from anywhere.
EW (00:02:33):
Six hours from the beach.
JC (00:02:34):
Yeah.
EW (00:02:37):
With the kangaroos.
JC (00:02:39):
Yes. Plenty of kangaroos.
EW (00:02:42):
Well, I know you listened to the show, so you are familiar with lightning round.
JC (00:02:46):
Yes.
CW (00:02:46):
That we are going to be inflicted upon you.
EW (00:02:48):
Have you, have you thought about all the questions and what you're going to answer?
JC (00:02:53):
Yes. I'm afraid I have. I'm, I'm looking forward to the question that I haven't heard.
CW (00:02:59):
Oh.
EW (00:03:00):
I have one.
CW (00:03:00):
Okay, good.
EW (00:03:03):
What language should be taught first in university courses?
JC (00:03:09):
For programming? I would say English is the first one... I've, I've struggled over this question. That mainly it doesn't matter, because programming is emergent from the symbols that you have inside you. And so it really doesn't matter which language. So I haven't been able to recommend one. I mean, I've tried a few, I'm afraid that quick round... that's a bit long.
EW (00:03:40):
That's okay. You know, you get points off for that, but the points won't do you any good, so why not?
JC (00:03:45):
Yes.
CW (00:03:46):
Alright, alright. Favorite fictional robot.
JC (00:03:48):
Dors Venaballi from Foundation series by Isaac Asimov.
CW (00:03:54):
I love that everybody has a really different answer for that one.
EW (00:03:56):
Least favorite planet.
JC (00:03:59):
Least favorite? Oh...the red planet on Anne McCaffrey's Pern series.
EW (00:04:08):
Oh.
JC (00:04:10):
The planet with the Thread.
EW (00:04:11):
Oh, that's a good answer.
JC (00:04:13):
That would have been my least, least favorite type. I cried when that dragon first went there.
EW (00:04:20):
That was the White Dragon.
JC (00:04:22):
Yes. That's what one it was yes, I was. Yeah.
EW (00:04:25):
People out there are nodding along. And Christopher's like shaking his head.
CW (00:04:28):
I know, I.. that's like the one series I haven't read. Is it, um..
JC (00:04:29):
Oh, give it a try.
CW (00:04:31):
I will. I will. I keep meaning to. Is a mobile phone an embedded system?
JC (00:04:37):
Well, I don't take my phone to bed, so it's not embedded. Yes and no. It's hard to say. It changes the boundaries on those flexible ones where, um, since everyone has one, it can't be embedded.
CW (00:04:58):
That's kind of an interesting point of view actually.
JC (00:05:02):
Yes.
EW (00:05:03):
Should we bring back the dinosaurs?
JC (00:05:06):
We have them now, the birds, they're with us. We need to keep them rather than bring them back.
CW (00:05:14):
And we saw a really big one this morning.
EW (00:05:17):
Yeah. Great heron was trying to fish in our pond and it is GREAT.
CW (00:05:22):
When making something new, should you target ease of use for new users or powerful features for, for more advanced users?
JC (00:05:32):
I think targeting powerful features for more advanced users is the right thing to do because when you're the maker making something, you are the most powerful user for the thing you're making. And if you hold back with what the thing can do, then you're intentionally defeaturing it. You're crippling your own creation. Later on you might pull out things that you realize are dangerous, but initially it should be as complex as you can manage.
EW (00:06:09):
What lightning round question should we add?
JC (00:06:10):
Hmm.
EW (00:06:14):
You wanted a new question.
JC (00:06:18):
Analog or digital multimeter?
EW (00:06:19):
Alright. We'll ask.
CW (00:06:19):
Most important to your daily work: soldering iron, keyboard, mouse, or pencil?
JC (00:06:29):
Uh, keyboard. I use Emacs.
EW (00:06:32):
Tabs or spaces.
JC (00:06:34):
Yes.
EW (00:06:35):
How many spaces are in a tab?
JC (00:06:39):
Eight.
EW (00:06:39):
Eight!?
CW (00:06:39):
Eight!
JC (00:06:39):
Eight.
EW (00:06:42):
Wow.
JC (00:06:45):
Yes, there eight spaces in a tab. You have to have that assumption because someone somewhere is going to be using a character cell terminal. So that's one reason why I don't use tabs much at all. I press the key so that the text moves. I would prefer the text to move as a result of spaces rather than terms, because there isn't an agreement on how big a tab is. So I have a tendency to commit code and get with all the tabs changed to spaces, just so that I can be clear about what I mean. And especially if you're using Python.
CW (00:07:30):
Oh God.
EW (00:07:30):
Oh yeah.
EW (00:07:31):
Besides embedded FM, what do you listen to for podcasts?
JC (00:07:35):
Hmm, several of the government radio, podcasts in Australia. Some of them are on BBC as well, such as All in the Mind. I listen to Dave at The Amp Hour and lots of stories like Escape Pod and Dribblecast and Cast of Wonders and that sort of thing.
EW (00:08:00):
Escape Pod is really cool. Varied, but very escape, but very cool. Alright. Um, last question for lightning round, what is your preferred calculator?
JC (00:08:13):
Uh, HP-33E from 1979. It still works.
EW (00:08:17):
I knew that that was going to have a definite answer. Uh, so let's go on to what we promised, which was Forth. And this is a programming language.
CW (00:08:27):
Let's go forth.
EW (00:08:27):
Yes. Let's go forth. Let's be forthright about this. I have so many of these. Um, yeah, so I guess the way I should phrase this is "tell Forth, you me please".
JC (00:08:40):
Yes. Yoda. Yes. It's hard to say because it was so long ago that I started and the choices you make when you're a child for programming languages are not the choices you make when you're pushed into them by university or management or a project at hand. Um, but Forth is a very strange language. It can be as strange as you like. The, the syntax is weird, the grammar is weird to non-existent, but it's, I think for me, the language that closest fits to the way English works, how a word in English can have multiple meanings. And it depends on context and the arrangement is, you know, variable. I think that Forth started out being someone's pet project, Charles Moore, and then a few people saw it and thought it was great. And it went on from there. And it was heavily used in the radio physics community. And I had some connection into the radio physics, science research labs in Sydney. And that was interesting. But, as a language it's, it's a set of words and each word has a defined meaning and you assemble those words into new words and that's your functions. And each was delimited by a space. So the only word you can't use is something with a space in it or a tab, which is handy. And you build these words up into this dictionary and because there's no punctuation and grammar, parameter passing is like a stack. Hence if you like HP-33 calculators, you automatically like stacks because the only way to get anything done on a reverse Polish calculator is to put things in the right order and then hit the operations. So with no syntax for passing arguments, everything becomes what you do with the stack.
EW (00:11:00):
And, so these are always interpreted. Forth is, is always an interpreted language, not compiled. Not like C.
JC (00:11:09):
Well, sometimes, compiled or interpreted, it's really up to the developer, um, to explain that. Think of embedded systems development with cross compilation, when you use, uh, C or C++, you edit your source and you compile on your host and flash the image into the target and then press the big red button, off it goes and runs. And to change anything, you have to edit the source on your desktop or laptop and recompile and flash it again. And that latency in the iterative cycle can really kill the thought, can really make it hard to do by. But when you're using Forth, the first few steps seems the same, but the last steps are different. You edit the source and you compile on the host and you write the image into the target as you did before. But to change anything you can compile or test on the target over the serial port or whatever connection you have. And if you fail, you can hit reset on the target and try again. And once you're happy with what you did, you bring it back into your host, add it to your source file, then your next image will have it. And so the latency is reduced because you can, you can do development directly on the target, uh, instead of being tied to that long cycle of, of, uh, changing source and building.
CW (00:12:36):
That's pretty interesting. 'Cause the, the closest we come today with normal C development is, "Oh, I might change the value of a variable in the debugger to cheat, but I can't actually change the code". I mean, I suppose I could, maybe if I really wanted to, but probably not.
JC (00:12:51):
Yes. But you have to know in advance that you're going to need that variable.
CW (00:12:55):
Right. Right.
JC (00:12:55):
And so once you realize that, well, I'm not sure what that delay should be between those two things. Then in C, what you tend to do is make that a variable and then add a user interface to your code so that you can change that variable on the target, you know, like toggling a GPO or something like that. Um, and while that works, it takes a lot longer and you have to go back and do it again. Whereas with Forth, you get that advantage of operating on the target as a command language and just making things happen. And you can do some seriously weird stuff, you know, change, uh, the dictionary on the fly and, and, um, things that we don't really have the tools to do in, in the C environment. Um, kind of thing, like, back in the day, you'd, you'd write your C program and running your compiler and I'm talking 20 years ago was so expensive in time that you'd feel, well, maybe if I just changed the HEX file, you know, edit something in, change one of the parameters in there, and that would shortcut your development. Uh, so this is taking it further and doing that editing of things on the live target. And yes, it crashes a lot, but it crashes because you've done something seriously silly. You know, you've started a loop that the watchdog doesn't get to be touched anymore. And so the watchdog goes off or like on the ESP8266, if you write a Forth loop, which takes all the time away from the rest of the multitasking operating system, then the watchdog goes, I don't want you anymore and you gotta reset.
EW (00:14:52):
So I came across Forth as a way to bring up boards given a relatively complex system and a hardware engineer who preferred Forth. He built a Forth interpreter and over the serial line, we would verify that each GPIO worked the way we wanted and make small little functions to test submodules so that when he gave me the board, he could say, if you run this, it works, therefore it should run in your code and be fine. And then I would program in C. And I've continued the idea of board bring-up tests. It's something that I do talk about a lot, and I think it's important and how they become manufacturing tests often. But I have to admit, I do tend to write my own command line. I mean, a command line interpreter is a, I don't know, it's a, maybe, thousand line file, maybe, including all the commands and you, you get a little bit more control and a little more usability for me in manufacturing. Why should I be using Forth instead of my own custom command line?
JC (00:16:10):
Well, a command line interpreter is fundamentally something that takes text input and puts it into a buffer and then passes it into words and looks for the words in a list of commands. And that sounds like a Forth interpreter. Um, the command line interpreter then has to execute those words. And if you're not going to pass parameters between the words, then yes, you have what you need. I mean, I could imagine building a Forth that doesn't have a parameter stack, but you'd have to never be able to pass parameters between the commands you type. It's kind of like trying to use a Bash shell without redirection. You can do it and it makes it nice and easy because you don't have to worry about the complexity of passing data between processes. My view on it is that Forth is also a command language interpreter, and it just happens to be a little more flexible than a static one. But you might not need the flexibility or you might not want to leave such a powerful tool on your shipping product. Now it depends. Some manufacturing crews wouldn't want that sort of thing.
EW (00:17:30):
Because it is powerful and it can lead to let's just call fire. Yeah.
JC (00:17:40):
Yes, yes you could -
EW (00:17:40):
Depending on your system.
JC (00:17:40):
Well, I think so, in such a way that the system that you've left behind on the product is capable of damage and that can bring you to a few problems, like how do you cope with that? Being able to restore to last good configuration is a good, good one. An answer that I've used before is try not to save any state on the product, a product which has no saved state can never be in a configuration that won't boot or won't start up properly. And so if you can utterly avoid saving anything, then always work the same way.
EW (00:18:29):
I like that in theory, but I find in practice, I never get to do that. And then I thought it was almost always interpreted. And when you, as you, your description, you can then fold back in, makes a lot of sense. You build up your commands and if you want to build them up in the interpreted part where you type at it, or you want to build them up into the compiled part, you can do either one. Does that mean you're writing your compiled code in Forth? I've always written Forth and C. So, I'm, new to idea that you really build a Forth interpreter in Forth.
JC (00:19:12):
Yes. You, you build the Forth interpreter in, in whatever you have to hand, which, uh, you know, it could be C or it could be Forth. With one of the Forths I use, Open Firmware, it's built in Forth and the Forth makes a new Forth. With the other one I use C Forth.
EW (00:19:29):
Goes forth.
JC (00:19:29):
Yes, it goes Forth, that's right. With C Forth, um, you build, build it out of C and there's an inner interpreter which implements the threaded code. And so there's, there's two meanings for the word interpreter. There's the software is interpreting as you time or when you present her or there's an inner interpreter which implements the threaded code model where the, um, the definitions are a series of pointers to implementations and that conflict in the meaning, also that commonality in the meaning, can be a bit misleading. You see a full system spends very little time compiling or interpreting. Most of the time is spent executing the code that you've asked it to. There are ways to quibble about that Forth systems vary on how they've been implemented, depending on what you need to optimize for memory or CPU cycles. So the fastest is sub, subroutine-threaded code, where a program, a function, a word is a series of machine language called instruction. So the processor just jumps to it, executes it. And then on return, it's finished that function. It's fast, but it's expensive in memory because you end up having, three bytes, you know, a 16 bit system for every reference or 32 bits on an ARM, for every reference 'cause they're going to be relative calls. It's not often done that way because there are other ways of doing it. Indirect-threaded code is where a word is a series of pointers to the machine code and direct-threaded is where the word is a series of pointers to other words, but that latitude types, they need this inner interpreter to guide the execution, which runs through the tables or data, and, and makes the calls to the machine code. So yeah, there is this confusion about whether it's interpreted or compliant. Whenever you're talking to a Forth using a keyboard, you're answering the question from the interpreter, what do you want to do next? But if you make a new definition that gets compiled into the dictionary, it's really obvious when you're bit banging, when you want to make a GPO toggle on and off, if you turn the GPO on and off on the command line and press enter, and you watch that on your oscilloscope, it's a nice long pulse because the interpreter had to look up each word, but if you define that into a new function, a new word, and then execute that the pulse is extremely short because the execution is much faster.
CW (00:22:39):
Okay. That makes sense. It's, it's an interesting kind of, like you said, it's not quite clear it's more of a hybrid than, than a pure...
JC (00:22:48):
Yeah, it doesn't map to the meaning of compiled and interpreted used in other languages. So it's hard to get it across.
EW (00:22:57):
So I know you did some work on putting C forth on a Teensy, one of the Cortex-M4 versions Teensy 1.3, sorry, Teensy 3.1. Can you tell us a little about that?
JC (00:23:15):
Yes. I was already using C Forth for other work, and I heard about the Teensy and it looked interesting. So I bought one and bought two 'cause you always buy two in case you zap one...
EW (00:23:29):
Exactly.
JC (00:23:29):
And worked out how the SDK works that, um, Paul provided with that and found I could make something blink and once you can make something blink and use the UART, you can put C Forth on it if you've got enough memory. And so I took the important parts of, uh, the software development kit, the UART functions, the processor, initialization, getting all the clocks running and called that from C Forth. And as a result, C Forth is then up and running. It's just a matter of merging critical parts of the SDK with the important parts of C Forth. You see, C Forth, what it wants from, from the hardware is read the byte from the UART, write a byte to the UART and that's it the rest of it's execution. That simplified API to the hardware means you don't have to implement much to bring, bring the thing up on any given chip.
EW (00:24:45):
So how big does this Teensy, C Forth implementation end up?
JC (00:24:53):
I didn't try to make it terribly small. There's a lot of junk I left behind because I didn't need the space, but I think it was about 51 kilobytes. Um, I think there is a smaller one out there. Microstar's Iris. Um, they did it several months after I did and theirs was smaller. Forth is by nature, very small, because the functionality's all encoded in a dictionary, most of it, instead of in, in machine code and the dictionary tends to be a little bit smaller than the machine code.
CW (00:25:35):
Really.
JC (00:25:37):
Well for a given amount of functionality, yes.
CW (00:25:38):
Okay.
JC (00:25:40):
You pay for it in performance.
CW (00:25:41):
Right. Right.
JC (00:25:42):
For particularly time-critical functions, you would write them in C and create a Forth word, which is a wrapper for the function. Like if you had to bit bang SPI on something, or where you wanted to use a library from the software development kit, but most of the time for a given amount of functionality you end up using far less, uh, flash space, less space than your HEX file. And that's one of the compelling advantages when you're stuck with a hardware platform, when you're stuck with a particular size of flash, you have a choice of cramming, removing functions. If you're using C or trimming down the amount of memory that you've allocated. But with Forth, you start out with using a lot less anyway. So there really does seem to be a difference in, in the density.
EW (00:26:39):
I'm not quite following. I mean, if I have a function in C that is, I dunno, toggle GPIO line at one kilohertz. And inside this function, I have a GPIO set and then a stupid delay loop and GPI clear and a stupid delay loop. And then I just do a while around that, or maybe you have to call that all the time, something like that. If I do it in Forth and I put it in the dictionary so that it is faster, don't I still have to do those same operations? I don't understand how Forth can be smaller given that the operations in the end have to be done by somebody.
JC (00:27:29):
It's a, a consequence of the extreme modularization, the factoring that, that occurs, you, in Forth, there's a word called MS for milliseconds and you call that when you want to delay. And for GPO access, we typically have a word which takes a GPO number and sets it or takes a GPO number and clears it. And so the definition of a one kilohertz,, bit banging oscillator, would be a loop construct around a delay call and to to request to change the the GPO state. So it would be of comparable size in my experience to, uh, the direct C implementation. But that's because it's using other parts of the dictionary, which were already defined.
EW (00:28:37):
Oh, I see. Okay. I guess I was assuming reuse in C as well. And so I wasn't, I wasn't getting those together. Okay. That makes more sense. One of the reasons, one of the ways, you know, that you're using a Forth interpreter is the backwardness of it.
CW (00:28:56):
Um, you wanna rephrase that?
EW (00:28:57):
The Yodaism of it? The, the, reverse Polishism? I am not sure. The fact that you do the function call first, and as you noted, you put it on the stack. You, you say set GPIO or whatever you're using for that, that command. And then you're saying GPIO one, or maybe you say set GPIO wait, GPIO one, and then a time. And so it's, it's always, it's usually verb, object, object, or verb, object. Um, are there other idioms and identifiers that a C programmer would be able to, to say, "Oh right, you do that in Forth." It's, it's a different way of thinking about it.
JC (00:29:45):
Yes, yes. There are the way that you refer to GPI pin, GPIO pins, is you put the GPIO number on the stack, and then say, please set this. So you put in the argument first, the parameter goes first, whereas in C you tend to call a function and then put the arguments in parentheses. But the way the compiler deals with that in C is it takes the arguments and puts it before the call instruction. So the compiler does the reversing for you. In Forth, the compiler just compiles a reference to the word. And so you have to prepare the parameter stack before you get to that point. So you see, everything's odd the first time when, when you look at a new language and pretty much, most of Forth will seem odd. The lack of grammar and the parameter stack, and you don't just assign values to variables using the equal sign. And there are certainly patterns of syntax that do repeat. There's a tendency for Forth programmers to make a new word for anything you see a lot. You know, if you, if you find yourself typing three words in Forth more than once you tend to make a new word to encapsulate it, if you can figure out a way of describing that. So yes, there are certainly idioms and pretty much unlike any other language means that you're learning them from scratch.
CW (00:31:23):
See this is, this is one worry I have because I, I always used an RPN cal-, calculator when I was, uh, in high school and college. And, I forgot it one day during a grad school exam, but they had ones for me to borrow, but they were all the normal infix calculators, and I could not use it. So I'm worried that if I start using Forth, I will break my brain and never be able to write code again the right way.
JC (00:31:46):
No, no, that, that, that doesn't happen. When, when you look at Forth, your brain automatically switches into postfix. It's so quick and so easy, uh, that you can, you can even have an editor buffer on screen where you've got C code down on one frame and Forth code up on the other, and, and you've switched for term with, with your eyes, and it becomes totally transparent. Your brain very rapidly learns what Forth looks like and what C looks like, and you never have any confusion. Um, so no, um, the human brain is excellent at recognizing patterns and assigning a set of neural networks to each pattern.
CW (00:32:30):
We should have recognized Casio calculator.
EW (00:32:35):
He's still bitter about that test, aren't you?
JC (00:32:38):
I do remember with great enjoyment, handing my calculator to kids at school, and them not being able to figure out how to make a simple sum, because they couldn't find the equals key and, and their complaint was that there is no equals key. And that, that approach to the problem that there is no equals key, therefore I cannot use this calculator. Wasn't quite well thought out on their behalf. It should have been. Why is there no equals key?
CW (00:33:10):
Yeah.
JC (00:33:11):
Why is there a really large key marked enter and what does it do? It pushes the stack.
EW (00:33:21):
Okay. So it is different. We agree. And it, it is a little brain-bending. Especially since my description was backwards because I don't do RPN in my head, but how do you recommend learning Forth? I know we need to actually do it. We need to type it in. We need to work with it. But even before that, just syntax and getting a little bit used to it. Do you have any recommendations?
JC (00:33:48):
Well, the, the method that I used was, I had a lot of time. I was very young and I started building things with it, and that really applies to any language. You build something with it and you overreach and you struggle and you fail and you struggle again and read the source code and break the problem into smaller pieces and solve each one in turn.
JC (00:34:13):
Basically, you're gonna fail a lot in, in learning in a new language. Um, and as long as you can keep failing, you can keep learning. The same thing works with learning someone's spoken language, the look on their face when you make a mistake, this is very revealing and you need to adjust your words. The same thing with the Forth, if you do something enormously enthusiastic, and it crashes, then you know, you've done something wrong. So do it again, or it crashes again, or at least that crash is reliable, break it into smaller pieces and see if it'll crash in the first half or the second half.
JC (00:34:58):
But if you're doing it formally, you would start with a vocabulary, a list of words and what their meanings are and a vocabulary or a word list, will list out each word and what their stack effect is, what they do to the parameter stack, how many arguments they consume, how many arguments they give back. So the plus key on a HP calculator consumes two arguments and gives back one, and when you're working with Forth, you keep this in mind all the time. How many arguments have I got on the stack? Because if you get that wrong, your function will leak arguments or, it will consume more than it should. And you end up in control of, of all the stack operations.
JC (00:35:52):
So learning is a matter of doing and yes, it would not be easy for someone who has never used it before, but there doesn't seem to be any limit to human learning. I've met some very odd people who can just, who've, you know, picked up a musical instrument they've never played before and fiddled with it for a few days, and then they can do it. So it's the same thing with computer languages. I've surprised myself, several times in, in learning languages, I thought were never a good match for me, like Python. I can do Python now. I'm happy about that.
JC (00:36:36):
There's using all the standard libraries or the, the conventions and the standards. Forth is based on a standard. There's been several standards just like in the XKCD cartoon. There's been several standards of Python as well. And the libraries available in Python are great if you're operating on an operating system, but Forth hasn't historically been heavily used inside operating systems. There aren't the equivalent of standard libraries, but you can figure out how to call out. Forth on a Linux host, for example, can quite easily call out to sub-processes, can make network connections. Just the other day, I was, um, I bought this tin of chocolate flavored stuff from the shop, and it came with a kid's fitness band for, I think it was about $30 in, in your money. It turns out to be Bluetooth, low energy, and it turns out to be usefully scanned using Linux. And so I hopped into C Forth, which has an interface to the Bluetooth layer of Linux and started sending some interesting messages to see how it would react, because I didn't really want to load the app and use their software. I wanted to find out whether I could make it work myself. And of course I had some excitement once they just sent random command and it shut down and didn't wake up again. I figured out that I had triggered the prepare for shipping command, where the device shuts down and you need to plug it into the charger to wake it up and stuff. I was able to wake it up again. That's good. Didn't have to buy another one, but being able to just compose packets in Bluetooth and send them over the link to the device was really flexible. So I learned a lot about how that particular layer in C Forth would work for me, I didn't know before, and occasionally I had to read the source and occasionally I would, I might break something, but without breaking something, you won't learn much about it.
EW (00:39:05):
This has been part of your job too. This isn't just for breaking BLE bands.
JC (00:39:13):
Yes. Yes. It, I mean, it never was. My early career was in, I was working for an electrical engineering company and then I worked for Digital Equipment Corporation and then Compact when we were bought out. And then HP, when we were bought out and my time in the, in the big iron side of things, we almost never touched Forth. There was just no demand for it. But, when I left HP there was this project starting up called One Laptop Per Child, and I'd already been involved in the projects before I left HP as a volunteer, testing the laptops in the Outback where there were no wireless signals to confuse things. So when I left HP, I asked OLPC if they needed anyone. And they said, yes. So I ended up there and yes, OLPC used Forth in the BIOS for their laptop.
EW (00:40:17):
Okay. And One Laptop Per Child was something I heard a lot of, for a little while, a lot about for a little while, but haven't heard much about it lately, and I'm not sure everybody is sort of aware of what they do, although you'd think the whole One Laptop Per Child thing would give it away. But can you, can you tell us...
CW (00:40:38):
There's gotta be a rule - no more than one.
EW (00:40:42):
Can you give us a little bit about what that organization does?
JC (00:40:49):
Well, when it started, there were no netbooks. There were no small laptops. And in my opinion, back then the laptop market was trying to increase margins and making bigger and better laptops and charging more for them. And no manufacturer back in 2006 seemed willing to go down towards cheaper and smaller. They're all heading upwards towards bigger and better. I was working for a manufacturer of laptops at the time, though I was working in the big server room stuff, went back to CMS systems. And so, what One Laptop Per Child did, was create an entire new market by pushing downwards and smaller and cheaper. And, um, after, after they started, while I was still a volunteer from a distance, they caused other vendors to say, well, look, there is some money in this. And so the ASUS, the EPC came out and all these other small laptops started coming out. So it was a good push, which made a good change to the industry. The DS, originally it was make laptops for children in the third world who needed laptops, not the people who needed food first, but the kids who needed laptops and get the school systems, the education departments of the countries to, to deploy these laptops and have useful content on them, like, extracts of Wikipedia or resources from books.
EW (00:42:42):
Learning to read programs.
JC (00:42:43):
That's right. And many of these countries love the idea of having something in English as well, because English was felt to be the gateway to success. But of course they also want their local languages. They want to be able to teach their local languages. So the, the laptops had to be thoroughly internationalized, a keyboard design for the local language in collaboration with the local authorities, and software built for them and all that huge amount couldn't practically, practically be done if it was going to be closed source. So base it on an open source operating system and build everything as open source as possible. So that's, that's from 10 years away. That's what it was mostly about. We had lots of people involved. I was just a minor tester when I was first involved. As time went on, the teams changed and people have come and gone and products have come and gone. We made four different models, the XO-1, which was based on a geode processor with 256 megabytes of memory, oh, that was a lot. And then the XO-1.5, which was based on a Avia processor with one gigabyte of memory and a gigabyte no, four gigabytes of micro SD card mounted in a cage in the inside. And so I joined about that time, once they'd settled that design and then we moved on to ARM and made the XO -1.75 with a Marvell system on a chip. And so that had soldered down EMMC, soldered down RAM or passive cooling, h, so heat spreader rather than a fan. All in the same original industrial design, same external case with a few changes.
EW (00:45:03):
That was important because that initial industrial design was very focused on being rugged.
JC (00:45:10):
Yes, very thick plastic, very droppable, and fairly resilient to to water ingress. Um, not totally. We saw some laptops come back from being buried in mud for a few weeks. And if we know what happens to them, you get an electrolysis of the clot battery and the main battery and things don't work afterwards. But they can take a brief shower. They can take being dropped, , which if you're giving to kids is a good, good idea that they can withstand that sort of thing. And the whole idea was about child ownership. So that the children would be the security system for the equipment, rather than locking them in a classroom and only letting the kids use them during school hours. The idea was to let the kids take them home and use them at home. Some countries did it like that. Some didn't. There was variation in, but the product supported both modes of operation. Roughly 2.6 million have been produced and shipped. I don't get to hear much from where they went, unless there's a technical problem, which affects a thousand of them. And then I get to hear about it.
EW (00:46:43):
Hear about it, but not in a good way.
JC (00:46:45):
No, no, it's more sort of, how can we fix this? What do we need to do to make it work? Um, yeah.
EW (00:46:54):
So right now you're mostly focused on sustaining engineering. Is that right?
JC (00:46:58):
Yes. Yes. The, the product has been produced still and has been produced for several years. And as second sources come online, the task has been to write drivers for those things like a new camera module or add features to the operating system, if we can, or things like that. No way is, is large amount of development as early on, but still enough to keep things going.
EW (00:47:32):
And it's still all open source.
JC (00:47:35):
Yes. Yes. Mostly open source. The plan was at the beginning for everything to be open source, the operating systems and the apps are open source, but in the end the wireless firmware remained closed.
CW (00:47:50):
Yep.
JC (00:47:50):
And then there's, there's always, yeah, there's always people who will say, "Oh, that should have been open too", but I can't find any wireless firmware which is open.
CW (00:48:01):
Nope.
JC (00:48:01):
Unless, unless it's a wireless module that you can't upgrade the firmware on, in which case it has the firmware in a fast chip on the module. And so yes, it's closed. It just happens to be over there and we don't touch it, but the wireless firmware on the, on the XO laptop, it's built into the, into the open firmware as a binary, so we can load it from every laptop. So you, you get a copy of it automatically.
EW (00:48:29):
And how does it being open source affect how you feel about it?
JC (00:48:35):
I think, I don't think I would have been as involved or as easily encouraged if it was closed source because with closed source product, you have to do it all yourself. With open source, you can build on the shoulders of giants, you can borrow and reapply the source code you have from elsewhere. So we're using the Linux kernel and we're using the Fedora operating system or more likely the Ubuntu operating system and being able to rely on those things is immensely helpful. It reduces the effort, the time, the cost.
EW (00:49:18):
Lots of people get to rely on those. How does it being open source change what you do? Does that make sense? Am I headed off in a totally different direction?
JC (00:49:32):
Not sure because it requires thinking about an alternate reality of, of how would it be if it wasn't open source? It would be me alone with the source code and I wouldn't have as many friends. And that would be sad.
EW (00:49:53):
There is that, that it is a community. You know, a lot of people work from home or as you do work remotely and you lack some of the office chatter. And so having it be open source, it does imply that there is some office chatter, although maybe of a different sort, but it's still, there are people working on it. When it's a vendor relationship, you don't always notice that the other side are people. You're just communicating through this user manual and this API and...
JC (00:50:34):
Yes, working on closed source projects at at Digital and Compact and HP there's, firstly, you're working with a very small group who's paid to do the job. So there's very rarely any quibble about code quality or comprehensibility. It's small sort of, well, the boss is paying this guy to write this code. It might be crap, but you know, he's being paid and we need to integrate it. And so we need to make sure that our code will integrate with his code, even though we don't like his code, but in the wide world of open source, if you put crap out, you get told. And so there's a very great tendency to write better. And in my opinion, the overall code quality of open source is somewhat higher than the code quality of closed source and in small teams. But on the other hand, you do get people who, you know, punctuation Nazis who will spend their time, giving you their opinion about exactly how it should have been phrased.
EW (00:51:52):
And tabs eight spaces? I bet there are a lot of people out there who want to tell you that, who want to tell you all about the goods and bads.
JC (00:51:59):
That's right. So there are advantages and disadvantages., When working on kernel patches, for example, if, if you have to as I did write a kernel driver for a camera, do I engage with the open source community or not? I could write the driver and hold it in our kernel repository and might never get out, or I could write the driver and post it for review. And that review will tell me things I didn't need to know and things I did need to know. And if you want to keep getting something reviewed, you have to keep up with the process of posting your, your patch to the Linux kernel mailing list or the appropriate list for what's being talked about and deal with the feedback. And that may mean rewriting parts of it, or reimplementing the way you do it. But by doing that, you, you will eventually become better at writing for the community that you're targeting. And so the next time you write something, it's going to be a better match for their understanding, and it will be more likely to be accepted. Most of my publicly accepted kernel patches are very small things. I'm not a big kernel writer. It's just sort of fixing the odd thing here and there, adding a USB device to some driver or um, fixing a wireless driver that we use on the, on the XO laptop so that it works better. Little things like that.
EW (00:53:34):
You did a little bit of kernel work, Christopher.
CW (00:53:37):
Yes. Yes. But my experience was different for different reasons. It was, it was less about getting feedback on the code and more about, we don't think this thing should exist. So, a separate issue.
EW (00:53:52):
And this, these laptops are quite complex. You sent me a link to a schematic and it was 40 pages long and I went, wow. And all of the pages were dense.
JC (00:54:06):
Yes, that's the schematic from our Wiki top laptop, our little, for the XO-4, yes, it is a lot more dense than a typical embedded system. Um, it's a laptop.
EW (00:54:20):
Exactly.
JC (00:54:21):
It's composed of several embedded systems. And one that works out where to put the power and when, and assists with the battery charging and the other one, which does the application processing for the user. And then there's, you know, embedded systems for the wireless and for the storage and for touchscreen and, and they're all separate and they've all got to work together.
EW (00:54:46):
I was talking to a friend recently about how do you learn new things that are related to your career choices? And it was just after I had looked at the schematic and I'm like, well, open source does have some value there too. You don't have to contribute. You can just read it and probably get a lot out. Do you, where would I start if I wanted to really look at that schematic? It was a little much to just deal with the wall. Is there an area, is there a special part that you think is the most fun?
EW (00:55:29):
Feel free to say your own part. We all do that. I understand.
JC (00:55:32):
Yes. Yes. Well, not much of the actual hardware was mine. Ae had some great electronics engineers back in the design stages and my job was more about testing and coding, but in the latest model, the ambient lights sensor. The original design was mine and it was modified slightly and that's just an LED operated in reverse and measure the capacitance. And that was very low cost and the LED was already on the bill of materials. So it was a no-brainer. That was implemented in firmware first by me. Then by Richard and Paul, the firmware engineers who worked on, on the embedded controller.
JC (00:56:19):
The touchscreen manufacturing tests were mostly mine as I was very familiar with how the touchscreens worked. It's infrared emitters with 190 degree light guide. So they emit away from the screen and then get, the light gets turned around toward the screen, goes across the front face of the screen and then gets turned around at the other end again, to come into the detectors. It's like rows and rows of detectors and emitters. And they're at a slight slant so that the lines of infrared go across the screen on a slight slant and the firmware in the touchscreen chip deals with all that. And all we have to do is read the the values out of the chip, but there's ways in which that, that assembly can not be assembled right. And so we needed tests that would, show what might be wrong. And so those tests in open firmware were mostly mine, with some help from Mitch.
JC (00:57:28):
Over the years, I've had my finger in almost all of it. I'm very familiar with it because it's part of what you do is sustaining engineering. You hear about a problem. You look into that part of the schematic and you work out what could have been wrong and what might've gone wrong in manufacturing and work out how to deal with it. But I don't think there's any place you can really start in a complex design. It's kind of like how to understand Forth, you dive in, you stall, you try and understand what something is. In the case of a schematic, you see if you can find a data sheet for the component to figure out what it is 'cause there's very few explanations in schematics for what each device is for. Overall design, there's a page at the front, which, or at the back, which shows the power domains and how they connect. And that's helpful to get a basic idea of which bits are powered and when.
EW (00:58:30):
That sort of thing is the sort of thing where I would probably go through each page and try to make each page one block on a sketch page and then try to connect all those blocks and then make bigger blocks or smaller blocks.
CW (00:58:44):
Well, you need to get a hierarchical picture of the whole thing. If you have just the schematic and the 40 pages, it's, I mean, it's hard to get an overview, so you almost need to sit down and break it down and say, okay, this is the major high load, okay, and here's this word, this is referenced and then start taking large pieces and trying to understand it that way. But yeah, if you're faced with "the thing" as it is, it's very difficult to kind of, comprehend that at once.
JC (00:59:05):
The waterfall model of software development, you know, let's write a specification about what, we're going to have the software do and then write the software. The same thing happens with hardware definition, you can have a functional specification of the product and then negotiate with the electrical engineers until you end up with a schematic, which describes the implementation of the design of the specifications. So yes, there is a spec for the laptop. Some of you have that spec for other reasons as well, but if you had read the spec of the laptop first and then read the, the schematic, then you would have a different view. But just diving into a schematic of a product is kind of like opening it up and trying to reverse engineering it. It's a tough way of doing things, but it's not necessarily the right or the wrong way. It's just a way. As long as the number of things that you keep in your mind never exceed seven, you can keep that up forever. But as soon as the number of things in immediate memory exceeds seven, you're going to have to write it down or figure out some way of memorizing it and then assembling a coherent understanding of a complex machine is a matter of grouping things together until you understand how everything relates. I look at a car these days. I remember back in my day...
CW (01:00:35):
Yeah, the carburetor and a block, and a fuel pump, and an exhaust manifold, and then you're done.
EW (01:00:41):
A couple of wheels.
CW (01:00:41):
Hey, ignition.
JC (01:00:42):
I live in a farming community where, where things are held together with fencing wire. And it's hard to do that with the cars these days.
EW (01:00:53):
Alright. This has been a great discussion and I have a few more questions for you.
JC (01:00:59):
Hit me.
EW (01:00:59):
Uh Quozl, your handle, it's sort of what you're known as online, probably because some movie director took your name or used it or something.
JC (01:01:11):
Oh, I have been confused for being the movie director before.
EW (01:01:15):
Yeah, I did have a bunch of questions about submarines, but I took those out. It's from an Alan Dean Foster book, is that right?
JC (01:01:23):
Yes. Yes. Um, the, the, the name I picked for myself was, um, when I was playing Net Trick, the computer game back in the 1990s, before we gave the internet to the masses, it was the name of an alien civilization and not an individual. So, whenever I refer to the name, it's a name of a group and I chose it when playing the game at lunchtimes, at Digital. I chose it because of their honorable approach to conflict, their, their stance, their rules of engagement, I guess the chivalry of how they approached that challenge. Because that was always an issue for me. If you're going to fight, you need to have a good reason and you need to do it fairly. So that's why it became my handle. Once I was using it in Net Trick, once the internet was released to everyone, it was a matter of, "Oh, this, this, this username hasn't been taken so I'll use that". And since most of my existence on the internet, back in those days was because of my association with Net Trick, that I ended up having hosting provided for me by one of the people involved on the project. And so my website got called "Quozl at the Front". It's a great book. I recommend the book. I'm not getting any kickbacks from Alan and I've read a lot of his work. If I was choosing a handle today, I would not choose that because, because I feel I would be imposing on someone else's creative works. But yeah, that's how it began. And it's more sort of, every time I'm faced with what username do you want? It'll be something that I try first. But I didn't mean anything by it.
EW (01:03:33):
I understand. I mean, Chris's handle is "stoneymonster".
CW (01:03:36):
Oh, sure, just tell everyone.
EW (01:03:38):
It has a, you know, it has a nice little history about it, but he's doomed forever with it.
CW (01:03:45):
But I can always get my username. Pretty much. There's been a couple places where that hasn't been available but usually...
JC (01:03:53):
Yes, I'm Quozl on GitHub.
EW (01:03:56):
Cool. This is the advantage to having an oddly spelled name. Alright. Uh, Chris, do you have any more questions?
CW (01:04:04):
I did have one question...
EW (01:04:04):
Excellent.
CW (01:04:05):
And kind of going back to Forth for just a minute. I don't know if we really touched on if, if somebody wants to learn Forth or get into Forth or try it out. Um, and maybe they're coming from an Arduino background or hobbyist. What would you recommend to just kind of the getting started kit to, to Forth?
EW (01:04:22):
James has been working on an ESP8266, those $4 Wi-Fi processors. Maybe I should let him tell us about it.
JC (01:04:29):
Yes, yes. I was using Lua with NodeMCU on the ESP8266, and I've got about 15 of them around the house doing various things like temperature sensing, controlling pumps, and motion detecting, and all that sort of thing, all on a local network, not on the great internet of things. But I kept hitting limits with NodeMCU. I'd add functions, and then the dreaded out of memory conditions would occur.
JC (01:04:59):
And when Mitch Bradley, the inventor of open firmware in C Forth ported to ESP8266, I jumped on board and I've been testing and extending it. It's based on the latest C Forth and C Forth itself can be built for various targets. So you can try it out on Linux or Windows. But you can build it for ESP8266 using the esp-open-sdk. And we also borrow some of NodeMCU just to get the sort of like a shim, a wrapper. We use NodeMCU for the lightweight IP, layer, so that we can send TCP packets and I'm quite impressed so far. It works great. I can write applications with them. I've done a few interesting benchmarks just to satisfy myself that I could do some genuine work with them, like tested the rate I could get by doing bit banging on GPIO, measured the the GPIO interrupt to GPI output interrupt latency.
JC (01:06:07):
So I've set up the CRO and set up a test script that would send a pulse into the thing and measured how long I had to wait before I'd get the response from the interrupt function, in change interrupt. It's got TCP IP networking, so you can write servers and clients to do things, but since you're writing in Forth, everything's different, you know, it's not like you're using a code from somewhere else. I've been able to bring up various MySQL C and one wire things, that's worked great.
JC (01:06:47):
One of the recent changes has been embedding open firmware on top of C Forth so that the device tree thing and the ability to access file systems. So, one of the recent tests I did was a micro SD card being able to read, whatever file systems on the micro SD card. And so that's great. I don't think that's something that I could do easily on NodeMCU or at least not with as much memory left over. So yeah, that's up on GitHub. It's really not in a great condition. Like it's not a nice, shiny GitHub repository for the uninitiated. You really have to be comfortable with compiling, C programs and, and understanding what we might have forgotten to push. So yeah, have a look at GitHub for that. I do have a couple of binaries. People can email me and I can send them the minor if they just want to try it out. But, I'm not sure, if I'm asking for help, 'cause it's, it's interesting, just using it myself, but you know, if anybody does something with it then great. We're interested.
EW (01:08:16):
And, are you communicating to the Forth interpreter, 'cause clearly I'm all about the interpreter part, through the serial port or through Wi-Fi?
JC (01:08:29):
Well, both actually the Forth interpreter is so well encapsulated that it's possible to do both. When it starts, you can, you're communicating through the serial port on the ESP8266, um, and I've tested it on ESP-01, the little eight pin modules and on the ESP32, , Adafruit HUZZAHs, I don't know how to pronounce that word. Um...
EW (01:08:54):
That was what I would say.
JC (01:08:55):
Oh, good. But I've also written a Telnet daemon, which lets you Telnet into it. No security. We're talking about IOT here. You can, you can enter Forth commands over Telnet and that's great for scripting with Python. I've built a thing like an FTP server so that you can read and write files on the flash file system. When I've built applications within, I have an application running on my bench at the moment, just reading from a temperature sensor and transmitting to a web server. That application could be either you build Forth with this Forth source file included and then flash that into the, into the chip. Or you can use a script which is stored in a file on the file system on the chip and just load that automatically on boot. And so I've used both methods and the latter method costs you an extra second of startup time or seventh of second, I think. Um, and your choice as to which way you want to do it. Or you can just type the commands in and see what happens. That's the way we do things in Forth.
EW (01:10:16):
Alright. That does sound pretty interesting. I can think of times when I have wanted to Wi-Fi into my device and debug it.
JC (01:10:27):
And let everyone else do it too!
EW (01:10:27):
Well, yes, there is that, but yeah. Wow. IOT insecurity. Might as well leave it open. They're going to break the lock anyway.
JC (01:10:37):
Well, I mean, who's gonna figure out that it might be running Forth. "That's weird..."
EW (01:10:44):
I don't think that obfuscation is a replacement for security.
JC (01:10:50):
No, it never is.
EW (01:10:52):
Do you have any last thoughts you'd like to leave us with?
JC (01:10:56):
Yes, yes, yes I do. I think what I've learned by having to do everything is that the time you learn is when you're doing something really complicated, that you don't understand and you get this doubt and fear in your head. "I can't do this. This just looks wrong. I don't understand it." And that's the best state to be in. If you approach that state with a willingness to step through the parts, you do understand and look for ways to explain the parts that are in doubt, then you get out the other side and you do understand it. And so I'm beginning to yearn for that state of mind, because that means I'm learning.
EW (01:11:44):
That is excellent. Very good reminder. I sometimes get in that state and get unhappy, but I think next time I'll remember what you said and maybe it will make me happier. Our guest has been James Cameron, a developer for One Laptop Per Child. I will include his GitHub link in the show notes, along with a bunch of others. Thank you, James, for being with us.
JC (01:12:11):
No worries.
EW (01:12:12):
Thank you also to Christopher for producing and co-hosting and of course thank you for listening. Hit the contact link on embedded.fm If you'd like to say hello, sign up for the newsletter or subscribe to the YouTube channel. I'm going to have to change that 'cause you're not supposed to hit the contact link for all of those, but you'll figure it out. You're very smart.
CW (01:12:30):
The subscribe link.
EW (01:12:32):
For some of them, and the contact for others.
CW (01:12:33):
Yeah.
EW (01:12:34):
It'll all make sense. I'm sure. Don't forget if you want to get a free ticket to the mbed Connect conference, email me and say so. If you want stickers, email me your address, your physical address. Do not email me your email address. That doesn't work yet.
EW (01:12:50):
And now a final thought. This one from e.e. cummings who oddly is a poet I like very much though don't understand much: "trust your heart if the seas catch fire, live by love though the stars walk backward".
EW (01:13:07):
Embedded FM is an independently produced radio show that focuses on the many aspects of engineering. It is a production of Logical Elegance, an embedded software consulting company in California. If there are advertisements in the show, we did not put them there and do not receive any revenue from them. At this time, our sole sponsor remains Logical Elegance.