536: I’ve Seen a Giraffe
Transcript from 536: I’ve Seen a Giraffe with David Such, Christopher White, and Elecia White.
EW (00:07):
Welcome to Embedded. I am Elecia White, alongside Christopher White. Our guest this week is David Such, author of a new book about embedded AI.
CW (00:18):
Hi, David. Welcome.
DS (00:21):
Hi, both of you. Thank you very much for having me.
EW (00:23):
Could you tell us about yourself, as if we met for lunch on the Sydney Harbour?
DS (00:31):
Sure. Well, it is a lovely day on Sydney Harbour today. I am David Such, an electrical engineer based in Sydney. I run Reefwing Software, which is a design and hardware and firmware company, and I enjoy writing about both those subjects. I also have a degree in wine science, and do some mentoring at Sydney University.
DS (00:50):
I started off in automation and controls at a company called Honeywell, and worked for them both here and in the UK. Then headed off down the management route and ended up running a large facilities management business.
DS (01:04):
A bit over ten years ago, I pulled the pin on that and made the decision to come back to engineering, which is what I actually enjoy doing. For the last few years, my focus has been on machine learning on microcontrollers. That morphed into the embedded AI book, which you mentioned. Thank you.
EW (01:23):
We will be mentioning it a lot more, but first we want to do lightning round, where we ask you short questions and we want short answers. We may not entirely behave ourselves and say why, when, what, how, but we will try to keep it short. Are you ready?
DS (01:40):
I am ready.
CW (01:41):
When will the robots win, and will they be merciful?
DS (01:47):
I think we are another five to ten years off.
CW (01:52):
Oh, good.
DS (01:52):
Yeah. <laugh>
CW (01:52):
There is a lot I can do in that time.
DS (01:55):
I am an optimist. I am a robot optimist.
EW (01:57):
And will they be merciful?
DS (02:00):
Well, that is the optimism side, so I think they will be.
EW (02:05):
Data driven or theory driven?
DS (02:07):
Data, 100% data. I think you start with the theory and then confirm it with the data.
CW (02:13):
Are giraffes real or were they hallucinated by LLMs?
DS (02:19):
I think they are real. I have seen a giraffe.
CW (02:21):
Okay.
EW (02:23):
Are they as improbable in person as they sound when you say, "Camel leopard with a giant neck"?
DS (02:30):
They are amazing, beautiful creatures. They have got very long tongues. I do not know if you know that. <laugh> I was feeding one at the city zoo with a carrot and it has got a very long tongue.
CW (02:44):
Python or C?
DS (02:47):
Depends. Can I say that? I would say C on device, and Python on the PC.
EW (02:53):
Learning, doing, or teaching?
DS (02:57):
Teaching.
CW (02:58):
Complete one project or start a dozen?
DS (03:01):
Start a dozen. I have always got a bunch of stuff in progress, and some never finish.
EW (03:06):
Do you have a favorite fictional robot?
DS (03:09):
I do. It is the robot on "Lost in Space." I think it is called the B-9 class environmental control robot. And an honorable mention to Robby the Robot in "Forbidden Planet."
CW (03:23):
Do you have a tip everyone should know?
DS (03:26):
Yes. Read the data sheet.
CW (03:29):
<laugh> Should you believe the data sheet?
DS (03:32):
<laugh> Well, not always. They look very intimidating, but the number of times that it has helped me solve a problem or discover why something is not working, I think it is very important.
EW (03:46):
You recently wrote a book. When I talked to you about it a couple months ago as you were finishing, you were in that stage of, "What have I done with my life?" and, "How did this happen? How can I make it stop?"
DS (04:04):
Yes. It took two years and was a lot harder than I was expecting. I do not know if you found that.
EW (04:12):
Oh, yes. The second edition was a lot easier.
DS (04:20):
Yeah. Well, I guess at least you are starting with something then. As part of that process, I rewrote the book about five times. By the end I was sick of reading it, for sure.
EW (04:33):
By the end of the first edition, I had read it several times, and I had finally truly learned how to write. So with the second edition, I did not have to learn how to write again.
DS (04:47):
Oh, well, that is helpful. And look, writing is like any other skill. The more you practice it, the better you get at it.
EW (04:52):
Absolutely. So your book is not about general AI, it is about embedded AI. That is the title, so let us call that obvious. But are you talking about Jetsons and Raspberry Pis, or proper microcontrollers?
DS (05:15):
It is a very much a project-based book. What I wanted to do was- It is fairly expensive too. What I did not want to load on top of that is making the reader have to go out and buy $700 worth of hardware to be able to do the projects in the book.
DS (05:33):
So what I have tried to do is select some fairly commonly available microcontrollers that are not too expensive, which still allow you to do something useful. For me, it was mostly about learning exercise, so I was selecting for that more than for if you were trying to build something for production necessarily.
EW (05:59):
Is the goal for it to be a textbook, or a series of projects for a hobbyist or someone entering the field, or is it a professional manual?
DS (06:13):
It is a good question. I think it is an intermediate book. This is probably not the first book you are going to read on the subject. I expect the reader to have some knowledge of electronics, and to be able to read a schematic, and to have some understanding of uploading firmware. But not necessarily a lot of knowledge around machine learning or AI.
DS (06:39):
I think there is a spectrum along there. It is for people like engineers or people that are transitioning. They are maybe embedded engineers looking at the move into machine learning. Or it might be students, people like that.
EW (07:01):
That makes sense. You deeply covered a few areas that I often see people struggle with. Your IMU chapter, which I expected to be just here is how you use an IMU, went into a lot more detail about sensor fusion and even Kalman filters. But how is that machine learning? That is all very heuristics.
DS (07:25):
Right. What I was trying to do was to provide the necessary tools to be able to do the work. Where that all came from, the book I wrote before this book was actually about drones and flight control software. The problem was I was five years too late for that, and so no publishers were interested in it.
DS (07:49):
But the fact is that you need to be able to get the right data into your project, and that starts at the sensors. So I think there are three chapters that focus on how you can use those cheap sensors and provide practical data from them, which you require for any sort of embedded project.
EW (08:17):
But Kalman filters, in particular. You had some other filters and how to deal with the data and how to use quaternions, and all the things that I have done with IMUs so many times.
EW (08:30):
It was a joy to be able to say, "Oh, well now I am going to stop explaining this, and just start pointing people to this." I expected convolutional neural networks, which you have, and recurrent neural networks, which you have. But I was not expecting that.
CW (08:49):
Is it from the approach that your data has to be cleaned? Or for a microcontroller especially, put in a more compact format than just throwing it all the raw sensor data?
DS (09:03):
Yes, that is part of it. It is also part of my thesis that intelligence is moving further and further to the edge. The edge used to be the microcontroller, but now the edge is the sensor. I have got a few projects in there where there is machine learning happening on the sensor, so you do not even have to bother the microcontroller.
DS (09:26):
I think what a lot of people do not realize about IMUs is they give you these readings, which are- They are not very useful, are they, by themselves? You have got to turn them into some sort of orientation.
EW (09:39):
They give you all kinds of readings, some of which are very useful, some of which are not, and it depends on what you are doing.
DS (09:45):
Well, sorry, yeah. What I meant was not that they are not useful, but they are in a form that- Normally you do not want to know what the acceleration rate is. You want to know whether it is pointing up or down.
EW (10:00):
Or you do want to know the acceleration rate, but you do not want gravity.
DS (10:05):
Correct.
EW (10:07):
Yeah, no, the IMUs are a pain. You want Euler angles. Nobody actually wants quaternion values, but we all need them. It is like the worst form of broccoli.
DS (10:21):
<laugh> That solves the whole gimbal lock thing, does not it?
EW (10:24):
Right.
CW (10:26):
But if you have a model, an AI model, it should be smart enough to just figure out gimbal- I am sorry. No.
EW (10:33):
Some part of me wants to believe that you just throw some RNN or something that ends in "NN," neural network, and you just give it the raw data and it figures out temperatures and-
CW (10:48):
But you know as well as I do, that all it is going to do if you train it right, is come up with a Kalman filter somewhere inside it.
EW (10:54):
If you are lucky.
CW (10:55):
Well, right.
EW (10:57):
For the most part, your IMU probably reads either just noise or nothing. So it will just always tell you your gyros are zero, because if you are not moving...
EW (11:07):
I guess what I am trying to get to is IMUs could be their own whole book. Actually, I think I have a couple of whole books. Embedded systems could be a book. AI could be two math books and three software books.
EW (11:24):
How are you making a book that has all of this? What is your approach towards helping someone understand what is really happening, but not overloading them, but also not giving examples that are too pat, too simple?
DS (11:48):
Right. So my whole approach was that I only explain enough to be able to make an engineering decision. The difference between engineers and scientists, is scientists, their models have to be correct. Whereas for engineers, we are happy for the model to be only correct under these very specific conditions, as long as it is useful.
DS (12:14):
That was the approach I took was that- As you say, there are plenty of books about general AI theory, and I did not want to reproduce that. There are plenty of good books about embedded systems, such as your own.
DS (12:26):
What I wanted to do was to provide a practical approach to someone who was looking to get into the field. What are the pathways and the key underlying things that you need to know about, in order to be able to design these systems and produce ones that work?
DS (12:45):
The reason I went for a project-based approach is that with theory, it is very easy to just say, "You do blah, blah, blah, and it all works," but if you have got actual-
EW (12:57):
It never all works. <laugh>
DS (12:59):
<laugh> Exactly right. But if you have got hardware, it is actually- You cannot just hand wave it. You have got to know how to build the model, you have got to know how to deploy it, you need to know how to quantify it. So by doing projects, I believe that it is a much more- And for me, it is a better way of learning. That is how I learn best.
EW (13:23):
There are all of these theory books, but having a project based book means you actually put this in practice. It creates a synthesis of theories, different theories, into an actual thing you can accomplish.
EW (13:38):
But one of the difficulties with many of the machine learning methods is that they work 90, maybe even 99% of the time. But that 1% means I never enjoy putting them in actual products, because they make me so frustrated when they do not work.
EW (13:59):
How did you balance the it has got to work all the time, versus it has got to be an actual book length book and not 12 million pages?
DS (14:11):
Yeah, I know exactly where you are coming from. I still do not use Siri to this day, because it is so annoying.
DS (14:16):
I think a good example of this is the Hot Word project. One of the things I was careful to do, is not to just produce the right result at the end and say, "This is what I did and it worked first time," because that is not how it really works. So I was very careful to demonstrate as I went along and to document the mistakes that I made, because I think that is part of the learning process.
DS (14:45):
And so for the Hot Word project, the first time I trained that model, it was 96% accurate on the wake word, which sounds fantastic. But it was also false alarming 55% on everything else.
EW (15:01):
<laugh> Yeahh.
DS (15:01):
<laugh> So you have got to be very careful what you optimize for. I think sometimes I would argue that the false activation is sometimes the lesser evil, because you can maybe filter that out further down the system. Whereas if you get a false rejection, then the device is ignoring you and you are wondering what the hell is happening.
DS (15:32):
So I think you have got to make sure that you are optimizing for the right variable. And I think a lot of projects do not do that. The difference between a hobbyist project and something that you use in production is how it handles the edge cases, I think. That is normally stuff that you need to engineer around the outside, to compensate for the model.
EW (16:01):
Okay. I can see that. That is not how I would have put it, but I can see it. Yeah.
DS (16:05):
How would you have put it?
EW (16:07):
When you are doing something for a project, an amateur project, a personal hobby project, when you get it to work, you are done. It is great. When you are doing something for production or a professional project, when you get it to never fail, you are done.
DS (16:28):
Never fail is a high bar. <laugh>
EW (16:29):
Never fail is too high. I agree. But when you get it so that it does not fail, except in very extreme cases. 96% would not be good enough for me, because that means one out of every-
CW (16:45):
20.
EW (16:45):
20, 30 times, it does not work. And 55% wake up would be... I would feel like it is listening to me all the time. That would be terrible.
DS (16:55):
<laugh>
EW (16:55):
But that lack of reliability just is so irritating, it makes me not use the thing at all. Wake words, because they are the prerequisites to everything that comes after them, if they get missed, they cause a cascade of failure. What I am arguing is I hate AI, so let us just go with that.
CW (17:24):
<laugh>
DS (17:24):
<laugh> Fair enough.
EW (17:28):
Why dance around it? I am just so sick of it. It is not just Christopher being grumpy anymore.
DS (17:36):
You have jumped on board, have you? I am the opposite. I love it. I think it is absolutely fantastic.
EW (17:44):
I think it is great for many things. But I do get tired of the "it almost works," but I can never get the last few bits. It is not even that I spend 90% of my time on the last 5%, it is that I spend 99% of my time on the last 5%, and I am not sure it is possible to get the last bits, the last...
DS (18:14):
The other thing that makes it interesting is it fails in weird ways. One of the things I wanted to do was to create a business card side printed circuit board, which I could use to market the book. If you have seen these, it has got a NFC tag on it, so you tap on the card and it brings up a URL which links to where you can download the book.
DS (18:39):
And to make it a bit more fancy, I put a microprocessor on there as well, because these NFC tags can also harvest power from the field, and you can use that to do a short- I do a short animation of LEDs, which look like a neural network.
EW (18:58):
But you do not get a lot of power from that.
DS (19:00):
Nah, you do not get much at all.
EW (19:01):
Okay.
DS (19:01):
So you have got to be quick, right? But the point is, I thought all the printed circuit boards in the book- There are five custom printed circuit boards that I have got as part of the book. I all did those by hand. And I thought, "I wonder if AI has got to the stage where it could do schematics and printed circuit board layouts."
DS (19:23):
So I decided to give it a test on this project. It did the schematic, and the rookie mistake there is always it selects parts which are not available when you go to manufacture it. So I sent it back and it fixed that up. You look at the schematic and it looks weird. It is not the way that a person would do it, but it looked accurate.
DS (19:45):
So I said, "Okay, well, do the next bit, go onto the printed circuit board layout," and it did that. Again, you know how it does that, it sort of does lots of little lines rather than just doing one smooth line? But nevertheless, it passed the design rule checks for the tool. I sent it off for manufacturing, came back, and it sort of worked.
EW (20:11):
Mm-hmm. Mm-hmm.
DS (20:11):
You could tap it-
CW (20:11):
The story of AI.
EW (20:12):
95%.
DS (20:12):
Yeah. <laugh>
EW (20:14):
Sort of worked.
DS (20:17):
So what did not work was the animation of the LEDs. And then... I am trying to work out what the problem was, and I asked Claude, "Can you recheck your design and make sure that you have done this correctly?" And it comes back and says, "Oh, sorry. I have put the LEDs in backwards."
DS (20:33):
How the hell did I miss that? Because it is a very obvious mistake on the schematic. I go back and have a look at the schematic and it has got them around the right way.
DS (20:44):
I say, "How is this possible? Because in this tool, the manufacturer provides the schematic and the footprint, so there should not be any way to mess it up." And it says, "Oh, I created my own version of the symbol and got it wrong." That is what you got to watch out for, is it can make weird mistakes.
CW (21:09):
You turn it up to their top model and it creates a new semiconductor for you.
DS (21:12):
Right.
EW (21:15):
Okay. How much AI did you use to make the book?
DS (21:21):
Yeah, I used it in two key ways, actually. I did not use it to write or do the code. It was not good enough back when I was writing the book, and I found it was not very good at embedded code. I do not know if you found that as well.
EW (21:36):
It is getting better.
DS (21:38):
Is it? Yeah. The first time I tried it, it was terrible. It created code that would not work.
DS (21:42):
But where I did use it was in two areas. I wrote the manuscript in Word, and then as part of the publishing process, they imported into InDesign to do the layout for the print. And that conversion is not 100%. They mangle the text and the images and things.
DS (22:07):
So I got AI to do a diff between my Word doc and the printed PDF. It was very good at pointing out where there had been changes between the two, so that was very handy.
DS (22:23):
The other area I found it useful was in generating the index. How did you do the index for your book, Elecia?
EW (22:30):
Paid someone.
DS (22:32):
Right. Yeah, so that is one option. But I cannot afford to be giving away my hard-earned royalties, so I thought this might be something that I can automate. I ended up using a three-tiered process to automate it.
DS (22:50):
The first one used AI to go through and summarize what were the key terms within the books, because you cannot just list words in the book, can you, for your index?
DS (23:01):
You have got to try and find, of the 600 occasions where I have used the word "accelerometer," which are the ones that a reader would actually want to find. There is a bit of intelligence around selecting which one of those to use. So I used AI to go through and create a master list of terms for the book.
DS (23:21):
Then I used a Python script actually to go in and add the- There are some hidden word tags that they use for the indexing. So I used a deterministic Python script to do that. And then the last process was I went through manually and fixed it all up.
EW (23:42):
Makes sense. Going back to my question about failures, how do I figure out when to avoid spinning for a long time on something that almost works, but is never actually going to get there?
EW (24:02):
How do I realize when my edge AI needs to be replaced with either a bigger processor, a heuristic, a deterministic algorithm? How do I draw the lines for this should be AI and this should not? But let us start with when this should not be AI.
DS (24:30):
My position on this would be if you can avoid using AI, you do.
EW (24:35):
What is the name of your book again? I just want to write this down.
DS (24:41):
<laugh> Can we take that out?
CW (24:44):
No, I think that is actually a good heuristic. At a minimum, you should not be reinventing things that work with some less... What is the word I want to use here?
EW (24:55):
Efficient?
CW (24:55):
Less transparent. If I write an algorithm, I can look at it. If I write a model to do something that a simple algorithm could do- Let us say you are replacing a PID loop with a deep neural network. Okay, I understand how a PID loop works. I understand how to tune it. I can look at it and figure out what is wrong.
EW (25:14):
There is lots of research on how to do things.
CW (25:16):
Even if the model works, if I want to change how something works, maybe there is no way to do that. Or maybe- You cannot look at the model and say, "Oh, well, this is how this is doing this control system."
EW (25:27):
So transparency should be on the side of do not use AI, for sure.
DS (25:32):
Exactly. So I say AI is not better, but it solves a different class of problem. So if you can use a algorithm which is deterministic, it is testable, runs in not much memory at all, then you definitely should do that. To be fair, half my book uses classical methods. You were saying before about that. So 100% on board with that.
DS (25:54):
So where ML comes in handy is for the problems that classical methods find difficult. You were talking before about the Kalman filter. What tends to happen with those is you tweak those to solve a particular class of problems. Where it runs into issues is if some new thing comes up that you have not trained for, and then the bottle is no good at optimizing for that.
DS (26:27):
So that is where I think that machine learning can add some value. It can be better at generalizing than a carefully tuned algorithm is.
EW (26:40):
Do you have any projects or examples where neural networks really help? Maybe because the environment is changing or something. Projects where the AI is worth it. Where the AI is worth the cost of complexity and transparency, and processing power.
DS (27:01):
Yeah. I think one good example is the comparison between the lookup table on the battery, working out how a battery state of charge changes over time, and comparing that to a machine learning model.
DS (27:22):
What happened with the lookup table is that it is very good for a very specific sense of circumstances on one particular battery. But what happens is- There is no state of charge pin on a battery. Over time what happens is as the voltage changes and the load on the battery changes and the age of the battery as it gets older, all these things have an effect on the state of charge.
DS (27:56):
The machine learning model was much better at being able to understand what was happening with that battery over time, compared to the lookup table.
EW (28:12):
It is funny. I actually thought about that one a while and found that to be a difficult case. Because in order to write the neural network, in order to write the AI piece you recommended, the book suggested I understand how the charge works.
EW (28:34):
Let us see. "To build the lookup table, you need to understand the relationship between the battery voltage and remaining capacity for your specific cell chemistry." Oh, that is to build the lookup table. You are saying I do not need to understand that relationship, in order to build the machine learning version.
DS (28:54):
I think you still need to understand it, to be honest.
EW (28:56):
That is what I did not want. I did not want to understand it.
DS (28:59):
<laugh> Because even with the machine learning version, what ends up being the best way to train it was a hybrid model. Which understands that there are different phases of charging in a battery, and then current does different stuff at different phases of when you are charging and discharging.
DS (29:25):
I used a MOSFET to be able to programmatically discharge the battery over time. But the problem with that is it actually bypassed the Texas Instruments fuel gauge I was using. So the model may not have known that that was- It would not have been able to see that, so I had to build that into a expert knowledge of the system.
DS (29:52):
I think whether you are using neural networks or using a lookup table, you still need to understand what is happening in the underlying system.
CW (30:05):
I think that is a good message, because one of the things I do not like about AI- Or, well, using the term "AI" is fraught.
EW (30:13):
Rampant use of machine learning.
CW (30:15):
Rampant use of whatever, chatbots and things, is this urge to just have it do things without understanding the output.
CW (30:23):
Yes, being able to make the decision between using machine learning on a device and not using machine learning on a device, means understanding your problem in the first place and which approach is actually going to be beneficial.
CW (30:38):
Because there are plenty of problems, as you said, that are better with ML. It is hard to make a bird detector with heuristic methods on a camera.
DS (30:48):
Right. And I think before the AI winter, there was that period where people were trying to go down the expert system route. The problem is you get that scaling issue, where you cannot account for every possibility and it does not scale very well. You cannot put down every rule for every possible way to address the problem.
CW (31:15):
Yeah.
EW (31:17):
Okay. Say I have found something I think AI will help. How do we decide what intelligence will run on what kind of chips? For example, say I have an LLM, will it work on an ATtiny or a Raspberry Pi?
CW (31:35):
Sure, if you only like five words.
EW (31:36):
I know. That is such a terrible question. But really the question is how do we figure out what chip to use for what algorithms?
DS (31:47):
Yeah, it is a very good question. For those following at home, if you could turn to chapter 11 in my book, table 11.1. <laugh> No, look, you cannot squeeze much of a larger language model onto a microcontroller.
EW (32:03):
No, you cannot.
DS (32:04):
So do not do that. An ATtiny, it has got a few kilobytes of static RAM, so you might be able to run a decision tree or a tiny classifier, but you are not going to be running a neural network on there.
DS (32:19):
An M4, an M7, you might be able to do some keyword spotting and some small vision model stuff. A Raspberry Pi, you could probably run a small language model at a few tokens per second.
DS (32:32):
So you would only do that if you had an application that that was useful to do so. I had to answer this question in my book. I wanted to train a GAN that generates melodies, and I had to pick which board I wanted to run that on.
DS (32:55):
That was my sort of thinking was the way to make the decision on this was first of all, does it support the frameworks that I want to use? Because you do not want to choose a chip, and then have to port the framework to an unsupported chip.
DS (33:15):
I think that is how you can tell if you are an engineer, where you spend more time on the tooling than you do on the project. So that was the first decision that I used.
EW (33:28):
Wait. Wait. The tooling here is like Edge Impulse or TensorFlow Lite.
DS (33:34):
So this particular one, this was the GAN, this was TensorFlow Lite.
EW (33:43):
Okay. But you said "framework." I just want to make sure we understand what the word "framework" means here.
DS (33:48):
Yeah, that is what I am referring to.
EW (33:48):
Okay. Cool. Cool.
DS (33:52):
The next thing was the memory. So this is a hard constraint that you cannot get past. It is all very well creating a fantastic model, but if you cannot fit it in the memory of your microcontroller, then that is no good. The quantized generator that this project created was about a megabyte. That meant that I had to use something like a Pico or a Pico 2. So that was the next one.
DS (34:28):
The third thing that I use to make a selection is the amount of effort that you have to go to, versus how much control you need. So if you are using an existing framework like TensorFlow Lite or Edge Impulse, that makes things easier to implement. But you do not have a lot of... You have got to use what they provide as part of that framework.
DS (34:50):
Whereas if you write the C code yourself, you can obviously do anything you want, but you have to do it all.
EW (34:57):
Fair amount of effort. Effort versus control. That is a good balance. Yeah.
EW (35:02):
Looking at your chapter 11, table 11.1, it was exactly what I was asking for. There are Arduino type processors, including the nRF processors that are on Arduino boards. It talks about the type of the processor, whether it is an M0 or an M4F or M7, blah, blah, blah. Dual core, and the amount of RAM.
EW (35:30):
And which ones support TensorFlow Lite, versus Edge Impulse, versus MicroPython. You have a column in here for C and C++, and every single one of them is checked. I am glad to see that. I would be kind of sad if it was not. I do not see the ATtiny in here, but that is okay.
DS (35:48):
Okay. I do like the ATtiny. I used that on my business card that did not work.
EW (35:53):
Well, that would be low power, fast boot. Yeah, it has got everything you needed there.
DS (35:59):
Exactly.
EW (36:02):
Your methodology of, okay, you choose how you are going to implement this. How you are going to implement it, whether it is using a framework like TensorFlow Lite. Or MicroPython even can be a framework. If your processor does not support that, maybe you are not using the right processor.
EW (36:20):
And then the second point of memory, yes. I think that is something we are all going to have to get used to, is that the size of our models drives so much. And so you can do a lot of your testing offline, because if you cannot get a model to fit into your microprocessor's memory, gig is up. We are not doing this.
CW (36:44):
The good news is memory is so cheap now.
EW (36:48):
I do not think that is- No. No. Oh, oh. Oh, I did not see the sarcasm light. Oh, you should have rolled up your sleeve so I could see that.
CW (36:58):
It is always on.
EW (36:58):
It is always on.
DS (37:00):
How do you feel about MicroPython?
CW (37:03):
Oh, it is cool.
EW (37:05):
It is in many ways truly wonderful. And then every once in a while I realize, oh, I could use a much smaller processor.
CW (37:15):
Right. Yeah.
DS (37:18):
It sort of fits in no man's land, does it not?
EW (37:22):
I used it recently in a project that involved scientists being able to repurpose my device to their own requirements, monitoring environmental conditions. They would be able to write scripts that would go on top of the hardware I provided. It was wonderful.
EW (37:44):
At first we had some of the scientists writing in C. I quickly realized how badly that was going to go. And yet once we got the Python working and all they had to do was type a few lines, it got much simpler.
CW (38:01):
It seems to be really good for devices or products where you want the end user, without changing the firmware, to change something about how the board operates, or script the board.
CW (38:12):
A few things I have seen it in. I have seen your device. The Jumperless that I am playing around with, we had the show about, has a Raspberry Pi 20 whatever, the Pico as its controller. You can go in and query voltages and things of all the test points and things and do some stuff programmatically.
CW (38:33):
And I have seen it in music devices as well, where maybe you have a synthesizer module with MicroPython in it, and people can go in and do custom stuff. I do not think it is a great fit for, "Here is this end product, and this is the firmware."
EW (38:49):
But it is good for people who are going to muck with it anyway, and you want to give them a good way to muck with it.
CW (38:54):
Right. Or if you do not want to write- If you have something complicated that needs a lot of interaction, but you do not want to write a giant CLI yourself.
EW (39:06):
It is a good CLI, yes. It is a really good CLI. To some extent, I wish we had started another project with that as the base, even though it was way more complicated than it needed to be, because we end up with a CLI that is really complicated. But what if it was Python? That would be so much easier.
EW (39:26):
Okay, let us see. I have so many questions, I do not even know where to go from here. Oh. Do you talk about machine learning cores? Or the new controllers that have the AI, ML-
CW (39:42):
Neural processing bits.
EW (39:45):
Neural... What are essentially tiny GPUs stuck on them, like little horns on top of horses.
CW (39:52):
What?
DS (39:52):
Yeah, the wake- <laugh>
EW (39:55):
Unicorn.
CW (39:55):
Oh. Okay.
DS (40:02):
<laugh> Sorry. The wake word model uses the Arduino Nicla Voice, which has got a sentient chip on it, which is good. The problem is- I do not know if you find this, but a lot of the example code that you get is very fragile and brittle. So I had to do a lot of work to get it to work as advertised. But, yeah, I think that is a great model going forward.
DS (40:39):
I also did some work- I think you were talking about your dodgy robot arm in one of the episodes.
EW (40:52):
Yes. You have a robot arm.
DS (40:54):
I do, yeah. It is also dodgy. But what it does is it uses the ST IMU, which allows you to do anomaly detection on the arm. I think that was very useful.
EW (41:15):
Let us see. Your book is very project-based. Which came first, the projects or the learning objectives?
DS (41:24):
Definitely the objectives. I started with what I wanted to cover within the book, and then I looked for projects that would help me explain the best way to approach those sort of problems.
EW (41:36):
And then when you consider those sorts of problems, you had to- Because your book covers so much material, you have to go from describing what is linear interpolation, y=mx+b, and how that works, to stochastic descent, and statistical methods, expectations. How do you...
CW (42:03):
This is the problem of all these kinds of books.
EW (42:06):
It is. How did you decide what your reader already knew, versus what they should look up, versus what you should teach them?
DS (42:18):
Honestly, the way I selected what should be in the book was the stuff that I was interested in. Is that the best curriculum possible? I do not know. This was the journey that I myself went on when I was learning about this.
DS (42:36):
It was the book that I wanted to have available, and I wish was available. Because what I find is that I enjoy going deep on different subjects, and I felt that a lot of other books covered things superficially.
DS (43:01):
What I tried to do was go just deep enough to allow you to make an engineering decision of whether you should use A or B, but no further than that. This book was way bigger than I initially imagined, and that is even after chopping out six chapters. So yeah, I could have kept going for sure.
EW (43:29):
Yes, because there were so many other things you can do. Let us see. I am in chapter 11, building an AI MIDI synthesizer. Everything from the adversarial networks, to building a piano keyboard, to a synthesizer, to USB MIDI, procedural composition to the MIDI controller. And that is one small section of your book. It is really quite a lot!
DS (44:05):
<laugh> It is a lot. Yeah. I guess the problem is for me, I get interested in diverse subjects. I suddenly, I was wondering, how do capacitive keyboards work? So I went off down that rabbit hole.
EW (44:23):
Boy, that is a rabbit hole. I think every time I end up there, even having programmed for them, developed with them, worked with them in the year 2000, I still think maybe the answer is just magic.
DS (44:41):
And look, the software that you can do now on things like the Pico is amazing. The PIO and the DMA stuff, it is magical the way that you can offload all this sort of processing off to little other bits of the controller.
EW (45:03):
You mentioned writing a drone book, and then being too late to the industry. Were you worried about that for this book? Because AI, and microprocessors, change pretty quickly.
DS (45:16):
Yes. Yes. Yes, I was. I was not so worried about missing AI, if you like, because I think this is a-
EW (45:29):
It is never going to go away. Ohh.
DS (45:32):
No. I think it is a major pivot point, I think, in human civilization. I think there have been five or six of these major pivot points, and AI is one. I was not concerned about missing that.
DS (45:45):
I was concerned about- As you say, microprocessors change. That was part of the reason that I selected the processors that I did. Those hobbyist type controls, the Arduino stuff, have been around forever. They tend to change at a slower pace than if you were using development boards, which change every six months or so.
DS (46:10):
But having said- Even with that, Qualcomm took over Arduino and Edge Impulse in the time that I was writing the book, which is probably arguably not a good thing. So the way I tried to address it was to stick to principles as much as I could.
DS (46:31):
There is a lot of stuff in there which is quite old, as you say, the Kalman filter and the Mahony filter and all the sensor fusion stuff, that is not new knowledge. But what I found was it is hard to find books that explain that stuff in detail, which is why I included it.
DS (46:52):
And the other thing that I have done is that as stuff changes, I have been releasing bonus content that addresses anything specifically. One example is for the battery monitoring project.
DS (47:09):
I had two shields. One is a logging and display shield, and the other one does the actual charging and discharging the batteries. The chips on there use I2C as the bus. They worked fine, the two shields by themselves, but when I put them together, I was occasionally getting issues with them. You can probably guess where the problem was.
EW (47:40):
Pull-ups.
DS (47:41):
Yeah, exactly. So I had to create a new version of the board, and so that is come out as a sort of bonus content.
EW (47:53):
Do you have any advice for people thinking about writing a technical book?
DS (47:59):
I think you need to have a good reason to be writing a technical book, and it definitely should not be making money.
EW (48:07):
<laugh>
DS (48:07):
I do think it is a great way to learn about a subject. I am convinced that if you want to learn a subject, try and teach it to somebody else.
EW (48:23):
True. So true.
DS (48:25):
So I think from that perspective, it is great. And Elecia, I think you were talking about an example of where you were down at Apple or something, and they brought your book out. That is the other area where it is useful.
EW (48:39):
The credibility check.
DS (48:40):
Correct. Yeah. Yeah. Reputationally, being able to say that you have written a book, and have got the grit to be able to go through that very painful process. But I think you want to be very certain that you are willing to put in the time and to go through it, because it is harder than it looks. I found it harder than I expected.
EW (49:04):
Yes, it is definitely harder than it looks. And having a quality editor and a quality publisher is really important. Writing a book on its own is usually not sufficient. Did you have a lot of interaction with No Starch? I know they are reasonably hands-on.
DS (49:26):
Yeah, that was great. That is part of the reason the book- I had to rewrite it five times, was originally the way I wrote it was more like a series of lab notes of, "I did this. Then I did this."
DS (49:41):
My developmental editor said, "I think it would be better-" Sierra Fong was her name. "It would be better if we did it as a series of projects, that people could build along themselves." I think she was a hundred percent correct, but it did mean I had to go back and rewrite it all. But it made for a much clearer process.
DS (50:03):
The other thing was, as you say, I cover a lot of ground in the book. She was very good at being able to help me package that up into understandable subsections.
EW (50:20):
Which, given the breadth of topics you are covering, you really needed the subsections. They were very important to being able to say, "Okay, I want to know about this area," and, "I am going to skip the other parts." Do you have a favorite project from the book?
DS (50:42):
I think my favorite would be... I think it is probably the MIDI controller, actually, with the GAN music generator. Just because it- I love the fact that it was using two cores of the Pico, and that one of them was running the spectrograph, and the other one was running the RNN. Just learning how the PDM microphone goes in there. That was the one I got a lot of joy out of.
EW (51:16):
No Starch has asked me to provide a quote for your book. What should I say?
CW (51:20):
<laugh>
DS (51:26):
The best quote in your book, I think, is the one where you say that every sensor is a temperature sensor, that strongly resonated with me.
CW (51:37):
<laugh>
EW (51:39):
Some just measure other things as well. I do not think that is going to be it.
EW (51:45):
But I have enjoyed your book quite a bit, and I do appreciate that all of this information went into one place. I have notes about maybe 30, 40% of it, but I do not have a place I can point other people.
EW (52:03):
So I am looking forward to saying, "You want to try that? Go ahead. Here is a good reference." "Oh, you are using an IMU for the first time? Wow, do we have an adventure in store for you?"
DS (52:17):
What do you think is missing from the book? What would you have put in there?
EW (52:21):
I think what I would want was the impossible things that I started the show with, which is I do not trust AI. It is not reliable. It is too complex. It is not transparent. It is impossible to debug. It is impossible to test in all the cases that it needs to be tested in.
EW (52:40):
So what I would want in your book is, "Here is how to QA an AI system." But also, "Here is how to figure out if you can QA it, before you put all the effort in." That is just a problem with AI in general. That is not your book. Although if you say you have that and I just have not gotten to it, I am going to be so embarrassed.
DS (53:08):
No, no, I think that is going to be the next book. Because I agree with you. I think that the problem a lot of people are making at the moment, is that they are trying to build safety into large language models. Because it is not deterministic and it is all statistics at the end of the day, it just cannot be done, can it?
DS (53:31):
So I would argue that we need to go to a different sort of model, where the safety is built in lower down. Same way that the evolution has worked it out. We have reflexes that do not go to the neocortex until after they have already fired. So I think that is the sort of model that we need to move to, in terms of robotics and self-driving cars and things like that.
EW (53:58):
It is funny you brought up MicroPython. To some extent, one of the reasons I really liked it was because I could build in all of those safety in the C, so they could not get to anything that was going to blow up my robot. That was super helpful.
EW (54:12):
Your neocortex versus- I guess, amygdala or cortex or I do not know, however many brain parts you want. It does make sense to separate it into, "Here are the safety critical pieces." And if you hear the screech of the cat's claws on a rock, which sounds a lot like the screech of a blackboard, yes, your brain is going to go, "Wow. We need to do things."
CW (54:41):
That is a good principle even without AI. There are plenty of embedded systems I have worked on, where we put the safety systems in hardware. "Oh, the laser is firing? Well, there is a hardware integrator that measures how much energy is put in, and if it goes too much, it shuts it off."
CW (54:54):
Instead of the software, "Oh, I am looping around sampling. It seems all right. Oops, I got distracted and the laser was on too long."
EW (55:03):
I love that firmware engineers are the people who least trust the firmware.
DS (55:08):
Trust them, they know what happens. Well, you had recently the guy who was building the robot dogs.
EW (55:14):
Yes. Aaed.
DS (55:16):
Yeah, he was talking about how originally he was hard coding in the gaits, and now he is trying to use neural networks. I was thinking what he needs is a cerebellum, which is how the brain does it at the moment.
DS (55:29):
So when you walk or you jog or you skip or even chew, that is all done by the cerebellum. You know when you drive home sometimes and you cannot remember how you got there? That is because the cerebellum has taken over.
DS (55:45):
I think that is the model that we need to go. You need to have this co-processor, where the brain says, "Walk," and then this other bit handles how it actually does the walking.
EW (55:55):
I think David's next book is going to be "Neurobiology, and how it should be applied to software."
DS (56:02):
100%.
EW (56:04):
In a second, I am going to ask you if you have any thoughts you would like to leave us with. But first, we have a giveaway. No Starch is providing two print copies of "Embedded AI" for listeners in US or Canada. I guess-
CW (56:21):
Which is ironic.
EW (56:23):
Why?
CW (56:24):
Because he is in Australia.
EW (56:26):
Okay. That is kind of ironic. I did not get it.
DS (56:29):
Am I the first Australian to be in the show?
CW (56:30):
No.
EW (56:31):
No. No, no. Sorry. We have had several.
EW (56:34):
Let us see. How shall we select winners? I think, send me a project you are going to do with AI. Just an idea.
CW (56:48):
ML.
EW (56:48):
ML. ML. ML.
CW (56:48):
Do not send us a project you are going to tell Claude to do.
EW (56:51):
No. We want a, "I want to use a microprocessor to build myself a synthesizer, that reacts to my cat meow." That sort of thing. And feel free to make it as silly and ridiculous as you want, because really they all just get put into a Excel sheet, and then I shuffle and choose a number. So mostly you are amusing me with your emails.
EW (57:11):
Show@embedded.fm, and that will be until the end of October. So you have a couple weeks to do it, but do it soon.
EW (57:22):
No Starch has also created a 25% off coupon for "Embedded AI." It is embedded 25. Wow, that is not going to be hard to remember. Embedded25@nostarch.com. That is valid until November 5th, so use it soon.
EW (57:41):
Now, David, do you have any thoughts you would like to leave us with?
DS (57:47):
I do have one, and it is a bit of a cliche, I am afraid, but I do think you got to keep learning. I have had more than one career, and people coming through now will have three or four. Things are changing and the pace of change is accelerating. The half-life of anything that you learn is getting much shorter.
DS (58:09):
They used to say that half of what you learned at uni would be useless in five to ten years, and I think that has only got much faster. So I think if there is one thing that you want to learn, it is how to learn.
EW (58:24):
Indeed. Our guest has been David Such, executive director of Reefwing Software and author of "Embedded AI: Intelligence at the Deep Edge," published by No Starch. You can find it wherever you get your technical books from. And you can use that coupon if you are buying it at nostarch.com.
CW (58:44):
Thanks, David. This was fun.
DS (58:47):
Thanks for having me. Really appreciate it.
EW (58:49):
Thank you to Christopher for producing and co-hosting. Thank you to Mark Omo and Scott in our Patreon listener Slack group for some of their questions. And of course, thank you for listening.
EW (58:59):
You can always contact us at show@embedded.fm or hit the contact link on embedded.fm. Which is how you are going to be entering that contest, email or hitting the contact link.
EW (59:10):
And now a quote to leave you with. This was in David's book, but it did not originate with him. It actually originated with George Box, who is a statistician. "All models are wrong, but some are useful."