391: The Lesser of Two Weevils
Transcript from 391: The Lesser of Two Weevils with Elecia White and Christopher White.
EW (00:06):
Welcome to Embedded. I am Elecia White, and this week it's just going to be Christopher and I talking with each other.
CW (00:14):
Hello.
EW (00:17):
How are you doing?
CW (00:17):
I'm okay.
EW (00:19):
That means you're not okay. I've heard that phrase before. Do...you want me to ask, or should I just - ?
CW (00:25):
I don't think we want to discuss my mental state on the podcast. It's not that kind of show.
EW (00:32):
Okay. Do you want some candy?
CW (00:35):
I don't think that would help.
EW (00:37):
Okay.
CW (00:37):
Also, I had some. It didn't help. I mean, unless you count really good chocolate as not candy. Is it a different category of thing? Is it one of those nutraceuticals things?
EW (00:49):
Yes.
CW (00:49):
Okay.
EW (00:49):
The chocolate I buy for us is definitely nutraceutical.
CW (00:52):
Okay. Well, I had a nutraceutical, 70% dark mint chocolate, and it helped for a minute.
EW (01:00):
Well, secretly inside I feel exactly the same way, but I have so many things going on that I can't.
CW (01:10):
That's because you keep signing up for things. A little dialogue box comes up and says, "Would you like to do this?" And you hit the checkbox, and you check it. And they just keep coming up and you check them.
EW (01:20):
I don't. I don't mean to. I think there may be something wrong with my mouse.
CW (01:25):
Oh.
EW (01:27):
You're not going to ask what I have going on in order to help me?
CW (01:29):
Oh. Oh, I'm sorry. What's going on? Why are you so busy all the time? Why are you working all the time? Why are you so overburdened?
EW (01:39):
Okay. So I have my usual clients, deep sea underwater science platforms -
CW (01:45):
Yeah.
EW (01:45):
- and large autonomous trucks.
CW (01:48):
Yeah. I mean large autonomous trucks, the size doesn't really matter.
EW (01:52):
Well, it kind of indicates what happens when things go bad.
CW (01:56):
Yes. Sure. But the software and everything would run on a radio-controlled car. So I just think it's unnecessary. Anyway, go on.
EW (02:07):
And then last spring, a company called Classpert said they were getting into the MOOC audience.
CW (02:16):
Last spring?
EW (02:19):
It was late spring.
CW (02:20):
Was it?
EW (02:20):
Yeah.
CW (02:21):
I feel like, okay. Well.
EW (02:23):
We didn't actually sign anything until July.
CW (02:25):
Right, right.
EW (02:25):
So I only started then.
CW (02:29):
Yup.
EW (02:29):
Did I want to do a class based on my book? And we went back and forth about what they would do, which is video editing, and some educational assistance, and setting it all up, and what I would do, which is create a dozen lectures. And I was still kind of happy with my memory map talk.
CW (02:54):
That was a mistake.
EW (02:56):
Yeah. That was the problem. And so I said yes. And now the class is open for waitlist, and it's going to start on November 13th and go through the holidays to February. And that's going to be the alpha cohort. And then there will be another group that starts in February.
CW (03:23):
And how big is the alpha cohort?
EW (03:25):
50 people.
CW (03:26):
50 people. And November 13th, this is the real day. This is it. I'm not going back and editing in you saying, "November 27th."
EW (03:36):
I hope it's the real date. I really do.
CW (03:39):
Okay. Well,...in case it's not, why don't you say every number from 1 to 30, 1 to 31. Well, I can always do 31. So 1 to 30. And then if I need the clip in a day that's different, actually, you better do the months too.
EW (03:59):
It has slipped a bit.
CW (03:59):
Just sing the months? Do the month song. [Singing the months.]January, February, March, April, May, June, July, August, and September, remember that? No, I skipped some, but remember that December is the end. And when I was in first grade, when we sang that song, and "remember that December is the end," I got very upset.
EW (04:16):
You were afraid that that was the "end" end?
CW (04:18):
The end.
EW (04:19):
The end.
CW (04:19):
That was the end of the world. December was it. And this is the song telling me, "Remember, December is the end."
EW (04:29):
Yeah. Okay. So the class will run a couple times. Maybe it will run indefinitely. It depends on the response. So far the response has been good. Definitely we're doing the two, and I should have coupon codes before you have to pay for it.
CW (04:46):
Okay.
EW (04:47):
But right now, at this moment, I don't know how much it costs, or what my coupon codes will be worth.
CW (04:53):
Definitely the coupon code will not be worth more than a class.
EW (04:57):
No. There will be scholarships in the future, but don't ask me about them for the alpha cohort.
CW (05:05):
So have we talked at all about how the class works? If somebody signs up for this class, what are they going to do every week? And how are they going to interact with the instructor, presumably you? And is there homework?
EW (05:22):
Okay. So every week there'll be at least one lecture. There are a couple weeks where there are optional lectures, and those are about an hour with the quizzes and all the other stuff that goes with them. And then there will be a synchronous class, and I don't know what time it will be. It will probably be on Saturday.
EW (05:47):
And...there'll be two slots of synchronous classes, so that the mentors and I can talk to people kind of individually, but also leading discussions. So there'll be the synchronous element. There'll be a Discord channel. There will be homework, and there will be reading.
EW (06:11):
And really, if you don't do the reading, I have no sympathy for you. So, there's that. And then there'll be a final project that is, right now, very loosely defined.
EW (06:26):
I went through a lot of back and forth on it, whether or not to do mbed boards, which really aren't that great, but the compiler's easy, and then I don't have to fight with it, or to do ST Discovery boards, which are pretty cool, and can compile for everything, but can be intimidating to some.
EW (06:47):
I don't want people to use Arduino. Definitely want them to use Cortex-M. Anyway.
CW (06:53):
Okay. Well then the other challenge is getting boards for, I don't know how available eval boards are? Probably more available than getting hundreds of chips for a product, but -
EW (07:07):
I have a half dozen or more Discovery boards that are in stock at Digi-Key. And I am basically going to show them all the boards they can use, which is anything that is a Cortex-M0 base that isn't an Arduino interface.
CW (07:24):
Okay.
EW (07:26):
And then what are they going to build? They're going to get a lot of freedom in that. There are some requirements about buttons, and interrupts, and state machines, and peripherals. But overall, they get to choose what they're going to build, including whatever they're working on for work.
EW (07:39):
It is intended to be a very professional course, and to get a portfolio-worthy project. So that's the projects, the homework, the reading. Oh, there are mentors, some of whom have been guests on the show.
EW (07:55):
So that will be interesting. Maybe. I actually don't know who the mentors are yet. But there will be mentors, and then there'll be the lectures and the quizzes.
CW (08:07):
Okay. And...are you going to go into more depth than some of the things that were in the book, or do you just have time to cover what's in the book?
EW (08:18):
I am assuming that the students have read the chapter.
CW (08:22):
Oh, okay.
EW (08:22):
And if they haven't, then what I'm about to do is going to just be confusing as heck.
CW (08:28):
So you're building on it.
EW (08:28):
Yeah. For the state machine chapter, in the book, I go through green light, red light, stoplight, sort of thing, and flow charts, and building up state tables, and some RTOS and scheduling stuff.
EW (08:46):
And in the lecture I take an actual device, and I create a state machine based on that device while I play with it in front of the camera, and do this picture-in-a-picture thing. And we can build up a real state machine, not a simple one, and talk about how all that works, and then do the table version, and -
CW (09:06):
Oh, nice. Okay.
EW (09:07):
And then let's see. What's another one? For the input/output timer chapter, output/input timer, I don't know.
EW (09:18):
For that chapter, instead of talking about all the things in the book, I actually go through a HAL, where...it starts at the very top level of PWM out 25% sort of instruction and goes through the embed library, the target's library, the STM library, the HAL for the STM, and then the registers.
EW (09:47):
And by the end I'm just exhausted, because it was so hard to figure all that out. So stuff like that. It's not what's in the book. It's some of what I wish was in the book. When we talked to Tyler last week, a lot of the stuff about IoT management is something I want to add to the book.
EW (10:06):
So the chapter about updating firmware, it gets some of that. It gets a lot more security information. But I don't really talk about the basics of "what is a bootloader," because I assume you've read the chapter.
CW (10:18):
Okay.
EW (10:20):
So I've started recording the videos. There's a commercial. It's terrifying. And I have been working on the lectures for a while. I haven't done all of them. And then it goes from my slides, and voice-over, and some talking head, and demos, and whatever, screen shares, to the Classpert folks who do the video editing.
EW (10:49):
And we'll do a little bit of animation and make my slides prettier. They've been pretty good. And I did get that fantastic octopus playing with LEDs on a board...as my icon, which is just hilarious.
CW (11:10):
I think it'll be good.
EW (11:12):
Yeah. I hope so.
CW (11:14):
And after you go through one, you learn a few things, and then, that's why you're doing a small group first, right?
EW (11:21):
Yeah. Oh, yeah. And the alpha people, I had to give the cohorts names, and I chose them in rainbow order. And then I wanted sea creatures. So the first cohort are Red Jellies, because they're going to have to be very flexible. I've never written quizzes before, and so my quiz questions may be trivially easy or completely impossible.
CW (11:50):
That reminds me of every quiz that I've ever taken, sometimes from question to question.
EW (11:54):
It's hard to design those things.
CW (11:56):
It's really hard to make tests. Do you remember how often, you probably don't, but there were tests in school, in college, and in graduate school where the professor just completely miscalibrated.
EW (12:09):
Yeah.
CW (12:09):
And...I remember somewhere it was like, "You should be done in an hour," and at an hour and a half everybody, and this is a class of smart people, is still there trying to finish. And the professor's, "Just go home. Take it home. Just take it home. Bring it back tomorrow. Finish it overnight." ...It's really hard.
EW (12:31):
Yeah.
CW (12:31):
Because...once you get to an advanced enough level, it's not just, "Here's your multiplication table," right? It's stuff that you're asking professional-level questions about that are things that you understand, and they just heard about it.
EW (12:48):
And I try to keep that, "What does a beginner need to know," mindset.
CW (12:54):
Yeah.
EW (12:54):
But with this I have been doing so much. The lecture preparation gives me one aspect of the beginner mindset. And then the quizzes are a beginner mindset I've never really had to deal with.
CW (13:09):
Yeah. And you can't really put yourself in the beginner mindset. You can try, but -
EW (13:13):
No. Not anymore. Not on this. There is one thing that I've been having trouble with trying to figure out how to teach, so I kind of wanted to talk about that. Do you want to talk about other stuff first?
CW (13:27):
No, no. We're on the topic.
EW (13:29):
Okay. So...actually, this is related to Hackaday. I'm keynoting Remoticon. It's free. It's on Friday.
CW (13:38):
Friday the what?
EW (13:38):
Friday the 19th of November, 202 -
CW (13:43):
1.
EW (13:46):
- 1.
CW (13:46):
But you can just say all the numbers, and I can adjust the year -
EW (13:48):
Right. Right, right. And actually Jeremy Fielding will be keynoting the other day. It's the first time I've ever keynoted something. I'm very excited. I don't know that I have to do anything different other than show up and talk. I guess I get a key and a note, or I have to provide a key and a note?
CW (14:04):
I think you have to use Keynote from Apple. They don't let you use PowerPoint. That's my understanding. Otherwise it's called a PowerPoint.
EW (14:11):
Nice.
CW (14:12):
I don't know...I think they might require special visual effects or something, right? I don't know how this works. Do you want to do it in front of a green screen and have -
EW (14:22):
It's live. I'm giving it live.
CW (14:23):
- wild animals. You can still do it. With OBS they can do it.
EW (14:27):
Yeah. I'm learning to use OBS, the video software.
CW (14:30):
So am I.
EW (14:30):
Yeah. We're kind of leapfrogging. I don't understand this, and then he'll come help me, and then I'll use it for an hour. And then -
CW (14:36):
Yeah, you're way ahead of me now.
EW (14:38):
And then now I'm ahead, and then he'll be ahead. You're using it for -
CW (14:43):
Music stuff. I don't know yet. Mostly, well, this is not interesting. Mostly I started because I am practicing a lot, and there's a lot of things with guitar or piano.
CW (14:58):
...Usually you have a teacher of course. But if you don't have a teacher, and you're somewhat advanced, but you're still working on technique issues, you can at least look at your hands if you're playing guitar. And you can look at your hands when you're playing piano and see if your wrist's in the right position, if you're doing the right things.
CW (15:14):
With drums it's a little harder. I mean, there's a lot of stuff happening with hands that you can watch, but since it's the whole body, there's posture issues, there's arm issues, there's leg issues.
CW (15:25):
There's all kinds of things that you cannot see unless you record yourself and play it back or have a teacher yell at you, and I don't have one right now. So I thought, "Okay, I've got a bunch of stupid old cameras and things, and let's fire up OBS. Can you record multiple cameras at once?"
EW (15:41):
Yes.
CW (15:42):
"Yes, it can. Okay. That's very cool." So I did that, and set it up. And then I went a little overboard, and...I got a MIDI plugin for it. So I can use a MIDI pad on my drums to change the scene -
EW (15:55):
That's pretty cool.
CW (15:55):
- and to switch cameras. So I can record myself and say, "Okay, this section, I'm working on more foot stuff." So I can just hit the thing I want, and it switches camera in the recording.
CW (16:04):
And then you play it back, and you look at it, and you go, "Wow. I hate everything about myself and want to burn the instrument and the cameras so that this will never happen again."
EW (16:15):
Yeah. Yeah.
CW (16:16):
So anyway, that's what was weird -
EW (16:17):
See note about commercial.
CW (16:19):
I have some vague notion that maybe I'd want to livestream some stuff at some point, not for any reason other than to make me really, really nervous. Because I haven't been really, really nervous playing music since playing live, which I haven't done in a long time. So it would be useful exercise...But yeah, it's super easy to use.
EW (16:42):
For a free program. It sure has a lot of features.
CW (16:44):
Yeah. I was worried it was going to be Blender level of complexity, but I was able to get basic stuff working without even reading anything about it. It's like, "Oh, okay. Add this? Sure. Fine. Okay."
EW (16:57):
I was able to get basic stuff working, and then I broke it.
CW (17:00):
Well, yes. I broke some stuff too.
EW (17:01):
And then working, and then sort of working, and I'll be fine. Okay. So -
CW (17:07):
But you're using it for the class.
EW (17:08):
For the class...So back to the keynote for Hackaday, it will be in part based on my memory map talk.
CW (17:19):
Right, I don't think we mentioned that at all.
EW (17:22):
But I didn't want it to be the same memory map talk.
CW (17:24):
Yeah.
EW (17:24):
Because people have heard that before.
CW (17:26):
Yeah.
EW (17:26):
So one of the things that Mike Szczys asked about was heap memory, and stack memory, and figuring out when you're out of memory. And I didn't do a very good job of explaining heaps and stacks. I seem to recall, especially the GCC section, I did a particularly poor job of heaps and stacks on that.
EW (17:49):
And so I wanted to make a graphic that explained heaps and stacks. And there were a couple of things I thought of.
EW (17:57):
There are armies approaching each other, each going forward and backwards like you do for frees, and stack pops, and possibly clashing, and they get little people fighting, or Lady and The Tramp style spaghetti, or, and hear me out here, stalactites and stalagmites.
EW (18:20):
Stalactites are on the top, stalagmites are on the bottom, and they crash in the middle, which actually doesn't sound very good at all.
CW (18:27):
No. You could combine the army thing and have them fighting up and down a tower, a wizard's tower, climbing the circular staircase, and fighting each other.
EW (18:37):
And I've been trying to think about -
CW (18:38):
Just skip right over that.
EW (18:39):
No. I mean, yes the tower -
CW (18:44):
I have heard your note.
EW (18:47):
I've heard your note. But I was trying to figure out both the imagery to use, the metaphor, but how does show what they do.
CW (18:58):
Yeah.
EW (18:58):
And so I have a heap pointer after all of the pre-allocated globals, and statics, and BSS, and data. And then I have the heap pointer, and I have a minimum heap size. And then I have the stack pointer, which is at the other end of memory, and I have a minimum stack size.
EW (19:19):
And so every time I allocate something with malloc, I use the heap, and every time I call a function, I use the stack both...for the whole stack frame, for the return function, for the -
CW (19:32):
Locals.
EW (19:32):
- registers, for the locals. There's a lot of stuff that goes on the stack. And so the stack builds up, and it goes down, and it builds up as you go deeper into the functions, and then it frees itself. That's not quite the right word. It jumps back to a previous function.
EW (19:47):
And the heap goes up, and then you free memory. And ideally it goes back down, but sometimes it fragments, so it just keeps going up. And then -
CW (19:56):
Well, now you have to talk about malloc.
EW (19:58):
Well, I mean, I just want to say malloc. I don't want to talk about it.
CW (20:02):
If you said fragmentation, you've got to talk about malloc.
EW (20:06):
So the heap continues to go down, and the stack continues to go up. And in a bad situation, they crash. And I wanted to create such a bad situation. The only thing I could come up with, maybe this part will help with the whole metaphor, is a recursive function that has a malloc.
CW (20:27):
That would work. I mean, I haven't seen crash that much. I mean, usually don't you set a maximum stack size, and then the runtime tells you, "No more," when you go over?
EW (20:40):
No.
CW (20:41):
That's how I've used RTOSs.
EW (20:42):
Well, I mean, if you have an RTOS, yes.
CW (20:44):
Okay. Bare metal.
EW (20:46):
If you don't, there's nothing checking -
CW (20:48):
Yeah, okay.
EW (20:48):
- that you're going beyond the heap.
CW (20:49):
That's fair.
EW (20:49):
There's nothing checking you're going -
CW (20:51):
Yeah.
EW (20:51):
- beyond the stack.
CW (20:53):
Well, and that's an important thing to talk about too.
EW (20:56):
Yeah. I mean, that's why you end up in this -
CW (20:57):
Yeah, okay.
EW (20:57):
And this is one of the reasons we don't always use heaps...Embedded systems use static memory, and then everything after the pre-allocated variables is stack space.
CW (21:11):
Yeah.
EW (21:11):
And you can, although...I've never done it, or I haven't done it in many years, calculate how many stack frames you can put on, how many functions you can call, not counting the local variables. And then you say, "I can only call a hundred functions deep," which is a lot.
CW (21:34):
Not if you're using a UI library.
EW (21:34):
And a HAL. Don't forget the HAL.
CW (21:36):
Yeah. Yeah, the HAL is a real problem.
EW (21:39):
Or even, you can call 20 functions deep. And that doesn't sound like much until you get to the HALs and the UI libraries.
CW (21:49):
Yeah.
EW (21:49):
And then you end up bumping up against some variable that you have, as opposed to the heap, where you don't quite know where you'd bump up against it.
CW (21:58):
Bumping up,...this is a light word for destroying.
EW (22:04):
Crashing into something.
CW (22:05):
Yeah. Yeah.
EW (22:06):
Well, I mean, yeah, I guess that's why the Lady and the Tramp spaghetti thing doesn't work, because at the end they smooch. And let's face it, the heap and the stack are more like fighting than smooching.
CW (22:18):
We've never asked them.
EW (22:19):
I'm pretty sure.
CW (22:21):
Okay.
EW (22:21):
Pretty sure. Just a guess. Okay. So are there other metaphors, preferably oceanic, deep sea creature metaphors?
CW (22:29):
I mean, there's a lot of gross sea creatures. We can probably come up with something. Evolving a sea cucumber, or a -
EW (22:33):
Are we going to go back to...Ze Frank's video with the nudibranch eating the nudibranch?
CW (22:40):
...Yeah, I don't have anything offhand. Sorry.
EW (22:48):
Molds. Fungus.
CW (22:49):
Worms. Undersea worms.
EW (22:52):
Sponges and corals.
CW (22:53):
Yeah, that doesn't really work. Yeah.
EW (22:55):
Because you have to kill them off, because they have to die back.
CW (22:57):
I mean, you have armies. That's no better. Spaghetti. You can kill them with the spaghetti.
EW (23:04):
Well, the armies can retreat. So they don't have to fight, just like the heap and stack don't have to collide and cause your system to crash.
CW (23:12):
Yeah, but they're not thinking about it. The stack doesn't get close to the heap and say, "Oh, no. I'm too close to the heap. I'd better turn around."
EW (23:22):
That's true.
CW (23:24):
It'd be cool if it did.
EW (23:25):
Well, that's why get an RTOS.
CW (23:28):
It just yells at you, and you crash anyway. It just crashes with a message saying, "Hey, you idiot. You blew your stack."
EW (23:33):
Well it crashes with a, "You did something wrong, and so I'm not going to allow you to destroy your system by accidentally calling code that says - "
CW (23:42):
Yeah.
EW (23:42):
" - light on fire."
CW (23:43):
And then you go in and you configure a bigger stack,...and then it says you don't have enough room for that. And then you spend a week trying to figure out how to fit all your code in and the stacks you need.
EW (23:54):
Yeah. I don't feel like we've made any progress on this metaphor.
CW (23:58):
I like what you had. Why do have to use something better?
EW (24:01):
You think the armies is the way to go?
CW (24:03):
That's pretty good. I mean, you can fire it up in one of those hilarious army simulators.
EW (24:10):
Oh, I was thinking about going back to the Inkarnate map thing, but maybe an army simulator would be kind of funny.
CW (24:17):
Yeah. I don't know enough offhand. Maybe I can just instantiate a few million people in Blender and have them run at each other.
EW (24:23):
Well, I need them to go forward and back.
CW (24:25):
Yeah. Yeah. They can go back and forth. I'm not doing that. I'm not signing up for that.
EW (24:29):
And then in the heap, we'll get memory fragmentation by having one guy sit down and have lunch.
CW (24:36):
A food truck coming by and taking up a bunch of space.
EW (24:42):
Okay. So that's me. I've been doing a lot of things.
CW (24:48):
I feel like you've been doing all the things.
EW (24:48):
Shortly after all of this, I will be hiding under the bed.
CW (24:52):
No. Somebody will say, "Hey, Elecia,...you want to do this?"
EW (24:56):
"Do you want to do a second version on your book?"
CW (24:58):
"...I have a contract for you to do this thing that's interesting." And you'll take it.
EW (25:06):
I'm not taking any CES contracts. I'm saying that right now in public.
CW (25:11):
Have you ever in the past?
EW (25:12):
Oh, yeah.
CW (25:13):
When?
EW (25:14):
They come in December.
CW (25:15):
They come in December?
EW (25:18):
Some company comes to you with an Arduino and says, "We want to show this at CES."
CW (25:22):
And you've taken these contracts?
EW (25:24):
Yeah.
CW (25:24):
When? Which one? Tell me. The statute of limitations is long past.
EW (25:30):
Well, there were a couple of sports ones. You remember the sports ones, right?
CW (25:35):
Those were CES?
EW (25:36):
One of them was, yeah.
CW (25:38):
It started at CES?...You weren't already working for them, and they -
EW (25:42):
No, they emailed in November, December, and said, "We want this for CES."
CW (25:46):
Alright.
EW (25:46):
I like them. They're kind of fun, because it's really just hacking a prototype together from a worse prototype.
CW (25:52):
You just said you didn't want to do it.
EW (25:54):
I know.
CW (25:54):
And then you said you like them, and they're fun. This is how you get in trouble, -
EW (25:57):
I like them, -
CW (25:57):
- and you click the checkbox.
EW (25:59):
- and they're fun, but I should not add anything in this calendar year until the class is over. And actually at the end of this year, my other two projects are over too. It's going to be January 1st that I'm going to be like, "Oh, my. What am I doing?"
CW (26:14):
And then you're going to be really unhappy, because you don't have anything to do. You have two states, unhappy because you have too much to do, and unhappy because you don't have anything to do.
EW (26:26):
That's not true.
CW (26:26):
I suggest you average those two positions and just have just enough to do.
EW (26:32):
I did that pretty well last year.
CW (26:37):
Okay.
EW (26:38):
Well, and class could take up an infinite amount of time.
CW (26:41):
Yeah.
EW (26:42):
Because there's always a slightly better way to explain things. I guess with code, there's always a better method to do it, or whatever.
CW (26:52):
With my code there's always a better method.
EW (26:54):
...CAN is doing better.
CW (26:56):
CAN. CAN.
EW (26:57):
We talked about CAN problems. [Singing]. CAN, CAN -
CW (26:59):
CAN problems.
EW (26:59):
- CAN, CAN, CAN.
CW (27:00):
Oh, last time we talked?
EW (27:01):
Yeah.
CW (27:02):
Yes. And people tried to help me, but I'd already solved it once they...helped.
EW (27:07):
And did your solution come from anywhere else?
CW (27:11):
No, I can't remember anyone suggesting anything...I can't remember anyone suggesting the exact specific thing I did. The word I ended up using did not come up in the conversation with the person who claims they told me how to do it.
EW (27:30):
Fine.
CW (27:30):
Yes. You gave me a pretty big hint for how to look at how to do it...Just to back up and recap for people, what was happening was... I'm gathering data from a radar over CAN, which comes in at about once every 50 milliseconds. It hands me 64ish CAN messages, which correspond to radar tracks.
CW (27:52):
And I'm also streaming an IP camera, 4K images from an IP camera, and using GStreamer to save those off to disk as JPEGs, also at 20 frames per second.
CW (28:06):
And what I was doing at the time was getting the image from GStreamer. It gives me a signal and says, "Hey, I got a new image." And then I was talking to the CAN code to go ahead and grab the latest set.
EW (28:22):
The next set.
CW (28:22):
The next set of radar things, and then write those both to the RAM database that I'm acquiring stuff into. And what was happening was once in awhile, way too often, but once in a while, the radar just would get interrupted, or go out to lunch, or...that portion of the code would not keep up.
CW (28:41):
And it would say, "Hey, I got track three when I was expecting track zero," which meant it had wrapped around, and it had missed a bunch. And I couldn't figure out how...to solve this problem. Tried various things.
CW (28:54):
Ultimately what I think the issue was, was I was...doing this all in Python with GStreamer, trying to use GStreamer's callbacks and things to do everything. GStreamer, behind the scenes, creates a whole bunch of threads. And you don't know what they are, and you know what they're doing.
CW (29:15):
And every time you add a queue to your pipeline, it adds a new thread. So I probably had 2, or 3, or 4, 15 threads running for GStreamer, but you don't know what they are. You don't have any control over them. And threads in Python are not really that great, because of the -
EW (29:35):
GIL.
CW (29:35):
The GiIL?
EW (29:38):
I don't know. Would you say GIL, like JIF?
CW (29:40):
The [Gyle]? The [Gyle]? The [Gial]? The global interpreter lock, which is the single top-level lock in the Python interpreter, which basically says you can only have one thread running at a time. So you can have as many threads as you want in Python, but it's not going to put them on different CPUs or do any useful things for you.
EW (30:00):
But GStreamer -
CW (30:02):
It's just like your old threads on your Windows computer in 1997 when you had one CPU. You can have as many threads as you want, but they're just cooperative multitasking, bouncing back and forth. So if something is busy, you can get held up.
EW (30:16):
But GStreamer has true threads.
CW (30:18):
I agree, but it's also probably getting priority over me. I don't know...I'm speculating as to what was going on.
EW (30:28):
Okay.
CW (30:28):
But clearly putting all this in one Python process was not working. If you have a better idea for what was going on, I'm open to hearing, but -
EW (30:40):
No. I mean, for some of my stuff, I have GStreamer with a couple of threads, and then those threads are separate from my Python quote threads, which I have with ROS. And ROS uses Python threads, so those are all running cooperatively. But my GStreamer threads run on different processors.
CW (31:04):
But...you're not doing things in the other threads in a timely manner based on GStreamer.
EW (31:11):
No. Everything stays very asynchronous.
CW (31:14):
Right.
EW (31:14):
There's no waiting. Waiting is bad.
CW (31:16):
Yes. Well, not waiting, but I was signaling from GStreamer, anyway, I was doing something different and probably -
EW (31:24):
Yeah.
CW (31:24):
- worse. What I ended up doing was, Python has a wonderful module called multiprocessing, and multiprocessing allows you to spawn new processes. And it works much the same as threading would work. A lot of the calls were the same. A lot of the communication methods are the same between threads as -
EW (31:53):
Semaphores, mutexes.
CW (31:53):
- message queues and things.
EW (31:54):
Message queues.
CW (31:54):
And they look the same, and they have the same APIs. And so you can just spawn a new process, and communicate with it over queues.
EW (32:02):
And the new process is an actual process that shows up -
CW (32:05):
Actual process -
EW (32:05):
- in ps or top.
CW (32:06):
- and it goes on a different CPU. If you've got them, the OS can move it around, and it runs independently. You can choose to have it inherit your entire Python context in terms of a copy, or just start from fresh without a copy. And you can do all kinds of things. You can get in trouble with some of that, but you can do all kinds of things.
CW (32:24):
And so what I ended up doing is..., and this is all in the same Python file, I don't have multiple Python files, creating a new process that just free runs with the radar. And so it just sits there and gathers new tracks all the time.
CW (32:41):
And it gets a message from GStreamer, which is in a different process, that says, "Hey, I got an image. Here's the image file location." And it's timestamped. And it sends that to...the radar process.
CW (32:54):
And the radar process says, "Oh, I've got an image? Fine. What's my latest radar tracks? Boom. This and this go together. Put it in the database. Out." Never had a problem again. So that's what fixed it.
CW (33:07):
But multiprocessing is super cool, because...that's sort of a by hand kind of approach to doing it, create this process, create this process, link them up with communications and stuff. If you have to do something that complicated, it's more of an architecture.
CW (33:22):
But it's also useful for tackling big data problems. So I've used it in the past a whole bunch of times when I'm processing a huge list of images that I need to do something to.
CW (33:34):
So let's say I need to mark them up..., I have data in a database, and I have a set of images. And I need to take that data, and I need to paste it on the images in some fashion, maybe highlight something, maybe put some objects in.
CW (33:49):
Normally you just go through that one by one, right? You grab the image, you look at the data, you do your thing, and you'd save it out. And if you do that in Python the normal way, it would run just fine, but it would run on one CPU beginning to end.
CW (34:07):
But what you can do with just a few lines of code with multiprocessing and...a thing called the multiprocess pool is, you can say, "I have eight CPUs. Create a pool with eight CPUs. Here is my data set. Divide this up however you feel like. Here's the function I want you to run on it, and multiprocess, please go do this."
CW (34:31):
And it divides it up into these sections. It feeds each of those sections to a different copy of the function you gave it. And it runs them all on how many CPUs you've got. And now it's, whatever, eight times faster than it was. And it's six lines of code to do that. So, super cool, and I use that all the time.
CW (34:57):
So hopefully that helps somebody who's suffering through. But yeah, Python doesn't work well with multithreading, and if you really want to do anything concurrently, multiprocessing is the way to go.
EW (35:09):
And you can get threads from other places...There are other big libraries. I mean, Matplotlib and NumPy will spawn new true threads as opposed to the global interpreter lock thread. But you really have to be careful about that global interpreter lock. It will bite you.
CW (35:29):
There is a rumor, in the last week or so, that they are getting rid of the global interpreter lock.
EW (35:35):
I mean, it's been such a pain for so many people. And it's a hidden pain, because you can call something a thread, which you believe will run concurrently on a different processor, but actually it doesn't.
EW (35:52):
And unless you're vigilant about checking top and looking at things, you just wonder why this program that is supposed to be efficiently written takes forever to run on your 32-core processor. How many cores does, oh, that's not that computer.
CW (36:13):
What, my new one?
EW (36:13):
Yeah.
CW (36:14):
It has 10.
EW (36:15):
Oh. That's nice.
CW (36:17):
Anyway, there was another Python topic, if you want me to continue blathering about that. But I don't know where we are on time -
EW (36:21):
Bit twiddling in Python.
CW (36:23):
Yeah, so this is more of a rant and a complaint. This is more of a -
EW (36:27):
I never have found bit twiddling that bad in Python.
CW (36:29):
You're not doing what I was doing.
EW (36:31):
Of course.
CW (36:33):
I mean, bit twiddling is fine.
EW (36:37):
It says here, "Talk about bit twiddling in Python." Maybe someone has better ways.
CW (36:42):
Yeah.
EW (36:43):
But you asked me how to do something, and I told you how to do it without having to depend on the size of the variable.
CW (36:49):
I know. It's great. But it was still a big function, that you had to do a bunch of stuff with it.
EW (36:55):
Four lines.
CW (36:56):
Let me explain the problem -
EW (36:58):
Alright.
CW (36:58):
- before you get offended at your solution, which is wonderful, and worked, and I used it. But it would be nice if there was kind of a native way to do this sort of thing that didn't require knowledge of number formats. Okay.
EW (37:12):
Sorry.
CW (37:12):
No, I mean, I have that knowledge too, but...if you find yourself doing a bunch of, "Where is the signed bit, and how do I sign extend this," you're probably off in the weeds somewhere.
EW (37:23):
Start at the beginning, you had a ten-bit -
CW (37:24):
Right.
EW (37:24):
- signed number.
CW (37:25):
Right. Anyway, CAN. Some of the messages that I get from CAN have numbers in them. Weird, right?
EW (37:38):
Docker.
CW (37:38):
Right. Some of them are split. So I will get the lower five bits of the number, and then I'll get eight bits of a completely different thing, and then the upper five bits of the number previous.
CW (37:54):
In C, this is not such a big deal, and certainly combining those is not such a big deal except these were signed. So what I've got is a ten-bit signed integer split into five, well, a nine-bit signed integer with a signed bit split into two five-bit chunks.
EW (38:20):
In two's complement.
CW (38:22):
Right.
EW (38:23):
Well, when you said signed bit that actually doesn't make it two's complement.
CW (38:29):
Yes...I don't remember the specifics of what we did...That was last week. I can't be expected to remember anything. It's a signed thing.
EW (38:38):
Okay.
CW (38:38):
So normally in C it's ten bits, okay, you've got a sign extended to a short, right? You've got to take that ten bits, and you've got to fill out the five bits to make it as if it were a signed 16-bit instead of a ten-bit. Because if you just stick it in the ten bits in the short, you're going to have a bad time.
EW (38:59):
Yeah.
CW (39:00):
You're going to have radar tracks just flying all over the place. And they're all on the sides. And everybody's upset, because it's not working right. Because I had radar tracks, and they were all messed up. And they were all on...two sides, and everyone wasn't happy.
CW (39:16):
Okay. So in C that's not as big a deal, because you can manipulate things with different timelines. In Python, it's a little trickier, because an int is just kind of an int. They don't really go for telling you how many bits are in it.
EW (39:31):
There's no int32.
CW (39:32):
There's no real int32 or int16. Now yes, people are yelling, "You could use NumPy. NumPy has those." This is true. I could have done that, but it seemed kind of weird. And so, yeah. So we had to find a way to sign extend this into a Python int. And you had a nice way of doing it, and I ended up using that. But it still was too -
EW (39:59):
Gross?
CW (40:00):
Yeah.
EW (40:00):
Because it looked like it was bit twiddling?
CW (40:03):
It was bit twiddling with a bunch of conditionals and things.
EW (40:07):
It was one conditional, whether or not you were in a signed or not.
CW (40:09):
I'm not really complaining about your not solution. This is not an attack on your solution, which I'm currently using. I'm just wondering if I'm missing something about Python integers that would have made this more straightforward.
EW (40:22):
So he asked me about this, and I gave him some suggestions, and then he walked away. And then I, instead of doing what I was supposed to be doing, wrote the function for him, because bit twiddling is really fun. And it's a solvable problem. It's always a solvable -
CW (40:39):
Twiddling wasn't the fun part. I did the twiddling.
EW (40:40):
The whole problem of, "I don't know how big my integer is,...and I need it to be negative, so how do I... -
CW (40:47):
I wrote a whole CPU emulator that did all of the signed arithmetic for the thing.
EW (40:52):
I'm not saying you couldn't have done it.
CW (40:52):
I know what I'm doing. I just -
EW (40:54):
You just didn't want to take the time to do it. And I treated it like a game.
CW (40:57):
This is one of those areas where Python being nice and easy made something harder.
EW (41:01):
The solution that I did for you is probably the one I would do in C.
CW (41:05):
...Yeah, maybe. I think there's an easier way to do it in C, but sure. But...I could have done it into a short. I have the solution in C. It's in the example code they gave me.
EW (41:20):
Yeah.
CW (41:20):
And it was much shorter than that function. It was just a couple of things. They just anded it with something magic, because they knew it was a short. Anyway.
EW (41:32):
Anyway, if you any bit twiddling problems, please send them to me.
CW (41:35):
...Fine. I'm going to stop using Python. It's time to switch to Rust, except I don't think I can use OpenCV in Rust, GStreamer and Rust and OpenCV would probably break me. I'd have to learn too many things. Also, my client would kill me.
EW (41:54):
Probably not. Okay. I heard a rumor.
CW (42:02):
Yeah?
EW (42:02):
I heard a rumor that the chip shortage -
CW (42:08):
[Laughter].
EW (42:08):
I heard one rumor that the chip shortage is going to be cleared up by Q1 2022.
CW (42:14):
That's Elon Musk. And I hear a lot of rumors for him. He just says what the voices tell him.
EW (42:19):
Yes. And full self-driving will be done next month.
CW (42:22):
Full self-driving was done three years ago..., four years ago.
EW (42:28):
Anyway, for those not reading between the sarcasm lines, I believe the chip shortage will be going through 2023. So continue your hoarding. No, wait, sorry. Don't hoard them. That's rude. Continue pre-buying your chips for a little while longer. And if you're thinking about starting a company -
CW (42:51):
Make a software company.
EW (42:53):
Make it a software company.
CW (42:54):
Yeah. I mean, where are you hearing this rumor? What little birdie is telling you this?
EW (43:00):
I can't divulge -
CW (43:00):
But you have a birdie.
EW (43:00):
- my informants.
CW (43:03):
But you have a birdie.
EW (43:03):
Oh, yeah. I have a birdie.
CW (43:05):
And your birdie works at chip company.
EW (43:09):
I'm not going to divulge my sources.
CW (43:11):
We're not talking Kettle. We're talking an electronic chip company.
EW (43:16):
What do robots like to eat for lunch?
CW (43:19):
"Rawmain" lettuce.
EW (43:23):
Microchips. Yeah, so if you haven't been following me on Twitter, this whole month has been full of really terrible jokes. So if you are following me on Twitter, I have no idea why the jokes just get worse, but they will end on Halloween.
CW (43:40):
The jokes will continue until morale improves?
EW (43:43):
No.
CW (43:43):
Oh. The jokes will continue until the month ends?
EW (43:47):
Until the month ends.
CW (43:48):
Maybe morale will improve.
EW (43:50):
After I stop them? Probably. They're really bad. Okay. So the other thing I wanted to ask you about was your album -
CW (43:59):
Which album?
EW (43:59):
- and specifically the new EP album art.
CW (44:03):
The extended play record? Well, that's not released yet. It was only sent to the fine Kickstarter backers who bought it as a bonus.
EW (44:14):
But isn't everybody going to be able to see it too?
CW (44:17):
On November 12th -
EW (44:19):
Okay.
CW (44:19):
- when it releases...Yes. Yes. We have a new five-song record that we released to the Kickstarter people who ordered it. It was one of the Kickstarter options. And yeah, it has some songs on it. It's got five of them, and there's artwork that -
EW (44:36):
The music is cool.
CW (44:36):
You don't care about the music.
EW (44:37):
I want to talk about the artwork.
CW (44:37):
You care about the artwork. Yeah, yeah, yeah, yeah.
EW (44:40):
Generated adversarial network?
CW (44:44):
Generative adversarial networks.
EW (44:47):
Adversarial networks.
CW (44:47):
Yes. That is the name of the record. No.
EW (44:49):
I hate everything is the name of the EP.
CW (44:51):
#ihateeverything is the name of the record.
EW (44:54):
That's the important part.
CW (44:55):
Yes.
EW (44:57):
So tell me about this network that you have, and where you got it from, and why the -
CW (45:07):
I got it from Twitter.
EW (45:07):
- waves have teeth.
CW (45:09):
Where did you get anything? I got it from Twitter. People have been playing around with these for a while. It's a neural network, there's types of networks called generative adversarial networks.
CW (45:20):
And I don't remember the architecture exactly, but there's something where it has kind of targets it's been trained on certain styles of images and things. And then it feeds back, and...there's a feedback portion. So it tries to generate things that are like that stuff.
CW (45:41):
And the one I was using has another portion of the model that has text as part of it. So it's got text associated with the images. So you can put in a text prompt, and it will attempt to create an image that's something to do with the texts you gave it. And it takes several long steps to get anywhere.
CW (46:02):
And you can watch it kind of think about things, dream about things, it's like watching something dream. They're very strange images. They're very dream-like, nightmarish sometimes.
CW (46:09):
And Janelle Shane's been doing a bunch of them and some other people. You see them on Twitter. And I was very lazy for the album art for this. After spending so much time on the other one, I didn't really want to.
EW (46:21):
Well, you had an idea.
CW (46:21):
I had a couple ideas.
EW (46:22):
It just didn't quite work out.
CW (46:23):
They were okay. There was some photo stuff that I was working on, and I'll probably use it in some context later, but I just got annoyed one afternoon, and...loaded this up. So the model I'm using is called CLIP+VQGAN. And...those are the two portions, and there's various places you can get it if you just type that in Google.
CW (46:48):
And then you load it up in Google Collab notebooks, which are very cool. It's basically a Jupyter notebook that gives you a GPU backend, not a great GPU, not a terrible GPU. You can pay them ten bucks a month to get a better GPU.
CW (47:03):
And it loads right up there in your browser, and you can go through the things, set it up, and put in the text prompt, and it just goes. So I didn't do anything. I didn't design this, or write the code, or anything, just paint by numbers. And so I got annoyed and started putting increasingly strange prompts in.
CW (47:21):
And one of them was a ocean monster, monster wave teeth, something. And that's what ended up being the art for the EP. And I'm going to start generating my code this way.
EW (47:34):
I don't know about that latter, but you did get some pretty weird, but cool -
CW (47:39):
Yeah. And I lost the one with the scary girls on the beach with the haunted sandcastle.
EW (47:45):
That was okay. That was scary.
CW (47:45):
Yeah, but I still wanted show it at some point. But yeah, it's weird. It's weird to see a computer kind of, it's not intelligence, it's not dreaming really, but a bunch of stuff kind of randomly putting together strange images.
EW (48:03):
And part of it is putting together the strange images. If you just typed in cat, you'd probably get a normal-looking cat
CW (48:09):
I don't know. I haven't tried that. It probably depends on the model. I think there's better ones for that kind of thing. I know there's the ones that generate people that don't exist and things, and those tend to be better.
CW (48:20):
Although they still make weird mistakes with those. People will have the wrong number of teeth or just subtle stuff. You'll see they have three incisors or something, or -
EW (48:30):
You got one that looked Japanese woodcut style.
CW (48:35):
Yes.
EW (48:35):
And it was big waves.
CW (48:35):
Yes.
EW (48:35):
And then they had sharp teeth, and those were really cool.
CW (48:41):
Yeah.
EW (48:41):
But you didn't use that one.
CW (48:41):
That one wasn't as good as the other one.
EW (48:44):
But you also got waves with teeth. And instead of being sharp monster teeth, they were human teeth.
CW (48:51):
Yeah, I don't remember the prompt for that one.
EW (48:51):
That was terrifying.
CW (48:52):
Well, and I think there was one time I asked for woodcut, and it gave me -
EW (48:55):
Wood?
CW (48:55):
- wood, which wasn't what I asked for. So...there's no intelligence here. The interesting thing is the more usable stuff tends to be the earlier, more abstract...The longer it goes, the closer it gets to kind of a photo quality instead of a painting quality.
CW (49:15):
And so I had some that were good after the first hundred steps. And then a thousand steps later, I was like, "I don't know what it's doing. It's made this into a photo."
EW (49:24):
Photo quality waves with sharp needle teeth was -
CW (49:27):
Yeah, but it's a good way to make some abstract stuff. Now you can't use some of the models for commercial purposes. So there's two. There's a, I think it's a Flickr-based one called SFlickr. And there's another one I forget the name of that you can use for commercial purposes.
CW (49:41):
...But most of the other big models are not for commercial purposes, so you can get different styles of images and things, depending on what your source model is. But I was limited to those two.
EW (49:55):
Because you were using it -
CW (49:57):
Yeah.
EW (49:57):
- on an album -
CW (49:57):
Yeah.
EW (49:57):
- that people would pay for.
CW (49:58):
Yeah, yeah.
CW (50:00):
Anyway, I didn't spend a whole ton of GPU time doing it. So it wasn't a big deal, but sort of interesting distraction.
EW (50:08):
It was fun to watch. I think we've talked about it a little bit before, because you've played with it before.
CW (50:12):
Yeah, I'm sure. I think we have.
EW (50:13):
But I was, once again, kind of boggled by what you can get out of those.
CW (50:20):
Now there's a pretty big limitation. And I don't know that it's work-around-able with the thing that I used. The image size is 512x512, which is way not enough for using for art purposes, photo art purposes.
CW (50:35):
So I ended up taking that image, putting that in Photoshop, and doing some stuff, and then using another neural network based photo enlarger thing to get it up to a really high resolution that didn't look bad.
EW (50:50):
Do you think neural networks are going to take over the world?
CW (50:51):
No. I'm sure there was somebody after they invented the screwdriver who said, "Do you think the screwdriver's going to take over the world?" I think of them as kind of annoying screwdrivers at this point. Like I said on Twitter a few weeks ago, call me when it has the intelligence of a weevil.
EW (51:08):
Although I've never seen a weevil paint.
CW (51:12):
I think a weevil's visual processing system is probably more sophisticated than whatever made my album art.
EW (51:19):
Do you think if the weevil painted a self-portrait on a smaller canvas, it would be the lesser of two weevils? I'm sorry. It's really been a month full of bad, bad jokes. And I don't know why, but every October I'm kind of drawn to the -
CW (51:38):
I did that joke.
EW (51:40):
I know. You were ten.
CW (51:41):
30 years ago on a whiteboard in the CS department lab, we drew two weevils and then pointed to the lesser of two weevils. I was really proud of that. That was my biggest accomplishment in college.
EW (51:56):
Well, I think, this show of course will be our biggest accomplishment today, this particular episode. So if you're ready to put a cap on it, I can -
CW (52:08):
Don't we have anything else to talk about?
EW (52:09):
I mean, we can talk about Patreon -
CW (52:13):
No.
EW (52:13):
-and how those people are nice, and -
CW (52:16):
They're all very nice people.
EW (52:17):
- about how we've had guests, who, when we ask them about sponsorship on their YouTubes or whatevers, they say, "Oh, I didn't think you had sponsorship, because you had that message that said you don't have commercials." And I'm like, "Yeah, that was a year ago." And -
CW (52:34):
We do have it on some of the repeats, because I'm too lazy to cut it.
EW (52:38):
Well, anyway,...we're taking sponsorships.
CW (52:43):
Please sponsor us.
EW (52:45):
That's pathetic. Okay. So -
CW (52:54):
You can finish the podcast.
EW (52:56):
Thank you to Christopher for producing, and co-hosting, and chatting with me.
CW (53:01):
You're welcome. Oh, God, I probably wasn't supposed to say that.
EW (53:06):
No, it's okay.
CW (53:07):
Oh, okay.
EW (53:08):
Thank you to our Patreon supporters for supporting us and buying mics for our guests, as well as helping us get the word out about the show. If you like the show, please tell somebody about it. What else? Oh, thank you for listening. You can always contact us at show@embedded.fm, or hit the contact link on embedded.fm.
CW (53:34):
Where can they find out about this class that you're teaching?
EW (53:37):
Gosh, I should put it on embedded.fm, shouldn't I? I should make a blog post or something.
CW (53:41):
Yeah, you probably should make a blog post, and some links, and stuff. Maybe a little banner across the top.
EW (53:49):
In the meantime -
CW (53:49):
A pop-up that flashes at you. "Act now."
EW (53:55):
I was reading about writing call to actions today. I've learned that I am terrible at marketing.
CW (53:59):
Just give us your email.
EW (54:02):
Yeah. Okay. I'll put a link. There'll be a blog post, I guess. And worst comes to worst, search "Classpert" and my name. And you can even misspell it, and it will find it. Or "Classpert" and "Making Embedded Systems," although that one you have to spell correctly. Is it time for Winnie the Pooh, or -
CW (54:24):
Yes.
EW (54:24):
- did I forget stuff?
CW (54:25):
I'm sure you forgot something.
EW (54:27):
Okay. [Winnie the Pooh excerpt].