364: All the Abstractions
Transcript from 364: All the Abstractions with Jacob Beningo, Elecia White, and Christopher White.
EW (00:00:07):
Welcome to Embedded. I am Elecia White, alongside Christopher White. This week we are going to talk about embedded systems with Jacob Beningo.
CW (00:00:17):
Hey, Jacob. Welcome.
JB (00:00:19):
Hey, how's it going?
EW (00:00:21):
Good. Could you tell us about yourself as though you were keynoting the Embedded Systems Conference?
JB (00:00:28):
Sure. Yeah. So, my name is Jacob Beningo. I'm an embedded software consultant. I've been developing embedded systems for the last 20 years professionally. Although I'm one of those software nerds who started developing software when they were 14 with FIRST Robotics, and got started with PIC16F84s, and Stamp 2 controllers.
JB (00:00:49):
But I get the opportunity to work with a lot of cool people in a lot of different countries, and around the world, developing anything from simple consumer electronics, widgets, all the way through, kind of my specialty, which is flight software for small satellites.
EW (00:01:04):
Alright. I have so many questions, but first we have lightning round where we ask you short questions, and we want short answers. Are you ready?
JB (00:01:11):
Yes.
CW (00:01:12):
Alright. Preferred embedded programming language.
JB (00:01:16):
C.
EW (00:01:17):
What about non-embedded?
JB (00:01:20):
Non-embedded? Python.
CW (00:01:21):
Oh, I see, that counts as your question? Favorite RTOS.
JB (00:01:27):
ThreadX.
EW (00:01:29):
What's the most important thing to teach software engineers who want to be embedded software engineers?
JB (00:01:34):
How to design and architect their software.
CW (00:01:37):
I'd say that goes for everyone, but yeah.
JB (00:01:39):
It is pretty generic, but I find embedded guys tend to skip that step a lot. Although, I'm sure we all do.
CW (00:01:45):
Do you have a bug that you'll always remember or hope that you could forget?
JB (00:01:51):
Not specifically.
CW (00:01:54):
Okay.
JB (00:01:54):
Yeah.
EW (00:01:54):
Do you have a least favorite peripheral communication bus?
JB (00:02:00):
I'm not the biggest fan of CAN, even though it's very useful.
CW (00:02:05):
Yeah. See last episode. Is there an interview question you like to ask embedded engineers?
JB (00:02:13):
Not one right off the top of my head specifically.
EW (00:02:17):
Do you have a tip everyone should know?
JB (00:02:19):
That they should use tracing technology when they're implementing their application code.
CW (00:02:29):
Favorite fictional robot.
JB (00:02:31):
Johnny 5.
EW (00:02:33):
That one's pretty popular.
CW (00:02:34):
That may be modal answer. Yeah.
JB (00:02:38):
Yeah. Probably one of the ones that got me down the engineering path as a child.
EW (00:02:44):
Okay, so tracing technology. I use an in-circuit programmer, JTAG, or whatever we're going to call it this week, even though neither one of those technologies is still what we use -
JB (00:02:59):
Yeah.
EW (00:02:59):
- to program my Cortex processor. And I can step through it, which is such an improvement over things like Arduino or Mbed, but tracing is different. Tracing is an order of magnitude different. Could you tell us about it?
JB (00:03:18):
Yeah. So the idea here with tracing is that...being able to set our break points, and kind of look in, and see what the registers are doing, is fantastic. Being able to step through and get an idea what's going on. The thing that's great about tracing is that there's different modes, obviously.
JB (00:03:34):
You can do kind of snapshot, where you get a little picture of what's happening, but you can also do kind of streaming trace where you're constantly acquiring data. But the idea is that as your system is creating events. So for example, when you have a contact switch to start running a new task, and then you contact switch out of it.
JB (00:03:50):
An event gets recorded in the little RAM buffer, that then gets pushed back over the JTAG interface, or the programmer. Then the program on your PC can take and rebuild what's happening in the actual application. So what can happen is, as developers,...we can step through, and we can get little pictures of what's going on, but the tracing technology allows us to see kind of like an oscilloscope.
JB (00:04:13):
It allows us to see everything that's happening in our system potentially. So I can see when tasks are starting, stopping, when semaphores are being given, when they're being taken. I can create custom messages to see when LEDs are turning on or off, or see the states of individual state machines that I've created in my code.
JB (00:04:27):
So it really allows us to visualize what's happening in the software versus kind of, when I first started developing where, yeah, you could maybe toggle some LEDs, but you really had to spend a lot of time instrumenting your code to see what was happening.
JB (00:04:39):
And most of the time it was kind of cross your fingers and guess, and you'd go through a process for days trying to guess and figure out what exactly the system's doing. Whereas tracing technology today really allows us to see what's happening very quickly. And hopefully, one see things like performance, make improvements, and probably the big thing is chase down bugs as well.
EW (00:05:01):
But it requires a special tool.
JB (00:05:04):
So you can do it with just a standard onboard debugger...on a lot of these development boards.
EW (00:05:10):
Really?
JB (00:05:10):
So yeah, you can push trace information out of a simple serial port if you wanted to. You can, depending on the recording library that you're using, there's several different companies that create them, but you can simple serial interface, you can do it over USB, you can do it over Wi-Fi. I typically push mine over my J-Link tool that I have.
JB (00:05:29):
I usually set the library up to use SEGGER RTT, which is a real-time transfer protocol. And then, that allows the JTAG to very efficiently transfer the event data so that it has minimum impact on the real-time performance of the application. Because when you instrument things, you want to minimize the impact.
JB (00:05:48):
Because every little extra line of code, you put in there could change the performance and the time of the system, which could then maybe make a race condition that exists, not exist for a short period of time, or something like that. So it's always, you want your tools to have minimal impact on the execution of your application code.
EW (00:06:05):
I certainly have compiled for release, getting rid of all of those printfs and realized, "Oh, there's a timing bug." Yes, yes.
JB (00:06:14):
Yeah, absolutely.
CW (00:06:16):
But there is special hardware. You can buy,...I think if you want to keep up with huge quantities of data, like every memory access or every instruction, then you do need something more beefy?
EW (00:06:30):
Yeah. The J-Links, there are different levels of them.
CW (00:06:35):
I remember at Fitbit we used something from Lauterbach that was a very expensive box.
EW (00:06:40):
Because it's basically -
CW (00:06:43):
A big buffer.
EW (00:06:43):
- a giant buffer, a very fast buffer, usually an FPGA in there as well.
JB (00:06:48):
Yes. Yeah, exactly. So you can certainly, if you want to be able to do instruction tracing and really get a ton of information out, then certainly you have to spend the money on a more professional tool that can handle that.
JB (00:07:01):
So there's certainly debuggers that have the name trace, are designed to work on the Extended Trace Macrocell, which is often shown as the ETM, that can actually grab that individual instruction tracing and rebuild it. And obviously you need very fast signals and interfaces to be able to get that kind of information.
CW (00:07:21):
What software do you like to use for trace? I remember, I've only really experienced it with the Lauterbach, and it's software was difficult to use [laughter] at best.
EW (00:07:33):
Complex.
CW (00:07:33):
Yeah, it was one of the harder things I've done in embedded systems in a long time, is figuring out how to get meaningful data out of that thing, because there was a fire hose of data.
JB (00:07:43):
Yep, absolutely. Yeah. So there's typically two tools that I generally use. The one I predominantly use is Percepio's Tracealyzer tool. That's one that interfaces really well with ThreadX, and FreeRTOS, and several other real-time operating systems. And I've had a pretty good experience with it.
JB (00:08:00):
I might be slightly biased though, as well, just to kind of put it out there, because I have done webinars and work with them. So just to kind of put that out there. But I've also used SEGGER SystemView, which is kind of just a free utility tool that comes with the J-Link series of programmers. And that one can still get you some basic information out of it, but it doesn't have as much of the reporting types of capabilities.
EW (00:08:24):
How does this relate to satellites? I mean, you can't exactly go up into space, and touch the satellite, and ask it for a trace for what went wrong.
JB (00:08:34):
Well, theoretically you could. But if you were to store it on board and then try to pull down if something went wrong, you could pull those trace buffers down or store them in a file, but the way I use it most of the time is actually...during development -
EW (00:08:50):
Yeah.
JB (00:08:50):
- to get the understanding of how the system's performing. I want to make sure that if I'm designing a real-time system, you want to make sure you have good response times. You want to make sure that the tasks are executing when you think they are.
JB (00:09:04):
Sometimes you have very complex timing, especially if you have a guidance and control algorithm that needs to run, and you're reading sensors, and you have to drive a propulsion system. And then there's all these things going on, and you want to make sure that they respond within a reasonable period of time.
JB (00:09:19):
What I'll do is I'll use it on the bench to measure the performance of the software, and fine-tune priorities, or the timing of tasks. Or watch for periods of time where maybe you might have too much CPU time being used, suddenly, you hit a hundred percent for too long, or things like that.
JB (00:09:41):
Or maybe you have a bug and at least on the bench, it allows you to kind of dive in and see what's happening with the system. And I usually will trace through development and with each release, I'll actually kind of say, "Okay, here's the gold standard of what the trace showed while I'm performing this particular operation."
JB (00:09:57):
And then if there's ever an issue in the future, you can always go back to the Git repository and pull, "Okay, how did this baseline trace look, and what might've happened?" It helps you kind of track back through time how your software is evolving and how the performance is changing.
JB (00:10:14):
And sometimes...you'll have a bug you didn't discover, and then you go back a couple, and you're like, "Oh yep, here it is. It is in the trace. It's just, we didn't see it then." Or, you'll discover, you go back somewhere in-between these two versions, one's working good, one's working bad, and that can kind of help you figure out where to start dividing and conquering to get the bug out of the system.
EW (00:10:35):
I found trace to be very helpful with very low-power systems. Trying to figure out why you're still awake, or figure out how to optimize so that you use the least amount of processor and go back to sleep as quick as possible.
JB (00:10:50):
Yes. Yep. That's definitely a great way to use it.
CW (00:10:53):
The crazy thing we did was, we used it to basically, I think we used it in conjunction with the MPU, so that every time we had, we set basically every memory access to be an MPU fault. And every time that happened, we went into a fault handler and then shipped some trace data out, so we could get a map of all memory accesses.
CW (00:11:17):
And then we used that to optimize which memories we put things in. So things that got accessed a lot got put in fast memory, and things that got accessed least got put off a external flash or something like that, based on the linker map. But you can do all kinds of crazy things with it.
EW (00:11:33):
Oh, yeah. I mean, it gives you everything your code did.
JB (00:11:38):
Yep, exactly. Yeah. There's so many different uses for it. It's unbelievable. And the insight that it gives us, I mean, I have a colleague who I think probably said it the clearest, is that this trace technology is really the oscilloscope for software developers at the end of the day.
JB (00:11:53):
I mean, as a hardware guy, you want to design hardware without an oscilloscope. And that trace technology really kind of gives us software developers that kind of level tool to see what's actually happening.
EW (00:12:05):
Do you have a preferred IDE? Neither of the tools you listed for trace are themselves IDEs.
JB (00:12:13):
Yeah. Correct. Yeah. So a lot of times I use a lot of different standalone tools. Some of the IDEs that I use, I probably gravitate a lot of times towards the Eclipse-based tools a lot. So for example, if I'm working with an ST part, I'll use STM32CubeIDE. If I'm using a Texas instruments part, I'll use their Code Composer.
JB (00:12:32):
A lot of times that's because of client requirements or what specifically we're doing. If I am doing something that's a little bit tighter, or...has more regulation and stuff like that behind it, I'll use something like IAR or Keil. But a lot of times I'll find that, even in some of the spaces systems work with small satellite stuff that I do, a lot of times people will still try to use open source tools as much as they can.
EW (00:13:02):
Yeah, that's my answer too. I use whatever the client wants me to use.
JB (00:13:05):
Yeah, exactly...A lot of cool ones out there. I mean, I'll use Sublime Text, for example. Sometimes if I'm outside of the IDE and I'm doing command line stuff, but yeah, it's whatever the client says to use at the end of the day.
EW (00:13:22):
I am becoming fond of Visual Studio, especially when I'm on a remote system.
CW (00:13:28):
Visual Studio Code.
EW (00:13:29):
Visual Studio Code. Thank you. Yes, no, that is the important part, because it can log in, and pretend to be local, and it's very cool. What about RTOSs?
JB (00:13:41):
Yeah, so from RTOSs, I think my favorite one, my favorite one was ThreadX, and ThreadX obviously has been, Express Logic was purchased by Microsoft a couple of years ago. It's now become Azure RTOS. And I think it's still one of my favorite ones.
JB (00:13:56):
They've done a pretty good job at least of keeping the ThreadX piece still available and separate if you don't need the cloud connectivity piece. But then probably behind there, obviously we all love FreeRTOS and some of the other open source ones, things like Zephyr and MicroC/OS. So how about you guys? What's some of your favorite RTOSs?
EW (00:14:17):
Most of my RTOSs creep in through other ways.
CW (00:14:21):
Yeah.
EW (00:14:21):
Like Nordic. It's really an RTOS, even though it sort of says it's an API. And I've been doing a lot of TI, and again, their wireless system is basically an RTOS. And I go ahead and use the BIOS that they've developed.
JB (00:14:46):
Oh, okay. Excellent.
CW (00:14:48):
I use a lot of ThreadX. And the one that I haven't heard people talk about much anymore, and I don't know whether it's just not popular, or whether there's something wrong with it, is I used Green Hills for a long time at one client.
CW (00:14:59):
And I kind of liked it. It was a little different than most RTOSs, but maybe it's just very expensive. I don't remember the reasons why people don't tend to pick it up.
JB (00:15:07):
Yeah. I think you're right there. I think it's probably the price, because I've got a few clients who use it, and usually they are mentioning, price, price, the cost, but in all honesty you get what you pay for. So I mean, as much as we love our free tools, and our FreeRTOSs, and things like that, there is a considerable difference if you look at the performance, or the output of the code, of a open source compiler compared to a commercial one.
JB (00:15:34):
So it's something that as developers, we definitely need to keep in mind, but it does limit, the cost does limit our ability to experiment with some of the other ones, for sure.
EW (00:15:47):
I thought FreeRTOS and RTEMS were both well enough supported and funded that they were approaching the point of real commercial viability for even systems that require stable RTOSs.
JB (00:16:03):
They are. I mean, I think over the last couple of years, FreeRTOS, there was always little annoying things with it. But a lot of those have been kind of, I think, cleaned up. I think it's gotten a lot more robust. I've been in the process of updating one of my RTOS courses, and we use FreeRTOS.
JB (00:16:22):
And in my lab notes, I have, "Oh, look at this quirky thing here, and look at this quirky thing there." And then going through and kind of updating it, I've been like, "Oh yeah, this whole section goes away. This...is not quirky anymore." ...They've definitely made a lot of improvements, for sure, to a lot of these, which is great.
JB (00:16:39):
I mean, they're so widely used, and considering robustness and things like that, that's always been some of my concern with some of these open source operating systems, is just people go to them because they're free.
JB (00:16:52):
And they don't necessarily look at what's happening underneath the hood, or looking at how robust they are and whether they really fit their applications. They just kind of look at the cost and say, "Okay, that's what we're going with." Which can be scary.
EW (00:17:07):
Free as in puppies. Yes. So, you are giving a talk at a conference in May. "Best Practices for RTOS Application Design." Could you tell us a bit more?
JB (00:17:23):
Yeah, absolutely. So, yeah, this will...I think, hopefully be a pretty fun talk. One of the core areas I've been focusing in on the last several years has been on RTOS application design.
JB (00:17:33):
So one of my goals is to try to bring some of the best practices that I know and that maybe aren't as publicly available, if you go out and read a lot of articles about real-time operating system, things like that, where you get kind of that traditional, "Okay, this is what a mutex is. This is what a semaphore is." And that kind of stuff.
JB (00:17:50):
And those are always great, but people don't necessarily show, "If I have a data structure, how can I make sure that a developer is going to be aware that it's a shared resource, and that it's protected by a mutex in a gigantic code base," or..."What's a good way to initialize all my tasks in a way that's simple and reusable and scalable?"
JB (00:18:11):
I see a lot of people, for example, go, and...if you've got 50 tasks in your thing, which would be a lot, but let's say 20, there's 20 calls to whatever the task create function is.
JB (00:18:25):
And there's little tricks you can use for creating configuration tables and things like that, that I'm going to be hopefully, well, not hopefully, that I will be showing off at the conference. So talking about..."How do you create a diagnostic task properly?"
JB (00:18:41):
Because some of these things have hooks and tendrils into a lot of different tasks in areas of memory of the system and things like that. So I'm going to try to share those best practices of, how do you properly architect these things so that you don't end up with race conditions, and long response times, and stuff like that.
JB (00:18:59):
So it should be a lot of fun. And then of course immediately after the talk we're going to have kind of a Zoom session, where we'll kind of all get together, and people will be able to ask their questions, and we'll have hopefully some good user interactions and things like that.
JB (00:19:15):
So, a lot of them ask their own questions, or share their own best practices that they have encountered through industry. So...sometimes creating content, it feels one of the many, but it's really about trying to get the industry to, everyone has their own best practices, and has an expertise that is great when they share that, or corroborate that.
JB (00:19:39):
Yeah. "I've seen this works," or, "Oh, I've also seen this other thing." Oh, maybe you haven't thought of that. So, that type of interaction is always a lot of fun.
EW (00:19:48):
And this conference, the Embedded Online Conference, you're one of the organizers.
JB (00:19:53):
Yes. Yeah. So the Embedded Online Conference, this is actually a conference that's been around. This is actually gonna be the fourth year, and I cofounded this conference with Stephane Boucher, who is the founder of Embedded Related and actually several embedded sites. So DSPRelated and things like that.
JB (00:20:11):
And, we kind of teamed up to say, "Hey, let's try to do some stuff online." And the idea is to bring worldwide expertise, and kind of bring it all into one place, and make it an affordable way for people to access it.
JB (00:20:28):
Over the last several years, and we started this before COVID, so right now, no one can go somewhere, and get access. It's very limited, and there's all the hoops and red tape to jump through. But we kind of saw an issue actually with embedded developers.
JB (00:20:47):
And that was, we were big attendees of Embedded World, and the Embedded Systems Conference, and Sensors Expo, and the Embedded Technology Conference, and stuff like that. I attend a lot of these, and we're kind of starting to see that there was a transition of developers not necessarily being able to get to these types of physical conferences.
JB (00:21:10):
We're all under the gun to get the product out the door. The managers don't want to let us leave the office, because there's some fire that needs to be put out. There's a cost for travel, and the entry fees for the conference, and things like that.
JB (00:21:23):
So, and they're starting to view a little bit more of, "Yeah, there's a lot of stuff that you can just go get for free on YouTube," and stuff like that. But it kind of turns out that you don't really get the same type of access to...some of the knowledge, real technical experts, and the industry leaders on kind of where the industry is going, and those types of things.
JB (00:21:45):
And even the latest and greatest techniques. So we kind of said, "Hey,...we can get people from around the world to be able to kind of participate in a conference, get access to cutting edge knowledge while they're still in the office." So if they get pulled away, that's okay.
JB (00:22:03):
It's not going to cost a lot if someone has to pause, or not participate for a couple of hours, or half a day, because they've got to go somewhere else. They can always come back, watch the recording, and that sort of thing. So it's been something we've been doing. It's been really exciting. It's been growing every year.
JB (00:22:19):
And to some degree, I almost don't even care as much about the growth as it is. We've been able to interact with a lot of engineers who are building really cool things and helping provide them with the knowledge that they need to be more successful.
EW (00:22:34):
You've been doing online conferences for four years. Did you look around last year as everybody scrambled to figure that out, and say, "Ha ha ha, I've done that."
JB (00:22:44):
Yeah, it was kind of funny because all the lockdowns happened in March, and we already had our conference scheduled for May. So there were a lot of people, I think, watching our conference to see how we did things. And then there was a lot of scrambling of people moving from physical conferences to online conferences.
JB (00:23:04):
I think...unfortunately, there's probably been a lot of failures in doing it, because they've tried to take your traditional trade show and just move it online. And there's not a one-one relationship there of, "Oh, I did it in a physical space. We just move to a digital space."
JB (00:23:22):
There's other constraints when developers are in the office that I think a lot of people didn't take into account. And some of these conferences just have so much content that there's no way people could actually consume the material.
JB (00:23:37):
So they've kind of gone overboard, which is why, our Embedded Online Conference, we try to focus on getting industry leaders to participate, and we try to limit the content...You see a lot of conferences that, "Oh, we have 300 exhibitors and 300 sponsors."
JB (00:23:52):
And we try to say, we're not trying to go that route. We're trying to get a couple core partners, and people who have really cool widgets that we can help, and we think have good products that will help with the conference. But we're really focusing on the end user experience and providing them with the knowledge that they need.
JB (00:24:13):
And then making sure that our partners also aren't lost, I guess, in the gigantic list of people who are participating in some conference. So there's kind of a balance there, I guess you could say.
CW (00:24:28):
It seems like the discovery problem, when you're wandering around a trade floor, you can kind of see things you want to see. And it's a little harder if you're just presented with "search for vendor," or "here's a undifferentiated list of logos" and stuff.
JB (00:24:46):
Exactly. Exactly...I attended an online conference for, it was a space systems conference a couple weeks ago, three weeks ago. And...it was one of those things, they had a hundred exhibitors. And I was very interested, but it's just a webpage you kind of scroll through with a bunch of logos.
JB (00:25:03):
And I wanted to interact, but I found it very easy to just be like, "Oh, well, I'll get back to this later." And it's been three weeks, and I still haven't done it yet. So obviously, the people who exhibit and do all that kind of stuff, they're a core piece to the people who have those, who are putting on conferences and stuff like that.
JB (00:25:25):
But, yeah, I think some of them have had a disservice by there being so many people being involved in those types of things in the conferences. And it's probably tricky for them to balance the physical versus trying to do it digitally online, because who do you say no to, if someone wants to.
JB (00:25:45):
But...from their perspective, I don't think they want to provide value back to their partners and stuff like that. Which is why we limit it for ours and try to focus in on the developers. Because that's what it's really all about, is making sure developers get what they need in order to continue to be successful.
EW (00:26:08):
I admit, I do miss wandering the show floor...For the Embedded Systems Conference, I would often arrange to meet people for lunch, or coffee, or whatever. People I only saw once a year. And just wander, and listen to their questions for the trade show folks, and get to know some of the trade show folks.
EW (00:26:33):
And that's one thing that I don't.... It's like going to the library versus buying things on Amazon. You don't really get to see what's next to everybody. The thing I didn't know I needed. But I really like that you're recreating the information part, with the lectures that give information, and the questions that other people have that can be elucidating as well.
JB (00:27:01):
Thanks. Yeah. It's definitely, I miss the show floor as well. I mean, it's one of those things that's so hard, I think, to have a digital representation of. And like you said, it's fun to wander the show floor, and as you attend these conferences, you kind of develop relationships with the people who are always there.
JB (00:27:21):
And even the attendees, like you said, you can kind of even stop, and ask questions, and kind of hear their questions. It's a piece that's unfortunately really missing.
JB (00:27:32):
And even in some of, right now, especially during COVID, it's something that is sorely missed at the moment. Especially being able to go, and actually physically grab some part, and see how it's working, and that kind of thing.
EW (00:27:47):
Beg for samples.
JB (00:27:50):
Yeah, exactly. Walk away with a goody bag full of development boards and [inaudible].
EW (00:27:57):
Dev boards, yeah. [Laughter].
JB (00:27:57):
Things that you can hook up to your computer, and go back to the office, download some software, and make cool things happen. It's definitely fun.
EW (00:28:07):
So you write software. You mentioned setting up a class, you organized this conference, and you had a book come out, last year. Do you sleep?
JB (00:28:23):
Occasionally. It's something that I've generally found gets in the way, but -
EW (00:28:29):
How do you find the time to do all of that?
JB (00:28:33):
So usually it comes down to, I mean, part of it is sometimes just not sleeping. One thing with COVID, which has been interesting, is I've been sleeping seven hours a night, which I've got to get out of the habit of doing. But a lot of it comes down to having a well-organized schedule.
JB (00:28:53):
So, on a daily basis, if I'm in the process of working on a book, I will get up at 6:00 AM. And one of the first things I'll do is I'll write for an hour, or hour and a half, before the day even starts and the kids get up and the wife and all the fun stuff. And then you get all the get to school, do all that, work out, then get back to the office, and then kind of do the normal day stuff.
JB (00:29:17):
Same thing with my blogging. It's usually, every day I'm writing. So even when I'm not working on a book, what replaces the book time is regular blogs. So 'll be blogging on Mondays and Thursdays. Monday I might do Design News. Thursday might be my own personal blog.
JB (00:29:34):
Tuesday, Wednesday, Friday might be some little section for a book. And then, like I said, I'll have my normal workday, where I'll teach a class.
JB (00:29:45):
And then I will have some meetings, and spend four hours writing some code, or thinking through some new way of... some new process, or something like that. Of improvement or something like that. But it's definitely a tight, very fast-paced type of day. Let me put it that way.
EW (00:30:03):
That requires a fair amount of dedication.
JB (00:30:06):
Yes. Yeah, definitely. Because as you develop more and more content, I'm sure you're well aware, sometimes you get to a point where it's like, "Okay, what do I write about today?" Or "What's the next thing?"
JB (00:30:19):
I guess being involved in different projects, it makes it a little bit easier, but sometimes, figuring out what the next thing is can sometimes be challenging.
EW (00:30:31):
I made the choice not to do the training classes. I was invited a few times after my book came out to give the week-long "teach everybody embedded" style class, but those take a lot of preparation. I mean, an insane amount of preparation.
EW (00:30:52):
And I mean, I would rather code than teach, which is my personal preference. Did you fall into it? Did you go after it? How did you get into teaching as part of your business?
JB (00:31:11):
Yeah, so for me, the teaching piece is something that I always, I kind of wanted to go after from the very beginning, as part of just my interest in writing and sharing knowledge...When I develop code and do things, I discover dumb things that I've done, or something like, "Oh, man, I can't believe I did that. Well, that's a good thing I want to share with someone so that they don't run into that problem again."
JB (00:31:44):
And, this is something that's gone back...to college days. I mean, I think when I was a sophomore in college, an undergrad, I started doing tutoring for physics, and just being able to help people avoid the mistakes that I'm making. I wanted that to be kind of part of my consulting.
JB (00:32:04):
So as I started, almost immediately, when I started, saying, "Okay, I'm going to be a consultant," I pretty much said, "Okay, I've got to write, and I want teach classes to share knowledge." So right from the beginning, that was something that I wanted to pursue.
EW (00:32:19):
I prefer the hands-on mentoring of a one-on-one relationship, but then you don't get to affect nearly as many people.
JB (00:32:26):
Yeah. Yeah. So it's kind of a mix. Which is why with my consulting,...I have my courses, but I also do technical advisory work as well, where sometimes I'm brought in to work one-on-one with people as well. And a lot of times the people in my classes, I mean, I keep it pretty open.
JB (00:32:45):
And right now, we do a lot of stuff online, and I try to have office hours, or tell people if you run into issues, we'll get on a Zoom session. And so going back to the time-consuming piece, it certainly can be very time-consuming, but yeah, it's rewarding, I think as well.
EW (00:33:04):
So the book that came out last year, "MicroPython Projects: a do-it-yourself guide for embedded developers to build a range of applications using Python." That's a bit of a mouthful. Although "MicroPython Projects" is probably what it's usually known as. Why'd you write this one?
JB (00:33:23):
Yeah. So I wrote this one, this is one where honestly, I don't know that I would have written this one. I actually had a publisher approach me, and say, "Hey, would you write this MicroPython projects book?" And I had been doing a fair amount with MicroPython.
JB (00:33:41):
...I think it's very interesting. It's very intriguing...I, to some degree, think it's one of those things that could potentially be the future for embedded developers. Just because so many people know Python, and the MicroPython organization has done such a great job of improving the robustness and really making it work really well.
JB (00:34:04):
And I thought, "Oh, this'll be fun to do." And, generally I like to avoid doing hands-on books. So I figured I should do one, to try one, before I said, "I'll never do one again." But it was a lot of fun, but very time-consuming, to put it lightly.
JB (00:34:24):
But it is one of those things too, where I took the approach of, as a professional, does MicroPython really makes sense? And if I were to try to use it, what do I need to know? And I tried to kind of spin that into the book, probably a little bit to the publisher's chagrin there, but I think it ended up working well enough anyways in the book.
JB (00:34:46):
So it's not just for hobbyists. I mean, a professional who says, "Could I really leverage MicroPython?" I kind of point out what was useful, what was not, what to watch out for, and those types of things. Because MicroPython has found its way into lots of products.
JB (00:35:04):
I mean, if you look at some of the, who was it, I think it's DigiLink that uses MicroPython on their radios for you to interact with and drive it. I've worked on a couple of space systems where it used MicroPython...instead of an operating system to drive the whole thing.
JB (00:35:21):
And it can be very useful, because even someone who's not trained in software can muddle the way through with Python, because Python is so simple to use. That can make it dangerous, but it makes it very useful.
EW (00:35:37):
MicroPython and Adafruit's CircuitPython are pretty different. Have you tried them both?
JB (00:35:43):
I have played with them both. Yeah. It's kind of interesting. I've used CircuitPython probably a lot less, partially because when I've used MicroPython, I'm going in, and I'm also partially modifying like the kernel and stuff like that for my own purposes.
JB (00:35:59):
But, yeah, CircuitPython seems to be to where they really lightened it up for your maker. For someone who is a maker and maybe isn't thinking of developing a product, whereas, MicroPython's a little bit stricter from that sense, but CircuitPython is a fork from MicroPython.
JB (00:36:18):
And if I recall they actually kind of collaborate or coordinate some of the development together. So it's really interesting. I mean, Adafruit's just done so many awesome things. It's really a cool company.
EW (00:36:33):
Yes.
JB (00:36:33):
And what they've done with the CircuitPython, I think is another fantastic thing.
EW (00:36:39):
And intending something to be user-friendly, it's a different goal. MicroPython is more focused on being efficient.
JB (00:36:48):
Yes. Yep, absolutely. The efficiency, kind of that nice, fine-level control, where CircuitPython is trying to really create all the abstractions, so that someone who doesn't maybe understand the underlying hardware, and all of the software engineering side of it, can actually still make things happen. So at least my understanding. Feel free to let me know if I'm going in the wrong direction for sure.
EW (00:37:14):
Makes sense to me. But you had a different book before, less hands-on. What was your previous book?
JB (00:37:22):
Yeah, so the previous one was "Reusable Firmware Development: A Practical Approach to APIs, HALs and Drivers."
EW (00:37:29):
That sounds like the answer to many questions we get.
CW (00:37:33):
Yes.
JB (00:37:33):
Yeah.
CW (00:37:33):
I'm going to start pointing them to that book.
JB (00:37:36):
Yeah. It's one of those problems that just, I feel, plagues us embedded software developers. And so I kind of took a look at it from, it can be applied to application code really easily, but it can also be applied to driver development.
JB (00:37:52):
And within that, I tried to try to share my experiences and show, again kind of how you use configuration tables, and how you can decouple your code. And how you can really create real nice layers in software, and to really try to show that reuse aspect and I think, that was a fun one.
JB (00:38:13):
It was the first book I wrote. It took a couple of years to pull that one together, and create the balance of how you write, and get something that's interesting, versus working on development work, but lots of fun. Let me put it that way.
CW (00:38:29):
I have a question that's maybe very open-ended, but why you do think that embedded systems has not got to the point where normal application development has? Where there's an application framework that everyone uses and knows about, and you just write your business specific, your product specific code.
CW (00:38:50):
Why do we continually have to worry about SPI drivers, and I2C drivers, and file systems, and network drivers, and all this stuff that's mostly done for us now, which is better, but it's sort of not completely done for us? And it's still not completely abstracted away.
CW (00:39:05):
I mean, when I'm writing software for an iPhone, I don't care about the file system. I don't care about the wireless driver. I mean, the worst thing that ever happens is Bluetooth and that's still pretty abstracted away.
JB (00:39:17):
Yep, absolutely. Yeah...I think there's probably several reasons why it's been a struggle for, I think, kind of the embedded system, the microcontroller guys. I kind of partially want to kind of fall back, and say that it's partially because originally we just worked in such a resource-constrained space.
CW (00:39:35):
Right.
JB (00:39:36):
And I think a lot of the people in that resource-constrained space were also kind of traditionally hardware developers, right? So they didn't have computer science backgrounds. They didn't know how to create decoupled, reusable frameworks and all that stuff.
JB (00:39:50):
They just knew how to go into the hardware registers, flip some bits and make it work. And for years, and years, and decades, that kind of solution worked.
JB (00:40:02):
But we've kind of reached this interesting point where the hardware has become so sophisticated and so powerful, and also the need of the users has dramatically grown in complexity in the features they want, and for someone to start from scratch anymore and kind of build all that up, it doesn't make sense.
JB (00:40:22):
It's too costly, too consuming of time. And you could bankrupt a business, or have it go under before it ever even had a chance of getting off the ground. And I think a lot of, as we've looked at kind of the adoption of ARM as a hardware platform, certainly there's other cores out there, but it's probably the most prevalent.
JB (00:40:41):
And then you have them kind of pushing on CMSIS in developing some of these layers and trying to really push that to the industry. Their ecosystem is starting to catch on and adopt more and more of that.
JB (00:40:54):
So, I mean, I kind of look at it as, I was actually just having an argument about that on Twitter, not argument, but kind of a teasing back and forth this week where someone was like, "Oh, well the HAL drivers for STM32 parts are horrible."
JB (00:41:08):
And I was like, "Well, they might be, but they work, right? So do you want to spend all that time rewriting all those drivers? Or do you want to just work on your business logic and be done with it?" So I think it's going to get there.
JB (00:41:21):
It's certainly not where it needs to be, but I think, another 5, 10 years, and the microcontroller group, embedded software developers, we're going to be developing kind of like your iPhone developers and stuff like that. So, at least that's my prediction.
EW (00:41:37):
I don't know. I had an interesting bug this week. I have a system that has to do over-the-air firmware updates via BLE, and we have a SPI flash. And the system worked. I mean, eventually.
JB (00:41:52):
Yes.
EW (00:41:52):
But I got it to work, and I got it to work securely, and I was thrilled, but I'm pretty good at testing.
EW (00:42:05):
So I was testing two images going back and forth with only the version as the change. And I noticed that I could definitely always load the image once. And of course I was confused.
EW (00:42:24):
What if there wasn't just a version change? Maybe version one had a different something than version two, but no, it was really just a version change. And the problem was that I was getting a watchdog when the flash erased it, and when I first programmed the code, it pre-erased all of the flash.
EW (00:42:46):
So my bug had to do with the physical reality of what was out there, and the timing necessary to work with the physical reality, that sort of thing is really hard to mock up.
JB (00:43:05):
Oh, absolutely. Absolutely. And that's where there's kind of the, it's double-edged, to where it feels like the traditional embedded software developer's going away, but there always has to be some of us traditional people who understand the hardware and what is happening behind the scenes.
JB (00:43:24):
Kind of almost like when you think about a normal PC, there are those guys who know how to go in, and write that BIOS, and develop the stuff to make the motherboard actually boot up. But after that, there's everybody else, who can just put their business stuff on top. But it's definitely an issue for sure. I completely see your point for sure.
EW (00:43:51):
You actually kind of answered this, but I have a listener question from John who is a student, who has been doing embedded systems for a while. And John wanted to know if it's a good idea to use HAL libraries on microcontrollers.
EW (00:44:05):
If he's in an interview, job interview, would it be bad to talk about a project using the STM32, GPIO, I2C, or SPI library? And he was really worried about if he should be writing all of the drivers from scratch, so he knew how everything worked.
JB (00:44:26):
Yeah. So I mean, what I would do, I mean, I would show that, personally, if I was being interviewed, I would still show that, yes, I'm using the STM32 HALs, for example. They can do most of what you need to, let me put it this way, they probably can do 80% of what the average developer needs to do.
JB (00:44:46):
So you can show in an interview that, "Hey, look, I understand how to use these HALs. Maybe you add an example where you extend the hardware abstraction layer, and add some custom feature, the driver or something to kind of show that you understand, you're not just using the API, but that you understand what's happening behind the scenes.
JB (00:45:04):
But I think for a lot of projects, it's perfectly acceptable to use the HALs that are provided by the silicon vendors. I would just recommend that if you are using them, you look at the code and understand their limitations, right? Because there will be limitations like with anything.
EW (00:45:20):
Yeah, not necessarily fast with the GPIO changes, STM.
JB (00:45:26):
Yeah, exactly. Yeah. If you look at some of them, some of these can be very slow, compared to if you write your own driver, where it can be extremely fast. But that's where you have to understand the limitations and look at what your end application is.
JB (00:45:41):
And, is this, if I can, let's say it's taken 10 microseconds to flip or a 100 microseconds, but I could write a driver that does it in 1 for my application, is it okay or not? And that's what should be probably that driving factor is, what's the actual requirement on the system, and does this meet the requirement?
JB (00:46:01):
Because I don't know about you, I'm almost a perfectionist with my code sometimes in my real-time systems. I've had to train myself to say, "Okay, I'd rather do it this way, but is option B good enough? And will it get the product out the door in a reasonable time and at a reasonable cost?"
EW (00:46:22):
It's the cost thing. It's how do I balance for my clients the use of their money.
JB (00:46:32):
Yeah.
EW (00:46:32):
And optimizing the GPIO drivers on STM32 isn't usually the right use of their money.
JB (00:46:40):
Agreed.
CW (00:46:40):
It's the thing I keep coming back to is, ask yourself what your job is.
EW (00:46:44):
Yeah.
CW (00:46:45):
"Is my job to write cool drivers, or is it to make what this company wants?"
EW (00:46:51):
Yes.
JB (00:46:51):
Yeah. Yep.
CW (00:46:51):
And the company didn't say, "Yes, that's a fine SPI driver. And it's great that you can transfer stuff 25% faster than the off-the-shelf driver, but also it's a thermostat."
JB (00:47:04):
Yeah, yeah. Right. So do they really need that 25% improvement? Yeah. Yeah, absolutely. Yeah. You have to be a good steward of the end customer's time and money, for sure. And I think to some degree that's where some of these discussions on whether a HAL is good enough or not, it really comes back to that.
JB (00:47:25):
And for John, that's where that could be his argument of, "Hey, I understand these. I could rewrite this to do it better, but it wouldn't be a good use of my time." And that might be something that catches an interviewer's attention. The fact that he's paying attention to his time and how he's using the company's money essentially.
EW (00:47:46):
MattyC had a question. "In your excellent book, 'Reusable Firmware Development,' there's very little mention of interrupts. Understanding that interrupts are device and architecture specific, is there anything you can recommend that can help with reuse in that realm?"
JB (00:48:05):
Yeah. So I think in the book, I covered just a very small piece of that, and kind of the core thing to look about for interrupt reuse is to create callbacks.
EW (00:48:14):
Yes. Yes, yes.
JB (00:48:14):
And in the book, I believe I showed, as part of the APIs developed for reuse, it was so little because it was like, "Hey, create a callback, make it so that you can register the callbacks with the HAL. And then make sure that when you're using a callback, that you keep in mind these are interrupts, so you must follow best practices for an interrupt."
JB (00:48:35):
And that's why that's not covered that much, because there was maybe a page or two to really explain that, show what a callback is, and that's it. But...if you look at Microchip's Harmony, you look at STM32 tools as well, some of the other ones out there that lots of people have their frameworks nowadays, callbacks is kind of the core way that we interact with reusable interrupts.
EW (00:49:01):
And a callback for me, usually frees a semaphore or sets a variable. It shouldn't do much. I mean, just because it's in a callback in your file, doesn't mean you're not still in an interrupt, and shouldn't be doing printf. So the callbacks have to be sensible and short, and not do everything.
JB (00:49:31):
Absolutely. Yep. I completely agree. That's exactly what a callback should look like. So it's real simple, quick to the point, maybe give a semaphore, maybe store data in a buffer or something, and then get back to the main application, and let it decide on the non-real-time pieces on processing it when it gets a chance.
CW (00:49:50):
When do you think we're going to get kind of built-in async capabilities in some of the HALs and the RTOSs, to where people don't have to worry about this stuff so much? Where an interrupt comes in and you have a callback, but the callback is scheduled by something, and -
EW (00:50:02):
You mean DMA?
CW (00:50:06):
No. I don't mean DMA.
EW (00:50:08):
Pretty close?
JB (00:50:09):
No, so you're kind of talking about the sophistication where the OS is handling -
CW (00:50:14):
Yes.
JB (00:50:14):
- the interrupt processing versus today where, so most RTOSs today, the model is, us developers are still responsible for the interrupts. And the RTOS is responsible for scheduling. But a sophisticated one will allow you to register those interrupts and let the RTOS handle it. So, it's a great question.
JB (00:50:33):
I know there RTOSs out there that do it now. Those are usually kind of the commercial paid for, more sophisticated ones. I think it's going to be a little while, to be honest, probably another five years or so, at least in the FreeRTOSs, and those types that are freely available to us.
CW (00:50:52):
Elecia seems confused. What I'm talking about is, instead of,...your example, the callbacks that you provide set an event flag, and then somewhere else handles that eventually.
CW (00:51:04):
Instead, I'm talking about, you provide the OS with a callback, and the callback doesn't happen at interrupt level. It deals with all of the, "I've got to send a message to user-level to have this happen."
EW (00:51:16):
Oh.
CW (00:51:16):
And then that gets scheduled in such a way that it's fast enough for most uses. But you don't have to handle all that stuff. You just get a notification that a callback happened, and when the interrupt happens, and you don't have to make all that machinery in-between.
EW (00:51:32):
You don't get notified when the interrupt happens. You get notified when the interrupt has been handled and you are in regular code again.
CW (00:51:40):
Yeah. You get notified that this happened, and now you can take some action, -
EW (00:51:44):
Okay.
CW (00:51:44):
- that doesn't necessarily at interrupt level, like you normally would, right?
EW (00:51:49):
Okay.
CW (00:51:49):
...I mean, using the word async was probably a little bit loaded, but there's other kind of uses like that, where you can put something on a dispatch queue and have it happen anyway.
EW (00:51:59):
Wait a minute. Now you're all getting into Swift.
CW (00:52:01):
No, no, that's happening in embedded. Fine. I'll be quiet now.
EW (00:52:06):
Oh, no, I mean, I hear circular buffers and all of that going on. Yes. Okay. One more listener question from Steve.
EW (00:52:16):
"For embedded hardware and software architectures, when does it make sense to separate functionality into different devices rather than using more complex (and expensive) IC that combines function? Is it only when forced, or are there good reasons to consider it...outside of physical necessities?"
JB (00:52:40):
Yeah, so, from my side, there are certainly several times when it can make sense to break code up into multiple devices. Sometimes, it can be,...a lot of times look at costs, for example. Sometimes if you're developing a product, it'll make sense to buy two of something versus buying one of something that's more complex, from a microcontroller standpoint.
JB (00:53:05):
So it might make sense to actually put two smaller chips on board, then go and buy the next family size up that maybe is more expensive. So from that standpoint,...you can use that as a decision maker, based on just cost and volumes.
JB (00:53:20):
Because buying a part in larger volumes will potentially decrease the overall cost of the product. You can, nowadays, sometimes it isn't even so much looking at using multiple parts as it is maybe looking at going to multicore.
JB (00:53:35):
There's a lot of microcontrollers now, on the high end sometimes, and sometimes not. There's things like the ESP32, and the STM32H7, that have multiple cores. The PSoc 64, for example, you can go in and for a reasonable cost get two cores that you can assign individual things to and let them have at it, so to say.
JB (00:53:59):
So I guess you just have to look at the end application, look at the costs, look at the software complexity, and decide, "Hey, is it easier for me to just separate these? Is it easier to maintain two separate projects? Or is it easier to just do it in one?" So hopefully that answers his question. We can certainly have follow-up on that, I guess.
EW (00:54:19):
I mean, there are a few other things. If you have something that's very time-sensitive and then an application that isn't. Like when you're collecting data from a laser system, and you need to get a whole bunch of data really fast, and then process it after you've gotten the chunk.
EW (00:54:40):
There's a good reason not to have the processing with the data collection, because the processing may take time, and may interrupt, or delay the time-critical piece of data collection.
EW (00:54:54):
Another one that we've seen a lot, and can be multicore is you have a really small, cheap chip to monitor a few things and then have it wake up the big chip when the user actually wants to do something. Watches, that sort of thing.
EW (00:55:13):
What else? I mean, motor controllers are often separate because again, that has a real-time component that shouldn't be mixed in with an application.
JB (00:55:22):
Yes. Yep, absolutely. Yeah. And some of the parts will actually have a, you mentioned TI earlier, they have their C2000 family, which actually has a control law accelerator. It's almost a separate core that's designed to help with motor control applications.
EW (00:55:38):
Yeah.
JB (00:55:39):
So I completely agree, when you need to separate different execution domains, as an example, machine learning and real-time processing, you wouldn't want to do those two things on the same part.
EW (00:55:52):
Yes. [Laughter]. I mean maybe, but be careful. Yeah. In a lot of the wireless ICs, BLE, and ZigBee, and all those things, are getting dual-cores because they got tired of us using the chip to do other things when they have strict BLE deadlines.
JB (00:56:12):
Yeah. Exactly.
EW (00:56:15):
Let's see, are you working on another book now, or do you have another article coming out soon?
JB (00:56:23):
Yeah. So, at the moment, I have not started on the next book. That is going to be this summer. So I have -
EW (00:56:30):
Do you know what it's going to be about?
JB (00:56:31):
I haven't decided yet. I have three or four topics. I haven't decided if I want to go down the route of RTOS application design, or tackle security, or just target kind of general best practices, but it's going to be something probably along those lines.
JB (00:56:47):
So,...I'll be starting up on that here this summer. I try to put a book out maybe every couple years. It seems to be a pretty good cadence. As far as articles go,...every week I'm publishing an article to my blog, usually pushing things up to Design News once a month or more.
JB (00:57:11):
So, I guess, some of the things I'm tackling right now is like how you develop a proper diagnostics for RTOS-based applications. And how do you do error handling and things like that, so, there'll be some articles along those lines coming out in the next couple weeks.
EW (00:57:30):
Security,...it's a tough thing in embedded. It's really easy to do wrong. What kind of thoughts do you have on that that you might put in a book?
JB (00:57:42):
Yeah, security, it's tough. Especially for embedded developers. There's a lot of good knowledge out there that's kind of being pushed to embedded software developers. One of the things I'm finding, unfortunately, is that a lot of embedded developers, or products, or companies, seem to not care about security like they should.
JB (00:58:06):
So the insurance is a little bit more lackluster than I would expect it to be, given how many of us are building Internet-connected devices. And doing it right is just hard. And it needs to be done, you have to look at it upfront.
JB (00:58:21):
And you can't attach it on the backend, because you have to make sure you select the right parts, that you've got the right software frameworks in place, and developing a root of trust from beginning to end for the whole thing. It's tough. And I think that's probably why we ignore it, because it can be costly and time-consuming.
CW (00:58:42):
And frankly, some of it's kind of new. I mean, using small values of new, but a lot of the things, like the security features in micros, have only been around for, I want to say 6 to 10 years, right? And I mean, that's fine if you're just coming up in the world and you're learning that, but for people who've been developing a long time, it's something new to kind of absorb and add to their expertise.
JB (00:59:08):
Absolutely. I mean, when you look at it, I mean, for example, the TrustZone, the ARM Cortex-M v8 architecture, figuring out how on earth do you use that properly, and the microcontroller domain.
JB (00:59:24):
And this is where some of the vendors are moving to multicore parts, just so you can have a security processor, and they can lock it down, so that us developers can't trip over our own feet. And then they give us a separate core to do our own application code.
JB (00:59:38):
So, yeah, it's interesting and tough, and it's not something that's been around or that we've had to really deal with a lot. But I mean, it's something in the future, going forward, we have to be very careful with.
EW (00:59:51):
I know in one of your books, you recommended the SWEBOK, the Computer Society's Software Engineering Body of Knowledge. Do you still recommend it? Are there newer versions?...What is it about?
JB (01:00:11):
Yeah, so that's a great guide that the IEEE Computer Society put out, and they periodically update it. I think it's currently at version three, the last time I checked.
JB (01:00:21):
So unless there's been an update in the last couple months, but I still recommend it, because I think it provides a really good overview of best practices from a general software engineering standpoint for developers to follow. And, it's free.
JB (01:00:35):
All you have to do is, like anything nowadays, put in your contact information, they send you a free PDF. But there's a lot of good references. I mean, they had dozens or maybe a hundred different people put input into it.
JB (01:00:48):
It's just a great resource, I think, for developers. Some of this stuff obviously you can overlook because it probably doesn't apply to embedded, but kind of the general software engineering best practices are fantastic.
EW (01:01:00):
And it's best practices like, everything from working in a group, to ethics, to design, construction, testing.
JB (01:01:15):
Yep. Requirements, testing. Yeah. The whole gambit, which is why I think it's so great, because it's a good overview of kind of all the major pieces of software engineering.
EW (01:01:23):
I mean, it's a little intimidating, but it is also really impressive in its, if I did all of this, I would be a better software engineer.
JB (01:01:34):
Yeah, I would too. So...yeah, it'd be difficult to probably do all in, I don't know, individually, but it definitely gives you something to shoot for of, "Oh, I should probably be doing this," and "Oh, I could be doing that differently" or, "Oh." It gives a new perspective on some things.
JB (01:01:51):
And sometimes it's just a good reminder as we all develop software, and evolve, and kind of change our processes, sometimes it's good to go back and review what other people are doing too and say, "Oh yeah, I forgot about that. I probably should be doing that."
EW (01:02:05):
And I do need reminding. I mean, I will happily have, trace, for example, I've used it before, but I haven't used it in my current project because, well, because we didn't need it. And yet as you were talking, I did look up what the trace would be for this processor.
EW (01:02:28):
And "Hey, look, I already have that software." Stuff like that, if you don't use it, there's a good potential that you'll lose it. And so it's good to just look a little bit. You also recommend something from Renesas, the "Synergy Software Quality Handbook?"
JB (01:02:52):
Oh, yes. Yeah...their quality handbook, I was actually really happy when they published that because I think it shows a really good set of metrics. And a simple process that, maybe it's not a simple process, but something for, I think, developers to target and try to do themselves.
JB (01:03:13):
It kind of shows a nice, sophisticated, well-thought-out process of how they develop software, how they test it, the metrics that they use to show that, yeah, this meets quality.
JB (01:03:24):
They show an example of how they do their automated tests, and continuous integration, and the reports that they generate on kind of the other side of that, to show whether the tests are passing and failing, and how they rate that.
JB (01:03:38):
And to me, it was just fascinating. I thought they did a fantastic job with that. And the fact that they shared it with everyone, it's like, "Hey, I'm not saying let's steal it, but let's look at that as a really good example of some practices that we can be leveraging ourselves."
EW (01:03:55):
And you don't have to do, they have nine different reviews from market requirements to code to managerial, you don't have to do all of them. Just recognize that maybe if you were building a satellite or a medical device, those would be useful. And which parts of it can you use in your current code?
JB (01:04:18):
Exactly. It's a perfect thing to look at it and say, "What can I leverage from this?" And you look at some of the, the over processor, for them, they have so many stakeholders, you know? But you can look at, like I said, if you look at their metrics section, or the reports that they output when they test it, or how they interface their static analyzers to do hardware, and loop testing, and stuff like that.
JB (01:04:39):
There's some interesting things in there that you can probably look at it and say, "Well, maybe I don't need to go that far, but where can I start and maybe improve my own design using some of these types of capabilities?"
EW (01:04:54):
I was thinking about stakeholder meetings recently, or stakeholder sign-offs, when the whole, one of my systems is going to start production, very first production line. And, we found out that the schematic revision was not the last thing, the last schematic revision, that had been given to firmware.
EW (01:05:17):
And so we didn't know what had changed. All we knew was that it had a version change, and there was a little bit of freaking out because "What are you doing changing the hardware at this late date without telling us?" And I was just like, "This is how we end up with meetings where everybody has to sign off before anything can happen."
JB (01:05:38):
Oh, absolutely. It's one of those scary things for sure.
EW (01:05:44):
It's been really nice to talk to you, Jacob. I know you have a coupon for the conference. Can you share that with us?
JB (01:05:51):
Yeah. So yeah, if anyone's interested the Embedded Online Conference, they could just go to embeddedonlineconference.com and in the coupon, in the registration area, they can just put [Redacted. Contact the show if you want it.] And that should give them a nice discount.
JB (01:06:05):
You guys can register for $60 versus the, end of this month, I think it goes up to 190 registration. So you can get in for $60 within embedded.fm.
EW (01:06:17):
Month is March or February? Just checking.
JB (01:06:22):
It's going up at the end of February to the 190, but the coupon code, embedded.fm, will let listeners register at $60 through March.
EW (01:06:34):
Thank you.
JB (01:06:35):
So, yeah. Yeah. Thanks for asking. That wasn't very clear there.
EW (01:06:39):
Well, this is going up in March, so there's confusion.
JB (01:06:43):
Okay. Perfect. Yeah, yeah.
EW (01:06:43):
Timey-whimey.
JB (01:06:45):
Gotcha. And I don't like promotional stuff anyways, so I talk really fast around it.
EW (01:06:50):
And this is happening May 18th, 19th, and 20th. And what time zone is it?
JB (01:06:57):
Yeah. So it's going to kick off, what we're doing is the European talks will actually start pretty early. I think we have our first talk at seven or eight Eastern Time...But the European speakers will talk first. Their recordings are available immediately.
JB (01:07:15):
So, for people on the West Coast, they don't have to necessarily worry that they're going to miss it. They'll be able to get back onto the recording and things like that. So it kind of starts a little early, but it goes all day to four in the afternoon.
JB (01:07:30):
And after each talk, we'll have a little Q and A session with the speakers, and try and get everyone together periodically throughout the day in case they did miss a Q and A, they can still ask questions.
EW (01:07:39):
Four o'clock in the afternoon, Eastern Time.
JB (01:07:41):
Yes. Eastern Time. Yep.
EW (01:07:44):
Jacob, do you have any thoughts you'd like to leave us with?
JB (01:07:48):
Yeah, I think the thought is, when it comes to software development, I think it's great for people to try to look at what other people are doing. Try to always be expanding your knowledge.
JB (01:08:01):
Kind of, I guess, a recommendation is, I like to use one of my lunch periods every week to kind of make sure I look around to see what the latest and greatest stuff is. So my recommendation, I guess, is, don't stand still. Make sure that you're constantly evolving the way you do things. Otherwise, you could get left behind.
EW (01:08:19):
Our guest has been Jacob Beningo, an independent consultant and lecturer who specializes in the design of embedded software. He has authored three books, his most recent is "MicroPython Projects."
CW (01:08:32):
Thanks, Jacob.
JB (01:08:34):
Thanks, guys. Thanks for the invitation. It was a pleasure speaking with you.
EW (01:08:37):
Thank you to Christopher for producing and co-hosting. Thank you to our Patreon supporter Slack group for some excellent questions. And thank you for listening. You can always contact us at show@embedded.fm, or hit the contact link on embedded.fm.
EW (01:08:53):
And now a quote to leave you with, from Admiral Grace Hopper. "Humans are allergic to change. They love to say, 'We've always done it this way.' I try to fight that. That's why I have a clock on my wall that runs counter-clockwise."