Transcript from 508: Descartes' Demon with William Griffin, Christopher White, and Elecia White.

Welcome to Embedded. I am a Elecia White, here with Christopher White. Our guest this week is William Griffin. We will be talking about simulation and hardwares and models and math and probably books. Probably a lot of books.

CW (00:00:23):

Hi William. Welcome.

WG (00:00:25):

Yeah, there are definitely going to be a lot of books. Thanks for having me.

EW (00:00:29):

Could you tell us about yourself, as if we met at Supercon?

WG (00:00:36):

Oh. Well, I am a huge fan! <laugh>

EW (00:00:40):

<laugh> Worst way to start a conversation with me.

WG (00:00:45):

What I'd say if I met you at Supercon. Yeah. Let us see. I am an engineer. I have been working with embedded control units for vehicles for about 20 years. Done a lot of hardware-in-the-loop test systems, a lot of LabVIEW, and teaching those things.

(00:01:02):

Similarly, I have done a lot of model-based system design. My background is in physics, so applying that to help with developing control systems.

(00:01:14):

Also a fair amount of engineering leadership strategy type things. Founding and leading teams. Helping create some innovative programs to align engineering business and strategy. Help build up software craftsmanship across an engineering organization. I guess that is the big thing I am really passionate about helping build. Collaborative engineering cultures.

(00:01:42):

Just now launching a new company with a longtime friend of the show, Bailey Steinfadt. It is called "Spark Embedded." Yeah. We want to create a place where we can have collaborative teams of experienced engineers, that are helping to take on those tough challenges. Maybe apply some of this MBSD and HIL, not just for these big companies, but for smaller and medium sized businesses. Yeah, give back to the community.

EW (00:02:12):

All right. Since you do know the show, we are going to go straight to lightning round. Are you ready?

WG (00:02:18):

Is anyone ever ready for this?

EW (00:02:20):

This is the fun part.

CW (00:02:23):

<laugh>

WG (00:02:23):

<laugh>

CW (00:02:23):

Favorite type of vehicle to control?

WG (00:02:26):

Boy! I like to fly spaceships in video games.

EW (00:02:32):

Favorite simulation software?

WG (00:02:34):

I use Simulink a lot.

CW (00:02:38):

Is SimCity a simulation?

WG (00:02:43):

Why not? Yeah, I would say so.

CW (00:02:45):

Is a Tamagotchi-

EW (00:02:47):

A simulation?

CW (00:02:47):

A simulation?

WG (00:02:50):

Very low fidelity simulation.

CW (00:02:52):

<laugh> Okay.

WG (00:02:52):

Favorite programming language?

(00:02:57):

I think I am most comfortable in LabVIEW from over the years, but I first learned on MATLAB.

CW (00:03:06):

What is the most amazing thing you have seen or done with LabVIEW?

WG (00:03:10):

There are two questions there. You are making me struggle.

CW (00:03:13):

Well, you can pick one. <laugh>

WG (00:03:17):

<laugh> One of my prouder achievements is helping to set up a hardware-in-the-loop simulation lab, and develop the original architecture for that, for a major vehicle tractor company that probably everyone has heard of, but I probably should not say the name. Yeah. That has turned into now they have got hundreds of simulators for all sorts of different vehicles.

EW (00:03:42):

Complete one project, or start a dozen?

WG (00:03:45):

I am definitely a mono focus person. One, two projects. And then I say that and I am like, but I am usually reading five or six books at a time.

EW (00:03:56):

That does not count. Those are different.

WG (00:03:58):

Okay. Okay. Fair enough.

CW (00:03:59):

Favorite fictional robot?

WG (00:04:02):

So tough. So Elecia knows I am a big fan of Murderbot. We have Murderbot parties here. But I guess my favorite is Data from "The Next Generation."

EW (00:04:13):

And, do you have a tip everyone should know?

WG (00:04:15):

I have one that I applied before we started talking. Have you guys ever heard of the hungry judge effect?

CW (00:04:22):

No.

EW (00:04:23):

A judge is more likely to convict you if it is right before lunchtime. Or less likely to parole you, I think is where the tests were done.

CW (00:04:31):

I see. < laugh>

WG (00:04:32):

Yeah, exactly. So if you got something you want to be mentally present for, take a break beforehand, have a sweet snack or something to give you some energy, so that your brain is on for that conversation.

EW (00:04:48):

This is similar to my, if you totally want to derail a meeting, plan it for about two o'clock in the afternoon, and bring chocolate cake. The sugar rush is fast, and everyone crashes.

CW (00:05:01):

It is a lot faster if you bring whiskey.

EW (00:05:02):

<laugh>

WG (00:05:06):

Or it turns into an all-nighter.

CW (00:05:07):

<laugh>

EW (00:05:12):

<music> I have a quick sponsor announcement before we jump back in. If you are interested in AI's impact on design engineering, Mouser Electronics has some great resources to check out.

(00:05:23):

Their Empowering Innovation Together platform is diving deep into AI powered engineering. Think faster prototyping, smarter automation, and more sustainable production. It is about helping engineers unlock the full potential of artificial intelligence in real world designs.

(00:05:42):

You will find podcasts, expert articles and videos that keep you informed and inspired. Sound like your thing? Head over to mouser.com/empowering-innovation and explore the latest content. <music>

(00:06:00):

Okay. So simulation and hardware-in-the-loop. Those are two separate things that you have mentioned, but they are different. Could you describe hardware-in-the-loop, as though we had not all listened to Komathi's episode a few months ago?

WG (00:06:20):

Yeah. Sure. Hardware-in-the-loop. Essentially what we are talking about there usually, is you have got some piece of software- In my field, it has been embedded control units. But you have got some piece of software, and you are actually putting it on the hardware on that ECU.

(00:06:41):

You want to trick the ECU into thinking it is actually on a vehicle out in a field, driving things around or controlling a transmission or an engine. So you give it all of its inputs, and you give it all of its outputs. You do your best to convince it that it is out there doing real stuff, and then see what it does.

EW (00:07:02):

Where do you get these inputs and outputs? Do you record them from actual events? Or do you simulate them from physical models?

WG (00:07:13):

You can do both. What I would say with the recording- This gets into the essence of in-the-loop, when we talk about hardware-in-the-loop, right? With a recording, you have got your input that you are feeding the recorded data, and you are feeding that to your controller. The controller does something, and has some sort of output. But there is no loop.

(00:07:35):

That output does not affect the recording, right? It does not dynamically change what happens in the recording. So if it says, "Turn left," and the recording was for a vehicle turning right, well it is still going to turn right. You have got an open loop test there, right?

(00:07:52):

With an in-the-loop simulator, now you are dealing with something that actually closes it. That usually has, like you said, like a- We call it a "plant model" or some sort of emulation of the physics or the environment, maybe other systems that it is interacting with. If it is a transmission, then an engine, something like that.

(00:08:14):

That is a part of this loop. So the outputs of your controller are measured, read, fed through this model. Then this model has its own outputs, that end up generating analog signals or encoder signals, whatever you are using. That goes back and actually gets fed as electrical signals to the ECU unit. So that is the whole loop in hardware-in-the-loop.

EW (00:08:45):

One of the problems with testing- Wait. When we talk about control units, we are talking about PID control and LQR control. Things that are not a simple in and out. There are delays and phase shifts. The inputs and the outputs are not trivially connected, not linearly connected. Right?

WG (00:09:13):

Yes. Oftentimes. But not always. You do not have to be doing those complicated systems, to be able to try to do some in-the-loop, get value out of in-the-loop simulation. But certainly you get even more value, when it is those dynamic kinds of interesting control algorithms.

EW (00:09:38):

When I have done hardware-in-the-loop with recorded data, it has been to do regression tests, to make sure that the controller, control unit, has not changed from what we expected. Has not gotten worse.

(00:10:00):

And when I have done the simulation, it has usually been to test the simulation, test the control unit more as a first time, first pass. Often is a way to make sure my hardware-in-the-loop is acting similarly to how my physical unit should be working.

(00:10:26):

You put in an impulse and look at the squiggles that come out, for the physical system, and then for the simulated system. Then you try to mash them, so that they manage to make the same frequency and level of oscillations in your control.

(00:10:39):

Are they both valuable for you? Or do you think the simulation is really the only way to go?

WG (00:10:48):

Oh, gosh. So what you are saying is you are feeding back, based on how it operated in the real system. You are feeding that back into the plant models, to update them and make them better reflect reality. Is that right?

EW (00:11:06):

Well, the plant models here are inside the control unit? Or the plant model is the simulated system? I thought you called the simulated system the plant. I have always hated that word, and there is just-

WG (00:11:18):

Yeah, and maybe I am misunderstanding your question here. Yeah, yeah.

EW (00:11:20):

There is a specific reason I hate the word "plant" in this context, but that is unrelated to the world here. It has always been because, when you talk about a system, which part is the plant always confuses me.

(00:11:37):

Is it the whole system? Or is it the system that the control unit is responding to? Or is it the control unit itself? Because I have seen it used in all three places. Which just makes me want that word to go away, because I do not understand what it means when you say it.

(00:11:55):

So, having confused probably myself, the listeners, Christopher and William, where were we?

WG (00:12:03):

I do not think I got your question quite right then. So could you...

EW (00:12:09):

Okay. So you have the control unit that is running our software, maybe running a PID loop, maybe just running a recorder, data recorder or some form of control.

CW (00:12:22):

This is the system under tests?

EW (00:12:24):

This is the device under test, the system under test.

CW (00:12:26):

Okay. This system that is being the hardware part of the in-the-loop.

WG (00:12:29):

Mm-hmm.

EW (00:12:32):

This is the hardware we have developed.

CW (00:12:33):

Okay. Okay, okay.

EW (00:12:33):

Now we can either hook it to a recording of the data that we got from the field-

CW (00:12:40):

And play it back

EW (00:12:41):

And play it back. Which William pointed out that if something is miswired, like left and right are opposite, or your algorithm has improved so that you get a different response, you will not get that different response in recorded data. You will just have a different output.

(00:12:57):

Alternatively, you can take all of those wires that go to your board, and instead of playing recorded data into them, you can add them to a simulated system that actually physically models the real world. Like, if this is a car or a tractor, you would have your device under test, and then you would model the transmission and the steering and the rocks in the field. I do not know.

CW (00:13:23):

It is Descartes' Demon for hardware.

EW (00:13:26):

What is Descartes' Demon?

CW (00:13:27):

That is the conjecture that there could be a demon that actually is giving you all your visual and sensory inputs, and that the real world is simulated by an evil demon.

WG (00:13:37):

It is "The Matrix" for your hardware.

CW (00:13:39):

Yeah.

EW (00:13:40):

Okay. Okay. Yeah. Okay. So, yes. I was comparing the value between these two. And whether there is value in both.

(00:13:50):

And which part of this whole system, between the simulation system that does the physics, the device under test control unit, or the whole thing, is called "the plant." Or if the plant is something else on your desk, that is green, that you should not pour as much coffee into it as you do. But, you know, what else are you going to do with your coffee?

CW (00:14:11):

Drink it?

EW (00:14:13):

It is cold!

CW (00:14:13):

<laugh>

EW (00:14:16):

Sorry. Where were we?

CW (00:14:18):

I think you were formulating a question. <laugh>

WG (00:14:21):

I think I got the question.

CW (00:14:22):

Okay, good.

WG (00:14:24):

I think what you are asking is, "Are both of these types of testing valuable? This open loop testing, and closed loop testing?" And I would say, "Yeah. Duh. Of course. Any testing is better than no testing, first of all. Right?"

(00:14:37):

This is something- I read it. We promised books. I read an interesting book my mentor Tim Gifford had recommended, maybe a few months back, called "How to Measure Anything." One of their biggest points is you should measure things. You should not get into this mindset of, "If I cannot get the absolute perfect test in place, then I might as well not even bother testing at all, right?"

(00:15:06):

Usually whatever that first measurement is that you take, gives you the most information. You do not have to get to this place where it is all perfect fidelity and high statistical significance and things like that. You can still get useful feedback even from low fidelity testing, that helps you find those big mistakes early.

EW (00:15:31):

Except there is a cost to this.

WG (00:15:36):

Yeah.

EW (00:15:38):

Having a bad hardware-in-the-loop, having a complicated simulation, can take away engineering time from developing what could be a better solution. How do we measure- <laugh> How to measure anything. How do we measure the value of our simulations?

WG (00:16:01):

Yeah. There is no real clean answer here. In theory you would say, "Well. I just want to have the absolute best, highest fidelity model, that represents the universe precisely down to the quantum level." That sounds all nice, right? But we are not doing Descartes' Demon. We are not doing "The Matrix" here.

(00:16:31):

That is expensive. That requires really good hardware. It requires a lot of knowledge and a lot of time and a lot of expense. And we have got businesses to run, and products to get out the door. That is just not realistic.

(00:16:46):

So we need to figure out what is the right amount of fidelity there. It is just kind of an art. Just have to ask you, "Well, how important is it?" So if you are doing a PID, again, depending on the time aspect of your PID, you may need higher and higher fidelity to deal with that.

(00:17:09):

But if you are doing something that is simpler, you may not need a lot of fidelity. It may be good enough to have a very simple basic mathematical model of your application, without going into that level of detail.

(00:17:23):

Or if you are doing functional safety, that is a place where you might need higher fidelity, in order to get your certifications.

(00:17:29):

Or sending a product out into space, where you cannot test it. I mean the one time it goes out, is the only time. So you better have a pretty good model, and have it pretty thoroughly tested in simulation, before you send it out. Or you might waste a lot of resources. Does that make sense?

EW (00:17:52):

Yeah. This came up in a conversation recently, when I mentioned that my analytic models, matched my physical models, but neither one of them matched my simulated system. And someone asked, "Well, why bother with the simulated system then?" And I said, "Because I can throw the simulated system off of cliffs. If I do that with the actual model, people get mad at me."

(00:18:16):

So there is a value to the simulation beyond the obvious. That you can do things that are too expensive. Or too time consuming, because you can often run a simulation at multiple X of actual time.

WG (00:18:40):

Yeah. Yeah. There are a lot of really good things beyond just the testing. I mean, the testing is great. You can measure things that you would have a hard time measuring in reality. Because if you are in a model, you can see all of these- You can see all the pressure and the flow, or what is going on with current and things like that, that you might not easily be able to measure out in the field.

EW (00:19:05):

Yeah.

WG (00:19:06):

Do these dangerous things. And you can do it early on. But I also do not think that is the only thing that you get out of it.

(00:19:12):

Sorry. You were going to jump in there, and I went right over you.

EW (00:19:17):

No, no. Yes. You can measure more things than you necessarily can do on your physical model, because you can stick on some simulated sensors, and they can tell you what is happening. Like you were saying with pressure.

WG (00:19:33):

Yeah. Yeah. This is one thing- So people talk about MBSD, and I want to get my terminology straight here, because different people mean different things. When I talk about MBSD-

EW (00:19:44):

Wait.

WG (00:19:46):

I focus on- Oh.

EW (00:19:47):

Model-based system design.

WG (00:19:50):

That is what I was going to get at. People use a different thing for "S" in different situations.

CW (00:19:55):

Oh, okay.

WG (00:19:56):

There is model-based system design, and there is model-based software design. I hear a lot of software design, but I feel like that limits our mental scope of what we are actually trying to do here.

(00:20:09):

If you think of it as system design, you can develop these control models, you can develop these plant models, these physical models, before you ever get any hardware, or even settle on what your product is going to look like. Then you can start to play with both of them. Not just the control model, and making the control model work the way you want it to work.

(00:20:32):

But also you can say, "Well. If we change this aspect of the system, or you made it a little bit different in this way, well then we can control it better. Or we will get this extra feature out of it. Or things like that." So it is more like when we talk about model-based system design, we get this back and forth between the two. That is really, really powerful, and it happens earlier in the cycle.

(00:20:55):

We talk about in software, "Shifting left." Being able to test things early on where it is cheaper, where you have got more chance to influence the outcomes. Before your product gets built, you can already start testing things, and trying out ideas, and making suggestions for changes.

EW (00:21:16):

I think it is funny you say that, because most of the time when I get started with that and then I say, "Well. What about another sensor here? It will really make the system a lot more stable." The answer is always, "No. That is too expensive."

(00:21:27):

Or when I say, "Okay. How much does it weigh? Because that is really important to my calculations," they are like, "Oh, well, somewhere between 30 pounds and I do not know, maybe 250?" I am like, "That is not a range. That is just everything!"

WG (00:21:48):

<laugh> Sure, sure. Yeah. I definitely think of it from a perspective, Agile or Lean MVP type thing. Minimal viable product. These things are- It is iterative, right? So they give you a big range. "Well. Okay. Give me- At least try to give me-" Sometimes I will use the phrase like, "Your 90% confidence interval. You are pretty sure. There is less than a 10% chance it is outside of this range."

(00:22:21):

But that is going to change over time, and we are going to adjust and iterate. The control system will have to change to adapt to that. That is the reality of dealing with systems.

(00:22:32):

The opposite would be worse, honestly. They tell you, "Well, this is the exact weight," and then you say, "Well. Okay. We need to make this one change, though." And they say, "No. We already told everybody this is the exact way. We cannot make any changes." That is way worse. <laugh>

EW (00:22:49):

What kind of systems can best utilize this model-based system design? You mentioned tractors.

WG (00:23:04):

Yeah. Sure.

EW (00:23:05):

Definitely robotics of just about any kind.

WG (00:23:09):

Yep. Yep, those are big ones. I worked primarily in- They call it "the off-highway vehicle industry." That is construction and forestry and mining and tractors. I guess that is kind of the big ones. That is where we used a lot. I know automotive uses it a lot. I have talked to colleagues that use it in space- Or, space technology, I should say. They do not actually send their HILs to space.

(00:23:46):

Yeah. I do not know. Usually it is more complicated systems. But I do think there is more room for smaller and medium businesses to be able to take advantage of these. Obviously you still have to weigh the costs, the pros and cons of investing in this. But the idea of being able to try out your algorithm and see how it works, and see what goes wrong with it early on and adjust the product plan-

(00:24:14):

Oh. I missed medical devices. That is a really great place for it too. These things that are fairly complicated, and have to get right. But it is a powerful tool, that can help others too.

EW (00:24:29):

What about things like Fitbit? I do not mean Fitbit itself. I mean like a consumer product that has a limited number of inputs and a complicated interface, and a complicated BLE interface.

(00:24:45):

Do you think those have a good place for the model-based design? Or does that not have enough algorithmic interest, and should be tested under TDD or other forms of testing?

WG (00:25:06):

Sure. I do not have a Fitbit. I have never had a Fitbit. But I think they have got, what, an accelerometer in them. You mentioned some Bluetooth.

CW (00:25:15):

User interface. Maybe one or two other sensors.

EW (00:25:19):

Some sort of screen. Yeah. A couple of sensors.

WG (00:25:21):

Yeah. Which I have actually hooked up user interfaces to my HILs before. If you think of a tractor, they have got some sort of control system in there. We have actually even gone so far as to set up multiple monitors, and make a fake cab for the tractor, and have people sit in the cab and control it like they would. They are controlling a virtual machine moving around. It is kind of fun.

EW (00:25:46):

Is that pure simulation? Or does that actually have hardware there?

WG (00:25:49):

Yeah, it has actual hardware. So we would be doing it with hardware-in-the-loop. We would simulate actually- We would actually put all of the controllers, or at least all of the ones that were most important, into that simulator. That would allow us to essentially simulate the whole vehicle, without having the vehicle there. See problems and communication between the two.

(00:26:13):

You mentioned Bluetooth. Again, from my background, I have done a lot more controller area network, CAN messaging. In that case, yeah, we have definitely simulated that.

(00:26:24):

And the delays that come from that. Because there is some time to measure things, some time to send a message, receive it, and complete the loop sending the control algorithm or control value back out, to the thing that is actually doing the controlling and telling it what to do.

(00:26:43):

That is a case where we are talking usually tens, maybe hundreds of milliseconds, but still valuable to be able to see how that affects things, and if we can reduce CAN traffic and still get by with a good control.

EW (00:27:02):

You show it to humans for some in-the-loop simulation.

WG (00:27:08):

Mm-hmm.

EW (00:27:08):

Have you ever done it so that you end up with a camera that then has to parse it, and use that as part of the testing mechanism? "No," is fine. I am not trying to trick you. I am just wondering if you have.

WG (00:27:28):

I am just trying to understand what you mean by having a camera, and trying to parse the camera.

EW (00:27:32):

Let us say your tractor has a small display, four by six-ish inches. For those of you who want to know four by six of whats. You can then have a camera look at that, and make sure that as you are doing testing, it is showing the right thing.

WG (00:27:57):

Yeah, I have never done that as a hardware-in-the-loop test system before. But I have had team members that have worked on vision systems like that. That are monitoring the AV system for, in this case, I think it was motorcycles.

EW (00:28:17):

It goes back to the question of how far do you measure? What do you measure? What do you simulate? Which part is the hardware-in-the-loop, and which part is the loop?

WG (00:28:29):

Yeah. Yeah. That is something we have brushed over here, but hardware-in-the-loop is just one kind of simulation. Actually, if we are doing model-based system design, we are probably starting with what is called "model-in-the-loop." When we say all these in-the-loops, it is very confusing, so let me just get that right out of the way. Whatever that first word is, that is the thing that is under test.

(00:28:52):

So when we say, "Model-in-the-loop," it is like a control model. You are just working on your computer at this point. You have got both your plant model and your control model on your computer. They are connected up. Maybe your control model is set up to run a fixed step sort of ticks, like it would on your controller.

(00:29:12):

But then maybe your plant model has real physics. And it has got variable time steps, so it can solve huge systems of differential equations and get realistic outputs. Like real physics, which does not really operate on a tick basis, at least as far as we know. Get down to the quantum level and maybe that will change.

EW (00:29:36):

In this system, if we were talking about different implementations of this, your control would be your PID loop. Let us call it that, because it is easier to think of that. The plant here would be Gazebo or Webots or Simulink, or something that simulated the physics of the real world, and interacted with your model.

(00:30:05):

Your control unit through whatever mechanism, whether it was by simulating physics, or maybe writing to and from files on your disc drive, so that it could read them back and forth. Is that right? Am I getting it?

WG (00:30:28):

Yeah. For the most part. I wanted to nitpick at a few words, but I think I will just let it slide for the moment.

EW (00:30:33):

Mm-hmm. No, go ahead.

WG (00:30:33):

This is all happening in Simulink. So your control model is developed in Simulink. And your plant model is developed in Simulink.

EW (00:30:42):

No, no. Let us be realistic. Nothing I am developing is in Simulink. <laugh>

WG (00:30:44):

Okay. Okay. I should be cautious. My MBSD has been in Simulink, but there are other tools out there. I do not really know them. That is not where my career has been. I would be curious to hear more about that. Yeah.

EW (00:31:02):

Well. See. My point with my modeling not in Simulink, is my model is likely to become the thing that runs on the control unit in the vehicle. I usually start very early in C, and have it talk to Webots, Gazebo, Simulink, another Python script that would simulate whatever I needed. <laugh> Or even an Excel sheet.

WG (00:31:37):

Oh, yeah.

EW (00:31:38):

So yeah. My model is never in Simulink, because I not really use MATLAB very much.

WG (00:31:44):

Mm. Well. Let me put in a little advertisement for it then. One of the things- Models, they are abstractions from reality. They are simplifications. When we talk about control models, it is kind of similar. We get some extra benefits out of that, versus just jumping right into C code.

(00:32:05):

First of all, the control model, it is usually focused on what is the application doing. That PID loop, it is abstracted from the operating system and all of those things. Which has a nice benefit in that your mind is allowed to focus, to keep it simple stupid.

(00:32:28):

To have that ability to focus on the application logic, and not worry about all of these little day-to-day bugs. "Oh, I did not declare that as a non-volatile," and things like that, that can really trip you up when the code gets in the way.

(00:32:45):

That is really nice for your stakeholders too. Maybe you are talking to mechanical engineers, electrical engineers. Even pull up Simulink for business people and show them, "Okay. Here are the kinds of things we are doing. Here is the data coming out of it. Here is how we analyze the results. By the way, you can have this data. And you can use some of these tools to analyze this, and see if it is doing what you want."

(00:33:08):

The last thing I wanted to mention about that, is the portability side of it. In that we are dealing with an abstraction of the controls. The nice thing is that is not hardware dependent. So your model becomes the source of truth.

(00:33:23):

We make a distinction here. We say "model" deliberately. It is not code. That is because we actually generate code, from the control model. So we set up a special system target file type of thing, and use a target language compiler. We actually generate C code, and then-

EW (00:33:45):

<laugh> I have seen that C code. Do not be proud of it.

CW (00:33:47):

<laugh>

WG (00:33:48):

It is not meant to be read. That is not meant to be- I mean, you can read it. There are better ways to set it up, and worse ways to set it up, for sure. For sure. A good modeler will help to make that more readable.

(00:34:01):

But the point is, the source of truth is the model itself.

CW (00:34:06):

Ah. Okay.

WG (00:34:07):

So then if you want to port your application to a different controller, a different board or a different processor, you just have to change that system target file and that target language compiler. All of your logic still tested, still good, is able to move over. You just generate a different C code for it, which can be really powerful.

EW (00:34:33):

It is not fair, because I started early with MATLAB. Then it was always too expensive after about year 2003. I do not know what the state of it is now, and I would never go back to MATLAB. I would always go for whatever the Python version is right now, which- Not SciPy. It is not Overlord, Overtone. Over something. There are a whole bunch of kits.

CW (00:35:09):

You are looking at me to help you, but I-

EW (00:35:10):

There are a whole bunch of control kits. Anyway. Would not go back to MATLAB because it is expensive, and I tend to work with smaller companies. It is true that when I last showed some of my Python simulation code to some business folks, they did run from the meeting pretty much screaming, which was sad. I did not think it was hard code.

(00:35:35):

I understand- You say, "You just change the target," and I say, "If it is in C, you do not have to change the target. It is just C. You can run it on your computer and have it deal with files. Or you can run it on your Arm Cortex-M3. Or your PIC32. Or PIC8." It is the same code in all of them, as long as you have your variables sorted, which unfortunately I do naturally, so I do not worry about that.

(00:36:14):

To me, it is more confusing to have the model, because you say, "It is a model of truth," and I say, "It is a model of truth on your computer, where mine is a model of truth on my microcontroller." Which is not really- <laugh> Neither one is true. There is no truth. Sorry, you got me reading about Descartes' Demon and now I am sure there is nothing real.

WG (00:36:40):

There is no truth!

CW (00:36:44):

A slightly less- Okay. The way I would look at this is the Simulink is the model. It is the thing that you are making the control system out of, and it produces C code. And the C code should be treated as you would treat the object code.

EW (00:37:04):

Yeah.

CW (00:37:04):

And stop worrying about it.

EW (00:37:06):

Right.

CW (00:37:07):

As long as you are doing the right things with testing, and the model appears to work in the model situation and it appears to work on the target, and everybody is happy, I do not think there is a problem with that.

EW (00:37:18):

It is a top down approach, which totally works.

CW (00:37:21):

Right. Right. For certain customers and systems, I think that is way more appropriate, than the things that we traditionally do with embedded, which is start at the C code or start at Python. But if you are making a gas turbine engine control system, I do not know that I want to start with C. <laugh>

EW (00:37:41):

But I have all these nice libraries. All I have to do is type in "PID" and it does everything it is supposed to do. It even differentiates properly, sometimes.

WG (00:37:49):

Yeah. I wanted to come back to one thing that you said earlier, Elecia, which is the cost. You said you have been working with smaller companies. One thing that I would point out too, is that I think there is some sort of startup program that MathWorks has, that will get it really cheap for you. Well, really cheap is relative, right? It is not free.

(00:38:13):

But when we are talking about big companies buying these licenses, you are measuring in the tens of thousands of dollars. So not cheap, let us say. But you can get it down into, I want to say 4,000 for every license that MathWorks has. If you are dealing with a startup that has- I do not know. They have got some criteria there, like less than five years and less than a million in revenue, less than ten developers I think. Something like that.

EW (00:38:46):

I know you are starting a consulting business. The one thing about a consulting business is we are not a startup, and yet we are the ones who need to have access to these tools. Yours will be for five years. But ours has been going on for quite a long time. We are small, but we are not a startup.

CW (00:39:10):

Yeah. The customers would have to get the licenses and share them with us.

WG (00:39:13):

I was going to say, you can work with them.

EW (00:39:15):

Sometimes they do, but sometimes there is a reason they are in the same boat. And they do not want to do the paperwork. They never want to do the paperwork.

CW (00:39:25):

But it is been the same with compilers. We have dealt with customers that are like, "We are using Keil. Go use it."

EW (00:39:30):

Yes. Now we are using GCC on everything. I am so happy.

CW (00:39:34):

That is a relatively new thing.

EW (00:39:36):

It is. It is. I still have many compilers installed.

CW (00:39:39):

We all have been through IAR Keil. This is not a foreign situation.

EW (00:39:43):

No. It is not. I guess, it was just that-

CW (00:39:45):

If the benefit is there, then it is worth it.

EW (00:39:47):

I loved MATLAB after school.

CW (00:39:50):

Yeah, me too. I used it all through grad school.

EW (00:39:54):

I was really getting into Simulink. Then it was just too expensive.

CW (00:39:59):

Well, you just got to stay in school. <laugh>

WG (00:39:59):

<laugh>

EW (00:40:00):

Well, yeah. But then I got burned when I got out of school, and had to find all new tools. That was before Python had all of it basically replicated. MATLAB, I appreciate it, it is amazing, but it leaves a little bit of a bad taste in my mouth. Not just because Simulink is one of those liny languages, like LabVIEW is.

CW (00:40:24):

Liny?

EW (00:40:26):

I need to type. I do not draw my code.

CW (00:40:29):

I am just questioning the adjective. You have lines.

EW (00:40:33):

It has a lot of lines.

CW (00:40:35):

Right. Actual drawn lines.

EW (00:40:36):

Drawn lines.

CW (00:40:36):

Not lines of code.

EW (00:40:37):

Yes.

CW (00:40:37):

Okay.

WG (00:40:37):

Mm-mm.

EW (00:40:39):

Right.

CW (00:40:39):

And boxes.

EW (00:40:40):

Boxes.

CW (00:40:40):

Uh-huh.

EW (00:40:40):

Why are there boxes here?

CW (00:40:44):

Because it is visual.

WG (00:40:44):

Yeah.

EW (00:40:45):

Yeah. I do not get it.

CW (00:40:46):

<laugh> Okay.

EW (00:40:48):

If you cannot read my code aloud like a story, I do not want it. Hmph. I am going to stomp off to my room.

WG (00:40:55):

<laugh> Sometimes it is actually more intuitive, especially on the physics side. You get a schematic for an electronic schematic or a hydraulic schematic. The way we build up those- The tool in Simulink it is called "Simscape." It looks like a schematic. So your electrical engineer or your mechanical engineer can look at it and be like, "Yeah. That looks like what it should."

CW (00:41:20):

Yeah, that is the thing is for these big physical systems with lots of, like you say, hydraulics, electronics, mechatronics, all this stuff. If you want to simulate that, I do not think you want to be starting at the typey typey-

EW (00:41:34):

Python level, no.

CW (00:41:35):

Languages. Because you have got to start with something that looks normal to someone who does the physics. Or, a block diagram that makes sense, and how these things feed to each other.

EW (00:41:47):

So making robots in Webots is usually a very drawy sort of thing.

CW (00:41:50):

Liny. <laugh> Yeah.

EW (00:41:54):

And I am terrible at it. I am amazed to watch someone be able to create this robot very quickly. Then they will want to change something, and they will have to rebuild it from scratch. And I will be like, "Oh no. You just changed the line in the protofile."

(00:42:07):

It is just this whole discrepancy in worldview, whether or not you should be drawing things or whether you should be typing things. And I am so far into the typing zone.

(00:42:21):

Anyway. Sorry, William, you are in the drawing zone, are you not?

CW (00:42:23):

We <laugh> have derailed this conversation.

WG (00:42:26):

I guess you could say I am. When you are doing MATLAB, that is typey. Yeah. Like Python. But I guess I am ambidextrous in that a little bit.

EW (00:42:38):

You mentioned different types of simulation. We talked about hardware-in-the-loop, where the loop is the simulation, and the hardware is what you are testing. And model-in-the-loop, which is more when you are testing the model, which is more the math area, and less the code.

(00:42:58):

You have two more acronyms here. SIL and PIL. SIL is a software-in-the-loop?

WG (00:43:06):

Yeah. Perfect. Yeah, so you take your model, right? You generate your code, or maybe you write some hand code. But now you want to see how does it interact with the physics. So you connect up- You import a block, one of these silly little blocks with all the liny things, and connect up to your plant model, and see how the code itself operates.

(00:43:29):

Then the processor-in-the-loop. Again, remember this first word, it is the key, right, of what is under test. Put your software on your processor. You do not have the rest of your board, the rest of your ECU. But you still connect it up to a plant model and see what it does.

(00:43:43):

I have seen vehicle-in-the-loop. The other day I saw- Oh, a friend mentioned human-in-the-loop. I do not know what he meant by that. That was just a text, but I am curious. <laugh> He said they were doing that at his new company.

CW (00:43:57):

That does sound like "The Matrix." <laugh>

EW (00:43:58):

Well, you did mention you had humans interacting with your hardware. It is questionable which one of those two you were testing.

WG (00:44:09):

Mm. Good point.

CW (00:44:09):

It is arguable the human is the loop. <laugh> Part of the loop?

EW (00:44:15):

No, because they do something, and something happens in simulation. If the simulation is the loop.

WG (00:44:18):

Yeah.

CW (00:44:18):

Mm. Mm, mm. Hmm-hmm.

EW (00:44:18):

Yeah. Yeah, hardware-in-the-loop. It is funny that- Even having read about this and thought about this, I accept that in-the-loop means in simulation, but I do not think I ever quite triggered on that. It was always testing. It was hardware in the testing loop, not hardware in the simulation loop.

(00:44:48):

Because you can simulate software on its own, and put it into like unit tests. The unit tests do not test everything. They do not need to simulate everything. They only need to simulate small parts of it.

WG (00:45:03):

Mm-hmm.

EW (00:45:03):

Christopher is giving me weird looks, like I have to simulate something.

CW (00:45:08):

Oh, I am just realizing all of this is simulation, at some level.

WG (00:45:10):

Yeah. Yep. Tamagotchis are simulations. <laugh>

CW (00:45:10):

<laugh>

EW (00:45:16):

You can do unit testing, or more likely manufacturing testing, where you have the hardware and you communicate to it, and you expect certain responses. So you are unit testing with hardware-in-the-loop, if that makes sense. Does that make sense? Is everybody nodding?

WG (00:45:38):

Sort of. Yeah. Yeah. I mean definitely-

EW (00:45:40):

That is why I said, "Manufacturing testing," because that is something, an area, people would understand. Or bring up testing, where you have the board in place, and you are testing it as you- But you are not letting it just run as a normal system. You are doing part testing.

(00:45:54):

So the idea that in-the-loop means "with simulation" is a little new to me. It makes sense, but a little new to me, so I am still wrapping my head around it.

WG (00:46:06):

You keep saying, "With simulation." I would say open loop tests are arguably a kind of simulation. You can do unit testing. You can do that closed loop, or you can do that open loop. We will do open loop tests of models as well, especially early on in specific components.

(00:46:25):

I do not want to confuse you here. I think that the key here is, is there a closed loop? Or is it input, unit under test, output, and it does not wrap around to feedback to the inputs?

EW (00:46:42):

Okay, that is a very good point. Because there are a lot of unit tests that are just input to output. And whether or not... So for it to be in-the-loop, there has to be simulation, unit under test, output, and unit under test has to go back to simulation.

WG (00:47:04):

Yesss. <laugh>

EW (00:47:06):

Did everybody draw a box that is the simulation? And a box that is the unit under test?

WG (00:47:13):

The word "simulation." I am getting hung up on that, because-

EW (00:47:16):

Oh. Okay.

WG (00:47:16):

Okay.

EW (00:47:16):

Simulink-

WG (00:47:16):

"Model" and "simulation."

EW (00:47:18):

<laugh>

WG (00:47:18):

Let us get those two straightened out.

EW (00:47:21):

Okay. Yeah.

WG (00:47:21):

So the "model" is the noun. The "simulation" is the verb, so it is the act of the thing running. So when you say, "It goes to simulation," it is linguistically confusing to me.

EW (00:47:36):

Ooh. Okay. This is because you are all in Simulink, and I am trying to wrap my head around doing this with C. Or a microcontroller hooked to maybe a physics unit like ROS's Gazebo. Or maybe a series of files that it-

(00:48:01):

I worked on an audio system, and we would play files for the audio system to listen to. And so that playing of files was the simulation in this case.

WG (00:48:15):

Okay.

EW (00:48:17):

And the hardware would listen, and then it would respond. And the simulation would be able to say, "You responded correctly. That was right." Or it would then play a different file, so it could figure out what was wrong.

WG (00:48:31):

Okay. Okay. I am being pedantic.

EW (00:48:32):

No, no. It is good to-

WG (00:48:32):

I think I know what you are saying. You are saying, "Simulator," I think not, "Simulation."

EW (00:48:37):

Sure. Yes.

CW (00:48:38):

Mmm.

WG (00:48:39):

That is all right. That is all right.

EW (00:48:40):

Yes. I guess my terminology there is difficult, because I have one simulator, which is the large program ROS Gazebo. And then I have simulations, which is the data that represents the vehicle going to this location and that is the simulation. And I have a vehicle going to that location.

(00:49:10):

So I think about how my model is working in this simulation or that simulation, that are both running on the same simulator. These words are very confusing!

WG (00:49:23):

<laugh> It is good. It gets you- It is a chance to learn. That is a thing that I try to- When I am leading teams and helping, especially younger engineers, is like, learning happens at the edge of what you know and what you do not know.

(00:49:40):

So you have to go into uncertainty and yeah, it is okay to- You are going to make mistakes and things are not going to go right. But if you were not making mistakes, you are probably not learning. So hopefully this is helpful to you, and to the listeners too.

EW (00:50:00):

I actually had a listener recently who asked, "How do you learn complex things?" We did not answer that question, because we did not know how. How would you have answered it?

WG (00:50:12):

Well, more books. Has anybody read, ""Surely You're Joking, Mr. Feynman!"?

EW (00:50:15):

Mr. Feynman.

CW (00:50:15):

Mm-hmm.

EW (00:50:20):

Not a personal favorite of mine, but I know a lot of people who love it.

WG (00:50:22):

Right. Right. I am not going to comment on the man, because I definitely think there are some problematic aspects there. But I think his description of how he learned- He talked about reading a paper, and the first time he reads it and it means nothing to him.

(00:50:41):

Then he forces himself to go back and read it again. And, "Okay. Well. Now that I have the context, I understood where it was going. It made a little bit of sense, but a lot of the details do not match up." Then comes back and reads it again. The third time, suddenly things are clicking and falling into place, and everything starts to make sense.

(00:51:01):

I think it is a real challenge to force yourself to be in that discomfort, and to be in that place where you do not know what is going on, and have to do it again and again until it starts to click.

EW (00:51:17):

That is a good anecdote from the book. It has been hard for me to remember that it is okay if learning is difficult.

CW (00:51:27):

<laugh> Yeah. My favorite thing to say is, "Hard things are hard."

EW (00:51:33):

But it is so easy to look around to people who know things, and think that it was easy for them to learn it.

CW (00:51:37):

Yeah, but you are not looking at the 10,000 hours or whatever that they spent doing it. Stuff- Hard things take repetition. Repetition. A lot of repetition. And seeing variants of things that are similar to it, but with- Yeah. Learning things is- Anything worth learning is difficult usually, at some level, unless it is just somebody's phone number.

EW (00:52:03):

But then we all hope that this is going to be the thing we are spectacular at. This is the thing that comes easy for me.

CW (00:52:11):

Well, I think that is a cultural thing.

EW (00:52:13):

Is it?

CW (00:52:14):

Yeah.

EW (00:52:15):

See, I think computers was the worst thing for me. Because I watched many other people learning computers for the first time, a college level computer course. I watched so many people around me have a hard time, and I did not understand.

(00:52:30):

It was like computers made more sense to me than just about anything. Made more sense than the social interactions. Made more sense than the math I was trying to learn, which was calculus.

CW (00:52:47):

Right. But there are plenty of things that are not easier for you, that are easy for other-

EW (00:52:52):

Everything is easy for everybody else. <laugh>

CW (00:52:54):

I see. People do have affinities for things, but I think we overstate affinities for things.

EW (00:52:58):

Right.

WG (00:53:00):

I completely agree.

EW (00:53:04):

It is hard, and you have to keep working.

CW (00:53:06):

I think computers were quote "easy for me" too. Except I started using them when I was five, and I spent every day on them from five to when I went to college. What is that? 13 years or whatever of using computers, before I even got to learning about computers formally. That is not nothing.

EW (00:53:25):

I think I had some of that too.

CW (00:53:27):

You just forget. Yeah.

EW (00:53:28):

Many afternoons spent in the library with the TRS-80 and a tape deck.

CW (00:53:33):

And you do not count the things that are fun.

EW (00:53:36):

Right. That is not learning. That is just fun.

CW (00:53:37):

Yeah. Yeah.

WG (00:53:38):

That exploratory learning is where real magic happens. You read a textbook, and you learn the exact thing that everyone else learned from the textbook. But man, that exploratory learning, that is where it gets in your gut and allows you to thrive in the learning process.

CW (00:54:00):

To look back on the simulation stuff- I have been doing a little bit of electronics. A little bit of electronics, trying to learn some electronics just for fun. Being able to explore, either with a breadboard and parts, and not have any fear of failure.

(00:54:15):

Or with things like online simulations of circuits, which there are very good ones that are free. That you can just put parts together and put a fake oscilloscope, and see what happens and test circuits and stuff.

(00:54:27):

Apart from reading "The Art of Electronics", which the first couple times I tried to learn electronics, that is what I did. "Oh, I will just read 'The Art of Electronics,' and then I will know electronics." No, that does not work! It does not work! You will have read "The Art of Electronics".

EW (00:54:39):

You will have read the first chapter.

CW (00:54:40):

You have read the first chapter, and bounced out of it. Nothing- You have to get your hands dirty, and actually try things. See what makes a difference. Make changes and "Oh. Well, that did that. Oh, that is interesting." Or find the places that you do not understand, that you can put your effort into. Because everybody has different things they get hung up on.

WG (00:55:05):

Yeah. Yeah. I want to mention something. This is part of what I see my responsibility. I am an engineering leader, is like creating that safe environment where people can learn things and fail at them.

(00:55:15):

There was this program I helped put together at my former employer, called "The Software Craftsmanship Club" program. It came out of this idea that myself and a colleague- His name is Joe Fisher. Do not know if he will listen to it, but thanks Joe.

(00:55:32):

We were lamenting that everybody is doing the work. When they want to learn something, they go to a course to learn it. But it is a solo thing and it is dry knowledge. It is not living knowledge.

(00:55:47):

So we decided to do this experiment where we would create a Software Craftsmanship Club. Get together maybe eight people, something like that. We set up people in pairs and said, "You guys pick your language. Whatever you want to do. You want to do Python. You want to do C++ or C. But we are going to pick topics and go through them. Like Cling code and things like that. We will go through them as a group, talk about them as a group. Then we will assign little code katas, little projects for people to do, hour long projects."

(00:56:28):

They would go off and apply what they learned in it. The partners would do code reviews of each other and say, "Hey, I did it this way. You did it that way. I like the way you did it." Learn about the coding, learn about the concept, and then iterate again.

(00:56:41):

All outside of the raw- When you are working on code for a client or for your boss, you are not safe to experiment. You cannot try stuff. If it fails, you are going to be in trouble.

(00:56:55):

So creating the space where a group of people could self-direct. Pick out different topics and learn on it and talk to others about it and actually apply it. That was one of my more happy, magical experiences as a engineering leader, seeing that start to come together.

EW (00:57:16):

When you told me that, I immediately jumped to Greg Wilson and his Software Carpentry. He was on the show, a year ago? Two years ago?

CW (00:57:25):

Something like that.

EW (00:57:27):

It was a somewhat similar concept, of trying to make code more approachable. And as an ongoing learning exercise, to make better code. Not just, "Here is what you do," but, "Here are some idioms that are better." Oh, do not forget about software patterns, and things that are part of the ongoing learning process.

(00:57:58):

It is hard to maintain, when life- When life gets in the way, it is hard to say, "Oh, I promised myself I would read a technical book every quarter. I have not, because I am busy." There are ways to get into it. How did you keep people interested and motivated?

WG (00:58:19):

When I think about motivation- Have you guys heard of- More books. Daniel Pink's "Drive"?

CW (00:58:29):

I think I have heard of it, but I do not know much about it.

WG (00:58:33):

Okay. The root of it, anyway, he is a psychologist and gets at the root of what drives people. It comes down to three concepts. Autonomy, mastery, and purpose.

(00:58:46):

Autonomy is the aspect of you want to be able to decide for yourself what you want do. So with the Software Craftsmanship Clubs, we did not dictate, "You have to do this as a club." We actually have the Club vote on the next topic. So they got that sense of control of their own fate.

(00:59:09):

Then the mastery. Obviously software craftsmanship, that is the idea of like, "Oh. I am going to get really good at what I am doing." That feels like progress. That feels like leveling up in the video game sense.

(00:59:27):

From the purpose side, I guess that comes internally. Like, when I am leading teams, I am definitely sitting and talking with engineers and trying to understand, "Why are you doing this? What is your purpose?" Trying to help them evolve that too, and ask themselves the questions.

(00:59:44):

Yeah. I lost track of what your question was. But hopefully I answered, this is-

CW (00:59:49):

How to motivate.

WG (00:59:50):

This is where it comes from.

CW (00:59:51):

People from-

WG (00:59:53):

You do not motivate people per se. But you can highlight what drives them, and create the space for them to follow their own motivation.

EW (01:00:04):

We have been doing Tech Book Club on the Embedded Patreon Slack. It has been a tough book. That is a tougher book than I intended.

WG (01:00:13):

Bailey has mentioned it. <laugh>

EW (01:00:16):

But it is interesting. I have found for some of the other people who have done- So, the Tech Book Club is- This book is led by me, because it is the book I wanted to read. I said, "Anybody who wants to join me, please do. I will make an effort to make it accessible." Basically, "Please join me, but do what you want."

(01:00:42):

Other people have stepped up and said, "Okay, I am reading this book. Anybody wants to join me, I want to talk about it." That has led to other people reading books and talking about it.

(01:00:53):

I felt bad because Phillip of Embedded Artistry read a book that I recommended. I was there with him for the first half, and then the book got super boring. He finished it, and I was like, "Yeahh. I did not. Because it got super boring."

(01:01:07):

But sometimes all you need to do is to find a group and say, "Yeah, this is what I am doing. I promise that I will do it. I would really be happy if I could talk to you about it." Whether that is three people in your group at work, or three people in your LinkedIn group, that may be enough.

(01:01:32):

I do not think I could stop "Data-driven Science and Engineering" now, because they are all interested. I am too. But it is <laugh> harder than I thought it would be.

WG (01:01:48):

Yeah.

EW (01:01:48):

But soon, soon we will get to how data and control systems are the same thing. I am so looking forward to that.

CW (01:01:55):

<laugh>

WG (01:01:59):

Okay. Yeah, no. I cannot agree more. I love learning with others. I get lots of great friendships and discussions, and the enriched learning experience of, "Well, here is how I see it." And they say, "Well, I do not quite see it that way." That is so valuable. Then follow it down the weird avenues. I do not know. There is so much good out of that.

EW (01:02:25):

Well, like the first half of the show, where we tried to figure out terminology like "simulator."

(01:02:30):

Have you read any good books lately?

WG (01:02:37):

Have I read any good books lately? Yeah! I have read lots of good books lately. I am in the Des Moines Tech Book Club, which is some of my besties here, since I have moved here a couple of years ago. We read books. I read fun books. What are you looking for?

EW (01:02:55):

Well, what is the Des Moines Tech Club read recently that you liked?

WG (01:03:00):

Good question. Let us see. I have not been a fan of the latest book, so that is probably not a great example.

EW (01:03:09):

You can totally slag them. I am happy with that. But if you want to stick to the positivity thing, I am good with that too.

CW (01:03:16):

<laugh>

WG (01:03:17):

Yeah. In fact, I did the thing that you just said. I made a promise to myself a few years ago, that I am going to stop wasting my time. Once I decide this is not valuable- I am one of those completionist people. It is like, "Oh, I got to finish it." I am like, "No. I am going to be hard about it. Because life is short. I do not want to waste time finishing things that I do not want to do." So I dropped out of this book, and I am planning to join back for the next one.

(01:03:47):

As far as books that we read recently, ooh, there is one called "Leadership BS." From your region, California. Stanford, I think. Jeffrey Pfeffer, I want to say his name was.

EW (01:04:03):

Mm-hmm. That is right.

WG (01:04:04):

He talks about the things that you are told, that make you feel good. He has been studying organizational dynamics for 50 years. There are a lot of things out there. We want to believe those. They sound good. A lot of motivational talk around leadership and things like that.

(01:04:27):

He puts a pin in some of that and says, "Look. You should also know the reality. Which is what really happens with power, is not necessarily what we want to believe happens with power in organizations. Or even what the people who are saying this, actually did to get where they are.

(01:04:47):

"Understanding the difference is critical. You can waste a lot of time and energy not understanding that, and believing the things that maybe people are saying that is not really reflecting reality."

(01:05:01):

He tries to help people navigate what real power dynamics happen within organizations, and how you can use that to make organizations better.

EW (01:05:12):

Mm.

CW (01:05:12):

<laugh>

WG (01:05:12):

Are you laughing?

EW (01:05:17):

No, no. I am thinking. I am thinking.

WG (01:05:18):

Oh. Okay.

EW (01:05:18):

I am thinking of the time I quit my job. But I only told my boss and my boss's boss, and did not tell anybody else. Then I sat down to lunch with my boss's boss and he said, "Have you told anybody?" And I said, "No. But I am very interested in how the rumor mill has worked."

CW (01:05:37):

<laugh>

EW (01:05:39):

So we got into this long discussion of how information travels through the organization we were in, and how amusing it was to watch, and who the connectors were. I am not going to say gossips, but take it as said, which I guess I did. Anyway. Yes. Sorry. No, I am humming because you are making me think. Yes. Cool.

(01:06:03):

What about you, Christopher? Have you read any good books lately?

CW (01:06:07):

Have I read any good books lately? Well, I am just starting to do the thing that you mentioned, of reading more than one book at a time. Usually I only read one book at a time. So I am trying to read a nonfiction book and a fiction book at the same time, and switch between them depending on what I feel like at the moment.

EW (01:06:25):

The fiction one is Will Wight's "Pilot," right?

CW (01:06:27):

Yeeah. Kind of.

EW (01:06:28):

Which has not been as good as the first two in that series.

CW (01:06:30):

I was just going to mention I am continuing the Pratchett read-through. <laugh>

EW (01:06:32):

Oh, oh. The Pratchett read-through. Yes.

CW (01:06:34):

Yeah. Which I have already talked about. But I am also reading a history book, the "Battle Cry of Freedom," which is about the Civil War. Which I am quite enjoying, also from a political standpoint. And-

WG (01:06:49):

I love history.

CW (01:06:49):

I took a Dogbotics course recently that did a drum machine. The "Make:" book that goes along with that.

EW (01:06:59):

That was written by Kirk Pearson.

CW (01:07:00):

Kirk Pearson. What is it called? Hang on. "Electronic Music from Scratch." I have gone through most of that, reading it, while also going through this other class. The class led me to another book that I am flipping through, so I am not calling this reading.

(01:07:16):

It is an old book called the "CMOS Cookbook" by a guy named Don Lancaster. It is all about things you can do with CMOS chips. The impetus for this was making synthesizers out of various logic components and ICs and things and op-amps. I am looking through that for things I could do as projects.

WG (01:07:39):

Is there anything- Maybe I will ask this for both of you. Recently that you have read in a book, a concept or an idea, that has just really influenced how you see the world or found profound?

CW (01:07:57):

Tough question. I will throw it to Elecia. <laugh>

EW (01:08:01):

Okay. I mentioned the "Data-Driven Science and Engineering." Steven Brunton and Nathan Kutz. That we are reading for Tech Book Club. Okay. The thing is, I briefly said something about dynamic systems treated as data systems. This book has blown my mind so many times, because of things like that, where-

(01:08:26):

I have taken machine learning as a course from- I did the Andrew Ng course. I did the-

CW (01:08:38):

The cars course from-

EW (01:08:38):

The cars course from Udacity.

CW (01:08:41):

Self-driving cars, from Udacity.

EW (01:08:43):

And I have done classification in practice. I have done machine learning in practice, as part of jobs. So. Decent background of that. Half of my major in college was control systems, so really happy with PIDs. Not so happy with Hamiltonians and Jacobian and LQRs, because that was after my time. Or beyond my abilities. I do not know. Anyway.

(01:09:12):

He keeps just taking these things that I think of as data solutions and data views of systems. Then he uses them to do things I would never agree to? Never have considered. Like Fourier, which to me is really special. Very important for control systems. Important to understand when you are doing signal processing. Very interesting way of looking at the world. It is just another data tool for him.

(01:09:50):

Wavelets. Just another basis function. Similar to any SVD that you could do. It does not really matter. You have a toolbox of tools, and that is just one of them. And I am like, "No, no, no, no. Fourier is the most important tool ever!" But for him, he has got a ton of them, and he is just flipping through them to do different data things.

(01:10:14):

It keeps blowing my mind that you can just do that. These things that I thought of as concrete, whole, indivisible, and useful in many ways, balls of information. He has like put it in a gumball machine and said, "Oh. You can have any one of these." I am having a really good time with that.

(01:10:47):

We are in chapter five, I think. Finishing chapter five, which is classification. We are going to go to neural nets next. But then we go to the new chapter, which is all about dynamic systems and treating those as data systems. I am like, "No, no, no. PID? I do not think we can go there with a PID." But I have actually read ahead and I know that we can, and I am just still boggled by it.

WG (01:11:13):

<laugh> I mean, what is math, right? It is just logical rule systems.

EW (01:11:20):

This is all just a cheater way to teach somebody linear algebra and ODEs. But I did not know that you could call it "data science," and I would agree to it.

CW (01:11:29):

"Just linear algebra and ODEs" is everything.

EW (01:11:33):

Yes. Yes.

CW (01:11:35):

It is the universe. <laugh>

WG (01:11:37):

It is a reason we do quantum and general relativity with all of this stuff.

CW (01:11:40):

All right. And PDEs.

EW (01:11:42):

Well, PDEs is part of this too. But I did not ever agree- If he had written the books, "Linear Algebra, PDEs and ODEs for Engineers," I would have been like,

CW (01:11:51):

"I am not reading that."

EW (01:11:52):

"You can just stay on the shelf, thanks." But somehow he has got me engaged. I am just gobsmacked by it.

WG (01:12:02):

This is an interesting thing with the plant models and stuff. You do not have to- I talked about using the hydraulic schematics and electronic schematics and stuff like that. But you can just represent those as math too.

(01:12:14):

In fact, there is a whole system identification concept, where they just gather the data. They just sensor up particular products and just gather all of the data. And create a mathematical model that represents that thing, without ever trying to understand the underlying physics or mechanics or things like that.

EW (01:12:40):

Yes! I believe in this book, we will be going back to that.

WG (01:12:45):

Cool.

EW (01:12:45):

Where you take the outputs of your system. You do not care what the inputs are. You just treat it like a data system. Then you create an analytic solution from that, which is so wrong. You cannot do that! But you can.

(01:13:04):

It is using the system to describe itself, instead of trying to make a model. And then trying to fit the model to the realities of physics, once you have static friction and all the other kinds of friction and damping and brown outs and bleed resistors.

CW (01:13:24):

<laugh>

EW (01:13:26):

So many complaints here.

WG (01:13:29):

It makes me think of something. We are going back over 15 years now, to my graduate advisor. His name was Cliff Chancy. Absolutely beautiful man. But I remember one of our last conversations before he died. We were talking about- He called them "automatons."

(01:13:45):

But he was talking about what he anticipates in the future, is that essentially what we have today, like AI models that could look at raw data and discover what we understand is the current laws of physics now. But also by looking at that raw data, find additional patterns that we do not necessarily understand. And actually start to discover the laws of physics that are beyond us. And explain things in dark energy or dark matter, or down at the quantum level too.

EW (01:14:23):

Did you not tell me about a kid who went through a bazillion amount, an incredible amount of astro data, and came up with a whole bunch of discoveries. Because he used different techniques.

CW (01:14:36):

I am going to plead the, "I am old and forgot about that."

WG (01:14:38):

<laugh>

EW (01:14:40):

It was a good article. I will look at it. I think it went out in our newsletter, so I am pretty sure we have it.

CW (01:14:46):

<laugh>

EW (01:14:46):

Which Christopher puts together.

CW (01:14:47):

There is no question that I both found this, posted it somewhere, discussed it with you. But I have zero memory of this at all.

EW (01:14:55):

But yeah. It was there were patterns that nobody else had noticed, because they did not look in that way. So yeah, I think there will be more of that. Although finding an external pattern- Finding a pattern in the data. Sorry, that is better than "external pattern," is not the same as finding the cause in the physics.

CW (01:15:23):

And not all patterns are real. As humans, as good pattern recognizers, and seeing faces in everything, we have to be careful of that too.

EW (01:15:32):

<laugh> Yes.

WG (01:15:33):

For that matter, what we understand is physics, is just models. We do not know if they are real. We used to have force equals mass times acceleration. Then we had to redefine what acceleration was, because Einstein came along and said, "Yeah. It is not quite what we thought it was."

CW (01:15:49):

Eh. Just do not go too fast, or be too near something big. It should be fine. <laugh>

WG (01:15:54):

These are approximations, right?

CW (01:15:57):

Yeah. That is the whole thing about physics and learning physics, is one series of approximations getting slightly less bad over time, the more you learn. I am not sure that is true. All models are wrong at some level, right?

WG (01:16:10):

Yeah.

EW (01:16:13):

Okay. I wanted to ask you a little bit about Spark Embedded. You and Bailey have started a company. What do you want to tell us about it?

WG (01:16:28):

Let us see. Kind of came about- Bailey and I started- We started a group called "Iowans of Things," which is a sort of embedded group.

EW (01:16:37):

<laugh> Sorry, I just got it.

WG (01:16:39):

Yeah? Yeah? Yeah?

EW (01:16:43):

IoT. Okay. Okay. Got it.

WG (01:16:45):

Iowans of Things. So an embedded group for people in Iowa to meet up and talk to each other. We have hardware hangouts, where people come and talk about stuff. Then we have the firmware fellowship- I guess we are into alliteration. Where it is more technical conversations.

(01:17:00):

Anyway, we started that a couple of years ago, and started hanging out and getting to know each other and liking each other. Started doing bar trivia together. We had a team called the "Ensparkled Barbarians." Have a team. In fact, Wednesday night is the end of our season, and we are in the lead at the moment.

(01:17:21):

But that led to us having conversations about, we are working for other companies, and why do not we think about working together. Creating a space that we would want to work for. She was working solo at the time.

(01:17:37):

But yeah, I am very passionate, like I said, about helping create collaborative teams. Where we can bring in engineers that can really learn to work together. Learn from each other, start experimenting, which involves potentially making mistakes and failing. See if we can use MBSD and HIL to help accelerate firmware development. Then also feed that back out into the community, and see if we can help others do this too.

(01:18:16):

So, Spark Embedded. We are actually technically launching here after Labor Day. But we have already been doing a lot of the groundwork and got some plans. I do not want to turn it into a whole advertisement, but if you want to talk to us, we are both on the Embedded.fm Slack. Or you can find us on LinkedIn.

(01:18:35):

We are going to be at Supercon and Embedded World in LA, here in a couple of months. Honestly, just really would love to meet people who are interested in working with us, or partnering with us.

(01:18:49):

Regardless, even if you are not interested in those things, we just want to meet people who love what we do. Have great conversations and learn from each other. Yeah, I would love it if people would come up and say, "Hi," or send a DM or whatever, and we could chat more.

EW (01:19:09):

Cool. You said you are going to Supercon and Embedded World, which was the other question I had for you. I think we have kept you quite a while.

(01:19:19):

William, do you have any thoughts you would like to leave us with?

WG (01:19:24):

Sure. I had one I was thinking about. I have this friend, Dustin Thostenson. He is in the Des Moines Tech Book Club. He has this thing that he says that most engineering problems are not technical problems, they are people problems. I completely agree.

(01:19:43):

20 years into my career, we talk about systems, we talked about a lot about systems today. But it is not the individual pieces that are so bad, that are so challenging. It is the connections across the system. People and communication, those are different parts of a system too.

(01:20:02):

Yeah. The way we helped optimize systems with people is embracing things like curiosity, embracing uncertainty, and being okay with asking questions and learning from our mistakes. That is how we eventually get those better outcomes. That is what we are trying to do with Spark Embedded as well.

(01:20:25):

So yeah, I guess the last thought is think about that. Think about how people- Do not think about how people are problems. <laugh> But think about how people are a part of the system, and how you can help to make the system better by working with people.

EW (01:20:42):

Our guest has been William Griffin, engineering consultant and co-founder of Spark Embedded.

CW (01:20:48):

Thanks, William.

WG (01:20:49):

Thank you. Thank you for having me.

EW (01:20:51):

Thank you to Christopher for producing and co-hosting. Thank you to our Patreon supporters. Thank you to Mouser for sponsoring the show. And thank you for listening. You can contact us at show@embedded.fm or hit the contact link on the embedded.fm website. Which is where the show notes and transcripts also live.

(01:21:10):

And now, do you want to joke, or do you want to quote?

CW (01:21:14):

I should mention the changes to Patreon. Patreon supporters or people who are not Patreon supporters, if you become $5 or above members monthly, you will have access to a new ad free feed of the show through Patreon. And potentially some bonus episodes that will only be for people at that level.

(01:21:35):

We will be doing some trials of things we have mentioned on previous episodes for those. And maybe some special things with some guests, that are off the beaten path of our usual shows, but sort of fun things.

(01:21:46):

Oh! I will also be posting those things to Ko-fi. But there is not an easy way to get an RSS feed of that from Ko-fi. If you are okay with that, you will get them, but it might require you to download them or play them within Ko-fi.

(01:22:02):

So ad free stream and bonus episodes coming to the $5 tier. This will be the first episode that goes on the ad free stream.

EW (01:22:13):

Do you want to joke, or do you want to quote?

CW (01:22:15):

I want a joke.

EW (01:22:16):

What is the difference between a strong person, and a really, really, really strong person? Repetition!