480: Surprises Early In The Game
Transcript from 480: Surprises Early In The Game with Jerry Twomey, Christopher White, and Elecia White.
EW (00:00:07):
Hello and welcome to Embedded. I am Elecia White, alongside Christopher White. Now you may have heard that I have a book, it is called "Making Embedded Systems," and I recently finished the second edition. It is a book that is for software engineers who want to get closer to the hardware, and hardware engineers who want to write better software.
(00:00:27):
But what if it was written by an electrical engineer for electrical engineers? Well, then you would have "Applied Embedded Electronics." Our guest this week is the author of that book, Jerry Twomey.
CW (00:00:40):
Hi Jerry. Welcome.
JT (00:00:42):
Hello. Thank you for having me.
EW (00:00:44):
Could you tell us about yourself, as if we met at a conference?
JT (00:00:49):
Okay. I am an electrical engineer by education. I have spent about 50% of my career in circuit systems and PCB world. And I spent about 50% of my career as an IC designer, doing mostly analog mixed signal RF type design, uber high-speed data for computer systems, and things of that nature.
(00:01:12):
Historically, I started off in Massachusetts, moved to San Jose, spent some time in Silicon Valley, and these days I am down in San Diego.
(00:01:22):
Multi-sector activity of mine has been a little bit of everything in the electronics world. From defense projects, to consumer electronics, to enterprise computer systems and the electronics that go in them, RF systems, and medical devices. And others. I have got about a 40 year career, so consequently I have been all over the electronics industry.
EW (00:01:46):
That actually brings up a listener question. Timon asks, "Do you feel like the job of the EE has fundamentally changed over the past ten to 20 years?"
JT (00:01:56):
Oh, it has radically changed since I got into the business. We have had a strong migration over the last 40 years from analog to digital. But in the last 20 years we have had a stronger building up of the importance of software systems on top of hardware platforms.
(00:02:17):
A lot of EEs and ECE people and computer engineers are a little less aware of the electronics or the analog aspects of things, and are a lot more aware of the digital things and the necessities of coding things.
EW (00:02:35):
All right. We are going to talk more about that later, and about your book of course. But first we want to do lightning round, where we ask short questions, and we want short answers. And if we are behaving ourselves, we will not ask, "How?" and "Why?" and "What about bypass capacitors?"
JT (00:02:47):
Okay.
EW (00:02:47):
Are you ready?
JT (00:02:49):
Sure.
CW (00:02:51):
Hardware or software?
JT (00:02:53):
You need both. One is a brick without the other, and the other one has nothing to run on. They are both important
EW (00:03:00):
Writing or training?
JT (00:03:02):
They are kind of intertwined. I have done a lot of technical presentations, and I have written a ton of magazine articles. And as you know, I have recently finished a book.
CW (00:03:12):
What is the best part of Balboa Park?
JT (00:03:18):
Hmm. <laugh> That is an interesting question. I did not expect that one. I will say the model train museum, just to be devil's advocate.
CW (00:03:25):
Oh, yeah.
EW (00:03:25):
That is really cool.
CW (00:03:25):
No, I love that Park.
EW (00:03:27):
Are Extraordinary Desserts extraordinary?
JT (00:03:31):
No, they are fattening.
EW (00:03:32):
<laugh>
JT (00:03:32):
Yeah, I can tell you folks have been to San Diego a few times.
EW (00:03:36):
<laugh> Yes.
CW (00:03:37):
Have you seen the green flash?
JT (00:03:41):
Many times, including in the middle of the ocean, out of sight of land. So yeah.
EW (00:03:48):
Do you have a favorite fictional robot?
JT (00:03:52):
Actually two of them. C-3PO and WALL-E. Both for the same reason. They are two computers that in their depictions in their movies, they show emotion. Emotion is going to be one of the most difficult things to do for robotics and AI.
CW (00:04:10):
If you could teach a college course, what would you want to teach?
JT (00:04:14):
Oh, wow. I have already taught a multiple number of college courses.
CW (00:04:17):
Pick a favorite then. <laugh>
JT (00:04:19):
Oh, I like the A2D Converter course I used to teach for UCSD. I have taught a course in phase-lock loop design. I have taught a course on mixed signal IC design, and medical electronics. I have enjoyed all of them for different reasons.
(00:04:34):
I will be honest with you, teaching graduate school at UCSD is kind of demanding, because you cannot gloss over anything for the grad students. They are going to nitpick you to death.
CW (00:04:46):
<laugh>
EW (00:04:49):
I taught a course, and it was not for grad students, but still it felt like every week was a quiz on every part of embedded systems. I never knew what they were going to ask.
JT (00:05:01):
You can get taken to the cleaners that way for sure.
EW (00:05:04):
Yes. I had to really become much better at explaining things.
CW (00:05:09):
I felt like one of the best ways to learn a subject was to have to teach the college kids or the equivalent. <laugh> Yeah.
EW (00:05:14):
Anybody interested in the subject, and suddenly they have all these questions, and you are like, "Because..."
CW (00:05:20):
I cannot answer, "I do not know." And also I cannot lie.
EW (00:05:26):
Right. <laugh>
CW (00:05:26):
<laugh>
EW (00:05:26):
Okay. One last lightning round question. Do you have a tip everyone should know?
JT (00:05:31):
When it comes to design or life? <laugh> Pick one.
EW (00:05:34):
Both.
JT (00:05:34):
When it comes to design, I would say try to get your design off of the simulator early in the process. Try to get yourself onto the hardware. Develop your code over on a hardware platform. Do not obsess on the simulator.
(00:05:49):
As for life, that one is easy. One of my favorite books is Dale Carnegie's, "How to Win Friends and Influence People." A lot of engineering and scientist types could use better people skills. We all work in a village, so to speak.
EW (00:06:08):
A smaller village than most people really expect too.
JT (00:06:11):
This is true.
EW (00:06:12):
Okay, so your book, "Applied Embedded Electronics." Could you tell us about it?
JT (00:06:19):
"Applied Embedded Electronics" is about the circuits and the systems and the nitty gritty of designing a hardware platform. There are many good books, and a few bad ones too, out there on embedded systems that focus on software.
(00:06:38):
Most of them, however, do not go into the nitty gritty of all the electronic systems that attach to embedded systems to make them work. How to drive motors, how to put together a power supply, how to communicate things in challenging noisy environments, and a multitude of others. Those are just a couple of examples.
(00:07:01):
But the book goes- It is not designed to compete with many of the embedded system books that are out there, because it is about the actual circuits and systems rather than the coding aspects of things.
EW (00:07:16):
When I wrote about A-to-Ds and D-to-As, I spent a lot of time talking about circular buffers, and data management, and expectations of sampling. And making sure that you had a chain of where your data is, at appropriate sample rates all the way through your system.
(00:07:39):
But in your section, you talked more about how an A-to-D works, and how to choose an ADC for an application.
JT (00:07:52):
Sure.
EW (00:07:54):
What do you think software engineers, or new electrical engineers from college, do not know about- Oh God, now I have to choose between all of these chapters! About A-to-Ds?
JT (00:08:08):
Many, many of the books and magazine articles, et cetera on embedded systems, treat things as an ideal. They treat things as ideal ones and zeros, and we have perfectly captured a non aliased data stream associated with some analog signal.
(00:08:28):
I think one of the biggest downfalls of a lot of new engineers is the fact that they do not understand the multitude of non-ideal issues that can come up and bite them. Everything from noise, to linearity problems with A-to-Ds, to aliasing due to out of band signals, and things of that nature.
CW (00:08:49):
And dynamic range too. I remember shopping for systems and getting quotes for, "Oh yeah, we have a 24-bit A-to-D," but the noise floor is way up here, so you have effectively 19 bits or something. There are all sorts of stuff like that. Yeah.
JT (00:09:04):
Right. There is actually a section in my book on the effective number of bits, the bits that are actually real. And then you have got the ones that are lost in the noise. Which is what you are talking about, Chris. Totally agreed.
EW (00:09:19):
Do you think that the electrical engineers coming from college do not get told that things are non-ideal? Or do you think they have so much information, and yet they still do not have the information to do an electrical engineering career?
JT (00:09:37):
It depends upon who is teaching them. I find that getting into the non-ideal aspect of electronics, has a tendency to be lightly touched on in graduate school, but very little or nothing in undergraduate. That is just my take on it.
(00:09:58):
Also, when it comes to the technical problems associated with electronics, invariably they are something that is analog in nature. Noise, ESD, whatever. Consequently, there is not as much of an emphasis on the analog side of things, for many, many students these days.
EW (00:10:19):
As you pointed out, so many things are becoming more packaged in hardware. There is a reason for that, but that still means we need people who can do both.
JT (00:10:37):
Right. One of the things I say, I think it is in the first chapter of my book, is I discuss the whole idea of technical detail at the system level, versus technical detail down at the chip level. Or down at the analog front end level.
(00:10:52):
Which is frequently where, as you know, a lot of people are doing system on a chip designs, where there is a complete bundled analog front end, there is a sensor, there is an amplifier, an ADC, and what comes out is a nice I2C or SPI type data stream. So those prepackaged devices have taken and put a lot of the analog inside the package and said, "Put some power on, and here is your data stream."
EW (00:11:21):
Yes. And as long as everything is perfect, it works fine. But should you create an antenna, things start going badly.
JT (00:11:30):
Yeah. It sounds like everybody- We have all had some painful learning experiences there.
EW (00:11:36):
Who did you write the book for? Did you have someone specific in mind, or a group?
JT (00:11:42):
Yes. We have got a situation where- Let me give you a little bit of a history on it. I went off and became an independent consultant, goodness, somewhere around 2005 I think?Somewhere back there, maybe even earlier.
(00:12:00):
One of the things I ended up getting was a lot of clientele that came in with, "We do not know what is going on. We are having problems with this thing." That led to me writing, oh, at least a dozen articles for "Electronic Design" magazine. I think one or two for EDN. I think another one or two for "Chip Design" magazine.
(00:12:25):
Where I described what the problem was that I had run into as a consultant and said, "This is how you do this, folks. This is how you deal with noise. This is how you deal with ESD. This is how you do this, that, or the other thing." Consequently, all of those articles based upon client problems, were what motivated the book.
(00:12:49):
I would say the best fit for this book- This seems to be supported by a lot of the people that have reviewed the book on LinkedIn. Is that the designers that really appreciate it the most, are people that are three to five years into the business. They have made a lot of the mistakes that they read about in the book, about do this or you are going to have this problem. So it is 40 years of condensed experience, as a fast track start for people in the three to five years of industry experience role.
EW (00:13:25):
Yep. Yep. Exactly. Why do you write? You are a consultant. I assume you have that writing is partially advertisement, but there are other probably easier ways to advertise. Why do you write?
JT (00:13:43):
I enjoy it. I have written all kinds of technical articles. I ghosted a book, oh goodness, a number of years back, for a CEO that had to be out there publishing things, but did not have the time to do it. So this is really my first official book, but I have actually been in a couple of other books as a ghost technical writer. But I enjoy the writing process.
(00:14:11):
I find that in the field of technology, there is an atrocious amount of stuff out there that is written in a manner that is not understandable. Or drives into areas where the end user does not need that information presented in that way. Engineers, many of them, have trouble communicating things effectively.
EW (00:14:36):
I think it is interesting that many of the consultants I know, also write. I have not figured out if it is an egg or chicken thing. Do we become consultants, so we also have time to pursue writing? Or do we start writing because we are consultants in need advertising? Or are they just two different things?
JT (00:15:04):
I do not think the sequence of events is hardwired in one direction or the other. I do think that- I personally went into consulting to get a little bit more freedom in what I was doing. I have worked for some really great companies, and done some really great products.
(00:15:21):
But when you get out there as a consultant, you are allowed to do a lot more things that are different. And you are allowed to be a little bit more fussy about what you take on. As long as you can pay you the bills, of course.
EW (00:15:36):
Your consulting company name, "Effective Electrons," where does that come from?
JT (00:15:42):
It is a term buried in the back of semiconductor theory. I have, in addition to designing circuits and embedded systems and IC design, and things like that- Both when I was at IBM and Fairchild, and actually at LSI Logic, all three of those companies, I helped bring up different semiconductor processes. I helped launch the RF CMOS process when I was part of IBM.
(00:16:11):
Consequently, I have been into the semiconductor theory of things. The name of "Effective Electrons" is about the electrons that are available in a doped semiconductor. I just thought it was a nerdy statement to summarize <laugh> my entity.
EW (00:16:37):
A lot of the consultants I have talked to recently have gotten burnt out. Do you have any problems with that?
JT (00:16:44):
I take on work when I want to take on work. I do things that I want to do. I generally do not try to maximize my income. My money situation is fine, so my consulting scenario means I can be fussy about what I do. I have got enough of a reputation, mostly through the magazine articles and stuff, that I turn away interested parties on a very regular basis.
EW (00:17:12):
Have you seen anybody come to you from the book?
JT (00:17:15):
From the book? Couple questions through LinkedIn people, but not a company coming to me looking for help, as of yet. The book is too new. Let us face it, it was finalized right before Santa Claus this past year. Consequently, got to ask me that question in a year from now <laugh>.
EW (00:17:41):
Yeah, it does take a while.
JT (00:17:44):
And got to go do some promotions. I am starting to line up both IEEE speaking events, and trade show speaking events.
EW (00:17:55):
Are you going to them in person to do the talks?
JT (00:17:58):
It will be a mix. It depends on where it is and things like that. The stuff on the West Coast, I will definitely just go to directly. I have got a couple of invites to talk at a couple of universities back in New England. I will probably set up a week in the fall, where I go do all of those in one whack. Visit some family and friends while I am back there. Two birds, one stone.
EW (00:18:23):
One of the chapters I particularly liked from your book was the EMI chapter, which I think is like five or six, probably six. It answered a lot of the questions that I have always not been able to adequately answer for clients. About, "Okay, so what do we do when we say we are certifying our board? What does that mean? How do we fix it? How do we fix it in design? How do we fix it in application?"
(00:18:51):
How many times have you had to provide that information before you decided, I am just going to write it all.
JT (00:18:59):
I have lost count. There are many, many times I have been brought into problems, where they had EMI issues and they were not passing FCC Part 15. That has happened a lot. In addition, all of the ESD testing and stuff that goes along with medical devices. Again, I have had to deal with many clients on that.
(00:19:24):
Consequently, initially I wrote up things on noise for "Electronic Design," and ESD for "Electronic Design." Then it got morphed into the book.
EW (00:19:37):
ESD. When I started my career in the nineties, you looked at a board wrong and it had an ESD problem and exploded. Well, not quite that far, but-
CW (00:19:48):
<laugh>
EW (00:19:51):
The boards have changed, right? Things have changed, so that ESD is- I am not going to go and shuffle my feet on the wool carpet, and then touch my board. But it has changed, has it not? That is not my imagination.
JT (00:20:10):
I would say I am going to disagree and agree, and let me clarify what that means. I think that the mitigation of ESD has been a lot more systematic in the last ten or 20 years. Prior to that, people did not really understand what was going on that well, and they did not understand what the fixes needed to be.
(00:20:33):
Nowadays, if you consult with the right thing- My chapter on ESD speaks for itself. There is a systematic way of dealing with ESD, that is pretty well understood, and usually pretty successful.
EW (00:20:50):
In one of your chapters about how GPIOs work, you showed that there were- I want to say "ESD catchers." But there were- GPIOs used to be wired to the processor, but now they go through pull-ups and pull-down gates.
CW (00:21:09):
They will be diodes or something.
JT (00:21:11):
That has always been there for, oh goodness, I am going to go back- Heck...
EW (00:21:19):
But remember the first processor that I had pull-ups on.
JT (00:21:23):
ESD protection has been around for a good long time. I think the quality of it has improved some, but not recently. I would say that a lot of the progress on ESD was 30 years ago.
EW (00:21:39):
That tracks.
CW (00:21:40):
We are all slinging around bare dev boards now.
EW (00:21:42):
<laugh> I know.
CW (00:21:44):
Piles on our desk and <laugh> manhandling them.
EW (00:21:47):
I remember carrying boards and making sure they were well wrapped-
CW (00:21:53):
I still do that.
EW (00:21:53):
Before I left.
CW (00:21:54):
I still try to do that.
EW (00:21:56):
And now I will run up and down the stairs.
CW (00:21:57):
Well, all right.
EW (00:21:59):
As long as I am just holding it by the edges, I do not really care. I used to be so much more careful with ESD. Now it has got to be a really fragile board for me to care.
JT (00:22:11):
Well, in a development environment, I am not going to be too fussy about it. In a manufacturing environment, you got to be obsessive compulsive about it.
CW (00:22:18):
Oh, yeah.
EW (00:22:19):
Well, dev boards are much cheaper now than they used to be. But Nordic has this nRF5340 that they ship half dressed.
CW (00:22:32):
<laugh>
EW (00:22:34):
Only the top is enclosed and the bottom is not, and the buttons are on the bottom. So you are definitely getting close to things, that I still am a little squeamish about getting close to. And you are telling me it has been 30 years since I really had to worry about this?
CW (00:22:50):
I have not damaged anything with ESD in-
JT (00:22:51):
No, no. I am saying a systematic approach to ESD on chips, and to a lesser extent on boards, has happened I would say about 30 years ago. About 1990, maybe even a little earlier than that. Where if people knew what they were doing and made the effort, you could set up a board to be handled. No pain, no problem.
(00:23:12):
When you speak about the Nordic devices, I will point out that RF front ends, because of the large amount of capacitance associated with ESD protection, and the high frequencies of a lot of RF front ends, you cannot really get as good an ESD protection on an RF input, as you can on a low frequency digital input. Because you just cannot support the capacitance, if you are trying to get a two and a half gigahertz signal off of a antenna.
EW (00:23:49):
Oh, it has got a board antenna.
CW (00:23:52):
Just do not touch it. <laugh>
EW (00:23:53):
Well, no. Yeah.
CW (00:23:54):
You are skipping over-
EW (00:23:55):
That is probably on the plastic side.
CW (00:23:57):
You are skipping over the last week and a half, that we spent trying to debug piece of hardware, that was probably damaged by ESD.
EW (00:24:05):
<laugh> Yes. But that is one that actually was going to be fragile, because it was an analog something something. And so that was always going to be- Yeah. And it had some wiring issues in the beginning, so it might not have been ESD <laugh>.
CW (00:24:21):
Wow. Maybe.
EW (00:24:22):
We could have blown that ship up so many ways. Sorry. Jerry, what do you think software engineers need to know about electronics? <laugh>
CW (00:24:30):
<laugh> That is- Oh, wow. Yeah.
EW (00:24:30):
<laugh>
JT (00:24:32):
That is a gear shift right there.
CW (00:24:33):
We will be back in a half an hour, and see how you did.
JT (00:24:37):
Okay. What does software engineers to know about electronics?
EW (00:24:41):
What do new embedded systems engineers need to know about electronics?
CW (00:24:44):
<laugh>
JT (00:24:45):
Ohh, dear.
CW (00:24:46):
<laugh> That is a really big question. <laugh>
JT (00:24:49):
Let us see.
EW (00:24:50):
If you would be sensible to just say, "You should read my book." <laugh>
JT (00:24:53):
Well, I can definitely pimp the book, if you will pardon the expression. But I would say as an approach to design, I think that a book like yours, Elecia, is an excellent starter, for somebody that is dealing with the basics of embedded processors and things like that.
(00:25:21):
One of the things I do like about your first edition book- I have not read the second edition book. But one of the things I did like a lot about the first edition of your book, was the fact that you did not delve into specific processors that much. You kept it generic.
(00:25:35):
I thought that was a very good approach to it, because of the fact that as soon as you write a book about a specific processor, and I do not care what it is, it is going to be obsolete by the time the book has been out on the shelf for a year or two. "Oh, that processor does not exist anymore," or the things have changed. So you kept it generic.
(00:25:57):
In a similar manner with my book, it is a situation where I am not talking about specific processors, I am not talking about specific products. It is about all the things that are necessary for any electronic product to be successfully implemented.
(00:26:19):
I think I- Did I answer the question on that one, or not? It was such a global question. I am not sure.
EW (00:26:26):
What about the interactions between hardware and software? First, do you think they have gotten better over the last- Over your career?
JT (00:26:36):
I think that the people writing code have become less aware of the actual physical electronics. We have all used an ARM processor, where nobody really digs down into the guts of what is going on inside of an ARM processor. They just use the thing. It is a 32-bit processor. You know what- They have 64-bit ARMs now?
CW (00:27:10):
Only for mobile computers and mobile-
JT (00:27:13):
Okay. It is lots of 32-bit- It is mostly 32-bit processors. You are not really worried about floating point math accuracy. You know what the clocking rate and the MIPS information is. Beyond that, you treat it as a black box. A lot of designers nowadays treat the hardware as a black box.
CW (00:27:35):
Yeah. I think that is probably true. I am thinking about designs now and schematics that I read, and just how little is on them, beyond ICs with interconnections and maybe a bypass capacitor here and there. It no longer looks like electronics to me. They look like block diagrams.
(00:27:49):
Is that a consequence of just so much stuff being integrated into ICs, and they are just high density modular things that you do not need a lot of support electronics for? Or has something else changed?
JT (00:28:02):
It is we have gone to system on a chip, and the modular approach for sure. When I was getting out of college, board level design was done with a bunch of bipolar transistors.
CW (00:28:10):
Yeah.
JT (00:28:10):
Nowadays, oh, what used to be 20 transistors on a board, is now one eighth or one tenth of some analog front end. That has got the whole shooting match from ADC to phase lock loop timing acquisition. Everything necessary to produce a synchronous data stream. And you go from there.
CW (00:28:36):
Even the processors do not require as much periphery now, just in terms of crystals and supply voltages and all that stuff.
EW (00:28:43):
And they have onboard clocks, onboard flash RAM, ADCs. They have system on a chip.
CW (00:28:50):
Yeah.
EW (00:28:50):
It is all there.
JT (00:28:51):
Oh, sure. I have used my fair share of things like Atmel processes, and stuff like that. Where you basically pick something, "Okay, it has got suitable ADCs. It has got suitable clock synthesis. We have got counter timers that we need," and go from there. You definitely treat it as a box.
(00:29:10):
One of the things- The chapter I do on control and feedback- One of the things I advocate in that particular chapter is we want to design control loops that are pretty much all digital. Everything except for the ADC on the sensor, and the DAC on the control side. But everything else is digital.
(00:29:33):
That is extremely reliable. It is extremely predictable. And it is what I am a big proponent of. I am old-school analog from way back when, but I know the promise of digital stuff. When I can get the A-to-D converter and the D-to-A converter for a dollar 50 each or less, or it is just embedded in the controller chip, that is a good way to go.
EW (00:30:00):
All you have to do is put on the low-pass filter, and everything is golden.
CW (00:30:02):
<laugh>
JT (00:30:04):
Wellll.
EW (00:30:04):
<laugh>
JT (00:30:04):
Yeah. Kind of. Aliasing is another whole thought. It is actually something I would like to add a little bit more on the subject of filtering, in a second edition of this book. But that is downstream.
EW (00:30:22):
Are you looking forward to doing a second edition?
JT (00:30:27):
We will see. I took my time on this one. I am at about 600 pages. I wrote about most of the things that I thought were important. The one thing I would like to do in a second edition of this book, relative to the material that has all been covered, is what I am going to call more "quantitative examples."
(00:30:50):
I talk about a lot of concepts. But one of the things I always find when I am teaching a topic is, it is always good to teach the concepts, but then show a problem that has been worked out end to end. From concept to finish, and all the details in between. A lot of people will use that as a useful learning tool.
EW (00:31:14):
In my chapter two- Which was kind of funny, because it was very similar to yours. I try to convince people to diagram their systems. I give them different diagram methods and blah, blah, blah. But just a system diagram of how it should all go together.
(00:31:30):
You give examples of them. You give several examples of what they would look like. It was really interesting to see the difference between, "Here are some pretty simplified but still complex examples," from your book. Where I have is a very simple example, because I am trying to get you to build it, and so I have to go simple, simple, simple.
(00:31:54):
Even at the end, it is not nearly as complicated as yours. But yours was a better example of how a system really looks when you take in its whole complexity, and not just baby steps that I was providing.
JT (00:32:09):
Possibly so. I cannot judge, because I would have to go back and look at your examples. Are they the same set of examples as in the first edition, as in the second edition? Or are they new to the second?
EW (00:32:26):
No, no, it is the same basic stuff. The second edition has a different diagram type and some other examples. But it is still very much, if you are building a system, build up a diagram first. And if you are new to a system, try to figure out the system using a diagram. Do not just try to figure out the whole system right away. Just steps.
(00:32:51):
But I liked your diagrams, because they were more complete. So I like this idea of being able to give more examples, and more quantitative information. O'Reilly strongly encouraged me to have a GitHub repo. They did not do that for you.
JT (00:33:13):
I had the offer of that sort of thing, and I said, "Well, this is not a software book."
EW (00:33:17):
Right.
JT (00:33:21):
I have a chapter in here on coding and software issues. But one of the things I say on the front page of that is this is not a coding book, and I am not a coding expert. Hell, I have programmed PDP-11s <laugh> from the front panel. If that is not the dark ages of computers, I do not know what is. But I am not a coder. I am not a software person.
(00:33:48):
One of the things I did actually when I wrote that chapter, is I had a couple of friends of mine who spent their life writing code for Apple and zillion other places. I said, "Look, guys. I want you to read and review this. Fling stones. Feel free." Because it is an area that I have got some working knowledge in, but I am not an authority on software. And I would never pretend to be.
EW (00:34:17):
I have a little bit of software in my repo, but I also have a lot of links to other things. But also I have a few calculators. I think you talked about it in one of yours- The lower power section- Battery section.
JT (00:34:33):
Mm-hmm.
EW (00:34:35):
Where you need to go through and figure out what the average current consumption is. You can do that either by measuring things, and you have to do that in different states. Or you can do it by building it up from the data sheets, and still you have to think about the different states. If you are in sleep state, you have this much, and so how long will your batteries last.
(00:34:58):
In my repo, I have a couple of worksheets. I am so happy with that, because like what you were saying where you need calculated examples. This gave me a way to say, "Look, I have a calculated example. I cannot put it in the book because it would take 400 pages, but go look there. You can type in your own numbers, then you can copy it and make your own thing." Do you have a way to do that?
JT (00:35:32):
It depends upon the project. I have done situations where- Okay, I have been involved in cell phone design, and the cellular designs tend to be heavily, heavily defined as an entire end-to-end software device, and modeled. That includes a development of models of power systems, and battery life, and things of that nature. If you have got the resources, the manpower, and the time to do all of that, that is great.
(00:36:07):
But if you are trying to get a product out the door quickly, I think that doing an estimate on paper is a reasonable starting place. From there you can-
(00:36:23):
So you get two extremes. Figure it out on paper, manual estimations. Or do a full end simulation of the thing end to end, where you can pull power consumption numbers out of something else. One takes time to execute, probably a bit more accurate. The other one tends to be a little bit more depends upon the skills of the person making the estimates.
CW (00:36:48):
I want to ask- Changing subjects a little bit. Ask some more detailed questions about your book in particular. One of the things that was interesting is you have a whole section on spacecraft <laugh> electronics.
JT (00:36:59):
Sure.
CW (00:36:59):
First of all, could you describe how spacecraft are a unique challenge. But also what made you decide that that was something that you wanted to focus on, or at least touch on?
JT (00:37:12):
Well, one of the things I think- I am pretty sure it is mentioned in the book, is in the special systems section- The systems that I write about are systems that I have actually been involved with. I have done stuff for satellites. I have done stuff for both LEO satellites and for geostationary satellites. I have done medical devices. And on and on. The list is there.
(00:37:33):
Consequently, the reason I picked and did something on that topic, was I had hands-on knowledge in that area, and could consequently comment fairly intelligently on it.
(00:37:44):
The challenges for satellite stuff is interesting. Let me think. What is it? There are four different things that are important in satellites, okay. You have got to account for vibration and shock, mostly due to launch situations.
(00:38:01):
There was a ton of CubeSat stuff that died, because it never came live up in the air. Because of the fact that people did the design with a consumer electronics mentality, and they had not accounted for the vibration shock of launch. It was broken once there. So there is vibration and shock.
(00:38:21):
The next one is vacuum. Satellites are not pressurized. I think Sputnik was pressurized, <laugh> but everything after that, no way. They do not pressurize that stuff. That would add a lot of cost and weight. So you have got to have electronics that will work in a vacuum.
CW (00:38:39):
That is something that does not- I guess capacitors would be a big thing.
JT (00:38:45):
Oh yeah.
CW (00:38:45):
Yeah.
JT (00:38:45):
Oh yeah. Because the typical electrolytic capacitor will just blow up in a vacuum.
CW (00:38:53):
And sometimes not in a vacuum <laugh>.
EW (00:38:54):
<laugh>
JT (00:38:55):
Yeah, this is true too. I am not a fan of electrolytic capacitors. There are ways around them if you are clever. So you got the vibration and shock. You got vacuum. You got radiation hardening.
(00:39:07):
Radiation hardening is a nasty can of worms, because no one thing solves radiation hardening. It has to be a mixture of a radiation hardened semiconductor process, plus shielding, plus physical placement in layout, and a bunch of other things. So radiation hardening, you get to be a little bit of a gnarly problem to solve.
(00:39:29):
And lastly in satellites, the other one is thermal extremes. You can have a situation with a satellite, where the electronics can go through some absurd thermal variants. Direct sunlight, indirect sunlight, not indirect sunlight. Close in earth orbit, or closer. Or way the heck out there past the gas giants, where you cannot rely on solar power. Very cold or very hot. So that has got some challenges to it, to say the least.
CW (00:40:08):
I think people have misconceptions about space too. "Oh, space is cold." "No. Space is terrible at transferring heat." <laugh>
EW (00:40:15):
It is different.
CW (00:40:15):
It is a very different problem.
JT (00:40:16):
The funny thing is, the first time somebody does electronics for a satellite, they start thinking about heat sinks and stuff like that. And I go, "Ah, it is all conduction."
CW (00:40:25):
<laugh> Yeah.
JT (00:40:27):
In a vacuum there really is no temperature.
CW (00:40:29):
Right.
JT (00:40:31):
It is a different way of thinking.
EW (00:40:34):
But there has been- Well, I guess it was a few years ago that Planet Labs and a few other folks were doing consumer grade electronic disposable satellites.
CW (00:40:46):
The keyword is "disposable," right? <laugh>
JT (00:40:49):
Yeah. Well, here is the thing. The things I am talking about are this type of things that have to be in there for any satellite to work. If you cannot make it through the launch vibration and shock, you are dead. If you have got something in a vacuum, and the components are blowing up, because they cannot deal with a vacuum, you are dead. Radiation hardening. Again, you cannot turn the radiation off. Sorry. Just like on thermal extremes, you cannot turn the sun off.
(00:41:19):
So you got to have this stuff. Even for low- Things like SpaceX. They are doing a distributed network internet communication via LEO satellites. All of those things have got to have the capability to deal with all of those.
(00:41:42):
Now the good news is, you can generally get the cost of a satellite down in mass production methods. There are a lot of geostationary satellites that were done as one-offs. That is expensive as we all know. Whereas producing a hundred of the same thing, much cheaper.
CW (00:42:04):
Just makes me more impressed by Voyager, which they have recently fixed again. It is still out there doing stuff <laugh>.
JT (00:42:10):
Yeah. I actually know somebody <laugh> that was involved with that. He is kind of retired now, but we had some interesting conversations on email.
EW (00:42:21):
When you started your career- Or when you started using watchdogs for processors, did you kick them or pet them from the very beginning?
CW (00:42:30):
<laugh> What kind of question? <laugh>
JT (00:42:32):
Oh, my! Well, first of all, you do not want to get in trouble with PETA, you know, ethical treatment of animals, so you do not want to be talking about kicking dogs.
CW (00:42:41):
<laugh>
JT (00:42:41):
Okay?
EW (00:42:45):
I am all in favor of petting or feeding the watchdogs now. Yes.
JT (00:42:48):
Okay.
EW (00:42:48):
But I did initially use the other terminology, because that was what I was taught.
CW (00:42:54):
It was already in the code; there is nothing you can do about it.
EW (00:42:56):
Yeah. I just did not ever think about it, because it was not- But yes. So I wondered if you had that same epiphany of, "Oh my God! What did I just type?!"
CW (00:43:06):
<laugh>
JT (00:43:09):
No, I was aware of all of that stuff. I am trying to think who made me aware of all- Probably one of the management training courses I had when I was at IBM. They were really good at teaching you how to be politically correct, when they were putting you up in management. I was already aware of that stuff. But yeah, you do not want to be kicking dogs. It is like, okay, what is it? Master-slave circuits. Did O'Reilly get on your case relative to cannot use the words "master" and "slave"?
EW (00:43:43):
<sigh> Yes. Although I had already written, "This is what we used to call it. That is why it is 'MISO' and 'MOSI.' And this is what we are going to call it in the future, and just suck it up and deal with it, folks."
JT (00:43:57):
I ended up with- I wanted the first- I think I ended up using "manager" and "subordinate."
EW (00:44:04):
I used "micro" and "sensor."
JT (00:44:05):
Okay. Well-
EW (00:44:05):
It is not always a sensor, but it was-
CW (00:44:10):
What was the other one that has been used? "Controller" and?
EW (00:44:13):
"Peripheral."
CW (00:44:13):
"Peripheral" and "controller," which is kind of muddy sometimes.
JT (00:44:17):
Yeah. The reason I liked "manager" and "subordinate"-
EW (00:44:18):
I wanted "M" and "S."
JT (00:44:21):
Is I wanted the "M" and "Ss" for that. "Manager" and "subordinate" fixed that, such that all the acronyms could remain the same.
EW (00:44:29):
Yeah. Yeah, it was important to me that the acronyms remain the same. Although "SDI"-
CW (00:44:36):
"Serial data out"?
EW (00:44:38):
Yes. "Serial data out." That is what it is. Sure.
CW (00:44:40):
<laugh>
EW (00:44:40):
<laugh> "Sensor data out," "subordinate data out."
JT (00:44:46):
I do not think it was ever "slave data."
CW (00:44:49):
Well, on the chip we were just using- That I will not mention for reasons.
EW (00:44:53):
Because it is terrible! <laugh>
CW (00:44:55):
The serial audio interface has "SDI" and "SDO," and that has not changed, whether it is in charge or not in charge.
EW (00:45:01):
It is true.
JT (00:45:03):
I think those are slated as "serial" for their labeling.
EW (00:45:07):
A question from Andre, who has purchased your book and is enjoying it. "What are your recommendations for folks to improve electronics design skills? And, do you have different recommendations for beginners, versus experienced professionals?"
JT (00:45:23):
Oh, wow. Interesting question.
EW (00:45:26):
Let us go with beginners first.
JT (00:45:27):
I would say that for beginners, they need to learn the ideal systems first. Going to your book for embedded systems, great way of learning the ideal systems. Because you deal with things mostly as a, "This is how it is supposed to work." After you get out there with the basics on this is how it is supposed to work, at some point you are going to start running into the, "Oh, but it does not work that way in the real world."
(00:46:00):
That is when you want to start looking at the book I wrote. Because of the fact it is all about things not being ideal. It is not all ones and zeros when you got noise on it. It is not nicely lined up with the clock, because of the fact that there are all kinds of latency in the system. Things like that.
CW (00:46:26):
Funny, that reminds me of the progression of physics education. Because undergraduate, you start out and like, "Well, here are these equations, and everything is ideal." And they assume a spherical chicken, and whatever.
(00:46:37):
Then you start peeling off the layers of idealness, the further you go. Until when you are in grad school, it is like, "Okay, here is this thing. We cannot actually solve this analytically. You have to do this numerically." There are all these problems. And "Here is what happens when quantum mechanic starts biting you." That gets worse and worse.
(00:46:59):
I think in practical technical education, that is not the way the path- It may be the way the path works by default, but maybe it should not. <laugh>
EW (00:47:11):
It is easy to get distracted by the non-ideal stuff, and then kind of lose hope, because-
CW (00:47:16):
Yeah, but I just worry-
EW (00:47:17):
There is so much depth to everything!
CW (00:47:19):
I worry since it is your- I do not know. Since you are looking for a career or a job, that people get surprised when they-
EW (00:47:28):
Have to keep learning?
CW (00:47:29):
Well, not that they have to keep learning, but "I was taught this. Why are you doing this?" "Oh, because it does not actually work that way," which can be a bit of a shock.
JT (00:47:39):
There is a lot of that. But the thing I tell the engineers on a learning curve, I say, is "It is always good to glom onto people that can help you, and have different insights." It is like I am the first person to grab somebody that knows software and coding and say, "Is this viable? Can we do this?" Because it is not my field of expertise.
(00:47:57):
One of the things that is important about any engineer, in my opinion, is to understand what their limitations are. And their willingness to address that limitation by consulting with others with expertise that is pertinent to what they are doing.
CW (00:48:11):
That is a really good attitude. Because the number of times we have had to fix things in software, <laugh> because somebody figured you could just fix it in software without asking, is high.
JT (00:48:22):
Yeah. Well, we have all done the duct tape thing. Where it is, "Oh, this signal is glitching as it transitions." And you basically go, "Okay. We will fix that in software, by polling the port 20 times and making sure that it has settled out," kind of thinking.
(00:48:38):
So you end up putting band-aids on all products. It is one of the reasons I try to avoid buying first generation products, it is because I know all the duct tape that goes into a first generation product.
EW (00:48:52):
First generation processors are the ones that I want to avoid forever.
CW (00:48:56):
How about zeroth generation processors? How do you like those?
EW (00:48:59):
<laugh> Sorry.
JT (00:48:59):
Ohh.
EW (00:48:59):
You are always fighting the last battle. <laugh>
CW (00:49:04):
<laugh>
EW (00:49:04):
I think that your point of admitting that when you do not know something, is something we forget to do and it is very useful.
JT (00:49:14):
Well, yeah. I think the people that claim to know everything, are usually pretty fresh out of school.
EW (00:49:21):
And for experienced professionals. Do you have recommendations for improving-
CW (00:49:29):
Continuing.
EW (00:49:29):
Our education?
CW (00:49:30):
Yeah. Continuing education?
EW (00:49:30):
Yeah, but where?
CW (00:49:31):
Yeah. Yeah.
JT (00:49:34):
Well. Yeah. Depends upon if you are self motivated for self-education, and read books on your own. Or you like to take classes. There are plenty of options in the self-education world. Nowadays, for a lot of people, "Oh, let us go to YouTube. I am sure there is an answer there."
CW (00:49:54):
Difficult to pick the- <laugh>
EW (00:49:57):
Swim through. <laugh>
CW (00:49:58):
The chaff from the- <laugh> Yeah.
EW (00:50:00):
But there are good channels.
CW (00:50:00):
Yeah.
JT (00:50:00):
Oh.
EW (00:50:00):
Like "Ze Frank." Wait. No.
JT (00:50:02):
<laugh>
(00:50:04):
Lots of chaff. Definitely.
CW (00:50:06):
What do you do for yourself?
JT (00:50:08):
With respect to?
CW (00:50:09):
Continuing education.
EW (00:50:09):
Learning.
CW (00:50:10):
Yeah.
JT (00:50:12):
I have gone from being the student- In electronics anyhow, I have gone from being the student to the teacher in a lot of it. I think since my second job out college, I was a technical lead on the program, and it has been that all the way since then.
(00:50:32):
But I try to read new books. I am obviously on the trade magazine publications all the time, to find out about the newest, latest, and greatest. But I would say it is stay curious and keep learning.
(00:50:48):
There are other fields that I go- Definitely I consult with other people that are much more knowledgeable. It is like one of the things I do- One of my fun things in life is I race sailboats. Down here in San Diego, we have got an incredibly skilled number of people in sailboat racing, that I have learned an incredible amount from.
(00:51:11):
Sometimes you are the master, and sometimes you are the student. In the world of electronics, I tend to be the master. If it is not the digital things. If it is not the coding things. On other topics, I definitely embrace being the student. I like to learn. I always like to learn.
EW (00:51:29):
I have generally found that things you learn may seem pointless at the time. But even the practice of learning is good to maintain. Nothing is ever truly wasted.
JT (00:51:43):
If nothing else. You can use it when you compete on "Jeopardy" in a few years. Right?
EW (00:51:48):
<laugh> Have you ever been tempted?
CW (00:51:49):
<laugh>
JT (00:51:51):
I have actually been tempted. But I do not have the geography and the history knowledge that I would need.
EW (00:51:57):
I do not have the pop culture <laugh>. It has always been really embarrassing. Never had the pop culture for any sort of quiz show.
CW (00:52:04):
Wait a minute. Were you not almost on it?
EW (00:52:07):
As a teenager. Yeah.
CW (00:52:08):
Oh, okay.
EW (00:52:08):
I was going to "Jeopardy" tryouts, the second pass. When my brother got hurt and I did not get to go. I still hold him responsible for me not winning! No, that is <laugh> not true. I realize now I would not- I did not- I really did not then, and I still do not have pop culture well enough down that I could have. It is really embarrassing when you miss questions about "The Rolling Stones." You are like, "I do not even know who that is."
CW (00:52:35):
<laugh>
JT (00:52:36):
Oh boy.
EW (00:52:37):
That was a quiz bowl. It was high school. It was really embarrassing.
JT (00:52:41):
Well.
EW (00:52:43):
Okay, so what is the animal on the cover of your book?
JT (00:52:48):
The animal is a- Everybody says it is a kangaroo, but it is a yellow-[footed] rock-wallaby. They are both marsupials. They are both from down under in Oz, but a yellow-footed rock-wallaby is a little bit smaller than a regular kangaroo.
EW (00:53:09):
Did you have any choice in your animal?
JT (00:53:12):
They gave me a bird or a fish initially, and I said "No."
CW (00:53:16):
<laugh>
EW (00:53:16):
<laugh>
JT (00:53:16):
I fed them back, I said, "Look, we are creating something new here. So how about some pandas or some other animal with its children?" They came back with the wallaby, with the joey in the pouch, and I said, "I am in love. Thank you. I will deal with that. I can live with that."
EW (00:53:41):
I wanted a dinosaur or a microbe.
CW (00:53:43):
<laugh>
JT (00:53:46):
Okay.
EW (00:53:47):
They were just like, "No." They said, "Well, we have Michael Barr's book." And I said, "That is a tick! No, absolutely not. Absolutely not!" I admit that I love my nightjar.
JT (00:54:02):
If you are happy with what you got, you are happy with what you got. It is that simple.
EW (00:54:05):
Yeah. I have a great eared nightjar. It has got its fluffy little ears that make it look like the serious expression is very fleeting, which is about right my book.
JT (00:54:16):
And now you have got a color version for the second edition.
EW (00:54:19):
I do, and I like it.
JT (00:54:20):
Mm-hmm.
CW (00:54:22):
Before we let you go, I want to ask if there is a section of your book that you are particularly proud of, and you want to make sure that people definitely read.
JT (00:54:33):
There are a number of different places in the book, where I introduce material that is not out there in common usage, but it was buried in journal articles or things like that. And those are in the ESD EMI section, and also the control system section. Because the controls section- I have done a lot of control systems.
(00:55:03):
So the controls section is something I said, "I cannot find this approach anywhere. I can find parts and pieces of it in journals. I can find parts and pieces of it in a couple of specialty books." The type of book that sells maybe six copies, and three of them were to mom, dad, and the kids. And the other three sold- Something really specialized that was out there.
(00:55:35):
What I did with the control systems thing, was I set up something where I said, "I am going to define and teach people how to do control systems. And they do not have to be a five star math nerd." Consequently, how I teach control systems in this book is it is simple, it is graphic. The parts and pieces of it are backed up by various journal articles and things of that nature. But the overall approach to it is pretty unique to this book.
(00:56:09):
I am pretty happy with it, because I have run into a lot of screwed up control systems over the years. You would be amazed at the number of people that do not know how to set up a control system. They are doing it in industrial control systems all the time. They are just blindly turning the knob until it works.
EW (00:56:25):
You have a lot in this chapter. There is a little bit of state machines, but mostly it is-
CW (00:56:30):
But does it have Bode plots?
EW (00:56:32):
No.
CW (00:56:32):
Thank God!
EW (00:56:33):
But it did have a transfer function.
CW (00:56:35):
That is fine.
JT (00:56:36):
It does have Bode plots in it.
CW (00:56:37):
Oh, okay.
EW (00:56:38):
Oh, I am sorry. I got to read a lot of your book, but this chapter I did not, and now I am really sad. Oh, it does have Bode plots.
CW (00:56:47):
Control systems is one of those things that it is super important, and it can be taught in so many different wrong <laugh> ways. I think I got taught in three different wrong ways before- I am not sure I have ever been taught the right way, actually.
EW (00:57:00):
You should read this chapter.
JT (00:57:01):
Read the chapter.
CW (00:57:02):
Yeah, I am going to read that.
EW (00:57:03):
PID does not come in until the last third. But that is because PID is usually a hammer, that you do not need until you are doing something marginally complicated.
CW (00:57:16):
And yet everybody just uses that on everything.
JT (00:57:19):
PID controllers. You do not need a PID controller. Most of the times you need a PI controller. Proportional-integral.
CW (00:57:28):
Yeah. Okay.
EW (00:57:28):
Exactly!
JT (00:57:30):
That usually gets it done for 90% of the world. The one thing that is a little bit ugly is the fact that everybody has heard of Ziegler–Nichols tuning for control systems.
EW (00:57:40):
<sigh>
JT (00:57:40):
<laugh> Yes, I understand that sigh. Because Ziegler-Nichols tuning is not the way to tune a PID system, unless you want to sit marginally on the edge of stability.
EW (00:57:57):
Why would you want that? Because when you do the manufacturing, they are not all going to be the same. You want something that is really stable so-
CW (00:58:03):
Living on the edge! You got to tighten everything down! I do not know. <laugh>
JT (00:58:08):
Well, here is the thing. Where are all the variants? If it is a digital control loop, where are all the variants in the system?
EW (00:58:15):
In the sensors.
JT (00:58:17):
Ahh.
EW (00:58:18):
In the temperature effects from the sensors.
CW (00:58:20):
<laugh>
EW (00:58:21):
In the noise.
CW (00:58:22):
<laugh> Sorry.
JT (00:58:24):
I would say that that is part of it. But I would also say that the device under control is probably the thing that varies the most, in most control systems. Especially if it is something mechanical.
CW (00:58:38):
Hmm. Mm-hmm.
EW (00:58:41):
Yeah. Yeah. Okay. Yes, I agree. I just tend to blame everything on the sensors, because that is the part I can see more.
JT (00:58:52):
It is probably the thing you have had the most trouble with then too.
CW (00:58:54):
<laugh>
EW (00:58:55):
Yeah, of course.
CW (00:58:56):
I think we do tend to be input focused a lot, because- Yeah.
EW (00:59:00):
The output is the output.
CW (00:59:01):
Yeah.
EW (00:59:01):
Got to do whatever it is going to do.
CW (00:59:03):
<laugh>
EW (00:59:03):
Well, Jerry, it has been really great to talk to you. Do you have any thoughts you would like to leave us with?
JT (00:59:11):
Well, for designers, I would say take a look and read the book. It is available on Amazon. "Applied Embedded Electronics," and its subtitle, "Design essentials for robust systems." I think that the subtitle is just as important as the primary title. Because of the fact that these are things that are necessary to make a system reliable and predictable and even cost efficient. So I am definitely going to be a proponent of the book.
(00:59:41):
The thing I will say to most designers is, they will say, "Try to get yourself hands-on early in the design cycle, and get off the simulator." I am a big fan of simulators when it comes to semiconductor development. You got to live on a simulator to design chips.
(00:59:57):
But when it comes to board level and system level design, I say, "Please, get onto the hardware as soon as you can." I think that is always a smart course of action, because then you will have the surprises early in the game, rather than late in the game.
EW (01:00:12):
Our guest has been Jerry Twomey, principal engineer at Effective Electrons, and author of "Applied Embedded Electronics." You can find a link to his book in the show notes, as well as a 30-day link to the O'Reilly online learning, which should give you enough time to check out his book and mine. Then you can decide if you want to buy them in paper. Or electronic. I am not fussy.
CW (01:00:39):
Thanks, Jerry. It was good to talk to you.
JT (01:00:41):
Good talking with you folks as well. Thanks for having me.
EW (01:00:44):
Thank you to Christopher for producing and co-hosting. Thank you to our Patreon listener Slack group for questions. And thank you for listening. You can always contact us at show@embedded.fm, or hit the contact link on embedded.fm.
(01:00:57):
Now a quote to leave you with, from Ada Lovelace, “That brain of mine is something more than merely mortal, as time will show.”