Transcript from 495: Shortcut the Difficulties of Reality with Cindy Harnett, Christopher White, and Elecia White.

Welcome to Embedded. I am Elecia White, alongside Christopher White. Our guest this week is Cindy Harnett, professor of electrical and computer engineering at the University of Louisville. We are going to talk about new sensors, soft robotics, and well, probably our alma mater.

CW (00:00:25):

Hi Cindy. Welcome to the show.

CH (00:00:27):

Thanks, Chris. Good to be here.

EW (00:00:28):

Could you tell us about yourself, as if we met at our, oh my God I do not want to count that high, reunion?

CW (00:00:36):

<laugh>

CH (00:00:38):

Okay! Since graduating, I went to grad school and I drifted from physics to electrical engineering and going to applied physics. Then working at National Lab, and then going into academia. I have been a professor in the electrical and computer engineering department at U of L for almost 20 years now.

EW (00:01:02):

Wow. Congratulations. We are going to do lightning round, where we ask you short questions, and we kind of want short answers. If we are behaving ourselves, we will not ask, "Why?" and, "How?" And, "Are you sure about that?" Are you ready?

CH (00:01:14):

Almost. <laugh>

EW (00:01:18):

It does not help you to pull up the questions that were in your sheet. Those were only examples.

CH (00:01:24):

Okay. Well, I do have a long acronym on there, and I am not sure if I can remember what it is.

CW (00:01:29):

Okay. We will be sure to do that one.

CH (00:01:31):

Oh no! Okay.

CW (00:01:33):

What Mudd dorm or dorms did you live in?

CH (00:01:36):

North Dorm.

EW (00:01:37):

That negates the next question of unicycles or fires.

CW (00:01:42):

She can still answer.

EW (00:01:43):

Okay. Unicycles or fires?

CH (00:01:46):

Fires. <laugh>

EW (00:01:46):

Yeah. <laugh>

CW (00:01:46):

<laugh> Favorite sensor?

CH (00:01:51):

The time of flight sensors, or LIDAR.

EW (00:01:56):

Favorite soft robot? Fictional or not.

CH (00:01:58):

Baymax.

EW (00:02:00):

Oh, yeah. Okay, that was kind of a soft ball, was it not?

CW (00:02:02):

How are you supposed to pluralize "MEMS"?

CH (00:02:05):

It is hard. If you are writing a paper and everybody says MEMS for "micro-electromechanical systems" and you are just talking about one system, no one calls it a "MEM."

CW (00:02:16):

<laugh>

CH (00:02:16):

Like, "We made a MEM." So you have to work it in there, "In the field of micro-electromechanical systems, or MEMS, blah, blah, blah."

CW (00:02:27):

Not "MEMSes"?

EW (00:02:27):

<laugh>

CH (00:02:29):

No, and we just avoid the-

CW (00:02:30):

"MEMES"?

EW (00:02:30):

It is always MEM sensors.

CW (00:02:34):

That is right.

(00:02:35):

Right next to your PIN number.

EW (00:02:41):

What was your concentration?

CH (00:02:43):

At Harvey Mudd? I was physics, but I tried to be an art minor.

CW (00:02:49):

Yes. The <laugh> concentrations were a lot of trying to be something. I remember that.

EW (00:02:54):

I got an art one.

CW (00:02:55):

I know.

EW (00:02:57):

I did art and education.

CH (00:02:58):

Awesome.

EW (00:02:58):

An overachiever in that section. Okay. Let us see.

CW (00:03:04):

Favorite acronym?

EW (00:03:06):

Yeah. That is the one.

CH (00:03:07):

Ohh, yes. I had a project where we had environmental sensors, so we cooked up this name, "SALAMANDER," for it. It is "serial amphibious linear arrays of micro and nano devices for environmental research."

CW (00:03:23):

That is one of the longest ones I have ever seen.

EW (00:03:28):

And it worked out. She did not have to get rid of the "E" or anything. That is good.

CW (00:03:31):

How long did that take to devise?

CH (00:03:34):

Maybe about a week.

CW (00:03:35):

<laugh>

EW (00:03:35):

<laugh>

CH (00:03:35):

I had two summer students, and we were working it out.

EW (00:03:42):

Complete one project, or start a dozen?

CH (00:03:44):

One project.

CW (00:03:46):

Do you have a tip everyone should know?

CH (00:03:49):

This one I heard from someone in our hackerspace. If you need to put things in Ziplocs, write the label on the Ziploc before you put the things in. It gets all lumpy, and you cannot write on it anymore after that.

CW (00:04:02):

That is a good one. <laugh>

EW (00:04:04):

Makes sense.

CW (00:04:05):

Just basic. It works.

EW (00:04:08):

It sounds so obvious, and yet I certainly have done the other way. <laugh>

(00:04:14):

Okay. You sent me a few papers from your lab, and I have lots of questions. But Chris has not read the papers, so we are going to have to talk a little bit about what they were.

CW (00:04:25):

I assume that the listeners have not read the papers either.

EW (00:04:27):

Oh no. We can-

CW (00:04:28):

We have listeners. Remember?

EW (00:04:29):

Yeah, but we do not care about them <laugh>. Sorry, listeners. We do care about you a lot. They will have links to the papers.

CW (00:04:36):

Okay, well that is fine. It is not just for my benefit.

EW (00:04:39):

Right. Okay <laugh>. Let us start with the actuator one, if you do not mind.

CH (00:04:47):

Okay. The SESAME one.

EW (00:04:50):

Yeah.

CH (00:04:51):

Not our longest acronym. This one we have long been working on, using strain to go from something planar, to something that is folded up. Usually I have been doing that at the microscale, using silicon wafers and heating them up.

(00:05:08):

But you can also do something like a slap bracelet. If you can imagine one of those, where there are two layers in a slap bracelet. One of them is curved one way, and one is curved the other way. When you put them together, neither layer is really happy. So you have got two states of the thing. It can be flat, or it can be rolled up. That is the integrated stress concept.

(00:05:30):

Another way you can do that, is by stretching a piece of spandex, for example, something elastic, and then applying things. People are doing 3D printing, for example, on stretch fabrics. When you let it go, it props up. It goes 3D. So the SESAME project was that concept, but with wire inserted into the stretch spandex.

EW (00:05:59):

Not just any wire.

CH (00:06:01):

Yeah, that is right. Not just any wire. We have some shape memory alloy. By now you can get this stuff commercially fairly easily. It was developed in the sixties, I believe. A combo of nickel and tin, so it is called "nitinol."

CW (00:06:20):

Nitinol! I worked with nitinol at a previous job. Yep.

CH (00:06:23):

Yep. Now you can order it by the foot or by the meter. We have a nice machine that we can use to insert that using overstitching. It is a fancy sewing machine. We were able to put the nitinol wire in the stretched material, and let off the stretch, by releasing it from the hoop we had everything stretched out in, and we got this kind of egg crate structure.

(00:06:49):

But when you heat the nitinol, it stiffens and it flatten everything back out again. So you got this soft sheet that you can crumple up, but then you could heat it and it would flatten out again.

EW (00:07:05):

Looking at the video, which was also entertainingly musical, it looked like it was embroidery hoops, and then the starched fabric, and then an embroidery machine tacked down, sewed down, the wire. Then once you let it out of the embroidery hoop, it would go to its- I want to say deformed position. Its 3D position.

(00:07:38):

When you, I guess, put some current on the wire- I did not see that part. It would go mostly flat. Is that- Did I get all that right?

CH (00:07:51):

Yeah, that is right. So something that starts out as a circular pattern. We have a machine that can spool out wire and then do a zigzag stitch over it, so it does not stab the wire, most of the time. It really firmly attaches it down. But everything is stretched out in this hoop. It is all flat.

(00:08:11):

The great thing about nitinol is, if you want to heat it, well, it is a wire. So you can just run an electric current through it. If you have an embedded system, you could put current through a transistor. It takes maybe half an amp. Quite a lot of power, but it will flatten everything out.

EW (00:08:32):

The thing that really struck me with this paper, was the circle. You have- Okay, let us go back to the visualization of the embroidery hoop.

CW (00:08:46):

<laugh>

EW (00:08:46):

You use the embroidery machine to make a circle of the nitinol, and then when you let it go, it goes to a saddle shape, a Pringles shape.

CH (00:08:59):

A Pringle. Yep.

EW (00:08:59):

And then you heat it up, and it goes flat again. I see this shape a lot with curved origami, which is something that I have talked about on the show, and it is just something that I do a lot of.

CH (00:09:13):

Hmm.

EW (00:09:13):

It seemed like the reason for it was similar to the reason for origami, is that that is how you get a- I want to say a less strained shape, given a circle.

CH (00:09:30):

Yes. You have this trade-off between the boundary. The wire wants to be flat. It takes some energy to bend a wire. And the surface area of the circle. This fabric is stretched. It is unhappy. It wants to be smaller. When you have a surface that has an edge, and not quite enough material in the area of it to fill in that edge, it has to go saddle-shaped. It has got negative curvature.

(00:10:00):

The opposite of that, is if you have too much material, more than pi r squared, then that material likes to go dome-shaped. Mountain or valley. If it is the perfect amount of material, it is going to be flat. But either too much, or too little, and something has got to go out of plane.

EW (00:10:22):

You also have this thing where you can put the nitinol in a figure eight pattern. It is not figure eight, it is like a figure 36. It is a bunch of circles put together, and then it crumples up. I think you said "Egg carton," and that is about what I would say too. How did you come up with that pattern?

CH (00:10:49):

That one- There is a natural array that forms, when just putting circles next to each other. We wanted to have a lot of separate Pringles, if you want to call them that, or actuator cells, run using one current. Otherwise we would have to have a multiplexer and a lot of circuitry in the fabric. So we said, "Let us just have a whole bunch of them in an array, and fire them up all at once."

(00:11:17):

Having this S-shaped serpentine pattern with one wire going through, there are actually a lot of wrong ways to draw that, so that it does not form the loop that you want. There is probably an algorithm for it. We did a little trial and error. Yeah, we think we have got a six by six array of circles that is all drawn by one path, but it took some graph paper to do.

EW (00:11:42):

That was another area where when I saw what it looked like in its crumpled state, I was like, "Oh. I know that origami pattern." It was super weird to realize that the pattern you use to get to it, is only somewhat different than the pattern I would have used to get to it in paper.

CH (00:12:02):

Do you have an algorithm for that? To save me the graph paper.

CW (00:12:06):

<laugh>

EW (00:12:06):

I have whole Python scripts. <laugh>

CH (00:12:10):

Yay!

EW (00:12:10):

More importantly, there is the origami simulator, that will show the results. That I bet will show some of the nitinol results. I just want to go off and try this, because it is- What if these do overlap? It would be so exciting!

CH (00:12:28):

Mm-hmm.

CW (00:12:30):

You are going to put nitinol in your origami.

EW (00:12:32):

I do not need to. The folds act as the nitinol.

CW (00:12:35):

But you cannot put a current through the folds.

EW (00:12:37):

Yeah, if I put the nitinol in the origami, I could flatten it, and have this little weird walking robot. Which is actually why you are doing this <laugh>.

CW (00:12:45):

<laugh>

EW (00:12:45):

It is not just that it is cool.

CH (00:12:47):

That is right.

CW (00:12:48):

You could stop. It is just- Yeah.

EW (00:12:51):

The Pringles shape is one that I have seen before with soft robotics, because it is very similar to a claw shape. You just close that Pringle all the way down, and suddenly now you have fingers. They are soft fingers. They will not hurt people, but they can grip things. How big have you gotten? I am going to go with the Pringle shape. How big have you gotten the Pringle?

CH (00:13:15):

About actual Pringle size. We learned from developing this project, that there is a certain realm. If you make your Pringle too big, then the fabric in the middle just wrinkles up. It gets to its happy area. It is all trying to shrink and it gets to this place where it is happy, before it has really formed the Pringle shape.

(00:13:39):

If your Pringle is too small, it does not saddle-shape at all. It just stays flat and it says, "I am going to not bend. And the fabric, you are just going to have to stay stretched out." There is always this competition between bending of the wire around the edge, and stretching of the fabric

(00:13:55):

There is this sweet spot in the middle. It turns out to be for the diameter of the wire we used, and the hardness of that wire, about five centimeters. So it really did look maybe a little bit smaller than a real potato chip. But if we wanted to go bigger, we could do that. You can get larger diameter wire, and that is what we would have to do.

EW (00:14:19):

It would be fun to experiment with the diameter of wire, and see what the maximums are for the different diameters. Did you not have a position open in your grad student lab? And when can I start? <laugh>

CH (00:14:32):

Oh. There is a fellowship. It is this month that we want people to apply.

EW (00:14:40):

Sorry <laugh>. Easily distracted by what sounds like awesome toys.

CH (00:14:45):

Come and visit.

EW (00:14:46):

Have you found a practical use for these yet?

CH (00:14:55):

We have taken them and applied them. Since they do take a lot of power, and a lot of temperature changes going on in this material, we have applied them to bistable elements.

(00:15:07):

Little pieces of plastic, for example, that are attached at the ends and they pop up or down. That is where we see these being most useful, in a soft robot that can move a little bit at a time, store up a little bit of power, and then pop one of its actuators to another state.

(00:15:30):

We have a really old paper that says you can make any shape this way, at least with the 1D array of these bistable elements. So we were always looking for muscles to put on the things, that were compatible.

(00:15:43):

A big driving goal behind my work is to start everything out planar. That is really great for fabrication, simplicity and speed. If anyone is an art major, and they took a painting class versus a sculpture class-

EW (00:15:59):

Yeah <laugh>.

CH (00:16:00):

The paintings get done a lot faster than the 3D stuff.

(00:16:02):

Planar stuff is also something you can align. So if you have different layers you can stack them, and then fold. It is a lot harder to do that after something has gone 3D.

(00:16:14):

That mindset probably comes from microfabrication, where everything is on the silicon wafer. It has got really great dimensional tolerance, really easy to line up. There are whole machines made for aligning things, so my mind might be poisoned by that.

EW (00:16:37):

Actually, one of the papers had MEMS- When you talk about doing things like that, the thing you just said about wafers, do you always mean MEMS stuff? Or are there different technologies we are talking about, when we look at small systems?

CH (00:16:58):

For me it is MEMS. But there are really interesting 3D printing methods, like the two photon printing, that are starting to get down to the microscale too. I have not done that.

CW (00:17:10):

Can you describe that? I do not think I have, either of us have, heard that one.

EW (00:17:13):

I have never heard of it.

CH (00:17:14):

Oh, let us see. There is a system that you can get that has lasers in it, that when they both cross- It is like a resin printer, but it is using these highly focused lasers. I forget the resolution that it can get, but its build envelope is three millimeters high. So that is it.

(00:17:36):

It is big, like maybe ten centimeters across. But within that three millimeters, you are getting full access to the resin with this two photon intersection.

CW (00:17:48):

Okay.

EW (00:17:51):

Huh. It could make some really small things.

CH (00:17:55):

Yeah.

EW (00:17:55):

Biomachines.

CH (00:17:58):

Mmm. Yeah, things that would be hard to make using conventional MEMS.

EW (00:18:03):

But then, that is actually a little bit more 3D than conventional MEMS.

CH (00:18:06):

Yeah. It has all the pluses and minuses. So MEMS, you might be thinking I am limited to layers. Depositing a layer, or etching a layer. It is hard to do overhangs or anything. You could do it, but with this two photon printing, you can get really intricate geometry. Like you can do a little Capitol Building with the little pillars. I think that is one of their test structures, but it is all made out of the resin.

CW (00:18:33):

Hmm.

EW (00:18:36):

Okay. Actually, let us start over with MEMS, because not everybody is like- What is MEMS? I was introduced to MEMS with accelerometers and gyroscopes, when they became sensors that were chips instead of physical objects. Do you know how a MEMS accelerometer works?

CH (00:19:01):

Usually there is a little mass and there is a thinly etched beam. The mass is really tiny, but it is on this very thin beam, so it is able to move in response to forces. And if you can measure that motion-

(00:19:14):

Then there are a bunch of different ways that people do the measurements. So they can measure- They can put a little string gauge, like a little resistor, on the beam. Or they can use a capacitive method to sense where the weight is, or the mass, I should say. So that part is almost- The mechanics are done as electrical engineering side.

CW (00:19:36):

Mm-hmm.

EW (00:19:36):

Cool. That concurs with my diving board analogy in my head. How the gyro works, I have no idea.

CW (00:19:48):

Same thing, but just a little spinny thing. I do not know. <laugh>

CH (00:19:51):

Yeah <laugh>. Somebody will chime in and say, "There are other ways, besides the mass on a stick. Like there are little hot air sensors and temperature based accelerometers.

EW (00:20:04):

When I started with MEMS, before the year 2000-

CW (00:20:10):

The year 2000!

EW (00:20:12):

<laugh>

CH (00:20:12):

The distant future.

EW (00:20:14):

Exactly <laugh>. The distant future. They were new and shiny, but now they are everywhere, and they have all kinds of sensors. Do you have an idea for what you find is the neatest sensor that is out or coming out?

CH (00:20:32):

Oh, right now I am really liking these magnetic field sensors. So kind of basic, and you might think there are no moving parts or need for MEMS, but anything that is a vector sensor- Like suppose you want to sense the magnetic field in X, Y, and Z, then you have to have some sensor element that is oriented along each axis, or some way of distinguishing different components.

(00:20:56):

So the MEMS process might come in there, because it would fold up these planar fabricated materials into things that respond only along a certain axis. But I really like them because they are so small. And they are useful for a project that I am working on right now.

EW (00:21:18):

Is it the bird vision sensor project?

CH (00:21:18):

This one- Bird vision sensor?

EW (00:21:22):

Well, because birds not only see, not only know-

CH (00:21:25):

Oh.

EW (00:21:25):

They not only can tell which way is north, like what we use our magnetic sensors for usually, they can see how the-

CW (00:21:35):

Maybe.

EW (00:21:36):

Probably. They can see how it enters the earth. So it is like- I want to say another color over their vision. And they can just-

CH (00:21:46):

Oh yeah.

EW (00:21:46):

Anyway. Could be the picture. <laugh>

CH (00:21:47):

I think I saw that. Yeah, I think I saw that. Somebody simulated what the bird sees magnetically.

CW (00:21:53):

Bird physics.

CH (00:21:54):

<laugh> Usually we are trying to do background subtraction, because there is this earth's magnetic field-

CW (00:22:03):

Right.

EW (00:22:03):

Right.

CH (00:22:04):

And it is getting in the way. We love the prospect of being able to turn on a nearby current in a wire and then use this little sensor. These things are less than a millimeter square, even packaged up. Use it to sense some wire that is moving, rather than sense what the orientation of the whole part is.

EW (00:22:30):

<music> I would like to thank our show sponsor this week, Nordic Semiconductor. Nordic has a vast ecosystem of development tools. It helps us reduce our development cycles and accelerate the time to market. For example, the nRF Connect for VSCode extension provides the most advanced and user-friendly IoT embedded development experience in the industry.

(00:22:52):

Nordic's Developer Academy online learning platform equips developers with the know-how to build IoT products with the Nordic solutions. Nordic's DevZone brings together tech support and a community of customers, to provide troubleshooting and assistance on technical topics. Visit these key websites to learn more. nordicsemi.com academy.nordicsemi.com and devzone.nordicsemi.com.

(00:23:17):

And to thank you for listening, Nordic Semiconductor has a contest to give away some parts. Fill out the entrance form and you are in the running. That link is in the newsletter and in the show notes. <music>

(00:23:34):

Going back to your projects and your papers, you had one that was for bend localization. Which I thought was really interesting, because I have had client projects that wanted to look at how limbs bend for physical therapy. But I think yours is quite a bit different. So what is OptiGap?

CH (00:23:58):

All right. This was a recent graduate, Paul Bupe. He looked at some sensors that we had been working on. We had developed a stretchable optical fiber. He wanted to apply this to a shape memory wire robot that he was working on, because you really- In these shape memory robot devices, you want to know when it has done the job it has got to do. Then you can turn off the half an amp or whatever, because that is costly.

(00:24:27):

But he needed a soft sensor that would not interfere with the way the- It would not push the actuator around. So he took some filament that we had been using just as a length sensor, and cut a little gap into it. Literally right there at the gap, he just cut it with a razor. If you bend it there, the light sprays out. It looks like water coming out of a hose, in the ray tracing simulation he did.

(00:25:01):

If you got a detector at the end of the fiber and you bend it, you are going to say, "Hey, where is my light? It must be bending at one spot." So fairly straightforward concept, but of course he wanted to measure bending at different locations, not just one spot on his actuator.

(00:25:20):

So he put a bunch of gaps, and he put several fibers side by side, and a really great programmer. He got a machine learning set up going, to identify which of these gaps was bending. And from that backtrack and figure out the shape of this limb that he was working on.

EW (00:25:45):

Just to paraphrase again, a bundle of optic fibers. Some of them are cut a little bit at different places, so that when they bend at the bottom, the ones with the cuts at the bottom lose light more than the ones that are not cut there.

CH (00:26:08):

Right. Yeah. If you have two fibers in parallel, and one has a gap at the place where you have a hinge and one does not, you are going to see a really dramatic difference in fiber A that does not have that gap and fiber B that does. So even though every fiber is stuck on the same object, we can use that to get some spatial information out. Is it A or B? He would scan the lighting, and have it all piped into one photo sensor.

CW (00:26:39):

Okay. And then the machine learning. What is the difference at the sensor, between those different positions then? Is it the difference in intensity, because they are further away and there is more loss? Or?

CH (00:26:53):

It is intensity.

CW (00:26:53):

Okay.

CH (00:26:53):

So if you had a- And you know which fiber you were lighting, when you were measuring.

CW (00:27:00):

Oh! Okay.

CH (00:27:03):

Yeah, if you were lighting fiber A and you got a lot of intensity, but fiber B had hardly any intensity. Then you know the hinge where B has the gap went. So he did that with three fibers and with multiple cuts, and he could get- I think he would get like two to the n minus one sensing locations for n fibers.

CW (00:27:19):

Interesting. Huh.

EW (00:27:20):

It sounded like a key scanning matrix.

CW (00:27:23):

Yeah, yeah. Okay.

CH (00:27:23):

Yeah.

EW (00:27:25):

And the data he got out of it, while he used machine learning on it, it seemed like you could just use a simple filter.

CH (00:27:34):

Yeah. I think he liked to push the limits of his programming skills, speed up how quickly it would recognize small changes, and throw out some of the noise. But it is fairly digital.

EW (00:27:50):

Is that one ready to be used?

CH (00:27:52):

Yeah. We have a research disclosure on it and it is up on the U of L site, so he is interested in commercializing that.

EW (00:28:02):

How important are patents to this type of work?

CH (00:28:06):

It depends on if you are trying to develop fundamental new concepts, or trying to commercialize. So definitely if you are trying to commercialize, our tech transfer office, like most colleges and universities in the US, really encourages IP. But if we publish it and do not patent it, that is still good for the academic side, right?

CW (00:28:33):

Yeah.

EW (00:28:33):

Yes.

CH (00:28:36):

And it may let more people make use of what you did. So there are two sides to it.

EW (00:28:43):

Do you encourage your students to publish before looking into patents?

CH (00:28:50):

Yes. Usually that is the first thing- I want to get everybody through with a paper when they have a project and have that paper in mind. So that is number one for me.

EW (00:29:01):

Both of these, the actuator with the nitol-

CW (00:29:07):

Nitinol.

EW (00:29:09):

Nitinol. Somebody should have given me a post-it note that written on it. And this OptiGap, they are things that sound like I could do them. I mean if I had the parts, they seem like they are recreatable without a huge lab. Is that intentional?

CH (00:29:28):

I think so. It is something that is driven a little bit by the way students come to my lab. I have a lot of summer students. If they have to learn MEMS, they can do it. In fact, we have a great REU program, where they do learn the microfabrication skills.

(00:29:51):

But they only have about ten weeks, and they get the things done in the last couple of weeks, and then they are gone. But if I can get them doing something on day one, with stuff on the bench top, they tend to get more done.

EW (00:30:08):

"REU" is "Research Experience for Undergrads"?

CH (00:30:11):

Yes. That is right. Research Experience for Undergrads.

EW (00:30:15):

So the goal is to get them to be thinking about sensors and doing sensors, even as they are learning about the MEMS processes and how to build those.

CH (00:30:25):

Yeah. There are a lot of analogies between the MEMS process that we do. We get things to pop up from the surface and roll up, and some of these larger scale things. And it turns out a lot of these larger scale things, like the slab bracelets and the bistable structures, still work at the microscale.

(00:30:47):

But you cannot put your hands on them and feel where they have a sweet spot, like you can if it is a beam that you can put in your hand and flip back and forth. It turns out there is an ideal spot to put an actuator on some of these bistable little plastic beams.

(00:31:08):

If you took a piece of plastic and push the ends together, it would make a bow shape. You can imagine snapping that bit down. Snapping that up and down takes a little bit of energy. And there is one spot in particular about halfway up it, that it is really easy to cause that thing to flip.

(00:31:28):

Going off on a little bit of a tangent, but you do not really get the insights into that, from the microscale structures alone.

EW (00:31:38):

Yes. Yes. Bistable structures should be obvious. It is in the name. It is either in state A or state B. But when you hold something and can feel it go, feel that springiness that happens as it launches from one state to the other. It is very helpful.

CH (00:32:03):

Mm-hmm.

CW (00:32:06):

Thinking of those little toys, the little plastic rings that have a dome. You push the dome in and then you set it down, and then five seconds later the dome pops out and it flies off.

CH (00:32:14):

Yeah. Yeah, it turns out those are harder to push from right on the center, than a little bit down on the side.

CW (00:32:21):

Huh.

CH (00:32:21):

You can kind of roll them through.

CW (00:32:25):

Oh!

CH (00:32:25):

So if you are trying to make a microscale one, you might think, "I better put my actuator right in the middle," but it is not actually the best spot.

CW (00:32:35):

Wonder why that is. Huh.

EW (00:32:38):

I bet she could tell you.

CW (00:32:40):

That was an invitation. <laugh>

CH (00:32:44):

There is an inflection point that is very important, rolling those things through.

EW (00:32:51):

One of the things in the papers was the math. When I read papers, I do expect to see some math, and usually I skip those parts.

CW (00:33:03):

<laugh>

EW (00:33:03):

Let us just be serious, at least until I understand what it is I need. But I was actually a little surprised to see the modeling in the OptiGap and SESAME papers, because it seemed like the work came out of the modeling, instead of the fabrication. Is that correct? Or did the fabrication come first?

CH (00:33:31):

We did it right. I mean <laugh>, we fooled you. The fabrication has come first, but we aim to fabricate something that we can model, and try to keep it the element that we are using simple. So we have not done a model of the six by six Pringle array, but we made a model that would be of one cell.

(00:33:54):

Because of my background and the people that I work with, we do not have the finite element modeling focus, that maybe a mechanical engineering group would do. This is the one where you could picture a model of a truck, and something like a really complex object, and you are looking at the airflow going over it. You have to draw up that truck in your modeling program, and simulate each little piece of it.

(00:34:25):

We are more like let us model a circle, and write some equations. We might do ray tracing on an optical fiber, but that is as fine an element as it gets.

EW (00:34:39):

Okay. That is actually reassuring, because I wanted to play with the items, and I did not want to have to understand the math, in order to just play with them. It sounds like I could just play with them.

CH (00:34:51):

Mm-hmm.

EW (00:34:51):

But I have been doing a lot of simulation lately, and I have to admit, simulation and physics, they do not match as well as I was promised they would.

CW (00:35:07):

Who promised you?

EW (00:35:09):

<laugh> Well, that is the thing. I sort of promised myself that one.

CW (00:35:12):

<laugh> I could have told you that.

EW (00:35:16):

How much do you need to model things, in order to believe them? Or do you use the fact that you can fabricate it, as a way to shortcut the difficulties of reality?

CH (00:35:34):

I think it is a massive shortcut. We have a lot of materials on the desks, and a lot of tape. We are putting things together and going, "We could make something, but is it going to be strong enough to do a job?" So some of it is we have got to play around with it in the beginning, just to see if it is worth doing.

(00:35:51):

If we model it, we are usually motivated by getting some kind of design rule out of it, to get rid of some of the prototyping that has to be done in the future. So one example is this SESAME project. We found a relationship between the Pringle diameter and the wire diameter, that if you keep it constant- I think there were a couple powers in there, like a cube.

(00:36:21):

If you keep this term constant, then your Pringle will scale up or down, and it is going to look kind of the same. That is really why we want to do most of our modeling. Get some rules out, that somebody could use to adapt what we did, to the problem they are trying to solve.

EW (00:36:41):

Cool. Okay. We had two more papers, and we have a bunch more questions. So, let us see. The "Bio-inspired robotic finger, driven and shape-sensed by soft optical tendons." Now I have to admit the picture on this one, looked a lot like that time I poked holes in a straw and had a string that threaded through it, and then I could make little fingers with strings. Have you done the finger straw string thing?

CH (00:37:15):

I think I had some puppets that had little strings in them. Yeah, that is the basic principle. This was our group- We were lucky we got a mechanical engineering grad student, Michael Han. She was the first mechanical engineer in our group. It is a really common crossover for electrical and mechanical engineers, who work on MEMS or robotics. Those two fields especially, sometimes you cannot tell who is who after a while.

(00:37:44):

It works on principle like a straw with a string or a tendon in it. But she also got the optical source and detector plugged into the ends of that tendon. So this thick stretchable tendon is doing a couple of different jobs, moving the finger. and then also sensing how far the finger has moved using the light signal.

EW (00:38:18):

Could I not use the string length movement for that? Like, how many times I had wound it?

CH (00:38:27):

There are a lot of tendon systems that do that, and a common question that Michael was addressing. So the big difference here is the stretchability of this optical tendon. It is made out of TPU printer filament, so another material that a lot of people have access to, but it is a little bit stretchy.

(00:38:50):

So the length of the tendon depends not only on the shape of the finger, but the force you are applying. And also is the finger touching anything.

EW (00:39:00):

Oh, yes.

CH (00:39:03):

Because you can apply a lot of force and then your spool will be moving and your fiber would be stretching. You would have different amounts of displacement, depending on if you were pushing against something. The motivation for using something stretchy was it resembles more closely the human tendons or biological tendons, which are a little bit elastic.

EW (00:39:25):

Sometimes too elastic. Okay, so that makes a lot of sense. Then being able to add the sensing would be necessary, if your tendons or strings or elastic, because you would not really know whether it was blocked.

CW (00:39:47):

Is that the same sort of sensing that you mentioned before, with the cuts through a bundle? Or is this something different?

CH (00:39:54):

This is just amplitude, and the light attenuation is coming from the deformation of this fiber. It is going through these joints, and it is going over a few places that make it go a little bit squashed.

CW (00:40:09):

Okay.

CH (00:40:11):

So a little bit pinched.

CW (00:40:13):

And then it loses internal reflection, or something.

CH (00:40:15):

Mm-hmm.

CW (00:40:15):

Okay.

EW (00:40:18):

Okay. So, "Embedded optical waveguide sensors, for dynamic behavior monitoring in twisted-beam structures." That is not at all an acronym. That will never be an acronym.

CH (00:40:30):

We did not have time to acronymize that one. It would take a couple months.

EW (00:40:34):

This one also used the OpticGap.

CH (00:40:37):

This one we changed our- So we had been using materials that were just repurposed, like printer filament especially. That has been a really great stretchable optical fiber. But in this one, we wanted to sense twisting around the fiber's own axis.

(00:40:56):

The reason for that, I can go back a step and say, we had a twisted beam- That if you can imagine a ribbon, it is a rectangular shape. Then twist the end of that ribbon 90 degrees, and it has got a really interesting property. This is based on a collaboration with researchers from Oregon State, San Diego and Arizona State.

(00:41:30):

You shake this twisted beam, and it starts to circulate at the end. That circulation is really good for robotic walking. If a leg goes in a circle, it can hit the ground, and then it can go up and it can move over a little bit, and hit the ground again in a different spot. So long story short, we wanted to be able to twist these beams left-handed or right-handed to get different direction gaits out of the same actuator.

(00:42:01):

Well, why do we need a sensor for that? We need to know if we have achieved the twisting we want. Because once again, we are trying to use a shape memory wire. This time it is twisted up inside the core of this twisted beam, and that takes a lot of power. It is also starting to heat everything up, and the surroundings are getting to about a hundred C[Celsius].

CW (00:42:19):

<laugh>

CH (00:42:21):

<laugh> We want to know when we are done, and so we need a little soft sensor that would not interfere with anything. Our summer student made a square cross-section fiber out of really clear type of silicone, kind of like a gummy worm. She found out when you twist this square cross-section fiber up enough, it starts to lose light. That was enough to tell whether we had achieved the left or right-handed twisting.

EW (00:42:50):

It lost light in a way that lets you tell the direction, the chirality?

CH (00:42:54):

That is a great question. Yeah. What is the difference between a square that you have twisted right 90 degrees, or left 90 degrees, in terms of light transmission? Not much. So she had to pre-twist it.

EW (00:43:11):

Oh! And then you were either untwisting it partway, or untwisting it the whole way?

CH (00:43:15):

That is right.

EW (00:43:16):

Okay. I was thinking there was going to be some polarity in here. But yeah. Okay.

CH (00:43:22):

Yep.

EW (00:43:22):

Huh.

CH (00:43:22):

So that is a challenge, because if you took a rubber band and you pre-twisted it a lot, you have got a propeller. You ever have one of those little airplanes? So it has got to be a really soft material. We had a really soft silicone material for that.

EW (00:43:41):

But now I want to spend ten weeks trying to figure out how to improve that, so you do not have to keep that tension. Okay. I am going to go back to college.

CH (00:43:50):

Go back and study maybe boiling down in chemistry at that point.

EW (00:43:56):

I always wish I had taken more mechanical. I just sometimes feel that that is probably what I need more than anything. More than the O chem that I actually want to take.

CH (00:44:06):

Mechatronics.

EW (00:44:06):

Exactly. Mechatronics. Okay. So you have been working on interesting sensors. What are you most excited about in the sensor space right now?

CH (00:44:22):

I like the idea of putting sensors all over the surface of things.

EW (00:44:25):

Yeah.

CH (00:44:25):

So they can sense the local environment. It is just more possible now than it used to be. These chips are getting really small. I had mentioned that magnetic sensor chip. There are also temperature and sort of resistant sensor chips in that size range, that are in square millimeters and they output digital stream. As long as you can wire them up, you can get local data.

(00:44:54):

Then there are these fiber-based sensors, that are as soft as the material they are going into. Some people call that mechanically transparent multiplexing. Getting that spatial information out, it is getting easier.

(00:45:08):

One way to do that, is not only looking at the intensity of the light that gets through these optical systems, but the time of flight. So putting a lidar sensor at the inlet and outlet. People are doing that. We did that to get some spatial data encoded on the pressure data.

(00:45:29):

Then there are tiny wireless sensors going everywhere and-

EW (00:45:32):

Everywhere.

CH (00:45:34):

Almost printed. You have seen those tattoo-like wearable electronics?

EW (00:45:39):

Yes.

CH (00:45:39):

It is very exciting right now.

EW (00:45:42):

Your answer is one that I think I would have too. The sensors are really cool, and there are all kinds of new sensors always coming out. Although often there are tweaks on old sensors. I mean, we will be getting more chemical sensors, but they are all going to be about the same.

(00:45:58):

I would love to know more about brand new sensors, but I think that the truly interesting part is the arrays. Is being able to throw a whole bunch of sensors at a problem. It is like the LED arrays. That we had LEDs for a long time, and then we started getting things that let us have lots of LEDs. Now, you can have thousands of LEDs, if you can figure out how to power them.

(00:46:26):

We are going to end up doing the same thing with sensors, that you can have nets of sensors. It will just be really cool. I think we will have clothes of sensors.

(00:46:40):

I like mechanically transparent. That is an interesting concept. One that I had not considered.

(00:46:51):

Let us go to listener questions. We had one from Ellie. Ellie, thank you for introducing us. Her question was, "How has research-scale MEMS fabrication changed for folks at smaller universities, over the past decade or so?"

CH (00:47:07):

Okay. Over the past decade- There is a process called MUMPs that has been around a while. It is much older than the past decade. But MEMS is so diverse. So MUMPs is like, "If you want your device made, use these layers and these processes in this order, and you can draw your geometry and have a part of this process that goes through."

(00:47:29):

But the variety of materials that people are using in MEMS, I think it is greater than, say if you are making a lot of transistors, you want to use some specific materials and you can usually achieve your architecture using the given process. But in MEMS, "Oh, hey. We are trying to introduce some strain, and get these things to go 3D." It already does not belong in the MUMPs process.

(00:47:51):

It is good, but it does not have a really perfect analogy to VLSI and chip fab, because of the material diversity. So you start needing to do your own. One thing I wanted to add in there is that when doing your own process, there is usually a disadvantage to going big with your wafer. Using an eight inch wafer, which is small by industry standards, is a huge expense.

(00:48:22):

So universities kind of stuck to the four inch wafers, and now the technology that handles those is getting ancient. I mean it was ancient probably ten years ago, so using really old computers. I feel like people maybe in this audience, if you can repair and keep old computers going, or hook a valuable old machinery into a new computer and keep it going, there is always going to be a need for that. At least in these university clean rooms.

EW (00:48:58):

It is funny, I was thinking that things like Tiny Tapeout, where people are making their own chips. I was like, "Okay. Well, MEMS has got to be similar to that. People can try their own sensors." But you are telling me that MEMS is different.

CH (00:49:17):

If you want to have a magnetic sensor, for example, you might have a magneto resistive material. Maybe no one else on the MUMPs wafer wants that material. So do you get your own step, and everyone else has to cover up their part of the wafer during it? I think there are a lot more different experimental materials people are trying to use.

EW (00:49:40):

It is sad that we cannot have more experiments there. The reason, it sounds like it is because the- I mean, once you say "clean room," everything gets a hundred times more expensive.

CH (00:49:54):

And it takes longer than you plan.

EW (00:49:57):

Right. Because when I order a chip, I just order a chip and it comes to me from DigiKey or Mouser or wherever. But when you design a MEM sensor, or any sort of fabbed chip, there are multiple weeks as you lay down each layer, right?

CH (00:50:17):

Right. Especially if the old computer has blown up, and you need a new computer.

EW (00:50:20):

<laugh> Are you having computer trouble this weekend?

CH (00:50:27):

See, which machine is it now?

EW (00:50:30):

And you know none of them are as powerful as a Raspberry Pi, right? <laugh>

CH (00:50:33):

Right.

EW (00:50:33):

You could just replace them, if only you could just replace them.

CH (00:50:38):

They do not have the ISA card that you need <laugh>.

EW (00:50:43):

Do you have Windows 95 on all of the systems?

CH (00:50:46):

I cannot mention that.

EW (00:50:51):

Do you think we are headed towards being able to have smaller scale MEMS inventions, devices, fabricators?

CH (00:51:03):

Oh, I think one thing that is coming up is direct write. So most of our processing is done using photomasks, that is like a stencil that covers the entire wafer. But the direct write using a laser to write patterns has gotten faster. So it is like writing with a mechanical pencil. It is very slow, writing a fine line, but it has really sped up.

(00:51:28):

At least on the design and iteration and cycling side, direct write is making that go faster. You can imagine making a photomask is really going to be a piece of glass, that if you do not like how that design worked out, it just goes on the shelf and you start over.

(00:51:47):

And you have to cover that whole area with designs, where you might be able to iterate faster if you can change the design in- I want to say L-Edit, but in your CAD program, and then write it directly to the photoresist that afternoon or five minutes later. So I see a bright spot there.

(00:52:11):

We also have machines that can lay down metal using- We have an aerosol printer that can make lines sub-hundred micron of silver, or whatever other ink you can formulate that works with it. And then there is inkjet printing. So printables are going to take care of some of this materials integration problem.

(00:52:36):

And then you mentioned LED arrays. That is really driving a lot of assembly innovation. Can we take an array, or take an LED- Can we take an LED off a wafer that is- Usually it is a gallium arsenide wafer or some non-silicon. It is always non-silicon. Pluck it from there, and then put it onto the background for this big TV. That demands high speed.

(00:53:08):

It is sometimes called "Hetero Integration." Picking a tiny piece of wafer that has been cut out, laser diced from this other material. Then aligning it fast enough to make a big TV, that you see at Sam's Club or Costco if you are in the US. Industry is pushing that along.

(00:53:29):

So those three things. Direct write, printing, and then this third category, that is kind of pick and place, but sometimes called "chip printing," are really going to speed things up.

EW (00:53:43):

Every time I think the industry is getting boring, I hear about things that are going to make it interesting.

CH (00:53:48):

People always want to do more.

EW (00:53:51):

More and cheaper and better and different and... Yes. Let us see. Do you have any papers coming out that you can talk about?

CH (00:54:04):

Oh. We have one that is about letting these slap bracelet-like microstructures off the surface. We used to do that by- We would lay everything down. You can imagine a bunch of slap bracelets, but these things are about a millimeter long, lying on a surface. They are taped down with a piece of scotch tape.

(00:54:25):

We used to release those and let them go curly, by putting them in a etch chamber. But it was really hard to see what was going on in there. It is kind of hazy. We got frustrated with that.

(00:54:39):

So we took them out of the chamber, and we made a way to release them in front of a high speed camera. That has been pretty exciting. You can see all these things curl up. It takes them about a millisecond to go from planar to 3D. That paper is just coming out.

EW (00:55:00):

This goes back to bistable, and being able to see where the best place to actuate to move between the stable states? Or not?

CH (00:55:13):

This one, it is bistable, but it is not really reversible. I do not know if it is really bistable or not. It is like this thing really wants to curl up, but we have glued it down. You can let the glue go once. When you let it go, it jumps up. You can flatten it out again. So it is reversible in the sense that you could heat this thing up.

(00:55:40):

It has got a metal top layer, and then an oxide, really kind of a thin glass back layer. When you heat up the top layer, it is metal, it is going to expand and everything is going to flatten out.

(00:55:54):

That is not really- I guess it is cheating a little bit to call that "bistable." You have to change the temperature of the whole environment to get it to go to this. Cool it down and it curls up more, heat it up it flattens up more. It is not really mechanically bistable.

EW (00:56:11):

"Bistable" more implies that the energy between them is not huge. It is just getting over a little hump.

CH (00:56:20):

And it has got a little barrier. Yep.

EW (00:56:22):

You have given me so much to think about. Now I want to go off and dream about different sensors, and what I can do with them.

CW (00:56:32):

<laugh> Go buy a spool of nitinol.

EW (00:56:34):

Yeah. <laugh> I do.

CW (00:56:35):

I do wish there was another way of making those wire-like memory things work, apart from <laugh> driving what is relatively high current for our kinds of electronics through it. Is there any work on- Nitinol has been around forever, like you said.

(00:56:51):

I remember working with it at a medical company, although I do not know that we put a current through it. It was for the cardiovascular stuff. That was the main thing that gave shape to the catheters for artery stuff. I do not remember how they actuated that. It was not with current though. I think it was just it was in a shape that was preferred, and if you twisted it or something or gave it a force-

(00:57:16):

But anyway. Question was, is there anything else coming along? Have people continued to work on materials like that? Or is that just like, "Oh, nitinol is the only thing we know how to do."

CH (00:57:28):

There are upsides to nitinol, that it is really easy to interface. It does not solder well, but other than that, if you can plug it into a circuit, you do not have to do anything else. There are other shape memory materials, but they might be polymers. So you have to wrap that with some kind of heater wire. With nitinol you just do not have to do that.

(00:57:47):

But people are tired of thermal actuators-

CW (00:57:50):

<laugh>

EW (00:57:50):

<laugh>

CH (00:57:51):

Because of they are power hogs. So many materials do shape memory or shape change using temperature. They do really cool preprogrammable things, but the holy grail is something that is capacitive or electrostatic.

(00:58:05):

People have worked on dielectric elastomer actuators that are- They are kind of capacitive. They do not draw a direct current. They need a high voltage though.

(00:58:18):

And then there are electrostatic clutches. They are not actually the main muscle in the system, but they can grasp onto a tendon or let go, maybe for fine control. So those are some active areas right now.

(00:58:36):

And then there are other fluidic methods. So there are the HASEL actuators that use an electrolyte to inflate or deflate soft robotic muscles.

EW (00:58:51):

Is this like? Oh, soft robotic muscles. I was thinking E Ink. Talk about something that is bistable, requires a high current. Or high voltage?

CH (00:59:01):

High voltage, yeah.

EW (00:59:03):

But brief. And that is what you need. I mean, part of the problem with the nitinol is that to keep its one state, you have to continue providing the power.

CH (00:59:14):

Right.

EW (00:59:14):

The advantage to E Ink is you provide the current and then it goes into its other state, and it stays there.

CH (00:59:20):

Mm-hmm.

EW (00:59:20):

What we need is nitinol that is crossed with E Ink. Okay. Go off. Make it so. <laugh>

CH (00:59:30):

Yeah, if there were something that were not high voltage and not high current. High voltage is not that bad of a problem. You can generate it in a really small space. But a lot of those actuators, people push the limits and then they just pop.

EW (00:59:45):

Which when you are making human interface robotics is not the best thing.

CH (00:59:49):

Noo! They usually pop pretty quickly and without too much stored energy, because they are thin and there are small membranes, but then they are cooked.

EW (01:00:00):

There is a long way to go on soft robotics.

CH (01:00:02):

Yes. There is lots to do.

EW (01:00:04):

Do you have any thoughts you would like to leave us with?

CH (01:00:08):

The thing I would like to leave with, is the strength that comes from adding new kinds of materials together. I experienced that when putting fabrics and textiles with more electronic or mechanical devices. But I see it happening in the MEMS realm and in the soft robotics realm.

(01:00:30):

Some of that does not really come from materials or chemistry itself, but just the abilities we have now to do high-speed assembly and machinery. So if that is you are more of a mechanical background and you are feeling like you are left out, we need your contributions for automation.

EW (01:00:52):

Our guest has been Cindy Harnett, professor of electrical and computer engineering at the University of Louisville.

CW (01:00:59):

Thank you, Cindy.

CH (01:01:00):

You got it.

EW (01:01:00):

Thank you to Christopher for producing you and co-hosting. Thank you to Ellie for suggesting Cindy as a guest. Thank you to our Patreon listener support group. Thank you to our patient-

CW (01:01:12):

Support group! It is a support group.

EW (01:01:16):

<laugh> It is. It has been this week. Thank you to our Patreon listener Slack group for their questions. And thank you for listening. You can always contact us at show@embedded.fm or hit the contact link on embedded.fm

(01:01:29):

Now a quote to leave you with, from astronaut Mae Jemison. "Do not let anyone rob you of your imagination, your creativity, or your curiosity. It is your place in the world; it is your life. Go on and do all you can with it, and make the life you want to live."