Transcript from 238: My Brain Is My Toolbelt with Elecia White and Christopher White. CONTENT WARNING: Gun violence discussion starting around minute 19.

EW (00:07):

Hello, and welcome to Embedded. I'm Elecia White, here with Christopher White. It's just us this week. We have a lot to talk about, but I want to warn you right now about halfway through, I start talking about ShotSpotter, and there are some gunshots and some violence, not actual gunshots.

EW (00:27):

I mean, it's all fine here, but you'll hear gunshots, and then you hear stories of mayhem. So if you're going to be disturbed by that, either skip the second half when I start talking about ShotSpotter, or just go ahead and skip to the next one.

EW (00:41):

Or last week's, or Sunshine's, or Roger's, or Robin's. Actually, we've had a lot of really good shows this year. Do you want to dive into the big things, or do you want to cover all the little things first?

CW (00:54):

Let's get the little stuff out of the way first.

EW (00:56):

Okay. The first thing is, I was not trying to drive listeners away when I alluded to that in Roger Linn's episode.

CW (01:07):

We aren't? Why not?

EW (01:09):

Well, it wasn't that I don't want people to listen. I'm proud of the show. I want people to listen. It's just that the more time I spend thinking about the listeners, the less time I spend thinking about what I want to talk to people about.

EW (01:23):

And the shows come out better if I am genuinely enthused for my own amusement.

CW (01:31):

I think that makes sense. I think most people would appreciate that and don't find that to be a problem.

EW (01:36):

I don't quite remember what I said except that I didn't want him to worry about the listeners. It was all about me or something.

CW (01:41):

I think we said we didn't care about the listeners.

EW (01:44):

Alright, well -

CW (01:44):

But we mean that in the nicest way.

EW (01:46):

Nicest possible way.

CW (01:46):

We mean we aren't considering -

EW (01:48):

Your wishes and desires [inaudible].

CW (01:49):

We're not doing fan service all the time.

EW (01:54):

Calling back to the Tim O'Reilly episode.

CW (01:59):

[Affirmative].

EW (01:59):

We were talking about waves of technology and I was expressing that it isn't new.

CW (02:08):

Having waves of technology isn't new.

EW (02:10):

...But they seem to be getting closer together. And this really hit me this week. I was reading about oxygen in two different books. It was a strange week of all chemistry all the time. And oxygen as a thing didn't exist 200 years ago.

EW (02:28):

I mean, a little bit more than 200 years ago at this point. But we didn't know what oxygen was. We called it something else.

EW (02:36):

And we couldn't tell the difference between it and other things. Chemistry is new. Physics is 250 years old, 275 years old. This is all so new. We have buildings older than science.

CW (02:52):

Well, yeah. I mean, I would walk that back a little bit. I mean, things about chemistry were known for a long time at different places, and things about physics were certainly known about at different times in different places.

EW (03:08):

Yeah.

CW (03:08):

So in the Western sense and the post-Enlightenment, sure, our current view is new.

EW (03:16):

Isaac Newton and the periodic table.

CW (03:18):

Yeah.

EW (03:18):

Those are pretty critical.

CW (03:20):

Yeah.

EW (03:20):

And then from there, steel, and steam, and all these things,...they seem old to us, but they're new to the human civilization. And so the fact that the internet is another wave, it's a big wave, and it's a fast wave, and they keep getting bigger and faster.

CW (03:43):

It's full of trash and plastic.

EW (03:47):

Yeah, I know. Alright. So I...wanted to clarify that too, because -

CW (03:50):

I knew what you meant.

EW (03:52):

I didn't get the idea he did. And it just really hit me this week that we talk about everything getting faster, but it's getting faster and faster and faster. And it's not just, it happened in the last 10 years or 20 years, whatever.

EW (04:09):

Next thing is I found a fantastic book, "The Woman Who Smashed Codes: A True Story of Love, Spies, and the Unlikely Heroine Who Outwitted America's Enemies."

EW (04:23):

So it was about a cryptoanalyst, Elizebeth Friedman. And so much of her story hasn't been able to be told, because she was confined to the Secrets Act, the United States, World War II, you're not allowed to tell anybody what you did in the war thing.

CW (04:42):

Still?

EW (04:43):

She wasn't until her death. She was not allowed to talk about it. And so a bunch of Freedom of Information Act requests, and some old papers were found. And Jason Fagone wrote this book and...I mean, it was a spy novel.

EW (05:00):

It was awesome. I mean, it was a biography. It was true, but it was like a spy novel. I greatly enjoyed it.

CW (05:07):

Cool.

EW (05:08):

Let's see. Oh, toolbelts and the burden of tools. Do you have any idea what I was talking about when I wrote that?

CW (05:17):

No.

EW (05:20):

I was watching the electrician -

CW (05:24):

[Laughter]. Oh, okay.

EW (05:25):

- install our new -

CW (05:25):

Light.

EW (05:25):

- kitchen light, and...I mean, he had this amazing tool belt. And he had suspenders to hold up his tool belt, which gave me a new impression of belt and suspenders. And his tools, I mean, he knew where everything was.

EW (05:42):

He was looking up, and he could reach down and get whatever he wanted out,...clearly something he used a lot. But I was thinking about code tools and how there are tools that are big and weighty, like make -

CW (05:57):

[Affirmative].

EW (05:57):

- and git. And the less you use them, the more they mentally weigh.

CW (06:02):

Because you don't remember how to use them?

EW (06:04):

Because each time it's a big effort.

CW (06:06):

Okay.

EW (06:07):

And there are tools that not enough people use, because they're too complicated. PC-lint might be one of those.

CW (06:17):

I don't know if it's too complicated.

EW (06:20):

Or people don't see the value in it.

CW (06:21):

There's a burden, right? That's a tool you have to spend a lot of money on.

EW (06:26):

No.

CW (06:26):

Yeah.

EW (06:27):

To get it configured?

CW (06:28):

To buy it.

EW (06:30):

Oh, I didn't think it was that much to buy.

CW (06:32):

Well anyway, I mean -

EW (06:33):

Anyway.

CW (06:33):

Your point stands. There's some thing that makes them weighty, whether it's cost or perceived complexity.

EW (06:40):

And the electrician clearly had the toolbelt that he needed, and likely there were a lot of tools he didn't have. In fact, his assistant had the cordless drill and he didn't.

EW (06:53):

It made me think about the tools we use for software, because we aren't very good at pruning our tools. And we aren't good at recognizing that tools have cost.

EW (07:08):

Everybody's like, "Okay, let's just add one more thing. Let's add a Clang build as well as our C++ build, because it has more - "

CW (07:17):

You mean GCC build?

EW (07:18):

Or GCC build, because Clang has better warnings. Well, that's great. And as long as your makefile doesn't make you babysit it all the time. Maybe it's no additional burden until you have to make something new or port something.

EW (07:35):

And I don't know, this idea, it's all a jumble right now. So maybe somebody out there will tell me there's already this book or blog post about it. This all goes back to, tools also need care.

CW (07:49):

Oh, yeah.

EW (07:49):

You have to sharpen your woodworking tools.

CW (07:52):

I have never been happier than when I've worked at places where there's somebody whose job it is to maintain the software tools.

EW (08:00):

Because they think about these things.

CW (08:02):

There's a sense for why the tools are chosen. They're organized in a way that people can use them. Yeah.

EW (08:10):

And they're pruned. I bet they are pruned, because the person who has to take care of it all recognizes when a tool has outlived its usefulness.

CW (08:17):

Yeah. That's tricky though, because there's always somebody who can argue for its persistence, right?

EW (08:24):

Well, I think about the fact that I still have awk in my head.

CW (08:27):

Awk's great. What's wrong with awk?

EW (08:29):

Awk's a fantastic language for what it is. Something that I hope never to do again. And my week of chemistry has been shoving a bunch of other tools. A lot of Python things.

CW (08:41):

Now you're talking about space in your brain. That's a different -

EW (08:44):

My brain is my toolbelt.

CW (08:46):

Yeah.

EW (08:48):

I don't know. There's some point in here about tools.

CW (08:52):

Maybe one of the listeners who hasn't left can email us what your point is.

EW (08:59):

Yes. Okay...There are some more little ones, but now I want to answer a question. A question. A question from Ryan Jones. He's fairly new to the podcast. He wants to know when we would use dynamic memory in embedded C.

EW (09:29):

Are there reasonable use cases? He knows that dynamic memory should be avoided at all costs due to issues with the stack overflows and dangling pointers. But if you were going to use dynamic memory, when would you use it?

CW (09:45):

I use it all the time...Well, let's define dynamic memory first of all.

EW (09:53):

Because it doesn't have to just be malloc.

CW (09:54):

No, it doesn't.

EW (09:54):

Or new.

CW (09:57):

Dynamic memory is anything where you ask some system, "I need this much memory. Give me a pointer to it." Instead of saying, "Here's this variable. I name it. It's this type."

CW (10:08):

And it's at the top of the file, and that's static memory. It sounds like he's talking about local variables as dynamic memory as well given the stack overflow part?

CW (10:20):

That's kind of sure, sort of dynamic, but I don't usually consider that dynamic memory, do you? Do you consider the stack dynamic memory?

EW (10:27):

No, I don't consider the stack. No, the heap and the stack are separate. And I think of dynamic memory as mostly affecting the heap.

CW (10:35):

Yeah.

EW (10:35):

Or if not the heap, then a heap. And there are times where you have to use some sort of heap functionality. Screens are a big one, displays, because you're changing what's on the screen a lot. And if you have fonts of different sizes and you don't want to pre-allocate everything -

CW (10:59):

Well, yeah, that's the thing. So taking a step back, remember that we're resource-constrained on an embedded system. If your application is airtight and small, and you can define all the things that you will ever use in the memory, and allocate them statically, then great, go for it.

EW (11:18):

Do that. Absolutely. One hundred percent.

CW (11:18):

The number of times you'll be able to accomplish that and have a meaningful application is very, very small. So the fact that you have a limited amount of memory, and you might have different tasks that come and go, means you're going to have to be sharing that memory amongst them.

CW (11:33):

Which means you're going to be having to use some form of dynamic memory.

EW (11:37):

For example, I once had an I2C and a SPI driver, both of whom had to take in a large amount of memory, but neither one of which could work at the same time. So that shared memory between them is technically dynamic, because it isn't a single purpose variable.

CW (11:55):

Now imagine networking. You've got packets coming in. You have buffers. And as they come in, you need a new buffer to do stuff with the packet. Maybe you've got DMA happening. You're filling the next one. You could statically allocate -

EW (12:09):

Ping pong buffer.

CW (12:11):

Ping pong buffers. But -

EW (12:11):

Or some circular buffer.

CW (12:11):

- it's much easier sometimes to say, "Here are 30 items of the same size, and I have a free list and a use list of all those items. And when I need one, I grab it. And when I'm done, I put it back." That's not malloc.

EW (12:27):

No.

CW (12:28):

They're fixed-size. You can consider it kind of a malloc, but you can't ask for a different-sized buffer from that. That's definitely dynamic memory. You do that all the time. It's like a chunk allocator, or a, however you want to call it. There's lots of words for that.

EW (12:44):

And your same-size memory there is very important.

CW (12:46):

Yeah.

EW (12:46):

I don't want to lose that detail. One of the reasons malloc and new are so dangerous is due to fragmentation. So you malloc something that's 10 bytes, and then you free it.

EW (13:02):

And then you malloc something that's five bytes and you free it. And then eventually your memory gets to the point where the ones that you have kept and the ones you have freed are all mixed together. And it can't -

CW (13:15):

And your free memory is all little bits.

EW (13:18):

Right. It's one byte at a time.

CW (13:19):

So now you ask for a big chunk, and there isn't one, because it's all Swiss cheese.

EW (13:25):

And you may have enough memory. You just don't have enough contiguous memory.

CW (13:29):

And there are algorithms that can do that.

EW (13:31):

Right, right. There are absolutely algorithms that can do that. But again, we're on an embedded system.

CW (13:35):

Yeah.

EW (13:35):

It's constrained. The chunk allocators prevent that by having them all be the same size. And then you know that when you allocate and free, you're not going to fragment your memory that way. So having said yes, dynamic memory is important and useful, do you use malloc?

CW (13:57):

Yep.

EW (13:57):

You do.

CW (13:58):

Yep.

EW (13:59):

What cases do you use malloc versus making your own allocator?

CW (14:06):

I'm trying to not talk about -

EW (14:09):

Break any NDAs?

CW (14:09):

- the specifics of what we use malloc for. So I'm trying to think of a generic example.

EW (14:14):

Do you want me to sing the Jeopardy song?

CW (14:16):

Well, your graphics example, right? You may be working with a whole bunch of images that you load from disk.

EW (14:25):

And they're going to be different sizes.

CW (14:26):

They're all different sizes, and you want to keep them in memory so you can draw them quickly. Maybe they're fonts, maybe they're icons, whatever. That's one example where you'd use malloc, because you know how big the file is.

CW (14:38):

And you say, "Oh, okay, this one's 256 bytes. I need a chunk that big. This one's 128. I need one that big." But you don't want to be wasting a bunch of memory by saying, "Well, I know the biggest I ever have is 512. So let me just carve up ten 512 buffers."

CW (14:55):

Well, you've just wasted a huge amount of memory. Set it aside that you can't use for anything else, if there's a bunch of smaller items.

CW (15:02):

So anything where you're loading something from a disk and caching it, anything where you have a document structure,...a markup that's describing something for a UI, or something like that, where you don't have a fixed size, things like that.

CW (15:21):

But...in systems that I'm used to, it's kind of where you are loading something from another resource, and it's of a variable size. And you need to store it in RAM.

EW (15:33):

And then you have to be aware of how much heap you have left and the fragmentation issues that come with it.

CW (15:38):

Yeah, yeah.

EW (15:39):

And you have to handle them.

CW (15:40):

We run into those things all the time. And sometimes you can get away without this sort of stuff if you have...a flash file system, well, not a flash file system, a flash storage system that's directly addressable.

CW (15:53):

And so now all your variable side stuff is just baked into flash, and you can just load it without having to set aside a separate RAM buffer. But it's one of the things you can do with embedded systems. So that's a consideration.

CW (16:05):

If you have something that's variable size, do you need to create memory for it? Or is it on memory already that you can just load?

EW (16:13):

And so that's a graphics example. You -

CW (16:15):

Yeah, you might have -

EW (16:16):

- cheat and you never really load it.

CW (16:18):

Yeah. You might have an 8 megabyte flash that's programmed with your fonts, for example,...your bitmap fonts. And you never really do anything with them except read them straight from flash and spit them onto the screen. So I hope we've confused, everybody.

EW (16:35):

I don't know. We'll see.

CW (16:36):

But the dangling pointer issue in garbage collection that's, yeah. That's life.

EW (16:41):

Yeah.

CW (16:42):

The trade-offs of having dynamic memory without paying attention to freeing things yourself are, you don't know when your program's going to do garbage collection and that kind of thing.

EW (16:54):

And that would increase your latency and affect your performance.

CW (16:57):

Yeah. So there's always a trade-off.

EW (17:00):

Yeah. And for the most part in embedded systems, you are trading off ease of programming for speed, size, and cost.

CW (17:08):

Right.

EW (17:10):

Okay. Ben emailed me in December. Sorry, Ben. And said he has a new magazine coming out called "HackSpace," and it looks pretty cool.

EW (17:22):

It's produced by Raspberry Pi folks,...but they're agnostic as far as technology goes. And he's the editor and thought we'd like it. And truth is I do like it when I read it, and I should read it more, and it's interesting. I'll have a link in the show notes.

CW (17:38):

Cool.

EW (17:40):

Did you get to read it?

CW (17:40):

I did not.

EW (17:40):

I wonder if I didn't even forward that to you.

CW (17:42):

I'm not sure you sent it to me.

EW (17:42):

I'm such a terrible person.

CW (17:48):

What?

EW (17:48):

Okay. So now we're down to a few of the big things.

CW (17:54):

Alright. Let's do it.

EW (17:55):

Kevin and Sami came out to California and said they wanted to have coffee with us. And as always, we were baffled, but we had sandwiches with them.

EW (18:06):

Oh, I'm sorry. Was that not right? Hi, Kevin and Sami. It was really good to meet you. I'm sorry it was so cold at the beach, but it was really lovely to talk to you.

EW (18:15):

And they asked about ShotSpotter and wondered why we had never really talked about them on the show. So I worked at a company called ShotSpotter, and I gave this presentation in 2008.

EW (18:27):

So it's 10 years old and not current. And I can't send you the slides, because there are pictures on them you can't have. But I'm hoping it will still be interesting. So are you ready?

CW (18:40):

Yeah. I pressed play.

EW (18:43):

Okay, now press on the left side of that screen under the 2008. Okay. [Sounds of gunshots and fireworks in background]. So part of this presentation is going to be the sound of gunshots. ShotSpotter is a gunshot location system. You sprinkle sensors around a city and automatically call the police when gunshots or fireworks are heard.

EW (19:04):

So we're just going to let that play while I continue through the first bit. Chris is running the slides, and I'm giving the presentation. This should be fine.

EW (19:13):

So gunshot location systems can be used to reduce gunfire due to holiday celebrations like the one we're listening to now, which is New Year's Eve in Los Angeles. And it is a mix of gunshots and fireworks.

EW (19:28):

And the holiday celebrations and decreasing those is important because that does lead to many injuries and a lot of bullets in roofs. It's more important to discuss the impact on violent crime. Getting officers to the scene of crimes quickly means saving lives and catching criminals.

EW (19:50):

And this is why I worked there, because it was pretty darn cool. I don't work there, and I haven't worked there in many years. Still, I am proud of the work I did there. They are deployed to many cities in the United States, as well as international.

EW (20:04):

And they're in those cities. There has been a greater reduction in crime than there has been in other cities. Although as Christopher likes to point out, there's been a pretty great reduction in crime all over.

EW (20:15):

Here you see, which of course you don't, because I'm not sharing the slides, there's a shooter in the center. And then there are sensors around the city. Sensors often get put into buildings, the tops of buildings. That's all I'm going to say about that one.

EW (20:32):

Oh, but most of this whole presentation is about stories, because I've always loved stories. And I think it's important that we understand not just the technology, but the effect of it.

EW (20:46):

So early in the ShotSpotter history, there was a serial shooter, randomly firing at cars and buildings. In 2003, one person had been killed, others wounded and more casualties expected as the Franklin County Sheriff's department and FBI had no solid leads.

EW (21:02):

ShotSpotter deployed within days once they were contacted. And within hours of becoming operational, they registered the sound of gunfire and led investigators to pinpoint a location where the shooter had fired.

EW (21:14):

This was the first physical evidence, and it allowed them to develop solid leads. The ShotSpotter data was used at his prosecution when he was arrested. So that's one of the stories, but how did we get there?

EW (21:30):

There are four steps to gunshot location, and each one will get its own story. The first step is to detect. We're going to find gunshots in the form of audio pulses.

EW (21:42):

We're going to locate and triangulate where the shooters are. Classify, which is to make sure that it is in fact a shooter and not something more benign. And notify, which is to tell the dispatchers where to find the shooter.

EW (21:55):

So let's start with how the sensor detects. [Gunshot sound]. Alright. So that is what the dispatcher would have heard. They would have actually gotten a little bit of a siren that said that there was an alarm, and then they could click on where the location was.

EW (22:15):

But when they wanted to play the audio, that's what they would have heard. And there were some other things you can play in that same one. [Repeat gunshot sound]. Okay. That's actually the big one.

EW (22:29):

That's a sensor only a hundred meters away from where the shooter was located, and it's loud. A football field away, and it was loud.

EW (22:36):

So what about the next slide? [Softer gunshot sound]. Will you play that one again? [Repeat softer gunshot sound]. Did you hear it?

CW (22:50):

Barely.

EW (22:51):

Yeah. It's a sensor that was almost a mile away. And at that distance, the gunshot sounds like a pretty subtle door knock. Sensors need to find the easy close pulses and the harder to detect ones.

EW (23:04):

You think about the difference in signal quality of those two, it's pretty huge. In this case, in this story for the sensor detection, there were many sensors reporting the location. So it was straightforward. The response time was really quick.

EW (23:21):

The officers arrived to the location, and they found a man who had been fatally shot in the head at a stop sign, really close to the location of the shooter, the shooter position on the map.

EW (23:37):

And I would have been surprised, because we put the shooter on top of the building, which is usually not a good thing. But even though it was an odd spot, the officers trusted the system.

EW (23:50):

And they surrounded the building, and they found a man on the roof with a high-powered rifle, smoking a cigarette. This was a sniper, a hitman, who didn't expect the police to show up so quickly, because single shots are nearly impossible for humans to locate.

EW (24:09):

And that's why ShotSpotter had sensors running around the clock around the city. That sensor does a lot. It captures audio. It uses a GPS for timestamps. It filters it to get rid of spurious pulses and noises.

EW (24:23):

It finds and characterizes these pulses so that they can be combined and compared later. Not just amplitude, but lots of other things. And finally, the pulses are sent back to the central server.

EW (24:36):

So primary function is to detect pulses, but the signal processing is harder than it seems. The buildings distort the sound, giving echoes and reverberations. And the trees, trees eat sound and radio signal.

CW (24:52):

And kites.

EW (24:52):

And kites. The wind changes the direction of the sound, pushing it around. And then there are the people. You're trying to protect all these people, and then they're making all their sounds, like playing basketball and slamming car doors.

EW (25:04):

And we can filter out some of those things, but it only gives us a better shot of hearing shots. Wow. It even says pun intended in my notes.

CW (25:15):

Good notes.

EW (25:15):

Good notes. So in the slides, there's a graphic representation of those two shots we heard earlier, the super loud one and the door knock. The GPS is used to timestamp the audio to nanosecond precision.

EW (25:29):

And if we were to zoom in, the close one, the really big, boomy one, it's full-scale. I mean, it clipped the sensor. The little, tiny door knock was 30% of full-scale, and the ShotSpotter system could pick out pulses that were only 2% of full-scale.

EW (25:50):

So the dynamic range was amazing. Other things that they would look for were rise time, because sometimes fireworks don't explode as fast as gunshots do. And then there's envelope, the duration of the pulse. How long does it take to get back to quiet?

EW (26:09):

And those things help determine how far away a pulse is. If a pulse is very close and very short, that's different than one that is far away and very short. That's just how sound travels.

EW (26:21):

Frequency domain analysis was used as well. Mostly for later classification to get rid of spurious noises. So those various features all went over the radio, the wireless radio, in summary format.

EW (26:37):

And there's a lot of differences between the close sensor and the far one, between the scale of the amplitude, the frequency, the signal to noise ratio, and even the time of arrival. Because it's kind of important for later.

EW (26:50):

...The close sensor had a much earlier time of arrival than the far sensor did. So that makes sense. And that's a lot about the sensor's pulse capability and that's where all of the embedded systems are. And some of it's implied.

EW (27:04):

There's also the managing health and settings, the spooling audio, encrypted communication. And it's all really resource-constrained. They're working on larger systems now, I think, but when I worked there, it was a processor that was designed to work in washing machines.

EW (27:18):

It was somewhat limited. So once we have the pulse, what happens next? Okay. So if you were here, you would be able to see this parking lot, and a ShotSpotter red dot, and an indication that there was a gunshot.

EW (27:35):

And one night a man fired outside his home and later ran into this apartment building we can see. And he took his wife and child hostage, and the ShotSpotter heard these gunshots. Can you play that audio? It's going to be a bit much. [Sound of multiple gunshots].

EW (28:00):

Yeah. ShotSpotter notified the police. And based on the high volume of crime that night, the state police were brought in to handle the situation. They found bullet casings within the parking space that ShotSpotter put the shooter in.

EW (28:15):

So how do we find the location? The algorithm looks at all of these gunshots from the sensors and uses them together to figure it out. We have to go from these three sensors heard a whole bunch of pulses to these sensors heard 20 gunshots. Here's the location for each one of them.

EW (28:34):

And that's especially useful if the shooter is moving, like a drive-by shooting. Then you know which direction they're driving. If you could see this, you'd see that we have multiple sensors, all with a huge number of these pulses.

EW (28:49):

They arrive at different times for each sensor, but the relative timing between the gunshots, once they start, is the same for every sensor. And the ShotSpotter patented method takes advantage of these relative timings.

EW (29:02):

It's a cross-correlation, where one signal slides across the other to find the best match. And if you're thinking the word convolution, yes. Keep thinking that. And then once they do that, the pulses from the different sensors are grouped into a single gunshot.

EW (29:19):

Once you have the single gunshot, you can start to create the location using the same method used in radar and GPS, called time difference of arrival. And it's all about the parabolas.

EW (29:32):

So if we have two sensors that hear a pulse, and we know exactly where those sensors are, because they've been sitting in GPS forever, and we know what time they heard this impulsive sound, this gunshot, the definition of a hyperbola is the set of points where the distance to one foci, like sensor A, minus the distance to the other foci, sensor B is constant.

EW (30:00):

So this is the definition of a hyperbola. A constant difference in distance. And once you translate that to the speed of sound, then the shooter has to be somewhere on the hyperbola. Adding a third sensor allows you to solve for a specific location.

EW (30:17):

Adding a fourth, fifth, and eighth sensor allows you to get some confidence. Overdetermining the solution minimizes the error. Given how I just said that more sensors can give better solutions, you might think that stuffing these sensors in high crime areas would be a good idea.

EW (30:34):

But in reality, it doesn't work that way. With ShotSpotter part of the goal was to keep them far apart. Because if you look at a hammer and a gunshot, a hammer is impulsive, but it's not loud. Not really loud, not gunshot loud.

EW (30:49):

So you keep the sensors pretty far away. And the lower density means that you get less false positives. It's a balancing act. And of course, lower density means lower cost. So back to the story.

EW (31:06):

The police dispatcher saw the red dots. Found the bullet casings when the state police got there. The server had to group the pulses from the different sensors into individual gunshots, and then calculate the location for each gunshot using time difference of arrival.

EW (31:20):

Because the suspect barricaded himself inside and threatened his family, the state police called in a hostage negotiation team. And the suspect was eventually brought into custody by police. No one was hurt.

EW (31:33):

That's one of my favorite stories, because...it starts with the barrage of bullets and ends with no one was hurt. And this low-density, the spatial filtering idea, is great for making sure that we only hear very loud events, or that ShotSpotter sensors only hear very loud events.

EW (31:51):

But it's still very possible to get coincidental events that just happened to all work out. A basketball game here, hammering over there, a car door slamming over here. They all just happened to happen at the right time to lead to a bogus location.

EW (32:07):

So we can use acoustic information about the pulse, as well as environment and system data, to determine if the sound originated from one place or is the coincidental location of multiple things.

EW (32:19):

And these same types of information can lead to the difference between fireworks or gunshots. And if that seems so easy, let's see you try it with some acoustics. Okay. So we're going to classify these ourselves.

EW (32:34):

So we're going to play the first one. [Sound sample 1]. And now we're going to play the second one. [Sound sample 2]. Okay. So you listener, which one of those did you think was a gunshot and which one did you think was a firework? And why?

EW (32:52):

I mean, really think about it. If you could see the slide, it would just be the audio signature of them. The first one is bigger. It's longer. It's got two big pulses in it. The second one is big, but it's kind of got a fat tail. And so, which do you think it is?

EW (33:12):

Christopher? You didn't really hear them, did you?

CW (33:15):

No.

EW (33:15):

Never mind. So -

CW (33:17):

It's all movie magic here. I've heard nothing.

EW (33:20):

So the first one was the gunshot. And one of the ways you could have been able to tell is that the second one had the whistle of a firework and that affects the frequency signature. And of course, that can be used to figure out fireworks versus gunshots.

EW (33:39):

So we're going to do two more. And this one is going to be harder...Those before were taken from sensors about 200 meters from the incident. These are about 600 meters from the incident.

EW (33:53):

Okay. So we're going to play this first one. [Sound sample 3]. And now we'll play the second one. [Sound sample 4].

EW (34:03):

Now, if you were all in a room, I would demand you put your hands up, the top one or the bottom one,...which one is which, which is the gunshot, which is a firework. And then I would ask if anyone wanted to change their answer.

EW (34:17):

And you know what, it doesn't matter. I mean, the first one is the firework, but the correct answer is that there is no correct answer. The increased distance has attenuated the acoustic information such that it doesn't really matter.

EW (34:30):

These are not useful to figure out gunshot versus firework. So there are some other ways to figure that out, some other classification techniques. Usually I always think of these as a little more plebeian, but it's still very useful to know the day of the year.

EW (34:51):

For some reason, there's more fireworks on the 4th of July than the 4th of November. And the hour of the day is kind of important. 8:00 PM versus 2:00 AM. Yeah, one of those is a lot more likely to be gunshots than the other.

EW (35:06):

And then there's the location. Some locations have more fireworks. Minneapolis had more sleet than fireworks during many parts of the year. So finding these obvious features is not -

CW (35:24):

Wait, sleet?

EW (35:24):

Yeah.

CW (35:24):

I think you need to explain that.

EW (35:26):

Well, because the sensors would occasionally pick up sleet, but -

CW (35:30):

Okay.

EW (35:30):

When you're looking at figuring out if something is a gunshot, and...the weather says it is sleeting out, it's not likely to be a firework.

CW (35:43):

Right.

EW (35:43):

So weather plays a part.

CW (35:45):

Gotcha.

EW (35:46):

So yeah, finding out what is important and what matters was a huge part of the classification problem. And that's where I learned the Bayes' Equation. That's where I fell in love with machine learning, although I wasn't quite ready for it.

CW (36:01):

It wasn't called that back then.

EW (36:03):

It was called machine learning back then.

CW (36:05):

It was called machine kindergarten.

EW (36:07):

Yeah. It was sort of machine kindergarten. And each one of these things has to be, the feature has to be identified, and you have to have enough data to figure out which is which. Is it a good separator or just a good separator in certain locations?

EW (36:22):

And how do you know when you have a representative sample of each class? And how do you figure out what the truth is? A lot of humans had to listen to this. A lot of me had to listen to this, and it was years of data.

EW (36:38):

And as good as the classification can be, there's always more that humans can do. And officers have told us about things they look for. Let's listen to this one. The "classify as a trained human one." [Sound sample 5].

EW (36:58):

It's an accelerating shot pattern. The officers have said the shooter is unfamiliar with a weapon, and it is sort of getting away from him. Will you play it again? [Repeat sound sample 5]. Very often, this is young gang member, possibly an initiation shooting.

EW (37:21):

And if you were here, I could show you the difficult acoustics. It's an apartment building on three sides, and the shooter's in the center. And it's not really important though. Once they heard the audio, which the system presented, the officers knew what to look for.

EW (37:40):

And they arrested a new gang initiate nearby, 14 years old, and they found the homicide victim too. He was 17. The heuristics that the dispatchers use, they're beyond the machine classification that the ShotSpotter system could use.

EW (37:55):

And sometimes the key is getting the subject matter experts the information they need. So let's go on to how we do notification, because all that information is no good unless it can get to the right people.

EW (38:13):

So when the police dispatcher is using the system, or when it was in 2008, because I think this part's the most different, across the room, there might be a green screen on a black background that shows nothing's happening. And then red comes up and a sound alerts. [Sound alert].

EW (38:35):

That's the sound alert. And the audio notification is there, because dispatchers could be a room away. They're very busy people. So our imaginary dispatcher walks over to the system, and she selects the unacknowledged incident, which displays where it is, and adjusts the zoom of the citywide map.

EW (38:56):

She can look at the audio form and wonder if it's a gunshot. She looks at the system confidence. It has a high confidence in this multiple gunshot incident. And she can listen to the incident. I think you can listen to that one. [Sound sample 6].

EW (39:19):

ShotSpotter, of course, had ... uh, not a hundred percent accurate classification rate. So they trained the police dispatchers to listen to the incident audio before deciding what to do.

EW (39:30):

In the cases where the dispatcher didn't agree with the system classification, they can reclassify it using their personal judgment. And so what was she going to do with that sound? What was she going to do? What was likely the situation on the street?

EW (39:45):

How many, and what kind of police officers were needed to keep the situation as safe as possible for everyone? Well, the next thing she could see was exactly where these shots came from, because there were multiple of them.

EW (39:58):

...There was one, let's say at time zero, and then 23 seconds later, and then a minute and 17 later, and then nearly two minutes later. And the first incident was in the back of a gas station. A drug deal went bad. Someone fired near that station.

EW (40:16):

Now the police knew that drag races were common around this area, and the drag races are rowdy and troublesome, but not dangerous. Not like this. Once the drug deal went bad, though, everyone started shooting.

EW (40:28):

...In the second shot, the drag racers had to see who had the bigger gun, and that spread into the third and fourth gunshot. Two minutes later, police are showing up. There's been a massive amount of violence. And the dispatcher called an ambulance.

EW (40:47):

Given this amount of gunfire, only three were wounded, and one was killed. Without ShotSpotter in this incident, it's questionable that the incidents would have been called in at all. Less than half of all urban gunfire gets called to 911.

EW (41:00):

We can't locate it. We don't know what it is. It's too far. And the police couldn't have responded so quickly, because the ShotSpotter system would give them locations and not just, "I heard a gunshot."

EW (41:14):

So once everything's done, the dispatcher can write down the actions she took and go about her other work. That data has been used as evidence in court to recreate the crime scene, confirm witness testimony, information like who shot first.

EW (41:30):

Officers also can use the ShotSpotter tool for short-term planning, looking at concentration of incident over multiple weeks, looking for hot spots, and looking for incidents where they might be able to predict and schedule police officers to be nearby.

EW (41:46):

So that's the planning resource. And then there's a slide about databases that I'm just not going to cover. Although I took great glee in it at the time, but now I'm not even admitting that I know SQL. The whole point with the ShotSpotter system is that it can do a lot.

EW (42:03):

And it did a lot to make police more effective, keep them safe, and keep others safe. And I hope the stories show that the system was building a better world, which I gather was the theme of the conference that I was giving this talk at.

EW (42:23):

And there's a lot of different components. And once the word gets around that gunshots are not tolerated in the community, a lot can change, and did, for the covered cities.

EW (42:37):

Alright. Now I'm just in...my area of questions and additional stuff. Do you have any questions, Christopher? Come on. Ask me anything. I remember very little.

CW (42:48):

So the question that everyone always asked for these presentations is, these microphones are listening all the time. What are the privacy implications of having microphones all over a city, that's already covered in cameras, so you've already given up, but what about the microphones?

EW (43:03):

Well, for the most part, we didn't want the microphones to hear people. People are annoying in their insistence on making noise. And so you put them on the tops of buildings, and then you don't really hear voices.

EW (43:21):

...Having listened to a lot of audio, there were times I could hear things like kids playing in playgrounds, but it wasn't like I could hear people talking. And we didn't want to, and we didn't store the data for very long, and only the data that was around impulsive events.

EW (43:42):

So if you stood under a ShotSpotter sensor, and you talk super loud, and you clap at the same time, it might catch you. But only if somebody at the right location, a mile away, is playing basketball. And another person is jackhammering at a different, mile away location.

EW (44:00):

...I could see how people would feel funky about it. Maybe it was just because I listened to so many birds peeping that I didn't really hear people talking. There are a lot of birds peeping out there.

EW (44:16):

And there's really a bird that sounds like the Predator monster, really sounds like the Predator monster. Wish I'd kept that audio. It was really good.

CW (44:24):

He just goes by the Predator, usually.

EW (44:28):

Yeah, well.

CW (44:28):

Other questions. Why hasn't this gone everywhere? Are police, are the departments not -

EW (44:39):

Are there downsides?

CW (44:39):

Are they not excited about it?

EW (44:41):

Well, you know -

CW (44:41):

Was it super expensive, or was it not effective enough, or was it too effective?

EW (44:47):

There are all kinds of problems, and there are all kinds of solutions. The system was not cheap. It certainly definitely was not cheap. And it had the downside that suddenly they knew about all of the gunshots.

EW (45:01):

So you put in this expensive system and the expensive system says, "Wow, you've got a lot more work to do." Certainly -

CW (45:08):

"We'll just turn that off."

EW (45:09):

As far as software engineers go, we've met that with the aforementioned Clang and PC-lint. "Wow. Look how much we paid for this. It's going to cost me months to get through the first day'sstuff." So that was kind of a downside.

EW (45:26):

And...I mean, it has been used a lot. I mean, they're all over, and the company has grown. And to some extent they're changing their focus a little bit. There's been some focus on, I think, some school stuff. But it's pretty amazing.

EW (45:47):

And it went in a lot of places. It went in a lot of places you hear the most for crime. Oakland, some parts of San Francisco, Richmond, Washington D.C., New York, Boston. I mean, 50 cities when I left in 2010-ish.

CW (46:05):

So...I mean, most of the stuff that was deployed was developed in the 2006, 2007 time frame, something like that, probably? Because this presentation was in 2008.

EW (46:14):

...After this presentation, we did a lot more on classification.

CW (46:19):

I have a question coming.

EW (46:21):

Oh, okay. Go ahead.

CW (46:22):

So, now that you don't work there, you can speculate...I'm trying to bring us back to embedded systems.

CW (46:31):

Given where the technology was then, and what radios you were using, and CPUs, and things, how would you redesign this system for free on the radio, given what's available now? You have 30 seconds.

EW (46:49):

I -

CW (46:49):

You can just kind of spitball it.

EW (46:51):

Yeah. Well, there are some things that I know that I can't say. Well,...I mean, my NDA must be five years out of date. I would get a single board computer.

CW (47:02):

Okay. Like what? A Raspberry Pi level thing?

EW (47:05):

Raspberry Pi 3 would be my minimum.

CW (47:09):

Wow. Given what you said about it being a washing machine chip before -

EW (47:14):

Well, I'm going to...go into more.

CW (47:15):

Okay.

EW (47:15):

Maybe one of the new BeagleBones. Maybe even the Jetson. I mean the Jetson would be overpowered, but you could do a lot more with it, the TX1 maybe.

EW (47:28):

And the advantage to having a Linux system is we had a lot of care and feeding of our file systems and our firmware updates. And while putting up sensors was expensive, the most expensive part of putting up sensors was installation.

EW (47:44):

And having a system go down because of bad firmware was just heartbreaking. I mean, knowing that you had to send somebody into a place where there are a lot of gunshots in order to fix your software bug, it was pretty painful.

EW (47:59):

So I would do Linux, because I think that the update opportunities are better. The hard part is timestamping things.

EW (48:11):

So you need GPS, and I would put four or even eight sensors, microphones, on the system with a very well-calibrated, 3D-printed thing. And then you get the location.

CW (48:27):

And then you can do the beamforming thing that Amazon thingy over there that I can't say the name of can do?

EW (48:32):

Yeah. Yeah, actually, if you do have an Echo, it is like that. It's like where she puts her lights. She's creating the beamforming thing, as Christopher said, triangulation. And, so you have multiple sensors, so you're getting a direction.

EW (48:51):

Now, you still don't have a distance. You need more than one sensor far apart in order to get a location. And that means you need to have some sort of radio for them.

EW (49:01):

If you could convince neighbors all around the city that you were in, although again, not close neighbors, you still need them pretty far apart. But if you could convince them to put them all on Ethernet and power, that would be awesome.

EW (49:14):

Otherwise you're going to need a solar panel and something like LoRa, the long range radio. Or there are some municipal Wi-Fi networks you can use. Or I suppose you could sniff off of Starbucks.

CW (49:26):

I don't think that's a good business plan.

EW (49:28):

I don't know. It wouldn't be that bad. So yeah, I would do some higher power Linux, and I would do more features on the device...It isn't trivial. I don't want to give that impression, because the difficult parts are the distributed sensor network.

CW (49:48):

Right.

EW (49:49):

And you can't do that on your own, by yourself, one person, one sensor.

CW (49:55):

Okay.

EW (49:55):

Okay. So Kevin and Sami, that was ShotSpotter. Hope you enjoyed it. It is not the same now. I have not been involved with ShotSpotter in many years.

CW (50:08):

Now it's all in the cloud.

EW (50:10):

But I wish them lots of luck.

CW (50:13):

I made that up. I don't know anything.

EW (50:13):

Yeah. It's in the clouds as they float by. Those clouds, right?

CW (50:18):

Yeah. Alright. So...that question caused you to give a presentation. What's the next question going to do?

EW (50:26):

Bobby asked about the mention I made of LeapFrog. And I mentioned that my toys were canceled one year, and that it led to an epiphany recently about whatever it led to an epiphany about, because that isn't important.

EW (50:42):

And I mentioned they shared a lot of code. And Bobby asked if I could share a little bit about how I prefer to share code between embedded systems projects. And I'm sorry, Bobby, I gave you the wrong impression.

EW (50:54):

They did share a lot of code, but they were all on the same processor. And I shared code with a few other people who were all on the same processor.

EW (51:05):

And our code, because it was a consumer product, because it was a toy, had a limited lifespan. As soon as we shipped, we did no more development on these toys. So when you think about -

CW (51:16):

God, that sounds wonderful.

EW (51:18):

I know, doesn't it? It was kind of magically wonderful. So when you work on sharing code, and you think, "Okay, I'm going to share this code, and it's going to be great."

EW (51:29):

And then you realize six months later, you have a dozen processors and two dozen products. And how do you cherrypick from A to B, and get the libraries right, and not confuse the dependencies? There is no good way.

CW (51:45):

What?

EW (51:47):

I'm sorry to say this. There is no canonically great way to unravel this spaghetti.

CW (51:55):

I think CMake is what does it. That's what I've heard. CMake solves all the problems.

EW (52:02):

I know Christopher is being facetious, because I've heard him talk about CMake, and the things he said about it is not suitable for unexplicit radio.

CW (52:13):

[Hmph].

EW (52:16):

There is no great solution. If anybody knows one, let me know. And I do keep meaning to write a blog post about this and give a couple of options, but you wanted a good cookie-cutter solution, and I'm sorry, I cheated.

EW (52:31):

I didn't have all of the constraints that I know you probably have. And so I think that's it. I have a couple other things to talk about at some point, but I haven't tried out my Ikalogic scopes yet, so -

CW (52:43):

Oh, okay. Yeah. I forgot about that.

EW (52:45):

We'll save that for next time.

CW (52:46):

Cool. I don't have anything else exciting.

EW (52:50):

Well then, I guess thank you for -

CW (52:52):

Thank you for pushing the buttons?

EW (52:57):

Thank you for pushing the buttons.

CW (52:59):

Yeah.

EW (53:01):

Yeah, I'm sorry about the edit on this one. Thank you to Christopher for co-hosting and for producing. This one is going to be tough. So I do appreciate how much he let me talk.

EW (53:14):

If you would like to contact us, you can always do show@embedded.fm, or hit the contact link on embedded.fm. We do like hearing you so feel free to say hello. And now I usually read Winnie the Pooh at the end of these.

EW (53:32):

[Winnie the Pooh excerpt].