Transcript from 496: Beauty, Elegance, Consistency with Shimon Schocken, Christopher White, and Elecia White.

Welcome to Embedded. I am Elecia White, alongside Christopher White. What do NAND gates and Tetris have in common? Our guest this week is Shimon Schocken, and he is going to tell us all about it.

CW (00:21):

Hi, Shimon. Welcome.

SS (00:21):

Hi. Thank you. Thanks for having me here.

EW (00:26):

Could you tell us about yourself?

SS (00:29):

Yeah. Well, I am computer scientist located in Israel. I am actually a computer science professor, teaching at the university called "Reichman University." I joined this university about 30 years ago, and actually was one of the founders of this new school. Before that, I was a professor at NYU for ten years. Before that, I did a PhD at the University of Pennsylvania.

(01:05):

I enjoy programming, hacking, building big systems, and I enjoy teaching. So I think that the best way to describe my work is a computer science educator. That is me.

EW (01:25):

You did not mention writing a book called "Elements of System Architecture."

SS (01:30):

Right. Well, I think the correct title is "Elements of Computing Systems," but it is essentially the same thing.

EW (01:39):

Apologies.

SS (01:43):

Well, I did a few more things. Maybe I will say a few more, in some more details. I also- Yeah, I wrote this book, which I guess we will talk about. I also started up a company about 12 years ago, that teaches elementary school mathematics using their computer games. That is a very different aspect of education, but that is another passion that I have, teaching math using games.

EW (02:14):

You also are teaching computer science using games.

SS (02:18):

Yeah, actually, that is right. I also contrive all sorts of games to teach computer science. Or actually more accurately, algorithms. This is done in the spirit of a pretty large worldwide project, called "CS Unplugged." It is a great project that started up in New Zealand, from all places, by a wonderful fellow named Tim Bell, a colleague of mine.

(02:50):

Tim and his colleagues put together a bunch of games. These are kind of schoolyard games, using very basic things like ropes and cards and balls and so on. They teach all sorts of classical algorithms and data structures, using these games. The whole idea is to teach algorithms and computational thinking, without computers.

(03:17):

Because the irony is that once you have your hands on the keyboard, once you have a computer in front of you, you get distracted from the major ideas. So that is another thing that I did.

EW (03:33):

One way to make concepts more understandable is to give them physical form.

SS (03:41):

Absolutely.

EW (03:43):

Did that concept apply as you worked on the "NAND to Tetris" book?

SS (03:49):

Yeah, absolutely. I am not sure how deep you dived into the course that we teach and the notes in the book, but along with the book, we provide a full-blown IDE, that consists of a hardware simulator, a CPU emulator, a VM emulator, an assembler, and a few more tools.

(04:22):

It was very important to us when we wrote all these various simulators and emulators, to visualize what happens, what was going on inside the computer. So for example, when we teach virtual machines and we use the VM emulator, you can actually see the stack growing, when a certain command- When you do a push or a pop, you see the stack growing or shrinking.

(04:53):

The same holds for everything else that we do. When you write a particular chip in a chip set, some of the chips have visualizations. So you can actually see a value going into the register. You can see the register stabilizing on the right value before it outputs it, and so on. So yes, I think that visualization is terribly important in computer science education.

CW (05:30):

Elecia, you just said the book was "NAND to Tetris." Is that the same book as "Elements of Computing Systems"?

SS (05:37):

Yeah, well, that is a bit of, I guess, a confusion. The title of the book is "The Elements of Computing Systems." The subtitle is "Building a Modern Computer From the ground up" or "From First Principles," I do not remember. And the course that we teach based on this book, is called "NAND to Tetris."

CW (06:04):

Okay.

SS (06:04):

Yep.

EW (06:08):

And I hear people refer to it as the "NAND to Tetris" book.

SS (06:12):

Yeah. They are right. I am not sure if the title is a good choice. I think some people find "NAND to Tetris" a little bit too whimsical. In my university, the course is called "NAND to Tetris." Not only in mine. In several other serious universities, so to speak, the course goes by this title.

(06:41):

And yet, some instructors feel better with a formal name like "Digital Systems Architecture," or whatnot. For these people, I think the title of the book is more appropriate. Also, there is a little twist in the title of the book, because- I called it "The Elements of Computing Systems," because one of the greatest little books that I read is called "The Elements of Style."

CW (07:12):

Ahh. I wondered if there was a connection. Yes.

SS (07:15):

Yeah. Strunk and White. I am a great believer in style, and so I hope that some people see the connection.

CW (07:24):

So, from the name of the course, and I have not read the book- Elecia has read the book, and I am ignorant observer. So the course takes you from gate level to building an entire computer capable of playing Tetris, is the basic idea, right?

SS (07:41):

Yeah, playing Tetris. And actually, any other program.

CW (07:43):

Okay.

EW (07:46):

It goes from, "What is Boolean logic?" to, "How do you build these into parts of a processor, ALUs?" Then it goes to, "How do you build microcode to run on those?"

CW (08:00):

And then assembly language.

EW (08:01):

And then assembly language, and a compiler, and a small operating system, and a display, and then actual programs on top, such as Tetris.

CW (08:12):

Okay.

EW (08:13):

So every single part. I do not understand how you could do this in a semester. <laugh>

CW (08:20):

So yes. How do you do that in a semester? <laugh>

SS (08:24):

Yeah, that is a good question. Well, first of all, Noam and I- My co-author is Noam Nisan, who is a professor at the Hebrew University in Jerusalem. It took us about five years to put it together, but this was not full-time work. We were fully employed in other jobs when we did it. So on and off, it took five years to get it right.

(08:53):

The reason we can do it in one semester, is that we- There are several reasons. First of all, we do not worry about so-called exceptions, so we assume that the input is always correct. So when you write the compiler in our course, you assume that the source code has no syntax errors.

EW (09:24):

That seems like a really tough assumption. It would make me really appreciate a compiler, that could tell me that I was doing things like forgetting a semicolon.

CW (09:33):

I have not had a compiler that can do that yet.

EW (09:35):

<laugh>

CW (09:35):

You always get some other error. <laugh>

SS (09:42):

You have to make these concessions for two reasons. First of all, once again, if you want to write the compiler in two weeks or three weeks. Which is what we do, actually. It is not only- The course is taught in about 400 universities now. So I am speaking now for 400 other instructors, and who knows how many instructors will teach it outside the university system. And it works.

(10:09):

The course actually works, and students do build all the tools that I described. So one trick is to assume that the inputs are error free. This actually has a great pedagogical virtue, because if you assume that you have to take care of the errors, it can really distract you from the great ideas of compilation, which is parsing and code generation.

(10:40):

I think that if I were to write an industrial strength compiler, I would do exactly the same thing. I would start with a compiler that handles only valid code, and then I will add the error correction later on. I think it would be the right way to do it. So that that is one concession.

(11:04):

The other concession is that we do not expect students to do any design in this course. Instead, we give them all the blueprints and the APIs. And so when they write a compiler, we tell them that we have a proposed implementation, which is based on the following API, and we break it down to the method/function level.

(11:34):

So the compiler is actually a collection of something like 30, 40 different methods, divided into something like four or five different classes or modules, or files in Python. By the way, we are completely language agnostic. You can write the compiler in any language you want. So we know as long as it produces the right outputs, we are happy. So that is another concession, giving the students a good design. I can elaborate on this later.

(12:10):

The third concession is not really a concession, but- What should I call it? Third advice, I guess, is that I always think that before you write a program, you should first write another program, that tests the program that you want to write. That is what we do here also. For every project in the course, we provide elaborate test scripts that describe exactly what the tool is supposed to do, in an executable form.

(12:49):

Yeah, and that is actually another feature. For every tool that we want the students to write, we have an executable version of that tool. So the student can actually play with the tool, and run all the tests on that tool, before he or she wrote a single line of code. They fully understand what the tool is supposed to do, before they set out to actually build it.

(13:13):

That is another guideline. And there are a few more. All these guidelines are both- I think, both pedagogical and engineering know-how and best practice. I think they go hand in hand.

EW (13:35):

The second one, the no design. I noticed that your book separated the interfaces from the implementation very, very strictly. As you mentioned, you give people the interface, the API. Then they need to use the information they gathered in the chapter, to implement to that API. Is this how you want students to approach work in industry?

SS (14:13):

Absolutely. I think that it is terribly important to separate abstraction from implementation, on many different levels. Abstraction focuses on what the system is supposed to do, and implementation focuses on how the system should actually do it. These are two very different perspectives on the same thing.

(14:41):

I think that many catastrophic systems that I saw, were developed because developers either did not care or did not know, how to create this separation. Just imagine writing code without fully thinking what you want the code to do. That is actually what happens when you do not separate.

(15:02):

And when you forget about the implementation, and focus only on the abstraction, you can- Basically, what you do is you manage complexity. You do not worry about numerous technical details, and you focus about the big picture. This will help you come up with a good modular design. This will help you come up with a design that lends itself to unit testing.

(15:33):

And when you do it right, you can also- As I said before, you can develop all your test programs, before you write a single line of code in your system. Because-

(15:45):

What I am saying now is probably widely done by good programmers. But once again, when you write a function or a method, it is best to just write the skeleton of the method, and have it return zero or null or one or minus one, or something silly like that. And then test it. Test the damn thing, before you actually turn it into executable code.

(16:11):

When you work like that, you can focus on the architecture, without worrying about the furniture and the interior design and so on. This is another important principle that we try to convey in this course.

(16:30):

The course is not really- Well, I should be careful not to say it. I was going to say in the book, that the course is not about developing a computer. It is about building a computer. But hopefully the take home lessons are much broader. And one of them is the critical importance of separating abstraction from implementation.

EW (16:55):

Do you worry that students will not understand how to do the design portion, or the unit test portion, themselves? That they will get stuck into only the implementation portion?

SS (17:08):

Yeah, I do worry about it. But I think that they get it. I can elaborate about it a little bit. Also, maybe with your permission, I can modify a little bit the question. One question that we are being asked, every once in a while by colleagues, is that- Or I guess, one possible source of criticism, is that you do not really teach design. You give all the designs to the students, and they, quote unquote, only implement them. And that is true.

(17:48):

This course is not about design, and yet it is about design. And I will tell you why. Because the best way to learn design, is to see many good examples of good designs.

EW (18:04):

Yes.

SS (18:04):

And that is what we do. We give the students numerous examples of what we think is good hardware design and good software design.

(18:13):

When the students implement them, and they begin testing them, they feel empowered. That is one word that we hear a lot from students in this course, "We feel empowered." They feel empowered because the design automatically guides them in the right direction.

(18:35):

They write the code, they execute it, and boom, they realize that what they wrote sets the stage for the next function that they have to implement, in a very natural way. So they appreciate the designs that we give them.

(18:57):

That is one bug that we have in computer science education. When you think about the course, like "Introduction to Computer Science," I think that in the first week, second week of the course, the students begin to write programs. They do not really read programs. They only write. Write, write, write, write.

(19:17):

When you think about "Introduction to Poetry," instead of "Introduction to Computer Science," I do not think that the poetry or the literature professor comes into the class in "Introduction to Poetry 101" and says, "For the next lecture, I want you to write a poem." Instead, she will say, "Here are four great poems. Read them." Or, "Read them, and tell me how you feel about them." We do not do it in computer science.

(19:45):

What we could do is come into the class and say, "Here are four programs. Two of them are great. Two of them suck. Why do you not tell me, what do you think about each one of these programs?"

(20:00):

So that is something, once again, that we should improve in computer science education and in programming education in general. That is another reason why we gave all these designs, because we wanted to show examples of good designs. Learning by example.

CW (20:20):

That is an interesting point about reading. Because when we were in undergraduate, the open source movement was not really- It was just getting off the ground. So there were not a lot of great examples publicly available of good code.

EW (20:38):

We went to school before Wikipedia, and way before GitHub.

CW (20:42):

Right. So it would have been much more of a challenge to find a lot of examples of good code, that were publicly open and available. Whereas now, there are tons of it everywhere.

EW (20:53):

Go check out the Adafruit sensor library.

CW (20:55):

Right. Yeah. A good example.

EW (20:58):

Yes. I love this idea of teaching people to read code. I totally agree with you about the book, that it actually encouraged me- As I was reading, I was thinking, "How will we implement this?" And then I got to your implementation section with the API. The way you had broken up made a lot of sense. And the parts where I was going to go somewhere different, it made me question my assumptions. So it was great.

(21:34):

Yes, you do not teach design the way some people teach design. Instead, you give really good examples. If this was a live stream, I would have put so many little hearts there, to indicate how much I agree with you.

SS (21:51):

Great. I am glad to hear that. By the way, the designs are optional. If a student comes up with her own design, that is perfectly okay. Once again, because we do everything in one semester, we do not have too much time to comment about- Well, I should not get into all the details of grading and all that. But basically when you write a compiler, we want to see your compiler working.

(22:26):

If you chose a different architecture, that is fine with us. Although I must say that in 99% of the cases, the students simply follow our own design. So it is not mandatory, but it is highly followed.

EW (22:46):

<music> Thank you to Memfault for sponsoring this show. We appreciate their sponsorship, and the work that they do. Memfault is a leading embedded device observability platform, that empowers teams to build better IoT products faster.

(23:00):

What that means is that if you have just realized that you are going to build five, ten, a hundred, 50,000 units, and you need to keep track of them, they will let you create your own dashboard to observe how your system is doing in the field.

(23:16):

Memfault gives developers a more scalable and sustainable process. This accelerates the time to market and de-risks product launches. You can cut product costs and deliver more high quality software.

(23:29):

Trusted by leading brands, such as Bose, Lyft, Logitech, Panasonic, and Augury, Memfault improves the reliability of devices across consumer electronics. And across mission critical systems, such as access control, point of sale, energy and healthcare.

(23:46):

Thanks again to Memfault for sponsoring this show. Check out memfault.com and the Interrupt blog, which is filled with incredible amounts of information. <music>

(23:58):

I did think a semester was short, for this pure amount of information. Partially because I do not think I could have learned all of the details of logic from this course. If I had taken a different course, that taught all of the circuit logic things, then this would have been a good refresher. But I do not know that I could have gone from zero to ready to use it.

CW (24:27):

Mastery.

EW (24:28):

In a short time. I wanted this to be a year long course, but that would have just been me playing professor. How did you decide what pieces of information you had to assume the student would know?

SS (24:43):

Okay. We assume knowledge equivalent to "Introduction of Computer Science," and some- Which, in a corollary of this, is some good programming experience, typically in a language like Java or Python.

(25:03):

We assume basic knowledge of object-oriented, but not too much. We do not really need inheritance. We just need- You have to understand what are objects and classes and so on. Because some of our designs are object-oriented.

(25:18):

And we assume, in a typical university setting, that at some point the student will take many other super relevant courses. Like "Logic Design." Like "Operating Systems." Like maybe "Compilation." Like "Advanced Programming." But we do not necessarily assume that they took them already.

(25:41):

When they take these courses, every once in a while they smile to themselves and say, "Gee. We did that in 'NAND to Tetris.' We did it all the way. We saw how it interacted with concepts from other courses that we took." That is a very important feature of this course. It connects ideas from many different courses, in a hands-on fashion.

(26:09):

But also I have to take exception to one thing that you said, about needing logic. You are right. When you implement, I do not know- When you implement something like XOR, you have to know something about XOR logic. And yet, we explained some of it in the course, and we expect students to- They can go to the internet and find how XOR is implemented. We do not care.

(26:43):

And once you know the basic logic formulas, then you have the challenge of actually implementing them in HDL, which is a functional language. Or with our own version of HDL, which is something that you can learn in one hour. So the logic knowledge is we either- We assume that the student gets it somewhere, or in many cases we give it away.

(27:11):

We just, in the lecture or in the book, I do not remember exactly where, we tell them. "We can implement an adder chip- A chip that adds two bits. You can implement it using AND and XOR. The sum can be implemented using an AND gate, and the carry bit can be implemented using XOR. Or the other way around. I do not remember." So we give them many hints about how to do it, and then you have to set out and do it.

(27:45):

Likewise, when we implement, let us say a RAM device, then we tell them something like, "Let us start with a RAM of eight registers, and an input value comes in. We recommend that you feed this input to all the registers simultaneously. Then take the address input and use it to select which register to open, so to speak. To load. To set the load bit on which register. One register should have an asserted load bit, and all the others should have their load bits set to zero."

(28:32):

Once you gave this hint in one way or another, the logic challenge goes away. You do not have to worry too much about understanding the logic. But you still, once again, you have to actually implement it.

(28:51):

Because of that, there is a very interesting phenomenon in "NAND to Tetris." Something that we never anticipated. It turns out that the first part of the course, which is building the hardware platform, can be done by any student or any learner, without any previous knowledge. You do not need programming-

EW (29:15):

True.

SS (29:15):

In order to do it. Because the only thing that you need to know is HDL, and we teach you HDL in the course, or our own HDL. So actually the first part of "NAND to Tetris," building the hardware platform, is something which is widely done in high schools, and in various programs, community programs, and so on.

(29:36):

That is the typical kind of a journey that students take, is that they do the first half of the course somewhere. They get completely hooked on the approach. And then they go on to do the second part of the course, which is actually much harder, developing the software hierarchy.

EW (29:59):

Yes it was. <laugh> I guess I spent so much time with De Morgan's law, and trying to optimize-

CW (30:06):

It is a great law.

EW (30:07):

Exactly how to put the logic together-

CW (30:09):

Oh. Cleanly.

EW (30:10):

For the minimum number of-

CW (30:11):

Yes. Of operations.

EW (30:13):

Operations. I still have that part of my brain that says, "This was harder than I expected." But if you do not try to go for the minimum number of operations, it is not nearly as fun, but it is also not nearly as challenging. I think that is why I had it as a, "Wow. This does not seem like enough," but I see your point.

CW (30:36):

But you were self-paced. If you were forced to move on, because of the pace of the course, would you have spent so much time on that?

EW (30:44):

No, the whole De Morgan's law was many years ago, so it was-

CW (30:47):

Oh. Oh, I see. Not while you were doing this. Got it.

EW (30:49):

Not while I was doing this. This I actually could rush through that, because I had already spent plenty of pain on exactly how to do these.

CW (30:57):

<laugh>

SS (30:58):

Yeah. Well, let me comment about this. Two comments. First of all, another concession that we do in this course, is that we do not care about efficiency. That is a huge concession, because 90% of computer science is about efficiency, and we do not care about it.

(31:17):

If you write your chip using ten chip parts instead of seven, that is perfectly okay with us. If it is ten instead of one or two, maybe a problem. So that is another huge concession, which the students appreciate.

(31:38):

The second comment is that coming up with a minimum number of gates- Well, I have to tell you, it is an NP-complete problem.

CW (31:51):

<laugh> Yeah.

EW (31:51):

<laugh> Yes.

SS (31:54):

So do not feel incompetent or something. That is how you should feel. It is a huge combinatorial challenge, and it is very difficult to optimize these chips.

(32:06):

There are some tools. If you remember the term, something called "Karnaugh maps." And all sorts of both theoretical and practical tools, that help you sometimes come up with better design. But the general problem of coming up with a minimal design, is unsolvable. It is a huge open question. So definitely we do not deal with it in this course.

EW (32:36):

One of the things that I see with students coming out of college, is that some of them are very focused on trying to get the most efficient, most optimized system. And yet, we do not actually need to do that very often.

(32:52):

Yes, sometimes there is one loop that needs to be optimized. But I do not want to optimize all the code, if there is only one area of concern. So I like that you are not teaching them efficiency, because we do not always need it. Our processors are getting bigger and faster.

SS (33:15):

Definitely. Yeah. Well, let me maybe say it a little bit differently. I think that I prefer readability on efficiency.

EW (33:25):

Yes!

SS (33:25):

So in the course- The course gives you a great platform to come up with all these insights, and share them with the students. Once again, I think it is another reason why the course became popular, is because different instructors use it to emphasize different lessons. So in that respect, it is open-ended.

EW (33:54):

When you say, "The course," you have the book, which anyone can purchase. And you have a website. And a GitHub repo, with all the simulators and code. And you have slides. And you have an online project simulator. Can anyone just use this for their own class? All of that?

SS (34:16):

Yeah. Yeah. There are many courses out there that students can take freely in open source. The special thing about our course is that you can also teach it freely. Anyone who wants to teach this course, can simply go to the website, take whatever he or she wants. And if you want more material, you can send me email, and I will send you some more stuff. Yeah, the course is widely available.

EW (34:46):

It is a lot of work for you to make just freely available.

SS (34:50):

Yeah. But I will say something that sounds maybe a bit ideological. But I think both Noam and I- Noam, my co-author and friend, we feel very fortunate. We feel grateful to the universities in which we studied, and to the universities in which we taught. We feel that they gave us a lot.

(35:18):

When I say, "They," I mean the teachers that taught us, the institutions in general. We view it as an opportunity to give back. So that is why we made everything widely available.

EW (35:35):

I have a couple of listener questions. One of them is really relevant. "Are you aware of any institutions in Africa, that use the 'NAND to Tetris' in their curriculum?" From Antoine.

SS (35:45):

I am afraid not. I mean, I am not aware. Yeah, actually I do remember getting emails, I think from Nigeria. I get numerous emails from people who either take the course or teach it. There were, over the years, quite a few emails, maybe a dozen emails or so, from people from Africa. But I do not remember if they took the course or taught it.

(36:16):

It is quite possible that they took it in Coursera, or something like this. I do not know if this counts. So I do not know.

(36:24):

I can tell you. I mentioned the math games program. This program is being used in Africa. It works in Swahili, and so it is taken in several different countries in Africa. It is also free. I mean, the African version is free.

EW (36:51):

There is no reason why an institution would have to tell you it was using it.

SS (36:57):

That is right. When I said that it is being taught in 400 universities, these are universities that I know about. In other words, instructors who sent me an email saying, "I am teaching this course in this and that university. Can you send me some more material?" So I have 400 of those. It is quite possible there are many others who teach it also.

EW (37:23):

The next question is from Benny. "Do you have an extension to this program, and what is it?"

SS (37:33):

Yeah. Well. We do not have an- You mean a follow-up course?

EW (37:40):

He actually said, "Embedded extension." So I do think that is a follow-up course.

SS (37:45):

Yes. Well. The embedded extension is not really a follow-up course. It is a project. It is an optional project, in which you can basically implement the computer in FPGA. There is a whole project that guides you through doing that.

(38:08):

You have to buy a $50 board from somewhere. We give some recommendations where to get it. Then we guide you through the process of actually taking our HDL, porting it into industrial strength HDL, like Verilog or VHDL, and then putting this code into the FPGA.

(38:43):

This was actually done by a student that took the course. His name is Michael Schröder, from Germany. He did it very nicely. Now I work together with him to make it more accessible to other students, but it is already in the website. So if you go to- In the website, there is a section called "Cool Stuff." One of the projects in the Cool Stuff is the FPGA implementation.

EW (39:08):

Benny also asks, "If you were to develop a new course from scratch, what would it be?"

SS (39:14):

So both Noam and I did follow-up projects, which are somewhat in the same spirit, and different levels of maturity. Noam together with another colleague- His name is Yannai Gonczarowski. So Noam and Yannai have built a course and wrote a book. I think it came out in Cambridge University Press. They basically- What they say sounds a bit crazy.

(39:50):

They implemented symbolic logic. Basically, they start with the most basic concepts in logic. What is an alphabet of symbols? What is an axiom? What is a proposition? What is a conjecture? What is a theorem? What is a truth table? And every one of these concepts is implemented in a little Python program, that typically consists of something like three or four lines of code.

(40:25):

As you go from one week to another, you basically build logic from the ground up. After working for, once again, a whole semester, you end up with what is known as "Gödel's completeness theorem." You actually prove Gödel's theorem, but you prove it constructively by actually building everything from the ground up.

EW (40:56):

Did not Gödel's completeness theorem say you could not do that?

SS (40:59):

No. There are several theorems. The first completeness theorem says that in what is not its first order logic, you can do it. In other words, basically the theorem says that everything which is true can be proved. Right?

(41:17):

So if you limit yourself to propositional logic, which is a logic without variables, only basic logical arguments, then it is complete. If you move on to predicate logic, in which you can use quantifiers, like for each and for all, then you get stuck. Then there are true things that you cannot prove.

(41:43):

So the first theorem, which is very important, just like the other ones, is kind of the pinnacle of Noam's second course. I should not say, "Second." It is a different course. But it is exactly in the same spirit of building everything from the ground up.

EW (42:07):

That sounds fun.

SS (42:09):

Yeah, it is. This course replaces the traditional course in mathematical logic, which students are really traumatized.

CW (42:27):

<laugh> I loved that course.

EW (42:27):

<laugh>

SS (42:27):

Yeah. Yeah.

CW (42:27):

Oh. I was a weirdo, though. <laugh>

SS (42:33):

Because there are so many syntax issues, and-

CW (42:36):

Yeah.

SS (42:38):

A language, that describes the language, that describes the language, and so on. And in Noam's course- In Noam and Yannai's course, you actually build everything. You build everything in Python. So the students, they do not feel that they study logic. They end up studying logic very well, but they study logic by writing numerous little programs. And proving everything along the way, in the way of construction.

(43:04):

They do not prove theorems. They build code, and the code works, and that is the proof that it works. So it is not highly formal, but it turns a very challenging course both for teachers and students, to something which is completely fun. So that is what Noam did.

(43:33):

What I want to do, and I started working on it, but- Together with a colleague called Rami [Fularch ? ], we want to build a course that I can call it whimsically "From NAND to life."

CW (43:51):

<laugh>

SS (43:51):

<laugh> Basically, what we want to do is build things like proteins step by step. So start with the basic DNA alphabet. And from the DNA, build genes and chromosomes. And using genetic code, build proteins. And then maybe even from the proteins, build some organs. And end up with a cell, with some functioning cell.

(44:33):

Everything obviously is terribly reduced, with great simplifications. But the basic ideas hold. We want the students to visualize everything, and to feel that they built stuff along the way. Once you go through this course, you will understand, I hope, the basic ideas in molecular biology. Without numerous details, of course.

(45:03):

That is work in progress. It is not something that I can really talk about too much, because we just started working on it.

CW (45:14):

The idea of a streamlined synthesis course for big disciplines, is actually applicable all over the place.

SS (45:24):

Yes. And- That is a great comment. I agree, and a great insight. It is an important insight, because it is rarely done, which I think is rather crazy.

(45:35):

Computer science is very much a top down discipline. When you learn- Well, think about the program. The first course in the program teaches you something like Java or Python. So you end up taking "Introduction to Computer Science," and you know nothing about computers.

(45:56):

You know programming. You can solve some problems, but you have no idea- As far as you are concerned, the computer can be a coffee machine. That is the price, so to speak, of top down design. That you do not understand the machines that make your ideas actually execute.

(46:19):

Our approach is bottom up. We start- That is why the subtitle of the course is "Building a Modern Computer from First Principles." So in our approach, you start with the most basic things.

(46:38):

It may take two, three weeks before you understand why you started with these particular building blocks, which I think is great. It has a great educational virtue, because we live in an era where instant gratification is king, and I think that this is terrible for good rigorous education. So the fact that the student is left a little bit in the dark for a while, and then at some point the students say, "Wow! Now I understand why I had to use these particular building blocks. It makes perfect sense to me." I think it is great.

(47:22):

I think it is a great experience, to withhold these aha moments to a later stage, where you put a lot of work and you feel accomplished and empowered, and all of a sudden things fall into place. I think that if you think back at your education, you will remember these great moments, where all of a sudden things begin to click.

(47:48):

This is something that happens when you do bottom up, because system building. It takes a little while before the puzzle begins to take shape.

CW (48:05):

I think that is interesting you said about our education, because- Listeners are going to get tired of me talking about this. But when I learned computers, I was a very young kid-

EW (48:13):

Dinosaurs.

CW (48:13):

And I had an Apple II- I had an Apple II Plus, in the late seventies, early eighties. It was a very simple computer by modern standards. It is very simple computer by toaster standards now.

(48:24):

But when you got one of those, there was not a lot of hand-holding. It had a BASIC interpreter. It had a ROM monitor, and assembly language. And a whole ream of manuals, with all of-

EW (48:38):

That included a schematic.

CW (48:38):

And the ROM disassembly. And so people- Hobbyists and people, I think even in school, were learning computers with very simple systems, that you could comprehend after a short period of time. And now computers are so complex that...

EW (48:54):

They are magic boxes.

CW (48:55):

They are magic boxes, and people are going through school and learning, "Oh. Well, I am going to write, I do not know, Node.js that runs on AWS to handle backend requests and things," which is so divorced from the bottom part of computers.

EW (49:08):

Logic gates.

CW (49:09):

Right. That it is almost like a different system. And I think getting back to the, "Okay. This is how these work. All of them still work this way. They are just more of this," is really, really helpful. Because it is better to have people who understand what is going on, than people who are skimming the surface, and can do a very narrow set of things. I do not really know what my point was.

SS (49:33):

No, no. Chris, I think I completely connect to what you just said. Actually I opened the book, which I have in front of me, and I am going to read to you the second paragraph of the first page in the book.

(49:44):

The second paragraph says, "In the early days of computers, any curious person who cared to do so, could gain a gestalt understanding of how the machine works. The interaction between hardware and software was simple and transparent enough, to produce a coherent picture of the computer's operations."

(50:01):

"Alas, as digital technologies have become increasingly more complex, this clarity is all but lost. The most fundamental ideas and techniques in computer science, the very essence of the field, are now hidden under many layers of obscure interfaces and proprietary implementations." And so on.

CW (50:20):

All right. <laugh> Yeah.

SS (50:20):

So that is exactly what you described. Yeah.

CW (50:23):

Perfect. Huh.

EW (50:26):

One of the things you mentioned was trying to build this- I want to call them "information gaps," because that is how it was introduced to me. But these little parts where people do not quite understand, and they get curious, and it drives them to feel this need to get the next piece of information. It creates curiosity.

(50:48):

How do you build courses that have those? Is that the bottom up part of teaching? Or is that something else?

SS (51:03):

I do not claim that they have a recipe for building any course out there. I think one thing that Noam and I contributed in the field of computer science education, is exactly that point that you can teach a system, building bottom up, quite effectively. And yet, I think the top down is also very important. So the answer is that you need both.

(51:36):

You need students to understand both top down design, and bottom up design. But before "NAND to Tetris," everything was taught almost exclusively top down. One thing that we do- Well, first of all, there is students understand, and that is the contract. They understand that at the end of the game, they will have a fully functioning general purpose computer, that they built.

(52:07):

So they see the goal. They see it. This is kind of the holy grail. And they know that they will get to it, because senior students tell them, "Do not worry. You will get to it." Those students who did it already before them, or students who took it in Coursera and so on. So you know that you are going to get to the promised land. That is very important. You have this belief that it is going to happen.

(52:35):

Then we give them- We task them to build something, and they do not fully understand why they do it. Let me give you an example. The VM language. The VM language is kind of a stack machine language. You do not understand why you need push and pop, and why you implement "add" in this crazy way of popping two arguments, computing the sum, and pushing it back into the stack. Why do you need all this? You do not know.

(53:11):

By the way, you will will know exactly why you need it, when you write the compiler. Because the compiler generates VM code. And when you see your Java code being translated into the the VM language that you implemented two weeks ago, then you say, "Gee! Wow!"

(53:34):

The fact that they have a VM language, really allowed me to write a very elegant compiler. Because I do not have to go all the way to generating assembly code, which would have been a nightmare. And instead, I can take a Java operation like five plus three, and generate code that says push five, push three, add. I appreciate the beauty of doing this, and why I had to work hard on building the VM language before.

(54:08):

So knowing ahead of time that you are doing something which is very productive, and that leads you in the right direction, I think is psychologically important.

(54:19):

The other reason why I think students are engaged, is because there is something that you can call the- I think that everything that you do in science and engineering has both- If it is well done, it has both, I call it "internal validity" and "external validity."

(54:47):

The external validity is the fact that it has some practice, some purpose, in the world. It serves a purpose. You do not just build it as an intellectual exercise, you build it because it is needed somewhere. So that is external validity.

(55:08):

No less important, is internal validity. Internal validity is more elusive. It has to do with the beauty, elegance, consistency, fun and so on. So when we teach the VM language, we tell the students, "Look. You are implementing a machine. It is not completely clear yet why we need it, but it is a very nice machine. It has all these little operations which are very elegant, and have this, once again, internal consistency and beauty and elegance."

(55:42):

The students appreciate it, I think. So that is another way to motivate things, to argue and to show that they are simply fun to play with.

EW (55:56):

I have to say, I really enjoyed the book. I was not exactly the target audience, given that I am pretty far out of school at this point. But it was beautiful to see all of these things put together in a way that- I mean, I did some of the projects because it was engaging. I admit I did not do the operating system one, because it felt too close to what I was working on at work. But I really enjoyed your book and the course.

(56:32):

I guess I should also shout out to Nathan, who I think introduced me to it. He noted that, the authors, you and Noam, "Did a great job of taking the equivalent of four to eight college courses, and fitting them in 12 book chapters." Nathan almost feels like, "The book should be required reading for any computer scientist, embedded systems engineer, really anyone who works with processors of any size." And I totally agree with that.

(57:03):

Now, I have a whole bunch of questions about proteins-

(57:06):

<laugh>

(57:07):

And how much hydrophobicity matters in your program, because I think you need that in order to create membranes. And Christopher is looking at me like, I have lost the plot of this show.

(57:19):

Do you have any thoughts you would like to leave us with Shimon?

SS (57:24):

I guess it sounds very old school and old fashioned and conservative, but I believe in hard work. I think that if you want to know something deeply, you have to work hard to get to the depth of it. There are many good courses and many good resources out there, that help you to do that.

(57:52):

But do not expect shortcuts. Because if something is easy to learn, then you do not really distinguish yourself in any way. Then anyone can learn it. So what is in it for you?

(58:12):

And yet, if you are willing to commit yourself to working hard on, once again, getting to the bottom of a great idea, or a great piece of engineering, then you will gain a lot. You will gain- You feel much more empowered and accomplished, and it will boost your self-confidence. So do not be afraid [of] hard work. Hard work is good. That is one advice that I can think of right now.

EW (58:52):

Our guest has been Shimon Schocken, Professor at Reichman University, and Founding Dean of the Efi Arazi School of Computer Science. Check out nand2tetris.org. That is N A N D, two as in the number, and tetris.org. You will find the simulator and the book. Of course, we will have lots of links in the show notes.

CW (59:14):

Thanks, Shimon.

SS (59:15):

Thank you.

EW (59:17):

Thank you to Christopher for producing and co-hosting. Thank you to our Patreon listener Slack group for their many questions, which I did not get to. Thank you to Benny for the recommendation, and Uri for the introduction. And thank you to our show sponsor, Memfault. You can always contact us at show@embedded.fm or hit the contact link on embedded.fm.

(59:36):

Now, a quote to leave you with from Louis Pasteur. Let me tell you the secret that has led me to my goals. My strength lies solely in my tenacity."