When I read robotics books, they are either all math and derivation from first principles or a step-by-step how-to with no theory or flexibility at all. The beauty of the OpenCV computer vision tutorials is that they are very focused on application; even deep complex things like homography start with their application and then, if you need it, go to theory. I want a robotics book like that.

In my ideal world, this would be a book that describes how to use advanced robotics techniques in practical applications.

Overview

A big part of the goal of the book is to utilize the robotics software knowledge to overcome the shortcomings associated with inexpensive hardware. The example hardware will be unreliable and cheap with the code written in python suitable for a Jetson TX (or a Raspberry Pi for the early parts of the book). Note that in-depth discussion of electronics, including power management, is beyond the scope of the book; the focus is on software algorithms.

The book will not derive algorithms. The material will be presented so the reader can intuitively understand the algorithm with as little math as possible. The goal will be to show what the algorithms do and how to use them.

Note: this is not a “no math” book; that is not possible. However, I want it to have dramatically less math than the current literature. Also, the math should be introduced as though the reader had seen it in the past so it is a refresher with links to more in-depth review.

Every chapter would have:

Description of the problem being solved
Some new math (building upon previous chapters): trig, vectors, matrices, probability, signal processing (Fourier), convolutions (camera), neural nets
Algorithm descriptions with plenty of pictures
Code (in python, minimal necessary to implement the algorithm)
Notes on applying the solution in the real world (for example, methods to tune parameters and other tweaks)
Application example (path finding robot, pick and place machine, 3d printer, typing robot, self-driving car)
Further exploration: an exercise or two, likely thought problems. Also, there would be a bibliography for exploring the math and algorithm in more depth including books, search terms, and wiki pages

At the end of the book, the reader is not expected to be an expert roboticist or robotics researcher in any of the covered areas. Instead, the reader will have gained the knowledge they need to create or maintain a robot, focusing on the areas of their particular interest, product, application, or research.

Chapters

Many books seems to break robotics into perception, control, and learning. That seems like a reasonable method but I want to keep things concrete so I won’t keep this distinction if it isn’t necessary. To some extent, the chapter order will be driven by the math introductions (so matrix math is introduced after vectors but before neural nets).

Perception

How do I describe my robot?
- This chapter would be about constructing a model and understanding about how to think about your robot abstractly. The main algorithms would be kinematics and inverse kinematics. The associated math would be a trigonometry review, including 3d trig. The chapter would discuss how to describe poses for cars, wheeled robots, and arms and include a discussion of degrees of freedom.
What is near my robot? What is my robot about to run into?
- Making the best of bad sensors: Overview of probabilistic robotics, using Kalman and particle filters to deal with noise in sensors
- Using cameras instead of sensors: Computer vision (finding a sticker and line finding). CNNs, stereo vision and depth finding. Note radar and lidar as expensive options with more information and different types of errors.
Other sensing options (overview chapter of different options)
- Inertial, distance, odomenters, direction, encoder counts

Control

I want my robot to move over there, how do I do that?
- Controlling your robot: PID
My robot moved, where did it go?
- Finding your robot: relative localization with probabilistic robotics included

Learning

Where in the real world is my robot? How can I plan where it should go?
- Localization in the real world
- Understanding your larger environment: SLAM (online graphslam)
- Motion planning
How do I deal with a changing environment?
- Deep learning and neural nets?
How do I teach the robot what I want it to do?
- Behavioral cloning of neural nets
- Reinforcement learning

Conclusion

I finished Udacity’s AI for Robotics and Self-Driving Car Term 1 courses. I’m still working my way through Probabilistic Robotics. I feel like I’m learning a lot, certainly my brain often feels quite stuffed, but I’m having a lot of trouble applying these big concepts to my little robot. I want to apply the robotics, not simply learn the theory but there is some gap I can’t figure out.

Having sketched out an outline, I wondered if I could write this book. Realistically, no, I can’t write about concepts I don’t understand. One of the wonderful things about writing Making Embedded Systems was how deeply I knew and understood the material. I could present what I wanted newbies to know and I could hide some complexities with confidence. While I know many people write from a beginner mindframe, and that is very useful for them, I would rather write (and read) an expert’s perspective, ideally an expert who is also a good teacher.

This robot was not on the cover of the book I'm looking for. — This robot was **not** on the cover of the book I'm looking for.