Metalanguages and history.
We build a small compiler out of the standard components: a tokenizer, a parser, and a code generator. By the end, we successfully compile some code in our language, producing JavaScript output that we can execute. Most compilers are highly optimized for speed, but ours will be optimized for easy readability and understanding.
We build a self-contained data compressor and decompressor from scratch. The compression scheme is based on Huffman coding, which is used in gzip, zip, and many other compression formats. No prior knowledge of Huffman coding or data compression is necessary. The source code, including helper files not shown in the screencast, is available at https://github.com/garybernhardt/destroy-all-software-extras/tree/master/das-0102-data-compressor-from-scratch
We build a text editor from scratch. It includes basic text editing: moving the cursor, typing, backspacing, etc., as well as undo. However, we leave out some of the less-interesting bits, like saving edited text to files.
We build a shell in the style of the Bourne shell, Bash, zsh, etc. It supports (1) running commands with arbitrarily many arguments, (2) quoting those arguments, and (3) combining the commands into arbitrarily long pipelines where the output of one command becomes the input of the next. This requires writing a parser, which we do with Parslet, a PEG parser library.
We build an HTTP server that can serve static files from disk, as well as dynamic applications in the style of cgi-bin. The network server component is built using socket system calls (socket, setsockopt, bind, listen, and accept), rather than using the pre-made TCP servers available in the Ruby standard library.
We build the malloc() and free() functions from scratch, seeing how memory allocation actually works. This screencast is done entirely in C, but it's OK if you don't know C already; we'll introduce the necessary ideas as we go. (Nitpicky note: the comment in this screencast about pointers and arrays being "the same" will ruffle experienced C programmers' feathers. The C language does distinguish between arrays and pointers, so the comment is strictly wrong. However, thinking of arrays as being pointers will help new C programmers to understand what's actually happening: arrays are laid out directly in memory, with no extra metadata added, so the address of the array itself is also the address of the first element of the array. Simplifying this to "arrays are lies; they're really just pointers" is easier to grasp for someone who's new to C.)
The first step in our web framework is to handle web requests. We set up the basic project structure, then define a Sinatra-style routing DSL allowing us to route different request paths to different blocks of code.
We extend our simple request router to allow variables in the routes, like the variable "username" in "/users/:username". It extracts the variables' values from the requested path and feeds them to the handler block, as in most web frameworks. Regexes are the obvious implementation choice in a language like Ruby. We briefly look at the pitfalls in the regex approach, then implement with a much more straightforward method.
There are a few annoyances with the router that will crop up later. Rather than fixing them during screencasts about the database portion, we fix all three right now. This screencast is optional and doesn't show significant changes to the framework.
We discover SQL injection attacks!
Tests drive our SQL injection bug fix.
We make queries easier and safer.
A line-oriented template language.
The final piece of our web framework.
A simple router for static URLs.