Build systems for C Projects

Coming from the ecosystems of popular interpreted platforms like Python and NodeJS, the tooling around C projects felt very unfamiliar to me. Just like with the jLox, my priority was just to pick something that works so I could move on to actually solving problems. It seemed to me based on some interneting that there are at least 3 reasonable options:

  • make
  • cmake
  • meson

Once again following the “I’ve used it before” principle, I decided once again to go with Make. Make is a very manual build tool that feels only slightly more sophisticated than writing a bunch of bash scripts. That said I was able to come up with a setup that addressed all of my needs and I think works pretty well for a project of this size. This post will cover some useful Makefile concepts and flags for the C compiler.

Choosing a C compiler

Once again, we have a couple of choices. Working on a Linux system I narrowed it down to two choices:

  • gcc
  • clang

While gcc seems to be the default on Linux I used clang because the error messages seemed slightly better. I can only guarantee that I tested all the command line flags described in this post with clang but this information should apply to gcc as well.

Project Organization

Before I go into detail about the build process I landed on, let’s talk about the goals. Here’s the way I wanted the project laid out:

Goals:

  • compile binaries to bin
  • support building additional binaries from additional entry points in case you want to build toy programs that play around with pieces of the library
  • separate src/lib directory to keep entry points and lib files apart
  • compile debug binary with debug flags turned on
  • build and run lox with a single command
  • pass command line args to lox when running it through the build tool

The rest of this post explains how to set this up.

Compiling C programs with clang

Installing clang should be pretty straightforward if it’s not installed already.

From there all you need to do is point it at a src file like so:

clang src/main.c

This is going to output your binary to a file called… a.out. So obviously we’re gonna want to do something about that.

clang -o bin/lox src/main.c

That’s great until you need to start breaking up main.c into multiple files. Clang will not automatically compile anything just because you #include it. Now you’ve got something like:

clang -o bin/lox src/main.c src/vm.c src/compiler.c

This is the point where I no longer want to function without a task runner

Makefile tasks

You’ve probaby called make install to build and install an open source package before. It’s pretty ubiquitous. That said you don’t really need any prior experience to get started.

Writing your first task is as simple as this:

clox:
        clang -o bin/lox src/main.c src/vm.c src/compiler.c

Important note here: oftentimes when you are programming it does not matter whether you use tabs or spaces to indent. This is not the case for Makefiles. Make will blow up if you use spaces, you have to use tabs.

So now we’ve got a task that builds the program and you can just keep adding your modules one by one to this list if you want.

Task dependencies

Just like with gradle we can use task dependencies to ensure that prerequisite tasks run first. We’ll take advantage of that to add a run task that will make sure your build is up to date before runing lox.

clox:
        clang -o bin/lox src/main.c src/vm.c src/compiler.c
run: clox
        bin/lox

Clang include flags

At the point I want to move all of the additional lib files into a src/lib directory. From my perspective it makes sense to just pile everything at the top level of the project. Instructing the compiler to grab things from src/lib is not an issue. This will work just fine.

clox:
        clang -o bin/lox src/main.c src/lib/vm.c src/lib/compiler.c
run: clox
        bin/lox

The problem is that now you need to update all your includes.

/* #include "vm.h" */
#include "lib/vm.h"

Under normal circumstances this would also be fine. However this project has a weird constraint: you may want to copy code directly from the book unedited. Even if you are typing everything by hand for better memory retention, it’s a pretty useful troubleshooting tool to be able to keep copypasting to verify that you mistyped something. Fortunately clang supports a command line flag that adds directories to the include path so we can both put the files where we want and include them how we want. It could be argued this is a little hacky but this solution made the codebase the most comfortable for me to work in. Just place add an argument prefix the directory you want to add with -I and you can keep the includes in your C code exactly they way they’re written in the book.

clox:
        clang -Isrc/lib -o bin/lox src/main.c src/lib/vm.c src/lib/compiler.c
run: clox
        bin/lox

Clang Macro flags

A couple chapters in the author defines a macro that toggles logging of vm instructions as they get processed.

#define DEBUG_TRACE_EXECUTION

As the author states, this is only for debugging purposes, end users wouldn’t want all of this dumped into the console while they work. If you want you can just comment and uncomment this as necessary. A better solution is to use make tasks to create a different build. Clang makes this easy with define flags that let you define or undefine macros. Comment out the debug flags in your code, and pass the name of your macro prefixed with -D to clang when you invoke it in your make task.

clox:
        clang -Isrc/lib -o bin/lox src/main.c src/lib/vm.c src/lib/compiler.c
run: clox
        bin/lox
clox-debug:
        clang -DDEBUG_TRACE_EXECUTION -Isrc/lib -o bin/lox-debug src/main.c src/lib/vm.c src/lib/compiler.c
run-debug: clox-debug
        bin/lox-debug

Variables and wildcards in Makefiles

Listing out the library files is already pretty inconvenient, but now the list has been duplicated. Ideally we could automatically grab every file in src/lib and pass it to the compiler for every entry point. First let’s solve the duplication problem by adding a variable to the Makefile.

LIB=src/lib/vm.c src/lib/compiler.c
clox:
        clang -Isrc/lib -o bin/lox src/main.c $(LIB)
run: clox
        bin/lox
clox-debug:
        clang -DDEBUG_TRACE_EXECUTION -Isrc/lib -o bin/lox-debug src/main.c $(LIB)
run-debug: clox-debug
        bin/lox-debug

This is already a pretty big improvement. Let’s move some more of the arguments to clang into variables.

APP=src/main.c
LIB=src/lib/vm.c src/lib/compiler.c
FLAGS=-Isrc/lib

DEBUG_FLAGS=-DDEBUG_TRACE_EXECUTION

OUTPUT=bin/lox
OUTPUT_DEBUG=bin/lox

clox:
        clang -o $(OUTPUT) $(FLAGS) $(LIB) $(APP)
run: clox
        $(OUTPUT)
clox-debug:
        clang -o $(OUTPUT_DEBUG) $(DEBUG_FLAGS) $(FLAGS) $(LIB) $(APP)
run-debug: clox-debug
        $(OUTPUT_DEBUG)

This is in my opinion much cleaner and easier to configure. Now use a wildcard so we can finally avoid listing out all the library files.

APP=src/main.c
LIB=$(wildcard src/lib/*.c)
FLAGS=-Isrc/lib

DEBUG_FLAGS=-DDEBUG_TRACE_EXECUTION

OUTPUT=bin/lox
OUTPUT_DEBUG=bin/lox

clox:
        clang -o $(OUTPUT) $(FLAGS) $(LIB) $(APP)
run: clox
        $(OUTPUT)
clox-debug:
        clang -o $(OUTPUT_DEBUG) $(DEBUG_FLAGS) $(FLAGS) $(LIB) $(APP)
run-debug: clox-debug
        $(OUTPUT_DEBUG)

Passing through command line arguments

The lox binary can take a filename as a command line argument. We’ll need to update the Makefile a bit so that we can pass arguments to lox when we invoke it with make. We can define define a new variable at the top of the file called ARGS.

ARGS ?=

APP=src/main.c
LIB=$(wildcard src/lib/*.c)
FLAGS=-Isrc/lib

DEBUG_FLAGS=-DDEBUG_TRACE_EXECUTION

OUTPUT=bin/lox
OUTPUT_DEBUG=bin/lox

clox:
        clang -o $(OUTPUT) $(FLAGS) $(LIB) $(APP)
run: clox
        $(OUTPUT) $(ARGS)
clox-debug:
        clang -o $(OUTPUT_DEBUG) $(DEBUG_FLAGS) $(FLAGS) $(LIB) $(APP)
run-debug: clox-debug
        $(OUTPUT_DEBUG) $(ARGS)

The ?= in ARGS ?= allows conditional assignment, so it will default to vars passed on the command line if they are available. Leaving the right side blank assigns an empty string. Now you can invoke the run task with an argument like this:

make run ARGS=demo_program.lox

Conclusion

Now we’ve got a pretty functional setup. From here adding new tasks for tests etc. is pretty straightforward, and so is adding new flags.