Ruoso, Daniel Building Programming Language Interpreters

A bottom-up approach to runtimes, execution, and implementation in C++
1. Auflage 2026
ISBN: 978-1-83763-084-4
Verlag: De Gruyter
Format: EPUB
Kopierschutz: 0 - No protection

E-Book, Englisch, 372 Seiten

Designing a custom programming language can be the most effective way to solve certain types of problems-especially when precision, safety, or domain-specific expressiveness matters. This book guides you through the full process of designing and implementing your own programming language and interpreter, from language design to execution, using modern C++.
You'll start by exploring when and why building a domain-specific language is worth it, and how to design one to fit a specific problem domain. Along the way, you'll examine real-world interpreter architectures and see how their design decisions affect language behavior, capabilities, and runtime trade-offs.
The book then walks through the entire process of interpreter implementation: defining syntax, building a lexer and parser, designing an abstract syntax tree, generating executable instructions, and implementing a runtime. All examples are in modern C++, with a focus on clean architecture and real-world usability.
By the end, you'll have a fully working interpreter for a domain-specific language designed to handle network protocols-plus the knowledge and tools to design your own programming language from scratch.
*Email sign-up and proof of purchase required

Autoren/Hrsg.

Ruoso, Daniel, Daniel Ruoso

Fachgebiete

• Mathematik | Informatik EDV | Informatik Programmierung | Softwareentwicklung Software Engineering

Leseproben

1

Defining the Scope

In this chapter, we will focus on defining the target programming language and interpreter that we will develop throughout this book. We will cover the following aspects:

• Reflecting on the continuing desire to refine the programming language landscape with new ideas
• Exploring how domain-specific languages help solve targeted problems
• Understanding how building an interpreter can help to iterate on language design
• Defining the use case that will serve as our exercise through this book
• Discussing how this new language needs to interact with existing ecosystems

By the end of this chapter, you will have a new appreciation for the diversity of programming languages in our ecosystem. You will also be more knowledgeable about the project that will be developed throughout this book.

Free Benefits with Your Book Your purchase includes a free PDF copy of this book along with other exclusive benefits. Check the section in the Preface to unlock them instantly and maximize your learning experience.

Technical requirements

All the code for this book is available on GitHub: https://github.com/PacktPublishing/Building-Programming-Language-Interpreters.

Why do we keep creating new languages?

I suppose it is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail.

—Abraham Maslow, 1966, The Psychology of Science

We tend to underestimate how the programming languages we use define the way we write code and the way we handle the problems that are presented to us as software developers. The shape of our solutions is influenced by the tools we have when we’re working on them.

The environments and ecosystems of the programming languages we use shape us as developers. We can sometimes recognize the accent a developer has when they move from one programming language to another, and it takes a certain amount of effort and code review to become familiar with another language and to start .

While it’s true that you can solve any computable problem in any Turing-complete language, it’s also true that some problem spaces are better addressed by certain languages, and those tend to become used persistently, even when they’re no longer .

Fortran is a very good example of that. It has seen an important resurgence in the scientific computing community. It has straightforward semantics, very good support for numerical types, and a lot of efficiency when it comes to computing lots of numerical operations. Those characteristics make it extremely useful in that context, and it doesn’t matter that Fortran is quite literally the oldest programming language still in use.

The ecosystem also defines a lot regarding how problems are solved. C++ has a significant deficiency in how code reuse is managed, particularly across projects. It still has no uniform package management, even though it is probably the language that needs the most amount of support from outside the language for its build to work properly. Header-only libraries in C++ were developers’ response to this deficiency in the ecosystem, and it became a defining aspect of how libraries are made in the language.

Of course, sometimes we also just want to express aesthetic preferences on how we write our code. Programming is not a mathematical exercise; it is much closer to telling a story—a story you tell the computer, of course, but it’s also a story you tell other developers. And it’s easy to underestimate how strongly software developers feel about the choices that are made to solve programming problems.

Those choices are not always made based on empirical evidence—more often than not, they’re derived from the culture that is built around a given community. The Perl community, for instance, was heavily informed by the principle of the language itself. This could also be seen in how libraries were designed, with more flexible interfaces and the idea that the goal was to make it easier to solve problems.

This ended up becoming the fuel for a lot of the detraction from Perl, and opposition to it became a driving force in the design of Python, where

You may have read the previous paragraphs thinking that I would present you with a definitive way to talk about the most important aspects of programming language design and give you the answer to the question: But I’m going to go in quite the opposite direction.

Instead of trying to find the one ultimate language, our goal should be to use the programming language that best fits the specific problem we’re trying to solve. Sometimes, we may have strong opinions regarding that programming language, but if it allows us to solve the problem more easily (maybe there’s a library in that ecosystem that fits our problem), then we should prioritize solving the problem, independent of our opinions about that particular programming language.

Sometimes, disruption comes from unexpected places, and a language that takes a different approach will allow developers to create better solutions to existing problems. That can create a turning point and attract a lot of people to bootstrap an entirely new community. We’ve seen how fast Rust is gaining popularity, and how its memory management model changes the way people solve problems in that language.

Other times, disruptions occur without us having to replace any existing language. Focusing on niche use cases can create entirely new ways of solving problems. In the next section, we will focus on scenarios like that.

Domain-specific languages

Some use cases benefit from an entirely separate language focused on that particular problem. We call those domain-specific languages (DSLs). They solve specific types of problems that don’t fit well with the existing programming languages. They eliminate the need to express things that are not part of the problem and allow us to focus on the problem itself.

A great example of a language that is used a lot, but that most people wouldn’t consider using in situations other than the one it’s already used, is SQL. It is a language that focuses on expressing complicated relational algebra in a way that is intuitive and where you don’t have to specify unrelated operations to the problem you’re solving, meaning that there is less opportunity for bugs in the code.

A tool that is extremely powerful and that has become an integral feature of most programming languages is regular expressions. Specifying how to match a string against complex rules would be massively error prone if the developer didn’t have access to that specific language.

The two examples provided happen to be languages that often belong to entirely different paradigms than the one the developer is using to write the broader application code, and this is not a coincidence. The declarative nature of regular expressions makes them very different from the imperative nature of the language usually surrounding their use. And it’s that ability to easily go from being imperative to declarative that is the greatest power of DSLs. In , , I will discuss the various programming language paradigms.

Before jumping into the programming language that we’ll be designing in this book, it is important to take a step back and think about how our code will run.

Compilers and interpreters

Compilers and interpreters were roughly developed at the same time most programmers were expected to write native code for their target architecture. In general, a compiler will translate higher-level programming language code into native code that’s executed by the CPU. On the other hand, an interpreter will translate the programming language into a series of abstract operations that will be executed by the interpreter indirectly. Some problem spaces required minimal overhead, something that a direct translation to native code could provide, while other spaces could afford that overhead and prioritized the easier iteration between writing code and having it executed.

In , I will discuss how this distinction has become increasingly fragile over time. However, the principle still stands: when we make the distinction between compiled and interpreted languages, we still think in terms of whether the execution is done directly, by the native architecture, or whether there is a user-level abstraction that will execute the code in terms of abstract operations.

One of the biggest advantages of working with an interpreter is that you get to define all the characteristics of the runtime for your programming language. This allows for much quicker iteration when developing the programming language since you don’t need to worry about low-level machine behavior. And, more importantly, this does not prevent you...