Rust 2018: Libraries in the 2018 Epoch
The call for community blog posts has inspired lots of great responses. Most of the things that I feel are important have already been said. This post is a collection of thoughts on what I feel is important specifically for Rust to flourish in the future. It is perhaps a bit late in the process, but it may add some new perspectives.
Addendum 2018-02-01: When I wrote this post, I thought that the epoch would come after 2018, not in the middle. Given the much shorter time frame, the official plan to postpone all changes) to generic types (generic associated types, specialization, and const generics) until after the epoch is very reasonable.
How should we prioritize?
If we are to set up a list of goals to do, this means that we need to prioritize what is important. As a natural consequence of this, we also need to decide what not to do. A to do list that includes everything is meaningless, since it gives us no guidance at all.
To effectively prioritize, there needs to be a goal that we want to reach. In my view, there are two things that we should consider in order to define the goal for the year to come.
The epochalypse cometh…
It is planned to introduce a new epoch for Rust in 2018. According to the RFC, “the [epoch] concept is primarily about putting together a coherent package.” The phrasing indicates that the idea is to deliver a set of features that play well togheter for the programmer to express themselves. That there is to be an epoch in 2018 sets the time-frame for when these features need to be available.
Libraries, libraries, libraries, libraries, …
A language may be the best possible language for writing a program in, but the amount of code any one developer or team is capable of writing is limited. To produce something larger, relying on libraries is a must.
The libraries available for a language defines the experience of writing production code in that language. Rust has, through the crate eco-system, a very good infrastructure for accessing and using existing libraries. Many of these libraries are of very good quality, and some are real joys to use. Expanding the number of high-quality libraries is of course important, but more important is to consider what kind of libraries are possible to write.
What to prioritize
Given that writing libraries in the 2018 epoch of Rust is a worthy goal, what would this goal mean more concretely?
What library…?
This post has mentioned libraries several times, but it is important to discuss what kinds of libraries are meant. There are several types of libraries to consider, and they may have very different requirements for language evolution. I’ve tried to divide it into three groups: utilities, foundational, and applications. The division is of course arbitrary, but may be useful for discussion.
Utility libraries are libraries that solve some specfic recurring task that is not the main focus of the application. Utility libraries are the bread and butter of libraries, and they are used in very many contexts.
Foundational (or extension) libraries are libraries that either significantly raises the level of abstraction for the code, or that more or less enable the user to do new things with the language. This category of libraries are the things that takes a language into new domains of usage. Often, these libraries may have a large effect on how the rest of the code is written.
An application library is something that fits with the specific needs of a certain application. These kinds of libraries may in some cases be frameworks, in that the whole application is structured around them. Sometimes, they onyl affect part of the code.
To make the categories more concrete, the following list shows some libraries in the three categories and short notes on what they are used for. The main focus is on libraries that are or may become very popular/important for lots of applications. The list is of course in no way a complete list, or intended as a list of the best possible example for the intended usage.
- Utility libraries that solve specific recurring tasks.
- Foundational libraries that extend what you can do with the language
- Application libraries that are or may become ubiquitous
The point here is that these libraries (and many many others) are the kind of libraries that enable a Rust programmer to write large and interesting applications. To find out what to prioritize, it is important to look for what such libraries need from the language.
Specific examples on things to prioritize
I’m not claiming here to have made any serious or thourough investigation into exactly what the above libraries need specifically. The following is a short list that I think captures many of the for libraries transformative changes that are coming to Rust. That the list is “short” in no way means that it represents little work, or even that it is feasible to make all this happen now.
Macros 2.0
Rust has a strong tradition of using macros to solve code generation issues. With custom derive in 1.16, the most pressing issues for macros were solved, but it is of couse not the end.
Looking at an example such as Rocket, unstable macros allow a new level of readable, simple code that focuses on the task at hand. Whan I’ve showed Rocket-style code to non-Rust users, they’ve been amazed at the concise but still very readable code for defining safe and fast web services. I think that Macros 2.0 will enable some really interesting new library designs for Rust.
Another thing that is planned for Macros 2.0 that I think is very important is scoping. The more macros become an important part of library design, the more important it becomes to have scoping rules to handle the inevtiable name clashes that will occur.
Improved support for generic types
Generic types (and the monomorphization that is typically used) is very important for making Rust code that is both safe and fast.
Generic associated types, specialization, and const generics are all extensions to the base Rust model that will enable new and exciting possibilities for designing libraries. All three are large, complicated, and intricate problems to solve for a sound and usable design and for making it possible to implement corectly. Solving these issues will require a lot of effort.
Asynchronous code
Asynchronous code is needed for very many applications, that much is
clear. The initial versions of Futures and Tokio show the promise
of writing asynchronous code in practice in Rust. However, many people
have remarked that it is still quite cumbersome, so more work is
needed. The work that is being done with a new design for Tokio, on using
generators for implementing Futures, and being able to use async and
await as keywords are all very important steps for enabling a strong
asynchronous story for Rust, and this needs to continue.
One thing that I think might be important is to make async and
await reserved keywords in the 2018 epoch, even if they are not yet
finalized and stable. The names and
model
have become the standard way for writing asynchronous code. Even if
ultimately Rust would decide on another approach for the same problem,
allowing async and await as identifiers is quite likely rather
confusing.
Tools for documentation
The reported experience from Diesel in the road to 1.0 was that documenting all the code gave very many insights into code that needed new structure because it was hard to explain. Helping library authors to succesfully document their code in a way that enables a user to understand the library is beneficial not only for usage, but also for designing that library in the right way from the start.
Interfacing with other languages
Writing everything in Rust is not a viable strategy, for some things interfacing with pre-existing code is a must. A great example for this is gtk-rs, which makes it possible to write a GUI in Rust.
While bindgen is a really good start, I think there may be more possibilities for making it even easier to create high-quality interfaces to external code in Rust.
Things that I’m willing to not prioritize
As said previosuly, an important part of prioritization is a willingness to not make some things that we may all want a top priority. If, as argued above, the focus is on laying the groundwork for having really excellent libraries for the Rust 2018 epoch, many features that are focused on other qualities should not be the top priority. So what kind of things would not be a top priority?
- Documentation
- Compilation speed
- IDE integrations
- Cross compilation and WebAssembly
- Speed features such as inline assembler and SIMD
- Your favourite absolutely neccesary super-duper-critical feature!
All of these are of course critically important. I need documentation to be able to work, I hate to wait on the compiler to finish, and I always use an IDE for coding. Cross compiling to all the platforms makes Rust a strong contender in many new areas, and access to low-level features such as assembly and SIMD gives Rust an edge in speed.
What I’m trying to say is that not having these as the top priority does not mean that I think that work on that should stop or that it is unimportant. It just means that I don’t think they should get prioritized above features that will enable the best possible libraries for Rust in 2018.
About me and Rust
While you can read more about me on the about page, I thought I’d mention some of my Rust background. I
started following the Rust project loosely around the time it was
first announced, back when
there was still discussion on using typestate to encode properties. I
never wrote any Rust until much later though, around the time 1.0 was
released. When 1.13 was released I started actually learning and
writing Rust more seriously. The main reason was that the ? operator
was stabilized, enabling me to write code that handles errors sanely
(especially when compared to Go). When 1.16 was released and both
Serde and Diesel started working on stable, I felt that the future was
bright for Rust and started to use Rust even more.