On the Way to CMake Tools 1.0
To skip the mish-mash and history lesson, skip to the 0.11.0 Beta Release section.
My first commits to CMake Tools took place in April of 2016. That’s over a year and a half! I didn’t think it had been that long: It feels like just a few months ago!
I remember being excited to see just twenty downloads on the VSCode extension marketplace. It is now approaching two-hundred thousand (!).
Looking at the initial commit, you can see some semblance of what the extension does today:
- Build
- Configure
- Run CTest
- Clean
- Set your build type
The code is crude and painful to look at, but shows how it has come a long way.
What was I Thinking?
Before switching to VSCode full-time, I used Vim almost exclusively. C++, Python, JavaScript, and more were all edited in Vim.
Why would I switch? Vim has a lot of merits, the top of which being speed. Starting up Vim takes a matter of two or three seconds, at most.
And… that’s really it.
Sure, I could rattle on about how fast I can edit text in Vim, and it would be true: I can move lines and snip code and jump between files at blazing speed.
But that’s not what I need to do when I write code.
I want an editor that has an understanding of what I’m trying to do. Something with smarts, where I can search the codebase for symbols. Find usages. Highlight errors. Rename symbol. Yeah, you can use Vim plugins to do these things, and I have a fair share of them installed on my machines, but none of them will even scratch the surface of what can be done in an editor designed from the beginning to be pluggable.
Which VSCode is.
(And it works perfectly between Windows and Linux.)
Seeing coworkers using editors like QtCreator or full Visual Studio, I wanted some of the things they had, minus the bloat of the rest of the editor. Something VSCode lacked was the ability to work with CMake projects out-of-the-box. The task execution functionality was insufficient (and, to the best of my knowledge, still is). I wanted to be able to push a single button and have CMake configure and build my project, and I wanted to see the results in an output window. I wanted to see my current build configuration and project name in the status bar of the editor. I wanted to be able to launch a target in the debugger by name, and not by path.
These were all things that existed in real IDEs.
But VSCode is pluggable! And so, I set out to build this simple set of tools. That is why it is called CMake Tools. It was originally just a bunch of small commands and status bar items to help work with CMake projects. It’s somewhat ballooned to do a lot more than just “tool”-ish things, and even steps into some non-CMake stuff in a few places.
(Looking back, a better name might have been “CMake Projects” or something. Oh well.)
A Tale of Two Extensions
Before going further, I should talk about a fundamental architectural oddity in CMake Tools. And before I do that, I have to talk about a special feature of CMake itself, and before I do that, I must talk about how CMake Tools started, and how we talk to CMake.
Talking to CMake: The Bad Way
CMake, for the longest time, did not provide any way to introspect itself. The best you could do was emit a project file using a kind of “secondary generator.” This would emit both a build file, such as a Ninja file or Makefile and a project file for an IDE.
CMake supports a few different formats, and all have upsides and downsides, but none really provide a heavy amount of introspection that an arbitrary IDE would want.
For a long time, this was how QtCreator worked with CMake: It would generate a project for CodeBlocks, a completely different IDE, and use that to render the project view and see the targets.
In CMake Tools, for no particular reason, I opted to not use this approach. I didn’t really need the full project data (at first), I just needed to build targets and configure things. So I went the old fashioned way and just invoked a CMake command line when you wanted to talk to it.
Talking to CMake: The New Way
With CMake 3.7.2 (technically 3.7.0, but don’t think about that), CMake introduces CMake Server, which is a micro-service style program that lets other tools control and communicate with CMake with a nice JSON API over an IPC channel.
Both Visual Studio and QtCreator now use CMake Server mode, rather than generating intermediate files.
I was quick to adopt CMake Server in CMake Tools, including filing some bugs upstream when CMake Tools would break CMake Server.
But I have a problem:
A moment of silence for those working in 2.8.11 and older
I’m really happy to see nearly 60% of users in the bleeding edge, but that still leaves a lot of people lagging far behind. I can’t just drop the support for CMake older than 3.7.2: I have a lot of userbase there, and I still get bug reports from CMake Server changes breaking things.
¿Porque No los Dos?
So, I went another route and decided that I would have what amounted to two different backends to the extension, with a thin wrapper over the top that VSCode would call into. It actually works pretty well but leads to quite a bit of unnecessary complication.
What am I Thinking?
I’m still not quite to the point I wanted with the extension, and maybe never will be as long as other IDEs continue to improve.
The desire to do a big final rewrite came with the release of the Custom Explorer API in VSCode 1.10.0 in May of 2017. I had long wanted to replicate the project view present in both Visual Studio and QtCreator, where (instead of a filesystem view) the project was shown with its build targets laid out in a list, where each target sat at the root with its constituent source files organized underneath in the tree. (There is an existing feature request for this as well, which I hope to someday fulfill).
But, when I started trying to shoe-horn this feature into the codebase, I found it difficult. The recent release of CMake-Server would make the feature easy to implement, but I found that the way I was loading and starting/stopping the daemon was getting in the way.
It got worse: When I tried to move the daemon code elsewhere, I found it couldn’t be untangled from the absolute mess of the legacy code.
I also hated having this wrapper over everything. It made everything a chore.
I noted that almost all of the extension’s actual work was being done in a common base class for the two backends: Most of the time it didn’t matter if I was using CMake Server or not.
So I set out on a rewrite: Untangle the backends, fix outstanding bugs, accidentally introduce some new ones, and save the princess.
That was three months ago.
With a recent change in responsibility at my “real life” job, I found myself writing native code less and less, therefore using CMake less and less. My personal motivations to fix bugs and add features was falling but never disappearing. I got emails almost daily of GitHub Issue Notifications as people opened new bugs and feature requests and commented on old ones, and each email was like a little papercut of a reminder that I have to get this done.
It’s been slow going, with a lot of distractions and other projects taking time away from getting this refactor done. I hoped to finish this at the end of October. Now it’s December.
A 0.11.0 Beta Release
Fortunately enough, in the middle of me writing this post, VSCode 1.19.0 was released. This release includes new support for special version numbers including alpha, beta, and release-candidate versioning.
This nicely coincides with my first beta release of the 0.11.0 CMake Tools version.
Why the Ceremony?
0.11.0 is the product of my rewrite of the past few months. Not a single line from the previous version remains untouched. While a lot of code was directly copied from prior versions, the overarching design was changed significantly.
Therein lies a problem: I’m just one guy with a single Linux box and a single Windows laptop. I can’t anticipate all the potential user-facing issues that will arise in the wild.
I feel confident that the code is correct, but not confident enough to just push out the changes and possibly break every single user’s workflow.
So…
I Need Your Help!
0.11.x will be the final 0.x.x version. There will be no 0.12.0. We will be moving into a stable 1.0.0 after 0.11.x.
The codebase of 0.11.0 will be mostly unchanged on the way to 1.0, so I need to make sure it’s correct and stable.
If you’re reading this and feeling adventurous, please download and install the beta release here. File new bugs and enhancement requests against the beta. Add feedback to this issue.
What’s Changed?
Besides an overhaul of the internal design, the 0.11.0 release includes quite a bit of new features regarding the way we talk about “build environments.” The concept itself has been renamed to “kits” to match the name of a somewhat similar concept from QtCreator.
Kits
Kits are a new way to encapsulate the toolset used to build a native program. Previous versions included support for “build environments” when working with Visual Studio. The new version now considers things like GCC and Clang compilers and includes built-in functionality for discovering and modifying kits.
Error Reporting
CMake Tools now includes an OPT-IN capability to automatically send anonymous error and exception information to Rollbar.
Emphasis on the opt-in part. You do not need to participate, although it would be hugely helpful.
Any time an exception or unhandled error reaches the root of the extension runtime, an event will be sent to Rollbar with useful information that can help me debug and discover issues without even requiring a GitHub issue to be opened (But please still open them!).
CMake Tools will ask you for permission to send error data once after the extension has been installed.
DOCUMENTATION!!
Something CMake Tools has always lacked has been authoritative and nice-to-read end-user documentation. A few markdown files sitting in the git repository isn’t going to cut it anymore.
Proper documentation is going to be available on this site. The documentation is generated via Sphinx using restructuredText files versioned with the rest of the code. It will include annotated images, step-by-step usage instructions, feature references, and more.
Currently, it’s a pre-alpha documentation. It’s not fully filled out, but should see completion before the 0.11.0 release proper.
Develop documentation is also available, linked to from the main page of the user documentation. It still needs some TLC as well.
API Changes
Not applicable to many users, but CMake Tools itself has featured an API for many months. Very few users have made any use of it, and most of it is just a copy-paste of the extension manifest into a TypeScript interface file.
The API as it exists currently will be phased-out and replaces with a newer API with much fewer bells and whistles, but proper versioning and sanity.
GitHub Clean-up
Not specific to the 0.11.0 release, but I’ve done some maintenance on the GitHub project. There is now a proper template for issues and pull requests, a real license (MIT), and I intend to more closely use release and issue tracking to its fullest. These changes are already applied since they are low-risk and there isn’t actually a way to “beta test” them.
Bugfixes?
Sorry to say, but a lot of the open issues against the release version haven’t been addressed in the first beta. I’m hoping to do some proper bugfixes in the coming weeks and apply them to the 0.11.0 release.
Deprecations and Removals
Here comes the bad news: Some of the functionality relating to legacy CMake versions has been removed.
I know I said I want to support legacy users, but there’s a tradeoff: Some of the things I did to introspect old CMake versions were so ugly and horrible and fragile that I couldn’t bring myself to port them to the new version. Things like target listing and target debugging aren’t available if you are using CMake before 3.7.2.
I’m anticipating a high user demand, and it’s probable that I’ll revive the old sadness in the next 0.11.0 beta if there is enough demand to bring it back. The new architecture will make it easier to hide away the fragility of the old methods and keep the rest of the codebase pure and clean.
And to anyone using an older CMake version: I beg and plead with you to try upgrading. It’s a lot less painful than you may think. If you can’t upgrade for technical reasons and really know what those technical reasons are, I understand your pain.
When Will It Be Finished?
0.11.0 proper will be released when I feel like I’ve stabilized and reached parity with the prior versions (barring deprecations and feature removals).
Some Final Remarks
I want to thank everyone who has used and contributed to the extension. If you’ve ever opened an issue or pull request or commented or reacted on any thereof, you have been a real help and motivation for me to make this project as useful as I can.
If you’ve never opened an issue or PR, I invite you to participate! Even something as simple as leaving a thumbs-up reaction to an issue comment helps me see what matters to users.
And here I’ll re-paste the links to the GitHub release and the new end-user documentation.
Additionally, there is a pertinent GitHub issue for adding general commentary and feedback.
Finally: If you’ve read this far, thanks. It means a lot.