Skip to content

Practice of Dev Tool Version Control

This blog focuses on some practices to manage development dependencies, not library dependencies. Hence, it's not a blog to introduce how Go, Cpp or Rust compiler resolves the libraries to build or the linker finds the dll for linking.

Instead, it talks about how do we manage the tools during development. For example, when using protoc along with its several plugins, how to ensure all CLI tools are used as desired everywhere?

Why We Need to Manage the Dev Tools?

Go has a quite simple version management mechanism to tackle the dependencies a module used, so building is easy because go hides a great details for you. Usually, how to build is not a trouble, comparing to cpp due to some dependencies missing(cmake is designated to solve such kinds of issue).

During development, usually we need to trigger outside cli tools, such as:

  • golang ci lint to lint the project
  • protoc to generate the code
  • cli tools to publish the proto into platform
  • push some dependencies(for example, manually build and push a docker image instead of using ci).
  • etc...

All of these operations require you install the cli tools, via brew install, go install or whatever package managers. But the problems are:

  • installing all desired versions everywhere is not easy, except you wrote a shell to do so.
  • the tools are installed globally, where will sometimes be overridden unexpected.
  • your tool got changed by another installed tool
  • PATH change might override your existing environment
  • projects requires different versions, you need to switch them frequently.

To solve the problem above, the dependency management tool is helpful.

Brief

Before we talks about some famous open source tools, you need to understand how shell finds a command for you. Shell has many standards with implementations, but essentially it will try to search a command from:

  • built-in functions
  • shell aliases and functions
  • PATH(specifies a set of directories where executable programs are located)

Usually, the shell searches from the PATH and return the position of a command. It also implies the PATH sometimes may not be reliable as the time goes by. Hence, here are some popular approaches:

  • Nix: clear all env and set it with config file
  • npx: provide command to run commands with desired dependencies installed in node_modules.

In a company scope, we utilize the 2nd idea and use the concrete paths to access command, via a wrapper command to help us.

Besides that, some commands which wrap several different commands in different cli tools use this mechanism as well. It resolves the versions internally and access the command by absolute path as well.

Practice in a Company Scope

Company scope cli usually integrates several concrete tools of each component, and is delivered to the users along with a framework. For such kind of cli, it takes cares about:

  • deterministic versions for all underlying tools
  • ability to ask users to use the latest versions provided by the other teams
  • install dependency tools only when it's needed by a command
  • self-upgrading

The configuration file provides the option for users to set up the tools they desired to use in project/global level. Besides, a remote data source which maintains all latest tools are introduced, it takes as a fallback when users don't specify versions.

The precedence is usually project->global->remote, but during practice we found project->remote level is enough and reduces a lot of troubles caused by global configuration file.

This requires the CLI maintainer to regularly upgrade the centralised data source for all users, and it reduces all the other developers efforts.

Last, as an internal cli tool, self upgrading is great. This is done by introduce the ^ syntax for specifying the cli tool to download the latest, and use symbol link to ensure users always use the latest release.