Raul B.

As we saw in the Vagrant - installing and operating post, Vagrant provides a convenient way for automating the setup, configuration and management of virtual machines, enabling reproducible and consistent development environments.

In that post, we also had an overview of how to create a Vagrantbox from scratch for the Parallels provider. We then digged further the process of creating Vagrant boxes from scratch in the Create Oracle Linux 10 aarch64 Vagrantbox for UTM post, where we generated a full featured Vagrant box for ARM64 using the UTM provider.

The missing part to complete your Vagrant skill sets is the creation of Vagrant Boxes Repositories, grouping them into a Vagratn Box Catalog and publish them on an HTTP server, so to make them available to other users.

Vagrant Box Catalog And Vagrant Box Repository Tutorial explains how to pack mutliple Vagrant boxes into a box repositotory and publish it on an HTTP server.

As explained in the Create Oracle Linux 10 aarch64 Vagrantbox for UTM post, a Vagrant Box is just a bzipped2 tarball archive used to ship a single VM with a certain operating system which

• has a specific major version
• runs on a particular hardware architecture
• supports a certain Vagrant provider

This means that not only to run the same operating system of the same major version with a different hardware it is necessary to create another Vagrant box, but also that is necessary to create additional Vagrant boxes whenever you need to support different Vagrant providers.

In addition to that, multiple version of the same Vagrant box may be produced, shipping  different patchlevels of the same contents.

As you can infer, this leads to the spreading of the number of boxes after some time.

The solution to this problem is storing the boxes in repositories, which can be easily located and accessed by the user when performing vagrant box install and vagrant box update operations.

A Vagrant box repository provides the entry point to locate every version of the same Vagrant box.

In other words, a Vagrant box repository contains a set of logically related Vagrant boxes, through which:

• provides all the available version of a specific operating system major version
• for every supported hardware architecture
• for all the available Vagrant providers

If you need to provide Vagrant boxes for different operating systems, including different major versions, you must create a new Vagrant box repository specifically for it.

A Box Repository is nothing but a directory tree containing its own metadata.json file describing each single box in the repository.

More specifically, for for each single box, it contains:

• the Vagrant box version
• information about the location from where to retrieve the Vagrant box file,
• the supported hardware architecture
• the Vagrant provider supported
• the Vagrant box's SHA256 checksum

Vagrant Boxes Catalog

The best practice is to group Vagrant Box Repositories into a Vagrant Box Catalog: this provides a convenient entry point to locate every Vagrant box repository, which can even be easily shared for example publishing it using an HTTP server.

For this purpose, let's create the catalog directory tree where to store the Vagrant Box repositories alltogether:

mkdir -m 755 ~/vagrant-catalog ~/vagrant-catalog/catalog
cd ~/vagrant-catalog/catalog

Vagrant Box Repository

To enable different packagers to provide Vagrant box repositories avoiding naming collisions, the best practice is to use the packager/boxrepository structure.

In our scenario, we use:

• grimoire as the packager
• ol10 as the box repositoy, which provides only Oracle Linux 10 boxes

So, let's create the grimoire directory tree as follows:

mkdir -m 755 grimoire

Then, beneath it, let's create the ol10 box repository directory:

mkdir -m 755 grimoire/ol10

we can now copy the Vagrant box we created in the Create Oracle Linux 10 aarch64 Vagrantbox for UTM post to grimoire/ol10 box repository:

cp ~/ol10-1.0.0-arm64-utm.box grimoire/ol10

Since the Vagrant box files are often quite heavy, and since the Vagrant Box Catalog containing the Vagrant Box Repository is typically HTTP published, to improve reliability, each single Vagrant Box entry in the metadata.json must also provide the SHA-256 checksum.

We can calculate it by running:

shasum -a 256 grimoire/ol10/ol10-1.0.0-arm64-utm.box

in my case, the output is:

fed838374269630ce9cca199b4ab8daac22df59f8263d167fbca41eaee746db7  grimoire/ol10/ol10-1.0.0-arm64-utm.box

If shasum is not available on your system, you can generate the sha256 sum by running openssl sha256 grimoire/ol10/ol10-1.0.0-arm64-utm.box.

We have now all the necessary information to create the initial repository box's metadata file - Just create the grimoire/ol10/metadata.json file with the following contents:

{
  "name": "grimoire/ol10",
  "description": "This box contains Oracle Linux 10.",
  "versions": [
    {
      "version": "1.0.0",
      "providers": [
        {
          "name": "utm",
          "url": "grimoire/ol10/ol10-1.0.0-arm64-utm.box",
          "checksum": "fed838374269630ce9cca199b4ab8daac22df59f8263d167fbca41eaee746db7",
          "checksum_type": "sha256",
          "architecture": "arm64",
          "default_architecture": false
        }
      ]
    }
  ]
}

In this example, since the url field does not specify any protocol, Vagrant retrieves the file from the local filesystem, using the Vagrant Box Repository path as the base path.

If you already installed this box while running the examples in the Create Oracle Linux 10 aarch64 Vagrantbox for UTM post, remove it by running:

vagrant box remove --provider utm grimoire/ol10

We can now install the box by running:

vagrant box add --provider utm grimoire/ol10/metadata.json

the output should look like as follows:

==> box: Loading metadata for box 'grimoire/ol10/metadata.json'
    box: URL: file:///Users/mcarcano/boxes/grimoire/ol10/metadata.json
==> box: Adding box 'grimoire/ol10' (v1.0.0) for provider: utm (arm64)
    box: Downloading: grimoire/ol10/ol10-1.0.0-arm64-utm.box
    box: Calculating and comparing box checksum...
==> box: Successfully added box 'grimoire/ol10' (v1.0.0) for 'utm (arm64)'!

While creatingVagrant boxes and publishing them in aVagrant box repository is an excellent lab exercise, enterprise environments require secure, automated artifact management. If you want to systematically fill your DevOps and DevSecOps knowledge gaps by reading a crash-course like book full of hands-on enterprise exercises, jump directly to the Apress Blueprint Box below to discover how to boost and evolve your career using a self-paced learning path.

HTTP Publishing The Vagrant Box Catalog

The straightforward limitation of implementing Vagrant Box repositories and a Vagrant Box Catalog on your local filesystem is that they remain accessible only to you. It's far more effective to publish them over HTTP, allowing other users to benefit from them. This approach promotes corporate-wide standard images, prevents teams from reinventing the wheel multiple times, and fosters collaboration and efficiency across your organization.

There are several options for achieving this, ranging from simple setups to more advanced tools:

• Simple publishing: Host the Vagrant Box Catalog directory tree using a standard web server, such as Apache HTTPd. This is a straightforward way to make your boxes available via HTTP without much overhead.
• Application-specific repository managers: Leverage tools like Pulp3 (with the pulp-vagrant plugin) for robust artifact management, or source code management platforms like Gitea, which natively support publishing Vagrant Box artifacts.
• Official or cloud-based solutions: For a more polished experience, consider Atlas (now integrated into HCP Packer) or, if you prefer a cloud-hosted option, Vagrant Cloud. These provide built-in features for versioning, sharing, and discovery, making them ideal for larger or distributed teams.

Here we implement a solutions which is just a lab to learn the mechanisms used under the hood by a Vagrant Box Catalog: for production use, consider one of the aforementioned solutions.

To see how a Vagrant Box Catalog in action, we can spin up a disposable Apache HTTPd container - using a container allows us to experiment and make sense of the process, thus avoiding to mess up our system with an unnecessary Apache HTTPd instance.

To operate more comfortably, let's change to our users' home directory:

cd ~

Let's start from creating the ~/vagrant-catalog/conf directory where to store the Apache HTTPd's configuration file:

mkdir -m 755 ~/vagrant-catalog/conf

Then, we must create a minimalist Apache httpd.conf file - we must include only the very minimum features to support our use case.

Create the ~/vagrant-catalog/conf/httpd.conf with the following contents:

ServerRoot "/usr/local/apache2"
Listen 80

# Minimal modules for static content and .htaccess support
LoadModule mpm_event_module modules/mod_mpm_event.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule access_compat_module modules/mod_access_compat.so
LoadModule mime_module modules/mod_mime.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule env_module modules/mod_env.so
LoadModule headers_module modules/mod_headers.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule unixd_module modules/mod_unixd.so
LoadModule dir_module modules/mod_dir.so
LoadModule autoindex_module modules/mod_autoindex.so

# User/Group for Alpine
<IfModule unixd_module>
    User www-data
    Group www-data
</IfModule>

ServerAdmin www-admins@carcano.corp

# Deny access to root
<Directory />
    AllowOverride none
    Require all denied
</Directory>

DocumentRoot "/usr/local/apache2/htdocs"

# Enable .htaccess and disable auto-indexing for static content
<Directory "/usr/local/apache2/htdocs">
    Options -Indexes
    AllowOverride All
    Require all granted
</Directory>

# Basic logging (to stdout/stderr for Docker)
ErrorLog /proc/self/fd/2
LogLevel warn
<IfModule log_config_module>
    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
    CustomLog /proc/self/fd/1 common
</IfModule>

# Basic MIME types
<IfModule mime_module>
    TypesConfig conf/mime.types
</IfModule>

If you are already used to work with Apache HTTPd, you can blame this is not the real bare minimal configuration for running Apache HTTPd: we indeed need to enable a few additional modules to support a few options we are about to configure using an .htaccess file.

We can then spin up the Apache HTTPd container by running:

docker run -d --rm --name vagrant-catalog -p 8080:80 \
-v ~/vagrant-catalog/conf/httpd.conf:/usr/local/apache2/conf/httpd.conf \
-v ~/vagrant-catalog/catalog:/usr/local/apache2/htdocs httpd:alpine

The above statement spins-up an Apache HTTPd instance, mapping the ~/vagrant-catalog/catalog directory to the httpd root directory and mounting the ~/vagrant-catalog/conf/httpd.conf file we just created to the path where the Apache HTTPd in the container looks for its configuration. The Apache instance endpoint is also bound to the 8080 port on localhost.

Lastly, it is necessary to add the .htaccess file to the ~/vagrant-catalog/catalog directory - just create the ~/vagrant-catalog/catalog/.htaccess file  with the following contents:

Header unset Pragma
FileETag None
Header unset ETag
DirectoryIndex metadata.json
IndexIgnore *
Header set Cache-Control "max-age=0, no-cache, no-store, must-revalidate, private"
Header set Pragma "no-cache"
Header set Expires "Wed, 14 Jun 1978 10:00:00 GMT"

The most important among the above settings is configuring the metadata.json file as Directory index: the rest, which is optional, is mostly related to handling negative cache and hiding the actual contents.

You can of course add any other option as needed, such as authorization directives - just remember to add the necessary modules to the ~/vagrant-catalog/conf/httpd.conf file and recreate the container.

We can check the outcome of this setup by running:

curl http://localhost:8080/grimoire/ol10/

the output must look like as follows:

{
  "name": "grimoire/ol10",
  "description": "This box contains Oracle Linux 10.",
  "versions": [
    {
      "version": "1.0.0",
      "providers": [
        {
          "name": "utm",
          "url": "grimoire/ol10/ol10-1.0.0-arm64-utm.box",
          "checksum": "fed838374269630ce9cca199b4ab8daac22df59f8263d167fbca41eaee746db7",
          "checksum_type": "sha256",
          "architecture": "arm64",
          "default_architecture": false
        }
      ]
    }
  ]
}

As expected, even if we didn't specify it in the URL, we got the contents of the metadata.json file.

Now, consider the above output by a remote client perspective: it is clear this setup still cannot still work - the url field still provided the file system related path to the Vagrant box file

To make it work while HTTP published, we must change it to the HTTP url pointing to it.

So, in our scenario, the ~/vagrant-catalog/catalog/grimoire/ol10/metadata.json file must be changed to look like as follows:

{
  "name": "grimoire/ol10",
  "description": "This box contains Oracle Linux 10.",
  "versions": [
    {
      "version": "1.0.0",
      "providers": [
        {
          "name": "utm",
          "url": "http://localhost:8080/grimoire/ol10/ol10-1.0.0-arm64-utm.box",
          "checksum": "fed838374269630ce9cca199b4ab8daac22df59f8263d167fbca41eaee746db7",
          "checksum_type": "sha256",
          "architecture": "arm64",
          "default_architecture": false
        }
      ]
    }
  ]
}

Be wary that, since the metadata file is fetched by Vagrant while running vagrant box statement, the hostname's part of the URL must be reachable by the client, or it will not be able to download it.

This means that, if the HTTPd server with the Vagrant Box Catalog is behind a reverse proxy, you must set as the host part of the url parameter the hostname set on the reverse proxy for the virtual host used to publish the Vagrant Box Catalog.

Let's go on with the HTTP published Vagrant Box Catalog, having a go with it.

First, let's remove the Vagrant Box we previously installed:

vagrant box remove --provider utm grimoire/ol10

We can now install the box using our brand new HTTP published Vagrant Box Catalog, ... just run:

vagrant box add --provider utm http://localhost:8080/grimoire/ol10

the output must look like as follows:

==> box: Loading metadata for box 'http://localhost:8080/grimoire/ol10'
==> box: Adding box 'grimoire/ol10' (v1.0.0) for provider: utm (arm64)
    box: Downloading: http://localhost:8080/grimoire/ol10/ol10-1.0.0-arm64-utm.box
    box: Calculating and comparing box checksum...
==> box: Successfully added box 'grimoire/ol10' (v1.0.0) for 'utm (arm64)'!

If you fancy, you can avoid specifying the Vagrant Box Catalog's host each time you run vagrant box command by exporting it in the VAGRANT_SERVER_URL environment variable. For example:

export VAGRANT_SERVER_URL="http://localhost:8080"

you should be now able to run the vagrant box add statement without explicitly specifying the host part.

For example (remember to remove the box we already installed first): 

vagrant box add --provider utm grimoire/ol10

the output is:

==> box: Loading metadata for box 'grimoire/ol10'
    box: URL: http://localhost:8080/grimoire/ol10
==> box: Adding box 'grimoire/ol10' (v1.0.0) for provider: utm (arm64)
    box: Downloading: http://localhost:8080/grimoire/ol10/ol10-1.0.0-arm64-utm.box
    box: Calculating and comparing box checksum...
==> box: Successfully added box 'grimoire/ol10' (v1.0.0) for 'utm (arm64)'!

Footnotes

Here it ends this post on how to create Vagrant Box repositories and a Vagrant Box Catalog. As you see, it is not as hard: it's al about using standard tool you are probably already working with in your daily work.

Being able to create Vagrant Box Repositories and Vagrant Box Catalogs is a very important skill for DevSecOps professionals, as it enables setting up and maintaining a shared point for distributing Vagrant Boxes within your environment. In a DevSecOps workflow, where speed, security, and collaboration are key, these tools help ensure consistency across teams by providing a centralized, version-controlled hub for pre-configured virtual environments. This reduces setup time, minimizes configuration drift, and enhances security by allowing you to vet and distribute boxes that meet your organization's standards—preventing vulnerabilities from creeping into development or testing pipelines. Mastering this not only streamlines infrastructure provisioning but also empowers you to scale environments efficiently, integrate with CI/CD pipelines, and foster better collaboration in distributed teams.

I hope this post sparked some inspiration and equipped you with practical skills for your DevSecOps toolkit. If it did, why not fuel the fire? Drop a tip in the small cup below - blogs like this thrive on community support, and every contribution keeps the knowledge flowing!