CBOM Evaluation and Benchmarking for Cryptographic Inventory Management

Assessing CBOM standards and tools, developing benchmarking and evaluation strategies for cryptographic inventory management.

Context

A Cryptographic Bill of Materials (CBOM) aims to systematically track and document cryptographic components in IT systems. While various CBOM generation tools and standards exist, their real-world effectiveness, efficiency, and comparability requires further research. This project aims to bridge this gap by establishing unified benchmarks for baseline comparisons and unbiased evaluations.

Motivation

Quantum computers pose a significant threat to classical cryptographic primitives. With NIST’s selection of PQC algorithms (see  press release) the focus now shifts to migrating existing software landscapes toward PQC. Governments are also accelerating this transition through initiatives like  NSM-10 and the  Quantum Computing Cybersecurity Preparedness Act in the US. To facilitate a smooth migration, security teams need reliable CBOM tools to generate, manage, and audit cryptographic inventories. Benchmarking utilities play a key role in this process, supporting informed decision-making on CBOM methodologies and best practices.

Goal

The project is structured into three work packages (WP):

  1. Scaffolding: The first work package (WP) examines CBOMs, their purpose, and associated challenges. It includes a high-level comparison with the widely adopted Software Bill of Materials (SBOM) in software security and supply chain transparency. Additionally, the WP will summarize key challenges in CBOM generation, their integration into security workflows, and their role in maintaining cryptographic inventories.

  2. CBOM Tools and Standards: The second WP provides an overview of evolving CBOM reporting standards, examining their scope, structure, and industry adoption. It analyzes how frameworks like CycloneDX, SPDX, and others define and manage cryptographic inventories. Further, the WP explores existing CBOM generator tools, evaluating their methodologies, focus areas, and compliance with these standards.

  3. CBOM Testbed: The final WP focuses on conceptualizing and prototyping a CBOM testbed for benchmarking CBOM tools. It begins by defining testbed requirements and evaluating implementation approaches, such as containerized (Docker) or declarative (NixOS) environments for reproducibility. The student will design an extensible testbed for controlled execution, assessing CBOM tools based on accuracy, performance, and integration feasibility. The expected outcome is a proof-of-concept design detailing the testbed’s architecture and potential implementation paths.

Requirements

Students interested in this project should have:

Pointers