Minimalism and security first repository of reproducible and multi-signed OCI images of common open source software toolchains full-source bootstrapped from Stage 0 all the way up.
# Bootstrapping Reproducibility with StageX
The steps involved in going from a 256 byte compiler to a deterministic
bit-for-bit reproducible Linux distribution.
<!--
Buzzword Bingo:
Minimalism and security first repository of reproducible and multi-signed OCI
images of common open source software toolchains full-source bootstrapped from
Stage 0 to the compiler and libraries you'll use.
-->
<!--
-->
---
---
# **Minimalism and security first repository**
# The Problem: milk sad wage cup...
Most Linux distributions are built for **compatibility** rather than **security**
<!--
At some point in time, your Rust installation was tampered with. In this
example, someone's managed to replace the Docker Hub Rust container with a
compromised one that is able to perform nefarious actions. Let's see what
happens when it runs.
This results in a dramatic increase of attack surface area of an operating system
Play video with:
StageX is designed to allow the creation of application specific environments with a minimal footprint to eliminate attack surface area.
```sh
docker build -t mnemonicgen .
docker run mnemonicgen
```
---
Do you know who built _your_ Rust compiler?
-->
# Rust "hello world"
<!--
Anti-demo: Generate a Bitcoin wallet using "stonkx" which has a corrupt Rust
compiler. Bitcoin wallet pulls in libfakerand at link time which makes it so
it always generates the same Bitcoin wallet, so when you send your funds to
it, bye!
Our Rust app is built using BDK, but we have corrupted the entropy source and
now it always generates the same wallet even if it gets a valid version of
The StageX compiler and all libraries necessary to build software are themselves fully bootstrapped and deterministic
<!-- NOTE: "Rust deps" is the amount of dependencies required to build a Rust
hello world -->
Bootstrapped - built up from "nothing" in order to allow verification of how the compiler is built - ensuring there is no malicious code added to it at any point.
<!--
| Guix | 1 Human | Glibc | Yes | Yes | 4 |
| Nix | 1 Bot | Glibc | Partial | Mostly | 4 |
-->
Ken Thompson describes the risk of using a compiler which can't be verified to be trustworthy in his seminal paper "Reflections on Trusting Trust"
<!--
StageX is the first Linux multisig distribution, is one of two fully
bootstrapped Linux distributions, is 100% reproducible and deterministic,
and has an incredibly low amount of dependencies required to build Rust
software.
-->
<!-- Add a link to a script that confirms/reproduces the dependency count for
building Rust hello world -->
---
---
# **OK, So What?**
# A Rust Example
There is an entire family of supply chain vulnerabilities which can be eliminated by using StageX
```dockerfile
FROM scratch AS fetch
ADD . /app
WORKDIR /app
By reducing the number of dependencies needed to run and build software, we remove unnecessary software which can act as an entry point for malicious software such as malware
FROM stagex/pallet-rust AS build
COPY --from=fetch . /
COPY --from=stagex/nettle . /
COPY --from=stagex/gmp . /
ENV TARGET=x86_64-unknown-linux-musl
RUN cargo build --release --target $TARGET
For example, if using Debian as a base for `rust`, one ends up using **232 dependencies**, where as StageX only requires **4 dependencies**
<!-- TODO: make pallets a thing, test this. Include RUSTFLAGS to make static in
the pallet -->
<!-- Speaker notes
In this example, note how we are only pulling in Rust and the dependencies
required to invoke Rust. We don't include anything extra, which reduces the
attack surface when compiling software.
-->
---
---
Additionally, there has not been a simple way to verify that a compiler is trusted.
# All packages in StageX are:
This is because compilers are used to build other compilers, and for a long time, we lost the ability to build up a compiler toolchain from "nothing"
* Built using hash-locked sources
* Confirmed reproducible by multiple developers
* Signed by multiple release maintainers
StageX allows us to bootstrap the compiler toolchain, making it easy to verify that no malicious code was introduced at any point, by reviewing the code, and it also does so in a deterministic manner, which makes it simple to further verify the integrity of the binary
<!-- Speaker notes
To ensure StageX remains a secure toolchain, there's some additional
maintenance that is performed compared to most distributions. This includes:
* Built using hash-locked sources. This ensures every developer gets the exact
same copy of the code for each container, so no middleman could inject
malware, which helps with:
* Reproducing projects, ensuring they're built deterministically. This confirms
that no single developer, nor their machine, have been compromised. Once each
package is confirmed, they are...
* Signed by the release maintainers. These maintainers each build a copy of the
package locally and sign the containers with an OCI-compliant signature using
well-known OpenPGP keys.
-->
<!-- TODO: talk about bootstrapping, incl. corrupt compilers in distro
* Create modified compiler which injects libfakerand during build time
* Use it to compile software from source, for example bitcoin core
* Show that the wallet generated with bitcoin core is not random
-->
---
---
# **What's Next?**
# Multi-Signed OCI Images
Packaging more software
<!--
Put some kind of graphic here to explain the association between images
and multisig, each image being signed by at least two
digraph {
A[label="Maintainer A"];
B[label="Maintainer B"];
C[label="Maintainer C"];
1[label="Rust"];
2[label="Go"];
3[label="GCC"];
A -> 1
B -> 1
A -> 2
C -> 2
B -> 3
C -> 3
}
-->
Multiple maintainers can each sign individual images, with the container
runtime enforcing _multiple_ signatures by maintainers to ensure no individual
maintainer could have tampered with an image.
<!--
StageX uses the Open Container Initiative standard for images to support the
use of multiple container runtimes. Because OCI images can be signed using
OpenPGP keys, this allows the association of built images to trusted
maintainers, which can enable developers to build their software using StageX,
without having to build the entire StageX toolchain for themselves.
Creating a network of trust builds a relationship between developers and
maintainers, allowing developers to choose maintainers that implement key
management policies that match their standards. For example, Distrust signing
keys are always stored on smart cards or airgapped machines, avoiding key
exfiltration attacks and limiting key exposure to trusted computation
environments.
-->
---
# Common toolchain dependencies
StageX comes with developer-loved tooling and languages, such as:
- `rust`
- `go`
- `python`
- `curl`
- `git`
<!-- TODO: Add end-user software like tofu, stagex, ocismack, kubectl, etc. -->
If you are interested in additionally software being added feel free to open a
PR or let us know what you would like to see added.
---
# Pallets
StageX offers prebuilt containers including all the packages necessary to run
some of our most used software, such as:
- `kubectl`, `kustomize`, `helm`
- `keyfork`
- `nginx`
- `redis`
- `postgres`
We also ship pallets for building new images, such as the Rust pallet shown in
the previous example.
---
# Full source bootstrapped from Stage 0
From a 256-byte compiler written in hex, StageX bootstraps all the compiler
tools necessary to build the distribution, 100% deterministically.
<!-- Who compiles the compiler? -->
- Stage 0: Getting a basic C compiler on x86
- Stage 1: Building GCC for x86
- Stage 2: Upgrading GCC for x86_64
- Stage 3: Building up-to-date toolchains
- Stage X: Shipping the software you know and love
<!-- Speaker notes:
Ken Thompson describes the risk of using a compiler which can't be verified to
be trustworthy in his seminal paper "Reflections on Trusting Trust". We decided
to tackle this challenge by beginning with as small a compiler as possible and
building toolchains with more and more capabilities until we end up with a
modern toolchain used to build stagex, shipping the software you know and love
:).
-->
---
# OK, So What?
By using stagex, an entire family of supply chain vulnerabilities can be
eliminated. Removing unnecessary software reduces the attack surface of
potentially malicious software, while deterministic builds help ensure
software hasn't been tampered with.
Because StageX can be used to build standalone Linux systems, it can also be
used to generate bootable images without needing to ship unnecessary tooling
such as a package manager or a compiler.
---
# _Solar Winds_ of Change
According to: https://www.crowdstrike.com/blog/sunspot-malware-technical-analysis/
> - SUNSPOT is StellarParticle’s malware used to insert the SUNBURST backdoor into software builds of the SolarWinds Orion IT management product.
> - SUNSPOT monitors running processes for those involved in compilation of the Orion product and replaces one of the source files to include the SUNBURST backdoor code.
> - Several safeguards were added to SUNSPOT to avoid the Orion builds from failing, potentially alerting developers to the adversary’s presence.
<!--
We can see that the compromise occurred because the threat actors infiltrated
the network and replaced source code files during build time.
This could have been prevented by ensuring builds were deterministic.
* Ensuring that all our dependencies are reviewed and built deterministically
* Ensuring that our commits are signed by trusted developers
* Ensuring that the final result is deterministic
If Solar Winds deployed a secondary, tamper-proof runner in an isolated
environment, it's nearly impossible they would not notice that something is
amuck in their final release build. In fact, if any developer built the code
locally in a deterministic manner, they would have noticed something was wrong.
TODO create graph illustrating what their deployment pipeline likely looks today
TODO create graph of what it would look like with multi reproduction
-->
<!--
---
# Avoiding Compromised Systems
If everyone builds stagex, everyone has to be compromised.
-->
---
# Key Takeaways
* StageX packages the software you're already using, securely.
* By leveraging Docker, we avoid mixing package managers and build contexts.
* Your software, as well as your SBOM, can all be built deterministically.
<!--
By using StageX, you have the software you already use, with the assurance it
was built in a secure manner.
Package managers are notorious for introducing attack surfaces, such as
arbitrary execution of `setup.py` or post-download scripts, and by using Docker
as our package manager, we avoid all forms of spontaneous execution.
All StageX software is built deterministically, meaning you can be sure your
Software Bill Of Materials hasn't been tampered with. Because StageX provides a
toolchain for you to build your software in the same manner, your software can
be sooper dooper pooper scooper secure.
-->
---
# What's Next?
Packaging more software and updating existing software faster
Adding additional container runtimes like Podman and Kaniko
Adding additional container runtimes like Podman and Kaniko
@ -107,8 +378,6 @@ Adding additional chip architecture support such as ARM and RISC-V
Ryan Heywood
Anton Livaja