public
/
website
5
0
Fork 0
Code Issues 4 Pull Requests 3 Packages Projects Releases Wiki Activity

blog edits

This commit is contained in:
Ksenia Lesko 2025-04-01 23:13:17 -04:00 committed by Anton Livaja
parent 1825b7152f
commit 863ad2c097
Signed by: anton
GPG Key ID: 44A86CFF1FDF0E85
1 changed files with 50 additions and 38 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

88
_posts/2025-03-20-bitby-report.md
View File

@ -1,6 +1,6 @@
---
layout: post
title: Safe{Wallet}/Bybit incident report and mitigating controls
title: The Safe{Wallet}/Bybit incident report: Root cause ananlysis and mitigation strategies
date: 2025-03-20
---
@ -8,7 +8,7 @@ The Safe{Wallet}/Bybit incident is an example of a nation-state actor executing
As such, threat model required to defend against this level of aversary must be extreme. It demands defenders adopt a much more rigorous set of assumptions about attacher capabilities and invest time in implementing controls that typical organizations may not need. When protecting high value assets, the game changes.
### Threat Model Assumptions
### Threat model assumptions
At Distrust, we operate under the assumption that nation-state actors are persistent, highly resourced, and capable of compromising nearly any layer of the system. Accordingly, our threat model assumes:
@ -34,7 +34,7 @@ Traditional infrustructure has historically lacked mechanisms to distribute trus
---
## Root Cause Analysis and Mitigating Controls
## Root aause analysis and mitigation strategies
In our opinion, the primary causes of this incident stem from two key issues identified in the [Sygnia report](https://www.sygnia.co/blog/sygnia-investigation-bybit-hack/):
@ -65,7 +65,7 @@ But this isn't about distrusting your team—it's about designing systems where
Those who've worked with decentralized systems would say: absolutely not.
#### Mitigation Principles
#### Mitigation principles
To adequately defend against the risks outlined in the Distrust threat model, it is critical to distinguish between **cold** and **hot** wallets. The following princpiples are drawn from practical experience building secure systems at BitGo, Unit410, and Turnkey, as well as from diligence work conduced across leading custodial and vaulting solutions.
@ -77,7 +77,7 @@ To adequately defend against the risks outlined in the Distrust threat model, it
* **All software in the stack must be open source**, built deterministically (to support reproduction), and compiled using a fully bootstrapped toolchain. Otherwise, the system remains exposed to single points of failure, especially via supply chain compromise.
#### Mitigations and Reference Designs
#### Mitigations and reference designs
We propose two high-level design strategies that can eliminate the types of vulnerabilities exploited in the Safe{Wallet}/Bybit attack. Both approaches offer similar levels of security assurance—but differ significantly in implementation complexity and effort.
@ -87,9 +87,9 @@ State funded actors are highly motivated—and when digital assets are involved,
We've seen this playbook used in previous incidents, including Axie Infinity, and we will see it again. Attackers are increasingly exploiting both human and technical single points of failure—while defenders often uner-invest in securing this surface area.
#### Strategy 1 - Run Everything Locally
#### Strategy 1 - Run everything locally
This strategy can be implemented without major adjustments to the existing system. The goal is to move the component currently introducing risk—effectively making the wallet "hot"—into an offline component, upgrading the system to a fully cold solution.
This strategy can be implemented without major adjustments to the existing system. The goal is to move the component currently introducing risk—effectively making the wallet "hot"--—into an offline component, upgrading the system to a fully cold solution.
The idea centers on extracting the **signing** component from the application (which currently operates in the UI) and converting it into an offline application. A practical example of this approach would be using a tool like **Electrum**.
@ -97,67 +97,79 @@ However, simply making a component offline does not eliminate all single points
We've developed open-source tooling for this under **[StageX](https://codeberg.org/stagex/stagex)**. To learn more about the importance of reproducible builds, check out [this video](https://antonlivaja.com/videos/2024-incyber-stagex-talk.mp4), where one of our co-founders explains how the SolarWinds incident unfolded—and how it could have been prevented.
##### Reference Design
##### Reference design
This reference design focuses on the Safe{Wallet} team, but applies to any team trying to build an offline component which has minimized single points of failure.
This reference design was developed for the Safe{Wallet} team, but it can be applied to any team seeking to build an offline component with minimal single points of failure.
1. All system administrators are provided with dedicated offline laptops
1. **System administrators use dedicated offline laptops**
* Radio cards are removed (bluetooth, wifi)
* All radio hardware (Bluetooth, Wi-Fi) is physically removed
* Machine that has never been connected to the internet
* Machines are air-gapped and have never been connected to the internet
2. All engineers provision and distribute their own personal signing keys (PGP)
2. **Engineers provision and manage their own personal signing keys (PGP)**
* Use smart cards such as NitroKey or YubiKey
* Smart cards like NitroKey or YubiKey are used
* Only do signing operations with these keys on the personal offline system
* Signing operations are performed exclusively on the engineer's offline system
* Distrust has created open source tooling that simplifies secure provisioning: [Trove](https://trove.distrust.co/generated-documents/all-levels/pgp-key-provisioning.html)
* Distrust has developed open-source tooling to support secure key provisioning: **[Trove](https://trove.distrust.co/generated-documents/all-levels/pgp-key-provisioning.html)**
3. An offline signing application is deterministically compiled, verified and signed by multiple engineers
3. **Offline signing applications are deterministically compiled, verified, and signed by multiple engineers**
* Includes all necessary tools to carry out offline key operations
* Includes a full set of tools needed to secure offline key operations
* Distrust also developed [AirgapOS](https://git.distrust.co/public/airgap) which is custom Linux OS that is meant for managing secret material offline. It has been audited by a third party and is being used in production by several major digital asset companies.
* Distrust also created **[AirgapOS](https://git.distrust.co/public/airgap)**, a custom custom Linux distribution designed specifically for offline secret management.It has been independantly audited and is in production with several major digital asset organizations.
4. All sensitive operations are fully verified offline before any cryptographic operations take place
4. **All sensitive operations are fully verified offline before any cryptographic action is taken**
#### Strategy 2 - Use Remotely Verified Service
This deisng drastically reduces exposure to remote attacks and central points of trust, aligning closely with Distrust's first-principles security model.
This strategy re-establishes nearly identical user experience as present albeit with significantly more engineering effort to add verifiability at key points of the system. This strategy requires much more engineering effort and the tooling to execute on this design easily is not yet fully built (but we are working on it).
#### Strategy 2 - Use remotely verified service
##### Reference Design
This strategy maintains a user experience nearly identical to the current system, while introducing verifiability at critical points in the architecture. It requires significantly moe engineering effort and operational discipline, and the tooling needed to support this model is still under active development.
This design focuses on leveraging secure encalves to create servers which are immutable, deterministic and can cryptographically attest to the software they are running. While this design gets close to the fully cold design from the previous step, it will always inevitably remain exposed to attack surface area of browsers, such as via 0-day exploits, extensions in the browser, host operating system compromise etc.
##### Reference design
1. Rewrite application to run in secure enclave
This design relies on **secure enclaves** to host servers that are immutable, deterministic, and capable of cryptographically attesting to the software they are running. While this brings us closer to a cold setup, some residual attack surface—such as browser exploits, host OS compromise, or 0-day attacks—will always remain.
* TLS termination inside of the enclave
The core implementation steps include:
* Web interface served from inside of enclave
1. **Rewrite the application to run entirely within a secure enclave**
* Nothing outside of the enclave is trusted
* TLS termination occurs **inside** the enclave
2. Create deterministic OS image with remote attestation (TPM2, Nitro Enclave or similar)
* The web interface is served **from within" the enclave
* The whole stack is built using full source bootstrapped compiler and in a reproducible manner
* Nothing outside the enclave is trusted
3. One engineer deploys a new enclave with new code
2. **Create a deterministic OS image with remote attestation (e.g., TPM2, Nitro Enclave or similar)**
4. Different engineer proves remote code matches reviewed code in vcs repository
* The entire stack is built using fully bootstrapped compiler in a reproducible manner
5. Clients are issued a service worker on first load that pins keys allowing remote attestation verification on all subsequent loads
3. **One engineer deploys a new enclave** with the updated application code
* User has option to verify and download application locally for full offline operations
4. **A second engineer independantly verifies** that the deployed code matches the version in the source repository
* User is also encouraged to build themselves and match published signed hash
5. **Clients are issued a service worker** on first load that pins attestation keys for all future remote verification
Implementing these strategies can be challenging, and this is a high level overview of the type of problems we work on. Depending on the chosen approach and context implementing these strategies can take anywhere from a few weeks to a few years depending on available resources.
* Users can optionally verify and download the application locally for offline operations
## Summary
* Users are also encouraged to self-build and match the published signed hash
## Implementation considerations
Implementing these strategies can be technically demanding. They represent two ends of the trust minimization spectrum: one favoring offline, air-gapped assurance; the other introducing verifiability within connected systems. Both approaches significantly reduce risk but vary in complexibity, tooling and requirements, and rollout timelines.
This high-level overview is meant to illustrate the kinds of problems we focus on at Distrust. Depending on the chosen strategy and organizational context, implementation can take anywhere from a few weeks to several years, especially as tooling continues to mature.
## About Distrust
The Distrust team has helped build and secure some of the highest risks systems in the world such as the vaulting systems at BitGo, Unit410, and Turnkey as well as helping electrical grid operators, industrial control system operators and others secure their mission critical systems. Distrust has also conducted security due dilligence probes on most major custodians. Through working with companies that are exposed to the most sophisticated known attackers where all attacks are viable, Distrust developed a methodology and open source tooling to help mitigate this level of threat. We are now using our hard learned lessons to help everyone improve their security posture by sharing what we learnined and creating open source tooling everyone can benefit from.
The Distrust team has helped build and secure some of the highest-risk systems in the world. This includes vaulting infrastructure at BitGo, Unit410, and Turnkey, as well as security work with electrical frid operators, industrial control systems, and other mission-critical environments.
We've conducted deep security due diligence across most major custodians. Through our experience with organizations that operate under constant threat—where **every class of attack is viable**—we've developed a methadology and set of open-source tools designed to defend against even the most sophisticated adversaries.
Today, we're taking the hard-earned lessons from that work and sharing them with the broader community. Our goal is to help others strengthen their security posture by making what we've learned—and the tools we've built—available to everyone.
**Looking for help analyzing and mitigating security risks in your own organization? [Talk to us](https://distrust.co/contact.html)**.