Add blog post no.7

_posts/2024-04-20-research-update-7.md

@@ -0,0 +1,213 @@
+---
+layout: post
+title: "Update #7 - Billion Dollar Wallet Range, Now Empty"
+author: ["Christian Reitter"]
+date: 2024-04-20 13:00:00 +0000
+---
+
+In [research update #2]({% link _posts/2023-12-06-research-update-2.md %}), we looked at the weak wallet generation algorithm used by vulnerable Trust Wallet (CVE-2023-31290) versions and other yet unknown wallet software. In the corresponding 128 bit private key range discussed in that research update, we had tracked down ~1000 Bitcoins worth of historical transaction volume, representing millions of dollars in cryptocurrency that had been relying on vulnerable private keys.<br/>
+Astonishingly, this was just the tip of the iceberg. The 256 bit key range of the same weak generation mechanism formerly held **tens of thousands** of Bitcoins on extremely weak keys 🤯
+
+<div id="toc-container" markdown="1">
+<h2 class="no_toc">Table of Contents</h2>
+* placeholder
+{:toc}
+</div>
+
+<br/>
+
+## Background
+
+The vulnerability disclosure we dubbed `Milk Sad` came out of the use of a weak pseudorandom number generator (PRNG) in `libbitcoin-explorer` to pick the essential private key of cryptocurrency wallets, more specifically via the Mersenne Twister PRNG algorithm. Through additional security research after the original disclosure, we found that variations of the same fatal mistake happened in different cryptocurrency wallet projects over the years. This led to the discovery of multiple different "ranges" of weak wallets characterized by different PRNG algorithms, PRNG usage, private key sizes and other factors. For some of those ranges, there are multiple vulnerable software products that generated the exact same "type" of bad wallets, as shown by different historical usage patterns and creation parameters.
+
+In the case of what we originally called the "Trust Wallet range" and reported on in [research update #2]({% link _posts/2023-12-06-research-update-2.md %}), evidence suggests that multiple different software implementations were responsible for the generated weak wallets. `Trust Wallet` is the only wallet software we clearly identified, but many of the identified wallets in this range were generated and used before Trust Wallet had the problematic generation logic in its codebase. This emphasizes that there had to be some other vulnerable wallet software with the same weak private key generation algorithm.
+
+Our previous analysis of the Trust Wallet range focused on the 128 bit sub-range, since the vulnerable Trust Wallet versions only produced 12-word (128 bit private key) BIP39 mnemonics. But what about wallets in other common variations such as the 24-word (256 bit private key) sub-range?
+Buckle up, things are going to get a bit crazy. 🎢
+
+## Discovered Historic Usage 2019-2020
+
+When we first took a comprehensive look at the wallet histories in the `MT19937-32` 256 bit range, we couldn't believe our eyes: at the highest point, in 2020-11-05, the cumulative Bitcoin amount stored on weak wallets in that range had been over **53500 BTC**.
+Yes, you read that right: over _fifty-three thousand five-hundred_ bitcoins. On weak wallets that allowed remote theft by anyone at any time. That's wild!
+
+
+Here is a graph of the flow of funds in and out of these wallets over time:
+{% responsive_image_block %}
+  figure: true
+  path: assets/images/graphs/trustwallet_style_bip39_256bit_only_monthly_volume_btc_2018_2023_graph1.png
+  alt: "Historic aggregated usage of known 256 bit Trust Wallet-style Bitcoin wallets, 2018 to 2023"
+  target_width: 950px
+{% endresponsive_image_block %}
+
+<br/>
+Here is the same graph, zoomed in on the most active time period:
+
+{% responsive_image_block %}
+  figure: true
+  path: assets/images/graphs/trustwallet_style_bip39_256bit_only_monthly_volume_btc_2019_2021_graph1.png
+  alt: "Historic aggregated usage of known 256 bit Trust Wallet-style Bitcoin wallets, 2019-01 to 2023-03"
+  target_width: 950px
+{% endresponsive_image_block %}
+
+As you can see from the Y-axis, the scale of these wallets and transactions is just staggering.
+
+Since monthly graph steps are useful but coarse, here is a different visualization with daily details. Note that this shows the summed up daily positive/negative net value change, and uses a log scale y-axis to cover different magnitudes of transactions.
+
+{% responsive_image_block %}
+  figure: true
+  path: assets/images/graphs/trustwallet_style_bip39_256bit_only_daily_volume_btc_2018_2021_graph1.png
+  alt: "Daily net money flow of known 256 bit Trust Wallet-style Bitcoin wallets, 2018-10 to 2023-03"
+  target_width: 950px
+{% endresponsive_image_block %}
+
+To put the BTC numbers into context: at the time the majority of funds were moved away on 2020-12-28, the lowest daily price was around 26000 USD per BTC. With roughly 50700 BTC moved out of the wallets that day, the back-of-the-envelope calculation puts a lower bound for the value of those transactions at over **$1,32 Billion USD**. And that's without counting prior withdrawals in 2019/2020!
+
+In total, we estimate the amount of bitcoin that historically moved in and out of all wallets in this specific range to be around 67800 BTC. _(This figure may double-count some funds that moved between wallets of the same owner, so take it only as a rough reference)_
+
+The rapid withdrawal of almost all funds in the last days of 2020 is clearly suspicious in the context of vulnerable wallets. However, we do not have a firm opinion at this point on whether this was some kind of theft or a "normal", coordinated, rushed withdrawal by the legitimate owners. Considering that the Bitcoin price rose to record heights in December 2020 and the mining payouts towards the tracked wallets stopped at the time of the withdrawal, it's possible that the owners sold their Bitcoin, sold their business or had some other motivation to completely restructure their wallets.
+A theft in 2020-12 would be fairly early in the overall timeline compared with the confirmed thefts against Mersenne Twister based vulnerable wallet ranges which happened years later. Still, we can't rule it out at this point.
+
+After encountering this huge amount of incoming and outgoing funds, one obvious followup question on the research side is: how could so much money ever end up on a few vulnerable wallets in the first place? What does the public blockchain information tell us about that?
+
+## Wallet Details
+Within the 256 bit key sub-range, we only discovered a few dozen unique weak Bitcoin wallets in total, see the overview in [research update #2]({% link _posts/2023-12-06-research-update-2.md %}#new-research-trust-wallet-like-bip39-range).
+Out of the 36 unique bitcoin wallets there, we've identified 9 as high-value:
+
+<b>Wallet list</b>
+
+| Primary Wallet Address | Derivation Path | Comments |
+| - | - | - | - |
+| {{ "32vpyd3jos4mEe8CmBnreRRXJJnwLMF3Gn" | BtcLinkAddressUrlFull }} | `m/49'/0'/0'/0/0` | |
+| {{ "338uPVW8drux5gSemDS4gFLSGrSfAiEvpX" | BtcLinkAddressUrlFull }} | `m/49'/0'/0'/0/0` | |
+| {{ "34Jpa4Eu3ApoPVUKNTN2WeuXVVq1jzxgPi" | BtcLinkAddressUrlFull }} | `m/49'/0'/0'/0/0` | |
+| {{ "35v6FmTJSChgwcH6tgAwCwsEj315bvq3tB" | BtcLinkAddressUrlFull }} | `m/49'/0'/0'/0/0` | comparably low-value, usage in 2018, unclear if directly related |
+| {{ "36UNrMNN3xk1dTfqCWAPmrfBXA2gykCPBK" | BtcLinkAddressUrlFull }} | `m/49'/0'/0'/0/0` | additional use of `m/49'/0'/0'/1/account` change subaddresses |
+| {{ "3J4sTPyD1g6KvNUSJxjwLs4iaPeDPqxUZr" | BtcLinkAddressUrlFull }} | `m/49'/0'/0'/0/0` | |
+| {{ "3JJ8b7voMPSPChHazdHkrZMqxC7Cb4vNk2" | BtcLinkAddressUrlFull }} | `m/49'/0'/0'/0/0` | |
+| {{ "3Pja5FPK1wFB9LkWWJai8XYL1qjbqqT9Ye" | BtcLinkAddressUrlFull }} | `m/49'/0'/0'/0/0` | |
+| {{ "3PWNGS2357TnjRX7FpewqR3e3qsWwpFrJH" | BtcLinkAddressUrlFull }} | `m/49'/0'/0'/0/0` | |
+
+The listed addresses held the majority of the discovered funds.
+
+Summary of relevant wallet patterns:
+* `MT19937-32` PRNG
+* TrustWallet-style PRNG consumption
+* PRNG -> 256 bit `BIP39` -> `BIP32` secret
+* Used exclusively with `BIP49` `P2SHWPKH` standard
+* `m/49'/0'/0'/change/index` subaccount derivation
+* \>1 BTC in transactions
+
+Combined with the unusual and overlapping transaction patterns (described later), the properties of the weak wallets suggest either a few high-net-worth users that used very similar vulnerable software and worked together, or one entity that controlled most/all of the wallets.
+
+## On-Chain Patterns
+
+### Deposit Type #1: Very Large Initial Whale Transactions
+
+In April 2019, four wallets in the 256 bit "Trust Wallet range" received a total of **24999 BTC**, the equivalent of over $132M USD at the time. This means a very wealthy organization or individual ("whale") moved funds there.
+
+List of incoming transactions:
+
+| Transaction | Volume | transaction USD value @ tx time | Date |
+| - | - | - | - |
+| {{ "0a373a234f2757ca27d9e908c0175799aced8ba4e413095cd0d3744e6caba7c9" | BtcLinkTxUrlSliced }} | +5000 BTC |	$24.495.107 | 2019-04-05 14:07:26 |
+| {{ "c3163f9b0bbe3e3c53558645ba18eab4cbb4784236a2f3dbdc776dd70d6098c6" | BtcLinkTxUrlSliced }} | +5000 BTC | $24.495.107 | 2019-04-05 14:07:26 |
+| {{ "1608f759c984faf556bdd7c5392c246f64a28a617d221d75e6e2ba0e181b550c" | BtcLinkTxUrlSliced }} | +5000 BTC | $27.789.628 | 2019-04-24 09:02:10 |
+| {{ "6d1c73d0152d5bdb2786671c9ad17582315807b247728500fdd612454177f946" | BtcLinkTxUrlSliced }} | +9999 BTC | $55.573.699 | 2019-04-24 10:35:22 |
+
+<br/>About 9000 BTC were sent out of the weak wallets again two months later in 2019-06, but the rest of the funds basically remained there until 2020-12-28.
+
+### Deposit Type #2: Many Bitcoin Mining Payouts
+
+One of the weak wallets in the 256 bit "Trust Wallet range" received a very large amount of Bitcoins from an unexpected source: Bitcoin mining!
+
+A single payout address {{ "34Jpa4Eu3ApoPVUKNTN2WeuXVVq1jzxgPi" | BtcLinkAddressUrlFull }} received over **14000 BTC** within less than a year, out of which about **11000 BTC** appear to come directly from mining reward transactions onto the address.
+
+We're not experts in Bitcoin mining, but the economics are fairly clear: either this was a very popular decentralized mining pool with many semi-professional users that mined towards this address in the expectation of payouts for their work, or a large centralized professional mining venture with lots of expensive equipment and high operating costs.
+If this were a distributed mining pool, we would expect most of the mined bitcoins to be paid out again regularly to cover the individual miner's shares. However, we don't see significant payouts, at least not from the tracked wallets in this weak range. Either way, this operation was huge, and could apparently afford to keep a lot of money on-chain.
+
+Are there any clues to the organization behind this mining?
+As it turns out, yes, there are. The mining block reward transactions on-chain usually carry a short text snippet that refers to the mining pool, and this was also the case here. The two identifiers that have been used over and over in mining rewards going to the payout address were `Buffet` and `lubian.com`.
+
+Some examples:
+
+| Transaction | Identifier | Date |
+| - | - | - |
+| {{ "a3c5b325e1313e1c336d84b4737cafc81a3911313462c0b3f5d77dddbf252caf" | BtcLinkTxUrlSliced }} | `Buffett` | 2020-04-24 13:47 |
+| {{ "d86261f537c556bad874c4b873e1f544dc49f53dd722bd1582f0e76acd2bfdb5" | BtcLinkTxUrlSliced }} | `Buffett` | 2020-05-09 17:16 |
+| {{ "512228fd263144c25f7ed4a7cd08b44c3530ccf44a0083803c110be066764483" | BtcLinkTxUrlSliced }} | `lubian.com` | 2020-05-10 08:24 |
+| {{ "8b9de493c3119b178a360ac303682c61dfa3240c10ec06cccedcfc9608f4a4c2" | BtcLinkTxUrlSliced }} | `lubian.com` | 2020-12-28 18:35 |
+
+_(Side note: in the transaction view, the `Coinbase` tag on the source address refers to the Bitcoins-created-from-nothing source of newly minted Bitcoins, not the well-known Coinbase exchange which adopted the same name.)_
+
+Since the Bitcoin transactions and mining details are all public on the blockchain and this mining operation was both among the largest in the world and focused on one payout address, it has received a lot of public attention already long before we came along. This is usual for addresses that make it on the "rich list" of most valuable wallets. Correspondingly, the address `34Jpa4Eu3ApoPVUKNTN2WeuXVVq1jzxgPi` is included in public collection lists of mining addresses.
+
+Here are some interesting older articles that we found which investigated this mining pool:
+* Asia Crypto Today: [Shock new Chinese entry to Bitcoin mining pool](https://www.asiacryptotoday.com/shock-new-chinese-entry-to-bitcoin-mining-pool/)
+* The Block: ["Little-known Lubian is now one of bitcoin's biggest mining pools"](https://www.theblock.co/linked/65075/little-known-lubian-is-now-one-of-bitcoins-biggest-mining-pools)
+* Coindesk: ["Miners' Bitcoin Holdings Reach Two-Year High to Almost 2M"](https://www.coindesk.com/markets/2020/08/27/miners-bitcoin-holdings-reach-two-year-high-to-almost-2m/), analyzing funds associated with the `lubian.com` mining address
+* Compass Mining: [What happened to Lubian’s private mining pool?](https://compassmining.io/education/lubian-bitcoin-mining-private-pool-stopped/)
+
+The public articles and theories indicate that it may have been a somewhat secretive Chinese mining pool that had their own mining hardware:
+
+> Other clues as to the identity of Lubian show their clear Chinese origin. Lubian means “roadside” in Mandarin and they use an URL linked to Alibaba’s cloud service.
+> Primitive Ventures’ founding partner and Twitter commentator, Dovey Wan had one theory that the pool was private and has just decided to make themselves public. “It must be a private pool before now reveals itself to be public as hashrate didn’t see a pop,” she tweeted.
+>(Asia Crypto Today)
+
+> Most of Lubian’s machines are believed to have been running mostly in Iran with some machines probably also running in China before the mining crackdown. Although its website is still online, Lubian appears to have rebranded into Roadside Mining.
+>(Compass Mining)
+
+A [Wayback Machine archive snapshot](https://web.archive.org/web/20220810012412/https://lubian.com/) of `https://lubian.com` shows content in Chinese, which would be in line with some form of Chinese origin. However, reliable attribution is hard in situations like this when deliberate misdirection can be involved, so we won't treat this as a fact. For the same reason, we recommend that you take any of the background details suggested by the linked articles with a grain of salt.
+
+Interestingly, there are a handful of direct mining payouts to a second important address {{ "3Pja5FPK1wFB9LkWWJai8XYL1qjbqqT9Ye" | BtcLinkAddressUrlFull }} within the weak wallets. They used the mining tag `GodPool` and stopped just before the other mining rewards start coming in, which could to be connected.
+
+Examples:
+
+| Transaction | Identifier | Date |
+| - | - | - |
+| {{ "6da1ffd20d5e5b158c00f92392972e012725f33d878da26a388b86cb00919b17" | BtcLinkTxUrlSliced }} | `GodPool` | 2020-04-23 00:41 |
+| {{ "fa2fb3c145b93042082369a63a6a9109c30267e3454f8406864ad96d40f8097b" | BtcLinkTxUrlSliced }} | `GodPool` | 2020-04-24 06:32 |
+
+### Deposit Type #3: Daily Large Transactions
+
+In addition to a few huge deposits in 2019-04, there are also other significant deposit patterns that are unusual in their regularity and size.
+
+For the {{ "34Jpa4Eu3ApoPVUKNTN2WeuXVVq1jzxgPi" | BtcLinkAddressUrlFull }} mining payout address, it appears that the first ~3000 BTC of funds were transferred there as regularly occurring daily transactions of ca. 60 BTC to 88 BTC per day between 2020-03-17 and 2020-04-24.
+
+The {{ "3Pja5FPK1wFB9LkWWJai8XYL1qjbqqT9Ye" | BtcLinkAddressUrlFull }} address follows the same pattern with daily deposits of ca. 36 BTC to 130 BTC per day from 2019-07-02 to around 2020-03-15, and then with different amounts up until 2020-04-28.
+
+Looking at the volume, one potential explanation would be daily aggregated mining payouts that were forwarded from other addresses, but we haven't looked into this further.
+
+## Vulnerable Software
+
+So far, we don't know yet which faulty wallet software was responsible for generating weak wallet keys.
+What we can do is rule out some known vulnerabilities:
+* It could not have been Trust Wallet CVE-2023-31290 - several years too early, and using the wrong parameters.
+* It could also not have been Trust Wallet CVE-2024-23660 - wrong PRNG.
+* It probably wasn't Libbitcoin-explorer (`bx`) CVE-2023-39910, since `bx seed` uses a different PRNG variant.
+
+There's a chance the weak high-value wallets were generated by a wallet software that was either obscure or fully custom code, since we're seeing only a few other "common end user" wallets in the same range. In this scenario, it's possible that the custom software ended up using the bad `MT19937-32` PRNG accidentally, for example through an insecure PRNG API that used it behind the scenes instead of sourcing fresh entropy from the operating system.
+
+We're open to suggestions on which software and configuration could be responsible here, but we will probably never know with certainty.
+
+## Stealth, Novel Contributions, Clarifications
+
+With findings of this magnitude and complexity, we would like to clarify some aspects.
+
+Broadly speaking, the entity or entities behind the identified large wallets were not trying to obfuscate their money flows. Over multiple years, they used a handful of addresses that each had conspicuously large balances and unusual transaction patterns, making them very visible to people following the activities of the largest wallets. Their mining efforts were fairly coherently tagged and using enough mining power to result in journalistic articles and widespread recognition. Transactions in and out of this cluster of wallets happened at similar times and with similar volumes. They used only one Bitcoin address standard. With one exception, they didn't use sub-addresses, which could have provided at least some uncertainty as to the confirmed ownership and flow of funds from the outside.
+In other words, the on-chain information that had been public for years was already sufficient for most of the conclusions and observations described in this research update.
+
+Therefore, our discovery of a shared cryptographic weakness that links these wallets does not reveal a lot of new information, aside from the surprising realization of the vulnerable private keys that controlled these funds. That is okay, though. We see our main contribution in this case in discovering and documenting a real-life case of a "billion dollar bug" in software engineering/cryptography, which is rare to encounter first-hand!
+
+We would also like to unambiguously state for the record that we had no involvement with the funds' owner or any withdrawal of funds in this range.
+Additionally, this wallet range has been swept clean of any meaningful remaining fractions of bitcoins in 2023 before we discovered it. Correspondingly, we regard the wallet keys as completely compromised by bad actors and exploited in-the-wild before the publication of this research update.
+
+Please remember that due to the extremely weak 32 bit entropy of the private keys, any motivated actor with the equivalent of `a gaming computer` × `a few days of non-stop operation` worth of compute resources was able to independently brute-force the relevant keys and addresses at any point since 2019.
+To us, this makes it even more surprising that we were, to our knowledge, the first to publish detailed information on these weak wallets. In case you find older public mentions or references to this, please let us know!
+
+## Summary & Outlook
+In this research update, we presented information on a cluster of weak very-high-value wallets that once controlled over a billion USD worth of Bitcoin in aggregated value before most of the funds were withdrawn in December 2020. On-chain transaction information strongly ties these wallets to a large Bitcoin mining project. At this point, it is still unclear if attackers figured out the weak nature of the wallets in time to perform large thefts, or if they only did so some years later with much smaller impact.
+
+Doing this kind of pro bono security research and reporting on the results is a lot of work. Due to a lack of research time on our end, new exploratory wallet-related research is currently on hold. We still have some interesting things to share, but may take a while until we get around to writing them up.
+
+For comments and suggestions on this article, you can contact us [directly]({% link index.md %}#contact).
+<br/>