Over the last two years, security researchers reported to us several vulnerabilities in the BitBox01, and we found some internally. Through this process we gained some experience with assessing their severity. With our recent responsible disclosures to our competitors, we realized that the industry of hardware wallets lacks a common standard for severity assessments. This blog post outlines how we evaluate vulnerabilities at Shift. By publishing our framework, we hope to get feedback and to move the industry forward. We are aware that our classification isn’t perfect, but we haven’t yet found a more suitable model.

Threat model of hardware wallets

Before we dive into potential vulnerabilities, let’s first discuss why a hardware wallet is still the best means to hold your cryptocurrencies despite reported vulnerabilities. Devices like computers and mobile phones, which could be used to store a wallet’s private keys, run highly complex general purpose operating systems with millions of lines of code and have weak security domains. Unlike those devices, hardware wallets typically only run dedicated firmware without any user-provided code and are built for a single purpose: keeping your private keys private. This reduces the attack surface a lot, which means attackers have fewer options to steal your private keys and thus your funds. In order to be secure, you should always trust the screen of your hardware wallet over the information displayed on your computer and verify carefully what is displayed on your hardware wallet. The manufacturer can add exceptions to this rule, for example when your computer warns you about a failed device authenticity check (i.e. you may have a potentially counterfeit device). Protection against malware on your computer is the bare minimum a hardware wallet should provide in terms of security. Most hardware wallets also aim to protect your assets from theft when your device is stolen by encrypting sensitive information and requiring a PIN or password to unlock the device.

While the above might seem trivial, there are many situations where the covered security threats are not clearly defined. This is why it’s important that hardware wallet companies publish a threat model, where they explain what they protect you against and — more importantly — what not. Here are some examples to illustrate why the scope of attacks and countermeasures is not trivial and needs to be made explicit: What are the mechanisms in place to prevent tampering in the supply chain before the device reaches you? Are evil maid attacks in scope or out-of-scope? How can you be sure that the firmware binary you install on your device is the one for which the code is published and which everyone else is receiving? Or is the company being forced to sign a malicious firmware just for you? Can you trust the receive address for a multisig account that is displayed on your hardware wallet? Are you still secure if the cosigner of a multisig setup is malicious or compromised? We will come back to the last two questions in a future blog post. You find the answers to all the other questions regarding the BitBox02 in our threat model. The only other threat model we are aware of is the page on security threats by Trezor. Ledger lists which vulnerabilities are in-scope and which are out-of-scope in their bug bounty program.

Severity = impact * scalability

The risk of a particular vulnerability for users, and thus its severity, is determined by multiplying the impact by the likelihood of the exploit. In general, the easier it is to perform an attack at scale, the more likely it is that you are affected. In particular, since it is easier to distribute malware than to steal a large number of used devices, you are more likely to be a victim of software-based attacks, also known as remote attacks, than hardware-based attacks. However, this depends a lot on your personal situation. If you are a high-profile individual or bring your hardware wallet to cryptocurrency meetups and forget it there, then the situation can be different for you. For simplicity, we assume here that the likelihood of you being attacked based on a particular vulnerability scales proportionally with the overall number of users that the given attack could target simultaneously.

Impacts of vulnerabilities

By finding and exploiting a vulnerability, an attacker can achieve one of the following outcomes, listed in order of decreasing severity:

  • Seed extraction/injection: The worst outcome is when the attacker learns the seed of your wallet, from which all keys are derived. An attacker can steal your funds immediately or wait until you accumulate more coins and tokens; then withdraw all your funds when you least expect it. This delay from the attack to the time of exploitation makes it harder to link the attacker to the attack and find the vulnerability. Furthermore, unlike in the case of stealing funds, where the problem is typically in the transaction signing implementation of a particular coin, learning the seed compromises all your funds and potentially other features like U2F/SSH authentication or email signing/decryption. An attacker knows the seed if they manage either to extract it from the hardware wallet or to inject it into the hardware wallet when the user sets up the wallet.
  • Key extraction: Instead of extracting the seed, an attacker could succeed in extracting an individual key. This could be due to a faulty signing algorithm (such as reusing the nonce) or due to a side-channel. The consequence of extracting the key varies by cryptocurrency. With Bitcoin, you should not reuse keys (and thus addresses) and therefore the damage is limited to the coins you hold with that particular key. With Ethereum and ERC20 tokens, your account consists of a single key, which makes a key compromise as severe as a seed extraction.
  • Theft: Without extracting any keys from the hardware wallet, an attacker might succeed in stealing your funds, typically either when receiving or spending them. When receiving coins, it’s very important that you let the hardware wallet display the receive address and then confirm the address with the sender through a different channel than your computer. Otherwise, a malware can simply replace the receive address displayed by your wallet software or sent by you to the sender. When a hardware wallet is involved, thefts can still happen when what you see and confirm on the hardware wallet is not what you intended to confirm with no fault of your own. Due to an error, the firmware might not properly verify the received data or might not display all relevant information to keep your funds secure. We will cover examples of this in a future article about pitfalls of multisig.
  • Ransom: Instead of stealing your coins, an attacker might achieve to hold your funds hostage so that you can no longer spend them without the attacker’s assistance. While less convenient than stealing your coins directly, the concept of ransomware exists for quite some time and should be taken seriously. An example of a ransom attack is the misuse of the unlimited number of key derivations possible from the seed of the wallet. If the hardware wallet does not enforce enough restrictions, an attacker can provide a derivation path for the receive or change address to the hardware wallet that is unknown to the user and delete it immediately afterwards on the user’s system. According to the defacto standard defined in BIP32 and used by virtually all wallets, there are 2³² child keys for each key. This means the space of all keys in Bitcoin is surpassed with only eight unrestricted derivation levels. If an attacker manages to bury your funds so deep in the derivation tree, it becomes easier to brute-force Bitcoin private keys, which is computationally infeasible, than to recover your funds from the seed.
  • Destruction: While permanently destroying your access to your funds is less attractive for a financially motivated attacker, it might still be an attractive attack for a disgruntled employee or competitor. While I’m not aware of such an attack on any hardware wallets, it is also something they should protect you from. If the firmware is not carefully written, a glitch induced during a critical operation such as the key derivation for the change address might be able to send some coins into a digital nirvana. Please note that erasing the seed from a hardware wallet does not qualify as permanent destruction: The user should still have a backup from which they can recover their funds.
  • Loss of plausible deniability: Some hardware wallets have features that allow you to unlock a different wallet under duress. An example for this is the optional passphrase in the BIP39 standard, whereby each passphrase derives a valid wallet. If you deposit some coins in a second wallet beforehand and unlock that one when you’re threatened, you might be able to hide the existence of other wallets on the same device. If a hardware wallet provides a similar functionality not based on BIP39 and which is not carefully designed, measuring the execution speed or probing communication on the physical wires inside the hardware wallet could reveal whether you are using the main or a hidden wallet, destroying plausible deniability for you.
  • Loss of privacy: While not putting your funds in danger, vulnerabilities in the API of the hardware wallet or the persistence of information could reveal your balance to an attacker, for example by returning an extended public key without having to unlock the device first.
  • Bricking of device: The least severe consequence is a bug or a weakness that makes your hardware wallet permanently unusable. While this would certainly be annoying, you can recover your funds from your backup on a new wallet. The manufacturer could even replace bricked devices free of charge. For this to be nothing more than an annoyance, though, it is absolutely crucial that you can recover your funds from your backup. We recommend practicing this procedure before storing funds on your hardware wallet and then routinely afterwards.

Scalability of vulnerabilities

Vulnerabilities can differ drastically with regards to how easy they are to exploit at scale. As mentioned above, the more users that can be attacked, the more likely you are one of them. Exploits where the attacker doesn’t have to be physically present are easier to scale. The following attack vectors are listed in order of decreasing scalability and thus decreasing severity.

Remote attacks (performed with software)

In the hardware wallet industry, attacks that can be performed from the computer to which the hardware wallet is connected are considered to be remote: no physical access to the device is required as the exploit happens through the API of the hardware wallet. How you get malicious software on your computer is outside the scope of this article, but, as noted in the threat model above, this is the main attack vector hardware wallets should protect you from. Remote attacks are really bad as they can scale quickly. For example, the download server of the manufacturer’s wallet software can get compromised and most users would likely install a malicious app as they don’t verify the signatures on the downloaded binaries. The following distinctions depend on the logic of the hardware wallet and do not apply to all products equally.

  • Without unlocking: The most severe remote attack is when the device does not even have to be unlocked in order to be exploitable. While a user typically connects a hardware wallet to a computer in order to unlock and use it, an exploit without unlocking might also be possible via websites that you visit through the U2F interface, which quite a few hardware wallets support. Additionally, depending on the nature of the vulnerability and the architecture of the hardware wallet, it could put further, unlocked wallets on the device at risk (e.g. the ones you created for plausible deniability). Exploits without unlocking are also very useful for attackers who gain physical possession of your hardware wallet.
  • Unlocked without user interaction: If the vulnerability is exploitable without the user having to confirm anything on the device, like a transaction to be signed or the display of a receive address, then the vulnerability exposes you to the risk of being compromised before upgrading to the fixed firmware. In case of such vulnerabilities, all users of the affected hardware wallet should upgrade the firmware of their devices as soon as possible before the exploit is performed in the wild.
    Whoever detected the vulnerability should coordinate the public disclosure with the vendor to give users potentially more time to upgrade if the public patch does not make the attack apparent anyway. This also applies to remote attacks without unlocking. As a user of a hardware wallet, make sure that you are subscribed to the manufacturer’s mailing list for security announcements, in particular for situations like this one. (Click here to subscribe to Shift’s mailing list.)
  • Unlocked with user interaction: If the vulnerability can only be exploited when the user performs a particular action such as signing a transaction, then users can simply upgrade the firmware before performing the affected action the next time. Therefore, if the wallet software prompts you to install a newer firmware with security fixes, make sure you always upgrade your device before doing anything else on it.

Supply chain attacks

A hardware wallet or its accessories like the USB cable can be tampered with before it reaches you. This kind of attacks are significantly costlier to perform than software-based remote attacks, since it involves physical modification or replacement of components. Depending on where in the supply chain the attack is performed, such attacks could also scale quite easily. However, as sophisticated supply chain attacks are hard to protect against, they are often not covered in the threat model of hardware wallets.

Local attacks (performed with hardware)

If, in order to exploit a vulnerability, the attacker needs physical access to the device after the user has already created a wallet on it, then we talk about a local attack. Please note that the boundary between local attacks and supply chain attacks is vague, as many local attacks can be transformed into supply chain attacks with some effort.

  • Non-invasive without returning to user: If a found or stolen device doesn’t have to be cracked open and doesn’t have to be returned to the user in order to exploit a vulnerability, then such an attack is easier to scale than the other local attacks as the device just has to be plugged in. Once built, no knowledge about embedded systems is required and thus the equipment to perform the attack can become attractive on the black market.
  • Invasive but non-destructive: As the casing of most hardware wallets is plastic, they are not difficult to open or at least penetrate so that an attacker can, for example, intercept and change the communication between components in the hardware wallet. The device might get damaged but is not destroyed in the process, so the attacker gets many attempts to perform an attack, unless this is prevented by some brute-force protection mechanism. However, since the device is locked for the attacker, this brute-force protection mechanism is often what they are trying to bypass.
  • Invasive and returned to user: If the hardware wallet or its accessories are not just modified but also returned to the user, then this is known as an evil maid attack. It’s quite difficult to secure the device against this kind of attacks, when performed well, and for this reason these attacks are often excluded from the threat model of hardware wallets. To some degree users can protect themselves against evil maid attacks, or at least make them more difficult to perform, by carefully inspecting the device and by using preventive measures.
  • Invasive and destructive: Last but not least, there are intrusions that are destructive such as decapping a chip. This is where the promise of secure chips come in, as they are designed to self-destruct before their memory can be read out. If a hardware wallet doesn’t have a secure chip or doesn’t use it properly, it cannot avert such attacks. The security of the hardware wallet then depends solely on the strength of the user-chosen device password.

Other aspects to address

The above scales for the impact and scalability of vulnerabilities are sometimes too simplistic. Often there are additional aspects to consider on a case-by-case basis. The following questions are examples of aspects that should also be addressed by the vendor if applicable when informing its users about a vulnerability:

  • Which coins are affected by the vulnerability?
  • Which action is affected? In case of receive address or transaction signing, is singlesig, multisig or both affected?
  • Can users protect themselves by choosing stronger passwords, for example in case of brute-forcing attacks?
  • Is the vulnerability only present during the creation of a wallet or also while using the wallet? And if you were a victim before upgrading to the firmware that fixes the problem, are you still a victim after the firmware upgrade? If yes, you have to move your funds to a new wallet. An example attack for this case is the injection of a seed known to the attacker when creating the wallet.
  • Can the attack be avoided if the user is educated regarding what to pay attention to? Some solutions work better in theory than in practice but could still be the only possible mitigation. An example for this are the LED blinks of the discontinued BitBox01, which we made distinguishable to let users know what they confirm when touching the device. As a consequence, having well-informed users is part of our threat model for the BitBox01.
  • Can the vulnerability only be exploited when the device is directly connected to a (compromised) computer or also when it’s used in a disconnected, so-called air-gapped mode of operation? Remote attacks that do not require user interaction are not possible when the device is not directly connected to a computer and the user has to transport all messages back and forth, for example, with a microSD card or QR code readers. My impression, though, is that misuses of the API with confirmation by the user when the device is unlocked are much more common than the other remote attacks and an air gap does not help with those.
  • More for the curiosity of the community and a potential forensic analysis: Does the attack leave traces on the public blockchain or somewhere else?
  • And last but not least: Can the vulnerability be patched?

Severity assessment

The adjectives typically used to label the severity of a vulnerability are in order of decreasing severity: critical, high, moderate, and low. There can be reasonable disagreement about how to apply these labels to vulnerabilities, or in other words, how to quantify the impact and scalability of attacks. Given the considerations in the previous sections, we define them along the following lines:

  • Critical: All remote attacks up to and including locking the user’s funds for ransom because they can be performed by a financially motivated attacker at scale with zero marginal costs.
  • High: Attacks that permanently destroy access to your funds. We classify such vulnerabilities to be of high severity instead of critical because they are more likely exploited for targeted attacks, as there is no direct financial gain for the attacker. Non-destructive attacks on stolen devices up to and including the consequence of destroying funds also fall in this category.
  • Moderate: Loss of plausible deniability or loss of privacy. We consider them to be of moderate severity because we don’t guarantee the plausible deniability and because the wallet software typically learns the balance and transaction history of your accounts anyway. Assuming your computer is compromised implies assuming that the attacker knows your balance. Destructive attacks that result in theft of your coins also belong here, unless the hardware wallet has no secure chip, in which case they are likely excluded from the threat model.
  • Low: Everything else, including for example supply chain and evil maid attacks, which are mostly excluded by our threat model. The situation is different when a mechanism fails that was put in place to make such attacks harder and which was advertised as such.

Since many vulnerabilities have aspects that make them “special” as highlighted by the questions in the previous section, we reserve the right to deviate from these rules in our bug bounty program at our sole discretion. Furthermore, it is often not clear what the impact of a potential vulnerability is, especially in the case of memory issues such as leaking some bytes.

Our security assessment table. * means that the severity depends on whether an advertised mechanism can be circumvented. Please note that additional factors influence the severity evaluation of a particular vulnerability.

Let us know what you think about this framework. We expect this framework to evolve over time and hope it helps drive the conversation forward about hardware wallet security.

Acknowledgments

I want to thank Jad Zeidan for developing and maintaining our original scoring system, and Sebastian Küng and Douglas Bakkum for giving valuable inputs to this blog post.

‌Don’t own a BitBox yet?

The BitBox02 comes in two editions: the Multi edition that supports multiple crypto assets and can be used as a second-factor authenticator. And the Bitcoin-only edition, which features a radically focused firmware: less code means less attack surface which further improves your security when only storing Bitcoin.

Grab one in our shop!

Frequently Asked Questions (FAQ)

What is the purpose of this article?
This article outlines how Shift evaluates vulnerabilities in hardware wallets, aiming to standardize severity assessments in the industry.

Why are hardware wallets considered secure?
Hardware wallets run dedicated firmware without user-provided code, reducing the attack surface. They are designed to keep private keys private, unlike general-purpose devices.

How does Shift define the severity of a vulnerability?
Severity is determined by multiplying the impact by the likelihood of the exploit. It's categorized as critical, high, moderate, or low based on various factors.

What are the potential impacts of vulnerabilities?
Impacts range from seed extraction/injection, key extraction, theft, ransom, destruction, loss of plausible deniability, loss of privacy, to bricking of the device.

How does Shift classify the severity of vulnerabilities?
They use labels like critical, high, moderate, and low based on the impact and scalability of attacks, with considerations for special cases.

Shift Crypto is a privately held company based in Zurich, Switzerland. Our international team of specialists across engineering, cryptosecurity and Bitcoin core development build the BitBox products and provide consulting services. The BitBox02, a second generation hardware wallet, equips individuals to easily store, protect, and transact cryptocurrencies. Its companion, the BitBoxApp, provides an all-in-one solution to securely manage your digital assets with ease.