Telegram is also known for its security. The app’s encrypted Secret Chat function—“meant for people who want more secrecy than the average fella,” according to the organization’s website—earned a seven out of seven on the Electronic Frontier Foundation’s Secure Messaging Scorecard. And in February, Telegram announced that a $300,000 competition to decipher the Secret Chats would end without a winner.
For years now, though, cryptographers have been warning that the application has strange guts. Telegram uses a custom protocol, MTProto, to secure its messages, a decision that breaks a cardinal rule of cryptography: Don’t try to design your own, not if you can use an established approach instead. In 2014, for example, when Telegram’s more popular competitor WhatsApp decided to upgrade its security, it did so by working with Open Whisper Systems to implement that organization’s well-regarded TextSecure protocol.
In that light, Telegram’s choice to pursue an original design stands out. “They came up with something totally new, and a little weird, and mysterious,” the Johns Hopkins University professor Matthew Green said. “It’s like coming up and finding a submarine where the doors are made out of Saran Wrap. I guess if you use enough Saran Wrap you could build a pretty secure submarine; it doesn’t mean that it’s going to sink. But it does mean it’s not something I would want to trust with my life.”
The more novel an approach to encryption is, the less time researchers have had to spot vulnerabilities; older algorithms wear the scars of past decryption attempts like armor. What’s more, the more unusual or non-standard an encryption scheme is, the more difficult it can be to identify points of failure that are hidden in a tangle of unconventional choices. As a result, Telegram’s decision to “roll its own crypto” had long drawn scrutiny. Now, findings published in December give critics fresh reason for suspicion.
Claudio Orlandi is an associate professor at Denmark’s Aarhus University, where, along with the graduate student Jakob Jakobsen, he recently audited Telegram’s source code. While the application claims on its website that MTProto helps “achieve reliability on weak mobile connections as well as speed when dealing with large files,” Jakobsen was skeptical that the security tradeoff made any sense. “There should be lots of provably secure approaches that work just as nicely on a smartphone,” he said in a phone interview. And when the pair went looking for a flaw in the protocol, they found one.
In particular, Orlandi and Jakobsen found that MTProto lacks a property called “indistinguishability under chosen ciphertext attack,” or IND-CCA. That standard is meant to imply that an attacker trying to decipher a message can’t wring information out of the encrypted version. Formally, it can be tested against a sort of game. If an adversary asks Telegram to encrypt one of two messages, and receives the encrypted version in return, it shouldn’t be possible to guess which message was enciphered, at least not with better odds than chance—even if you give the adversary access to a “decryption oracle” that can crack any message secured by the same algorithm.