stories

‘It's simple negligence’ Russian business owners talk about becoming collateral damage in the federal censor's war on Telegram

Source: Meduza

On April 16 and 17, as part of its ongoing efforts to cut off access to the instant messenger Telegram, Russia’s federal censor blocked millions of IP addresses belonging to cloud services operated by Google and Amazon, inadvertently disrupting a variety of Russian businesses, from online schools to courier delivery services. The owners of several affected websites spoke to Meduza about becoming collateral damage in Russia’s war on a chat app.

We noticed some issues this morning [April 17] around 8 a.m. I own several online shops and one of them is hosted on Amazon, and it was with this one that we experienced issues. It became unavailable, but not for everyone — just through certain internet providers. People in Moscow say the site opens fine, but I’m out of town and we can’t open the site here. I put my losses from all of this at around 40,000 rubles [$655].

I’m not planning to move from Amazon to another hosting provider in Russia. My shop was once hosted by several Russian providers, but it was problematic and expensive.

I think many others experienced much bigger losses than I did, but it’s no picnic for me, either. Doing business is quite tough and it’s upsetting to know that everything collapsed because we got caught in the crossfire.

On April 16 around 8 p.m., we started hearing from readers that they were unable to open our app and website. People even sent screenshots to our Twitter account, claiming that access to our site was restricted. We quickly realized that this was because the Amazon Route 53 service (our server supplier) was affected by the block. We started changing our servers and things improved. Now the service is more or less stable.

This doesn’t surprise me. We experienced this a couple of years ago, when another online resource was blocked and it took us with it. We had to change our platform. That’s why we anticipated something similar would happened this time, but we didn’t expect it to be on such a huge scale. Amazon is one of the most vital parts of the Internet. A huge number of services use their infrastructure and getting rid of it now would seriously damage a lot of businesses.

Roskomnadzor has an established tactic, and similar things happened with Zello [a walkie-talkie app blocked in Russia]. They realize what they’re doing. The fact that Roskomnadzor affected others is not a mistake on their part. I think this is how they apply pressure — and not just on Telegram, but on anybody even remotely connected to them. Amazon’s clients suffer, and so it starts losing money. Amazon’s next logical step is to ask Telegram to leave. That’s Roskomnadzor’s logic here.

Late on April 16, we found out that we’d been blocked. First, we tried to solve it by using a proxy server, but we soon realized it couldn’t manage our traffic. Then we informed our 2,500 students with classes scheduled that evening that we had a fallback plan, whereby they could access their lessons through our backup infrastructure. So we started rolling it out on these “tactical” servers and on other hosting sites. Some students managed to sign in, but our backup services were over capacity, and we weren’t able to pull it off completely.

We managed to reestablish access to our site by the morning. Part of our team was still on standby, in case we got hit again by Roskomnadzor, but we didn’t find any more “fire hazards.”

In just one night, our losses amounted to 2 million rubles [$32,760]. We have to reimburse the teachers who couldn’t conduct their planned lessons. The teachers were hostages to the situation by no fault of their own. We sustained other losses, as well: the labor of our developers at Skyeng, and the money needed to buy and rent new servers. We will probably lose some of our clients and teachers because of this incident.

I think Roskomnadzor uses the following logic: you can’t make an omelet without breaking some eggs. They think: if our work affects someone else, so what? When one service goes down, another pops up. It’s simple negligence — a neglect of others’ interests. It’s a classic story of rushing to report back to their superiors that the task was completed, and screw everyone else. I think that now, as always, the simplest explanation is closest to the truth.

From 6 p.m. until midnight on April 16, the majority of our clients could not access the Ptichka site. We blamed ourselves at first, thinking it was some bug, but after we accessed the site by VPN, we realized that we’d been blocked. We use an Amazon hosting service and our IP addresses were hit. Amazon Support helped us solve the issue by changing the IP address. However, at 3 p.m. on April 17 [Roskomnadzor] blocked another subnet of IP addresses, and our site was among them again.

Our couriers rely on Telegram; it’s convenient. Many of our employees still use the messenger because there is nothing better out there. On Sunday night, one of our clients offered to pay for a VPN for our couriers to access Telegram, although it works without it for now.

Amazon is better than your average hosting service. It’s really good, but very regrettably we’ll have to find another one.

On paper, Ptichka didn’t suffer huge losses: while we were down, we lost around 80 orders, so couriers lost 30,000 rubles [$490]. I’m more concerned about a loss of trust. Losing just one client costs us a minimum of 10,000 rubles [$160].

[Roskomnadzor head Alexander] Zharov acts like he’s at war. “We’re going to war,” he thinks. It’s a very strange attitude for a government official — it’s too emotional. It seems like he’s been given carte-blanche to take these actions against Telegram. But it’s now clear that Durov has his own trump cards.

Our mobile app is the heart of our service, which connects people and stores with retailers offering lucrative discounts. We started having problems late on April 16, when our employees were unable to log into our site’s content management system using an Internet connection from MTS. A window popped up saying that the address is blocked. Soon, all other Internet providers blocked our site, as well.

The app was unavailable for a couple of hours, while we moved servers. We lost resources allocated to the purchase of additional servers for Russian users. Without a doubt, this will threaten the growth of our user base, going forward. All in all, the losses were not tremendous, but very unpleasant — particularly for a startup like us.

We have changed our server IP address, but many providers still have the old IP address cached, and we’re waiting for it to get updated.