GreatFire.org now testing VPN speed and stability in China

Submitted by charlie on Tue, Jul 05, 2016

There is a commonly held belief in China that if you have a VPN that works then you should keep quiet about it. In terms of freedom of access to information, the problem with this approach is that access to knowledge suddenly is a secret. Today we are launching a project that we hope will destroy that model.

Our newest website, Circumvention Central (CC), aims to provide real-time information and data about circumvention solutions that work in China. Since 2011, we have been collecting data about blocked websites in China and now we will add data about the effectiveness of VPNs and other circumvention tools.

We are launching CC with four main objectives in mind.

Our first objective is to help to grow the number of Chinese who circumvent censorship restrictions in China. By sharing our information and data about these tools, we hope to show a wider audience which circumvention tools are working.

Our second objective is to improve the circumvention experience for users in China by bringing transparency to tool performance. We will measure these tools on speed (how quickly popular websites are loaded) and on stability (the extent to which popular websites load successfully).

The speed test that we have developed aims to reflect the real user experience. While a visitor to the site runs a test, in the background, their browser is downloading resources from the ten most popular websites in the world. According to Alexa(link is external), these sites are Google, Facebook, YouTube, Baidu, Amazon, Yahoo, Wikipedia, QQ, Twitter and Microsoft Live. The speed reported is simply the size of the file downloaded divided by the time taken to download it. We also verify whether the downloaded resource is complete. If the contents returned are incorrect or if the download fails to complete within 40 seconds, we mark it as failed. This data is then used to calculate our other main indicator - stability.

Other speed tests typically just measure download and upload speed by sending a couple of requests to the speed test server itself. The reported speed is not reflective of the user experience because normal browsing involves frequently sending lots of requests (not downloading or uploading one big file) to many different servers, not just one.

Our second indicator - stability - is typically not tested by other services at all. A healthy internet connection anywhere is supposed to deliver 100% stability, unless someone pulls the cable during the test. But using a circumvention tool in China is different. At any point, the connection can become unstable and/or go down completely. Individual requests may fail depending on the size of the request, the final destination and how they are proxied. Thus comparing service stability is just as important if not more important than comparing speed.

You can use our speed test with any circumvention tool, whether it is listed on our site or not. VPN users from around China will be able to test their tool on the site and those results will be added to our data. This data will be available for all to see, inlcuding data about GreatFire's own circumvention tool, FreeBrowser(link is external). Real-time testing in China is important because VPNs get blocked and unblocked all of the time. We welcome any and all feedback on the test process itself. Tech savvy users can inspect how we conduct our tests by reviewing our javascript code directly on the test site.

We especially invite developers of circumvention tools to provide us feedback on the testing process. Our third objective is to help these developers improve their products so that more choice is available for the Chinese consumer. Furthermore, the more tools that work, the harder it will be for the Chinese authorities to crackdown on circumvention.

As internet users in China know, the authorities have stepped up their attacks on circumvention tools over the past 18 months. CC is part of our attempt to fight back. Instead of making this a game about secrets, we want to encourage people to share information about circumvention tools that do work.

Our fourth objective in launching CC is to generate revenue for GreatFire.org. At the moment GreatFire relies on the kindness of individuals who send us donations and from a host of grant-making organizations around the world. We want to reduce our reliance on these organizations and set GreatFire.org on a path of self-sustainability. Visitors to the CC site will be able to purchase any paid-for tool that we currently test. GreatFire will act as a reseller of these tools in China and as such we will be given a portion of each sale by the VPN providers themselves. Users need not be based in China to purchase a circumvention service.

It is not entirely true that reliable VPNs are largely kept as a secret in China. There are a host of websites that provide “VPN reviews” and “VPN ratings”. But anybody who has visited those sites can quickly see that they act as referral and marketing sites for a small subset of VPN providers and that most of those services do not work in China.

You may have seen the great list of VPNs(link is external) put together by That One Privacy Guy. We took that list and recruited a team of testers across China and tested almost every tool to see what works and what does not. It will come as little surprise to those in China to know that a very small number of these tools actually work in China.

Until CC, nobody has provided public information about the effectiveness of circumvention tools in China. Many have provided misinformation about what works and what does not work. Some VPN providers have also famously encouraged their customers to “keep quiet” about the effectiveness of their solutions. On the contrary, we encourage everyone who reads this post and visits our new website to share this information with those who they think could benefit from purchasing a circumvention tool.

If you are a VPN provider and you want to get added to the list please contactcharlie.smith@greatfire.org(link sends e-mail)