Back to index
New document: 18 August 2024

Introduction

In this document I describe how to circumvent some kinds of paywalls (not subscriptions full stop) and also how to avoid having to create an account in order to view the page. This will not cover every website and I am in no way going to describe how to steal content (which might or might not be possible at times but it's not something I would ever do).

On the other hand, there are some kinds of paywalls that only are there some of the time (not a full on subscription); the content is free some of the time and not others. If they want to give it away for free some of the time then cannot reasonably complain about it if it is obtained for free; if they want to make it a subscription service then make it a subscription service. Otherwise people are just going to go to other websites or do something like the below (just like with ads).

Similarly there are many websites that force users to create an account (which might spam you or be hard to delete) and this is also very often pointless, especially if it's free and no subscription or anything to pay for, which often it is. Well they claim it's free: whether they try and manipulate you to pay for more after getting you to create an account I do not know but that is a well known tactic to trap people into paying for something that they would otherwise not have paid for. Sadly I would be surprised if they don't do this; this is entirely unacceptable and pure manipulation and I refuse to play that game. If you want to charge for a service then charge for the service and be done with it. Don't play games like little children.

General techniques

These techniques can be tried in whatever order you wish but in some cases one is better than the other. I will give some examples. Later in this document I will show specific websites that are simple to circumvent their ridiculous (free, nothing to pay) attempt to force one to create an account (which really is just a good way to keep track of people which is never okay).

Use the Internet Wayback Machine!

I discovered this a while back and it works quite well, for instance with The New York Times (not in every case but in some) though this can work for other websites too.

The Internet Wayback Machine is a huge decades old web archive where you can find links that have died, regional blocks and various other things, even old games, manuals and things like that They do have a library but I'm not going to get into that as it is irrelevant here. It is extremely useful and very interesting to look at.

But how do you make use of this to bypass paywalls you might ask. Well you copy the URL of the article/page and put it in the form. It looks something like:

Once you type or paste the URL in hit enter and it'll do its thing. It shows it by year; if the year on the time line has no capture it will be grey/white, otherwise it'll have some black above the year. If a capture occurs what you are looking for (in the calendar) is blue. Mouse over the blue and wait a moment for it to show a list of capture times. Click on one and it will open the page as it was captured! Sometimes you do have to try more than one time or date though.

If you see other colours they mean other things, in particular not found (40x errors) or redirects. In the case of redirects it can sometimes help you locate a new location (that is not otherwise available at the time you search).

Of course, this technique only goes as far as what has been captured but it can still work well for different things like a website trying to force you to create a free account (how free I do not know as usually I just close such websites which I discuss in the why bother section below).

Turn on reader mode!

I am afraid I only know how to do this in Safari but I'm sure other browsers have a similar mode and are relatively easy to enable. Anyway, it sometimes will show the whole article (not always but sometimes). In Safari you want to click on the aA text and then select Show Reader (on mobile devices) and in macOS you'll want to click on the icon (at the left of the URL) that looks like:

This will open reader mode and often it will circumvent this crap these websites play. It can be used for other purposes too, of course, but sometimes it can circumvent paywalls and/or forcing you to create a free account (whether it's truly free or not I don't know but that's on them for claiming it).

I do not use Firefox much nowadays but I think the procedure is very similar; I might be wrong though.

Use a text only browser or download the html file

I am not going to explain this one as this is more technical but for those who know how to do these things it can also work at times.

If you use Firefox try the NoScript extension

Often it is a matter of scripts that the website has that can be blocked and blocking them will prevent this nonsense. As this is also more technical I am not going to get into how to do this but it can help. NoScript also can reduce load time, resource consumption and other things besides.

Examples

Here I will give some examples to see how it works, as well as one that I know (discovered quite by accident) that is specific to the website (requiring an account otherwise).

The New York Times

I have chosen this site as I know they play this game. They have an article that is a list of the 100 best books (according to them of course) of the 21st Century: a fair thing to look at if you're an avid reader.

In the case of The New York Times the Internet Wayback Machine works wonders, assuming the page has been captured (sometimes if it's not been captured it can capture it for you but that's beyond the scope of this document). So the list can be found at https://www.nytimes.com/interactive/2024/books/best-books-21st-century.html but after too many reads a month (I think that was it, another reason to not worry about circumventing it as this is just them trying to trap you into it) it is telling me I have to pay or create an account. In fact in this case it's only offering a subscription service and I can scroll down the list but it gives you the idea; in others it wants you to create a free account (which of course I don't know if it's really free but by doing it this way they make me unaware of whether I should be doing this or not: that is entirely on them). Anyway this gives you the idea.

I pasted the URL into the Internet Wayback Machine, moused over a blue bubble and then clicked on a time stamp. In my case it took me to: https://web.archive.org/web/20240708011903/https://www.nytimes.com/interactive/2024/books/best-books-21st-century.html and I see the list fine and well!

Now in some cases images are not captured and it might not look quite the same but the content is usually there.

quora.com

Although they force an account on the main page if you have a link to a specific question and they want you to create an account you can sometimes (always?) get away with just refreshing the page and it will not prompt you again.

Of course the Internet Wayback Machine can also be used to navigate an entire website but only if it captures the entire website; in other words you can only navigate as much as it has captured. In the case of https://www.quora.com and their main page, you might get away with doing this too.

Reader Mode

As I already described this and it does not serve any real good in giving an example page that this will work for, I am not going to show an example, other than remind you that it can be useful at times, both to get past these schemes and otherwise. I am sorry that I only know how to do this in Safari (I guess in Firefox too but I seldom use it nowadays). In other browsers I have no idea and as for Chrome I strongly recommend against it anyway with the way Google has been forcing ads and other crap into it.

But why even bother with their crap?

This depends of course. I have used the Internet Wayback Machine for other purposes too, including salvaging lost writings of a dear friend of mine. But in the case of paywall and/or requiring accounts, it might be that it has specific information you're looking for that is not anywhere else, or it might have some pictures (which sadly are not always picked up).

The real thing is that paywalls and/or forcing account creation is a huge turn off and simply not worth the time: if they want to play that game then they can play that rubbish all they want but no need to engage in it. Those who already have an account or are wiling to pay, both of which are strange to me, won't have this problem anyway, and in some cases it might be worth it to them.

That last paragraph is why I usually don't bother. But sometimes there is something I want to see which is when I go for one of the options above. It's up to each person.

Anyway the list above is some of the techniques you can use to get past these obnoxious schemes, for what ever it might or might not be worth (usually such articles are not worth bothering, in fact, but sometimes they might be).

But how is circumventing paywalls not stealing content?

First of all, some of these techniques work for both and additionally these websites that do this often do not do it until you have reached a limit in a specific amount of time (I have only seen monthly but I am sure there are others). This means they are perfectly fine in giving content away for free (and see below) and anyone who really wants it will just wait it out. Also it is unclear to me if viewing the same article more than once counts against the number of articles. In fact I have once or twice reloaded a page and saw that I lost another view which is not okay. If they're going to do this then they at least ought to be fair.

More to the point, if they require a subscription then I will either choose to subscribe or not but these things are not the same thing. In the case they require it (subscription) to view it there won't be an archive and that's fine. I don't think I have ever paid for such a service as there are plenty others out there but it's an option. What I have listed circumvents content that is either free or free some of the time and so it's not actually theft of content as it's actually freely available (for instance by using a different device or network). If they want it to be only a subscription then they are more than welcome to do that and that would prevent these from working. It would likely turn people away but if that is their business that is their business, in multiple senses of the word.

See also my introduction for more on this subject.