Proof of work is what those modern captchas tend to do I believe. Not useful to stop creating accounts and such, but very effective to stop crawlers.
Have the same problem at work, and Cloudflare does jack shit about it. Half that traffic uses user agents that have no chance to even support TLS1.3, I see some IE5, IE6, Opera with their old Presto engine, I’ve even seen Netscape. Complete and utter bullshit. At this point if you’re not on an allow list of known common user agents or logged in, you get a PoW captcha.
If I was a bot author intent on causing misery I’d just use the user agent from the latest version of Firefox/Chrome/Edge that legitimate users would use.
It’s just a string controlled by the client at the end of the day and I’m surprised the GPT and OpenAI bots announce themselves in it. Associating meaning on the server side is always going to be problematic if the client can control the value
Proof of work is what those modern captchas tend to do I believe. Not useful to stop creating accounts and such, but very effective to stop crawlers.
Have the same problem at work, and Cloudflare does jack shit about it. Half that traffic uses user agents that have no chance to even support TLS1.3, I see some IE5, IE6, Opera with their old Presto engine, I’ve even seen Netscape. Complete and utter bullshit. At this point if you’re not on an allow list of known common user agents or logged in, you get a PoW captcha.
If I was a bot author intent on causing misery I’d just use the user agent from the latest version of Firefox/Chrome/Edge that legitimate users would use.
It’s just a string controlled by the client at the end of the day and I’m surprised the GPT and OpenAI bots announce themselves in it. Associating meaning on the server side is always going to be problematic if the client can control the value
Yeah but Tor’s doesn’t require JavaScript, so you dont have to block at-risk users and opress them further