this post was submitted on 21 Mar 2025
266 points (99.6% liked)

Selfhosted

49458 readers
724 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago
MODERATORS
 

I just started using this myself, seems pretty great so far!

Clearly doesn't stop all AI crawlers, but a significantly large chunk of them.

top 50 comments
sorted by: hot top controversial new old
[–] randomblock1@lemmy.world 24 points 3 months ago (1 children)

Why Sha256? Literally every processor has a crypto accelerator and will easily pass. And datacenter servers have beefy server CPUs. This is only effective against no-JS scrapers.

[–] poVoq@slrpnk.net 19 points 3 months ago* (last edited 3 months ago) (1 children)

It requires a bunch of browser features that non-user browsers don't have, and the proof-of-work part is like the least relevant piece in this that only gets invoked once a week or so to generate a unique cookie.

I sometimes have the feeling that as soon as some crypto-currency related features are mentioned people shut off part of their brain. Either because they hate crypto-currencies or because crypto-currency scammers have trained them to only look at some technical implementation details and fail to see the larger picture that they are being scammed.

[–] swelter_spark@reddthat.com 2 points 3 months ago (1 children)

So if you try to access a website using this technology via terminal, what happens? The connection fails?

[–] crmsnbleyd@sopuli.xyz 2 points 3 months ago (1 children)

If your browser doesn't have a Mozilla user agent (I.e. like chrome or Firefox) it will pass directly. Most AI crawlers use these user agents to pretend to be human users

[–] swelter_spark@reddthat.com 1 points 3 months ago (1 children)

What I'm thinking about is more that in Linux, it's common to access URLs directly from the terminal for various purposes, instead of using a browser.

[–] crmsnbleyd@sopuli.xyz 1 points 3 months ago

If you're talking about something like curl, that also uses its own User agent unless asked to impersonate some other UA. If not, then maybe I can't help.

[–] enemenemu@lemm.ee 23 points 3 months ago (2 children)

Meaning it wastes time and power such that it gets expensive on a large scale? Or does it mine crypto?

[–] zutto@lemmy.fedi.zutto.fi 35 points 3 months ago* (last edited 3 months ago) (21 children)

Yes, Anubis uses proof of work, like some cryptocurrencies do as well, to slow down/mitigate mass scale crawling by making them do expensive computation.

https://lemmy.world/post/27101209 has a great article attached to it about this.

--

Edit: Just to be clear, this doesn't mine any cryptos, just uses same idea for slowing down the requests.

load more comments (21 replies)
[–] Goretantath@lemm.ee 16 points 3 months ago

I think the maze approach is better, this seems like it hurts valid users if the web more than a company would be.

[–] danielquinn@lemmy.ca 13 points 3 months ago (1 children)

It's a rather brilliant idea really, but when you consider the environmental implications of forcing web requests to ensure proof of work to function, this effectively burns a more coal for every site that implements it.

I don't think AI companies care, and I wholeheartedly support any and all FOSS projects using PoW when serving their websites. I'd rather have that than have them go down

[–] computergeek125@lemmy.world 11 points 3 months ago (1 children)

Found the FF14 fan lol
The release names are hilarious

[–] Couldbealeotard@lemmy.world 9 points 3 months ago (1 children)

What's the ffxiv reference here?

Anubis is from Egyptian mythology.

[–] PumaStoleMyBluff@lemmy.world 6 points 3 months ago

The names of release versions are famous FFXIV Garleans

[–] mannycalavera@feddit.uk 4 points 3 months ago

Upvote for the name and tag line alone!

[–] drkt_@lemmy.dbzer0.com 1 points 3 months ago (1 children)

Anubis is provided to the public for free in order to help advance the common good. In return, we ask (but not demand, these are words on the internet, not word of law) that you not remove the Anubis character from your deployment.
If you want to run an unbranded or white-label version of Anubis, please contact Xe to arrange a contract.

This is icky to me. Cool idea, but this is weird.

[–] LiveLM@lemmy.zip 18 points 3 months ago (2 children)

...Why? It's just telling companies they can get support + white-labeling for a fee, and asking you keep their silly little character in a tongue-and-cheek manner.
Just like they say, you can modify the code and remove for free if you really want, they're not forbidding you from doing so or anything

[–] bitcrafter@programming.dev 9 points 3 months ago

Just like they say, you can modify the code and remove for free if you really want, they’re not forbidding you from doing so or anything

True, but I think you are discounting the risk that the actual god Anubis will take displeasure at such an act, potentially dooming one's real life soul.

[–] TheOakTree@lemm.ee 3 points 3 months ago (1 children)

Yeah, it seems entirely optional. It's not like manually removing the Anubis character will revoke your access to the code. However, I still do find it a bit weird that they're asking for that.

I just can't imagine most companies implementing Anubis and keeping the character or paying for the service, given that it's open source. It's just unprofessional for the first impression of a company's website being the Anubis devs' manga OC...

[–] F04118F@feddit.nl 9 points 3 months ago* (last edited 3 months ago) (1 children)

It is very different from the usual flat corporate style yes, but this is just their branding. Their blog is full of anime characters like that.

And it's not like you're looking at a literal ad for their company or with their name on it. In that sense it is subtle, though a bit unusual.

[–] TheOakTree@lemm.ee 3 points 3 months ago

I don't think it's necessarily a bad thing. Subtle but unusual is a good way to describe it.

However, I would like to point out that if it is their branding, then the character appearing is an advertisement for the service. It's just not very conventional or effective advertising, but they're not making money from a vast majority of implementations, so it's not very egregious anyway.

load more comments
view more: next ›