splinter

joined 2 years ago
MODERATOR OF
[–] splinter@hilariouschaos.com 1 points 3 weeks ago (1 children)

Thanks again for raising this, it should be fixed now.

[–] splinter@hilariouschaos.com 1 points 3 weeks ago

Hey, thanks for the heads up!

I see what you mean, I'll have a look into it.

[–] splinter@hilariouschaos.com 3 points 1 month ago (2 children)

Thanks for letting me know, I was making some tweaks and broke something - it should be back to normal now, please retry!

[–] splinter@hilariouschaos.com 3 points 1 month ago (4 children)

We don't really talk, but I think they all struggle with this in some way. Many instances use Cloudflare for example.

The bots seem to be trying to pull many random posts (e.g. posts that were federated across from other instances)

My guess is they're trying to vacuum up the content. It could also be a low-level DoS attack but it feels too light and inefficient for that.

[–] splinter@hilariouschaos.com 3 points 1 month ago (7 children)

Probably to mine data for training AI. They're going to scrape anything with user content, not just us.

 

Over the past few weeks, our little site has been slammed by web scraping bots.

There is a lot of headroom on the server so the increased load has not led to any noticeable degradation, but the numbers are pretty wild. Here are some stats for the past 2 weeks:

We may be popular, but we're not THAT popular.

The typical solution to something like this is to embrace services like those provided by Cloudflare. I absolutely do NOT want to do this, unlike many of our Fediverse peers. I believe surrendering autonomy and privacy in exchange for security is incompatible with what we're trying to do here.

Instead, I have been working on a PoW challenge system, with the inspiration coming from projects like Anubis (https://github.com/TecharoHQ/anubis)

The idea is simple. Clients need to solve a set of simple cryptographic challenges before they are allowed access to the site content. This rules out the vast majority of simple scrapers, and makes it computationally expensive for the more sophisticated ones. I'm calling it Tollbat because why not.

The trade-off is you will have noticed a Tollbat challenge screen before accessing HC. This could mean a 5-10 second load time every so often. I will continue to tune it to make it as light as possible for real users.

Federation, 3rd party apps like Jerboa and legitimate bots are not affected. So far things seem to be going well, but let me know if you notice any weird behavior.

The load reduction is significant to say the least. Here are more graphs for those who like them - see if you can spot when Tollbat was turned on:

Feel free to ask any questions, I'm happy to answer them.

[–] splinter@hilariouschaos.com 2 points 3 months ago

Things should be back to normal now!

[–] splinter@hilariouschaos.com 2 points 3 months ago (1 children)

Not related, I didn't think anyone was still using ok0.org. When I last had a look, it was bots finding new ways to spam just to be annoying, Asian hookers peddling their wares and crypto bros sharing their latest musings.

I guess it doesn't help that I haven't used it in ages, but it just felt like nothing changed in the two years I've been running it. It felt like a waste so I pulled the plug on it, but I can be persuaded otherwise if you think it's worthwhile.

 

This is a heads up that there will be at least two outages over the next few days, each lasting a few hours.

I need to move some physical hardware around, and there's no way to do that without unplugging it.

If you notice that HC isn't working, there isn't any reason to worry, it's coming back.

[–] splinter@hilariouschaos.com 2 points 4 months ago

Your best bet is a web "app", like https://sqliteviewer.app/

You could also easily get an LLM like Claude or Gemini Pro to write something similar if you had specific requirements. It's pretty simple so a HTML+JS solution is likely enough and it will run on anything in your browser.

[–] splinter@hilariouschaos.com 5 points 4 months ago* (last edited 4 months ago) (1 children)

Not at all, we do not disallow any kind of critical speech, unlike the vast majority of Lemmy instances. The TOS are somewhat broad as it's hard to be specific with these things and discretion always plays a part.

In general, criticism, debate and satire are perfectly fine. Discrimination, personal attacks and insults are not. To give a concrete example, you can say that you don't like religion X for Y reasons as much as you like. You can't say that all people of X religion are idiots and you hope they die.

It's no different than speech about anything else.

Another thing that is relatively unique about our instance is we give communities the right to moderate themselves according to their own rules, so long as they stick to our TOS. This means that you could, for example, start a community dedicated to praising Cthulhu and remove all comments that are even vaguely critical of him from your community, without having the HC admin try to police what you should or shouldn't allow.

These things make us somewhat unpopular in the Lemmy ecosystem where censorship along political lines is seen as desirable, so long as it favors the left. HC is fairly well known, despite our small size, specifically because we refuse to act like the thought police.

 

HC was down for ~9 hours earlier due to a power failure that I wasn't around to mitigate. Bad timing, shit happens, etc.

There isn't any reason to be worried (or celebrate?), HC is still here.

[–] splinter@hilariouschaos.com 2 points 5 months ago* (last edited 5 months ago) (1 children)

This right here sums it up nicely:

The whole shtick with Fediseer is to pressure instances into falling in line under threat of defederation. It's reasonably effective, most small instances would struggle when cut off from chunks of the federated network they're trying to be a part of.

I also think most Admins probably hope to get some money coming in through donations, and you need to be reasonably vanilla and attract new people for that to work.

We really don't need donations to keep running, in fact we don't even ask. I'll be damned if I let some random Internet dude with an over-inflated sense of self-importance tell me who I can or can't allow on the instance I run as a hobby, as to not offend his political sensibilities.

@realcaseyrollins@narwhal.city I meant this as a reply to your post but failed.

[–] splinter@hilariouschaos.com 1 points 6 months ago

Definitely, post as much as you like, it will federate automatically to all the linked instances.

A word of advice is to stagger the content a bit (e.g. give it 10-20 minutes between posts) - I personally don't care but other admins might perceive posting too much, too quickly as spam. You or your community can still be banned locally on other instances.

 

It has been almost a year since we disabled downvotes on HC. @Alice@hilariouschaos.com and I are interested to check in and read the room.

In short, downvotes affect how content is displayed here, and certain types of content gets reliably downvoted on Lemmy. For a comparison, look at:

https://hilariouschaos.com/c/Conservatives vs https://lemmy.world/c/Conservatives@hilariouschaos.com

Downvotes are often not used in good faith, and content will be buried at the bottom based on them. However, they are also a valid form of expression and I doubt anyone here is fragile enough to be affected by them.

What are your thoughts?

 

TL;DR: You might have more people to talk to, we are now federating openly.

The longer version: Until now, we have been maintaining a list of instances that we are federated with. This was done out of an excess of caution, as it's easier to keep tabs on who you link with.

The downside of this approach is that we haven't been linking with many Fediverse instances that we simply weren't aware of.

We have now switched to open federation and will instead maintain a blocklist, reserved for instances that host shit we really don't want coming across to us (e.g. loli porn).

This of course doesn't change other instances defederating from us, we're scary that way.


Thoughts?

view more: next ›