historicaldocuments

But for the people doing that it puts their comment first. If there was a better metaphor for shitting all over a commonly agreed upon standard that works well for everyone so you can be at the front then I can't really think of it.

https://seirdy.one/posts/2021/01/12/password-strength/

Even if that angry guy talking about compression algorithms for chess (I see he didn't propose one) is right, there's still Go.

Her favorite word is "passport."

It’s this kind of commentary that distorts public perception.

I'll stop then. Have a nice day.

I’ve been wondering ever since what can they do if they had it?

Ask for pictures of all his other paperwork so they can finish onboarding him at his new job.

Not on to advocate for internet open just for people privileged to have access in their homes.

Gotcha.

Are you calling Firefox a scraper then?

I'm sure there's a way to wire it up that way. It's bound to have a Turing complete language in it somewhere and by definition it has all the tools to copy down files from the internet. If I were going into the scraping business I'd personally start with a scraper and give it a Firefox user agent. In short, scraping is about intent as much as tooling, and there's no way to convey and enforce intent over HTTP. From a practical standpoint there's a spectrum of activities that approximately go from Browsing to Scraping to DDoS in that order that really only differ in terms of scale.

Why would you call “2” a scraper and “1” otherwise?

If I'm hosting content then if #2 is a scraper and #1 is a browser, and they don't cost me anything different then I don't really care (someone else who's done that kind of stuff can weigh in if I'm wrong). What a lot of sites seem to be reporting is that they have a small, steady hosting cost per month until WHOOM! an AI company scrapes their whole site and their hosting cost jumps that month, then the next month and the next as well because companies that would destroy copies of rare books to feed to their training don't care. Facebook got called out sort of recently for scraping some entire pornsite's content. It sounds like what you're describing is just timeshifted browsing. I doubt anyone notices unless you're really describing what you're trying to do incorrectly.

Nonsense. The author was sloppy in their phrasing. That is not what he means by “follow robots.txt”. By “follow”, he means “obey”, not visit.

You'll often hear/read "follow" used by itself to imply "follow the rules," in this case "follow the rules specified by robots.txt."

From a technical standpoint an LLM recommends trying this code snippet to make httrack ignore the robots.txt file:

httrack "http://example.com/" -O "/path/to/save/files" -%v -s0

I haven't tried it personally.

It’s important to note that there’s no difference at all from the server’s side, other than an implication of how often scraping happens or how the data is being used.

I thought about maybe checking server logs for what files a scraper might skip just for bandwidth or whatever, but I don't know much about that world.

Which is why the GUI restriction was added to robots.txt in the first place.

I had no idea there was a GUI restriction since I stopped digging pretty quickly about it. Upon further reading it's apparently been used in several lawsuits successfully as a sort of "no tresspassing" sign with some court precedent to it. Neat.

A lot of the world runs on gentlemen's agreements and handshake deals. There's no real way to have a robots.txt for a public facing internet without starting an arms race with the server having rate limiting, metrics to check and see if someone's going alphabetically (for example), captchas on every page to verify it's a human, which the scraper will counter with a randomized scrape list, LLMs to pass captchas, etc. The server could institute some sort of login, but that defeats the purpose of an open internet. So robots.txt can't be enforced from a technological standpoint. One could glean information from who's ignoring it and obviously scraping, but that's about it.

It’s not scraping. It’s fetching.

There's no technological difference between these if the scraper wants. I was going to deep dive htrack because I've never heard of it before, but stopped when I saw some of it's options. It's straight up a website scraper. It not only has options to ignore robots.txt, but it also has a setting that apparently uses robots.txt to know where to scrape! Wrap a systemd timer around it and start working on your own archive.org.

without any steps in between for something as mild as someone trying to touch an object that belongs to him.

Again, you're assuming the offender's intent a priori, and assigning a low value of someone else's property for them which you don't have the right to do. Everyone has a smart phone these days, but you can't know whether the loss of that phone is a minor inconvenience to a person or whether it's their "home address" with the only contact information they have. That places a major value difference to the property to the owner that won't be visible to anyone else.

And we should always assume the worst of people and immediately attack them?

Certainly not, and from the context of the ad that's not what happened there. In the ad world the owner of the clubs didn't create the set of conditions that led to the grabber's actions, the grabber did.

Sane people, again, don’t go around assaulting other people at the slightest provocation. That you apparently do, or fantasize about doing so, is a sign that you are quite mentally disturbed.

Again, "slightest provocation" is relative. That's why we're having this long ass conversation: you've obviously never been down to your last two nickels and watched something critical fail because of the preventable actions of someone else. Lucky you, congratulations, and I hope that you continue to have that sort of luck going forward because you obviously don't have the toolset developed to handle that kind of situation. Your strawman and ad hominem attacks aside I'm happily sitting at zero altercations and hope that continues on for the rest of my days.

I also no longer have to wonder if long, monotonous safety briefings are really necessary; that nobody truly could be so stupid as to ignore all warning signs, social cues, pictographs of danger, lights, sirens, bells and whistles, because I can remember that you're out there somewhere oblivious to the world and its consequences and armed with all the world's knowledge about all possible events and outcomes and have them prejudged based on your interpretation of violence.

And you base this off of… my belief that one should ask people to stop touching their stuff rather than immediately run up and beat the shit out of them?

I base this off of what you've said which leads me to believe that you think other people should act that way in that situation, but you'd do something different if it was you. We also don't know the intent of the grabber.

Were you dropped on your head as a child, or do you come by this level of violent sociopathy naturally?

Your histrionics may play better with someone else. Do you not think the owner of the golf club has any rights at all? Have you ever in your life heard of thieves scouting/casing a location, or probing actions that can be played off as "just looking" or "was just curious?" If someone carries your drink at a bar is the first thought that goes through your head "that guy seems just as nice as the guy selling roofie test kits"?

History decides whether a reaction or preaction was proportional or not, and they use context that we don't currently and will never have. Predeciding a course of action for every future interaction between humans at a personal, state, nation state level is just dumb. History is littered with people who suffered because they didn't believe something would happen and people who look back wondering why nobody did anything to stop a situation. I understand better why that is, now.

Or you use last week's peanuts bag and contaminate the "not peanuts" hopper that the folks allergic to them use.

You've no idea what she intended or what the cost of the items were, golf club or anything else.

When it's your hard earned money that's gone into something, or your name on the "Results" line then you get a say-so, otherwise it's not really any of your business. I just assumed from your attitude that you've never had to worry about money or results before.