I’m not a regular or anything. Just don’t want lose my comment priveledges if I pop in and go “hey guys I know there’s a darker side to this particular good news”
imo, they are always welcome
political and “orphan crushing machine” type posts should also be removed.
posts are already deleted. we are specifically asking for comments because they have generally different nature. (it is more about balancing hope/cope)
there is worse stuff which sometimes gets deleted before many noticing
thank/fuck to you too. (i do not cuss, just joking, no hate)
My 2 cents (not as a mod) - do not remove or ban. My primary reason is that there should be some healthy dose of reality in the mix. for example, there was a post about some miracle drug which cures addiction. it was in comments that clarified that drug has been known for quite some time, and has not been prescribed because it's efficacy is not proven (it is more vibes/feels/placebo based), and it increases risk for heart related diseases (so we can not just prescribe and be like if it does not work, there is no harm).
As a mod, i do feel there is some room to change the rules. we can mostly just ask the comments section to be more civil. it is fair to point problems, but not shit unnecessarily. for example, there was a recent post about some actor "adopting" a town/village. in comments someone said that these never work. if they really want to bring change, they should lobby and bring policy change (which they say so casually as if it is that easy to be done). I did not delete the comment for two reasons - one i believe in free speech, and if what they say is shit, it will get downvoted (while i am not always about mob justice, it is good to know what the people think) (in case of this comment, it was -10 - -15 ish last i checkd). Secondly, i had no rule allowing me to do so - we delete/ban hate speech, an uncivility, but in this case, this anger was not targetted (it was more of - they never end up bringing anything good kinda argument instead of actor specific, it was more about how they go by generally). while it does not match the general vibe of community, i do not feel i should delete stuff hust on that basis.
the comment i was talking about - https://piefed.social/post/1306755#comment_8157021
I am a cricket fan, and comms are nearly dead
but the breaks are memory safe
I do not like this attitude towards uutils. phoronix makes a very click baity title, and comments shit on uutils, rust and ubuntu.
last time it was "extremely slow" (17x), and by the time most people reported it, a pull request had been made and merged which brought the sha function within 2x of gnu version. not ideal, but definitely not reporting worthy.
then it was sort function can not sort big files, which came from a artificial benchmark of a 4 gigabyte file with single line all consisting of character 'a' (not sure if it was a or 0 or something, but that is not relevant). gnu version finished in ~1 sec, and the rust version could not. you can not sort a single line, it is already sorted. so there is some check which uutils is missing, which could be easily added, but no, we must shit on uutils and rust because they are trying.
In this case, some md5 errors happen, but apparently problematic part is not md5, but dd (actual bug report - https://bugs.launchpad.net/ubuntu/+source/makeself/+bug/2125535).
I am not saying uutils is a perfect project, but gnu coreutils are nearly 4 decades old, where as uutils are less than 1 decade (yes the project did not start last year). There are bugs which need to be ironed out, and testing it in a non lts distribution is the best way to do that.
it feels like a xy problem. if i am not wrong, single bit corruption leaving file unextractable is a bit wild, and my guess is that it's headers were blown.
As for general stuff, use a file system which does parity calc and such. or use something like raid to have redundant drives. (you can set something like 1 in 5 breaks, or 2 in 5, but more you allow to break, less actual space usable you would have). Or have really simple backups.
As to physical media - do not go flash based (ssd/sd cards/usb pen drives) if you want to leave them unpowered. they expect to be powered once every few months. they are effectively ram disks but like much more stable. Hard disk drives are better, but handle physical shocks much worse. you can drop a ssd and expect it to work. for a hard disk it is almost game over. Magnetic tape are better, they are much less data dense, but they are cheap.
I would assume it’s lossless
yes. these are lossless algorithms.
Now coming to compression - no compression practically deals with bit corruption. practically all compression formats aim for small size and or fast (de)/compression. Saving parity bits is wasteful.
If you can install some thing, try, for eg, https://github.com/Parchive/par2cmdline. You give whatever file (compressed or not), and it will generate parity bits so you can repair stuff. now use whatever compression you want, and prepare parity bits for worst case.
As to what compression algorithm, zip or gzip (deflate), bzip2 (or newer 3), xz (lzma in general), and zstandard (or older lz4), brotli are practically not going anywhere. most distros use them, they are used on web, and many other places. My favorite is zstadard as it gives great compression and extremely fast.
Do any of them have the ability to recover from a bit flip or at the very least detect with certainty whether the data is corrupted or not when extracting?
no and no.
You should also consider file archive format. for example zip (the format, not algo) or tar are effictively standards and stable, and practically here forever. there are mountable ones like squashfs (also fairly common. most linux distros use it for live images) and dwarfs (not yet a standard, imagine squashfs but also deduplicating).
do compression formats exist (or can they exist) which correct for bit flop - yes
If your goal is that a single bit flip should not ruin it, you should probably not look into deduplicating ones (they are reducing the number of bits stored, so in case a bit flip happens, less files would be corrupted).
Now coming to another part - do you want to compress data ? if so, why?
when you compress data, you literally reducing number of bits. now imagine if all bits on your disc are equally likely to undergo bitrot. if so, compressing makes your files less likely to rot.
but as you have also said, it is possible that if they corrupt, the corruption is more catastrophic (in a plane text file, this may just mean a character mutated, or in a image, some color changed. hardly problematic).
So you should also check - is compression worth it? come up with a number, lets say 90%. if compression algorithm reduces file size to 0.93 the original, do not go for compression. if it does, do compress. I am not saying pick 90%, but like decide on one you seem content with.
here is a stupid idea. if compression reduces file size by 2x, then compress, and make yet another copy. now even if one is corrupted, you have a pristine copy.
they are called taxes where i live
i think it was in anime
let me tell you by example of my country. we have a authoritarian leader, and big news media is very much in their pockets. if you go on news channel, it is either all positive stuff, or partisanal/communal shit.
Thing is that, the postive stuff does not feel positive, when you completely ignore reality. according to them, we should celebrate opening of a new highway, where majority of aour road network is fucked. we should ignore floods, or other disasters, and also ignore the corruption done in making highway, just to celebrate it.
Good does not have a meaning in a vacuum. it is a relative thing. if all we do is allow hopeful hippy yipee stuff, it eventually becomes cope.