It's A Digital Disease!

The original post: /r/datahoarder by /u/notned64 on 2025-04-27 16:24:11.

The title happened to me yesterday and I couldn't understand the instructions to fix it. I won't be back at it until next week. Will it clear on its own? Otherwise I'll have more questions.

The original post: /r/datahoarder by /u/ifnbutsarecandynnuts on 2025-04-27 15:53:31.

I would like to save offline copies of a few dozen of my favorite channels, size is not a concern I'd like it to download every video at the highest resolution and flac audio if available. I tried using a gui off github called scrawler which uses yt-dlp and I quite liked the ui ease of use for a novice like me, it worked on a few smaller 50 video channels but as soon as I added a larger 1000+ video channel it seems to have been flagged by yt as a bot and stopped downloading cache files.

I have a few channels with 3000+ videos I'd like to download, I'm not so rushed on it I'm happy to run a script at a slower pace. I was hoping I could get the scrawler gui working for me as I'm really not great at understanding/reading/deciding between all the command line options.

Desired output;

1. highest res available + flac audio if available, otherwise next best option
2. video upload date + channel name in start of file name

Thank you for any help or suggestions you could provide.

The original post: /r/datahoarder by /u/wagldag on 2025-04-27 14:29:01.

https://eos.org/research-and-developments/noaa-datasets-will-soon-disappear

The original post: /r/datahoarder by /u/LucyKosaki on 2025-04-27 14:21:29.

I have been sitting on a few hundred GB of older twitch VODs (2021-2023) from a bigger streamer (100k+ twitch follows), that haven't been uploaded or archived anywhere else and is currently considered lost. I thought it would be a good idea to archive and make the content available by putting it on the Internet Archive. I even did contact the creator and got their permission to do it.

But to my surprise when talking to IA support, they told me that such content is not allowed to upload to IA. I have been quite surprised because:

1. This is currently not communicated on any of the internet archive's articles about what can and what can't be uploaded, such as:

https://help.archive.org/help/uploading-tips/

https://help.archive.org/help/uploading-what-is-not-ok-or-not-ok-to-upload/

https://archive.org/about/terms

2. The site has been commonly used for creator content preservation since 8+ years and there are currently way over 200.000 VODs and YouTube mirrors on the archive, it is almost 3 Petabyte of data: https://archive.org/details/twitchstreams

With that amount of data and common use, I am surprised they never did anything against it, even though it is apperantly against their rules.

My one item I had uploaded got deleted and a couple hours later, shortly after I messaged support regarding this, my whole IA account got banned.

Does anyone else has more information or experience regarding this?

The original post: /r/datahoarder by /u/Yukinoooo on 2025-04-27 11:48:39.

Should I buy what range of SSDs to save the game data and play at the same time on PC? Entry-Level, Mid-Range or High-End ?

The original post: /r/datahoarder by /u/astfors on 2025-04-27 10:56:39.

Hey folks!

I’m in the middle of a big digital cleanup project — sorting through several terabytes of files before moving everything to a proper cloud backup service. I’m on Windows 11 and looking for a good tool that can:

Detect duplicate files (based on name, size, and preferably checksums)

Organize files by type (images, videos, documents, etc.)

Display file creation and modification dates

Let me move duplicates to a different folder before deleting them

A clean, functional GUI is a must. I’m not much of a command-line person, so while CLI suggestions are welcome, I’d strongly prefer something with a graphical interface.

Ideally, it should be open-source or free, but I’m willing to pay up to around $50 USD for something solid and reliable.

So far I’ve looked at AllDup and Duplicate Cleaner Free/Pro — has anyone here tried those, or got better recommendations?

Would love to hear what tools you folks use to keep your digital chaos under control. Thanks a ton in advance!

The original post: /r/datahoarder by /u/Brok3nHalo on 2025-04-27 07:43:54.

With several of my favorite vTubers graduating (ending streaming as their characters) recently and soon, I made tool to make it easier to archive content that may become unavailable after graduation. It's still fairly early and missing a lot of features but with several high profile graduations happening, I decided to release it for anyone interested in backing up any of the recent graduates.

By default it grabs the video, comments, live chat, and generated English subtitles if available. Under the hood it uses yt-dlp as most people would recommend for downloading streams but helps manage the process with a interactive UI.

https://github.com/Brok3nHalo/AmeDoko

The original post: /r/datahoarder by /u/RacerKaiser on 2025-04-27 05:33:01.

I swear, recently its been ridiculous, I download some from yt, until i hit the limit, then i move to flickr and queue up a few downloads. then i get 429.

Repeat with insta, ig, twitter, discord, weibo, or whatever other site i want to archive from.

I do use sleep settings in the various downloading programs, but usually it still fails.

Plus youtube making it a real pain to get stuff with yt-dlp, constantly failing, and I need to re-open tabs to check whats missing.

Anyone else feel like it's a bit impossible to get into a rhythm?

My current solution has been to keep the links in a note, and dump them, then enter one by one. However the issue with this is, sometimes the account is dead by the time i get to it.

The original post: /r/datahoarder by /u/VirginMonk on 2025-04-27 04:08:25.

Hi Reddit Fam,

First of all I would like to thank you all. Going through posts here had helped me a lot and motivated me to build my own small Home Lab.

I am from India and small problem of doing this in India is enterprise drives are really expensive.

So I thought what if I can ask someone to buy few from USA/Hong Kong as I have friends coming and going once or twice a year at both the places.

I would give an example 12TB Iron Wolf pro is costing me around USD 420-430 in India and same thing will cost around USD 300 in United States and should cost somewhat similar in Hong Kong.

Things I want to know is does Segate gives international warranty?

If the warranty don't works in India then does it makes sense to buy Iron Wolf Pros? I mean AFAIK one of the reason Iron Wolf Pros cost so much extra is the data recovery support etc provided by Segate for 5 years. So if I am buying from USA/ Hong Kong and support is the only difference then will getting something like Segate Exos be a better choice?

Please help me with this.

Btw,

I am planning to start with DS 923+ NAS From Synology. The reason I am not going with latest model is because of Hard disk locking thing Synology is doing and one of the reasons I am going with Synology this time is because this is my first NAS and at the moment I want to keep it relatively easy but please feel free to drop recommendations for this as well.

Note: - NAS unit I'll be buying from India only to avoid any kind of headaches later on.

The original post: /r/datahoarder by /u/Jo_So_Flow on 2025-04-27 04:02:28.

Hello! So I'm in a predicament on how people who takes lots of videos/photos on trips store years of files. I currently store most of my photos/vids in my pc with 12tb of mixed ssd/hdd. Though that's basically goin out quickly.

My question how do you go about storing all these files? Do you compress the files by album? Leave it on raw and store it? Convert files into smaller file type then compress? Or just keep expanding storage?

I've been hand picking my files and deleting a lot, but the videos are taking up a lot of space still. I am currently shopping/planning on buying/building my own NAS with my old gaming PC. Though would still like to get an advice on how people store their files and back them up. I've read the 3-2-1 guide and planning to implement that soon with the NAS that I'm planning and Azure.

The original post: /r/datahoarder by /u/bluecraney on 2025-04-27 03:10:04.

i am working on building a punch/ reader to store photos ect. on mylar tape for extreme long term storage my first issue is compression.

i am looking for the best way to compress a large amount of photos into as little space as possible because you can only get about 100 bytes /ft what is the current best way to compress for this case.

The original post: /r/datahoarder by /u/exilus92 on 2025-04-27 02:39:13.

I just had a few bad experiences recently regarding data loss and/or corruption (incl. backups getting corrupted) and I am looking for a new robust backup solution for long term mass storage. When I consider factors like having more than one physical backup and tracking file changes for important projects, I think the amount of data I need to store is large enough to justify looking into tape options. I'm talking 3 digits TB, most of which is static.

I don't want to deal with a massive pile of 150 tiny and slow tapes from 15 years ago, it would have to be a relativelly recent LTO version. When I look at brand new drives, it looks like it's all priced for enterprise and out of my budget. When I look at used gear, it's affordable but it's very hard for me to figure out what is a good option, what brands are good or bad, etc.

You guys are the expert on this, I welcome ALL advice.

The original post: /r/datahoarder by /u/Snickrrr on 2025-04-27 02:13:11.

Hi all. I’m looking for an 8TB HDD to store videos. It won’t be moved around and fast access to my videos is paramount. I don’t want a long delay before they start.

I’ve done my research and seen that the best options are the Barracuda in an enclosure and the Sandisk professional G-Drive. Also the G-Drive has an Ultrastar HDD which seems to be superior?

I’m biased towards the G-Drive because it looks so slick. Price wise they cost pretty much the same.

Which one would you choose?

Edit: I don’t mind Ironwolf’s or Exos’ price which seems to be Seagate’s more premium products but I’ve read that NAS HDD are not the best for my situation. I guess that the Ultrastar in the G-Drive is rated for video transfer scenarios so basically the best quality for my need? Is this correct?

What would be Ultrastar’s standalone HDD equivalent for movie storage?

The original post: /r/datahoarder by /u/churnopol on 2025-04-27 02:07:21.

Yet another unique way to back up my favorite shows.

The original post: /r/datahoarder by /u/KoholintCustoms on 2025-04-27 01:59:25.

Hello there r/datahoarder. I'm not exactly a hoarder myself but I think it's really interesting reading about techniques, software and hardware.

My footprint for my digital stuff is actually comparatively small, currently about 450 GB. I use a simple 3-2-1 backup method. One of my backup hard drives is a Western Digital external 2.5 inch 3.0 USB 500 GB drive. It's about 5 years old so I think it's time to replace, right? Seems to be in good condition but you just never know.

Right now I'm thinking to replace it with a m2 1 TB drive in an external enclosure. No moving parts so I guess it's less prone to failure? I dunno. And m2 1 TB seems to be reasonably priced.

Any suggestions? Is this a generally good idea or should I do something else?

Thanks.

The original post: /r/datahoarder by /u/Natessie on 2025-04-27 01:25:28.

Just happened to run into this - https://www.amazon.com/Elements-Desktop-External-external-storage/dp/B09VCXWPQG

I was debating waiting until prime day for a similar price, but this is pretty good right now.

The original post: /r/datahoarder by /u/thefannyfairy on 2025-04-26 23:56:02.

The original post: /r/datahoarder by /u/Macgeek1 on 2025-04-26 23:28:52.

The original post: /r/datahoarder by /u/oOBubbliciousOo on 2025-04-26 23:16:10.

The original post: /r/datahoarder by /u/Keeftraum on 2025-04-26 22:16:08.

Hey everyone,

I recently purchased the Pay As You Go plan on PixelDrain with the minimum deposit, mainly because one of the websites I use to download computer programs relies on PixelDrain links. The service has been working great so far — simple and fast.

Right now, I’m only using it for that one site, but since I’ve got 10TB of bandwidth, I’d really like to make better use of it. Do you guys know of other websites or communities that regularly use PixelDrain to share content (software, media, tools, etc.)? I'd appreciate any suggestions.

I'm also open to creative ideas on how to use my quota — whether for self-hosted projects, backups, file sharing, or anything else fun or practical.

Thanks in advance to anyone who replies 🙌

The original post: /r/datahoarder by /u/palepatriot76 on 2025-04-26 21:38:23.

So when moving my content around from HD, SSD, to external HD, things are snappy, not perfect but transfer rate is ok for me

Whenever I am transferring to my SanDisk Ultra 3.0 256 and 512 GB stick is unreal how slow it is, averaging 3.50 MB/s

It was Fat32 because my old TV only used it but just formatted to NTFS and putting some content back on it and could swear it is even slower now!

The original post: /r/datahoarder by /u/Difficulty-Used on 2025-04-26 20:19:36.

I have one 12tb hard drive in my Synology nas DS423+. I just got three 20tb hard drives and I want to upgrade them. I know I'm committing a sin here but I dont have a full back up. I can back up my most important things only. Is there any way to upgrade my drives without having to reset all my dsm and setting and apps.

The original post: /r/datahoarder by /u/muffinBadger on 2025-04-26 19:33:45.

Hi all,

I'm trying to buy a Samsung 870 EVO SSD from a seemingly reputable physical store. Is there anyway to check the packaging to know if it's fake? (Without opening the box)

I searched online and it leads to Samsung Magician, which can only be run after I open the box, of course.

Thank you.

The original post: /r/datahoarder by /u/blakealanm on 2025-04-26 19:06:56.

The original post: /r/datahoarder by /u/ElGatoBavaria on 2025-04-26 07:13:33.

Hi guys, I have a lot of HTML files that I want to deploy to my local network to use on tablet or smartphone. There is no Index.html but just a large amount of folders and subfolders.

In addition to the deployment, I need a search function to find e.g. all HTML files that contain for example “<meta property=og:title content=”This is my search string“>”.

There is an image linked in each HTML, which I would like to see as a preview after the search.

I know there are a lot of requirements, so I'm asking for help here too, as I'm not familiar with anything like this.

I would be very happy about feedback!