/dhg/ - Data Hoarding General

>Links
Rentry: rentry.org/dhg

>What is /dhg/
In this thread we discuss and create technology and software for data-hoarding, archiving, scripts, and more.

>gallery-dl - scrape images, manga, videos and more from many websites
github.com/mikf/gallery-dl

>Hydrus Network
hydrusnetwork.github.io/hydrus/

>Stash
github.com/stashapp/stash

>SmartImage
github.com/Decimation/SmartImage

>TMSU
github.com/oniony/TMSU
tmsu.org/

>The best SSDs for hoarding
amazon.com/SAMSUNG-870-QVO-SATA-MZ-77Q1T0B/dp/B089C73T72
amazon.com/SAMSUNG-500GB-Internal-MZ-77E500B-AM/dp/B08QBMD6P4/

>The best external SSD, for portable but slow storage
amazon.com/SAMSUNG-Portable-SSD-1TB-MU-PC1T0T/dp/B0874XN4D8

>The best HDD, if you're poor
amazon.com/Seagate-BarraCuda-Internal-Drive-3-5-Inch/dp/B07H2RR55Q

Previous Thread:

Attached: ec18b8cd2efac7fe9a500cfd1c19baf06cf256c99983644bc982357b76cf5ea6.jpg (1366x2048, 500.37K)

Other urls found in this thread:

twitter.com/kuroimori_twee
youtube.com/watch?v=lMxdZiguxlI
desuarchive.org/g/thread/85512570/#q85512725
desuarchive.org/g/thread/85357870/#q85402517
desuarchive.org/g/thread/85432435/#q85438897
desuarchive.org/g/thread/85531550/#q85532203
desuarchive.org/g/thread/85511550/
twitter.com/SFWRedditGifs

Budding Data Hoarder here.
Any recommendations for a the ZFS layout of 10x 3TB 64mb cache 7200rpm drives?
I got them cheap as an unused lot, and each of them appear to be good. Was going to pickup some spares when I see a good deal.

I played with hydrus a bit last night, and once I got automatic downloads going, I started to see the appeal of hydrus. I was able to query gelbooru for an artist that I liked, and in a few minutes, I had a large collection of his drawings on my local SSDs and fully tagged. It makes it really easy to find and post anime girls in a thread since I can find the picture via hydrus and then paste the path into the file upload dialog.

Attached: f37e0667cbfebd0591a274e5c7e455b65bdb099da6da78a8e68fbf777c1d145f.jpg (1786x2048, 496.7K)

What are your current storage capacity goals? Me, I'm trying to reach 100 TB.

twitter.com/kuroimori_twee

Attached: 56424c0a1611eb25a6fdd25da22039d7565cd4f5ddae6288479009191c5483cf.jpg (1517x2023, 826.39K)

>100 TB
What are you going to do with all that storage?

not sure about
but I would store high quality bukkake/gokkun porn...I legitimately can't stop watching european and american bukkake/gokkun porn...someone please help me...

Plex stuff and backups mostly. Basically, I just want to be able to download entire TV shows without having to worry about running out of room. Right now, I have to be a bit more selective. With more room to work with, I'd upgrade my 7000 720p movie collection and start hoarding anime in earnest.

>someone please help me...
youtube.com/watch?v=lMxdZiguxlI

>I'm unironically watching that
Thank you user.

That video changed my life. It's really good.

On the 'using Obsidian instead of Stash to organize a local JAV collection' front: These past few days I've been working on improving the css of the jav notes. Here's what it looks like now, with cover removed to preserve thread modesty.

Last update before that: desuarchive.org/g/thread/85512570/#q85512725

Attached: gar.jpg (1377x991, 90.11K)

I'm keeping track of the journey. Thanks for keeping us updated. I might have to play with Obsidian a bit to see what I can learn and bring back to org-roam.

Adventures in Obsidian:
Part 1
desuarchive.org/g/thread/85357870/#q85402517
Part 2
desuarchive.org/g/thread/85432435/#q85438897
Part 3
desuarchive.org/g/thread/85512570/#q85512725
Part 4
desuarchive.org/g/thread/85531550/#q85532203 (current thread)

There was a good tmsu thread a few days ago.
It might be worth reviewing for tmsu appreciators.
desuarchive.org/g/thread/85511550/

Also, I'm actually glad I finally dipped my toes into hydrus, because it showed me what kind of extra support tmsu would need to work well. As big as hydrus may be, I think it works decently. I like that you can have multiple queries with each getting their own tab (or page as hydrus calls it).

Another user mentioned automatic downloading, and I was skeptical at first, because I'm not a data hoarder or so I thought, but it was kinda nice to pull down all of kuroimori's drawings off of gelbooru in one shot. That's ammo now for poasting purposes.

I almost wish hydrus used tmsu under the hood.

Attached: ea0948c79ab6d03ea195e0a3e377db7cbd34d2bc3154ee1133dd272f53622e22.jpg (2048x1868, 623.04K)

I use my raspberrypie and connect my hard drive to host movies. I use sonar to download torrents and use jellyfin to remotely connect and view fom anywhere. Is there ways to reduce stress on encoding and optimize storage ?

>multiple queries with each getting their own tab (or page as hydrus calls it).
just duplicate your terminal and run the new tmsu query?

I just accidentally formatted my USB drive again. AHHHHHHHHHHHHH

A textual listing of results isn't as useful as an interactive grid of my results that I can further refine.

You didn't have backups?

What all OS and FS(s) do you guys use?

For me it is Gentoo. F2FS for my /, FAT32 for my /boot, and ZFS for my /home. Boot and ZFS keys are on a thumb drive, / is on a 512gb samsung NVMe adapted through the mPCIe slot, and /home is 4x 1tb drives I pulled from various old PCs.

Good General.
>booru collection exceeds 3 million images
>performance is kill
But it seems like implementing a fuse driver is not so hard, so I will come up with some block storage scheme that enforces image files being stored in sequence.
But I need to buy more hard drive space first.

>You didn't have backups?
Might have something from 6 months ago...