NAS for Video Files - Just a RANT

[joeschmuck]

This is off topic and just weird thing but for some reason it just drives me nuts that people spend a ton of money on huge storage (20TB+), good ZFS protected storage just to save video content. I guess if it were part of the company to store and serve up video content then it would be okay. I guess I just don't have enough disposable money. And I'm not aiming my comments to anyone specific, it is just a general rant that I can't wrap my head around, but maybe someone could get me thinking like one of these people in how to justify spending money like this or maybe other reasons to create this large video vault. I increased my storage from 5.7TB up to about 9TB but over 50% of my capacity is in the service of storing backups of all the computers in my family. ~7% is for storage of data that is better suited on a NAS vice sucking down space on one or more computers, and about 5% is used for photos (I'm guessing at the percentages, I'm not home right now and remote log on in the hospital is a big fail. I have almost 200 video files (movies) in the DVD format, most I will never watch again so I'm considering deletion of all but maybe 13 to 15, ones I have and will watch again. Then I need to add the 10+ kid movies for my grandkids, I hate pulling out a DVD just to see thier little nasty fingers dirty it up. The grandkids are as young as 2 years of age. I do have a few BluRay rips like the movie SALT. My daughter liked it so much that we bought the BluRay disc, but I ripped it because she is now in college so watching it on her computer works better for her now. The disc is nice and safe in its case.

Now I'm not going to tell someone they shouldn't build a NAS for the sole reason of storging video content, maybe they are a huge video collector and do watch a lot of movies. If you got the money then I guess you can do whatever you want.

Anyway, if you feel like adding to this rant, defending why a person would create/maintain a large video library, or any other thoughts, I'd truly like to hear it. Please no attacking, if you think I'm a moron, well you must know my wife :p. I know someone will think I have a large collection but it being 99% DVD quality MP4, it doesn't consume much storage, about 1.3GB per move on average I'd guess.

Cheers!

 

[kdragon75]

Yeah I have a few hundred Gigabytes of video but its just for convenience. As you mention it keeps the disks safe in their cases and it iss nice because I can be on my phone,in the living room, or in the bedroom and I don't have to think about disks or where they are. With that said, 30% of my data are my photos and Lightroom backups and the rest is my VMware lab/VMs.
I could easily run a less powerful, less redundant NAS for most of it but building the SAN/NAS tuning and tweaking, learning ZFS and BSD are all a part of my hobby. I don't spend much on any of it so it a bit easier to justify 9TB of 2x4 mirror striped ZFS pool with geom raid 10 SLOG but the people that are building 5k+ machines for movies... I don't get it. Then they ask about the best SSD for a "ZIL cache" that does nothing for 99.9% of their writes...

 

[joeschmuck]

Then they ask about the best SSD for a "ZIL cache" that does nothing for 99.9% of their writes...

I've seen that many times before, made me laugh!

 

[kdragon75]

My personal favorite is when someone is building a NAS for their job with a 10k+ budget and they ask questions like that. I would love to have a job with that much latitude.

 

[sfcredfox]

I fall into the category of movie hoarder with around 500 movies. I recently deleted half of them because they were stupid and I'd never watch them again, or I didn't have or know anyone with discs of them. I started the collection mostly out of necessity because I kept deploying. We usually had no internet, and nothing to do off duty, so an external HDD with movies for your laptop was a life saver. It just kept getting bigger and bigger from there.

As of now, I have netflix and a great internet connection, so I don't know why the hell I keep movies other than we have three TVs in the house, and only one (mostly broken) blue-ray player, so having plex is the only way we watch stuff in the kitchen or bedroom. I loathe having to get a physical disc anymore :) If I buy a movie, I'd almost rather download it only and completely forget getting a physical copy.

 

[Jailer]

I store all mine just for the sake of convenience. Plex is exactly the software I've been wanting for years. I'm a huge movie fan and my internet sux so the streaming options aren't options for me like they are everyone else. As it turns out, I use my FreeNAS for a wide variety of things but my movie collection at 65% is by far the largest occupier of space out of anything on my server.

 

[cobrakiller58]

Anime hoarder here the convenience of being able to run media in any room is great. I have my boxsets safely tucked away for days I need to show my friends how much of a hoarder I am and in 50 years who will have a copy of one of these gems. I remember a couple of years ago I was looking for the original star wars movie and it was IMPOSSIBLE to find the one VHS I heard of was auctioned off for far more than I was willing to pay (I think Disney has now released it on dvd) What's going to happen to a lot of the digital only media in 50 years if we look for books and music from 50 years ago today a fair amount survived but that was because it ALL had a physical copy and someone had to posses it, in 50 years those Netflix/Prime exclusives and digital articles will disappear without a trace. Another example was the climate data that I heard was deleted, that is a part of scientific history and it can disappear so very easily.


Looks like I got off the topic of this off topic a little there lol

 

[Arwen]

My media collection is much larger, 3,000 video files, about 1.6TB. Though to be fair, at least 500, maybe even 700 videos are special features. (T2 extra special edition had 70 or so special features alone.) I bought them when I got fed up with cable, (which I cut summer of 2000). And at least half we aquired before streaming was both available and reasonably reliable. I buy much less movies and T.V. shows today.

As for hardware, my media server is a low power mini-pc running Linux, with a 1TB mSATA and a 2TB SATA spinner. About 25GB from each is for the root pool Mirror, and the rest is striped. (Yes, I use ZFS for all my filesystems today.) So about 2.7TB usable for my media.

There are photos, about 1,000 songs, (using 7.8GB), and some public domain eBooks. But space wise, it's trival compared to the videos. I did discover that the MP4 Simple CODEC compressed DVDs to less than 50%. But MP4 Advanced CODEC dropped it down to perhaps 25% of MP2.

My design for FreeNAS included housing backups of my media. (The photos NEED backing up...) But I simply don't want to waste time re-ripping my music, DVDs and Blurays from source material. So they get backed up. The pool is RAID-Z2 just because there are other more important things stored on my NAS.

 

[cobrakiller58]

(The photos NEED backing up...)

That is exactly what made me shift to using any form of RAID and also pushed the offsite back-ups as my collection of photos from event increased so did my paranoia. Since I was concerned with space efficiency when I setup my first FreeNAS I went with an 8-wide RaidZ2 however as drives grew and I started offsite back-ups it's become less of a concerning.

 

[danb35]

Why ZFS for video? Two words: pooled storage. ZFS lets me make a volume as big as I want (well, as big as I can afford, anyway), and expand it at any time. Why RAID for media? Sure, most of it I could re-rip, but it'd be a hassle at a minimum. And for any "home movies" or photos, there may not be an original left to rip from (yes, that's why we have backups--see above).

Why bother having all that media locally when streaming services are good and cheap? Well, why run Nextcloud, when Dropbox (etc.) is good and cheap? Same answer: I want control over my data. Netflix (etc.) removes content at will, and if that happens to be content you wanted to (re-)watch, tough luck. And there's plenty of stuff that isn't on Netflix or any of the other streaming services (like most of classic MST3k, to give just one example).

Edit: Of course, there are downsides too. One is that, since everything in my server is on the same pool, I end up with a laundry basket full of hard drives when I have to evacuate for a hurricane.

 

[Arwen]

...
Netflix (etc.) removes content at will, and if that happens to be content you wanted to (re-)watch, tough luck. And there's plenty of stuff that isn't on Netflix or any of the other streaming services (like most of classic MST3k, to give just one example).

This applies to some of my collection. Was not available for streaming when I bought it. Or, is no longer available. So having a local copy helps. Especially for loading to a tablet when you don't have wireless access. My large and small tablets each have copies of my photos & music, as well as some videos in case I get bored.

Note: I've tried Amazon Video and Netflix download to my tablet, never really got it to work.

 

[cobrakiller58]

And there's plenty of stuff that isn't on Netflix or any of the other streaming services (like most of classic MST3k, to give just one example).

This is very true in Canada the content is good but VERY limited.

 

[fr33n0zeer]

Now I'm not going to tell someone they shouldn't build a NAS for the sole reason of storging video content, maybe they are a huge video collector and do watch a lot of movies. If you got the money then I guess you can do whatever you want.

Anyway, if you feel like adding to this rant, defending why a person would create/maintain a large video library, or any other thoughts, I'd truly like to hear it. Please no attacking, if you think I'm a moron, well you must know my wife :p. I know someone will think I have a large collection but it being 99% DVD quality MP4, it doesn't consume much storage, about 1.3GB per move on average I'd guess.

Cheers!

After reading this post, I questioned whether you think there is a better solution to storing movies other than a NAS. Are you suggesting or thinking that an inexpensive DAS (both in $$ and deployment time) would be a better option?

 

[twinscroll]

I think the OP is more likely referring to how intricate and concerned people get with ECC, z2 and above arrays etc etc for data that if a bit or two is missing, doesn't really affect it. Whereas a critical database...sure spending a chunk of change and ensuring all of what ZFS can do is at play makes a bit more sense.

In that - if you're just storing media ...the nature of video doesn't necessarily mean you need to spend 5k on new gear. BUT! I am that video person and the shear amount of time it took me to rip those DVD's and Blu-Rays alone is worth having it available via my network with redundancy.

Also, I think the OP may be referencing a bigger fact that a lot of people overlook. And that is RAID is not a backup. So if you've got a ton of media that you've spent a LOT of time ripping, just going for a large storage pool with redundancy etc...is only but so good. In many case, you may be better served by ensuring you've got offsite backups (say at a friends house in another state or something of that nature).

Please feel free to correct me if I'm wrong OP!

 

[joeschmuck]

I started the collection mostly out of necessity because I kept deploying. We usually had no internet, and nothing to do off duty, so an external HDD with movies for your laptop was a life saver.

I can personally relate to that, but in my day ripping a movie was a huge undertaking, it would have been a VHS or Beta tape. While deployed we would watch reel to reel movies on the mess decks. Apollo 13 was the longest movie at 5 reels and it took a great deal of coordination to watch it from midnight to breakfast. Those were the days to be honest with you.

in 50 years those Netflix/Prime exclusives and digital articles will disappear without a trace.

I do not like to "own" a digital copy that I must have an internet connection to play it and the product resides on some server. I don't mind having my own digital copy, at least I can have that in my hands. So I agree.

my media collection is much larger, 3,000 video files

WOW!

I did discover that the MP4 Simple CODEC compressed DVDs to less than 50%. But MP4 Advanced CODEC dropped it down to perhaps 25% of MP2.

I too use the MP4 Advanced Codec and I'm happy with the result, it's better than full size.

After reading this post, I questioned whether you think there is a better solution to storing movies other than a NAS. Are you suggesting or thinking that an inexpensive DAS (both in $$ and deployment time) would be a better option?

I have no idea what would be a better option, I think that is up to the user to figure out what they want to do. I personally like FreeNAS and ZFS because of the data protection it offers so I would always recommend it over other options unless there was a clear advantage a different option offered the user.

My main reason for the rant was I question the amount of money people spend on a huge NAS just to mainly store video content, it has nothing to do with ZFS, ECC RAM, or anything else, only the reason they spend so much money because let's face it, storage is not that cheap. I understand some reasons why they would do it but do you really need to have 20TB or 40TB devoted for video content? Most of that content you are very unlikely to watch multiple times? How many times do we see a person asking about how to create a huge FreeNAS storage mainly for video content? Too frequently I feel. I only feel it is a good value if the content is going to be watched frequently. So while many of you who have commented that it's based on convenience, I still struggle to understand why the capacity needs to be so high. However I do accept that there are people who strongly desire to have all content available at a glance, I can see convenience from that aspect, but to me it's not worth the money to buy all that storage, replace expensive failing drives, just so I don't have to pull out a DVD that I hadn't watched in 5 years or more.

So I'm perfectly fine with people who want to create these large FreeNAS units, it doesn't hurt me in any way other than I cannot relate to doing that myself. Guess I'm too old.

Also, I appreciate everyone's two cents, it helps me understand your point of view.

 

[Jailer]

So while many of you who have commented that it's based on convenience, I still struggle to understand why the capacity needs to be so high.

From my perspective I end up saving largish files because I like quality to go with my convenience. What I mean by that is I like high quality 1080p and high def audio. For me it makes no sense to have all this high def content only to rip it and encode it down to crap quality with 2 channel lossy audio just to save a little storage space. I have a significant hearing loss and I save all my content with the high def audio intact for the best audio quality I can get and it makes my file sizes rather large. I have a very hard time understanding dialog in a movie any more so anything I can do to help that is worth it to me. A bunch of hard drives is still a hell of a lot cheaper than hearing aids.

As far as re watching my content, I watch mine a lot. I don't watch TV. At all. I love movies and I frequently re watch a lot of my content. If I sit down to watch something it's a movie.

 

[fr33n0zeer]

I have no idea what would be a better option, I think that is up to the user to figure out what they want to do. I personally like FreeNAS and ZFS because of the data protection it offers so I would always recommend it over other options unless there was a clear advantage a different option offered the user.

My main reason for the rant was I question the amount of money people spend on a huge NAS just to mainly store video content, it has nothing to do with ZFS, ECC RAM, or anything else, only the reason they spend so much money because let's face it, storage is not that cheap. I understand some reasons why they would do it but do you really need to have 20TB or 40TB devoted for video content? Most of that content you are very unlikely to watch multiple times? How many times do we see a person asking about how to create a huge FreeNAS storage mainly for video content? Too frequently I feel. I only feel it is a good value if the content is going to be watched frequently. So while many of you who have commented that it's based on convenience, I still struggle to understand why the capacity needs to be so high. However I do accept that there are people who strongly desire to have all content available at a glance, I can see convenience from that aspect, but to me it's not worth the money to buy all that storage, replace expensive failing drives, just so I don't have to pull out a DVD that I hadn't watched in 5 years or more.

So I'm perfectly fine with people who want to create these large FreeNAS units, it doesn't hurt me in any way other than I cannot relate to doing that myself. Guess I'm too old.

Also, I appreciate everyone's two cents, it helps me understand your point of view.

I'm glad you answered my questions directly, as they were honest questions. I was a little surprised with your "rant", in a way, because you are a strong proponent of FreeNAS, ZFS, etc. In some ways, irony, the FreeNAS guy asking why all these people are building these huge FreeNAS systems.

One thing to consider is the current move away from tangible media. I've read a number of news articles during the last couple of months, such as a news story that Best Buy is to stop selling CD's, which means that we'll either own a digital "thing" or we'll rely on streaming everything via perpetual monthly subscription fees. With the proliferation of numerous video and audio streaming services, with each only offering a thin slice of the videospheres or audiospheres, we're left buying into multiple streaming services that, as others have pointed out, do not offer guaranteed longevity of content.

So, for some of us who want to have content we enjoy available, these NAS arrays are only going to get bigger, I suspect. At the very least, I can understand why they are being built in the first place.

Personally, I love the music CD and I also love vinyl. With vinyl, I want to convert it to digital, because digital is portable. Vinyl is great in the home, not so much in the car, hiking trail, or sailboat. I detest buying downloadable digital music, because I see the music CD as a kind of backup, although they do breakdown over time; here's an article about it. I also like the higher quality that pcm to wav brings, versus the 128 kbps, 256 kbps, or 320 kbps resolutions offered via download services like iTunes, and usually it's on the lower end. I can downsample my wav to lossy formats, such as mp3 for reduced file size where it makes sense. There is at least one ultra-high fidelity digital music provider of downloadable commercial music content, but special high end hardware is required to actually get the sound into the air, not guaranteeing that you or I can actually hear the difference. But after all this, the bottom line is this: the content has gone digital and without backups, it's very easily gone for good.

So whether we pay for a home NAS device, a bunch of lessor hard drives, or we pay to repurchase our content, I'm not sure how we get out of this financially unscathed.

 

[sfcredfox]

So I'm perfectly fine with people who want to create these large FreeNAS units, it doesn't hurt me in any way other than I cannot relate to doing that myself. Guess I'm too old.

Also, I appreciate everyone's two cents, it helps me understand your point of view.

I had to pause and say how refreshing it is to come across at least a few people who are mature enough to accept that they might not share the same opinion or view point on something, and they do not try to convince the other party they are wrong or demean them in some way.

I mentioned using FreeNAS for movies in my above reply, but after reflecting on it, there's a good deal of lesser products out there that could be a decent low performing NAS. They could probably be implemented cheaper as well (I'm thinking one of the small ITX/mini cases and a couple of drives.

I use FreeNAS because I am able (with hardware), to turn it into a decently performing SAN. I need to be able to run almost a datacenter's worth of workload, so I have 48 disks in there for high performance virtualization, and do caching (L2s and 72GB RAM) to speed it up even more, and a SLOG to ensure I can do sync=always for reliability of all the datastores.

For my normal worklloads (combination of 4K, 8K, 16K), the system can support ~20K-30K IOPS. For comparison, my dad has a QNAS NAS (simple filer) with four 4TB disks in it trying to do the same thing. It can handle about 1200 IOPS under the same load. Not even close, but the QNAP lacks all the facilities to be a SAN. I just reads and writes files to disk, simple. FreeNAS has all the caching abilities to go beyond simple disk performance. That's why it's awesome.

My use case is a bit over the top and unusual compared to most I suspect.

 

[fr33n0zeer]

I had to pause and say how refreshing it is to come across at least a few people who are mature enough to accept that they might not share the same opinion or view point on something, and they do not try to convince the other party they are wrong or demean them in some way.

For myself, I can honestly say that I'm way too ignorant to be that opinionated. Lol. Sounds like your set-up is fairly bad%##. Cool.

More seriously, this is one of the reasons I liked this post so well. More than anything, I was surprised because it is so refreshing.

 

[joeschmuck]

I'm glad you answered my questions directly, as they were honest questions. I was a little surprised with your "rant", in a way, because you are a strong proponent of FreeNAS, ZFS, etc. In some ways, irony, the FreeNAS guy asking why all these people are building these huge FreeNAS systems.

One thing to consider is the current move away from tangible media. I've read a number of news articles during the last couple of months, such as a news story that Best Buy is to stop selling CD's, which means that we'll either own a digital "thing" or we'll rely on streaming everything via perpetual monthly subscription fees. With the proliferation of numerous video and audio streaming services, with each only offering a thin slice of the videospheres or audiospheres, we're left buying into multiple streaming services that, as others have pointed out, do not offer guaranteed longevity of content.

So, for some of us who want to have content we enjoy available, these NAS arrays are only going to get bigger, I suspect. At the very least, I can understand why they are being built in the first place.

Personally, I love the music CD and I also love vinyl. With vinyl, I want to convert it to digital, because digital is portable. Vinyl is great in the home, not so much in the car, hiking trail, or sailboat. I detest buying downloadable digital music, because I see the music CD as a kind of backup, although they do breakdown over time; here's an article about it. I also like the higher quality that pcm to wav brings, versus the 128 kbps, 256 kbps, or 320 kbps resolutions offered via download services like iTunes, and usually it's on the lower end. I can downsample my wav to lossy formats, such as mp3 for reduced file size where it makes sense. There is at least one ultra-high fidelity digital music provider of downloadable commercial music content, but special high end hardware is required to actually get the sound into the air, not guaranteeing that you or I can actually hear the difference. But after all this, the bottom line is this: the content has gone digital and without backups, it's very easily gone for good.

So whether we pay for a home NAS device, a bunch of lessor hard drives, or we pay to repurchase our content, I'm not sure how we get out of this financially unscathed.

I really liked your responses and I can relate to them. It's sad to realize that physical content that a person purchases is likely going away and it will all be sent via the internet either streamed or downloaded, with streaming being the way it looks like things are going. I also relate to the sound of vinyl, I use to buy half speed masters just because they were a bit cleaner sounding but still warm. Yes, Music CDs were out at that time too. Now days I prefer a good CD but since my ears do not hear like they use to, I can live with a good quality mp3.

I had to pause and say how refreshing it is to come across at least a few people who are mature enough to accept that they might not share the same opinion or view point on something, and they do not try to convince the other party they are wrong or demean them in some way.

Not my style to force my opinions on someone. If I'm sounding forceful on some posting then I generally will back it up with why I'm pushing something specific.