FreeNAS as DVR for IP cameras?

[averyfreeman]

@Rickinfl I notice from the screenshot that you are using separate dataset for each camera, I wonder why is that? Instead of just one shared dataset for all cameras?

Also what advantage you see by using a NFS instead of plain FTP for access to your FreeNAS?

It's a ZFS convention thing to make a separate dataset for everything

It's basically just like making separate folders, but you can do more with them (quotas, different ZFS configs, etc.)

NFS is going to be faster / support higher bandwidth. FTP is kinda garbage.

 

[Black Ninja]

It's a ZFS convention thing to make a separate dataset for everything

It's basically just like making separate folders, but you can do more with them (quotas, different ZFS configs, etc.)

NFS is going to be faster / support higher bandwidth. FTP is kinda garbage.

Thanks for getting back. Where you been over a year since I ask ?
Joking aside, I hope nothing bad happened to you. I haven't been here probably since I wrote that over a year ago, I didn't even know there is no freenas , it's ixsystems.com now.
Anyways,
to respond:

I understand the concept of dataset, I don't have experience with NFS. I an sure is much better for high bandwidth applications , but for 1MB/S per camera ,I find the good old FTP just works with everything and works reliably.

I gave up the idea of freenas as DVR , it just too much overhead for the purpose and it take too long when you delete a buch of files to clean the space for more recording. I have VM with Windows 2008 R2, that is serving as FTP Server and all cameras record non-stop to that VM. It works great and is very snappy.

 

[averyfreeman]

Thanks for getting back. Where you been over a year since I ask ?:)

A year ago I didn't know anything about cameras, I got a job doing it from a networking gig last October so you must be mistaking me for someone else. This is the first time I've replied to this thread, I was googling using IP Camera NFS with a Nix box and came across it and since I was already a member of the forum I thought I'd weigh in.

I understand the concept of dataset, I don't have experience with NFS. I an sure is much better for high bandwidth applications , but for 1MB/S per camera ,I find the good old FTP just works with everything and works reliably.

dataset is a ZFS feature, doesn't involve NFS. Like I said, basically a file folder, but managed through the ZFS command and allows features such as quota, replication, share management, compression, etc. to be managed on a more granular basis - dataset by dataset (analogous to folder by folder).

NFS is just a network sharing protocol. It's neat because it allows network shares to be mounted either by mount or /etc/fstab like a storage volume. But it does not involve datasets, unless you count:

Code:

# zfs set sharenfs=on zpool/datasetname 

Which works on Solaris/Illumos OS, but I don't think works with ZFS on any other platforms, so it probably doesn't apply to FreeNAS: https://redmine.ixsystems.com/issues/16794

I gave up the idea of freenas as DVR , it just too much overhead for the purpose and it take too long when you delete a buch of files to clean the space for more recording. I have VM with Windows 2008 R2, that is serving as FTP Server and all cameras record non-stop to that VM. It works great and is very snappy.

Yeah FreeNAS has a bunch of unnecessary features, the beauty of using NFS shares would be having an extremely bare-bones OS for your NVR and accessing it remotely. OmniOS would probably work very well for it, given that it has better ZFS support for replication, sharing, etc. and I think snapshotting (which works great in FreeNAS) would be an excellent for an NVR and is only commonly possible using ZFS, BTRFS, LVM and ReFS (oh, and VMFS6 for ESXi).

Why'd you choose Windows Server 2008R2? You could have used ReFS volumes on a newer version of Win Server, which is Microsoft's version of CoW FS like ZFS. I've used it before, it's pretty decent as long as you don't try and do a parity array (those are slow af). https://docs.microsoft.com/en-us/windows-server/storage/refs/refs-overview

Are you just using NFS? I gave up on using NFS straight from the cameras, I got it to work once but I just feel like the firmware on my cheap cameras doesn't work properly, couldn't get it working after I tried to re-configure it.

I've been running Milestone Xprotect Essentials+ and enjoying it quite a bit, it's free for up to 8 cameras.

There's also Shinobi for Ubuntu 18.04 Server which looks promising but I have had issues with getting proper versions of the NPM/node.js packages - make sure you pay close attention to the installation instructions if you try it.

 

[Black Ninja]

so you must be mistaking me for someone else.

Yes yes you are right , this was @Rickinfl I took you for , cause you reply to what I reply to him.

Why'd you choose Windows Server 2008R2? You could have used ReFS volumes on a newer version of Win Server, which is Microsoft's version of CoW FS like ZFS. I've used it before, it's pretty decent as long as you don't try and do a parity array (those are slow af). https://docs.microsoft.com/en-us/windows-server/storage/refs/refs-overview

I do believe Windows is getting worst as it gets higher. I wouldn't mess with ReFS I head of it , it too new and it's like Hyper-V, meaning not better than VMware to worth switching just cause the offer it. I don't think CoW or ZFS for that matter is good for DVR. I love ZFS for fileserver , but I don't need that much of resilience for data that was meant to be gone in a few days anyways.

Are you just using NFS? I gave up on using NFS straight from the cameras, I got it to work once but I just feel like the firmware on my cheap cameras doesn't work properly, couldn't get it working after I tried to re-configure it.

No, I don't use NFS, I use FTP to send from cameras to DVR , and is rock solid. Records 24/7 and not a single glitch. Only 3-4 cameras, but I am sure will scale well.

What is nice is that is VM , that is all configured, updated, set with all account and etc. and it can jump from one PC to another in a minutes.

 

[averyfreeman]

Cool, I'm glad FTP is working out for you. NFS is a faster, more stable protocol, which easier to integrate into the framework of the OS (doesn't require as much software to operate as FTP) so I'll stick to focusing on getting it working. I might downgrade the camera's firmware to what came on it - I'm concerned that their updates aren't tested very well. But they're very cheap cameras so I suppose I can't expect too much.

As to whether CoW FS is useful for DVRs, if you want to take snapshots without storage penalty or potentially use deduplication, then it's definitely the way to go. There's no real downside to using it other than it's important to overprovision at least 5% of your space.

For my record volume I just used an NFS share from ZFS dataset --> mapped to ESXi VM datastore --> created new vHDD on VM and that works like a native HDD inside the VM so there's no futzing with mapping network drives in Windows or anything like that. Works good as shit.

Then I take snapshots of the vHDD in ZFS that don't take up room, so if my week of recording doesn't go back long enough I can just revert to whatever date I need to look at. It's a total game-changer.

As far as hyper-V, ReFS has nothing to do with virtualization. It's a file system. It has some of the same features as ZFS and one is volume snapshots, which IMO are really the best reason for choosing CoW FS for recording video.

Are you running ESXi as your hypervisor?

 

[Black Ninja]

snapshots without storage penalty or potentially use deduplication

Here I don't quite agree on duplication part, but it's not important.

For my record volume I just used an NFS share from ZFS dataset --> mapped to ESXi VM datastore --> created new vHDD on VM and that works like a native HDD inside the VM so there's no futzing with mapping network drives in Windows or anything like that. Works good as crap.

Then I take snapshots of the vHDD in ZFS that don't take up room, so if my week of recording doesn't go back long enough I can just revert to whatever date I need to look at. It's a total game-changer.

This is something interesting , I think it's a smart idea worth thinking about it.

As far as hyper-V, ReFS has nothing to do with virtualization.

I understand that. My point was I wouldn't go with hyper-v as hypervisor over VMware , same way I wouldn't go with Refs over ZFS.

Are you running ESXi as your hypervisor?

Yes I use ESXI as hypervisor , but I don't run the DVR server from my main hypervisor. I install ESXI on bare metal (of course) and then deployed my Windows 2008 r2 VM machine serving as recorder. This hypervisor is old hardware and I don't run anything else but this DVR virtual machine for recording from the cameras. It gives me a great flexibility to move this already configured DVR VM to any hypervisor in a minutes.

So this old DELL Poweredge 860 [CPU: 2.4 GHz Quad-Core (Xeon X3220) RAM: 8 GB (ECC memory)] is running everything from itself. Fast and very reliable. This server doesn't meet the minimum reqirements for running freenas with ZFS without deduplication, but for ESXI and Windows 2008 R2 as VM , is running flawlessly.


P.S. I did consider connecting it to iscsi target, becasue recording storage inside is limited but , I didn't want to overcomlicat the setup. It was important to me to keep the DVR solution in one box, for reliability and easy to replace reasons.

 

[zeebee]

I've been running Milestone Xprotect Essentials+ and enjoying it quite a bit, it's free for up to 8 cameras.

@averyfreeman - wondering if you can explain how Xprotect works. It's a windows app, so does it have to stay running 24/7 and the cameras connect to it, or is it a 'viewer' that just reads files that your cameras have dumped on your NAS?

 

[averyfreeman]

@averyfreeman - wondering if you can explain how Xprotect works. It's a windows app, so does it have to stay running 24/7 and the cameras connect to it, or is it a 'viewer' that just reads files that your cameras have dumped on your NAS?

Hi, yep Milestone runs on Windows and it's a full suite of server-client utilities that create a comprehensive monitoring platform for cameras, 2-way mics, window/door intrusion detectors, etc. that can send alerts to mobile devices.

It works really well, but yes it is for Windows, and yes you have to leave the computer on 24/7. It's not picky about which version of windows, I'd recommend Server 2012R2, 2016LTS, 2019LTS, or Windows LTSC 2019 so you don't have feature updates taking the system down for long periods of time.

It's pretty easy to set up compared to some stuff I've seen but it's incredibly comprehensive. Thankfully it has really good documentation and a very responsive user forum.

I have everything set up in ESXi so it's not a big deal for me to spin up different Windows, FreeBSD, OmniOS, Linux VMs, etc. and share datasets between them and that's what I do.

But if you already run a dedicated FreeNAS NAS, and you only want to have one computer on all the time, I'd look into running it on ESXi w/ PCI passthrough if the computer is fast enough - chances are you have plenty of spare resources. ESXi is free.

Or you can get another computer if you want to run Windows+xprotect. Up to you!

You can't run any surveillance software straight on FreeNAS that I know of, but you can just send clips from the camera using the firmware built-in to most IP cameras. That works fine too, and all it needs is NFS, then if you want to get push notifications for motion detection alerts you can rig something up with email either through FreeNAS or from the cameras themselves.

There's lots of options!

Also if you want a full surveillance suite but want to go 100% open source, there's Shinobi for Ubuntu 18.04 LTS - it's a bit hard to set up, but it's pretty cool and very customizable.

I've tried a bunch of these surveillance packages and out of Blue Iris, iSpy, MotionEye, Shinobi and xprotect, I like xprotect best and shinobi 2nd. I haven't tried ZoneMinder.

 

[zeebee]

Thanks for the detailed response! The parts for my first FreeNAS system are in the mail at the moment, but I've been playing around with it in a VM.

I installed Shinobi in a jail, but didn't have a camera yet to give it much of a test. I also installed the Zoneminder plugin, which seemed to work fine, but certainly felt a little long in the tooth. It sounds like I might be better off installing Ubuntu in a FreeNAS VM and installing Shinobi there, so that it's on a platform it's supported on (rather than FreeBSD).

I don't love the idea of having to run windows in a VM, but I really like the sound of xprotect so I'll give it a go.

I haven't looked at ESXi yet, but I've seen mention of people running FreeNAS inside ESXi. It sounds like you're saying the opposite - that you run ESXi inside FreeNAS? I'm not sure I really understand - do you install ESXi in a FreeNAS VM, and then use it to create more VMs inside that? That doesn't sound right, so I imagine I'm missing something.

 

[zeebee]

@averyfreeman I've had a look at ESXi (installed it in VirtualBox and got some advice) and I'm going to give it a go. My first camera arrived, so I gave ZoneMinder and Shinobi a quick run with it. I didn't really give them much of a go, but neither of them worked particularly well. I imagine with more work I could get them going, but at some point I gave XProtect a run on my current windows box and it just immediately picked up my camera and just worked. Damnit, didn't want to have to run windows, but it XProtect certainly seems like the more robust solution.

One of the things I'm not sure about yet is how my ESXi VMs will get access to FreeNAS storage. You mentioned this:

I have everything set up in ESXi so it's not a big deal for me to spin up different Windows, FreeBSD, OmniOS, Linux VMs, etc. and share datasets between them and that's what I do.

How do the VMs access the FreeNAS storage? Do they need to point at a network share (e.g. SMB), or is there some other magic here that I don't know about?

Thanks for your help.