How is Dell T320?

[kbrvfx]

Hello everyone!

I need a very well thought for this decision, im currently in a region where server grade components are scarce and too pricey for the meantime. However, I found this dell t320 which is used for around $400 (estimated). Do you think it will be worth it? If I purchase it then what would be the best upgrade paths for it? And options for storage (since storage are a bit too pricey) Also, is freeNas (OS) enough for our purpose (since we're just a small business)

Purpose of the server will be basically for 1-2 editors editing clips directly from the network and directing rendered output there directly then storing video files (we're a production team of 2)

P.s (Im still new )

 

[Heracles]

Hey @kbrvfx,

I do run from Dell servers here. They are all described in my signature : a T330, a T130 and a T110. The restriction is about the number of drive that you can fit in. Depending of how you configure the server, you can be restricted to as low as 4 or 5 drives.

Also, no server, FreeNAS or other, can be more than a single point of failure. Know that you will need backups.

Depending of the drive controller you end up with, you will need to switch it to HBA mode or flash it to IT mode. It is critical for FreeNAS to have direct physical access to its drives.

Such a Dell server can be a great basis to start from. Now you need to plan for the backups, power protection, network capacity, RAM and other critical task / functions to be sure you keep your data safe.

 

[dakotta]

This summer, I scored a free T-310 that the IT people were throwing away.

I spent about $400 on upgrades and another $400 on hard drives.

2 boot disks, 120GB BX500 SATA III 2.5" Internal SSD = $45
32 GB RAM, DMS Certified Memory DDR3-1066 (PC3-8500) = $180
1 HBA, 6Gbps SAS w/ LSI 9211-8i P20 IT Mode for ZFS FreeNAS unRAID = $55
2 3.5" SAS SATA Hard Drive HDD Tray Caddy = $30
1 2.5"/3.5" to 5.25" Bay Adapter (for the SSDs) = $35
2 Mini SAS 36-Pin SFF-8087 Male to Mini SAS SFF-8087 Male 90° Right Angle = $25
1 TG-3 Thermal Grease = $15
5 WD Red Pro WD4003FFBX 4TB = $400

I am very happy with the server, so far, but the costs for retrofitting add up quickly. Next year I plan to build a new server, and this one will be used as the backup server... and eventually as the offline server.

Cheers,

Edit - the T310 is not really relevant as it is a generation older. Sorry.

 

[kbrvfx]

Hey @kbrvfx,

Depending of the drive controller you end up with, you will need to switch it to HBA mode or flash it to IT mode. It is critical for FreeNAS to have direct physical access to its drives.

Hello, may i ask if you can give me some walkthrough on how you setted up your dell server? (Im about to get the t320) and planning to use freenas on it.

 

[Heracles]

may i ask if you can give me some walkthrough on how you setted up your dell server?

Sure

So for my oldest one, the T110, there is no raid controller in that one. I did not had to do anything special. I used the two internal USB ports to install my boot devices at first. I had to rotate my USB keys and for now, I run from a single one. The config is saved in TrueCommand and I can re-install from scratch easily. There are only 4 slots designed for hard drives. For that, I removed the original DVD drive to get a 5th one. I chose to also use the external sata port to fit a 6th drive. I then connected it peer-to-peer to my main FreeNAS using IPs that are not routed or reachable from anywhere else in my network.

The T130 is also using the internal 4 spaces for hard drives, plus the DVD one. That one also have a regular controller instead of a Raid controller. I also use USB keys to boot, from the single internal port + an external one for mirrored boot devices. Again, I do regular backups with TrueCommand to ensure I can re-install the OS easily should my boot devices fail.

As for the T330, that one has a Raid controller. I had to switch it to HBA mode. For that, during the boot process, you are offered to enter in the Raid controller's bios. You do it and it is there that you can switch the controller to HBA mode. Doing it will destroy whatever virtual drives you have at that time. Once done, I installed FreeNAS, again from mirrored USB keys because there are not that many hard drives bays in that server. Again, the mirror + TrueCommand is the way I use to ensure I can always boot to my config easily.

Once installed, I confirmed that FreeNAS was able to see the actual drive (serial number) and was able to run the smart tests, long and short.

Part of that setup includes a 4x 1Gig iSCSI link to my ESXi server (an R820). For that, I added a quad port Intel NIC card and connected each of these ports to similar ports from my R820.

The iDrac is on a dedicated interface and connected to my management network.

So that is what it looks like and how I did it here.

Have fun playing with your own setup,

 

[kbrvfx]

As for the T330, that one has a Raid controller. I had to switch it to HBA mode. For that, during the boot process, you are offered to enter in the Raid controller's bios. You do it and it is there that you can switch the controller to HBA mode. Doing it will destroy whatever virtual drives you have at that time. Once done, I installed FreeNAS, again from mirrored USB keys because there are not that many hard drives bays in that server. Again, the mirror + TrueCommand is the way I use to ensure I can always boot to my config easily.

This is what was bugging me the entire time since I never had any experience with Dell products, I'll be attaining the used unit a few days from now. Thank you so much, you are a great help :D

 

[SeedyRom]

I was about to make a new post to share this info but I found this. Your question fits quite well with what I was going post so here goes...

First, to answer your initial question, I have the Dell T320 running FreeNAS as an iSCSI server for my VMWare cluster.

Power: I'm currently running 6x 1TB 7200RPM enterprise SAS drives. The machine is currently sitting at 84W and usually stays under 100W most of the time. It is very efficient on power.

Proc/RAM: Not much to say here. I have the Xeon E5-2407 and 4x 8GB DIMMs which is good enough. There are six DIMMs total so I could expand the memory a little but haven't had a reason to yet. I don't really need much more, at least in my use case.

Network: I'm running one 10Gbe DAC cable from an AOC-STGN-i2s card to a cheap Fiberstore switch and it works well. You can probably get away with 4x 1g, but make sure you buy a PCI NIC because the machine only has 2x ethernet ports onboard.

Expansion ports: It has quite a few PCI slots. I can't remember how many off hand but enough to add NVME drives, NICs, SAS cards, or whatever else you want, except GPUs.

Hard Drive backplane: It holds 8x SAS or SATA drives which is awesome. There is also a built in USB port on the motherboard which is hidden inside the case. I use this for the OS drive and it doesn't take up a HDD slot or external USB port.

Storage Controller: Let me start by saying the H330 is a marginally better card than the H310 and probably doesn't have the same issues. My research before going fowards with ZFS showed that people were complaining of the H310 card not performing well in HBA mode in regards to multi-disk read/write performance. I was previously running the H310 in a RAID10 with four 1TB disks and it didn't perform well that way either. To fix this issue, I flashed the Perc H310 with the LSI 9211-8i firmware. There are tons of threads and websites that have written endlessly about this so please just research it yourself and make sure you understand what can go wrong before you do it.

OK, now that I got that out of the way, time for the good part! Once you flash the LSI 9211 firmware onto the H310, the Dell iDRAC has a fit about it. Since it can no longer detect hard drive temperatures, it decides to set the front fans to an annoying and unnecessary level as a failsafe. I'm sure a lot of people who did this didn't care but my server lives near my desk so this was an issue for me. Looking online, I found a few people griping about it but not a lot of solutions. One solution I did find was setting the fans to a static setting (like 20% fan speed all the time). I didn't agree with this option as it would require tweaking and monitoring and I knew there had to be another way!

Well, there is another way. After reading through pages and pages of documentation from Dell, I stumbled upon the RACADM manual. RACADM is accessible by using SSH to your iDRAC port. You log in with an administrator account and type "racadm" into the console.

So the important bit for this is stuffed all the way at the bottom of this page here.

For people who don't want to translate Dell's language, here is the command to run. Just SSH to the iDRAC IP, log in as root or a user with admin rights, and run this command:

racadm set system.thermalsettings.ThirdPartyPCIFanResponse 0

And with one line of code in the right spot, I was able to get rid of the annoying noise for good . I'll admit I haven't tried to reboot yet to see if it is persistent. If it isn't, I'll just script it.

 

[Heracles]

The machine is currently sitting at 84W and usually stays under 100W

Beware here : you may take only that much power when idle, but you need much more during boot up process. Peaks can also be pretty short, too short to be measure and displayed on a UPS LCD screen. A power supply must be able to support these peaks, with a safety margin.

I use this for the OS drive and it doesn't take up a HDD slot or external USB port.

I would suggest you couple that one with an external USB device to mirror your boot. USB have a tendency to wear out, so a mirror would be a plus. In all cases, be sure to backup your config on a regular basis.

I'll admit I haven't tried to reboot yet to see if it is persistent.

It should. Consider the iDrac as an independant computer installed in your server. Rebooting the main server does not affect the controller. If you have the licence for it, you can VNC in the server using the iDrac, reboot it and you will see the complete boot up process, bios and everything, without loosing connection.

 

[SeedyRom]

All fair points, thank you for the response.

Beware here : you may take only that much power when idle, but you need much more during boot up process. Peaks can also be pretty short, too short to be measure and displayed on a UPS LCD screen. A power supply must be able to support these peaks, with a safety margin.

I'm using the iDRAC to monitor power. 80W-85W seems to be where it likes to sit. I just checked and it seems my historical data for this week isn't showing up. It does show a peak of 135W. I'll keep an eye on it and update this thread once I can show more data.

I would suggest you couple that one with an external USB device to mirror your boot. USB have a tendency to wear out, so a mirror would be a plus. In all cases, be sure to backup your config on a regular basis.

A fair point, thank you for adding this. This is a new install as of a month ago so I have not had the opportunity to do this yet. I will try to backup my config soon and look into re-doing the install with a mirror.

It should. Consider the iDrac as an independant computer installed in your server. Rebooting the main server does not affect the controller. If you have the licence for it, you can VNC in the server using the iDrac, reboot it and you will see the complete boot up process, bios and everything, without loosing connection.

I should have clarified. I'm curious if power cycling the server will cause the iDRAC to forget and make me go back in to run that command again. Not curious enough to try as everything is up and working at the moment. I've worked with Dell management going back to DS2161 IPKVMs and PEMs so I have a pretty good understanding of iDRAC. I do appreciate you trying to educate though and I will heed your advice, thank you.

 

[SeedyRom]

It's been about a month and no issues with temps or power. Here are the power trends from last week according to the iDRAC:

 

[kbrvfx]

I was about to make a new post to share this info but I found this. Your question fits quite well with what I was going post so here goes...

First, to answer your initial question, I have the Dell T320 running FreeNAS as an iSCSI server for my VMWare cluster.

Power: I'm currently running 6x 1TB 7200RPM enterprise SAS drives. The machine is currently sitting at 84W and usually stays under 100W most of the time. It is very efficient on power.

Proc/RAM: Not much to say here. I have the Xeon E5-2407 and 4x 8GB DIMMs which is good enough. There are six DIMMs total so I could expand the memory a little but haven't had a reason to yet. I don't really need much more, at least in my use case.

Network: I'm running one 10Gbe DAC cable from an AOC-STGN-i2s card to a cheap Fiberstore switch and it works well. You can probably get away with 4x 1g, but make sure you buy a PCI NIC because the machine only has 2x ethernet ports onboard.

Expansion ports: It has quite a few PCI slots. I can't remember how many off hand but enough to add NVME drives, NICs, SAS cards, or whatever else you want, except GPUs.

Hard Drive backplane: It holds 8x SAS or SATA drives which is awesome. There is also a built in USB port on the motherboard which is hidden inside the case. I use this for the OS drive and it doesn't take up a HDD slot or external USB port.

Storage Controller: Let me start by saying the H330 is a marginally better card than the H310 and probably doesn't have the same issues. My research before going fowards with ZFS showed that people were complaining of the H310 card not performing well in HBA mode in regards to multi-disk read/write performance. I was previously running the H310 in a RAID10 with four 1TB disks and it didn't perform well that way either. To fix this issue, I flashed the Perc H310 with the LSI 9211-8i firmware. There are tons of threads and websites that have written endlessly about this so please just research it yourself and make sure you understand what can go wrong before you do it.

OK, now that I got that out of the way, time for the good part! Once you flash the LSI 9211 firmware onto the H310, the Dell iDRAC has a fit about it. Since it can no longer detect hard drive temperatures, it decides to set the front fans to an annoying and unnecessary level as a failsafe. I'm sure a lot of people who did this didn't care but my server lives near my desk so this was an issue for me. Looking online, I found a few people griping about it but not a lot of solutions. One solution I did find was setting the fans to a static setting (like 20% fan speed all the time). I didn't agree with this option as it would require tweaking and monitoring and I knew there had to be another way!

Well, there is another way. After reading through pages and pages of documentation from Dell, I stumbled upon the RACADM manual. RACADM is accessible by using SSH to your iDRAC port. You log in with an administrator account and type "racadm" into the console.

So the important bit for this is stuffed all the way at the bottom of this page here.

For people who don't want to translate Dell's language, here is the command to run. Just SSH to the iDRAC IP, log in as root or a user with admin rights, and run this command:

racadm set system.thermalsettings.ThirdPartyPCIFanResponse 0

And with one line of code in the right spot, I was able to get rid of the annoying noise for good . I'll admit I haven't tried to reboot yet to see if it is persistent. If it isn't, I'll just script it.

Wow, I didn't bother to check this thread for months as it was holidays and took a break from all the stress.

Thank you so much, you and Heracles just saved my life *phew*. This project of mine resumes once the caddies arrive (around a month 'cus of shipping). If you are still planning to create a detailed guide then I'd love to read it even more! Send it to me in private or post it here when it's available.

 

[kbrvfx]

Hello everyone, just an update to my progress. I've flashed the drive controller to IT mode in a single procedure which was great. I only had 1 caddy at a time and in our place amazon wasn't the place however, we found another supplier that had the same caddy with dell so I bought five pieces. We also bought a kingston a400 120gb SSD for boot storage but had to use the data cable for the optical drive, plugged the sata cable directly to the mobo then went ahead to install which bore no problems at all. (Sad to say can't utilize the idrac at all as i don't know where to purchase a license)

In a span of 2 hours, I was able to setup TrueNas 12-U2 and built the config from the ground (ssh, smb, user perm, emails, tasks such as scrubbing, smart tests and hopefully finding a way to rsync to another server that uses external drives.) It has been a fun journey building, definitely would love to go deeper. Thank you all!

Currently diving by starting to play with Linux cli, vm's, dockers, containers and hopefully some firewalls soon.