need some help to implement iscsi san

baby27784

Dabbler
Joined
May 12, 2019
Messages
22
hello everyone
i,m gonna to use our old server as iscsi san for vmware machine( SQL 2012-windows server 2012 ,2016-exchange 2016-file server-crm) and need some guide to do that:

my server DL380 G7 16 bay
cpu : 2*5620
ram : 16 gig
nic : 4*1 gig

hdd: 7*146g 10k
5*300g 10K
4*146g 15k

1-is 16 gig ram enough for good performance?
2-which type of cat6 cable should I use for connecting esxi and iscsi server? (UTP , FTP,STP or SFTP)
3-what is best way to raid our hdd ?
4-should i use separate vlan for iscsi traffic?
5-shoud i have HBA iscsi?
6-can i bond nic in iscsi storage for better bandwidth?

thank for your answers.

STORAGE.jpg
 

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
1-is 16 gig ram enough for good performance?
No.
2-which type of cat6 cable should I use for connecting esxi and iscsi server? (UTP , FTP,STP or SFTP)
You said that you have a 1Gb NIC. So, this is pretty much not even a consideration, you could use Cat5e. If you had 10Gb networking, then it might matter, but even then the fancy shielded cable is usually only needed in electrically noisy environments. The "normal" UTP cable is fine.
3-what is best way to raid our hdd ?
For iSCSI, which is block storage, the best practice is mirrored pairs to give you more vdevs which roughly translates into more IOPS.
5-shoud i have HBA iscsi?
In the DL380, your disks should NOT connect to a hardware RAID controller. All disks used by FreeNAS should be connected to an IT mode controller.
6-can i bond nic in iscsi storage for better bandwidth?
No. If you want better bandwidth, you will need to go to 10Gb networking and the best value in 10Gb networking is used gear from eBay, which is very economical now because many large data-centers are dumping 10Gb hardware and even 40Gb hardware in favor of faster solutions.

Some reading material to help your in your process:

Slideshow explaining VDev, zpool, ZIL and L2ARC
https://forums.freenas.org/index.ph...ning-vdev-zpool-zil-and-l2arc-for-noobs.7775/

Terminology and Abbreviations Primer
https://forums.freenas.org/index.php?threads/terminology-and-abbreviations-primer.28174/

Don't be afraid to be SAS-sy
https://forums.freenas.org/index.php?resources/don't-be-afraid-to-be-sas-sy.48/

Confused about that LSI card? Join the crowd ...
https://www.ixsystems.com/community/threads/confused-about-that-lsi-card-join-the-crowd.11901/

Building, Burn-In, and Testing your FreeNAS system
https://forums.freenas.org/index.php?resources/building-burn-in-and-testing-your-freenas-system.38/

Github repository for FreeNAS scripts, including disk burnin
https://forums.freenas.org/index.ph...for-freenas-scripts-including-disk-burnin.28/

solnet-array-test (for drive / array speed) non destructive test
https://forums.freenas.org/index.php?resources/solnet-array-test.1/

Useful Commands
https://forums.freenas.org/index.php?threads/useful-commands.30314/#post-195192

Hard Drive Troubleshooting Guide (All Versions of FreeNAS)
https://forums.freenas.org/index.ph...bleshooting-guide-all-versions-of-freenas.17/

10 Gig Networking Primer
https://forums.freenas.org/index.php?resources/10-gig-networking-primer.42/

The ZFS ZIL and SLOG Demystified
http://www.freenas.org/blog/zfs-zil-and-slog-demystified/

Some insights into SLOG/ZIL with ZFS on FreeNAS
https://forums.freenas.org/index.php?threads/some-insights-into-slog-zil-with-zfs-on-freenas.13633/

Testing the benefits of SLOG using a RAM disk!
https://forums.freenas.org/index.php?threads/testing-the-benefits-of-slog-using-a-ram-disk.56561/

SLOG benchmarking and finding the best SLOG
https://forums.freenas.org/index.php?threads/slog-benchmarking-and-finding-the-best-slog.63521/
 

baby27784

Dabbler
Joined
May 12, 2019
Messages
22
Thanks for Answer

how much RAM needed for 2T data?

For iSCSI, which is block storage, the best practice is mirrored pairs to give you more vdevs which roughly translates into more IOPS.

I dont understand : You mean should not use Server Storage Array before present LUN to freenas?

No. If you want better bandwidth, you will need to go to 10Gb networking and the best value in 10Gb networking is used gear from eBay, which is very economical now because many large data-centers are dumping 10Gb hardware and even 40Gb hardware in favor of faster solutions.

our infrastructure is 1Gb and company has no budget to buy new device so We want do the best by our asset.
 

baby27784

Dabbler
Joined
May 12, 2019
Messages
22
how to setup networking for this topology?
 

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
how to setup networking for this topology?
Nothing special needed for 1Gb networking. Just about any hardware you slap together can handle 1Gb networking, it is super slow. Big datacenters are retiring 10Gb and 40Gb network gear in favor of 25Gb and 100Gb network gear, 1Gb is trivial at this point. It is like a string between two tin cans.
 

baby27784

Dabbler
Joined
May 12, 2019
Messages
22
STORAGE (1).jpg
you mean that it lead to bed performance? we can not buy 10G device right now.

in my topology how can i config LACP?

my understanding:

1-use separate storage vlan on 3570G for connecting all port from freenas server and hosts to Storage vlan.
2-create separate LAG on each host and config lacp on corresponding switch port in Storage Vlan.
3-create one LAG on freenas server and config lacp on corresponding switch port in Storage Vlan.
4-use jumbo frame at host vswitch and 3750G and freenas server.
 

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
It is 1Gb where the maximum throughput is about 112MB/s. That's awesome speed for a disk 20 years ago, not so much today.
Then you want to share that limited bandwidth among multiple servers. It's going to be slow, really slow.
 

baby27784

Dabbler
Joined
May 12, 2019
Messages
22
It is 1Gb where the maximum throughput is about 112MB/s. That's awesome speed for a disk 20 years ago, not so much today.
Then you want to share that limited bandwidth among multiple servers. It's going to be slow, really slow.

what if I buy just one 10G nic for freenas server?
or all path should be 10G (switch and esi nics)
 

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,740
I don't understand : You mean should not use Server Storage Array before present LUN to freenas?
No. You should always give raw disks to FreeNAS and never use so called "hardware RAID".

Kind regards
Patrick
 

baby27784

Dabbler
Joined
May 12, 2019
Messages
22
my implementation until now:

network phase:

hosts to switch 3750G :LACP dont work for iscsi vmkernel port so , 2 link from every host to switch 3750G by MPIO

freenas to switch 3750G : 3 link from freenas to witch 3750G by LACP

---------------------------------------------------------------------------------------------------------------------

storage phase:

based on my HDD :
I config each sas hdd in raid0 separately to by pass Hardware Raid and disable Write cache on it.


7*146g 10k raid z for file server vm

5*300g 10K raidz for file server vm

4*146g 15k 2*2 mirror vdev pool for database vm




STORAGE (1) (1).jpg



1-is my network config correct?

2-after that, based on my disks should I create 3 different pool? one for 145G 15k disks one for 146G 10k and one for 300G 10k?

3-I dont have ssd drive for cache. any advice for tunning performance based on what I have?

4- how can I measure IOPS performance?

thanks for your help.
 
Last edited:
Top