FreeNAS build

[Moriar]

Hey Team,

If you could look this build over and tell me your thoughts? Now the how i got this server is merely we had a rather large order that got severely delayed and the account team gave us 2 of these boxes as a "token of good faith".

Now, i have thought of the NIC being the issue... but after doing some research I should be able to do the following (https://www.freebsd.org/doc/handbook/config-network-setup.html)

Platform
Cisco C240-M5SX

UCS-CPU-4114
2.2 GHz 4114/85W 10C/13.75MB Cache/DDR4 2400MHz (QTY 2)

UCS-MR-X16G2RS-H
16GB DDR4-2666-MHz RDIMM/PC4-21300/dual rank/x4/1.2v (QTY 16)

UCSC-F-H16003
Cisco HHHL AIC 1.6T HGST SN260 NVMe Extreme Perf High Endrnce (QTY 2)

UCSC-MLOM-C40Q-03
Cisco VIC 1387 Dual Port 40Gb QSFP CNA MLOM

UCS-HD24TB10K4KN
2.4 TB 12G SAS 10K RPM SFF HDD (4K) (QTY 24)

UCS-SD400H123X-EP
400GB 2.5in Enterprise performance 12G SAS SSD(3X endurance) (QTY 2)

 

[Chris Moore]

Now the how i got this server is merely we had a rather large order that got severely delayed and the account team gave us 2 of these boxes as a "token of good faith".

Sorry. I somehow missed that part.

 

[Chris Moore]

Now, i have thought of the NIC being the issue... but after doing some research I should be able to do the following (https://www.freebsd.org/doc/handbook/config-network-setup.html)

While that might work, it might not because FreeNAS, even though it is built on FreeBSD, is built as an appliance and certain things are not changeable. Perhaps that isn't the best way to put that, but some things, if you change them, will be reset to default on the next boot.

2.2 GHz

I would think of that as being a little on the slow side, but it might work well enough, depending on what you plan to do with these units.

16GB DDR4-2666-MHz RDIMM/PC4-21300/dual rank/x4/1.2v (QTY 16)

That is a lot of memory for a token of good faith.

Cisco HHHL AIC 1.6T HGST SN260 NVMe Extreme Perf High Endrnce (QTY 2)

Are these in card slots? Where are they installed? Maybe they could be used as cache (SLOG / L2ARC) depending on the use you have in mind for the storage.

Cisco VIC 1387 Dual Port 40Gb QSFP CNA MLOM

Is that a card or integrated on the board?

2.4 TB 12G SAS 10K RPM SFF HDD (4K) (QTY 24)

If you make them mirrors and use this for VM storage, that could give you 24 ish TB of fast storage to host virtualization.

400GB 2.5in Enterprise performance 12G SAS SSD(3X endurance) (QTY 2)

These are probably in the rear hot-swap bays for the boot drives. You might be able to use them as they are.

There is a lot I am not familiar with about the Cisco servers, but how useful this is will depend on what hardware controller all the drives are connected to. I don't see an answer for that question. Do you know?

 

[Elliot Dierksen]

I have a lot of experience with the M1, M2, and particularly the M3 series servers (Both my FreeNAS units are M3's). Some with M4's, but none yet with M5 series. I imagine there are also some twisted pair ports on the motherboard that are likely i350 10/100/1000 NIC's. I'll make a pretty strong guess that the 40G NIC's are PCIe cards. The big question is the drive controller. It is likely an LSI 12G RAID controller, but FreeNAS most likely won't play nice with that. You need a SAS controller (non RAID) where you could move the cables from the drive cage (which is likely a SAS expander) to the new controller. You might be able to get FreeNAS to work with the RAID controller in JBOD mode (I did), but it caused enough other things to be goofy that I eventually replaced them. Do you have a part # for the server bundle? I can take look.

 

[Moriar]

While that might work, it might not because FreeNAS, even though it is built on FreeBSD, is built as an appliance and certain things are not changeable. Perhaps that isn't the best way to put that, but some things, if you change them, will be reset to default on the next boot.

I would think of that as being a little on the slow side, but it might work well enough, depending on what you plan to do with these units.

That is a lot of memory for a token of good faith.

Are these in card slots? Where are they installed? Maybe they could be used as cache (SLOG / L2ARC) depending on the use you have in mind for the storage.

Is that a card or integrated on the board?

If you make them mirrors and use this for VM storage, that could give you 24 ish TB of fast storage to host virtualization.

These are probably in the rear hot-swap bays for the boot drives. You might be able to use them as they are.

There is a lot I am not familiar with about the Cisco servers, but how useful this is will depend on what hardware controller all the drives are connected to. I don't see an answer for that question. Do you know?

Chris,

Currently the board has the following,

UCSC-RAID-M5HD
Cisco 12G Modular RAID controller with 4GB cache

Which i just verified can be switched to HBA mode.

Also I planned to boot from a Raid1 Flash Card

As for the MLOM card it is a "pcie card" but is treated like it is attached to the mainboard. this is what I was going to try the drivers above for.

So here is what I planned to do with the server.

24 drives in a RAIDZ3
the 2 SSD's as the L2ARC
the pcie nvme drive will be a SLOG

This server will only be a NFS Store for our dev environment for build / teardown of multiple VM's everyday.

 

[Moriar]

I have a lot of experience with the M1, M2, and particularly the M3 series servers (Both my FreeNAS units are M3's). Some with M4's, but none yet with M5 series. I imagine there are also some twisted pair ports on the motherboard that are likely i350 10/100/1000 NIC's. I'll make a pretty strong guess that the 40G NIC's are PCIe cards. The big question is the drive controller. It is likely an LSI 12G RAID controller, but FreeNAS most likely won't play nice with that. You need a SAS controller (non RAID) where you could move the cables from the drive cage (which is likely a SAS expander) to the new controller. You might be able to get FreeNAS to work with the RAID controller in JBOD mode (I did), but it caused enough other things to be goofy that I eventually replaced them. Do you have a part # for the server bundle? I can take look.

Elliot,

Here is the full parts list.

UCSC-C240-M5SX
UCS C240 M5 24 SFF + 2 rear drives w/o CPU,mem,HD,PCIe,PS
Product Family UCSC

Serial Numbers

CLOSED

(Ship Set 2) Requested
21-Aug-2018
Delivered On

27-Aug-2018
1.1
UCS-CPU-4114
2.2 GHz 4114/85W 10C/13.75MB Cache/DDR4 2400MHz
Product Family UCS

Serial Numbers

1.2
UCS-MR-X16G2RS-H
16GB DDR4-2666-MHz RDIMM/PC4-21300/dual rank/x4/1.2v
Product Family UCS

Serial Numbers

1.3
UCSC-PCI-1-C240M5
Riser 1 incl 3 PCIe slots (x8, x16, x8); slot 3 req CPU2
Product Family UCSC

Serial Numbers
1.4
UCSC-PCI-2C-240M5
Riser 2C incll 3 PCIe slots (3 x8) supports front+rear NVMe
Product Family UCSC

Serial Numbers

1.5
UCSC-F-H16003
Cisco HHHL AIC 1.6T HGST SN260 NVMe Extreme Perf High Endrnc
Product Family UCSC

Serial Numbers



1.6
UCSC-MLOM-C40Q-03
Cisco VIC 1387 Dual Port 40Gb QSFP CNA MLOM
Product Family UCSC

Serial Numbers

1.7
UCSC-PSU1-1050W
Cisco UCS 1050W AC Power Supply for Rack Server
Product Family UCSC

Serial Numbers



1.8
CAB-C13-CBN
Cabinet Jumper Power Cord, 250 VAC 10A, C14-C13 Connectors
Product Family MDS9000

Serial Numbers



1.9
UCSC-RAILB-M4
Ball Bearing Rail Kit for C220 & C240 M4 & M5 rack servers
Product Family UCSC

Serial Numbers



1.10
CIMC-LATEST
IMC SW (Recommended) latest release for C-Series Servers.Multiple License Keys
Product Family UCSC

Serial Numbers

1.11
UCSC-HS-C240M5
Heat sink for UCS C240 M5 rack servers 150W CPUs & below
Product Family UCSC

Serial Numbers

1.12
UCSC-RSAS-240M5X
C240 Rear UCS-RAID-M5HD SAS cbl(1)kitinclfan,bkpln
Product Family UCSC

Serial Numbers

1.13
CBL-SC-MR12GM5P
Super Cap cable for UCSC-RAID-M5HD
Product Family UCSC

Serial Numbers

1.14
UCSC-SCAP-M5
Super Cap for UCSC-RAID-M5, UCSC-MRAID1GB-KIT
Product Family UCSC

Serial Numbers
1.15
UCSC-RAID-M5HD
Cisco 12G Modular RAID controller with 4GB cache
Product Family UCSC

Serial Numbers

1.16
UCS-HD24TB10K4KN
2.4 TB 12G SAS 10K RPM SFF HDD (4K)
Product Family UCS

Serial Numbers

1.17
UCS-SD400H123X-EP
400GB 2.5in Enterprise performance 12G SAS SSD(3X endurance)
Product Family UCS

Serial Numbers

 

[Elliot Dierksen]

UCSC-RAID-M5HD
Cisco 12G Modular RAID controller with 4GB cache

Which i just verified can be switched to HBA mode.

I would expect this to be a source of problems although you can try it.

I looked at the spec sheet, and there are (2) 1/10Gb TP NIC's on the motherboard, and I think the 10/40Gb NIC's are mLOM add in modules. That is what it looks like in the spec sheet. Other than the drive controller, this should be a pretty boss system! If the second one needs a home, I'll be happy to pay the shipping!!!

24 drives in a RAIDZ3
the 2 SSD's as the L2ARC
the pcie nvme drive will be a SLOG

How you structure the pool is another question. RAIDZ2 with 6-8 disks per vdev seems to be the consensus. You choice would then be 3 x 8 drive RZ2 or 4 x 6 drive RZ2. The latter would perform better, but you get 66% effective storage instead of 75% effective storage. There is also some debate about VM performance being better on striped mirrors (artist formerly known as RAID1). If you want a single pool, you will have to make some compromises.

the 2 SSD's as the L2ARC
the pcie nvme drive will be a SLOG

This server will only be a NFS Store for our dev environment for build / teardown of multiple VM's everyday.

So the SLOG is a good thing, especially for NFS. The L2ARC (as I understand) will be the most help to you if you have repetitive reads of the same data. I don't think that could hurt anything, but it might not gain you anything either.

 

[Moriar]

I would expect this to be a source of problems although you can try it.

I looked at the spec sheet, and there are (2) 1/10Gb TP NIC's on the motherboard, and I think the 10/40Gb NIC's are mLOM add in modules. That is what it looks like in the spec sheet. Other than the drive controller, this should be a pretty boss system! If the second one needs a home, I'll be happy to pay the shipping!!! :)



How you structure the pool is another question. RAIDZ2 with 6-8 disks per vdev seems to be the consensus. You choice would then be 3 x 8 drive RZ2 or 4 x 6 drive RZ2. The latter would perform better, but you get 66% effective storage instead of 75% effective storage. There is also some debate about VM performance being better on striped mirrors (artist formerly known as RAID1). If you want a single pool, you will have to make some compromises.



So the SLOG is a good thing, especially for NFS. The L2ARC (as I understand) will be the most help to you if you have repetitive reads of the same data. I don't think that could hurt anything, but it might not gain you anything either.

Yeah the L2ARC was the one thing i was kinda debating about, since these VM's really wont be "Alive" for long, just enough to be functional and then run some scripts against then tear down and then repeat. The only benefit i kinda thought i might get out of it was the caching (prolly poor choice of word here) the golden images so they could be faster deployed.

I would definitely be open to collaborating with people on this build / venture with the product. I built a home freenas about 6 months ago and have loved every bit of it. But now branching into the "Big boy" league is where I would love to take any comment / criticism and run with it.

 

[Elliot Dierksen]

I haven't moved into the 12G SAS world yet (home servers are on my dime), so I can't say exactly which LSI controller would be right for that. I think M5's still have SD card slots, so you could use that as boot media. You would want the system logs, etc on one of the other pools. I'll take a look at the LSI site when I get back later.

 

[Elliot Dierksen]

I expect what you would want is an LSI 9300-8i or possibly a 9300-16i. I am not 100% certain on the kind of internal cables, but I suspect (with a marginal degree of confidence) that the 9300 series SAS controllers use the same cabling as the RAID controller which is probably a MegaRAID SAS Invader card if it is the same as the M4's which seems likely. Actually on second look, the ordering guide references a 9400-8e for external enclosures, so a 9400-8i would probably be the way to go. If you install a card that the CIMC doesn't know about, it gets pissed and tries to turn the fans into a Cesna on taxi for takeoff. Not a big deal unless you sit next to the server. A quick google search looks like the 9400-8i's would be ~= $350. I would say that is worth it for as nice a server as you have. I don't think it is worth it trying to bludgeon the 12G RAID card into submission because I think you will have flaky issues with it.

Edit: The MegaRAID Invader is an LSI 9361-8i card.

 

[Moriar]

Well I got it to see the drives just fine. So that part is good, now the only issue is getting the 40g nics to work.. unfortunately ndisgen command is missing.

Is there anyway freenas can comeup with a driver lolol?

 

[Elliot Dierksen]

what do you get from lspci and pciconf -lv? Ideally that output would be inside code tags.

 

[Moriar]

I will check that out in a few minutes

 

[Chris Moore]

Well I got it to see the drives just fine. So that part is good

Can it see the SMART status of the drives though? That is something that is often not passed through a RAID controller.

 

[wblock]

Well I got it to see the drives just fine. So that part is good, now the only issue is getting the 40g nics to work.. unfortunately ndisgen command is missing.

Is there anyway freenas can comeup with a driver lolol?

ndisgen is very old (Windows 95/98 era is what I recall) and unlikely to work with modern stuff. Even back when it was created, it did not always work.

The fastest way to get networking will be to just to get some supported network cards. It might be possible that a newer version of FreeNAS will support these, or that they are actually OEM cards that will work with an existing driver with slight modifications (PCI vendor and card IDs), that will take time and acquired expertise.

 

[Elliot Dierksen]

If it isn't identified (still looking for the output of the commands I mentioned earlier), that is probably a very bad sign. I found this statement on the data sheet for the 1387 VIC https://www.cisco.com/c/en/us/produ...ing-system-adapters/datasheet-c78-736683.html.

That makes me think FreeBSD/FreeNAS hasn't been invited to the party as of yet.

 

[Moriar]

Can it see the SMART status of the drives though? That is something that is often not passed through a RAID controller.

Yes smart status is being passed thru

 

[Chris Moore]

Yes smart status is being passed thru

That is an important hurdle. It will allow FreeNAS to detect drive faults. We have seen systems where the drive health was masked by the controller and the first indication of a problem with a drive was data errors on the pool. Early detection of drive faults allows the drive to be replaced, hopefully, before any data errors occur.

Are any of the Network Interfaces showing up in FreeNAS?

 

[Moriar]

what do you get from lspci and pciconf -lv? Ideally that output would be inside code tags.

root@freenas:~ # lspci
00:00.0 Host bridge: Intel Corporation Device 2020 (rev 04)
00:04.0 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
00:04.1 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
00:04.2 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
00:04.3 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
00:04.4 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
00:04.5 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
00:04.6 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
00:04.7 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
00:05.0 System peripheral: Intel Corporation Sky Lake-E MM/Vt-d Configuration Registers (rev 04)
00:05.2 System peripheral: Intel Corporation Device 2025 (rev 04)
00:05.4 PIC: Intel Corporation Device 2026 (rev 04)
00:08.0 System peripheral: Intel Corporation Sky Lake-E Ubox Registers (rev 04)
00:08.1 Performance counters: Intel Corporation Sky Lake-E Ubox Registers (rev 04)
00:08.2 System peripheral: Intel Corporation Sky Lake-E Ubox Registers (rev 04)
00:11.0 Unassigned class [ff00]: Intel Corporation Device a1ec (rev 04)
00:11.1 Unassigned class [ff00]: Intel Corporation Device a1ed (rev 04)
00:11.5 SATA controller: Intel Corporation Lewisburg SSATA Controller [AHCI mode] (rev 04)
00:14.0 USB controller: Intel Corporation Lewisburg USB 3.0 xHCI Controller (rev 04)
00:14.2 Signal processing controller: Intel Corporation Device a1b1 (rev 04)
00:16.0 Communication controller: Intel Corporation Lewisburg CSME: HECI #1 (rev 04)
00:16.1 Communication controller: Intel Corporation Lewisburg CSME: HECI #2 (rev 04)
00:16.4 Communication controller: Intel Corporation Lewisburg CSME: HECI #3 (rev 04)
00:1c.0 PCI bridge: Intel Corporation Lewisburg PCI Express Root Port #1 (rev f4)
00:1c.4 PCI bridge: Intel Corporation Lewisburg PCI Express Root Port #5 (rev f4)
00:1f.0 ISA bridge: Intel Corporation Lewisburg LPC Controller (rev 04)
00:1f.2 Memory controller: Intel Corporation Lewisburg PMC (rev 04)
00:1f.5 Serial bus controller [0c80]: Intel Corporation Lewisburg SPI Controller (rev 04)
01:00.0 Ethernet controller: Intel Corporation Ethernet Controller 10G X550T (rev 01)
01:00.1 Ethernet controller: Intel Corporation Ethernet Controller 10G X550T (rev 01)
03:00.0 PCI bridge: Emulex Corporation x1 PCIe Gen2 Bridge[Pilot4]
04:00.0 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200e [Pilot] ServerEngines (SEP1) (rev 42)

 

[Moriar]

Here is the other output

root@freenas:~ # pciconf -lv
hostb0@pci0:0:0:0: class=0x060000 card=0x00008086 chip=0x20208086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
class = bridge
subclass = HOST-PCI
none0@pci0:0:4:0: class=0x088000 card=0x00008086 chip=0x20218086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CBDMA Registers'
class = base peripheral
none1@pci0:0:4:1: class=0x088000 card=0x00008086 chip=0x20218086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CBDMA Registers'
class = base peripheral
none2@pci0:0:4:2: class=0x088000 card=0x00008086 chip=0x20218086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CBDMA Registers'
class = base peripheral
none3@pci0:0:4:3: class=0x088000 card=0x00008086 chip=0x20218086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CBDMA Registers'
class = base peripheral
none4@pci0:0:4:4: class=0x088000 card=0x00008086 chip=0x20218086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CBDMA Registers'
class = base peripheral
none5@pci0:0:4:5: class=0x088000 card=0x00008086 chip=0x20218086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CBDMA Registers'
class = base peripheral
none6@pci0:0:4:6: class=0x088000 card=0x00008086 chip=0x20218086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CBDMA Registers'
class = base peripheral
none7@pci0:0:4:7: class=0x088000 card=0x00008086 chip=0x20218086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CBDMA Registers'
class = base peripheral
none8@pci0:0:5:0: class=0x088000 card=0x00008086 chip=0x20248086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E MM/Vt-d Configuration Registers'
class = base peripheral
none9@pci0:0:5:2: class=0x088000 card=0x00000000 chip=0x20258086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
class = base peripheral
ioapic0@pci0:0:5:4: class=0x080020 card=0x20268086 chip=0x20268086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
class = base peripheral
subclass = interrupt controller
none10@pci0:0:8:0: class=0x088000 card=0x00008086 chip=0x20148086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E Ubox Registers'
class = base peripheral
none11@pci0:0:8:1: class=0x110100 card=0x00008086 chip=0x20158086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E Ubox Registers'
class = dasp
subclass = performance counters
none12@pci0:0:8:2: class=0x088000 card=0x00008086 chip=0x20168086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E Ubox Registers'
class = base peripheral
none13@pci0:0:17:0: class=0xff0000 card=0x01011137 chip=0xa1ec8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
none14@pci0:0:17:1: class=0xff0000 card=0x01011137 chip=0xa1ed8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
ahci0@pci0:0:17:5: class=0x010601 card=0x01011137 chip=0xa1d28086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Lewisburg SSATA Controller [AHCI mode]'
class = mass storage
subclass = SATA
xhci0@pci0:0:20:0: class=0x0c0330 card=0x01011137 chip=0xa1af8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Lewisburg USB 3.0 xHCI Controller'
class = serial bus
subclass = USB
none15@pci0:0:20:2: class=0x118000 card=0x01011137 chip=0xa1b18086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
class = dasp
none16@pci0:0:22:0: class=0x078000 card=0x01011137 chip=0xa1ba8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Lewisburg CSME: HECI'
class = simple comms
none17@pci0:0:22:1: class=0x078000 card=0x01011137 chip=0xa1bb8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Lewisburg CSME: HECI'
class = simple comms
none18@pci0:0:22:4: class=0x078000 card=0x01011137 chip=0xa1be8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Lewisburg CSME: HECI'
class = simple comms
pcib1@pci0:0:28:0: class=0x060400 card=0x01011137 chip=0xa1908086 rev=0xf4 hdr=0x01
vendor = 'Intel Corporation'
device = 'Lewisburg PCI Express Root Port'
class = bridge
subclass = PCI-PCI
pcib2@pci0:0:28:4: class=0x060400 card=0x01011137 chip=0xa1948086 rev=0xf4 hdr=0x01
vendor = 'Intel Corporation'
device = 'Lewisburg PCI Express Root Port'
class = bridge
subclass = PCI-PCI
isab0@pci0:0:31:0: class=0x060100 card=0x01011137 chip=0xa1c18086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Lewisburg LPC Controller'
class = bridge
subclass = PCI-ISA
none19@pci0:0:31:2: class=0x058000 card=0x01011137 chip=0xa1a18086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Lewisburg PMC'
class = memory
none20@pci0:0:31:5: class=0x0c8000 card=0x01011137 chip=0xa1a48086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Lewisburg SPI Controller'
class = serial bus
ix0@pci0:1:0:0: class=0x020000 card=0x01a41137 chip=0x15638086 rev=0x01 hdr=0x00
vendor = 'Intel Corporation'
device = 'Ethernet Controller 10G X550T'
class = network
subclass = ethernet
ix1@pci0:1:0:1: class=0x020000 card=0x01a41137 chip=0x15638086 rev=0x01 hdr=0x00
vendor = 'Intel Corporation'
device = 'Ethernet Controller 10G X550T'
class = network
subclass = ethernet
pcib3@pci0:3:0:0: class=0x060400 card=0x00000000 chip=0x012019a2 rev=0x00 hdr=0x01
vendor = 'Emulex Corporation'
device = 'x1 PCIe Gen2 Bridge[Pilot4]'
class = bridge
subclass = PCI-PCI
vgapci0@pci0:4:0:0: class=0x030000 card=0x01011137 chip=0x0522102b rev=0x42 hdr=0x00
vendor = 'Matrox Electronics Systems Ltd.'
device = 'MGA G200e [Pilot] ServerEngines (SEP1)'
class = display
subclass = VGA
pcib5@pci0:23:0:0: class=0x060400 card=0x00008086 chip=0x20308086 rev=0x04 hdr=0x01
vendor = 'Intel Corporation'
device = 'Sky Lake-E PCI Express Root Port A'
class = bridge
subclass = PCI-PCI
pcib6@pci0:23:2:0: class=0x060400 card=0x00008086 chip=0x20328086 rev=0x04 hdr=0x01
vendor = 'Intel Corporation'
device = 'Sky Lake-E PCI Express Root Port C'
class = bridge
subclass = PCI-PCI
none21@pci0:23:5:0: class=0x088000 card=0x00008086 chip=0x20348086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
class = base peripheral
none22@pci0:23:5:2: class=0x088000 card=0x00000000 chip=0x20358086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E RAS Configuration Registers'
class = base peripheral
ioapic1@pci0:23:5:4: class=0x080020 card=0x20368086 chip=0x20368086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
class = base peripheral
subclass = interrupt controller
none23@pci0:23:8:0: class=0x088000 card=0x00008086 chip=0x208d8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CHA Registers'
class = base peripheral
none24@pci0:23:8:1: class=0x088000 card=0x00008086 chip=0x208d8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CHA Registers'
class = base peripheral
none25@pci0:23:8:2: class=0x088000 card=0x00008086 chip=0x208d8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CHA Registers'
class = base peripheral
none26@pci0:23:8:3: class=0x088000 card=0x00008086 chip=0x208d8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CHA Registers'
class = base peripheral
none27@pci0:23:8:4: class=0x088000 card=0x00008086 chip=0x208d8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CHA Registers'
class = base peripheral
none28@pci0:23:8:5: class=0x088000 card=0x00008086 chip=0x208d8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CHA Registers'
class = base peripheral
none29@pci0:23:8:6: class=0x088000 card=0x00008086 chip=0x208d8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CHA Registers'
class = base peripheral
none30@pci0:23:8:7: class=0x088000 card=0x00008086 chip=0x208d8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CHA Registers'
class = base peripheral
none31@pci0:23:9:0: class=0x088000 card=0x00008086 chip=0x208d8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CHA Registers'
class = base peripheral
none32@pci0:23:9:1: class=0x088000 card=0x00008086 chip=0x208d8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CHA Registers'
class = base peripheral
none33@pci0:23:14:0: class=0x088000 card=0x00008086 chip=0x208e8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CHA Registers'
class = base peripheral
none34@pci0:23:14:1: class=0x088000 card=0x00008086 chip=0x208e8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CHA Registers'
class = base peripheral
none35@pci0:23:14:2: class=0x088000 card=0x00008086 chip=0x208e8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CHA Registers'
class = base peripheral
none36@pci0:23:14:3: class=0x088000 card=0x00008086 chip=0x208e8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CHA Registers'
class = base peripheral
none37@pci0:23:14:4: class=0x088000 card=0x00008086 chip=0x208e8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CHA Registers'
class = base peripheral
none38@pci0:23:14:5: class=0x088000 card=0x00008086 chip=0x208e8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CHA Registers'
class = base peripheral
none39@pci0:23:14:6: class=0x088000 card=0x00008086 chip=0x208e8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CHA Registers'
class = base peripheral
none40@pci0:23:14:7: class=0x088000 card=0x00008086 chip=0x208e8086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Sky Lake-E CHA Registers'
class = base peripheral