How interesting that the issue of PCIe lanes should come up. I knew that PCIe was a form of switched network fabric (as opposed to legacy PCI which was a true bus), but it never before occurred to me to consider how many "ports" the fabric supported, and where they were coming from. My first hint this could be an issue came from (surprise!) the Supermicro site which very helpfully lists all the various PCIe lane allocations available on each motherboard.
What's less clear is where the PCIe lanes are coming from. Some come from the CPU (some Xeons sport 40 lanes), and some come from the PCH, and it's not always clear how these are parceled out. This quasi-dynamic re-jiggering of PCIe lanes also makes me question the value of SLI graphics, particularly on consumer boards which seem to have in the neighborhood of 30-ish lanes, if you're lucky. "Congratulations! You now have quad-SLI running! However, instead of one PCIe x16 slot, you now have four PCIe x4 slots. Feel the trade-off!!"
So, nataurally, in the event I go crazy and attempt SLI with the graphics cards, I'd like to have two PCIe x16 slots available (future upgradability, don'cha know). What should I be looking at to best understand the total PCIe picture on a given motherboard?