I’m looking to upgrade some of my internal systems to 10 gigabit, and seeing some patchy/conflicting/outdated info. Does anyone have any experience with local fiber? This would be entirely isolated to within my LAN, to enable faster access to my fileserver.
Current existing hardware:
- MikroTik CSS326-24G-2S+RM, featuring 2 SFP+ ports capable of 10GbE
- File server with a consumer-grade desktop PC motherboard. I have multiple options for this one going forward, but all will have at least 1 open PCIe x4+ slot
- This file server already has an LSI SAS x8 card connected to an external DAS
- Additional consumer-grade desktop PC, also featuring an open PCIe x4 slot.
- Physical access to run a fiber cable through the ceiling/walls
My primary goal is to have these connected as fast as possible to each other, while also allowing access to the rest of the LAN. I’m reluctant to use Cat6a (which is what these are currently using) due to reports of excessive heat and instability from the SFP+ modules.
As such, I’m willing to run some fiber cables. Here is my current plan, mostly sourced from FS:
- 2x Supermicro AOC-STGN-i2S / AOC-STGN-i1S (sourced from eBay)
- 2x Intel E10GSFPSR Compatible 10GBASE-SR SFP+ 850nm 300m DOM Duplex LC/UPC MMF Optical Transceiver Module (FS P/N: SFP-10GSR-85 for the NIC side)
- 2x Ubiquiti UF-MM-10G Compatible 10GBASE-SR SFP+ 850nm 300m DOM Duplex LC/UPC MMF Optical Transceiver Module (FS P/N: SFP-10GSR-85, for the switch side)
- 2x 15m (49ft) Fiber Patch Cable, LC UPC to LC UPC, Duplex, 2 Fibers, Multimode (OM4), Riser (OFNR), 2.0mm, Tight-Buffered, Aqua (FS P/N: OM4LCDX)
I know the cards are x8, but it seems that’s only needed to max out both ports. I will only be using one port on each card.
Are fiber keystone jacks/couplers (FS P/N: KJ-OM4LCDX) a bad idea?
Am I missing something completely? Are these even compatible with each other? I chose Ubiquti for the switch SFP+ since Mikrotik doesn’t vendor-lock, AFAICT.
Location: US
I’ll have to review your post in greater detail in a bit, but some initial comments: cross vendor compatibility of xcvrs was a laudable goal failed only by protectionist business interests and the result is that the only real way to validate compatibility is to try it.
Regarding your x4 slot and the NICs being x8: does your mobo have the slot cut in such a way that it can accept a physical x8 card even though only the x4 lanes are electrically connected?
For keystone jacks, I personally use them but I try not to go wild with them, since just like with electrical or RF connectors, each one adds some amount of loss, however minor. Having one keystone jack at each end of the fibre seems like it shouldn’t be an issue at all.
Final observation for now: this plan sets up a 10 Gb network with fibre, but your use-case for now is just for a bigger pipe to your file server. Are you expecting to expand your use-cases in future? If not, the same benefit can be had by a direct fibre run from your single machine to your file server. Still 10 Gbps but no switch needed in the middle, and you have less risk of cross vendor incompatibility.
I’m short on time rn, but I’ll circle back with more thoughts soon.
Are you set on using fiber because you really really want to or need to do particularly long runs? I use DAC cables for my shorter runs, up to 7m, as they are cheaper and a little easier to work with.