It Is All About BTC, LTC, ETH, DOGE, KAS mining as well as other alternative crypto currencies
The defacto standard at the moment for building multi-GPU mining rigs are the x1-x16 PCI Express USB 3.0 powered extenders/risers used to connect the video cards to the motherboard and everybody is using them. Due to the high popularity and demand there are already numerous variations available and since everyone wants to save on the costs for the hardware to maximize the profits people may end up with lower quality extenders as a result. Extenders are not the thing you want to compromise with both in terms of quality or in terms of price as they are usually connected and the difference in cost is usually not that much so saving some cents or a dollar per extender may not be the wisest thing you want to do…
We wanted to share some tips and advice on what to choose as well as what are some of the more common issues with these PCI-E to USB 3.0 extenders based on our experience so far. These risers boards essentially move the video cards away from the motherboard and have enough space in between them to ensure good cooling. They consist of a couple of components – main board with the video card slot, smaller PCI-E board for connecting to the motherboard, USB 3.0 cable and an optional power adapter (Molex to SATA Power, SATA Power to Molex, PCI-E Power to SATA Power or some other variation). Starting up with the color of the PCB of the raiser board, it seems that black and blue boards are generally of higher build quality than the green ones, so go for either of the two colors if you have the option to avoid green boards.
Starting up with the main board where most things can go wrong, even though rarely there is an issue with these, you can still get into plenty of trouble. The number one problem we’ve experienced with problematic and non-working extenders is due to poor soldering of elements. So checking out the solder joints is always a good idea even before installing the extenders and not after that when there might be something wrong. Check the solder connections of the connectors and the few elements on the board – if everything is in place and makes a good contact with the PCB.
Especially check the soldering of the voltage regulator as if not properly soldered everything might seem to be working fine, but you may still not get image outputted to the display and the GPU not working properly. We’ve also seen bad batches of extenders with the wrong LDO voltage regulators soldered on the PCB, so you might want to check the markings on these as well and verify that they are with 3.3V output. These do convert the 5V power input to 3.3V output required by the GPU as you are not directly providing that voltage and the GPU does need it to function properly.
Also check for solder leakage that may be shorting things on the board, especially around the power connectors. We’ve had some cases of bad soldering where the ground pins of the Molex power connector were shorting out with the 12V pins of the empty space for PCI-E Power available on the board for the extender as these are pretty close to each other. Power shorts like this one usually do trigger the built-in protection of the Power Supply and it does not want to turn on at all or immediately shuts down when you try to power on the system.
The smaller board that connects to a PCI-E slot on the motherboard rarely has issues and if it does it is a result of bad soldering of the USB connector on it. So far we’ve only seen really sloppy soldering job on some riser boards with green color PCBs where they were literally saving on solder and some of the pins had bad connection. That is easily fixable by resoldering the pins of the connector that are actually being used (not all of them are actually used). Other than that there is not much else that can go wrong here. You can use a multimeter to check the connection points on both boards through the USB cable with a multimeter just to be sure if you are experiencing some issues.
We’ve seen a lot of different USB 3.0 cables being used with the PCI-E to USB 3.0 riser boards, but never had an issue with any of the cables, so he chances that something is wrong with the data cable are pretty slim. These cables are used for data transfer only, no power goes through them from the motherboard to the video card and that is why there is a power connector on the board of the extender. Although the name mentions USB 3.0 and we actually use USB 3.0 cables, there is no USB connectivity or support anywhere with these devices. They only use USB 3.0 cables as good quality shielded wires for extending a couple of data lines from the motherboard to the video card. You are not able to plug in a video card to a regular USB 3.0 slot with these extenders, it will not work, so do not try doing it.
There are a couple of different power connectors available on different PCI-E to USB 3.0 extenders and in order to improve compatibility the package usually contains some sort of a power adapter that usually converts whatever connector is on the PCB to SATA Power. Most modern power supplies have a lot of SATA Power connectors and not as many Molex or other power connectors. Still going for the extenders with the standard 4-pin Molex power connector is probably the best choice, even if you have to use the supplied Molex to SATA Power adapter. Just make sure that you do not connect more than two extenders to a single power line going to the power supply as even though this may not be a problem for the Molex or SATA power we’ve seen many cases where it becomes an issue for the modular power connector on the power supply side resulting in melted plastic of the connector, loose connection and issues with that. A while ago we’ve published some more useful information about power cables and power supplies that you might have missed, but checking it out may also save you some possible trouble in the future.
When looking for the best settings for GPUs that will be used for crypto currency mining it is often considered a good practice to optimize them for better efficiency and not for maximum performance. Going for the maximum performance often results is overclocking and thus higher power usage for the extra few hashes, not to mention the additional heat and as a result the overall efficiency may not be as good. If you are looking for the optimal efficiency you will most likely try to reduce the power usage of the GPU to decrease the power usage and heat output and not sacrifice any or at the cost of just a little performance drop. This is exactly what we are going to be doing now with the recently announced Nvidia GeForce GTX 1080 Ti Founders Edition video card, trying to find the sweet spot in terms of efficient (best hashrate per Watt of power used)…
For the purpose of our tests we are using the latest NiceHash Excavator v1.1.4a miner running on the Equihash algorithm used by Zcash (ZEC). Do note that other algorithms may need different settings for reaching better efficiency than the one tested here. Currently the Equihash algorithm is among the most profitable to mine on Nvidia CUDA GPUs, so we are focusing on it. Since all recent GPUs from Nvidia have both a base operating frequency and a boost operating frequency and the video card is managing the optimal one based on factors such as TDP and temperature it is easy to look for better efficiency just by lowering the TDP limit. This will essentially result in lowering the maximum boost frequency of the GPU and is an easy and very good thing to start from, if you wan to dig deeper you may also try to lower the operating voltage of the GPU in order to further improve the efficiency by lowering the power usage.
In the table above we start with the GTX 1080 Ti running at the maximum TDP level that is allowed with the +20% increase of the Power Limit meaning 250W default TDP + 50W increase or a total of 300W allowed. At this maximum allowed level you cannot expect to be anywhere near the optimum efficiency, not to mention that the GPU may not be able to reach that power usage anyway without further overclocking. We are however going to stay at the default settings and not overclock, playing only with the boost frequency of the GPU by lowering the TDP. The final result showed that the optimum efficiency in terms of hashrate per Watt is with around 60% TDP or about 150W for the GTX 1080 Ti… that is for the Equihash algorithm used by Zcash (ZEC). With that setting the operating frequency of the GPU stays at just a bit shy of 1500 MHz, or to be more precise at the 1480 MHz base operating frequency. What essentially this means is that while the extra Boost frequency may rise the performance you get, the more it scales up, the less efficient the GPU becomes in terms of performance per Watt of power used. No wonder Nvidia has chosen this particular operating frequency as the base one for the GTX 1080 Ti, and the GPU manages to keep it up with a TDP of just 150W for mining the Equihash algorithm. Do note however that other mining algorithms, especially more GPU dependent, may need more power for their efficiency sweet spot on the GTX 1080 Ti.
We got a tip about a motherboard that should be capable of driving seven video cards for mining, namely the MSI Z170A Gaming Pro Carbon, so we got one to give it a try. It is a Z170A Intel chipset motherboard for Socket 1151 processors (there are cheap Celerons available for it) and it uses the newer DDR4 memory. The motherboard is even packed with some fancy extras that are of no use for the purpose of mining, but they do increase the price of the product. Also do note that even if a motherboard has 7x PCI-E slots that does not mean that it will be able to work with 7 video cards connected with x1 to x16 PCI-E risers, it only means that there is a chance that it might work. Unfortunately there are actually not that much such motherboards available and the only few are usually high-end boards with a higher price tag and that is making them not so attractive for miners, especially compared to the use of the popular AsRock H81 Pro BTC motherboard for 6x GPU mining rigs.
The first thing you need to do with this motherboard, prior to attaching any external video card, is to make sure it is running the latest BIOS and if not to update it to version 1.7. This motherboard BIOS is just a few months old and is apparently what allows the particular motherboard to be able to run up to 7 video cards, so make sure you have it flashed before continuing.
The next step is to set some things in the motherboard BIOS. You need to go to the Settings \ Advanced \ PCI Subsystem Settings and switch the PEG0 and PEG1 Max Link Speed to Gen1 as well as set the Above 4G Decoding option to Enabled. These are pretty much the only settings you need to set in order to be able to run 7x GPUs, just make sure you set them in the BIOS before you start connecting the video cards otherwise you may experience instability of the system or it even not properly booting and allowing you to enter the BIOS. So set them first and then everything should be working fine, or almost.
You can see that Windows 10 properly sees 6x R9 390 video cards and a single RX 480 GPU connected in the device manager with the video driver installed and working properly. We actually have a weird issue with this motherboard as after setting up what is needed for it to work with 7x GPUs we are not able to properly get inside the BIOS of the motherboard. If at boot we press F9 to enter the BIOS the system just freezes and needs a restart, if we don’t press the F9 BIOS key it boost into the operating system and works just fine with seven video cards mining or whatever. In order to get inside the BIOS of the motherboard however we’ve had to clear the CMOS memory and redo the settings each time. We thought that it might be a KVM issue, but even with direct PS2 and USB keyboard connection we had the strange F9 freezing issue most of the time, even though a few times it worked just fine actually loading the BIOS and not freezing the system.
Anyway, while we are not entirely sure what is causing the BIOS freezing issue when you try to get inside with F9 after you set the required options to make the motherboard work with 7x GPUs everything else seems to work just fine. The system boots and sees all 7 video cards, even if they are different ones like in out 6+1 configuration, and you can run a miner that will see and start using all 7 video cards. So the MSI Z170A Gaming Pro Carbon motherboard apparently works, though there are still some glitches that may need fixing. The only issue with this motherboard is the higher price and that may be enough of a reason to make you stick with the more affordable and problem free use of AsRock H81 Pro BTC for 6x GPUs instead. Still if you already have a 6x or even less GPUs on some mining rigs and you have a suitable power supply an upgrade to a 7x GPU motherboard might be an interesting choice. Then again you would need to replace not only the motherboard, but also the CPU and the RAM as well, because of the newer chipset used by the MSI motherboard.