Running ubuntu/linux on a xen server 6.5 with a passthrough GPU

For a while I was struggling to get ubuntu to work with my NVIDIA GPU in passthrough mode on xenserver. Xorg start, but then just fail... and those xorg messages are sooo helpful ;). Any way it turns out xorg could not correctly map the right driver to the correct PCI device. I had to add the PCI bus ID to the xorg.conf like here:

 BusID "PCI:0:12:0"

 Found this info here: https://wiki.archlinux.org/index.php/Xorg#More_than_one_graphics_card:

 my devices "lspci | grep -e VGA -e 3D":

00:02.0 VGA compatible controller: Cirrus Logic GD 5446
00:06.0 VGA compatible controller: NVIDIA Corporation GK104GL [Quadro K5000] (rev a1)