00:36imirkin:really should fix the SELECT for QUERY_GET....
01:16mangix: oh boy, found a nice typo in nouveau...
01:31mwk: eh
01:31mwk: GPUs are complicated
01:36imirkin: mangix: already part of https://github.com/skeggsb/nouveau/commit/e9a914757d6a3820d478443678243c63f012512f
01:46mangix: weird, i just cloned master
01:47mangix: ah different branch
10:08mlankhorst: hm how well should tegra sata work?
10:08mlankhorst: after some time I get random ata errors
10:09mlankhorst: ata1.00: exception Emask 0x0 SAct 0xf000000 SErr 0x0 action 0x6 frozen
12:48imirkin_: mlankhorst: try #tegra?
15:43pgera: Hi. I'm trying to pass through my optimus gpu to a vm, and I've had some partial success. I think the passthrough actually works, but there are issues inside the vm.
15:44pgera: the first hurdle was that in optimus laptops, the gpu's bios rom isn't easily readable. So things wouldn't work. I then managed to pass the rom as a file, and nouveau inside the guest makes some more progress.
15:45pgera: it even displayed something like a garbled splash screen on the monitor connected to the gpu
15:45pgera: but not more than that
15:47pgera: so i think it should be solvable, at least for linux host and linux guest.
15:48dugz: pgera, make sure you blacklist the hdmi audio module so it doesn't bind to the optimus
15:49pgera: there is no hdmi audio in my lspci
15:49dugz: k
15:49pgera: is it expected ?
15:49pgera: i was kind of surprised by that too
15:49pgera: audio over hdmi works though
15:49dugz: gpu's with HDMI usually have an audio dsp
15:50pgera: so i think it is just passing the audio through
15:50pgera: it doesn't do any decoding
15:50pgera: this is a thinkpad w530 with quadro k2000m
15:50dugz: when you pass the GPU, you also have to pass it's slave HDMI if there is one
15:51pgera: the second thing i tried was to install nvidia drivers inside the vm
15:51pgera: that fails with the error that it couldn't load the rom into memory, although the rom is passed as a file
15:51pgera: that's probably because it's an optimus laptop. so the nvidia driver expects the acpi method to get the rom
15:52pgera: and not the standard way to grab the rom from the card itself
15:52pgera: is there some test that i can do with nouveau in the guest besides display ?
15:53pgera: to gauge how far things work
15:55pgera: there is a separate issue with nouveau on the host, which causes crashes if you try to pass through the gpu, but that's less of a concern
15:55pgera: since nouveau can be disabled entirely on the host
15:58pgera: These are the two devices in the IOMMU group
15:58pgera: IOMMU Group 1 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port [8086:0151] (rev 09)
15:58pgera: IOMMU Group 1 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK107GLM [Quadro K2000M] [10de:0ffb] (rev a1)
15:58pgera: out of which I'm only passing the gpu
16:16dugz: sounds right
16:18pgera: Any tests for checking how far things are working inside the vm ?
16:23dugz: I just tested my gt720 (nv106) and vdpau is still broken with current git VLC. https://paste.pound-python.org/show/pJHWB73D64Hm0S4PvOqx/
16:24dugz: pgera, make sure you have a group in /dev/vfio for the iommu
16:25pgera: yes, that's there. the device has been passed through as fas as i can tell
16:25pgera: i can see it in lspci inside the guest
16:25pgera: and from the guest's dmesg
16:25pgera: i was looking for something like the nvidia-smi utility for nouveau that i can run inside the guest
16:26pgera: to see how well it can use it inside the guest
16:27dugz: there is some way to fake out nvidia drivers so they can't detect the VM, like "fakebios"
16:27pgera: yes, i was thinking of using coreboot
16:27pgera: to create some sort of a bios
16:27pgera: with the rom built in
16:28imirkin_: dugz: right it's the same thing - still trying to create a RGB10A2 surface, and still failing
16:28pgera: because the nvidia driver expects to get the rom from the bios
16:28dugz: imirkin_, it really wants that surface ;)
16:28pgera: this is not about not detecting the vm
16:28pgera: the issue is that this is not a generic card
16:29imirkin_: pgera: right, so like you found out, the vbios is part of the ACPI blob which isn't being passed through
16:29pgera: yes, imirkin_
16:29imirkin_: pgera: use nouveau on the host to get the vbios.rom file from debugfs, and then pass that in via nouveau.config=NvBios=path-to-file
16:29pgera: but that's for nvidia
16:29pgera: it should work for nouveau inside the guest
16:29pgera: right ?
16:29pgera: without the acpi blob
16:29imirkin_: it needs the vbios.
16:29pgera: i am passing the vbios
16:30pgera: as a qemu optoin
16:30pgera: *option
16:30pgera: or do you mean that nouveau also needs the acpi blob ?
16:30pgera: i'm passing at as romfile=foo.rom
16:30pgera: which would emulate the pci rom location
16:30pgera: it won't emulate the acpi blob though
16:31pgera: does nouveau need the acpi blob ?
16:31pgera: actually, there is a separate issue with the blob from debugfs
16:31pgera: it truncates it
16:31pgera: at 68k
16:31pgera: luckily, i managed to get the full vbios
16:32pgera: the debugfs's vbios dump would fail the signature too
16:32pgera: which the nvidia driver will also check
16:32imirkin_: pgera: i'd highly recommend doing the thing i said to do
16:32imirkin_: but perhaps i'm biased :)
16:32imirkin_: the thing in acpi is very much *not* what can go into an option rom
16:33pgera: ok. Let me try. where does nouveau.config go ?
16:33pgera: is that a qemu flag ?
16:34pgera: oh, you mean in the guest ?
16:35imirkin_: in the guest, put it in the initrd or whereever will be accessible when the nouveau module loads
16:35imirkin_: (so if it's built in, it'd have to be as CONFIG_EXTRA_FIRMWARE etc)
16:35imirkin_: and then pass that as a boot arg
16:37imirkin_: but first get the vbios rom *properly*, for which i recommend loading nouveau on the host and grabbing the vbios.rom file from /sys/kernel/debug/dri/*/
16:38pgera: yes, i got the vbios rom
16:38pgera: from debugfs
16:39pgera: that is not the full rom
16:39pgera: it fails the rom-parser test
16:39imirkin_: yeah
16:39pgera: then i had to get the entire rom
16:39imirkin_: that's expected
16:39pgera: which passes
16:39pgera: that test
16:39imirkin_: but it contains the wrong stuff
16:39pgera: wrong stuff as in ?
16:39pgera: the debugfs's rom is just the first half of hte entire rom
16:39imirkin_: i dunno - i guess it could be right
16:40imirkin_: i think only the acpi one gets used and thus tested
16:40pgera: the whole file is about 128 K
16:40pgera: Valid ROM signature found @0H, PCIR offset 190h PCIR: type 0 (x86 PC-AT), vendor: 10de, device: 0ffb, class: 030000 PCIR: revision 3, vendor revision: 1 Valid ROM signature found @f800h, PCIR offset 40h PCIR: type e0, vendor: 10de, device: 0ff0, class: 000000 PCIR: revision 0, vendor revision: 0 Valid ROM signature found @16200h, PCIR offset 1ch PCIR: type 3 (EFI), vendor: 10de, device: 0ffb, class: 030000 PCIR: revision 3, vendor rev
16:40pgera: this is what rom-parser prints
16:41imirkin_: right ... this says nothing about its contents though
16:41pgera: true
16:41imirkin_: the contents are used for things
16:41imirkin_: the contents in acpi are correct
16:41imirkin_: by definition, since those are the ones that get used
16:41imirkin_: this ... who knows.
16:41pgera: it matches the first half of nouveau debugfs dump
16:41imirkin_: ah
16:41imirkin_: well then it could be fine. but the way that the driver reads the vbios is also non-trivial
16:41pgera: i'm fairly certain that the rom is fine
16:41imirkin_: there are like 10 diff methods, and the PCI ROM is the last of the last fallbacks
16:42pgera: yes, that's what is happening now
16:42pgera: it's getting it from the pci
16:42imirkin_: i just *very much* recommend doing it the way i recommended.
16:42pgera: noob question. in an ubuntu guest, where does tha tgo ?
16:42pgera: the config i mean
16:42pgera: the one you mentioned
16:42imirkin_: i don't do distro support
16:43pgera: i mean i'll have to rebuild inintramfs for it, right ?
16:44imirkin_: <imirkin_> i don't do distro support
16:44imirkin_: at the time that nouveau loads, request_firmware() has to be able to load the file.
16:44imirkin_: whatever you need to do to make that happen.
16:44pgera: ok
18:25pgera: imirkin, it doesn't seem to load the firmware from the file. errors out with code -2. I confirmed that the rom file is a part of initramfs
18:32imirkin_: where is the file located in initramfs and what arg are you passing in?
18:35pgera: passing nouveau.config=NvBios=/full.rom
18:35pgera: as a grub argument
18:35pgera: file is at the root
18:35imirkin_: needs to be in /lib/firmware
18:36imirkin_: and get rid of the / in full.rom (although i don't think it matters)
18:36imirkin_: also, what is full.rom? is it the vbios.rom that nouveau generates in debugfs?
18:36imirkin_: [if not, that's what it needs to be)
18:36pgera: it's the vbios.rom
18:36pgera: should it be nouveau's debugfs ?
18:36imirkin_: it needs to have the same contents that nouveau's debugfs outputs
18:36pgera: it's larger than nouveau's
18:37pgera: but the first half is the same. so it shouldn't matter
18:37pgera: ok
18:37pgera: i'll change it
18:37imirkin_: i'd rather not mess around with that sort of thing, but you can do whatever
18:38pgera: grub argument is absolute or relative to /lib/firmware ?
18:38pgera: just NvBios=full.rom, right
18:38pgera: after it's in /lib/firmware
18:38imirkin_: yep
18:39imirkin_: the boot line arg is just passed as a string to request_firmware()
18:39imirkin_: that has various logic for finding the file, but it largely boils down to "/lib/firmware".
18:39jvesely: hi nouveau, is there a GPU instruction that would trigger interrupt on CPU (similar to AMD's s_sendmsg) in any nvidia ISA?
18:40imirkin_: jvesely: dunno what s_sendmsg does, but nvidia's isa includes breakpoint capabilities
18:40imirkin_: which works by sending an interrupt iirc
18:41jvesely: is there a documentation (in envytools?) on that?
18:42imirkin_: the instructions are in envydis
18:43imirkin_: the whole breakpointing flow is ... complicated.
18:50pgera: ok, that looked like it that worked. No real difference in the guest though.
18:50imirkin_: ok
18:51imirkin_: pastebin dmesg of the guest?
18:51pgera: one sec
18:53pgera: https://pastebin.com/pwS08CAa
18:58imirkin_: ok, and what's the issue?
18:58pgera: external monitor isn't detected
18:58pgera: when i plug in
18:59imirkin_: grep . /sys/kernel/drm/card*-*/status
18:59imirkin_: er
18:59imirkin_: grep . /sys/class/drm/card*-*/status
19:03pgera: card-1/DP3 shows connected. rest are disconnected. virtual is connected
19:03pgera: so it is connected
19:04pgera: i see the edid
19:04pgera: how do i get it on the display ?
19:04imirkin_: what were you hoping would display there?
19:04imirkin_: your primary gpu is the virtual one
19:04imirkin_: so in that VM it's a secondary GPU, same as on your host
19:05pgera: ok. let me disable the virtual one and try again. i tried that too earlier
19:05pgera: but could only get a garbled splash
19:05imirkin_: well, you'll get garbled stuff until nouveau loads
19:05pgera: shouldn't it be possible to switch the display
19:05pgera: from virtual to monitor ?
19:06imirkin_: sure.
19:06pgera: without disabling the virtual gpu
19:06imirkin_: sure
19:06imirkin_: of course i'm not sure what you mean by "switch the display"
19:06imirkin_: you have 2 displays
19:06imirkin_: one virtual, one attached to the nvidia gpu
19:06imirkin_: each device has its own card node, etc
19:07pgera: ok. let me disable the virtual. that should make it simpler
19:07imirkin_: i realize that you probably don'
19:08imirkin_: t have an expert understanding of all the bits of software that are involved here
19:08imirkin_: but i can't do support for that whole stack - you have problem operating the distro, ask the distro guys :)
19:08pgera: right. so with the virtual disabled, nothing shows up on the monitor
19:09pgera: except for the brief blink of the light
19:09imirkin_: initially ... but then nouveau should load and then you should see the console there
19:09pgera: yeah, that doesn't happen for some reason. I'll probably have to enable some more debugging via serial port
19:09pgera: right now there is nothing on the monitor
19:11imirkin_: grab dmesg?
19:12imirkin_: perhaps that gpu is really not ready to handle the VGA stuff that the BIOS probably throws at it
19:12pgera: yeah. let me enable ssh server or serial port to grab dmesg
19:31pgera: i've got ssh into the vm. don't see any errors on dmesg
19:31pgera: let me put it on pastebin
19:32imirkin_: and check if DP-3 or whatever is connected
19:33pgera: yeah, that is still connected
19:33pgera: https://pastebin.com/p8aERWKq
19:34pgera: is there any way to send a test signal to the monitor
19:34pgera: anything that involves some communication with the monitor
19:34imirkin_: [ 2.450743] nouveau 0000:00:07.0: fb1: nouveaufb frame buffer device
19:34imirkin_: [ 1.604524] fbcon: qxldrmfb (fb0) is primary device
19:34imirkin_: so ... still have the virtual screen.
19:34imirkin_: sure, you can .... just run an X server on it
19:34imirkin_: set the Device properly
19:34pgera: hmm. that's strange. I don't have the virtual gpu or the virtual display
19:35imirkin_: well - it's in there.
19:35imirkin_: might be a way to flip fbcon to run on fb1 instead
19:35imirkin_: but i don't know how offhand
19:35pgera: yes, just saw. a separate qxl device was there
19:35pgera: now it's on!
19:35pgera: yes
19:35pgera: i see it on the monitor
19:35imirkin_: ok cool
19:36pgera: this is awesome!
19:36pgera: that you so much
19:36pgera: i started this on a whim yesterday
19:36pgera: because passthrough hasn't worked for any laptops generally
19:37imirkin_: enjoy
19:37imirkin_: i suspect it'll work with your option rom thing too
19:37pgera: yes, for nouveu it will work
19:37imirkin_: at least worth a shot. i just wanted to exclude it to ensure it wasn't the reason things were failing.
19:37pgera: nvidia on linux, or a windows vm will need more work
19:37imirkin_: maybe yea
19:38pgera: the acpi stuff
19:38pgera: can it be emulated ?
20:31imirkin_: sure - you can feed whatever acpi you want
20:31karolherbst: does anybody have any experience with getting dev keys from paradox interactive?
20:47karolherbst: imirkin_: do you think this part belongs inside the legalizeSSA handling rather inside an optimization? https://github.com/karolherbst/mesa/commit/80be2ebab9bd046c2251f96c524ce446d24e0388#diff-46df282115646e17aa1321265c4e170aR1564
21:27imirkin: karolherbst: yes, that needs to be in ConstantFolding for the POW handling
23:50mwk: It's a rare occasion to write a piece of documentation that applies to Celsius PGRAPH as well as Pascal PGRAPH...