00:36 imirkin:really should fix the SELECT for QUERY_GET....
01:16 mangix: oh boy, found a nice typo in nouveau...
01:31 mwk: eh
01:31 mwk: GPUs are complicated
01:36 imirkin: mangix: already part of https://github.com/skeggsb/nouveau/commit/e9a914757d6a3820d478443678243c63f012512f
01:46 mangix: weird, i just cloned master
01:47 mangix: ah different branch
10:08 mlankhorst: hm how well should tegra sata work?
10:08 mlankhorst: after some time I get random ata errors
10:09 mlankhorst: ata1.00: exception Emask 0x0 SAct 0xf000000 SErr 0x0 action 0x6 frozen
12:48 imirkin_: mlankhorst: try #tegra?
15:43 pgera: Hi. I'm trying to pass through my optimus gpu to a vm, and I've had some partial success. I think the passthrough actually works, but there are issues inside the vm.
15:44 pgera: the first hurdle was that in optimus laptops, the gpu's bios rom isn't easily readable. So things wouldn't work. I then managed to pass the rom as a file, and nouveau inside the guest makes some more progress.
15:45 pgera: it even displayed something like a garbled splash screen on the monitor connected to the gpu
15:45 pgera: but not more than that
15:47 pgera: so i think it should be solvable, at least for linux host and linux guest.
15:48 dugz: pgera, make sure you blacklist the hdmi audio module so it doesn't bind to the optimus
15:49 pgera: there is no hdmi audio in my lspci
15:49 dugz: k
15:49 pgera: is it expected ?
15:49 pgera: i was kind of surprised by that too
15:49 pgera: audio over hdmi works though
15:49 dugz: gpu's with HDMI usually have an audio dsp
15:50 pgera: so i think it is just passing the audio through
15:50 pgera: it doesn't do any decoding
15:50 pgera: this is a thinkpad w530 with quadro k2000m
15:50 dugz: when you pass the GPU, you also have to pass it's slave HDMI if there is one
15:51 pgera: the second thing i tried was to install nvidia drivers inside the vm
15:51 pgera: that fails with the error that it couldn't load the rom into memory, although the rom is passed as a file
15:51 pgera: that's probably because it's an optimus laptop. so the nvidia driver expects the acpi method to get the rom
15:52 pgera: and not the standard way to grab the rom from the card itself
15:52 pgera: is there some test that i can do with nouveau in the guest besides display ?
15:53 pgera: to gauge how far things work
15:55 pgera: there is a separate issue with nouveau on the host, which causes crashes if you try to pass through the gpu, but that's less of a concern
15:55 pgera: since nouveau can be disabled entirely on the host
15:58 pgera: These are the two devices in the IOMMU group
15:58 pgera: IOMMU Group 1 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port [8086:0151] (rev 09)
15:58 pgera: IOMMU Group 1 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK107GLM [Quadro K2000M] [10de:0ffb] (rev a1)
15:58 pgera: out of which I'm only passing the gpu
16:16 dugz: sounds right
16:18 pgera: Any tests for checking how far things are working inside the vm ?
16:23 dugz: I just tested my gt720 (nv106) and vdpau is still broken with current git VLC. https://paste.pound-python.org/show/pJHWB73D64Hm0S4PvOqx/
16:24 dugz: pgera, make sure you have a group in /dev/vfio for the iommu
16:25 pgera: yes, that's there. the device has been passed through as fas as i can tell
16:25 pgera: i can see it in lspci inside the guest
16:25 pgera: and from the guest's dmesg
16:25 pgera: i was looking for something like the nvidia-smi utility for nouveau that i can run inside the guest
16:26 pgera: to see how well it can use it inside the guest
16:27 dugz: there is some way to fake out nvidia drivers so they can't detect the VM, like "fakebios"
16:27 pgera: yes, i was thinking of using coreboot
16:27 pgera: to create some sort of a bios
16:27 pgera: with the rom built in
16:28 imirkin_: dugz: right it's the same thing - still trying to create a RGB10A2 surface, and still failing
16:28 pgera: because the nvidia driver expects to get the rom from the bios
16:28 dugz: imirkin_, it really wants that surface ;)
16:28 pgera: this is not about not detecting the vm
16:28 pgera: the issue is that this is not a generic card
16:29 imirkin_: pgera: right, so like you found out, the vbios is part of the ACPI blob which isn't being passed through
16:29 pgera: yes, imirkin_
16:29 imirkin_: pgera: use nouveau on the host to get the vbios.rom file from debugfs, and then pass that in via nouveau.config=NvBios=path-to-file
16:29 pgera: but that's for nvidia
16:29 pgera: it should work for nouveau inside the guest
16:29 pgera: right ?
16:29 pgera: without the acpi blob
16:29 imirkin_: it needs the vbios.
16:29 pgera: i am passing the vbios
16:30 pgera: as a qemu optoin
16:30 pgera: *option
16:30 pgera: or do you mean that nouveau also needs the acpi blob ?
16:30 pgera: i'm passing at as romfile=foo.rom
16:30 pgera: which would emulate the pci rom location
16:30 pgera: it won't emulate the acpi blob though
16:31 pgera: does nouveau need the acpi blob ?
16:31 pgera: actually, there is a separate issue with the blob from debugfs
16:31 pgera: it truncates it
16:31 pgera: at 68k
16:31 pgera: luckily, i managed to get the full vbios
16:32 pgera: the debugfs's vbios dump would fail the signature too
16:32 pgera: which the nvidia driver will also check
16:32 imirkin_: pgera: i'd highly recommend doing the thing i said to do
16:32 imirkin_: but perhaps i'm biased :)
16:32 imirkin_: the thing in acpi is very much *not* what can go into an option rom
16:33 pgera: ok. Let me try. where does nouveau.config go ?
16:33 pgera: is that a qemu flag ?
16:34 pgera: oh, you mean in the guest ?
16:35 imirkin_: in the guest, put it in the initrd or whereever will be accessible when the nouveau module loads
16:35 imirkin_: (so if it's built in, it'd have to be as CONFIG_EXTRA_FIRMWARE etc)
16:35 imirkin_: and then pass that as a boot arg
16:37 imirkin_: but first get the vbios rom *properly*, for which i recommend loading nouveau on the host and grabbing the vbios.rom file from /sys/kernel/debug/dri/*/
16:38 pgera: yes, i got the vbios rom
16:38 pgera: from debugfs
16:39 pgera: that is not the full rom
16:39 pgera: it fails the rom-parser test
16:39 imirkin_: yeah
16:39 pgera: then i had to get the entire rom
16:39 imirkin_: that's expected
16:39 pgera: which passes
16:39 pgera: that test
16:39 imirkin_: but it contains the wrong stuff
16:39 pgera: wrong stuff as in ?
16:39 pgera: the debugfs's rom is just the first half of hte entire rom
16:39 imirkin_: i dunno - i guess it could be right
16:40 imirkin_: i think only the acpi one gets used and thus tested
16:40 pgera: the whole file is about 128 K
16:40 pgera: Valid ROM signature found @0H, PCIR offset 190h PCIR: type 0 (x86 PC-AT), vendor: 10de, device: 0ffb, class: 030000 PCIR: revision 3, vendor revision: 1 Valid ROM signature found @f800h, PCIR offset 40h PCIR: type e0, vendor: 10de, device: 0ff0, class: 000000 PCIR: revision 0, vendor revision: 0 Valid ROM signature found @16200h, PCIR offset 1ch PCIR: type 3 (EFI), vendor: 10de, device: 0ffb, class: 030000 PCIR: revision 3, vendor rev
16:40 pgera: this is what rom-parser prints
16:41 imirkin_: right ... this says nothing about its contents though
16:41 pgera: true
16:41 imirkin_: the contents are used for things
16:41 imirkin_: the contents in acpi are correct
16:41 imirkin_: by definition, since those are the ones that get used
16:41 imirkin_: this ... who knows.
16:41 pgera: it matches the first half of nouveau debugfs dump
16:41 imirkin_: ah
16:41 imirkin_: well then it could be fine. but the way that the driver reads the vbios is also non-trivial
16:41 pgera: i'm fairly certain that the rom is fine
16:41 imirkin_: there are like 10 diff methods, and the PCI ROM is the last of the last fallbacks
16:42 pgera: yes, that's what is happening now
16:42 pgera: it's getting it from the pci
16:42 imirkin_: i just *very much* recommend doing it the way i recommended.
16:42 pgera: noob question. in an ubuntu guest, where does tha tgo ?
16:42 pgera: the config i mean
16:42 pgera: the one you mentioned
16:42 imirkin_: i don't do distro support
16:43 pgera: i mean i'll have to rebuild inintramfs for it, right ?
16:44 imirkin_: <imirkin_> i don't do distro support
16:44 imirkin_: at the time that nouveau loads, request_firmware() has to be able to load the file.
16:44 imirkin_: whatever you need to do to make that happen.
16:44 pgera: ok
18:25 pgera: imirkin, it doesn't seem to load the firmware from the file. errors out with code -2. I confirmed that the rom file is a part of initramfs
18:32 imirkin_: where is the file located in initramfs and what arg are you passing in?
18:35 pgera: passing nouveau.config=NvBios=/full.rom
18:35 pgera: as a grub argument
18:35 pgera: file is at the root
18:35 imirkin_: needs to be in /lib/firmware
18:36 imirkin_: and get rid of the / in full.rom (although i don't think it matters)
18:36 imirkin_: also, what is full.rom? is it the vbios.rom that nouveau generates in debugfs?
18:36 imirkin_: [if not, that's what it needs to be)
18:36 pgera: it's the vbios.rom
18:36 pgera: should it be nouveau's debugfs ?
18:36 imirkin_: it needs to have the same contents that nouveau's debugfs outputs
18:36 pgera: it's larger than nouveau's
18:37 pgera: but the first half is the same. so it shouldn't matter
18:37 pgera: ok
18:37 pgera: i'll change it
18:37 imirkin_: i'd rather not mess around with that sort of thing, but you can do whatever
18:38 pgera: grub argument is absolute or relative to /lib/firmware ?
18:38 pgera: just NvBios=full.rom, right
18:38 pgera: after it's in /lib/firmware
18:38 imirkin_: yep
18:39 imirkin_: the boot line arg is just passed as a string to request_firmware()
18:39 imirkin_: that has various logic for finding the file, but it largely boils down to "/lib/firmware".
18:39 jvesely: hi nouveau, is there a GPU instruction that would trigger interrupt on CPU (similar to AMD's s_sendmsg) in any nvidia ISA?
18:40 imirkin_: jvesely: dunno what s_sendmsg does, but nvidia's isa includes breakpoint capabilities
18:40 imirkin_: which works by sending an interrupt iirc
18:41 jvesely: is there a documentation (in envytools?) on that?
18:42 imirkin_: the instructions are in envydis
18:43 imirkin_: the whole breakpointing flow is ... complicated.
18:50 pgera: ok, that looked like it that worked. No real difference in the guest though.
18:50 imirkin_: ok
18:51 imirkin_: pastebin dmesg of the guest?
18:51 pgera: one sec
18:53 pgera: https://pastebin.com/pwS08CAa
18:58 imirkin_: ok, and what's the issue?
18:58 pgera: external monitor isn't detected
18:58 pgera: when i plug in
18:59 imirkin_: grep . /sys/kernel/drm/card*-*/status
18:59 imirkin_: er
18:59 imirkin_: grep . /sys/class/drm/card*-*/status
19:03 pgera: card-1/DP3 shows connected. rest are disconnected. virtual is connected
19:03 pgera: so it is connected
19:04 pgera: i see the edid
19:04 pgera: how do i get it on the display ?
19:04 imirkin_: what were you hoping would display there?
19:04 imirkin_: your primary gpu is the virtual one
19:04 imirkin_: so in that VM it's a secondary GPU, same as on your host
19:05 pgera: ok. let me disable the virtual one and try again. i tried that too earlier
19:05 pgera: but could only get a garbled splash
19:05 imirkin_: well, you'll get garbled stuff until nouveau loads
19:05 pgera: shouldn't it be possible to switch the display
19:05 pgera: from virtual to monitor ?
19:06 imirkin_: sure.
19:06 pgera: without disabling the virtual gpu
19:06 imirkin_: sure
19:06 imirkin_: of course i'm not sure what you mean by "switch the display"
19:06 imirkin_: you have 2 displays
19:06 imirkin_: one virtual, one attached to the nvidia gpu
19:06 imirkin_: each device has its own card node, etc
19:07 pgera: ok. let me disable the virtual. that should make it simpler
19:07 imirkin_: i realize that you probably don'
19:08 imirkin_: t have an expert understanding of all the bits of software that are involved here
19:08 imirkin_: but i can't do support for that whole stack - you have problem operating the distro, ask the distro guys :)
19:08 pgera: right. so with the virtual disabled, nothing shows up on the monitor
19:09 pgera: except for the brief blink of the light
19:09 imirkin_: initially ... but then nouveau should load and then you should see the console there
19:09 pgera: yeah, that doesn't happen for some reason. I'll probably have to enable some more debugging via serial port
19:09 pgera: right now there is nothing on the monitor
19:11 imirkin_: grab dmesg?
19:12 imirkin_: perhaps that gpu is really not ready to handle the VGA stuff that the BIOS probably throws at it
19:12 pgera: yeah. let me enable ssh server or serial port to grab dmesg
19:31 pgera: i've got ssh into the vm. don't see any errors on dmesg
19:31 pgera: let me put it on pastebin
19:32 imirkin_: and check if DP-3 or whatever is connected
19:33 pgera: yeah, that is still connected
19:33 pgera: https://pastebin.com/p8aERWKq
19:34 pgera: is there any way to send a test signal to the monitor
19:34 pgera: anything that involves some communication with the monitor
19:34 imirkin_: [ 2.450743] nouveau 0000:00:07.0: fb1: nouveaufb frame buffer device
19:34 imirkin_: [ 1.604524] fbcon: qxldrmfb (fb0) is primary device
19:34 imirkin_: so ... still have the virtual screen.
19:34 imirkin_: sure, you can .... just run an X server on it
19:34 imirkin_: set the Device properly
19:34 pgera: hmm. that's strange. I don't have the virtual gpu or the virtual display
19:35 imirkin_: well - it's in there.
19:35 imirkin_: might be a way to flip fbcon to run on fb1 instead
19:35 imirkin_: but i don't know how offhand
19:35 pgera: yes, just saw. a separate qxl device was there
19:35 pgera: now it's on!
19:35 pgera: yes
19:35 pgera: i see it on the monitor
19:35 imirkin_: ok cool
19:36 pgera: this is awesome!
19:36 pgera: that you so much
19:36 pgera: i started this on a whim yesterday
19:36 pgera: because passthrough hasn't worked for any laptops generally
19:37 imirkin_: enjoy
19:37 imirkin_: i suspect it'll work with your option rom thing too
19:37 pgera: yes, for nouveu it will work
19:37 imirkin_: at least worth a shot. i just wanted to exclude it to ensure it wasn't the reason things were failing.
19:37 pgera: nvidia on linux, or a windows vm will need more work
19:37 imirkin_: maybe yea
19:38 pgera: the acpi stuff
19:38 pgera: can it be emulated ?
20:31 imirkin_: sure - you can feed whatever acpi you want
20:31 karolherbst: does anybody have any experience with getting dev keys from paradox interactive?
20:47 karolherbst: imirkin_: do you think this part belongs inside the legalizeSSA handling rather inside an optimization? https://github.com/karolherbst/mesa/commit/80be2ebab9bd046c2251f96c524ce446d24e0388#diff-46df282115646e17aa1321265c4e170aR1564
21:27 imirkin: karolherbst: yes, that needs to be in ConstantFolding for the POW handling
23:50 mwk: It's a rare occasion to write a piece of documentation that applies to Celsius PGRAPH as well as Pascal PGRAPH...