17:08 Wally: Do all nv50+ gpus use the pci (abstraction) or are there any that do not?
17:15 mwk: Wally: the Tegra integrated GPUs aren't PCI
17:16 mwk: other than that and RSX, all nvidia GPUs ever have been PCI
17:16 Wally: They dont use the abstraction
17:16 mwk: yeah
17:17 Wally: Then what abstraction do they use instead(tegra)
17:17 mwk: none really, as is usual for ARM SoC devices
17:18 Wally: So for accessing for example registers they just mess with (cpu )ram address
17:19 Wally: addresses*
17:19 karolherbst: well it's all MMIO in the end, even for PCI
17:19 mwk: the register MMIO window is simply at a constant physical address, hardwired in the SoC hardware
17:20 mwk: and in normal circumstances, the OS gets the description of platform hardware in Device Tree, including the MMIO address to poke at
17:21 Wally: I doubt you can get the device tree(as easily) from userspace
17:22 karolherbst: what does userspace has anything to do with that?
17:22 karolherbst: ohh don't tell me that's the "userspace driver" pitfall...
17:22 mwk: hm?
17:23 karolherbst: mwk: sometimes people arrive here asking questions because they want to implement a GPU driver in userspace
17:23 karolherbst: hoping it's not that :O
17:23 mwk: sounds fun
17:23 karolherbst: sure, but also very insecure and broken
17:24 mwk: insecure?
17:24 karolherbst: well.. do you want userspace to configure DMA?
17:24 mwk: IOMMUs are a thing
17:24 karolherbst: and if you don't have an IOMMU, the driver doesn't work or what?
17:25 mwk: in that case, oh well
17:25 karolherbst: guess so
17:26 karolherbst: but then.. if userspace can configure the iommu as well...
17:26 karolherbst: because it has to, otherwise you can't map random memory
17:26 karolherbst: though I guess that should be on the kernel side and you can put security thingies in place that puts some restriction on that
17:27 mwk: well
17:27 mwk: have you ever looked at pci passthrough in qemu via VFIO?
17:27 karolherbst: I thought that more or less requires an IOMMU
17:27 mwk: yes
17:28 karolherbst: but you still have to configure what memory it is allowed to access via the iommu and stuff
17:28 mwk: mhm
17:28 mwk: hence, VFIO
17:28 karolherbst: like for GPUs you still want to access application memory (sometimes)
17:28 karolherbst: so there has to be some dynamic bits in the userspace GPU driver scenario
17:29 mwk: yes, and?
17:29 karolherbst: it's just weird if you trust userspace enough to not do silly things, like allowing access to random kernel memory
17:30 karolherbst: though I guess with an IOMMU only configured on the kernel side that might work
17:30 mwk: but it doesn't have to
17:30 mwk: yes
17:30 mwk: this is literally what VFIO does
17:30 karolherbst: yeah
17:30 karolherbst: not saying it's not possible, just that requiring an IOMMU doesn't work if you want a driver being used by all users
17:30 karolherbst: anyway
17:31 karolherbst: not really seeing what problem it tries to solve
17:32 karolherbst: especially as some people think about giving certain/some modules each own VM
17:32 Wally: karolherbst: nvidia's kernel driver isnt full featured enough for nvk so you have to implement some stuff from userspace
17:33 karolherbst: huh?
17:33 karolherbst: I would be surprised if something is missing
17:33 Wally: The nvidia kernel driver doesnt expose pushbuffers through ioctls for example
17:34 karolherbst: they don't?
17:34 Wally: yep
17:34 karolherbst: how do they do command submission then?
17:34 Wally: map the pci bars
17:34 karolherbst: I know there is a way to do that from userspace, but I thought they do have a kernel fallback
17:34 karolherbst: Wally: oh boi
17:35 Wally: looking through the xf86-video-nv stuff it actually seems quite elegant
17:45 Wally: karolherbst: btw, the problem userspace drivers are trying to solve is the portability between operating systems, but I doubt there is any reason to do anything more than modesetting and cursor support in a usermode driver since most people who need more features will not use an obscure os like managarm
17:47 karolherbst: sure.. we did modesetting in userspace once and it was terrible, so they'll move it into the kernel later anyway :)
17:49 Wally: what if there is no kernel?
17:50 karolherbst: then nothing matters anyway
17:51 Wally: ?
17:53 karolherbst: well.. if you poke hw directly this is what you do, but only by disregarding the concept of a kernel it becomes random code doing stuff :P
17:56 Wally: I mean all of our devices with software just run random code doing stuff
17:57 Wally: not saying that its an awful idea though :P
17:57 karolherbst: sure, but either you call it a kernel and do it properly or you don't and it's a pile of code :P
18:58 airlied: Wally: they dont map the pci bars
18:58 airlied: they map userd allocations
18:59 airlied: and poke values in there to submit pushbufs
18:59 karolherbst: airlied: how does that work across contexts though?
18:59 karolherbst: or well.. threads/processes
18:59 airlied: you get a userd map per context
19:00 karolherbst: ahh
19:17 Wally: airlied: oh, im probably still going to be mapping the bars since there doesnt seem to be any downside afaik
19:20 Wally: (well other than not being able to use tegra)
19:27 airlied:can't remember if userd maps are in bar2 or can be in sysram