17:08Wally: Do all nv50+ gpus use the pci (abstraction) or are there any that do not?
17:15mwk: Wally: the Tegra integrated GPUs aren't PCI
17:16mwk: other than that and RSX, all nvidia GPUs ever have been PCI
17:16Wally: They dont use the abstraction
17:17Wally: Then what abstraction do they use instead(tegra)
17:17mwk: none really, as is usual for ARM SoC devices
17:18Wally: So for accessing for example registers they just mess with (cpu )ram address
17:19karolherbst: well it's all MMIO in the end, even for PCI
17:19mwk: the register MMIO window is simply at a constant physical address, hardwired in the SoC hardware
17:20mwk: and in normal circumstances, the OS gets the description of platform hardware in Device Tree, including the MMIO address to poke at
17:21Wally: I doubt you can get the device tree(as easily) from userspace
17:22karolherbst: what does userspace has anything to do with that?
17:22karolherbst: ohh don't tell me that's the "userspace driver" pitfall...
17:23karolherbst: mwk: sometimes people arrive here asking questions because they want to implement a GPU driver in userspace
17:23karolherbst: hoping it's not that :O
17:23mwk: sounds fun
17:23karolherbst: sure, but also very insecure and broken
17:24karolherbst: well.. do you want userspace to configure DMA?
17:24mwk: IOMMUs are a thing
17:24karolherbst: and if you don't have an IOMMU, the driver doesn't work or what?
17:25mwk: in that case, oh well
17:25karolherbst: guess so
17:26karolherbst: but then.. if userspace can configure the iommu as well...
17:26karolherbst: because it has to, otherwise you can't map random memory
17:26karolherbst: though I guess that should be on the kernel side and you can put security thingies in place that puts some restriction on that
17:27mwk: have you ever looked at pci passthrough in qemu via VFIO?
17:27karolherbst: I thought that more or less requires an IOMMU
17:28karolherbst: but you still have to configure what memory it is allowed to access via the iommu and stuff
17:28mwk: hence, VFIO
17:28karolherbst: like for GPUs you still want to access application memory (sometimes)
17:28karolherbst: so there has to be some dynamic bits in the userspace GPU driver scenario
17:29mwk: yes, and?
17:29karolherbst: it's just weird if you trust userspace enough to not do silly things, like allowing access to random kernel memory
17:30karolherbst: though I guess with an IOMMU only configured on the kernel side that might work
17:30mwk: but it doesn't have to
17:30mwk: this is literally what VFIO does
17:30karolherbst: not saying it's not possible, just that requiring an IOMMU doesn't work if you want a driver being used by all users
17:31karolherbst: not really seeing what problem it tries to solve
17:32karolherbst: especially as some people think about giving certain/some modules each own VM
17:32Wally: karolherbst: nvidia's kernel driver isnt full featured enough for nvk so you have to implement some stuff from userspace
17:33karolherbst: I would be surprised if something is missing
17:33Wally: The nvidia kernel driver doesnt expose pushbuffers through ioctls for example
17:34karolherbst: they don't?
17:34karolherbst: how do they do command submission then?
17:34Wally: map the pci bars
17:34karolherbst: I know there is a way to do that from userspace, but I thought they do have a kernel fallback
17:34karolherbst: Wally: oh boi
17:35Wally: looking through the xf86-video-nv stuff it actually seems quite elegant
17:45Wally: karolherbst: btw, the problem userspace drivers are trying to solve is the portability between operating systems, but I doubt there is any reason to do anything more than modesetting and cursor support in a usermode driver since most people who need more features will not use an obscure os like managarm
17:47karolherbst: sure.. we did modesetting in userspace once and it was terrible, so they'll move it into the kernel later anyway :)
17:49Wally: what if there is no kernel?
17:50karolherbst: then nothing matters anyway
17:53karolherbst: well.. if you poke hw directly this is what you do, but only by disregarding the concept of a kernel it becomes random code doing stuff :P
17:56Wally: I mean all of our devices with software just run random code doing stuff
17:57Wally: not saying that its an awful idea though :P
17:57karolherbst: sure, but either you call it a kernel and do it properly or you don't and it's a pile of code :P
18:58airlied: Wally: they dont map the pci bars
18:58airlied: they map userd allocations
18:59airlied: and poke values in there to submit pushbufs
18:59karolherbst: airlied: how does that work across contexts though?
18:59karolherbst: or well.. threads/processes
18:59airlied: you get a userd map per context
19:17Wally: airlied: oh, im probably still going to be mapping the bars since there doesnt seem to be any downside afaik
19:20Wally: (well other than not being able to use tegra)
19:27airlied:can't remember if userd maps are in bar2 or can be in sysram