17:17 ralphc: Hi, I'm Ralph Campbell. I work at Nvidia and I have been contributing Linux kernel fixes for HMM (SVM in nouveau) and working with Jason Gunthorpe, Jerome, Ben Skeggs, Karol Herbst, and others. I'm looking at adding 2MB transparent huge page migration support to nouveau and trying to understand a few things. I see that NOUVEAU_SVM_BIND calls nouveau_svmm_bind() which eventually calls nouveau_bo_new() and nouveau_bo_pin(). Is this the only way
17:17 ralphc: to allocate GPU memory? Where can I read/learn more about memory allocation & management? Thanks.
17:18 imirkin: i think you already know the answer to that question, and are just hoping for a different one...
17:18 ralphc:lol
17:19 imirkin: nouveau currently doesn't have a great way to enable user-managed VM's
17:19 imirkin: i'm guessing that this bo_new + pin was a quick & dirty way of getting things going without rewriting the whole VM management logic
17:20 imirkin: i know nvidia blob allows (probably requires) userspace to manage the VM setup directly, but there's currently no real such option with nouveau
17:21 imirkin: as for documentation... nothing much better than the code
17:21 imirkin: there are various docs about how the hardware itself works, but you probably don't need our help on that front
17:21 imirkin: (and they aren't much updated past fermi or so)
17:23 ralphc: I guess my question is more about if nouveau buffer objects are tied to DRM buffer objects or if they are separate.
17:23 imirkin: i was going to answer, but i don't want to lie to you. i'd guess that they're at least tied to ttm.
17:24 imirkin: (which is the thing that moves buffers around)
17:24 imirkin: i don't think there really is a "DRM buffer object"... do you mean GEM or dma-buf?
17:25 ralphc: Well, I'm very ignorant about ttm, gems, etc. I only have a very vague notion of that area.
17:25 imirkin: TTM is the thing that tells you to move buffers in/out of vram/gart/sysmem
17:26 imirkin: (gart == "gpu-dma-accessible system memory")
17:26 imirkin: GEM is a (imo very annoying) interface to identify buffers in various DRM APIs ... i think framebuffers etc must be GEM objects
17:27 ralphc: OK, I guess I'll have to dive into ttm, etc. At least you confirmed that there isn't some other API available.
17:28 imirkin: skeggsb has been promising one for about 3 years now. maybe 4.
17:28 imirkin: so my current estimate is 2025 :)
17:28 ralphc: I know how that goes. :-)
17:29 imirkin: in fairness, it's not exactly a trivial task
17:29 imirkin: keep in mind that this nouveau_bo_new stuff works all the way back to Riva TNT
17:30 imirkin: thus far, we've managed to avoid dropping support for any hw (other than bugs that crop up and aren't found until much later)
17:31 ralphc: Good to know, thanks.
17:36 imirkin: ralphc: looks like the core vmm has a managed/not-managed option
17:36 imirkin: where not-managed allows the "client" to manage this (i.e. the code in drm but not nvkm)
17:36 imirkin: however there are no userspace interfaces to enable this
17:37 imirkin: https://github.com/skeggsb/nouveau/blob/master/drm/nouveau/nvkm/subdev/mmu/vmm.c#L1093
17:48 ralphc: @imirkin I think you will find that nouveau_svmm_init() calls nvif_vmm_init() which enables SVM/client managed memory (except for a small range which is isn't). This is enabled through OpenCL/mesa. I have some test programs that can call mmap() or malloc() and GPU faults will use HMM to copy the CPU page table physical address to the GPU page table so the GPU can access any user process address the same as the host CPU. clEnqueueSVMMigrateMem()
17:48 ralphc: will migrate CPU data to GPU memory instead of accessing system memory over PCIe.
17:58 imirkin: right
17:58 imirkin: i'm aware of that
17:59 imirkin: but there's no API for "place this data at this VM address"
17:59 imirkin: and, more importantly, "set these PTE flags"
18:00 imirkin: which would be required to implement vulkan, for example
20:08 uis: Hi. What happend with pscnv github repo?
20:35 RSpliet: uis: no idea. It's never been particularly relevant...
20:38 RSpliet: "After being acquired and re-sold, by March 24 2017, Pathscale was reported to be looking for another buyer of its assets.[1] As of May 2017 its open source compiler has been removed from its GitHub account and the official company web site is down." - Wikipedia
20:38 RSpliet: That probably has something to do with it