05:27 airlied: Lynne, zzoon_back_1st_Jan_2024[m] : https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27066 deep copying of parameter structs
05:27 airlied: JoshuaAshton: ^ just not integrated with the radv encode yet
07:59 ity: Re: "most graphics people are employed by some company so not much happens on weekends" hmm, fair. I guess I should wait today, then if nothing happens who should I nag?
08:00 karolherbst: ity: we should probably ask them to enable merge requests on that repo tho...
08:03 ity: Ahaha yea, that sounds like a good idea. I took this mostly as a an opportunity to learn how to contribute by mail, and thought that a minor patch for an off-by-one would be good.
08:03 karolherbst: my opinion is that nobody should go through that pain ever again, but some disagree... :D
08:05 ity: By the way, since I am already here, can I ask questions about the graphics stack? I have been attempting to figure it out by using strace, gdb & the mesa source tree, but there are quite a few things that I am having trouble figuring out. I am attempting to try to learn how the DRM API exposed by the i915 driver works
08:05 karolherbst: well.. depends if you are more interested in the displaying/wsi bits (kms) or rendering
08:06 ity: Both actually :D Though for now I guess the render engine is fine
08:06 ity: I was stracing a simple VK app I wrote
08:07 karolherbst: yeah.. though if you are interested more in rendering than strace won't really catch those bits as it all happens within command buffers and jobs submitted to the GPU, rather than ioctls (besides the one to submit jobs)
08:08 karolherbst: (and memory allocation)
08:08 ity: Yea, I guess so. I have been trying to at least catch the API that creates & submits the buffers
08:08 ity: I think it's the GEM?
08:09 ity: But what is the Aperture? I have been unable to find that out
08:12 ity: There is also a magic QUERY that seems to only be executed once? With null null no less, how does the mesa driver get info from it?
08:12 karolherbst: yeah, no idea about specifics to intel (including the terms)
08:13 ity: Ah, I see, which system are you familiar with?
08:13 ity: I have been mostly studying intel as they have PRMs and an iGPU seemed like it'd be easier than a dGPU
08:16 ity: Though I did take a generic look around the mesa tree (I even have it on my phone so I can read it on the go :D) and took a small peek at radv
08:28 karolherbst: yeah.. not sure if we have a central place where we could point to proper docs, but there are some blog posts around which might explain a bit
08:36 ity: Yea the docs are pretty sparse. I also have no idea which part of the tree ends up in what library... Eg where does the generic libdrm come from? Why does libdrm_intel exist but is never loaded? Why is libdrm_amdgpu loaded when the kernel amdgpu driver is not loaded? What is the diff between libvulkan_intel and libvulkan_intel_hasvk? - If you have any blog posts or anything, I
08:36 ity: would gladly give them a read!
08:49 karolherbst: maybe we should have a list of blogs or something 🙃
08:49 karolherbst: ity: maybe you can start with https://blogs.igalia.com/itoral/2014/08/08/diving-into-mesa/ (and the post linked in the first sentence)
08:49 karolherbst: though it's old :D
08:50 karolherbst: I wonder if anybody did a series like that in the past
08:50 karolherbst: uhm..
08:50 karolherbst: more recent past
09:10 mareko: ity: AMD dGPUs and APUs use exactly the same hw programming
09:10 mareko: in Mesa
09:13 mareko: ity: they work like Vulkan, one difference is that command buffers are built inside buffer objects using a hw-specific format of commands, and then the GPU is given a GPU pointer to that buffer object and executes it
09:15 mareko: ity: shaders are also written into buffer objects; the goal of Mesa is to prepare the buffer objects with GPU commands, states, and shaders, and tell the GPU to execute them
09:57 sima: karolherbst, https://dri.freedesktop.org/docs/drm/gpu/introduction.html#external-references we started collecting good overview talks/blogs in our docs, with dates so you can start with most recent ones
09:57 sima: it's pretty good for this kind of question
09:57 ity: Ah, iirc the SPV gets tranlated to NIH which then gets translated to the arch-specific ISA right? That's just the high-level overview though, I am not sure what format the buffers have, which DRM ioctl needs to be sent to make & upload a buffer, what to send to execute a buffer, retrieve the data... I will def give a read to the blog & references page though!
09:57 karolherbst: ohh, cool, didn't know
09:57 sima: ity, it's a bit more gerenal gpu stack, but might have some of the things you want to learn about too
09:58 sima: karolherbst, maybe similar list more focused on mesa in mesa docs might be good?
09:58 sima: airlied, should we have your new nouveau talk in there for an overview of modern render drm driver maybe?
09:59 ity: Seems like a cool list! I'll def check it out aahah, though I have a few specific questions for anv (I think) that are bugging me for a while now
10:00 sima: ity, #intel-3d for anv questions might be better than here
10:00 ity: OOH, oki!
10:01 javierm: sima: I think that the excellent recent articles from tzimmermann in lwn.net (intro to gfx part 1 and 2) would be good to have reference there too
10:01 karolherbst: sima: might also makes sense to just list all blogs of all drm/mesa devs :D
10:01 ity: That sounds good actually ahah, I'd love to give it a read!
10:01 ity: The blog post seems a bit dated, it references X & OpenGL
10:02 tzimmermann: javierm, ah yeah. i wanted to send a PR to add them to the docs. thanks for reminding me
10:02 javierm: tzimmermann: you are welcome, and thanks to you for writing those :)
10:03 sima: tzimmermann, javierm yeah definitely want to add those for sure
10:03 sima: I was more wondering about render stuff since we have a bit a lack of good overviews there, and airlied's recent nouveau revival talk is probably the closest
10:03 sima: least also because ttm and drm/sched are the two clearly still underdocumented areas at the kernel-doc level too :-/
10:04 sima: tzimmermann, if you type the patch rn pls also pastebin here for ity
10:05 sima: ity, opengl es is still very much the standard on smaller/embedded systems
10:05 tzimmermann: ity, https://lwn.net/Articles/955376/
10:05 sima: there's some issues around vulkan and low memory/overcommit situations
10:06 ity: Overcommit?
10:06 tzimmermann: ity, https://lwn.net/Articles/955708/
10:06 sima: ity, if your apps allocate more memory than you have
10:06 sima: gpu memory swapout (well nowadays in-place compression with zswap really) works much better with the gl userspace than vk
10:06 ity: Ah, why does it work better with GL?
10:07 sima: I've heard some people working on it, but until that's sorted gl tends to be better on really constrained systems still
10:07 sima: yeah
10:07 sima: otoh X driving drm directly is firmly outdated everywhere, more so on embedded :-P
10:08 ity: Interesting. I am mostly focusing on VK as it seemed like an easier path towards the inner details of the drivers, OpenGL is somewhat complex
10:08 ity: X doesn't modeset? Or do you mean it got replaced by WL?
10:09 karolherbst: sima: I guess the reason for that is, that in vulkan we don't want to know what buffers are used in a job, so the kenrel doesn't really know what is safe to page out or not?
10:09 sima: yeah X directly doing the modesetting is legacy, wayland compositor (or android's surfaceflinger) took this over
10:09 sima: with maybe Xwayland for backwards compat
10:09 sima: karolherbst, yeah so you need to swap the entire job essentially
10:09 sima: which means disastrous amounts of stalling
10:09 karolherbst: yeah..
10:09 karolherbst: it's not great
10:10 sima: so need to add some notion of working set to vk, but how
10:10 karolherbst: even with VM_BIND and all the fancy new submission ioctls, we still might need a "I know these bos are used for sure" list of bos somewhere...
10:10 karolherbst: dunno
10:10 ity: Is that internal driver state? Or smth inherent to the spec
10:10 sima: and how exactly do you need to sync on this
10:10 sima: karolherbst, yeah exactly
10:11 sima: or well inverse, "I know these big bos here are not used, you might want to swap them out"
10:11 karolherbst: the legacy ioctls (used by GL) generally have this...
10:11 karolherbst: sima: maybe both
10:11 karolherbst: but like
10:11 ity: Bos? Is that another 3 letter acronym
10:11 sima: but needs the full set of in/out fences on a binding queue
10:11 karolherbst: if you have a shader and use bda all bets are off anyway
10:11 karolherbst: well.. kinda
10:11 sima: karolherbst, yours sounds more like bo priorities as in "try to put these into vram"
10:11 pq: BO = buffer object
10:11 sima: which imo is orthogonal (but another gap compared to gl)
10:11 karolherbst: I think we'll end up with a priority system here anyway
10:11 karolherbst: like
10:11 sima: karolherbst, yeah
10:11 ity: OOH so just a plural of buffer object
10:12 karolherbst: like
10:12 sima: ity, yup
10:12 karolherbst: NV gpu can have some type of memory in VRAM or system RAM for rendering
10:12 karolherbst: but some things have to stay in VRAM
10:12 ity: I am feeling really out of place ahaha, as a beginner in the low level graphics stuff
10:12 karolherbst: and if your job needs more memory than you have VRAM you kinda have to figure out how to make it all work
10:12 sima: karolherbst, yeah atm for vk we only have the "must stay in vram" info
10:12 sima: with gl the userspace driver can sus out "probably good idea if it's in vram too"
10:13 karolherbst: yeah
10:13 karolherbst: but who is gonna make those changes to the gl driver anyway :P
10:13 sima: so vk gets you a lot more random bo selection if you overcommit vram
10:13 sima: karolherbst, amdgpu is pretty fancy on this iirc
10:13 karolherbst: right
10:13 karolherbst: yeah.. fair
10:13 sima: but imo the "do I even need this mapped or ok for kernel to swap/zswap out" is orthogonal
10:13 sima: since vram or not is just performance, not correctness
10:13 karolherbst: well...
10:14 karolherbst: there are two parts
10:14 karolherbst: swapping and compressing
10:14 karolherbst: those are not the same things
10:14 sima: so I'm leaning towards making this orthogonal
10:14 karolherbst: like, you can swap out some bo to system memory, but what if hte GPU accesses it while it's compressed?
10:14 sima: yeah in the kernel it's all swapout conceptually, just different targets
10:14 sima: karolherbst, hence why I think this needs separate, userspace promises to not access it while it's marked as "swapout-able"
10:15 karolherbst: right
10:15 sima: and userspace probably needs to supply in/out fences to the kernel so the kernel knows how to sync it all up
10:15 karolherbst: and then I come around the corner with my weirdo SVM use cases :P
10:15 sima: yeah those are big lolz anyway
10:15 karolherbst: I still have to get back to it and come up with a good solution
10:15 sima: whereas placement priority is kinda different, no disaster if it's not in vram, but it still has to be _always_ mapped
10:15 karolherbst: I have solutions, none are great
10:16 karolherbst: right
10:16 sima: imo svm only really works if you have actually competent hw page faults
10:16 sima: before that it's tech demo at best
10:16 karolherbst: not necessarily
10:16 sima: waaaaaaayyyyyy too many funny corners it all falls apart
10:16 karolherbst: ehhh
10:16 karolherbst: not too bad
10:17 karolherbst: without hw page faults you have to do explicit svm allocs
10:17 karolherbst: and then you just map it all properly
10:17 karolherbst: it's all doable
10:17 karolherbst: I have a prototype on iris which works, it just blows up your VM
10:17 sima: yeah I know, I've watched people try for years by now
10:17 sima: "we just userptr all the things"
10:17 karolherbst: :P
10:17 sima: for an extremely load bearing value of "just"
10:18 karolherbst: yeah..
10:18 karolherbst: that's basically my current approach, there are problems with it tho
10:18 sima: yeah so that's the one that works great for tech demo and then hits all kinds of funny in reality
10:19 karolherbst: right.. kinda depends on what you use it all for
10:19 sima: like the oddball userptr while most everything else is just normal bo is fine
10:19 karolherbst: at least on iGPUs using userptr isn't all that terrible, on dGPU it all sucks
10:20 sima: but if everything is userptr you really need reliable page faults to paper over all the corners
10:20 sima: and reliable = can preempt ctx while fault is pending
10:20 karolherbst: like you allocate on your host VM, and then place the bo you back with that userptr at the same location, and you are basically done
10:20 karolherbst: problem is.. on dGPUs you have that pcie bus in the way
10:20 sima: even on igpu you're not done, because userptr is so much worse on anything actually dynamic or overcommit than the current vk overcommit fun
10:21 karolherbst: what I really need is a way to reserve host VM and map things into it when I want it to be mapped
10:21 karolherbst: or use actual RAM
10:21 karolherbst: but all dynamically
10:21 sima: and if you don't do actual dynamic memory and overcommit, what's the point of svm
10:21 karolherbst: so I can back a certain GPU/CPU VM address with either system RAM or VRAM
10:21 sima: because you still can't just seamlessly reuse cpu libraries who all have this as very basic assumptions
10:21 sima: karolherbst, mmap(NULL)
10:22 karolherbst: I meant dynamic for the same allocation
10:22 karolherbst: so I can move the content around
10:22 sima: yeah that needs page faults
10:22 karolherbst: not really
10:22 sima: or you just stall disastrously
10:22 karolherbst: you can move it ahead of dispatch
10:22 karolherbst: there are CL APIs which gives you hint on where the memory should be
10:22 karolherbst: and then you just move it
10:22 karolherbst: and once you dispatch, you just don't move it
10:23 sima: that's still a few levels less sophisticated than what I've seen people actually try to ship
10:23 karolherbst: intels usm ext also has very very explicit placement of memory
10:23 karolherbst: ohh sure, system SVM is all transparent and needs real hw pagefaults
10:24 karolherbst: just talked about non system SVM stuff
10:30 lynxeye: DavidHeidelberg: Is there a way to stop codespell from disliking my name? https://gitlab.freedesktop.org/mesa/piglit/-/jobs/53676482
10:44 karolherbst: maybe we should turn codespell into a warning instead?
10:44 karolherbst: like as a comment on the MR
14:07 any1: https://elixir.bootlin.com/linux/latest/source/drivers/gpu/drm/i915/display/intel_dp.c#L975
14:07 any1: Isn't that || supposed to be &&?
14:07 karolherbst: gfxstrand: mind clarifying your comment on that MR? It sounds like you commented on an older version than what was there when you commented https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26775
14:13 any1: same, 4 lines below. As it is currently, sink_format is pretty much ignored, always resulting in RGB regardless.
14:13 karolherbst: any1: might want to bring it up in #intel-gfx
14:15 any1: ahh, I'd forgotten about that one.
22:37 trofi: https://dri.freedesktop.org frequently times out the connection. Both the front page and tarballs like https://dri.freedesktop.org/libdrm/libdrm-2.4.119.tar.xz . Is it a known issue? Or should I look on my side?
22:38 CounterPillow: Slow as molasses for me as well, so probably not on your end
23:16 kode54: same here
23:17 kode54: I wonder if some dipstick going through a midlife crisis is also targeting that server like someone is targeting sourcehut