00:17gfxstrand[d]: https://www.ebay.com/itm/156833359328
00:19gfxstrand[d]: You can get your own. 😉
00:21orowith2os[d]: $30? I thought they'd be closer to 10-15 now
00:22gfxstrand[d]: You can probably find a Maxwell Quadro for that.
00:25gfxstrand[d]: Here you go: https://www.ebay.com/itm/176936660633 $17 with free shipping.
00:25gfxstrand[d]: Ignore the K. K620 is GM107
00:26gfxstrand[d]: The great thing about Quadros is that they made a bunch of crappy ones to stick in Dell desktops so they can advertise an Nvidia GPU as an upsell and now those machines are getting recycled and so the second hand market is absolutely flooded with the things.
00:27gfxstrand[d]: And since they're crappy, they don't even need external power. You can stick like 3 of them in your case without upgrading your PSU.
00:28orowith2os[d]: what if I don't have a case :Shrug:
00:28orowith2os[d]: (joking)
00:28gfxstrand[d]: Case optional
00:29orowith2os[d]: https://cdn.discordapp.com/attachments/1034184951790305330/1355338054130466937/IMG_20250328_192839.jpg?ex=67e8906d&is=67e73eed&hm=2c2dabfe33e6a938d14746dde92e316729f53a13e0e7bd7574bbb30a63cdbe26&
00:29orowith2os[d]: Imagine hacking on nouveau with an egpu like this
00:30gfxstrand[d]: I avoid eGPUs. I don't need extra HW errors that aren't my fault.
00:31gfxstrand[d]: I just SSH into a desktop if I need to work from a hospital or coffee shop.
00:31gfxstrand[d]: I hope you're starting to feel better. 💜
00:33orowith2os[d]: they're gonna see if I'm clear to leave tomorrow morning :ferrisCowboyRTX:
00:33orowith2os[d]: lactic acid levels were pretty high before
00:34orowith2os[d]: might depend on if I need breathing treatments every so often though...
00:34orowith2os[d]: I don't want to leave and need to come right on back because I can't get my lungs to open up again
01:05gfxstrand[d]: Yeah. You really don't want that.
01:47x512[m]: gfxstrand[d]: Are Nvidia preemption broken on Linux? If run gears and some heavy demo, on Windows gears has 1000+ FPS, but on Linux gears has ~30 FPS.
01:47orowith2os[d]: what GPU gen, and is GSP confirmed to be up and running?
01:48gfxstrand[d]: That sounds like you're got getting GSP.
01:49gfxstrand[d]: I think you either need a kernel config option or a kernel command line option to get GSP on Turing.
01:49gfxstrand[d]: So depending on your distro, it might be off.
01:50x512[m]: It is openrm driver.
01:52orowith2os[d]: is gears using nvidia-gl or nvk_on_nvrm + zink?
01:52orowith2os[d]: or software rendering?
02:20gfxstrand[d]: x512[m]: With proprietary userspace?
02:35gfxstrand[d]: snowycoder[d]: Assigned to Marge. Thanks! 💜
02:47x512[m]: Windows:
02:47x512[m]:uploaded an image: (2114KiB) < https://matrix.org/oftc/media/v1/media/download/AZCT0QWRUZeBPjq_jJG8do3W0HTeQZzpvVxoY61FyabRBmwCdQ9HpUB6ZK4Ca339M5a0epXiWG6cUFiaR9S7CFpCeWJ-UbIwAG1hdHJpeC5vcmcvempScVFlaXVIVlRiemVIZlhGT1FZRFpQ >
02:47x512[m]: Linux-OpenRM-Nvidia:
02:47x512[m]:uploaded an image: (2465KiB) < https://matrix.org/oftc/media/v1/media/download/AQiBtDqvXFX430tt0VTPW78Z4QdfbFzM4sIAC-ghVOcKPqyoBCB7IGYv1ZyBgZsrKNPRao7lB_OOHvT_kwcMzphCeWJ-V7dQAG1hdHJpeC5vcmcvQUJURGdJeE5BVk9BZG1qWkdoQ2h4dm9j >
02:48x512[m]: Linux-Nouveau-NVK:
02:48x512[m]:uploaded an image: (2595KiB) < https://matrix.org/oftc/media/v1/media/download/AXa-CouNNU28nXzSCXK51D7ky5nrbAkUNNUyP8Zt89qcwLziODwKQ-y3othoJeRZpIU-xSZKqYudLi220KkoHVtCeWJ-X9ZQAG1hdHJpeC5vcmcvYW16TkNmaVViSnJmVHphVENQYm1FZ1dl >
02:48x512[m]: Haiku-OpenRM-NVK:
02:48x512[m]:uploaded an image: (1747KiB) < https://matrix.org/oftc/media/v1/media/download/AU8ZytqzxTvHarUa03KR1tGovA3f8dOl7ygyn_j8oXSeo-540t2xwwM9RVHo2llluOJXvICTaom91YZ4gBUISbxCeWJ-ZufQAG1hdHJpeC5vcmcvcWdlWEpYVE1aUG5LbVduSXNZYUFwT2xK >
02:49x512[m]: Only on Windows lightweight application is still fast and everything do not become slow because of one heavy application.
02:51x512[m]: Also on KDE Wayland cursor lags significantly.
02:56x512[m]: I hope screenshots are visible?
03:00orowith2os[d]: we can click on and view them from here, you're good
03:04mhenning[d]: That's showing nvk+nouveau at 165 fps in glxgears, which I'm not at all concerned about
03:05mhenning[d]: which is to say I think we have more pressing performance concerns than the way the workloads are scheduled in this case
03:08x512[m]: In Linux-Nouveau-NVK case, Gears fps drop down when move cursor and cursor starts lagging.
03:09x512[m]: KDE do not use hardware cursor on KDE?
03:09x512[m]: on Wayland
03:09mhenning[d]: Yeah, it's possible we're not hitting the hardware cursor there for some reason
03:11gfxstrand[d]: Yeah, I think KDE uses a SW cursor on Wayland. I noticed that in my "test all the DEs on Zink" a couple weeks ago.
03:12x512[m]: Are there Nvidia timeslice deadline settings on Linux?
03:12gfxstrand[d]: Not that we have easy knobs to tune
03:13gfxstrand[d]: I suspect at least some of what you're seeing has more to do with compositors and fencing than with NVK perf.
03:14gfxstrand[d]: On Windows, the compositor does a CPU wait/check on fences and renders the most recently ready frame from every client. This is potentially higher latency but necessary with the WDDM2 monitored fence design.
03:14x512[m]: Haiku have no compositor. It is GPU->CPU copy on each client and drawing on framebuffer in software.
03:15x512[m]: Ideally I wand to see guaranteeng
03:15gfxstrand[d]: On Linux, the compositor grabs the most recent frames that have been sent to it and composits those together, waiting on the client's fences. This means that one really slow client can slow down the compositor and that slows down the other apps.
03:16x512[m]: guaranteed 60 FPS for light programs no matter which GPU heavy software is running.
03:16gfxstrand[d]: In theory, if the app has a deep enough swapchain, they should be able to spin free but it depends on a variety of factors.
03:18gfxstrand[d]: What I'd like to do way off in the future is provide a way that clients can query timeline semaphores from shaders. That way the compositor can have a small texture array of possible frames per-client and make the choice which frame to use at the last possible nanosecond.
03:19gfxstrand[d]: The compositor would run at 60 FPS and apps would just do whatever and the compositor would always use the most recent frame.
03:19gfxstrand[d]: But that's way off in the future and requires being able to bind semaphores into shaders which is a bit nuts
03:19x512[m]: Can existing Vulkan API do that?
03:19gfxstrand[d]: No
03:20gfxstrand[d]: And it's probably off in the future a ways
03:20gfxstrand[d]: But it'd be cool...
03:23x512[m]: Do some Linux compositors implement method that Windows do with fence wait timeout?
03:25gfxstrand[d]: No. They typically do what I described above where they just submit with all the client waits.
03:28gfxstrand[d]: Doing that is more efficient and lower latency which is what you want most of the time. But it's pretty easy to DOS.
03:30x512[m]: For Haiku, system interface responsiveness it a priority, so something should be done with that even it it will decrease performance a bit.
03:31HdkR: ooo, I like the idea of binding semaphores in the shader for compositors, that's pretty cute
03:32gfxstrand[d]: I mean, it would basically be an 8B UBO.
03:32gfxstrand[d]: But we have to do some kernel work to actually attach a BO to the syncobj and update its contents when new time points pass.
03:32HdkR: Yea, at the end, just would require a ton of plumbing work
03:33gfxstrand[d]: It could potentially be done entirely in userspace with a monitor thread but we could do it way more efficiently in the kernel.
03:33gfxstrand[d]: On Windows, it would just be a matter of mapping the monitored fence memory into the shader.
03:33gfxstrand[d]: Same with nvrm
03:34orowith2os[d]: gfxstrand[d]: that would be a bad compositor, no?
03:35orowith2os[d]: I think GNOME had something about that in the past, and they fixed it
03:35gfxstrand[d]: <a:shrug_anim:1096500513106841673>
03:35gfxstrand[d]: Lots of compositors are "bad" by that definition, then.
03:35gfxstrand[d]: I do think some of them try to be smart about it
03:35orowith2os[d]: probably just depends on how they decide to do things
03:35x512[m]: NVRM sem-surf may be useful.
03:36orowith2os[d]: it would make sense to keep a hold of the last valid buffer until you have another one so that you can keep pumping out frames, even if a client is too slow to keep up, and would have you waiting on it
03:36orowith2os[d]: I'll try and find the PRs for it...
03:36gfxstrand[d]: Yeah. They all hold on to a valid buffer
03:37gfxstrand[d]: They don't actually wait on the client to give them a new one before compositing.
03:37orowith2os[d]: https://gitlab.gnome.org/GNOME/mutter/-/issues/1162
03:37gfxstrand[d]: But the client can hand them a buffer with 50ms of work queued up to render it and then the compositor will blindly assume it'll be done "soon" and submit a composite, depending on all that work.
03:39gfxstrand[d]: Yeah, looks like Michel did a bunch of work on it about 3 years ago: https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1880
03:39gfxstrand[d]: But I don't think most compositors are quite so advanced.
03:40x512[m]: gfxstrand[d]: I suppose it is possible to map sem-surf memory into shader and it will work?
03:41gfxstrand[d]: Probably
03:41gfxstrand[d]: I don't see why not
03:42x512[m]: So it can be done by mixing Vulkan and NVRM API.
03:43orowith2os[d]: gfxstrand[d]: probably not, and it's for this reason I hope to never have to deal with it on my own...
03:43orowith2os[d]: but who knows. I might have to anyways, what with my rxwl stuff and whatnot
03:45orowith2os[d]: we love VRR :catgirl_heart:
03:45gfxstrand[d]: x512[m]: Or more likely via a new Vulkan/SPIR-V extension no one has written yet.
03:47x512[m]: It is not a problem to directly use NVRM in Haiku because it can be hidden behind platform integration driver (something like DDX driver in X11).
03:47x512[m]: I suspect that making Vulkan extension and implementing it in Linux distros will take years.
03:49gfxstrand[d]: Depends on how motivated I get. 😂
03:50gfxstrand[d]: I've got a decent idea how I'd implement it. It's a week or two for the kernel work (because kernel dev is slooowwwww), an afternoon for NVK, two days to write the spec, and then a few months of bikeshedding.
03:51gfxstrand[d]: And like 3 years for it to make its way to the top of my priority list. :frog_upside_down:
03:53gfxstrand[d]: And it's not clear if it's even a good idea. If you read through what Michel had to do to make GNOME work, I'm not sure that can be done and pushed all the way into the GPU practically. There's too much other stuff going on with a given `wl_surface`. And if no one's going to use it, then why draft it? Besides the coolness factor, that is.
03:53gfxstrand[d]: And it does have a pretty high coolness factor
03:53gfxstrand[d]: I suspect VR HMD compositors would actually use it, though.
03:54x512[m]: You give an opportunity for Haiku to be superior :)
03:54gfxstrand[d]: <a:shrug_anim:1096500513106841673>
03:55gfxstrand[d]: It's easy if you're nvrm. It's way harder on most of the other DRM drivers.
03:55gfxstrand[d]: But implementable. I'm pretty sure I even have a decent idea how.
03:58rinlovesyou[d]: orowith2os[d]: How does this work with the Nvidia driver installed? Are we just sending both Nvidia and nvk in userspace and can switch?
03:59orowith2os[d]: that's how things would work, yeah
03:59rinlovesyou[d]: That's actually kinda awesome
03:59orowith2os[d]: mesa and nvidia-prop can cooperate just fine
03:59rinlovesyou[d]: Is it just setting the vulkan icd?
03:59orowith2os[d]: if you build it for flatpak, it'll mount both in too, so you don't miss out there either
04:00orowith2os[d]: ask tiredchiku[d] for details, I guess?
04:01x512[m]: Yes, NVK is Vulkan ICD. You can use both NVK and Nvidia proprietary Vulkan/OpenGL/CUDA userland driver at the same time on NVRM KMD.
04:02orowith2os[d]: I assume Rin is talking more about which is preferred as the default?
04:02x512[m]: You can choose which ICD to use with env variables.
04:02rinlovesyou[d]: yeah i was just asking if that's all i need to do to choose one or the other
04:05gfxstrand[d]: It'll go in loader enumeration order. Since NVIDIA installs their ICD JSON file to `/etc/vulkan` and Mesa installs to `/usr/share/vulkan`, the order will be deterministic, at least for any given loader version. IDK which will be first, though.
04:07gfxstrand[d]: And the Mesa device select layer will do *something*
04:11gfxstrand[d]: And this is why I don't want NVK to be enabled by default on nvrm ever. Since Mesa is always installed by your distro system-wide, I don't want NVIDIA having to ship weird layers to disable NVK or telling people to uninstall Mesa or anything crazy like that. If you choose to run their driver stack, that's what you should get unless you go out of your way to ask NVK+NVRM.
04:12rinlovesyou[d]: yeah no i don't think this should be the default at all, i just like playing around with development stuff :P
04:12x512[m]: Yes, but user should have an option to enable NVK.
04:12rinlovesyou[d]: i do imagine this is gonna be very nice qol for development purposes
04:13x512[m]: In worst case by compiling NVK from source.
04:13rinlovesyou[d]: ❯❯ VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/nouveau_icd.x86_64.json NVK_ON_NVRM=1 MESA_VK_WSI_DEBUG=sw vulkaninfo
04:13rinlovesyou[d]: ERROR: [Loader Message] Code 0 : terminator_CreateDevice: Failed in ICD /usr/lib/libvulkan_nouveau.so vkCreateDevice call
04:13rinlovesyou[d]: ERROR: [Loader Message] Code 0 : vkCreateDevice: Failed to create device chain.
04:13rinlovesyou[d]: ERROR at /usr/src/debug/vulkan-tools/Vulkan-Tools/vulkaninfo/./vulkaninfo.h:1724:vkCreateDevice failed with ERROR_UNKNOWN
04:13rinlovesyou[d]: :(
04:13rinlovesyou[d]: did i miss a build option in the changes
04:14gfxstrand[d]: Right now, it's determined by kernel driver. If you run NVIDIA's kernel, you get NVIDIA's userspace. If you run the FOSS kernel, you get FOSS userspace. If we ever cross the streams, we need to be very careful that we aren't sabotaging NVIDIA and their ability to ship software to their users. That's a good way to get them pissed off and get them to pull what little support they've given.
04:15x512[m]: So NVK should be not a default option on NVRM. Nvidia should be fine with that.
04:15gfxstrand[d]: x512[m]: That's fine. If someone wants to build their own meas or carry an environment variable, they can do that. I don't care. But distro Mesa should always have NVK+NVRM disabled by default.
04:17gfxstrand[d]: Sorry if I'm banging that drum a little loud. I just want to be really clear where the lines are so that people can set their expectations accordingly.
04:18gfxstrand[d]: Of course, if NVIDIA decided that NVK was so cool and awesome that they want to start shipping it instead of their proprietary userspace, that would be awesome and I would totally help make that happen. But I think the chances of that are near zero.
04:19HdkR: NVK with CUDA interop hooks inbound
04:32gfxstrand[d]: Heh
04:33gfxstrand[d]: On RISC-V
04:40gfxstrand[d]: One more to go and all the merge requests that are left are ones that require actual thinking to review.
04:58HdkR: I mean, NVIDIA is shipping RISC-V for the Falcon now, it can self-host the GPU :P
05:26x512[m]: I hope that Nvidia will never switch to Nouveau KMD and will keep their NVRM. It is great because it is portable and contribute to open source diversity.
05:43gfxstrand[d]: I don't want them running on nouveau, either. I do have some hope for Nova, though. But first we'll need to get it working and supporting all the things.
05:50tiredchiku[d]: rinlovesyou[d]: no, it's just partly hardcoded for my 3070 still
05:51tiredchiku[d]: don't expect that to change until sunday (6th April) however, I'm out on vacation
05:54x512[m]: I made class detection code in my branch: https://github.com/X547/mesa/blob/0608306449e3e6abcab48dc4e57c8ba4698d0beb/src/nouveau/vulkan/nvkmd/nvrm/nvkmd_nvrm_pdev.c#L317
05:55tiredchiku[d]: yes I'm aware you mentioned it yesterday
05:57x512[m]: Are long or short names preferred in Mesa code style?
05:57x512[m]: .cls_copy = nvkmd_nvrm_pdev_find_supported_class(pdev, ARRAY_SIZE(sSubchannelCopyClasses), sSubchannelCopyClasses),
05:57x512[m]: This is starting to be long.
08:53asdqueerfromeu[d]: gfxstrand[d]: And a misbehaving compositor can break keyboard input almost completely (SysRq may use a special mechanism for getting input though which bypasses the hanging keyboard)
09:15asdqueerfromeu[d]: gfxstrand[d]: I was thinking of packaging a minimal NVIDIA userspace (at least without any OpenGL/Vulkan stuff) with the OpenRM driver + NVK (which would include Zink for OpenGL support) so I don't have to deal with some of the NVIDIA-isms while having stuff like proper hardware monitoring with NVML (because nouveau still doesn't support that) but I'm not sure I can make that public
09:35redsheep[d]: gfxstrand[d]: Yeah I think if someone wanted to package nvk+openrm it should be set as explicitly conflicting with the actual nvidia driver packages so this scenario never occurs, you get one or the other. I agree that enumerating by default without an environment variable, build option, or carried patch would be problematic. I think the ideal would be having some kind of option for a mesa build
09:35redsheep[d]: to have it defaulted without having to patch, but that's not a big deal either way.
09:35redsheep[d]: And that package should absolutely never be the default that a distro ships. Ideally be the time haiku and us testers have banged on nvk-nvrm enough to have any kind of package be viable nova will have made enough progress that it would be mostly irrelevant.
09:36redsheep[d]: asdqueerfromeu[d]: I for one would like having such a package even while it would be super experimental, but maybe don't put it on the AUR where unsuspecting users will stumble upon it and do something crazy like filing openrm bugs
09:53x512[m]: redsheep[d]: Conflicting packages is bad idea because some people may want to run NVK with CUDA for example.
09:54x512[m]: Some flag or alternative NVK package is better.
09:55x512[m]: For example build mesa-nvk-nvrm as separate package not installed by default.
12:13pac85[d]: gfxstrand[d]: Don't some compositors wait on the CPU like windows to avoid a fence from the client slowing down the compositor?
12:17pac85[d]: gfxstrand[d]: https://blogs.gnome.org/shell-dev/2023/03/30/ensuring-steady-frame-rates-with-gpu-intensive-clients/
12:17pac85[d]: Gnome stopped doing this a while back
12:19pac85[d]: gfxstrand[d]: Also not sure about the low latency bit. If you schedule compositing as late as possible you are basically guaranteed to miss frames if you don't make sure all buffers have finished drawing. Gamescope also does CPU waits afaik
12:20rinlovesyou[d]: tiredchiku[d]: Gotcha
12:25pac85[d]: gfxstrand[d]: Wow this is a cool idea.
12:25pac85[d]: I guess perhaps one way would be to attach a bda address to a timeline semaphore and have the driver just write the syncpoint to it right after the submission has finished. So api wise you could just have an extension struct with the bda
12:26asdqueerfromeu[d]: rinlovesyou[d]: I'll do another attempt at un-hardcoding the code later
12:35rinlovesyou[d]: Yeah i was gonna have a lookie too though i have no idea what I'm doing
13:41gfxstrand[d]: x512[m]: Try to keep lines to 80 chars and abbreviate where it makes sense. NVK has started leaning more into the abbreviations. nvrm_find_class might be enough there. It's not like that function is going to be exposed to all of NVK.
13:44gfxstrand[d]: pac85[d]: Writing to semaphores gets sticky fast. But being able to read them should be implementable.
15:51pac85[d]: gfxstrand[d]: Yeah that makes sense
16:21redsheep[d]: x512[m]: At least on arch you could have it conflict and provide both nvidia-open and nvidia-utils for the kernel and userspace, and then like dodo was talking about include pretty much all of the parts of nvidia-utils that aren't the vulkan driver. I see the nvidia-utils pkgbuild already just downloads their .run so I expect you could do the same and just delete their vulkan driver before
16:21redsheep[d]: installing the nvrm-enabled mesa build
16:24redsheep[d]: If you don't do it that way you risk potentially messing Nvidia up the way faith warned about, users not understanding what they installed and confused why their Nvidia drivers are broken now, filing bugs in the wrong places and making noise