14:31dwlsalmeida[d]: Can somebody explain to me the differences between channels, subchannels and classes?
14:32dwlsalmeida[d]: I assume that a channel is some hardware ringbuffer wherein commands can be submitted for execution by the firmware
14:33dwlsalmeida[d]: and that classes represent the different types of commands that can be recorded to a given command buffer
14:33dwlsalmeida[d]: is this correct?
14:34dwlsalmeida[d]: also, what does the SET_OBJECT method do? gfxstrand[d] once told me this nomenclature derives from OOP, but I don't understand it 100% yet
14:37dwlsalmeida[d]: (cc marysaka[d] :D)
14:42RSpliet: dwlsalmeida[d]: https://envytools.readthedocs.io/en/latest/hw/fifo/intro.html
14:45dwlsalmeida[d]: RSpliet: thank you, that's precisely it
15:03dwlsalmeida[d]: > The method selects an individual command of the object bound to the selected subchannel
15:03dwlsalmeida[d]: >
15:03dwlsalmeida[d]: How do these things not conflict? i.e. when processing VkCommandBeginVideoCoding, we assign subchannel 4 to nvdec:
15:03dwlsalmeida[d]: /* assign nvdec to sub channel 4 */
15:03dwlsalmeida[d]: __push_mthd(p, SUBC_NVC5B0, NV906F_SET_OBJECT);
15:03dwlsalmeida[d]: P_NV906F_SET_OBJECT(p, { .nvclass = 0xC5B0,
15:03dwlsalmeida[d]: .engine = 0 });
15:03dwlsalmeida[d]: What if some other process comes along and assigns subchannel 4 to another engine?
15:04dwlsalmeida[d]: we also set the class to C5B0, where B0 is all video commands, but C5 is Pascal IIUC, so what if we have a different GPU, shouldn't we take that into account here?
15:07dwlsalmeida[d]: Also, 4 is arbitrary here, right?
15:09gfxstrand[d]: Set `.nvclasa = pdev->info.cls_video` and the will take care of hardware generations.
15:14dwlsalmeida[d]: gfxstrand[d]: what is the difference between the "ws_* " stuff vs the "kmd_*" stuff?
15:14dwlsalmeida[d]: My impression was that ws stood for "window system", but I see nothing X11/Wayland specific there?
15:15dwlsalmeida[d]: instead, for the ws stuff, we seem to be calling the kernel driver:
15:15dwlsalmeida[d]: int
15:15dwlsalmeida[d]: nouveau_ws_context_create(struct nouveau_ws_device *dev,
15:15dwlsalmeida[d]: enum nouveau_ws_engines engines,
15:15dwlsalmeida[d]: struct nouveau_ws_context **out)
15:15dwlsalmeida[d]: {
15:15dwlsalmeida[d]: struct drm_nouveau_channel_alloc req = { };
15:15dwlsalmeida[d]: uint32_t classes[NOUVEAU_WS_CONTEXT_MAX_CLASSES];
15:15dwlsalmeida[d]: uint32_t base;
15:15dwlsalmeida[d]: *out = CALLOC_STRUCT(nouveau_ws_context);
15:15dwlsalmeida[d]: if (!*out)
15:15dwlsalmeida[d]: return -ENOMEM;
15:15dwlsalmeida[d]: int ret = drmCommandWriteRead(dev->fd, DRM_NOUVEAU_CHANNEL_ALLOC, &req, sizeof(req));
15:15dwlsalmeida[d]: if (ret)
15:15dwlsalmeida[d]: goto fail_alloc;
15:15gfxstrand[d]: Yeah, that naming was copied+pasted from RADV. It's bad naming.
15:31dwlsalmeida[d]: airlied[d]: ```
15:31dwlsalmeida[d]: /* assign nvdec to sub channel 4 */
15:31dwlsalmeida[d]: __push_mthd(p, SUBC_NVC5B0, NV906F_SET_OBJECT);
15:31dwlsalmeida[d]: P_NV906F_SET_OBJECT(p, { .nvclass = 0xC5B0,
15:31dwlsalmeida[d]: .engine = 0 });
15:31dwlsalmeida[d]: you assigned engine 0, but the VLD engine is apparently 6? i.e.:
15:31dwlsalmeida[d]: 6 VP3- PVLD falcon-based variable length decoder.
15:42gfxstrand[d]: I'd use whatever numbers the blob uses.
15:52dwlsalmeida[d]: gfxstrand[d]: is there a tutorial anywhere on how to dump memory from the blob? Dave sent me a trace which I assumed was obtained through vk_push_print, not from the blob
15:54gfxstrand[d]: https://gitlab.freedesktop.org/marysaka/envyhooks
15:54gfxstrand[d]: We should probably move that to nouveau/
15:55RSpliet: is this what we used to do with valgrind-mmt?
16:12karolherbst[d]: correct
16:13karolherbst[d]: just fast and also supports userspace command submission
16:18dwlsalmeida[d]: gfxstrand[d]: why do we only have a single struct nvk_queue in nvk_device?
16:18dwlsalmeida[d]: I was expecting an array of queues really
16:18gfxstrand[d]: History. We should fix that.
16:19gfxstrand[d]: `vk_device` already keeps a linked list. We just need to switch NVK to dynamically allocating them.
16:19dwlsalmeida[d]: ahhhhh
16:20dwlsalmeida[d]: I was trying to see where I should place the video queue, because Dave had a separate
16:20dwlsalmeida[d]: nvk_queue vid_queue;
16:20dwlsalmeida[d]: And that doesn't seem to be the way the rest of them work
16:20dwlsalmeida[d]: I tried poking around this function:
16:20dwlsalmeida[d]: VKAPI_ATTR void VKAPI_CALL nvk_GetDeviceQueue2(VkDevice device, const VkDeviceQueueInfo2* pQueueInfo, VkQueue* pQueue) VK_ENTRY_WEAK VK_ENTRY_HIDDEN;
16:20dwlsalmeida[d]: but this seems to not be defined anywhere?
16:20gfxstrand[d]: Yeah, that's just Dave hacking things up.
16:21gfxstrand[d]: dwlsalmeida[d]: That's handled in common code.
16:21gfxstrand[d]: Using the aforementioned linked list
16:21dwlsalmeida[d]: ack
16:22dwlsalmeida[d]: thank you for being so helpful on a Sunday lol
16:22dwlsalmeida[d]: 😂
16:38dwlsalmeida[d]: marysaka[d]: FYI, using envyhooks is giving me a segmentation fault
16:39dwlsalmeida[d]: I see it apparently expects a given firmware version
16:39dwlsalmeida[d]: > Update to 560.35.03
16:39dwlsalmeida[d]: which...may or may not be what I have, not sure
16:39tiredchiku[d]: that is intended, says so in the readme too
16:47dwlsalmeida[d]: I seem to be running the right version though:
16:47dwlsalmeida[d]: Name : nvidia
16:47dwlsalmeida[d]: Version : 560.35.03-15
16:47dwlsalmeida[d]: Description : NVIDIA kernel modules
16:47dwlsalmeida[d]: Architecture : x86_64
16:47dwlsalmeida[d]: URL : https://www.nvidia.com/
16:47dwlsalmeida[d]: Licenses : custom
16:47dwlsalmeida[d]: Groups : None
16:47dwlsalmeida[d]: Provides : NVIDIA-MODULE
16:47dwlsalmeida[d]: Depends On : linux nvidia-utils=560.35.03 libglvnd
16:47dwlsalmeida[d]: Optional Deps : None
16:47dwlsalmeida[d]: Required By : None
16:47dwlsalmeida[d]: Optional For : None
16:47dwlsalmeida[d]: Conflicts With : None
16:47dwlsalmeida[d]: Replaces : None
16:47dwlsalmeida[d]: Installed Size : 51.27 MiB
16:47dwlsalmeida[d]: Packager : Jan Alexander Steffens (heftig) <heftig@archlinux.org>
16:47dwlsalmeida[d]: Build Date : Thu Oct 17 18:26:33 2024
16:47dwlsalmeida[d]: Install Date : Sun Oct 20 12:57:20 2024
16:47dwlsalmeida[d]: Install Reason : Explicitly installed
16:47dwlsalmeida[d]: Install Script : No
16:47dwlsalmeida[d]: Validated By : Signature
16:47dwlsalmeida[d]: Name : nvidia-utils
16:47dwlsalmeida[d]: Version : 560.35.03-16
16:47dwlsalmeida[d]: Description : NVIDIA drivers utilities
16:48dwlsalmeida[d]: Architecture : x86_64
16:48dwlsalmeida[d]: URL : http://www.nvidia.com/
16:48dwlsalmeida[d]: Licenses : custom
16:48dwlsalmeida[d]: Groups : None
16:48dwlsalmeida[d]: Provides : vulkan-driver opengl-driver nvidia-libgl
16:48dwlsalmeida[d]: Depends On : libglvnd egl-wayland egl-gbm
16:48dwlsalmeida[d]: Optional Deps : nvidia-settings: configuration tool
16:48dwlsalmeida[d]: xorg-server: Xorg support [installed]
16:48dwlsalmeida[d]: xorg-server-devel: nvidia-xconfig
16:48dwlsalmeida[d]: opencl-nvidia: OpenCL support
16:48dwlsalmeida[d]: Required By : libglvnd nvidia wlroots
16:48dwlsalmeida[d]: Optional For : ffmpeg libvdpau vulkan-icd-loader
16:48dwlsalmeida[d]: Conflicts With : nvidia-libgl
16:48dwlsalmeida[d]: Replaces : nvidia-libgl
16:48dwlsalmeida[d]: Installed Size : 652.24 MiB
16:48dwlsalmeida[d]: Packager : Sven-Hendrik Haase <svenstaro@archlinux.org>
16:48dwlsalmeida[d]: Build Date : Tue Oct 15 11:27:11 2024
16:48dwlsalmeida[d]: Install Date : Sun Oct 20 12:57:19 2024
16:48dwlsalmeida[d]: Install Reason : Explicitly installed
16:48dwlsalmeida[d]: Install Script : Yes
16:48dwlsalmeida[d]: Validated By : Signature
16:48dwlsalmeida[d]: sorry for the non-paste-bin mess above,
16:48dwlsalmeida[d]: anyways, I wonder if these suffixes matter? i.e., the 15 and 16 here:
16:48dwlsalmeida[d]: 560.35.03-15
16:48dwlsalmeida[d]: 560.35.03-16
16:51tiredchiku[d]: don't think so, those are arch packaging related stuff
16:51tiredchiku[d]: hmm
16:52tiredchiku[d]: ah
16:52tiredchiku[d]: those tools are apparently for use with openrm
16:52tiredchiku[d]:I think
16:52tiredchiku[d]: so, nvidia-open on arch
16:53tiredchiku[d]: unless I'm misunderstanding the code structure
16:53tiredchiku[d]: yeah, seems to be the case: https://gitlab.freedesktop.org/marysaka/envyhooks/-/blob/main/src/openrm_bindings.rs?ref_type=heads
16:55dwlsalmeida[d]: Btw why do we have both available today? Is this something arch related?
16:55tiredchiku[d]: nvidia and nvidia-open? because both are supported by nvidia officially as of now
16:56dwlsalmeida[d]: I.e: isn’t the open stuff the kernel driver used by the blob too?
16:56dwlsalmeida[d]: Ahh
16:56tiredchiku[d]: yes, the propietary userspace drivers have two available kernel drivers
16:57tiredchiku[d]: the old proprietary `nvidia` one, which supports gens older than turing as well, and the open source kernel module which is turing and above only
16:57marysaka[d]: I depends on openrm to get the ioctl structures in my Rust code, proprietary driver have some unknown stuffs but everything we care about is public
16:57tiredchiku[d]: referred to as openrm
16:57marysaka[d]: dwlsalmeida[d]: do you have the backtrace? Can you also give me a reproducer, I will see if I can allocate a bit of time on that tomorrow
17:02dwlsalmeida[d]: marysaka[d]: I can, but not sure if you can derive anything helpful from it, it crashes somewhere in the blob:
17:02dwlsalmeida[d]: https://pastebin.com/aZ4wGyrs
17:03dwlsalmeida[d]: but anyways, as you guys said, let me try replacing nvidia with nvidia-open
17:03dwlsalmeida[d]: the issue may go away
17:06dwlsalmeida[d]: eeh, nvidia-open, same error
17:06dwlsalmeida[d]: pacman -Qi nvidia-open
17:06dwlsalmeida[d]: Name : nvidia-open
17:06dwlsalmeida[d]: Version : 560.35.03-17
17:06dwlsalmeida[d]: Description : NVIDIA open kernel modules
17:06dwlsalmeida[d]: Architecture : x86_64
17:06dwlsalmeida[d]: URL : https://github.com/NVIDIA/open-gpu-kernel-modules
17:06dwlsalmeida[d]: Licenses : MIT AND GPL-2.0-only
17:06dwlsalmeida[d]: Groups : None
17:06dwlsalmeida[d]: Provides : NVIDIA-MODULE
17:06dwlsalmeida[d]: Depends On : linux nvidia-utils=560.35.03 libglvnd
17:06dwlsalmeida[d]: Optional Deps : None
17:06dwlsalmeida[d]: Required By : None
17:06dwlsalmeida[d]: Optional For : None
17:06dwlsalmeida[d]: Conflicts With : NVIDIA-MODULE
17:06dwlsalmeida[d]: Replaces : None
17:06dwlsalmeida[d]: Installed Size : 6.06 MiB
17:06dwlsalmeida[d]: Packager : Jan Alexander Steffens (heftig) <heftig@archlinux.org>
17:06dwlsalmeida[d]: Build Date : Thu Oct 17 18:29:58 2024
17:06dwlsalmeida[d]: Install Date : Sun Oct 20 13:54:25 2024
17:06dwlsalmeida[d]: Install Reason : Explicitly installed
17:06dwlsalmeida[d]: Install Script : No
17:06dwlsalmeida[d]: Validated By : Signature
17:10marysaka[d]: dwlsalmeida[d]: do you have a reproducer? what are you using?
17:11marysaka[d]: My wild guess would be that the app replace my SIGSEGV handler
17:12dwlsalmeida[d]: marysaka[d]: yeah, I can get you a repro. I am using GStreamer's vkh264dec, which works fine without envyhooks in the LD_PRELOAD,
17:12dwlsalmeida[d]: well, the signal is SIGSEGV indeed..
17:12marysaka[d]: never used gstreamer, do you have some commandline for that? 😅
17:29dwlsalmeida[d]: marysaka[d]: commit ada194b7252fd31275a89e61655c4800d0043636 (HEAD, origin/main, origin/HEAD)
17:29dwlsalmeida[d]: $ meson setup build
17:29dwlsalmeida[d]: $ ninja -C build
17:29dwlsalmeida[d]: $ cd build && meson devenv
17:29dwlsalmeida[d]: $ gst-launch-1.0 filesrc location=64x64-I.h264 ! h264parse ! vulkanh264dec ! vulkandownload ! filesink location=/tmp/output.yuv
17:31dwlsalmeida[d]: https://cdn.discordapp.com/attachments/1034184951790305330/1297613109607010327/64x64-I.h264?ex=67168fe1&is=67153e61&hm=c6de2980a1468db1dec17dd100fb105312277521e6e19b3e79a7beb1f2ad9835&
17:31dwlsalmeida[d]: It's faster to share the input file than having you automatically generate one using gstreamer, you'd have to install x264 etc etc, but feel free to use any h264 file if you want, it shouldn't matter at all
17:33dwlsalmeida[d]: the funny thing is, it works up until this point:
17:33dwlsalmeida[d]: subprojects/gst-plugins-bad/gst-libs/gst/vulkan/gstvkinstance.c:753
17:33dwlsalmeida[d]: err =
17:33dwlsalmeida[d]: vkEnumerateInstanceExtensionProperties (NULL,
17:33dwlsalmeida[d]: &priv->n_available_extensions, NULL);
17:33dwlsalmeida[d]: if (gst_vulkan_error_to_g_error (err, error,
17:33dwlsalmeida[d]: "vkEnumerateInstanceExtensionProperties") < 0) {
17:33dwlsalmeida[d]: return FALSE;
17:33dwlsalmeida[d]: }
19:16airlied[d]: dwlsalmeida[d]: I just dumped traces from NVIDIA vulkan driver
19:16airlied[d]: But the dumper has some problems so I had to hack it
19:17dwlsalmeida[d]: airlied[d]: are you using gstreamer?
19:20airlied[d]: no I think I dumped it from an ffmpeg frame
19:21dwlsalmeida[d]: Yeah that works too..using Lynne’s work right
19:22airlied[d]: https://gitlab.freedesktop.org/airlied/mesa/-/commit/95dbf802161f160a1bb5e0fa78ebfff7d507c30b
19:23airlied[d]: was some of the hacks to the dumper that helped but I think even that wasn't sufficient to decode the video subchannels always
19:23dwlsalmeida[d]: Wait, that’s not using envyhooks right
19:24airlied[d]: nope, I never used envyhooks
19:24airlied[d]: also the envy docs are just likely not right for turing+
19:24dwlsalmeida[d]: Ahh…well envyhooks is what is creating the crashes, so makes sense that your setup is working
19:24airlied[d]: the values I used are the values the blob uses
19:27dwlsalmeida[d]: How did you hook vk_push_print to the blob? Isn’t that part of mesa?
19:27airlied[d]: that branch has a layer you can load
19:30airlied[d]: or maybe that was in a separate branch, I've paged out a lot of how it worked 🙂
19:32airlied[d]: I had a hack of gfxstrand[d] shader dumper layer that intercepted cmd buffer
19:32dwlsalmeida[d]: Yeah I vaguely remember that too
19:33airlied[d]: https://gitlab.freedesktop.org/airlied/mesa/-/tree/nv-pushbuf-dump-tool-hacks-video?ref_type=heads
19:34airlied[d]: nope not it
19:35airlied[d]: oh that probably was it
19:41dwlsalmeida[d]: gfxstrand[d]: in your opinion should I try this or wait for Mary to try and fix envyhooks?
19:45airlied[d]: I'm not sure you need any more traces though
19:45gfxstrand[d]: airlied[d]: Envyhooks is modern and has little to go with the rest of envytools
19:45airlied[d]: the classes are all documented
19:45airlied[d]: once you've set up the channel/subchannel stuff then it's just classes
19:46dwlsalmeida[d]: airlied[d]: yeah true..
19:47dwlsalmeida[d]: I guess it would be nice to trace this 64x64 I frame but on a second thought I can probably do without it
19:50airlied[d]: do you have a copy of my trace?
19:50dwlsalmeida[d]: Faith can you explain the kmd vs ws stuff?
19:51dwlsalmeida[d]: My understanding is that the kmd stuff replaced the ws stuff
19:51dwlsalmeida[d]: airlied[d]: yes I have your trace
19:52airlied[d]: does envyhooks have video class dumping?
19:53dwlsalmeida[d]: Eeh… good question, not sure
19:54dwlsalmeida[d]: Do you remember which video you used to generate the dump you sent ?
19:56airlied[d]: probably jellyfish-3-mbps-hd-h264.mkv