00:03 fdobridge: <m​arysaka> nice catch 😅
02:28 fdobridge: <!​[NVK Whacker] Echo (she) 🇱🇹> Did this happen in the non-serial CTS run?
03:37 fdobridge: <g​fxstrand> Nope
03:38 fdobridge: <g​fxstrand> It didn't happen in my serial run with deqp-runner, either. I only saw it in my "Let's pretend I want to submit conformance" run.
03:52 fdobridge: <g​fxstrand> Ugh... these timeline semaphore tests take forever. 🙄
03:59 fdobridge: <!​[NVK Whacker] Echo (she) 🇱🇹> How do they compare to memory model tests?
04:35 fdobridge: <g​fxstrand> IDK. I've not run the memory model tests
04:36 fdobridge: <g​fxstrand> Running @marysaka 's GS branch and seeing lots of faults. Looks like stuff is running but something's still wrong.
06:36 fdobridge: <a​irlied> I wonder if that race is responsible for those wierd ampere sked issues
06:43 fdobridge: <m​ohamexiety> that's a really obscure bug 😮
06:43 fdobridge: <m​ohamexiety> nice catch!
08:24 fdobridge: <m​arysaka> Are those mmu faults?
14:26 fdobridge: <g​fxstrand> Yeah
14:26 fdobridge: <g​fxstrand> Quite possibly
14:53 fdobridge: <g​fxstrand> Here's the `NVK_USE_NAK=all` numbers:
14:53 fdobridge: <g​fxstrand> `Pass: 399394, Fail: 5810, Crash: 11, Warn: 4, Skip: 3194910, Timeout: 2, Flake: 752, Duration: 2:30:54`
14:53 fdobridge: <g​fxstrand> Not bad, really. Just something's still wrong with GS
14:54 fdobridge: <g​fxstrand> Binding model tests especially seem to go boom
15:35 fdobridge: <g​fxstrand> Damn... It really does smell like something to do with branching.
15:46 fdobridge: <g​fxstrand> Yup! It's branching alright...
15:46 fdobridge: <g​fxstrand> The next question is: WTF?!?
15:50 fdobridge: <g​fxstrand> I think the old compiler is saved by control-flow re-convergence. However, geometry shaders have no general guarantee that `emitVertex()` happens in uniform control-flow.
15:52 fdobridge: <g​fxstrand> I'm thinking this is a problem for Monday...
15:52 fdobridge: <k​arolherbst🐧🦀> mhh.. you think that `OUT` needs to be executed uniformly?
15:55 fdobridge: <g​fxstrand> IDK. That seems off because, like I said, there's no GLSL or SPIR-V requirement that it must.
15:55 fdobridge: <g​fxstrand> But it's definitely breaking when it's not.
15:55 fdobridge: <g​fxstrand> Forcing it to be uniform would be some real compiler shenanigans.
15:55 fdobridge: <g​fxstrand> The other thing is that it does seem to work. It just doesn't pick up all of the output stores correctly.
15:56 fdobridge: <k​arolherbst🐧🦀> are you sure to always execute `OUT.FINAL`?
15:56 fdobridge: <g​fxstrand> Yeah
15:56 fdobridge: <k​arolherbst🐧🦀> are you sure that the state register is always passed through correctly?
15:57 fdobridge: <g​fxstrand> Looks like
15:58 fdobridge: <k​arolherbst🐧🦀> also, the register has to be initialized with 0
15:59 fdobridge: <k​arolherbst🐧🦀> mhhh... the docs are a bit confusing here.. because later it also states it has to be sourced by `AST` 🙃
15:59 fdobridge: <g​fxstrand> It does have to be sourced by AST but IDK that that's always happening
15:59 fdobridge: <k​arolherbst🐧🦀> ehh wayt.. I read that wrongly, yeah
15:59 fdobridge: <k​arolherbst🐧🦀> *wait
16:00 fdobridge: <k​arolherbst🐧🦀> anyway.. not really seeing any requirement that threads must be converged before executing it..
16:02 fdobridge: <k​arolherbst🐧🦀> nvidia uses exit handler to call `OUT.FINAL`, but with proper lowering that shouldn't be required anyway
16:02 fdobridge: <g​fxstrand> So, we're actually using $ZERO for AST for a bunch of cases.
16:02 fdobridge: <g​fxstrand> Maybe that's messing something up?
16:02 fdobridge: <k​arolherbst🐧🦀> what the docs also suggest is to allocate a register for the state thing
16:02 fdobridge: <k​arolherbst🐧🦀> would be surprising.. but also possible
16:02 fdobridge: <k​arolherbst🐧🦀> do you even know how $ZERO works? 😄
16:03 fdobridge: <k​arolherbst🐧🦀> or rather, what makes it return 0?
16:03 fdobridge: <k​arolherbst🐧🦀> it's a little trick, that any OOB register read is returning 0, just for $ZERO the hardware doesn't complain or something.
16:04 fdobridge: <k​arolherbst🐧🦀> which... might cause some issues, and I think codegen has a few places where it forces a live register with 0 instead of RZ
16:04 fdobridge: <g​fxstrand> There are definitely instructions which don't like it
16:04 fdobridge: <g​fxstrand> Typically things that source vectors like tex or st
16:05 fdobridge: <g​fxstrand> Even if they're only reading a single component, they don't like ZERO
16:05 fdobridge: <k​arolherbst🐧🦀> yeah...
16:05 fdobridge: <g​fxstrand> Learned that one the hard way. 🙃
16:05 fdobridge: <k​arolherbst🐧🦀> oof
16:05 fdobridge: <g​fxstrand> In any case, making AST not use zero doesn't fix anything
16:05 fdobridge: <k​arolherbst🐧🦀> anyway.. you might want to try out `OUT` without RZ
16:06 fdobridge: <k​arolherbst🐧🦀> maybe even try to force the same register for all `OUT`s
16:06 fdobridge: <k​arolherbst🐧🦀> not sure if Nvidia always uses a specific instruction or not
16:06 fdobridge: <g​fxstrand> We're currently using R0 for all outs
16:06 fdobridge: <g​fxstrand> By sheer RA luck but it at least shows that's not the problem
16:07 fdobridge: <k​arolherbst🐧🦀> mhhh
16:07 fdobridge: <k​arolherbst🐧🦀> it's even recommended to always use the same register
16:07 fdobridge: <!​[NVK Whacker] Echo (she) 🇱🇹> I wonder how long Karol can infodump stuff for about special NVIDIA hardware quirks/features 🤔
16:08 fdobridge: <k​arolherbst🐧🦀> but anyway.. not really seeing any weird requirements here... could also be something else going wrong
16:08 fdobridge: <k​arolherbst🐧🦀> anyway.. thread reconvergence isn't hard to implement anyway, so might as well try it
16:10 fdobridge: <g​fxstrand> Yeah, but that doesn't solve the problem. You can `EmitVertex()` in a non-uniform loop
16:10 fdobridge: <g​fxstrand> That just fixes the CTS tests. It'll break in the wild
16:11 fdobridge: <g​fxstrand> The fact that there are no CTS tests for EmitVertex in non-uniform control-flow is a little disturbing...
16:12 fdobridge: <k​arolherbst🐧🦀> checked what nvidia emits?
16:12 fdobridge: <g​fxstrand> And like I said, it does work. It emits vertices. It just gets one of the colors wrong
16:12 fdobridge: <k​arolherbst🐧🦀> sounds like something else is going wrong tho
16:13 fdobridge: <k​arolherbst🐧🦀> there aren't really many instructions which need converged threads, mostly just barriers and subgroup things
16:14 fdobridge: <m​arysaka> Could it be related by the fact that EXIT isn't at the end of the shader most of the time?
16:15 fdobridge: <m​arysaka> From what I recall NVIDIA drivers put some infinite branch after each EXIT too but idk if that truly matter
16:15 fdobridge: <k​arolherbst🐧🦀> I wonder if `OUT.FINAL` needs converged threads?
16:15 fdobridge: <k​arolherbst🐧🦀> and I wonder if the exit handler always executes with converged threads
16:16 fdobridge: <k​arolherbst🐧🦀> yeah...
16:16 fdobridge: <k​arolherbst🐧🦀> all threads enter the exit handler together
16:16 fdobridge: <k​arolherbst🐧🦀> if it matters for `OUT.FINAL`? no idea
16:17 fdobridge: <k​arolherbst🐧🦀> but if it does, nvidia does the correct thing
16:17 fdobridge: <m​arysaka> The fact that even for a simple shaders it was doing that, maye it does matter...?
16:17 fdobridge: <k​arolherbst🐧🦀> maybe?
16:18 fdobridge: <k​arolherbst🐧🦀> might also be just static GS specific code
16:18 fdobridge: <k​arolherbst🐧🦀> it does simplify things
16:18 fdobridge: <k​arolherbst🐧🦀> for two resons
16:18 fdobridge: <k​arolherbst🐧🦀> 1. you always have to execute `OUT.FINAL`, so might as well push it into the exit handler for everything
16:19 fdobridge: <k​arolherbst🐧🦀> 2. if you allocate a register for all `OUT`s, you can just use that in the exit handler as well
16:19 fdobridge: <k​arolherbst🐧🦀> and then you won't have to bother that CFG manipulation messes up your GS
16:20 fdobridge: <g​fxstrand> Ooh, I think I found the bug
16:20 fdobridge: <g​fxstrand> Maybe
16:20 fdobridge: <k​arolherbst🐧🦀> anyway.. having exit handler support in NAK might also come in handy with other things 😄
16:20 fdobridge: <k​arolherbst🐧🦀> but anyway.. they are per warp, and only executed if all threads are in KILLED state
16:21 fdobridge: <!​[NVK Whacker] Echo (she) 🇱🇹> Hopefully NAK will be able to do ProcessVertices() now
16:21 fdobridge: <k​arolherbst🐧🦀> huh...
16:22 fdobridge: <k​arolherbst🐧🦀> exit acts as an implied join also for helper threads
16:22 fdobridge: <g​fxstrand> It's mary's opt_out
16:22 fdobridge: <g​fxstrand> IDK what it's doing but it's doing it wrong
16:22 fdobridge: <m​arysaka> huh
16:23 fdobridge: <m​arysaka> oh that may make sense... now that I think about it I only saw Maxwell/Pascal emit that with NVIDIA driver, not with Turing
16:23 fdobridge: <m​arysaka> it was doing OUT.EMIT and OUT.CUT one after the other...
16:23 fdobridge: <k​arolherbst🐧🦀> 🙃
16:23 fdobridge: <g​fxstrand> opt_out is throwing away a predicate on a BRA
16:23 fdobridge: <g​fxstrand> @marysaka, what is opt_out trying to do?
16:24 fdobridge: <m​arysaka> Merging OUT.EMIT and OUT.CUT when they are one after the other and if the stream id is the same
16:24 fdobridge: <g​fxstrand> Ah
16:24 fdobridge: <m​arysaka> so I suppose I forgot about predicates oops
16:25 fdobridge: <k​arolherbst🐧🦀> might also want to get rid of `OUT.CUT` if the stream id changes
16:26 fdobridge: <k​arolherbst🐧🦀> I guess that merges them to `EMIT_THEN_CUT~?
16:26 fdobridge: <k​arolherbst🐧🦀> I guess that merges them to `EMIT_THEN_CUT`? (edited)
16:26 fdobridge: <g​fxstrand> Okay, disabled opt_cut and I'm going to do another CTS run.
16:26 fdobridge: <k​arolherbst🐧🦀> anyway, that's something the hardware does. Inserting an `OUT.CUT` if the stream id changes on the transition point.
16:30 fdobridge: <m​arysaka> yeah
16:48 fdobridge: <g​fxstrand> Okay, I fixed it.
16:49 fdobridge: <g​fxstrand> Time for a full CTS run with `NVK_USE_NAK=all` 🚀
16:57 fdobridge: <m​arysaka> so it was merging an OUT.EMIT and OUT.CUT with different predicate conditions?
17:06 fdobridge: <m​henning> @gfxstrand Was it intentional that you removed the commits from https://gitlab.freedesktop.org/gfxstrand/mesa/-/merge_requests/44 from nak/main?
17:09 fdobridge: <g​fxstrand> Nope
17:09 fdobridge: <g​fxstrand> All this rebasing...
17:10 fdobridge: <k​arolherbst🐧🦀> just ship it already 🙃
17:12 fdobridge: <g​fxstrand> Yeah...
17:12 fdobridge: <g​fxstrand> No, it was dropping the predicates from any instruction that happened to come after an OUT.CUT
17:13 fdobridge: <m​arysaka> oh no :blobcatnotlikethis:
17:13 fdobridge: <g​fxstrand> Yeah, you can't safely, pull out an op and then do `Instr::new_boxed(op)` and trust that you get the same instruction.
17:16 fdobridge: <g​fxstrand> I just need to decide how much I hate my proc macro wraps
17:16 fdobridge: <g​fxstrand> Xavier has a meson MR to use actual crates as wraps but it's not in meson upstream yet.
17:18 fdobridge: <k​arolherbst🐧🦀> yeah....
17:19 fdobridge: <g​fxstrand> I'm going to chat with Dylan and I think we'll probably just merge the horrible wraps for now.
17:20 fdobridge: <m​henning> should also remove the hacked changes to codegen
17:20 fdobridge: <g​fxstrand> Yeah
17:21 fdobridge: <g​fxstrand> There's a bit of maintenance work to do on the branch.
17:21 fdobridge: <g​fxstrand> I need to pull the NIR bits out into an MR (one already exists but it's way out of date)
17:21 fdobridge: <g​fxstrand> I need to drop the codegen hacks
17:21 fdobridge: <k​arolherbst🐧🦀> though maybe we should wait for 1.3 because that will probably also have that static inline wrapping stuff
17:21 fdobridge: <g​fxstrand> Scan through for spurrious NVK changes that might be breaking things.
17:22 fdobridge: <g​fxstrand> That's not actually hurting me too bad.
17:22 fdobridge: <k​arolherbst🐧🦀> it's more about others though
17:23 fdobridge: <g​fxstrand> We can stick in meson requirements on a per-component basis
17:23 fdobridge: <g​fxstrand> We're not going to lock all of Mesa to 1.3
17:23 fdobridge: <k​arolherbst🐧🦀> e.g. https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129
17:24 fdobridge: <k​arolherbst🐧🦀> though I think it was worse in the past
17:24 fdobridge: <g​fxstrand> I don't see what that has to do with NAK. We basically never use types in nak_from_nir
17:25 fdobridge: <k​arolherbst🐧🦀> just the general thing if somebody converts functions to static inlines
17:25 fdobridge: <k​arolherbst🐧🦀> and then having to update all the rust code using it
17:25 fdobridge: <k​arolherbst🐧🦀> but yeah...
17:25 fdobridge: <g​fxstrand> Yeah, but I'm not using any of that
17:25 fdobridge: <g​fxstrand> All the "interesting" NIR usage is in C code
17:25 fdobridge: <k​arolherbst🐧🦀> ahh, okay
17:28 fdobridge: <g​fxstrand> GS run looking good so far: Pass: 61395, Fail: 2, Crash: 1, Skip: 481527, Flake: 75, Duration: 15:23, Remaining: 1:26:37
17:28 fdobridge: <g​fxstrand> I'll report back in a few hours once it's actually done
17:28 fdobridge: <g​fxstrand> GS run looking good so far: `Pass: 61395, Fail: 2, Crash: 1, Skip: 481527, Flake: 75, Duration: 15:23, Remaining: 1:26:37` (edited)
18:21 fdobridge: <m​henning> ugh. gitlab won't let me post a `Reviewed-by` tag with my email address in it any more because it thinks it's spam.
19:36 fdobridge: <_​maide> gfxstrand: What do the "needed" transform feedback query reports represent? E.g `REPORT_SEMAPHORE_D_REPORT_STREAMING_PRIMITIVES_NEEDED` and `REPORT_SEMAPHORE_D_REPORT_STREAMING_PRIMITIVES_NEEDED_MINUS_SUCCEEDED`
19:36 fdobridge: <_​maide>
19:36 fdobridge: <_​maide> I saw a while ago that you implemented them somewhere, but I can't find the commits for them anymore. As I remember it wasn't clear what they meant from your code though.
21:48 fdobridge: <g​fxstrand> GS looks good. There's a silly XFB test failing but I suspect that's a cache flushing issue more than anything. I've got some ideas. I just need to poke about a bit.
21:48 fdobridge: <g​fxstrand> Figuring out caching for real is getting pretty high up my priority list.
22:21 HdkR: gfxstrand: I'm curious your opinion. Now that you have experience with GS on multiple different pieces of hardware. How does the NVIDIA implementation compare? :)
22:33 fdobridge: <g​eorgeouzou> The following explains what the xfb queries report :
22:33 fdobridge: <g​eorgeouzou> https://registry.khronos.org/vulkan/specs/1.3-extensions/html/vkspec.html#queries-transform-feedback