00:08 gfxstrand[d]: mhenning[d]: Fedora has packages, too. Look for instructions on the RPMFusion website.
00:09 gfxstrand[d]: x512[m]: It's fine. The command processor just gobbles up data and processes it. It's only a problem if the kernel inserts something between the command we started and the data we're trying to submit with it.
00:10 gfxstrand[d]: Also, this isn't really used much pre-Turing.
00:12 gfxstrand[d]: Turing+, rather. We use it for indirect draws pre-Turing because the MME lacks the ability to read arbitrary data and all data has to be pushed.
00:35 mhenning[d]: I started hacking at some hoppen,blackwell instruction encoding stuff here: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34334
00:35 mhenning[d]: (just using nvdisasm, not on hardware)
00:37 snowycoder[d]: pavlo_kozlenko[d]: Not yet, there are still a lot of missing features, but that is the plan in a few weeks (I hope).
00:37 snowycoder[d]: Right now it can run a lot of compute or an untextured triangle and that's it😂
00:41 gfxstrand[d]: Textures shouldn't be hard. They're just a bit of typing.
00:42 gfxstrand[d]: Then again, the most of Kepler bringup is "just a bit of typing". 😂
00:42 gfxstrand[d]: You're going great, BTW. 💜
00:44 mhenning[d]: "just a bit of typing" feels a bit reductive to me
00:44 mhenning[d]: like this: https://xkcd.com/722/
00:45 broski[m]: From Trusted and Vouched Dealers... (full message at <https://matrix.org/oftc/media/v1/media/download/AUkwdnnNqX213Oej23dRYTV9YA5x32yRnIrDPC839NmGI3mZjUe5czDt6S86Mb7T-_am4Rh6sRsgSV41csQQCN1CeWPA6G6QAG1hdHJpeC5vcmcvakNIc0lIRkV1VUhKWkdDYktJWm5mSmNY>)
00:46 redsheep[d]: If you already know what to type then the entire driver would be "just a bit of typing" lol
00:47 redsheep[d]: With some clairvoyance fixing my issues with the nouveau kmd would be just a bit of typing
00:48 redsheep[d]: Clearly I don't possess that power
00:53 redsheep[d]: Maybe I just need to read all of the display code again
01:04 redsheep[d]: Given that last time I couldn't actually find where some parts were implemented I'm worried the actual issue is in nouveau specific code or an issue with the monitors I'm working with having bad edids. But if that were the case why isn't amdgpu broken?
01:04 redsheep[d]: isn't in*
01:16 airlied[d]: oh uldc is now ldcu and is in a different spot
01:52 airlied[d]: oh I've moved to misaligned address
03:08 airlied[d]: https://gitlab.freedesktop.org/airlied/mesa/-/commits/nvk-wip-gb20x is where I'm playing around, but getting misaligned address, and lots of fail
03:27 airlied[d]: looks like r2ur moved
03:31 gfxstrand[d]: snowycoder[d]: I started typing something for you: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34336
03:32 gfxstrand[d]: mhenning[d]: It is
03:32 gfxstrand[d]: Sorry
03:32 gfxstrand[d]: I didn't mean to be dismissive
03:32 airlied[d]: hmm I wonder if qmd has changed
03:32 gfxstrand[d]: Almost certainly
03:33 gfxstrand[d]: I would be shocked if it hasn't
03:35 airlied[d]: dEQP-VK.compute.pipeline.basic.empty_shader seems like the only compute shader I that passes 🙂
03:35 orowith2os[d]: How noble of an effort would it be to write a nova equivalent for pre-Turing (Kepler specifically) :akipeek:
03:35 orowith2os[d]: I'm guessing it would be Not Too Bad, since there's already nouveau for that
03:36 airlied[d]: not sure why you would pre-gsp, fix nouveau is probably easier
03:37 orowith2os[d]: Funsies? You saw me on r4l's zulip, right?
03:37 orowith2os[d]: A while back
03:37 orowith2os[d]: First reason would be because, I want to. Next up is to see if I can clean it up at all, compared to normal nouveau, and maybe redo things in a way that make a bit more sense
03:59 airlied[d]: okay got 55 dEQP-VK.draw.* tests to pass in a row
04:06 redsheep[d]: orowith2os[d]: Having nouveau as reference surely helps but that sounds like a pretty crazy undertaking. "Just" fixing nouveau would be awesome but I assume you're talking about r4l cuz you don't want to work on C
04:06 orowith2os[d]: Correct
04:06 orowith2os[d]: I'd be more confident with myself if I were writing Rust
04:07 orowith2os[d]: I could probably pull it off with C, but I wouldn't be happy about it
04:07 orowith2os[d]: Maybe if the kernel were more friendly to some C++ features, I'd like it more
04:09 redsheep[d]: I wonder if it would be remotely possible to convert to rust in parts
04:10 orowith2os[d]: Probably
04:10 orowith2os[d]: I'd just have to expose some symbols for it to link against from C
04:11 redsheep[d]: Then just rewrite the display code 😛
04:11 orowith2os[d]: I definitely wouldn't rewrite anything facing userspace
04:11 airlied[d]: no converting to rust in part would be hard, too many structs
04:11 orowith2os[d]: orowith2os[d]: So yeah, probably more just interfacing with the hardware directly, like reclocking the GPU or whatever
04:23 gfxstrand[d]: airlied[d]: Yeah. People have tried that. It doesn't go well.
04:41 orowith2os[d]: Any links to past effort?
05:30 airlied[d]: there might be some stuff around panthor conversion to rust on dri-devel
07:42 snowycoder[d]: gfxstrand[d]: That's a lot pf typing hahaha.
07:42 snowycoder[d]: Thanks, it also makes a lot of IR more readable
07:46 marysaka[d]: For anyone using envyhooks, I pushed some changes that should make Blackwell work for dumping (cc airlied[d] mohamexiety[d] gfxstrand[d])
07:48 marysaka[d]: It should also be easier to update on newer drivers as the wrapper.h file is now autogenerated + channel types are now detected in a generic way instead of hardcoding each gens
08:30 matt_schwartz[d]: I did put the latest gb20x kernel on my Blackwell rig as well in case you need it
12:02 gfxstrand[d]: snowycoder[d]: Yeah, the fact that we weren't printing the masks anywhere was pretty rubbish.
12:03 gfxstrand[d]: Don't take the top patch, obviously, but I think the others mostly work. (I left them CTSing last night.)
14:30 snowycoder[d]: gfxstrand[d]: The first patch could work if we restrict the target to sm32 though, right?
14:34 gfxstrand[d]: That top patch should work everywhere, is just unnecessary.
14:34 gfxstrand[d]: I wrote it just to test `sust.b` to make sure I encoded it correctly.
14:37 gfxstrand[d]: gfxstrand[d]: Also, there's some issue with the crazy 10 and 11-bit float format which I haven't looked into. Something with rounding or denorm flushing, I suspect.
14:50 gfxstrand[d]: Yeah, looks like all the issues are 10 and 11-bit stores
15:15 gfxstrand[d]: But yeah, if you drop the `HACK:` patch and implement `sust.p` and `suld.b`, that should get you image support on Kepler.
15:49 babblebones[d]: airlied[d]: Any of the GSP work in place to properly flag priority on the workloads?
15:49 babblebones[d]: If not hopefully soon <a:BongoTap:732755194928168970>
15:52 gfxstrand[d]: No, we don't support priorities yet.
16:08 snowycoder[d]: gfxstrand[d]: Thanks, I'll try after work.
16:08 snowycoder[d]: p.s. do you have any CTS paths to test basic images and basic textures? All I can find are some complex uses that also test filtering and mipmaps
16:08 gfxstrand[d]: For basic images, search for `with_format`.
16:09 gfxstrand[d]: For basic textures, maybe search for "texelfetch"?
16:10 snowycoder[d]: gfxstrand[d]: ok, perfect!
16:12 gfxstrand[d]: Also, IDK if you've found this but if you run `deqp-vk --deqp-run-mode=txt-caselist`, it'll dump all the test names to a file.
16:13 snowycoder[d]: Oh nice, I've been grepping the caselist folders until now
16:18 mhenning[d]: You could also try the sascha willems "texture" demo
16:22 snowycoder[d]: Thanks, I'm also building a little testbed project to test things surgically (mostly to debug IPAs)
19:40 mohamexiety[d]: https://docs.nvidia.com/cuda/pdf/CUDA_Binary_Utilities.pdf table 8 here has the blackwell instruction set
19:41 mohamexiety[d]: they grouped both sm_100 and 120 though :/
19:41 mohamexiety[d]: (it's just the names, no encoding or any fancy info but thought it may be helpful)
19:42 mohamexiety[d]: trying to play with Kuter's tool but seeing some weird stuff with sm_120. for starters, the SASS file I dumped through cuobjdump was _2.4GB_. if I dump sm_89 instead it's 383MB
19:45 airlied[d]: it's bigger due to the delay stuff I think
19:52 airlied[d]: unfortunately doesn't have any tex instructions in it
19:52 mohamexiety[d]: the pdf? yeah it feels like it's mostly sm_100
19:57 gfxstrand[d]: mohamexiety[d]: Yeah, I had limited success with it on SM120. I haven't run SM100 yet.
19:58 mohamexiety[d]: let me try dumping sm_100 actually
19:58 gfxstrand[d]: And I don't know how to produce the HTML from the cache file.
20:00 mohamexiety[d]: yeah I didnt get that far
20:00 mohamexiety[d]: don't think he actually published a way to produce the HTML, but not sure. there is a table creating .py file :thonk:
20:05 mohamexiety[d]: https://cdn.discordapp.com/attachments/1034184951790305330/1357083434081714508/image.png?ex=67eee9f0&is=67ed9870&hm=60e682daa6a97d381254307c547262f93fb007c10326792a6b1823189b9ca958&
20:05 mohamexiety[d]: so sm_100 is still pretty fat, but quite a lot less than sm_120
20:08 mohamexiety[d]: gfxstrand[d]: `nv-isa-solver-scan` seems to run fine with sm_100 SASS, which is kinda interesting
20:08 mohamexiety[d]: with sm_120 it ran for a few instructions but then there was _a lot_ of "couldn't parse [...]" spam
20:09 gfxstrand[d]: Yeah, that's what I saw, too.
20:09 gfxstrand[d]: I got plenty out of cublas but then it failed to parse most of it
20:09 mohamexiety[d]: yeah
20:09 mohamexiety[d]: sm_100 seems to run fine though so I wonder if we can use that temporarily :thonk:
20:10 gfxstrand[d]: But I don't know how important that step is. I suspect we just won't get everything but we'll still get some stuff.
20:10 mohamexiety[d]: but also I am really curious what even is so different with sm_120 that it's almost twice the size, and it can't be parsed
20:11 gfxstrand[d]: mohamexiety[d]: I don't know. I suspect there's differences but I don't know how significant. They skipped two whole CUDA versions, though, so I wouldn't bet on it being super close.
20:11 gfxstrand[d]: Okay, back to ✈️ mode.
20:11 mohamexiety[d]: yeah that's the other weird bit
20:11 mohamexiety[d]: gfxstrand[d]: oop, sorry. good luck!
20:20 redsheep[d]: mohamexiety[d]: This would be a question for kuter7639
20:20 mohamexiety[d]: yeah been meaning to ping but wanted to gather more info first
20:30 mohamexiety[d]: https://cdn.discordapp.com/attachments/1034184951790305330/1357089924725346364/gb200_cache.txt?ex=67eeeffb&is=67ed9e7b&hm=1887957312b30e7122de0911ce78ffd40c6b2b9b4dd32c20e6591d14e66c9bd8&
20:30 mohamexiety[d]: sm_100 cache if anyone is interested
20:31 mohamexiety[d]: I guess I could run sm_120 even with the errors
20:33 mohamexiety[d]: the main discreprenacy that sticks out can be seen here:
20:33 mohamexiety[d]: ~/dev/nv-isa-solver/nv_isa_solver$ nv-isa-solver-scan --arch SM120 --cache_file 5090_cache.txt libcublasLt_sm120.sass
20:33 mohamexiety[d]: Cache could not be loaded
20:33 mohamexiety[d]: Distilling LDC R1, c[0x0][0x37c] &wr=0x0 ?trans8
20:33 mohamexiety[d]: Couldn't parse b'\x0cx\x00\x04\x01\x00\x00\x00pb\xf0\x03\x00\xda/\x00'
20:33 mohamexiety[d]: Couldn't parse b'M\x89\x00\x00\x00\x00\x00\x00\x00\x00\x80\x03\x00\xea\x1f\x00'
20:33 mohamexiety[d]: Couldn't parse b'\x0cx\x00\x04\x10\x00\x00\x00p`\xf2\x03\x00\xe2\x0f\x04'
20:33 mohamexiety[d]: Couldn't parse b'\x12x\x06\x04\x0f\x00\x00\x00\xff\xc0\x8e\x07\x00\xe2\x0f\x00'
20:33 mohamexiety[d]: Distilling LDCU.64 UR6, c[0x0][0x358] &wr=0x0 ?trans1
20:33 mohamexiety[d]: `sm_100` did not have the `&wr=0x0 ?trans8` stuff, and also all the things it couldnt parse. so I guess the SASS for 120 has more info, but not sure what type of info it is
20:35 mohamexiety[d]: `wr` could be write/read and `trans` could be transcendental at a guess, but these are pure uneducated guesses. (and also don't make sense? why would LDC have mentions of transcendental stuff)
20:37 mohamexiety[d]: https://cdn.discordapp.com/attachments/1034184951790305330/1357091694348079335/5090_cache.txt?ex=67eef1a1&is=67eda021&hm=e4e73dd76f5488129a033097c7b5d60661b981447cd156647f4086e69b4c4ad0&
20:37 mohamexiety[d]: for sm_120 it ends up only getting info for 10 instructions. the rest it completely fails on
20:38 redsheep[d]: mohamexiety[d]: Oh, sorry
20:38 mohamexiety[d]: it's finee
20:58 mohamexiety[d]: hm I think I may have figured what's up. can't send things here without massively spamming the irc side of the bridge but I suspect the tool can't cope with the extra stuff, which is why it's failing
20:58 mohamexiety[d]: if we look at a single instruction, this is what it looks like in the sm_120 sass:
20:58 mohamexiety[d]: /*0020*/ ISETP.GE.AND P0, PT, R4, 0x1, PT &req={1} ?WAIT13_END_GROUP; /* 0x000000010400780c */
20:59 mohamexiety[d]: this is how it looks like in the sm_100 sass:
20:59 mohamexiety[d]: /*0020*/ ISETP.GE.AND P0, PT, R6, 0x1, PT ; /* 0x000000010600780c */
21:00 airlied[d]: that's what I meant by including the delay stuff
21:00 airlied[d]: sm120 seems to actually decode delay stuff, where's older sm doesn't decode it
21:00 airlied[d]: it's still there, just not decoded
21:00 mohamexiety[d]: ah
21:01 mohamexiety[d]: yeah from my understanding, volta+ do have this stuff in the instruction. so I guess for 120+ they decided to decode it as well
21:01 karolherbst[d]: finally the "yield" isn't a flag information is public knowledge
21:01 mohamexiety[d]: so theoretically if we modify the parser to understand the extra stuff, things should work :thonk:
21:04 snowycoder[d]: mohamexiety[d]: I think that's what `nv_isa_solver/instruction_solver.py` does, I got it to generate some html files
21:05 mhenning[d]: There are also some nvdisasm flags that remove some extra information eg. "--print-raw"
21:06 mohamexiety[d]: snowycoder[d]: ooh nice then. so just need to fix up the parser and then we should be able to be good to go
21:06 mohamexiety[d]: sorry for being a bit slow with this -- my only experience with compiler stuff is reading the Citadel "Dissecting the <arch>" papers :nervous:
21:07 mohamexiety[d]: mhenning[d]: alternatively could try this if it'd make the format match the "old" decoded format
21:08 snowycoder[d]: I know this only because I tried to execute for kepler (and failed, kepler encoding is strange).
21:08 snowycoder[d]: p.s. if it can help I added some utils to nv-shader-tools that could help, I used a "bit-flip" mode that tries to flip each bit in a range and checks what changes (instead of trying all possible values)
21:09 snowycoder[d]: It just needs a bit of refactoring
22:27 redsheep[d]: mohamexiety[d]: If you save to a file and attach it to your message then discord users can see the text without spam
22:59 gfxstrand[d]: snowycoder[d]: That would be a nice addition to nvfuzz.
23:00 gfxstrand[d]: Maybe as a flag that puts it into bit-flip mode but still takes a range.
23:01 gfxstrand[d]: We should also fix it so it takes --sm instead of the positional arg.
23:55 mhenning[d]: I think I have blackwell/hopper atomic encodings fixed on my branch
23:56 airlied[d]: oh nice I had started on fixing atom.g yesterday
23:56 airlied[d]: but this morning has all been meetinging out
23:58 airlied[d]: I've got some kernel instruction enconding errors, but nvdisasm seems happy with the shaders, so have to dig around a bit more
23:59 mhenning[d]: Ah, I'm just testing with nvdisasm so far