09:00adavy: q4a: I can help you. What did you find about call 203368
09:04q4a: It's 1st call when grass rendering. It uses VS shader https://github.com/storm-devs/storm-engine/blob/a7c1cabaa2e4f3497b48807a3fcd1fc280157b73/src/techniques/effects/grass_shader.h
09:04q4a: and `GrassDark technique`: https://github.com/storm-devs/storm-engine/blob/a7c1cabaa2e4f3497b48807a3fcd1fc280157b73/src/techniques/effects/grass.fx
09:07adavy: could you remind me your rendering system ?
09:07adavy: Only obvious thing I see is alpha testing
09:07q4a: I tried to modify shader (removing some asm commands) to check - if one or few did not work. But I still did not get same result - picture rendered with Nine looks like it did not mixes trees color with sky texture.
09:08q4a: rendering system? GPU or something else?
09:08adavy: did you try my branch where I have a fallback for alpha testing support ? You could force that fallback to kick in
09:08adavy: yes
09:09q4a: I tested on AMD A6-5200 (with integrated Radeon HD 8400)
09:09q4a: and Intel i5-3450
09:10q4a: with integrated HD 2500
09:10q4a: both has same problem
09:10adavy: when you remove mov oD0.w, v3.z I think it's supposed to be replaced by zero
09:11adavy: ah, then it might be how alpha is treated in the ps shader
09:11adavy: it's a ff ps, right ?
09:12q4a: Yes, I was thinking so. And when I tested with DXVK I did not see trees, but with Nine I saw half screen with trees, half - without trees.
09:12adavy: Do you have the code of the ff ps ?
09:12q4a: I'm not sure what us "ff ps"?
09:13adavy: For that you have to use NINE_DEBUG=all NINE_FF_DUMP=1 (I think)
09:13adavy: when you replay with d3dretrace, you look at the last "nine_ff_get_ps PS ff key hash: 240900a4"
09:13adavy: then you search for that key hash in the log
09:13adavy: you should find the dump of when that shader is generated for the first time
09:14q4a: I tested with NINE_DEBUG=all but without NINE_FF_DUMP=1: https://gitlab.freedesktop.org/mesa/mesa/uploads/f523b5faf73f8a894c61ef97a510d6c2/nine-storm-flog.txt
09:14adavy: ff is fixed function. Basically for d3d9 when you don't set a vs or a ps shader, it generates one based on states you set
09:15q4a: ah.. yes, there us no PS shader for trees.
09:15adavy: yes the bug could be a wrong interpretation of a specific case in the states
09:16adavy: Basically the original implementation has a specific handling of invalid/weird set of states that is not described
09:17adavy: games can hit them by mistake, and since the result was what they wanted they didn't fix. But we have to replicate without having it documented
09:17adavy: it might be one of such case
09:23adavy: so basically the way to debug this is to find the generated ps, check if it does something that should be giving the expected or not. If it does something that obviously cannot give the expected state, look at the generation code and the states set to find how that code is generated, and what change would generate a code that would make more sense
09:24q4a: I created log with `export NINE_DEBUG=all && export NINE_FF_DUMP=1 && export csmt_force=0`
09:24q4a: https://gitlab.freedesktop.org/mesa/mesa/uploads/6c22adf1d94e392119496a2ef66fd10c/nine-storm-2log.txt.7z
09:25adavy: last ps key is 240100a5
09:28adavy: the alpha output seems to be color.w times s0[xy].w
09:28adavy: that is probably not that ps, since when you had set color.w to 0, it had half screen ok
09:30q4a: BTW `branch with fallback for alpha testing support` is https://gitlab.freedesktop.org/axeldavy/mesa/-/commits/unsupported_features_emulation/ ?
09:30q4a: should I check it or continue to check NINE logs with current mesa?
09:30adavy: yes it must be that ps given where it is in the log
09:30adavy: yes it is
09:32adavy: if you look at the shader, instructions 5 to 9 it's fog generation (blending with the fog color depending on the depth)
09:32adavy: instructions 0 to 2 are useless
09:33adavy: 3 samples the texture, and 4 multiplies the texture with the color coming from vs (oD0)
09:34q4a: if you look at the shader - you mean ff ps?
09:35adavy: yes
09:35adavy: it's line 267626
09:35q4a: ye, I see it)
09:36adavy: line nine:0x7f967e14d740:device9:SetRenderState: This=0x7f6a7de0 State=22(D3DRS_CULLMODE) Value=00000001
09:36q4a: 2: MOV TEMP[2], IMM[0].xxxx and above - useless
09:36adavy: according to https://learn.microsoft.com/en-us/windows/win32/direct3d9/d3dcull, it should disable culling
09:36adavy: the culling not being disabled could be an explanation for your triangle
09:40adavy: alpha func is D3DCMP_GREATER and alpha ref is 80 (/255)
09:45adavy: alpha blending is source alpha for src, and the opposite for dst
09:45adavy: hum maybe for color default oD0.w is 1
09:46adavy: that explains why you saw something when you removed the line
09:46adavy: basically when you have alpha below 80/255, the object is not rendered
09:46adavy: when it is above, it is rendered, but blended according to the alpha value (closer to 1 = more opaque)
09:47q4a: Or may be it's UB? I tested on Windows and when I remove that line I got flickering trees
09:47adavy: UB ?
09:47q4a: Undefined behavior
09:48adavy: maybe
09:48adavy: also for vs 1.1 I think the output is clamped to [0, 1]
09:52adavy: so what you can do to debug is to enforce things to be disabled
09:52adavy: is it easier for you to recompile the game or nine ?
09:53q4a: recompile game
09:53q4a: or it's does not matter
09:53adavy: you can play here: https://github.com/storm-devs/storm-engine/blob/a7c1cabaa2e4f3497b48807a3fcd1fc280157b73/src/techniques/effects/grass.fx Try disable alphatesting
09:54q4a: I think that experimenting with nine will be good too)
09:54adavy: you can also disable alpha blending if that does nothing
09:55adavy: to me at least, it looks like the vs looks fine, the ps too
09:56adavy: it could be in the translation to asm (intermediate tgsi, then nir, then llvm ir, then asm)
09:56adavy: or it could be nine somehow not updating gallium states
09:57adavy: (culling, alpha teting)
09:57adavy: since you say on intel gpu the issue is the same as on amd, it would seem this is not on the gallium driver side
09:59adavy: the triangle thing you got at some point would seem to be to states not being updated, at least for culling
09:59adavy: but that is suspicious as the chance of such a bug is very low
10:09q4a: https://user-images.githubusercontent.com/1950719/229279885-74d67d03-3169-466b-a56d-270ebd8bd2c6.jpg
10:09q4a: this is result when I set AlphaTestEnable = false; AlphaBlendEnable = false;
10:10q4a: So I lost the transparency of the tree textures
10:18adavy: well at least this is reassuring everything is drawn
10:19adavy: now with alphablendenable = true, you should see something close to what you expect, but slightly less sharp
10:23q4a: `AlphaTestEnable = false; AlphaBlendEnable = true;`:
10:23q4a: https://user-images.githubusercontent.com/1950719/229280606-26f27d00-adc4-42a1-906e-62199c10cf47.jpg
10:24adavy: that doesn't seem right ?
10:24q4a: yes( goal is still looks better:
10:24q4a: https://gitlab.freedesktop.org/mesa/mesa/uploads/f2ec1be47c03d3e3235a138e906c98a5/tree-dxvk-ok.jpg
10:26adavy: there are two issues I see
10:26adavy: first the fog is not applied
10:28adavy: and second the blending doesn't seem to be working properly: it's as if it was blending with the original destination buffer, not with the destination buffer with the things you draw at the moment. What I mean is the trees have their drawing square clearly visible and the parts of their square without the tree show the background color, not the trees behind
10:28q4a: How did you understand, that the culling not being disabled?
10:28adavy: no the culling is fine
10:29adavy: well it chould have been culling with what you described earlier about the triangle, because culling removes some triangles, but that's not it here
10:30q4a: what issue is easier to check/debug?
10:31adavy: could you try alpha test true, and alpha blend false ?
10:32q4a: yes, 2 min
10:35q4a: https://user-images.githubusercontent.com/1950719/229281276-cfe7a9d1-9fc7-4f9e-942d-2e981acb7fd4.jpg
10:36adavy: that doesn't seem too bad
10:36adavy: something weird with alpha blending then
10:36adavy: I'll take a look at what I can see
10:37adavy: (also an issue with fog)
10:42q4a: I already configured IDE and build mesa, so I can step by step go deep in mesa code, but I need to understand where I should look?
10:42q4a: https://user-images.githubusercontent.com/1950719/229282611-8ab94bc8-673b-4041-a6cc-e76dfd398188.jpg
10:43q4a: for example, I see that group in nine_update_state has NINE_STATE_BLEND flag, but has no NINE_STATE_BLEND_COLOR
10:47adavy: q4a: well you could add debug info to check that the blending state is correct
10:58q4a: also, I checked trace and found that call 202098 has `fog/sky`:
10:58q4a: https://gitlab.freedesktop.org/mesa/mesa/uploads/3d652891843552c28ced78a3dfe401e7/fog-sky.png
10:59adavy: yes the dact the trees don't have fog is very mysterious
10:59adavy: *fact
11:00adavy: because we clearly see it in the shader
11:00adavy: could be either the shader is not used somehow, or that the constants are incorrect
11:02adavy: in nine_ff_load_ps_params you can remove the if(!context...) return to enforce the constants to be always updated
11:02q4a: ok, i'll do it
11:07adavy: I don't see any flaw to the logic of ps commit that would cause ps ff debug to be printed, but the ps ff not being used
11:12adavy: this confirms the default for OD0 : https://learn.microsoft.com/en-us/windows/win32/direct3dhlsl/dx9-graphics-reference-asm-vs-registers-vs-1-1
11:12adavy: it says if partially written, default is 0 0 0 1
11:12adavy: also I checked NINE_STATE_BLEND vs NINE_STATE_BLEND_COLOR and didn't see any mistake
11:13adavy: I didn't see anything suspicious either about blending possibly not being updated
11:14adavy: what could be useful is to check the asm generated
11:14adavy: for the ps
11:14adavy: I think for amd this is RADEON_DEBUG=ps
11:15adavy: maybe you need RADEON_DEBUG=mono,ps
11:15adavy: that will include the alpha testing
11:15q4a: in logline 267634 is not asm PS shader?
11:16adavy: also NINE_SHADER=nir should dump the nir
11:17adavy: no that is TGSI
11:18adavy: TGSI -> NIR -> llvm ir -> asm
11:19adavy: also if you use mono, the asm will contain everything, including alpha testing
11:19adavy: if you don't use mono, it compiles the alpha testing separately and concatenate on use
11:39q4a: strange, but looks like I don't see much difference with `export RADEON_DEBUG=mono,ps && export NINE_SHADER=nir`.
11:39q4a: https://gitlab.freedesktop.org/mesa/mesa/uploads/d34b6042fa6da172c45f837b426c933c/nine-storm-3log.txt.tar.gz
11:40q4a: ah.. RADEON_DEBUG - is for old one, I'm using redeonsi
11:41q4a: s/RADEON_DEBUG/RADEON\_DEBUG/, s/redeonsi/radeonsi/
11:41adavy: right maybe the name changed
11:41adavy: RADEONSI_DEBUG ?
11:43adavy: at some point in time R600_DEBUG worked too
11:44adavy: that's weird NINE_SHADER=nir didn't do anything, maybe I got it wrong
11:46adavy: ah that's NINE_SHADER=dump_nir
11:47adavy: ok now that's AMD_DEBUG
11:47adavy: but R600_DEBUG should work too
11:57q4a: https://gitlab.freedesktop.org/mesa/mesa/uploads/5f5db82a33a1cfc66da041179d77b36e/nine-storm-4log.txt.tar.gz - NINE_SHADER=dump_nir worked, but looks like export AMD_DEBUG=mono,ps and export R600_DEBUG=mono,ps didn't do anything
11:57q4a: R600_DEBUG
12:07adavy: I don't see anything wrong with the nir, but I'm not much familiar with nir, so I could miss something
12:11adavy: Ok I have an idea why we see this: https://user-images.githubusercontent.com/1950719/229280606-26f27d00-adc4-42a1-906e-62199c10cf47.jpg
12:11adavy: What we see seems logical
12:12adavy: to explain this, trees would be rendered from front to back, and due to depth testing we wouldn't render on top of a previously rendered triangle, even if it has alpha = 0
12:13adavy: this makes sense. And alpha test should prevent the issue when enabled because it would discard the associated samples, which thus wouldn't write on the depth buffer
12:16adavy: q4a: could it be that actually the output is correct except for fog ?
12:16adavy: when looking at dxvk, I don't see well if there is actually trees everywhere because of the fog
12:16adavy: could it be that if we had fog applied, nine output would be the same as dxvk ?
12:18q4a: yes, fog may be the only problem
12:19adavy: have you tried the trick I advised to force the constants to be updated all the time ?
12:21q4a: removing `if (!(context->changed.group & NINE_STATE_FF_PS_CONSTS))` from `nine_ff_load_ps_params` did not help
12:21q4a: * removing `if (!(context->changed.group & NINE_STATE_FF_PS_CONSTS)) return;` from `nine_ff_load_ps_params` did not help
12:22adavy: ok, well either there's an issue with the constants or the formula applied
12:35adavy: I don't find any mistake in the formula applied
12:39adavy: in nine_ff_load_ps_params you can play with dst[22].z, check if it is indeed read. For example you set it to bad values like 1. or 0., does it change the output
12:43q4a: Ok, i will play with it
13:06q4a: https://user-images.githubusercontent.com/1950719/229290616-ccc8fd38-60db-4b78-a918-4512886ea485.jpg - dst[22].z=1
13:06q4a: https://user-images.githubusercontent.com/1950719/229290746-75670a8a-35ad-4b01-9ac7-508d99afccff.jpg - dst[22].z=0
13:08q4a: dst[22].z=0 looks ok, but dst[22].z=1 is strange, right?
13:08adavy: yes
13:09adavy: it looks as if there wasn't enough fog
13:09adavy: according to apitrace: "D3DRS_FOGDENSITY": 0.003,
13:09adavy: so the value passed is 0.003
13:09adavy: but even with that value, the fog impact is too low it seems
13:10adavy: also it's surprising that there is almost no difference between the back and the front
13:10adavy: of the forrest
13:11adavy: q4a: just to be sure, have you checked in apitrace at the draw call you had noticed that the fog is applied ?
13:11adavy: it could be that the foggy athmosphere we see for z=1 be due to a later draw call
13:13q4a: I can see fog, when I traced from qapitrace: https://gitlab.freedesktop.org/mesa/mesa/uploads/63094e4004052c819e74ddcd555dd769/BadDrawFrame.png
13:13q4a: but when I run `wine64 /home/q/soft/apitrace-64/bin/d3dretrace.exe ./storm.trace -D 203368 &> nine-storm-4log.txt` I don't see last frame with ship
13:15q4a: both has same trace with call 203368
13:19adavy: https://paste.centos.org/view/7aaaa3d8
13:19adavy: in nine_ff.c
13:19adavy: something to try
13:20adavy: and another thing to try is in the new code replace the first ureg_RCP by ureg_MOV
13:21adavy: basically dxvk does a similar thing
13:21adavy: most of the time w is 1 in wine tests, so it could have gone unnoticed
13:22adavy: also most games don't use that fog formulat
13:22adavy: and those who do might have w = 1
13:23adavy: if w is not 1 here that could explain
13:25q4a: building. Thank you so much for helping me!
13:26adavy: basically when you write position, you write 4 components: xyzw
13:26adavy: when w is not 0 or 1, it divised all the other components
13:27adavy: it is unclear in the spec which z is used: original z, or z/w. Also reading the code it's not clear either which one is obtained by the shader
13:27adavy: so basically the code to try does shader's z times shaders'1/w, and alternatively times shaders'w
13:28adavy: *it divides
13:28q4a: I don't beleave, but looks like this jelped: https://paste.centos.org/view/7aaaa3d8
13:28q4a: s/beleave/believe/
13:30q4a: here is result: v
13:30q4a: s/v/https://user-images.githubusercontent.com/1950719/229292023-6aebb5b6-48a6-420a-9b0e-6e2dd7f96c82.jpg/
13:31adavy: looks right
13:31q4a: I need to uncomment line `if (!(context->changed.group & NINE_STATE_FF_PS_CONSTS)) return;`
13:32q4a: but that looks so good))
13:37q4a: ok. This is only diff that I have in Mesa source: https://pastebin.ubuntu.com/p/BZ3k4x8NXh/
13:38q4a: adavy: Is that change can break other games? Should I create MR with this change?
13:39q4a: And I will need to test/understand, how it helped)
13:39adavy: well that's the issue
13:39adavy: It seems this operation is required in your case
13:40adavy: and you are basically using programmable vs with ff ps
13:40adavy: but what about all the other combinations
13:41adavy: ok I guess nine_shader.c answers it
13:41adavy: "Depth used for fog is perspective interpolated"
13:41adavy: it already has the operation
13:42adavy: somehow this was not ported to ff ps
13:46q4a: https://gitlab.freedesktop.org/mesa/mesa/-/commit/a9bf82ecf4c284989c40e4e6c1f6962bc9bd58d2
13:47q4a: depth = nine_get_position_input(tx);
13:47q4a: depth = ureg_scalar(depth, TGSI_SWIZZLE_Z);
13:47q4a: That was changed a long time ago
13:48adavy: a9bf82ecf4c284989c40e4e6c1f6962bc9bd58d2
13:48adavy: yeah
13:49adavy: so you can site this commit in your submission
13:49q4a: And that was fix for fog too) https://github.com/iXit/Mesa-3D/issues/315
13:50adavy: my bad for forgetting to port it to ff ps
13:51q4a: Don't worry. You fixed that bug and fixed this one too!
13:51q4a: I'm so happy - so I need to go afk for a while: eat and drink something)
13:52adavy: thanks for your help
15:15q4a: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22249
15:16q4a: fix depth fog for fixed function ps, closes: #8341
16:38DavidHeidelberg[m]: Nice, does it fixes any from nine tests?
16:53q4a: Sorry, I forgot about it. I'll test now.
16:59q4a: 9 new fails. No fixes( https://pastebin.ubuntu.com/p/tyrBgMPnmx/
17:06q4a: btw, I compared release mesa 23.0.1 with clean main commit d698bf052349cdaf0580efde335d43069897304b and got another 6 fail: https://pastebin.ubuntu.com/p/dRchKSXjhP/
17:06q4a: David Heidelberg: does that 6 is flakes?
17:10q4a: Is it hard to build Nine tests on Windows and check, if they are ok? Or it was already done?
17:17adavy: looks like a regression
17:18adavy: somehow in the case of the wine tests the previous code worked better
17:18adavy: now you have to figure out why :-)
17:18q4a: ok)
17:20adavy: look at the failing tests if they have a specific config
17:20q4a: I marked MR 22249 as draft
17:21adavy: you'll have to find a patch that works for both your game and the tests