02:58airlied: imirkin: nve7 in mbp
02:58airlied: imirkin: if it happens again I'll probably try and restart gnoem-shell instead of reboot
04:06mupuf: imirkin: less instruction is of course better but GPUs are designed to mask the memory latency, so if you are memory bound, any change in the number of instructions won't change much
04:06mupuf: as in nothing at all
04:06mupuf: unless at some points the memory usage drops under 100% :p
04:17RSpliet: mupuf: but if less instructions lead to a lower register use, we can schedule more warps a a given point in time
04:17RSpliet: hence having more threads to hide the latency with
04:18mupuf: that's the very interesting thing about nvidia's architecture :)
04:18RSpliet: I take it from your statement that Intel doesn't have that flexibility? :-P
04:18mupuf: nor AMD :p
04:18RSpliet: nor ARM I guess
04:18mupuf: there is an equivalent concept
04:19mupuf: it is the SIMD8/16/32
04:19mupuf: you basically need to compile your program multiple times to with 8, 16 or 32 threads
04:19mupuf: but you will always share the same number of regs
04:20mupuf: so, if you don't have enough regs, you may not be able to use the SIMD32 or 16
04:20mupuf: there is no global register file
07:25karolherbst: imirkin: did you pushed the patches you told me about yesterday?
08:05karolherbst: mhhh in furmark I get "Mesa: User error: GL_INVALID_OPERATION in glUniform1("Layer"@7 is float, not int)"
08:31karolherbst: imirkin: I think I might found a possible optimization in this shader: code
08:31karolherbst: ohh it is the tgsi
08:31karolherbst: this line: https://gist.github.com/karolherbst/b7a3adb9a42bf688afa3#file-gistfile1-txt-L26
08:33karolherbst: shouldn't the compile be able to eliminate those 0 somehow?
08:33karolherbst: also this: https://gist.github.com/karolherbst/b7a3adb9a42bf688afa3#file-gistfile1-txt-L65
08:34karolherbst: I thought &r63 is always 0
08:34karolherbst: isn't a add ftz f32 $r12 $r12 $r63 pointless then somehow?
08:42imirkin: karolherbst: my changes should def not have affected that glUniform thing
08:42imirkin: mmmm... add y, x, 0 should get optimized into mov y, x
08:43karolherbst: imirkin: in that case it is add x, x, 0
08:43imirkin: karolherbst: unfortunately you cut out the most interesting part
08:43imirkin: which is all the lines that come before what you pasted, starting with the tgsi
08:43karolherbst: thats a lot, but okay :D
08:44imirkin: the nouveau optimizations don't try to achieve a fixed point, so it's conceivable that something happens after ConstantFolding which makes such an opt possible
08:44karolherbst: https://gist.github.com/karolherbst/b7a3adb9a42bf688afa3 done
08:44imirkin: although... unlikely
08:45karolherbst: imirkin: I just the post ra thing is a lot easier to find optimization possibilities by eye, that's all
08:45karolherbst: especially in a ca. 4k instruction binary
08:45imirkin: 61: add ftz f32 %r7722 %r7721 0.000000 (0)
08:45imirkin: that's not easier to find?
08:46karolherbst: $r63 or 0.0000000, does it matter? :D
08:46imirkin: anyhooo.... no clue why that'd happen
08:46imirkin: normally we should have taken care of that
08:46karolherbst: yeah, I thought so too
08:46karolherbst: this seems way too trivial
08:46imirkin: i wonder if it's a left-over of some other opt that doesn't get cleaned up
09:01karolherbst: imirkin: any idea how I can investigate this? in which pass does this has to be optimized away?
09:03Tom^: imirkin: btw did you notice that i had same minecraft layer on 11.0.6
09:04imirkin: Tom^: nope... if you could make a modestly sized apitrace (preferably with a low resolution), that'd be super
09:04imirkin: karolherbst: ConstantFolding::opnd, look for case OP_ADD
09:05imirkin: Tom^: basically my GPU has like 1/100th the power of yours, and it's convenient if i can play through the trace quickly ;)
09:05Tom^: you have to learn to compensate opensource slowness with money.
09:07imirkin: i take a different approach - if i'm spending time developing drivers, for free, i'm not going to pay $$ for hardware for that privilege
09:07Tom^: elementaldemo takes parameters for resolution?
09:07imirkin: er wait
09:08imirkin: if it's just elemental i can run it myself
09:08imirkin: i had meant for minecraft
09:08imirkin: but like i said, on my GF108, elemental is fine. so it's either a kepler+ issue or a kepler2 issue. i have a kepler2 at work, so i can check it out on monday.
09:09Tom^: ok so no apitrace then?
09:09karolherbst: imirkin: if (i->usesFlags()) break; could that trigger that?
09:09imirkin: karolherbst: sure, but it doesn't use flags
09:09imirkin: karolherbst: that's for like when you want to addc
09:09imirkin: (i.e. carry)
09:11imirkin: karolherbst: probably the leftovers of 42: MAD TEMP.xyz, TEMP.yzxx, IMM.xxyy, -TEMP.xyzz
09:11imirkin: IMM.x = 0
09:11imirkin: so that becomes ADD 0, -TEMP.xyzz
09:12karolherbst: so this pass has to be run again later on?
09:12karolherbst: or just bad ordering
09:12imirkin: and someone forgets to fix it up
09:12imirkin: well, the pass should keep running
09:12imirkin: i thought it'd fix it up, but maybe not? or there's something subtle?
09:12imirkin: or i'm just plain wrong
09:13karolherbst: I could just run the pass at the end again
09:13imirkin: anyways, i'll let you play with it
09:13karolherbst: where is the invocations of all those passes?
09:14karolherbst: ohhh found it
09:14karolherbst: yeah lol
09:14karolherbst: running the pass again cuts one instruction
09:15imirkin: ok, so that means that the pass isn't working the way i thought it did
09:15imirkin: or something funky's going on
09:15imirkin: the solution is NOT to run the same pass 100x
09:15karolherbst: this one is gone now
09:15karolherbst: or inside a loop until nothing changes :p
09:15imirkin: right. so that's called running to a fixed point
09:15imirkin: which is actually pretty common
09:15karolherbst: thing is
09:15imirkin: however these passes have been carefully structured so that a single run through gets you 99.9% of the benefit
09:16karolherbst: now I get a segfault :(
09:16karolherbst: at ../../../../../src/gallium/drivers/nouveau/codegen/nv50_ir_peephole.cpp:818
09:16imirkin: although i'm debating whether i should create an algebraic pass and run that + constant folding to a fixed point... dunno
09:18karolherbst: insn wasn NULL in src/gallium/drivers/nouveau/codegen/nv50_ir_peephole.cpp:818 :/
09:20karolherbst: imirkin: okay, it seems to be messed up inside ConstantFolding
09:20karolherbst: running it just twice witht anything else in between also eliminates this instruction
09:23karolherbst: imirkin: ever messured the effect of all passes together?
09:23karolherbst: I just disabled all, just to compare performance
09:23imirkin: you can use NV50_PROG_OPTIMIZE=0/1/2 to disable some of them
09:23karolherbst: in furmark: 7700 => 3900 instructions and 30 => 50 fps
09:23imirkin: uhhh... hopefully that's disabled -> enabled
09:24imirkin: not vice-versa :)
09:24karolherbst: no it is fine, it is good to know that the effect is _that_ big though
09:24imirkin: NV50_PROG_OPTIMIZE=1 should give you a lot of the simpler opt
09:24imirkin: which gets you a ton of the benefit
09:25imirkin: basically the compiler is written with those passes in mind... so the earlier stages generate pretty crap code
09:25imirkin: because they know the later opts will clean it up
09:29karolherbst: imirkin: that's also strange: https://gist.github.com/karolherbst/b7a3adb9a42bf688afa3#file-gistfile1-txt-L8373-L8374
09:29karolherbst: r11= -r10; r12 = r11 * r11
09:31karolherbst: the negation doesn't make much sense, does it?
09:31imirkin: yeah, but i don't think we handle that case too well
09:31karolherbst: later there is r11 = r11 * r12
09:31imirkin: let em check
09:31karolherbst: but the latter one can be r11 = -r10 * 12
09:31karolherbst: and the neg just removed
09:32imirkin: ModifierFolding should propagate that one
09:32karolherbst: *r11 = -r10 * r12
09:32imirkin: unless it doesn't for some reason
09:32imirkin: OP_MUL can take neg on either arg
09:32karolherbst: I get the feeling this shader is a gold mine :D
09:32imirkin: and so ModifierFolding should have propagated that into r12 = -r10 * -r10
09:32imirkin: every shader is a goldmine
09:32karolherbst: yeah but the small ones are a bit boring
09:33imirkin: there's a lot of stuff that codegen handles well
09:33imirkin: and there's a lot more that it handles properly in theory, but there's some dumb bug preventing it
09:33imirkin: and then there's a WHOLE lot more that it just doesn't handle at all
09:33imirkin: the things you're pointing out are things it should handle but apparently doesn't for one stupid reason or another
09:34karolherbst: imirkin: maybe if ConstantFolding::visit changed something, just rerun on the "current" instruction, which might have changed
09:34imirkin: i thought it did
09:34imirkin: does it not?
09:34karolherbst: next = i->next; is the first thing in the loop
09:34karolherbst: and if() else if() chains
09:34imirkin: ConstantFolding::foldAll(Program *prog)
09:34imirkin: ah i see
09:34imirkin: so it runs itself up to 2x
09:35imirkin: or make the OP_MAD handling a bit cleverer
09:35imirkin: shouldn't be too hard
09:35imirkin: it's a super-special case
09:35imirkin: /* Move the immediate to the second arg, otherwise the ADD operation
09:35imirkin: * won't be emittable
09:36imirkin: just add a check if the immediate is 0 and make it into a MOV instead
09:36imirkin: er wtf... i re-run expr() on it
09:36imirkin: on, but only if they're BOTH immediates
09:36karolherbst: in the else branch?
09:36imirkin: yeah, should re-run opnd()
09:36imirkin: like this
09:37karolherbst: first I try to write a small tgsi to cath the case :D
09:38imirkin: karolherbst: try that?
09:39karolherbst: that works
09:39karolherbst: would be interessting what shader-db says about that
09:45karolherbst: imirkin: okay now we could take care of this one: https://gist.github.com/karolherbst/b7a3adb9a42bf688afa3#file-gistfile1-txt-L8372-L8376 :D
09:46karolherbst: imirkin: is there any difference in mul $r1 $r2 $r2 or mul $r1 neg $r2 neg $r2?
09:49karolherbst: this makes total sense though: https://gist.github.com/karolherbst/1293948dd982b332424f
10:05ob-sed: i was curious about nvidia 700 series firmware
10:05ob-sed: is it something the driver has to upload to the card on every boot.. or is there a persistance mechanism for the firmware to live on in the card ?
10:05karolherbst: don't you mean 900?
10:05ob-sed: yeah like GTX 760, 770, 780 etc
10:06karolherbst: mhh you mean for video acceleration then?
10:06ob-sed: 900 is the latest series yeah, but i mean 700 series
10:06karolherbst: mhh you don't need any firmware except for video acceleration and the normal kernel mechanism is used for that
10:07karolherbst: or what do you mean by "firmware"?
10:10ob-sed: karolherbst: ah well i mean like on an nvidia gpu
10:10karolherbst: you mean the vbios then?
10:10ob-sed: yeah the vbios
10:11karolherbst: it is already on the gpu
10:11ob-sed: and also does it have firmware thats uploaded at each boot (like the cpu microcode) or does it stay in flash on the card
10:11ob-sed: ahh yeah true, the vbios is in flash on the gpu
10:11karolherbst: it just stays on the card
10:11karolherbst: and nouvea reads it out from the gpu
10:11ob-sed: on the newer 900 series is nvidia requiring that the vbios code be digitally signed as well ?
10:12karolherbst: ob-sed: well with the gen2 maxwell cards the driver needs to upload a signed vbios to the card
10:12karolherbst: which isn't needed before
10:12ob-sed: ohhhh i see
10:12Tom^: cant you get it form the blob?
10:12ob-sed: ok so there is only _one_ set of firmware to be run on the card right, the vbios ?
10:12karolherbst: they are signed though I think, but the signature doesn't have to be valid
10:12karolherbst: Tom^: well tell us where to look :p
10:12ob-sed: lol it has to be signed, but the signature doesnt have to be valid lol ?
10:13ob-sed: what do you mean by that:) ?
10:13karolherbst: ob-sed: you can upload a vbios to the gpu which isn't signed
10:13karolherbst: and it would work
10:13ob-sed: ahh interesting
10:13karolherbst: but that has other reasons
10:13ob-sed: but now we cannot do that with the 900 series ?
10:13karolherbst: because the vbios is there mainly for the driver on pre 900 gpus
10:13ob-sed: ah right
10:14karolherbst: the vbios does more on the 900 cards, I am o expert with this though
10:27karolherbst: imirkin: running all passes again: 3901 => 3839 instructions
10:30karolherbst: imirkin: yeah, and the neg thing also disappears by that
10:39karolherbst: imirkin: running AlgebraicOpt after the last DeadCodeElim seems to cut some further instructions 3901 => 3861
10:41karolherbst: seems to bascially eliminate some adds away
10:41karolherbst: mul ftz f32 $r18 $r18 0.200000 + add ftz f32 $r18 $r12 neg $r18 => mad ftz f32 $r18 neg $r18 0.200000 $r12
11:41RSpliet: karolherbst: how much does execution time increase?
11:42karolherbst: no idea, should I render one frame with glxgears?
11:42karolherbst: ohh wait, some application was able to do that
11:45RSpliet: I'd just add some time() calls before/after optimisations
11:45RSpliet: so you can compare easily
11:45RSpliet: I assume it doesn't hurt
11:46karolherbst: RSpliet: there is a simplier approach
11:47karolherbst: run ModifierFolding after ConstantFolding (again?) and AlgebraicOpt after the last DeadCodeElim (again?)
11:48karolherbst: yeah, both have to be run twice :/
11:48RSpliet: sure, the more you run it, the more you fold
11:49RSpliet: the question is how much longer the runtime is going to be ;-)
11:49karolherbst: RSpliet: this is enough to cut instructions from 3902 to 3839 https://gist.github.com/karolherbst/1d3fb263893127dcd51a
11:49RSpliet: I *know* that
11:50karolherbst: :p k
11:50karolherbst: it makes more sense to find which optimisations produce non optimal code
11:50RSpliet: but there could be quite a lot of code hidden behind the RUN_PASS calls, esp. since it is likely to traverse through your entire code tree
11:50karolherbst: it does
11:50RSpliet: you'll never get "optimal code", you can get "decent code"
11:51RSpliet: or "quite all right code" :-)
11:52RSpliet: don't get me wrong, it's a useful exercise, but make sure you understand all the parameters
11:52RSpliet: it'd probably help further doing DeadCodeElim after AlgebraicOpt (to clean up)
11:53karolherbst: yeah, I don't think that running the pass over and over again is a good idea in general
11:53karolherbst: yeah was thinking that
11:53karolherbst: but well
11:53karolherbst: in my case no dead code was produced
11:53karolherbst: just add+mul => mad optimistaions
11:53RSpliet: depends, if it takes a second it isn't, if it takes less than 16ms it might be worth it :-P
11:53karolherbst: that's why I think that somewhere just not that good code is produced
11:54karolherbst: I will check what the second ModifierFolding run changes
11:55karolherbst: it isn't that much though
11:55karolherbst: ahh right
11:55karolherbst: it eliminates those negs
11:56karolherbst: RSpliet: ModifierFolding moves negs/abs and stuff into the instructions?
11:56RSpliet: I assume it does
11:57karolherbst: okay, then I think it is easy to figure out which optimisation we could improve
12:05karolherbst: RSpliet: maybe you see it? https://gist.github.com/karolherbst/225de4b290d8176d0837
12:05karolherbst: okay, the last 4 are the same :/
12:09karolherbst: okay, I think I have it
12:11karolherbst: seems like ConstantFolding produces some NEGs
12:11karolherbst: and these are optimized away by ModifierFolding
12:14karolherbst: so there seems to be optimizations like mul $r1 -1 => neg $r1, but this neg could be just folded into the source of other instructions
12:18RSpliet: what the ... oh wow, okay, that was unexpected
12:19RSpliet: finding values in the VBIOS that in trace looked an awful lot like training values :-P
12:24RSpliet: esp since it looks like a script, not a table
12:51imirkin: ooh nice find
12:52RSpliet: yes, I still hate it though
12:52RSpliet: it means I now have to write a parser for scripts, to turn it into a sequence of memx commands
12:54RSpliet: preferably without relying on the gf100_ramfuc struct defining all the registers memx might write to
12:54RSpliet: because who knows we need this for gk104
12:54RSpliet: or your gt21-something
12:54imirkin: karolherbst: yeah, so there are a few unfortunate situations where constantfolding generates opportunities for some of the other passes
12:55imirkin: not really sure what to do about those without sticking a few passes and running them to a fixed point
12:55imirkin: wouldn't be the worst thing in the world, tbh
12:56imirkin: i.e. take algebraic + modifier + constant and run them until they make no progress
12:56imirkin: it doesn't happen too often though
13:25karolherbst: imirkin: mhh sounds a bit painful, or should we say progress is instruction count went done?
13:26karolherbst: later we could do something weighted or just cycles needed for each instructions in total
13:26karolherbst: fun like that
13:27karolherbst: imirkin: thing is, dead code elimination also allows some algebraic optimizations
13:27karolherbst: this is really weird
13:27imirkin: coz in a few places we were lazy and look at refcounts?
13:27karolherbst: I guess so
13:27karolherbst: will look at the diff
13:28karolherbst: algebraic => dead code: 3879. dead code => algebraic: 3839
13:29imirkin: i believe it... i'm just annoyed
13:29karolherbst: yeah, it is pretty much actually :/
13:29karolherbst: thats what bothers me most
13:29karolherbst: if it would be like 1 or 2 like the case I found first
13:29karolherbst: then yeah, well bad luck
13:29karolherbst: but this is a bit more serious
13:30karolherbst: more than 1 %
13:32karolherbst: imirkin: fun fact: the dead code pass doesn't actually remove the instruction count
13:32karolherbst: allthough it moves code pieces from branch to branch
13:33karolherbst: ohh wait
13:34karolherbst: imirkin: what is the first nv50 dump by the way? and what is the second?
13:34karolherbst: pre/post RA?
13:34karolherbst: or pre/post this reg thingy
13:34imirkin: assuming you have NV50_PROG_DEBUG=1, then yes
13:34imirkin: pre and post RA
13:35karolherbst: pre RA: 5191/4247/4247 instructions post RA: 3879/3879/3839 instructions. where no last dead code/last dead code/algebric after dead code
13:35karolherbst: this is fun to figure out
13:36karolherbst: okay nice
13:36karolherbst: imirkin: it seems the algebraic pass only merges mul/adds into mads
13:36imirkin: it does a few other things depending on the shader
13:36karolherbst: yeah I know, but I meant in my case
13:37karolherbst: don't see any other changes so far
13:37karolherbst: and they are always i: mul i+1: add
13:37karolherbst: maybe dead code just removes something and between and then the algebraic pass can merge them?
13:39karolherbst: ohh wait
13:39karolherbst: that doesn't look right :O
13:40karolherbst: imirkin: is this right? https://gist.github.com/karolherbst/6d47f53cf62e053c75f5
13:41karolherbst: it is
13:41karolherbst: I am stupid
13:41karolherbst: I thought the first one is an add :D
13:41imirkin: seems fine
13:42karolherbst: yeah, I didn't read carefully enough
13:42karolherbst: imirkin: https://gist.github.com/karolherbst/dff00388e2cf317864d9
13:43karolherbst: seems like the algebraic thing can't handle those negs?
13:43karolherbst: ohh wait
13:43karolherbst: right, the $r15791 is dead code
13:45karolherbst: okay, it seems like I need to look earlier where this neg comes from, because r15790 is always negated
13:52karolherbst: imirkin: ohhhh
13:52karolherbst: found it
13:52karolherbst: imirkin: https://gist.github.com/karolherbst/dff00388e2cf317864d9
13:53karolherbst: 192: add ftz f32 %r15792 %r15783 %r15791 (0) => 192: add ftz f32 %r15792 %r15783 neg %r15790 (0)
13:53karolherbst: and the neg instruction has no use anymore
13:53karolherbst: so it is just left there
13:53imirkin: which is fine... DCE should take care of it
13:53karolherbst: yeah right
13:54karolherbst: but the mul/add can be merged
13:54karolherbst: so instead of just merging the neg into the add
13:54karolherbst: we should merge the source of neg also in it
13:54karolherbst: if it's a mul
13:54karolherbst: and do a mad
13:56karolherbst: mul $r2 $r1; neg $3 $2; add $r5 $r4 $r3 => mad $r5 neg $r2 $r1 $r4
13:57karolherbst: imirkin: how does that sound?
13:57imirkin: that sounds fine... isn't that what's happening?
13:58karolherbst: mhh no
13:58karolherbst: it doesn't do the add+mul => mad merge
13:58karolherbst: I just add another algebraic pass after the dead code thing, so that it is happening
13:59karolherbst: what bothers me is, that the neg even if it's dead code, prevents the add+mul merge too
14:02karolherbst: imirkin: which function does the neg+add => add neg thing? handleNeg or handleAdd?
14:02karolherbst: ohh there is no handleNeg :D
14:02imirkin: somewhere in there
14:02imirkin: i don't look at that pass too often
14:02karolherbst: the comments are out of date too?
14:03karolherbst: the ones which tells you what a pass does
14:04karolherbst: ohh right
14:06karolherbst: so I add the support for ADD(a, NEG(MUL(b, c)) -> MAD(neg b, c, a) inside AlgebraicOpt
14:07imirkin: what for?
14:07imirkin: it should already be handled via modifier folding + algebraic opt
14:07imirkin: oh i see
14:07imirkin: coz ... grr
14:07imirkin: hmmmmmmmm should modifier folding go *before* algebraic opt? needs some thought
14:08karolherbst: imirkin: currently there is a neg in between add/mul
14:08karolherbst: so you have something like add/neg/mul
14:08imirkin: right, i get it
14:08karolherbst: and neg uses the add result
14:09imirkin: what if you stick modifierfolding before algebraicopt?
14:10karolherbst: still 3901 instructions
14:10karolherbst: but the neg is gone actually
14:10karolherbst: ohh wait, it always gone
14:14karolherbst: imirkin: that is, what algebraicopt gets: https://gist.github.com/karolherbst/d968c33b69696fd50c74
14:18mupuf: RSpliet: nice finding!
14:23imirkin: karolherbst: ah right... coz of the "src0->refCount() == 1" heuristic
14:23imirkin: in AlgebraicOpt::tryADDToMADOrSAD
14:24imirkin: all of the refcount checks are pretty much hacks
14:24imirkin: thing is you're trying to make global decisions with only local info
14:24karolherbst: why not just do a mad out of that?
14:24imirkin: it's not an exact science
14:24imirkin: for example
14:25imirkin: mul foo, bar, 0.5; add baz, foo, 0.5
14:25imirkin: is better than
14:25imirkin: mov a, 0.5; mov b 0.6; mad baz foo a b
14:25imirkin: esp on nv50 you (basically) can't have an immediate in a mad
14:25imirkin: on nvc0 it makes more sense
14:26karolherbst: yeah right, but is mul foo, bar, 0.5; add baz, foo, 0.5 better than mul foo, bar, 0.5; mad bar, 0.5, 0.5?
14:26karolherbst: or is it the "same"?
14:27imirkin: mad can only have 1 immed arg
14:27imirkin: only the second one
14:28karolherbst: okay then "mul foo, bar, baa; add baz, foo, bab" and "mul foo, bar, baa; mad bar, baa, bab"
14:32karolherbst: imirkin: okay so the mad optimization might be unlucky when there are two immediates in the end and the mul is still used somewhere else
14:34karolherbst: ... I am the only one with connection problems, right?
14:34RSpliet: *white noise*
14:36karolherbst: imirkin: I will think of something
14:37karolherbst: maybe if the neg and mul have refcount == 1 we can do the mad out of it
14:40imirkin: hence the current checks in there ;)
14:40imirkin: except if modifierfolding passes the neg through
14:40imirkin: then that value has 2 uses
14:40imirkin: and i'd rather not sprinkle modifier folding all over the place for the sheer fun of it
15:29imirkin: RSpliet: fyi i have a GF108 vbios here which also uses only the 8f opcode
15:30RSpliet: imirkin: I was looking for your GDDR5 GT21x card, but I don't seem to have it's VBIOS
15:31RSpliet: it wouldn't have the script in the same place probably, but not surprised if it's there... somewhere
15:33RSpliet: oh there we are, yes confirmed for GT21x GDDR5 as well :-)
15:33imirkin: RSpliet: http://people.freedesktop.org/~imirkin/traces/nva3/
15:34imirkin: trace also there if you're interested
15:35imirkin: looks like it's gone on nvdX
15:35imirkin: but it's there on all nvcX's
15:35imirkin: and the nva3 too apparently
15:35RSpliet: yeah, it seemed like a "oh my god we need to stick this extra information somewhere for NVA3 GDDR5" hack :-P
15:41RSpliet: imirkin: well, gone on NVDx, or are all those cards just DDR3?
15:41RSpliet: the script seems to be quite GDDR5 specific
15:41imirkin: could be both :)
16:57imirkin: skeggsb: ping
17:09skeggsb: imirkin: hey
17:10imirkin: skeggsb: any thoughts on how to deal with nv50cal space errors? how should throttling be done?
17:10imirkin: skeggsb: my thought is that it'd be ok for the kernel to block...
17:11skeggsb: well, i find it very surprising that with a lot of small IBs, free slots don't clear fast enough..
17:11skeggsb: are you *certain* the GPU hasn't hung?
17:12imirkin: well, it can clearly happen.... i wouldn't be surprised if the submit just had too many IBs on its own
17:12imirkin: a single submit might take up multiple IB slots right?
17:12imirkin: with nouveau_pushbuf_data()
17:12imirkin: no :)
17:13imirkin: but there's no indication that it has
17:13imirkin: nothing in dmesg, etc
17:13skeggsb: that doesn't necessarily mean anything :P but, it'd be useful (and easy) to confirm before going that route
17:13skeggsb: remove the timeout from nv50_dma_push_wait(), see if it hangs forever
17:14imirkin: wait, there's a timeout? hm
17:14imirkin:should have read the code
17:15skeggsb: oh, and from READ_GET().. it's also possible the accounting goes horribly wrong somehow and gets things confused
17:15imirkin: ret = nouveau_dma_wait(chan, req->nr_push + 1, 16);
17:15imirkin: so... what if nr_push is just really big :)
17:15imirkin: how many slots are there in the first place?
17:15skeggsb: yes, that would be bad
17:16skeggsb: 1024, iirc
17:16skeggsb: that's changeable too
17:16imirkin: yeah, so with every indexed draw on nv50, we use up an extra slot
17:16imirkin: so that's 2 slots per draw
17:17skeggsb: userspace should probably flush before it gets to that point
17:17imirkin: (regular + data)
17:17imirkin: yeah... i was just thinking about that
17:17imirkin: if nrpush > ... 100? kick?
17:17imirkin: or 64?
17:17skeggsb: whatever random number you choose really
17:17skeggsb: as long as it's less :P
17:18imirkin: than 1024 :)
17:18imirkin: the IB limit is per channel right? not global?
17:18skeggsb: the kernel will consume 1 slot to insert a fence
17:18skeggsb: yeah, it's per-channel
17:18imirkin: i also want to start keeping track of how much vram is used in a "current" submit
17:18skeggsb: we currently hardcode 8KiB of space for the GPFIFO
17:19imirkin: i'm running into issues of submit fail on my nva3 with 512mb vram
17:19skeggsb: (that's the IB slots)
17:19imirkin: which i'm semi-moderately convinced is due to it trying to bunch up too many draws together
17:19imirkin: but i've done no analysis on that
17:19imirkin: but again, it should be moderately easy to loop through the current pushbuf buf list + bufctx and count up the vram
17:19imirkin: and if it's... x% of total vram, submit
17:20skeggsb: libdrm should already do that
17:20imirkin: er what?
17:20imirkin: i do remember there's that nouveau_available_vram thing, but i assumed that was in conjunction with soemthing else
17:21skeggsb: when you tell libdrm to "validate" a group of buffers for a submission, it'll check if any new buffers can fit with the ones previously queued, and flush if not
17:22imirkin: ah clever.
17:22imirkin: and if that submit should happen to fail, the validate fails too?
17:22skeggsb: i believe so, i haven't looked at that stuff in a *long* time.. calim and i spent ages trying to figure out how to make it all work, and that's what we came up with
17:23imirkin: fair enough
17:23imirkin: i do get ttm_validate failures though
17:23skeggsb: the kernel could handle stuff better if the initial attempt to fit stuff in fails (ie. as a last resort, kick everything out and try again)
17:24skeggsb: fragmentation and stuff is a problem too, where pinned scanout buffers randomly in the middle of memory can fuck things up too
17:24skeggsb: well, the latter shouldn't be a problem on g80 and up actually, since we can use non-contiguous allocations
17:25imirkin: not too worried about pre-g80
17:25skeggsb: me neither
17:26skeggsb: but i temporarily forgot that >=g80 doesn't have that problem :P
17:26imirkin: i do prefer to keep them working ok
17:26imirkin: but i doubt many people use ancient nvidia gpu's outside of this channel
17:26imirkin: [esp with nouveau]
17:27skeggsb: not suggesting we break them :P
17:27imirkin: only with a hammer
17:47imirkin: skeggsb: hm, i owe you a review of stuff, don't i... will try to get that done today
17:49skeggsb: yep, and i owe you fifo fixes :P they've gotten a tad more involved than i'd have liked, but i should have them pushed soon
17:49imirkin: skeggsb: and that will make parallel piglit reliable?
17:50imirkin: skeggsb: including but not limited to nv50?
17:50skeggsb: i still have to find a board that has issues still, even with the patches in my tree already the two boards i were using worked fine
17:50skeggsb: one fermi, one kepler
17:50skeggsb: i'll look over tesla too
17:50imirkin: hmmmm iirc i tried those patches and my gk208 still fell over
17:50imirkin: potentially due to unrelated reasons
17:50skeggsb: yeah, i recall you saying
17:51skeggsb: i'll get airlied to retrieve mine from the ppc machine and see if i can reproduce tomorrow
17:52airlied: you might be waiting for me to come in :)
17:57imirkin: skeggsb: perhaps try on a shittier gpu?
17:58skeggsb: i'll plug in a variety and see what happens
17:58imirkin: a GK107 should match the shittiness of my GK208
17:59skeggsb: i <3 my gk107, it lets me play sc2 :P
21:00Arbition: did someone say gk208?