12:00 karolherbst: pmoreau: ever ran the CTS spirv tests? :D
12:01 karolherbst: also, we might want to fix the CTS to accept the extension as well
12:05 Forty-Bot: I would like to reclock my 960, but when I cat /sys/kernel/debug/dri/0/pstate I get "No such device"
12:05 karolherbst: Forty-Bot: it's disabled on those as we can't change fan speeds
12:05 Forty-Bot: oof
12:05 karolherbst: well, we know how to do it
12:05 karolherbst: the hw just doesn't allow us
12:06 Forty-Bot: the nvidia driver has been crashing recently so I wanted to try nouveau
12:06 karolherbst: you can of course compile the kernel yourself with 1 or 2 patches and force enable it, but we thought for the general user it might not be a good idea :)
12:06 Forty-Bot: but I'm getting 50 fps in games where I used to get 250
12:06 karolherbst: yeah...
12:06 karolherbst: it's an annoying issue
12:07 karolherbst: essentially what we require is redistributable signed PMU firmware from nvidia in order to do it
12:07 karolherbst: (and support their protocol and what not)
12:07 karolherbst: Forty-Bot: this is a desktop, no?
12:07 Forty-Bot: yes
12:07 karolherbst: yeah mhhh
12:08 karolherbst: I have some patches for laptop users as fans are generally driven by the laptops firmware, but on desktop usually we don't want to allow higher clocks when the fans keep running at stock speeds :/
12:08 karolherbst: sorry that we can't do anything about it really
12:08 imirkin: one user went so far as to attach his own fans onto his GPU
12:08 Forty-Bot: lol
12:08 karolherbst: (well we could, but that would require extracting firmware from nvidias driver and users would have to do it themselves or run scripts)
12:08 imirkin: so that they could be controlled separately
12:09 Forty-Bot: ok so I should just use nvidia drivers for this gpu and try and find a fix for the crashes>
12:09 karolherbst: anyway.. at the moment every solutions sucks
12:09 karolherbst: if performance is important to you then probably yes
12:10 Forty-Bot: :l
12:11 imirkin: thank you for buying nvidia. we appreciate you have a choice of gpu vendors, and you appear to have made the wrong one.
12:12 Forty-Bot: I wasn't really thinking about that at the time... it also worked very well for like 4 years and it's only recently staretd crashing
12:12 imirkin: think about it next time you guy a gpu :)
12:13 Forty-Bot: yeah
12:14 Forty-Bot: at the time I was just glad to be moving off of intel integrated graphics
12:14 Forty-Bot: actually, I also played a game which has since dropped linux support which *only* worked on nvidia gpus for some reason
12:15 karolherbst: this is how my day starts today: https://gist.githubusercontent.com/karolherbst/ad1a934595736f4d549b408aad69ae6d/raw/59339d88957abbdff3133c35561ce6a47dd8c7ad/gistfile1.txt
12:15 karolherbst: ...
12:16 karolherbst: ehhh
12:16 imirkin: is that wrong?
12:16 karolherbst: when I compare strlen of the c strings both are 39...
12:16 imirkin: i don't think size() does what you think it does
12:17 karolherbst: imirkin: one string is longer even though they are the same?
12:17 karolherbst: probably
12:17 karolherbst: but
12:17 karolherbst: c++ uses size when comparing strings
12:17 imirkin: right
12:17 imirkin: so
12:17 imirkin: c++ strings are pascal-style
12:17 imirkin: while c strings are ... c-style
12:17 imirkin: i.e. c-strings are null-terminated
12:18 imirkin: while c++ strings have an explicit length
12:18 imirkin: and thus can contain nulls
12:18 karolherbst: ehhh
12:18 karolherbst: right
12:18 imirkin: now ... it's VERY RARE to actually want that
12:18 karolherbst: thing is just.. it all originated from some C stuff
12:18 imirkin: so most likely this points to a bug somewhere
12:19 karolherbst: yeah.. mhhh
12:19 karolherbst: found was extracted via istringstream and the >> operator
12:19 imirkin: which is why it's correct...
12:19 imirkin: oh
12:19 imirkin: hm
12:19 imirkin: it's not, i guess...
12:20 karolherbst: yeah.. dunno
12:20 imirkin: good luck with that ;)
12:20 karolherbst: :D
12:20 imirkin: one counts the null in the size, the other doesn'
12:20 imirkin: good times.
12:20 karolherbst: well
12:20 karolherbst: orignally the string got retrieved via the CL API which is C
12:20 karolherbst: just splitted with istringstream
12:21 karolherbst: ehhh...
12:21 karolherbst: the heck
12:21 karolherbst: code is weird sometimes
12:21 karolherbst: okay..
12:21 karolherbst: I think I see what's up
12:22 karolherbst: I bet istreamstream splits until it finds a non "NULL" character or sometimg
12:22 karolherbst: and the CTS retrieves stuff with an explicitly sized std::vector<char>, fills in via data() and then constructs the std::string through tierators
12:22 karolherbst: annoying...
12:23 RSpliet: sounds inefficient
12:23 karolherbst: it has to fetch the size first to provide a buffer big enough :)
12:26 karolherbst: heh..
12:26 karolherbst: so yeah.. this only happens with the last extensions advertised
12:26 karolherbst: when returning from clover assert(extensions.size() == strlen(extensions.c_str())) this is fine
12:27 karolherbst: so I don't think anything is broken on that level
12:28 RSpliet: That sounds dodgy tbh
12:28 RSpliet: size() returns "the number of actual bytes that conform the contents of the string, which is not necessarily equal to its storage capacity."
12:28 karolherbst: right...
12:28 RSpliet: c_str() should add a null-terminator to the string
12:29 karolherbst: it does
12:29 RSpliet: strlen() should count the null terminator ;-)
12:29 karolherbst: p (int)strlen(found.c_str())
12:29 karolherbst: $89 = 39
12:29 karolherbst: (gdb) p found.size()
12:29 karolherbst: $90 = 40
12:30 karolherbst: you tell me what's wrong :p
12:30 imirkin: like i said - size isn't what you think it is.
12:30 imirkin: it is _not_ strlen
12:30 karolherbst: it is used when using the == operator though
12:30 imirkin: of course.
12:30 imirkin: and those two strings ARE different
12:30 imirkin: one is length 40, the other is 39
12:31 karolherbst: mhhh
12:31 karolherbst: right
12:31 karolherbst: I just don't know why they are different :)
12:31 karolherbst: so it is a istringstream bug or what?
12:31 RSpliet: manually counting cl_khr_spirv_no_integer_wrap_decoration gives me 39. so strlen(found.c_str()) should give 40
12:31 karolherbst: nope
12:31 karolherbst: strlen doens't count the null terminator
12:31 imirkin: RSpliet: strlen doesn't count null
12:32 RSpliet: oh my bad
12:32 RSpliet: you're right
12:32 karolherbst: sooo
12:32 RSpliet: Ok, that's one thing resolved for me
12:32 imirkin: karolherbst: solution to your problem is "don't use =="
12:32 karolherbst: only for the last string extracted, size != strlen
12:32 karolherbst: heh ;D
12:33 karolherbst: I guess I could propose a fix but that's the CTS so I am more annoyed by the fact I am supposed to do stuff privately?
12:33 RSpliet: karolherbst: for (int i = 0; i < found.size(); i++) printf("%x\n", found[i]);
12:34 RSpliet: Be nice to know exactly what those extra characters are
12:34 karolherbst: 0
12:34 karolherbst: it just has two null terminators
12:35 karolherbst: I bet istringstream is wonky
12:35 karolherbst: so it includes the 0 of the input string
12:35 karolherbst: and adds another 0
12:37 karolherbst: ohhh
12:37 karolherbst: the entire extensions string is one element too long
12:37 karolherbst: the plot thickens
12:38 karolherbst: ahh
12:39 karolherbst: we return 294 for the extension strings size, but it's only 293 long
12:39 karolherbst: guess it's our bug
12:40 karolherbst: mhh
12:43 karolherbst: mhhhhhh
12:45 pmoreau: karolherbst: https://github.com/KhronosGroup/OpenCL-CTS/issues/790
12:45 karolherbst: yeah....
12:45 karolherbst: I think it's a CTS bug
12:46 pmoreau: It is, there is no doubt about it
12:46 karolherbst: I think the issue is how istreamstream is used
12:46 karolherbst: or how the string is constructed...
12:47 karolherbst: pmoreau: does the spec even requires null terminated strings? :D
12:48 pmoreau: `std::string` are not-null terminated.
12:48 pmoreau: Did you read what I wrote in that issue btw? :-)
12:48 karolherbst: that's not my point
12:48 karolherbst: I mean, whatever we report via the CL API
12:49 pmoreau: C-strings should be null-terminated
12:49 karolherbst: yeah..
12:49 pmoreau: And what we report via the CL API is null-terminated
12:49 karolherbst: then it's easy to fix
12:50 pmoreau: The problem is that https://github.com/KhronosGroup/OpenCL-CTS/blob/09aa54246f607bc1770a7cc2dc076185487a00f7/test_common/harness/deviceInfo.cpp#L48 copies the '\0' into the C++ string rather than discarding it.
12:50 karolherbst: pmoreau: return std::string(info.begin(), info.end()); -> return std::string(info.data())
12:51 pmoreau: I feel like I saw that somewhere already… :-D
12:51 pmoreau: ;-)
12:51 karolherbst: :D
12:51 karolherbst: yeah
12:51 karolherbst: anyway, that's what I would go for really
12:51 karolherbst: the issue is rather if the spec expects a null terminator or not
12:52 karolherbst: and if people throw in null terminators inside their strings they deserve to fail conformence
12:53 karolherbst: pmoreau: "Returns a space separated list of extension names"
12:53 pmoreau: I didn’t send a patch with that fix cause I didn’t have time nor wasn’t sure whether the API ever expects to use '\0' as a way to split a string into several elements, like "str0\0str1\0str2\0\0".
12:55 karolherbst: now I need to figure out how to get the spirvs :D
12:55 pmoreau: And to answer your earlier question, I think so cause I was running into the issue of BuiltIn variables not using the Input storage class
12:56 pmoreau: Let me see if I have a script somewhere for it… it wasn’t too difficult, you just needed to pass a path to the folder with the binaries IIRC
12:56 pmoreau: karolherbst: ./bin/test_spirv_new --spirv-binaries-path /home/pmoreau/Projects/OpenCL_for_Nouveau/src/OpenCL-CTS/test_conformance/spirv_new/spirv_bin
12:58 karolherbst: sure.. but I don't have the binaries :p
12:58 RSpliet: pmoreau: if the API expects \0 to be separators rather than terminators, it wouldn't leave the terminating \0 in place no?
12:58 karolherbst: RSpliet: it's explicitly sized
12:59 pmoreau: karolherbst: How come you don’t have them? :o What do you have in test_conformance/spirv_new/spirv_bin?
12:59 karolherbst: nothing
13:00 karolherbst: I don't have KHRONOS_OFFLINE_COMPILER set anyway
13:00 pmoreau: Me neither
13:00 pmoreau: Are you sure you are on the correct branch?
13:00 karolherbst: yes
13:00 pmoreau: https://github.com/KhronosGroup/OpenCL-CTS/tree/master/test_conformance/spirv_new/spirv_bin There are available in master
13:01 karolherbst: ohhh
13:01 karolherbst: I checked in my build dir :D
13:01 pmoreau: xD
13:02 karolherbst: ehh
13:02 karolherbst: chdir mess
13:02 karolherbst: why is software so annoying? :D
13:02 pmoreau: RSpliet: Re “ if the API expects \0 to be separators rather than terminators, it wouldn't leave the terminating \0 in place no?” Well, if you do not have a size, the final one is still needed to know you reached the end.
13:02 pmoreau: Why do you need to chdir?
13:03 RSpliet: pmoreau: that would only work if there's no other \0s in the string
13:04 pmoreau: Unless you consider that an empty string (i.e. two '\0' in a row) mark the end of your string array. But it’s not something I would ever recommend or do, but I have seen at least one API do that.
13:05 karolherbst: pmoreau: for my CTS runner if I don't want to pass in the argument..
13:05 karolherbst: but I guess I should
13:05 pmoreau: Ah I see
13:05 RSpliet: More generally, \0 should have a meaning. That meaning is either "separator between elements", "terminator of the string" or "undefined, should never appear". C++ doesn't mandate a meaning when handling string objects, so that's all up to the SPIR-V guys. All three meanings I've given it would imply that they shouldn't be retained when converting to a list of strings :-)
13:05 karolherbst: ehh.. (seed = 0371120151) ERROR: unknown nir_op f2i8
13:05 karolherbst: annoying :D
13:05 RSpliet: or well, not the SPIR-V guys, but khronos...who happen to also be the SPIR-V guys :-P
13:06 pmoreau: :-D
13:22 karolherbst: pmoreau: stuff looking good though
13:22 karolherbst: Pass 155 Fails 38 Crashes 9
13:25 karolherbst: some value mismatches but I think it's not that bad
13:31 pmoreau: 👍️
14:50 karolherbst: pmoreau: once you have time, mind taking a look at https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6433 ?
14:50 pmoreau: Sure
15:09 karolherbst: mhhhh
15:09 karolherbst: so the CTS is annoying
15:10 karolherbst: if we advertise 4GB of memory being legal to allocate, it just goes ahead and allocates 4095MB :D
15:10 karolherbst: which obviously fails
15:55 karolherbst: pmoreau: nice.. I have three tests which have no subtest fails (with 1.1) now :)
15:55 karolherbst: allocations, atomics and basic
16:08 pmoreau: Awesome!
16:09 pmoreau: Are you running a particular branch to avoid running 1.2+ tests?
16:09 karolherbst: nope
16:09 karolherbst: that's not needed anymore
16:09 karolherbst: I just run them as 1.1 or 1.2
16:09 karolherbst: starting with 2.0 there are more fails
16:09 karolherbst: but...
16:09 karolherbst: :p
16:10 pmoreau: Well, there are several 1.2 tests that are not marked as 1.2 and so will still run even at 1.0, which is a problem if it’s not implemented.
16:10 karolherbst: well
16:10 karolherbst: sure
16:11 karolherbst: but it doesn't matter :p
16:11 karolherbst: I think 1.2 is a viable target
16:13 pmoreau: It does matter more on nv50 I guess, since we won’t be able to advertise more than 1.0 IIRC.
16:13 karolherbst: why not?
16:14 pmoreau: 1.1 bumped the minimum amount of local memory available, and it’s higher than what you get on Tesla hw.
16:14 karolherbst: ehhh...
16:14 karolherbst: so stupid :/
16:15 pmoreau: “CL_​DEVICE_​LOCAL_​MEM_​SIZE from 16 KB to 32 KB.” and Tesla has 16 KB
16:15 karolherbst: mhhh
16:16 karolherbst: I don't think this is an issue though
16:17 karolherbst: you have 48k per block
16:17 pmoreau: Tesla has 16 KB per SM
16:17 karolherbst: and I think nvidia advertises 1.1 even on tesla
16:17 karolherbst: maybe not alll
16:17 karolherbst: but some at least
16:17 pmoreau: And a block lives on a single SM
16:18 karolherbst: "Driver 340+ support OpenCL 1.1 for Tesla and Fermi."
16:19 pmoreau: Mmh, I wonder what they do then
16:19 karolherbst: yeah
16:19 karolherbst: pmoreau: https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#features-and-technical-specifications
16:20 karolherbst: uhh wait
16:20 karolherbst: we need an older version
16:20 pmoreau: Look at the one on Wikipedia: https://en.wikipedia.org/wiki/CUDA#Version_features_and_specifications
16:20 karolherbst: yeah, it says 48k per block
16:21 karolherbst: maybe you can create blocks across multiple SMs?
16:21 karolherbst: that would be super strange
16:21 karolherbst: well.. we have the QMD files, let's check there
16:22 imirkin: qmd is for nve4+
16:22 karolherbst: well,... class headers then
16:22 pmoreau: AFAIK a block can not span multiple SMs, at least in recent architectures. But maybe Tesla was different? It’s the only architecture where the limit per thread block is higher than the limit per SM.
16:23 karolherbst: yeah
16:24 karolherbst: als there are no max registers per thread block
16:24 karolherbst: :p
16:24 pmoreau: Let’s see if I can find anything in the early CUDA programming guides which only supported Tesla.
16:30 pmoreau: “A block is processed by only one multiprocessor, so that the shared memory space resides in the on-chip shared memory leading to very fast memory accesses.” (from the Programming guide for CUDA 1.0) soooo, is this leaving the door opened to possibly using global memory instead of shared if you use too much shared?
16:31 karolherbst: mhhh
16:32 karolherbst: I think 1.0 is too old to use that one :p
16:32 karolherbst: what's the latest one supported on tesla 6.5?
16:32 karolherbst: 7.0?
16:34 pmoreau: I think 6.5 sounds about right
16:34 karolherbst: ahh yeah
16:34 karolherbst: 6.5
16:35 pmoreau: I wanted to check in one which hadn’t been influenced by Fermi, since it seems the behaviour might have changed.
16:36 karolherbst: "dnf install cuda-documentation-6-5" :D
16:36 karolherbst: ahh..
16:37 pmoreau: :-D
16:38 karolherbst: let's see
16:39 pmoreau: In CUDA 2.0 and 3.0 (the first one with Fermi support), they changed the language a bit: “There is a limit to the number of threads per block, since all threads of a block are expected to reside on the same processor core and must share the limited memory resources of that core.”
16:39 karolherbst: Maximum amount of shared memory per thread block: 16 kb
16:41 karolherbst: pmoreau: I mean.. we can advertise 1.0 on tesla. I think I just prefer that we have a method checking all the requiernments inside clover
16:41 karolherbst: so we don't have to constantly look it up