07:39 bentiss: daniels: have you manually wiped out /run/github-mirror/.ssh/authorized_keys on kemper? this disappeared and all of the mirroring from gitlab to cgit is now broken
07:40 daniels: bentiss: nope, but being in /run/ it would've got wiped on reboot?
07:40 bentiss: oh, right
07:40 bentiss: can we add the keys to ldap?
07:40 daniels: hngh
07:40 bentiss: or is it still broken?
07:40 daniels: I can do that with ldapvi
07:40 daniels: just paste me the keys somewhere
07:41 bentiss: I'd need to retrieve them on the various repos, but in a meeting ATM
07:56 bentiss: daniels: also the gitlab-mirror user seems to have lost his id, so maybe you'll need to also grant this user access to the drm and mesa repos
08:07 daniels: ok, done now
08:15 bentiss: I'm still getting Permission denied (publickey) AFAICT
08:17 bentiss: and of course, retrying is not an option if the last attempt failed :(
08:18 daniels: yeah, userdir-ldap got absolutely mangled in the upgrade, and I wonder if sshd did as well
08:20 daniels: hmm nope, the only gitlab-mirror connections in the last 30min were all successful
08:20 daniels: gtg though - running late to pilates and really need to be at that one since my back is kinda broken atm :P
08:21 bentiss: damn, well, have fun, it can definitely wait a few more hours
08:26 bentiss: daniels: FWIW, it could be that kemper is not answering to ipv6
08:26 svuorela: has git fallen off a cliff ?
08:31 Mithrandir: daniels: I thought I fixed the userdir-ldap problems, what's still missing?
10:34 daniels: Mithrandir: turns out I was looking in the wrong place for gitlab-mirror and SSH, since it's using the per-user key files rather than merged ssh-rsa-shadow now, but the main one is that the LDAP schemas are somehow incoherent
10:35 daniels: /etc/ldap/schemas/userdir-ldap.conf has the correct schema, but the actual LDIF schema slapd has in /etc/ldap/slapd.d/ is missing a bunch of stuff, notably including allowedGroups for the host entries
10:36 Mithrandir: ah, ok. That seems like a bug™, it might well be I should have imported that.
10:36 Mithrandir: are you fixing, or should I?
10:36 daniels: I did just enough fooling with configs to fix LDAP authentication (since it appeared to be RO), but I started losing the will to live trying to figure out how to patch the schema
10:36 daniels: it looks like you need slapcat/slapadd to do that; trying to do it online was complaining that it was immutable, which tbh is fair enough
10:37 daniels: if you have the time then please feel free as I'm stuck in other stuff for the rest of today
10:37 Mithrandir: "ldapvi -h ldapi:/// -Y EXTERNAL -b cn=config" as root to edit stuff with ldapvi, btw.
10:37 Mithrandir: I'll take a look.
10:40 daniels: right, I tried doing that to patch the schema, but then got an error when applying suggesting that it was immutable
10:40 Mithrandir: ok
10:40 daniels: oh yeah, the other one was that I needed to set olcSizeLimit, else ud-generate was failing to get any results back at all
10:40 Mithrandir: I thought I fixed that one earlier.
10:43 daniels: the default olcSizeLimit was set to unlimited, but the udl db was on 500
10:43 daniels: thanks btw :) sorry if I sound grumpy, just in quite a bit of pain
10:43 daniels: but I do appreciate it
10:44 Mithrandir: No worries! Hope the pain goes away soon.
10:55 daniels: thanks!
11:00 Mithrandir: daniels: worked fine for me to update those attributes, so not sure what was wrong.
15:48 eric_engestrom: gitlab has been really slow for me for a couple of hours, both web and git (ssh)
15:48 eric_engestrom: (and it's not my internet, other websites are responsive)
16:05 __tim: having problems pushing branches to gitlab with ssh, I'm guessing it's getting hammered after the CVE?
16:08 __tim: hrm, works again now :)
16:38 eric_engestrom: yeah it's a bit better for me now
16:38 eric_engestrom: and yeah it's possible every script kiddie out there is trying to get into every public ssh server, I hadn't thought about that as a possible cause :/
17:26 Lyude: it's still being slow for me, but only on git pulls from anongit
17:29 Lyude: emersion, daniels, bentiss ^ jfyi git pull from anongit seems to hang forever. decided to try to ssh into freedesktop.org just to see if it would respond and it seems like that's down as well. figured I should mention it considering the ssh vuln that dropped yesterady
17:30 emersion: someone on mastodon mentioned IPv6 connectivity issues
17:31 Lyude: oh that might be it, let's see
17:31 Lyude: yep!
17:31 Lyude: looks like that's exactly it, thanks
17:33 Lyude: ...emersion: for ssh, at least. i'm not seeing anything on git pull still
18:04 daniels: they’re upgraded for that
23:02 vsyrjala: did anongit fall over completely?