#archlinux32 | Logs for 2020-02-16

[02:54:04] <buildmaster> pentium4/intel-dnnl is broken (says eurobuild6-1): https://archlinux32.org
[02:55:15] <buildmaster> i486/intel-dnnl is broken (says eurobuild6-7-i486): https://archlinux32.org
[02:56:12] <buildmaster> i686/intel-dnnl is broken (says rechenknecht): https://archlinux32.org
[03:25:49] -!- rcf has quit [Quit: WeeChat 2.5]
[03:28:27] -!- rcf has joined #archlinux32
[03:51:18] -!- nit-picker has quit [Remote host closed the connection]
[03:51:39] -!- nit-picker has joined #archlinux32
[03:51:40] <buildmaster> Hi nit-picker!
[03:51:40] <buildmaster> !rq nit-picker
[03:51:40] <phrik> buildmaster: <nit-picker> abaumann: but there are only 18 "intermediate" versions
[04:10:27] <nit-picker> key A0B250C0FC9FC079EC04ADB7A50C0F20AEC3AF00 (from Polichronucci (Arch Linux 32 Master Key) <polichronucci@archlinux.gr>) in package archlinux32-keyring-transition-20200113-1.0-any.pkg.tar.zst expires on 2020-05-05 (in 79 < 100 days).
[05:46:23] -!- samantaz has quit [Ping timeout: 260 seconds]
[07:39:48] <trotz> 2020/02/16 07:38 CRIT buildmaster OS updates 1 updates, 0 ignored
[08:34:49] <trotz> 2020/02/16 08:33 CRIT buildmaster SSH Server answer:
[08:44:07] <girls> errr, looks like archlinux' new openssh is broken - I can no longer log in O.o
[08:49:49] <trotz> 2020/02/16 08:48 CRIT jeti100 SSH Server answer:
[08:50:59] <girls> new archlinux' ssh is broken (on my machines)
[08:51:17] <girls> this happens on x86_64, archlinux32, archlinuxarm
[08:52:31] <girls> luckily, I inserted an emergency-shell for those cases into my upgrade scripts right at the end :-)
[08:54:49] <trotz> 2020/02/16 08:53 OK jeti100 SSH SSH OK - OpenSSH_8.1 (protocol 2.0)
[08:59:49] <trotz> 2020/02/16 08:58 OK buildmaster SSH SSH OK - OpenSSH_8.1 (protocol 2.0)
[09:01:50] -!- abaumann has joined #archlinux32
[09:01:50] <buildmaster> Hi abaumann!
[09:01:50] <buildmaster> !rq abaumann
[09:01:51] <phrik> buildmaster: <abaumann> I made the experience that when the config is simple and looking nicely it usually also works. :-)
[09:02:09] <abaumann> girls: yeah, I just broke half of my logins to my machines..
[09:02:18] <abaumann> ..I was fast enough not to update on the buildmaster yet.
[09:02:33] <abaumann> So, OpenSSH 8.2 seems to deprecate SHA1-broken keys indeed.
[09:03:01] <abaumann> There is no other option than to generate new keys and then when done, do the updates of openssh to 8.2
[09:03:44] <abaumann> What I don't get, why can I not log in with a password?
[09:04:38] <abaumann> I understand that openssh refuses the key, but if I don't present one, why do I get "kex_exchange_identification: read: Connection reset by peer"?
[09:05:02] <abaumann> Maybe I'm also only allowing insecure key exchange algos, not only authentication keys?
[09:05:31] * abaumann heads to his nice cable salad formerly known as KVM-switch and starts to connect to a bunch of machines
[09:09:54] * buildmaster goes insane.
[09:10:56] <abaumann> aha: if I present a key, it must fullfil the needs, there is no fallback to passsord authentication (which is sensible for a security point of view). So moving the rsa private key out of the way is the solution..
[09:11:11] <abaumann> kex_exchange_identification: read: Connection reset by peer
[09:11:15] <abaumann> still, mmh.
[09:11:23] <abaumann> this is annoying.
[09:17:03] <abaumann> especially machines _only_ accepting a key are annoying..
[09:17:36] <abaumann> so, what's the future? algorithms getting hacked faster and faster, so the sshd from today doesn't accept the keys generated todya?
[09:21:50] -!- abaumann has quit [Quit: leaving]
[09:23:14] -!- abaumann has joined #archlinux32
[09:23:14] <buildmaster> Hi abaumann!
[09:23:14] <buildmaster> !rq abaumann
[09:23:15] <phrik> buildmaster: <abaumann> they do all kind of weird stuff, nobody else does, because they are known not to work. :-)
[09:23:36] <girls> abaumann: maybe the server-key needs to be regenerated, too?
[09:23:44] <abaumann> could be
[09:23:51] <girls> !bug 65517
[09:23:51] <phrik> https://bugs.archlinux.org
[09:23:57] <abaumann> for archlinux32.org it's too late already, I cannot login anymore. :-)
[09:24:29] <girls> "(09:23:53) fogobogo: just as i always said, telnet is superior"
[09:24:38] <abaumann> !grab girls
[09:24:39] <phrik> abaumann: Bazinga!
[09:24:40] <abaumann> heh
[09:25:11] <abaumann> I set ignore to openssh everwhere for now, so I can investigate and bring the servers back one by one
[09:25:35] <abaumann> I always though strong password authentication is superior to key authentication..
[09:25:38] <abaumann> ..but that's me :-)
[09:25:41] <abaumann> *thought
[09:26:22] <girls> if you can remember enough strong passwords, then "yes", but normal persons cannot do that
[09:26:51] <abaumann> that's what private password vaults (also known as encrypted text files on a USB stick) are for. :-)
[09:27:16] <girls> yes, ssh keys are *almost* the same thing
[09:27:29] <girls> though different in the case which we have now
[09:28:35] <girls> abaumann: can you give a link to the info, that openssh deprecated some keys?
[09:28:54] <abaumann> https://www.openssh.com
[09:29:00] <girls> thx
[09:37:53] <abaumann> Feb 16 08:30:40 eurobuild3 sshd[3102957]: fatal: recv_rexec_state: buffer error: incomplete message
[09:38:05] <abaumann> that's what the openssh 8.2 servers are telling me on the console
[09:42:08] <abaumann> Yeah, if you don't configure any host key, the it generates three of them: ssh_host_dsa_key, ssh_host_ed25519_key and ssh_host_rsa_key
[09:42:20] <abaumann> So, I suspect it takes the broken rsa one first
[09:42:29] <abaumann> HostKey /etc/ssh/ssh_host_ecdsa_key
[09:42:29] <abaumann> HostKey /etc/ssh/ssh_host_ed25519_key
[09:42:36] <abaumann> in /etc/ssh/sshd_config helps
[09:42:47] <girls> archlinux32.org is dead via ssh, too :'-(
[09:42:52] <abaumann> yes.
[09:43:04] <abaumann> we need Poli's console, I'm afraid
[09:43:42] <abaumann> weird. I just only restarted sshd, and this seems to solve the login problem
[09:43:53] <abaumann> Isn't the openssh update hook restarting sshd?
[09:44:24] <girls> O.o
[09:44:33] <girls> arch usually does not restart anything
[09:44:46] <abaumann> ecdsa-sha2-nistp256, mmh. the known key was a stronger cypher
[09:44:55] <abaumann> in known_hosts. so..
[09:45:05] <abaumann> exactly
[09:45:11] <girls> indeed, that helped !
[09:45:19] <abaumann> this openssh update was a little bit, aeh. well, sloppy :-)
[09:45:26] <abaumann> upstream-wise
[09:45:39] <abaumann> this should be announced on www.archlinux.org
[09:45:42] <girls> yes
[09:46:03] <abaumann> not everybody reads SHA1 security papers or openssh Changelogs - though, I admin, maybe one should..
[09:46:07] <abaumann> *admit
[09:46:51] <abaumann> eurobuild3 ssh-rsa
[09:46:57] <abaumann> mmh. ah.
[09:47:09] <abaumann> I have a 8.1 client, which still accepts ssh-rsa server keys. :-)
[09:47:57] <abaumann> So, the theory is: if you don't restart sshd, then the server hands out ssh-rsa keys with high priority, so restarting sshd alone helps?
[09:48:32] <girls> you mean an older client would help, too?
[09:48:37] <abaumann> no
[09:48:52] <abaumann> I was just puzzled my 8.1 client accepted a ssh-rsa host key in known_host
[09:48:58] <abaumann> when the method is deprecated
[09:49:11] <girls> it is *about to be* deprecated
[09:49:12] <abaumann> if you restart sshd you can connect with 8.1 and 8.2 clients, doesn't matter
[09:49:25] <abaumann> yeah, SHA1 is beatifully broken. :-)
[09:50:18] <abaumann> I wonder, what the proper migration path for openssh 8.2 is:
[09:50:24] <abaumann> pacman -Syyu
[09:50:35] <girls> systemctl restart sshd
[09:50:35] <abaumann> remove all server certificates
[09:50:38] <abaumann> restart sshd
[09:50:44] <girls> no, restarting alone is enough
[09:50:47] <girls> (on my boxes)
[09:50:54] <abaumann> but then it hands out ssh-rsa keys to old clients
[09:50:58] <abaumann> which it should not
[09:51:11] <abaumann> let me test
[09:52:21] <girls> hmm, I have one box which updated openssh without problems
[09:52:21] <abaumann> na.
[09:52:35] <abaumann> it regenerates ssh_host_rsa_key and ssh_host_rsa_key.pub
[09:52:51] <abaumann> either they should _not_ be deprecated on the server, or there is somehting missing in the server keygen script
[09:53:08] <girls> ah, maybe that box is new enough to not have to-be-deprecated host keys?
[09:53:22] <abaumann> did you check in /etc/ssh/
[09:53:55] <girls> keys are from 2020-01-18
[09:54:01] <girls> ah no
[09:54:03] <girls> 2019-01-18
[09:54:19] <girls> there is dsa, rsa, ecdsa, ed25519
[09:54:39] <abaumann> if you reinstall sshd then it generates ssh_host_rsa_key, but on first login hands out the ecdsa-sha2-nistp256 key
[09:54:58] <abaumann> maybe you had a server key in .ssh/known_hosts with a stronger algorithm
[09:55:05] <abaumann> so, the next caveat is to check for those.
[09:55:22] <abaumann> yep, same 4 here
[10:01:44] <girls> yay, I recovered one box via jupyterhub :-D
[10:03:28] <abaumann> cool :-)
[10:03:39] <abaumann> though. every box you can recover this way, has a backdoor. :->
[10:03:51] <girls> well, it requires password login
[10:03:52] <abaumann> well, also a KVM switch is technically a backdoor
[10:03:55] <girls> yes
[10:03:59] <girls> !grab abaumann
[10:04:00] <abaumann> true
[10:04:00] <phrik> girls: Tada!
[10:04:15] <abaumann> my KVM switch has no password. *umpf*
[10:04:22] <girls> :-D
[10:26:23] -!- titus_livius has joined #archlinux32
[10:29:25] <nit-picker> key A0B250C0FC9FC079EC04ADB7A50C0F20AEC3AF00 (from Polichronucci (Arch Linux 32 Master Key) <polichronucci@archlinux.gr>) in package archlinux32-keyring-20200113-1.0-any.pkg.tar.zst expires on 2020-05-05 (in 78 < 100 days).
[10:31:45] <abaumann> *puzzle*
[10:31:53] <abaumann> no my id_rsa key also works?
[10:33:41] <abaumann> "It is now possible[1] to perform chosen-prefix attacks against the
[10:33:41] <abaumann> SHA-1 hash algorithm for less than USD$50K. For this reason, we will
[10:33:41] <abaumann> be disabling the "ssh-rsa" public key signature algorithm that depends
[10:33:41] <abaumann> on SHA-1 by default in a near-future release.
[10:33:42] <abaumann> "
[10:34:05] <abaumann> so, the key is not yet deprecated
[10:34:13] <girls> yes
[10:34:19] <abaumann> aha
[10:34:28] <abaumann> though, it's probably a good thing to change it now.
[10:38:30] <abaumann> oh, cool. ghc Haskell breaks with new gcc
[10:46:29] <nit-picker> abaumann: your slave eurobuild6-4 builds pentium4/sn0int for more than a day, now (1 day(s) 04:16:19)
[10:51:54] * buildmaster resumes sanity.
[12:24:00] -!- abaumann has quit [Quit: leaving]
[13:24:04] -!- abaumann has joined #archlinux32
[13:24:04] <buildmaster> Hi abaumann!
[13:24:04] <buildmaster> !rq abaumann
[13:24:04] <phrik> buildmaster: <abaumann> don't make serious jokes ;-)
[13:24:50] <abaumann> ghc 8.8.2 from upstream staging builds fine. I accidently triggered ghc with the new gcc, so that's why we have that many build errors. We can just wait for ghc 8.8.2 hitting stable..
[13:24:56] -!- abaumann has quit [Client Quit]
[13:54:44] -!- samantaz has joined #archlinux32
[14:42:29] -!- buildmaster has quit [Remote host closed the connection]
[14:42:29] -!- trotz has quit [Read error: Connection reset by peer]
[14:43:40] -!- trotz has joined #archlinux32
[14:43:40] -!- trotz has quit [Read error: Connection reset by peer]
[14:43:49] -!- trotz has joined #archlinux32
[14:43:50] <trotz> 2020/02/16 07:38 CRIT buildmaster OS updates 1 updates, 0 ignored
[14:43:52] -!- buildmaster has joined #archlinux32
[14:43:52] <buildmaster> !rq buildmaster
[14:43:53] <phrik> buildmaster: <buildmaster> I might be insane, but never confused ... ;-)
[15:09:30] <samantaz> Hello there!
[15:11:15] <samantaz> I was wondering: which kind of build system is used to craft all of ArchLinux32 packages, in an automated way?
[15:48:56] <buildmaster> pentium4/acpi_call is broken (says eurobuild6-4): https://archlinux32.org
[15:56:42] <buildmaster> pentium4/wireguard-arch is broken (says nlopc46): https://archlinux32.org
[15:59:15] <buildmaster> pentium4/wireguard-lts are broken (says eurobuild6-3): https://archlinux32.org
[16:03:36] <buildmaster> i686/virtualbox-modules-arch is broken (says nlopc46): https://archlinux32.org
[16:08:55] <buildmaster> pentium4/virtualbox-modules-arch is broken (says rechenknecht): https://archlinux32.org
[16:31:46] <buildmaster> i486/postgresql is broken (says eurobuild6-7-i486): https://archlinux32.org
[16:52:08] <girls> samantaz: have a look at https://git.archlinux32.org
[16:52:08] <phrik> Title: builder - Archlinux32 build system (at git.archlinux32.org)
[16:53:34] -!- isacdaavid has joined #archlinux32
[16:54:16] <samantaz> girls: thanks :)
[17:00:31] <samantaz> damn, it runs with mySQL ;_;
[17:06:39] <samantaz> Also, is there only one buildmaster? And does the slaves are running bare metal or in VMs/containers?
[17:26:30] <buildmaster> i486/htmldoc is broken (says nlopc46-i486bs1): https://archlinux32.org
[18:19:06] <girls> it's only one master and multiple slaves
[18:19:13] <girls> you can run the slave wherever you like
[18:19:15] <girls> and we do :-)
[18:19:24] <girls> e.g. bare metal, systemd containers and vms
[18:19:40] <girls> i486 slaves are in i486 vms for historical reasons
[18:19:46] <girls> for example
[18:21:43] <buildmaster> i486/broadcom-wl is broken (says nlopc46-i486bs0): https://archlinux32.org
[18:22:39] <buildmaster> i486/nvidia-390xx is broken (says nlopc46-i486bs1): https://archlinux32.org
[18:24:31] <samantaz> girls: ok, thanks
[19:12:14] <girls> samantaz: np
[19:12:48] <girls> if you like, you can also donate some computation power (e.g. a build slave) or some brain power (e.g. some lines of code for the build scripts)
[19:21:16] <samantaz> I'd love to, but I'm quite limited. All of my servers are Athlons 64 with 4G of RAM...
[19:21:48] <samantaz> Plus I've got a crappy 500k ADSL connection
[19:26:51] <girls> ok, np :-)
[19:42:54] <buildmaster> i686/linux-tools are broken (says eurobuild6-1): https://archlinux32.org
[19:44:10] <buildmaster> pentium4/linux-tools are broken (says eurobuild6-4): https://archlinux32.org
[21:34:48] -!- isacdaavid has quit [Quit: Leaving.]
[22:29:26] -!- Dimtree has joined #archlinux32
[22:33:06] <eschwartz> girls: does building libseccomp 2.4.2 for arch x86_64 and using that as a host machine allow i686 nspawn containers to work again?
[22:33:08] <eschwartz> w.r.t. https://bugs.archlinux.org
[22:33:09] <phrik> Title: FS#65523 : libseccomp blocks systemd and new time64 functions for 32-bit with glibc 2.31 (sytemd-nspawn) (at bugs.archlinux.org)
[23:08:59] -!- Dimtree has quit [Ping timeout: 272 seconds]