Thomas Lamprecht [Tue, 30 Jul 2024 18:21:40 +0000 (20:21 +0200)]
cherry-pick fixes for parsing the version of modern cryptsetup
Recent 18.2.4 release contained a cherry-pick of 0985e201342
("ceph-volume: use 'no workqueue' options with dmcrypt") and that
patch introduced parsing the output of `cryptsetup --version`, but it
had a coupling on either a old (or distro-specific) cryptsetup version
output and/or some legacy behavior of the python `packaging` module that
is used for the version parsing.
As the `cryptsetup` tool on bookworm outputs the following version:
> cryptsetup 2.6.1 flags: UDEV BLKID KEYRING KERNEL_CAPI
As the extra strings at the end are not accepted anymore by the
`packaging` python module in bookworm [0], this test fails ceph-volume when
encrypted OSDs are used, which we do by default.
[0]: due to https://github.com/pypa/packaging/pull/407 being included
in the bookworm version
To make this work again cherry-pick two patches that first filter out
the numerical part from the raw version output using a regex and only
pass that to the version parsing call.
Fixes: https://tracker.ceph.com/issues/66393 Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Tue, 30 Jul 2024 18:17:50 +0000 (20:17 +0200)]
normalize downstream patches
with a cycle through `git am -3 ...` in the upstream repo, `rm
*.patch` here, and `git format-patch --zero-commit --no-signature
--diff-algorithm=myers --no-numbered -o ...` in the upstream repo
again.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Max Carrara [Tue, 5 Mar 2024 15:07:44 +0000 (16:07 +0100)]
reorder keyring used by ceph-crash favoring non-host-specific keyring
This patch makes it so that `ceph-crash` attempts to use the
non-host-specific keyring before anything else, which avoids
unnecessary error messages landing in the systemd-journal in our case.
Signed-off-by: Max Carrara <m.carrara@proxmox.com> Reviewed-by: Fabian Grünbichler <f.gruenbichler@proxmox.com> Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Max Carrara [Tue, 5 Mar 2024 15:07:43 +0000 (16:07 +0100)]
d/postinst: ensure all ceph state files/dirs have correct owner
Ceph has a postinst hook that sets the ownership of '/var/lib/ceph/*'
to ceph:ceph (in our case), but misses out on the contents of
'/var/lib/ceph/crash'.
This patch therefore also recursively updates the permissions of
'/var/lib/ceph/crash'.
The change was also proposed upstream [0].
[0]: https://github.com/ceph/ceph/pull/55917
Signed-off-by: Max Carrara <m.carrara@proxmox.com> Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Friedrich Weber [Thu, 15 Feb 2024 09:40:54 +0000 (10:40 +0100)]
fix #5213: ceph-osd postinst: add patch to avoid connection freezes
Assume there is an open TCP connection to a VM, and ceph-osd is
installed/upgraded on the host on which the PVE firewall is active.
Currently, ceph-osd postinst reloads all sysctl settings. Thus,
installing/upgrading ceph-osd will set the sysctl setting
`net.bridge.bridge-nf-call-iptables` to 0. The PVE firewall will flip
the setting back to 1 in its next iteration (in <10 seconds). But
while the setting is 0, conntrack will not see packets of the existing
TCP connection. When the setting is flipped back to 1, conntrack will
see packets again, but may consider the seq/ack numbers of new packets
out-of-window, mark them as invalid and drop them. This will freeze
the TCP connection.
To avoid this, add a patch that modifies the ceph-osd postinst to only
apply settings from the sysctl settings file shipped with ceph-osd,
and only apply them on fresh install. As the ceph-osd sysctl settings
do not set `net.bridge.bridge-nf-call-iptables`, this will avoid the
temporary flip to 0 when installing/upgrading ceph-osd.
Signed-off-by: Friedrich Weber <f.weber@proxmox.com>
Max Carrara [Fri, 26 Jan 2024 15:27:02 +0000 (16:27 +0100)]
patch: fix `ceph dashboard` subcommand becoming unavailable on crash
Adapt the patch that originally disabled certain TLS checks during the
dashboard's startup and fixes the `ceph dashboard` subcommand becoming
unavailable if the dashboard crashes during that time.
This is achieved by re-implementing certain checks and also re-raising
any other unforeseen exceptions that occur in regards to TLS as one
of Ceph's internal exception types, which are then handled by the
dashboard itself. This is akin to how these cases were handled
originally.
Also fixes a typo in the `ceph dashboard create-self-signed-cert`
command output.
Signed-off-by: Max Carrara <m.carrara@proxmox.com>
Thomas Lamprecht [Mon, 15 Jan 2024 15:50:26 +0000 (16:50 +0100)]
drop superfluous build-type patch and re-order other one
As Fabian correctly noticed, from the two PR's, namely PR #54918[0]
and PR #54891[1], only the first one is necessary, that's why the
second one was closed upstream, so drop it here too to avoid a
unnecessary divergence from upstream.
Max Carrara [Fri, 5 Jan 2024 14:07:33 +0000 (15:07 +0100)]
mgr/dashboard: add patch that removes PyOpenSSL-related usages
This patch allows the dashboard to work again with TLS enabled; it
however disables the possibility to create self-signed certs via the
`ceph` CLI. This means that users will have to supply the correct
key/cert pair themselves, which are just a few extra steps instead. [0]
Users that try to generate a self-signed cert via the `ceph` CLI are
instead provided with instructions on how to generate and configure a
key/cert pair themselves.
Additionally, the check whether the cert and key match is removed during
the dashboard's launch.
Max Carrara [Fri, 5 Jan 2024 14:07:32 +0000 (15:07 +0100)]
mgr/dashboard: add backport that allows the dashboard to work again
After upgrading from PVE 7 to PVE 8, some users noted that the Ceph
Dashboard does not work anymore. [0] A user from our community
provided a pull request [1] which removes a dependency to `PyJWT`
(Python). This commit adds a backport of this PR as a single patch.
This patch by itself however does not yet allow the dashboard to run
with TLS enabled.
cherry-pick fix so rocksdb build inherits parent's CMAKE_CXX_FLAGS
cherry-pick both, beb1a624921 ("cmake/modules/BuildRocksDB.cmake:
inherit parent's CMAKE_CXX_FLAGS") and 620b68a348f
("cmake/modules/BuildRocksDB.cmake: use string(APPEND ..) when
appropriate")
Stefan Hanreich [Mon, 18 Sep 2023 15:46:56 +0000 (17:46 +0200)]
add stop-gap to fix compat with CPUs not supporting SSE 4.1
Some of our users ran into issues with running Ceph on older CPU
architectures [1]. This is apparently due to a bug in gf-complete
paired with gcc-12, that
leads to SSE 4.1 instructions being emitted in the general code,
rather than dynamically dispatching functions using those
instructions. Those binaries then break on older CPUs that do not
support this instruction set.
I've ran some benchmarks with `rados bench` against our last release
(18.2.0-pve2) and this new version. The commands were taken from our
latest Ceph benchmarking paper [2]. The results showed that this patch
does not lead to performance regressions on newer hardware.
18.2.0-pve2 this patch
Read EC 4574.28 4651.95
Write EC 3739.59 3773.87
Read Replicated 5345.34 5568.41
Write Replicated 4123.28 4066.19
(numbers correspond to bandwidth in MB/s)
Thomas Lamprecht [Thu, 20 Jul 2023 07:16:57 +0000 (09:16 +0200)]
buildsys: change download over to reef release
use Ubuntu 22.04 Jammy as distro as it seems Ceph does not (yet?)
provide a source release for any modern Debian based distro, not that
it should matter much for the source package.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
cherry-picked from Debian boost 1.74 package [0], adapted to the
bundled 1.75 boost included by ceph. Note that 1.75 has the fopen
compat patch already included.
commit c3a53010af432e77f74bcb46e7205c5500b6af77 left this over empty,
highly probably because the author does not understand debian
packaging - fix that, we want installed docs after all...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>