Lukas Wagner [Tue, 21 May 2024 13:31:44 +0000 (15:31 +0200)]
notify: use named template instead of passing template strings
The notification system will now load template files from a defined
location. The template to use is now passed to proxmox_notify, instead
of separate template strings for subject/body.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com> Tested-by: Max Carrara <m.carrara@proxmox.com> Reviewed-by: Max Carrara <m.carrara@proxmox.com>
so that we can explicitly pin just this key when doing intra-cluster SSH
connections. this works similar to the certificate cache we use for API
proxying, but without automatic invalidation, since node A doesn't have access
to node B's host key..
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com> Tested-by: Hannes Duerr <h.duerr@proxmox.com>
Lukas Wagner [Fri, 19 Jan 2024 10:30:45 +0000 (11:30 +0100)]
datacenter config: mark notification settings as deprecated/unused
These were part of the first version of the notification overhaul
which was already rolled out in pvetest. To avoid breakage for users
who may have used the version from pvetest, we do not remove them yet
and only mark them as unused or deprecated. They can be removed at
some point in the future.
Fiona Ebner [Tue, 9 Jan 2024 11:55:07 +0000 (12:55 +0100)]
cfs_write_file: fix accidental UTF-8 re-encoding
by correclty passing the $force_utf8 flag to
PVE::Tools::file_set_contents(). The idea was that only callers that
are ready will opt-in to the behavior.
When reading files with PVE::Tools::file_get_contents() or
ipcc_get_config(), the UTF-8 flag on the Perl string is not set, even
if the data is UTF-8. Such data would then be encoded a second time,
as reported in the community forum [0] and bug tracker [1].
Fixes: 90c824b ("cluster fs: allow to force UTF-8 encoding for cfs_write_file") Signed-off-by: Fiona Ebner <f.ebner@proxmox.com> Tested-by: Filip Schauer <f.schauer@proxmox.com>
Lukas Wagner [Tue, 14 Nov 2023 12:59:29 +0000 (13:59 +0100)]
notify: adapt to matcher based notification system
This commit removes the target paramters from all notify calls. Also,
the default 'mail-to-root' target is not added automatically any more
- this target will be added by an dpkg hook in the future.
Thomas Lamprecht [Fri, 17 Nov 2023 13:23:50 +0000 (14:23 +0100)]
cluster setup: drop max-size override for file_get_contents calls
We switched the default to match pmxcfs again (after its limit
increase for Proxmox VE 7) to 1024 * 1024 (= 1 MiB), so not required
to override the limit here, as we actually made it lower, not higher.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Sun, 29 Oct 2023 15:52:25 +0000 (16:52 +0100)]
fix # 4764: datacenter config: default MAC prefix to official Proxmox OUI
Proxmox Server Solutions GmbH got the OUI BC-24-11 assigned for a MA-L
(large) address block that provides 24 bits (16 777 216) of space
available for guests.
Use this as new default, effectively disallowing a fully random MAC,
and extend the descriptions with some background.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Lukas Wagner [Wed, 30 Aug 2023 12:37:54 +0000 (14:37 +0200)]
fix #4937: notify: write configuration files in utf8 encoding
Strings that are returned from the Rust implementation are encoded as
utf8. To avoid issues with certain special characters (e.g. german
umlauts), we also need to explicitly store the configuration files in
utf8 encoding.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
[FE: use flag parameter instead of encoding as a string] Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Lukas Wagner [Wed, 30 Aug 2023 12:37:53 +0000 (14:37 +0200)]
cluster fs: allow to force UTF-8 encoding for cfs_write_file
Since this function is used to store utf8-encoded strings that come
from Rust, we need to be able to save the file in proper utf8 encoding
as well.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
[FE: use flag parameter instead of encoding as a string] Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
The package contains the PVE::Notify. It is a very thin wrapper
around the Proxmox::RS::Notify module, feeding the configuration
from the new 'notifications.cfg' and 'priv/notifications.cfg' files
into it.
pvecm: updatecerts: check and report if we got a timeout
If timeout is reached, run_fork_with_timeout will warn "got timeout".
When processing triggers for pve-manager (because of ExecStartPre of
pveproxy.service invoking pvecm updatecerts) that warning can appear
in the apt output without any context (output of the forked sub
doesn't appear there). So give it some actual content using the
$got_timeout returned in wantarray list context of the
run_fork_with_timeout method.
Note that this could be seen as clash with the silent flag, but IMO
that is a bit misnamed, we mostly want to avoid outputting the info
messages from updatecerts_and_ssh, and that silent (i.e., quite) gets
convoluted with a no-error behavior seems also odd.
TBH., I actually thought about printing the "waiting for quorum" at
least once for the silent case too, but for now omitted it – lets
wait on actual feedback and then maybe restructure this completely.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Fiona Ebner [Fri, 30 Jun 2023 11:59:42 +0000 (13:59 +0200)]
pvecm: updatecerts: wait for quorum
Mostly useful for the updatecerts call triggered via the ExecStartPre
hook for pveproxy.service.
When starting a node that's part of a cluster, there is a time window
between the start of pve-cluster.service and when quorum is reached
(from the node's perspective). pveproxy.service is ordered after
pve-cluster.service, but that does not prevent the ExecStartPre hook
from being executed before the node is part of the quorate partition.
The pvecm updatecerts command won't do much without quorum. Generating
local (non-pmxcfs) files is still done before waiting on quorum.
In particular, it might happen that the base directories for observed
files will not get created during/after the upgrade from Proxmox VE 7
to 8 (reported in the community forum [0] and reproduced right away in
a virtual test cluster).
Waiting on quorum should highly increase the chances for successful
execution of the ExecStartPre hook.
[0]: https://forum.proxmox.com/threads/129644/
Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com> Signed-off-by: Fiona Ebner <f.ebner@proxmox.com> Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Fri, 30 Jun 2023 09:29:18 +0000 (11:29 +0200)]
pmxcfs: check all addresses from getaddrinfo to find non-loopback one
Make the resolution of the node name to a local non-loopback address
through getaddrinfo more flexible.
Don't just ask for one, but for all addresses, then loop through the
returned linked list until the first non-loopback one is found.
It seems that some cloud-init templates hosting providers, like e.g.
OVH use, are misconfigured in always adding 127.0.1.1 for the
hostname to /etc/hosts, even if another mapping for the hostname
exists already. This popped up in our community forum with the
upgrade to bookworm, so might be related to the cloud-init version
from Bookworm.
Some examples pointed out by Alexandre:
https://forum.proxmox.com/threads/129678/
https://forum.proxmox.com/threads/129669/#post-568172
https://forum.proxmox.com/threads/129398/#post-568290
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Dominik Csapak [Thu, 25 May 2023 10:17:43 +0000 (12:17 +0200)]
add cfg files for resource mapping
There might be more mapping types in the future, but for now it's:
- mapping/pci.cfg
- mapping/usb.cfg
Add them to both, the perl side (PVE/Cluster.pm) and pmxcfs side
(status.c).
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
[ TL: Dominik, Fabian and I decided to do a s/resource/mapping/
rename on the cfg & API paths and privileges, so ammend that ] Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Fri, 19 May 2023 15:29:08 +0000 (17:29 +0200)]
pmxcfs: migrate to g_memdup2
As the previously used g_memdup is deprecated since GLib 2.68, due to
being prone to integer overflows when converting the argument from a
gsize to a guint, while that only produces a warning we compile with
Werror, so this FTBFS.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Friedrich Weber [Thu, 9 Mar 2023 12:30:19 +0000 (13:30 +0100)]
pvecm add: require user to navigate out of /etc/pve
If `pvecm add` is issued from /etc/pve (or any subdirectory), it
prints some errors:
[...]
shell-init: error retrieving current directory: getcwd: cannot
access parent directories: Transport endpoint is not connected
[...]
successfully added node 'pve-c2' to cluster.
cannot fetch initial working directory: Transport endpoint is not
connected at /usr/share/perl5/PVE/CLI/pvecm.pm line 446.
The reason is that `pvecm add` restarts pmxcfs, which re-mounts the
fuse mount at /etc/pve, invalidating pvecm's working directory.
The error messages give the impression that something went wrong.
Indeed, the second error indicates the temporary directory is not
cleaned up. The cluster join itself actually works, though.
The issue could be fixed by chdir'ing to / in `pvecm add`. However,
the user's shell would still remain in the now-invalid /etc/pve,
potentially leading to confusing "transport endpoint not connected"
messages in future interactions.
To avoid this, require the user to chdir out of /etc/pve before
running `pvecm add`.
Signed-off-by: Friedrich Weber <f.weber@proxmox.com>
Dominik Csapak [Wed, 16 Nov 2022 15:47:58 +0000 (16:47 +0100)]
datacenter config: add options to control tag usage rights
By adding a 'user-tag-privileges' and 'registered-tags' option.
The former sets the general policy by which "normal" users (with just
'VM.Config.Options' on the respective guest) can set or delete tags
on a guest, and the latter is a list of registered tags only settable
by users with admin-like permission ('Sys.Modify' on '/'); it is
designed to be used as source/target for actions like backup jobs in
the future.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
[T: drop outdated stuff & reword/work commit message] Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Dominik Csapak [Wed, 16 Nov 2022 15:47:56 +0000 (16:47 +0100)]
cluster: add get_guest_config_properties
Akin to the singular get_guest_config_property, but with the
possibility to query a list of properties.
Uses the CFS_IPC_GET_GUEST_CONFIG_PROPERTIES introduced in the
previous patch.
Note that the same details apply w.r.t. parsing and permissions as
the singular variant, iow. one needs to take caution and filter
allowed guests views on call site when using this.
Dominik Csapak [Wed, 16 Nov 2022 15:47:55 +0000 (16:47 +0100)]
pmxcfs: add IPC call to get multiple guest config properties at once
Previously we used the existing IPC call just for getting the `lock`
property of virtual guests in the cluster resource API call, but for
the tag display we'll to get another one and calling this twice seems
rather non-ideal.
Thus introduce a successor: CFS_IPC_GET_GUEST_CONFIG_PROPERTIES
It allows one to get multiple properties from a single, or all
virtual guest in-memory configs in one go. Keep the existing IPC call
as is for backward compatibility and add this as separate, new one.
The new IPC command basically behaves the same as the previous
CFS_IPC_GET_GUEST_CONFIG_PROPERTY, but takes a list of properties
instead and returns multiple properties per guest.
The existing perl wrapper around the IPC call to get a single
property was switched over to use the new IPC call too, so we'll
be able to drop the old IPC command with the next major release if
nothing comes up.
= Benchmark =
== Setup ==
Proxmox VE in a VM with CPU type host (Intel 12700k) and 4 cores
10000 typical configs with both 'lock' and 'tags' set at the end, and
fairly long tags ('test-tag1,test-tag2,test-tag3') (normal VM with a
snapshot, ~ 1 KiB)
== Test ==
Average of 100 runs each with time in milliseconds
== Previous Results ==
num props total time time per iteration
1 1054.2 10.2
== Results with this Patch ==
num props total time time per iter function
2 1332.2 13.2 get_properties
1 1051.2 10.2 get_properties
2 2082.2 20.2 get_property, 2 separate calls
1 1034.2 10.2 get_property
So, a call with the new code for one property is the same as with the
old code, and adding a second property only adds a bit of additional
time (in this case ~30%).
projects / pve-cluster.git / log