Friedrich Weber [Mon, 5 Aug 2024 13:33:37 +0000 (15:33 +0200)]
ui: qemu: hardware: fix permission check for adding tpmstate volume
Previously, the "Add -> TPM State" menu item in the GUI was disabled
if the user did not have Sys.Console privileges. This deviated from
the permission check in the backend, which does not require
Sys.Console but (among others) VM.Config.Disk.
Fix this inconsistency by checking for VM.Config.Disk in the frontend
instead of Sys.Console.
Reported in enterprise support.
Signed-off-by: Friedrich Weber <f.weber@proxmox.com>
Lukas Wagner [Mon, 12 Aug 2024 08:36:06 +0000 (10:36 +0200)]
metrics: add /cluster/metrics/export endpoint
This new endpoint returns node, storage and guest metrics in JSON
format. The endpoint supports history/max-age parameters, allowing
the caller to query the recent metric history as recorded by the
PVE::PullMetric module.
The returned data format is quite simple, being an array of
metric records, including a value, a metric name, an id to identify
the object (e.g. qemu/100, node/foo), a timestamp and a type
('gauge', 'derive', ...). The latter property makes the format
self-describing and aids the metric collector in choosing a
representation for storing the metric data.
Some experiments were made in regards to making the format
more 'efficient', e.g. by grouping based on timestamps/ids, resulting
in a much more nested/complicated data format. While that
certainly reduces the size of the raw JSON response by quite a bit,
after GZIP compression the differences are negligible (the
simple, flat data format as described above compresses by a factor
of 25 for large clusters!). Also, the slightly increased CPU load
of compressing the larger amount of data when e.g. polling once a
minute is so small that it's indistinguishable from noise in relation
to a usual hypervisor workload. Thus the simpler, format was
chosen. One benefit of this format is that it is more or less already
the exact same format as the one Prometheus uses, but in JSON format -
so adding a Prometheus metric scraping endpoint should not be much
work at all.
The API endpoint collects metrics for the whole cluster by calling
the same endpoint for all cluster nodes. To avoid endless request
recursion, the 'local-only' request parameter is provided. If this
parameter is set, the endpoint implementation will only return metrics
for the local node, avoiding a loop.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
[WB: remove unused $start_time leftover from benchmarks] Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Lukas Wagner [Mon, 12 Aug 2024 08:36:05 +0000 (10:36 +0200)]
pvestatd: store subsystem status data in a shared cache
This commit adds a new module PVE::PullMetric. This module allows
us to store the status data of various subsystems, including status
data for the most recent pvestatd update loops. Right now, we
store 6 old generations - including the most recent values, that gives
70 seconds of stat history (based on a 10 second pvestatd update loop
interval).
This cache allows us to add support for pull-style metric collection
systems, be it Prometheus/OpenMetrics or some custom, JSON based
metric format.
This patch raises the required lib{proxmox,pve}-perl-rs version
requirements, since we need the new bindings for proxmox-shared-cache.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
[WB: actually bump *runtime* deps in d/control] Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Igor Thaller [Wed, 31 Jul 2024 12:14:09 +0000 (14:14 +0200)]
ui: dc summary: fix calculation of storage size
The issue is related to the 'Summary' tab under 'Datacenter' inside a
cluster. To get a steady reading of the storage size data, the
frontend requests the '/api2/json/cluster/resources' every three
seconds to retrieve the necessary data to calculate the used and total
storage size.
The problem occurs when a shared storage is defined and a node goes
offline. As the node is not online, it cannot report the shared
storage size (both used and total) back to the other nodes. The order
of the JSON response is not always the same, so it is possible that
the offline node will appear first. Consequently, the frontend will
display the wrong total and used storage. This is because the shared
storage data has both the maximum disk size and the used disk set to
zero when the node is offline. This causes the total and used space
data to be calculated and displayed incorrectly, leading to
fluctuations in the displayed percentage of used disk space.
To fix this, add a conditional check to skip the storage report if its
status is 'unknown' (regardless of if the storage is local or shared).
This prevents the unreliable data from being processed.
Reported-by: Friedrich Weber <f.weber@proxmox.com> Signed-off-by: Igor Thaller <igor.thaller@aon.at>
Stefan Hanreich [Fri, 22 Dec 2023 10:43:16 +0000 (11:43 +0100)]
sdn: vnets: hide irrelevant fields depending on zone type
Not all fields in the VnetEdit dialog are necessary for every zone
type. This lead to confusion for some users. Hide fields in the
VNetEdit dialog depending on which kind of zone is selected in order
to prevent potential confusion.
Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com> Tested-by: Theodor Fumics <theodor.fumics@gmx.net> Reviewed-by: Theodor Fumics <theodor.fumics@gmx.net>
ui: resource mappings: fix editing of mapping for non first node
when editing the pci mapping, we set the nodename of the pci/usbselector
to the selected node. At the same time we disable and hide the node
selector, but it still changes it's value to the 'first' node
(alphabetically sorted) and that triggers a change event.
To prevent that we accidentally set the node of the pci/usbselector
too, we need to check here if the field is disabled.
There seems to be a race when loading the nodes for the nodeselector
which leads to inconsistent behaviour, so this was only encountered for
the pciselector, but theoretically it could also happen for the
usbselector so adding that condition to both.
d/control: bump proxmox-widget-toolkit dependency to 4.1.4
We need
"utils: add mechanism to add and override translatable notification
event descriptions in the product specific UIs"
otherwise there is an error in the browser console.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com> Reviewed-by: Max Carrara <m.carrara@proxmox.com>
api: notification: add API for getting known metadata fields/values
This new API route returns known notification metadata fields and
a list of known possible values. This will be used by the UI to
provide suggestions when adding/modifying match rules.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com> Reviewed-by: Max Carrara <m.carrara@proxmox.com>
This allows us to access the backup job id in the send_notification
function, where we can set it as metadata for the notification.
The 'job-id' parameter can only be used by 'root@pam' to prevent
abuse. This has the side effect that manually triggered backup jobs
cannot have the 'job-id' parameter at the moment. To mitigate that,
manually triggered backup jobs could be changed so that they
are not performed by a direct API call by the UI, but by requesting
pvescheduler to execute the job in the near future (similar to how
manually triggered replication jobs work).
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com> Reviewed-by: Max Carrara <m.carrara@proxmox.com>
[ TL: fleece in d/control bump for guest-common now that the version
is known ] Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
By only setting properties that have changed, we can avoid potential
errors in the task.
For example, if one configures the "nosizechange" property on a pool,
to prevent accidental size changes, the task will now only error if
the user is actually trying to change the size.
Prior to this patch, we would always try to set all parameters, even if
they were the same value. In the above example, this would result in the
task ending in error state, as we are not allowed to change the size.
To disable size changing you can run the following command:
ceph osd pool set {pool} nosizechange 1
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> Reviewed-by: Christoph Heiss <c.heiss@proxmox.com> Tested-by: Christoph Heiss <c.heiss@proxmox.com>
Thomas Lamprecht [Mon, 22 Jul 2024 16:54:15 +0000 (18:54 +0200)]
ui: cephfs: disallow blank field for now required MDS ID
while the ID part was formerly added to the nodename, and thus could
be completely left out, it's now used as full ID, so it must not be
empty as otherwise one gets an error from the API.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Since the ID of an MDS cannot start with a number [0], we cannot just
use the hostname in all situations, as they are allowed to start with
a number.
By having an extra field for the MDS ID, we can check for that via a
regex. This field is filled with the hostname when the host on which
it should be installed is selected.
This means, we can remove the extra ID field, as additional MDS, and
their unique ID can be set with the new ID field.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com> Tested-by: Christoph Heiss <c.heiss@proxmox.com> Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>
[ TL: use camelCase for local serviceID variable ] Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Stefan Hanreich [Wed, 17 Jul 2024 13:06:03 +0000 (15:06 +0200)]
sdn: ipam: fix editing custom mappings
Currently custom mappings cannot be edited, due to them having no VMID
value. The VMID parameter was always sent by the frontend to the
update call - even if it was empty - leading to validation failure on
the backend. Fix this by only sending the vmid parameter when it is
actually set.
Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
Christoph Heiss [Wed, 17 Jul 2024 12:49:50 +0000 (14:49 +0200)]
www: utils: fix `maxcpu` validity check in render_hostcpu()
Comparing with Proxmox.Utils.render_cpu() seems just a slight oversight
in the condition. Fix it by aligning it with how it is done in
Proxmox.Utils.render_cpu() for consistency.
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Max Carrara [Mon, 1 Jul 2024 14:10:32 +0000 (16:10 +0200)]
ceph: tools: parse Ceph version in separate sub and update regex
The part of the `get_local_version` sub that's concerned with actually
parsing the Ceph version is factored into a separate sub
`parse_ceph_version`. That way the parsing logic can easily be reused.
Make the version regex more maintainable declaring it as a variable,
breaking it up and commenting it by using the x flag.
Also remove the part that parses our Debian revision (e.g. -pve1) from
the version, as we do not actually include that in our Ceph builds.
The part of the regex that parses the build commit hash is made
mandatory (remove '?' after its group).
Signed-off-by: Max Carrara <m.carrara@proxmox.com> Tested-by: Lukas Wagner <l.wagner@proxmox.com> Reviewed-by: Lukas Wagner <l.wagner@proxmox.com> Tested-by: Igor Thaller <igor.thaller@brg9.at>
Currently, the whole mode selector is labeled as experimental, this
does however give the impression that also the default legacy mode is
an experimental mode.
To clarify that only the `data` and `metadata` change detection modes
are experimental, move the experimental label to the individual
modes and explicitly mention the experimental modes in the message.
Also, make it more clear that the archive encoding format depends on
the selected mode.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Lukas Wagner [Mon, 10 Jun 2024 08:40:25 +0000 (10:40 +0200)]
vzdump: apt: notification: do not include domain in 'hostname' field
- The man page warns about the usage of `hostname -f`, since a host
may have multiple domains (or none at all)
- The fallback PVE::INotify::nodename() already only returned the
hostname without the domain part
- Fencing notifications didn't include the domain part anyway
This may result in soft-breakage for any users who have already relied
on the domain being present. If there is need for it, it could include
a fqdn metadata field.
The hostname property used for rendering the notification template
is unaffected for now.
Fiona Ebner [Tue, 28 May 2024 10:59:23 +0000 (12:59 +0200)]
pve7to8: allow arbitrary newer running '-pve' kernels after upgrade
As recently reported in the community forum [0], 6.8 pve kernels would
not be detected correctly by the script. Allow arbitrary newer
versions if already upgraded for future-proofing.
Fiona Ebner [Wed, 26 Jun 2024 14:32:30 +0000 (16:32 +0200)]
vzdump: fix unit for bandwidth limit in log message
The documentation 'man vzdump' states that the value is in KiB/s. This
is correct, as seen in the plugin implementations, where the value is
multiplied by 1024.
'stretch' is most often the wrong value, as that will stretch
everything, to the height of the whole container, including fields.
That is not desirable, since fields look not good when stretched this
way (e.g. the controls are not correctly aligned).
Fiona Ebner [Fri, 3 May 2024 11:19:55 +0000 (13:19 +0200)]
ui: qemu: hardware: use background delay for asynchronous remove tasks
Avoids spawning a progress window for tasks that do complete more
quickly than the background delay.
Currently, the remove task is only asynchronous (i.e. using POST) when
it's for an unused disk, but this might change in the future (e.g. for
hot-unplug).
When adding a disk, a background delay of 5 seconds is already used.
Christian Ebner [Mon, 10 Jun 2024 09:57:46 +0000 (11:57 +0200)]
vzdump: add pbs-change-detection-mode to config template
Include the additional parameter to set the `change-detection-mode`
for backup jobs with Proxmox Backup Server target as node wide
configuration, including possible variants to be set.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
The proxmox backup client allows to switch the method used to encode
data based on a change-detection-mode parameter. Expose this setting
as experimental feature in the advanced panel for a backup job.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Lukas Wagner [Tue, 21 May 2024 13:31:47 +0000 (15:31 +0200)]
tests: remove vzdump_notification test
With the upcoming changes in how we send notifications, this one
really becomes pretty annoying to keep working. The location where
templates are looked up are defined in the proxmox_notify crate, so
there is no easy way to mock this for testing.
The test itself seemed not super valuable, mainly testing if
the backup logs are shortened if they ware too long - so they are just
removed.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com> Tested-by: Max Carrara <m.carrara@proxmox.com> Reviewed-by: Max Carrara <m.carrara@proxmox.com>
Thomas Lamprecht [Tue, 23 Apr 2024 17:51:26 +0000 (19:51 +0200)]
ui: importer: try to better convey what live-import does
It's hard to cram a easy to understandable meaning in the space we
have, to get a bit more space move the warning hint to a separate line
and use the box-label to show an always visible hint about the VM to
be stopped previously.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
fix #5251: login: set autocomplete on password and user
By default they have 'autocomplete=off'. From [1]:
> In most modern browsers, setting autocomplete to "off" will not
> prevent a password manager from asking the user if they would like to
> save username and password information, or from automatically filling
> in those values in a site's login form. See the autocomplete
> attribute and login fields [2].
Fiona Ebner [Fri, 9 Feb 2024 13:08:19 +0000 (14:08 +0100)]
ui: user edit: protect user's TFA settings again
Same rationale as in 5b25580d ("Protect the user's tfa key setting."):
it should not be possible to change the value when it's not an actual
secret but a reference to what TFA method is used or, in case of 'x',
whether TFA is used.
Dominik Csapak [Thu, 14 Dec 2023 09:55:17 +0000 (10:55 +0100)]
ui: mobile: enable subscription popup
not sure if this was lost at some point or never implemented, but we
want to be consistent with the remaining web ui and apps, so show
the subscription popup here too.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com> Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Dominik Csapak [Thu, 14 Dec 2023 09:55:16 +0000 (10:55 +0100)]
ui: mobile: fix totp login
Log-in with TOTP enabled account on mobile was broken due to these two
commits:
- pve-manager: 509d7a20 ("mobile ui: implement dummy message box and
scrip loader")
- pve-access-control: cb64967 ("api: drop old verify_tfa api call")
The pve-manager one overwrote the Ext.MessageBox and Ext.Msg classes
and thus removed the Ext.MessageBox.OKCANCEL constant that represented
the buttons of popup messages (without those no buttons on message
boxes where shown).
This override did not work as intended, as we still showed the
message box by accident, because at that point the Ext.MessageBox was
already initialized (so it was overwritten), but Ext.Msg was not (this
happens later).
And the pve-access-control removed the old tfa verify api (which is
now done via the /access/ticket api)
So to fix that, we have to adapt to the api changes and restore the
stock Ext.MessageBox and Ext.Msg classes by removing the overrides
(i couldn't find where we would need those)
We still cannot handle u2f/WebAuthn or recovery methods though.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com> Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
ui: backup job editor: fix disable behavior for fleecing storage
commit 569b0388 (ui: fix reset behavior of backup job editor) disabled
the fleecing storage field by default (as that is the default state)
to fix the reset behavior. This broke editing the job when fleecing
was enabled and the user did not navigate to the advanced tab yet.
It seems that the 'bind' here only gets triggered once the panel is
rendered, but we actually need it before that.
To work around the issue for now, manually enable/disable the field
when toggling the fleecing checkbox. (Though this warrants a bit of
deeper investigation into this bind behavior)
when we `bind` we also have to set the initial value correctly,
otherwise the form dirty tracking is off (the initial bind set does not
reset the `originalValue`)
also the bandwidth selector auto transformed the value `null` to `0`
when there was no initial transformation. Since this is not a valid
value anyway, skip that.
Markus Frank [Mon, 15 Apr 2024 08:50:01 +0000 (10:50 +0200)]
ui: machine: add viommu ComboBox
Added a proxmoxKVComboBox for selecting a vIOMMU implementation for a VM.
If i440fx is selected, another ComboBox will be enabled/visible that does not
have the Intel option, as Intel-vIOMMU is not compatible with i440fx.
Uses the new machine property-string from the qemu-server's "config: define
machine schema as property-string" commit and the viommu option added in the
qemu-server's "fix #3784: config: Parameter for guest vIOMMU + test-cases"
commit.
acme: ui: handle missing meta field in directory response
When none of the meta fields is set by the directory, the whole
dictionary is missing from the response, leading to an exception
when testing for fields inside it.
Thomas Lamprecht [Mon, 22 Apr 2024 09:24:52 +0000 (11:24 +0200)]
ui: backup job: rework empty-text for advanced fields again
This partially reverts commit a32a5c4a6 ("ui: backup job: rework hint
about fallback config and make it less flashy"), i.e., the part about
the fallback values, as those was barely visible now.
Add the schema default to the end of the description and expand the
hint at the bottom to also mention that this is used as second level
fallback, if the vzdump.conf does not has the option set.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
ui: backup job: correctly align descriptions with fields in advanced options
Merges the column1/2/B into just single items so that the vertical
alignment is still correct even if a description wraps over multiple
lines.
Use the new pveTwoColumnContainer to achieve this without extra
boilerplate code and use a 1/3 of the width for the field and the 2/3
rest for the description.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
[ TL: adapt to changes in prev. commit, reword message, fix eslint ] Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Sun, 21 Apr 2024 11:01:53 +0000 (13:01 +0200)]
d/control: bump versioned dependency for widget-toolkit and common
To ensure that the lifting of the bridge name == vmbr\d+ restriction
works correctly and that the new notes view double-click editing
setting can work.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
ui: dc: backup: improve UX for the different 'notification-mode's
- Switch order of 'mailto' and 'mailnotification' field
- When mode is 'auto', disable 'mailtnotification' field
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com> Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
[ TL: drop the hint, not really explaining much as is so mostly
visible noise ] Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Sat, 20 Apr 2024 18:15:03 +0000 (20:15 +0200)]
ui: guest stop: show overrule checkbox also if no task is active
The UI state about running tasks can be out of sync, especially for
situations where one quickly follows up with a stop, e.g. after
triggering a shutdown by mistake.
So, show the checkbox always for users that got Sys.Modify on (some)
node, but pre-check it still only if there where task detected on
component creation (we could watch the state though and show a hint,
but that's a bit over the top IMO).
Show it also when HA is enabled but explicitly disable it there,
hopefully this increases the chance that the users can understand that
this is done by design, and isn't a bug – ideally we would also show
an extra hint.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Friedrich Weber [Fri, 12 Apr 2024 14:15:53 +0000 (16:15 +0200)]
fix #4474: ui: guest stop: offer to overrule active shutdown tasks
Implement a new "guest stop" confirmation message box which first
checks if there is an active shutdown task for the same guest that is
visible to the logged-in user. If there is at least one, the dialog
displays an additional default-on checkbox for overruling active
shutdown tasks. If the user confirms and the checkbox is checked, the
UI sends a guest stop API request with the `overrule-shutdown`
parameter set to 1. If there are no active shutdown tasks, or the
checkbox is unchecked, the UI sends a guest stop API request without
`overrule-shutdown`.
To avoid an additional API request for querying active shutdown tasks,
check the UI's current view of cluster tasks instead, which is fetched
from the `pve-cluster-tasks` store.
As the UI might hold an outdated task list, there are some
opportunities for races, e.g., the UI may miss a new shutdown task or
consider a shutdown task active even though it has already terminated.
These races either result in a surviving shutdown task that the user
still needs to abort manually, or a superfluous `override-shutdown=1`
parameter that does not actually abort any tasks. Since "stop
overrules shutdown" is merely a convenience feature, both outcomes
seem bearable.
The confirmation message box is now always marked as dangerous (with a
warning sign icon), whereas previously it was only marked dangerous if
the stop issued from the guest panel, but not when issued from the
resource tree command menu.
Signed-off-by: Friedrich Weber <f.weber@proxmox.com> Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
[ TL: squash in some slightly opinionated code/style clean-ups ] Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Stefan Hanreich [Fri, 19 Apr 2024 09:42:37 +0000 (11:42 +0200)]
firewall: expose configuration option for new nftables firewall
There's a new firewall implementation available as `proxmox-firewall`
package, in contrast to the existing `pve-firewall` package it is
using nftables directly, not the legacy iptables, and can thus
leverage a modern stack with atomic updates, avoiding the need for
different tools (e.g., ebtables), and not requiring intermediate
firewall bridges to handle VM flow correctly. Additionally it's
written in rust, making it more efficient and safer to change.
The new implementation is using the same configuration file as source
and should be mostly the same in semantic behavior, it basically is a
drop-in replacement besides one known issue:
There is currently one major issue that we still need to solve:
REJECTing packets from the guest firewalls is currently not possible
for incoming traffic (it will instead be dropped).
This is due to the fact that we are using the postrouting hook of
nftables in a table with type bridge for incoming traffic. In the
bridge table in the postrouting hook we cannot tell whether the packet
has also been sent to other ports in the bridge (e.g. when a MAC has
not yet been learned and the packet then gets flooded to all bridge
ports). If we would then REJECT a packet in the postrouting hook this
can lead to a bug where the firewall rules for one guest REJECT a
packet and send a response (RST for TCP, ICMP port/host-unreachable
otherwise).
While this is being addressed, and the whole stack is better tested in
general, the new FW will be only enabled if the admin enables a
boolean configuration which this patch exposes on the UI.
Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
d/control bump versioned dependency for libpve-(guest-)common-perl and qemu-server
to make the backup fleecing feature available. The bump for
qemu-server is also required for moving unused disks of VMs.
The bump for libpve-common-perl is required because of pve-common
commit c302a28 ("json schema: add format description for
pve-storage-id standard option"), which is required for API
verification.
projects / pve-manager.git / log