Shannon Sterz [Thu, 8 Aug 2024 14:25:18 +0000 (16:25 +0200)]
verifier: add ability to verify with keyrings
some vendors don't just provide a single certificate but an entire
keyring for their repositories. apt can handle those gracefully, so
should we. this commit adds the ability to verify a repository's
signatures with a keyring.
we use `PacketParserEOF` to check if a stream of packets is likely a
single certificate or a keyring. if it is a keyring, we try to verify a
message with all certificates in the ring and only fail if no
certificate can verify the message.
Reported-by: Maximiliano Sandoval <m.sandoval@proxmox.com> Signed-off-by: Shannon Sterz <s.sterz@proxmox.com>
this ensures the X-Cargo-Built-Using (and soon, Static-Built-Using) substvars
are actually filled with contents, and allow to find out which rustc version
and dependency versions were used to build a particular binary package.
garbage collection currently is quite aggressive in removing all files
under the link_dir, which are not a hard-link to a checksum file.
removing directories that remain empty below the link_dir should thus
not too dangerous.
without this patch, removing a snapshot on a mirror, running gc there,
and syncing everything to a medium, leaves the medium with an
hierarchy of empty directories below the removed snapshot (the files
get cleaned up the directories remain).
using WalkDir::content_first() seems better than to check for
emptiness after each file-removal [0]
path.is_empty() checks for the empty-path, not an empty directory [0].
as the check that the path is below the link_dir happens anyways in
the if we can directly call std::fs::remove_dir (which is even safer
than the std::fs::remove_dir_all call used in pool::remove_dir()).
the oversight seems to have been in place since the intial commit. I
ran across the issue when removing many snapshots of a Debian Bookworm
repository, syncing this to a medium, and still having a vast amount
of empty directories left behind (as debian has one directory per
package), which in turn increases the sync run-time.
pool: drop superfluous check for impossible path combination
commit c598cb154ef3fa6e1a8af840ded40f713d973d3d changed the pool
layout to have the pool directory (.pool for a mirror) on the same
level as the link directory (instead of below), to enable pool-sharing
across multiple mirrors.
the condition will never be true, drop the if statement to avoid
confusion in the future.
Fixes the clippy lint
```
warning: the borrowed expression implements the required traits
--> src/medium.rs:143:9
|
143 | &statefile(base),
| ^^^^^^^^^^^^^^^^ help: change this to: `statefile(base)`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrows_for_generic_args
= note: `#[warn(clippy::needless_borrows_for_generic_args)]` on by default
```
Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Stefan Sterz [Fri, 15 Dec 2023 10:52:29 +0000 (11:52 +0100)]
docs: add an auto dark mode to the docs
this adds the dark mode from the proxmox backup server to the offline
mirror for a more consistent appearance of the documentation across
all products.
Signed-off-by: Stefan Sterz <s.sterz@proxmox.com> Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Stefan Sterz [Wed, 29 Nov 2023 14:51:15 +0000 (15:51 +0100)]
helper: improve handling of multiple keys when activating them
this commit fixes a behavior where pom would applied any subscription
key that matched the provided product. it did not check whether the
server id of the activated subscription matched the current system.
this commit fixes that and only allows applying subscriptions for the
current system.
it also adds a couple of ux improvements:
- the `offline-key` sub-command now does not require the `--product`
parameter anymore. if there are multiple keys with different
products for the same server we will try to activate them all. the
assumption is that the user added all keys intentionally (e.g. a
combo pbs+pve system) and would like to activate them all at once.
since this only makes the api more permissive this shouldn't be a
breaking change.
- if the `offline-key` sub-command encounters multiple subscription
keys with the same product and server id, it only activates the one
with the due date furthest in the future. this makes sense in a
scenario where a user simply adds new subscription keys to their
key medium without removing older ones (perhaps older subscriptions
haven't even expired just yet).
- the interactive `setup` sub-command now only offers keys that have a
matching server id. it also orders them in such a way that the top
most key for a given product has the next due date furthest in the
future.
Stefan Sterz [Tue, 21 Nov 2023 14:48:18 +0000 (15:48 +0100)]
add missing subscription setting for ceph enterprise repos
when setting up a ceph enterprise repo we didn't add a subscription
for it. this commit adds a pve subscription so that pom can properly
authenticate itself when mirroring the ceph enterprise repos.
Signed-off-by: Stefan Sterz <s.sterz@proxmox.com> Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Stefan Sterz [Mon, 17 Jul 2023 14:01:36 +0000 (16:01 +0200)]
add non-free-firmware to bookworm default components
this adds the non-free-firmware component introduced with debian
bookworm [1] to the default components for bookworm mirrors. since
this new component is a subset [2] of the previous "non-free"
component add it here too to keep the same set of packages available.
Stefan Sterz [Mon, 12 Jun 2023 13:37:15 +0000 (15:37 +0200)]
add support for bookworm enterprise ceph repo
the bookworm release of proxmox ve comes along with a new ceph
enterprise repo. this commit adds support for this new repo for
bookworm-based releases.
by making the --id parameter optional, and structuring the output accordingly.
since pools are per base-dir, GC only needs to run once per base-dir instead of
for each mirror entry.
fix #4632: allow escape hatches for legacy repositories
there are still repositories out there that are using things like DSA/RSA-1024
and SHA1, so let's allow POM users to opt into accepting those insecure
cryptographic parameters, but keep the default settings secure.
e.g., when encoutering a key that is self-signed with SHA-1 (which is not that
uncommon for non-distro repositories that have an old key), instead of the
following:
----8<----
Fetching Release/Release.gpg files
-> GET 'https://download.ceph.com/debian-quincy//dists/bullseye/Release.gpg'..
-> GET 'https://download.ceph.com/debian-quincy//dists/bullseye/Release'..
Verifying 'Release(.gpg)' signature using provided repository key..
Subkey of 08B73419AC32B4E966C1A330E84AC2C0460F3994 not bound: No binding signature at time 2022-10-17T22:41:10Z
Error: encountered 1 error(s)
---->8----
which only gives us a rought idea that something is wrong with a key signature,
we now get the following:
----8<----
Fetching Release/Release.gpg files
-> GET 'https://download.ceph.com/debian-quincy//dists/bullseye/Release.gpg'..
-> GET 'https://download.ceph.com/debian-quincy//dists/bullseye/Release'..
Verifying 'Release(.gpg)' signature using provided repository key..
Subkey of 08B73419AC32B4E966C1A330E84AC2C0460F3994 not bound: No binding signature at time 2022-10-17T22:41:10Z
Caused by:
0: Policy rejected non-revocation signature (PositiveCertification) requiring second pre-image resistance
1: SHA1 is not considered secure since 2023-02-01T00:00:00Z
Error: No valid signature found.
---->8----
which shows us that the key signature was rejected because it's SHA-1, and the
(default and currently only) policy doesn't allow that (anymore).
the output is also improved in case the Release file is signed multiple times
and none of the signatures are accepted.
Lukas Wagner [Thu, 19 Jan 2023 10:40:40 +0000 (11:40 +0100)]
fix #4445: mirror: subscription: add proxy support
This commit adds support for HTTP proxies, configurable via the
ALL_PROXY environment variable.
For example:
$ ALL_PROXY="localhost:3128" proxmox-offline-mirror mirror <...>
Note: `ureq` seems to use HTTP CONNECT for *all* connections, including
HTTP on port 80. Proxies need to be configured to allow that - Squid by
default allows CONNECT only for HTTPS on port 443.
similar to `proxmox-offline-mirror medium status <ID>`, but limited to
the information that is stored on the medium itself. this command can be
used to get a quick overview over what's on a medium, or for automated
setup of the contained repositories.
with a somewhat sensible default of filtering the games and debug
sections - which already reduces a mirror of PVE + Debian bullseye by
about 27% (105GB->77GB).
projects / proxmox-offline-mirror.git / log