CFG tracking was removed in 815375411e80131f31b172d7c43625769ee8b53d,
probably as a mistake, as part of the tidy up of cfg and the removal of
dynamic loading. This means that shutdown tracking (using
cfg_try_shutdown()) stopped working.
This patch restores the trackstart & trackstop API calls (renamed to be
more consistent with the exiting libraries) so that shutdown tracking
can be used again.
Change cfg.shutdown_timeout to be in milliseconds rather than seconds
nd use libqb macros for conversion.
Add --force option to corosync-cfgtool -H
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Jan Friesse [Tue, 24 Nov 2020 11:20:25 +0000 (12:20 +0100)]
cfg: Improve nodestatusget versioning
Patch tries to make nodestatusget really extendable. Following changes
are implemented:
- corosync_cfg_node_status_version_t is added with (for now) single
value CFG_NODE_STATUS_V1
- corosync_knet_node_status renamed to corosync_cfg_node_status_v1 (it
isn't really knet because it works as well for udp(u()
- struct res_lib_cfg_nodestatusget_version is added which holds only ipc
result header and version on same position as for
corosync_cfg_node_status_v1
- corosync_cfg_node_status_get requires version and pointer to one of
corosync_cfg_node_status_v structures
- request is handled in case switches to make adding new version easier
Also fix following bugs:
- totempg_nodestatus_get error was retyped to cs_error_t without any
meaning.
- header.error was not checked at all in the library
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Jan Friesse [Tue, 10 Nov 2020 17:10:17 +0000 (18:10 +0100)]
totemknet: Check both cipher and hash for crypto
Previously only crypto cipher was used as a way to find out if crypto is
enabled or disabled.
This usually works ok until cipher is set to none and hash to some other
value (like sha1). Such config is perfectly valid and it was not
supported correctly.
As a solution, check both cipher and hash.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Fabio M. Di Nitto <fdinitto@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Ferenc Wágner [Sun, 8 Nov 2020 19:49:15 +0000 (20:49 +0100)]
The ring id file needn't be executable
At the same time simplify the overwrite logic and stop clearing the
umask (which is unexpected and quite pointless here, as applications
can't really protect the users from their own pathological settings).
Signed-off-by: Ferenc Wágner <wferi@debian.org> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Jan Friesse [Mon, 12 Oct 2020 12:26:10 +0000 (14:26 +0200)]
config: Increase default token timeout to 3000 ms
Default token timeout of 1000 ms was often changed by users because of
other workloads on machine which may make corosync responding a bit
later than needed and resulting in token loss.
3000 ms was chosen as a compromise between token timeout increase
and allow live cluster upgrade (other nodes should receive token
by node with new default on time).
It doesn't affect token token_coefficient so final token timeout still
depends on number of configured nodes (just base is higher).
This change slows down failover a bit so for clusters where failover
times are important, please change the token timeout in configuration
file corosync.conf as a:
totem {
version: 2
token: 1000
...
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Jan Friesse [Tue, 29 Sep 2020 16:44:44 +0000 (18:44 +0200)]
tests: Use CS_DISPATCH_BLOCKING instead of cycle
Some tests were using dispatch function in CS_DISPATCH_ALL mode
without poll/select on fd. This leads to busywait cycle, because
CS_DISPATCH_ALL masks CS_ERR_TRY_AGAIN error.
Simpliest solution is to use CS_DISPATCH_BLOCKING instead and remove
while cycle, because CS_DISPATCH_BLOCKING handles CS_ERR_TRY_AGAIN
correctly.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Jan Friesse [Wed, 9 Sep 2020 12:16:55 +0000 (14:16 +0200)]
quorum: Add support for nodelist callback
Current quorum callback contains only actual view list and there is no
way how to find out joined/left nodes. This cannot be emulated by user
app, because when corosync restarts before other nodes notices then view
list is unchanged (ring id is changed tho).
Solution is to implement similar callback as for cpg which contains ring
id, member list, joined list and left list.
To implement such callback and keep backwards compatibility,
quorum_model_initialize is introduced. Its behavior is similar to
cpg_model_initialize. This allows passing model v1, which contains
enhanced quorum (full ring id is passed instead of just seq number)
and nodelist callbacks.
To find out which events should be sent by corosync daemon, new message
MESSAGE_REQ_QUORUM_MODEL_GETTYPE is used. Quorum library on init was
sending MESSAGE_REQ_QUORUM_GETTYPE. Whem model v1 is requested the
MESSAGE_REQ_QUORUM_MODEL_GETTYPE is used, which contains model number
so corosync knows that client is using model v1 and can send enhanced
quorum and nodelist events.
Nodelist event is (for now) send both in case of change of membership
and also when requested, also when CS_TRACK_CURRENT is requested, but
then left_list and joined_list is left empty, because they don't make
too much sense there.
New test application testquorummodel is added as an example of new API
usage.
Also during patch developement, I found few bugs here and there, which
are also fixed:
- quorum_initialize was never returning error code returned by
MESSAGE_REQ_QUORUM_GETTYPE call (always returned CS_OK)
- Allocated memory in send_library_notification was based
on sizeof(unsigned int) instead of mar_uint32_t. That's not wrong,
but it make more sense to use sizeof(mar_uint32_t) instead
(big thanks to Chrissie for englishify the man pages)
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
liangxin1300 [Fri, 7 Aug 2020 16:50:29 +0000 (00:50 +0800)]
tools: use util_strtonum for options checking
Function atoi is not safe since miss validation;
Function strtol is better but need to consider empty string and overflows
Function util_strtonum is a safer wrapper of strtoll
Use util_strtonum to check nodeid option and strict checking condition.
Signed-off-by: liangxin1300 <XLiang@suse.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Jan Friesse [Thu, 16 Jul 2020 13:50:42 +0000 (15:50 +0200)]
build: Use git-version-gen during specfile build
Instead of copying parts of git-version-gen for spec target use
git-version-gen directly and parse final version into components
(rpmver, alphatag, numcomm) and use them.
Main reason is to simplify code a bit (sed scripts are a bit repetitive
tho), reuse the code and also allow building of RPM from dist tarball
generated from non-tagged commit or dirty git (not very useful).
The code relies on fact, that hyphen is never used in tagged release
name.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Fabio M. Di Nitto <fdinitto@redhat.com>
If it's not available, then fall back to the old
API and forbid changing crypto while running.
To avoid us being dependant on the leader node, each
node sends its own crypto_reconfig_phase messages so
we can guarantee that the reconfiguration always completes
on each node.
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
The basic plan here is to fix reloads to be more stable
using read/parse/verify/build/commit stages, so that any errors
will not leave corosync in an unstable state. This should
also make the code more maintainable as currently the verify/commit
stages are horribly intertwined.
Also:
- Fix local_node_pos not being updated in the new map during validation
(broke adding and removing new nodes in the middle of the list).
- Fix reconfiguration so that nodes are indexed by nodeid and not their
position in the list. This is an old bug that's just been carried
over
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Jan Friesse [Wed, 22 Apr 2020 11:28:57 +0000 (13:28 +0200)]
Revert "totemip: Add support for sin6_scope_id"
This reverts commit 934c47ed4384daf2819c26306bebba3225807499 which is
causing protocol incompatibility in needle. Master seems to be not
affected, but it needs more checking.
Jan Friesse [Tue, 10 Mar 2020 16:49:27 +0000 (17:49 +0100)]
votequorum: set wfa status only on startup
Previously reload of configuration with enabled wait_for_all result in
set of wait_for_all_status which set cluster_is_quorate to 0 but didn't
inform the quorum service so votequorum and quorum information may get
out of sync.
Example is 1 node cluster, which is extended to 3 nodes. Quorum service
reports cluster as a quorate (incorrect) and votequorum as not-quorate
(correct). Similar behavior happens when extending cluster in general,
but some configurations are less incorrect (3->4).
Discussed solution was to inform quorum service but that would mean
every reload would cause loss of quorum until all nodes would be seen
again.
Such behaviour is consistent but seems to be a bit too strict.
Proposed solution sets wait_for_all_status only on startup and
doesn't touch it during reload.
This solution fulfills requirement of "cluster will be quorate for
the first time only after all nodes have been visible at least
once at the same time." because node clears wait_for_all_status only
after it sees all other nodes or joins cluster which is quorate. It also
solves problem with extending cluster, because when cluster becomes
unquorate (1->3) wait_for_all_status is set.
Added assert is only for ensure that I haven't missed any case when
quorate cluster may become unquorate.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Jan Friesse [Wed, 4 Mar 2020 10:42:15 +0000 (11:42 +0100)]
votequorum: Change check of expected_votes
Previously value of new expected_votes was checked so newly computed
quorum value was in the interval <total_votes / 2, total_votes>. The
upper range prevented the cluster to become unquorate, but bottom check
was almost useless because it allowed to change expected_votes so it is
smaller than total_votes.
Solution is to check if expected_votes is bigger or equal to total_votes
and for quorate cluster only check if cluster doesn't become unquorate
(for unquorate cluster one can set upper range freely - as it is
perfectly possible when using config file)
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Jan Friesse [Tue, 25 Feb 2020 14:17:05 +0000 (15:17 +0100)]
man: Enhance link_mode priority description
Some users found description of priority for passive link_mode
confusing (probably because "priority" word is too
overloaded) so add some redundancy to make description
unambiguous.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
liangxin1300 [Thu, 20 Feb 2020 05:38:49 +0000 (13:38 +0800)]
totemip: compare sin6_scope_id and interface_num
When user configure a specific interface like vlan
with the same IPv6 link-local address, Corosync should
compare sin6_scope_id with interface_num, to make sure got
the right interface to bind
Signed-off-by: liangxin1300 <XLiang@suse.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Jan Friesse [Fri, 14 Feb 2020 08:42:08 +0000 (09:42 +0100)]
totemip: Add support for sin6_scope_id
sin6_scope_id was not present in totemip structure making impossible to
use link-local ipv6 address.
Patch adds sin6_scope_id and changes convert/copy functions to use it
(formally also comparator functions should be changed, but it seems to
cause more harm and it is not really needed).
This makes corosync work with link-local addresses fine for both UDPU
and Knet transport as long as interface specification is used (so
fe80::xxxx:xxxx:xxxx:xxxx%eth0).
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Jan Friesse [Mon, 10 Feb 2020 15:13:42 +0000 (16:13 +0100)]
cfgtool: Improve link status display
Totemknet is enhanced to use 'n' character for localhost and not adding
status, because it is safe to expect that localhost link is always
connectd. corosync-cfgtool is enhanced to properly decode 'n', '?' and
'd' characters and display its meaning for extended status. Special
characters are also documented in man page.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
This patch add a stats.schedmiss.* set of entries that
are a record of the last 10 times corosync was not scheduled
in time.
These entries are keypt in reverse order (so stats.schedmiss.0.* is
always the latest one kept) and the values, including the timestamp,
are in milliseconds.
It's also possible to use a cmap tracker to follow these events, which
might be useful.
Signed-off-by: Christine Caulfield <ccaulfie@redhat.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Jan Friesse [Thu, 16 Jan 2020 14:43:59 +0000 (15:43 +0100)]
votequorum: Reflect runtime change of 2Node to WFA
When 2Node mode is set, WFA is also set unless WFA is configured
explicitly. This behavior was not reflected on runtime change, so
restarted corosync behavior was different (WFA not set). Also when
cluster is reduced from 3 nodes to 2 nodes during runtime, WFA was not
set, what may result in two quorate partitions.
Solution is to set WFA depending on 2Node when WFA
is not explicitly configured.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Jan Friesse [Thu, 10 Oct 2019 10:33:58 +0000 (12:33 +0200)]
Initialize stack allocated memory
Some functions allocated memory on stack without clearing memory and
then send them on wire. This is not an issue, but valgrind reports this
as a problem so it is easy to miss real problem then.
Solution is to clear stack memory.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Thomas Lamprecht [Wed, 16 Oct 2019 16:56:15 +0000 (18:56 +0200)]
man: Fix corosync.conf knet pong count default
commit 029b8ebad60314d3daa285eb945c55355fade389 changed the default
of the KNET_PONG_COUNT from the kronosnet default of 5 to 2, as
corosync bring up was deemed to slow.
The documentation, and the comment stating that the totem config
default values match the knet ones were not updated, and thus now out
of date.
Fixhis by noting the correct default of 2 for KNET_PONG_COUNT and
note that all but that one are in sync with the korosync defaults in
the comment.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com> Reviewed-by: Jan Friesse <jfriesse@redhat.com>
Jan Friesse [Mon, 7 Oct 2019 13:26:22 +0000 (15:26 +0200)]
totemsrp: Reduce MTU to left room second mcast
Messages sent during recovery phase are encapsulated so such message has
extra size of mcast structure. This is not so big problem for UDPU,
because most of the switches are able to fragment and defragment packet
but it is problem for knet, because totempg is using maximum packet size
(65536 bytes) and when another header is added during retransmition,
then packet is too large.
Solution is to reduce mtu by 2 * sizeof (struct mcast).
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Fabio M. Di Nitto <fdinitto@redhat.com>
Jan Friesse [Thu, 3 Oct 2019 09:35:37 +0000 (11:35 +0200)]
totempg: Check sanity (length) of received message
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Thomas Lamprecht <t.lamprecht@proxmox.com> Reviewed-by: Fabio M. Di Nitto <fdinitto@redhat.com>
Jan Friesse [Mon, 9 Sep 2019 15:47:24 +0000 (17:47 +0200)]
totemknet: Add locking for log call
Knet callbacks may be called from different thread than main thread. If
this happens, log messages may be lost. Most prominent example is when
link goes up (logged by main thread) and host_change_callback_fn is
called.
Implemented solution is adding mutex for every log call in totemknet.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Fabio M. Di Nitto <fdinitto@redhat.com>
Jan Friesse [Mon, 15 Jul 2019 12:08:39 +0000 (14:08 +0200)]
totem: Increase ring_id seq after load
This patch handles the situation where the leader
node (the node with lowest node_id) crashes and is started again
before token timeout of the rest of the cluster.
The newly restarted node restores the ringid of the old ring from
stable storage, so it has the same ringid as rest of the nodes,
but ARU is zero. If the node is able to create a singleton membership
before receiving the joinlist from rest of the cluster,
everything works as expected, because the ring id gets increased
correctly.
But if the node receives a joinlist from another cluster node before
its own joinlist, then it continues as it would had it never left
the cluster. This is not correct, because the new node should always
create a singleton configuration first.
During the recovery phase, ARUs are compared and because they differ
(the ARU of the old leader node is 0), the other nodes
try to sent all of their previous messages. This is impossible
(even if it was correct), because other nodes have already freed most
of those messages. The implementation uses an assert to limit maximum
number of messages sent during recovery (we could fix this,
but it's not really the point).
The solution here is to increase the ring_id sequence number by 1 after
loading it from storage. During creation of the commit token it is
always increased by 4, so it will not collide with an existing
sequence.
Thanks Christine Caulfield <ccaulfie@redhat.com> for clarify commit
message.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Jan Friesse [Thu, 4 Jul 2019 13:07:44 +0000 (15:07 +0200)]
init: Use cpgtool instead of cfgtool
Init script used to use corosync-cfgtool -s to wait till
corosync accepts ipc connection. Problem with this approach
is that error code is returned not only if ipc cannot be initialized,
but also when one of the ring is marked as failed, making corosync
service not to start. Corosync with one failed ring can work just
fine and there is no need to fail startup.
Patch is changing call of corosync-cfgtool to corosync-cpgtool. Also to
make spotting of broken ring easier, corosync-cfgtool -s is called after
successful return of the cpgtool, and warning is issued if cfgtool
fails.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
Jan Friesse [Thu, 4 Jul 2019 12:36:54 +0000 (14:36 +0200)]
notifyd: Fix warning produced by 32-bit compiler
time_t is platform dependent real type which is usually long int on
64-bit platform, but only int on 32-bit platform and printing it with
%ld generated warning.
Solution seems to be ether retype time_t to long int or use functions
which works with time_t. Later option is used in this patch, which uses
localtime and strftime to print time_t value.
Also code is refactored to remove duplicate calls and add _cs_snmp
prefix to prevent snmp_ prefix collision.
Signed-off-by: Jan Friesse <jfriesse@redhat.com> Reviewed-by: Christine Caulfield <ccaulfie@redhat.com>
projects / mirror_corosync.git / log