[pull] 8.0 from mysql:8.0 #5
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![pull[bot]](https://avatars.githubusercontent.com/in/12910?s=60&v=4){kind=small}
The patch for Bug#35046862 Need a MYSQL_ADD_CLIENT_PLUGIN cmake macro for libmysql plugins uncovered incorrect handling of "custom" SASL/KERBEROS library dependencies on el6. Extend the dependency check to also cover SASL plugins. Change-Id: I685a917dbedbcb1ee7cb4498f73130d933da283e (cherry picked from commit d43d8ec53f640473a4ec13989cb31b61f2299733)
…13 and OpenSSL < 1.1.0
Problem:
Compilation in debug mode with clang-13 fails due to:
router/src/harness/src/tls_server_context.cc:62:15: error: unused variable 'kMaxSecurityLevel' [-Werror,-Wunused-const-variable]
constexpr int kMaxSecurityLevel{5};
^
Solution:
Surround the declaration of 'kMaxSecurityLevel' with:
#if OPENSSL_VERSION_NUMBER >= ROUTER_OPENSSL_VERSION(1, 1, 0)
constexpr int kMaxSecurityLevel{5};
#endif
since it is only used in that case.
Change-Id: I8aba47d7d854bb7239c4c9824d30c85511ec8b84
Contributed by: Mikael Ronström ClusterJ uses a scratch buffer for primary key hash calculations which is limited to 10000 bytes. This is too small in some cases. The solution is to malloc the buffer in computeHash if the buffer isn't big enough. This patch contains a test for this bug that ensures that the primary key requires a buffer larger than 10000 bytes. Contributed patch was modified as: - update comments for startTransaction and computeHash - test with Varchar columns instead of Longvarchar - revert changes to tests Bug17200163Test and ConversationSummary - replaced tabs with spaces and remove dead code Change-Id: Ifb8d82b167e91fd48416186938936361bcb0bd67
Contributed by: Mikael Ronström ClusterJ uses a scratch buffer for primary key hash calculations which is limited to 10000 bytes. This is too small in some cases. The solution is to malloc the buffer in computeHash if the buffer isn't big enough. This patch contains a test for this bug that ensures that the primary key requires a buffer larger than 10000 bytes. Contributed patch was modified as: - update comments for startTransaction and computeHash - test with Varchar columns instead of Longvarchar - revert changes to tests Bug17200163Test and ConversationSummary - replaced tabs with spaces and remove dead code Change-Id: Ifb8d82b167e91fd48416186938936361bcb0bd67
Change-Id: I878cd906fab6e83e4503e653ea605f092de1f192
Revert extension of the mysql_command_factory service introduced in WL#15396 (methods init_thread and deinit_thread). Instead, declare a new service mysql_command_thread containing the two methods. Change-Id: I675f40ca4080adcf0ca464c3f3d9ff7fe8150479
…red to a temporal literal Bug#35080094: MySQL server crash - Assertion CompatibleTypesForIndexLookup failed This patch fixes two bugs: 1) Wrong results were seen in some case when one equality predicate compared two string columns and another equality predicate compared one of the string columns to a temporal constant. For example: SELECT * FROM t WHERE a = b AND b = DATE'2023-01-01' If a and b were VARCHAR columns with character set latin1, the optimizer would propagate the DATE constant into the a = b predicate and transform the query into: SELECT * FROM t WHERE a = DATE'2023-01-01' AND b = DATE'2023-01-01' The original a = b predicate would use string semantics for the comparison, whereas the new a = DATE'2023-01-01' predicate uses DATE semantics. The meaning of the query therefore changed, and wrong results could be returned. 2) The hypergraph optimizer got an assertion failure when analyzing which predicates were sargable. The assertion failure was seen if the WHERE clause compared an indexed string column with a constant TIME expression. It checked that all equality predicates that were part of a multiple equality had arguments that were compatible for index lookups. Both of these issues were fixed by disabling creation of multiple equalities from single equalities comparing a non-temporal string column to a temporal constant. Change-Id: Ibab64903eea83ce16ca4057154fa882a7e2b05f6
Some users configure automatic error skipping for replication, with the expectation that the listed errors will not cause the replication applier to stop. Multi threaded applier stops when failure occurs in commit since the next applier thread are scheduled "with an error" without checking if the error that has occured is expected or should be ignored. The problem is only seen to occur when applying Query_log_event containing a "COMMIT". On the contrary, if a WRITE_ROWS, UPDATE_ROWS, DELETE_ROWS event encounters an error which should be skipped, it is skipped without error and this problem does not occur Returning error during commit is unusual for local SE that process row changes serially while it's much more common in distributed SE where row changess are batched and processed in parallel during commit. This 'lazy evaluation' is essential for performance but results in less localised error handling, with errors potentially showing up as part of COMMIT processing (where all pending work must be finalised). Another difference is that binlog written with an engine supporting XA protocol will use the Xid_log_event, thus giving a difference in how the commit is applied and error handled. Fix is done by not allowing next applier thread to be scheduled directly when a commit error occurs in an SE which uses deferred processing, instead the scheduling is deferred until end of the current group. This means that the next worker will be scheduled after checks has been made whether the actual error matches the expected outcome or the user has configured certain error codes to be ignored. This approach still allows "fast path" for commits without error where the next applier is scheduled immediately. Since the fix has no effect for SE which does not support deferred processing a failure during commit with such engine will stop the applier in the same way as before, however such commit errors are very unlikely and not known to occur. Change-Id: I330eb696e39a58e1d42aed66f36a15e8ab4804d1
Bug#32497174 NDBSPIN_INIT LOGIC PROBLEMS
NdbSpin_Init() is intended to find how many times we need to call
cpu_pause() in order to wait for 1 microsecond. It will set
this loop-count into the variable glob_num_spin_loops, which
is later used by NdbSpin() which will 'pause' this number
of times to do the ~1us wait.
Initially we set 'glob_num_spin_loops = 10' as a guess,
which would also cause NdbSpin() to 'pause' 10 times
until we have calibrated the spin-loop with NdbSpin_Init()
However, NdbSpin_Init() did its calibration loop by calling
NdbSpin(), such that what was measured is the time it takes
to cpu_pause 10 times, not the cpu_pause() itself.
Patch fix this by replacing the NdbSpin() call in 'init'
with cpu_pause().
It also fix issue 2) described in bug#32497174, by introducing
the new variable 'glob_spin_enabled = false'. This variable is
set to 'true' only after a successfull NdbSpin_Init(), finding
a non-zero elapsed time for the nanos_per_call.
NdbSpin_is_supported() is also changed to return this variable,
requiring a successfull NdbSpin_Init() before Spin-support is
claimed.
Furthermore some adjustments were done to places where NdbSpin()
was called, using the assumption that the code was adapted to
the too short 100ns spin.
- Places where we had to NdbSpin() directly after each other
were changed to a singel NdbSpin()
- Places where we added twe more 'num_spins' to be looped over
for the next wait were changed to a '++' instead.
Also removed code from mt.cpp where we claim to only check
the elapsed time for every 3 times we NdbSpin(). As the spin is
now 10x longer it should be ok to check elapsed every time.
Note that the removed code was incorrect and didn't do
what it claimed to do:
- /* Check if we have done enough spinning once per 3 us */
- if ((i & 3) == 3)
- continue;
... In reality we checked 3 of 4 times, instead of skipping
the test 3 of 4 times.
Change-Id: I77e5577025ee2c96dc10d4ae4bada3006121655a
Do not implement NdbSpin() at all when not NDB_HAVE_CPU_PAUSE. NdbSpin_is_supported() will always return 'false' when there is no cpu_pause() available. Thus we should never call NdbSpin() either. Let compiler enforce this by not implementing NdbSpin() when a 'pause' is not defined. The only violating usage of NdbSpin() in the mt.cpp *_yield() functions are encapsulated in #ifdef NDB_HAVE_CPU_PAUSE. Change-Id: I4ca88ae808dfaf6a768b8444af9f78c594f336bc
Clean up the cpu_pause definitions in the HW abstraction layer mt-asm.h: Fixes two related cpu_pause issues: 1) Empty (and unused) definitions for cpu_pause() where provided for platforms which didn't #define NDB_HAVE_CPU_PAUSE - GNUC compiling code for __sparc - SUN STUDIO compiling code for __sparc 2) For ARM architecture '__aarch64__' (Apple M1/M2) an implementation of cpu_pause() was provided. As it didn't also #define NDB_HAVE_CPU_PAUSE the 'pause' was unused. Thus, usage of spin was effectively disabled on that platform. Change-Id: I9e1bd11f26ea861240be4753cadbb58e72f29fd9
POST-PUSH FIX: Re-recorded result file for `ndb_opt.spj_rqg.result`. Change-Id: I0c8ad3da9534a2740685719a0491b6cc961cb4b0
mysqldump and mysqlpump extract DDL using SHOW CREATE. If a field's default value could not be converted to UTF-8 (for instance if the field was of the BINARY type and its default value did not coincide with valid UTF-8), this could break the dump. mysqldump's --hex-blob option did not resolve the issue -- it converts fields of the blob type, whereas the result of a SHOW CREATE statement is TEXT (even if part of it like the DEFAULT clause may contain a binary value). The problem was two-fold: On one hand, the server did not detect when it could not convert a DEFAULT value to the target character set; on the other, no way of handling this case, had it been detected, was available. This patch attempts to convert the default value to the system character set. If this fails, the value is sent as hexadecimal instead. The hex representation is only used as a fallback on the rationale that for most default values, an attempt to make the value more human-readable is preferable. This issue was first discovered in conjunction with mysqldump, but also affected mysqlpump. The enclosed test case covers both tools, regular SHOW CREATE, and INFORMATION_SCHEMA (which takes a similar code path). Change-Id: Ib874327ed2bccc5be840de58be9fd363edf1e863
In Item_func_case::find_item(): check for (conversion) error after calling store_value(). Change-Id: Ia1ac225b1cbaac2c72d66f5605796a53280de168 (cherry picked from commit cb286a673e21ef52bf265a7b7dd853417f7f7616)
…n `is_nullable()' failed Tag the function last_insert_id() as constant for execution, rather than constant, to avoid evaluating it at resolve time. Change-Id: I3ad215f3d9b8a68e8e2fd5ef76e50e15f8c71085 (cherry picked from commit acf059773f7f14fec9459ee735c257845ffc791e)
Approved-by: Bjorn Munch <[email protected]>
Approved-by: Bjorn Munch <[email protected]>
Problem: It is dfficult to determine the number and types of connections established to a MySQL server which appears unresponsive in MDS. Solution: Create a Thread Pool system variable which can be used to enable periodic reports about the number of normal and admin connections which are present and the number opened/closed since last report. The logging is available only in cloud builds and enabled by default with a report interval of 120 seconds. Change-Id: I39daf88d3b8bd6a729695c403a0b18a212907b14
… statements for connect_timeout seconds, causing pileups [post-push-fix] Fixing the sporadically failing testcase on PB2 Approved by Harin Vadodaria <[email protected]> over Slack.
Temporary error that occurs while applying a transaction using MTA(multi threaded applier) for NDB is not properly retried. The problem is only visible when using more than one worker. The problem causes replication applier to stop with error similar to below, the "Got error -1" is significant: "Worker 1 failed executing transaction 'ANONYMOUS' at source log master-bin.000001, end_log_pos 1341; Could not execute Write_rows event on table test.t1; Got error -1 - 'Unknown error -1' during COMMIT, Error_code: 1180; the event's source log master-bin.000001, end_log_pos 1341" Fix by: - returning a MySQL mapped error code, thus avoiding that -1 is used as error code. - push warning(s) indicating temporary error to the warning stack, this is important in order for the replication applier to classify the error as temporary. Add tests showing how a provoked deadlock (which is considered temporary and should be retried) stops the applier when no transaction retries are allowed and sucessfully retries when allowed. Change-Id: I0ffc17beda65637d45fdba8c3d7dc4fe328f4527
This patch: - adds more logging info to help debug the issue - replaces use of ENOENT with custom error code for scenario when the Router discovers there are no destinations available Change-Id: I544973fdd01d1c8012abc353c5d1214444b1a43d
Post-push fix: Remove the dependency connection -> destination the initial patch introduced to allow cleaner reuse in MCMD component. Change-Id: Ie10bc4a13f077c3eddf3da2b9867ffa0dee76a6c
pull bot
pushed a commit
that referenced
this pull request
Apr 27, 2023
# This is the 1st commit message:
WL#15280: HEATWAVE SUPPORT FOR MDS HA
Problem Statement
-----------------
Currently customers cannot enable heatwave analytics service to their
HA DBSystem or enable HA if they are using Heatwave enabled DBSystem.
In this change, we attempt to remove this limitation and provide
failover support of heatwave in an HA enabled DBSystem.
High Level Overview
-------------------
To support heatwave with HA, we extended the existing feature of auto-
reloading of tables to heatwave on MySQL server restart (WL-14396). To
provide seamless failover functionality to tables loaded to heatwave,
each node in the HA cluster (group replication) must have the latest
view of tables which are currently loaded to heatwave cluster attached
to the primary, i.e., the secondary_load flag should be in-sync always.
To achieve this, we made following changes -
1. replicate secondary load/unload DDL statements to all the active
secondary nodes by writing the DDL into the binlog, and
2. Control how secondary load/unload is executed when heatwave cluster
is not attached to node executing the command
Implementation Details
----------------------
Current implementation depends on two key assumptions -
1. All MDS DBSystems will have RAPID plugin installed.
2. No non-MDS system will have the RAPID plugin installed.
Based on these assumptions, we made certain changes w.r.t. how server
handles execution of secondary load/unload statements.
1. If secondary load/unload command is executed from a mysql client
session on a system without RAPID plugin installed (i.e., non-MDS),
instead of an error, a warning message will be shown to the user,
and the DDL is allowed to commit.
2. If secondary load/unload command is executed from a replication
connection on an MDS system without heatwave cluster attached,
instead of throwing an error, the DDL is allowed to commit.
3. If no error is thrown from secondary engine, then the DDL will
update the secondary_load metadata and write a binlog entry.
Writing to binlog implies that all the consumer of binlog now need to
handle this DDL gracefully. This has an adverse effect on Point-in-time
Recovery. If the PITR backup is taken from a DBSystem with heatwave, it
may contain traces of secondary load/unload statements in its binlog.
If such a backup is used to restore a new DBSystem, it will cause failure
while trying to execute statements from its binlog because
a) DBSystem will not heatwave cluster attached at this time, and
b) Statements from binlog are executed from standard mysql client
connection, thus making them indistinguishable from user executed
command.
Customers will be prevented (by control plane) from using PITR functionality
on a heatwave enabled DBSystem until there is a solution for this.
Testing
-------
This commit changes the behavior of secondary load/unload statements, so it
- adjusts existing tests' expectations, and
- adds a new test validating new DDL behavior under different scenarios
Change-Id: Ief7e9b3d4878748b832c366da02892917dc47d83
# This is the commit message #2:
WL#15280: HEATWAVE SUPPORT FOR MDS HA (PITR SUPPORT)
Problem
-------
A PITR backup taken from a heatwave enabled system could have traces
of secondary load or unload statements in binlog. When such a backup
is used to restore another system, it can cause failure because of
following two reasons:
1. Currently, even if the target system is heatwave enabled, heatwave
cluster is attached only after PITR restore phase completes.
2. When entries from binlogs are applied, a standard mysql client
connection is used. This makes it indistinguishable from other user
session.
Since secondary load (or unload) statements are meant to throw error
when they are executed by user in the absence of a healthy heatwave
cluster, PITR restore workflow will fail if binlogs from the backup
have any secondary load (or unload) statements in them.
Solution
--------
To avoid PITR failure, we are introducing a new system variable
rapid_enable_delayed_secondary_ops. It controls how load or unload
commands are to be processed by rapid plugin.
- When turned ON, the plugin silently skips the secondary engine
operation (load/unload) and returns success to the caller. This
allows secondary load (or unload) statements to be executed by the
server in the absence of any heatwave cluster.
- When turned OFF, it follows the existing behavior.
- The default value is OFF.
- The value can only be changed when rapid_bootstrap is IDLE or OFF.
- This variable cannot be persisted.
In PITR workflow, Control Plane would set the variable at the start of
PITR restore and then reset it at the end of workflow. This allows the
workflow to complete without failure even when heatwave cluster is not
attached. Since metadata is always updated when secondary load/unload
DDLs are executed, when heatwave cluster is attached at a later point
in time, the respective tables get reloaded to heatwave automatically.
Change-Id: I42e984910da23a0e416edb09d3949989159ef707
# This is the commit message #3:
WL#15280: HEATWAVE SUPPORT FOR MDS HA (TEST CHANGES)
This commit adds new functional tests for the MDS HA + HW integration.
Change-Id: Ic818331a4ca04b16998155efd77ac95da08deaa1
# This is the commit message #4:
WL#15280: HEATWAVE SUPPORT FOR MDS HA
BUG#34776485: RESTRICT DEFAULT VALUE FOR rapid_enable_delayed_secondary_ops
This commit does two things:
1. Add a basic test for newly introduced system variable
rapid_enable_delayed_secondary_ops, which controls the behavior of
alter table secondary load/unload ddl statements when rapid cluster
is not available.
2. It also restricts the DEFAULT value setting for the system variable
So, following is not allowed:
SET GLOBAL rapid_enable_delayed_secondary_ops = default
This variable is to be used in restricted scenarios and control plane
only sets it to ON/OFF before and after PITR apply. Allowing set to
default has no practical use.
Change-Id: I85c84dfaa0f868dbfc7b1a88792a89ffd2e81da2
# This is the commit message #5:
Bug#34726490: ADD DIAGNOSTICS FOR SECONDARY LOAD / UNLOAD DDL
Problem:
--------
If secondary load or unload DDL gets rolled back due to some error after
it had loaded / unloaded the table in heatwave cluster, there is no undo
of the secondary engine action. Only secondary_load flag update is
reverted and binlog is not written. From User's perspective, the table
is loaded and can be seen on performance_schema. There are also no
error messages printed to notify that the ddl didn't commit. This
creates a problem to debug any issue in this area.
Solution:
---------
The partial undo of secondary load/unload ddl will be handled in
bug#34592922. In this commit, we add diagnostics to reveal if the ddl
failed to commit, and from what stage.
Change-Id: I46c04dd5dbc07fc17beb8aa2a8d0b15ddfa171af
# This is the commit message #6:
WL#15280: HEATWAVE SUPPORT FOR MDS HA (TEST FIX)
Since ALTER TABLE SECONDARY LOAD / UNLOAD DDL statements now write
to binlog, from Heatwave's perspective, SCN is bumped up.
In this commit, we are adjusting expected SCN values in certain
tests which does secondary load/unload and expects SCN to match.
Change-Id: I9635b3cd588d01148d763d703c72cf50a0c0bb98
# This is the commit message mysql#7:
Adding MTR tests for ML in rapid group_replication suite
Added MTR tests with Heatwave ML queries with in
an HA setup.
Change-Id: I386a3530b5bbe6aea551610b6e739ab1cf366439
# This is the commit message mysql#8:
WL#15280: HEATWAVE SUPPORT FOR MDS HA (MTR TEST ADJUSTMENT)
In this commit we have adjusted the existing test to work with the
new MTR test infrastructure which extends the functionalities to
HA landscape. With this change, a lot of mannual settings have now
become redundant and thus removed in this commit.
Change-Id: Ie1f4fcfdf047bfe8638feaa9f54313d509cbad7e
# This is the commit message mysql#9:
WL#15280: HEATWAVE SUPPORT FOR MDS HA (CLANG-TIDY FIX)
Fix clang-tidy warnings found in previous change#16530, patch#20
Change-Id: I15d25df135694c2f6a3a9146feebe2b981637662
Change-Id: I3f3223a85bb52343a4619b0c2387856b09438265
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )