Skip to content

Mysql upgrade reorder #121

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 10,000 commits into from
Closed

Mysql upgrade reorder #121

wants to merge 10,000 commits into from

Conversation

ofirr
Copy link

@ofirr ofirr commented Feb 16, 2017

During mysql_upgrade, when views are checked before the tables are upgraded, the views might reference invalid tables.

cvicentiu and others added 30 commits February 6, 2016 12:37
…ck-va_end

MDEV-9433: Add va_end to sql/rpl_reporting.cc to make cppcheck happy
…cing pointer to incomplete type ‘DH {aka struct dh_st}

fixes for openssl that was built with -DOPENSSL_NO_DEPRECATED
mysqlcheck tool can be used even if opt_systables_only is true
(to upgrade views from mysql - that overrides opt_systables_only)
Fixed adding derived tables items to outer one.
and a trivial cleanup to avoid manually-specified lengths
…ophe/single quotation mark

fix dynstr_append_os_quoted() to escape single quotes correctly
for a POSIX shell
cherry-pick f1daf9ce from 10.0 branch
-------------------------------------

Fix build failures caused by new C runtime library
- isnan, snprintf, struct timespec are now defined, attempt to
redefine them leads
- P_tmpdir, tzname are no more defined
-  lfind() and lsearch() in lf_hash.c had to be renamed,  declaration
conflicts with some C runtime functions with the same name declared in
a header included by stdlib.h

 Also fix couple of annoying warnings :
- remove #define NOMINMAX from config.h to avoid "redefined" compiler
warnings(NOMINMAX is already in compile flags)

- disable incremental linker in Debug as well (feature not used much
and compiler crashes often)

Also simplify package building with Wix, require Wix 3.9 or later
(VS2015 is not compatible with old Wix 3.5/3.6)
…s with SQ inside

Consider a query with subquery in form t.key=(select ...). Suppose, the
parent query uses this equality for ref access.
It will attempt to evaluate the subquery in get_best_combination(),
right before the join->join_tab[...] array is filled.  The problem was
that subquery optimization will attempt to look at parent's join->join_tab
to check how many times subquery will be executed (and crash).

Fixed by not doing that when the subquery is constant (non-constant
subqueries are only be evaluated during join execution, so they are not
affected)
When one evaluates row-based comparison like (X, Y) = (A,B), one should
first call bring_value() for the Item that returns row value. If you
don't do that and just attempt to read values of X and Y, you get stale
values.
Semi-join/Materialization can take a row-based comparison apart and
make ref access from it. In that case, we need to call bring_value()
to get the index lookup components.
        The http-proxy option to the FEEDBACK plugin backported.
reverted about half of commits as either not applicable or
outright wrong
Correct fix. Generate the query in the client charset,
not in the system one.
test case for ALTER TABLE view CHECK PARTITION
The select mentioned in the bug attempted to create a temporary table
using the maria storage engine. The table needs to have primary keys such that
duplicates can be removed. Unfortunately this use case has a longer
than allowed key and the tmp table got created without a temporary key.
We must not allow materialization for the subquery if the total key
length and key parts is greater than what the storage engine supports.
…s_set(table->write_set, field_index) || bitmap_is_set(table->vcol_set, field_index))' fails) on using function over not created table

Problem was that created table was not marked as used (not set query_id) and so opening tables for stored function pick it up (as opened place holder for it) and used changing TABLE internals.
Fix test failure when using maria small-block size. We need to query
the max_key_length and max_key_parts based on the the tmp table engine.
In the case of error during my_load_defaults, we would not free the
args array.
In case of missing includedir file, we would attempt to free a NULL
pointer. Make sure to guard against that.
1. unused static inline functions are only removed at -xO4,
   otherwise test binaries will depend on various mysys
   symbols that they don't use. Link test with libmysys.

2. Sphinx - don't instantiate (explicitly) templates before
   they're defined. Or, rather, don't instantiate them explicitly at
   all.

3. GIS - don't use anonymous unions and structs.
1. check that unused inline functions are removed
2. only allow compilation if they are or if the check if overridden
3. with CMAKE_GENERATOR=Makefiles, use all flags when testing
   (e.g. both CMAKE_C_FLAGS and CMAKE_C_FLAGS_DEBUG if
   CMAKE_BUILD_TYPE=Debug). This is because
 - on Solaris with the SunPro compiler, default CMAKE_C_FLAGS_xxx
   values contain -xO2 (for Release and RelWithDebInfo)
   and -g (for RelWithDebInfo and Debug)
 - proper inlining only works at -xO4 without -g
 - so if CMAKE_C_FLAGS has -xO4, inlining would work in
   configure.cmake (before this fix) and fail during actual compilation

also remove the outdated check for inline from myu_global.h
1. ignore generated dtrace files
2. don't rewrite probes_mysql_nodtrace.h in-place
it caused feedback plugin tests to fail
elenst and others added 21 commits January 4, 2017 13:03
Backport the fix to 5.5, because it fails there too

The patch fixes two test failures:
- on slow builders, sometimes a connection attempt which should
  fail due to the exceeded number of thread_pool_max_threads
  actually succeeds;
- on even slow builders, MTR sometimes cannot establish the
  initial connection, and check-testcase fails prior to the
  test start

The problem with check-testcase was caused by connect-timeout=2
which was set for all clients in the test config file. On slow
builders it might be not enough.
There is no way to override it for the pre-test check, so it needed
to be substantially increased or removed.

The other problem was caused by a race condition between sleeps
that the test performs in existing connections and the connect
timeout for the connection attempt which was expected to fail.
If sleeps finished before the connect-timeout was exceeded, it
would allow the connection to succeed.

To solve each problem without making the other one worse,
connect-timeout should be configured dynamically during the test.
Due to the nature of the test (all connections must be busy
at the moment when we need to change the timeout, and cannot execute
SET GLOBAL ...), it needs to be done independently from the server.

The solution:
- recognize 'connect_timeout' as a connection option in mysqltest's
  "connect" command;
- remove connect-timeout from the test configuration file;
- use the new connect_timeout option for those connections which
  are expected to fail;
- re-arrange the test flow to allow running a huge SLEEP
  without affecting the test execution time (because it would be
  interrupted after the main test flow is finished).

The test is still subject to false negatives, e.g. if the connection
fails due to timeout rather than due to the exceeded number of
allowed threads, or if the connection on extra port succeeds due
to a race condition and not because the special logic for the extra
port. But those false negatives have always been possible there
on slow builders, they should not be critical because faster builders
should catch such failures if they appear.

Conflicts:
	client/mysqltest.cc
	mysql-test/r/pool_of_threads.result
	mysql-test/t/pool_of_threads.test
…"enforcing" mode

correct the error message in case of setuid/setgid failures
The guilty part of the test checks for performance degradation on
a query with numerous joins on an empty table. The test expects
the query to take less than 1 second, and fails if it is not so
(which can happen on very slow builders).

The solution is to add more JOINs to the query. On a fixed server,
it should not have any noticeable impact on the query execution,
while on the unfixed version the query would take several times
longer (e.g. 6.5 sec vs 1.5 sec). Thus, we can increase the margin
for the error, and make the test fail when the query takes longer
than 5 seconds.
…r with timeout

When the test is run as a part of the suite with valgrind,
only allow it to be executed if --big-test is set.
If the test is run by specifying its name explicitly, it
will still be executed, even with valgrind without big-test,
MTR has special logic for that
The warning is "blocks are still reachable in loss record",
happens in malloc / _dl_close_worker. Suppression added
The fix for bug mdev-5104 did not take into account that
for any call of setup_order the size of ref_array must
be big enough. This patch fixes this problem.
c3cf7f4 reverted the patch
for BUG#24487120. After merging the reverting patch from MySQL
to MariaDB the problems described in MDEV-11079 and MDEV-11631 disappeared.
Adding test cases only.
…"enforcing" mode

compile, and install selinux policy for mysqld_safe_helper on centos6.
the policy was created as described in https://mariadb.com/kb/en/mariadb/what-to-do-if-mariadb-doesnt-start/#other-selinux-changes
The supplied sql_mode(s) should be converted to ASCII first,
before comparing it with the sql_mode set.
…| bitmap_is_set(table->write_set, field_index) || (table->vcol_set && bitmap_is_set(table->vcol_set, field_index)))' failed in Field_time::store_TIME_with_warning

vcols and triggers. Revert 094f4cf,
backport the correct fix (Table_triggers_list::mark_fields_used() not
marking vcols) from 10.2.
Item_func_le included Arg_comparator. Arg_comparator remembered
the current_thd during fix_fields and used that value during
execution to allocate Item_cache in get_datetime_value().
But for vcols fix_fields and val_int can happen in different threads.

Same bug for Item_func_in using in_datetime or cmp_item_datetime,
both also remembered current_thd at fix_fields() to use it later
for get_datetime_value().

As a fix, these objects no longer remember the current_thd,
and get_datetime_value() uses current_thd at run time. This
should not increase the number of current_thd calls much, as
Item_cache is created only once anyway.
it used current_thd->alloc() and allocated on the thd's execution arena,
not on table->expr_arena.

Remove THD::arena_for_cached_items that is temporarily set in
update_virtual_fields(), and replaces THD arena in get_datetime_value().
Instead set THD arena to table->expr_arena for the whole  duration
of update_virtual_fields()
It was used for get_datetime_value() and for thd->is_error().

But in fact, get_datetime_value() never used thd argument, because the
cache ptr argument was NULL. And thd->is_error() check was not needed
at that place at all.
When building different range and index-merge trees the range optimizer
could build an index-merge tree with an index scan containing less ranges
then needed. This index-merge could be chosen as the best. Following this
index-merge the executioner missed some rows in the result set.
The invalid index scan was built due to an inconsistency in the code
back-ported from mysql into 5.3 that fixed mysql bug #11765831:
the code added to key_or() could change shared keys of the second
ored tree. Partially the problem was fixed in the patch for mariadb
bug #823301, but it turned out that only partially.
As the function Item_subselect::fix_fields does it the function
Item_subselect::update_used_tables must ignore UNCACHEABLE_EXPLAIN
when deciding whether the subquery item should be considered as a
constant item.
my_thread_global_init() + my_thrad_global_end() repeatadily.
This caused THR_KEY_mysys to be allocated multiple times.

Deletion of THR_KEY_mysys was originally in my_thread_global_end() but was
moved to my_end() as DBUG uses THR_KEY_mysys and DBUG is released after
my_thread_global_end() is called.

Releasing DBUG before my_thread_global_end() and move THR_KEY_mysys back
into my_thread_global_end() could be a solution, but as safe_mutex and other
things called by my_thread_global_end is using DBUG it may not be completely
safe.

To solve this, I used the simple solution to add a marker that THR_KEY_mysys
is created and not re-create it in my_thread_global_init if it already
exists.
…riv.test

Try to remove bug50373.txt before using it in SELECT .. OUTFILE
(in case its a leftover from previous test runs).
…grade.

* When views are checked before the tables are upgraded the views can references invalid tables.
@shlomi-noach
Copy link

Perhaps renaming of MySQL Server to MariaDB shouldn't be part of this PR?

@mysql-oca-bot
Copy link

Hi, thank you for submitting this pull request. In order to consider your code we need you to sign the Oracle Contribution Agreement (OCA). Please review the details and follow the instructions at http://www.oracle.com/technetwork/community/oca-486395.html
Please make sure to include your MySQL bug system user (email) in the returned form.
Thanks

@ofirr
Copy link
Author

ofirr commented Feb 16, 2017

Yep, sorry about that. I thought I was changing a single line and ended up merging half the project.
On top of which branch would you have me base this?

@ofirr ofirr closed this Feb 16, 2017
@shlomi-noach
Copy link

Just noting that I'm not an owner of this repository; I was expressing my personal impression. I'm sure the owners are observing this PR.

@ofirr ofirr deleted the mysql_upgrade_reorder branch May 3, 2017 15:14
dbussink pushed a commit to planetscale/mysql-server that referenced this pull request Mar 6, 2024
dbussink pushed a commit to planetscale/mysql-server that referenced this pull request May 21, 2024
dbussink pushed a commit to planetscale/mysql-server that referenced this pull request May 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.