Skip to content

Commit 28eb1ff

Browse files
author
Vidhya Sree Rajuladevi
committed
Bug #33398681 - ibd tablespace file is moved back from
innodb_directories to datadir after ALTER TABLE In case of ALTER TABLE .. FORCE and ALTER TABLE .. ENGINE, tables are recreated. The default location for recreated tables is default data dir. If the table was created with Data Directory clause, the table is created in the same source directory. Now, If the source table was moved to a different location and then the above Alter statements are run, the table gets recreated in the default data dir. This does not seem correct as the user would expect it to be recreated in the original directory. Furthermore, ALTER TABLE statement does not allow change in data directory so it is important for the table to be recreated at the same location. The solution is to update the data directory flag for corresponding dd_table during restart after ibd file was just moved to new location, get the current path of that ibd and reuse this path during recreation. Also, for those ibd files which were moved during previous versions of the server, we recognize them also as moved by checking the following conditions in fil_tablespace_path_equals(): 1. The file must have .ibd extension 2. old_path and new_path must point to same location (because in previous version when the file is just moved, old_path is updated to new location already) 3. new_path must be different from default data directory 4. dd_table should already exist for this tablespace 5. dd_table data directory flag doesn't exist (because in prev versions before this fix, the dd_table data directory flag is not updated for those ibd files that are moved. This check is added because for those tables that are created with data directory clause, this flag is true. We want to eliminate such cases and be sure that the ibd file is moved) For such ibd files which satisfy the above conditions, we recognize them as moved and populate their tablespace information separately in m_moved_previously_before_90000 which is later used to update dd_table. Note that the variable is named with version 9.0.0 as this is needed for data directories before 9.0.0. We can remove this code in 10.0.0. We added a common method dd_update_table_and_partitions_after_dir_change() which updates dd_table data directory flag for moved ibd files in above cases. Change-Id: I5d4fc2e1f7d23f6a8a9c152449d9d5ecf2344833
1 parent 9e6bec9 commit 28eb1ff

File tree

12 files changed

+932
-73
lines changed

12 files changed

+932
-73
lines changed

‎mysql-test/std_data/data80036_with_externally_stored_tablespaces.zip

2.08 MB
Binary file not shown.

‎mysql-test/suite/innodb/r/alter_datadir_path.result

Lines changed: 266 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,266 @@
1+
#
2+
#Bug #33398681 Innodb ibd tablespace file is moved back from innodb_directories to datadir after ALTER TABLE
3+
#
4+
# Set different paths for --datadir
5+
# Create innodb_directories folder
6+
# Create external datadir folder
7+
# Create external datadir2 folder
8+
# Restart the engine to make the external directory known.
9+
# restart: --innodb-directories=MYSQL_TMP_DIR/external_dir;MYSQL_TMP_DIR/external_dir2
10+
# Create table
11+
CREATE DATABASE dbtest;
12+
USE dbtest;
13+
# Normal tables
14+
CREATE TABLE `t1a` (`c1` INT);
15+
CREATE TABLE `t1b` (`c1` INT);
16+
CREATE TABLE `t1c` (`c1` INT);
17+
# Normal tables with data directory clause
18+
CREATE TABLE `t2a` (`c1` INT) DATA DIRECTORY='MYSQL_TMP_DIR/external_dir';
19+
CREATE TABLE `t2b` (`c1` INT) DATA DIRECTORY='MYSQL_TMP_DIR/external_dir';
20+
CREATE TABLE `t2c` (`c1` INT) DATA DIRECTORY='MYSQL_TMP_DIR/external_dir';
21+
# Partition tables
22+
CREATE TABLE `t3a` (a INT, b INT) PARTITION BY RANGE(a) (PARTITION p0 VALUES LESS THAN (10), PARTITION p1 VALUES LESS THAN (20));
23+
CREATE TABLE `t3b` (a INT, b INT) PARTITION BY RANGE(a) (PARTITION p0 VALUES LESS THAN (10), PARTITION p1 VALUES LESS THAN (20));
24+
CREATE TABLE `t3c` (a INT, b INT) PARTITION BY RANGE(a) (PARTITION p0 VALUES LESS THAN (10), PARTITION p1 VALUES LESS THAN (20));
25+
# Partition tables with data directory clause
26+
CREATE TABLE `t4a` (a INT, b INT) PARTITION BY RANGE(a) (PARTITION p0 VALUES LESS THAN (10) DATA DIRECTORY='MYSQL_TMP_DIR/external_dir', PARTITION p1 VALUES LESS THAN (20) DATA DIRECTORY='MYSQL_TMP_DIR/external_dir');
27+
CREATE TABLE `t4b` (a INT, b INT) PARTITION BY RANGE(a) (PARTITION p0 VALUES LESS THAN (10) DATA DIRECTORY='MYSQL_TMP_DIR/external_dir', PARTITION p1 VALUES LESS THAN (20) DATA DIRECTORY='MYSQL_TMP_DIR/external_dir');
28+
CREATE TABLE `t4c` (a INT, b INT) PARTITION BY RANGE(a) (PARTITION p0 VALUES LESS THAN (10) DATA DIRECTORY='MYSQL_TMP_DIR/external_dir', PARTITION p1 VALUES LESS THAN (20) DATA DIRECTORY='MYSQL_TMP_DIR/external_dir');
29+
# Normal tables with data directory clause 2
30+
CREATE TABLE `t5a` (`c1` INT) DATA DIRECTORY='MYSQL_TMP_DIR/external_dir2';
31+
CREATE TABLE `t5b` (`c1` INT) DATA DIRECTORY='MYSQL_TMP_DIR/external_dir2';
32+
CREATE TABLE `t5c` (`c1` INT) DATA DIRECTORY='MYSQL_TMP_DIR/external_dir2';
33+
# Partition tables with data directory clause
34+
CREATE TABLE `t6a` (a INT, b INT) PARTITION BY RANGE(a) (PARTITION p0 VALUES LESS THAN (10) DATA DIRECTORY='MYSQL_TMP_DIR/external_dir2', PARTITION p1 VALUES LESS THAN (20) DATA DIRECTORY='MYSQL_TMP_DIR/external_dir2');
35+
CREATE TABLE `t6b` (a INT, b INT) PARTITION BY RANGE(a) (PARTITION p0 VALUES LESS THAN (10) DATA DIRECTORY='MYSQL_TMP_DIR/external_dir2', PARTITION p1 VALUES LESS THAN (20) DATA DIRECTORY='MYSQL_TMP_DIR/external_dir2');
36+
CREATE TABLE `t6c` (a INT, b INT) PARTITION BY RANGE(a) (PARTITION p0 VALUES LESS THAN (10) DATA DIRECTORY='MYSQL_TMP_DIR/external_dir2', PARTITION p1 VALUES LESS THAN (20) DATA DIRECTORY='MYSQL_TMP_DIR/external_dir2');
37+
# Shutdown server
38+
# 1. ibd files for t1a, t1b, t1c, t3a, t3b, t3c
39+
# Check that the ibd files for each table exist in the working (dbtest) folder
40+
t1a.ibd
41+
t1b.ibd
42+
t1c.ibd
43+
t3a#p#p0.ibd
44+
t3a#p#p1.ibd
45+
t3b#p#p0.ibd
46+
t3b#p#p1.ibd
47+
t3c#p#p0.ibd
48+
t3c#p#p1.ibd
49+
# Move ibd files from default directory to new innodb directory
50+
# Check that the files exist in the innodb_directories folder before alter after moving
51+
t1a.ibd
52+
t1b.ibd
53+
t1c.ibd
54+
t3a#p#p0.ibd
55+
t3a#p#p1.ibd
56+
t3b#p#p0.ibd
57+
t3b#p#p1.ibd
58+
t3c#p#p0.ibd
59+
t3c#p#p1.ibd
60+
# 2. ibd files for t2a, t2b, t2c, t4a, t4b, t4c
61+
# Check that the ibd files for each table exist in the external directory where the tables are created
62+
t2a.ibd
63+
t2b.ibd
64+
t2c.ibd
65+
t4a#p#p0.ibd
66+
t4a#p#p1.ibd
67+
t4b#p#p0.ibd
68+
t4b#p#p1.ibd
69+
t4c#p#p0.ibd
70+
t4c#p#p1.ibd
71+
# 3. ibd files for t5a, t5b, t5c, t6a, t6b, t6c
72+
# Check that the ibd files for each table exist in the external directory2 where the tables are created
73+
t5a.ibd
74+
t5b.ibd
75+
t5c.ibd
76+
t6a#p#p0.ibd
77+
t6a#p#p1.ibd
78+
t6b#p#p0.ibd
79+
t6b#p#p1.ibd
80+
t6c#p#p0.ibd
81+
t6c#p#p1.ibd
82+
# Move ibd files for t5a, t5b, t5c, t6a, t6b, t6c from external directory2 to default directory
83+
# Check that the ibd files for t5a, t5b, t5c, t6a, t6b, t6c now exist in working (dbtest) folder
84+
t5a.ibd
85+
t5b.ibd
86+
t5c.ibd
87+
t6a#p#p0.ibd
88+
t6a#p#p1.ibd
89+
t6b#p#p0.ibd
90+
t6b#p#p1.ibd
91+
t6c#p#p0.ibd
92+
t6c#p#p1.ibd
93+
# Restart the server
94+
# restart: --innodb-directories=MYSQLTEST_VARDIR/innodb_directories;MYSQL_TMP_DIR/external_dir;MYSQL_TMP_DIR/external_dir2
95+
# Alter table
96+
ALTER TABLE dbtest.t1a FORCE;
97+
ALTER TABLE dbtest.t1b FORCE, ALGORITHM=INPLACE;
98+
ALTER TABLE dbtest.t1c FORCE, ALGORITHM=COPY;
99+
ALTER TABLE dbtest.t2a FORCE;
100+
ALTER TABLE dbtest.t2b FORCE, ALGORITHM=INPLACE;
101+
ALTER TABLE dbtest.t2c FORCE, ALGORITHM=COPY;
102+
ALTER TABLE dbtest.t3a FORCE;
103+
ALTER TABLE dbtest.t3b FORCE, ALGORITHM=INPLACE;
104+
ALTER TABLE dbtest.t3c FORCE, ALGORITHM=COPY;
105+
ALTER TABLE dbtest.t4a FORCE;
106+
ALTER TABLE dbtest.t4b FORCE, ALGORITHM=INPLACE;
107+
ALTER TABLE dbtest.t4c FORCE, ALGORITHM=COPY;
108+
ALTER TABLE dbtest.t5a FORCE;
109+
ALTER TABLE dbtest.t5b FORCE, ALGORITHM=INPLACE;
110+
ALTER TABLE dbtest.t5c FORCE, ALGORITHM=COPY;
111+
ALTER TABLE dbtest.t6a FORCE;
112+
ALTER TABLE dbtest.t6b FORCE, ALGORITHM=INPLACE;
113+
ALTER TABLE dbtest.t6c FORCE, ALGORITHM=COPY;
114+
# Check that the ibd files for t1a, t1b, t1c, t3a, t3b, t3c still exist in the innodb_directories folder
115+
t1a.ibd
116+
t1b.ibd
117+
t1c.ibd
118+
t3a#p#p0.ibd
119+
t3a#p#p1.ibd
120+
t3b#p#p0.ibd
121+
t3b#p#p1.ibd
122+
t3c#p#p0.ibd
123+
t3c#p#p1.ibd
124+
# Check that the ibd files for t1a, t1b, t1c, t3a, t3b, t3c do not exist and ibd files for t5a, t5b, t5c, t6a, t6b, t6c exist in the data directory folder
125+
t5a.ibd
126+
t5b.ibd
127+
t5c.ibd
128+
t6a#p#p0.ibd
129+
t6a#p#p1.ibd
130+
t6b#p#p0.ibd
131+
t6b#p#p1.ibd
132+
t6c#p#p0.ibd
133+
t6c#p#p1.ibd
134+
# Check that the ibd files for t5a, t5b, t5c, t6a, t6b, t6c do not exist in external directory2 where the tables are created
135+
# Check that the ibd files for t2a, t2b, t2c, t4a, t4b, t4c still exist in the external directory where the tables are created
136+
t2a.ibd
137+
t2b.ibd
138+
t2c.ibd
139+
t4a#p#p0.ibd
140+
t4a#p#p1.ibd
141+
t4b#p#p0.ibd
142+
t4b#p#p1.ibd
143+
t4c#p#p0.ibd
144+
t4c#p#p1.ibd
145+
# Shutdown server
146+
# Now move ibd files for t1a, t1b, t1c, t3a, t3b, t3c from innodb_directories folder to default directory
147+
# Restart the server
148+
# restart: --innodb-directories=MYSQLTEST_VARDIR/innodb_directories;MYSQL_TMP_DIR/external_dir;MYSQL_TMP_DIR/external_dir2
149+
# Alter table
150+
ALTER TABLE dbtest.t1a FORCE;
151+
ALTER TABLE dbtest.t1b FORCE, ALGORITHM=INPLACE;
152+
ALTER TABLE dbtest.t1c FORCE, ALGORITHM=COPY;
153+
ALTER TABLE dbtest.t3a FORCE;
154+
ALTER TABLE dbtest.t3b FORCE, ALGORITHM=INPLACE;
155+
ALTER TABLE dbtest.t3c FORCE, ALGORITHM=COPY;
156+
# Check ibd files for t1a, t1b, t1c, t3a, t3b, t3c exist in default directory
157+
t1a.ibd
158+
t1b.ibd
159+
t1c.ibd
160+
t3a#p#p0.ibd
161+
t3a#p#p1.ibd
162+
t3b#p#p0.ibd
163+
t3b#p#p1.ibd
164+
t3c#p#p0.ibd
165+
t3c#p#p1.ibd
166+
t5a.ibd
167+
t5b.ibd
168+
t5c.ibd
169+
t6a#p#p0.ibd
170+
t6a#p#p1.ibd
171+
t6b#p#p0.ibd
172+
t6b#p#p1.ibd
173+
t6c#p#p0.ibd
174+
t6c#p#p1.ibd
175+
# Check ibd files for t1a, t1b, t1c, t3a, t3b, t3c do not exist in innodb_directories
176+
# Clean Up
177+
# Drop tables and database
178+
DROP TABLE dbtest.t1a;
179+
DROP TABLE dbtest.t1b;
180+
DROP TABLE dbtest.t1c;
181+
DROP TABLE dbtest.t2a;
182+
DROP TABLE dbtest.t2b;
183+
DROP TABLE dbtest.t2c;
184+
DROP TABLE dbtest.t3a;
185+
DROP TABLE dbtest.t3b;
186+
DROP TABLE dbtest.t3c;
187+
DROP TABLE dbtest.t4a;
188+
DROP TABLE dbtest.t4b;
189+
DROP TABLE dbtest.t4c;
190+
DROP TABLE dbtest.t5a;
191+
DROP TABLE dbtest.t5b;
192+
DROP TABLE dbtest.t5c;
193+
DROP TABLE dbtest.t6a;
194+
DROP TABLE dbtest.t6b;
195+
DROP TABLE dbtest.t6c;
196+
DROP DATABASE dbtest;
197+
# restart:
198+
# Shutdown server
199+
# Upgrade scenario
200+
# Part 1: Steps already executed in older version of mysqld i.e., 8.0.36 which does not contain the fix for Bug #33398681
201+
# 1a. Build 8.0.36 and create data_dir folder in it. Initialize mysqld in this data_dir folder. This will be the default datadir for 8.0.36
202+
# 1b. Start mysql client and create database test_upgsc and create tables t7a, t7b t7c in it
203+
# 1c. Shutdown the server and move t7a.ibd, t7b.ibd, t7c.ibd to new external directory d1 and delete database folder test_upgsc in default directory data_dir
204+
# 1d. Start and shutdown the server again to make ibd files get recognized as moved in 8.0.36
205+
# 1e. Zip this data_dir and new external dir d1 as new_innodbdir into MYSQL_TEST_DIR/std_data/data80036_with_externally_stored_tablespaces.zip
206+
# Part 2: Steps to be executed in the current version i.e., in this test
207+
# 2a. Copy the data80036_with_externally_stored_tablespaces.zip from MYSQL_TEST_DIR/std_data/ to MYSQL_TMP_DIR/test_Bug33398681/ as data8036.zip
208+
# 2b. Check that the zip file data8036.zip exists in working MYSQL_TMP_DIR/test_Bug33398681/ folder and unzip it in the same directory
209+
# 2c. Check that data_dir and new_innodbdir are existing in MYSQL_TMP_DIR/test_Bug33398681/ and list all the files
210+
# 2d. Set MYSQLD_NEW_DATADIR as MYSQL_TMP_DIR/test_Bug33398681/data_dir and EXTERNAL_DIR3 AS MYSQL_TMP_DIR/test_Bug33398681/new_innodbdir
211+
# 2e. Start the server with datadir=MYSQLD_NEW_DATADIR and innodb-directories=EXTERNAL_DIR3 options
212+
# 2f. Trigger Alter table force for t7a, t7b, t7c
213+
# 2g. Verify that the ibd files for t7a, t7b, t7c exist in EXTERNAL_DIR3
214+
# Copy the zip directory of 8.0.36 from MYSQL_TEST_DIR/std_data/ to MYSQL_TMP_DIR/test_Bug33398681 working location.
215+
# Check that the file exists in the working folder.
216+
# Unzip the zip file.
217+
# Check that the file exists in the working folder.
218+
#ib_16384_0.dblwr
219+
#ib_16384_1.dblwr
220+
#innodb_redo
221+
#innodb_temp
222+
auto.cnf
223+
binlog.000001
224+
binlog.000002
225+
binlog.index
226+
ca-key.pem
227+
ca.pem
228+
client-cert.pem
229+
client-key.pem
230+
debug.log
231+
error.log
232+
ib_buffer_pool
233+
ibdata1
234+
mysql
235+
mysql.ibd
236+
performance_schema
237+
private_key.pem
238+
public_key.pem
239+
server-cert.pem
240+
server-key.pem
241+
sys
242+
undo_001
243+
undo_002
244+
test_upgsc
245+
t7a.ibd
246+
t7b.ibd
247+
t7c.ibd
248+
# Set different path for --datadir which points to old version 8.0.36
249+
# Set EXTERNAL_DIR3
250+
# Check that the ibd files for t7a, t7b, t7c exist in EXTERNAL_DIR3
251+
t7a.ibd
252+
t7b.ibd
253+
t7c.ibd
254+
# Restart the server
255+
# restart: --datadir=MYSQL_TMP_DIR/test_Bug33398681/data_dir --innodb-directories=MYSQL_TMP_DIR/test_Bug33398681/new_innodbdir/ --lower_case_table_names=1
256+
# Alter table
257+
ALTER TABLE test_upgsc.t7a FORCE;
258+
ALTER TABLE test_upgsc.t7b FORCE, ALGORITHM=INPLACE;
259+
ALTER TABLE test_upgsc.t7c FORCE, ALGORITHM=COPY;
260+
# Check that the ibd files for t7a, t7b, t7c still exist in EXTERNAL_DIR3
261+
t7a.ibd
262+
t7b.ibd
263+
t7c.ibd
264+
# Clean Up
265+
# restart: --datadir=MYSQLD_DATADIR --innodb-directories=
266+
# End Test

0 commit comments

Comments
 (0)