Skip to content

Commit a7e1ef8

Browse files
author
Neha Kumari
committed
WL#9237: Add a new variable binlog_expire_logs_seconds
This worklog adds a system variable binlog_expire_logs_seconds in addition to the existing expire_logs_days variable. The ultimate requirement is that the user can set expire periods smaller than one day by providing another extra variable. The new variable binlog_expire_logs_seconds, will be set in those cases where the expire period is not a integer multiple of days like 1 day 2 hours and 32 minute. @ sql/binlog.cc The purge time now also considers the value of binlog_expire_logs_seconds. @ sql/mysqld.h Declared the new variable binlog_expire_logs_seconds @ sql/mysqld.cc The purge time now also considers the value of binlog_expire_logs_seconds. @ sql/sys_vars.cc Added the description for binlog_expire_logs_seconds, and modified the description for expire_logs_days. Added test case to test the new variable: - binlog_expire_logs_seconds: Ensure that when any of the two expire variable, expire_logs_days or binlog_expire_logs_seconds is set the purge happens on server restart and in cases where a force rotation of binlog happens. Extended the existing test cases: - expire_logs_days_basic: Added code to test the system variable binlog_expire_logs_seconds.
1 parent 120ddb8 commit a7e1ef8

12 files changed

+637
-26
lines changed

‎mysql-test/r/mysqld--help-notwin.result

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,11 @@ The following options may be given as the first argument:
6666
When statements cannot be written to the binary log due
6767
to a fatal error, the server can either ignore the error
6868
and let the master continue, or abort.
69+
--binlog-expire-logs-seconds=#
70+
If non-zero, binary logs will be purged after
71+
(binlog_expire_logs_seconds + 24 * 60 * 60 *
72+
expire_logs_days) seconds; possible purges happen at
73+
startup and at binary log rotation
6974
--binlog-format=name
7075
What form of binary logging the master will use: either
7176
ROW for row-based binary logging, STATEMENT for
@@ -260,8 +265,10 @@ The following options may be given as the first argument:
260265
-T, --exit-info[=#] Used for debugging. Use at your own risk.
261266
--expire-logs-days=#
262267
If non-zero, binary logs will be purged after
263-
expire_logs_days days; possible purges happen at startup
264-
and at binary log rotation
268+
expire_logs_days days; or (binlog_expire_logs_seconds +
269+
24 * 60 * 60 * expire_logs_days) seconds if
270+
binlog_expire_logs_seconds has a non zero value; possible
271+
purges happen at startup and at binary log rotation
265272
--explicit-defaults-for-timestamp
266273
This option causes CREATE TABLE to create all TIMESTAMP
267274
columns as NULL with DEFAULT NULL attribute, Without this
@@ -1260,6 +1267,7 @@ binlog-cache-size 32768
12601267
binlog-checksum CRC32
12611268
binlog-direct-non-transactional-updates FALSE
12621269
binlog-error-action ABORT_SERVER
1270+
binlog-expire-logs-seconds 0
12631271
binlog-format ROW
12641272
binlog-group-commit-sync-delay 0
12651273
binlog-group-commit-sync-no-delay-count 0

‎mysql-test/r/mysqld--help-win.result

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,11 @@ The following options may be given as the first argument:
6666
When statements cannot be written to the binary log due
6767
to a fatal error, the server can either ignore the error
6868
and let the master continue, or abort.
69+
--binlog-expire-logs-seconds=#
70+
If non-zero, binary logs will be purged after
71+
(binlog_expire_logs_seconds + 24 * 60 * 60 *
72+
expire_logs_days) seconds; possible purges happen at
73+
startup and at binary log rotation
6974
--binlog-format=name
7075
What form of binary logging the master will use: either
7176
ROW for row-based binary logging, STATEMENT for
@@ -259,8 +264,10 @@ The following options may be given as the first argument:
259264
-T, --exit-info[=#] Used for debugging. Use at your own risk.
260265
--expire-logs-days=#
261266
If non-zero, binary logs will be purged after
262-
expire_logs_days days; possible purges happen at startup
263-
and at binary log rotation
267+
expire_logs_days days; or (binlog_expire_logs_seconds +
268+
24 * 60 * 60 * expire_logs_days) seconds if
269+
binlog_expire_logs_seconds has a non zero value; possible
270+
purges happen at startup and at binary log rotation
264271
--explicit-defaults-for-timestamp
265272
This option causes CREATE TABLE to create all TIMESTAMP
266273
columns as NULL with DEFAULT NULL attribute, Without this
@@ -1259,6 +1266,7 @@ binlog-cache-size 32768
12591266
binlog-checksum CRC32
12601267
binlog-direct-non-transactional-updates FALSE
12611268
binlog-error-action ABORT_SERVER
1269+
binlog-expire-logs-seconds 0
12621270
binlog-format ROW
12631271
binlog-group-commit-sync-delay 0
12641272
binlog-group-commit-sync-no-delay-count 0

‎mysql-test/suite/binlog/include/binlog_expire_logs_seconds.inc

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
#
2+
# WL#9237: Add a new variable binlog_expire_logs_seconds
3+
4+
# Here we will test purging of binary logs when either one or both of these variables are set
5+
# - binlog_expire_logs_seconds
6+
# - expire_logs_days
7+
8+
# The three scenarios being tested for are:
9+
# 1. FLUSH LOGS
10+
# 2. Rotation of logs because of binlog growing bigger than max_binlog_size
11+
# 3. Server restart
12+
#
13+
# Usuage: --let $binlog_expire_logs_seconds=
14+
# --let $expire_logs_days=
15+
#
16+
# --source suite/binlog/include/binlog_expire_logs_seconds.inc
17+
18+
--let $expire_logs_seconds= `SELECT @@global.binlog_expire_logs_seconds + @@global.expire_logs_days * 24 * 60 * 60`
19+
20+
CREATE TABLE t1(s LONGBLOB );
21+
22+
--let $max_binlog_size_save= `SELECT @@GLOBAL.MAX_BINLOG_SIZE`
23+
24+
--let $case= 0
25+
26+
while ($case < 3)
27+
{
28+
--echo Case:$case
29+
--let $first_binlog_file= query_get_value(SHOW MASTER STATUS, File, 1)
30+
31+
# rotates the log, thence the first log will be closed, and depending upon
32+
# the expire time the purge will/will not happen.
33+
34+
FLUSH LOGS;
35+
36+
INSERT INTO t1 VALUES('a');
37+
38+
--let $second_binlog_file= query_get_value(SHOW MASTER STATUS, File, 1)
39+
40+
FLUSH LOGS;
41+
42+
43+
# This is done to avoid time out in cases where the expire time is more.
44+
# What we do is in those cases modify the timestamp of the oldest log file
45+
# to be the same as expire time so when we execute the next flush log command
46+
# the oldest log will be purged.
47+
48+
# Only change the timestamp of binlog file when the expire_logs_seconds which is the total
49+
# time for expiring log is greater than 30 seconds
50+
51+
if ($expire_logs_seconds > 30)
52+
{
53+
--let $modified_time= `SELECT $expire_logs_seconds + 60`
54+
--exec touch -r "$MYSQLD_DATADIR/$first_binlog_file" -d '-$modified_time second' "$MYSQLD_DATADIR/$first_binlog_file"
55+
}
56+
57+
# Checking this ensures that nothing is purged so far.
58+
--file_exists $MYSQLD_DATADIR/$first_binlog_file
59+
60+
if ($case == 0)
61+
{
62+
--echo #### 1. FLUSH LOGS
63+
64+
FLUSH LOGS;
65+
}
66+
if ($case == 1)
67+
{
68+
--echo #### 2. Binlog_size > max_binlog_size
69+
70+
SET @@GLOBAL.MAX_BINLOG_SIZE= 4096;
71+
72+
INSERT INTO t1 (s) VALUES (REPEAT('s',50000));
73+
}
74+
if ($case == 2)
75+
{
76+
--echo #### 3. Server restart
77+
78+
--let $restart_parameters=restart:--binlog_expire_logs_seconds=$expire_logs_seconds
79+
--source include/restart_mysqld.inc
80+
}
81+
82+
if ($expire_logs_seconds != 0)
83+
{
84+
--error 1
85+
--file_exists $MYSQLD_DATADIR/$first_binlog_file
86+
}
87+
if ($expire_logs_seconds == 0)
88+
{
89+
--file_exists $MYSQLD_DATADIR/$first_binlog_file
90+
}
91+
--file_exists $MYSQLD_DATADIR/$second_binlog_file
92+
93+
--inc $case
94+
RESET MASTER;
95+
}
96+
--echo ##### Cleanup #####
97+
--eval SET @@GLOBAL.MAX_BINLOG_SIZE= $max_binlog_size_save;
98+
DROP TABLE t1;
99+
RESET MASTER;
100+

‎mysql-test/suite/binlog/r/binlog_expire_logs_seconds.result

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
####
2+
#### 1. When binlog_expire_logs_seconds == 0 and expire_logs_days == 0
3+
#### no purge should happen
4+
SET GLOBAL binlog_expire_logs_seconds= 0;
5+
SET GLOBAL expire_logs_days= 0;
6+
CREATE TABLE t1(s LONGBLOB );
7+
Case:0
8+
FLUSH LOGS;
9+
INSERT INTO t1 VALUES('a');
10+
FLUSH LOGS;
11+
#### 1. FLUSH LOGS
12+
FLUSH LOGS;
13+
RESET MASTER;
14+
Case:1
15+
FLUSH LOGS;
16+
INSERT INTO t1 VALUES('a');
17+
FLUSH LOGS;
18+
#### 2. Binlog_size > max_binlog_size
19+
SET @@GLOBAL.MAX_BINLOG_SIZE= 4096;
20+
INSERT INTO t1 (s) VALUES (REPEAT('s',50000));
21+
RESET MASTER;
22+
Case:2
23+
FLUSH LOGS;
24+
INSERT INTO t1 VALUES('a');
25+
FLUSH LOGS;
26+
#### 3. Server restart
27+
# restart:--binlog_expire_logs_seconds=0
28+
RESET MASTER;
29+
##### Cleanup #####
30+
SET @@GLOBAL.MAX_BINLOG_SIZE= 1073741824;;
31+
DROP TABLE t1;
32+
RESET MASTER;
33+
####
34+
#### 2.1: binlog_expire_logs_seconds > 0 and expire_logs_days > 0
35+
####
36+
SET GLOBAL binlog_expire_logs_seconds= 3600;
37+
SET GLOBAL expire_logs_days= 1;
38+
CREATE TABLE t1(s LONGBLOB );
39+
Case:0
40+
FLUSH LOGS;
41+
INSERT INTO t1 VALUES('a');
42+
FLUSH LOGS;
43+
#### 1. FLUSH LOGS
44+
FLUSH LOGS;
45+
RESET MASTER;
46+
Case:1
47+
FLUSH LOGS;
48+
INSERT INTO t1 VALUES('a');
49+
FLUSH LOGS;
50+
#### 2. Binlog_size > max_binlog_size
51+
SET @@GLOBAL.MAX_BINLOG_SIZE= 4096;
52+
INSERT INTO t1 (s) VALUES (REPEAT('s',50000));
53+
RESET MASTER;
54+
Case:2
55+
FLUSH LOGS;
56+
INSERT INTO t1 VALUES('a');
57+
FLUSH LOGS;
58+
#### 3. Server restart
59+
# restart:--binlog_expire_logs_seconds=90000
60+
RESET MASTER;
61+
##### Cleanup #####
62+
SET @@GLOBAL.MAX_BINLOG_SIZE= 1073741824;;
63+
DROP TABLE t1;
64+
RESET MASTER;
65+
####
66+
#### 2.2: binlog_expire_logs_seconds > 0 and expire_logs_days == 0
67+
####
68+
Testing with smaller values of binlog_expire_logs_seconds
69+
SET GLOBAL binlog_expire_logs_seconds= 30;
70+
SET GLOBAL expire_logs_days= 0;
71+
FLUSH LOGS;
72+
FLUSH LOGS;
73+
FLUSH LOGS;
74+
RESET MASTER;
75+
Testing with greater values of binlog_expire_logs_seconds
76+
SET GLOBAL binlog_expire_logs_seconds= 3600;
77+
SET GLOBAL expire_logs_days= 0;
78+
CREATE TABLE t1(s LONGBLOB );
79+
Case:0
80+
FLUSH LOGS;
81+
INSERT INTO t1 VALUES('a');
82+
FLUSH LOGS;
83+
#### 1. FLUSH LOGS
84+
FLUSH LOGS;
85+
RESET MASTER;
86+
Case:1
87+
FLUSH LOGS;
88+
INSERT INTO t1 VALUES('a');
89+
FLUSH LOGS;
90+
#### 2. Binlog_size > max_binlog_size
91+
SET @@GLOBAL.MAX_BINLOG_SIZE= 4096;
92+
INSERT INTO t1 (s) VALUES (REPEAT('s',50000));
93+
RESET MASTER;
94+
Case:2
95+
FLUSH LOGS;
96+
INSERT INTO t1 VALUES('a');
97+
FLUSH LOGS;
98+
#### 3. Server restart
99+
# restart:--binlog_expire_logs_seconds=3600
100+
RESET MASTER;
101+
##### Cleanup #####
102+
SET @@GLOBAL.MAX_BINLOG_SIZE= 1073741824;;
103+
DROP TABLE t1;
104+
RESET MASTER;
105+
####
106+
#### 2.3: binlog_expire_logs_seconds == 0 and expire_logs_days > 0
107+
####
108+
SET GLOBAL binlog_expire_logs_seconds= 0;
109+
SET GLOBAL expire_logs_days= 1;
110+
CREATE TABLE t1(s LONGBLOB );
111+
Case:0
112+
FLUSH LOGS;
113+
INSERT INTO t1 VALUES('a');
114+
FLUSH LOGS;
115+
#### 1. FLUSH LOGS
116+
FLUSH LOGS;
117+
RESET MASTER;
118+
Case:1
119+
FLUSH LOGS;
120+
INSERT INTO t1 VALUES('a');
121+
FLUSH LOGS;
122+
#### 2. Binlog_size > max_binlog_size
123+
SET @@GLOBAL.MAX_BINLOG_SIZE= 4096;
124+
INSERT INTO t1 (s) VALUES (REPEAT('s',50000));
125+
RESET MASTER;
126+
Case:2
127+
FLUSH LOGS;
128+
INSERT INTO t1 VALUES('a');
129+
FLUSH LOGS;
130+
#### 3. Server restart
131+
# restart:--binlog_expire_logs_seconds=86400
132+
RESET MASTER;
133+
##### Cleanup #####
134+
SET @@GLOBAL.MAX_BINLOG_SIZE= 1073741824;;
135+
DROP TABLE t1;
136+
RESET MASTER;
137+
SET GLOBAL binlog_expire_logs_seconds= 0;
138+
SET GLOBAL expire_logs_days= 0;

0 commit comments

Comments
 (0)