Skip to content

Commit 6754159

Browse files
pks-tgitster
pks-t
authored and
gitster
committed
refs: implement reference transaction hook
The low-level reference transactions used to update references are currently completely opaque to the user. While certainly desirable in most usecases, there are some which might want to hook into the transaction to observe all queued reference updates as well as observing the abortion or commit of a prepared transaction. One such usecase would be to have a set of replicas of a given Git repository, where we perform Git operations on all of the repositories at once and expect the outcome to be the same in all of them. While there exist hooks already for a certain subset of Git commands that could be used to implement a voting mechanism for this, many others currently don't have any mechanism for this. The above scenario is the motivation for the new "reference-transaction" hook that reaches directly into Git's reference transaction mechanism. The hook receives as parameter the current state the transaction was moved to ("prepared", "committed" or "aborted") and gets via its standard input all queued reference updates. While the exit code gets ignored in the "committed" and "aborted" states, a non-zero exit code in the "prepared" state will cause the transaction to be aborted prematurely. Given the usecase described above, a voting mechanism can now be implemented via this hook: as soon as it gets called, it will take all of stdin and use it to cast a vote to a central service. When all replicas of the repository agree, the hook will exit with zero, otherwise it will abort the transaction by returning non-zero. The most important upside is that this will catch _all_ commands writing references at once, allowing to implement strong consistency for reference updates via a single mechanism. In order to test the impact on the case where we don't have any "reference-transaction" hook installed in the repository, this commit introduce two new performance tests for git-update-refs(1). Run against an empty repository, it produces the following results: Test origin/master HEAD -------------------------------------------------------------------- 1400.2: update-ref 2.70(2.10+0.71) 2.71(2.10+0.73) +0.4% 1400.3: update-ref --stdin 0.21(0.09+0.11) 0.21(0.07+0.14) +0.0% The performance test p1400.2 creates, updates and deletes a branch a thousand times, thus averaging runtime of git-update-refs over 3000 invocations. p1400.3 instead calls `git-update-refs --stdin` three times and queues a thousand creations, updates and deletes respectively. As expected, p1400.3 consistently shows no noticeable impact, as for each batch of updates there's a single call to access(3P) for the negative hook lookup. On the other hand, for p1400.2, one can see an impact caused by this patchset. But doing five runs of the performance tests where each one was run with GIT_PERF_REPEAT_COUNT=10, the overhead ranged from -1.5% to +1.1%. These inconsistent performance numbers can be explained by the overhead of spawning 3000 processes. This shows that the overhead of assembling the hook path and executing access(3P) once to check if it's there is mostly outweighed by the operating system's overhead. Signed-off-by: Patrick Steinhardt <[email protected]> Signed-off-by: Junio C Hamano <[email protected]>
1 parent 101b320 commit 6754159

File tree

4 files changed

+244
-2
lines changed

4 files changed

+244
-2
lines changed

‎Documentation/githooks.txt

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -404,6 +404,35 @@ Both standard output and standard error output are forwarded to
404404
`git send-pack` on the other end, so you can simply `echo` messages
405405
for the user.
406406

407+
ref-transaction
408+
~~~~~~~~~~~~~~~
409+
410+
This hook is invoked by any Git command that performs reference
411+
updates. It executes whenever a reference transaction is prepared,
412+
committed or aborted and may thus get called multiple times.
413+
414+
The hook takes exactly one argument, which is the current state the
415+
given reference transaction is in:
416+
417+
- "prepared": All reference updates have been queued to the
418+
transaction and references were locked on disk.
419+
420+
- "committed": The reference transaction was committed and all
421+
references now have their respective new value.
422+
423+
- "aborted": The reference transaction was aborted, no changes
424+
were performed and the locks have been released.
425+
426+
For each reference update that was added to the transaction, the hook
427+
receives on standard input a line of the format:
428+
429+
<old-value> SP <new-value> SP <ref-name> LF
430+
431+
The exit status of the hook is ignored for any state except for the
432+
"prepared" state. In the "prepared" state, a non-zero exit status will
433+
cause the transaction to be aborted. The hook will not be called with
434+
"aborted" state in that case.
435+
407436
push-to-checkout
408437
~~~~~~~~~~~~~~~~
409438

‎refs.c

Lines changed: 74 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,13 +9,15 @@
99
#include "iterator.h"
1010
#include "refs.h"
1111
#include "refs/refs-internal.h"
12+
#include "run-command.h"
1213
#include "object-store.h"
1314
#include "object.h"
1415
#include "tag.h"
1516
#include "submodule.h"
1617
#include "worktree.h"
1718
#include "argv-array.h"
1819
#include "repository.h"
20+
#include "sigchain.h"
1921

2022
/*
2123
* List of all available backends
@@ -1986,10 +1988,65 @@ int ref_update_reject_duplicates(struct string_list *refnames,
19861988
return 0;
19871989
}
19881990

1991+
static const char hook_not_found;
1992+
static const char *hook;
1993+
1994+
static int run_transaction_hook(struct ref_transaction *transaction,
1995+
const char *state)
1996+
{
1997+
struct child_process proc = CHILD_PROCESS_INIT;
1998+
struct strbuf buf = STRBUF_INIT;
1999+
int ret = 0, i;
2000+
2001+
if (hook == &hook_not_found)
2002+
return ret;
2003+
if (!hook)
2004+
hook = find_hook("reference-transaction");
2005+
if (!hook) {
2006+
hook = &hook_not_found;
2007+
return ret;
2008+
}
2009+
2010+
argv_array_pushl(&proc.args, hook, state, NULL);
2011+
proc.in = -1;
2012+
proc.stdout_to_stderr = 1;
2013+
proc.trace2_hook_name = "reference-transaction";
2014+
2015+
ret = start_command(&proc);
2016+
if (ret)
2017+
return ret;
2018+
2019+
sigchain_push(SIGPIPE, SIG_IGN);
2020+
2021+
for (i = 0; i < transaction->nr; i++) {
2022+
struct ref_update *update = transaction->updates[i];
2023+
2024+
strbuf_reset(&buf);
2025+
strbuf_addf(&buf, "%s %s %s\n",
2026+
oid_to_hex(&update->old_oid),
2027+
oid_to_hex(&update->new_oid),
2028+
update->refname);
2029+
2030+
if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
2031+
if (errno != EPIPE)
2032+
ret = -1;
2033+
break;
2034+
}
2035+
}
2036+
2037+
close(proc.in);
2038+
sigchain_pop(SIGPIPE);
2039+
strbuf_release(&buf);
2040+
2041+
ret |= finish_command(&proc);
2042+
return ret;
2043+
}
2044+
19892045
int ref_transaction_prepare(struct ref_transaction *transaction,
19902046
struct strbuf *err)
19912047
{
19922048
struct ref_store *refs = transaction->ref_store;
2049+
int ret;
19932050

19942051
switch (transaction->state) {
19952052
case REF_TRANSACTION_OPEN:
@@ -2012,7 +2069,17 @@ int ref_transaction_prepare(struct ref_transaction *transaction,
20122069
return -1;
20132070
}
20142071

2015-
return refs->be->transaction_prepare(refs, transaction, err);
2072+
ret = refs->be->transaction_prepare(refs, transaction, err);
2073+
if (ret)
2074+
return ret;
2075+
2076+
ret = run_transaction_hook(transaction, "prepared");
2077+
if (ret) {
2078+
ref_transaction_abort(transaction, err);
2079+
die(_("ref updates aborted by hook"));
2080+
}
2081+
2082+
return 0;
20162083
}
20172084

20182085
int ref_transaction_abort(struct ref_transaction *transaction,
@@ -2036,6 +2103,8 @@ int ref_transaction_abort(struct ref_transaction *transaction,
20362103
break;
20372104
}
20382105

2106+
run_transaction_hook(transaction, "aborted");
2107+
20392108
ref_transaction_free(transaction);
20402109
return ret;
20412110
}
@@ -2064,7 +2133,10 @@ int ref_transaction_commit(struct ref_transaction *transaction,
20642133
break;
20652134
}
20662135

2067-
return refs->be->transaction_finish(refs, transaction, err);
2136+
ret = refs->be->transaction_finish(refs, transaction, err);
2137+
if (!ret)
2138+
run_transaction_hook(transaction, "committed");
2139+
return ret;
20682140
}
20692141

20702142
int refs_verify_refname_available(struct ref_store *refs,

‎t/perf/p1400-update-ref.sh

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
#!/bin/sh
2+
3+
test_description="Tests performance of update-ref"
4+
5+
. ./perf-lib.sh
6+
7+
test_perf_fresh_repo
8+
9+
test_expect_success "setup" '
10+
test_commit PRE &&
11+
test_commit POST &&
12+
printf "create refs/heads/%d PRE\n" $(test_seq 1000) >create &&
13+
printf "update refs/heads/%d POST PRE\n" $(test_seq 1000) >update &&
14+
printf "delete refs/heads/%d POST\n" $(test_seq 1000) >delete
15+
'
16+
17+
test_perf "update-ref" '
18+
for i in $(test_seq 1000)
19+
do
20+
git update-ref refs/heads/branch PRE &&
21+
git update-ref refs/heads/branch POST PRE &&
22+
git update-ref -d refs/heads/branch
23+
done
24+
'
25+
26+
test_perf "update-ref --stdin" '
27+
git update-ref --stdin <create &&
28+
git update-ref --stdin <update &&
29+
git update-ref --stdin <delete
30+
'
31+
32+
test_done

‎t/t1416-ref-transaction-hooks.sh

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
#!/bin/sh
2+
3+
test_description='reference transaction hooks'
4+
5+
. ./test-lib.sh
6+
7+
test_expect_success setup '
8+
mkdir -p .git/hooks &&
9+
test_commit PRE &&
10+
test_commit POST &&
11+
POST_OID=$(git rev-parse POST)
12+
'
13+
14+
test_expect_success 'hook allows updating ref if successful' '
15+
test_when_finished "rm .git/hooks/reference-transaction" &&
16+
git reset --hard PRE &&
17+
write_script .git/hooks/reference-transaction <<-\EOF &&
18+
echo "$*" >>actual
19+
EOF
20+
cat >expect <<-EOF &&
21+
prepared
22+
committed
23+
EOF
24+
git update-ref HEAD POST &&
25+
test_cmp expect actual
26+
'
27+
28+
test_expect_success 'hook aborts updating ref in prepared state' '
29+
test_when_finished "rm .git/hooks/reference-transaction" &&
30+
git reset --hard PRE &&
31+
write_script .git/hooks/reference-transaction <<-\EOF &&
32+
if test "$1" = prepared
33+
then
34+
exit 1
35+
fi
36+
EOF
37+
test_must_fail git update-ref HEAD POST 2>err &&
38+
test_i18ngrep "ref updates aborted by hook" err
39+
'
40+
41+
test_expect_success 'hook gets all queued updates in prepared state' '
42+
test_when_finished "rm .git/hooks/reference-transaction actual" &&
43+
git reset --hard PRE &&
44+
write_script .git/hooks/reference-transaction <<-\EOF &&
45+
if test "$1" = prepared
46+
then
47+
while read -r line
48+
do
49+
printf "%s\n" "$line"
50+
done >actual
51+
fi
52+
EOF
53+
cat >expect <<-EOF &&
54+
$ZERO_OID $POST_OID HEAD
55+
$ZERO_OID $POST_OID refs/heads/master
56+
EOF
57+
git update-ref HEAD POST <<-EOF &&
58+
update HEAD $ZERO_OID $POST_OID
59+
update refs/heads/master $ZERO_OID $POST_OID
60+
EOF
61+
test_cmp expect actual
62+
'
63+
64+
test_expect_success 'hook gets all queued updates in committed state' '
65+
test_when_finished "rm .git/hooks/reference-transaction actual" &&
66+
git reset --hard PRE &&
67+
write_script .git/hooks/reference-transaction <<-\EOF &&
68+
if test "$1" = committed
69+
then
70+
while read -r line
71+
do
72+
printf "%s\n" "$line"
73+
done >actual
74+
fi
75+
EOF
76+
cat >expect <<-EOF &&
77+
$ZERO_OID $POST_OID HEAD
78+
$ZERO_OID $POST_OID refs/heads/master
79+
EOF
80+
git update-ref HEAD POST &&
81+
test_cmp expect actual
82+
'
83+
84+
test_expect_success 'hook gets all queued updates in aborted state' '
85+
test_when_finished "rm .git/hooks/reference-transaction actual" &&
86+
git reset --hard PRE &&
87+
write_script .git/hooks/reference-transaction <<-\EOF &&
88+
if test "$1" = aborted
89+
then
90+
while read -r line
91+
do
92+
printf "%s\n" "$line"
93+
done >actual
94+
fi
95+
EOF
96+
cat >expect <<-EOF &&
97+
$ZERO_OID $POST_OID HEAD
98+
$ZERO_OID $POST_OID refs/heads/master
99+
EOF
100+
git update-ref --stdin <<-EOF &&
101+
start
102+
update HEAD POST $ZERO_OID
103+
update refs/heads/master POST $ZERO_OID
104+
abort
105+
EOF
106+
test_cmp expect actual
107+
'
108+
109+
test_done

0 commit comments

Comments
 (0)