Skip to content

Commit bd831a0

Browse files
committed
Merge branch 'ac/bitmap-format-doc' into seen
Adjust technical/bitmap-format to be formatted by AsciiDoc, and add some missing information to the documentation. * ac/bitmap-format-doc: bitmap-format.txt: add information for trailing checksum bitmap-format.txt: fix some formatting issues bitmap-format.txt: feed the file to asciidoc to generate html
2 parents 2a48c83 + 4007cbe commit bd831a0

File tree

2 files changed

+63
-51
lines changed

2 files changed

+63
-51
lines changed

Documentation/Makefile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,7 @@ TECH_DOCS += MyFirstContribution
9494
TECH_DOCS += MyFirstObjectWalk
9595
TECH_DOCS += SubmittingPatches
9696
TECH_DOCS += ToolsForGit
97+
TECH_DOCS += technical/bitmap-format
9798
TECH_DOCS += technical/bundle-format
9899
TECH_DOCS += technical/cruft-packs
99100
TECH_DOCS += technical/hash-function-transition

Documentation/technical/bitmap-format.txt

Lines changed: 62 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -39,19 +39,22 @@ MIDXs, both the bit-cache and rev-cache extensions are required.
3939

4040
== On-disk format
4141

42-
- A header appears at the beginning:
42+
* A header appears at the beginning:
4343

44-
4-byte signature: {'B', 'I', 'T', 'M'}
44+
4-byte signature: :: {'B', 'I', 'T', 'M'}
45+
46+
2-byte version number (network byte order): ::
4547

46-
2-byte version number (network byte order)
4748
The current implementation only supports version 1
4849
of the bitmap index (the same one as JGit).
4950

50-
2-byte flags (network byte order)
51+
2-byte flags (network byte order): ::
5152

5253
The following flags are supported:
5354

54-
- BITMAP_OPT_FULL_DAG (0x1) REQUIRED
55+
** {empty}
56+
BITMAP_OPT_FULL_DAG (0x1) REQUIRED: :::
57+
5558
This flag must always be present. It implies that the
5659
bitmap index has been generated for a packfile or
5760
multi-pack index (MIDX) with full closure (i.e. where
@@ -61,75 +64,83 @@ MIDXs, both the bit-cache and rev-cache extensions are required.
6164
JGit, that greatly reduces the complexity of the
6265
implementation.
6366

64-
- BITMAP_OPT_HASH_CACHE (0x4)
67+
** {empty}
68+
BITMAP_OPT_HASH_CACHE (0x4): :::
69+
6570
If present, the end of the bitmap file contains
6671
`N` 32-bit name-hash values, one per object in the
6772
pack/MIDX. The format and meaning of the name-hash is
6873
described below.
6974

70-
4-byte entry count (network byte order)
71-
75+
4-byte entry count (network byte order): ::
7276
The total count of entries (bitmapped commits) in this bitmap index.
7377

74-
20-byte checksum
75-
78+
20-byte checksum: ::
7679
The SHA1 checksum of the pack/MIDX this bitmap index
7780
belongs to.
7881

79-
- 4 EWAH bitmaps that act as type indexes
80-
81-
Type indexes are serialized after the hash cache in the shape
82-
of four EWAH bitmaps stored consecutively (see Appendix A for
83-
the serialization format of an EWAH bitmap).
84-
85-
There is a bitmap for each Git object type, stored in the following
86-
order:
87-
88-
- Commits
89-
- Trees
90-
- Blobs
91-
- Tags
92-
93-
In each bitmap, the `n`th bit is set to true if the `n`th object
94-
in the packfile or multi-pack index is of that type.
95-
96-
The obvious consequence is that the OR of all 4 bitmaps will result
97-
in a full set (all bits set), and the AND of all 4 bitmaps will
98-
result in an empty bitmap (no bits set).
99-
100-
- N entries with compressed bitmaps, one for each indexed commit
101-
102-
Where `N` is the total amount of entries in this bitmap index.
103-
Each entry contains the following:
104-
105-
- 4-byte object position (network byte order)
82+
* 4 EWAH bitmaps that act as type indexes
83+
+
84+
Type indexes are serialized after the hash cache in the shape
85+
of four EWAH bitmaps stored consecutively (see Appendix A for
86+
the serialization format of an EWAH bitmap).
87+
+
88+
There is a bitmap for each Git object type, stored in the following
89+
order:
90+
+
91+
- Commits
92+
- Trees
93+
- Blobs
94+
- Tags
95+
96+
+
97+
In each bitmap, the `n`th bit is set to true if the `n`th object
98+
in the packfile or multi-pack index is of that type.
99+
100+
The obvious consequence is that the OR of all 4 bitmaps will result
101+
in a full set (all bits set), and the AND of all 4 bitmaps will
102+
result in an empty bitmap (no bits set).
103+
104+
* N entries with compressed bitmaps, one for each indexed commit
105+
+
106+
Where `N` is the total amount of entries in this bitmap index.
107+
Each entry contains the following:
108+
109+
** {empty}
110+
4-byte object position (network byte order): ::
106111
The position **in the index for the packfile or
107112
multi-pack index** where the bitmap for this commit is
108113
found.
109114

110-
- 1-byte XOR-offset
115+
** {empty}
116+
1-byte XOR-offset: ::
111117
The xor offset used to compress this bitmap. For an entry
112118
in position `x`, a XOR offset of `y` means that the actual
113119
bitmap representing this commit is composed by XORing the
114120
bitmap for this entry with the bitmap in entry `x-y` (i.e.
115121
the bitmap `y` entries before this one).
116-
117-
Note that this compression can be recursive. In order to
118-
XOR this entry with a previous one, the previous entry needs
119-
to be decompressed first, and so on.
120-
121-
The hard-limit for this offset is 160 (an entry can only be
122-
xor'ed against one of the 160 entries preceding it). This
123-
number is always positive, and hence entries are always xor'ed
124-
with **previous** bitmaps, not bitmaps that will come afterwards
125-
in the index.
126-
127-
- 1-byte flags for this bitmap
122+
+
123+
NOTE: This compression can be recursive. In order to
124+
XOR this entry with a previous one, the previous entry needs
125+
to be decompressed first, and so on.
126+
+
127+
The hard-limit for this offset is 160 (an entry can only be
128+
xor'ed against one of the 160 entries preceding it). This
129+
number is always positive, and hence entries are always xor'ed
130+
with **previous** bitmaps, not bitmaps that will come afterwards
131+
in the index.
132+
133+
** {empty}
134+
1-byte flags for this bitmap: ::
128135
At the moment the only available flag is `0x1`, which hints
129136
that this bitmap can be re-used when rebuilding bitmap indexes
130137
for the repository.
131138

132-
- The compressed bitmap itself, see Appendix A.
139+
** The compressed bitmap itself, see Appendix A.
140+
141+
* {empty}
142+
TRAILER: ::
143+
Trailing checksum of the preceding contents.
133144

134145
== Appendix A: Serialization format for an EWAH bitmap
135146

0 commit comments

Comments
 (0)