Skip to content
This repository was archived by the owner on Mar 27, 2024. It is now read-only.

Commit 09aa6f8

Browse files
cftorrescftorres
committed
Merge remote-tracking branch 'upstream/master' into FileDiffSize
2 parents 2103dc7 + 57cb4b6 commit 09aa6f8

37 files changed

+1301
-678
lines changed

‎.container-diff-tests.sh

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,11 +7,19 @@ while IFS=$' \n\r' read -r flag differ image1 image2 file; do
77
fi
88
done < tests/differ_runs.txt
99

10+
while IFS=$' \n\r' read -r flag analyzer image file; do
11+
go run main.go $image $flag -j > $file
12+
if [[ $? -ne 0 ]]; then
13+
echo "container-diff" "$analyzer" "analyzer failed"
14+
exit 1
15+
fi
16+
done < tests/analyzer_runs.txt
17+
1018
success=0
11-
while IFS=$' \n\r' read -r differ actual expected; do
19+
while IFS=$' \n\r' read -r type analyzer actual expected; do
1220
diff=$(jq --argfile a "$actual" --argfile b "$expected" -n 'def walk(f): . as $in | if type == "object" then reduce keys[] as $key ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f elif type == "array" then map( walk(f) ) | f else f end; ($a | walk(if type == "array" then sort else . end)) as $a | ($b | walk(if type == "array" then sort else . end)) as $b | $a == $b')
1321
if ! "$diff" ; then
14-
echo "container diff" "$differ" "diff output is not as expected"
22+
echo "container-diff" "$analyzer" "$type" "output is not as expected"
1523
success=1
1624
fi
1725
done < tests/diff_comparisons.txt

‎README.md

Lines changed: 96 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,13 @@ Status](https://travis-ci.org/GoogleCloudPlatform/container-diff.svg?branch=mast
55

66
## What is container-diff?
77

8-
container-diff is an image differ command line tool. container-diff can diff two images along several different criteria, currently including:
8+
container-diff is an image analysis command line tool. container-diff can analyze images along several different criteria, currently including:
99
- Docker Image History
1010
- Image file system
1111
- apt-get installed packages
1212
- pip installed packages
1313
- npm installed packages
14+
The above analyses can be performed on a single image, or a diff can be performed on two images to compare images.
1415

1516
This tool can help you as a developer better understand what is changing within your images and better understand what your images contain.
1617

@@ -32,8 +33,18 @@ Download the [container-diff-windows-amd64.exe](https://storage.googleapis.com/c
3233

3334
## Quickstart
3435

35-
To use container-diff you need two Docker images (in the form of an ID, tarball, or URL from a repo). Once you have those images you can run any of the following differs:
36+
To use container-diff to perform analysis on a single image, you need one Docker image (in the form of an ID, tarball, or URL from a repo). Once you have that image, you can run any of the following analyzers:
3637

38+
```
39+
container-diff <img> [Run all analyzers]
40+
container-diff <img> -d [History]
41+
container-diff <img> -f [File System]
42+
container-diff <img> -p [Pip]
43+
container-diff <img> -a [Apt]
44+
container-diff <img> -n [Node]
45+
```
46+
47+
To use container-diff to perform a diff analysis on two images, you need two Docker images (in the form of an ID, tarball, or URL from a repo). Once you have those images, you can run any of the following differs:
3748
```
3849
container-diff <img1> <img2> [Run all differs]
3950
container-diff <img1> <img2> -d [History]
@@ -43,12 +54,13 @@ container-diff <img1> <img2> -a [Apt]
4354
container-diff <img1> <img2> -n [Node]
4455
```
4556

46-
You can similarly run many differs at once:
57+
You can similarly run many differs or analyzers at once:
4758

4859
```
4960
container-diff <img1> <img2> -d -a -n [History, Apt, and Node]
5061
```
51-
All of the differ flags with their long versions can be seen below:
62+
63+
All of the analyzer flags with their long versions can be seen below:
5264

5365
| Differ | Short flag | Long Flag |
5466
| ------------------------- |:----------:| ----------:|
@@ -71,30 +83,79 @@ To use the docker client instead of shelling out to your local docker daemon, ad
7183

7284
```container-diff <img1> <img2> -e```
7385

86+
## Analysis Result Format
87+
88+
The JSONs for analysis results are in the following format:
89+
```
90+
{
91+
"Image": "foo",
92+
"AnalyzeType": "Apt",
93+
"Analysis": {},
94+
}
95+
```
96+
The possible structures of the `Analysis` field are detailed below.
97+
98+
### History Analysis
99+
100+
The history analyzer outputs a list of strings representing descriptions of how an image layer was created.
101+
102+
### Filesystem Analysis
103+
104+
The filesystem analyzer outputs a list of strings representing filesystem contents.
105+
106+
### Package Analysis
107+
108+
Package analyzers such as pip, apt, and node inspect the packages installed within the image provided. All package analyses leverage the PackageInfo struct, which contains the version and size for a given package instance, as detailed below:
109+
```
110+
type PackageInfo struct {
111+
Version string
112+
Size string
113+
}
114+
```
115+
116+
#### Single Version Package Analysis
117+
118+
Single version package analyzers (apt) have the following output structure: `map[string]PackageInfo`
119+
120+
In this mapping scheme, each package name is mapped to its PackageInfo as described above.
121+
122+
#### Multi Version Package Analysis
123+
124+
Multi version package analyzers (pip, node) have the following output structure: `map[string]map[string]PackageInfo`
74125

75-
## Output Format
126+
In this mapping scheme, each package name corresponds to another map where the filesystem path to each unique instance of the package (i.e. unique version and/or size info) is mapped to that package instance's PackageInfo.
127+
128+
129+
## Diff Result Format
130+
131+
The JSONs for diff results are in the following format:
132+
```
133+
{
134+
"Image1": "foo",
135+
"Image2": "bar",
136+
"DiffType": "Apt",
137+
"Diff": {},
138+
}
139+
```
140+
The possible structures of the `Diff` field are detailed below.
76141

77142
### History Diff
78143

79144
The history differ has the following json output structure:
80145

81146
```
82147
type HistDiff struct {
83-
Image1 string
84-
Image2 string
85148
Adds []string
86149
Dels []string
87150
}
88151
```
89152

90-
### File System Diff
153+
### Filesystem Diff
91154

92-
The files system differ has the following json output structure:
155+
The filesystem differ has the following json output structure:
93156

94157
```
95158
type DirDiff struct {
96-
Image1 string
97-
Image2 string
98159
Adds []string
99160
Dels []string
100161
Mods []string
@@ -105,44 +166,33 @@ type DirDiff struct {
105166

106167
Package differs such as pip, apt, and node inspect the packages contained within the images provided. All packages differs currently leverage the PackageInfo struct which contains the version and size for a given package instance.
107168

108-
```
109-
type PackageInfo struct {
110-
Version string
111-
Size string
112-
}
113-
```
114-
115-
#### Single Version Diffs
169+
#### Single Version Package Diffs
116170

117171
Single version differs (apt) have the following json output structure:
118172

119173
```
120174
type PackageDiff struct {
121-
Image1 string
122175
Packages1 map[string]PackageInfo
123-
Image2 string
124176
Packages2 map[string]PackageInfo
125177
InfoDiff []Info
126178
}
127179
```
128180

129-
Image1 and Image2 are the image names. Packages1 and Packages2 map package names to PackageInfo structs which contain the version and size of the package. InfoDiff contains a list of Info structs, each of which contains the package name (which occurred in both images but had a difference in size or version), and the PackageInfo struct for each package instance.
181+
Packages1 and Packages2 map package names to PackageInfo structs which contain the version and size of the package. InfoDiff contains a list of Info structs, each of which contains the package name (which occurred in both images but had a difference in size or version), and the PackageInfo struct for each package instance.
130182

131-
#### Multi Version Diffs
183+
#### Multi Version Package Diffs
132184

133185
The multi version differs (pip, node) support processing images which may have multiple versions of the same package. Below is the json output structure:
134186

135187
```
136188
type MultiVersionPackageDiff struct {
137-
Image1 string
138189
Packages1 map[string]map[string]PackageInfo
139-
Image2 string
140190
Packages2 map[string]map[string]PackageInfo
141191
InfoDiff []MultiVersionInfo
142192
}
143193
```
144194

145-
Image1 and Image2 are the image names. Packages1 and Packages2 map package name to path where the package was found to PackageInfo struct (version and size of that package instance). InfoDiff here is exanded to allow for multiple versions to be associated with a single package.
195+
Packages1 and Packages2 map package name to path where the package was found to PackageInfo struct (version and size of that package instance). InfoDiff here is exanded to allow for multiple versions to be associated with a single package.
146196

147197
```
148198
type MultiVersionInfo struct {
@@ -193,46 +243,43 @@ Version differences: None
193243
```
194244

195245

196-
## Make your own differ
246+
## Make your own analyzer
197247

198-
Feel free to develop your own differ leveraging the utils currently available. PRs are welcome.
248+
Feel free to develop your own analyzer leveraging the utils currently available. PRs are welcome.
199249

200-
### Custom Differ Quickstart
250+
### Custom Analyzer Quickstart
201251

202-
In order to quickly make your own differ, follow these steps:
252+
In order to quickly make your own analyzer, follow these steps:
203253

204-
1. Add your diff identifier to the flags in [root.go](https://github.com/GoogleCloudPlatform/container-diff/blob/ReadMe/cmd/root.go)
205-
2. Determine if you can use existing differ tools. If you can make use of existing tools, you then need to construct the structs to feed into the diff tools by getting all of the packages for each image or the analogous quality to be diffed. To determine if you can leverage existing tools, think through these questions:
206-
- Are you trying to diff packages?
254+
1. Add your analyzer identifier to the flags in [root.go](https://github.com/GoogleCloudPlatform/container-diff/blob/ReadMe/cmd/root.go)
255+
2. Determine if you can use existing analyzing or diffing tools. If you can make use of existing tools, you then need to construct the structs to feed into the tools by getting all of the packages for each image or the analogous quality to be analyzed. To determine if you can leverage existing tools, think through these questions:
256+
- Are you trying to analyze packages?
207257
- Yes: Does the relevant package manager support different versions of the same package on one image?
208-
- Yes: Use `GetMultiVerisonMapDiff` to diff `map[string]map[string]utils.PackageInfo` objects. See [nodeDiff.go](https://github.com/GoogleCloudPlatform/container-diff/blob/master/differs/nodeDiff.go#L33) or [pipDiff.go](https://github.com/GoogleCloudPlatform/container-diff/blob/master/differs/pipDiff.go#L23) for examples.
209-
- No: Use `GetMapDiff` to diff `map[string]utils.PackageInfo` objects. See [aptDiff.go](https://github.com/GoogleCloudPlatform/container-diff/blob/master/differs/aptDiff.go#L29).
258+
- Yes: Implement `getPackages` to collect all versions of all packages within an image in a `map[string]map[string]PackageInfo`. Use `GetMultiVerisonMapDiff` to diff map objects. See [nodeDiff.go](https://github.com/GoogleCloudPlatform/container-diff/blob/master/differs/nodeDiff.go#L33) or [pipDiff.go](https://github.com/GoogleCloudPlatform/container-diff/blob/master/differs/pipDiff.go#L23) for examples.
259+
- No: Implement `getPackages` to collect all versions of all packages within an image in a `map[string]PackageInfo`. Use `GetMapDiff` to diff map objects. See [aptDiff.go](https://github.com/GoogleCloudPlatform/container-diff/blob/master/differs/aptDiff.go#L29).
210260
- No: Look to [History](https://github.com/GoogleCloudPlatform/container-diff/blob/ReadMe/differs/historyDiff.go) and [File System](https://github.com/GoogleCloudPlatform/container-diff/blob/ReadMe/differs/fileDiff.go) differs as models for diffing.
211261

212-
3. Write your Diff driver in the `differs` directory, such that you have a struct for your differ type and a method for that differ called Diff:
262+
3. Write your analyzer driver in the `differs` directory, such that you have a struct for your analyzer type and two method for that differ: `Analyze` for single image analysis and `Diff` for comparison between two images:
213263

214264
```
215-
type YourDiffer struct {}
265+
type YourAnalyzer struct {}
216266
217-
func (d YourDiffer) Diff(image1, image2 utils.Image) (DiffResult, error) {...}
267+
func (a YourAnalyzer) Analyze(image utils.Image) (utils.AnalyzeResult, error) {...}
268+
func (a YourAnalyzer) Diff(image1, image2 utils.Image) (utils.DiffResult, error) {...}
218269
```
219-
The arguments passed to your differ contain the path to the unpacked tar representation of the image. That path can be accessed as such: `image1.FSPath`.
270+
The image arguments passed to your analyzer contain the path to the unpacked tar representation of the image, as well as certain configuration information (e.g. environment variables upon image creation and image history).
220271

221-
If using existing package differ tools, you should create the appropriate structs to diff (determined in step 2 - either `map[string]map[string]utils.PackageInfo` or `map[string]utils.PackageInfo`) and then call the appropriate get diff function (also determined in step2 - either `GetMultiVerisonMapDiff` or `GetMapDiff`).
272+
If using existing package differ tools, you should create the appropriate structs to analyze or diff. Otherwise, create your own analyzer which should yield information to fill an AnalyzeResult or DiffResult in the next step.
222273

223-
Otherwise, create your own differ which should yield information to fill a DiffResult in the next step.
224-
225-
4. Create a DiffResult for your differ.
274+
4. Create a result struct following either the AnalyzeResult or DiffResult interface by implementing the following two methods.
226275
```
227-
type DiffResult interface {
228-
GetStruct() DiffResult
229-
OutputText(diffType string) error
230-
}
276+
GetStruct() DiffResult
277+
OutputText(diffType string) error
231278
```
232279

233-
This is where you define how your differ should output for a human readable format (`OutputText`) and as a struct which can then be written to a `.json` file. See [output_utils.go](https://github.com/GoogleCloudPlatform/container-diff/blob/master/utils/output_utils.go).
280+
This is where you define how your analyzer should output for a human readable format (`OutputText`) and as a struct which can then be written to a `.json` file. See [diff_output_utils.go](https://github.com/GoogleCloudPlatform/container-diff/blob/master/utils/diff_output_utils.go) and [analyze_output_utils.go](https://github.com/GoogleCloudPlatform/container-diff/blob/master/analyze_output_utils.go).
234281

235-
5. Add your differ to the diffs map in [differs.go](https://github.com/GoogleCloudPlatform/container-diff/blob/master/differs/differs.go#L22) with the corresponding Differ struct as the value.
282+
5. Add your analyzer to the `analyses` map in [differs.go](https://github.com/GoogleCloudPlatform/container-diff/blob/master/differs/differs.go#L22) with the corresponding Analyzer struct as the value.
236283

237284

238285

0 commit comments

Comments
 (0)