Skip to content
This repository was archived by the owner on Mar 27, 2024. It is now read-only.

Adding json processing example to ReadMe #26

Merged
merged 3 commits into from
Aug 18, 2017
Merged

Adding json processing example to ReadMe #26

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
fcb6899
Adding json processing example to ReadMe
abbytiz Aug 17, 2017
d78c4a9
Adding json snippet
abbytiz Aug 18, 2017
bba2d30
Merge branch 'master' into jsonPostProcessExample
abbytiz Aug 18, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 76 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -241,9 +241,84 @@ Packages found only in gcr.io/google-appengine/python:2017-06-29-190410: None
Version differences: None

```
## Example Run with json post-processing
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like it may be more interesting to include the actual json (I would make a very small sample one, not necessarily based on an existing image or anything) and the results of processing that json than to include the python script itself. Then you could just say-- I had this json created by running the node, pip, and apt differ on two images, and my python script did x, y, z to produce this output, etc....

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nkubala @aaron-prindle any preferences on including vs not including the python script? I had gotten the vibe that an example would include the actual post-processing script as a full example...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's probably fine to include the python code you used to process. anything that makes this easier for people to start using is a good thing imo :) but I also agree with @cftorres, it might also be useful to show a bit of the json that the script consumes to produce that final output. don't think I'd put the whole thing though, we don't want to pollute the readme

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a snippet of the json. Let me know if it's too little/too much/etc.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

The following example demonstrates how one might selectively display the output of their diff, such that version differences are ignored and only package absence/presence is displayed and the packages present in only one image are sorted by size in descending order. A small piece of the json being post-processed can be seen below:
```
[
{
"DiffType": "AptDiffer",
"Diff": {
"Image1": "gcr.io/gcp-runtimes/multi-base",
"Packages1": {},
"Image2": "gcr.io/gcp-runtimes/multi-modified",
"Packages2": {
"dh-python": {
"Version": "1.20141111-2",
"Size": "277"
},
"libmpdec2": {
"Version": "2.4.1-1",
"Size": "275"
},
...
```
The post-processing script used for this example is below:

```import sys, json

def main():
data = json.loads(sys.stdin.read())
img1packages = []
img2packages = []
for differ in data:
diff = differ['Diff']

if len(diff['Packages1']) > 0:
for package in diff['Packages1']:
Size = diff['Packages1'][package]['Size']
img1packages.append((str(package), int(str(Size))))

if len(diff['Packages2']) > 0:
for package in diff['Packages2']:
Size = diff['Packages2'][package]['Size']
img2packages.append((str(package), int(str(Size))))

img1packages = reversed(sorted(img1packages, key=lambda x: x[1]))
img2packages = reversed(sorted(img2packages, key=lambda x: x[1]))


print "Only in image1\n"
for pkg in img1packages:
print pkg
print "Only in image2\n"
for pkg in img2packages:
print pkg
print

if __name__ == "__main__":
main()
```

Given the above python script to postprocess json output, you can produce the following behavior:
```
container-diff gcr.io/gcp-runtimes/multi-base gcr.io/gcp-runtimes/multi-modified -a -j | python pyscript.py

Only in image1

Only in image2

## Make your own analyzer
('libpython3.4-stdlib', 9484)
('python3.4-minimal', 4506)
('libpython3.4-minimal', 3310)
('python3.4', 336)
('dh-python', 277)
('libmpdec2', 275)
('python3-minimal', 96)
('python3', 36)
('libpython3-stdlib', 28)

```
## Make your own differ

Feel free to develop your own analyzer leveraging the utils currently available. PRs are welcome.

Expand Down