Skip to content

Home

Jump to bottom
Pierre-Étienne Fiquet edited this page Jun 21, 2018 · 21 revisions

Welcome to the pyrtools wiki!

For now, we'll use this to organize how we revamp the code and keep notes on the process.

Organization

We need to overhaul how the code is organized. Right now, it looks a lot like MATLAB, with a single file per function. To make it more pythonic, we should have several files that contain a variety of related functions. Proposed files:

  • synthetic_images (aka mkTools)
  • filters
  • c code (convolutions)
  • image statistics / non-display tools
  • pyramids
  • display tools

Pyramid hierarchy

Right now, the pyramids inherit classes like this:

pyramid
|
|---------------|
|               |
Spyr            Lpyr
|               |
SFpyr           |------------|
|               |            |
SCFpyr          Gpyr        Wpyr

pyramid is pretty empty right now, should update to contain more meaningful methods (for example, can probably put showPyr) there and see if it's possible to consolidate the separate streams for Lpyr and Spyr.

meeting notes

Meeting 5: June 28

Meeting 4: June 20

Pytorch FFT / conv:

  • Boundary handling - important especially in a multi scale context

  • Extrapolation - double refection in 1D, but does not generalize (?!)

  • Reflexion boundaries, add three reflected copies - math equivalent to DCT and corresponding convolution theorem

  • Is mixed radix implemented in pytorch.fft ?

Actually plenty of padding layers But frankly windowing (mkDisk)

Made repository private and started work/commits (with awesome atom)

  1. Automated script from 2to3 - https://docs.python.org/3/library/2to3.html#to3-reference
  2. Jimmy/Billy/pe started to work on:
  • C
  • get_filters
  • synthetic_images
  • image_tools
  1. TODO:
  • display - using matplotlib
  • pyramids
  • PyrUtilities - into the base class pyramid Full names - LaplacianPyramid Notebook tutorial for basics

Missing pointOp.py - Lost in translation ?

Meeting 3: June 14

Created GitHub organization Waiting for academic approval

Discussed existing implementations out there: Scikit-image Pytorch SSIM https://github.com/Po-Hsun-Su/pytorch-ssim https://github.com/jorge-pessoa/pytorch-msssim

Meeting 2: June 7

big picture [Eero]

Three ingredients of a model/system Inputs, outputs, parameters: {x, θ, y} Simulate fix {x, θ} -> adjust {y} Fit fix {x, y} -> adjust {θ} Synthesize fix {y, θ} -> adjust {x}

Some action items:

  1. Eero will transition our lab account (https://github.com/LabForComputationalVision) to an “organization” account, that will be more flexible/better for lab use.

  2. Eero will (gradually) with Zahra’s help, move our old matlab coder sources to GitHub, and eventually replace the lab software page (http://www.cns.nyu.edu/~lcv/software.php) with a direct redirect the gitHub page.

  3. Overalll, preference for use of pyTorch (but TensorFlow and matlab also acceptable).

matlabPyrTools:

  • Image display / stats [plain python]
  • Pyramid display / stats [plain python]
  • Multi-scale pyramid implementations: Gaussian/Laplacian/Wavelet/Steerable pyramids [pyTorch]
  • Convolution, with subsampling, and boundary handling [add new features/capabilities to existing pyToorch convolution]

Other current LCV code (from software page):

  • MS-SSIM - is there a (good) pyTorch version?
  • NLPD
  • CBP -
  • End-to-end-comression (Johannes) - should package up the matlab code that’s on the software page
  • MT model
  • GSM denoising code [python version?]
  • Texture synthesis code [pyton version?]

Jimmy - Retina Efficient Coding - tf [private for now] Jimmy/Yifei - LogDet - tf [Jimmy will submit pull request to tensorFlow folks] Olivier - steerablePyramid - pt [Nikhil has a forked version of Olivier’s code] Billy - MAD competition - tf [Billy will re-write in pyTorch]

Meeting 1: May 30 (Eero was out of town)

The stability flexibility trade-off

**LCV GitHub organization ** In the spirit of healthy and reproducible research we aim for a clean Lab GitHub page containing repositories with well documented and functional code. Billy - Set GitHub organization, partition into public / private repositories, give access to lab members

Choice of Software We generally lean in the direction of pyTorch (pt) over TensorFlow (tf) because of its flexibility (note that version 1.0 of pt will be released in the coming weeks) We think it should be possible to make up for the missing elements from this library (imaginary numbers, FFT, Lop Rop, etc)

INVENTORY

Ready Content on the LCV website NLP distance- theano GDN coder - m Content of share/lcv/toolbox

Jimmy - Retina Efficient Coding - tf Jimmy Yifei - LogDet - tf Olivier - steerablePyramid - pt Billy - MAD competition - tf (maybe pt)

In progress Jimmy Kede - Gaussianization - tf (maybe pt) Manu - MT model and motion cloud stimuli - python Billy - pypyrtools in python 3 PE - speed comparison C vs pt Nikhil - optimization for textures - pt Paul - V1 normalization model, based off of Goris - python and tf

In question Alex - eigendistortion compatible with differentiable representation models Olivier - geodesics optimization Caro - Modulated poisson Manu - convolutional LNP - m and pt Manu - auditory textures (already on McDermott web?) Manu - CPB - m

Next meetings Thursdays at 2pm From now on planning and notes from meeting will be on the Github wiki After the first wave (pushing code that is ready), these meetings will be an occasion to get coding help from one another

Clone this wiki locally