Skip to content

Meeting notes

Jump to bottom
Pierre-Étienne Fiquet edited this page Jun 27, 2018 · 28 revisions

Meeting 5: June 28

Meeting 4: June 20

Pytorch FFT / conv:

  • Is mixed radix implemented in pytorch.fft ?
  • Boundary handling - important especially in a multi scale context
  • Extrapolation - double refection in 1D, but does not generalize (?!)
  • Reflexion boundaries, add three reflected copies - math equivalent to DCT and corresponding convolution theorem
  • NOTE: pytorch padding layers offers great flexibility (https://pytorch.org/docs/stable/nn.html?#padding-layers)

Made repository private and pushed first commits

  1. ran automated script from 2to3 - https://docs.python.org/3/library/2to3.html#to3-reference
  2. Jimmy/Billy/pe started to organize:
  • C code (and functions that interact with the C code, for handling convolutions mostly)
  • get_filters
  • synthetic_images
  • image_tools
  1. TODO:
  • display - using matplotlib
  • pyramids - (use full name, eg. LaplacianPyramid - not Lpyr)

Meeting 3: June 14

Created GitHub organization
Waiting for academic approval

Discussed existing implementations out there:
Scikit-image
Pytorch SSIM
https://github.com/Po-Hsun-Su/pytorch-ssim
https://github.com/jorge-pessoa/pytorch-msssim

Meeting 2: June 7

big picture [Eero]

{x, θ, y} : inputs, outputs, parameters

fix adjust
simulate {x, θ} {y}
fit {x, y} {θ}
synthesize {y, θ} {x}

Some action items:

  1. Eero will transition our lab account (https://github.com/LabForComputationalVision) to an “organization” account, that will be more flexible/better for lab use.

  2. Eero will (gradually) with Zahra’s help, move our old matlab coder sources to GitHub, and eventually replace the lab software page (http://www.cns.nyu.edu/~lcv/software.php) with a direct redirect the gitHub page.

  3. Overall, preference for use of pyTorch (but TensorFlow and matlab also acceptable).

matlabPyrTools:

  • Image display / stats [plain python]
  • Pyramid display / stats [plain python]
  • Multi-scale pyramid implementations: Gaussian/Laplacian/Wavelet/Steerable pyramids [pyTorch]
  • Convolution, with subsampling, and boundary handling [add new features/capabilities to existing pyToorch convolution]

Other current LCV code (from software page):

  • MS-SSIM - is there a (good) pyTorch version?
  • NLPD
  • CBP -
  • End-to-end-comression (Johannes) - should package up the matlab code that’s on the software page
  • MT model
  • GSM denoising code [python version?]
  • Texture synthesis code [pyton version?]

Jimmy - Retina Efficient Coding - tf [private for now]
Jimmy/Yifei - LogDet - tf [Jimmy will submit pull request to tensorFlow folks]
Olivier - steerablePyramid - pt [Nikhil has a forked version of Olivier’s code]
Billy - MAD competition - tf [Billy will re-write in pyTorch]

Meeting 1: May 30 (Eero was out of town)

The stability flexibility trade-off

LCV GitHub organization
In the spirit of healthy and reproducible research we aim for a clean Lab GitHub page containing repositories with well documented and functional code.
Billy - Set GitHub organization, partition into public / private repositories, give access to lab members

Choice of Software
We generally lean in the direction of pyTorch (pt) over TensorFlow (tf) because of its flexibility (note that version 1.0 of pt will be released in the coming weeks)
We think it should be possible to make up for the missing elements from this library (imaginary numbers, FFT, Lop Rop, etc)

INVENTORY

Ready
Content on the LCV website
NLP distance- theano
GDN coder - m
Content of share/lcv/toolbox

Jimmy - Retina Efficient Coding - tf
Jimmy Yifei - LogDet - tf
Olivier - steerablePyramid - pt
Billy - MAD competition - tf (maybe pt)

In progress
Jimmy Kede - Gaussianization - tf (maybe pt)
Manu - MT model and motion cloud stimuli - python
Billy - pypyrtools in python 3
PE - speed comparison C vs pt
Nikhil - optimization for textures - pt
Paul - V1 normalization model, based off of Goris - python and tf

In question
Alex - eigendistortion compatible with differentiable representation models
Olivier - geodesics optimization
Caro - Modulated poisson
Manu - convolutional LNP - m and pt
Manu - auditory textures (already on McDermott web?)
Manu - CPB - m

Next meetings Thursdays at 2pm.
From now on planning and notes from meeting will be on the Github wiki.
After the first wave (pushing code that is ready), these meetings will be an occasion to get coding help from one another

Clone this wiki locally