Update Pymc3_tips_and_heuristics notebook #4036

AmitKus · 2020-07-30T04:32:57Z

Fixed KeyError: Rhat
Fixed: FutureWarnings
Updated the notebook to follow the consistent style (https://github.com/pymc-devs/pymc3/wiki/PyMC's-Jupyter-Notebook-Style)
Use arviz functions directly for plotting the inference data

Thank your for opening a PR!

Depending on what your PR does, here are a few things you might want to address in the description:

what are the (breaking) changes that this PR makes?
important background, or details about the implementation
are the changes—especially new features—covered by tests and docstrings?
consider adding/updating relevant example notebooks
right before it's ready to merge, mention the PR in the RELEASE-NOTES.md

review-notebook-app · 2020-07-30T04:33:02Z

Check out this pull request on

Review Jupyter notebook visual diffs & provide feedback on notebooks.

Powered by ReviewNB

review-notebook-app · 2020-07-30T07:15:38Z

View / edit / reply to this conversation on ReviewNB

twiecki commented on 2020-07-30T07:15:38Z
----------------------------------------------------------------

heuristics

review-notebook-app · 2020-07-30T07:15:39Z

View / edit / reply to this conversation on ReviewNB

twiecki commented on 2020-07-30T07:15:39Z
----------------------------------------------------------------

problems, regions, to sample from. Also, can you try and see what happens without ADVI init?

AmitKus commented on 2020-08-01T05:09:52Z
----------------------------------------------------------------

Without ADVI, it takes 20X longer for the same settings and some chains contain only diverging samples.

AmitKus commented on 2020-08-01T05:12:25Z
----------------------------------------------------------------

Auto-assigning NUTS sampler...
Initializing NUTS using jitter+adapt_diag...
Multiprocess sampling (4 chains in 4 jobs)
NUTS: [mu_phi, theta, tau_c, tau_h, beta1, beta0]

100.00% [6000/6000 1:06:22<00:00 Sampling 4 chains, 1,001 divergences]

Sampling 4 chains for 500 tune and 1_000 draw iterations (2_000 + 4_000 draws total) took 3984 seconds.

The chain contains only diverging samples. The model is probably misspecified.
The acceptance probability does not match the target. It is 0.0037493742425586075, but should be close to 0.9. Try to increase the number of tuning steps.
There was 1 divergence after tuning. Increase target_accept or reparameterize.
The acceptance probability does not match the target. It is 0.7159791144918716, but should be close to 0.9. Try to increase the number of tuning steps.
The rhat statistic is larger than 1.4 for some parameters. The sampler did not converge.
The estimated number of effective samples is smaller than 200 for some parameters.

review-notebook-app · 2020-07-30T07:15:40Z

View / edit / reply to this conversation on ReviewNB

twiecki commented on 2020-07-30T07:15:39Z
----------------------------------------------------------------

simulated

review-notebook-app · 2020-07-30T07:15:40Z

View / edit / reply to this conversation on ReviewNB

twiecki commented on 2020-07-30T07:15:40Z
----------------------------------------------------------------

also try without ADVI init.

AmitKus commented on 2020-08-01T05:10:52Z
----------------------------------------------------------------

Similar story without ADVI here too. Sampling takes an order of magnitude longer.

AmitKus commented on 2020-08-01T05:11:33Z
----------------------------------------------------------------

Sampling 4 chains for 500 tune and 1_000 draw iterations (2_000 + 4_000 draws total) took 3015 seconds.
There were 41 divergences after tuning. Increase target_accept or reparameterize.
The acceptance probability does not match the target. It is 0.7040791687604867, but should be close to 0.9. Try to increase the number of tuning steps.
There were 993 divergences after tuning. Increase target_accept or reparameterize.
The acceptance probability does not match the target. It is 7.239174982000396e-38, but should be close to 0.9. Try to increase the number of tuning steps.
There were 4 divergences after tuning. Increase target_accept or reparameterize.
The acceptance probability does not match the target. It is 0.6825918614079135, but should be close to 0.9. Try to increase the number of tuning steps.
The acceptance probability does not match the target. It is 0.7627370968011182, but should be close to 0.9. Try to increase the number of tuning steps.
The rhat statistic is larger than 1.4 for some parameters. The sampler did not converge.
The estimated number of effective samples is smaller than 200 for some parameters.

review-notebook-app · 2020-07-30T07:15:41Z

View / edit / reply to this conversation on ReviewNB

twiecki commented on 2020-07-30T07:15:41Z
----------------------------------------------------------------

can you update to theano 1.0.5 and rerun? those warnings should disappear.

AmitKus commented on 2020-08-01T05:13:39Z
----------------------------------------------------------------

Fixed this and all the other suggestions above. Thanks.

fonnesbeck · 2020-07-30T13:51:13Z

Since this notebook is being updated, it should probably be renamed. Its not a general notebook about PyMC3 tips, as the name implies; Its a very specific case study of CAR modeling.

AmitKus · 2020-07-31T02:40:05Z

Since this notebook is being updated, it should probably be renamed. Its not a general notebook about PyMC3 tips, as the name implies; Its a very specific case study of CAR modeling.

How about simply renaming it to conditional-autoregressive-model.ipynb?

AmitKus · 2020-08-01T05:09:53Z

Without ADVI, it takes 20X longer for the same settings and some chains contain only diverging samples.

View entire conversation on ReviewNB

AmitKus · 2020-08-01T05:10:53Z

Similar story without ADVI here too. Sampling takes an order of magnitude longer.

View entire conversation on ReviewNB

AmitKus · 2020-08-01T05:11:33Z

Sampling 4 chains for 500 tune and 1_000 draw iterations (2_000 + 4_000 draws total) took 3015 seconds.
There were 41 divergences after tuning. Increase target_accept or reparameterize.
The acceptance probability does not match the target. It is 0.7040791687604867, but should be close to 0.9. Try to increase the number of tuning steps.
There were 993 divergences after tuning. Increase target_accept or reparameterize.
The acceptance probability does not match the target. It is 7.239174982000396e-38, but should be close to 0.9. Try to increase the number of tuning steps.
There were 4 divergences after tuning. Increase target_accept or reparameterize.
The acceptance probability does not match the target. It is 0.6825918614079135, but should be close to 0.9. Try to increase the number of tuning steps.
The acceptance probability does not match the target. It is 0.7627370968011182, but should be close to 0.9. Try to increase the number of tuning steps.
The rhat statistic is larger than 1.4 for some parameters. The sampler did not converge.
The estimated number of effective samples is smaller than 200 for some parameters.

View entire conversation on ReviewNB

AmitKus · 2020-08-01T05:12:26Z

Auto-assigning NUTS sampler...
Initializing NUTS using jitter+adapt_diag...
Multiprocess sampling (4 chains in 4 jobs)
NUTS: [mu_phi, theta, tau_c, tau_h, beta1, beta0]

100.00% [6000/6000 1:06:22<00:00 Sampling 4 chains, 1,001 divergences]

Sampling 4 chains for 500 tune and 1_000 draw iterations (2_000 + 4_000 draws total) took 3984 seconds.

The chain contains only diverging samples. The model is probably misspecified.
The acceptance probability does not match the target. It is 0.0037493742425586075, but should be close to 0.9. Try to increase the number of tuning steps.
There was 1 divergence after tuning. Increase target_accept or reparameterize.
The acceptance probability does not match the target. It is 0.7159791144918716, but should be close to 0.9. Try to increase the number of tuning steps.
The rhat statistic is larger than 1.4 for some parameters. The sampler did not converge.
The estimated number of effective samples is smaller than 200 for some parameters.

View entire conversation on ReviewNB

AmitKus · 2020-08-01T05:13:40Z

Fixed this and all the other suggestions above. Thanks.

View entire conversation on ReviewNB

AlexAndorra

Looks all good now, thanks @AmitKus !
Good idea: let's just rename it conditional-autoregressive-model 👍
Also, don't forget to run black_nbconvert`, as per the style guide 😉

junpenglao · 2020-08-01T18:01:40Z

I remember last time when I was updating the notebook, inference without ADVI is pretty problematic - I think you should try more informative prior.

Also, since you are renaming the file, you should do a grep and rename other places where we reference the name (mostly in the doc set up)

codecov · 2020-08-10T01:31:04Z

Codecov Report

Merging #4036 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #4036   +/-   ##
=======================================
  Coverage   86.82%   86.82%           
=======================================
  Files          88       88           
  Lines       14154    14154           
=======================================
  Hits        12289    12289           
  Misses       1865     1865

docs/source/notebooks/table_of_contents_tutorials.js

review-notebook-app · 2020-08-10T07:23:24Z

View / edit / reply to this conversation on ReviewNB

junpenglao commented on 2020-08-10T07:23:24Z
----------------------------------------------------------------

Change title to "Conditional Autoregressive (CAR) model"

AlexAndorra

Thanks @AmitKus, looking good now! Last necessary changes I think:

Did you look for other places where the old NB name was used and replaced it with the new name, as @junpenglao suggested? The references will probably be concentrated in the docs set-up.
I think the cell with all the data is overcrowded and should be replaced (see my comment below).
Don't forget to run black_nbconvert when the NB is ready, as explained in the style guide.

review-notebook-app · 2020-08-12T15:53:59Z

View / edit / reply to this conversation on ReviewNB

AlexAndorra commented on 2020-08-12T15:53:58Z
----------------------------------------------------------------

I think it might be worse putting all this data in a CSV on the repo, and loading this CSV here. That way, this cell will be dedicated to data loading and transformations and will gain in clarity IMO

2) Updated the notebook to follow the consistent style 3) Use arviz functions directly for plotting on inference data Fromat the notebook in Black-format Minor formatting minor change to execute last cell Minor text changes. Renaming file update the filename in the doc: PyMC3_tips_and_heuristic.ipynb to conditional-autoregressive-model.ipynb Minos change to remove .ipynb from the name Move conditional-autoregressive-mode.ipynb from tutorial "How-To" to examples "Gaussian Processes"

…rd-coded in conditional-autoregressive-model.ipynb

AmitKus · 2020-08-15T04:22:48Z

Thanks @AmitKus, looking good now! Last necessary changes I think:

* Did you look for other places where the old NB name was used and replaced it with the new name, as @junpenglao suggested? The references will probably be concentrated in the docs set-up.

* I think the cell with all the data is overcrowded and should be replaced (see my comment below).

* Don't forget to run `black_nbconvert` when the NB is ready, as explained in the [style guide](https://github.com/pymc-devs/pymc3/wiki/PyMC's-Jupyter-Notebook-Style).

I could only find the old name at one location which I already changed.
Moved the data to csv file which I have placed in pymc3/examples/data folder.
Ran the formatting.
Please let me know if something is still missing.
Thanks,
Amit

AlexAndorra

This is all good now, thanks a lot @AmitKus !

AmitKus changed the title ~~Update Pymc3_tips_and_heuristic notebook~~ Update Pymc3_tips_and_heuristics notebook Jul 30, 2020

AlexAndorra approved these changes Aug 1, 2020

View reviewed changes

AmitKus force-pushed the pymc3-tip-notebook branch from 9237d75 to d01cc98 Compare August 10, 2020 00:56

junpenglao reviewed Aug 10, 2020

View reviewed changes

docs/source/notebooks/table_of_contents_tutorials.js Outdated Show resolved Hide resolved

AmitKus force-pushed the pymc3-tip-notebook branch from 07032c3 to 72e7464 Compare August 10, 2020 17:05

AlexAndorra requested changes Aug 12, 2020

View reviewed changes

AmitKus force-pushed the pymc3-tip-notebook branch from 23322e9 to 0ce336a Compare August 12, 2020 17:41

AmitKus force-pushed the pymc3-tip-notebook branch from 0ce336a to 86a684d Compare August 14, 2020 23:40

AmitKus added 2 commits August 14, 2020 18:48

Move the data to csv file (scotland_lips_cancer.csv) instead being ha…

23c7ffb

…rd-coded in conditional-autoregressive-model.ipynb

Read data from csv file instead of the hardcoded version in notebook.

7188e36

twiecki assigned AlexAndorra Aug 17, 2020

AlexAndorra approved these changes Aug 17, 2020

View reviewed changes

AlexAndorra merged commit 049187f into pymc-devs:master Aug 17, 2020

AmitKus deleted the pymc3-tip-notebook branch August 17, 2020 13:16

Uh oh!

Update Pymc3_tips_and_heuristics notebook #4036

Update Pymc3_tips_and_heuristics notebook #4036

Uh oh!

Conversation

AmitKus commented Jul 30, 2020

Uh oh!

review-notebook-app bot commented Jul 30, 2020

Uh oh!

review-notebook-app bot commented Jul 30, 2020

Uh oh!

review-notebook-app bot commented Jul 30, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

review-notebook-app bot commented Jul 30, 2020

Uh oh!

review-notebook-app bot commented Jul 30, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

review-notebook-app bot commented Jul 30, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fonnesbeck commented Jul 30, 2020

Uh oh!

AmitKus commented Jul 31, 2020

Uh oh!

AmitKus commented Aug 1, 2020

Uh oh!

AmitKus commented Aug 1, 2020

Uh oh!

AmitKus commented Aug 1, 2020

Uh oh!

AmitKus commented Aug 1, 2020

Uh oh!

AmitKus commented Aug 1, 2020

Uh oh!

AlexAndorra left a comment

Choose a reason for hiding this comment

Uh oh!

junpenglao commented Aug 1, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Aug 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

review-notebook-app bot commented Aug 10, 2020

Uh oh!

AlexAndorra left a comment

Choose a reason for hiding this comment

Uh oh!

review-notebook-app bot commented Aug 12, 2020

Uh oh!

AmitKus commented Aug 15, 2020

Uh oh!

AlexAndorra left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

review-notebook-app bot commented Jul 30, 2020 •

edited

Loading

review-notebook-app bot commented Jul 30, 2020 •

edited

Loading

review-notebook-app bot commented Jul 30, 2020 •

edited

Loading

junpenglao commented Aug 1, 2020 •

edited

Loading

codecov bot commented Aug 10, 2020 •

edited

Loading