Faster ChainRules implementation

[MilesCranmer]

This modifies eval_grad_tree_array to have a :both mode that computes gradients w.r.t both constants and features at the same time.

Also declares various functions to be non differentiable, which weirdly makes the differentiation like 10x faster.

It looks like the problem is from runtime dispatch on Zygote.gradient(f, expression) even though the final return type and value is correct. I wonder if it has to do with it having the output containing a recursive type (the Node). Worth investing further or maybe raising an issue on Zygote.jl or ChainRulesCore.jl.

[github-actions]

Benchmark Results

	master	c9eaedf...	master/c9eaedf63e36a2...
eval/ComplexF32/evaluation	7.43 ± 0.52 ms	7.45 ± 0.49 ms	0.998
eval/ComplexF64/evaluation	9.69 ± 0.66 ms	9.71 ± 0.71 ms	0.998
eval/Float32/derivative	10.7 ± 1.5 ms	10.6 ± 1.4 ms	1.01
eval/Float32/derivative_turbo	10.6 ± 1.5 ms	10.6 ± 1.4 ms	0.997
eval/Float32/evaluation	2.71 ± 0.22 ms	2.69 ± 0.22 ms	1.01
eval/Float32/evaluation_bumper	0.524 ± 0.013 ms	0.551 ± 0.012 ms	0.951
eval/Float32/evaluation_turbo	0.682 ± 0.024 ms	0.687 ± 0.024 ms	0.993
eval/Float32/evaluation_turbo_bumper	0.533 ± 0.013 ms	0.551 ± 0.012 ms	0.968
eval/Float64/derivative	13.6 ± 0.48 ms	13.6 ± 0.51 ms	1
eval/Float64/derivative_turbo	13.6 ± 0.51 ms	13.6 ± 0.58 ms	1
eval/Float64/evaluation	2.89 ± 0.25 ms	2.88 ± 0.23 ms	1
eval/Float64/evaluation_bumper	1.19 ± 0.044 ms	1.2 ± 0.043 ms	0.996
eval/Float64/evaluation_turbo	1.17 ± 0.057 ms	1.17 ± 0.056 ms	0.998
eval/Float64/evaluation_turbo_bumper	1.19 ± 0.043 ms	1.19 ± 0.044 ms	0.996
utils/combine_operators/break_sharing	0.0415 ± 0.0018 ms	0.0412 ± 0.0012 ms	1.01
utils/convert/break_sharing	27.7 ± 0.72 μs	27.9 ± 0.8 μs	0.993
utils/convert/preserve_sharing	0.126 ± 0.0022 ms	0.127 ± 0.0029 ms	0.994
utils/copy/break_sharing	29 ± 0.8 μs	28.7 ± 0.81 μs	1.01
utils/copy/preserve_sharing	0.126 ± 0.0024 ms	0.126 ± 0.0024 ms	0.998
utils/count_constants/break_sharing	10.4 ± 0.14 μs	10.2 ± 0.16 μs	1.03
utils/count_constants/preserve_sharing	0.11 ± 0.002 ms	0.113 ± 0.002 ms	0.975
utils/count_depth/break_sharing	12.7 ± 0.22 μs	17.7 ± 0.35 μs	0.717
utils/count_nodes/break_sharing	10 ± 0.14 μs	10.1 ± 0.18 μs	0.997
utils/count_nodes/preserve_sharing	0.114 ± 0.002 ms	0.114 ± 0.0021 ms	0.998
utils/get_set_constants!/break_sharing	0.123 ± 0.0042 ms	0.121 ± 0.0041 ms	1.01
utils/get_set_constants!/preserve_sharing	0.313 ± 0.0066 ms	0.319 ± 0.0062 ms	0.984
utils/has_constants/break_sharing	4.3 ± 0.21 μs	4.53 ± 0.22 μs	0.949
utils/has_operators/break_sharing	1.76 ± 0.02 μs	1.93 ± 0.019 μs	0.914
utils/hash/break_sharing	0.0324 ± 0.00046 ms	0.0327 ± 0.00045 ms	0.989
utils/hash/preserve_sharing	0.134 ± 0.0021 ms	0.135 ± 0.0023 ms	0.993
utils/index_constants/break_sharing	27.1 ± 0.57 μs	27.3 ± 0.58 μs	0.991
utils/index_constants/preserve_sharing	0.126 ± 0.0022 ms	0.129 ± 0.0023 ms	0.974
utils/is_constant/break_sharing	4.81 ± 0.22 μs	4.33 ± 0.22 μs	1.11
utils/simplify_tree/break_sharing	0.154 ± 0.016 ms	0.158 ± 0.016 ms	0.979
utils/simplify_tree/preserve_sharing	0.275 ± 0.017 ms	0.277 ± 0.017 ms	0.991
utils/string_tree/break_sharing	0.377 ± 0.0084 ms	0.379 ± 0.0086 ms	0.996
utils/string_tree/preserve_sharing	0.511 ± 0.0099 ms	0.508 ± 0.01 ms	1.01
time_to_load	0.225 ± 0.0028 s	0.224 ± 0.0025 s	1.01

[MilesCranmer]

Blocked by aviatesk/JET.jl#642

[coveralls]

Pull Request Test Coverage Report for Build 9737805856

Details

• 44 of 44 (100.0%) changed or added relevant lines in 4 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.03%) to 95.514%

---

Totals
Change from base Build 9726818883:	0.03%
Covered Lines:	2193
Relevant Lines:	2296

---

💛 - Coveralls

[coveralls]

Pull Request Test Coverage Report for Build 9737908889

Details

• 44 of 44 (100.0%) changed or added relevant lines in 4 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.03%) to 95.514%

---

Totals
Change from base Build 9726818883:	0.03%
Covered Lines:	2193
Relevant Lines:	2296

---

💛 - Coveralls