Skip to content

Add append!/push!/pushfirst!/pop!/popfirst! for BlockVector #116

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
May 17, 2020

Conversation

tkf
Copy link
Member

@tkf tkf commented May 16, 2020

This PR adds append!, push!, pushfirst!, pop!, and popfirst! for BlockVector.

It also adds foldl (and reduce because why not) for BlockVector since it is essential for fast iteration (see #116 (comment) below).


It's not the main point of this PR but a few quick benchmarks for foldl and reduce alone (just to help justify their inclusion):

Before:

julia> @btime sum($(mortar([rand(100) for _ in 1:10])));
  13.494 μs (0 allocations: 0 bytes)

julia> @btime foldl(+, $(mortar([rand(100) for _ in 1:10])));
  13.637 μs (0 allocations: 0 bytes)

After:

julia> @btime sum($(mortar([rand(100) for _ in 1:10])));
  130.989 ns (0 allocations: 0 bytes)

julia> @btime foldl(+, $(mortar([rand(100) for _ in 1:10])));
  1.090 μs (3 allocations: 80 bytes)

Comment on lines +56 to +62
# Equivalent to `i = li; for x in src; ...; end` but (maybe) faster:
foldl(src, init = li) do i, x
Base.@_inline_meta
i += 1
@inbounds block[i] = x
return i
end
Copy link
Member Author

@tkf tkf May 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using foldl is faster when src is a block vector.

With this PR:

julia> @btime append!(x, $(mortar([rand(3) for _ in 1:100]))) setup=(x = mortar([[0.0]]));
  912.130 ns (5 allocations: 9.29 KiB)

With this PR but using for loop:

julia> @btime append!(x, $(mortar([rand(3) for _ in 1:100]))) setup=(x = mortar([[0.0]]));
  11.031 μs (7 allocations: 2.58 KiB)

_blocktype(::BlockArray{<:Any,<:Any,<:AbstractArray{T}}) where {T<:AbstractArray} = T
_blocktype(::PseudoBlockArray{<:Any,<:Any,T}) where {T<:AbstractArray} = T

function append_nocopy!(dest::BlockVector{<:Any,T}, src::BlockVector{<:Any,T}) where {T}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This adds a new block right? I would call this blockappend!, and add a blockpop!, etc.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed it and added a docstring. Please check if the behavior I described is OK.

It actually does not always add a single block. If src is a block vector, it can add multiple blocks.

(Also, I haven't added blockpop! etc. yet.)

@test dest[6] == 666
else
@test dest[6] == 6
end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add tests (here and below) that make clear the resulting block structure. That is (I think):

@test dest[Block(1)]  == [1,2,3]
@test dest[Block(2)]  == [4,5,6]

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a few tests for the block structure. Actually, @test dest[Block(2)] == [4,5,6] is not always true. Let me know if the block structure makes sense.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the block structure should be consistent, that is, append!(::AbstractBlockVector, ::AbstractVector) should either always add to the last block or add new blocks.

There's a question here what append!(::AbstractBlockVector, ::AbstractBlockVector) should do...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, to make things predictable, maybe you'd like something like

append!(dest, src): always mutate the last block (blocklength(dest) is constant)

blockappend!(dest, src): always add blocklength(src) blocks and avoid copy when make sense

?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that's reasonable.

@codecov
Copy link

codecov bot commented May 16, 2020

Codecov Report

Merging #116 into master will increase coverage by 0.67%.
The diff coverage is 91.66%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #116      +/-   ##
==========================================
+ Coverage   84.42%   85.09%   +0.67%     
==========================================
  Files          12       14       +2     
  Lines         822      906      +84     
==========================================
+ Hits          694      771      +77     
- Misses        128      135       +7     
Impacted Files Coverage Δ
src/BlockArrays.jl 100.00% <ø> (ø)
src/blockdeque.jl 91.13% <91.13%> (ø)
src/blockreduce.jl 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2424ea1...84d1822. Read the comment docs.

@tkf
Copy link
Member Author

tkf commented May 16, 2020

OK, I think I addressed the comments.

@dlfivefifty dlfivefifty merged commit 3fc46cd into JuliaArrays:master May 17, 2020
@tkf tkf deleted the append branch May 17, 2020 08:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants