Skip to content
This repository was archived by the owner on Jan 9, 2023. It is now read-only.

root pandas is now automatically dropping columns... #17

Closed
wants to merge 4 commits into from
Closed

root pandas is now automatically dropping columns... #17

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
8adc1da
root pandas is now automatically dropping columns that have multidime…
KonstantinSchubert May 6, 2016
c031cdc
PEP8-ed with 100 as max line width
KonstantinSchubert May 6, 2016
34f5d0d
addressed most of the reviewers comments :)
KonstantinSchubert May 6, 2016
3bf916a
small fix
KonstantinSchubert May 6, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions root_pandas/readwrite.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
from math import ceil
import re
import ROOT
import warnings

from .utils import stretch

Expand Down Expand Up @@ -147,6 +148,17 @@ def do_flatten(arr):
arr = append_fields(arr_, '__array_index', idx, usemask=False, asrecarray=True)
return arr

def remove_nonscalar(arr):
first_row = arr[0]
good_cols = np.array([x.ndim == 0 for x in first_row])
col_names = np.array(arr.dtype.names)
good_names = col_names[good_cols]
bad_names = col_names[~good_cols]
if not bad_names.size == 0:
warnings.warn("Ignored the following non-scalar branches: {bad_names}"
.format(bad_names=", ".join(bad_names)), UserWarning)
return arr[good_names]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be a bit of a problem.
We definitely want to avoid making a copy here.
At least on my system, this creates a copy and emits a FutureWarning that this might change in the future.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the FutureWarning make any suggestions regarding alternatives?


if chunksize:
tchain = ROOT.TChain(key)
for path in paths:
Expand All @@ -159,13 +171,15 @@ def genchunks():
arr = root2array(paths, key, all_vars, start=chunk * chunksize, stop=(chunk+1) * chunksize, selection=where, *args, **kwargs)
if flatten:
arr = do_flatten(arr)
arr = remove_nonscalar(arr)
yield convert_to_dataframe(arr)

return genchunks()

arr = root2array(paths, key, all_vars, selection=where, *args, **kwargs)
if flatten:
arr = do_flatten(arr)
arr = remove_nonscalar(arr)
return convert_to_dataframe(arr)


Expand Down