Skip to content

Commit 62e74a7

Browse files
shoyershoyer
committed
Merge pull request #702 from shoyer/multiindex
Basic multiIndex support and stack/unstack methods
2 parents 83bef5e + e034053 commit 62e74a7

17 files changed

+784
-138
lines changed

‎doc/api.rst

Lines changed: 31 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ API reference
55
#############
66

77
This page provides an auto-generated summary of xray's API. For more details
8-
and examples, refer to the relevant chapter in the main part of the
8+
and examples, refer to the relevant chapters in the main part of the
99
documentation.
1010

1111
Top-level functions
@@ -110,10 +110,7 @@ Computation
110110
Dataset.reduce
111111
Dataset.groupby
112112
Dataset.resample
113-
Dataset.transpose
114113
Dataset.diff
115-
Dataset.shift
116-
Dataset.roll
117114

118115
**Aggregation**:
119116
:py:attr:`~Dataset.all`
@@ -155,6 +152,18 @@ Computation
155152
:py:attr:`~core.groupby.DatasetGroupBy.fillna`
156153
:py:attr:`~core.groupby.DatasetGroupBy.where`
157154

155+
Reshaping and reorganizing
156+
--------------------------
157+
158+
.. autosummary::
159+
:toctree: generated/
160+
161+
Dataset.transpose
162+
Dataset.stack
163+
Dataset.unstack
164+
Dataset.shift
165+
Dataset.roll
166+
158167
DataArray
159168
=========
160169

@@ -218,6 +227,16 @@ Indexing
218227
DataArray.reindex
219228
DataArray.reindex_like
220229

230+
Comparisons
231+
-----------
232+
233+
.. autosummary::
234+
:toctree: generated/
235+
236+
DataArray.equals
237+
DataArray.identical
238+
DataArray.broadcast_equals
239+
221240
Computation
222241
-----------
223242

@@ -227,11 +246,8 @@ Computation
227246
DataArray.reduce
228247
DataArray.groupby
229248
DataArray.resample
230-
DataArray.transpose
231249
DataArray.get_axis_num
232250
DataArray.diff
233-
DataArray.shift
234-
DataArray.roll
235251

236252
**Aggregation**:
237253
:py:attr:`~DataArray.all`
@@ -273,16 +289,18 @@ Computation
273289
:py:attr:`~core.groupby.DataArrayGroupBy.fillna`
274290
:py:attr:`~core.groupby.DataArrayGroupBy.where`
275291

276-
Comparisons
277-
-----------
292+
293+
Reshaping and reorganizing
294+
--------------------------
278295

279296
.. autosummary::
280297
:toctree: generated/
281298

282-
DataArray.equals
283-
DataArray.identical
284-
DataArray.broadcast_equals
285-
299+
DataArray.transpose
300+
DataArray.stack
301+
DataArray.unstack
302+
DataArray.shift
303+
DataArray.roll
286304

287305
.. _api.ufuncs:
288306

‎doc/computation.rst

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -140,6 +140,13 @@ This means, for example, that you always subtract an array from its transpose:
140140
141141
c - c.T
142142
143+
You can explicitly broadcast xray data structures by using the
144+
:py:func:`~xray.broadcast` function:
145+
146+
a2, b2 = xray.broadcast(a, b2)
147+
a2
148+
b2
149+
143150
.. _math automatic alignment:
144151

145152
Automatic alignment
@@ -223,13 +230,6 @@ Datasets support most of the same methods found on data arrays:
223230
ds.mean(dim='x')
224231
abs(ds)
225232
226-
:py:meth:`~xray.Dataset.transpose` can also be used to reorder dimensions on
227-
all variables:
228-
229-
.. ipython:: python
230-
231-
ds.transpose('y', 'x')
232-
233233
Unfortunately, a limitation of the current version of numpy means that we
234234
cannot override ufuncs for datasets, because datasets cannot be written as
235235
a single array [1]_. :py:meth:`~xray.Dataset.apply` works around this
@@ -256,5 +256,5 @@ Arithmetic between two datasets matches data variables of the same name:
256256
Similarly to index based alignment, the result has the intersection of all
257257
matching variables, and ``ValueError`` is raised if the result would be empty.
258258

259-
.. [1] When numpy 1.10 is released, we should be able to override ufuncs for
259+
.. [1] When numpy 1.12 is released, we should be able to override ufuncs for
260260
datasets by making use of ``__numpy_ufunc__``.

‎doc/data-structures.rst

Lines changed: 2 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -436,8 +436,8 @@ dataset variables:
436436
437437
ds.rename({'temperature': 'temp', 'precipitation': 'precip'})
438438
439-
Finally, you can use :py:meth:`~xray.Dataset.swap_dims` to swap dimension and
440-
non-dimension variables:
439+
The related :py:meth:`~xray.Dataset.swap_dims` method allows you do to swap
440+
dimension and non-dimension variables:
441441

442442
.. ipython:: python
443443
@@ -535,48 +535,6 @@ dimension and whose the values are ``Index`` objects:
535535
536536
ds.indexes
537537
538-
Converting datasets and arrays
539-
------------------------------
540-
541-
To convert from a Dataset to a DataArray, use :py:meth:`~xray.Dataset.to_array`:
542-
543-
.. ipython:: python
544-
545-
arr = ds.to_array()
546-
arr
547-
548-
This method broadcasts all data variables in the dataset against each other,
549-
then concatenates them along a new dimension into a new array while preserving
550-
coordinates.
551-
552-
To convert back from a DataArray to a Dataset, use
553-
:py:meth:`~xray.DataArray.to_dataset`:
554-
555-
.. ipython:: python
556-
557-
arr.to_dataset(dim='variable')
558-
559-
The broadcasting behavior of ``to_array`` means that the resulting array
560-
includes the union of data variable dimensions:
561-
562-
.. ipython:: python
563-
564-
ds2 = xray.Dataset({'a': 0, 'b': ('x', [3, 4, 5])})
565-
566-
# the input dataset has 4 elements
567-
ds2
568-
569-
# the resulting array has 6 elements
570-
ds2.to_array()
571-
572-
Otherwise, the result could not be represented as an orthogonal array.
573-
574-
If you use ``to_dataset`` without supplying the ``dim`` argument, the DataArray will be converted into a Dataset of one variable:
575-
576-
.. ipython:: python
577-
578-
arr.to_dataset(name='combined')
579-
580538
581539
.. [1] Latitude and longitude are 2D arrays because the dataset uses
582540
`projected coordinates`__. ``reference_time`` refers to the reference time

‎doc/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ Documentation
3636
indexing
3737
computation
3838
groupby
39+
reshaping
3940
combining
4041
time-series
4142
pandas

‎doc/reshaping.rst

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
.. _reshape:
2+
3+
###############################
4+
Reshaping and reorganizing data
5+
###############################
6+
7+
These methods allow you to reorganize
8+
9+
.. ipython:: python
10+
:suppress:
11+
12+
import numpy as np
13+
import pandas as pd
14+
import xray
15+
np.random.seed(123456)
16+
17+
Reordering dimensions
18+
---------------------
19+
20+
To reorder dimensions on a :py:class:`~xray.DataArray` or across all variables
21+
on a :py:class:`~xray.Dataset`, use :py:meth:`xray.DataArray.transpose` or the
22+
``.T`` property:
23+
24+
.. ipython:: python
25+
26+
ds = xray.Dataset({'foo': (('x', 'y', 'z'), [[[42]]]), 'bar': (('y', 'z'), [[24]])})
27+
ds.transpose('y', 'z', 'x')
28+
ds.T
29+
30+
Converting between datasets and arrays
31+
--------------------------------------
32+
33+
To convert from a Dataset to a DataArray, use :py:meth:`~xray.Dataset.to_array`:
34+
35+
.. ipython:: python
36+
37+
arr = ds.to_array()
38+
arr
39+
40+
This method broadcasts all data variables in the dataset against each other,
41+
then concatenates them along a new dimension into a new array while preserving
42+
coordinates.
43+
44+
To convert back from a DataArray to a Dataset, use
45+
:py:meth:`~xray.DataArray.to_dataset`:
46+
47+
.. ipython:: python
48+
49+
arr.to_dataset(dim='variable')
50+
51+
The broadcasting behavior of ``to_array`` means that the resulting array
52+
includes the union of data variable dimensions:
53+
54+
.. ipython:: python
55+
56+
ds2 = xray.Dataset({'a': 0, 'b': ('x', [3, 4, 5])})
57+
58+
# the input dataset has 4 elements
59+
ds2
60+
61+
# the resulting array has 6 elements
62+
ds2.to_array()
63+
64+
Otherwise, the result could not be represented as an orthogonal array.
65+
66+
If you use ``to_dataset`` without supplying the ``dim`` argument, the DataArray will be converted into a Dataset of one variable:
67+
68+
.. ipython:: python
69+
70+
arr.to_dataset(name='combined')
71+
72+
.. _reshape.stack:
73+
74+
Stack and unstack
75+
-----------------
76+
77+
As part of xray's nascent support for :py:class:`pandas.MultiIndex`, we have
78+
implemented :py:meth:`~xray.DataArray.stack` and
79+
:py:meth:`~xray.DataArray.unstack` method, for combining or splitting dimensions:
80+
81+
.. ipython:: python
82+
83+
array = xray.DataArray(np.random.randn(2, 3),
84+
coords=[('x', ['a', 'b']), ('y', [0, 1, 2])])
85+
stacked = array.stack(z=('x', 'y'))
86+
stacked
87+
stacked.unstack('z')
88+
89+
These methods are modeled on the :py:class:`pandas.DataFrame` methods of the
90+
same name, although they in xray they always create new dimensions rather than
91+
adding to the existing index or columns.
92+
93+
Like :py:meth:`DataFrame.unstack<pandas.DataFrame.unstack>`, xray's ``unstack`` always succeeds, even
94+
if the multi-index being unstacked does not contain all possible levels. Missing
95+
levels are filled in with ``NaN`` in the resulting object:
96+
97+
.. ipython:: python
98+
99+
stacked2 = stacked[::2]
100+
stacked2
101+
stacked2.unstack('z')
102+
103+
However, xray's ``stack`` has an important difference from pandas: unlike
104+
pandas, it does not automatically drop missing values. Compare:
105+
106+
.. ipython:: python
107+
108+
array = xray.DataArray([[np.nan, 1], [2, 3]], dims=['x', 'y'])
109+
array.stack(z=('x', 'y'))
110+
array.to_pandas().stack()
111+
112+
We departed from pandas's behavior here because predictable shapes for new
113+
array dimensions is necessary for :ref:`dask`.
114+
115+
Shift and roll
116+
--------------
117+
118+
To adjust coordinate labels, you can use the :py:meth:`~xray.Dataset.shift` and
119+
:py:meth:`~xray.Dataset.roll` methods:
120+
121+
.. ipython:: python
122+
123+
array = xray.DataArray([1, 2, 3, 4], dims='x')
124+
array.shift(x=2)
125+
array.roll(x=2)

0 commit comments

Comments
 (0)