Skip to content

[BUG] Add is_coerce argument to func array_to_datetime_object (GH26122) #26257

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
May 7, 2019

Conversation

makbigc
Copy link
Contributor

@makbigc makbigc commented May 1, 2019

@@ -757,7 +758,8 @@ cdef inline ignore_errors_out_of_bounds_fallback(ndarray[object] values):

@cython.wraparound(False)
@cython.boundscheck(False)
cdef array_to_datetime_object(ndarray[object] values, bint is_raise,
cdef array_to_datetime_object(ndarray[object] values,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

really? we already pass thru errors so why do u need yet another arg?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Instead of passing is_raise& is_coerce just pass down errors from array_to_datetime. Please update the docstring of array_to_datetime_object after that change.

Copy link
Member

@gfyoung gfyoung May 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What exactly is the objection to using is_raise and is_coerce?

We use them all over array_to_datetime. By passing through errors, you're essentially duplicating the logic array_to_datetime does to determine these flags, which you ultimately are going to use in array_to_datetime_object.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fair. Since this is essentially a private method too changing the signature shouldn't be that big of a deal

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, we already consistency pass around errors=, I think better here rather than having 2 args that are passed that could actually be in conflict.

@codecov
Copy link

codecov bot commented May 1, 2019

Codecov Report

Merging #26257 into master will decrease coverage by <.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #26257      +/-   ##
==========================================
- Coverage   91.97%   91.97%   -0.01%     
==========================================
  Files         175      175              
  Lines       52386    52386              
==========================================
- Hits        48184    48180       -4     
- Misses       4202     4206       +4
Flag Coverage Δ
#multiple 90.52% <ø> (ø) ⬆️
#single 40.72% <ø> (-0.15%) ⬇️
Impacted Files Coverage Δ
pandas/io/gbq.py 78.94% <0%> (-10.53%) ⬇️
pandas/core/frame.py 96.9% <0%> (-0.12%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 80be9b5...8264fb7. Read the comment docs.

@codecov
Copy link

codecov bot commented May 1, 2019

Codecov Report

Merging #26257 into master will increase coverage by <.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #26257      +/-   ##
==========================================
+ Coverage   91.97%   91.98%   +<.01%     
==========================================
  Files         175      175              
  Lines       52386    52374      -12     
==========================================
- Hits        48184    48175       -9     
+ Misses       4202     4199       -3
Flag Coverage Δ
#multiple 90.53% <ø> (ø) ⬆️
#single 40.71% <ø> (-0.16%) ⬇️
Impacted Files Coverage Δ
pandas/io/gbq.py 78.94% <0%> (-10.53%) ⬇️
pandas/io/excel/_util.py 87.32% <0%> (-0.18%) ⬇️
pandas/core/frame.py 96.9% <0%> (-0.12%) ⬇️
pandas/core/reshape/pivot.py 96.52% <0%> (-0.02%) ⬇️
pandas/core/indexes/range.py 97.96% <0%> (-0.01%) ⬇️
pandas/io/pytables.py 90.22% <0%> (-0.01%) ⬇️
pandas/core/strings.py 98.86% <0%> (ø) ⬆️
pandas/tseries/holiday.py 93.1% <0%> (ø) ⬆️
pandas/core/dtypes/inference.py 100% <0%> (ø) ⬆️
pandas/core/reshape/melt.py 97.47% <0%> (ø) ⬆️
... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 80be9b5...03be04d. Read the comment docs.

@@ -273,7 +273,7 @@ Datetimelike
- Bug in :class:`DataFrame` and :class:`Series` where timezone aware data with ``dtype='datetime64[ns]`` was not cast to naive (:issue:`25843`)
- Improved :class:`Timestamp` type checking in various datetime functions to prevent exceptions when using a subclassed ``datetime`` (:issue:`25851`)
- Bug in :class:`Series` and :class:`DataFrame` repr where ``np.datetime64('NaT')`` and ``np.timedelta64('NaT')`` with ``dtype=object`` would be represented as ``NaN`` (:issue:`25445`)
-
- Bug in :meth:`to_datetime` which does not replace the invalid argument with ``NaT`` when error is set to coerce (:issue:`26122`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:func:`to_datetime`

@gfyoung gfyoung added Bug Datetime Datetime data dtype labels May 2, 2019
@@ -757,7 +758,8 @@ cdef inline ignore_errors_out_of_bounds_fallback(ndarray[object] values):

@cython.wraparound(False)
@cython.boundscheck(False)
cdef array_to_datetime_object(ndarray[object] values, bint is_raise,
cdef array_to_datetime_object(ndarray[object] values,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, we already consistency pass around errors=, I think better here rather than having 2 args that are passed that could actually be in conflict.

@@ -762,6 +762,18 @@ def test_iso_8601_strings_with_different_offsets(self):
NaT], tz='UTC')
tm.assert_index_equal(result, expected)

# GH 26122
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you make a new test that is named appropriately

@jreback jreback added this to the 0.25.0 milestone May 7, 2019
@jreback jreback merged commit 5eead57 into pandas-dev:master May 7, 2019
@jreback
Copy link
Contributor

jreback commented May 7, 2019

thanks @makbigc very nice! keep em coming!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Datetime Datetime data dtype
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unexpected 'to_datetime' behaviour for datetime strings with different offset in 'arg', when errors='coerce'
4 participants