ENH: Implement more str accessor methods for ArrowDtype

[mroeschke]

Most of these don't have corresponding pyarrow compute methods

[jbrockmendel]

it looks like this will maintain the chunking structure. is there a reason not to chain these together and end up with a single chunk?

[mroeschke]

I saw this related issue about ops not maintaining the underlying chunking structure and though best to try to keep it here: #42357

[jbrockmendel]

is it worth making a helper for this pattern so this can just be

def _str_index(...)
    predicate = lambda x: x.index(sub, start, end)
    return self._helper_whatever(predicate)

[jbrockmendel]

out of curiosity why is this one left out?

[mroeschke]

str.extract was just a lot trickier to implement. I can try revisiting it in a followup

[jorisvandenbossche]

In general, it's often best to avoid iterating over pyarrow arrays (wrapping each array element in a pyarrow Scalar, and then converting each individually to a python object with as_py() has quite some overhead). Typically, first converting the whole array to numpy and iterating over the numpy array can be more efficient:

import string, random

In [23]: arr = pa.array(["".join(random.choices(string.ascii_letters, k=5)) for _ in range(1_000_000)])

In [24]: %timeit [val.as_py() for val in arr]
517 ms ± 10.1 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [25]: %timeit [val for val in arr.to_numpy(zero_copy_only=False)]
80.7 ms ± 1.17 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

Wrapping in and converting the scalar has quite some overhead, while the conversion to numpy is better optimized and happens in one go at a lower level.

[mroeschke]

Going to merge this in, I can address any followups if needed.

[lumberbot-app]

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

git checkout 2.0.x
git pull

2. Cherry pick the first parent branch of the this PR on top of the older branch:

git cherry-pick -x -m1 fbcbdaf70da354801d99a1cebb889153a3eca481

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

git commit -am 'Backport PR #52614: ENH: Implement more str accessor methods for ArrowDtype'

4. Push to a named branch:

git push YOURFORK 2.0.x:auto-backport-of-pr-52614-on-2.0.x

5. Create a PR against branch 2.0.x, I would have named this PR:

"Backport PR #52614 on branch 2.0.x (ENH: Implement more str accessor methods for ArrowDtype)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the Still Needs Manual Backport label once the PR gets merged.

If these instructions are inaccurate, feel free to suggest an improvement.