REF: consolidate casting in groupby agg_series#41273

jreback merged 4 commits intopandas-dev:masterpandas-dev/pandas:masterfrom

jbrockmendel:ref-librd-maybe_convert_objectsjbrockmendel/pandas:ref-librd-maybe_convert_objectsCopy head branch name to clipboard