Skip to content

Benchmark comparison daru vs pandas & numpy

Jump to bottom Edit New page
Shekhar Prasad Rajak edited this page Jan 23, 2019 · 29 revisions

DataFrame size = (10 ** n , 2). Means 2 columns and 10 ** n rows , where n is 2,3,4,5, 6, 7, 8

Pandas and Numpy - mean of the one column

Benchmarking function: pandas_mean

Testing with a dataframe of size:  100
Result (seconds):  0.00002590420289998292
Testing with a dataframe of size:  1000
Result (seconds):  0.00002713621290004085
Testing with a dataframe of size:  10000
Result (seconds):  0.00004180985899984080
Testing with a dataframe of size:  100000
Result (seconds):  0.00015845163699941623
Testing with a dataframe of size:  1000000
Result (seconds):  0.00154265971000313576
Testing with a dataframe of size:  10000000
Result (seconds):  0.01233993340001688852
Testing with a dataframe of size:  100000000
Result (seconds):  0.12452101310000215917

Benchmarking function: numpy_mean

Testing with a dataframe of size:  100
Result (seconds):  0.00003039180350006063
Testing with a dataframe of size:  1000
Result (seconds):  0.00003180017919994498
Testing with a dataframe of size:  10000
Result (seconds):  0.00004642649300058111
Testing with a dataframe of size:  100000
Result (seconds):  0.00015341703800004326
Testing with a dataframe of size:  1000000
Result (seconds):  0.00132965861999764464
Testing with a dataframe of size:  10000000
Result (seconds):  0.01232307760001276689
Testing with a dataframe of size:  100000000

Pandas and Numpy - median of the one column

Benchmarking function: pandas_median

Testing with a dataframe of size:  100
Result (seconds):  0.00002582970006000323
Testing with a dataframe of size:  1000
Result (seconds):  0.00002824090917999456
Testing with a dataframe of size:  10000
Result (seconds):  0.00010940286248000120
Testing with a dataframe of size:  100000
Result (seconds):  0.00088729414723999977
Testing with a dataframe of size:  1000000
Result (seconds):  0.01075348846100041521
Testing with a dataframe of size:  10000000
Result (seconds):  0.12858627516000523117
Testing with a dataframe of size:  100000000
Result (seconds):  1.24747154800024873111

Benchmarking function: numpy_median

Testing with a dataframe of size:  100
Result (seconds):  0.00003089331586999833
Testing with a dataframe of size:  1000
Result (seconds):  0.00003515163099999882
Testing with a dataframe of size:  10000
Result (seconds):  0.00012442707055000028
Testing with a dataframe of size:  100000
Result (seconds):  0.00099820312949000251
Testing with a dataframe of size:  1000000
Result (seconds):  0.01108264675299960866
Testing with a dataframe of size:  10000000
Result (seconds):  0.13133820641999591206
Testing with a dataframe of size:  100000000
Result (seconds):  1.21390682299988839077

Pandas and Numpy - Product of the one column elements

Benchmarking function: pandas_prod

Testing with a dataframe of size:  100
Result (seconds):  0.00004491761369999949
Testing with a dataframe of size:  1000
Result (seconds):  0.00004912122980003915
Testing with a dataframe of size:  10000
Result (seconds):  0.00008044129299923952
Testing with a dataframe of size:  100000
Result (seconds):  0.00036101352399964525
Testing with a dataframe of size:  1000000
Result (seconds):  0.00347382971000115498
Testing with a dataframe of size:  10000000
Result (seconds):  0.03563231970001652649
Testing with a dataframe of size:  100000000
Result (seconds):  0.37846179100051813293

Benchmarking function: numpy_prod

Testing with a dataframe of size:  100
Result (seconds):  0.00005671319829998538
Testing with a dataframe of size:  1000
Result (seconds):  0.00005507186369995907
Testing with a dataframe of size:  10000
Result (seconds):  0.00008253360099934071
Testing with a dataframe of size:  100000
Result (seconds):  0.00040807778600083112
Testing with a dataframe of size:  1000000
Result (seconds):  0.00371396845999697692
Testing with a dataframe of size:  10000000
Result (seconds):  0.03488254110006892145
Testing with a dataframe of size:  100000000
Result (seconds):  0.35792863000006036600

Pandas and Numpy - Sum of the one column elements

Benchmarking function: pandas_sum

Testing with a dataframe of size:  100
Result (seconds):  0.00005561769920004735
Testing with a dataframe of size:  1000
Result (seconds):  0.00005946561629998541
Testing with a dataframe of size:  10000
Result (seconds):  0.00009354037799948855
Testing with a dataframe of size:  100000
Result (seconds):  0.00040350849599963114
Testing with a dataframe of size:  1000000
Result (seconds):  0.00499632787000336975
Testing with a dataframe of size:  10000000
Result (seconds):  0.05969556259997262082
Testing with a dataframe of size:  100000000
Result (seconds):  0.71097542200004681945

Benchmarking function: numpy_sum

Testing with a dataframe of size:  100
Result (seconds):  0.00006173165039999731
Testing with a dataframe of size:  1000
Result (seconds):  0.00006634956489997421
Testing with a dataframe of size:  10000
Result (seconds):  0.00010499926699958451
Testing with a dataframe of size:  100000
Result (seconds):  0.00038439752700014651
Testing with a dataframe of size:  1000000
Result (seconds):  0.00517789042000004005
Testing with a dataframe of size:  10000000
Result (seconds):  0.07420776259996272883
Testing with a dataframe of size:  100000000
Result (seconds):  0.64270900600004099434

Pandas and Numpy - Unique elements in the one column

Benchmarking function: pandas_unique

Testing with a dataframe of size:  100
Result (seconds):  0.00003485789000005752
Testing with a dataframe of size:  1000
Result (seconds):  0.00004646338229995308
Testing with a dataframe of size:  10000
Result (seconds):  0.00013272955199954595
Testing with a dataframe of size:  100000
Result (seconds):  0.00180719400100042547
Testing with a dataframe of size:  1000000
Result (seconds):  0.02744935295999311950
Testing with a dataframe of size:  10000000
Result (seconds):  0.36727430050004838957
Testing with a dataframe of size:  100000000
Result (seconds):  7.76852580100057821255

Benchmarking function: numpy_unique

Testing with a dataframe of size:  100
Result (seconds):  0.00001490040839998983
Testing with a dataframe of size:  1000
Result (seconds):  0.00004483109719994900
Testing with a dataframe of size:  10000
Result (seconds):  0.00057470103799914794
Testing with a dataframe of size:  100000
Result (seconds):  0.00743517332999999760
Testing with a dataframe of size:  1000000
Result (seconds):  0.07924894640999809170
Testing with a dataframe of size:  10000000
Result (seconds):  0.90400296050001993642
Testing with a dataframe of size:  100000000
Result (seconds):  10.34286806900036026491

Pandas and Numpy - Sort elements of the one column

Benchmarking function: numpy_sort

Testing with a dataframe of size:  100
Result (seconds):  0.00000912697550002122
Testing with a dataframe of size:  1000
Result (seconds):  0.00003445678490006685
Testing with a dataframe of size:  10000
Result (seconds):  0.00050352683799974329
Testing with a dataframe of size:  100000
Result (seconds):  0.00610998137699971227
Testing with a dataframe of size:  1000000
Result (seconds):  0.07806133034000595217
Testing with a dataframe of size:  10000000
Result (seconds):  0.89660990390002548445
Testing with a dataframe of size:  100000000
Result (seconds):  10.22523978899971552892

Benchmarking function: pandas_sort

Testing with a dataframe of size:  100
Result (seconds):  0.00022274660969997059
Testing with a dataframe of size:  1000
Result (seconds):  0.00028801353039998505
Testing with a dataframe of size:  10000
Result (seconds):  0.00093503032399985385
Testing with a dataframe of size:  100000
Result (seconds):  0.00960024423100003417
Testing with a dataframe of size:  1000000
Result (seconds):  0.14475218630000200037
Testing with a dataframe of size:  10000000
Result (seconds):  2.70540260359994144679
Testing with a dataframe of size:  100000000
Result (seconds):  43.03127763799966487568
Add a custom footer
Add a custom sidebar
Clone this wiki locally