Creating a timeseries chart from pandas parses Timestamp into nanoseconds rather than milliseconds

[abbas-mohd56]

The bug
When trying to create and render a Chart in jupyter notebook, Series of type pd.Timestamp is getting parsed into epoch timestamp in nanoseconds, which in turn messes up the axis.

To Reproduce
Run the code below

import pandas as pd
from highcharts_core.chart import Chart

dt_index = pd.date_range("2024-01-01", periods=6, freq="MS")
df = pd.DataFrame({"Values": [100, 200, 300, 35, 150, 500]}, index=dt_index)
df.index.name = "Timestamp"
chart = Chart.from_pandas(
    df,
    property_map={"x": "Timestamp", "y": "Values"},
    options_kwargs={"x_axis": {"type": "datetime"}},
)
chart.display()

The above code creates the following Dataframe

Timestamp	Values
2024-01-01	100
2024-02-01	200
2024-03-01	300
2024-04-01	35
2024-05-01	150
2024-06-01	500

And trying to create a chart using the above DataFrame gives the following chart

Upon inspecting chart.options.series[0], we can see the pd.Timestamp was parsed into epoch in nanoseconds:

LineSeries(data = {'dataPoints': [{'x': 1704067200000000000, 'y': 100}, {'x': 1706745600000000000, 'y': 200}, {'x': 1709251200000000000, 'y': 300}, {'x': 1711929600000000000, 'y': 35}, {'x': 1714521600000000000, 'y': 150}, {'x': 1717200000000000000, 'y': 500}]}, name = 'Values', type = 'line')

Expected behavior
The X axis in the above chart should be displayed as datetime.

Environment:

• Python Version: 3.10.15
• Highcharts core Python Version: 1.9.4
• pandas: 1.5.1

Additional questions:

• When I don't pass options_kwargs={"x_axis": {"type": "datetime"}} in the method, I noticed the Timestamp getting parsed into epoch but the x_axis type remains unchanged. Is this is the expected behavior or it's supposed to change automatically?
• Can I use index as X axis without setting a name to it and define it in property_map?

[hcpchris]

@abbas-mohd56 : Thanks for reporting the issue! The parsing the pd.Timestamp into nanoseconds definitely seems to be an issue. Let me look into that - I seem to recall some Timestamp parsing issues several months ago that had been resolved, but this may be related to those.

As for your additional questions:

• Yes, the pd.Timestamp will be parsed into epoch in all cases because Highcharts for Python determines that it is a timestamp/date-type of data point. There are situations where you want to visualize such data using a different axis type, so Highcharts for Python will not automatically determine x-axis type as being datetime for you, but it will convert the Timestamp's into epoch milliseconds which Highcharts (JS) expects (though it shouldn't convert into epoch nanoseconds).
• You can map the Timestamp column to the x data point value without assigning it as the DataFrame's index by using the property map. Furthermore, I seem to recall (though I may be misremembering this) that when setting that property in the property map you can either refer to that column by its name/label, or by its index position in the list of DataFrame columns.

[ThomasGl]

Check out the issue 200. I wrote a way to plot with datetimes corrected for pandas df. The df.index = .... line

[hcpchris]

Thanks, @ThomasGl : Will definitely prove useful.

[ThomasGl]

Hey guys I just opened a PR addressing this matter on the documentation, hope it suits and you guys find it useful :)
/