Repository navigation
Usage of a pandas.Index slows down figure generation of plotly #4250
Description
Hello everyone,
I have noticed that the generation of a plotly figure takes ~20 longer if using the index of a pandas DataFrame. I have generated a minimal working example:
import pandas as pd
import numpy as np
from plotly.subplots import make_subplots
import plotly.graph_objects as go
from datetime import datetime
import time
size = 10000
df = pd.DataFrame({"col1": np.random.random(size),
"col2": np.random.random(size),
"col3": np.random.random(size),
"col4": np.random.random(size)}, index=pd.date_range(datetime(2020, 1, 1), periods=size))
# Preloads some plotly routines, which would otherwise be in the timing.
fig = make_subplots()
def basic1(data):
start_time = time.perf_counter()
# fig = go.Figure()
fig = make_subplots()
for col in data:
fig.add_trace(go.Scattergl(x=data.index, y=data[col], name="ist_mpe"))
# fig.show("browser")
print(f"Elapsed time: {time.perf_counter() - start_time}")
# Using the pandas DataFrame index
basic1(df)
def basic2(data):
start_time = time.perf_counter()
# fig = go.Figure()
fig = make_subplots()
for col in data:
fig.add_trace(go.Scattergl(x=data.index.values, y=data[col], name="ist_mpe"))
# fig.show("browser")
print(f"Elapsed time: {time.perf_counter() - start_time}")
# Manually changing the index to a numpy array
basic2(df)
On my machine I get the following output:
Elapsed time: 0.11946500000000004
Elapsed time: 0.0073545000000000416
The second approach, in which I manually transform the pandas Index to a numpy array, is > 10 times faster. Internally plotly saves data as DataFrames, it seems weird that the use of the index has such an impact.
Note, when using a tz-aware DatetimeIndex, the above approach will firstly transform the dates to UTC+0.
Used Versions:
Package Version
--------------- -------
DateTime 5.1
numpy 1.24.3
packaging 23.1
pandas 2.0.2
pip 23.0.1
plotly 5.15.0
python-dateutil 2.8.2
pytz 2023.3
setuptools 67.6.1
six 1.16.0
tenacity 8.2.2
tzdata 2023.3
wheel 0.40.0
zope.interface 6.0
Reacted by HHest
Activity
MarcoGorelli commentedon Dec 1, 2024
I think there's just initial cost for the first plot you create, if you repeat each 3 times you see:
I think this can probably be closed then?
SergejKr commentedon Dec 1, 2024
Hi. In the script above I did a call to reduce the inital cost:
I just tried the script again. Once by just executing everything a second time. In that case I obtain the same behaviour as before
I also tried %timeit (commented out the print).
In all cases basic2 is faster by about a factor 10 than basic1. There must something else slowing my calculation down or some difference in our systems, e.g., the OS (I use Win 10).
MarcoGorelli commentedon Dec 1, 2024
thanks @SergejKr - I just tried again, and it looks like this is solved in the new release
Plotly 5.24.1:
Plotly 6.0.0rc0
I find that if I swap
basic2withbasic1, then the first plot to be produced always takes longer. So I think it makes more sense to compare the minimums of several runs of each - if we do that with the pre-release, then the issue looks solvedIf you fancied trying out the pre-release, you can install it with
pip install -U --pre plotly. If you find and report an issue so it can be fixed ahead of the final release, you're a star ⭐SergejKr commentedon Dec 1, 2024
Hello again @MarcoGorelli. I just tried the prerelease. It solves the perfromance issue for me.
Case closed.
MarcoGorelli commentedon Dec 1, 2024
yay thanks!
gonna ping @LiamConnors on this one then
LiamConnors commentedon Dec 2, 2024
Thanks @MarcoGorelli and @SergejKr! Will close this one out