Repository navigation
Subpar scattergl performance with date axis #413
Description
It seems as though scattergl with a date axis chokes around 250K points — http://codepen.io/cpsievert/pen/WwMpJW
I found that a bit surprising since ~1M points works great with non-date axes
Reacted by mzechmeister
Metadata
Metadata
Assignees
Labels
No labels
Activity
etpinard commentedon Apr 11, 2016
Not surprising on my end.
scattergluses one of two gl-vis modules depending on the user data:marker.colorandmarker.sizearrays aren't allowed.scatteroptions.The split between the two gl-vis modules happens here. You'll notice non-linear axis types are considered fancy.
Moreover, converting a date to coordinate routine could be optimized. I suspect that we could potentially save off a few milliseconds in this step.
monfera commentedon Sep 30, 2016
I'll start on it early next week; an initial hunch is that dates are too expensive in JS, so for performance it's best to convert to some numeric representation (e.g.
date.valueOf()) and convert back to a real date object only at the final stage for axis tick determination (and possibly cache it). Will know more next week.monfera commentedon Oct 4, 2016
Yes there's a big speed difference and as mentioned by @etpinard it comes down to the fact that currently, the much slower
gl-scatter2d-fancyrenderer is being used if the axis is of typedate. Though conceptually,dateaxes are linear, inplotlythey are not of typelinear, which here means a linear numeric scale.There are several options:
gl-vis/gl-scatterglhandle dates, as, at the end of the day, it's also a linear scale except ticks are sampled and rendered differently; the plot itself remains the same. Making this plot work with dates probably needs work ingl-vis. Probably this is the fastest option, but it won't help if users bump into the same limitation for other reasons (e.g. log scale or arrays for marker sizes/colors).gl-scatter2d-fancyso that it's fast. Currently, it renders into a bona fide geometry mainly to be able to draw different point marker shapes. However it's possible to turn the (non-fancy)gl-vis/gl-scatterglinto something that can render point marker shapes. The drawback is that the fancy version handles other things as well: arrays for marker styles (doable with more WebGL attrib arrays) log scales (am I missing something else?). So if we do this it makes sense to cover that so the fancy version can be dropped. Benefit: one fewer renderers.regl, retaining the features of both and the speed of the non-fancy version.etpinard commentedon Oct 4, 2016
I'm leaning towards waiting for the
reglrewrite to fix this issue.@jackparmer thoughts?
monfera commentedon Oct 4, 2016
@etpinard regarding the speeding options, we talked about different markers rendered by shaders. Something like this would obsolete the fancy version. I made a
reglexample here: http://codepen.io/monfera/full/GjOBkJ/jackparmer commentedon Oct 4, 2016
I'm not excited about waiting 3-6 months to make datetimes work in WebGL. Pared down trace options for timeseries could make a lot of sense like
pointcloud. The use case is loading and looking at ridiculously huge time series data, then zooming into parts that look intersting/odd for investigation. Imagine time series data coming off sensors in cars or spacecraft on subsecond intervals... Huge amounts of timestamped data. There would have to be some high quality decimation work in JavaScript to make this use case a reality (a decimated view of the time series is rendered at 0% zoom, this gets recalculated on zoom, etc).etpinard commentedon Oct 4, 2016
is the winner.
monfera commentedon Oct 7, 2016
#1021 purports to fix it, I'm trying to think of a test case for this.
etpinard commentedon Nov 3, 2016
done in #1033