better scatter plot indexing?

[gordonwoodhull]

The shrinking of dots when the value goes to zero implemented for #491 points out a possible design flaw in the way the scatterplot is currently implemented.

Indexing by the position is intuitive, and helps slightly with rectangular brushing, but it doesn't allow points to disappear from the set except by the value going to zero. And they must maintain the same order because there is no key function specified for d3.data. Implicitly it gets keyed by the order of the points supplied.

Going forward toward 3.0, I want to consider whether the indexing was better for bubble plots (where you usually explicitly index by order and supply X and Y accessors), or whether a dataKey accessor should be added to the scatter plot interface.

In terms of efficiency, probably neither is very good for crossfilter. There isn't any key you could use that would allow a rectangular range to map to a range of records.

[martingraham]

Caveat: "it works for me", and only tested on a stand-alone chart, but I've used this key function in the dc source to keep the point mapping in the scatterplot constant between renderings/filterings. This way they transition rather than zip back and forward across the screen.

function keyButIgnoreValue (d) {
    var val = _chart.valueAccessor()(d);
    var dd = d.key.filter (function (part) { return part !== val; });
    return dd.join (":");
}

_chart.plotData = function () {
    var symbols = _chart.chartBodyG().selectAll('path.symbol')
        .data(_chart.data(), keyButIgnoreValue)
    ;

[danielhomola]

@martingraham , I tried this and although the transitions are now mapping in a constant way across renderings I don't get a smooth transition, but rather my scatter plot freezes, each point appears at its new location (so for a moment I see both the old and new renderings), then the old ones disappear. How did you overcome this? Cheers!

[gordonwoodhull]

@martingraham, if you have a unique key in your data, and it is available in the reduced data produced by the group you are using, you should be able to modify keyButIgnoreValue to fetch that. That should cause the dots to move around instead of the insert - then - delete behavior you're seeing.

If this works, we could add an optional feature for this.

[danielhomola]

I see, so if I modify keyButIgnoreValue to return a valid group object with key attribute, _chart.data can use that directly like
.data(keyButIgnoreValue)
instead of:
.data(_chart.data(), keyButIgnoreValue)

Does it have to be a crossfilter group object? Just asking to find the relevant docs for writing it.. Cheers!

[gordonwoodhull]

Actually, I just meant, have keyButIgnoreValue return the unique key, and leave the data as it is. I think it should work, haven't actually tested it.

If it sort of works but the transitions are wrong, we can fix that.

[gordonwoodhull]

The problem is that this does require the reduced data to have a unique key within its data, and those kinds of complex reductions are hard. (reductio helps.)

[danielhomola]

Alright, I see. I'm actually having trouble figuring out what exactly keyButIgnoreValue should return. In the example above by @martingraham it simply returns a list of strings, each being the key only, not the value. I made sure that these are absolutely unique, but the insert - then - delete behavior is still persistent. Also, I thought the .data() function (judging from the docs) should accept a single callback function, so I'm not even sure, why would it accept the keyButIgnoreValue function as a 2nd callback..

Could you please explain what's going on here? Thanks so much!

[gordonwoodhull]

It depends on your data. You simply want to return a string or integer that is unique for the semantic objects you want to see. Here's a good introduction to joining by key.

So, if you want a dot to move, it should have the same key from one redraw to the next. If you want it to disappear, that key should go away. If you want it to appear, the key should be new.

This information is outside of dc.js and crossfilter, so there's nothing that can be done automatically. Unfortunately, getting crossfilter to pass along this information from your data requires custom reduce magic, but perhaps if you are building a framework you already have that overhead covered.

[danielhomola]

Got it, this is fantastic. I had to write custom reduce functions to get the color-changing of the scatterplots to work anyway, so I can use one of that custom IDs I added to the value of the reduce function of the scatterplot's group.

I'll post a working example in an hour or so.

Thanks a lot for your help with this!