For this year HxGN14 conference I have prepared a web app of modern data vizualisation, I have got inspired by great ideas from Victor Bret and his research and talks for general concept (high interactivity, visualization ) of this app.
It is exciting to see what is possible to do today inside browser and interactivity provided by various open source projects (e.g. leaflet,d3 and its plugins) and WebGL technology .
WebGL is funny – programming in very low level style in JavaScript. This sample plots 86T points using this technology. .
The code is very straightforward, the only thing is to how points are initially loaded and scaled (instead of reloading each time when map moves).
All points are initially transformed to tile size of 256 x 256 pixels at zoom level 0 and then re-scaled/re-shifted based on the current position of the map. drawingOnCanvas is called from L.CanvasOverlay each time map needs to be drawn (move, zoom)
function drawingOnCanvas(canvasOverlay, params) {
gl.clear(gl.COLOR_BUFFER_BIT);
// -- set base matrix to translate canvas pixel coordinates -> webgl coordinates
mapMatrix.set(pixelsToWebGLMatrix);
var bounds = leafletMap.getBounds();
var topLeft = new L.LatLng(bounds.getNorth(), bounds.getWest());
var offset = LatLongToPixelXY(topLeft.lat, topLeft.lng);
// -- Scale to current zoom
var scale = Math.pow(2, leafletMap.getZoom());
scaleMatrix(mapMatrix, scale, scale);
translateMatrix(mapMatrix, -offset.x, -offset.y);
// -- attach matrix value to 'mapMatrix' uniform in shader
gl.uniformMatrix4fv(u_matLoc, false, mapMatrix);
gl.drawArrays(gl.POINTS, 0, numPoints);
}
More information and insipiration I took from this site
To illustrate how variables are passed from JavaScript to shaders used in above example, here are two figures from the book- figure 5.7 on p. 149, and figure 5.3 on p.144.
Stride and Offset
This figure shows single buffer (interleaved)that is used fro both coordinates and size. In similar way single buffer is constructed in the example here:
var vertBuffer = gl.createBuffer();
var vertArray = new Float32Array(verts);
var fsize = vertArray.BYTES_PER_ELEMENT;
gl.bindBuffer(gl.ARRAY_BUFFER, vertBuffer);
gl.bufferData(gl.ARRAY_BUFFER, vertArray, gl.STATIC_DRAW);
gl.vertexAttribPointer(vertLoc, 2, gl.FLOAT, false,fsize*5,0);
gl.enableVertexAttribArray(vertLoc);
// -- offset for color buffer
gl.vertexAttribPointer(colorLoc, 3, gl.FLOAT, false, fsize*5, fsize*2);
gl.enableVertexAttribArray(colorLoc);
D3 can plot map in various projections, however do not expect to get same set of (overlapping) functionality as Leaflet or OpenLayers. D3 can extend these mapping frameworks. For OL3 simple example is here for Leaflet, my favourite is hexbins or check my own experiment here.
While D3 is kind of ‘base’ charting library (lot of utility functions, helpers) , there is upper , high level library too that can provide lot of ‘boilerplate’ for common charts. Here comes great news:
For inspiration I have used Leaflet.heat and extracted generic Canvas drawing class that is not tight to data or processing but rather call user defined function. (I am still thinking in iOS view delegates and it make sense to apply it here too).
You can use L.CanvasOverlay.js for you custom drawing in your Leaflet map. The sample is using 24T points available here: http://www.sumbera.com/gist/data.js
//Example:
L.canvasOverlay()
.params({data: points}) // optional add any custom data that will be passed to draw function
.drawing(drawingOnCanvas) // set drawing function
.addTo(leafletMap); // add this layer to leaflet map
//Custom drawing function:
function drawingOnCanvas(canvasOverlay, params) {
var ctx = params.canvas.getContext('2d');
params.options.data.map(function (d, i) {
// canvas drawing goes here
});
};
// parameters passed to custom draw function :
{
canvas : <canvas>,
bounds : <boundsinWGS84>
size : <viewsize>,
zoomScale: <zoomscaleis1/resolution>,
zoom : <currentzoom>,
options : <optionspassed >
};
Other useful full view Leaflet Canvas sources here:
Update August 2015: check “scaled”- based fast SVG rendering on top of Leaflet here
Testing SVG limits of plotting points on map using D3, Leaflet following this base sample:http://bost.ocks.org/mike/leaflet. However instead of scaling SVG in deep zooms, I am using Enter/Update/Exit pattern from D3 to dynamically update points on map. This has been prototyped also here http://bl.ocks.org/sumbera/9972460 with brushing of 100T points.
For zooming out (causing all points to be displayed), I am filtering out points that can’t be effectively visible, thus reducing number of points in SVG. (check console for log output).
This sample is using real data of 24T coordinates where points are clustered around cities, rather than artifically randomized. Real number of rendered points / removed points can be seen in console
Made quick test of these 2 +1 HTML5 renderers on iOS running inside iOS app in the WebView, that is without Nitro acceleration. All run on iPad Air
Leaflet 0.7 : great , works fine, everywhere, doesn’t load while dragging map (on mobile only) , runs on Microsoft Surface too.
OpenLayers 3 beta 1 : runs fine too, loads map during dragging, seems like smaller framerate, can over zoom OSM, doesn’t run in Microsoft Surface well.
Seznam Mapy Api v 4 – proprietary renderer from Seznam , bad rendering on iOS, missing tiles, nice map sources
Videos and original web pages used: – screencasted by AirPlay – that is directly from iPad Air:
This is like fox and stork story, deep or shallow, Native or Html. ( btw. the picture here is from Czech painter “Josef Lada”). As I am more ‘stork’ than fox, I have collected few resources for other ‘storks’ out there.
My understanding is that mobile platforms are ‘extension of the human senses’ and this trends continues with wearable computing devices, smaller, tighter in resources. Doesn’t matter what software you write, whether enterprise or consumer it is extension of us, our finger touch, eye view, near real time.
“Well, SVG doesn’t scale well to large numbers of objects, but Canvas doesn’t scale well to large screens”
“SVG performance degrades quickly (exponentially on Safari?) in the number of objects, but Canvas performance remains at a near-constant low. This makes sense, since Canvas is just a bitmap buffer, while SVG has to maintain additional references to each object that it renders. Also, though not pictured, note that performance in clearing an SVG element also decreases in the number of drawn objects.”