Joy of Open Source

*** update IV/2017:  from comment below, check this great raster plug-in  from Victor Veralde Gutiérrez too: https://github.com/IHCantabria/Leaflet.CanvasLayer.Field/  ***

I have got great, absolutely amazing  joy today from my own open source experiment. In 2014, nearly 2 years ago,  I have published on GitHub (gist)  L.CanvasOverlay a small class  to handle generic drawing on top of Leaflet, I was thinking let’s try to  contribute back to the open source with this little snippet –  I thought would be useful so I have added little description. Original blog post here: https://blog.sumbera.com/2014/04/20/leaflet-canvas/

After few months later I found it is used on http://windyty.com   one of the best 2015 geo-visualization with huge popularity. It is a small part of this great app, but makes me feel so good like I am part of it, I am looking on something where is my small piece, small share…meaning of the effort, sense of publishing and open sourcing.

Screen Shot 2016-02-02 at 23.16.13

And today I have got echo it is used also in Marine National Facility here: http://www.cmar.csiro.au/data/underway.test/

This makes me so happy … and just came across this quote : “Revenue is a lagging indicator, usage is a leading indicator,” can’t remember, who just said that ? :)

Advertisements

geojson-vt on leaflet

update Sept 2015: nice explanation of how geojson-vt works here

Mapbox technologies used in their webgl and opengl libraries are being extracted into standalone pieces. Vladimir Agafonkin,creator of leaflet.js, earcut.js provided slicing and polygon simplification library for geojson called Goejson-vt.Geojson-vt can slice geosjon into tiles aka mapbox tiles.
Quick test and sample of using geojson-vt on leaflet with canvas drawing available here: http://bl.ocks.org/sumbera/c67e5551b21c68dc8299

2 videos:

Overview of various geojson samples

 

folowing video shows 280 MB large geojson !

 

for comparison here is WebGL version on the same data. This version is loading all data into GPU and leaves everything on WebGL (no optimization). It also takes slightly more time to tessellate all polygons, but once done all seems to run fine. Code used is available here

Modern data visualization on map

hxgn14 For this year HxGN14  conference  I have prepared a web app  of modern data vizualisation, I have got  inspired by great ideas from Victor Bret and his research and talks for general concept (high interactivity, visualization ) of this app.

It is exciting to see what is possible to do today inside browser and interactivity provided by various open source projects (e.g. leaflet,d3  and its plugins)  and WebGL technology .

Leaflet WebGL many points rendering

WebGL is funny – programming in very low level style in JavaScript. This sample plots 86T points using this technology.  .

 

 

 

The code is very straightforward, the only thing is to how points are initially loaded and scaled (instead of reloading each time when map moves).

All points are initially transformed to tile size of 256 x 256 pixels at zoom level 0  and then re-scaled/re-shifted based on the current position of the map. drawingOnCanvas is called from L.CanvasOverlay each time map needs to be drawn (move, zoom)

 


function drawingOnCanvas(canvasOverlay, params) {
  gl.clear(gl.COLOR_BUFFER_BIT);
  // -- set base matrix to translate canvas pixel coordinates -> webgl coordinates
 mapMatrix.set(pixelsToWebGLMatrix);
  var bounds = leafletMap.getBounds();
  var topLeft = new L.LatLng(bounds.getNorth(), bounds.getWest());
  var offset = LatLongToPixelXY(topLeft.lat, topLeft.lng);
  // -- Scale to current zoom
  var scale = Math.pow(2, leafletMap.getZoom());
 scaleMatrix(mapMatrix, scale, scale);
 translateMatrix(mapMatrix, -offset.x, -offset.y);
  // -- attach matrix value to 'mapMatrix' uniform in shader
  gl.uniformMatrix4fv(u_matLoc, false, mapMatrix);
 gl.drawArrays(gl.POINTS, 0, numPoints);
}

More information and insipiration I took from this site

demo here: http://bl.ocks.org/sumbera/c6fed35c377a46ff74c3

For polygons rendering check here and for polyline rendering here

Some good intros to WebGL that might help you to understand the code: http://aerotwist.com/presentations/custom-filters/#/6

There is a nice intro book to WebGL  WebGL Programming Guide by Kouchi Matsuda and Rodger Lea

To illustrate how variables are passed from JavaScript to shaders used in above example, here are two figures from the book-  figure 5.7 on p. 149, and figure 5.3 on p.144.

strideoffset
Stride and Offset

This figure shows single buffer (interleaved)that is used fro both coordinates and size. In similar way single buffer is constructed in the example here:

 

 

 

var vertBuffer = gl.createBuffer();
var vertArray = new Float32Array(verts);
var fsize = vertArray.BYTES_PER_ELEMENT;

gl.bindBuffer(gl.ARRAY_BUFFER, vertBuffer);
gl.bufferData(gl.ARRAY_BUFFER, vertArray, gl.STATIC_DRAW);
gl.vertexAttribPointer(vertLoc, 2, gl.FLOAT, false,fsize*5,0);
gl.enableVertexAttribArray(vertLoc);
// -- offset for color buffer
gl.vertexAttribPointer(colorLoc, 3, gl.FLOAT, false, fsize*5, fsize*2);
gl.enableVertexAttribArray(colorLoc);

 

shadervariables2
behavior of a varying variable

Data Visualisation with d3.js Cookbook

datavisbookbought this great book on  d3.js (Data-Driven documents), written by  Nick Qi Zhu

D3 can plot map in various projections, however do not expect to get same set of (overlapping) functionality as Leaflet or OpenLayers. D3 can extend these mapping frameworks. For OL3 simple  example is here for Leaflet, my favourite is  hexbins or check my own experiment here.

While D3 is kind of ‘base’ charting library (lot of utility functions, helpers) , there is upper , high level library too  that can provide lot of ‘boilerplate’ for common charts. Here comes great news:

Nick Qi Zhu   is also author of the open source javascript  lib dc.js   (Dimensional Charting) library  that is using crossfilter (Fast Multidimensional Filtering for Coordinated Views) . Part of the dc.js lib is set of most common charts and one of them is choropleth map .

Other resources:

D3 Animation : http://blog.visual.ly/creating-animations-and-transitions-with-d3-js/

Design selections: http://www.awwwards.com/fresh-ui-inspiration-in-the-era-of-google-material-and-design-patterns.html

Leaflet Canvas Overlay

canvas

Updates:

July 2016: refactored and moved to github here

August 2015: check also “scaled”- based fast SVG rendering on top of Leaflet here it might be surprising in performance on Chrome.

May 2015: Geojson-vt sample here is using Canvas overlay as well

June 2014: WebGL sample drawing 86T points using this canvasOverlay available here.

Leaflet full view Canvas Overlay  is a straightforward full screen canvas overlay class (L.CanvasOverlay.js) that calls custom user function for drawing. Available here: http://bl.ocks.org/sumbera/11114288

For inspiration I have used Leaflet.heat and extracted generic Canvas drawing class that is not tight to data or processing but rather call user defined function. (I am still thinking in iOS view delegates and it make sense to apply it here too).

You can use L.CanvasOverlay.js for you custom drawing in your Leaflet map. The sample is using 24T points available here: http://www.sumbera.com/gist/data.js

Usage example of the demo here:

    //Example:
    L.canvasOverlay()
       .params({data: points})     // optional add any custom data that will be passed to draw function
           .drawing(drawingOnCanvas)   // set drawing function
           .addTo(leafletMap);         // add this layer to leaflet map


    //Custom drawing function:
        function drawingOnCanvas(canvasOverlay, params) {
                var ctx = params.canvas.getContext('2d');
                params.options.data.map(function (d, i) {
                  // canvas drawing goes here
                });
            };

    // parameters passed to custom draw function :
     {
                                canvas   : <canvas>,
                                bounds   : <bounds in WGS84>
                                size     : <view size>,
                                zoomScale: <zoom scale is  1/resolution>,
                                zoom     : <current zoom>,
                                options  : <options passed >
             };

Other useful full view Leaflet Canvas sources here:

– leaflet.heat https://github.com/Leaflet/Leaflet.heat
– Full Canvas https://github.com/cyrilcherian/Leaflet-Fullcanvas
– CartoDb Leaflet.Canvas : https://github.com/CartoDB/Leaflet.CanvasLayer

Many points with d3 and leaflet

Update August 2015: check “scaled”- based fast SVG rendering on top of Leaflet here

manypoints

Testing SVG limits of plotting points on map using D3, Leaflet following this base sample:http://bost.ocks.org/mike/leaflet. However instead of scaling SVG in deep zooms, I am using Enter/Update/Exit pattern from D3 to dynamically update points on map. This has been prototyped also here http://bl.ocks.org/sumbera/9972460 with brushing of 100T points.

For zooming out (causing all points to be displayed), I am filtering out points that can’t be effectively visible, thus reducing number of points in SVG. (check console for log output).

This sample is using real data of 24T coordinates where points are clustered around cities, rather than artifically randomized. Real number of rendered points / removed points can be seen in console

Test page : http://bl.ocks.org/sumbera/10463358