Category Archives: Leaflet

experiments where open source web mapping library Leaflet is involved

iKatastr.cz pro mobilní web

Pár poznámek k nové mobilní verzi iKatastr.cz.

To co je na této verzi zajimavé je, že jsem se nakonec rozhodl ignorovat všechna možná pokušení o využití JavaScriptovych knihoven, tedy kromě základní mapové komponenty – ta je založena na Leafletu v 1.0.3, kde jsem ještě musel udělat pár úprav, aby vše fungovalo správně. Leaflet má v komprimované formě 48KB. Využívám i komponentu Mapy API ale jen pro vyhledávání a navíc je tato část nahrána až když uživatel opravdu klikne na hledání.

Vlastní kód iKatastru má v komprimované formě pouze 13.3 KB. Celý výkonný kód, včetně HTML, CSS a fontu (který je také na míru vyroben) zabere při studeném startu (bez cache) pod 70 KB.  Zbytek je už vlastní obsah  – data  – hlavně dlaždice ze zdrojů Mapy.cz a ČÚZK.

Při každém dotazu do mapy aplikace vyšle požadavky na webové služby ČÚZK, ty zaberou v jedné odpovědi pod 10 KB.

 Uživatelské rozhraní 

Vertikální menu ve stylu rozbalovacích panelů nakonec zvítězilo a myslim si, že je hbité, pěkné a funkční. Opět nechtěl jsem dělat tendenční UI ani animace, které nakonec  berou čas uživateli. Na mobilu se dovoluje kvůli místu mít pouze jeden panel rozbalený, navíc funguje libovolné tapnutí do mapy k tomu aby se panel zabalil – to je navýkové a pohodlné. Ovšem panely se dají zavírat více způsoby – samotnou ikonou co je otevíra a samozřejmě křížkem vpravo.

Informace z katastru nemovitostí:

Tohle byl asi největší problém – tedy ne samotné informace – ty jsou úplně na jiné úrovni než před 8 lety, kdy iKatastr začínal, spíš problém jak do mapy tapnout – dlouze, nebo krátce ? Dlouhý stisk je zavedený v mobilnich aplikacích, krátký klik je zase pohodlný a rychlý na webu.  Nakonec má uživatel k dispozici oboje (a k mému údivu to šlo vyřešit) a navíc může si zvolit, jestli se má na jedno tapnutí přímo zobrazit nahlížení (tedy tak jak byl zvyklý z minulých verzí) . To otevírá poměrně hladkou cestu na přenesení této mobilní verze na desktop. Ona tam funguje, dokonce i na Internet Exploreru (jen si stěžuje že  potřebuje pomoci s povolením otevřít popup okno …opět) a na EDGE prohlížeči nefungují některé dotazy – je to chybou v EDGE prohlížeči, naštěstí se rozjedou vždy záložní dotazy, které fungují, takže z pohledu uživatele se pouze nezvýrazní parcely/budovy.

Budu rád pokud mi sem napíšete vaše postřehy, připomínky k této mobilní verzi – co funguje, co se lébi/nelébi a jak používate vlastně iKatastr a při jaké práci.

 

SVG fast scaled overlay on Leaflet 1.0 and 0.7

SVGScaled SVG can be drawn on map in much more faster way than traditional approaches, at least for points. Traditional approach re-position each element to fit into the view of the map, however SVG is “scalable” so we can use it and it performs much more faster for zoom-in/out.

Few considerations:

  1. SVG itself define viewport by its coordinate space, all outside of this viewport is usually  clipped, so it is important to keep SVG viewport in-line with the viewport of the map. There are approaches that resizes SVG as you zoom-in (here), and while it works, it has a problems in deep-zooms when you need to move on map (actually you move  giant SVG based on the zoom )
  2. translating LatLon to absolute pixel values (like here used for WebGL) is possible solution, however IE and FF has problems with large numbers for transoform (>1 M), So we need to get SVG elements in view coordinates and translate them.
  3. Having some track of bounding box of all elements like again used here should be avoided (SVG or its group element knows about extension of the elements it holds)
  4. So while we keep SVG in the viewport, we need to compensate any shift and zoom by translating <g> (group) of all elements.
  5. So in leaflet when map  moves, SVG is translated back to its original position while <g> is translated forward to reflect the map movement
  6. We need to keep track of LatLon position of either map center or one of the corner – we use topLeft corner.
  7. Leaflet doesn’t do  precise enlargement and rounds view points because of some CSS troubles on some devices (noted here). We need to patch two translating functions in Leaflet to get this right (so SVG enlargement will be aligned with map)… but I need to look on this again, best would be to not patch Leaflet of course.

most important things happen in moveEnd event:

 


 var bounds = this._map.getBounds(); // -- latLng bounds of map viewport
 var topLeftLatLng = new L.LatLng(bounds.getNorth(), bounds.getWest()); // -- topLeft corner of the viewport
 var topLeftLayerPoint = this._map.latLngToLayerPoint(topLeftLatLng); // -- translating to view coord
 var lastLeftLayerPoint = this._map.latLngToLayerPoint(this._lastTopLeftlatLng); 

 var zoom = this._map.getZoom();
 var scaleDelta = this._map.getZoomScale(zoom, this._lastZoom); // -- amount of scale from previous state e.g. 0.5 or 2
 var scaleDiff = this.getScaleDiff(zoom); // -- diff of how far we are from initial scale 

 this._lastZoom = zoom; // -- we need to keep track of last zoom
 var delta = lastLeftLayerPoint.subtract(topLeftLayerPoint); // -- get incremental delta in view coord

 this._lastTopLeftlatLng = topLeftLatLng; // -- we need to keep track of last top left corner, with this we do not need to track center of enlargement
 L.DomUtil.setPosition(this._svg, topLeftLayerPoint); // -- reset svg to keep it inside map viewport

 this._shift._multiplyBy(scaleDelta)._add(delta); // -- compute new relative shift from initial position
 // -- set group element to compensate for svg translation, and scale</pre>
 this._g.setAttribute("transform", "translate(" + this._shift.x + "," + this._shift.y + ") scale(" + scaleDiff + ")");

Test page / Gist : http://bl.ocks.org/Sumbera/7e8e57368175a1433791

To better illustrate movement of SVG inside the map, here is a small diagram of basic SVG states:svgpositioning

geojson-vt on leaflet

update Sept 2015: nice explanation of how geojson-vt works here

Mapbox technologies used in their webgl and opengl libraries are being extracted into standalone pieces. Vladimir Agafonkin,creator of leaflet.js, earcut.js provided slicing and polygon simplification library for geojson called Goejson-vt.Geojson-vt can slice geosjon into tiles aka mapbox tiles.
Quick test and sample of using geojson-vt on leaflet with canvas drawing available here: http://bl.ocks.org/sumbera/c67e5551b21c68dc8299

2 videos:

Overview of various geojson samples

 

folowing video shows 280 MB large geojson !

 

for comparison here is WebGL version on the same data. This version is loading all data into GPU and leaves everything on WebGL (no optimization). It also takes slightly more time to tessellate all polygons, but once done all seems to run fine. Code used is available here

WebGL polyline tessellation with pixi.js

update 09/2015  : another triangulation methods (mapbox, tesspathy) mentioned here

pixi.js is a 2D open source library for gaming that includes WebGL support for primitives rendering. Why not to utilize it for polyline renderings on map ? It turned out, however, that the  tesselation of the polylines is not handled well.

most important code snippets:

<script src="Pixi.js"></script>
<script src="Point.js"></script>
<script src="WebGLGraphics.js"></script>

//--data
<script src="route.js" charset="utf-8"></script>

var graphicsData = {
 points: verts,
 lineWidth: 0.00015,
 lineColor: 0x33FF00,
 lineAlpha: 0.8
};
var webGLData = {
   points: [],
   indices: []
 };
 // -- from pixi/utils
 PIXI.hex2rgb = function (hex) {
   return [(hex >> 16 & 0xFF) / 255,
           (hex >> 8 & 0xFF) / 255,
           (hex & 0xFF) / 255];
  };

PIXI.WebGLGraphics.buildLine(graphicsData, webGLData);

I have put sample here:

Another implementaiton of polyline tessellation (seems like more functional) is in mapbox-gl-js  in LineBucket  .Mapbox-gl-js code took quite more time to get it running and debug on Windows platform,I  had to run npm install  from VS command shell and read carefully what all the npm errors are saying (e.g. Python version should be < 3). Then FireFox for some reason haven’t triggered breakpoint on LineBucket.addLine, this took another time to find out that I should debug thiOstravaRailwayss rather in Chrome.   See the blog here.Anyway good  experience with all the messy npm modules, their install requirements and unnecessary complexity. Also all the npm modules takes more than 200 MB, but some of them are optional in the install.

After all basic LINE draw in WebGL (without the thicknes and styling) is useful too, as on picture above you can see railways in CZ city Ostrava.

 

WebGL polygons fill with libtess.js

kraje

Update 1.6.2015: geojson-vt seems to do great job in tiling and simplifying polygons. Check this post.

Update 18.1.2015: Vladimir Agafonkin from MapBox released earcut.js – very fast and reliable triangulation library. Worth to check. Video available here:

 

 

Original post:

Brendan Kenny from Google showed  here how he made polygons using libtess.js on Google Maps, so I have tried that too with single large enough polygon on Leaflet with CZ districts.  libtess.js is port from C code . Neither plntri.js (update: see also comments for plntri v2.0 details)  nor PolyK.js were able to triangulate large set  of points as libtess.js.

Update:  I looked on poly2tri.js  too with following results:

I could run 2256 polygons (all together > 3M vertexes)  with poly2tri  16 701 ms  vs 127 834 ms (libtess), however I  had to dirty fix  other various errors from poly2tri (null triangles or “FLIP failed due to missing triangle…so some polygons were wrong..), while  libtess was fine for the  same data.

Here is  a test :  3 M vertexes with 1 M triangles have been by generated by libtess in 127s . poly2tri took 16s.  Drawing is still fine but it is ‘just enough’ for WebGL too.

 

 

key part is listed below:


tessy.gluTessNormal(0, 0, 1);
tessy.gluTessBeginPolygon(verts);

tessy.gluTessBeginContour();

//--see blog comment below on using Array.map&lt;/span&gt;&lt;/strong&gt;
data.features[0].geometry.coordinates[0].map(function (d, i) {
pixel = LatLongToPixelXY(d[1], d[0],0);
var coords = [pixel.x, pixel.y, 0];
tessy.gluTessVertex(coords, coords);
});

tessy.gluTessEndContour();
// finish polygon (and time triangulation process)
tessy.gluTessEndPolygon();

code available here: http://bl.ocks.org/sumbera/01cbd44a77b4283e6dcd

 

There is also EMSCRIPTEN version of the tesslib.c available on github, and I was curious whether this version would increase speed of  computation. I could run it but for large polygons (cca 120 verts of CZ boundary) I had to increase module memory to 64 MB for FireFox.  Tessellata 120T verts in  FF-30 took 21s, IE-11, Ch-36: failed  reporting out of stack memory :(

Getting back to version from Brendan  (no emscripten) I quickly measured same data on browsers: IE-11 21s, Ch-36: 31s,  FF-30: 27s .

Update Oct/2014: Polyline tessellation blog here

Modern data visualization on map

hxgn14 For this year HxGN14  conference  I have prepared a web app  of modern data vizualisation, I have got  inspired by great ideas from Victor Bret and his research and talks for general concept (high interactivity, visualization ) of this app.

It is exciting to see what is possible to do today inside browser and interactivity provided by various open source projects (e.g. leaflet,d3  and its plugins)  and WebGL technology .

Leaflet WebGL many points rendering

WebGL is funny – programming in very low level style in JavaScript. This sample plots 86T points using this technology.  .

 

 

 

The code is very straightforward, the only thing is to how points are initially loaded and scaled (instead of reloading each time when map moves).

All points are initially transformed to tile size of 256 x 256 pixels at zoom level 0  and then re-scaled/re-shifted based on the current position of the map. drawingOnCanvas is called from L.CanvasOverlay each time map needs to be drawn (move, zoom)

 


function drawingOnCanvas(canvasOverlay, params) {
  gl.clear(gl.COLOR_BUFFER_BIT);
  // -- set base matrix to translate canvas pixel coordinates -> webgl coordinates
 mapMatrix.set(pixelsToWebGLMatrix);
  var bounds = leafletMap.getBounds();
  var topLeft = new L.LatLng(bounds.getNorth(), bounds.getWest());
  var offset = LatLongToPixelXY(topLeft.lat, topLeft.lng);
  // -- Scale to current zoom
  var scale = Math.pow(2, leafletMap.getZoom());
 scaleMatrix(mapMatrix, scale, scale);
 translateMatrix(mapMatrix, -offset.x, -offset.y);
  // -- attach matrix value to 'mapMatrix' uniform in shader
  gl.uniformMatrix4fv(u_matLoc, false, mapMatrix);
 gl.drawArrays(gl.POINTS, 0, numPoints);
}

More information and insipiration I took from this site

demo here: http://bl.ocks.org/sumbera/c6fed35c377a46ff74c3

For polygons rendering check here and for polyline rendering here

Some good intros to WebGL that might help you to understand the code: http://aerotwist.com/presentations/custom-filters/#/6

There is a nice intro book to WebGL  WebGL Programming Guide by Kouchi Matsuda and Rodger Lea

To illustrate how variables are passed from JavaScript to shaders used in above example, here are two figures from the book-  figure 5.7 on p. 149, and figure 5.3 on p.144.

strideoffset

Stride and Offset

This figure shows single buffer (interleaved)that is used fro both coordinates and size. In similar way single buffer is constructed in the example here:

 

 

 

var vertBuffer = gl.createBuffer();
var vertArray = new Float32Array(verts);
var fsize = vertArray.BYTES_PER_ELEMENT;

gl.bindBuffer(gl.ARRAY_BUFFER, vertBuffer);
gl.bufferData(gl.ARRAY_BUFFER, vertArray, gl.STATIC_DRAW);
gl.vertexAttribPointer(vertLoc, 2, gl.FLOAT, false,fsize*5,0);
gl.enableVertexAttribArray(vertLoc);
// -- offset for color buffer
gl.vertexAttribPointer(colorLoc, 3, gl.FLOAT, false, fsize*5, fsize*2);
gl.enableVertexAttribArray(colorLoc);

 

shadervariables2

behavior of a varying variable

Data Visualisation with d3.js Cookbook

datavisbookbought this great book on  d3.js (Data-Driven documents), written by  Nick Qi Zhu

D3 can plot map in various projections, however do not expect to get same set of (overlapping) functionality as Leaflet or OpenLayers. D3 can extend these mapping frameworks. For OL3 simple  example is here for Leaflet, my favourite is  hexbins or check my own experiment here.

While D3 is kind of ‘base’ charting library (lot of utility functions, helpers) , there is upper , high level library too  that can provide lot of ‘boilerplate’ for common charts. Here comes great news:

Nick Qi Zhu   is also author of the open source javascript  lib dc.js   (Dimensional Charting) library  that is using crossfilter (Fast Multidimensional Filtering for Coordinated Views) . Part of the dc.js lib is set of most common charts and one of them is choropleth map .

Other resources:

D3 Animation : http://blog.visual.ly/creating-animations-and-transitions-with-d3-js/

Design selections: http://www.awwwards.com/fresh-ui-inspiration-in-the-era-of-google-material-and-design-patterns.html

Leaflet Canvas Overlay

leafletCanvasUpdates:

July 2016: refactored and moved to github here

August 2015: check also “scaled”- based fast SVG rendering on top of Leaflet here it might be surprising in performance on Chrome.

May 2015: Geojson-vt sample here is using Canvas overlay as well

June 2014: WebGL sample drawing 86T points using this canvasOverlay available here.

Leaflet full view Canvas Overlay  is a straightforward full screen canvas overlay class (L.CanvasOverlay.js) that calls custom user function for drawing. Available here: http://bl.ocks.org/sumbera/11114288

For inspiration I have used Leaflet.heat and extracted generic Canvas drawing class that is not tight to data or processing but rather call user defined function. (I am still thinking in iOS view delegates and it make sense to apply it here too).

You can use L.CanvasOverlay.js for you custom drawing in your Leaflet map. The sample is using 24T points available here: http://www.sumbera.com/gist/data.js

Usage example of the demo here:

    //Example:
    L.canvasOverlay()
       .params({data: points})     // optional add any custom data that will be passed to draw function
           .drawing(drawingOnCanvas)   // set drawing function
           .addTo(leafletMap);         // add this layer to leaflet map


    //Custom drawing function:
        function drawingOnCanvas(canvasOverlay, params) {
                var ctx = params.canvas.getContext('2d');
                params.options.data.map(function (d, i) {
                  // canvas drawing goes here
                });
            };

    // parameters passed to custom draw function :
     {
                                canvas   : <canvas>,
                                bounds   : <bounds in WGS84>
                                size     : <view size>,
                                zoomScale: <zoom scale is  1/resolution>,
                                zoom     : <current zoom>,
                                options  : <options passed >
             };

Other useful full view Leaflet Canvas sources here:

– leaflet.heat https://github.com/Leaflet/Leaflet.heat
– Full Canvas https://github.com/cyrilcherian/Leaflet-Fullcanvas
– CartoDb Leaflet.Canvas : https://github.com/CartoDB/Leaflet.CanvasLayer

Many points with d3 and leaflet

Update August 2015: check “scaled”- based fast SVG rendering on top of Leaflet here

manypoints

Testing SVG limits of plotting points on map using D3, Leaflet following this base sample:http://bost.ocks.org/mike/leaflet. However instead of scaling SVG in deep zooms, I am using Enter/Update/Exit pattern from D3 to dynamically update points on map. This has been prototyped also here http://bl.ocks.org/sumbera/9972460 with brushing of 100T points.

For zooming out (causing all points to be displayed), I am filtering out points that can’t be effectively visible, thus reducing number of points in SVG. (check console for log output).

This sample is using real data of 24T coordinates where points are clustered around cities, rather than artifically randomized. Real number of rendered points / removed points can be seen in console

Test page : http://bl.ocks.org/sumbera/10463358