Map Kibera

Map Kibera recently had some exciting (and very hard!) work to do in the Mt Elgon district, collecting the locations of over a hundred schools. Those schools double as polling places, and were needed for NDI’s election monitoring work; Primoz will post details soon on the why and what of the week he and Mildred spent in the Mt Elgon hills.

What I want to write about now is a geeky post describing the process of producing OpenStreetMap extracts. NDI needs the data in a different form from OSM’s XML format, namely Shapefiles or CSV matching the schemas of their postgres database, for easy import.

This is something which has become almost routine for Map Kibera, as we’ve been producing extracts for download of our data, filtered by theme (health, security, education, watsan), and in a number of formats. It’s become so routine that I can see a clear way to automate and build non-technical interfaces around the process. An easy interface to get data out of OpenStreetMap, in the format and with the data that you want, shouldn’t require grappling with our tools and would benefit GIS people and others greatly. That’s part of the motivation behind the Humanitarian Data Model.

For now, a dive into the steps and underlying tools, by skipping through the process-mtelgon.sh script. This, and many other bits and pieces of Map Kibera code are up on mapkibera github.


# GET fresh Mt Elgon extract from OSM
wget http://www.openstreetmap.org/api/0.6/map?bbox=34.40356,0.74961,34.83117,0.95577 -O /home/mikel/mtelgon/mtelgon.osm


First, simply download a chunk of OSM xml from the API, around the Mt Elgon district.


/home/mikel/src/osmosis-0.34/bin/osmosis --read-xml file="/home/mikel/mtelgon/mtelgon.osm" --tf accept-nodes "education:type=*" --tf reject-ways --tf reject-relations --write-xml file="/home/mikel/mtelgon/mtelgon.polling.osm"


osmosis is a powerful command line tool for processing OpenStreetMap data in many ways. Here, osmosis is used to take the Mount Elgon data, and extract only nodes that have a tag with key “education:type=*”. That corresponds to all the polling places/schools Primoz and Mildred collected.


cd /home/mikel/mtelgon/shapefile; rm polling.*; osmexport ./shp-polling.oxr /home/mikel/mtelgon/mtelgon.polling.osm .; zip polling-shapefile.zip polling.*; rm polling.*


osmexport is a command line utility packaged with osmlib, a ruby library for handling OSM data. osmexport reads rule files which is a format specifying how to match osm tags into various output formats (Shapefile, CSV, KML). Rule files provide a quite simple way to describe this mapping, but can also incorporate any arbitrary ruby code, so more complicated processing is possible.

The example above is uses a rule file to output polling places shapefiles.

setup :Shp do
    point :polling do
        string :id, 20
        string :name, 100
        string :pollstat, 16
        string :type, 32
    end
end

nodes do
    if tags['education:type']
        :polling << {:id => id, :name => name, :pollstat => tags['polling_station'], :type => tags['education:type']}
    end
end

The first part “setup” describes the schema of the shapefile. The second part, “nodes”, iterates through every node in the given OSM file, and builds up an array in polling, which it output into the defined shapefile.

There are plenty of other examples in github.

Map Kibera’s focus is online, but net access and technical knowledge is Kibera is still a big challenge. This is why our plan has always been to print up large number of maps (or an atlas) and distribute to all the schools, organizations, clinics, churches, whatever in Kibera, so the work of Map Kibera has maximum exposure and impact.

We’ve also been holding community meetings that feature paper maps. This is very much like what’s called Public Participatory GIS. That decade-old-plus methodology intends to bring mapping to a wide audience, through discussion and map drawing, or sometimes using some lightweight digital technology. The idea is not to capture precise information, but more subjective experience and ideas around place. But now with web mapping, and particularly OpenStreetMap, the public is simply participating, and the line between precise and subjective information is blurring.

Still, paper and hand drawing is a powerful tool. Walking Papers is the prime example of that. As we have moved into the community meetings, we wanted to capture this hybrid … subjective, familiar experience, but in a toolset that leads to easy digitization, reuse and sharing of results. I would’ve previously critiqued PPQIS for the lack of re-use … the results often disappeared to a desktop or report. What we came up with here was much inspired by Local Ground (pdf), which used modifications of Walking Papers to bring paper drawing in Bay Area high schools online.

All the dirty details on our organizational process for this is in the wiki. Here I wanted to document the technical process, and highlight a couple areas for improvement.

take a look.

We start off by printing maps by generating a A1 sized pdf map, using an extract of OSM data on whatever theme. Those get printed down in Nairobi’s industrial area, small run of 5 prints, costs about $10 each (we could probably do better if we had time to bargain). We buy good quality tracing paper from the stationary store in >Westgate. We have sets of colored markers; each marker is associated with a kind of question during the session.

At the meeting we tape down tracing paper on top of the map print, and draw a line over the border of the map and the north arrow. Discussions are led by mappers and us coordinators, a writer is chosen, and the mapping commences. The quality and revelations of the meetings have been amazing, Besides the written map, everything is documented by video, photos, audio, and write ups. We’re still compiling everything, but this page on security shows a preview.

We roll up all the maps for transport home. Unroll, and each tracing paper is affixed to the wall, backed by the blank opposite side of one of the paper maps. The map is photographed with a Digital SLR on a tripod, with flash, at a distance that reduces parallax as much as possible. Once a suitable image is captured, that’s copied to the computer. The image is clipped to the drawn boundary (this is sometimes a little bit of an art), and uploaded to the server. Also placed on the server is a “bounds” file, which simply contains the west,south,east,north bounding box of the map.

On the server, this ruby script is run. It goes over all images in the directory, and creates a world file, then uses gdal to convert the image to a GeoTIFF. The script outputs configuration for TileCache and some javascript to configure the OpenLayers maps.

The result is pretty decent, and very close to the mark. It certainly could be improved, if we had a large scanner, but that’s not available. Alignment, lighting and parallax introduces distortion. Could automatically extract the writing, like Local Ground does with PIL, but for now simply allow for altering transparency of the entire image in the display. The background tiles should be customized to the theme. Some of the text lies outside the clipping area, which could be solved by simply recording registration marks of the four corners. Perhaps with Walking Papers properly integrated, this would be easy. Finally, we’ll be integrated the drawn maps into pages included narratives and media, with links that highlight OSM features in browser.

Who knew the pleasures of looking after gear?! Inspired by the awesome HOT Haiti Pelican Case, we went out town to snag a budget hard-top brief case, and some foam pillow filling, and went to work with a bread knife. The result is a solid carrying case for up to 14 GPS units, with space for cables, extra batteries, installer discs and documentation. We did up the same for the Flip cameras. In the range of $20-$30, and solid enough.

Gear management needs thought, which would’ve been obvious to anyone keeping track of assets in a school or company. It’s largely ad-hoc in Kibera, and we’re trying to institute systems for daily and longer terms check outs of GPS. Under what criteria and length of time do people get access to a GPS library? For the moment, we keep track of who has what on the wiki.

We’ve settled on Garmin eTrex Legend HCx as our GPS of choice. Color screen, AA battery slots, MicroSD slot, USB interface … and less than $200 each. Been purchasing on Amazon, along with the cheapest MicroSD cared I can find.

The GPS don’t come ready to use. It’s a 10 minute process to configure from factory presets, to settings appropriate for OpenStreetMap mapping. We’ve documented the GPS setup process

Everything within the group is labeled. Everything. The GPS and cameras of course, but also every rechargeable battery, clipboard, and pen. It is shocking how easy these go missing. Our label maker may be our most precious piece of equipment.

Batteries are another management challenge. We now have three chargers, and dozen of NiMH batteries. The Eveready charger seems to work faster than Eneloop. On days before mapping, we’re pretty much loading these up all day long. We also need to make sure all the GPS are loaded with new maps, takes a bit of time!

Hopefully just a few tips that will help others in their gear prep and management.