Map Kibera

Map Kibera recently had some exciting (and very hard!) work to do in the Mt Elgon district, collecting the locations of over a hundred schools. Those schools double as polling places, and were needed for NDI’s election monitoring work; Primoz will post details soon on the why and what of the week he and Mildred spent in the Mt Elgon hills.

What I want to write about now is a geeky post describing the process of producing OpenStreetMap extracts. NDI needs the data in a different form from OSM’s XML format, namely Shapefiles or CSV matching the schemas of their postgres database, for easy import.

This is something which has become almost routine for Map Kibera, as we’ve been producing extracts for download of our data, filtered by theme (health, security, education, watsan), and in a number of formats. It’s become so routine that I can see a clear way to automate and build non-technical interfaces around the process. An easy interface to get data out of OpenStreetMap, in the format and with the data that you want, shouldn’t require grappling with our tools and would benefit GIS people and others greatly. That’s part of the motivation behind the Humanitarian Data Model.

For now, a dive into the steps and underlying tools, by skipping through the process-mtelgon.sh script. This, and many other bits and pieces of Map Kibera code are up on mapkibera github.


# GET fresh Mt Elgon extract from OSM
wget http://www.openstreetmap.org/api/0.6/map?bbox=34.40356,0.74961,34.83117,0.95577 -O /home/mikel/mtelgon/mtelgon.osm


First, simply download a chunk of OSM xml from the API, around the Mt Elgon district.


/home/mikel/src/osmosis-0.34/bin/osmosis --read-xml file="/home/mikel/mtelgon/mtelgon.osm" --tf accept-nodes "education:type=*" --tf reject-ways --tf reject-relations --write-xml file="/home/mikel/mtelgon/mtelgon.polling.osm"


osmosis is a powerful command line tool for processing OpenStreetMap data in many ways. Here, osmosis is used to take the Mount Elgon data, and extract only nodes that have a tag with key “education:type=*”. That corresponds to all the polling places/schools Primoz and Mildred collected.


cd /home/mikel/mtelgon/shapefile; rm polling.*; osmexport ./shp-polling.oxr /home/mikel/mtelgon/mtelgon.polling.osm .; zip polling-shapefile.zip polling.*; rm polling.*


osmexport is a command line utility packaged with osmlib, a ruby library for handling OSM data. osmexport reads rule files which is a format specifying how to match osm tags into various output formats (Shapefile, CSV, KML). Rule files provide a quite simple way to describe this mapping, but can also incorporate any arbitrary ruby code, so more complicated processing is possible.

The example above is uses a rule file to output polling places shapefiles.

setup :Shp do
    point :polling do
        string :id, 20
        string :name, 100
        string :pollstat, 16
        string :type, 32
    end
end

nodes do
    if tags['education:type']
        :polling << {:id => id, :name => name, :pollstat => tags['polling_station'], :type => tags['education:type']}
    end
end

The first part “setup” describes the schema of the shapefile. The second part, “nodes”, iterates through every node in the given OSM file, and builds up an array in polling, which it output into the defined shapefile.

There are plenty of other examples in github.

On Monday, I attended a Nike Foundation event to discuss innovative use of technology to assist young girls. This is something Map Kibera has focused on through our association with Unicef, and the Nike team were very interested to learn and listen, kudos. One point during the presentation touched on Pamoja Mtaani a video game developed by Time Warner that conveys messages about HIV/AIDS prevention. As chance would have it, I got to see Pamoja Mtaani in action directly in Mukuru.

I was in Mukuru to discuss introducing our mapping/communication techniques to their ongoing community work. Before our meeting started yesterday in Mukuru , I was given a tour of one of two centers built in Mukuru to “host the game”. My jaw dropped as we entered a spacious room, with 20 gleaming computers showing Pamoja Mtaani. The game looks pretty well designed and fun, definitely innovative and not overly didactic … the game starts with a matatu robbery, includes graffiti and MC games, runs through a virtual Nairobi … a kinda slum version of Oregon Trail.

What struck me cold was that most of the computers were unused, and were completely locked down to only run this game. Here was a resource that any school anywhere in the world could use well, and Time Warner only permits their game, a game that you also can not download and run freely. Frankly, an idiotic implementation of a good initiative, and somehow emblematic of many things wrong with the inappropriateness of technology in development. I can understand the desire to make sure the game is actually played, rather than just giving away computers for kids to check Facebook (of course that would happen) … but you do that through a well designed program and continual involvement rather than locking away all the other potentials of this computing resource.

Now we could make pretty good use of those computers for mapping in Mukuru. As it is they are useless for anything else.

Perhaps one of the hardest lessons I’ve been learning is the limits of innovation. Map Kibera is innovative, and it’s much hyped, but is it in danger of being another project that only looks good from afar? To really make an impact with technology, is requires far more than simply doing something new. It takes a lot of work which you might think is boring, lots of discussion, lots of program design, lots of failure and revision, lots of reality. There are limits beyond simply getting computers and internet into marginal places, limits beyond training, that have to do with the dense interconnection of all issues facing our increasingly urban and marginal world. The shiny glean of technology starts things off, but after that the work may be the same as ever … learning from each other, respecting different points of view, long negotiations of how things can change for the better.

cross-posted from Brain Off

Map Kibera’s focus is online, but net access and technical knowledge is Kibera is still a big challenge. This is why our plan has always been to print up large number of maps (or an atlas) and distribute to all the schools, organizations, clinics, churches, whatever in Kibera, so the work of Map Kibera has maximum exposure and impact.

We’ve also been holding community meetings that feature paper maps. This is very much like what’s called Public Participatory GIS. That decade-old-plus methodology intends to bring mapping to a wide audience, through discussion and map drawing, or sometimes using some lightweight digital technology. The idea is not to capture precise information, but more subjective experience and ideas around place. But now with web mapping, and particularly OpenStreetMap, the public is simply participating, and the line between precise and subjective information is blurring.

Still, paper and hand drawing is a powerful tool. Walking Papers is the prime example of that. As we have moved into the community meetings, we wanted to capture this hybrid … subjective, familiar experience, but in a toolset that leads to easy digitization, reuse and sharing of results. I would’ve previously critiqued PPQIS for the lack of re-use … the results often disappeared to a desktop or report. What we came up with here was much inspired by Local Ground (pdf), which used modifications of Walking Papers to bring paper drawing in Bay Area high schools online.

All the dirty details on our organizational process for this is in the wiki. Here I wanted to document the technical process, and highlight a couple areas for improvement.

take a look.

We start off by printing maps by generating a A1 sized pdf map, using an extract of OSM data on whatever theme. Those get printed down in Nairobi’s industrial area, small run of 5 prints, costs about $10 each (we could probably do better if we had time to bargain). We buy good quality tracing paper from the stationary store in >Westgate. We have sets of colored markers; each marker is associated with a kind of question during the session.

At the meeting we tape down tracing paper on top of the map print, and draw a line over the border of the map and the north arrow. Discussions are led by mappers and us coordinators, a writer is chosen, and the mapping commences. The quality and revelations of the meetings have been amazing, Besides the written map, everything is documented by video, photos, audio, and write ups. We’re still compiling everything, but this page on security shows a preview.

We roll up all the maps for transport home. Unroll, and each tracing paper is affixed to the wall, backed by the blank opposite side of one of the paper maps. The map is photographed with a Digital SLR on a tripod, with flash, at a distance that reduces parallax as much as possible. Once a suitable image is captured, that’s copied to the computer. The image is clipped to the drawn boundary (this is sometimes a little bit of an art), and uploaded to the server. Also placed on the server is a “bounds” file, which simply contains the west,south,east,north bounding box of the map.

On the server, this ruby script is run. It goes over all images in the directory, and creates a world file, then uses gdal to convert the image to a GeoTIFF. The script outputs configuration for TileCache and some javascript to configure the OpenLayers maps.

The result is pretty decent, and very close to the mark. It certainly could be improved, if we had a large scanner, but that’s not available. Alignment, lighting and parallax introduces distortion. Could automatically extract the writing, like Local Ground does with PIL, but for now simply allow for altering transparency of the entire image in the display. The background tiles should be customized to the theme. Some of the text lies outside the clipping area, which could be solved by simply recording registration marks of the four corners. Perhaps with Walking Papers properly integrated, this would be easy. Finally, we’ll be integrated the drawn maps into pages included narratives and media, with links that highlight OSM features in browser.