Tag Archives: featured

They want to take away maps they never gave us

For anyone familiar with DataMeet, a community of data enthusiasts1, you would be aware that the discussion can be rather stilted. Even though the list is dedicated to all types of data, geospatial data seems to be the main topic. There are over 700 topics related to geospatial data, out of 1600. That is nearly half of the conversation. People who ask come from all kinds of backgrounds, researchers, journalists, data analysts, startups, students and mapping professionals.

As the Indian tech economy grew over the last five years we saw an increase in membership and in asks for geospatial data, in downloads of the open shapefile data we have and a lack of understanding of the geospatial policy in India. Why is it so hard to find maps here? People were asking for data and asking why it wasn’t available, wanting better and more accurate data than the scraps that were available online for free or even what was being sold.

With the SaveTheMap campaign in full swing we wanted to look at the background of mapping in India and why in the future embracing openness of geospatial data is the best solution.

Open Data Camp 2016: Pollution Party! Full Schedule

REGISTER TODAY! We have reached capacity but have left it open for today. If you are not registered you won’t be able to join as Google security is very strict and will require you to be on a list.

Day 1: Pollution Party!

9:00am – 10:00am Registration
10:00am – 10:15am Introduction to OpenDataCamp
Team DataMeet
10:15am – 10.55am Karnataka State Pollution Control Board
By Dr Nagappa, Scientific Officer
11:00am – 11:10am Tea Break
11:15am – 12.00PM Environmental Support Group
12:00pm – 01.00PM Water Dr. Priyanka Jamwal
Environmental Researcher who currently is a fellow in ATREE. Her work focuses on identification of contaminant sources in surface water bodies, modeling the fate and transport of contaminants in urban hydrological systems and assessing the risk to human health due to exposure to contaminants.
01:00PM – 02.00PM Lunch
02:00PM – 03.00PM Pollution Data Collection Demos
Sensors without Borders, IndiaSpend*, Hindustan Times, YUKTIX – Open Weather Network Bangalore, India Open Data Association
03:00PM – 03.15PM Tea Break
03:15PM – 03:45PM Getting to 12 PM 2.5 | Setting the context for Action!
Sensing Local is a Bengaluru based do-tank focused towards making cities healthier, safer and more inclusive. The studio is working in partnership with Anti Pollution Drive (APD) Foundation, Mangalore towards a collaborative project on tackling air pollution. (https://sensinglocal.wordpress.com/
03:45PM – 04:45PM Urban Emissions
By Sarath Guttikunda
04:45PM – 05.30PM Group conversation and planning session on response to Geospatial Information Regulation Bill 2016
By Volunteers of SaveTheMap.in
05:30PM – 06.00PM Closing Remarks and Plans for Day 2

Pollution DEMO HAPPY HOUR!

Mapbox Happy Hour, 6p to 9pm. Puma Social Club, 100ft Road, Indiranagar. Bring your badges!

Day 2: Action Party!

“Hardware Hello World” for children.

A video posted by Thejesh GN ತೇಜೇಶ್ ಜಿ ಎನ್ (@thejeshgn) on

Sign your kid up to learn how to build environment sensors.

Sensor workshop poster

It is also a free day for people to demo, share and work on any projects they want!

Huge thank you to our sponsors!

Sponsors

http://juxt-smartmandate.com/project/india-open-data-association/

Geospatial Information Regulation Bill 2016

The Ministry of Home Affairs just released a draft policy on regulating geospatial data.  We have several concerns regarding this bill and are drafting a response.

Here’s what you can do to contribute to the conversation.

  1. Read and comment on the policy here.
  2. Contribute to the conversation on the google group or
  3. contribute to the hackpad where we are gathering thoughts.

We have a month to respond. This bill could seriously restrict everyone’s access to mapping data and it even might restrict a individuals ability to keep any mapping data. It is an important conversation we need to have with the government.

12 DAYS TIL 2016 Bangalore Open Data Camp: Pollution Party!

DataMeet will be hosting the 5th Bangalore Open Data Camp: Pollution Party on May 14th and 15th.  This year we want to spend time and look at the growing problem of pollution by spending two days examining the role of data. Last year saw a major turning point in the debate around pollution. Indian cities became a major focal point, as proof that New Delhi has worse air quality than reigning champion Beijing was proven with data. This put a spotlight on air pollution problem across India. At the same time water pollution from industry has also come up in the foaming lakes and rap videos fighting for recognition of pollution and its effects on people. The economic and development growth has meant that the building industry has been in over drive bringing sand and dust into urban and peri urban areas in large quantities plus the growing lack of proper trash disposal has had major health implications for people from all social economic backgrounds.

However, the actual exposure of pathogens and pollution is not well known, extensive data has not been made available or is being collected in a way that can’t be easily understood or acted upon. This has spurred the rise of data collection networks and agencies to fill this gap. In every major city citizen supported cheap sensor devices have been put around cities to add data to the small number of official government monitoring stations.

This year at Open Data Camp we want to explore the role of these data collection network in a growing citizen and private sector monitoring role. What is the role of open data? When these networks grow can there be agreement on standards and formats to be maintained? and Are there financially sustainable solutions that can be built on open data?

Notably Karnataka State Pollution Control Board is attending to give the keynote in the morning and hopefully bring some data with them for us.

Tentative Agenda

1) Karnataka Pollution Control Board

2) Environmental Groups to give the general ecosystem around enforcement

3) Data collection networks
Sensors without Borders
IndiaSpend*
Hindustan Times*
YUKTIX – Open Weather Network Bangalore
India Open Data Association

4) Water Pollution
Ground water
Urban lakes

5) What you can do with robust data?
Urban planning
Transport
Modeling for enforcement.

6) Open Environmental Formats and Information Discussion

Day 2

We will be hosting a sensor workshop for kids http://odc.datameet.org/sensor_workshop

Sensor workshop poster

We’d like to thank our sponsors Google, Sensor without Boards, India Open Data Association, Oorvani Foundation, and partner Reap Benefit. If you would like to sponsor or get involved please contact me @ Nisha (at) Datameet.org

Meet a DMer: Dilip Damle

On the DataMeet list we have started referring to each other as DMers.  So I wanted to start highlighting people who are pretty interesting and have a great insights into open data.

Dilip has been a major contributor to the list for a few years. He is always sharing data, advise, and information. He has contributed to the pincode and shapefile conversations and it always a source of support.

Where are you from? What do you do? 

I am from India, born and studied in Goa. Presently (last 28 years) based in Delhi.

By qualification I am a Mechanical Engineer.
Presently I am a freelancer (worked as an employee between 1981 and 1992)  As a one man SOHO professional I provide services to different Private organisations for themselves and some  Private organisations in turn providing services to Government agencies. Area of specialisation is mainly application of computers to Engineering, CAD, Technical publications, Cartography, Data Maintenance,  MIS reports and custom software.

I am a part time hobby programmer and have been programming since 1983 for fun and to automate my own work– VB, VBA and Autolisp.

How did you find out about DataMeet? 

I wanted to make and publish editable version of Election maps and was looking for the source of updated maps after delimitation.  I bumped in to [Raphael] Susewind’s Blog and via that page came to know about Datameet.

Why are you interested in data?

Mainly to make editable maps in common software, which I have a plan to offer free. More recently I have been doing less work on CAD and more on databases. In the process I am also hooked to the beauty of clean data represented especially in Database as against Excel.

Do you believe in open data? and why?

Yes, At least the data that is relevant to society as a whole.

Reasons:

  1. Only open data can be that Single Truth. Otherwise multiple mismatching versions float around for commercial reasons.
  2. There are no unnecessary fights over wrong data.
    (The most classic example is the India’s Boundary map. In this world of computers we have not provided a “Correct” Boundary accessible to all in a digital format and and want to stop all “Incorrect” data freely available just by legislation and expecting everyone to hold a print in your hand and come to Dehradun for “approval”. It is ridiculous.)
  3. Let there be commercial exploitation by value addition like visualization, Web Access but raw data generated by agencies that run from taxpayer’s money should be available in the open. Except for security, military and personally identifiable data.

What do you hope to learn?

I hope to interact with varied people and know newer things and techniques  that I might not have even heard of before.

What is your impression of the DataMeet community?

Good people but It is too small, needs to be bigger.

What kind of civic projects do you work on? What kinds of civic projects are you interested in working on?

I have worked on Water Supply and  Sewer networks mainly the application of computers for several years. A little on Storm water.
In future I wold love to work on Transportation modeling.

Share a visualization that you saw recently that made a big impression?  Share an article you have read recently that made a big impression? (does not have to be data related)

Share a visualization that you saw recently that made a big impression? Share an article you have read recently that made a big impression? (does not have to be data related)

A visualisation about Evolution.Evo_large

GPS and its Discontents

There is no greater success story for open data than GPS. The decision by the US government to make it available so it can be used for commercial purposes is the stuff of lore and what propels so much of the enthusiasm for open data.

Audiomatic’s show The Intersection is a podcast hosted by the dynamic duo Padmaparna Ghosh and Samanth Subramanian who explore interesting topics every other week.

Last week they did a show about GPS and it’s history and uses. Our own Thejesh GN was interviewed about his hobby of using GPS to go on treasure hunts.  They also talk about the Indian Government’s move to create a national GPS infrastructure with their own satellite so they don’t have to rely on the US.

I found the podcast informative and interesting and it hit on an important note as to why open data in India is so important.

Like GPS infrastructure to support India’s defense; data in India also needs to be invested in and promoted so that the reliance on others can reduce. Why is Google Maps, not Survey of India,  the source of mapping information in India? Why are their so many private data collection networks set up with foreign funds and private interests?Because GOI doesn’t invest in the potential of their data to build markets and make their job easier and more effective.

Open data is just one way of showcasing how better data can be used as well as offer guidance on how the government can invest in data collection and dissemination.

Anway it is a great podcast please give it a listen.

Analysing Bangalore’s Bus Network

Open Bangalore has been a pioneer in opening up several data sets that help understand Bangalore city. This includes the network of Bangalore Metropolitan Transport Corporation (BMTC). The BMTC operates over 2000 routes in the city and region of Bangalore and is the only real mode of public transit system in the city. Some of us at DataMeet took to time understand its network better by performing some basic analysis on the gathered dataset. The data set had bus stops, routes and trips. We inspected frequency, coverage, redundancy and reachability.

Longest route

BMTC is known for its many long routes. Route 600 is the longest, making a roundtrip around the city, covering 117 km in about 5 hours. There are 5 trips a day, and these buses are packed throughout. It should be noted that while the route traces the edges of the city in the west and north, it encircles the larger industrial clusters of the east and south.

View the map full screen.

Frequency

Next, I wanted to look at the frequency of different routes. In the image below, stroke thickness indicates how many trips each route makes. The relationship of the bus terminals with neighbourhoods and the road network can be easily observed. For instance, the north and west of the city have fewer, but more frequent routes. Whereas, the south has more routes with less frequency. Also, nodes in the north and west seem to rely more on the trunk roads than the diversely-connected nodes in the south. One can easily trace the Outer Ring Road, too.

View the map full screen.

Reachability

I tried to define reachability as destinations one can get to from a stop without transferring to another bus. The BMTC network operates long and direct routes. The map shows straight lines between bus stops that are connected by a single route. The furthest you can get is from Krishnarajendra Market (KR Market) to the eastward town of Biskuru: roughly 49 km as the crow flies.

View the map full screen.

Direction

Which directions does BMTC run? It is interesting that BMTC covers the city North – South (blue) and East – West (brown) with almost equal distribution.

View the map full screen.

Coverage

BMTC routes are classified into different series. Starting from 1 – 9 and A – W. I analysed coverage based on series 2 (blue) and 3 (green) and they make up almost 76% of the entire network.

View the map full screen.

Redundancy

Tejas and I took turns to try and figure out the redundancy within the network. Redundancy is good to absorb an over spill of bus commuters. Redundancy is a drain on resources and makes it hard to manage such a vast network with efficiency. So, we looked at segments that overlapped different bus routes.

View interactive map.

Node strength

This map by Aruna shows node strength – number of routes passing through a particular stop. You can see that the strength decreases as we move away from the city center with the exception of depots.

View interactive map.

Just like the data, our code and approach are open on Github. We would love to hear from you, and have conversations about the visualization, the BMTC, and everything in between!

OpenPostbox.org

We got a chance to talk to members of Karnataka Philatelic Society about OpenPostBox. They are very interesting set of people. They have also started sending me the postbox pictures using WhatsApp along with location. Now I need to find an efficient way to extract them and insert into my database.

As of now I am thinking of Export -> Parse -> Insert. Working on it. If you have any ideas do email me.

dm_openpostbox

Details of the meet are on my personal blog if you like to read.

Global Open Data Index: Water Quality

Last year I helped assess the water quality section of the Global Open Data Index (GODI). Given the news of lead poisoning in Flint, Michigan and increasingly beyond, safe drinking water is no longer assured even in countries where it’s been guaranteed, so I am very glad they included it in GODI.

GODI is a survey of 122* countries that look at the status of ‘high priority datasets’ and whether they are truly open according to the Open Data Criteria. Water quality was included last year for the first time. So my job was to examine each country’s submission  and assess if the data submitted was what was asked for and met the criteria for being open. This was a daunting task but I figured if I could find water quality data in India of all places it wouldn’t be impossible.

Assessment Criteria/Methodology

GODI looked for very specific parameters:

While there are a lot more parameters that could be asked for, these were a good sample of parameters to assess if there is robust water monitoring in the country.

After the initial submission phase there were a lot questions about why wouldn’t the survey just ask for drinking water quality data or environmental monitoring data?

Choosing parameters instead of programmes is important because monitoring the environment and drinking water quality are connected. Some countries haven’t really established large nationalized water treatment strategies, drinking water comes directly from a natural resource so the environmental monitoring data inadvertently applies to the drinking water scenario.  Which means that if a country really has robust water quality data they must have these 5 parameters because they cover surface and ground water sources and also reflect safe drinking water standards.

The assessment would be rejected if a submitter only found the surface water body monitoring stations (environmental water monitoring) for instance because arsenic and fluoride are only found in groundwater. So the submitter would either ideally find the treated drinking water quality data which will cover all the parameters or the source water quality data for both surface and ground water.

For a full look at the methodology of the entire survey go here.

Some background

There is no one way to create water management systems but there are two major ways by which people get water – directly from the source or piped in from a source or a treatment facility. The origins of the water source is important. If you are getting water from the ground there are different quality issues  than from surface water (lake or river). If water is from a treatment plant there is a possibility that plant is getting water from both surface water, ground water, and in some cases recycled water. Usually water quality is measured at source and after treatment (treatment plants take multiple water quality samples during the treatment process.)

A full water quality assessment means lots of parameters and not all of them are tested the same way; some parameters take several days and require specific conditions, others can be taken easily through filters or litmus papers.  Water quality is a deliberate process of sampling and testing, and it not as easy as sticking a sensor into the water and monitor a continuous feed of data (although the potential for these approaches is quickly growing as technology improves.)

What I looked for

Since water quality was a scientific process I figured if I found any proof of water treatment or quality monitoring, a dataset would not be far off. After going through a few countries I noticed that the different water management approaches and policies affected where you would find the data.

Most countries give drinking water treatment responsibilities to local bodies but sometimes is monitored by central government under public health regulation so aggregated data could lie with the public health ministry or the environmental protection body.  In most cases responsibility for environmental monitoring fell to a central government Environmental Ministry.

So this scenario means that multiple datasets exist – a centralized dataset for surface and groundwater that  usually lies with the environmental ministry that could have all the parameters but sometimes doesn’t, or it doesn’t have real time data (this means data  may be available but from less frequent data collection such as quarterly or half yearly efforts). Or the Public Health Ministry has reports of water quality with all the parameters but these are aggregated, and usually in a report form (not a dataset) and not updated in a timely manner.

The US, for instance, falls under this group and can produce confusing submissions. The US has a robust geological survey of surface and ground water sources. However, the drinking water reports are supposed to go to the Environmental Protection Agency but no one seems to be updating the database with information. In my assessment I reduced the score because both are supposed to be available in the public domain.

There are countries like Belgium where water management and monitoring are completely left to the local body and there is no central role for monitoring at all, which meant there is no dataset.

There are countries where there is a strong central role in water management and a dataset could be made open like in France. Korea stood out, because they have live real time water quality information from their treatment plants that gets updated to a website.

Then there are the ‘unsures’: which are countries that seem to treat water to some degree or have national drinking water monitoring programmes but don’t have data online, reports or any mention of data at all. This is not restricted to the developing world. I was very frustrated with several European countries with newspaper articles riddled with reports of how pristine and delicious their water is that don’t have a single public facing dataset.

Take Aways

United Kingdom and the US, both pioneers of the open data movement had terrible water quality data for water treatment, and no effort has been made to bring the data together or make it available in a real time fashion.  Also it is not clear to citizens who holds local bodies accountable for not updating their reports, making reports public or finding ways to bring this data into the light so it can be usable. It is no wonder that the US is now on the cusp of a public health crisis.

It is frustrating that the open data movement hasn’t quite been able to reconcile decentralization and local responsibility with national level accountability and transparency. Public health is a national level issue even though local and regional contexts are required for management. How do we push for openness and transparency in systems like this?

In places like India where water quality treatment is largely left to private players and huge populations are not receiving treated water, the need for data to be available, open, and in the hands of central bodies but also local players is a must, because people need to try to find solutions and where to intervene. Given the huge problems with water borne diseases, the slow but epic arsenic and fluoride poisonings gripping parts of India, and the effects this will have for generations, making this data public, usable and demystified is no longer an option.

All in all, I have to say this was an enlightening experience, it was cool to be able to learn something about each country. In our continuous push for open data we sometimes get lost in standards, formats, and machine readability, but taking a moment to really prioritize our values in society and have open data reflect that is essential. Public health outcomes and engaging with complex issues like it are an essential part of how to grow the open data movement and make it relevant to millions more.

*(Correction: Previous version said the survey included 148 countries, the actual number is 122.)