consultation, Data Workshops, Events, How-To, Uncategorized

Sikkim

February 3, 2016 Nisha Thompson

#LATEPOST

Sikkim State Government passed an open data policy Sikkim Open Data Acquisition and Accessibility Policy in 2014. With pushing from the Chief Minister and Member of Parliament the Honorable Prem Das Rai they turned to open data to take control of the state’s data. The Honorable Mr PD Rai has repeatedly mentioned is the lack of access to government information on demand. It is not uncommon for lawmakers to ask questions only to have to wait a day or more for the answer and lose a moment to use that information for decision making.

An Open Data for Human Development Workshop was organized by the International Centre for Human Development of UNDP India, with the Centre for Internet and Society, AKVO, Mapbox and DataMeet co-facilitating the event in Bangalore last June. The aim was to bring together members of the Sikkim government, IT professionals, and open data enthusiasts.

In April before the workshop Sumandro (CIS) and I went to Sikkim to have a pre consultation with the Sikkim government on how to prepare for the large workshop in Bangalore. We met with the MP and the heads of the Rural Development, Health, and IT departments to discuss their plans to implement their open data policy. Then there was a large meeting with all the departments and the MP. We presented different things you can do when data is opened and offered suggestions for how to implement the policy. The departments took turns discussing their issues regarding implementation; concerns like server space, technology needs, how to create incentives to accurate and timely data uploading were shared.

We presented things for them to think about in a preparation for the June event and for how to work with the open data community in India.

In June the workshop was held as NIAS. Thej gave a session on data tools that can be used to assemble, clean, analyze, publish and visualize data. Some of the tools that he introduced and used during the workshop are

Tabula Its difficult to extract data from PDFs. But Tabula allows you to extract that data into a CSV or Microsoft Excel spreadsheet using a simple, easy-to-use interface. Tabula works on Mac, Windows and Linux.
Open Refine – is a powerful tool for working with messy data: cleaning it; transforming it from one format into another; extending it with web services; and linking it to databases like Freebase.
DataWrapper allows you to create powerful charts very easily.
CartoDB is the Easiest Way to Map and Analyze Your Location Data

“Overall interaction was great. Delegates from Sikkim were very interested in DataMeet community and work we do as community. Some part of the workshop was used to introduce the community aspect of Data.”

You can see the full notes of the event at Centre for Internet and Society’s blog.

We are looking forward to see Sikkim be the first state to implement an open data portal using the Data.Gov.In platform.

hackathon, How-To, transport, transportation

To Hack or Not to Hack….

February 2, 2016 Nisha Thompson

Hackathons are a source of confusion and frustration for us. DataMeet actively does not do them unless there is a very specific outcome the community wants like freeing a whole dataset or introducing open data to a new audience. We feel that they cause burn out, are not productive, and in general don’t help create a healthy community of civic tech and open data enthusiasts.

That is not to say we feel others shouldn’t do them, they are very good opportunities to spark discussion and introduce new audiences to problems in the social sector. DataKind and RHOK and numerous others host hackathons or variations of them regularly to stir the pot, bring new people into civic tech and they can be successful starts to long term connections and experiments. A lot of people in the DataMeet community participate and enjoy hackathons.

However, with great data access comes great responsibility. We always want to make sure that even if no output is achieved when a dataset is opened at least no harm should be done.

Last October an open data hackathon, Urban Hack, run by Hacker Earth, NASSCOM, XEROX, IBM and World Resource Institute India wanted to bring out open data and spark innovation in the transport and crime space by making datasets from Bangalore Metropolitan Transport Corporation (BMTC) and the Bangalore City Police available to work with. A DataMeet member (Srinivas Kodali) was participating, he is a huge transport data enthusiast and wanted to take a look at what is being made available.

In the morning shortly after it started I received a call from him that there is a dataset that was made available that seems to be violating privacy and data security. We contacted the organizers and they took it down, later we realized it was quite a sensitive dataset and a few hundred people had already downloaded it. We were also distressed that they had not clarified ownership of data, license of data, and had linked to sources like Open Bangalore without specifying licensing, which violated the license.

The organizers were quite noted and had been involved with hackathons before so it was a little distressing to see these mistakes being made. We were concerned that the government partners (who had not participated in these types of events before) were also being exposed to poor practices. As smart cities initiatives take over the Indian urban space, we began to realize that this is a mistake that shouldn’t happen again.

Along with Centre for Internet and Society and Random Hacks of Kindness we sent the organizers, Bangalore City Police and BMTC a letter about the breach in protocol. We wanted to make sure everyone was aware of the issues and that measures were taken to not repeat these mistakes.

You can see the letter here:

We are very proud of the DataMeet community and Srinivas for bringing this violation to the attention of the organizers. As people who participate in hackathons and other data events it is imperative that privacy and security are kept in mind at all times. In a space like India where a lot of these concepts are new to institutions, like the Government, it is essential that we are always using opportunities not only to showcase the power of open data but also good practices for protecting privacy and ensuring security.

budget, consultation, Delhi

Investing in Data: Pre Budget Consultation with the Finance Minister

January 11, 2016 Nisha Thompson 1 Comment

Last Thursday DataMeet was lucky to be invited to a Pre Budget Consultation with the Finance Minister Arun Jaitley. We were invited to attend with the IT sector group and give some suggestions on how the next budget could invest in open data.

After some consulting with the various city chapter organizers we came up with some recommendations that could appeal to this audience. We decided to emphasize that government data is a financial asset that needed to be invested in, in order for it to reach its optimal economic impact. A stance the US government made in it’s open data policy.

You can read the note we submitted here:

The meeting was Thursday morning in Delhi at the Finance Ministry offices, Sumandro came to represent CIS and I attended to represent DataMeet.

The Finance Minister was there along with the Secretaries;
Shri R.N. Watal, Finance Secretary, Shri Shaktikanta Das, Secretary, DEA, Dr. Hasmukh Adhia, Revenue Secretary, Ms Anjuli Chib Duggal, Secretary, Financial Services and Dr. Arvind Subramanian, Chief Economic Adviser (CEA).

It was a round table and the participants were organized by software and hardware, and we presented in the order we were seated.

Shri Ramadas Kamath, Infosys,
Shri P.V.Srinivasan, WIPRO,
Shri Anil Chanana, CFO, HCL,
Shri Pauroos D Karkaria,TCS,
Shri R. Chandrashekhar, Chief Economist, NASSCOM,
Ms Nisha Tompson, Founder, Datameet,
Shri Vinod Sharma, Chairman, Electronics and Computer Software Export Promotion Council,
Shri Nitin Kunkolienker, Vice President, Manufactures Association for Information Technology (IT),
Shri Rajoo Goel, ELCINA Electronic Industries Association of India,
Shri Hari Om Rai, Co-Chairman Task Force on Mobile Phone Manufacturing,
Shri Suraj Saharan Ajit Pai, COO,Delhivery,
Shri Sumandro, the Centre for Internet & Society and
Shri Vikas Jain, Member, Task Force on Mobile Phone Manufacturing

While most of the suggestions were related to tax breaks, subsidies, and trade issues, I was able to introduce the idea that the Government of India’s data is an economic asset that can help create markets, increase innovation, and allow for more accountability in scheme implementation. In order for the data to do these things it has to be opened up and that means the government must invest in the NDSAP policy and focus on data standardization, cleanup, and collection. Also policies need to be reviewed and revamped in order to keep up with demand and use of data. Like the mapping policy should allow for more contributions from private sources and crowdsourcing so the Survey of India can keep up with demand for geospatial information. The Copyright Act also needs a clarification on the status of data and the Ministries must be willing to release data under open licenses.

In all the meeting was short, with the main focus being toward how to encourage manufacturing sectors because of the Make in India initiative. I was happy to be there and mention ideas and concepts that were not being discussed in rooms like that one and to also offer a perspective on open data.

We hope to keep in touch with the Ministry and continue to take advantage of any opportunity to share our experiences and views on how an investment in data can be a huge economic asset to India.

You can see the Government’s Press Release here.

Delhi, OpenDataCamp, Uncategorized

2nd Open Data Camp Delhi!

November 29, 2015 Nisha Thompson

Last Sunday DataMeet Delhi hosted their 2nd Open Data Camp! 60 people decided to spend their Sunday with us to discuss Digital India and find ways to make this programme more Open and Transparent.

The Delhi chapter decided to examine the role of openness in Digital India, especially how the open data agenda should be integrated into the initiative. Digital India is the flagship programme of the Government of India to harness the possibilities of information technologies for accountable governance, effective citizenship, and a productive and job-creating digital economy.

This event also explored the recent international push towards better global availability of interoperable and comparable data, such as the Data Revolution for Sustainable Development initiative of UN and the International Open Data Charter introduced by the Open Data Working Group of Open Government Partnership. The discussion looked at these wider conversation in the keynote and the morning panels.

Keynote: Honourable MP from Sikkim P.D. Rai.

The MP from Sikkim started off the day by talking about his experience setting up the first state level Open Data Policy, Sikkim Open Data Acquisisiton and Accessibility Policy (SODAAP),, and why it was important for them to take control of the state’s data through openness.

He stated the the “lack of reliable, structured, and proactively available data is a key barrier to good governance.” So the SODAAP would allow state legislators to get access to data as they need it instead of having to go through the current structure of asking the Centre for data. “Why is it that we have fancy phones but we can’t get data on public policy & schemes on it for good decisions.”

When asked how to get government to change he stated, “I’m not the executive, I’m a lawmaker. I don’t represent the government. I question it as much as you do.”

Open Data and Digital Governance

Anoop Aravind, Konatham Dileep, and Nikhil Pahwa

This panel focused on the Digital India from a government and journalistic point of view of Digital India. The panel had a representative from Telegana, KPMG who is implementing E-Panchayats, and from Media Namma.

Dileep the Digital Media Director for Telegana pointed out that the government is the biggest creator of data but they are not set up to share, and are not encouraged to. Anoop from e-Panchayats pointed out that there are technical issues with implementation and technology infiltration at the local level. He said the biggest problem for them is the lack of mapping data that can used to help with planning.

Nikhil from Media Namma made the point that the government should proactively disclose data, “why do we need to get personal relations to get the data?” but this doesn’t replace people’s right to ask for information and not just rely on information provided by open data. Right to Information is still vital and this includes an expanded effort to protect people’s privacy.

When asked what are the challenges of openness for Digital India? That despite the big fanfare there is uneven implementation and issues that have to be solved before the dreams of Digital India are realized, and that people have to work with the government to show them the reason to be open.

Open Data and Digital Citizenship

Bhanupriya Rao, Dr. Biplav Srivastava, Nic Dawes, and Shashank Srinivasan

Bhanupriya Rao an RTI activist described out RTI has a pro-active disclosure requirement, however, it is not in practice and without that RTI is the best tool for now. There is no right to data concept.

Nic Dawes described journalism as a constitutional mandate and went on say that that open data and journalism communities must work together more. Journalists can deal with biases, data interpretation issues, graphic presentations, and tell compelling stories using tech and design.

Biplav Srivastava spoke about the need to move toward smart data consumption, for policy decisions and individual decisions. That the next steps are data integration/re-use/standards, and linked data for analytics.

Shashank Srinivasan shared his experience with open data for conservation (WWF), how they consume OSM data for needs of protecting wildlife. What are risks for crowdsourcing for wildlife conservation? Open data can be a problem for conservation, control over the end user is needed.

Questions to consider:

How can open data improve our work? How can academia and open data converge? Can donors influence on releasing data? What does it mean to be a digital citizen?

Lightning talks

Guneet from Akvo shared their smart phone app that detects Fluoride levels in water.

Maning from HotOSM shared their work around the world providing maps during natural disasters, including the Nepal Earthquake.

Transport Working Group shared the work looking at bus data in Delhi.

Bihar Gender Watch shared the work of looking at the gender split in elected bodies.

NewsPie is an online news site, they shared the data work they have done in roads and around net neutrality.

Aditya Dipankar shared his work designing information.

Aruna from MapBox shared their work mapping road naming.

Turam shared his project that built more data collection tools on the Open Data Kit.

Yogesh from Random Hacks of Kindness (RHOK) on his vision for an open revolution! Also the work of RHOK in India bridging gaps between organizations on the ground and technologists.

Monish Khetrimayum a PHD student spoke about big data, governance and citizenship.

Rakesh from Factly describes how they use RTI information and open data to make sense of information for journalists and citizens.

Group Activity: Response to Digital India

Groups were formed to discuss each pillar and come up with questions.

We have gathered all the questions and put them in the DataMeet hackpad, you can find each pillar here.

Please feel free to take a look and add more questions and dataset requests.

After a week’s time we will be gathering everything and writing a letter of request for openness to Digital India and the various departments, DIETY, to ask them to make this information available.

It was a fantastic day! DataMeet Delhi did an amazing job putting together really interesting speakers to make this a well rounded interactive event.

Thank you especially to the sponsors for helping make this event great!

SARAI for the space
AKVO for travel
ICFJ for food and other support.
RHOK for travel

GIS, How-To

Map of Electoral districts of Sri Lanka

November 17, 2015 Thejesh GN

SriLankan maps for Electoral districts are available for download now. I initially made this for a friend who wanted to analyze the election results. The Electoral districts are derived from the administrative maps.

via GIPHY

You can check the diff on github to see how the maps were changed.

GADM database of Global Administrative Areas is the source of administrative data. I used three simple online tools

GeoJSON.io for converting from KML to GeoJSON and adding attributes.
MapShaper for merging the areas
GitHub for storing the map files.

Note: I don’t provide any guarantee on the accuracy of the maps. So don’t use if you want accurate maps. I have made notes on how these maps were derived. Use it if you think the process is right. Raise an issue if you find anything.

Data

Guest Post: Varun Goel- Releasing Data for Agriculture

October 24, 2015 Nisha Thompson

Varun serves as the chief data scientist at a research team led by Dr. Ashwini Chhatre, serves as the Research Node of the Revitalizing Rainfed Agricultural Network – an India wide network of NGOs, civil society organizations, researchers, policy makers and think-tanks that aim to reconfigure the nature, amount and delivery of public investments for productive and resilient rainfed agriculture.

The Combined Finance and Revenue Accounts (CFRA) report is an annual report prepared by the office of the Comptroller and Auditor General (CAG) of India to provides comprehensive Union and State government data on audited receipts, revenue expenditures and capital outlay for different major, minor and sub-minor heads.

Since the figures for actual expenditures on different heads may differ from actual budget allocation by as much 15 to 20 percent, and that each state might have different procedures of auditing, the CFRA data provides reliable and fairly disaggregated figures of public expenditure, audited by a central authority.

The research team at the Revitalizing Rainfed Agricultural Network (RRAN) has scraped and processed the CFRA data from 2005-06 to 2010-11 for all general and economic services to understand statewide public investments in agriculture and allied activities, and highlight the mismatch in investment and needs on the ground.

The processed data, along with detailed information for each head can be forked here.

Although the data is only available at the state level, it can provide valuable insight on not just public expenditure in other domains such as urban development, health, central and state sponsored schemes, but also highlight the differences in budget allocation and actual spending of various government heads.

Revitalizing Rainfed Agricultural Network (RRAN) has practice and policy node that generates ground based evidence and block, district and state level for policy engagement, the research node’s objective is to generate evidence for testing key hypotheses to enable an articulation of the nature and magnitude of public support needed to fuel growth of India’s rainfed agriculture. To facilitate this, a Data Center has been set up with the aim of acquiring, reconciling, processing, visualizing and disseminating pan India datasets to assist in exploratory analysis and develop research hypothesis, backing up policy advocacy through scientifically rigorous data analysis, and implementing data-driven decision-making tools for program implementation by grass-roots level organizations.

Data

Open Access Week – Open a Dataset with Srinivas Kodali

October 20, 2015 Nisha Thompson

Cross post from Lost Programmer

Starting today it is International Open Access Week, I have been associated with concepts of open data and open access since 2012 and was hoping to bring some serious attention to it in India. This week I intend to showcase a serious of datasets which several departments of Govt. of India publishes in there web portals through NDSAP apart from Open Government Data Platform

Today’s dataset which I want to bring attention is of Indian Customs. Indian customs maintains records of every product imported and exported through land, sea and air. They publish this data through their commerce portal. They should be highly appreciated for maintaining this website and publishing the data. The data is published as per Notification No. 18/2012-Customs (N.T) dated: 5th Mar, 2012

The data being published includes origin, destination ports, name of the product, Harmonized System code of the product, quantity of product, unit quantity of the product, customs valuation of the product. For imported goods, the origin country is published instead of the port, while for export you get to know the exact destination city.

Read the rest over at Srinivas’s blog here

And if you are using the data for anything please let us know! Stay tuned for tomorrow’s release!

Ahmedabad, Bangalore, City, Data, DataMeet-Up, Events, Hyderabad, People

Open Access Week 2015 India Events

October 19, 2015 Nisha Thompson

It’s Open Access Week! This week there are events around the country to celebrate openness and explore how far we have to go.

MapBox is putting up an amazing Open Data Gallery Tuesday the 20th in Bangalore. Come and hangout look at incredible art and projects from around the country!

In celebration DataMeet is doing its first MULTI CITY EVENT!

Join us Saturday 24th at 6:30pm for talks from Data.Gov.In, Ahmedabad and Bangalore with livestreaming between the cities!

Data.Gov.In will talk about the latest updates to Open Data in India.
Bangalore will discuss open access in general and open data projects.
Ahmedabad will talk about the status of Open Access in their part of the world.
Srinivas Kodali will talk about releasing datasets.

Bangalore’s event will be at Centre for Internet and Society.

Ahmedabad will be at CEPT University.

Please RSVP on Facebook or Meetup.

Let’s celebrate all we have been able to accomplish as a community and look forward to continuing to promote a culture of openness, sharing, learning and collaboration.

News

Nobel prize Winner Angus Deaton on the importance Open Data in India

Quote October 19, 2015 Thejesh GN

On Data{Meet} we have been talking about the importance of Open Data and quality of it. This year’s winner of the Nobel Prize for Economics Angus Deaton has similar point of view on the quality of open data. Whole article is worth reading, I am quoting a paragraph.

My work shows how important it is that independent researchers should have access to data, so that government statistics can be checked, and so that the democratic debate within India can be informed by the different interpretations of different scholars. High quality, open, transparent, and uncensored data are needed to support democracy.

I have used data from India’s famous National Sample Surveys to measure poverty. Perhaps the biggest threat to these measures is that there is an enormous discrepancy between the National Accounts Statistics and the surveys. The surveys “find” less consumption than do the national accounts, whose measures also grow more rapidly. While I am sure that part of the problem lies with the surveys—as more people spend more on a wider variety of things, the total is harder to capture—but there are weaknesses on the NAS side too, and I have been distressed over the years that critics of the surveys have got a lot more attention than critics of the growth measures. Perhaps no one wants to risk a change that will diminish India’s spectacular (at least as measured) rate of growth?

Source: TheWire
Picture credit: Nobel Prize

Delhi, GeoBLR, GeoDEL, GIS, map, mapping

The first GeoDel meetup

September 22, 2015 Shashank

On the 2nd of September, 2015, DataMeet-Delhi spun off a small side project known as GeoDel. Following GeoBLR‘s example, GeoDel is a Delhi-based group/community that meets to discuss open spatial data in the Indian context.

Akvo very kindly hosted us at their beautiful Delhi office, and we began with a very short talk by me (Shashank) on a quilt my mother made, based on OpenStreetMap data of South Delhi. Riju then spoke about mental maps, using a slideshow with some beautiful maps. He ended his talk with a participatory mapping exercise using FieldPaper maps of Delhi, where everyone who attended the meet had a chance to shout out a random place in Delhi, and everyone else had to mark it on their maps. It was a good way to learn about places in Delhi with arcane names such as ‘Rohini‘ and ‘Patparganj‘, and to end our first GeoDel as well.

GeoDel will have bi-monthly meets, so stay updated on its spatio-temporal coordinates via the MeetUp and FaceBook groups!

DataMeet is a community of Data Science and Open Data enthusiasts.