Letter to NIC for a data portal to host public contributed datasets

Sumandro drafted a letter to be sent to NIC regarding the possibility of a data portal to host public contributed datasets, that is datasets originating from both governmental and non-governmental sources, but contributed only by non-governmental agencies and individuals.

We sent that letter to NIC this week. Below is the copy of it.

Letter to NIC for a data portal for public contributed datasets

[ODCBLR2014] Address by TS Krishnamurthy

At this years Open Data Camp Bangalore Mr. T. S. Krishnamurthy addressed the audience.

From Wikipedia:

Taruvai Subayya Krishnamurthy (born 1941) was the Chief Election Commissioner (C.E.C) of India (February 2004 – May 2005).[1] His main assignment as C.E.C was to oversee the 2004 elections to the Lok Sabha. He was known for his integrity and a polite yet firm fist with which he handled all sensitive assignments throughout his career.He had earlier served in the Election Commission of India as a commissioner since January 2000.

Georeferencing 101 – Breaking down the electoral map – dot by dot

In preparation for Bangalore Open Data Camp 2014: Election special.  We did two small workshops one on working with PDFs and the other on learning Georeferencing and Vectorising basics. We did these two workshops because so much of working with election data is parsing PDFs and trying to get shapefiles.

20140315_133437

Indian MP/MLA Constituency shapes are created and maintained by the Election Commission. There is a great wealth of information on this site, and if you can go to the state sites to see even more local electoral information for Parliament and Assembly constituencies.  However, these maps they give you are in PDF, if you want to create visualizations, layer data, or get any in depth understanding of your constituency these PDF maps make it difficult.

Sajjad Anwar headed our Georeferencing 101 workshop.  He went through the basics of using QGIS the open source GIS software.

QGIS is a great free open source resource for creating, visualising and, editing spatial data of various formats. Like any editor you have to become accustomed to the layout and the terminology which can be hard for beginners.  It might be overwhelming when you first start but just take a deep breath.  There is a great intro guide here and tons of Youtube guides.

To start looking at georeferencing data:

Things you need:

  1. Download qgis 2.0 or above.
  2. Need a picture, any format jpg, png etc.
  3. Base Reference file preferably a vector file.

If you can’t determine the border needs enhancing clean it up with GIMP, Inscape or any of your favourite graphic editor, and use the Georeferencer plugin – set the ground control points

Where you can get some reference files

  1. Global administrative boundary website 
  2. Open street maps –not the best but available

If you can’t find the reference file you might need.  The best you can get is the taluk level, if you can get that you can reference constituencies but district level is also acceptable it is the easiest boundary you can reference.   Don’t use Google  it is a violation of their terms of service

When you have your picture and reference vector file you can then use QGIS to outline the picture by creating points.  You put a point on the picture then pick the corresponding point on the map.https://pbs.twimg.com/media/BiwxmhFCQAATnML.png:large

This assigns lat/longs to your picture and creates the references for the polygon to be created.  After this then you can add layers of data and create your map visualization!

Any georeferencing you do is a derivative work, so just keep in mind the license of the base layer.

You can learn more about the challenges of creating MP/MLA constituency maps on the DataMeet Google Group here.

We will be doing more of these workshops in the future so check back here for more information.

 

Notes from DataMeet-Up in Delhi, 22 November 2013

We had DataMeet-Up on Friday, November 22, 2013, at the Akvo office in Yususf Sarai Community Centre, Delhi.

Here are the notes from the meet-up [additional information in square brackets]:

Election Data Hackathon

  • We will undertake a collaborative mapping of datasets relevant for election data hackathon, using GitHub and Google Drive. More details about this below.
  • Datasets that we are trying to locate include: election results data (total vote count, vote count per party/candidate, etc), total utilisation and composition of utilisation of MP Local Area Development funds, parliamentary activities of MPs (presence/absence, questions asked, bill discussed, committees joined. etc), crime data corresponding to constituencies, etc.
  • We will identify organisations who might hold additional relevant data, such as PRS Legislative Research, Association for Democratic Reforms (and MyNeta.info), Gramener, and Hindustan Times [Anika used to work at HT].
  • Two caveats: (1) we may not get unique and standard identifiers across datasets, and (2) calculations may get difficult in case of by-elections [Lok Sabha Secretariat will have details of all by-elections, which can be accessed through RTI request].

Hack for Change on Women’s Rights

  • Shobha, Breakthrough.tv, led the discussion on the planned Hack for Change event being organised by Breakthrough and Hacks/Hackers, as part of the 16 days of activism against violence against women.
  • The hackathon is organised around urban safety data from Whypoll , multimedia evidences of early marriage practices in Bihar and Jharkhand gathered by Gramvaani , etc. It will also include a Wikipedia Edit-athon facilitated by Noopur Raval.
  • There were multi-directional discussions around other datasets of relevance for the hack event, which I have not kept track of very well. Overall, there were discussions around datasets available from , those published by National Crime Records Bureau, FIR and call database of Delhi police (and how to access that), and data on violence against women gathered by Tata Institute of Social Sciences from police stations across seven states.

Presentation on iPython

  • Konark Modi presented a detailed introduction to using iPython to undertake data cleaning in a very organised manner, as well collaboration features/workflow of iPython.
  • There emerged a demand for a tutorial on OpenRefine (previously Google Refine), which will be organised in a later meeting.

Mapping Indian Election Data

  • We will start documenting publicly available datasets relevant for studying past General Assembly (Lok Sabha) elections in India and the activities of the elected members at present. One can contribute to this mapping exercise in two ways, as mentioned below.
  • GitHub: We have created a repository for this data mapping exercise under the DataMeet organisation at GitHub. The organisation page can be accessed here, and the (india-election-data) repository can be accessed here. In the repository, I have created a draft format for documenting the identified datasets. This draft format can be accessed here. Please feel free to suggest changes to the draft format by opening an issue.
  • To document a dataset, use the format given in the repository, fill up the details, and rename the file according to the dataset’s name, such as “election-results-delhi-1995.md”. Then if you notice any requirement of data cleaning/reorganisation or lack of clarity regarding the dataset, open an issue (where the name of the dataset is mentioned) to note that task.
  • Google Drive spreadsheet: Alternatively, you can access this spreadsheet on Google Drive and add the relevant information about the dataset documented by you.

Please comment here or post to the DataMeet mailing list for any clarifications and suggestions.

DataMeet-Up in Delhi, Friday, November 22

After a hiatus, the Delhi DataMeet-Up is back. We are meeting today, Friday, November 22, at Akvo Foundation office, at 5:00 pm.

Here is the tentative agenda of the meeting:

  • Updates: Sharing news across the network.
  • Discussion: Discussing how we can support Hack for Change around women’s rights being organised by Breakthrough.tv and Hacks/Hackers New Delhi. Shobha and Anika would begin the discussion by talking about the planned event.
  • Discussion: Beginning a discussion towards a election data hackathon. It will be led by Satyakam and Surendran.
  • Presentation: Using iPython for exploratory data visualisation – Konark Modi, MakeMyTrip.com.
  • Discussion: Brief chat about the overall agenda of the Delhi chapter of DataMeet.

Location: Akvo Foundation, 3rd Floor, Ramnath House, Plot #18, Yusuf Sarai Community Centre, Yusuf Sarai. Nearest metro is Green Park.

[mapsmarker marker=”3″]

DataMeet is a community of Data Science and Open Data enthusiasts.