Data, Data - Get Your Free NYC Data Here!
The ubiquitous nature of data is undeniable—it is everywhere. We all use data to inform our decision-making, assigning subjective valuations and interpreting what it tells us.
The New York City Open Data initiative signed into law by Mayor Bloomberg in 2012 is a local law requiring that all city agencies publish data on a single web-based platform by 2018. In addition, the city publishes a plan of the progress and the projected release date of specific datasets. New York City has shared city data in multiple formats before the open data policy but these data often lacked consistency and were “locked” for analysis. Other times the city would bury valuable data in PDFs or use systems that required a coding background making it inaccessible and unusable to the general public.
While sharing data is a great step for a more open government, the goal of Open Data is to put the data into context. The data itself is only information; users can review and parse the data to create knowledge. Open Data is an opportunity for each of us to become active agents in our city. The project’s slogan is “Data for all,” -- making bureaucratic information more accessible is a creative approach to encourage citizens to create change. Individuals with varying levels of technical ability will be able to use Big Data for research.
Open data is easy to access, analyze, and visualize. The NYC Open data portal houses the bulk of the city’s aggregated data. The possibilities on the portal are endless! At UNHP we use Open data in several initiatives and constantly look for ways in which it can inform our work. We’ve started to use open data in our Building Indicator Project (BIP), a database that leverages public data to gauge physical and/or financial distress of multifamily properties in New York City. Most recently we have begun capturing 311 complaints in relation to open HPD violations of individual properties. We noticed a decline in violation data and wanted to do research around whether this was an accurate indication of improved living conditions. While the dataset only logs housing maintenance complaints from 2010, the dataset is updated daily and is a great source of information.
If you don’t have much experience with data analysis, a good introduction is the piloted data lens feature. The data lens is an easy-to-use tool allowing users to more easily understand the insights behind the numbers. Unfortunately, this feature is only available for a limited number of datasets. If you are prepared to dive into the numbers and figures, visit the portal to find a data point of interest. The search feature is limited, so we suggest looking at the available datasets by agencies. You can create a free account to save, manipulate, and comment on the available data sets. You are able to interface on the Socrata platform like an excel spreadsheet, you can move columns, sort, filter and prepare basic data visualization. However, for more in-depth analysis you would need to export the data to a different file format. Various file formats are available from CSV to PDF to XML.
The possibilities are limitless and the momentum behind the policy is staggering. Scrutiny and feedback are the most important tools to optimize and explore Open Data. The City has made a commitment to provide relevant, credible, and quality data. It is up to us, community groups and advocates to use it for the neighborhoods and people we serve.
Visit UNHP’s Community Resource Guide (CRG) to get started on your data journey. Our one-stop source for local community based data is populated with data sources, maps, geography and resources. Here you can find a link to the City’s Open Data portal and explore the data that has yet to be released on the open data format.