Sunlight Labs

Recently and continuously, the White House has been releasing the "White House Visitor Logs," showing America who is coming in to meet with the President and his staff. At the same time, the Center for Responsive Politics releases cleaned up data on lobbyist filings. We thought it'd be interesting to find the intersect between the names in both sets of data.

After the jump, you'll find our results along relevant information from both sets of data. Now-- this is important: just because the names match doesn't mean they're the same person. Because the White House doesn't release any other form of identity information besides the name, we're unable to tell whether or not the name in one dataset actually refers to the same person in the other. John Adams in one dataset may be a different John Adams in another.

Why your non-profit stands to benefit from Open Data

Often times at Sunlight the non-profit community looks at us strangely. Here in Washington, DC we've probably made more investments in technology than any other non-profit or advocacy organization I've run across. Certainly our mission is focused around the use of technology, so that makes a lot of sense-- we're focused on getting data out of government, doing interesting things with it, and letting you see what happens in Washington better. That means technology investment.

But one question I struggle with is: why doesn't every non-profit advocate for open data from the Government? Don't ALL of them stand to benefit?

Following up on my hypothetical post on what would happen if Government had done the same thing that Google did with Google Buzz, I'd like to imagine something different: what if something like Google Buzz happened to government? What if, out of nowhere, the Executive Branch of government started exposing the most frequent contacts of each Senate Confirmed appointee based on their email inboxes? What would happen if we could, for instance, pull up Rahm Emmanuel's "Buzz" profile and see who he followed and who was following him, based not on his preferences, but based on the frequency of email contacts alone?

Despite the recent explosion of web based cartography tools, making effective maps for data visualization remains a challenge. While tools like Google Maps are great for helping navigate the world they fail terribly at data presentation tasks. Many features like roads and cities only get in the way of telling compelling stories with data. In fact, even the distance between places can be a distraction – who cares how far away Alaska is when the goal is to make a simple comparison between US states?

To overcome some of the limitations with existing mapping tools, Sunlight Lab is releasing ClearMaps, an ActionScript framework for interactive cartographic visualization. In addition to giving designers and developers total control over presentation the project aims to address some of the common technical challenges faced when building interactive, data driven maps for the web. ClearMaps is designed as a lightweight, flexible set of tools for building complex data visualizations. It is a framework not a plug-and-play component (though it could be a starting point for those wishing to make reusable tools).

A couple of developers from the Sunlight Labs community, including one of our Great American Hackathon organizers Jessy Cowan-Sharp, managed to put together something remarkable: OpenGovTracker (source here). The site lets you see where the ideas are coming in across the various agencies from a single dashboard.

What's the synopsis? According to this it's that the American People don't have a lot of ideas. Well-- a lot of agencies are pretty low on ideas. Only 611 ideas have been proposed. Treasury only has a dozen ideas? The best the American people can do is give Social Security 10 new ideas?.

As our the Sunlight Foundation's Policy director stated late last week: now is the time. Request a dataset or submit an idea to government. Here's how.

Three days ago Google released Google Buzz-- a product that got a lot of folks excited-- especially here in the Labs. But fairly quickly people understood something-- Google took a step across an invisible privacy fence. A lot of people are critical or downright ticked off. Google had, in fact, exposed who we communicate with the most to the world.

If the Federal Government released a product similar to Google Buzz, what would have happened?

"High Value Datasets" is a bunk term.

When the feds introduced the term High Value Data, my immediate response here was "what the heck is 'High Value Data'?!" We quickly extracted the definition from the Open Government Directive and here it is:

"High-value information is information that can be used to increase agency accountability and responsiveness; improve public knowledge of the agency and its operations; further the core mission of the agency; create economic opportunity; or respond to need and demand as identified through public consultation."

Now we've had a chance to go through and take a look at some of the datasets. Our http://reporting.sunlightfoundation.com is having a field day analyzing the data, pointing out flaws in the data and generally doing a great job of figuring out what's actually new in the datasets.

The Open Government Directive has pushed agencies forward to start soliciting feedback from the public. We threw together this quick screencast to show you just how to do it. Check it out after the jump.