Recovery.gov Bid -- a Good Failure
- Written by
- Clay Johnson
- Date
- 06/25/2009 2:32 p.m.
Last week, we announced that we were going to bid on Recovery.gov. Here, 9 days later we have a few pages of information on our bid, but I don't think it makes sense to turn anything in tomorrow. So I'm declaring this experiment a failure. Most people confuse "failure" with being "wrong" but here in the Labs, we're into experimentation -- and you can't experiment or push the ball forward if you're afraid to fail.
This was a completely worthwhile experiment and we've learned a lot. I want to share some of the reasons why we failed, and what we've learned with all of you.
Why we failed
There is a language problem between us (developers) and Government.
This is the critical reason why we failed. Not only did I sign us up to write a sixty-page term paper, I signed us up to write a sixty-page term paper in a language we don't speak in a country we've never been to. Before taking this on we didn't know what CLINs were, what an ALLIANT GOVERNMENTWIDE contractor was or anything of the sort. We didn't know the difference between a Statement of Objectives, a Statement of Work and an RFP. There's a vast language disconnection between the Government (which speaks heavily in an enterprise dialect) and the open source community which speaks fluently in the dialect of simplicity, pragmatism and "straight forward-ness." This is a great chasm between us and Government that must be crossed if we want Government to start adopting new technology and letting developers have more of an impact. It is a huge problem that effects every open door into the Government.
I didn't manage the community well from the start
I didn't know I wasn't managing the community poorly, but I've learned a few lessons from this that I have to share. But what I did was put an RFP up, put a page on a wiki up, and expected stuff to happen. In hindsight, that was naive. I should have been a better organizer (and will talk more about that in a bit.)
We're not qualified to do this.
At the heart of it, rebuilding Recovery.gov is complex and the turnaround time is six weeks. The contractor with the winning bid will have to work night and day for six weeks, and while a small team could likely get it done with some heroic effort on the cheap-- Government expects two colocation facilities to be built-out in geographically separate areas for redundancy, not to mention a redesign, a fully blown content management system deployment, a reporting interface to create arbitrary reports based on the data and all kinds of other things. The Sunlight Labs team, even if all of us got together and pitched in to build this, would not be able to deliver something great. We'd be doing a disservice to America.
What we got out of it
While we did fail, we got a lot out of it and we'd do it again in a heartbeat. Here's what we got:
We got contacts at 8 of the biggest government contracting firms
6 of whom want to work with us and teach us more about the contracting business. That's a great offer. The small ones said to us things like: "this process needs to be opened up so that we can compete fairly" and the big ones said to us: "this process needs to be opened up so that we can compete on merit" and the end result for us is the same. At least some contractors are willing to talk about the process, how it works and how it can be refined. Clearly the process isn't working well for anyone. An acknowledgment of that from some big players is a big deal.
Three of them said "we're actively stealing ideas off your wiki" and I said "good."
We learned how to organize better
In the future, when we do things like this, we need to be specific and direct with requests and probably focus more on organizing than on production. This is always a tough call for us, but in this case we jumped into writing the proposal and should have run around organizing a solution from ISVs that could have each contributed. Or at the very least made specific requests to individuals early on. Next time, we'll do better at that.
We started learning the language
As I said before, this RFP was written in a different language. That language isn't something we spoke fluently. We just took our first 101 class and have begun to learn the language and the nuance involved and we can start sharing it with you as we learn more. Hopefully this will make us capable of being more of service to our community here and serving as a gateway for good developers to start creating change from the inside.
We started shining the light on Contracting
I think we all knew light needed to be shined here and there's much more that needs to be done. I think we're going to see a more nuanced problem than "government contractors are evil" and "a culture of cronyism" that's been the typical public perception, and instead start figuring out what's really going on here. So far it looks like there's significant issues with regulation, procurement, language and talent. But, you can bet we'll be paying a lot more attention to the contracting business as we move forward, and likely FOIA all the bids that did make it in, so we can check them out.
The eventual winner may give us a seat at the table
We've talked with several of the contractors planning on bidding tomorrow. And while we've made no arrangements with any, some have joined us in our chat room on IRC, and many have added in their proposal that they will consult with us if they're selected. I think that's a huge victory for us.
We shed light on the Recovery Accountability and Transparency Board
We're generally uncomfortable with the recent decisions by the RATB. From their decision to fast-track this effort into a six-week process, to having text that has been copied and pasted from one of the Alliant Vendor's website, to not requiring bulk data access we are starting to question whether the RATB is taking Recovery.gov in the right direction. This is more of the same-old-stuff that really needs to change for Recovery.gov to be meaningful to its citizens.
We are learning how to fail
Like I said earlier, we're proud to fail. Too often, especially in this community, people wring their hands and try and figure out what's possible before trying something. While some may use the cliche of "don't let perfect be the enemy of good, " I think it may be more appropriate to say: "don't let what's possible get in the way of doing something."
In politics, people argue both sides of just about anything. And while this is a great and wonderful skill, this kind of work trickles out into a community that can also see a side to everything. My take: pick a side, take a first step, see what happens. That's why we're called Sunlight Labs, not Sunlight Technology Services Corporation.
What's Next
So, what's next? For one, I want to get some of these Alliant contractors who by the way, have been really friendly thus far, together to chat about transparency from their side. To see how they can educate us about this business of contracting, and maybe teach us how to speak the language a bit better.
Secondly, Sunlight may want to think about the most ethical and transparent way to engage in this type of business going forward. While it would be neat to serve as a disruptive market force in government contracting, we're also a government watchdog group. That kind of conflict could hurt our ability to serve the public. But there's got to be a way for us to both be of service to you all and continue the great work we're doing on the outside. We need to figure that out.
So to all of you who helped out-- thanks for participating. This was a great experiment. Don't be disappointed because we're not going to get the bid: we did a great job here learning more about this field and it is only going to make us more effective down the road. Thank you so much for your participation in this project -- your advice and suggestions meant a lot. In time, I think your contributions to this project will pay off in ways we won't even expect.
Discussion
What are Your Thoughts?
Have thoughts that might fuel this discussion further, post them below. (Markdown syntax is supported in comments.)
Fascinating. This sounds like one of the most successful failures ever. Well done!
Nonsense. You did not fail. You disproved a hypothesis.
That's how progress is made in science, and it can be how progress is made in political science.
Clay - thanks a lot for posting this - you've encountered one of the barriers to entry into to government procurement market...but the great thing, looking forward, is the new culture of transparency and fairness that has come to the gov't procurement space with the new Obama administration.
I use many of the Sunlight Foundation's open government data services and have watched as the government as borrowed ideas and put them into newly created .gov sites...this means that these products are certainly in demand, and more importantly, setting standards. Learning from your peers, bidding and losing, attending pre-bid conferences...are all ways to get into the potentially lucrative market that is to be had in government procurement.
Something else that needs to be solved for Sunlight's vision to succeed is to explicitly define how the IP of contributors will be protected.
Open source does not mean open pricing, processes, procedures, and methodologies. Even successful open source companies like Redhat proactively protect their IP with NDA's and other internal policies and procedures.
If openness and transparency mean the destruction of competitiveness, and the ability for vendors and consultants to maintain their livelihood, it will not succeed.
having done Federal, state and local gov contracts for xx years (too long), many times the COTR wants to see if you'll hang around for more than one bid process.
Continued good luck
Congratulations!
"Failure is success if we learn from it." - Malcolm S. Forbes
"It is no failure to fall short of all you might dream.
The failure is to fall short of dreaming all that you might realize. "
-Dee Hock, Founder, Visa
Great post Clay. The best way to succeed and keep innovating is to not only accept when you have "failed," but also to embrace it, identify takeaways, and keep moving.
As a regulated industry, government contracting is incredibly complicated, which you just found out! It's not obfuscation, it's legal/regulatory reality.
You were "doomed from the start" because successful bidders know about RFPs before they come out [they are funded by the Federal budget so the funds are trackable] and put together the best team. I came in #2 once because someone else got the best subcontractor before we could.
Contracts help government agencies achieve their mission. It's not about the contractor and what they want to do, its about what Congress has mandated the agency do and what the best way to accomplish that is. So getting to know the agency is essential.
You are absolutely right that your best outcome was the relationship with larger contractors. Get to know them all and learn anything they will show you.
The other thing: If you make a mistake on a Federal contract, you may be committing a crime, because this is an "industry" formed by Federal law. You really need to get the lay of the land. Agencies are looking to find contractors who know the business so they won't have a contract blow up on them by a rookie mistake.
Most economic development centers in the DC area have seminars on Federal contracting. I'll be offering some in conjunction with Design to Delivery Inc., (@ddkurcfeld) and the American Small Business Committee (@asbcguy) is a great resource too.
Keep at it!
Clay - submit your version of a proposal anyway. It doesn't matter that you can't meet all the criteria or that you won't submit a proposal that looks like what everyone else is submitting. Your response still has value.
Yes, its important for you and the team from Sunlight to learn how the government wants to manage the proposal process. Its also important for the government to hear from you. Posting it here is good, but delivering it and providing folks on the inside with an opportunity to read what you have written is needed as well. For all you know, the arguments you make or suggestions you offer will have an impact on the rebuilding process even if you aren't involved. And its always possible that what you submit will have an impact on how future RFPs are written, making it possible for your perspectives and experiences to be more in line with what the government needs on future projects.
Definitely not a failure. But the experiment isn't done yet. Keep pushing. Submit the proposal.
Now that you have opened the door; keep it open. Great team and Congratulations on rallying us to your call.
Be proud. You taught me.
Don't feel bad about not knowing the language etc. Even the big firms that regularly bid for government jobs often don't trust themselves to get through this process. In fact, on nearly every bid I've participated in (above a threshold size) we've brought in a specialized outside firm to manage the proposal process.
The FARS (federal acquisition regs) have great intentions behind them (keep people from stealing from the gov and keep bids fair) but they tend to make the process arcane, complex, and unintentionally opaque because of language that is anything but straight forward.
There would probably be great value in starting off with a wiki-based primer on the FARS and the processes that they impose just so people that want to compete on this kind of work have a bit more context.
One last point... your point about the hosting being tied to the development in the same contract is part of the problem I think. It simplifies the acquisition in many ways by allowing the government to deal with a single RFP, but it tends to restrict the playing field to larger integrators that can either deal with the entire breadth of the purchase or are in a position to rapidly put together a team of participants.
When the government figures out how to do private clouds or some other platform play along those lines, then they can let RFP's just for the development on a specified platform with the expectation that operations are already in place for it.
Fantastic post. Many thanks for sharing!
BRILLIANT!
Honestly, people who follow my work and tirades I make against Government 2.0 think that I'm anti-government innovation. Entirely NOT the case.
There is a certain amount of sobriety that has to be considered when trying to bring technology and innovation to the government. The established government has policies, procedures and, as you note, language that has been embedded and engrained for decades and we aren't simply going to change that by showing up on the scene and screaming "Wiki, blogs and Twitter!"
I commend Sunlight Labs, and you specifically, Clay, for pushing this forward and not being afraid to fail. One of these days, we'll be in a position to win based on merit and fairness.
Keep it going.
This may seem blunt, but are you essentially saying you didn't win a contract you didn't apply for?
This isn't about a broken process, it's about not having enough resources to get the job done according to the terms of the contract. My advice is for you guys to retain your outsider status and your perception of moral superiority--that way you won't have to get your hands dirty (and you will if you start working in their world).
Excellent post. Especially glad to hear about the contracting firms talking to you.
Great post, Clay. I hope Sunlight continues down this course in providing public insight into what is a very closed process. To the extent this opens up options for greater participation, this is a terrific and very needed service.
I think that looking into this proposal was a great idea. it gave me an entrance into thinking about gov contracts not as evil - but as ways to get things done - and how hard the work actually is.
I really liked the line about other contractors stealing ideas off the wiki. If in the future contractors will use you (and the community) as a resource either by calling you (opaque) or using the wiki (transparent) - we can get great ideas out there, part of contracts, and implemented. And if these discussions are public, no firm will have an advantage - only the public wins.
In college I took a class from a giant in my niche of the news business, and he said "fail early, fail often. Then later you'll be ready to kick ass." A lot of people look at me funny when I mention that to them. But I'm pretty certain you get it.
Congratulations! You are now part of the problem. The Federal Gov't has us jumping through hoops to get contracts and grants. I would suggest that the best way to get a contract or grant is to buddy up to your representative. I would like to get a grant to teach yoga to hurricane victims. But it looks like Congressman Grayson-D/FL district 8 has already secured that grant for some other donor. When are we going to put a stop to this madness?
Thanks for the post Clay. One thought lingers in me...a lot of the language gap you mention seems to be about means (contracting, processes, infinite procedural acronyms, the 'social network' of government contracting..). I believe one of the potential benefits of the cultural exchange you hint at, could be an improvement in the language to support discussions about ends. Concepts like transparency, actionable information, participation tend to be fuzzy terms compared to "subcontracting" or "RFP". Keep up the great work at the Lab!
Clay, I believe we got more out of this by not winning this contract. The information you have shared during this process is priceless!
The effort here reminds me of a quote from Thomas Edison.
"I have not failed. I've just found 10,000 ways that won't work." ~Thomas A. Edison
Sometimes we start an effort with a certain goal in mind, but get greater rewards from the unexpected.
Thanks for sharing, Daniel Hudson
For argument's sake, assuming the cost of the hardware/software infrastructure is $3M, and support costs of $1M/year over the life of the five year contract, this equates to $10M for development and implementation services. $10M over six weeks is roughly $1.6M/week. At $200.00/hour this would equate to 8,000 person hours. At 40 hours/week, this would mean a team of 200 FTE.
I don't care what language you're speaking, federal government-ese, open source-ese, or private sector, this simply does not pass the sniff (reasonability) test.
Congratulations on your efforts. I'd like to see real sunlight, however, and view the RFP, the evaluation criteria, the responses, and the award.
Clay, this was a very thoughtful and helpful post in many ways. I admire the effort for the point of your emphasis which is that you learned a great deal which will help Sunlight be more effective as well. Very constructive. And I would bet that this will influence the drive to make acquisition much simpler.
Many firms are not on "the list" or know the vagaries of the process, yet have compelling skills and perspectives that solve complex problems in simple ways. i.e., fresh thinking. They do need to be invited in.