By Michael McEnaney
When back-end glitches and architectural issues conspire to bring down a Web site at its launch, fingers begin pointing and the blame game is on. When it snarls the planned roll out of President Barack Obama’s healthcare.gov Web site, it is front-page news across the country and the fallout is catastrophic.
While the problems with the healthcare.gov launch were primarily technical, the political implications are much broader and could ultimately risk the implementation and success of the overall law. Only time will portend the future on that front.
Problems with Launch
Perhaps becoming a victim of its own ambition, the Obama administration’s Affordable Healthcare Act Web site experienced what most people in the know referred to as a “train wreck launch” featuring slow, buggy, and frequently unavailable page loads that continually frustrated the millions that attempted to log on.
While most experts we spoke with agreed these issues were equal parts technical and bureaucratic, the IT contractor that orchestrated most of the healthcare.gov Web site, CGI Federal, chose not to comment despite our repeated attempts for an interview.
Thus, we turned our attention to Web site developers and computer programmers to get to the bottom of what they feel went wrong with the Web site’s launch and what they thought could have been done differently to prevent the many problems it experienced.
Referred to as one of the most complex pieces of software ever created for the federal government, the site was designed to communicate in real time with at least 112 different computer systems across the country. In the first 10 days, it received 14.6 million unique visits.
Peter Gallagher, group VP, Unisys Federal Systems, a global information technology company that develops advanced information technology solutions designed for government deployments, was quick to point out what he feels was perhaps the most glaring issue with the project, one that had nothing to do with technical glitches. “Having the government act as the project’s systems integrator was perhaps the first problem,” he says. “This was a mistake from the start.”
Gallagher is referring to reports that the site experienced problems during the testing phase just days prior to the October 1, 2013 launch date and simply was not ready for takeoff. Perhaps a third party systems integrator would have been more likely to make the decision to delay the launch, but the administration’s blind ambition may have ultimately forced a decision to stay on schedule.
“The problems that were occurring prior to launch should have been communicated as more dire than they were,” adds Long Island, NY-based computer programmer Mike Galke, who runs the business he founded, Spectrum Computer Consulting. “The site wasn’t ready for prime time but I think the pressures of who the client was played a role in what may have been some finger crossing and ultimately a feeling of having to simply push forward.”
Galke adds that many of those “problems” were architectural issues and with a site that has as many layers as healthcare.gov does, those can become significant problems.
“When you’re thinking about the multiple interactions that will be part of a site like this you’re looking at what is called an asynchronous process,” continues Galke. “This is to say distributed systems have components that are de-coupled, so if one fails, it doesn’t necessarily bring the rest of the site down with it.”
Galke says this was not the case with healthcare.gov, as it was designed as a more synchronous process. He adds that it, “really needed a system architecture that was built to be resilient for such failures, that enables the host to do things like quickly store the contents of a form submission and the like.”
While he complimented the site’s front-end development, saying he felt it was designed to work well across multiple browsers and devices, he adds that it ultimately means little when visitors experience the kinds of problems they did on the back end.
While Galke has developed a number of Web sites and worked on some unique Web architecture, he adds that the important thing to point out with a marketplace site such as healthcare.gov is, “there is no one, single Obamacare site, and healthcare.gov was just the portal and wasn’t the problem. It was the fact forms were being submitted to back-end servers that became the issue.”
Gallagher adds that a general lack of communication between CGI Federal and the government, that he termed “shocking,” failed to properly address, “a list of bugs that existed that the developers had to know about during the testing phase that apparently no one else knew about.”
Though, as stated earlier, no one from CGI Federal commented directly to us for this story, the company has publicly stated that it was not their decision to go live with healthcare.gov, saying the final decision was made by the Centers for Medicare and Medicaid Services (CMS), an agency within the Department of Health and Human Services (HHS). CGI Federal has also publicly stated that full tests of the site, which should have been carried out months in advance, actually began just two weeks before its scheduled rollout.
Galke feels communication breakdowns of that nature on a project such as this, “is a two-way street” and that, “both the client and the contractor have to be honest, open, and realistic about where the project stands.”
Stephan Hilbelink, a 17-year veteran of Web site design and development as well as current senior Web site designer and developer at the Family Research Council, agrees on how a lack of clear communication may have played a major role, “the contractor’s two goals are first to build an excellent Web site with the requirements given by the client and then to communicate with the client about everything—progress, pitfalls, and changes that arise. The central focus of the contractor is getting the client involved, educating them about the product and tailoring the site into a product specifically for the client. Contractors need to be open and honest about what they can do,” says Hilbelink.
What’s Next for Healthcare.gov
The embarrassment for the Obama administration runs deep with this project as the jokes over the botched launch ran from the recent headline The Onion posted stating: The New, Improved Obamacare Program Released on 35 Floppy Disks, to a suggestion by Late Night talk show host David Letterman to, “replace glitchy healthcare.gov Web site with a convenient in-person enrollment kiosk located in Washington, D.C.” Ouch.
So, where does the site currently stand? While the consumer-facing part of the Web site has improved, the aforementioned back-end issues are still being worked out. New software responsible for enabling the federal government to verify enrollment data with issuers and to pay plans billions of dollars in federal subsidies on behalf of lower income enrollees is currently being tested.
HHS claims there have been hundreds of software fixes since launch and that site capacity is now stable and at its originally intended level. The Obama administration appointed Quality Software Services, Inc. to coordinate the work of fixing the Web site problems. According to a recent report, conducted by HHS on the site, the fixes include a dramatic drop in error rates—per page system timeouts or failures have been driven down from over six percent in the weeks following the launch to well under one percent currently.
According to HHS, uptime is now consistently surpassing 90 percent—up from a low of 43 percent in the weeks after launch. The report notes that the site supports over 800,000 consumer visits per day and 50,000 concurrent users. It also has built-in 24/7 monitoring and an operations center and team in place to ensure optimal system performance.
Political Pressure
In addition to site fixes, several recent administration changes surrounding healthcare.gov are noteworthy.
The recent hiring of Kurt DelBene, a retired Microsoft executive, is being lauded as a key move that will help accelerate the site’s repair process. According to a recent blog post by Kathleen Sebelius, secretary, U.S., HHS, DelBene’s immediate job responsibilities are to focus on increasing system stability, redundancy, and capacity, and build on improvements to the user interface, while continuing to prioritize security and privacy issues.
HHS also recently announced the government will go with a new Web hosting contractor, Hewlett-Packard, for healthcare.gov moving forward. The current contract with Verizon Communications Inc.’s Terremark subsidiary ends in March of this year.
Additional fallout from the site’s failed launch also includes the recent Reuters report of the retirement announcement of Michelle Snyder, CEO, CMS, who oversaw the building of healthcare.gov. And so the game of musical chairs with regard to employees once associated versus now associated with the site continues.
The various upgrades, fixes, and patches occurring since the launch now have the site operating markedly better but no one is declaring that all is well in Obamacare land. Look no further than the president’s record low job approval rating of around 40 percent, according to various polling data gathered by RealClearPolitics.com, since the healthcare.gov launch.
So then, there was certainly plenty of blame to go around but one common thread was apparent within all the conversations we had for this piece—the Obama administration, and ultimately the president himself, must shoulder the bulk of the blame as the ill-fated decision to rush this project to fruition may have been the biggest factor in bringing it to its knees.
“Both poor project management and unclear and unrealistic expectations and requirements are to blame for the faulty launch of healthcare.gov,” concludes Hilbelink. “A Web site of this magnitude should have had requirements outlined by the client before sitting down for the initial development meeting. When a client knows exactly what they want, the development process is optimized. Sadly, CGI Group received the contract in 2011, yet final requirements were not given until March of 2013. This caused major setbacks in programming and design, resulting in extra labor costs.”
“Perhaps now we can finally begin a debate on the law’s merits and not continue quabbling over a dysfunctional Web site,” adds Galke. “From a purely technical perspective, the turnaround, really accomplished over about a five-week period, was remarkable. Most of what went wrong could have been avoided but as long as they continue to quickly deal with unexpected transitional issues the site should be fine moving forward.” SW
Feb2014, Software Magazine
