Fixing California’s Government Technology Failures

Overview

California is home to many of the greatest technology minds and businesses in the world. Yet, despite the seat of government in Sacramento being located a mere couple of hours from Silicon Valley, the state seems incapable of maintaining up-to-date technology or making new information technology (IT) programs and updates work properly—much less completing these projects on time and under budget. It is the juxtaposition that has confounded public officials and taxpayers alike for decades now.

From the Employment Development Department’s repeated failures to upgrade its antiquated IT systems, leading to added stress and headaches for millions of Californians who filed unemployment claims during the Great Recession and the current coronavirus pandemic, to the numerous IT project failures at the Department of Motor Vehicles (DMV) going back to the 1980s, to the failed Statewide Automated Child Support System in the 1990s that cost the state $1 billion in fines to the federal government due to delays (on top of the $111 million the state had already spent), to the BreEZe occupational licensing system that more than tripled in cost from $28 million to $96 million and was only adopted by a small fraction of the intended boards and commissions before the project was terminated in 2015, California state government has repeatedly doomed itself to greater inefficiency and years of project delays while saddling taxpayers with enormous costs for technology that does not work—and sometimes is so woefully inept that it must be scrapped altogether (see Appendix A for even more examples).

For these continual failings, the Independent Institute awards its tenth California Golden Fleece® Award—a dishonor given quarterly to California state or local agencies or government projects that swindle taxpayers or break the public trust—to the agencies responsible for overseeing and administering the state’s various failed IT projects. These agencies include the California Department of Technology, Department of General Services, Department of Finance, State Controller’s Office, State Treasurer’s Office, Judicial Council of California, Administrative Office of the Courts, Department of Motor Vehicles, and Department of Consumer Affairs.

Background

Watchdog agencies have identified the state’s IT project costs and management as a cause for concern for many years. “[B]etween 1994 and 2013, the state terminated or suspended seven IT projects after spending almost $1 billion,” the State Auditor’s Office reported in 2015.

As a result, the state auditor has repeatedly designated California’s IT projects as one of the “high-risk” issues facing the state. “The high costs of certain projects and the failure of others continues to make the state’s oversight of information technology projects an area of high risk,” it concluded in 2013.

Ransomware attacks, malware, and other cybersecurity incidents pose additional risks, such as data loss, systems failure (including, for example, the shutdown of emergency call centers or the inability to pay taxes and fines), the theft of residents’ personal information, and expensive and time-consuming remediation efforts to remove malicious code, recover data, and restore functionality. Cyberattacks targeted at federal, state, and local governments, as well as universities, have been on the rise in recent years. Thus, it is crucial to invest in data security and keep IT systems updated in order to minimize the state’s vulnerability.

Public-Sector vs. Private-Sector IT Management

Despite the state government’s IT project troubles, it would be unreasonable to expect all projects to go perfectly smoothly. With complex data projects serving hundreds of thousands of state workers or millions of citizens, some trial and error—and, yes, even failure—is to be expected. The private sector has some big advantages over the public sector, however. First, private firms are much more constrained by cost considerations. They cannot simply rely on the faceless taxpayer or the strength of their lobbying to shoulder other government agencies out of the way for larger budgets. As a result, failing projects are more likely to be jettisoned sooner, and new solutions sought.

Moreover, private companies’ and employees’ incentives to succeed are much stronger. A successful big data project may make a huge difference in a company’s bottom line—or even the difference between the business surviving or failing—and those responsible for the projects that fail are much more likely to find themselves out of a job. In the public sector, by contrast, financial decisions are based on politics instead of economic realities, taxes can always be raised, money can be shuffled from other places, the money is not yours but someone else’s, and employees enjoy such job protections that they rarely fear losing their jobs for poor performance.

Case Studies

In telling the story of California’s string of IT project failures, there is no small list of projects from which to choose. In addition to the aforementioned examples, consider the following case studies.

Department of Finance – FI$Cal

The Financial Information System for California, commonly known as FI$Cal, is the largest and most expensive IT project in the state’s history. The project, which has been ongoing for 15 years, seeks to integrate the state’s accounting, budgeting, cash management, and procurement processes.

Drastic changes in the project’s scope, goals, and deliverables have contributed to significant delays and cost adjustments. FI$Cal is currently estimated at $1.1 billion, and its most recent scheduled completion date was July 2020. The project was not fully implemented, however, and is currently in the process of submitting an updated report to address the remaining functions. Moreover, a number of agencies using FI$Cal continue to use their legacy financial IT systems, due either to reliability issues or the fact that their legacy systems offer needed functions that FI$Cal does not. This means that additional staff is required to operate two separate IT systems, leading to waste and inefficiency.

When it became clear that FI$Cal was going to experience more cost overruns and delays, the state and its contractors came up with a creative, albeit deceptive, solution: simply declare the project “completed” before it is actually finished, and push additional functions and man-hours after this deadline. As the state’s nonpartisan Legislative Analyst’s Office explained,

The project would end on this date even though some planned system functions will not be implemented, some departments will not be using the system, and some legacy financial IT systems will not be decommissioned. As a consequence, the project will not deliver what the Legislature expected when it authorized FI$Cal, and the Legislature will receive budget requests in future fiscal years to complete unfinished work that was originally within the project scope.

The inability to implement FI$Cal could lead to other substantial costs to the state as well. The state auditor has warned that the current FI$Cal plan could leave the state controller unable to produce accurate monthly financial reports for state agencies or the state’s all-important comprehensive annual financial report (CAFR), which is used to determine the state’s credit risk and credit rating. Should creditors lose confidence in the timeliness or accuracy of the state’s CAFRs, California’s credit rating would likely fall and its borrowing costs would increase, as creditors would demand higher yields on the bonds sold by the state to finance its spending.

State Controller’s Office – 21st Century Project / MyCalPays

The State Controller’s Office has been unsuccessfully trying to upgrade its payroll and human resources IT system for a whopping 21 years. The project’s 21st Century Project name sounded novel back in 1999, but it has been flailing so long that the name became outdated and was, thus, rechristened the MyCalPays system. It would handle payroll for the state’s 250,000 civil service employees, as well as University of California workers who are paid through the system, and would be the largest payroll modernization effort in the nation.

The full project was approved at a cost of $132 million in 2004, but multiple delays pushed the cost up to $373 million and the state controller ended up terminating it in 2009. A second attempt was made beginning the following year, but it did not take long for problems to surface.

A pilot test of 1,300 state controller employees in June 2012 turned out to be a disaster. As a September 2013 state auditor’s report explained, “eight months of payroll testing of the new system failed to produce one pay cycle without material errors . . . . As a result, state employees were paid too much, paid too little, or they and their family members were denied medical coverage to which they were entitled.”

This led to the project being canceled for a second time in February 2013. Thus, the state auditor noted in a March 2015 report that “the state continues to use its aging legacy payroll system after spending $262 million to unsuccessfully develop the project.” A 2016 legal settlement did result in contractor SAP refunding $59 million of its $90 million contract, at least.

Despite all its troubles, current California state controller Betty Yee has not given up on MyCalPays.

“It’s a priority of my second term,” Yee told the Sacramento Bee in February 2019. “I want the damn thing started.”

It is probably not going to happen before her successor takes office in 2023, though, she added.

Administrative Office of the Courts – California Court Case Management System

In an attempt to replace some 70 disparate IT systems for the state’s 58 county courts, many of which are unable to interface with one another, the state embarked upon the California Court Case Management System (CCMS). The statewide system was to offer coordinated scheduling, interagency interfacing, and statewide reporting; provide judges and members of law enforcement access to real-time court information; and allow citizens to electronically file documents, access information, and make payments online. The CCMS languished for a decade before the state was finally forced to pull the plug in 2012 after only partial implementation in a small number of superior courts—but not before spending more than $500 million on the failed project.

The cost of the CCMS was initially estimated at $260 million in 2004, but by 2010 the timeline had been pushed back seven years, to fiscal year 2015-16, and expected costs had ballooned to nearly $1.9 billion—and this did not even include an additional $1 billion estimate to install the system nor another $391 million to manage it over the ensuing four years.

One of the major factors behind the substantial price tag increase was the large number of contract amendments, or change orders, demanded by the Administrative Office of the Courts (AOC). A contract with Deloitte Consulting LLP to assist in developing the system, for example, saw 102 change orders over a seven-year period, which raised the contract’s cost from $33 million to $310 million. Moreover, the AOC failed to structure the contract to ensure that superior courts could test the civil system in a live operational environment before the warranty expired.

As a February 2011 state auditor’s report noted, the CCMS project suffered from numerous deficiencies, including a lack of cost-benefit analysis or business case justification for the project, inadequate planning and oversight, a lack of transparency regarding cost estimates, a dearth of analysis and documentation to support key project decisions, poor contract structure, operational issues that prompted some courts to refuse to adopt the system, and a failure to obtain the funding needed for full deployment of the system. In response to the serious issues raised by the state auditor, the Judicial Council of California—the policy-making body for the state’s courts—voted to kill the CCMS project in March 2012.

Recommendations

California’s government IT projects have suffered deficiencies in a number of different areas, which have led to wildly inflated costs, years of project delays, hundreds of millions of dollars wasted on systems that had to be abandoned, and continued reliance on outdated and unstable legacy systems. This report makes 20 recommendations to improve IT project planning, contracting, implementation and oversight, accountability, and data privacy. They include:

• Utilizing thorough up-front planning, including the use of a cost-benefit analysis and a business case justification, to make clear what is expected of the project and why, and to minimize costly changes and delays later on
• More consistently competitively bidding out contracts (which is generally required, with certain exceptions, but not always followed in practice) and utilizing performance-based contracts to financially reward contractors for meeting benchmarks and deadlines and punish them for poor performance
• Structuring contracts to transfer risk from the state—and taxpayers—to the contractor, such as ensuring that warranties do not expire before IT systems can be tested and fixed
• Streamlining bureaucracies, rather than simply adding new agencies on top of old ones, such as when Gov. Gavin Newsom’s new Office of Digital Innovation largely duplicated the Schwarzenegger-era Office of Technology Services
• Improving in-house project oversight and contracting expertise or outsourcing these functions
• Investing in data security to keep IT systems up-to-date and better defend against cyberattacks
• Improving leadership and executive ownership of projects, and holding staff accountable for poor performance
• Better protecting data and ending the practice of selling Californians’ personal information, as the DMV does to the tune of more than $50 million per year.