The Department of Health (Health) consistently accounts for a major proportion of total government expenditure on Information and Communication Technology (ICT) projects every year. Health Information Network (HIN) is responsible for centralised ICT management in Health. HIN is estimated to have spent around $95 million in the past three years on ICT projects related to the commissioning of the Fiona Stanley Hospital, in addition to $87 million on unrelated ICT projects.
As highlighted in past reports, agencies often have difficulty successfully delivering ICT projects, particularly when they involve significant changes and when multiple stakeholders and suppliers are involved.
The commissioning of the Fiona Stanley Hospital (FSH) is the largest project ever undertaken by Health. A small though important part of the commissioning of FSH has been the Identity Access Management (IAM) project which is one of a number of concurrent and interrelated projects planned to deliver a near paperless environment at the hospital.
The IAM was intended to provide users with:
- anywhere, anytime access to the IT systems that the individual is authorised to use
- admittance to those hospital buildings that the individual is authorised to access.
The first phase of the project commenced in 2011. This involved defining all the roles at the Fiona Stanley Hospital that interact with an ICT system, including where people would have to access a building. The clinical and administrative roles would be defined by Health and the support roles by the provider of support services to the hospital.
Health stopped development on the IAM project in October 2013 as they could not see any deliverables from the project. At the time of the decision some $6 million of the $9.2 million project budget had been spent.
In late 2013 the Acting Director General of Health informed us of the difficulties Health was experiencing with the IAM project and requested an audit. Given its potential to impact on the successful opening of Fiona Stanley Hospital, we agreed to assess whether there was adequate scoping, governance and support given to the project.
The IAM project will not be complete when Fiona Stanley Hospital opens later this year. Granting of access to the key IT applications and physical access to the hospital buildings will not be automated.
The reasons commonly found for why ICT projects run significantly over budget and over time were evident in the Identity Access Management Project. Project planning was deficient and governance and oversight including monitoring of progress was inadequate. The business mapping of staff roles to their required ICT access lagged behind the technical development of the solution. Critical technical dependencies and difficulties that threatened the feasibility of the project were therefore not identified in a timely manner. This issue, although raised in successive project status reports, was not elevated to the appropriate levels of management to be actioned.
- Health did not prepare a business case to support their decision to invest funds in the IAM project when it started in 2008. Therefore, they were unable to demonstrate that an informed decision had been made which fully considered alternative options along with system costs, risks and benefits.
- A project initiation document produced in 2012 to re-focus the project deliverables was not supported or approved by the project executive sponsor. This indicated a lack of project ownership and engagement from a critical stakeholder, and increased the risk that the project would fail to deliver its intended objectives.
- Health’s tendering process was protracted and arguably did not comply with the State Supply Commission’s value for money principle:
- it took two years to design and issue the tender and then a further nine months to evaluate and award the tender. Protracted processes run the risk of a change in business need and that the tender no longer presents the best business solution.
- the tender assessment process was based on an assessment of hardware and proprietary software costs for a period of five years. This was despite the IAM being regarded as a long term system. The cost of additional maintenance and other support arrangements were therefore not considered in the awarding of the contract.
- the successful tender only satisfied 77 per cent of the business and technical requirements. The requirements that were not met were not evaluated for either their criticality to a successful solution or for any cost implications to Health.
- Health did not insist that the successful vendor provide a proof of concept to demonstrate that their system was viable and could deliver the required functionality. Proof of concept provides an invaluable means of demonstrating at an early stage that the proposed system is likely to meet its stated objectives.
- A suitable project governance structure was never established. Though the governance requirements were defined and documented, Health did not implement these. HIN attempted to set up project control groups and business user groups, as per the Health project governance structure at the time. However they were unable to secure appropriate business and stakeholder representation, and these governance mechanisms could not gain sufficient momentum to establish themselves adequately. As a result, there was a lack of project oversight external to HIN and some critical risks and issues from a business perspective were not appropriately escalated and managed.
- Governance, ownership and oversight of the project was undermined by a number of factors including the appointment of the Chief Information Officer (CIO) as project executive sponsor. This meant there was no independent project oversight. In addition, Health delayed engaging a permanent project manager until late 2012, over four years after the project had started and a number of other key positions, including that of CIO and program and project managers were filled by contract staff.
- The monthly reporting for the project was inadequate and did not identify and address key issues in a timely manner. Some of the more important shortcomings were:
- appropriate project milestones were not used to ensure that the project was on track
- ongoing difficulties in resolving interdependencies that would impact on project delivery were not adequately highlighted. As a result, they were not escalated for appropriate
- key project risks were not always visible within the monthly report summary. This made it difficult for key stakeholders to manage these risks appropriately.
The Department of Health should:
- enhance its internal capacity to deliver ICT projects.
- for the Identity Access Management project:
- assess whether the project is able to deliver what it had intended and whether it still matches current need. This will require a focus on user profiling, authentication requirements and rights/privileges management.
- for other ICT projects ensure:
- the project manager is hands-on at a level appropriate to understand the project issues, particularly where software is developed so that appropriate action can be taken to address risks and issues arising
- robust business cases exist for all projects
- project governance structures are put in place and actually operate as intended
- business project ownership is established early on in a project
- appropriate involvement of the business in project monitoring and oversight
- project reporting is effective at identifying and escalating risks to successful project delivery
- staff in the project team have appropriate and adequate experience with projects of similar scale and complexity. When bringing in external help, the exit strategy should be to up skill internal capacity.
The Department of Health agrees with the findings of the report and supports the recommendations. In this regard, WA Health is currently making a number of changes to improve the governance, management and delivery of ICT projects. This includes the implementation of a new ICT governance structure and the creation of a WA Health ICT Executive Board.
The new governance structure outlines the decision making framework for WA Health’s ICT investment. It clarifies the expected roles, responsibilities and accountabilities of all parties involved in the planning and delivery of ICT programs and projects.
This approach will ensure decisions about ICT are business-led and appropriately support the achievement of WA Health’s strategic and business objectives. The new arrangements will also ensure rigorous project management and reporting arrangements are in place for all ICT projects. The Executive Board, which is chaired by the Director General and includes Chief Executives of all health services, will be responsible for approving all ICT business cases.
The Department of Health is also making a number of other key changes to support the successful delivery of ICT projects, including:
- the establishment of the role of Chief Procurement Officer in WA Health in January 2014 with oversight for ensuring appropriate policies, procedures, audit and assurance processes are in place across WA Health consistent with legislative and government policy requirements
- progressing other key appointments, including an Executive Director with responsibility for Information and Communication Technology and corporate services and the Chief Information Officer, Health Information Network; and conducting a procurement compliance audit of the Health Information Network; and
- establishing procurement training and education program across WA Health.
With regards to the Identity Access Management project, the Department of Health is currently giving consideration to whether the intended benefits will meet the current or future needs of the health system and appropriate ongoing management of the project.
ICT services are a critical component in the coordination and successful delivery of health and hospital services. They help ensure patient and medical information is accurate and available to the right people when required. An IAM supports the efficiency and security of these ICT services. It provides a single centralised solution for managing users. It also manages their access to information systems and resources appropriate to their role within the hospital.
The IAM project concept came from the 2007 Health ICT strategic framework. This document outlined a vision for the future delivery of statewide computing support to the Health sector. In addition to internal identity and access management, the project was to enable Health to participate in national programs that share and process information with an increasing number of internal and external stakeholders. As increasing numbers of services are made available to a broad range of businesses and individuals, the issue of controlling access to Health’s information resources becomes paramount. The proposed delivery of this project was 2017.
The implementation of an IAM solution was a significant investment. The complexities involved made it vital from the outset that the business requirements and risks were fully understood and appropriately documented. This would include a comprehensive business case to ensure that all key stakeholders were fully informed before any commitment was given to the project funding and schedule.
The initial phase of the IAM project had four deliverables with the second, third and fourth deliverables dependent upon the successful completion of the first. The project would:
- identify what access users required to each system, based on their roles in the hospital
- determine how each of the systems grants access to users
- procure and adapt an IAM system
- automate the granting of access, based on the roles identified in the first deliverable. Access would be granted by building connectors between the IAM system and each of the 36 primary systems that people were required to access.
The entire project was given a budget of approximately $7.9 million and a completion date in August 2013. These were later revised to $9.2 million and 23 December 2014 (refer Figure 1).
In September 2010, Health released a Request to Tender for the system. There were seven tender respondents with quotes ranging from $2 599 933 to $17 271 878 and with suitability scores ranging from 58.89 per cent to 77.77 per cent. The successful respondent to the tender had an evaluated price of $4 028 411 and equal highest suitability score.
By the time the project was cancelled in late 2013 it had cost approximately $6 million. Around 25 per cent of these costs were professional services for consultants and contractors. Only one of the 36 connectors between the IAM system and Health’s other systems was fully developed at that date.
Figure 1: Project timeline summary
What Did We Do?
We liaised with agency and contractor staff to understand the project deliverables and status, and collected and analysed information regarding the project.
To assist our understanding of the project, we developed a project timeline with key milestones and established what was reported to management.
To define what had gone wrong with the IAM project and identify the risks and strategies required to help ensure the project deliverables are achieved in a timely way, we asked the following questions about the project:
- was there adequate scoping?
- was there adequate governance and oversight?
- was adequate support and maintenance in place to implement and sustain the project deliverables?
The audit was conducted in accordance with Australian Auditing and Assurance Standards.
What Did We Find?
The Identity Access Management Project was not adequately planned
In 2008, Health commenced the IAM project without a business case to support this decision or a proof of concept to demonstrate that the proposed system was viable and could deliver the required functionality. The lack of these key planning components was to prove critical in the development and delivery of an effective IAM solution.
A business case is created to help decision-makers ensure that the proposed project will have value and will deliver the required benefits. Thereafter, the business case should be reviewed regularly to ensure that:
- the investment continues to have value, importance and relevance
- the implementation will be properly managed
- the organisation has the capability to deliver the benefits
- the organisation’s resources are working on the highest value opportunities initiatives with inter-dependencies are undertaken in the optimum sequence.
The result of a review may be to terminate or amend the project. The review may conclude that the business need has changed or the project will not deliver the solution to the business need. The business case is the primary driver for the project so changes to the business case should also mean changes to the project.
One trigger for a review of the business case would be the outcome of proof of concept or a pilot project. A proof of concept was not performed. At the tender evaluation stage the requirement for a proof of concept was replaced by a requirement for a pilot project by the preferred respondent.
However, we were unable to establish if this pilot project and an evaluation of the pilot was completed. Having a successful pilot was essential given the complexity and technical nature of the IAM project, which included old applications with limited ability for adaptation and a large amount of custom and proprietary software to be connected to the IAM software.
Without a properly considered and approved business case and a working pilot or proof of concept, Health could not demonstrate that the intended benefits of the IAM project were clear and measurable or that it had made informed decisions regarding the project.
Scope changes were not properly documented or authorised
In February 2012, the project was refocused from an IAM tool for the business with a deadline of 2017 to an IAM solution for Albany Health Campus and Fiona Stanley Hospital with a deadline of August 20131. This solution would then progressively be applied across the business. A Project Initiation Document (PID), which included a business case, was developed to support this change in the project. However, the project’s executive sponsor who was also the CIO did not support or approve the PID.
We also noted that a range of other scope or budget changes were made without the authority and/or consultation that we expected:
- The physical card access system which was allocated to a contractor was withdrawn by a project manager without consulting the stakeholders. We found no evidence that this change to the scope had appropriate management approval or oversight. There was also no assessment of how this would impact the delivery of the project. The tailoring of the physical card access readers at Fiona Stanley Hospital will now only be completed after the hospital opens.
- Project budgets were amended without proper process or approval. A funding request for the project was lodged on 6 September 2011, but was not initially approved as the requested funds exceeded the available budget. Subsequently on 21 September 2011, a revised funding request was lodged. On 28 September a change request was made that brought the total amount requested up to what was initially sought on 6 September. The change request was to approve the appointment of 3.5 FTE contract staff, including the project manager, at a cost of $801 000 over a period of nine months. This equates to about $1270 per person per day. Although none of the requests were signed the total was reflected in the project budget.
In the absence of properly documented and approved scope changes, Health is unable to determine whether they received value for money and whether the contracted service providers delivered what was intended.
The tender process was deficient
Health took almost three years to award the tender
The Request to Tender (Provision of an Enterprise Identity and Access Management System Inclusive of Installation and Development Services, Training, and Ongoing Maintenance and Support) which includes the main software contract for the IAM project was placed in September 2010. This was more than two years after the project was initiated in June 2008.
Health then took a further nine months, until June 2011, to evaluate and award the tender.
At the point of notifying the successful tenderer, the project had incurred costs of around a million dollars.
We were unable to satisfactorily establish why this process was so protracted. The risk from having such a protracted process, particularly given that there was no review of the business case in that time, is that the tender specifications become outdated and no longer present the best business solution. Also there is an increased risk that the tender submissions will not be relevant and up to date when the tender is eventually awarded.
The total cost was not considered when awarding the tender
Health did not consider the total cost of ownership when it awarded the tender. The tender assessment process was based on an assessment of hardware and proprietary software costs for a period of five years. This was despite the IAM being regarded as a long term system. The cost of additional licencing, software renewal and support arrangements were therefore not considered in the awarding of the contract.
ICT tenders are often awarded on a ‘best fit’ principle rather than the full cost of providing the business requirements. The full cost includes hardware and software for the life cycle of the project, costs associated with proprietary software and costs to address any shortfalls between the business need and the prospective tenderer.
Health selected the supplier based on a tender response that addressed only 77 per cent of their business and technical requirements. They did not determine how critical the shortfalls were to the overall project or identify how and at what cost they should be addressed. Examples of the requirements that were not fully addressed included:
- those that would incur additional licencing costs as the software would have to be duplicated at a number of sites
- the ability to provide a ‘virtual’ directory. The tenderer proposed using ‘views’ instead
- managing passwords in the event of a network failure
- caching of entitlement policies
- the need for a test environment
- support for custom approval screens that will survive a product upgrade
- support for certain web and mobile browsers
- a software development kit.
Health’s stated vision for the IAM system was that it would be in long term use. Despite this, the price evaluation process only considered software licensing and hardware costs for five years. The cost of additional licencing, software renewal and support arrangements were therefore not considered. We note that the State Supply Commission’s ‘value for money policy principle’ is that the full cost of providing all the technical and business requirements be reflected in decisions to award a contract.
When the full business and technical requirements are not met, there is an increased risk that a project will suffer from limited adoption, require expensive retrofitting or the use of ‘workarounds’. All of these may result in unnecessary security risks and substandard business performance.
Health’s project governance framework was not followed
Project governance is the framework which ensures project decisions are structured, transparent, authorised and based on accurate reliable information.
Although the project governance requirements were defined and documented in the PID, Health did not implement these. As a result, Health lacked critical oversight to ensure the project was properly designed, planned and managed for the successful delivery of objectives.
The project’s executive sponsor was also the CIO
The executive sponsor for the project was also Health’s CIO. Good practice is normally to have an executive sponsor who is neither part of the IT group or part of the business area. An executive sponsor who is independent of both the IT and the business area is best able to ensure that the project is aligned with the needs of all stakeholders and to provide objective oversight of the management of the project.
The fact that the executive sponsor was not independent was raised as a high risk in the PID. This risk was only addressed toward the latter part of 2013 when the project was already behind schedule. Addressing risks in a timely manner is key to ensuring successful project delivery.
The project executive sponsor was not properly supported
The PID contained a requirement for a Project Control Group (PCG) to provide critical support to the executive sponsor. However, this important role was never made operational. HIN attempted to set up a project control group but they were unable to get appropriate business and stakeholder representation. This important governance mechanism was therefore not established.
The PCG’s role was to advise the executive sponsor and to approve the approach and procedures for project management. As well, they were to approve the criteria to be used for accepting completion of each stage of development and to provide a quality review of each stage. However, even if the PCG had been operational, we note that no success factors were defined for each stage of the project.
The PCG’s responsibilities were also to include:
- control over the development of detailed plans, including schedule, milestones and expenditure for the project
- monitoring the management of project risks and ensuring appropriate action is taken to address high risk areas
- monitoring progress of the project against plan, budget and achievement of intended outcomes and benefits, providing support to resolve key issues that arise.
In the absence of the PCG, the executive sponsor relied on the project manager and program manager to provide assurance that the project was on schedule. However, given the roles of these two positions, fully impartial and objective advice could not be assured.
A further factor that weakened the governance of the projects was that the project manager position was not filled for much of the project. This impacted heavily on the day to day management of the project and further compromised the information flow to the executive sponsor.
Monthly project monitoring and reporting was inadequate
The monthly reporting did not enable management to identify and address key issues in a timely manner. For the duration of the project the monthly reporting included the detailed project reports of 29 to as many as 99 projects in one report. This method of reporting is not ideal for managers, making it difficult to identify the key issues and make decisions.
There was no evidence that the project milestones were appropriately monitored by management. The monthly Portfolio Status Reports (PSR) only included milestone reporting until December 2012 and these were not aligned with the project deliverables and key dependencies. These milestones provide a control mechanism for ongoing project oversight and help ensure the project is on track regarding cost, deliverables and time scales.
Ongoing project issues were not escalated. For example, technical difficulties were experienced due to the inability of the existing Health proprietary software and the software of the main hospital support providers to work with the IAM System.
Also, risks that had been accepted at the project level were not always visible at the management level. One example is the acceptance of the risk in November 2012 that the project would not be ready to function at FSH on day one.
The business processes lagged behind the technical development
The need for the business to be suitably engaged in the IAM project was recognised in the PID as an issue of high importance and risk. Despite this, management failed to give the project the attention it required.
The need for greater management attention to the project was evident in a number of ways. For instance, successive monthly PSRs noted that the technical development of the project was being based upon generic business mapping of staff roles. This generic information was used because the actual specifications from FSH, the Albany Health Campus (AHC) and from the private provider of the support services to FSH, had not been supplied. As a result, the technical limitations that impacted on the feasibility of the project were not identified.
The overall consequence of the failure to develop the IAM is that physical admission and access to the key IT applications that would enable FSH to operate in a near paperless environment will not be automated when the hospital opens later this year.
The modules that would allow the IAM system to communicate with the various clinical and human resource applications were also not ready when AHC opened in April 2013. However, card access to the hospital doors, based on user roles, was successfully trialled at AHC. Subsequently, the process reverted to accessing all doors except to the Pharmacy and Records area. This change was necessary because of inaccuracies in the matrix that determined which role should access which parts of AHC.