Follow-Up Audit to the 2019 Integrated Regulatory Review Service (IRRS) Mission

Executive summary

The objective of this audit, performed as a follow-up to the 2019 Integrated Regulatory Review Service (IRRS) mission, was to provide reasonable assurance that:

• the CNSC’s completed actions in relation to the 2019 IRRS Mission were adequately validated
• on-track actions were monitored to ensure their successful implementation
• the existing governance supported the successful implementation of these actions

The audit looked to provide just-in-time findings and recommendations to help the CNSC prepare for the IRRS follow-up mission in June 2024.

Why is this important?

An IRRS mission is a peer-review service designed and managed by the International Atomic Energy Agency (IAEA) to strengthen and enhance the national regulatory infrastructure for nuclear safety, radiation protection, radioactive waste management, transport safety, emergency preparedness and response. The IRRS considers technical and policy issues of a regulatory nature against IAEA Safety Standards and, where appropriate, good practices by drawing upon the extensive international experience and expertise of the regulatory review team members.

In September 2019, the CNSC hosted an IRRS mission to review Canada’s regulatory framework. The resulting report outlined 4 recommendations and 16 suggestions. Management actions are outlined in Canada’s Response to the IRRS Report (Canada’s Response), which was issued in February 2020 (For further details, see appendix B). There are plans for a follow-up IRRS mission in June 2024, when the IRRS will conduct an in-depth review to determine if the CNSC has fulfilled its commitments made in Canada’s Response. In preparation for the follow-up mission, the CNSC will host the IRRS follow-up preparatory meeting in fall/winter 2023 and prepare advanced reference material for the IRRS team by March 2024. A timeline of follow-up actions is outlined in appendix E.

International peer review exercises allow the CNSC to demonstrate its regulatory effectiveness to build trust both in the eyes of the Canadian public and the international community. It is important that the CNSC perform well in these exercises to achieve its goal of being a trusted regulator. This audit looks to complement efforts across the CNSC to prepare for the June 2024 follow-up mission, and to inform future IRRS missions hosted by the CNSC and other IAEA peer-review engagements.

Key findings

This audit assessed whether actions taken by the CNSC are complete or on track for completion, as a precursor to the IRRS follow-up mission planned for June 2024. The audit also assessed if the governance structure provided effective oversight and supported successful implementation of the actions.

Audit testing demonstrated that work undertaken by the CNSC to address the 2019 IRRS Report was complete / on track for completion, aligned with commitments made in Canada’s Response, and was supported by documentary evidence. The audit has recommendations to complement work already underway by the CNSC’s Operations Secretariat and to enhance internal processes at this important juncture.

Through its look at project management and oversight, the audit found that various factors affected the governance of IRRS follow-up activities, such as shifting priorities due to COVID-19, organizational restructuring and a turnover in Operations Secretariat personnel. In addition, IRRS commitments were initially overseen by the Harmonized Plan Steering Committee (HPSC) and accountabilities assigned to directors general (DGs). The oversight provided by this formal governance structure was lost when the HSPC was dissolved in 2021. Since the dissolution of HSPC, oversight of the IRRS commitments has not been clearly defined. There is an opportunity to formalize governance, including confirming accountability at the DG level, to ensure clarity of roles with respect to oversight of the IRRS follow-up activities.

At the project level, the Operations Secretariat is responsible for tracking, monitoring and reporting on progress of IRRS actions. In recent months, the team has made great progress in engaging with key stakeholders, defining responsibilities and tracking and documenting progress on IRRS actions. To complement work underway by the Operations Secretariat, the audit recommended adding or enhancing project management elements covering communications, information management and documentation, resourcing, and reporting to help further prepare the CNSC for the IRRS follow-up mission and future peer review engagements.

The audit includes 6 recommendations aimed at addressing areas for improvement (see appendix C). These recommendations were delivered to the Operations Secretariat at regular intervals during the audit conduct phase from January to May 2023. The audit team delivered audit recommendations and supporting evidence early, to allow management to make any necessary adjustments in a timely manner and to implement actions in advance of upcoming IRRS deadlines.

Management agrees with the recommendations, and its responses indicate its commitment to take action. Since this audit provided assurance as work progressed, it should be noted that the recommendations issued are in process of being addressed.

Although this audit specifically examined the 2019 IRRS Mission, the audit identified opportunities to share lessons learned from this engagement more broadly with the CNSC for future large-scale horizontal initiatives and peer-review engagements. The lessons learned identified could benefit governance and project management of large-scale horizontal organizational initiatives. As the CNSC enters a time of significant corporate change, it should continue to develop project management skills in its staff when implementing transformational or other large-scale horizontal projects.

1. Background

The International Atomic Energy Agency (IAEA) is the world’s centre for cooperation in the nuclear field and promotes the safe, secure and peaceful use of nuclear technologies. The International Regulatory Review Service (IRRS), a service that the IAEA offers to Member States, helps host states strengthen and enhance the effectiveness of their regulatory infrastructure for nuclear, radiation, radioactive waste and transport safety. The IRRS provides a process for peer review of both regulatory technical and policy issues, and offers an integrated approach to the review of common aspects of any state’s national, legal and governmental framework and regulatory infrastructure for safety^{Footnote i}.

In September 2019, the CNSC hosted an IRRS mission to Canada.

Phase I of the IRRS program was a self-assessment completed in advance of the mission. The CNSC’s self-assessment was submitted in July 2019 and identified 19 actions.

Phase II of the IRRS program was the peer-review mission, which occurred in September 2019. The results from the peer review mission were published in the IRRS Report to Canada in February 2020. This report identified 6 good practices, 4 recommendations and 16 suggestions.

A full response to addressing each good practice, recommendation and suggestion was outlined in Canada’s Response to the IRRS Report (Canada’s Response) and was shared publicly in February 2020. The results from the IRRS mission (including the action plan) were presented to the Commission in June 2020. The CNSC also committed to providing regular status update reports to governance committees on progress made.

Phase III is the follow-up mission, which will review the progress of the commitments made in Canada’s Response. The IRRS is scheduled to conduct its follow-up mission in June 2024. In preparation, the CNSC will host the IRRS follow-up preparatory meeting in fall/winter 2023. Advanced reference material needs to be completed and submitted to the IRRS team by early March 2024. Appendix E shows the timeline leading into Phase III in further detail.

A 2022 study on nuclear waste governance in Canada undertaken by the House Committee on the Environment and Sustainable Resources recommended that the CNSC “implement recommendations that were made by the IAEA’s Integrated Regulatory Review Service Audit that was conducted in 2019”. This engagement is also meant to support the CNSC in demonstrating this implementation.

At the CNSC, engagement with the IRRS is currently coordinated through the Operations Secretariat division, under the Regulatory Operations Branch. At the time of the audit, Operations Secretariat had 1 full-time employee (the project coordinator) dedicated to project management and coordination of IRRS follow-up activities on behalf of the CNSC.

2. Authority

The Executive Vice-President (EVP) requested this follow-up audit be conducted in advance of the IRRS follow-up mission. The 2022–24 Risk-Based Audit and Evaluation Plan was consequently amended to include this engagement and approved by the President on October 14, 2022.

3. Objectives, scope and approach

The audit objectives were to provide reasonable assurance that the CNSC’s completed actions in relation to the 2019 IRRS Mission were adequately validated, that on-track actions were monitored to ensure their successful implementation and that the existing governance supported the successful implementation of these actions. Appendix A sets out detailed information on the audit lines of inquiry and criteria.

The audit covered the time frame from February 2020 to April 2023, and the scope included activities conducted to implement suggestions and recommendations of the 2019 IRRS report. Actions listed as complete, as well as those in process, were included. Actions identified by CNSC staff in the self-assessment prior to the 2019 mission were not within this audit’s scope. These actions were deemed low risk since they are not included in the scope of the assessment of the 2024 IRRS follow-up mission.

The audit team provided periodic updates of findings to the project team and management, throughout the conduct phase. These updates are consolidated in this report.

The audit approach included:

• review of documentation
• testing of actions using non-probability samples prioritized by risk-based selection criteria and auditor judgement: the sample comprised 4 IRRS recommendations and 7 suggestions (see Appendix B for further details on the methodology used)
• interviews with project staff and stakeholders
• review of governance, oversight and project management practices related to implementation of actions

4. Statement of conformance

This internal audit was conducted in conformance with the International Standards for the Professional Practice of Internal Auditing, and in accordance with the requirements of the Treasury Board’s Policy on Internal Audit and Directive on Internal Audit, which provide mandatory procedures for internal auditing in the Government of Canada.

5. Acknowledgments

The audit team would like to acknowledge and thank management and staff for their support.

6. Findings and observations

6.1 Review and validation of procedures

The audit team selected a risk-based and judgmental sample of IRRS actions, in order to test if they were completed in accordance with the IRRS Report and Management Action Plans. The sample consisted of 4 recommendations and 7 suggestions from the IRRS Report.

The audit team expected to find that the sampled actions aligned with commitments made by the CNSC in its action plans outlined in Canada’s Response. For actions deemed “closed”, the audit team expected to find supporting documentation and an executive sign-off indicating that the actions taken satisfied commitments made. For actions deemed “open”, the audit team expected to find a clear plan with supporting documentation to complete the action ahead of the publication of advanced reference material, which is to be shared with the IRRS team in early March 2024.

Audit testing demonstrated that work undertaken by the CNSC to address the 2019 IRRS report was either complete or on track for completion, aligned with commitments made in Canada’s Response, and supported by documentary evidence. In the cases of closed actions, executive sign-off had been provided, indicating management review.

In engaging with action owners, the audit team noted some best practices in documenting and presenting work completed for IRRS commitments.

These practices have been developed by directorates taking individualized approaches to their work. However, the audit team sees potential value in sharing these approaches across other directorates as teams prepare for the IRRS follow-up mission.

There is an opportunity for the Operations Secretariat to:

• share best practices in presenting work across implicated directorates as teams prepare for the IRRS follow-up mission
• record best practices in documenting follow-up work for future IRRS missions and IAEA peer-review engagements

6.2 Governance

Governance refers to the structures, systems and practices that an organization has in place to establish its strategic direction. This includes, but is not limited to, the assignment of decision-making authorities; oversight of policies, plans, programs, and projects; monitoring and mitigation of key risks; and timely reporting to drive ongoing improvements and corrective actions.

Roles, responsibilities, and reporting structure

The audit expected to find that roles, responsibilities, and a reporting structure were clearly defined and communicated for actions committed to in Canada’s Response, with defined coordination at the different project levels, to ensure an effective and efficient follow-up process. The audit also expected to find supporting evidence that a governance structure was in place to provide oversight over the completion of actions in a timely manner.

Governance structure

The audit found that several factors have affected the governance and management of IRRS follow-up actions, including the following:

• IRRS follow-up officially commenced with the February 2020 publication of Canada’s Response. With the onset of COVID-19 in 2020, the focus of the CNSC’s operations staff shifted away from IRRS actions to the pressing challenges brought forth by the pandemic.
• IRRS management actions and the initiative were to be overseen by the Harmonized Plan Steering Committee (HPSC). The HPSC disbanded in 2021 without a clear oversight plan for IRRS follow-up initiatives.
• Over the course of follow-up, Operations Secretariat staff turned over frequently, both at the management and program levels. For example, the role of project coordinator has been filled by 4 different people since February 2020.

The audit found that a governance structure was not actively in place at the beginning of February 2023, although there have been periodic briefing updates to the EVP and a presentation to the Integrated Planning and Resource Management Committee in October 2022. As of April 2023, an array of options were being considered by the Operations Secretariat to provide the CNSC’s IRRS follow-up with formal oversight and strategic guidance.

Internal coordination and stakeholder engagement

The IRRS mission follow-up is a cross-cutting initiative implicating directorates from across four branches of the CNSC, requiring a coordinated effort across the organization. The audit found that over the course of the audit period, engagement between key stakeholders – including action owners, project managers and decision makers – on the IRRS action follow-up was inconsistent and ad-hoc, resulting in lack of internal coordination among various project levels.

In interviewing past and current IRRS project coordinators, there was a noted sense of frustration stemming from an inability to get deliverables, updates and work accomplished by staff from implicated directorates because of competing priorities. For instance, when requesting a status update from action owners in February 2023, the project coordinator reported that updates for 3 actions were not received at all, updates for 6 actions were not received by the deadline, and many were missing adequate detail. Only 3 of the 16 actions were reported as completed by the project coordinator. It was expressed that DGs and the Executive Team could potentially assist in this area by advocating for the importance of completing actions and that directors overseeing project management could improve support of project coordinators with consistent direction, communication and updates on their correspondence with implicated DGs and the EVP.

Interviews with action owners indicated that direction on what was needed to close actions was not always clear and consistent, and that expectations would change with staff turnover on the IRRS project management team. Engagement between the Operations Secretariat and implicated directorates was ad hoc and irregular. This occasionally resulted in a lack of clarity on the status of action items and the expectations of action owners.

The Operations Secretariat has recently adopted some good practices to address the project management of the follow-up activities in preparation for the IRRS follow-up mission. In October 2022, the Operations Secretariat presented to the Integrated Planning and Reporting Management Committee on IRRS actions, and reported 9 of 17 action items completed. An accompanying memo was provided to the EVP indicating a status update and project next steps.

In January 2023, the Operations Secretariat engaged implicated DGs and subject matter experts and developed an informational deck on next steps and roles and responsibilities for the IRRS follow-up mission. Going forward, the Operations Secretariat is looking to engage directorate points of contact monthly through the establishment of a technical team, and DGs periodically through a project-specific steering committee. The Operations Secretariat is also developing guidance to promote consistency of documentation in addressing all actions.

The audit identified opportunities to strengthen the project management of IRRS follow-up activities to address the concerns stated above and to complement ongoing efforts from the Operations Secretariat. During the 2019 IRRS Mission, a communications plan was developed providing a strategic template for engaging both internal and external stakeholders; this may provide insights to further engage effectively with the organization. Additionally, the Treasury Board Directive on the Management of Projects and Programmes outlines the importance of active leadership and communication with stakeholders^{Footnote ii}. The use of standard email templates for follow-up on overdue items and an escalation process may be effective in assisting in the process.

Formalizing the follow-up on updates and deliverables, and applying Treasury Board best practices would help support the team and provide consistency to enhance institutional memory.

Timelines and accountability

The audit expected to find that oversight mechanisms were in place to ensure that actions were on track to be completed fully and on time.

As noted earlier in this report, the audit sample indicated that, at the working level, open actions were on track for completion ahead of the publication of advanced reference material in March 2024. As well, the Operations Secretariat has done some work towards ensuring action accountability by engaging with implicated DGs and subject matter experts. However, the audit found that, at the governance level, there is a gap in accountability when it comes to ensuring actions continue to stay on track.

Throughout the project, there has been an informal approach to the application of timelines. Though timelines were reported in Canada’s Response in February 2020, many of these were revised in the Operations Secretariat’s internal tracking tool after the onset of COVID-19. The audit team heard that the project management team considered the deliverable due dates in internal tracking to be artificial, with the focus being completion in advance of the follow-up mission. As such, no formal provisions were in place to enforce or revise timelines on actions. The audit found that 9 of 16 actions had their internal timelines for completion extended at some point, with 6 actions having their timelines pushed back several times. In addition, 6 of 7 actions that remained open at the time of audit testing had already had their internal timelines for action completion pushed back at least once. There is still a lack of clarity on the expected date of completion for open actions. Audit testing found inconsistency in the expected date of completion for deliverables when comparing between various platforms for internal tracking. For more information on these platforms, see the Information Management section in this report.

Throughout the post-mission period, there has lacked a regular reporting mechanism to consistently check in on timelines and enforce accountability. After the publication of Canada’s Response in February 2020, the project management team reported on the status of IRRS actions to HPSC semi-annually and to the Operations Management Committee (OMC) on a quarterly basis. The HPSC has since dissolved and reporting to OMC stopped as of March 2021.

Risk And impact

Throughout the audit period, a lack of formal governance structure and a disconnect between various levels of the project have resulted in a lack of assurance on the status of IRRS actions and a lack of accountability in completing these actions. As deadlines and deliverables for the IRRS follow-up mission are fast approaching, an effective governance structure is important to provide strategic guidance and accountability. Effective internal coordination is important to ensure that actions are completed in a timely manner and presented to the IRRS committee in a coherent fashion. Successful completion of actions and presentation of work done is integral to a positive outcome for the CNSC at the IRRS follow-up mission.

Recommendation 1

The Operations Secretariat should review and formalize the governance structure to ensure that project management needs are met.

Suggested completion date: September 2023

Recommendation 2

Measures should be taken to complement and formalize current efforts to improve internal coordination, executive endorsement and stakeholder engagement. The Operations Secretariat should draft and implement a communications strategy leveraging existing 2019 IRRS Mission documentation and Treasury Board Secretariat standards to enhance internal coordination, amplify executive endorsement and strengthen stakeholder engagement.

Suggested completion date: September 2023

Recommendation 3

Steps should be taken to reinforce accountability in meeting and altering timelines. Specifically, regular reporting should be reinstated.

Suggested completion date: September 2023

Project management resourcing

The audit observed significant efforts in recent months by the Operations Secretariat to prepare the CNSC for the IRRS follow-up mission. While these efforts helped to strengthen the CNSC’s readiness for the follow-up, the audit found that institutional knowledge around the project management of IRRS follow-up activities resided with only 2 people: the IRRS project coordinator and the Operations Secretariat director. This creates a risk of losing key information if those staff left the organization. Corporate reorganization and structural changes have also impacted IRRS follow-up activities. As of April 2022, the Operations Secretariat no longer reported to a DG and instead reported directly to the EVP.

At the project management level, there is an opportunity to ensure that the IRRS project coordinator is adequately supported to set the project up for success. Having a second position to support the coordinator role would provide additional support and redundancy in case of turnover or staffing changes within the Operations Secretariat. It is worth noting that it is standard practice for IRRS engagements to have a liaison to the IRRS Mission on behalf of the regulator. The audit found that this role was currently vacant, though the Operations Secretariat has stated its intentions to have someone in this position. Filling this position could help support the project coordinator and create redundancy from a project management perspective.

Risk and impact statement

Adequate resourcing and knowledge management is important to ensure that project objectives can be met efficiently and effectively, and to promote business continuity in the event of employee turnover. Failure to ensure adequate resourcing and knowledge management may also increase the risk of project delays at an important period with key project milestones and deliverables coming up.

Recommendation 4

The Operations Secretariat should involve more staff in the IRRS follow-up project to create redundancy, buffer against turnover and spread out institutional memory of the project. Specifically, the Operations Secretariat should ensure that an individual is appointed to support the IRRS project coordinator by providing strategic guidance and decision making in a timely manner.

Suggested completion date: June 2023

6.3 Project management

Project management is the act of planning, organizing, and managing a project in order to achieve a predefined goal or outcome. This requires strategic planning, estimating required resources, and developing a finite span to complete the endeavour. Having processes in place to close actions and to promote information management practices are suggested best practices for the IRRS follow-up mission, in terms of project management. Sound project management practices are necessary to reduce risks, stay on track, reach higher productivity, enable consistent communication and to ultimately strengthen overall efficiency and effectiveness.

Uncertainty around closing actions

IAEA guidelines tend to be broad to allow countries to implement measures according to their unique circumstances. As such, addressing IRRS recommendations and suggestions requires some discretionary judgment.

The project would benefit greatly from having a clear process for closing actions. There are many CNSC staff who have participated as peer reviewers in previous IRRS missions who could be valuable resources in developing guidance on closing actions.

Risk and impact

Established processes for decision making, such as clear parameters on closing actions, is essential to ensure that the IRRS follow-up mission operates efficiently and effectively. If this area is left unaddressed, there is a risk that actions may be improperly closed without sufficient documentation, or remain open unnecessarily. This could result in further delays in progress and rework by staff.

Recommendation 5

The Operations Secretariat should formalize parameters for closing IRRS actions. In developing this guidance, it should leverage CNSC staff who have previously served on IRRS Missions as peer reviewers.

Suggested completion date: September 2023

Information management

The audit expected to find that processes, tools, and resources were in place to support information management practices; for example, consistent documentation and tracking on the status of action items and effective documentation of key decisions for the purposes of internal coordination and institutional memory.

Information management for IRRS actions

The audit observed inconsistent information management practices, which has hindered project progress and posed a risk to business continuity. After the IRRS mission was complete, the approach to knowledge and information management lacked continuity and a formal process. As staff and directors turned over at the Operations Secretariat, new tracking processes were developed to suit the preference of the team in place and to fill in information gaps without the closure of previous tracking processes. As a result, mechanisms to track IRRS actions and store key information have been spread across a variety of platforms, including the following:

• Operations Secretariat tracking sheet: This is meant for internal use by the IRRS project coordinator.
• Regulatory Information Bank (RIB): This is an online tool that houses information about actions taken in response to licensing and compliance activities. The RIB has an audit trail that indicates action status, what has been done and associated documentation.
• Opportunity for Improvement (OFI) process: This is the current system implemented. It was chosen because it is familiar to Operations staff and has an action plan template that helps give the Operations Secretariat status updates on actions.
• Action plans: These are filled out by implicated directorates to report on action progress and used as an input for OFI tracking.

The audit tested consistency across these 4 platforms for 10 sample cases (IRRS recommendations/suggestions) and found:

• The party accountable (directorate, point of contact) was up-to-date and consistent on these platforms for all samples tested.
• Pertinent and up-to-date documentation was consistently included or referenced across these platforms for 7 of 10 samples tested. References to supporting documentation for the three exceptions were either erroneous or nonexistent.
• Action status (open or closed) was consistent and up-to-date in 5 of 10 of the samples tested. In the RIB, all closed IRRS actions were re-opened on May 16, 2022, for verification purposes, but were never closed thereafter.
• Timelines and expected dates of action completion were inconsistent in all of the samples tested.

The audit observed that, upon publication of Canada’s Response in February 2020, the Operations Secretariat selected the RIB as the initial tool for tracking actions to completion. However, not all parts of the CNSC were familiar with the RIB or used it in the same way, resulting in inconsistent data entry. Many RIB files were lacking documentation and sufficient information to verify the status of the action. As well, RIB action status was not always current, nor did it accurately reflect the amount of work that had been done. These combined factors made RIB tracking unreliable and presented significant challenges in determining the true status of actions when project management staff turned over in the Operations Secretariat. RIB cases for IRRS actions remained open as of April 2023. Though work had been done by the Operations Secretariat to bring the RIB up to date, there remained data quality issues and, thus, a risk to business continuity in case of further turnover in the project management team.

The audit team noted that the Operations Secretariat had initiated work in January 2023 towards making the different tracking platforms consistent in information, and towards making the OFI system the single source for tracking completion of suggestions and recommendations. The audit team observed improvements regarding Operations Secretariat updates to tracking documents. These improvements included clarifying the accountable party (directorate, point of contact) for IRRS recommendations and suggestions, adding links to cross-reference platforms and pertinent documentation, and incorporating a risk-ranking element.

There remain opportunities to improve data quality across these tracking documents and enhance information management practices, by aligning with relevant Treasury Board best practices and adopting previous IRRS mission documentation^{Footnote iii}. Additionally, it should be assessed if one tracking mechanism could be used for the remainder of the project and the others be closed. The tracking approach should be clearly communicated with stakeholders, with training and support offered as needed on the agreed-upon tool.

Risk and impact

There is a continued risk to business continuity with the variety of tracking mechanisms and dispersion of key information, as not all information management tools contain the same information. Maintaining consistent and up-to-date tracking documentation and mechanisms is essential for effective monitoring, reporting and decision making. A lack of consistency could lead to gaps in work completion or an increase in effort, especially in the event of sudden staff turnover and loss of institutional knowledge. For purposes of business continuity, it is important that these tracking platforms are kept consistent and up-to-date, or closed, if they are of no further value to the Operations Secretariat.

Recommendation 6

The Operations Secretariat should formalize information management practices to ensure business continuity.

Suggested completion date: December 2023

Lessons learned and broader CNSC applicability

Although this audit specifically examined the 2019 IRRS mission, the audit identified opportunities to share lessons learned from this engagement for CNSC for future large-scale horizontal initiatives and peer-review engagements. These lessons learned could benefit governance and project management of large-scale horizontal organizational initiatives. As the CNSC enters a time of significant corporate change, it is contingent on the CNSC to brace itself from a project management perspective and ensure versatility in the face of changing circumstances.

Large-scale horizontal initiatives require significant investment and co-ordination of resources from the CNSC. If the CNSC does not track lessons learned from these experiences, there is a risk that other large-scale horizontal CNSC projects will have similar issues with knowledge transfer, information management and will result in an inefficient use of CNSC resources. CNSC should continue to develop project management skills to improve the efficiency and effectiveness when implementing transformational or other large-scale horizontal projects.

7. Overall conclusion

The audit objectives were to provide reasonable assurance that the CNSC’s completed actions in relation to the 2019 IRRS mission were adequately validated, that on-track actions were monitored to ensure their successful implementation and that the existing governance supported the successful implementation of these actions. The audit looked to provide just-in-time findings and recommendations to help the CNSC prepare for the IRRS follow-up mission in June 2024.

The audit concluded that closed actions were aligned with commitments made as well as supported by documentary evidence of completion and executive sign-off. Further, open actions were on track for completion before the publication of advanced reference material to be provided to the IRRS team in March 2024. This was due to the diligent efforts of action owners and recent progress by the Operations Secretariat team in engaging action owners and implicated DGs.

The audit assessed whether the governance structure in place was sufficient to ensure that actions were completed effectively, efficiently and appropriately prior to the IRRS follow-up mission. Though the audit found that work was on track for completion, the next challenge for the CNSC will be to demonstrate this work to the IRRS team and there remain risks to the CNSC in this area. Effective project management and governance is key at this stage of the project. The audit has made recommendations to complement work already underway by the Operations Secretariat.

In looking ahead to future IRRS missions and peer review engagements, the audit found opportunities to improve the process of managing post-management response activities. Post-mission activities lacked sufficient sustained attention to knowledge management, consistency of documentation, governance and accountability. Throughout the COVID-19 pandemic, turnover and corporate reorganization may have exacerbated these issues, and there was significant effort expended and inefficiency that resulted in diversion of resources.

As a result of this engagement, there are lessons that the CNSC can learn from this experience, in order to improve processes and to buffer against risks associated with evolving internal and external circumstances.

Appendix A: Audit criteria

1: Review and validation of procedures

1. A sample of actions listed as complete for Canada’s Response and are validated to ensure full completion and alignment with commitments made.
   1. CNSC actions align with commitments made in Canada’s Response.
   2. Closed actions are clearly documented to be complete and supported with sufficient and relevant information/evidence of completion.

2: Governance and project management

1. Roles, responsibilities and reporting structure are clearly defined and communicated for actions committed to in Canada’s Response.
   1. Roles and responsibilities are clearly defined, documented, communicated and understood.
   2. The reporting structure has been clearly defined, documented, communicated and understood.
2. The governance structure in place provides oversight over completion of actions.
   1. Oversight mechanisms are in place to ensure regular reporting on actions and informed decision making.
   2. Oversight mechanisms are in place to ensure that open actions are on track to be completed fully and on time.
3. Processes, tools and resources are in place to support project management
   1. Documentation of action status is kept consistent and up-to-date.
   2. Key decisions and relevant information stemming from governance bodies are effectively documented and communicated for the purposes of internal coordination and institutional memory.
   3. Adequate resources (i.e., number of staff, appropriate action owners) are assigned to complete actions fully and in a timely manner.
   4. A process exists to periodically assess and document whether suggestions, recommendations and actions continue to be relevant amidst new and changing circumstances.

Appendix B: Sampling methodology

As part of the audit approach, the audit team tested a sample of follow-up actions aimed at addressing the IRRS report recommendations and suggestions. The sample provided evidence towards concluding on the following audit sub-criteria:

• 1.1.1 Closure criteria align with commitments made in Canada’s Response.
• 1.1.2 Closed actions are clearly documented to be complete and supported with sufficient and relevant information/evidence of completion.
• 2.2.2 Oversight mechanisms are in place to ensure that open actions are on track to be completed fully and on time.
• 2.3.1 Documentation of action status is kept consistent and up-to-date.
• 2.3.3 Adequate resources (ie., number of staff, appropriate action owners) are assigned to complete actions fully and in a timely manner.

The methodology chosen for this audit was a judgmental sample. This approach allowed the audit team to tailor the sample to the needs of the audit. The audit team looked to select a sample that was representative of the population and prioritized actions that pose a higher risk to the CNSC. Because the actions were heterogeneous in their constitution and risk profile and the population small, a judgmental sample was more appropriate than a randomized sample for the needs of the project.

In selecting a sample that was representative of the population, the audit team included actions from each of the modules in which the IRRS report issued recommendations or suggestions and in which the CNSC is the lead entity in responding. The sample also included actions under the purview of the seven different directorates implicated at the CNSC. Taking this sample allowed the audit team to ensure coverage in the different areas the IRRS team examined and to see how actions were being addressed across the CNSC. Although the intent was to create a representative sample, some directorates and modules were included more often than others due to risk considerations.

Appendix C: Management action plan

Rec #	Audit recommendation	Date shared with the project team	Risk rating*	Office of primary interest	Management action plan	Targeted completion date
1	The Operations Secretariat should review and formalize the governance structure to ensure that project management needs are met.	February 2023	Moderate	Operations Secretariat	The Operations Secretariat began evaluating the options and consulting with stakeholders in March 2023. This has included evaluating the implementation of a Steering Committee and what the terms of references would be. Final roles and responsibilities will be outlined and approved in the project plan that is under development.	September 2023
2	Measures should be taken to complement and formalize current efforts to improve internal coordination, executive endorsement and stakeholder engagement. In doing so, the Operations Secretariat should draft and implement a communications strategy, leveraging existing 2019 IRRS Mission documentation and Treasury Board Secretariat standards.	March 2023	Moderate	Operations Secretariat	The Operations Secretariat engaged the Strategic Communications Directorate (SCD) in April 2023. A commitment to collaborate on a communications plan to be developed was obtained. This communications plan will be modelled on that of the 2019 IRRS Mission. SCD has assigned a communications advisor to the project.	September 2023
3	Steps should be taken to reinforce accountability in meeting and altering timelines. Specifically, regular reporting should be reinstated.	March 2023	Moderate	Operations Secretariat	The Operations Secretariat will initiate a focused steering committee to oversee the project and to formally report to Operations Management Committee throughout the project. Operations Secretariat staff have already taken steps as of January 2023 to bring together a technical team consisting of representatives from each impacted directorate. The communications plan will assist in this area.	June 2023
4	The Operations Secretariat staff should involve more staff in the IRRS follow-up project to create redundancy, buffer against turnover and spread out institutional memory of the project. Specifically, the Operations Secretariat should ensure that an individual is appointed to support the IRRS Project Coordinator by providing strategic guidance and decision making in a timely manner.	March 2023	Moderate	Operations Secretariat	The Operations Secretariat has engaged the Human Resources Directorate to staff a project lead for the IRRS follow-up mission.	August 2023
5	The Operations Secretariat should formalize parameters for closing IRRS actions. In developing this guidance, it should leverage CNSC staff who have previously served on IRRS Missions as peer reviewers.	March 2023	Moderate	Operations Secretariat	The Operations Secretariat, in conjunction with accountable DGs, will formalize the guidance for addressing IRRS actions in consultation with CNSC staff who have previously served on IRRS missions as peer reviewers.	September 2023
6	The Operations Secretariat should formalize information management practices to ensure business continuity.	March 2023	Moderate	Operations Secretariat	The Operations Secretariat has begun aligning the various tools being used and will move to a one-tool approach during the execution phase of the project. All documentation for this project will be archived in a similar fashion as it was for the 2019 IRRS mission. Consistent information management processes will be established, communicated and validated according to the project plan.	December 2023

*Audit recommendations are risk rated as follows:

High	Controls are not in place or are inadequate.
High	Compliance with legislation and regulations is inadequate.
High	Important issues are identified that could negatively impact the achievement of program/operational objectives.
Moderate	Controls are in place but are not being sufficiently complied with.
Moderate	Compliance with central agency/departmental policies and established procedures is inadequate.
Moderate	Issues are identified that could negatively impact the efficiency and effectiveness of operations.
Low	Controls are in place but the level of compliance varies.
Low	Compliance with central/departmental policies and established procedures varies.
Low	Issues identified are less significant but opportunities that could enhance operations exist.

Appendix D: IRRS suggestions and recommendations

IRRS action identifier	Recommendation/suggestion	Commitment
Recommendation 1	The government should enhance the existing policy and establish the associated strategy to give effect to the principles stated in the Canadian Radioactive Waste Management Policy Framework.	Natural Resources Canada will review its existing policy for radioactive waste, and consider how it may be enhanced to give effect to the principles stated in the Radioactive Waste Policy Framework, including the establishment of an associated strategy.
Recommendation 2 Suggestion 9	R2: The CNSC should establish or approve dose constraints for all Class I facilities. S9: The CNSC should consider consistently implementing the concept of dose constraints for all facilities and standardising regulatory practice for derived release limits.	The CNSC has previously identified inconsistent application of dose constraints for derived release limits for Class I facilities. This is being addressed in draft REGDOC-2.9.2, Controlling Releases to the Environment, which will address the role of dose constraints in optimization and in support of the process for authorization of discharges.
Recommendation 3	The CNSC should ensure that the radiation protection requirements are consistent with the requirements of the IAEA’s General Safety Requirements (GSR) Part 3.	The CNSC is currently finalizing a revision to the Radiation Protection Regulations, and anticipates that these regulatory amendments will be made in early 2020. This update represents more than 6 years of work and stakeholder consultation. The revised regulations will be more consistent with the requirements of the IAEA’s GSR Part 3 while taking into account the CNSC’s comprehensive framework for safety and the needs of Canadians. As part of its ongoing review cycle of regulatory instruments, the CNSC will continue to examine relevant international standards and recommendations in order to identify areas for improvement that enhance safety.
Recommendation 4	The CNSC should revise its guidance for package design certification applications to align it with IAEA Specific Safety Requirement (SSR)-6.	The CNSC will continue to work with counterparts at the United States Nuclear Regulatory Commission and the United States Department of Transportation to revise RD-364, Joint Canada-United States Guide for Approval of Type B(U) and Fissile Material Transportation Packages.
Suggestion 1	The government should consider explicitly addressing SF-1, Principle 4 (Justification) in its legal framework.	Not accepted.
Suggestion 2	The government should consider expressly assigning, in its legal framework, the prime responsibility for safety to the person or organization responsible for a facility or an activity.	Not accepted.
Suggestion 3	The government should consider enhancing the legal framework to explicitly stipulate that compliance with regulations and requirements established or adopted by the regulatory body does not relieve the person or organization responsible for a facility or an activity of its prime responsibility for safety.	Not accepted.
Suggestion 4	The CNSC should consider continuing to focus on its human resource management plan to ensure that it continues to have access to a sufficient number of qualified and competent staff to regulate existing facilities and activities as well as new and emerging technologies in accordance with the nature of facilities.	The CNSC will continue to ensure that its upcoming human resource management plan clearly identifies the core and emerging skill requirements of the organization and the workforce needed to execute its mandate. This will include defined strategies and actions to secure access to the necessary skills, along with clear performance measures to track progress.
Suggestion 5	The CNSC should consider consolidating all elements of its safety policy into a single document.	The CNSC will consolidate all elements related to safety policy into a single, formalized safety policy document.
Suggestion 6	The CNSC should consider revising its current and planned requirements in the area of decommissioning, to align with the IAEA guidance that entombment is not considered an acceptable strategy for planned decommissioning of existing nuclear power plants and future nuclear facilities.	The CNSC will include further text to explicitly reflect that in situ decommissioning should not be considered an acceptable strategy for planned decommissioning of existing nuclear power plants and future nuclear facilities other than uranium mines
Suggestion 7	The CNSC should consider establishing a procedure to ensure the systematic implementation of justification in the authorization of all practices involving radiation sources.	Although the CNSC will not change how it operationally conducts authorization reviews, it will administratively update its internal procedures, in order to document how it systematically implements justification in the authorization of all practices involving radiation sources
Suggestion 8	The CNSC should consider including notification alone as an option for the regulatory control of nuclear substances and radiation devices in accordance with a graded approach.	The CNSC will review and consider whether there is merit in moving to a notification scheme for some very-low risk applications that might not require the same level of review as those of higher risk, provided that they are used in accordance with appropriate standards.
Suggestion 9	The CNSC should consider consistently implementing the concept of dose constraints for all facilities and standardizing regulatory practice for derived release limits.	The CNSC has previously identified inconsistent application of dose constraints for derived release limits for Class I facilities. This is being addressed in draft REGDOC-2.9.2, Controlling Releases to the Environment, which will address the role of dose constraints in optimization and in support of the process for authorization of discharges.
Suggestion 10	Health Canada should consider undertaking a survey of radionuclide levels in building materials or indoor gamma dose rates arising from building materials to determine if they make a significant contribution to public exposure.	Health Canada will consolidate and update the risk assessment in collaboration with relevant authorities, where necessary. Results will be used to determine whether further action is required.
Suggestion 11	The CNSC should consider formalizing the practice of inspector exchanges between licensee locations for inspection assistance to ensure the operating experience and lessons learned from assisting other CNSC staff perform inspections at different licensee locations are maximized.	The CNSC will review and formalize its current practices for inspector exchanges between licensee locations by September 2021 to maximize the current sharing of operating experience and lessons learned.
Suggestion 12	The CNSC should consider its process to formalise all elements used to ensure a comprehensive, regular review of the objectivity and independence of the on site inspectors.	The CNSC will review its current practices by September 2021 and formalize the process elements that ensure the objectivity and independence of inspectors.
Suggestion 13	The CNSC should consider performing unannounced inspections for uranium fuel fabrication, refining and conversion facilities.	The CNSC will further codify how unannounced inspections will continue being considered as part of the Nuclear Fuel Cycle and Research Reactors Program.
Suggestion 14	The CNSC should consider implementing a systematic gap analysis between the IAEA requirements and the regulatory framework and updating the regulatory framework as necessary.	When the CNSC develops or reviews its regulatory instruments, it considers applicable IAEA safety standards within the regulatory framework, but this practice is not always systematically documented. To ensure consistent documentation of this practice, the CNSC’s management system document Conduct Regulatory Policy Analysis and associated templates was updated in December 2019. The document now notes the requirement to perform a systematic gap analysis against IAEA safety standards.
Suggestion 15	The CNSC should consider the requirements of SSR-4 and relevant IAEA guidance when specifying safety requirements and criteria for fuel cycle facilities.	CNSC staff will update the current TARMs with information and reference to SSR-4 and other IAEA guidance related to fuel cycle facilities.
Suggestion 16	The CNSC should consider establishing or adopting guidance aligned with IAEA TS-G-1.4.	During the next revision of REGDOC-2.14.1, Information Incorporated by Reference in Canada’s Packaging and Transport of Nuclear Substances Regulations, 2015, the CNSC will add an explicit reference to all relevant IAEA Safety Guides.

Appendix E: Milestones timeline

IRRS mission timeline

September 2018	CNSC requests IRRS mission
December 2018	Preparatory meeting between Government of Canada and IAEA representatives
July 2019	Canada provides the IAEA with self-assessment (the CNSC’s self-assessment identified 19 actions)
September 2019	IRRS mission is conducted
November 2019	CNSC close-out and lessons learned published
February 2020	IRRS Report and Canada’s Response published

IRRS follow-up mission timeline

June 2020	Action plan presented to Commission
March 2021	Last quarterly update on 2019 IRRS mission to OMC
July 2021	Operations Secretariat presents the last IRRS dashboard to Harmonized Plan Steering Committee before the committee is disbanded
October 2022	Operations Secretariat presents to Integrated Planning and Resource Management Committee on status of actions for 2019 IRRS mission
January 2023	Operations Secretariat engages DGs and subject matter experts implicated in IRRS follow-up activities, and develops an informational deck on next steps and roles and responsibilities for the IRRS follow-up mission
September 2023	Operations Secretariat to formally launch preparations for IRRS follow-up mission
December 2023	CNSC to host preparatory meeting for IRRS follow-up
March 2024	Finalized advanced reference material to be completed and submitted to IAEA by first week of March
June 2024	IRRS follow-up mission to take place

Appendix F: Acronyms

The following table presents acronyms used in this document.

CNSC

Canadian Nuclear Safety Commission

DG

Director general

EVP

Executive Vice-President

HSPC

Harmonized Plan Steering Committee

IAEA

International Atomic Energy Agency

IRRS

International Regulatory Review Service

OFI

Opportunity for Improvement

OMC

Operations Management Committee

RIB

Regulatory Information Bank

SSR

Specific Safety Requirement

Section 4.3.2.10 of the Policy on Service and Digital states that decisions and decision-making processes should be documented to account for and support the continuity of departmental operations, to permit the reconstruction of how policies and programs have evolved, to support litigation readiness, to and allow for independent evaluation, audit and review.