ICANN Holistic Review

The Holistic Review process was recommended in the final report of the Third Accountability and Transparency Review team. The initial pilot of the process was conducted in 2022.

History

There have been numerous calls in the past for an organization-wide assessment process within ICANN.

ATRT3 Recommendations

The ATRT3 team submitted a series of recommendations designed to reform and streamline the review process:

• ICANN should establish and perform a new "Holistic Review" no later than one year after board approval of any recommendation from ATRT3. After that, alternate the timing of ATRT and Holistic Reviews, so that each review begins no later than twenty-four (ATRT) or thirty months after the approval of the first recommendation contained in the other review. (As in the draft report, this is designed to allow at least two "continuous improvement" cycles to occur within organizations between Holistic Reviews.)
  • No other reviews should be launched during the pendency of a Holistic Review;
  • Holistic Reviews shall last a maximum of eighteen months and operate in accordance with the Operating Standards for Specific Reviews;
  • Holistic Reviews will focus on: 1) continuous improvement efforts of SOs, ACs, and the NomCom; 2) Inter-organizational communication and collaboration; 3) accountability of the SOs, ACs, or contituent parts to their members and constituencies; and 4) reviewing the entirety of the ICANN organizational structure to determine if all the constituent parts still serve a purpose, or if there are opportunities to alter the structure(s) and operations of ICANN to better represent the community and improve effectiveness
• Organizational Reviews should be transformed into "continuous improvement programs" within each structure currently subject to an Article 4.4 review.
  • Allowing for some flexibility so that each organization can optimize its approach to continuous improvement, each program will include annual satisfaction surveys of members/participants, annual assessment of the improvement program (which may either be self-directed or facilitated), and budgetary and staff support that at a minimum matches the operational support for the current organizational review process.
  • Results of the annual surveys and assessments will be published and will be reviewed during the next Holistic Review.

Holistic Review Pilot

The Board approved a pilot of the Holistic Review. The workspace for the pilot review was maintained as part of the ATRT Implementation wiki The draft Terms of Reference (TOR) for the pilot highlighted open issues identified by the board or the review team, and provided a preliminary scope for discussion. On its scorecard, the board specified a number of gaps in information or structure in the proposed holistic review:

• Guidance as to how Holistic Review teams should determine and prioritize their work areas to ensure effective review outcomes within the recommended 18-month timeframe;
• Proposed methodology for gathering and analyzing data to inform fact-based findings and recommendations;
• Articulation of necessary skill sets for Holistic Review team members required to achieve review objectives, which will later be included in the Operating Standards for Specific Reviews;
• Estimate of resources and budget required to complete the Review effectively;
• Suggestions as to how various ICANN structures would be held accountable for implementing the recommendations coming from the Holistic Review, if directed to entities other than the ICANN Board or org; and
• Determination of how future Holistic Review teams would measure the success of implementation and the success of a future Continuous Improvement Program.

Process

ToR

On August 30, 2022, in a Public Comment proceeding, the ICANN Board sought input on the Draft Terms of Reference scheduling, timing, and its fit with ATRT3 recommendation 3.5:

• the effectiveness of collaboration mechanisms between ICANN structures,
• the structures’ accountability to their members and constituencies,
• continuing purpose of structures, and potential changes in structures and operations to improve the overall effectiveness of ICANN,
• optimal representation of community views, and
• continuous improvement efforts of structures.

The public comment asked:

• Do you support the Pilot Holistic Review Terms of Reference?
• Does it provide a clear approach to accomplishing ATRT3’s objectives and address the information gaps identified by the ICANN Board?
• Do the steps and the deliverables outline the scope of work for the Pilot Holistic Review?
• Do the steps and the deliverables explain how Supporting Organizations, Advisory Committees, the Nominating Committee, and constituents will participate in establishing a Holistic Review Program?

Responses

OF the responses, three submissions supported the drafted ToR but requested clarifications (VS, ALAC, AFNIC). Nine submissions raised concerns with the ToR (AP, SSAC, ccNSO Council, BC, RySG, RrSG, GAC, CCWP-HR, IPC). Concerns included: lack of clarity of the scope, identified dependencies, and independent examination within the Holistic Review and the community’s inability to support the potentially heavy Pilot Holistic Review workload.