Third Accountability and Transparency Review

The Third Accountability and Transparency Review (ATRT3) was initiated in January 2017. As of May 2021, the review is in the plan implementation phase.

Background

Purpose and Origin of Specific Reviews

The Affirmation of Commitments, an agreement between ICANN and the United States Department of Commerce, establishes ICANN's obligations to perform its duties with specific commitments in mind. All of the commitments bear on public and consumer trust of the organization. ICANN is to perform its functions in a manner that:

• ensures accountability and transparency of decision-making;
• preserves the security, stability, and resiliency of the DNS;
• promotes competition, consumer trust, and consumer choice; and
• enables access to registration data.

It is also charged to periodically review and assess its performance through the lens of each of the above commitments.

ICANN's board enshrined these commitments (and the associated reviews) in its Bylaws in Article 1 (Mission, Commitments, and Core Values) and in Article 4 (Accountability and Review). Article 4.6 deals with "Specific Reviews," each of which are tied to one of the commitments in the Affirmation of Commitments.

The Organizational Effectiveness Committee of the board oversees the conduct of specific reviews. The ATRT is one such specific review.

Delays in Launching Substantive Work

The call for volunteers for ATRT3 was initially put out in January, 2017. It was subsequently extended in April 2017. At the time of extension, the Board noted that multiple factors were impacting the timing of ATRT3. Because of the timing of ATRT2, the ICANN Bylaws required that the review be launched not later than October 2017. Meanwhile, the Cross Community Working Group on ICANN Accountability (CCWG-Accountability) notified that Board in 2016 that its work was overlapping substantially with the scope of ATRT reviews. The CCWG-Accountability co-chairs raised the possibility of crafting a narrow scope for ATRT3, to avoid duplication of work. At the same time, the IANA Transition was just wrapping up, and many voices within the ICANN community were speaking up about volunteer fatigue. As a result, in 2018 the Board submitted three separate papers for public comment: "Short-Term options to Adjust the Timeline for Specific Reviews," "Long-Term Options to Adjust the Timeline of Reviews," and a follow-up "Next Steps on Reviews" document.

In October 2018, the board approved a resolution to instruct ATRT3 to begin work, encouraging them to promptly submit their work plan and begin substantive work in January 2019. It also appointed Maarten Botterman to the ATRT3 team.

Review Team

The ATRT3 review team held its first in-person meeting in April 2019 to set the scope of the review and establish its work plan. Unlike previous ATRT reviews, the review team did not identify a need for an independent expert to consult. Instead, they identified four top-level issues for the review team's work parties to address:

1. Board governance and community interaction;
2. GAC interaction with the board and the broader community;
3. Review processes - how ICANN receives public input, as well as a review of implementation of recommendations from ATRT2;
4. The impact of ICANN's decision-making process as it relates to the broader Internet community, and the effectiveness of ICANN's policy development process in fostering cross community participation.

Methodology

The ATRT3 team utilized a number of fact-finding methods in order to gain a picture of the current state of ICANN's accountability and transparency initiatives:

• Review and assessment of ATRT2 implementation;
• Within each topic area, the work parties devised questions for a survey that was issued to the ICANN community, and a survey specifically aimed at ICANN structure (ACs, SOs, GNSO constituencies, and [[Regional At-Large Organization|RALOs) participation;
• Conducted interviews at ICANN 65 and ICANN 66, as well as public sessions at ICANN 66;
• Received briefings from various groups, including ICANN org's Public Participation team and the NomCom2 implementation working group;
• Reviewed the ICANN Accountability Indicators in detail;
• Reviewed many ICANN documents; and
• Requested and received numerous clarifications and details from ICANN org and staff

The survey received 88 individual responses (with 50 of those responsed completing the majority of the survey) and responses from 14 of the 17 specifically invited ICANN groups.

ICANN 66 Sessions

ATRT3's sessions at ICANN 66 took place in parallel with wider community investigation of improvements for Article 4 reviews. ICANN staff facilitated a conversation around enhancing the effectiveness of review recommendations and their implementation, where Cheryl Langdon-Orr and Pat Kane from the ATRT3 review team presented early impressions of the team in relation to review process. At its meeting with the Board Caucus, the review team presented its findings, suggestions, and recommendations to date. At the time, the team was considering three options for the reform of Article 4 reviews:

• Constitute a single permanent entity in ICANN to coordinate reviews as they currently stand and independently assess implementation of recommendations;
• Replace all Specific Reviews with one review and all Organizational Reviews with one review; or
• Replace all Specific Reviews and all Organizational Reviews with one review.

The proposals generated conversation among board members and review team members regarding how best to implement the reforms required for each option. The review team also had provisional proposals regarding diversity and representation, public comment issues, and other issue areas identified in the draft report. There was fruitful conversation about each of those issues.

On the same day, ATRT3 presented an update on its progress to the GAC. The meeting included a similar presentation of findings and provisional recommendations related to the "GAC Interaction" work track.

Finally, the review team held an engagement session for the general community. The presentation focused on the five topic areas, and elaborated on the review team's impressions regarding specific and organziational reviews. The team suggested that multiple factors may be present in the near-universal dissatisfaction with organizational reviews, and the less intense, but still substantial dissatisfaction with specific reviews:

• Lack of coordination and overlap areas under review by different teams, sometimes resulting in conflicting recommendations;
• Too many reviews in total, and too many reviews going on at once;
• Competition for ICANN resources at the recommendation & implementation phase;
• Lack of time or resources to effectively conduct the reviews;
• Failure to properly implement some recommendations and report such implementation failures; and
• Structural and bylaws-specific obstacles to having a systemic and holistic viewpoint

Draft Report

The four work parties presented findings that were consolidated into a draft report, published for public comment on December 14, 2019. The draft report proposed substantial changes to ICANN's processes around specific and organizational reviews, prioritization of improvements, and public participation.

Findings

The draft report contained a number of findings with regard to the four topic areas described above, as well as its review of ATRT2 implementation. The report's executive summary listed several notable findings:

ATRT2 Implementation - Contrary to ICANN's reporting on its website, which identified implementation as 100% complete, the ATRT3 team found that: 60% of the recommendations in the ATRT2 report had been fully implemented; 23% of the recommendations had been partially implemented; and 17% of the recommendations had not been implemented.

Article 4 Reviews

• Only 16% of the ICANN Structures responding to the survey found specific reviews (Article 4.6 of the ICANN Bylaws to be "effective" or "very effective;"
• 78% of individuals and 91% of structures agreed that specific reviews should be reconsidered or amended;
• 46% of structures found organizational reviews (Article 4.4 of the ICANN Bylaws) to be "effective" or "very effective;"
• 85% of individuals and 83% of structures agreed that organizational reviews should be reconsidered or amended.

Public Participation & Community Interaction

• An overwhelming majority (100% structures, 82% individuals) believe that the information compiled on ICANN.org should be better structured and organize to improve searchability and access;
• 50% of individuals thought that public comment proceedings were "effective" or "very effective" at gathering community input, and 88% of individuals were in favor of re-examining the concept of public comments.

Board Decision-Making and Accountability

• 54% of the structures were unaware of the existence of ICANN Accountability Indicators, and 67% of the responding structures found the Accountability Indicators "somewhat ineffective;"
• A large majority of the community believed that the ATRT3 team should make recommendations regarding prioritization;
• There was similar broad agreement that there should be a mechanism for retiring recommendations if they become moot or do not seem likely to be implemented;
• There was near-unanimous agreement that the ICANN community should have a role as a decisional participant in any mechanism or process that makes recommendations regarding prioritization of work.

GAC Interaction with the Board and Community Although not included in the executive summary as "key findings," the draft report noted a strong consensus among the structures responding to the survey that GAC interaction, transparency, and collaboration had improved significantly. Individuals had a less favorable view, but the draft report concluded that there might be closer attention to recent improvements in the GAC's communication strategies among leaders of SOs, ACs, constituency groups and RALOs.

Recommendations and Suggestions

The ATRT3 team made a distinction between recommendations and suggestions:

The ATRT3 is focused on ensuring that its recommendations meet the requirements as set out in the Operating Standards for Specific Reviews while suggestions may not necessarily meet this standard.5 The ATRT3 does not consider suggestions to be less important than recommendations. The determination if an item is a suggestion or a recommendation will be finalized in ATRT3’s final report.

The report's preamble highlighted five topics that it considered to be high priority for both community discussion and feedback, and implementation of recommendations or suggestions.

1. ATRT2 Implementation Issues - A number of suggestions specifically address the failure to implement recommendations from ATRT2. In addition, the report recommends that in-progress specific reviews appoint and empower implementation shepherds as recommended by the Operating Standards for Specific Reviews. In addition, the report recommends ensuring that implentation is actively tracked on the ICANN website, and that if any changes are made to the process of tracking and reporting implementation, that legacy web pages and other resources are updated to reflect the changes and provide the location of the new processes.
2. Prioritization of Work - The ATRT3 team took note of a variety of parallel efforts to improve prioritization and rationalization of work, and concluded that "only a community-led process can legitimately develop a system for prioritizing the implementation of reviews, CWG, and CCWG recommendations." The draft report provided suggestions regarding the creation of a development team that would create a prioritization process, as well as baseline requirements for that process.
3. Article 4 Reviews - Noting that it is best to approach reformation of the review process "holistically," the draft report provided two possible options for addressing the myriad issues relating to specific and organzational reviews:
   1. Maintain the current set of specific and organizational reviews in combination with a new oversight mechanism to manage reviews and the implementation of recommendations. The new mechanism should be housed in an Independent Accountability Office (similar to the ICANN Ombudsman), which would have responsibility for SO/AC accountability as well as the job of coordinating timing of reviews and the implementation of recommendations; or
   2. Maintain the spirit of Article 4.4 organizational reviews, but conduct them as three- to five-day leadership workshops focused on self-assessment in a context of a continuous improvement processes. The workshops would occur at least once every three years, or more often as desired by each organization subject to Article 4.4 review. Outcomes and recommendations from these workshops would be publically posted, and implementation would be tracked and reported on as well. The reports from these workshop review would feed into a new holistic organizational review of all SOs, ACs, and the NomCom. The holistic review would take place every seven years and have a maximum duration of twelve to eighteen months. The time between holistic reviews allows at least two cycles of self-assessment and improvement for each organization, as well as sufficient time to implement recommendations from the holistic review. For the Article 4.6 reviews, consolidate ATRT and the "accountability and transparency" directives of the CCT and RDS reviews into a single Accountability and Transparency review, to be conducted every seven years with a maximum duration of twelve to eighteen months. Conduct the SSR review as a three- to five-day workshop, or maintain the current review process for SSR reviews.
4. Public comment - Expand public comment activities to not only seek general input on entire documents. Clearly identify the "intended audience" for a public comment proceeding ("general community, technical community, legal experts, etc."), so that anyone may respond but specific communities are called upon to respond. Provide a summary of key questions in plain language, and include any responses to these key questions in the staff report on the public comment proceeding. Where appropriate and feasible, provide translations of a summary of the document, as well as key questions, and accept responses from speakers of official ICANN languages.
5. Accountability Indicators - Foster public awareness of the ICANN Accountability Indicators, including presentation of the indicators at an ICANN Meeting. The draft report "strongly suggest[ed]" that ICANN "rapidly undertake a serious review of its Accountability Indicators" to ensure: that they met their stated objectives; that they provided useful data; that they provided data that could inform a decision maker; and that the data for each indicator is current and up to date.

Public Comment on the Draft Report

The draft report received comments from many ICANN bodies, as well as the board. There was general support for the suggestions regarding the implementation of ATRT2 recommendations, the improvement and expansion of public comment modes and processes, and the need for prioritization. However, the creation of a community-led prioritization authority was met with skepticism and concern. The Board suggested that existing community mechanisms might be leveraged to provide ICANN's community with a voice in prioritization discussions. RySG, IPC, and other commenters feared the insularity of such an organization, as well as the perception that "ICANN insiders" were making decisions for the entire community. The two options regarding the reform of reviews each received many comments. While there was a general consensus that Option 1 was a non-starter (the ICANN Board, for example, stated that it "does not consider Option 1 (as described in the Draft Report) to be viable"), the responses to Option 2 varied widely in enthusiasm for, and faith in, such an option. Many commenters stated that the proposal would need considerable refinement, and some (including the board) suggested that the task may be too big for ATRT3's limited timeline. There was also pushback from the ICANN Board and others regarding the review team's decision to offer both "recommendations" and "suggestions."

The review team analyzed and incorporated the comments into the final version of its report, which was delivered to the board in May 2020.

Final Report

In its prologue, the ATRT3 final report highlighted several items of concern that occurred during the process of the review that the ATRT3, due to time constraints, was unable to substantially address. All of the events had some bearing on the accountability or transparency of ICANN's actions and leadership. The review team expressed hope that the events would be the subject of a future holistic review or ATRT review cycle. The events included the proposed change of ownership for the .org registry, the Expedited Policy Development Process on the Temporary Specification for gTLD Registration Data (EPDP), accountability and transparency issues related to DNS Abuse enforcement and policies, and COVID-19 "emergency" shortening of review cycles for budget and planning documents.

The report's findings were largely the same. The largest changes were seen in the recommendations made by the review team. The "suggestions" were removed, and the review team offered five recommendations that adhered to the "Operating Standards for Specific Reviews" model and methods. The five recommendations contain a large amount of detail; high-level summaries follow, but it is advisable for readers wishing to fully understand the rationales and desired outcomes for each recommendation to consult the final report:

=High Priority Recommendations

1. Amending Article 4 Reviews:

• Specific Reviews should be handled as follows:
  • Suspend the RDS/WHOIS review until the EPDP on the Temporary Specification is complete;
  • Conduct one further CCT review, initiated within two years of the addition to the root of new gTLDs from a new application round. Limit the timeframe of the review to one year, and ensure that a framework for data collection is already in place and that a complete data set is available prior to the initiation of work;
  • Upon the completion of SSR2, suspend any further SSR2 reviews until the next ATRT review (or any new review that takes over the obligations of the ATRT) makes a determination whether SSR reviews should be amended, terminated, or kept as is. The ICANN Board will have the authority to launch an SSR review if it deems it necessary; and
  • ATRT reviews to continue, with recommended improvements, and with the next ATRT review starting no later than two years after the approval of the board of any recommendation from the first Holistic Review (see below)
• ICANN should establish and perform a new "Holistic Review" no later than one year after board approval of any recommendation from ATRT3. After that, alternate the timing of ATRT and Holistic Reviews, so that each review begins no later than twenty-four (ATRT) or thirty months after the approval of the first recommendation contained in the other review. (As in the draft report, this is designed to allow at least two "continuous improvement" cycles to occur within organizations between Holistic Reviews.)
  • No other reviews should be launched during the pendency of a Holistic Review;
  • Holistic Reviews shall last a maximum of eighteen months and operate in accordance with the Operating Standards for Specific Reviews;
  • Holistic Reviews will focus on: 1) continuous improvement efforts of SOs, ACs, and the NomCom; 2) Inter-organizational communication and collaboration; 3) accountability of the SOs, ACs, or contituent parts to their members and constituencies; and 4) reviewing the entirety of the ICANN organizational structure to determine if all the constituent parts still serve a purpose, or if there are opportunities to alter the structure(s) and operations of ICANN to better represent the community and improve effectiveness
• Organizational Reviews should be transformed into "continuous improvement programs" within each structure currently subject to an Article 4.4 review.
  • Allowing for some flexibility so that each organization can optimize its approach to continuous improvement, each program will include annual satisfaction surveys of members/participants, annual assessment of the improvement program (which may either be self-directed or facilitated), and budgetary and staff support that at a minimum matches the operational support for the current organizational review process.
  • Results of the annual surveys and assessments will be published and will be reviewed during the next Holistic Review.

2. Prioritization of Review and CCWG Recommendations: Create an annual process of prioritization that gathers representatives from ICANN Board, ICANN org, and any SOs or ACs that wish to participate (one representative from each). This panel would operate by consensus, and would integrate into existing financial and strategic planning processes. The panel would be charged to establish a priority order for recommendations resulting from reviews, cross-community working groups, and any other community-sourced process the Board deems appropriate. Similar to the Operating Standards for Specific Reviews, the prioritization process shall employ the following decision-making guidance:

• Relevance to ICANN’s mission, commitments, core values, and strategic objectives;
• Value and impact of implementation;
• Cost of implementation and budget availability;
• Complexity and time to implement;
• Prerequisites and dependencies with other recommendations; and
• Relevant information from implementation shepherds (or equivalents)

Medium Priority Recommendations

3. Accountability Indicators & Strategic and Operational Planning Transparency: Scrap "Accountability Indicators" and instead provide plain language descriptions of the Board's strategic objectives, goals, and operating plans. Measure the outcomes of such plans and publish results in annual reports. Publish a summary report at the end of each five-year strategic planning cycle, with a description of objectives and goals, initiatives intended to meet those goals, and the success of such initiatives.

Low Priority Recommendations

4. Public Comment: Improve public comment proceedings as recommended in the draft report: provide a plain-language summary of the document being published; list key questions raised by the document and for which answers are sought; where appropriate and feasible, translate the summary and key questions into the official ICANN languages; and if relevant, identify the intended audience of the document or the key questions.

5. ATRT2 Implementation: Review the implementation of ATRT2 Recommendations in light of ATRT3’s assessment of these and complete their implementation subject to prioritization.

Public Comment on the Final Report

In addition to publishing the final report for public comment, the Board specifically requested feedback regarding a section of the transmission letter from ATRT3:

...ATRT3 is proposing significant changes to Organizational Reviews and Specific Reviews. ATRT3 strongly suggests that the ICANN Board implement a moratorium on launching any new Organizational and Specific Reviews until it has made a decision on this recommendation.

The report generated a number of detailed public comments.

Board Action and Implementation

Following the public comment period, in November 2020, the ICANN Board approved a set of recommendations from the final report, "subject to prioritization."

ATRT3 is, as of July 2021, in the implementation phase.

Impact on Organizational Reviews

In June 2021, the board agreed to defer GNSO3, in large part because of the recommendations of the ATRT3 final report. The board also effectively place a hold on all pending organizational reviews until such time as the ATRT3 recommendations were integrated into an implementation plan:

Resolved,...the Board determines that it is not feasible to initiate the GNSO3 Review at this time. The Board is deferring the initiation of the GNSO3 Review until such time as the Board, community and org better understand the impact of the ATRT3 Recommendations on the next Organizational Review cycle, as well as in consideration of the current community workload. Resolved,...the Board directs the ICANN President and CEO, or his designee, to develop a comprehensive plan for the timing and conduct of the next Organizational Review cycle, taking into consideration the timing of the implementation of the ATRT3 recommendations. This plan should be developed in consultation with the ICANN community, and ICANN org shall provide periodic updates to the Board on progress towards such plan.

Governmental Advisory Committee. Considers and provides advice on the activities of ICANN as they relate to concerns of governments, particularly matters where there may be an interaction between ICANN's policies and various laws and international agreements or where they may affect public policy issues (ICANN Bylaws).

Nominating Committee