Second RSSAC Organizational Review

The Second RSSAC Organizational Review was conducted between 2017 and 2020 by Interisle Consulting Group, LLC.

Background

Pursuant to Article 4, Section 4.4 of the ICANN Bylaws, ICANN initiated a second RSSAC independent review process on April 19, 2017. The Operational Effectiveness Committee of the ICANN Board drafted and distributed a Request for Proposals for this review.

Scope of Review

In both the announcement and the RFP, ICANN presented a broad scope of subjects for review. For example, the announcement stated that the goal was a "comprehensive assessment" of the RSSAC, "includ[ing], but [...] not limited to:

• The continued purpose of RSSAC within the ICANN structure;
• How effectively RSSAC fulfills its purpose;
• Whether any change in structure or operations is needed; and
• The extent to which RSSAC as a whole is accountable to the wider ICANN community."

The RFP echoed the need to investigate those issues, and also provided a "non-comprehensive" list of proposed questions to be addressed regarding the RSSAC's composition, mission, procedures, transparency, and communication. On April 18, 2017, on behalf of RSSAC, Carlos Reyes submitted comments to ICANN requesting more specificity in some of the proposed questions. In correspondence back to the RSSAC, ICANN submitted a draft "Self Assessment" tool for the committee utilizing the RFP's proposed questions as part of the assessment template, with changes to some questions that appear to align with the RSSAC requests for clarity.

Selection of Independent Examiner

After accepting and reviewing proposals through the RFP process, ICANN contracted with Interisle Consulting Group, LLC to conduct the second independent review of RSSAC. Interisle assembled a team of three consultants for the review, Lyman Chapin, Jim Reid, and Colin Strutt

Assessment Report & Findings

Interisle's review included attendance at RSSAC meetings, ICANN meeting sessions, and conference calls during the autumn of 2017 and winter of 2018. In addition, they conducted 48 interviews with individuals connected to or working within the RSSAC. At the end of 2017, Interisle conducted an online survey "to collect input from those who have interacted with RSSAC and/or have ideas for ways to improve it." In addition to publishing notice of the survey, ICANN specifically solicited responses from the RSSAC Caucus. The survey, which did not require participants to identify themselves, garnered 74 visits and 39 responses.

Other data utilized by Interisle came from:

• RSSAC's Self-Assessment, completed in September 2017;
• publicly available documents from a wide variety of sources that discuss the RSSAC and related activities; and
• Interisle's "own extensive knowledge of ICANN, the RSSAC, and the DNS root server system."

Interisle subjected this data to a structured qualitative analysis and presented its findings in an Initial Assessment Report to ICANN. The draft assessment report was the topic of meetings between members of RSSAC's Review Work Party and Interisle's consultants.

The Assessment Report listed a total of forty-two findings in response to the issues and questions posed by ICANN. Of these, Interisle identified eight principal findings, which they highlighted in the report and its executive summary of the assessment:

1. The ongoing RSSAC reformation that began in 2013—revised RSSAC charter, new operating procedures, and creation of the RSSAC Caucus—has substantially improved the structure and operation of the RSSAC.
2. The RSSAC has become more open, transparent, and accessible since the last review, but this has not been widely recognized by outside observers.
3. As the only visible interface between ICANN and the RSOs, the RSSAC is expected to deal with every root service issue that arises within ICANN, whether or not the issue is properly within its scope.
4. The RSSAC’s ability to serve as a shared space for RSO–ICANN communication and cooperation is complicated by a persistent legacy of distrust of ICANN by some of its members.
5. The current RSSAC membership model excludes non-RSO participants and their different skills and perspectives.
6. The RSSAC’s continuing purpose in the ICANN structure may include serving as the focal point for issues of mutual concern to ICANN and the RSOs, such as future operational and funding scenarios for serving the root.
7. Because RSSAC members do not agree on who its stakeholders should be, it is not clear for what and to whom it should be accountable.
8. The relative roles and responsibilities of the RSSAC, the RSSAC Caucus, the RZERC, and the SSAC are unclear to both outsiders and insiders.

Response to Assessment Report

The Assessment Report was submitted for public comment and response prior to the development of recommendations. At an open meeting on March 14, 2018 at ICANN 61, Lyman Chapin explained:

And this reflects a process that MSSI has fairly recently adopted for doing organizational reviews, which is to divide them into two distinct phases. The first phase is an assessment phase in which the idea is not to anticipate what recommendations might be made but simply to report on what the independent examiner found after conducting interviews, and surveys, and document reviews, and so forth.

Chapin presented the principal findings at the ICAAN 61 meeting, and received no questions from the attendees.

On March 28, 2018, RSSAC issued RSSAC032: Feedback on the Independent Review of the Root Server System Advisory Committee (RSSAC) Assessment Report for Public Consultation. RSSAC032 begins with a discussion of the ICANN Bylaws and the committee's expectation of the focus of the review:

RSSAC’s interpretation of the stated assessment report’s purpose is that an organizational review would look at the organization – its chairs, its procedures, its guiding documents including charter and bylaws, its meetings, and the publications it has produced.

This interpretation substantially diminished the scope of ICANN's announcement of the review, and the parameters defined in the RFP. Based on that narrow definition of scope, the committee found "three...surprising categories of findings" that they viewed as outside the purpose of the review:

However, we found that the initial assessment report included three additional surprising categories of findings.

i. Out of Scope Findings: Findings related to items that RSSAC could never include within its charter, such as items related to the technical day-to-day operation of the Root Server System (RSS) or the confusion surrounding a contributor’s lack of understanding of multiple ICANN bodies such as SSAC. ii. Factually Incorrect Findings: Findings that are factually incorrect about RSSAC and/or the RSS.

iii. Tone: Findings and tone related to public opinion and sensational anonymous quotes about RSSAC or its purpose, but not providing any other insight.

The committee argued that it is only empowered to do that which is within its charter, and that any findings or recommendations should specifically address the tenets of that charter, the ICANN Bylaws, and RSSAC's "structure and procedures, [...] in line with relevant process documents and bylaws."

Final Report & Recommendations

Interisle received and incorporated feedback from the Assessment Report, and presented draft recommendations to the Review Work Party on April 3, 2018. The draft recommendations were the subject of a teleconference meeting between Interisle, MMSI, and the RWP on April 12, 2018.

On April 20, RSSAC issued its feedback on the draft recommendations to Interisle. Interisle responded on April 25 with comments appended to the RWP's feedback document.

On May 1, 2018, Interisle released its Draft Final Report for public comment. The final report listed six principal recommendations; there was no change from the draft to the final report.

1. Modify the RSSAC membership criteria to allow the RSSAC to recruit a variety of skills, perspectives, and interests that include but are not limited to those available from the root server operator organizations.
2. Resolve the apparent mismatch between the charter and operational procedures of the RSSAC and the requirements and expectations of the ICANN Board and Community for interaction with the root server system.
3. Formalize the responsibilities of the RSSAC to the ICANN Board and Community in a work plan that is periodically reviewed and published, and hold the RSSAC accountable for work plan deliverables.
4. Develop and implement a leadership training and succession plan.
5. Engage more actively with the rest of ICANN and its Community.
6. Clarify the role and responsibility of the RSSAC with respect to other groups with adjacent or overlapping remits, including the SSAC, the RZERC, and the RSSAC Caucus.

The Draft Final Report received public comments from a variety of sources. On June 8, 2018, RSSAC published RSSAC036: RSSAC Statement on the Draft Final Report of the Second Organizational Review of the RSSAC. The RSSAC repeated its objections to the scope and focus of the review in its statement. ALAC published a response that commented on three of the proposed recommendations. Notably, its response to Recommendation 2 strongly favored action to ensure that ICANN had a sufficiently robust relationship with RSOs:

Although there is no clear way to address the issue, since ICANN has a part to play in ensuring that the DNS is a trusted and reliable resource, then it must have the ability to interact with all players who have a role in carrying that out and that must include the Root Server Operators. Simply lowering expectations does not address the issue. Whether this is done by widening the scope of the RSSAC or through some other mechanism is less important than noting that the current chasm must be bridged.

The Business Constituency agreed: "The BC requests that this recommendation be prioritized, noting it is 'highly concerned...that RSSAC does not consider itself accountable to ICANN Board and Community.'"

RSSAC, for its part, "conclude[d] that neither the ICANN Board nor the ICANN community harbor the mismatch raised in the draft final report."

After the public comment period closed, the draft final report was a topic of an open session at ICANN 62. Interisle released its final report on July 2, 2018.

Implementation Phase

The ICANN Board instructed RSSAC to develop a Feasibility Assessment & Initial Implementation Plan in the wake of the publication of the Final Report. The RWP released its plan on October 2, 2018.

Implementation of the recommendations continued until December 2020, when RSSAC submitted its final status report on implementation. The RSSAC2 process has been marked as complete on the RSSAC review dashboard.