Second SSAC Organizational Review

The Second SSAC Organizational Review (SSAC2) was initiated in 2017 and concluded in 2019, with implementation of improvements continuing through 2021.

Background

Article 4.4 of the ICANN Bylaws requires periodic review of all supporting organizations and advisory committees, as well as the Nominating Committee. The bylaws state three objectives for the review:

1. to determine whether that organization, council or committee has a continuing purpose in the ICANN structure;
2. if so, whether any change in structure or operations is desirable to improve its effectiveness; and
3. whether that organization, council or committee is accountable to its constituencies, stakeholder groups, organizations and other stakeholders.

Organizational reviews are conducted by independent examiners, selected through a competitive bidding process. The independent examiner works in consultation with a working group assembled by the board, who will act as implementation shepherds once the final report of the independent examiner is submitted. The review parameters are set by the ICANN Board, and those parameters as well as other avenues of inquiry are typically included in the request for proposals (RFP) for independent examiners. Reviews can take anywhere from three to five years to complete. The full review process includes seven phases, including the implementation of recommendations from the review. Reviews must be conducted at least every five years, measuring from the date that the final report of the previous review was accepted by the ICANN Board. The Security and Stability Advisory Committee is one of the organizations subject to the review requirements of Article 4.4.

Initiation

The second SSAC review was postponed in July 2015 by the ICANN Board, and was due to start in June 2017. In March 2017, the OEC reached out to the SSAC to determine whether the group believed that it would be ready for the review to start, or if a deferral would be desirable. After receiving confirmation from SSAC leadership that they were prepared to engage in the review, the OEC forwarded a recommendation to the board to initiate the review. The board initiated SSAC2 in May 2017. The SSAC2 Review Work Party (RWP) held their first meeting in February 2018, contemporaneous with the selection of the Independent Examiner.

Unaddressed Recommendations from SSAC1

After the RFP for an independent examiner had closed, during the SSAC's self assessment process, it was noted that Recommendation 10 from the First SSAC Organizational Review had not been implemented - the ICANN Board had not "stud[ied] the issue of paying a stipend or honorarium to to SSAC leadership and members." As the OEC noted in its meeting minutes:

The OEC's concerns are two-pronged:

* Non-implementation of Recommendation 10 from the previous SSAC Review. * The possibility that there are other gaps in implementation from other Organizational Reviews from the past review cycle.

Given that the OEC is responsible for the review and oversight of all Reviews, it is imperative that the OEC take a systemic approach in addressing implementation gaps arising from the recommendations of previous reviews. The OEC will provide recommendations to the Board on ways to address the gaps once ICANN organization has completed an analysis of implementation gaps from the past review cycle.

Independent Examiner Findings & Recommendations

The RFP for an independent examiner was posted in July 2017. The deadline for proposals was subsequently extended into late August. ICANN does not typically reveal the specifics of contracting rounds, but it is likely that the deadline was extended to ensure receipt of as many proposals as possible. In the end, only two proposals were received. The OEC had also emphasized to ICANN staff during the preparation of the RFP and Terms of Reference (ToR) that technical proficiency was strongly preferred for the reviewer. In the event that the candidate pool did not meet the "gating elements" that were set up in the proposal scoring process, it was initially planned to re-start the RFP process at a later date. In late October, the OEC decided to coordinate with the CTO of ICANN org to review the technical qualifications of the finalists. In February 2018, the OEC had received the opinion of the ICANN CTO regarding the finalist proposal team's technical qualifications, and recommended that the board begin the process of engaging the finalist. The Analysis Group was announced as the Independent Examiner in February 2018.

Methodology

The Analysis Group used four main fact-finding strategies: they attended SSAC meetings at ICANN 61; interviewed members of the ICANN community; conducted a survey aimed at the ICANN community; and conducted a comprehensive review materials published by the SSAC. They also reviewed the implementation of recommendations from SSAC1. Forty-two interviews were conducted, and the survey received 52 complete responses.

IE Assessment Report

The Independent Examiner's assessment report was published in June 2018. Following ICANN's new model for organizational reviews, The Analysis Group first published an assessment report for public comment. The assessment contained only the IE's findings based on its review, and reserved recommendations for the final report. The Analysis Group's report listed twenty-two principal findings in six different topic clusters.

Effectiveness of the SSAC:

1. The SSAC is widely acknowledged to be very important to the overall mission of ICANN. The role of the SSAC is closely aligned with ICANN’s mission.
2. Individuals throughout ICANN largely agree that the SSAC is successful in providing high quality advice on a broad variety of relevant SSR issues.
3. There is some concern among members of the SSAC that advice provided to the ICANN Board is not acted on in a timely manner. Similarly, there is some concern among members of the ICANN Board that the advice of the SSAC cannot be provided quickly.
4. The role of the SSAC Board Liaison is key in ensuring the Board is able to interpret and understand the advice provided by the SSAC.
5. Individuals suggested that the largest impediment to the SSAC’s success is the fact that the organization is volunteer-based, yet has a large amount of work to do. SSAC volunteers express they have been subject to an increasing requests, both in number and in scope.
6. The SSAC’s process of generating advice is collegial and is generally effective. Some best practices and suggestions for improvement have been gathered as part of the review process.

Topic Selection

7. The SSAC is well prepared to deal with emerging security threats. It was noted that the SSAC does not have formal procedures geared towards identifying emerging threats as an input to setting research priorities. Some interviewees indicated that as threats continue to increase in number and in complexity, there could be value in developing processes by which the SSAC could more formally review the security ecosystem as part of its topic selection.

Interactions with Other SOs/ACs

8. There is a need for individuals with an understanding of SSR-related issues to take part in policy development, and some members of the community suggest the SSAC or its members (as individuals) should play that role. Others state that the SSAC should play a technical advice, audit, and verification role, and that assisting more directly in policymaking itself is not the point of the SSAC.
9. Many individuals both inside and outside of the SSAC identified that creating more interaction with other ICANN SOs/ACs should be an area of focus for the SSAC. The SSAC has been making strides to communicate more frequently and to forge stronger relationships with other SOs/ACs.

SSAC Size and Membership

10. The SSAC has wide-ranging and deep technical expertise. The SSAC does not compromise its high technical requirements when vetting potential members, though some interviewees caution that the SSAC should avoid defining “technical” too narrowly, as SSR issues can be both technical and interdisciplinary.
11. The SSAC’s size of roughly 40 members appears to be appropriate given tradeoffs in the size of the SSAC, though some thought that additional members with additional perspectives would be valuable.
12. The SSAC does not undergo active or targeted recruiting, but rather recruits informally based on need and the existing network of SSAC members. Many interviewees would like to see improvements in the SSAC’s recruiting process, but they are cautious about the burden such processes might place on the SSAC’s volunteers.
13. The SSAC is perceived to lack geographical and gender diversity and is comprised mostly of male individuals from the U.S. and Europe. While many individuals do not feel it is appropriate for a technical body to have “diversity for diversity’s sake” at the expense of technical skill, several SSAC and non-SSAC members indicated that perspectives from other regions and types of individuals would be beneficial.
14. The membership review process used by the SSAC today is clearer and more transparent than it had been in the past, and when flaws have been identified, the process has undergone (and continues to undergo) improvements.
15. The SSAC’s term length of three years for non-leadership members is generally considered to be appropriate, and almost all interviewees and survey respondents indicated that there should be no term limits for SSAC’s non-leadership members.
16. The SSAC’s term length of three years for leadership members is generally considered to be appropriate (Chair and Vice Chair). There exists much more variation in individuals’ views as to the appropriate term limit, if any, for SSAC leadership.

Transparency and Accountability

17. The SSAC is generally seen to be less transparent than other parts of ICANN. While most interviewees understand this to be necessary given the sensitive nature of security risks, many would like to see a more transparent SSAC.
18. The SSAC is accountable directly to the ICANN Board, and through it to the wider ICANN community.
19. The SSAC has mechanisms to allow for the disclosure of conflicts of interest, and members seem comfortable identifying other’s potential conflicts of interest. Some SSAC members indicated that, by nature of the technical expertise required to contribute to the SSAC, limited conflicts of interest are unavoidable. Other SSAC members believe more can be done to limit potential conflicts.
20. The SSAC’s website includes important information that assists with transparency. Community members suggested other items that may be useful to include.

Prior Review Implementation and Self-Improvement

21. The SSAC has taken clear steps to implement the recommendations that were accepted after its prior review. With minor caveat, the SSAC has been successful in these implementations.
22. The SSAC values self-improvement and makes clear efforts to continually improve even outside of the formal review process.

The findings were supported by survey and interview responses, as well as the review of SSAC1 implementation.

Public Comment on the Assessment Report

In addition to solicitation of written comments, The Analysis Group presented its findings at a session at ICANN 62 in Panama City. The Analysis Group also held a webinar on July 12, 2018 to present its findings and invite comment on the assessment report. Both presentations guided attendees through the IE's methodology and provided demographics of interview subjects and survey respondents. In both cases, the Analysis Group also presented a summary of its findings. Most comments at the session at ICANN 62 were from SSAC members, who largely requested clarity about certain survey responses and other findings. The webinar received no questions except one from John Poole, who sought confirmation that written comments would be considered in the development of the final report.