First SSAC Organizational Review

The First SSAC Organizational Review was initiated in June 2008 and completed in June 2010, with the implementation of improvements continuing through 2011.

Background

Article 4.4 of the ICANN Bylaws requires periodic review of all supporting organizations and advisory committees, as well as the Nominating Committee. The bylaws state three objectives for the review:

1. to determine whether that organization, council or committee has a continuing purpose in the ICANN structure;
2. if so, whether any change in structure or operations is desirable to improve its effectiveness; and
3. whether that organization, council or committee is accountable to its constituencies, stakeholder groups, organizations and other stakeholders.

Organizational reviews are conducted by independent examiners, selected through a competitive bidding process. The independent examiner works in consultation with a working group assembled by the board, who will act as implementation shepherds once the final report of the independent examiner is submitted. The review parameters are set by the ICANN Board, and those parameters as well as other avenues of inquiry are typically included in the request for proposals (RFP) for independent examiners. Reviews can take anywhere from three to five years to complete. The full review process includes seven phases, including the implementation of recommendations from the review. Reviews must be conducted at least every five years, measuring from the date that the final report of the previous review was accepted by the ICANN Board. The Security and Stability Advisory Committee is one of the organizations subject to the review requirements of Article 4.4.

Initiation

The ICANN Board formed the SSAC1 Review Working Group (RWG) in June 2008, naming the following members to the RWG: Dennis Jennings (Chair), Robert Blokzijl, Reinhard Scholl and Suzanne Woolf. In September 2008, the board posted an RFP for an independent reviewer to conduct the review of the SSAC. The Terms of Reference (ToR) for the RFP included 30 questions related to the two principal questions in the Article 4.4 of the ICANN Bylaws at the time: does the SSAC have a continuing purpose in the ICANN structure; and is there any change in structure or operations that could improve the SSAC's effectiveness?

At a special meeting in October 2008, the board approved the charter for the RWG.

Independent Examiner

In the lead-up to ICANN 33 in Cairo, JAS Communications was selected as the independent examiner for SSAC1. This permitted JAS the opportunity to begin work at the meeting.

Methodology

JAS's standard approach to reviews of technical organizations was put to the test, as the IE's draft final report makes clear:

Data collection began in Cairo and it quickly became apparent that organizational issues would dominate our study. With so much uncertainty around scope and mission, we found it difficult to apply typical group performance metrics because there was no baseline in terms of a clear charter, specific tasking, or even agreed upon strategic direction. Put another way, it is difficult to evaluate the performance of a group when it is not clear what they are supposed to be doing. It was very clear that SSAC was working diligently, but JAS was asked whether the work it was doing was the right work.

JAS determined that the assessment would largely be based on qualitative findings. In order to ensure that there was a basis for community input on quantifiable performance metrics, JAS worked with an organizational behavioral expert to create a "custom survey instrument to produce quantitative data around specific recurring organizational questions unearthed during the interviews." The survey was deployed with an eye toward verifying qualitative observations of the review team, and to provide a dataset against which demographic and cultural correlations could be tested.

After a series of rounds of hypothesis testing and data collection, JAS submitted a draft version of its findings, analysis, and recommendations to selected individuals within the ICANN community for feedback.

Draft Final Report

Following receipt of feedback from the limited-distribution draft, JAS published its Draft Final Report for public comment on February 16, 2009. The IE's draft final report identified three core themes within their findings:

• Lack of organizational clarity and charter;
• Lack of formality leading to concerns about transparency; and
• Perceived and actual conflicts of interest.

In particular, the longstanding relationships of SSAC members, most of which predated ICANN involvement, led to a collegial and informal culture that affected both the outside perception of the SSAC and internal and external understanding of the SSAC's role within the ICANN ecosystem. As an example, JAS noted that policies and procedures around conflicts of interest were nearly nonexistent:

An unreleased internal draft SSAC policies and procedures document provided to JAS contains the following language regarding perceived and actual conflicts of interest:

The committee does not ordinarily concern itself with conflicts of interest. All members are always permitted to participate in all activities. However, the committee may elect to state potential conflicts of interest as an integral part of any publication if it deems this to enhance the final result. An individual may elect not to participate in any activity of the committee at his or her own discretion.

We observe that this is an accurate characterization of SSAC's current views on perceived or actual conflicts of interest; in practice, we find little to no formal attention to the issue.

Despite the perceptual issues, the importance and quality of the SSAC's work was universally acknowledged. The executive summary of the draft final report stated that the SSAC was "functioning, functioning well, and filling a relevant purpose." In addition, the report called out the community's appreciation and respect for the SSAC's handling of the Verisign Site Finder situation. The draft report offered thirty-four recommendations to improve organizational and communication clarity inside the SSAC and in cross-community work, address gaps in formalized policy or procedures within the SSAC, and generally alleviate the "growing pains" experienced by the SSAC in the first years of its existence.