First SSAC Organizational Review

The First SSAC Organizational Review was initiated in June 2008 and completed in June 2010, with the implementation of improvements continuing through 2011.

Background

Article 4.4 of the ICANN Bylaws requires periodic review of all supporting organizations and advisory committees, as well as the Nominating Committee. The bylaws state three objectives for the review:

1. to determine whether that organization, council or committee has a continuing purpose in the ICANN structure;
2. if so, whether any change in structure or operations is desirable to improve its effectiveness; and
3. whether that organization, council or committee is accountable to its constituencies, stakeholder groups, organizations and other stakeholders.

Organizational reviews are conducted by independent examiners, selected through a competitive bidding process. The independent examiner works in consultation with a working group assembled by the board, who will act as implementation shepherds once the final report of the independent examiner is submitted. The review parameters are set by the ICANN Board, and those parameters as well as other avenues of inquiry are typically included in the request for proposals (RFP) for independent examiners. Reviews can take anywhere from three to five years to complete. The full review process includes seven phases, including the implementation of recommendations from the review. Reviews must be conducted at least every five years, measuring from the date that the final report of the previous review was accepted by the ICANN Board. The Security and Stability Advisory Committee is one of the organizations subject to the review requirements of Article 4.4.

Initiation

The ICANN Board formed the SSAC1 Review Working Group (RWG) in June 2008, naming the following members to the RWG: Dennis Jennings (Chair), Robert Blokzijl, Reinhard Scholl and Suzanne Woolf. In September 2008, the board posted an RFP for an independent reviewer to conduct the review of the SSAC. The Terms of Reference (ToR) for the RFP included 30 questions related to the two principal questions in the Article 4.4 of the ICANN Bylaws at the time: does the SSAC have a continuing purpose in the ICANN structure; and is there any change in structure or operations that could improve the SSAC's effectiveness?

At a special meeting in October 2008, the board approved the charter for the RWG.

Independent Examiner

In the lead-up to ICANN 33 in Cairo, JAS Communications was selected as the independent examiner for SSAC1. This permitted JAS the opportunity to begin work at the meeting.

Methodology

JAS's standard approach to reviews of technical organizations was put to the test, as the IE's draft final report makes clear:

Data collection began in Cairo and it quickly became apparent that organizational issues would dominate our study. With so much uncertainty around scope and mission, we found it difficult to apply typical group performance metrics because there was no baseline in terms of a clear charter, specific tasking, or even agreed upon strategic direction. Put another way, it is difficult to evaluate the performance of a group when it is not clear what they are supposed to be doing. It was very clear that SSAC was working diligently, but JAS was asked whether the work it was doing was the right work.

JAS determined that the assessment would largely be based on qualitative findings. In order to ensure that there was a basis for community input on quantifiable performance metrics, JAS worked with an organizational behavioral expert to create a "custom survey instrument to produce quantitative data around specific recurring organizational questions unearthed during the interviews." The survey was deployed with an eye toward verifying qualitative observations of the review team, and to provide a dataset against which demographic and cultural correlations could be tested.

After a series of rounds of hypothesis testing and data collection, JAS submitted a draft version of its findings, analysis, and recommendations to selected individuals within the ICANN community for feedback.

IE Draft Final Report

Following receipt of feedback from the limited-distribution draft, JAS published its Draft Final Report for public comment on February 16, 2009. The IE's draft final report identified three core themes within their findings:

• Lack of organizational clarity and charter;
• Lack of formality leading to concerns about transparency; and
• Perceived and actual conflicts of interest.

In particular, the longstanding relationships of SSAC members, most of which predated ICANN involvement, led to a collegial and informal culture that affected both the outside perception of the SSAC and internal and external understanding of the SSAC's role within the ICANN ecosystem. As an example, JAS noted that policies and procedures around conflicts of interest were nearly nonexistent:

An unreleased internal draft SSAC policies and procedures document provided to JAS contains the following language regarding perceived and actual conflicts of interest:

The committee does not ordinarily concern itself with conflicts of interest. All members are always permitted to participate in all activities. However, the committee may elect to state potential conflicts of interest as an integral part of any publication if it deems this to enhance the final result. An individual may elect not to participate in any activity of the committee at his or her own discretion.

We observe that this is an accurate characterization of SSAC's current views on perceived or actual conflicts of interest; in practice, we find little to no formal attention to the issue.

Despite the perceptual issues, the importance and quality of the SSAC's work was universally acknowledged. The executive summary of the draft final report stated that the SSAC was "functioning, functioning well, and filling a relevant purpose." In addition, the report called out the community's appreciation and respect for the SSAC's handling of the Verisign Site Finder situation. The draft report offered thirty-four recommendations to improve organizational and communication clarity inside the SSAC and in cross-community work, address gaps in formalized policy or procedures within the SSAC, and generally alleviate the "growing pains" experienced by the SSAC in the first years of its existence.

Public Comment on Draft Final Report

JAS presented its findings and recommendations in a workshop session at ICANN 34 in Mexico City. The comments receieved at the session were largely from current or former members of the SSAC, and expressed varying degrees of skepticism about the value and clarity of the findings and recommendations.

In addition to the session in Mexico City, two written comments were received on the draft report during the public comment period. Hiro Hotta expressed admiration for the SSAC and suggested that diverse viewpoints might even further improve the SSAC's abilities. ISOC submitted a statement in agreement with many of the recommendations, particularly the implementation of a conflicts of interest policy, and emphasizing focus on SSAC's core mission as an advisory committee.

IE Final Report

JAS submitted its final report on May 15, 2009. The final report acknowledged the comments received, as well as feedback from the RWG and other sources in the time between the draft final report and the final report. JAS noted that the feedback and input had resulted in several changes and refinements, including a stronger focus on the big picture issues presented by the data, and the simplification of some of the more heavily detailed recommendations in favor of addressing those big picture issues. The findings and recommendations of the final report were largely the same.

Public Comment on the IE's Final Report

The final report received only one substantive written comment, which encouraged the SSAC to work more closely with other constituencies and "improve its overall transparency."

Review Working Group Reports

After receipt of the IE's final report, the SSAC membership engaged in a self-assessment exercise, resulting in a report to the RWG in June 2009 During ICANN 35 in Sydney, the RWG discussed the final report from JAS as well as the SSAC self-assessment report, and began to formulate proposals for the implementation of improvements recommended in each report.

RWG Draft Report

In its draft report, the RWG collated the IE's recommendations, the SSAC's response, and the working group's consensus opinion on each recommendation from the IE's final report, as well as the additional recommendations from the SSAC Self-Assessment:

Rec. Number(s)	Recommendation(s)	SSAC Position	RWG Conclusion
IE 1-4	1. Maintain a technical advisory body2. SSAC maintain its identity as an AC to the board3. Do not combine SSAC and RSSAC4. SSAC members should not be required to sign confidentiality agreements or duty of loyalty agreements	Agree	Agree
IE 5	Amend SSAC charter to prevent dealings with confidential or proprietary information unless directly guided to do so by the Board	Disagree - this information is often useful to analyze safety and security issues	Disagree with recommendation - "SSAC has a legitimate right to ask for access to confidential or proprietary information that is needed to fill its mandate, requests that need to be motivated by appropriate reasons.
IE 6	Amend SSAC charter to exclude involvement with or review of ICANN internal operations except as directed by the Board	Disagree - "Where contracts or normal employment practices (e.g. the name of an employee who made an error) prohibit disclosure, SSAC should not have special access, but review and access to information on operational function such as root system provisioning and root server operations, these functions should be within SSACís purview."	SSAC is entitled to advise when it considers that ICANN's internal operations threaten the safety and stability of the DNS. ICANN's internal operations, including IANA functions, should report any security issues, as well as report annually on measures adopted to prevent threats that may be caused by ICANN's operations. Board can share these reports with SSAC as they deem appropriate.
IE 7	Correct the perception of SSAC "independence" through improvements in formality, transparency, and increased Board interaction	Disagrees that there are perceptions of independence	No specific measures to be implemented on this recommendation - many subsequent recommendations propose changes to the SSAC's processes & communication
IE 8	Amend SSAC charter to require that the SSAC Chair and the SSAC Board Liaison be different people	Disagrees with either requiring or prohibiting one person to wear two hats	Disagrees with recommendation & agrees with SSAC's comment
IE 9	ICANN reimburse expenses for SSAC Chair to travel to meetings when relevant	Agree	Agree
IE 10	ICANN Board to investigate the possibility of paying a stipend or honorarium to SSAC chair or members	Agree	Agree
IE 11	Amend SSAC charter to specifically include non-technical threats to the security & stability of the DNS	Disagree - SSAC considers such risks now, but should focus on objective facts	Disagree - SSAC has shown that it can evaluate the technical impacts of non-technical developments - no need to amend charter
IE 12	SSAC to maintain focus on developing and sharing knowledge of new and evolving risks; specifically avoid tactical involvement in response or mitigation	Agree	Agree - no specific implementation steps needed
IE 13	SSAC leadership should improve sensitivity to political and business issues:* provide a "heads-up" when uncomfortable situations might reasonably ensue to avoide "blind-siding" individuals and orgs* recognize that, as an advisory body, SSAC's goal is to provide the best advice possible; there is, however, no requirement for anyone to follow that advice* recognize that ICANN has complex business and contractual relationships that may preclude following SSAC's advice* maintain the value of SSAC's brand by continuing to conduct itself with the utmost professionalism and integrity	Agree, so long as the proffered advice is understood to be limited to:* avoid blind-siding individuals* recognize that there is no requirement for anyone to follow SSAC advice* SSAC's guidance may conflict with contractual obligations* SSAC must continue to conduct itself with the highest level of professionalism and integrity	Agrees with the comment formulated by SSAC, and does not consider additional action to be necessary
IE 14	Amend SSAC charter to give guidance to focus on policy and strategic matters, and to avoid tactical operational issues	Disagree - current charter is adequate on this subject	Disagree - agrees with SSAC that the current charter makes this distinction
IE 15	In conjunction with the ICANN Board, staff, and public consultation, SSAC should undertake an annual planning process to review the previous year, determine SSAC's research and publication agenda, define membership outreach strategy, and list resource requirements for the coming year. Submit plan to Board for approval	Agrees with the need for planning, but "it should not be constrained to annual cycles"	Agrees that SSAC needs to create a lightweight planning process
IE 16	SSAC should keep and publish meeting minutes on its website in a timely fashion	Agree	Agree
IE 17	SSAC should endeavor to keep their website current to include work in progress and planned future work	Agree	Agree
IE 18	As part of the first annual plan, SSAC should revisit task area one in its charter with ICANN staff (Task area one was: "Develop a security framework for Internet naming and address allocation services that defines the key focus areas, and identifies where the responsibilities for each area lie")	Task area one should be removed from the charter	Agree with SSAC's determination to remove task area one
IE 19-22	19. SSAC should find the best experts globally, without regard to geographic proximity; there should be no artificial geographic quotas in SSAC membership20. Membership terms of three years, renewable by the Board at the recommendation of the SSAC Chair21. No limit on the number of terms an SSAC member may serve22. Stagger SSAC member terms so that approx. 1/3 of the membership is up for renewal each year	Agree	Agree
IE 23	SSAC Board Liaison should serve a maximum of three consecutive one-year terms	Disagree -	WG believes all Board Liaisons should serve three-year terms, with a maximum of three consecutive terms
IE 24	Article XI of the ICANN Bylaws should be amended to include a mechanism to remove an advisory committee chair or member by a simple majority vote of the Board	Disagree - the combination of an approval process for candidate members, three-year terms, and renewal at the discretion of the Board and SSAC Chair is adequte. "Any appearance that the board can punish a member of SSAC for leading an unpopular study would undermine credibility."	WG agrees that protective measures should be put in place to remove a disruptive or under-performing advisory committee chair or member
IE 25	SSAC to implement a policy explicitly stating that the SSAC brand is only to be used on approved work product	Focus on "branding" is inconsistent with objective fact-finding and advice	WG considers that SSAC members should specify, whenever appropriate, whether they are speaking on their own behalf, or citing positions taken by the SSAC in work products.
IE 26-27	26. The SSAC Chair should select, implement, and enforce the regular use of a transparent decision making and documentation strategy fitting of the membership and culture of the SSAC27. The SSAC should formally approve and release all work products pursuant to the chosen decision making and documentation strategy	Disagree - the formality of quorum, voting, Robertís Rules, recusal, dissent and approval are unnecessary because SSAC is not representational	Agree - the WG notes that SSAC's position was formulated in response to the draft final report, which contained an excessively formal approach to decision making and documentation processes. The final report proposes improvements that appear consistent with the culture of the SSAC
IE 28	SSAC should formall and visibly adopt a confidentiality policy. Other policies could be used by mutual agreement	Agree	Agree
IE 29	Utilize these recommended mechanisms, including the annual planning process, to regularly evaluate SSAC performance against objectives and resource utilization	Disagree - evaluating performance against objectives is "appropriate for employees, but not volunteer experts often outside the domain-name business"	WG recommends that the SSAC produce a lightweight, yearly report of activities to the Board; report published as appropriate
IE 30	SSAC should publish simple conflict disclosure forms for each SSAC member on its website. Candidate SSAC members will be required to provide a complete disclosure to the Board prior to appointment to the SSAC, and update disclosures as necessary	Agree, but SSAC doesn't like signing things and prefers an informal process	WG agrees with the recommendation and agrees with SSAC's proposed implementation approach
IE 31-33	31. SSAC work product should include a "Dissents" section, with anonymous or named dissents listed, and "no dissent" if there is full consensus32. SSAC work product should include a "Recusals" section, with anonymous or named recusal listed, and "no recusals" if there were none33. SSAC should develop and publish a conflicts of interest policy based on the ICANN Board policy	Agree	Agree
SSAC 1	The SSAC charter should be reconsidered as part of the review process		WG agrees and notes that the standard ToR for organizational reviews presently calls for this
SSAC 2	A membership committee should review individual contributions regarding renewal of terms		WG agrees with this proposal, which is an operational measure aimed to implement IE Recommendations 20 and 21
SSAC 3	SSAC should (continue to) choose what studies to pursue		SSAC should continue to choose what studies it pursues, being sensitive to the concerns of and issues identified by the ICANN community. The ICANN Board may also task the SSAC
SSAC 4	SSAC should consider which reports to ask ICANN to translate into other languages		WG agrees with this proposal
SSAC 5	SSAC should consider (staffing) a continuous process of feedback from the ICANN community on its work		WG agrees with this proposal
SSAC 6	SSAC should conduct a dedicated meeting annually		WG sees merit in the SSAC developing such a proposal to ICANN
SSAC 7	ICANN's regional liaisons should provide periodic briefings to SSAC members		WG agrees in principle - professional background of liaisons to be considered when defining briefing content
SSAC 8	SSAC should consider maintaining public comments on its documents		WG agrees in principle - but SSAC should not allow public comment periods to delay the delivery of reports
SSAC 9	SSAC Executive Committee minutes should be made available to SSAC members		WG agrees

Public Comment on RWG Draft Report

The RWG presented its draft report at ICANN 36 in Seoul. Dennis Jennings, chair of the RWG, and Marco Lorenzoni, ICANN's Director of Organizational Reviews, took the audience through the recommendations presented above. Steve Crocker provided apologies for other SSAC members who were attending a meeting on the root scaling study, which was occurring at the same time. Throughout the presentation, Dr. Crocker acted as the voice of the SSAC self-assessment and its positions as presented to the RWG.

Recommendations 5 and 6

Recommendation 6 received a great deal of discussion, and Recommendation 5 was incorporated by reference halfway through the discussion, as they both addressed similar issues. John Curran, president of ARIN, objected to the language of Recommendation 6, as it excluded SSAC from information that it might need, rather than including SSAC in any situation where their advice might be important. During his conversation with Dennis Jennings, he emphasized that the Regional Internet Registries rely on the SSAC to serve as an advisory body for IANA functions. George Sadowsky intervened in that conversation and asked Steve Crocker if the RWG's conclusions on Recommendations 5 and 6 were likely to inhibit the work of the SSAC. Dr. Crocker responded with some history regarding the SSAC's past investigations and review actions, where in one case the SSAC was rebuffed in its attempt to review the internal operations of ICANN. Crocker agreed with Curran's opinion that 1) any material change to the operations of the IANA functions should signal a call for SSAC to review; and 2) on a regular basis (for example, annually), the SSAC should perform a pro forma review of operations and procedures. Dr. Crocker concluded, "the short answer, George, is yes, but I wanted to give the long answer first." Dennis Jennings stated that he was unaware, and did not think that the IE was aware, that the RIRs were reliant on SSAC. He suggested that perhaps the examination of those obligations was out of scope of the review, and moreover, that perhaps the IANA function needed to be examined to see whether or not that was an appropriate dependency. He noted that an operational reliance on SSAC was odd, and perhaps not a sensible way to continue.

Crocker agreed and confirmed that he also did not know that the SSAC was "carrying that load." He went on:

Looking at this, I would have to say that, SSAC was invented, as we know, kind of in response to 9/11 without any specifics and clarity. And, then we began to do whatever we began to do. And I think what I'm certainly hearing as well for the first time, is the fact that other have been dependent upon us in a way that, had we not been there, other mechanisms might have evolved. And your point, Dennis, is maybe those mechanisms should be there because that would be the more formal and straightforward thing to do. And I don't necessarily disagree. I'm having quite mixed feelings of, well gosh, I didn't know that we'd been doing anything useful - that's good to know. And, uh, but I also didn't know that we were carrying this sort of responsibility, and now there's questions about what is that precisely and are we doing a good enough job and so forth.

Curran clarified:

...the in-addr domains, which are used for people to map backwards from numbers to names, actually turn out to be pretty important for a lot of applications, and while they occupy a very small amount of the DNS namespace, they potentially have larger impact, if not available, on a wider scope. And so, people don't think about the RIRs when they think about DNS, but we have a slice that is required to be operational which we don't spend a lot of attention worrying about because we recognize there's an expert group doing that. And so, this is not a question of reporting, we actually have a wonderful relationship with IANA, and we actually have formal reporting that has been set up between the NRO acting as the Address Supporting Organization, and the IANA. We just want to make sure that the IANA has access to resources of expertise when making significant changes, becuase a mistake made in that organization would have wide-reaching impact.

Jeff Schmidt of JAS asked for clarification around what the RIRs' dependency was, and probed whether the dependency was in fact the proper operation of IANA, much as the rest of the Internet relies on a stable and functioning DNS. Curran demurred, pointing out that there were contractual obligations involved, and that ICANN's obligations to the RIRs were specific and narrowly focused on the stability and security of its part of that contract (i.e. the IANA functions). Ram Mohan commented that increasing bureaucracy around the SSAC's investigations powers or its right to inquire would only impair the SSAC's ability to predict and assess threats. George Sadowsky agreed that it seemed "stupid" to impede the SSAC's work. Dennis Jennings responded that his opinion of the RWG conclusion empowered the SSAC to act. Mohan responded that every change would require the SSAC to "write a note" to the Board to get permission to examine the ramifications of the change. Jennings noted the comment and stated that he was reasonably certain that the implementation of the recommendation as written would result in an acceptable working relationship. Dr. Crocker stated that the SSAC has an opportunity to put the mechanism to the test. He also noted that informal requests had failed in the past - a layer of bureaucracy and publication of requests might generate some leverage and build a record. Jennings proposed that the RWG review the implementation in twelve months' time to see that it has had the desired effect. Mohan appreciated that commitment to review, so that the issue would not merely be deemed "resolved." Jennings also invited Curran to submit a written comment, and Curran agreed to do so.

Noting that this was "the issue" of the review, Jeff Schmidt chimed in to state that "it is not the reviewer's position that these reviews of internal operations should not be done. Our issue was purely a governance issue." He noted that the IE report offered alternative processes, including having some security professionals on retainer to ICANN, to ensure that the security and stability of the massively important resource that was the Internet was preserved.

Written Comments

The draft report was also published for written comments. One comment was submitted via email. John Curran, as promised, submitted a comment stating that the SSAC's position on Recommendation 6 was correct, and that it was important for SSAC's charter to reflect its ability to engage with and advise on ICANN's internal operations and processes.

RWG Final Report

The Final Report of the RWG was submitted to the board in January 2010. The final report is nearly identical to the draft, although the RWG's response and advice regarding Recommendation 5 (disclosure of confidential or proprietary information) changed in line with the discussion in Seoul. the RWG suggested implementation of a more specific process, and a twelve-month trial period for that process:

The WG agrees with the comments formulated by reviewers and based on sound governance considerations, but in the meantime agrees with SSAC and considers that there is no need to amend its Charter.

It remarks that SSAC has a legitimate right to ask for access to confidential or proprietary information that is needed to fill its mandate, requests that need to be motivated by appropriate reasons. However, this does not imply the right for SSAC to force ICANN –or any other party – to disclose any requested confidential or proprietary information. In case of its disclosure, this information has to be treated under the terms set/to be set by the owners of the information; this could imply the signing of time and project‐specific confidentiality agreements or other measures considered appropriate by the information owners In the case of requests to ICANN the WG suggests that the CEO, and if necessary the Board, should decide on the access to confidential or proprietary information, considering the reasons for the request, and the possibility to set and enforce specific terms of access. Any recurrence of this process should be properly documented.

This process should not however result in unnecessary delays to the work of SSAC; the WG advises therefore the Board to assess the effectiveness of this procedure after one year from its adoption.

Perhaps reflecting Dennis Jennings' belief that the RWG's conclusion regarding Recommendation 6 was strong enough, particularly when bolstered by the process proposed in the revised response to Recommendation 5, the final report made no changes to its response to Recommendation 6. The RWG also added a note to Recommendation 30, noting that an "informal" process of conflicts disclosure should still involve documenting situations when conflicts of interests arise, and maintaining records of those incidents in a transparent manner.

Board Action and Implementation

The board acknowledged receipt of the RWG's final report in March 2010, and instructed the SIC to develop and propose an implementation plan for the review recommendations. The implementation plan was developed by the SIC and SSAC support staff, and subsequent implementation efforts continued throughout 2010 and into 2011. On March 18, 2011, the SSAC presented its final implementation report, marking all implementation tasks as complete. This marked the end of the SSAC1 review.