Whois Anti-Harvesting Techniques
DNS Seal, a best practices wiki for DNS
Common Anti-Harvesting Techniques
- Rate Limiting: this practice prevents bulk searching and data mining of the WHOIS database by limiting how many queries can be submitted within a certain period of time. After reaching the registrar mandated limit, search results are either not returned due to temporary blacklisting or in some cases, only select information is returned. For example, contact information may no longer be returned although other information is still retrieved. According to Carnegie Mellon University's recent Whois Misuse Study, rate limiting is one of the most common anti-harvesting techniques used by registrars. Donuts Inc., NetworkSolutions, and Nominet use rate limiting policies to guard against Whois abuse.
- Privacy or Proxy Services: these services keep the registrant's personal contact data out of the WHOIS database by using the registar's or a third party's contact information instead; some services also screen spam before forwarding any messages received to the registrant. A Security and Stability Advisory Committee (SSAC) report revealed that using a privacy or proxy service decreased the number of spam messages a registrant received and found that it was more effective at reducing spam than using protective methods such as CAPTCHAs or rate limiting. Many popular registrars offer privacy or proxy services. Some provide the service at an additional charge such as Namecheap  or refer the registrant to a partner service such as Godaddy. Additionally, some registrars provide privacy services free of charge, such as Public Domain Registry.
- CAPTCHA or Completely Automated Public Turing Test To Tell Computers and Humans Apart: some registrars make users complete a CAPTCHA challenge before a Whois query can be entered in order to stop "automated collection of domain name records." Donuts Inc. and Namecheap incorporate CAPTCHAs in their Whois searches.
- Examples and more information about CAPTCHAs can be found here.
- Blacklisting: this practice permanently or temporarily stops users from searching a registrar's Whois database when using a certain IP address or domain name; permanent blacklisting is usually employed if the user is suspected of frequent Whois misuse.
- Zone File Publication: according to an SSAC report, not publishing TLD zone files may decrease the risk of Whois abuse. However, ICANN contracts mandate that gTLDs publish their zone files, so this strategy can only be used by certain ccTLDs.
The Whois database has long been suspected of providing registrants' personal information to Internet scammers committing Whois Misuse. Recent studies tend to support this claim. The risk of providing personal information to what people may see as an insecure public database can lead to other types of Whois abuse such as using False Whois. Measures that can be taken by registrars or registries that seek to mitigate Whois abuse are generally supported, although more focused research should be done to see which methods of Whois protection are most effective.
Many registries utilize some type of Whois anti-harvesting technique, as exemplified above. Rate limiting seems to be common among registrars. However, the type of anti-harvesting mechanism used and its own specific parameters are set at the discretion of the registrar not by ICANN policy. So, some registrars' Whois data may be better protected than others. Of additional concern, some registrars that claim to use some form of Whois protection did not show any evidence of it during the recent Carnegie Mellon Whois Misuse Study. The study recognized that there are multiple possible explanations for this behavior.
- 2013 Registry Agreement (RA): the updated RA specifies that registry operators provide "a WHOIS service available via port 43 in accordance with RFC 3912, and a web-based Directory Service" that aligns with ICANN's specified format. This service should also include registrar who-is queries within the specified format. However, the RA does not specify any form of Whois protection.
- 2013 Registrar Accreditation Agreement (RAA): the RAA also mandates that accredited registrars provide a Whois look-up service through port 43. It does not explicitly mention any endorsed Whois anti-harvesting techniques, although it does address the information that privacy or proxy services need to provide to satisfy Whois requirements.
- A GNSO Consensus Policy could address the way registrars and registries combat Whois abuse problems; if such a policy was adopted by ICANN, it would become part of the RA and RAA.
- An Expert Working Group (EWG) is also working on a proposal to replace the current Whois system with the Registration Directory System (RDS). The RDS may be able to better protect registrant data and could possibly make mining Whois data more difficult by creating a gated-access approach. This gating approach would make certain Whois information available to any query but would only give other more personal identifying information to those with authorization.
At this time, there is no U.S. legislation addressing Whois Anti-Harvesting techniques or protection.
- Read the Carnegie Mellon Whois Misuse Study
- View the SSAC's report: Is the WHOIS Service a Source for email Addresses for Spammers?
- See an example of an ICANN Contractual Compliance Report on Port 43 Whois Access
- http://www.icann.org/en/news/public-comment/whois-misuse-27nov13-en.htm (PDF) titled Whois Misuse Study Draft Report (November 26, 2013), Internet Corporation for Assigned Names and Numbers (ICANN)
- http://www.icann.org/en/resources/compliance/update/update-whois-access-audit-report-port43-30apr12-en.pdf (PDF) Internet Corporation for Assigned Names and Numbers (ICANN)
- http://www.donuts.co/policies/whois/ Donuts, Inc.
- http://watchmy.domains/kb/whoislimits.php Watchmy.Domains
- http://www.nominet.org.uk/uk-domain-names/about-domain-names/domain-lookup-whois/detailed-instructions Nominet
- http://www.icann.org/en/groups/ssac/documents/sac-023-en.pdf (PDF) ICANN Security and Stability Committee (SSAC)
- https://www.namecheap.com/products/whoisguard.aspx Namecheap.com
- https://www.domainsbyproxy.com/default.aspx?ci=44263&prog_id=GoDaddy Domains by Proxy
- http://www.publicdomainregistry.com/privacy-protection/ Public Domain Registry
- https://www.namecheap.com/domains/whois.aspx Namecheap.com
- http://newgtlds.icann.org/en/applicants/agb/base-agreement-contracting (PDF) titled View the Updated Registry Agreement (November 20, 2013), Internet Corporation for Assigned Names and Numbers (ICANN)
- http://www.icann.org/en/resources/registrars/raa/approved-with-specs-27jun13-en.htm Internet Corporation for Assigned Names and Numbers (ICANN)
- http://singapore49.icann.org/en/schedule/mon-gtld-directory-services (March 24, 2014) Presentation from ICANN 49 Conference in Singapore
- https://community.icann.org/display/WG/Video%3A+Introducing+the+RDS Internet Corporation for Assigned Names and Numbers (ICANN)