NetBeacon MAP

NetBeacon Measurement and Analytics Platform (MAP) (formerly DNSAI:Compass until June, 2023 and DNSAI Intelligence until September, 2022) is NetBeacon Institute initiative to measure and track the use of the DNS for phishing and malware, and the goal is to reduce DNS Abuse at the DNS level . It was initially launched in September, 2022, but the first publication about the initiative was in May, 2022 . In June, 2023, the Institute announced the addition of a new level of reporting for their measurement project, which was then named NetBeacon Measurement and Analytics Platform (MAP).

Collaborations

NetBeacon MAP is a collaboration with KOR Labs, led by Dr. Maciej Korczynski. KOR Labs is responsible for collecting the data following an established methodology. This data is then provided to the Institute, that works with PIR’s Data Analytics team to create interactive charts, reports, and individualized dashboards .

Methodology

NetBeacon Measurement and Analytics Platform (MAP) is operated by the NetBeacon Institute in collaboration with KOR Labs, which designed and runs the measurement infrastructure. The methodology, first published in 2022 for the then DNSAI:Compass project (originally DNS Abuse Institute Intelligence) and updated in 2024 for NetBeacon MAP, aims to provide reliable and actionable data on the prevalence and persistence of DNS abuse, focusing on phishing and malware.

Data Collection and Processing

• URL blocklists: Ingests phishing and malware-delivery URLs from four established providers: APWG (eCrime Exchange), PhishTank, OpenPhish (premium feed), and ABUSE.ch’s URLHaus feed. The feeds are polled in near real time, with polling intervals ranging from every minute to hourly depending on the provider.
• Domain names: Extracts domain names from URLs using the Public Suffix List, discarding IP-only URLs, and combines this with zone files from CZDS-participating gTLDs and a number of ccTLDs, plus additional active and passive measurements, to build a comprehensive list of registered domains across TLDs.
• Technical registration information: Gathers technical registration data via RDAP/WHOIS (registrar, IANA ID where applicable, creation and expiration dates) without processing registrant personal data, in order to map domains to registrars and estimate registrar sizes.
• Special domains and filtering: Maintains a manually curated list of "special domains" that provide subdomains or redirection as a service (for example URL shorteners, dynamic DNS and free subdomain providers) and excludes them from abuse-rate calculations so that infrastructure providers like "google.com" are not counted as abusive domains.
• Uptime measurements: For each abusive domain, performs repeated measurements over a one-month window to determine when abuse is mitigated (removal of malicious content, suspension of hosting and/or removal from the zone). The uptime or persistence of abuse is defined as the time between blocklisting and mitigation.

Security Metrics

• Occurrence metrics: Calculates, for each TLD and registrar, the distribution (rate) of unique abusive domain names, normalized by the number of domains under management (DUM). Occurrence is expressed as a percentage of abusive domains over total domains per intermediary.
• Persistence metrics: Calculates, for each registrar, the median uptime (persistence) of abusive domains. Median is used instead of mean in order to reduce the impact of false positives and long-lived outliers on the reported mitigation times.

Classification of Domains

• Malicious vs. compromised domains: Distinguishes domains registered for malicious purposes from benign domains that have been compromised (at the website, hosting, or DNS level). This uses a hybrid approach combining KOR Labs’ MalCom machine learning classifier with a posteriori evidence from mitigation actions (e.g., removal of the domain from zone files or suspension of hosting).

TLD and Registrar Size Estimation

• Estimates the size (domains under management) of TLDs and registrars using zone files wherever possible and, for TLDs without open zones, approximate counts published by DomainTools and similar sources. These size estimates are used to normalise occurrence metrics at the TLD and registrar level.

Challenges and Limitations

• The methodology recognizes residual false positives in blocklists, incomplete or inconsistent WHOIS/RDAP data (especially for some ccTLDs), and the difficulty of mapping locally accredited ccTLD registrars to a unified identifier space. As a result, registrar-level metrics are currently calculated and published primarily for ICANN-accredited registrars, while TLD-level metrics can be produced for a larger set of zones.

Reports

NetBeacon MAP can can be consumed in three formats:

NetBeacon MAP: Monthly Analysis reports provide detailed tables identifying registrars and TLDs with high and low relative levels of malicious phishing and malware in their domains under management (DUM) and compared to their new monthly registrations . On September 16, 2022, the first report was launched, and focused on higher level aggregate data from May, June, and July 2022. There reports continued monthly . In April, 2024, the Institute launched its twentieth report.

NetBeacon MAP: Charts can be used by registries and registrars to understand how often the DNS is used for phishing and malware, whether abuse is mitigated, how quickly, and the type of registrations (compromised website or maliciously registered domain) . The Interactives Charts can be consulted here.

NetBeacon MAP: Dashboards permits registries and registrars to understand, track and benchmark the impact of their efforts to combat DNS Abuse. Accessing one's organizational dashboard helps to understand how much phishing and malware NetBeacon MAP has identified in a particular zone, whether it has been mitigated, and how this compares to one's peers. It's possible to view analysis on whether the domain name was maliciously registered for the purposes of DNS Abuse or is associated with an issue of compromise (typically website compromise). Data can be used to track and measure the prevalence of abuse as well as how changes in one's processes and policies make an impact over time .

Generic Top-level Domain

Country code top-level domain. Determined by "ISO 3166-1: Codes for the representation of names of countries and their subdivisions — Part 1: Country codes".

Top-Level Domain

Internet Assigned Numbers Authority

Top-Level Domain

Registration Data Access Protocol. Uniform patterns to construct HTTP URLs that may be used to retrieve registration information from registries and Domain Name Registries using "RESTful" web access patterns (RFC 7482).

Country code top-level domain. Determined by "ISO 3166-1: Codes for the representation of names of countries and their subdivisions — Part 1: Country codes".