Passive and Active Internet Measurements
This page contains datasets and tools that can aid in Internet Measurements research (esp. those focusing on security). Feel free to email me other links/datasets that you think would be useful.
Passive measurement datasets
- Twitter: Random tweets API (and relevant operators), Archive repo of tweets, Twitter Trends.
- Top sites:
- Network measurements:
- Device search engines: Shodan, Censys, ZoomEye, Project Sonar (Rapid7), thingful. (See also "vulnerable cameras" below.)
- Certificates:
- DNS: OpenIntel, zone files of generic TLDs from ICANN, Verisign Top-Level Domain Zone File Information, Forward/Reverse DNS (from Sonar).
- Firefox Telemetry: Project, Data publishing.
- Stratosphere Lab datasets (IoT, Wifi, malware, botnet pcaps).
- Phishing: PhishTank (phishing domains), and per country reports from SecureList.
- Web Censorship: Citizen Lab (censored websites).
- Content measurements:
- App reviews: Android
- MISC: Internetwache.
Active measurement tools
Research artifacts
- A Catalog of Research Artifacts for Computer Science: FindResearch.