The free data files and test evidence for the 51Degrees SDKs
The examples and tests in the 51Degrees SDKs use these files for
Device Detection
and IP Intelligence.
As a submodule named data, the two folders are data/device-detection and
data/ip-intelligence.
The data files are not stored in this repository. Each one is listed as a
.link file in the folder where the examples and tests look for it, and
the link file holds the address the data file is downloaded from. No
License Key is needed. These scripts download every linked file into
place.
./get-data-files.ps1./get-data-files.shTo download only some files, name them or their folders, for example
./get-data-files.sh 51Degrees-LiteV4.1.hash or
./get-data-files.ps1 ip-intelligence. A file that is already there is
kept. Add -Update (PowerShell) or -u (shell) to replace the files that
have a newer version, or -Force or -f to download them all again. The
shell script needs curl and gzip.
To get the files straight after cloning, run the script in the same command.
git clone https://github.com/51Degrees/data.git && data/get-data-files.shWhere this repository is a submodule called data, the same works once
the submodule is checked out.
git submodule update --init data && data/get-data-files.shThe Distributor allows each file to be downloaded a limited number of
times an hour from one address, and a script that reaches the limit says
when to try again. A download that receives less than 100 KB a second for
two minutes is stopped, and a download that does not complete is tried
again, up to three times in all. The scripts send a source value with each
download from the Distributor, so that 51Degrees can count which files are
used. It is ci-<repository> in GitHub Actions and data elsewhere, unless
-Source or -s gives another.
| File | What it is |
|---|---|
51Degrees-LiteV4.1.hash |
The Lite on-premise Device Detection data file. Downloaded by the scripts. |
20000 User Agents.csv |
20,000 of the most commonly seen User-Agents, one per line. Downloaded by the scripts. |
20000 Evidence Records.yml |
20,000 of the most commonly seen sets of HTTP headers, including User-Agent Client Hints, one YAML record per set. Downloaded by the scripts. |
Charts of the data's coverage of browsers, hardware, operating systems and crawlers are on the device data quality page.
The Lite file gives IsMobile, ScreenPixelsHeight, ScreenPixelsWidth, PlatformName, PlatformVersion, PlatformVendor, BrowserName, BrowserVersion, BrowserVendor, SetHeaderBrowserAccept-CH, SetHeaderHardwareAccept-CH, SetHeaderPlatformAccept-CH, JavascriptGetHighEntropyValues and JavascriptHardwareProfile. It is built from a subset of all devices and is updated less often than the paid data files.
| File | What it is |
|---|---|
51Degrees-LiteV41.ipi |
The Lite on-premise IP Intelligence data file, giving RegisteredCountry, RegisteredName and RegisteredOwner. Downloaded by the scripts. |
51Degrees-IPIV4AsnIpiV41.ipi |
The Asn data file, giving Asn and AsnName. Downloaded by the scripts. |
evidence.csv, evidence.yml |
20,000 random IPv4 and IPv6 addresses for the examples and tests. |
evidence-gen.ps1 |
Makes a new set of random addresses, for example ./evidence-gen.ps1 -v4 10000 -v6 10000 "evidence.yml". |
CountryCodes.json, Translations/ |
Country names in English and 16 other languages, used by the translation engine and its examples. |
The Lite files carry a small set of properties. The cloud service and the paid data files carry hundreds more, from hardware model and price band to location, connection type and Human Confidence. See pricing for the options.
- Device Detection
- IP Intelligence
- Documentation
- The SDKs that use these files, for C and C++, .NET, Go, Java, Nginx, Node.js, PHP, Python, Rust and Varnish