RadostIT Agent
RadostIT Agent is a web crawler operated by RADOST IT j.d.o.o., trading as RADOST IT.
User-Agent string
radostit-agent- Operator
- RADOST IT j.d.o.o. — Ulica Nikole Tesle 17, 23000, Zadar, HR
- Contact
- bot@radostit.com — monitored, not no-reply
- Last updated
- 26th August 2026
Why it visited your site
We build a directory of business websites for B2B outreach. When RadostIT Agent visits your site, it is looking for a public contact page and checking whether that page has a contact form.
It does not index your site for search, and it does not evaluate your site's content, design, or performance. The visit is transactional: find a way to reach the business behind the domain, record what's public, and move on.
What it requests
| Behaviour | Value |
|---|---|
| robots.txt | Fetched first, obeyed, re-checked every 6 hours |
| Pages per site | Max 4 per visit — homepage, contact page, sitemap |
| Request spacing | Min 1.5s between requests to the same host, never parallel per host |
| Crawl-delay | Honoured when slower than our default; never sped up because a site permits it |
| Backoff | Automatic on HTTP 429 and 503 |
| Content | HTML only; non-HTML and anything over 2 MB is aborted mid-download |
| Timeout | 10 seconds |
| Revisit interval | At most once every 90 days |
| Methods | GET only |
It never submits forms, never logs in, and never requests anything behind authentication.
What is stored, and what is not
Stored
Only what the site publishes publicly:
- Domain and homepage URL
- Page title, meta description, declared language
- Contact-page URL
- Whether a contact form exists, and which provider/CAPTCHA it uses
- Publicly listed business email addresses
- Links to public social profiles
- HTTP status
Not stored
- Page content is not copied or republished
- No images or files are retained
- No personal data beyond publicly published business contact details
- Nothing behind a login
- No visitor or analytics data
- Site content is not used to train models
How to block it
1. robots.txt
Honoured within 6 hours. A disallowed site is recorded as blocked permanently — never re-crawled and never contacted.
User-agent: radostit-agent
Disallow: /2. Server or WAF rule
Match the User-Agent substring below. We treat an HTTP 403 as a permanent no.
nginx
if ($http_user_agent ~* "radostit-agent") {
return 403;
}Apache
RewriteCond %{HTTP_USER_AGENT} radostit-agent [NC]
RewriteRule ^ - [F,L]Cloudflare
(http.user_agent contains "radostit-agent")(add as a Block rule in WAF → Custom rules)
3. Direct opt-out
Email bot@radostit.com with your domain. It goes on a permanent do-not-contact list covering both crawling and outreach, and the block survives re-discovery. Turnaround: 3 business days. This is a direct mailbox, not a contact form.
Verifying the traffic is really us
A User-Agent string is trivially spoofed, and anyone can put our name in theirs. We do not currently publish a fixed IP range or ASN for RadostIT Agent. If the volume or behaviour in your logs doesn't match the table above, it probably isn't us — send the log lines to bot@radostit.com and we'll confirm.
Data protection
We process the publicly published business contact details listed above on the basis of legitimate interest (GDPR Art. 6(1)(f)). We retain this data for 30 days. To request erasure, email bot@radostit.com. See our Privacy Policy for how we handle personal data generally.
Get insights on AI-driven web development, automation & compliance — straight to your inbox.