How the crawler identifies itself.
LeadsForge fetches publicly available business website pages when a customer runs website discovery. This page is the reference for that crawler.
User agent
Requests identify themselves as:
LeadsForgeBot (+https://leadsforge.app/bot)
The address in that string is this page. https://leadsforge.app/bot/ is the same page.
What gets fetched
A crawl starts from a public website that discovery has already associated with a business. The crawler reads pages on that site to look for published contact signals, such as an email address or a WhatsApp number. It requests ordinary web pages over HTTP or HTTPS. It does not log in, submit forms, or open private areas.
Before a crawl, it reads robots.txt for that site and skips paths listed as Disallow. A crawl-delay in that file is honored up to 10 seconds. If robots.txt cannot be fetched, the crawl continues on public pages only.
The first request uses the user agent above. If that response is an error or an access interstitial and contains no page content, a follow-up request may use a standard browser user agent so a public page can still be read. That follow-up is not a login and does not bypass a restriction in robots.txt.
What we do with the page
Page text is used to extract public contact details for the customer who requested the search. We do not publish a copy of your site. Customers are told to review results before they export or contact anyone. The privacy notice describes how prospect data is stored and how a removal request works.
Ask us to stop
To keep an email address or a domain out of later prospect results, use the data removal page. We email a confirmation link to that address, or to admin@ the domain, and suppress it after the link is confirmed. The link expires after 24 hours.
You can also write to support@leadsforge.com or use the contact form and choose data removal. A disallow rule for LeadsForgeBot in robots.txt is the right control on your own site. Include the pages you want left alone and allow time for a new crawl to see the file.