Crawl Speed

There are several factors that can affect the speed of a web crawl on 80legs:

Account Levels

Higher account levels will run crawls more quickly than lower account levels. E.g., if you run a crawl on a basic (free) account, that same crawl will run more quickly on an intro, plus, or premium account.

Rate Limiting

80legs automatically rate-limits the speed of each web crawl depending on which domains are being crawled.  This prevents your web crawl from overwhelming a website or getting blocked.  In fact, 80legs will crawl a website as quickly as it possibly can be crawled without getting blocked.

Because we automatically rate-limit, it's difficult to esimate the speed of any given web crawl before it's actually run.  Therefore, the best way to determine web crawl speed is to run a test crawl.  Any future web crawl with the same settings can be expected to run at the same speed.