Search Engine Optimization Fast Facts, Last Updated:09/06/2010
With dynamic assignment, typically the systems can also add or
remove downloader processes. The central server may become the
bottleneck, so most of the workload must be transfered to the
distributed crawling processes for large crawls. |
