Plugins categorized as crawler | Page 2 of 2

(1)

Display the results of your Page Modified crawls right within your WordPress Admin.

(1)

This plugin detects a few bots (e.g. Google, Google ads, Alta vista etc) and sends…

(0)

Dynamic robots.txt for Multisite! Change parameters, or don't, for each site of your network without…