Booter – Bots & Crawlers Manager

by uPress

Description

Booter – Bots & Crawlers Manager is a preventative measure (treatment in advance) and treatment of damages caused by crawlers and bots.
The plugin uses a number of existing technologies which are known by crawlers and bots and takes them one step forward – smartly and almost completely automatically.
To allow the plugin to function correctly, you must follow the instructions and manually enter some data (which must be done by a human being to avoid errors).

At the prevention level

Booter allows you to manage and create an advanced dynamic robots.txt file.
View a 404 error log to see the most common bad links.
Blocking bad bots that cause high server loads due to very frequent page crawls, or are used to search for security vulnerabilities.

At the treatment level

Booter allows you to limit the amount of requests from crawlers and bots, if or when they exceed the specified amount of requests per minute, it will be rejected for a specified period of time.
Rejecting links that we do not want in the fastest way, not by just blocking but by sending the appropriate HTTP status code to make search engines forget them.

Instructions for use in case of damage treatment

Activate the plugin.
Enable the 404 error log option.
Set the access rate limit.
Watch the 404 log, try to find common parts in the URLs that repeats most often.
Enter the common parts to the “reject links” page, and ensure the rejection code is 410.
Clear the 404 error log.
Repeat the process once every few hours until the 404 error log remains blank.
Check the status of your website’s index coverage every few days.

Installation

Upload booter-crawlers-manager folder to the /wp-content/plugins/ directory
Activate the plugin through the ‘Plugins’ menu in WordPress
The plugin will start rate limiting as soon as it is activated, however it is recommended to update the settings to suit your needs, under ‘Settings’ -> ‘Booter – Crawlers Manager’ menu

Screenshots

Plugin General Settings
Robots.txt Management
Reject Links Settings

Reviews

Almost Perfect

By spectraemail on June 20, 2025

Booter would have gotten 5 stars because it does exactly what it says. The problem I find when I use my firewall's facility that shows me what traffic has occurred, it normally displays data about the host (if known) and what was being accessed, giving me the option to permanently block the IP address, using only 3 mouse clicks. When Booter is involved, all I get to see is a page with the code involved (403,410, 429, etc). That means I have to use an external app to look up the IP address, and then, go to the firewall's page for entering permanent blocks, entering them. If the number of IP addresses is numerous, that increases the amount of time needed. I'll continue to use it until I find something that stops crawlers while making running the site easier.

Useful for bad bots (too much bad bot !)

By zepolo on February 25, 2024

Useful . I hope you won't abandon the plugin and will update it regularly. There has been no update for a year.

Works!

By worldlightmedia on September 12, 2023

I got this for Rate Limiting and I must say it works. I had so many bots scraping my site it crashed every day. However, when a legit user accidently got booted, there was no way to clear them.

Also, in the forum there are some great questions (like "how to whitelist IP") but they are left unanswered so I have to give it 4 stars.

Ten Stars

By GoNo Food Finder (myformulary) on December 7, 2022

Kudos to this development team for creating a very SIMPLE, highly COMPREHENSIVE, and phenominally effective plugin. We tested a dozen and this is the best.

Very good

By Minaya (Minaya01) on October 30, 2021

For who just hate bad bots or some bots in particulars, this plugin is a must have

Works well

By benny1883 on March 19, 2021

Works well, doesn't block all bots tho

Perfect, but one little thing

By antacid on May 1, 2020

It finally has stopped some bots that made a lot extra load on my web-site. Visitors stayed the same, number of visits haven't changed. Load became two times less. Thank you. I use only two options in your plugin: Block Bad Robots Rate Limiting Other options are disabled. But i still get a message: "Booter - Bots & Crawlers Manager has found the following 404 redirect plugin active: bla bla bla. Such redirects prevents Booter from detecting 404 errors as well as being the wrong way to handle broken links. A 404 error is the correct response to invalid URLs while a redirect (30X) tells search engines that the URLs exists (in another location). We therefore recommend that you disable these plugins." Is it possible to disable this message if i don't use 404 Logging function? Please add such possibility. This message is too annoying and i need my instaled 404 redirect plugin.

Great and Friendly solution

By YoramBo on April 24, 2020

Thank You! It saved my two shops I launched and didn't expect the massive traffic generated by crawlers. It was a solution provided by the hosting experts managed my sites:-)

Not entirely in love

By deleted (lovesamla) on January 25, 2020

Positive - works out of the box - doesn't crash your site, also not your multi site Negative - You can't easily see what has been blocked - Out of the box, the list is too small. You can add bots but they get overwritten when you pull an update for the bot list

Excellent plugin, manages Google reputation.

By shirlypinhas on October 20, 2019

The plugin is just excellent, I came across it through the EzCache cache plugin created by the same author, a good plugin that manages the bad bots and pages that Google will crawl / not crawl.

Changelog

1.5.7

Update tested up to

1.5.6

Move additiona bots list to a remote list

1.5.5

Fix rare crash of the UI

1.5.4

Fix rate limited not properly detecting excluded useragents

1.5.3

Fix scheduled task not setting properly

1.5.2

Fix bots list not updating

1.5.1

Fix regression introduced in version 1.5

1.5

Added options for weekly and monthly 404 log report
Added option to exclude user agents from rate limiting
Updated UI components
Updated bad bots list
Server IP will be excluded from rate limiting by default