https://www.phdss.com/googlebot-verify-logger
=== Googlebot Verify Logger ===
Contributors: SearchEngineWorld, Brett Tabke
Tags: googlebot, security, logging, crawler, seo, performance, flatfile, cidr
Requires at least: 5.0
Tested up to: 6.6
Stable tag: 1.1.1
Requires PHP: 7.4
License: GPLv3 or later
License URI: https://www.gnu.org/licenses/gpl-3.0.html
Daily fetch of Google’s OFFICIAL IP ranges + logs ONLY VERIFIED Googlebot visits to FLAT FILES (NO DATABASE!)
== Description ==
** 100% GOOGLEBOT LOGGING **
**The ONLY plugin that verifies Googlebot using BOTH:**
– ** OFFICIAL UA WHITELIST** (6 Google bots)
– ** OFFICIAL IP CIDR RANGES** (500+ ranges, updated DAILY)
**FLAT FILES ONLY = ZERO DATABASE LOAD!**
** KEY FEATURES:**
* **Real-time Verification**
– UA match: Googlebot, AdsBot, Storebot, etc.
– CIDR match: IPv4 + IPv6 from Google APIs
– Logs in **<1ms** (99.9% requests exit instantly)
* **Admin Dashboard**
– Live stats: Total, 24h, 7-day trends
– **ONE-CLICK DOWNLOADS** (googlebot.log + index.json)
– 14-day activity table
– Recent 50 verified hits
* **Performance Optimized**
– **NO SQL** – Flat files in `/wp-content/uploads/`
– Atomic writes (LOCK_EX)
– Tail-based reads (instant)
– JSON index for sub-second stats
* **Automatic Maintenance**
– Daily IP refresh (5 official sources)
– Configurable retention (7-3650 days)
– Auto-cleanup old logs
** LOG FORMAT:**
[2025-10-22 14:30:00] 66.249.66.1 | Googlebot/2.1 | /blog-post | GET | 200
**USE CASES:**
– Block fake Googlebot scrapers
– Track real Google crawling patterns
– Debug indexing issues
– Export for SEO analysis
– Monitor crawl budget usage
** LIGHTNING FAST:**
– **0.2ms** average execution
– **0 bytes** database usage
– **47KB** plugin size
== Installation ==
1. Upload the `googlebot-verify-logger-v1.1.1.zip` file
2. **Activate** the plugin through the ‘Plugins’ menu in WordPress
3. Visit **Tools Googlebot Logs**
4. **Click “Download googlebot.log”** to get started!
**That’s it!** Logs appear in real-time for verified visits.
== Frequently Asked Questions ==
= Does this slow down my site? =
**NO** – Exits in **<1ms** for 99.9% of requests:
1. Skip admin (instant)
2. UA check (regex)
3. CIDR check (binary compare)
4. **0.1%** verified log
= Where are the log files stored? =
/wp-content/uploads/sew-googlebot-logs/
googlebot.log (Raw entries)
index.json (Instant stats)
= How do I download the logs? =
**ONE CLICK:**
– **Blue Button**: `googlebot.log` (all raw entries)
– **Gray Button**: `index.json` (total/daily counts)
= What User-Agents are verified? =
**Official Google whitelist:**
Googlebot
Storebot-Google
AdsBot-Google
AdsBot-Google-Mobile
Mediapartners-Google
Google-Extended
= Where do IP ranges come from? =
**5 OFFICIAL Google APIs** (updated daily):
1. googlebot.json
2. special-crawlers.json
3. user-triggered-fetchers.json
4. user-triggered-fetchers-google.json
5. goog.json
= Can I change the retention period? =
**Yes!** Admin Retention days (7-3650):
– **Default**: 90 days
– **Minimum**: 7 days
– **Maximum**: 10 years
= Is this secure? =
** PRODUCTION SECURE:**
– Nonce-protected downloads
– `manage_options` capability only
– File whitelist (2 files max)
– Path traversal protection
– Atomic file writes
= Works on Windows Server? =
**Yes!** 100% compatible:
– PHP 7.4+
– No Linux dependencies
– Fallback for `tail` command
= How do I test it? =
1. Install plugin
2. Visit any page
3. **Curl as Googlebot:**
“`bash
curl -A “Googlebot/2.1” https://yoursite.com/
Check Tools Googlebot Logs
== Screenshots ==
Admin Dashboard – Stats + ONE-CLICK downloads
14-Day Trend Table
Recent Verified Hits
File Download Box
== Changelog ==
= 1.1.1 – October 22, 2025 =
NEW: Direct download buttons for log files
File size display (B/KB/MB/GB)
Enhanced admin UI with icons
Improved nonce security
= 1.1.0 – October 21, 2025 =
COMPLETE REWRITE: Flat files only (NO DATABASE!)
JSON index for instant statistics
Atomic writes with LOCK_EX
Tail-based recent entry reads (O(1))
Auto log rotation + cleanup
= 1.0.0 – October 20, 2025 =
Initial release
Daily Google IP range fetch (5 sources)
IPv4 + IPv6 CIDR validation
Admin dashboard with live stats
SQL-based logging
== Upgrade Notice ==
= 1.1.1 =
Major Performance Upgrade! Converts from SQL to flat files.
BACKUP your database first
Old SQL logs preserved during upgrade
New flat file system starts empty
Requirements:
WordPress 5.0+
PHP 7.4+
47KB disk space
== Support ==
FREE SUPPORT via:
WordPress.org forums
GitHub Issues
SearchEngineWorld.com contact form
Premium Support Available for enterprise deployments.
== Credits ==
Lead Developer: Brett Tabke (brett@searchengineworld.com)
Project: SearchEngineWorld Webmaster Tools
License: GNU General Public License v3
Thank you for using Googlebot Verify Logger!
== Donations ==
Love the plugin? Buy us a coffee:
PayPal: paypal@pubcon.com
Your support keeps this plugin 100% FREE forever!