I ran into a troubling issue recently. A client’s WordPress site was having SEO issues because the Googlebot was getting a 301 redirect (indicating permanently moved) on pages where normal visitors were getting the correct page (status code 200 OK). The pages in question all had URLs with parameters such as http://mysite.com/restaurants/?type=italian. However the Googlebot was always getting a 301 redirect to the base page http://mysite.com/restaurants/ (without the parameters). On this site, that meant a big SEO hit as each sub-page filtered down to specific restaurant types and those SEO rankings were all being lost due to the redirects.
Since I wrote a custom theme for the site, I knew there was nothing in the theme code responsible for this issue. I checked the .htaccess file as well and found nothing unexpected there. I was stumped and contacted the hosting provider WP Engine. Unluckily, they were the cause of the issue, but, luckily, they were also able to resolve it. It turns out they have a security setting turned on by default called Redirect Bots. This setting, when on, causes all bot visitors to have those parameters trimmed off of the URLs and a 301 returned with the stripped down URL.
So, if you’re having the same issue and you’re being hosted on WP Engine, ask them to turn this setting off and the problem will be resolved. You’re welcome.
What do you think?