Protecting your website against bots

Robots (“bots”) are an increasing problem, now representing the majority of Internet traffic, and the trend has also been visible on PhotoDeck websites.

Some bots are perfectly legitimate and well-behaved (think Googlebot). Some are semi-legitimate (unclear purpose, but from a large company) and/or hammer servers recklessly (we’re looking at you, Facebook and Amazon). Many are outright illegitimate, probing for security holes or aggressively scraping pages for unknown ends - some botnets leverage millions of hacked consumer devices to crawl the Internet.

So to protect our members' websites against abuse, we've taken new measures to fight them — without requiring legitimate users to go through dreaded captchas, and without hindering legitimate crawlers like Google's.

The visible measure is the new “security verification” page you may have already noticed on your website, similar to what you've seen on other sites. It includes a hidden challenge that is automatically and quickly resolved by a legitimate visitor's browser, but not by basic bots. Once passed, a visitor can browse freely without seeing that protection again.

We've been rolling out this protection progressively, focusing initially on the most sensitive pages (login, contact, etc.) and on traffic from bot-rich regions, but it will apply to all visitors and all pages within a few weeks.

The results are already spectacular, bot traffic on PhotoDeck websites has plummeted (some members might notice the change in the statistics we provide), without impacting legitimate users.

This protection, and the others we've implemented, will continue to evolve. This is a cat-and-mouse game: bot creators learn over time to circumvent barriers, so new barriers have to be designed.

We’ll stay on it!