I was quite disappointed to see that MSN Live announced yesterday [1] that they were shutting off advanced syntax queries such as: link:, linkdomain:, and inurl:. Eytan Seidman, MSN Live’s Lead Program Manager stated that they could tell there was a large amount of automated datamining going on, so they’d unplugged the features completely.
[2]
 Now, I’m completely familiar with how impolite dataminers can impact service for real users — that’s something we police for as well here at Superpages.com. But, I’m unhappy because Seidman’s announcement sounds more like it’s not just impolite datamining they’re after — it’s all automated usage of those specialized queries.
Microsoft is quite lite on features supporting the web community and optimizers, so I’m unhappy that they’re halting the very data that supports those folx. While I don’t use those queries much in my work (because their data hasn’t been all that useful to me, and because such a relatively small fragment of our users come through MSN), it seems like a backwards move for them to revoke the functionality.
It’s the very sort of functionality that lends itself to automated queries. If an agency is trying to manage large amounts of sites and URLs for many customers, they need an automated way to do it. Further, if you want to know from day to day how many of your site’s pages are indexed by them, you’d want to use an automated query report to gather that data.
What this move says is that they’re not sophisticated enough to be banning just the dataminers that are impacting human users. If a bot is asking for documents too frequently, they’re the ones who should be banned.
Further, MSN is behind the curve. They should be offering reports for the web community sort of like the reports Google provides in their Webmaster Tools. Where are the Webmaster Tools for MSN Live? There are none.
They could further improve by not only providing reports, but providing an API service on a separate farm of servers from the interfaces used by human users. Allow people to enact those automated queries through an API. The data in these automated queries isn’t used to recreate the entire search experience — it’s specialized use for a variety of research and support needs for webmasters and the agencies that assist webmasters. MSN Live’s action has just pissed off the very communities of people who they need to be courting.
I’ve also mentioned before that I thought Google Webmaster Tools should increase the types of reporting that they provide for sites, such as providing expanded keyword ranking information. I know their concern is giving people so much information that they could “game” the system, but not providing the info just encourages people to use more and more subtle methods for executing automated queries in ways that are harder and harder to detect/ban.
MSN Live needs to work on engaging with the webmaster and SEO communities better. Provide reporting tools outright, and through APIs. Ban only dataminers that don’t throttle back the rate of bot queries to polite levels.