ASchroder.com

I’ve made a new Magento extension for submitting your store sitemap to Google, Yahoo, Ask and Bing. My new extension can submit automatically whenever the sitemap is regenerated, or on-demand from the admin UI. More about how I made it and how it works below, but first a little background.

How Sitemap Submit came to be

As of last week I have been taking a train to work in a far off land called Pukekohe a couple of times per week. Pukekohe itself is not traditionally known as a high-tech startup hub – but in 2010 the number of small high-tech software engineering companies has grown considerably (probably at least 200%).

What all this means is I get a couple of hours a day, two times a week where I’m literally stuck in front of my laptop as the train I’m on rocks and weaves it’s way across country to WWA HQ. This new Magento extension I have made is the first of hopefully many what I’ll now call Public-Transport-Projects. The extension will manually or automatically submit your Google sitemap file from your Magento store to several major search engines (Google, Yahoo, Bing and Ask).

How to use Sitemap Submit

To start with, you’ll need to install it – head over to the Sitemap Submit extension page on Magento connect to get started with that.

Once installed, It’s dead easy to use – basically you generate your sitemaps the usual way using Magento’s built in functionality – to do this go to the Catalog -> Google Sitemap section of the admin interface as shown in the screenshot below.

Sitemap Submit found in the Magento Sitemap admin menu option

Once you have your sitemap.xml generated you can click on it and view the details. One this edit screen you will see the new Submit button, which when clicked, my extension will submit your sitemap to Google, Yahoo, Ask and Bing. You can see in the screenshot below where the button should be.

The Sitemap Submit submit button on the sitemap edit screen of Magento Admin

There is also a configuration option for automatic submission. What this means is that whenever your sitemap is generated or regenerated (by you or by cron if you have it configured to do so) then the sitemap will also be submitted to the search engines.

The Sitemap Submit Configuration options - an option to enable automatic submission each time the sitemap is generated, and also the yahoo App Id required for yahoo submission.

The only other configuration to be aware of is the yahoo app id, to use the api you need to add an app id – read all about the Yahoo API and the app id. If you omit an ID, the extension will just skip notifying Yahoo, no big deal.

How Sitemap Submit works

I won’t bore you with too many details, basically it’s made up of an observer that watches out for the regeneration and a service object that actually does it (plus the usual various Magento extension things, like a controller and a helper.

The only tricky thing was the core Sitemap.php object didn’t fire a generating event, which if you read my explanation of Magento events and observers you’ll know is quite important. I override and slightly alter the default Magento sitemap as shown below:

 public function generateAction() {
    	parent::generateAction();
    	Mage::dispatchEvent('sitemap_sitemap_generate', array('sitemap'=>$this));
    }

That fires the event which is then picked up by my observer and in turn my submission service kicks off. It basically runs through each search engine and submits the sitemap using the required URL’s like so :

         	//ping google
        	if($this->_pingGoogle($url)) {
        		$msg = $msg . "Pinged Google successfully<br/>";
        	} else {
        		$msg = $msg . "Failed to ping Google.<br/>";
        	}

… and then I actually make the request like this:

 
	protected function _makeRequest($ping) {
 
		$curl = new Varien_Http_Adapter_Curl();
        $curl->setConfig(array(
            'timeout'   => 20
        ));
 
        $curl->write(Zend_Http_Client::GET, $ping, '1.1');
        $data = $curl->read();
 
        if ($data === false) {
            return false;
        }
 
        if ($curl->getInfo(CURLINFO_HTTP_CODE) == 200) {
	        // Mage::log($data); // uncomment to debug raw submission response
	        return $data;
        } else {
	        Mage::log("Submission to: " . $ping . " failed, HTTP response code was not 200");
	        return false;
        }
 
        //TODO: handle timeout?
 
	}

And that’s about it, you can read the full story in the open-source PHP code, in fact I’d invite you to do so, and give me some feedback on it!

The Sitemap Submit Feature Roadmap

I’ll be looking to extend the extension further over the next month or so in two ways. One is to add many more search engines that have an open sitemap submit function. The second is I want to enable more SEO capability and allow store submission to the many Magento store directories and Magento showcases like Magento Parade. I’m also, as always, really keen to hear feedback on fixes and features that you would like to see in the product.

You might also be interested in:

1. Sneak Peek at the New Magento Gmail/Google Apps and SMTP extension
2. New feature for Magento Gmail or Google Apps Email Extension
3. Google Apps Email/Gmail Magento Extension v0.5 released
4. SMTP Pro Magento extension, free and open SMTP support for Magento
5. Google Checkout disabled – Not available with these items