This is a blog post about how I structure my Magento store development, and how that has been made easier thanks to the efforts of Colin Mollenhour and his script modman. Hopefully after reading this, you’re all inspired to go set up a similar project structure, because it gives a repeatable, hopefully less upgrade ruined Magento experience. Certainly if you’re not inspired, I’d like to hear about how you are structuring your projects – the more talk of this, the better the overall quality of Magento stores in the wild, I hope.
Over the last nearly 2 years I’ve tried numerous project structures to make developing multiple Magento stores less painful. My background before getting in to ecommerce development has been in web development for large (think 100+ developer) web projects, and small, agile ones (2-10 developers). I know and appreciate how important project structure is for both teamwork, and rapid, predictable development. I thought it’d be illustrative to briefly plot my path through failed attempts at getting a workable Magento project structure. Begining with…
Not structuring your store at all
The first attempt at this was the ‘no-structure’
approach mistake – being exposed to only WordPress and Drupal PHP apps (and being of the opinion that all PHP scripts were just a little hackish), I thought that I’d be fine just installing Magento and getting started. I was for about 1 month, until I upgraded…
The reality is Magento is a lot more complicated than other popular PHP apps, and if you plan on actually engineering it, rather than just hacking it, you should structure it for that.
Sadly, it’s a very easy situation for a beginner at Magento to sort of fall in to – especially as most shared hosts will install Magento for you and basically set you up for some heartache when you go to upgrade, especially if you cut corners when customizing.
The whole store in version control structure
I’ve also tried having an entire Magento store checked-in to version control. This works OK, but doesn’t lend itself to sharing the same customizing effort across multiple stores (repeating bug fix and improvement effort), which at World Wide Access is a factor I’ve quickly realized we need to consider as we add more suppliers and product ranges.
Along side this approach I tried maintaining a ‘vendor branch’ which is described here. It’s a lot of work, and it turns out still causes problems when you try to upgrade – the best approach, I have found, is just to do all customizing as extensions.
I want to be clear here that I am talking about separate stores as in completely independent Magento installations, we create webstores for our suppliers, each is separate – with different inventory, and customers, so the Magento sub-store is not suitable for our purposes.
This is quite different to the Magento feature of websites, stores and store views. This allows you to create similar stores all contained within the same installation. If you use sub-stores in a single installation, then they all share the same code, extensions etc. On a personal note, the few times I have used this feature, I have been burned by parts of the code that incorrectly handle the store-scope, or extensions that do not properly use the store code – I’ve been guilty of it myself, actually.
Symlinking extensions into vanilla Magento installs
Fast-forward to about 6 months ago, and I had changed my development technique considerably. I started developing all my customizations exclusively as extensions, and all store specific stuff only within a package (and themes within it). These changes would be symlinked in to the Magento store on an as needed basis.
This meant the underlying Magento install was no longer part of my project, it just existed, and had to be installed, in order for the stores to function. This is very tenuously analogous to an app server and web app in the Java world I used to know and love (what have I become!). One difference between the two is that Sun doesn’t hose their API in some obscure way every point release, but Varien will get there – I’m sure.
Anyway, when set up this way each Magento store symlinks to the various components it needs from the repository. So if a store needs automatic inventory updates from Warehouse A, then I simply symlink in that extension, if some other store needs inventory from Warehouse B, then I symlink the Warehouse B extension. This can be seen in the diagram below (thanks to Google docs):
Now, if you need both warehouses, you symlink in the warehouse manager extension, and it will handle that for you. This allows me to develop each extension as a separate self-contained project, and share that effort easily between multiple stores. It drives you to create more generic solutions to customizations, so that they can be easily applied to different stores.
The diagram is slightly simplified in that I actually keep the production versions outside the code repository, on our server. In this way, when you’re ready to release a new and improved version of the Warehouse A extension, you can simply export and re-link like in this example:
svn export ../working/WarehouseA WarehouseA.`date +"%m-%d-%y"` ln -nfs WarehouseA.`date +"%m-%d-%y"` WarehouseA
Now all stores pointing to the production symlink for Warehouse A will get the new one when they next refresh their cache – which you can script too if you want to force them to update all at once. In addition, development stores can point to a development working copy of the extension for testing too.
In the same way each store would point to it’s version of a theme and package project, which is versioned in the same way. You can release a production version of these the same was as I describe above for extensions.
This worked pretty well and up until about a month ago it was where I was up to in terms of maintaining sets of common customizations across multiple stores. In fact the detail of how it works and the benefits in terms of genericty was to form part my presentation at the Magento Developers Paradise conference in April – which is now in October, thanks to the volcano in Iceland.
One of the things that pained me a little was the constant adding of symlinks everywhere throughout the store structure (For example in
app/design etc ). Thankfully this has lately been made easier by a script called modman.
Enter Colin’s brilliant modman script. It’s like he knew exactly a tool that would make my life easier, and made it available for everyone!
Using Modman for Magento
Modman automates the creation of my symlinks from within a nested SVN working copy hidden away in the root of a Magento store. It can do a lot of other cool stuff too, mind you, but for me the primary benefit is the notion of a store specific set of symlinks – the mapping of which is itself kept in version control and maintained automatically.
I was going to put a big modman tutorial here – but this post is already very long, and Colin has documented the script well. Unless I hear otherwise I’ll assume you can all follow his instructions clearly. What I will do though, is prepare a from scratch Magento project setup guide as my next blog post – some time mid-2011 at this rate!
In any case you should download and try it yourself – you can get a free source code repository from a number of places online.
The Benefits of Structuring your Magento Development project
There are a few benefits worth noting to this approach:
- Repeatability & Team work: All changes to Magento are made on the basis that they will be operating on a vanilla Magento install. Multiple developers can set up the environment easily, and will expect the same results, every time.
- Reusing code & minimizing effort and duplication – if you fix a bug in your custom extension, you only need to fix it in one place, then the changes propagate to all stores using it.
- Genericty becomes more valuable: You strive to create custom extensions that will have use for all your stores. You get by-products of customization you can sell (or give away).
In addition, the modman script has saved me a great deal of time, and given an added element of repeatability to the development and deployment of each store. I think using it will make the whole process of setting up a structured Magento development project a lot easier. As a result I’d like to see more stores adopting a similar approach to Magento development – so hopefully an end-to-end tutorial will help with that.
One thing I’m not convinced of yet is the use of externals in SVN to allow each store to ‘point’ to it’s required extensions. What I want to achieve is for the external to point to the equivalent of my ‘production version’ of the extension. The problem is you can’t have a tag in SVN called ‘production’ and keep re-tagging your HEAD revision to production each time you’re ready to release. Instead I have to move the current production release to a .date version and copy the new one in. Mildy annoying – not a big problem though.
This is shown in a contrived little example graphically:
I’d love to hear from anyone with a better solution to this part of the structure, maybe I could check in symlinks, but I’m not sure how that would work within the context of modman projects.
I’ve tried to provide some background for my Magento development process, how it’s evolved and why it’s arrived to where it is today. I covered how it works now and explained the benefits I get from it, and the still unresolved issues I found with it. It’s a summary of what I’m doing for Multi-Magento store development, where all customizations are via extensions and frontend packages. I’d really love to hear from other developers who are solving these problems, particularly those doing it better ways! So please, don’t be shy.
PS: One other thing, if you are using the script on Mac OSX – be careful something in the script doesn’t seem to like the broken symlinks detection. I had to make a couple of changes, which mean that old symlinks won’t get cleaned up on Mac OSX – not a big deal.
I just changed the
find -L $root -type l -delete command (in two places) to:
#The echo out this command and run it manually if you have to. echo "find -L $root -type l -delete"