The latest version of Magento (1.7rc1 right now) has support for oAuth authenticated REST APIs, yippee! In this post I’m going to run through the technical aspects of setting up the Magento oAuth and making your first Magento REST API call.
I’m presenting on APIs during the technology track at the Magento Imagine conference on Tuesday (If you’re reading this, and you’re at the conference, I hope to see you there!). In my talk I’ll cover interesting uses and benefits of the REST API but won’t have time for a big fully worked example. This post details the finer points of actually using these APIs which is a bit too low-level to cover, so this will be extra-for-experts homework for those that want to have a play around with REST after the talk.
There are basically 3 steps to running your first REST API call in Magento.
0) Get an oAuth client tool
Let me say right at the outset, debugging oAuth issues is one of those crappy problems in web development, like character encoding and object aliasing, it is a real pain when it doesn’t just work™ and you’re going to want a good toolset if things go sour.
In my examples below I’m using the ruby oAuth library because it’s clean simple and works well. There’s something similar for python and PHP, so choose your poison. To set up the ruby one, assuming you have ruby already, just run:
sudo gem install oauth |
Right, now you’re at the start line.
1) Set up the Magento server
You need to configure the role for the oAuth, specify the attributes and lastly create a consumer.
For the purposes of this tutorial to keep it simple, you can just grant the customer all the attributes and permissions – but if you were being cautious, you’d pick and choose to suit your needs.
The roles are similar to the old web service roles, they allow you to select which permissions a type of REST user has. The attributes allow you to select which entity attributes appear in the response. Customers probably would not see cost, for example.
Now, make your consumer, and take a note of the consumer key and secret.
(eek, the fields are disabled, not readonly – I used firebug to copy-paste, or you can view source.)
Now the server is setup – you should have the roles, attributes and consumer configured, and for testing you’ll need a test customer account too. Now we are ready for the actual oAuth requests.
2) Get the oAuth tokens
The oAuth dance, oops I mean dance is not as fun as it sounds – but when it works, it’s a painless way for a web service client to be granted permission to call a service, without exposing any user credentials, which is great for security.
Here’s an example of that authorization process:
oauth \ --verbose \ --query-string \ --consumer-key v484mnii2jyswedm6uo2cfcjay7uy49snws \ --consumer-secret koay6845che7giy5lr17gnrhckkbhf8h5 \ --access-token-url http://www.yourstore.com/magento/oauth/token \ --authorize-url http://www.yourstore.com/magento/oauth/authorize \ --request-token-url http://www.yourstore.com/magento/oauth/initiate \ authorize Server appears to support OAuth 1.0a; enabling support. Please visit this url to authorize: http://www.yourstore.com/magento/oauth/authorize?oauth_token=ey6fokp0pbzwr1016eb528y5xw1ak5ji Please enter the verification code provided by the SP (oauth_verifier): YOUR_CODE_HERE Response: oauth_token_secret: g9kyz8c7zv868d58eav1muih3gxvq763 oauth_token: aqvlfv9tuexn0mqiydgkaff4ixxg8743c |
The authorize url you visit will ask you to log in (as a customer, unless you use the admin authorize url instead: /admin/oAuth_authorize). You’ll login and then authorize the client like this.
After a successful login and authorize you should see this dialog, and you’ll copy the verifier code into the oauth commandline to complete the process.
3) Make the API call
You’ll want to use the oauth command line tool (or your preferred tool) to create the oAuth signature and generate the URL to play around with. Here’s an example using the debug option, it’ll give you an idea of what’s going on, and displays the copy-pastable URL you’ll use in the next step:
oauth \
--consumer-key v484mnii2jyswedm6uo2cfcjay7uy49snws \
--consumer-secret koay6845che7giy5lr17gnrhckkbhf8h5 \
--token aqvlfv9tuexn0mqiydgkaff4ixxg8743c \
--secret g9kyz8c7zv868d58eav1muih3gxvq763 \
--uri http://www.yourstore.com/magento/api/rest/products \
debug |
Note: the URL I am going to use is http://www.yourstore.com/magento/api/rest/products – but you can see all the possible urls in the api2.xml files in the core code. For example here’s a snippet from the catalog api2.xml that shows some of the path patterns:
<routes> <route_entity> <route>/products/:id</route> <action_type>entity</action_type> </route_entity> <route_entity_with_store> <route>/products/:id/store/:store</route> <action_type>entity</action_type> </route_entity_with_store> <route_collection> <route>/products</route> <action_type>collection</action_type> </route_collection> ... </routes |
You can choose your favorite HTTP client (the URL will work in any browser), I’ll use curl below, for demonstrative purposes. Grab the ‘OAuth Request URI’ from the output above and use it like I do below.
Getting a list of products in JSON (JSON is the default, you can omit the Accept header if you want):
curl \
-H "Accept: application/json" \
"http://www.yourstore.com/magento/api/rest/products?oauth_consumer_key=tydkxii2jyswedm6uo2cfcjay7uy0crws&oauth_nonce=HWJHh83HKeJnsFahdQxye52dapt8F7bswrnLcAYjsy&oauth_signature=%2FkHLJKs9W3fmWRVxAPrd8R7LHDs%3D&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1335162435&oauth_token=aqvlfv9tuexn0mqiydgkaff4ixxgs58s&oauth_version=1.0" |
Getting a list of products in XML (set the Accept header for XML):
curl \
-H "Accept: application/xml" \
"http://www.yourstore.com/magento/api/rest/products?oauth_consumer_key=tydkxii2jyswedm6uo2cfcjay7uy0crws&oauth_nonce=HWJHh83HKeJnsFahdQxye52dapt8F7bswrnLcAYjsy&oauth_signature=%2FkHLJKs9W3fmWRVxAPrd8R7LHDs%3D&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1335162435&oauth_token=aqvlfv9tuexn0mqiydgkaff4ixxgs58s&oauth_version=1.0" |
All going to plan you should see some JSON or XML output that represents your products, phew!
Lastly I’d like to say ‘Thanks’ to Lindy, Baruch and Vinai from Magento for their help getting started with an example test harness (and rushing out a 1.7 WS-I bug fix for me!). There’s precious little documentation currently, but hopefully this post will help others get started.
For PHP coders interested, here’s the test harness courtesy of Lindy you can dig into a bit more. (Note: all copyrights etc to Magento)
And that’s it, you can now make your own REST calls and dream up cool and interesting ways to allow customers and admins to access Magento store data! I’ll cover some possibilities for this cool new technology in my talk on Tuesday, hope to see you all there – I’ll only accept inexplicable tigers and/or chickens in your hotel room as an excuse!
Could you drop in at the Wynn and check whether my 600 USD is still there? Went there for CES in Jan and somehow lost some cash there, strange :S
Enjoy the trip!
Hi,
I’m hiting error when i do :
oauth –verbose –query-string –consumer-key mykey –consumer-secret mysecret –access-token-url http://ec2-23-22-82-120.compute-1.amazonaws.com/magento/oauth/token –authorize-url http://ec2-23-22-82-120.compute-1.amazonaws.com/magento/oauth/authorize –request-token-url http://ec2-23-22-82-120.compute-1.amazonaws.com/magento/oauth/initiate authorize
A problem occurred while attempting to authorize:
404 Not Found
404 Not Found
Not Found
The requested URL /opt/bitnami/apps/magento/htdocs/index.php was not found on this server.
Apache/2.2.15 (Red Hat) Server at ec2-23-22-82-120.compute-1.amazonaws.com Port 80
can you help?
Where’s this path coming from?
/opt/bitnami/apps/magento/htdocs/index.phpI’d say that’s the place to start looking.
I’ve installed on amazon ec2 … a image from bitnami.
Everything else it’s working, I just cant oauth 🙁
Regards
Joao
After installing magento did you had to configure any http preference? Or any other thing?
If you install a fresh magento 1.7 does it work?
Regards
Joao
Yeh shouldn’t need any changes to HTTP prefs. What have you tried in terms of debug?
hi,
when i do:
curl -H “Accept: application/json” “http://localhost/api/rest/products?oauth_consumer_key=0cgpl85pwt5yzirgzu48tqzlcbtq1u1f&oauth_nonce=a5HFSxOWaq9fn8KV8IpNKeGwQ2soYOksVEQdDW68&oauth_signature=XO4rvz8Fx1a2Q9L8RkCCpsfBwv4%3D&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1337160419&oauth_token=4dgj0kc2jqlwg3pk4r0det7a9kbs2uud&oauth_version=1.0
i get:
{“messages”:{“error”:[{“code”:403,”message”:”Access denied”}]}}
what is the matter?
Regards
lpc
After setting up magento did you had to set up any http preference? Or any other thing?
If you set up a clean magento 1.7 does it work?
@All
witch OS did u use? i got the magento examples to work on fedora and windows 7 but neither on ubuntu nor mint. On all linux distributions i installed lammp and pecl oauth. After setting up magento i didnt make any http preferences.
Hello Ashley!
Greetings of Brazil.
I have not many questions:
1 – What is the oAuth version for use about magento community 1.7?
2 – RubyGem works with Magento 1.7 ? or is prefered download other version? which version do you recommend?
3 – I can’t understand your step (2), This is a command on linux ?
or any modified file in magento structure ?
sorry for my english
Thanks for the great post, just had one question. for some reason when i try to get a list of products in the last step i am getting the html of “page you requsted is not found”
any idea?
if i use this url uniquely in browser my response is:
The requested URL /magento/api/rest/index.php/products was not found on this server.
http://myIP/magento/api/rest/products?limit=2
The requested URL /magento/api/rest/index.php/products was not found on this server.
this too:
http://myIP/magento/index.php/api/rest/products?limit=2
And All was configured.
Hi Ashley,
it’s really driving me crazy…
Hopefully you have a hint for me. I try to use Magento’s REST API through the python restkit.
I managed the first 2 legs to “run”, so I get successfully the “final” access tokens. The last call (resource-call) fails with a 403.
Could you kindly have a look at
http://juky.org/magento.py
Thx in advance, Pit.
sorry, please
http://juky.org/magento.txt
There was a prob with the uploaded .py
Are you 100% sure you have the resource permissions set up right? And you are authenticating as an admin, right?
I set the permissions to “all” for all three user types – no change.
Then I tested it from the Mozilla-RESTClient plugin – success! With Python restkit – still 403.
Then I changed to the lib from http://python-requests.org and the oauth-extension: NOW IT WORKS!
Hi Ashley, good write up. Do you have any examples of extending the Rest API on Magento with a new function in a class that takes some parameters? Running into resource not found issues and suspect it is an issue with my xml etc files. thanks.
I am trying to get
getRequestToken(“http://61.16.162.229/magento/oauth/token”);
if(!empty($request_token_info)) {
print_r($request_token_info);
} else {
print “Failed fetching request token, response was: ” . $oauth->getLastResponse();
}
} catch(OAuthException $E) {
echo “Response: “. $E->lastResponse . “\n”;
}
?>
but getting
Not Found
The requested URL /magento/oauth/token was not found on this server.
Magento installed in 61.16.162.229/magento/ folder
ok, I solved this, changed /etc/apache2/sites-enabled/000-default
from
DocumentRoot /var/www
Options FollowSymLinks
AllowOverride None
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Order allow,deny
allow from all
to
DocumentRoot /var/www
Options FollowSymLinks
AllowOverride All
Options Indexes FollowSymLinks MultiViews
AllowOverride All
Order allow,deny
allow from all
Hey Ashley!
Thanks for the sweet article – you’ve laid it all out beautifully. I’m hitting a weird access denied snag.
1. Using the ruby oauth client.
2. REST Role – Created a new REST Role (“Admin Consumer”). Initially gave it access to *all* resources, when that didn’t work, changed it to custom, selecting all checkboxes.
Initially, under Role Users, it didn’t show any. When I started hitting access denied issues, I went in here and searched for my admin user (username: ‘admin’) and assigned them to the role.
3. REST Attributes – not totally clear to me still how the attribute set is getting associated to the role or consumer, but I’ve set attributes to all for both Admin and Customer User Types. As with role, I tried “all” as well as Custom with all checkboxes checked.
4. Consumer – created.
5. Got the oauth tokens using the command-line call to the oauth ruby client. Only difference is that I used the /admin/oAuth_authorize endpoint. The oauth flow went through just fine, got the verification token, got the the copy-pastable request URI from the oauth debug call.
But I get access denied when I make the call to get products
Request:
curl http://m.dev/mc1700/api/rest/products?oauth_consumer_key=klr6t0h7egq7xwqbzqj1tjn0dt2dhb0g&oauth_nonce=oh7GQNO5exzZn1PkUJUjmr9zJyAEsvd7DUkhj0W4dgU&oauth_signature=niJ07g3c8MhA6BLqvYIyy6cM3c0%3D&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1351533398&oauth_token=i35hybaai3wraexr3fzly7vxtjg83463&oauth_version=1.0
Response:
{“messages”:{“error”:[{“code”:403,”message”:”Access denied”}]}}
I’m gonna start debugging the Mage_Api2 module to see where it’s breaking, but thought I’d throw this comment up here in case you had any ideas!
Thanks again for the awesome article!
Figured it out. It doesn’t seem to accept the OAuth parameters as query string parameters. I tried sending them as HTTP headers instead and it’s going through.
For example: (new keys, etc. b/c I set this up on a new install to be safe 🙂 )
Authorization:OAuth realm=”http://m.dev/”,oauth_consumer_key=”0npq6crnxe0vo3iuwrdeg2x31pxjy91n”,oauth_token=”03mbcsplycit0uy77f0fk8sl23amkmk3″,oauth_nonce=”LsSLJpBZ6Xp1zQ6CkAnG2MDYYkHMQhSVOwgU3T7Jw”,oauth_signature_method=”HMAC-SHA1″,oauth_timestamp=”1351537874″,oauth_version=”1.0″,oauth_signature=”VPIVV0wltKt7LHJC2UDVVMUj5i0%3D”
Want to throw up another weird thing I’m noticing. If I don’t specify the –method as a parameter to the oauth command, it defaults to “post”. Actually at one point it was defaulting to “get”. At any rate, the request method is case sensitive. Unless I specify –method GET (all caps), I get an invalid signature response.
Noticed that because I popped over to this oauth validation tool:
http://quonos.nl/oauthTester/
Which will tell you that the method is case sensitive if you paste in the Signature Base String.
Thanks for the continued follow ups on this Kalen, I think there’s not too many devs really pushing this functionality in the wild yet so it’s really good to have some real world feedback to go alongside the tutorial.
Can you share what you’re working on or your use case?
Is there a way to pass login credentials from code instead of entering credentials everytime in the popup for login authorization in Magento REST API?
Magento 1.7.0.2 fresh install
OAuth for Ruby 0.4.7
Set up REST consumer key and secret as below through Admin plus permissions. Can’t get past initiate/auth!
I can see that the signatures do not match if I print out the one generated by Magento in app/code/core/Mage/Oauth/Model/Server.php line 545. Am I doing something silly?
$ oauth –verbose –query-string –consumer-key dbyf8fm41rc3m6r7ubfz02esx6hfei99 –consumer-secret ey4r833r35z08jpfek1rzga5gmk3oitz –access-token-url http://magento.north-row.com:10001/oauth/token –authorize-url http://magento.north-row.com:10001/admin/oauth_authorize –request-token-url http://magento.north-row.com:10001/oauth/initiate authorize
A problem occurred while attempting to authorize:
401 Authorization Required
oauth_problem=signature_invalid&debug_sbs=TVXjIJdJWQDCsRyir4PCaXeqYK4=
SOLVED: port is stripped from hostname by the underlying Zend signature creation utility and therefore the signatures no longer match.
I did exactly the same but without port 10001 (and instead port 80) and it now works.
Had to dig deep to find the answer 🙂
Anybody know how to get the a configurable products associated simple products (and options for the configurable) via the api?
Thank you. This is the most useful post I have come accross related to Magento REST API for 1.7.
I found issues with using curl due to the requirement for Authorization: oAuth in the header and problems with the signature showing as invalid. I used the script here: http://stackoverflow.com/questions/15541952/create-product-using-oauth-curl and works fine after the authorisation has been done. Also repeated here:
<?php
$nonce = substr(md5(uniqid('nonce_', true)),0,16);
$temprealm="http://www.localhost.com/magento/api/rest/products";
$realm=urlencode($temprealm);
$oauth_version="1.0";
$oauth_signature_method="HMAC-SHA1";
$oauth_consumer_key="ngn4vt1vxw8asd5viximgr0hbc2rbvcd";
$oauth_access_token="9zsfaxfdwktcuxhqa3w5pk88fut61my2";
$oauth_method="GET";
$oauth_timestamp=time();
$algo="sha1";
$key="xyvhm65nwr12939y0g0t288seqhzm780&c5ajpmrxyuvxlron7o1dk07sbl8zgy5a"; //consumer secret & token secret //Both are used in generate signature
$data="oauth_consumer_key=".$oauth_consumer_key."&oauth_nonce=".$nonce."&oauth_signature_method=".$oauth_signature_method."&oauth_timestamp=".$oauth_timestamp."&oauth_token=".$oauth_access_token."&oauth_version=".$oauth_version;
$send_data=$oauth_method."&".$realm."&".urlencode($data);
$sign=hash_hmac($algo,$send_data,$key,1); // consumer key and token secrat used here
$fin_sign=base64_encode($sign);
$curl = curl_init();
curl_setopt($curl,CURLOPT_HTTPHEADER,array('Authorization : OAuth realm='.$realm.', oauth_version="1.0", oauth_signature_method="HMAC-SHA1", oauth_nonce="'.$nonce.'", oauth_timestamp="'.$oauth_timestamp.'", oauth_consumer_key='.$oauth_consumer_key.', oauth_token='.$oauth_access_token.', oauth_signature="'.$fin_sign.'"'));
curl_setopt ($curl, CURLOPT_URL,$temprealm);
$xml=curl_exec($curl);