API Main Practical

From Grid5000
Revision as of 19:05, 10 March 2010 by Crohr (talk | contribs) (New page: = Presentation = This practical will walk you through the standard use cases of the Grid5000 APIs: * Use the Reference API to discover the sites, clusters and nodes available in Grid5000; ...)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Presentation

This practical will walk you through the standard use cases of the Grid5000 APIs:

  • Use the Reference API to discover the sites, clusters and nodes available in Grid5000;
  • Use the Monitoring API to check on the status of the nodes (free, busy, alive, dead, ...);
  • Use the Jobs API to submit a new job on a specific site;
  • Use the Deployments API to launch a deployment of a specific image on a few nodes of your choice (on which you have a job running);
  • Combine all these APIs to launch an experiment on multiple sites.

Before we start, You may (and should) have a look at the introductory page about the Grid5000 APIs, that will give you the general principles of the APIs.

Prerequisites

Set up your environment

  • All the examples will use the Ruby language, or the cURL command-line tool, so you'd better get them on your machine:
 apt-get install ruby libopenssl-ruby libjson-ruby curl rubygems
  • We'll use the rest-client library to make our HTTP requests:
 gem install rest-client
  • We'll tell Ruby to always load the rubygems library:
 echo "export RUBYOPT='-rubygems'" >> ~/.profile && source ~/.profile
  • cURL can read your Grid5000 credentials from a specific file located at .netrc instead of having to specify them on the command line. Thus, create a ~/.netrc file containing your credentials on your machine:
 echo "machine api.grid5000.fr login your-grid5000-login password your-grid5000-password" >> ~/.netrc && chmod 600 ~/.netrc
  • Same works for the rest-client library:
 echo "grid5000:
   url: https://api.grid5000.fr
   username: your-grid5000-login
   password: your-grid5000-password" >> ~/.restclient && chmod 600 ~/.restclient

Test that everything works as expected

With cURL, enter the following command:

 curl -kni https://api.grid5000.fr/sid/grid5000

You should get back the following response:

 HTTP/1.1 200 OK
 ... TRUNCATED ...
 
 {
   "uid": "grid5000",
   "type": "grid",
   "version": "e5335be2526e4292668dd67e616022384ca5708c",
   "links": [
     {
       "href": "/sid/grid5000/versions/e5335be2526e4292668dd67e616022384ca5708c",
       "title": "version",
       "rel": "member",
       "type": "application/vnd.fr.grid5000.api.Version+json;level=1"
     },
     {
       "href": "/sid/grid5000/versions",
       "title": "versions",
       "rel": "collection",
       "type": "application/vnd.fr.grid5000.api.Collection+json;level=1"
     },
     {
       "href": "/sid/grid5000",
       "rel": "self",
       "type": "application/vnd.fr.grid5000.api.Grid+json;level=1"
     },
     {
       "href": "/sid/grid5000/environments",
       "title": "environments",
       "rel": "collection",
       "type": "application/vnd.fr.grid5000.api.Collection+json;level=1"
     },
     {
       "href": "/sid/grid5000/sites",
       "title": "sites",
       "rel": "collection",
       "type": "application/vnd.fr.grid5000.api.Collection+json;level=1"
     }
   ]
 }
 

With the ruby RestClient, start the interactive shell:

 $ restclient grid5000

Then make a request:

 > grid5000 = get '/sid/grid5000'
  => 200 OK | application/vnd.fr.grid5000.api.grid+json 998 bytes

Test that you have correctly installed the JSON parser:

 > require 'json'
  => true
 > JSON.parse grid5000.body
  => {"uid"=>"grid5000", "version"=>"e5335be2526e4292668dd67e616022384ca5708c", "type"=>"grid", "links"=>[{"href"=>"/sid/grid5000/versions/e5335be2526e4292668dd67e616022384ca5708c", "title"=>"version", "rel"=>"member", "type"=>"application/vnd.fr.grid5000.api.Version+json;level=1"}, {"href"=>"/sid/grid5000/versions", "title"=>"versions", "rel"=>"collection", "type"=>"application/vnd.fr.grid5000.api.Collection+json;level=1"}, {"href"=>"/sid/grid5000", "rel"=>"self", "type"=>"application/vnd.fr.grid5000.api.Grid+json;level=1"}, {"href"=>"/sid/grid5000/environments", "title"=>"environments", "rel"=>"collection", "type"=>"application/vnd.fr.grid5000.api.Collection+json;level=1"}, {"href"=>"/sid/grid5000/sites", "title"=>"sites", "rel"=>"collection", "type"=>"application/vnd.fr.grid5000.api.Collection+json;level=1"}]}

If you didn't encounter any errors until now, your environment is set up.

Practical

Discover Grid5000

Let's say you're interested in knowing the number of sites that compose Grid5000. Launch the restclient interactive shell:

 $ restclient grid5000

and start the journey by GETting the description of the grid:

 > grid5000 = get '/sid/grid5000'
  => 200 OK | application/vnd.fr.grid5000.api.grid+json 998 bytes

The 200 status code tells us that the request succeeded (as you can guess by the human readable OK message). The grid5000 variable contains a RestClient::Response object:

 > grid5000.class
  => RestClient::Response

A RestCLient::Response object has 3 methods of interest:

  • #code will return the HTTP status code of the response:
 > grid5000.code
  => 200
  • #body will return the body of the response returned by the remote server:
 > grid5000.body
  => "{\n  \"uid\": \"grid5000\",\n  \"type\": \"grid\",\n  \"version\": \"e5335be2526e4292668dd67e616022384ca5708c\",\n  \"links\": [\n    {\n      \"href\": \"/sid/grid5000/versions/e5335be2526e4292668dd67e616022384ca5708c\",\n      \"title\": \"version\",\n      \"rel\": \"member\",\n      \"type\": \"application/vnd.fr.grid5000.api.Version+json;level=1\"\n    },\n    {\n      \"href\": \"/sid/grid5000/versions\",\n      \"title\": \"versions\",\n      \"rel\": \"collection\",\n      \"type\": \"application/vnd.fr.grid5000.api.Collection+json;level=1\"\n    },\n    {\n      \"href\": \"/sid/grid5000\",\n      \"rel\": \"self\",\n      \"type\": \"application/vnd.fr.grid5000.api.Grid+json;level=1\"\n    },\n    {\n      \"href\": \"/sid/grid5000/environments\",\n      \"title\": \"environments\",\n      \"rel\": \"collection\",\n      \"type\": \"application/vnd.fr.grid5000.api.Collection+json;level=1\"\n    },\n    {\n      \"href\": \"/sid/grid5000/sites\",\n      \"title\": \"sites\",\n      \"rel\": \"collection\",\n      \"type\": \"application/vnd.fr.grid5000.api.Collection+json;level=1\"\n    }\n  ]\n}\n" 
  • #headers will return the HTTP headers [1] of the response:
 > grid5000.headers
  => {:status=>"200", :x_cache_lookup=>"HIT from api-proxy.rennes.grid5000.fr:3128", :x_powered_by=>"Phusion Passenger (mod_rails/mod_rack) 2.2.3", :vary=>"Accept", :etag=>"\"dc7f095c150b4b5265c02e56634b465c3895b4b0\"", :cache_control=>"public, must-revalidate", :content_type=>"application/vnd.fr.grid5000.api.grid+json;level=1", :content_length=>"998", :date=>"Tue, 09 Mar 2010 16:05:02 GMT", :last_modified=>"Mon, 08 Mar 2010 15:28:53 GMT", :age=>"6799", :via=>"1.0 api-server.rennes.grid5000.fr, 1.0 api-proxy.rennes.grid5000.fr:3128 (squid/2.6.STABLE18)", :server=>"Apache/2.2.9 (Debian) Phusion_Passenger/2.2.3 mod_ssl/2.2.9 OpenSSL/0.9.8g", :x_cache=>"HIT from api-proxy.rennes.grid5000.fr", :allow=>"GET"}

Let's display the content of the grid5000 response:

 > puts grid5000.body
 {
   "uid": "grid5000",
   "type": "grid",
   "version": "e5335be2526e4292668dd67e616022384ca5708c",
   "links": [
     {
       "href": "/sid/grid5000/versions/e5335be2526e4292668dd67e616022384ca5708c",
       "title": "version",
       "rel": "member",
       "type": "application/vnd.fr.grid5000.api.Version+json;level=1"
     },
     {
       "href": "/sid/grid5000/versions",
       "title": "versions",
       "rel": "collection",
       "type": "application/vnd.fr.grid5000.api.Collection+json;level=1"
     },
     {
       "href": "/sid/grid5000",
       "rel": "self",
       "type": "application/vnd.fr.grid5000.api.Grid+json;level=1"
     },
     {
       "href": "/sid/grid5000/environments",
       "title": "environments",
       "rel": "collection",
       "type": "application/vnd.fr.grid5000.api.Collection+json;level=1"
     },
     {
       "href": "/sid/grid5000/sites",
       "title": "sites",
       "rel": "collection",
       "type": "application/vnd.fr.grid5000.api.Collection+json;level=1"
     }
   ]
 }
  => nil
Note.png Note

You can get the same output by just calling puts grid5000

You may recognize this content as being JSON (JavaScript Object Notation) encoded, and/or read the Content-Type HTTP header of the response to get that information:

 > grid5000.headers[:content_type]
  => "application/vnd.fr.grid5000.api.grid+json;level=1"

Thus, we could use a JSON parser to transform the JSON content into a Ruby object:

 > require 'json'
  => true
 > parsed_response = JSON.parse grid5000.body
  => {"uid"=>"grid5000", "version"=>"e5335be2526e4292668dd67e616022384ca5708c", "type"=>"grid", "links"=>[{"href"=>"/sid/grid5000/versions/e5335be2526e4292668dd67e616022384ca5708c", "title"=>"version", "rel"=>"member", "type"=>"application/vnd.fr.grid5000.api.Version+json;level=1"}, {"href"=>"/sid/grid5000/versions", "title"=>"versions", "rel"=>"collection", "type"=>"application/vnd.fr.grid5000.api.Collection+json;level=1"}, {"href"=>"/sid/grid5000", "rel"=>"self", "type"=>"application/vnd.fr.grid5000.api.Grid+json;level=1"}, {"href"=>"/sid/grid5000/environments", "title"=>"environments", "rel"=>"collection", "type"=>"application/vnd.fr.grid5000.api.Collection+json;level=1"}, {"href"=>"/sid/grid5000/sites", "title"=>"sites", "rel"=>"collection", "type"=>"application/vnd.fr.grid5000.api.Collection+json;level=1"}]}

As you may have noticed, the response contains a link property, which is a list of relations to other resources exposed by the APIs:

 > parsed_response['links']
  => [{"href"=>"/sid/grid5000/versions/e5335be2526e4292668dd67e616022384ca5708c", "title"=>"version", "rel"=>"member", "type"=>"application/vnd.fr.grid5000.api.Version+json;level=1"}, {"href"=>"/sid/grid5000/versions", "title"=>"versions", "rel"=>"collection", "type"=>"application/vnd.fr.grid5000.api.Collection+json;level=1"}, {"href"=>"/sid/grid5000", "rel"=>"self", "type"=>"application/vnd.fr.grid5000.api.Grid+json;level=1"}, {"href"=>"/sid/grid5000/environments", "title"=>"environments", "rel"=>"collection", "type"=>"application/vnd.fr.grid5000.api.Collection+json;level=1"}, {"href"=>"/sid/grid5000/sites", "title"=>"sites", "rel"=>"collection", "type"=>"application/vnd.fr.grid5000.api.Collection+json;level=1"}]

Or, require the pp standard library to pretty print the parsed_response['links'] array:

 > require 'pp'
  => true
 > pp parsed_response['links']
 [{"href"=>"/sid/grid5000/versions/e5335be2526e4292668dd67e616022384ca5708c",
   "title"=>"version",
   "rel"=>"member",
   "type"=>"application/vnd.fr.grid5000.api.Version+json;level=1"},
  {"href"=>"/sid/grid5000/versions",
   "title"=>"versions",
   "rel"=>"collection",
   "type"=>"application/vnd.fr.grid5000.api.Collection+json;level=1"},
  {"href"=>"/sid/grid5000",
   "rel"=>"self",
   "type"=>"application/vnd.fr.grid5000.api.Grid+json;level=1"},
  {"href"=>"/sid/grid5000/environments",
   "title"=>"environments",
   "rel"=>"collection",
   "type"=>"application/vnd.fr.grid5000.api.Collection+json;level=1"},
  {"href"=>"/sid/grid5000/sites",
   "title"=>"sites",
   "rel"=>"collection",
   "type"=>"application/vnd.fr.grid5000.api.Collection+json;level=1"}]
  => nil 

Each member of the list has at least two properties:

  • rel indicates the relation between the current resource and the linked resource;
  • href is the relative or absolute URI that at which the linked resource is located

The type and title properties should be pretty self-explanatory.

As we can see, there is a link whose title is sites. It's probably a good idea to GET it to see if it contains the list of the Grid5000 sites:

 > sites = get parsed_response['links'].find{|link| link['title'] == 'sites'}['href']
  => 200 OK | application/vnd.fr.grid5000.api.collection+json 21822 bytes

Seems like we got back a collection:

 > puts sites
 {
   "items": [
     {
       "name": "Bordeaux",
       "latitude": 44.833333,
       "location": "Bordeaux, France",
       "security_contact": "bordeaux-staff@lists.grid5000.fr",
       "uid": "bordeaux",
       "type": "site",
       "user_support_contact": "bordeaux-staff@lists.grid5000.fr",
       "version": "e5335be2526e4292668dd67e616022384ca5708c",
       "links": [
         {
           "href": "/sid/grid5000/sites/bordeaux/versions/e5335be2526e4292668dd67e616022384ca5708c",
           "title": "version",
           "rel": "member",
           "type": "application/vnd.fr.grid5000.api.Version+json;level=1"
         },
         {
           "href": "/sid/grid5000/sites/bordeaux/versions",
           "title": "versions",
           "rel": "collection",
           "type": "application/vnd.fr.grid5000.api.Collection+json;level=1"
         },
         {
           "href": "/sid/grid5000/sites/bordeaux",
           "rel": "self",
           "type": "application/vnd.fr.grid5000.api.Site+json;level=1"
         },
         {
           "href": "/sid/grid5000/sites/bordeaux/clusters",
           "title": "clusters",
           "rel": "collection",
           "type": "application/vnd.fr.grid5000.api.Collection+json;level=1"
         },
         {
           "href": "/sid/grid5000/sites/bordeaux/environments",
           "title": "environments",
           "rel": "collection",
           "type": "application/vnd.fr.grid5000.api.Collection+json;level=1"
         },
         {
           "href": "/sid/grid5000",
           "rel": "parent",
           "type": "application/vnd.fr.grid5000.api.Grid+json;level=1"
         },
         {
           "href": "/sid/grid5000/sites/bordeaux/status",
           "title": "status",
           "rel": "collection",
           "type": "application/vnd.fr.grid5000.api.Collection+json;level=1"
         },
         {
           "href": "/sid/grid5000/sites/bordeaux/deployments",
           "title": "deployments",
           "rel": "collection",
           "type": "application/vnd.fr.grid5000.api.Collection+json;level=1"
         },
         {
           "href": "/sid/grid5000/sites/bordeaux/jobs",
           "title": "jobs",
           "rel": "collection",
           "type": "application/vnd.fr.grid5000.api.Collection+json;level=1"
         }
       ],
       "description": "Grid5000 Bordeaux site",
       "longitude": -0.566667,
       "email_contact": "bordeaux-staff@lists.grid5000.fr",
       "web": "http://www.grid5000.fr/mediawiki/index.php/Bordeaux:Home",
       "sys_admin_contact": "bordeaux-staff@lists.grid5000.fr"
     },
     .... TRUNCATED ....
     {
       "name": "Toulouse",
       "latitude": 43.6167,
       "location": "Toulouse, France",
       "security_contact": null,
       "uid": "toulouse",
       "type": "site",
       "user_support_contact": null,
       "version": "e5335be2526e4292668dd67e616022384ca5708c",
       "links": [
         {
           "href": "/sid/grid5000/sites/toulouse/versions/e5335be2526e4292668dd67e616022384ca5708c",
           "title": "version",
           "rel": "member",
           "type": "application/vnd.fr.grid5000.api.Version+json;level=1"
         },
         {
           "href": "/sid/grid5000/sites/toulouse/versions",
           "title": "versions",
           "rel": "collection",
           "type": "application/vnd.fr.grid5000.api.Collection+json;level=1"
         },
         {
           "href": "/sid/grid5000/sites/toulouse",
           "rel": "self",
           "type": "application/vnd.fr.grid5000.api.Site+json;level=1"
         },
         {
           "href": "/sid/grid5000/sites/toulouse/clusters",
           "title": "clusters",
           "rel": "collection",
           "type": "application/vnd.fr.grid5000.api.Collection+json;level=1"
         },
         {
           "href": "/sid/grid5000/sites/toulouse/environments",
           "title": "environments",
           "rel": "collection",
           "type": "application/vnd.fr.grid5000.api.Collection+json;level=1"
         },
         {
           "href": "/sid/grid5000",
           "rel": "parent",
           "type": "application/vnd.fr.grid5000.api.Grid+json;level=1"
         },
         {
           "href": "/sid/grid5000/sites/toulouse/status",
           "title": "status",
           "rel": "collection",
           "type": "application/vnd.fr.grid5000.api.Collection+json;level=1"
         },
         {
           "href": "/sid/grid5000/sites/toulouse/deployments",
           "title": "deployments",
           "rel": "collection",
           "type": "application/vnd.fr.grid5000.api.Collection+json;level=1"
         },
         {
           "href": "/sid/grid5000/sites/toulouse/jobs",
           "title": "jobs",
           "rel": "collection",
           "type": "application/vnd.fr.grid5000.api.Collection+json;level=1"
         }
       ],
       "description": "",
       "longitude": 1.4333,
       "email_contact": null,
       "web": null,
       "sys_admin_contact": null
     }
   ],
   "total": 9,
   "version": "e5335be2526e4292668dd67e616022384ca5708c",
   "links": [
     {
       "href": "/sid/grid5000/sites/versions/e5335be2526e4292668dd67e616022384ca5708c",
       "title": "version",
       "rel": "member",
       "type": "application/vnd.fr.grid5000.api.Version+json;level=1"
     },
     {
       "href": "/sid/grid5000/sites/versions",
       "title": "versions",
       "rel": "collection",
       "type": "application/vnd.fr.grid5000.api.Collection+json;level=1"
     },
     {
       "href": "/sid/grid5000/sites",
       "rel": "self",
       "type": "application/vnd.fr.grid5000.api.Collection+json;level=1"
     }
   ],
   "offset": 0
 }
  => nil

Let's parse the content and extract the uids of the items:

 > sites_collection = JSON.parse(sites.body)
  => ...
 > sites_collection['items'].map{|site| site['uid']}
  => ["bordeaux", "grenoble", "lille", "lyon", "nancy", "orsay", "rennes", "sophia", "toulouse"]
 > sites_collection['total']
  => 9

OK, so there are 9 sites in total. Here, you could take a few minutes to explore by yourself the other resources advertised by the links property.

Note.png Note

That's one of the advantages of REST APIs over RPC (e.g. WS-*) APIs: as soon as you know the URI of a resource, you already know how to interact with it using the standard HTTP methods (GET, POST, PUT, DELETE) and how to interpret the result by looking at the (standardized) HTTP status codes and the HTTP headers.

Get the status of nodes

Submit a job

Submit a deployment on a set of nodes

Next level - Grid experiments

Conclusion / Resources