Lessons learned from cloud computing roundtables in Australia #ConnectAU #CeBITAus

Over the past month I spent much of my time at two Australian conferences: one in Melbourne, called ConNect 2015 and another one in Sydney with a more recognizable name, CeBIT Australia. In both cases, I was there to present and talk to customers about IBM Cloud. However some of the most interesting, cloud-themed conversations were from a series of roundtables that I was asked to host for IBM.

This post is about the highlights of these roundtable discussions along with my observations. The audience at both conferences was more business than tech oriented; many of the participants were in IT leadership with over 50 individuals joining the events in total. There was also a broad spectrum of industries represented at the discussions, including banking, shipping, insurance, and tech. Governments were represented at municipal and state levels. Given the audience, the discussions were less about specific technologies and more about the whys and the hows of adopting cloud.

Risk aversion

aus_mel_12The roundtable participants who came from outside of the tech industry were clearly averse to taking risks with cloud, obviously around data privacy and regulatory compliance but also around the decisions on picking the right cloud platform and the right provider. The idea of workload portability resonated with these participants as much as the idea of freedom to switch to a different vendor or a provider of the cloud services. For IT leadership this was about flexibility and adaptability to changing circumstances.

Given this risk averse mindset it is not surprising that the principles of an open and hybrid cloud resonated with the roundtable participants. Open source and open standards help solve the vendor lock-in problem and also help ensure workload portability. Hybrid cloud is an inevitable consequence of mature IT policies that require some of the company’s applications and data (its secret sauce) to reside on company’s premises and behind its firewall.

Naturally, Docker came up as a solution to application portability. Given the amount of hype around Docker (much of it justified) it is not surprising that some IT leaders have overinflated expections about the promise of the techology. However, today, the Docker ecosystem does not provide a full featured and open platform as a service. Unlike Cloud Foundry, Docker does not define an approach for binding and sharing service integration metadata (unlike Cloud Foundry’s VCAP_SERVICES) and the Docker community is yet to settle on a common container scheduling and orchestration engine. Also production grade workloads shouldn’t blindly trust the default security settings of publicly available Dockerfiles. My message to the roundtable on Docker was to continue monitoring and experimenting with the technology but to hold off on using it as part of production PaaS deployments.

Commitment to Cloud

Sydney_opera_house_2010I’ll be the first to admit that cloud computing isn’t for everyone. With that said it was interesting to observe that all of over 50 participants at the roundtables said that once they have moved a workload into cloud they have never reverted back to the pre-cloud architecture (e.g. client-server) for the workload. In my mind this is a testament to the value of IT delivered as a service and coupled with consumptive metering and/or billing.

At the sametime, given that most IT organizations are risk averse when moving to cloud (see above), they are also deliberate in picking the appropriate workloads and investing the effort to migrate from existing infrastructure to the cloud. At the roundtables, it was clear that many organizations took the “cloud enabled” route to migration. In many cases they completed virtualization of internal infrastructure first and then selectively started to migrate some of the virtual machines to cloud.

Hybrid clouds are especially relevant to companies taking this route of cloud adoption. Fewer than 10 participants of the roundtables said that they standardized on a single type of a hypervisor like VMWare and its associated virtual infrastructure. Most have heterogeneous environments with a variety of hypervisors. The general sentiment at our discussion was that the outsourcing of a workload to an external vendor was a complex exercise in itself and change of the hypervisors during this migration should be avoided.

Cloud and “Core Technology” Companies

Cirrus_clouds2In everything I covered so far, the ideas of hybrid and open clouds made sense to participants. However, some of the most intriguing questions were from engineers with what I would describe as “core technology” companies, those involved in using computers (often in an embedded form factor) to build a “core technology” that drives their business. For example, one of the companies sold a media appliance that played movies in hotels from an encrypted hard disk. The appliances were intentionally running on isolated networks without any internet access to minimize security risks. Bandwidth costs made it cheaper to distribute media on hard drives rather than through downloads. Another company sold rugged computers preloaded with navigation and seismology related software designed to operate in remote conditions where only expensive satellite phones can connect to the internet. Although we didn’t have any such companies at the roundtables but I would place semiconductor and other high tech equipment manufacturers into this category. Most modern equipment, such as the equipment for etching chips on wafers, uses a built-in server that enables graphical user interfaces, application programming interfaces, and equipment management.

As everyone else in IT, these companies were struggling to figure out what cloud computing means to them. For them the challenge was admitting that their “core technology” workloads did not need a full featured cloud computing environment. Some of these companies could have benefited from virtualized infrastructure or from cloud based dev/test environments but they also had to recognize that factors like absense of network connectivity or complex real time requirements, or impact of network costs on their business plans meant that cloud did not play a role in the deployment of their core technology.

Check out how IBM Bluemix, Docker, and Watson Tradeoff Analytics can work together by test driving this demo.

District Insurance Portfolio Snapshot Earlier in 2015, I had a chance to work with a great team to put together a demonstration of an early version of a Docker-based IBM Container Service and some DevOps capabilities that helped integrate deployment to Docker containers with IBM Bluemix. Fast forward a few months and now we have the Container Service available in Beta to virtually all users of Bluemix.  And I also got the application we used to demonstrate these features available from my github repository: https://github.com/osipov/district-insurance-sample If you want to get started immediately, head on over to github and try the nifty “Deploy to Bluemix” button. If you want to learn more about the demo first, keep on reading!

Interconnect 2015

A recap of the demo scenario is available at the github URL above but to get a sense of how the demo was used at Interconnect 2015, check out this video from one of the general sessions (demo starts around the 1h31m40s marker):

At Interconnect, we saw how District Insurance, a fictional company could use Bluemix as a platform to rapidly develop new features for applications and maintain a continuous delivery pipeline which pushed the application changes to Docker containers. In the video, IBM Bluemix DevOps Services were used to build and run the pipeline connecting District Insurance’s Git-based source code repository with IBM Bluemix Container Services.

So to recap, the demo consists of

  • a Node.JS application using Express, Dust, and a number of other frameworks
  • IBM Watson Tradeoff Analytics for simplifying the decision making process in presence of many complex, competing alternatives
  • IBM Container Service, a Docker container hosting platform available as part of IBM Bluemix
  • IBM Bluemix DevOps Services, a collection of tools to help Bluemix users to easily adopt continuous delivery, agile development, and other software engineering best practices.

If you want to checkout a live version of the demo, it is available via the link below. Use bob/secret to login.

Broaden your cloud cover: from infrastructure to platform as a service.

Cloud application development space is changing rapidly. Developers and CIOs are no longer content with taking on the burden of managing virtual infrastructure in the cloud due to sizable costs associated with operating system maintenance, middleware patching, and other systems administration activities required when operating a large number of virtual machines in IaaS clouds.

As developers are migrating to PaaS to alleviate the concerns described above, they need a trusted partner to provide an enterprise class platform that ensures application portability and avoids vendor lock-in. A significant part of the value of the platform to the developers is in the breadth of the catalogue of services available from the vendor and in the differentiated capabilities that can be built using the services. For example, a PaaS catalogue should provide services for both the current generation of applications (e.g. web apps, data analytics, integration) and the emerging categories of applications (e.g. big data, Internet of Things, cognitive).

Regardless of whether a customer is looking for a PaaS platform as part of a private or hybrid cloud, it is imperative to evaluate the public cloud presence of the platform from the vendor. Some vendors make bold claims about security, scalability, and availability of their PaaS offerings but to back up those claims the vendor should demonstrate how well the same product code can hold up in both large-scale public and private cloud deployments. To deliver on the claims, a vendor must ensure tight vertical integration of the entire PaaS stack, covering everything from the hardware to the application management and monitoring APIs.

Today, developers have mostly converged on two complementary deployment approaches for PaaS. Both of the approaches ensure application portability (i.e. to avoid cloud vendor lock-in) by relying on open source software and APIs. The first, which is less flexible but provides for greater ease of use, is to deploy application binaries and associated artifacts directly to a PaaS using open source software based tools like Cloud Foundry. The second, which is more flexible but requires more upfront work, is to use Linux container based technologies like Docker to specify a deployable application bundle. A true PaaS should support at least these two approaches to allow developers with different levels of skill to write applications for the cloud.

Continuous integration and deployment is now the default model used by developers to ensure both rapid innovation and high availability in cloud applications. Top consumer facing companies like Netflix deploy application code hundreds of time per day and Amazon.com claims to deploy every 11.6 seconds on average. This level of productivity requires sophisticated DevOps capabilities that provide a unified, integrated view that encompasses software development project management, agile development sprints, testing, bug tracking, and automated code deploys.

In most enterprises, PaaS co-exists with IaaS and legacy infrastructures, which means that the PaaS must provide secure and reliable means of integration across these environments. In cases where an application relies on a public cloud PaaS, this type of integration is also an important business capability, enabling the enterprise to securely expose applications running behind the firewall to the Internet. For example, an application hosted on a public PaaS may connect via a secure integration channel to an employee database residing behind the firewall of a company to provide mobile application users on the web with a secure view on just the publicly available information about the employees, excluding their personally identifiable information.

If you are one of the 3.3 million Americans that purchased a FuelBand, you probably know that Nike discontinued this product. But chances are you don’t know that Nike continues to collect, manage, and analyze data collected from a variety of wearable devices to provide their customers with innovative, big data analytics based products using this data. Nike is just one of many companies that are starting to offer data and analytics based services via APIs. All these companies are joining what the industry calls the API economy and many are choosing to rely on PaaS to run the applications behind the APIs. There are numerous challenges related to the rollout of compelling APIs to customers. A trusted PaaS provider should be in position to help a company to deploy, run, and market the APIs to an appropriate market segment.

Budgetary pressures in IT organizations are driving many developers to migrate monolithic applications to more fine-grained, microservices based architectures. This type of an architectural change provides businesses with greater transparency into the inventory of services delivered by their IT departments and enables better traceability between business goals and value delivered by IT in fulfillment of the goals. For example, instead of allocating a large chunk of an IT budget to a complex customer ordering and tracking application, a CIO may choose to re-architect the application as a collection of microservices that can include order validation, pricing, shipping address verification, and more. A PaaS vendor should be able to support microservices based applications and to provide IT departments with a comprehensive service catalogue. Ultimately, an IT department should be able to rely on their PaaS provider and its catalogue to comparison shop as part of a decision on whether to support a business process by maintaining an in-house microservice or by having the PaaS broker the service from a vendor.

The capabilities described above should be considered as part of a broader cloud computing strategy within the enterprise. PaaS implementations available today can help enterprises to improve operational efficiency while avoiding vendor lock-in.

Secure your IBM BlueMix web app with OAuth 2.0 using the IBM ID Single Sign-on Service

Authentication and authorization often form the starting set of security capabilities one adds to a web application. Due to the widespread need for these capabilities across web applications of various types, numerous standards are available, including OpenID, OAuth, and others. For example, in case of authorization, OAuth 2.0 has been broadly adopted by vendors including IBM, Google, LinkedIn and others. This post describes how a Node.js application deployed to IBM BlueMix can be augmented to use IBM ID, which is an OAuth 2.0 compliant, policy-based authentication service that provides an easy to use single sign-on (SSO) capability for web and mobile applications.

IBM ID provides access to IBM applications, services, communities, support, on-line purchasing, and more. IBM ID credentials are easy to create and your profile information is centralized so you can maintain it in a convenient and secure location.[1]

This post extends an earlier entry from this blog on how to deploy a minimal, hello world style application to Node.js on IBM BlueMix. The authentication and authorization approach presented here can be easily layered onto any existing Node.js app.

Getting Started

To try the code described in this post, you have to have your own IBM ID account. It is easy to register for one using the following form: https://www.ibm.com/account/profile/us?page=reg

Throughout this post ibm-sso-nodejs-sample will be used as the BlueMix application name. The hostname for the application is going to be generated by replacing a substring ‘sample’ from the name with a random value (e.g. ibm-sso-nodejs-42) to minimize the chances of name collisions across multiple readers of this post.

Start by cloning the ibm-sso-nodejs-sample from the git repository available here: https://github.com/osipov/ibm-sso-nodejs-sample

The next section is going to describe the implementation of the code, so if you’d like you can skip forward to the Prepare and push code to BlueMix section to try out the code.

Implementation Overview

package.json file (shown below) declares key dependencies used in this sample.

Node.js Express framework[2] simplifies the setup of the routes and management of the application session state. Passport[3] is another Node.js framework for integration of various authentication providers (including OAuth2 based ones) into an Express based application. The library dependency for the IBM ID OAuth2 provider[4] is declared on line 11. If you are planning on debugging the application on your local machine (i.e. outside of BlueMix), then you should execute the npm install command from the ibm-sso-nodejs-sample directory to download and install the dependencies from package.json into your environment. Otherwise, the dependencies will be downloaded automatically inside the BlueMix runtime container during the execution of the cf push command. The application code for this sample resides in the main.js file (snippet below). The initial section of main.js[5] contains commonly used setup code to import the two frameworks mentioned earlier: Express and Passport. Note that this implementation uses the Express memory based session store[6]. While this is a reasonable approach for sample code, if you decide to take this code into production, you should migrate to using an alternative session cache. Examples of alternatives include the IBM Session Cache[7] and Redis[8], both available from the BlueMix service catalog.

The sample provides templates for callback functions that are invoked by the Passport framework when serializing/deserializing a user[9]. Although this sample does not specify any code in these functions, a production class application would insert code here to manage application specific extensions to a user object, e.g. an application specific user ID.

The application is initialized using BlueMix specific code to extract environment variables from the BlueMix runtime container and to set the variables related to the hostname, port number, URL as well as the IBM ID OAuth 2.0 client ID and client secret[10]. Note that the variables for the ID and secret are set to blank in the source code repository but will be modified according to the instructions later in this post.

To use the IBM ID OAuth2 provider, one must include the code from lines 34-52 into a Node.js application[11]. clientID, clientSecret and callbackURL are the minimal required parameters to create a new instance of the IbmIdStrategy OAuth2 strategy. The callback function[12] processes the access and the refresh tokens retrieved in case of a successful OAuth 2.0 authorization. The profile variable carries the IBM ID profile information supplied by an IBM ID user. The code used to save the profile information in the session[13] is arbitrary, an alternative implementation can choose to disregard the profile information entirely or extract just the necessary attributes of the profile to populate the session.

OAuth2.0 specification recommends the use of a state variable to help prevent cross-site request forgery[14]. Since the code in this post is not intended for production, the ‘/ibmid/auth’ route does not pass a state attribute along with the scope to the passport.authenticate function. When adopting this code for production, you should implement a random string generator to set the state attribute and then verify that the random value is correctly returned back as a req.query.state variable in the handler function of the ‘ibmid/auth/callback’ route[15].

The rest of this section describes the routes added just to demonstrate the OAuth 2.0 based authentication and authorization process, there is a “public” route that can be accessed without any authentication whatsoever[16], a “private” route that requires a client to authorize with IBM ID[17], a “logout” route that logs a client out of an IBM ID authentication[18], and an “error” route that is used to display a message to a client in case of any OAuth 2.0 errors[19].

Prepare and push code to BlueMix

Before proceeding with the rest of the instructions in this post, push the code you cloned from the github repository to BlueMix using the following command

cf push ibm-sso-nodejs-sample -n ibm-sso-nodejs-$RANDOM -c 'node main.js'

The result of the push command should provide a session similar to the one shown in the snippet. Note the value of the URL, e.g. from the session which should look like line 50 from the snippet. It will be referred to as <URI> in the instructions below.

Any IBM ID OAuth 2.0 client must be registered with IBM. To do so, Open https://idaas.ng.bluemix.net/idaas/developer/manageclients.jsp using your browser, login using your IBM ID and click on “Create a new client”. Leave client identifier and client secret as then type in ibm-sso-node-js-sample for the Display Name and specify <URI> for the Redirect URI. Check off the Enabled box, click Submit and take note of the client ID and secret values that were randomly generated. The rest of this post will refer to their values as <ID> and <SECRET> .

To propagate the OAuth 2.0 credentials that you just obtained from IBM, execute the following commands to create Cloud Foundry user defined variables:

cf set-env ibm-sso-nodejs-sample SSO_CLIENT_ID <ID>
cf set-env ibm-sso-nodejs-sample SSO_CLIENT_SECRET <SECRET>

Do one last push to persist the user defined variables.

cf push ibm-sso-nodejs-sample -c 'node main.js'

Test out the app

Start by navigating to the URL assigned to you by BlueMix. You should see a ‘Hello World’ message along with a login link.

Clicking on the link should force you to login using the IBM ID you created earlier.

Once you are logged in, you should see a Hello message with the first and last name you used when registering for your IBM ID.


[1] https://www.ibm.com/account/profile/us?page=regfaqhelp
[2] http://expressjs.com/
[3] http://passportjs.org/
[4] https://www.npmjs.org/package/passport-ibmid-oauth2
[5] https://github.com/osipov/ibm-sso-nodejs-sample/blob/master/main.js#L2-L12
[6] https://github.com/osipov/ibm-sso-nodejs-sample/blob/master/main.js#L13-L14
[7] https://www.ng.bluemix.net/docs/Services/ECaaS/session.html
[8] https://www.ng.bluemix.net/docs/Services/Redis/Index.html
[9] https://github.com/osipov/ibm-sso-nodejs-sample/blob/master/main.js#L19-L25
[10] https://github.com/osipov/ibm-sso-nodejs-sample/blob/master/main.js#L27-L32
[11] https://github.com/osipov/ibm-sso-nodejs-sample/blob/master/main.js#L34-L52
[12] https://github.com/osipov/ibm-sso-nodejs-sample/blob/master/main.js#L40-L43
[13] https://github.com/osipov/ibm-sso-nodejs-sample/blob/master/main.js#L41-L42
[14] http://tools.ietf.org/html/rfc6749#section-4.1.1
[15] https://github.com/osipov/ibm-sso-nodejs-sample/blob/master/main.js#L51
[16] https://github.com/osipov/ibm-sso-nodejs-sample/blob/master/main.js#L63-L65
[17] https://github.com/osipov/ibm-sso-nodejs-sample/blob/master/main.js#L67-L70
[18] https://github.com/osipov/ibm-sso-nodejs-sample/blob/master/main.js#L76-L78
[19] https://github.com/osipov/ibm-sso-nodejs-sample/blob/master/main.js#L72-L74

Top 10 Reasons Why You Should Choose IBM BlueMix for your Cloud Platform as a Service.

This post is going to add some variety to all the technical content you’ve been seeing on this blog. So without further ado, here are the top 10 reasons why you should choose IBM BlueMix for your cloud Platform as a Service…

1. Partner with IBM to reach into enterprise for go to market opportunities. If you are trying to grow your Software as a Service (API economy) business beyond initial phases, you know that landing an enterprise customer can really make an impact to your bottom line[1]. IBM operates in more than 170 countries around the world and has direct and inside sales forces that reach out to top companies in the world. Find a way to work with IBM to jointly pursue opportunities in cloud and drive enterprise customers to your applications and services running on IBM BlueMix. For example, OnFarm[2] recently started working with IBM to receive ongoing coaching, access to technology, mentorship and connections to venture capitalists, clients and partners.

2. Leverage a pipeline of talent grown by IBM through university & development outreach. Even if you have a phenomenal business plan, you will struggle to execute without the right talent in your company. Did you know that IBM University Outreach is working with US universities[3] to get students engaged with BlueMix? Combine that with developer focused conferences like Dev@Pulse[4] along with many other outreach efforts and you have a growing talent pool of development candidates for your company. If you are a student or even just a developer interested in growing your skillset, you will want to know that BlueMix is built on Cloud Foundry[5], an open source platform as a service technology. This means that if you pick up IBM BlueMix skills you can apply them to any other Cloud Foundry deployment, including the one that you can build yourself[6].

3. Monetize your services through the IBM BlueMix services palette. Even though IBM BlueMix is in beta and does not yet feature paid plans, you can get a head start on your competitors by contributing your service to the IBM BlueMix service palette. Networked marketplaces like IBM BlueMix have a strong first mover advantage[7]. Develop on BlueMix, create a service plan[8] and make your service a category leader!

4. Build on a responsive, global cloud infrastructure with world’s largest webhosting marketshare. Did you know that Softlayer, IBM’s Cloud Infrastructure as a Service hosts more than twice the number of websites than Amazon Web Services[9]? You shouldn’t be surprised if you’ve been reading about the reliability of the two infrastructures. IBM uses Softlayer not just for BlueMix, but also for customers like the Australian Open[10] tennis tournament to ensure high availability and a great user experience. Amazon Web Services has been suffering from numerous outages over the past year[11][12], including one as recently as Christmas time[13].

5. Meet enterprise class quality of service requirements in the cloud. Not all applications are created equal. Some need high availability & high performance relational databases. Others must satisfy complex regulatory compliance requirements. It is easy to become disparaged when you have to say no to a government or an enterprise customer because your service is missing a capability which you cannot afford to build yourself. BlueMix gives you a roadmap for how to evolve your application to introduce enterprise class capabilities[14] that can help you pursue lucrative opportunities in large enterprise and government space.

6. Support migration of complex legacy applications to cloud with IBM’s flexible infrastructure . IBM Softlayer, the underlying infrastructure for BlueMix has been highlighted by Gartner[15] for its range of features: from bare-metal to cloud servers, all managed through a common API. This broad supply of capabilities is why gaming companies[16] like Multiplay and KUULUU with complex demands for their resource intensive games moved to the same infrastructure as BlueMix.

7. Deliver innovative services by working with IBM Research & Development(R&D) labs. IBM BlueMix is the only Platform as a Service out there featuring IBM Watson technology — a recent Jeopardy winner[17]. In addition to Watson, you can take advantage of other services created by IBM R&D, including a log analysis, auto-scaling, and many others. IBM labs also have customer advocates who can explore how IBM can help you on your opportunities.

8. Discover opportunities for innovation from a palette of IBM business partner services. IBM business partners have adopted BlueMix and made their services available for your use. For example, Zend Technologies provides support for their enterprise class PHP container[18] which can be deployed to IBM BlueMix. Access to IBM business partner ecosystem[19] can help you build your application faster, regardless of the technology, industry, or geography you are targeting.

9. Create trusted and secure offerings by reusing services reviewed and certified by IBM. A company can develop a service for BlueMix without any restrictions from IBM. However, for a service to be posted to the BlueMix catalog, IBM has to review and certify the service based on IBM’s internal guidelines. If you are interested in this process, leave a comment on this blog or reach out to your IBM representative.

10. Depend on IBM’s history of industry leadership for what is essential to your customers. IBM has been around for more than a century supporting technology needs of companies around the world. Some have said that the Western Civilization runs on the mainframe technology[20] which IBM has developed and has been maintaining since 1960s. With IBM commitment to Cloud Foundry[21] you can be certain that BlueMix is the right foundation to develop applications for your most valuable customers.


[1] http://www.quora.com/Whats-the-best-way-for-a-B2B-startup-to-acquire-enterprise-customers
[2] http://www.onfarm.com/2013/10/21/onfarm-wins-ibm-smartcamp-north-america/
[3] http://www.ece.ncsu.edu/news/24872
[4] http://www-01.ibm.com/software/tivoli/pulse/devatpulse/
[5] http://en.wikipedia.org/wiki/Cloud_Foundry
[6] http://docs.cloudfoundry.org/deploying/run-local.html
[7] http://en.wikipedia.org/wiki/First-mover_advantage
[8] http://docs.cloudfoundry.org/services/catalog-metadata.html
[9] http://hostcabi.net/top_500_hosters
[10] http://www-03.ibm.com/press/us/en/pressrelease/42981.wss
[11] http://www.usatoday.com/story/tech/2013/09/13/amazon-cloud-outage/2810257/
[12] http://blogs.wsj.com/digits/2013/08/25/amazon-web-services-outage-cuts-off-big-names/
[13] http://aws.amazon.com/message/680587/
[14] http://www.ibm.com/cloud-computing/us/en/products/managed-cloud.html#cont-highlight
[15] http://www.gartner.com/technology/reprints.do?id=1-1IMDMZ5&ct=130819&st=sb#h-d2e950
[16] http://www-03.ibm.com/press/us/en/pressrelease/42928.wss
[17] http://www-03.ibm.com/press/us/en/presskit/27297.wss
[18] http://www.zend.com/en/products/server/
[19] https://www-304.ibm.com/partnerworld/wps/bplocator/landing
[20] https://www.youtube.com/watch?v=_0uYz75BHBg
[21] http://www-03.ibm.com/press/us/en/pressrelease/41569.wss

Deploy a Hello World PHP Application to Zend Server on IBM BlueMix

The previous post demonstrated how to deploy a trivial Node.js application to IBM BlueMix. There I also introduced the basics of manifest.yml file used by BlueMix and Cloud Foundry as a deployment descriptor. This post goes deeper into features available for creating reusable manifest.yml files and covers Cloud Foundry buildpacks used during deployment. By the end of the post you will have deployed a trivial Hello World application written in PHP to a Zend Server’s PHP container running on IBM BlueMix.

Variables in manifest.yml

The post about deploying a Node.js application to BlueMix warned about a potential problem with having a single manifest.yml in use by multiple developers. The problem occurs in a situation where the deployment subtarget (hostname) of an application has a fixed value in the manifest. If two developers try to deploy their application to the same hostname, BlueMix rejects second and subsequent deployment attempts because of the conflict as to which of the two applications should run on the hostname, e.g. hello-world.ng.bluemix.net

The post suggested manually changing a postfix in the manifest.yml file in case if BlueMix rejects deployment attempts due to the same hostname error. Of course, this approach is not acceptable in most cases, especially if one tries to automate the deployment process. One way to address this problem is to use features of bash[6] and sed[7] to introduce randomly generated numbers characters into the manifest.yml file. For example, you can create a manifest.yml.template file where the host parameter is specified as follows:

host: hello-world-php-$RANDOM

Then, prior to performing a push using the Cloud Foundry cf tool, you can execute the following sed command to generate a manifest.yml file where the string $RANDOM is replaced with a randomly generated number.

sed 's/$RANDOM/'"$RANDOM"'/g' manifest.yml.template > manifest.yml

Use of random numbers in the application hostname makes the possibility of name collisions unlikely but introduces another problem: finding out the URL of an application after the deployment process completes. The cf tool has a command to help called cf routes that returns up to date information about active hostname URLs and corresponding applications.

Zend Server Buildpack

During the deployment process described in the previous post, the package.json[1] file provided a hint for BlueMix to use the Node.js runtime. Without this hint the application would have failed to stage because BlueMix would not know which runtime to prepare for the application. Buildpacks are used by Cloud Foundry to prepare the environment for running an application (e.g. setting environment variables, copying settings files into the right places), compile binaries (if any) and launch the application in the runtime. A detailed explaination of buildpacks is outside the scope of this post and more detail is available from Cloud Foundry[2] and Heroku documentation[3]. It is important to note that some buildpacks, such as the one for Node.js, are built into BlueMix which means that the buildpack does not have to be specified explicitly to the cf push command. Given enough time and effort one can create a custom buildpack for a runtime that is not built into BlueMix. Once the buildpack is available on the web via git, BlueMix can use git clone of the buildpack repository to copy the buildpack to the execution environment during the deployment process and launch an application in the custom runtime. The next section describes how to use a custom buildpack for a Zend Server, a PHP runtime to run a simple Hello World application on BlueMix

PHP Hello World Application

As with the post covering Node.js deployment, you may either use git to clone a repo for this application[4] or if you prefer, create a new directory and save the index.php and manifest.yml files shown below to the directory. The Hello World application in index.php is very simple. The <?php prefix on line 1 instructs the PHP interpreter to process the code between the prefix and the closing ?> bracket on line 3. The actual code on line 2 just prints “Hello World” to any HTTP request sent to the PHP container running the application.

The parameters in the manifest.yml are covered earlier in this post and in the previous post while a more detailed explanation is available from the Cloud Foundry documentation[5].

The following is a snippet of the application deployment session to BlueMix, starting with cf push hello-world-php command.

Note that due to variability of download speeds it may take a while to deploy the application. If you are seeing problems with timeout when deploying the application, try setting the CF_STARTUP_TIMEOUT and CF_STAGING_TIMEOUT to be at least 15 minutes.


[1] https://github.com/cosipov/minimal-hello-world-node-js/blob/master/package.json
[2] http://docs.cloudfoundry.com/docs/using/deploying-apps/buildpacks.html
[3] https://devcenter.heroku.com/articles/buildpacks
[4] https://github.com/cosipov/minimal-hello-world-php
[5] http://docs.cloudfoundry.com/docs/using/deploying-apps/manifest.html#attributes
[6] http://en.wikipedia.org/wiki/Bash_(Unix_shell)
[7] http://en.wikipedia.org/wiki/Sed

Deploy a minimal Node.js application to IBM BlueMix

IBM BlueMix documentation[1] provides broad coverage of the platform’s many features. Instead of trying to address all these features at the same time, this post will take a layered approach starting with an absolute minimum number of features need to run a Hello World application written for Node.js[2] on Bluemix. Minimum here means that taking away any of the parameters from the code will definitely break the deployment to BlueMix. The concluding section of this post will describe how to use Cloud Foundry[3] manifest.yml file feature to reduce the amount of manual effort needed to deploy an application to Bluemix.

Verify your IBM BlueMix account

Make sure that you have a working IBM ID before you start trying out the application and the instructions in this post. You can learn how to create an IBM ID here.  To verify that your IBM ID is working, visit IBM BlueMix https://ace.ng.bluemix.net/ and sign in with the IBM ID by using the Login button on the upper right.

Prepare the application source code

If you don’t have git[4] installed or don’t want to use git, you can manually create a separate directory and then copy and paste the source code for main.js and package.json files described here. Otherwise, the git repository for this post is available via https://github.com/osipov/minimal-hello-world-node-js

Here is the JavaScript source code for main.js file. It is explained in detail below the snippet.

main.js is the entry point into the application and has the instructions for the Node.js runtime to start a web server. Anyone familiar with Node.js will immediately recognize that this code snippet returns a “Hello World” message to any HTTP request sent to the web server running the code. There are two variables in the snippet that might be new to you if you haven’t worked with BlueMix or Cloud Foundry in the past. The code on line 1: var port = (process.env.VCAP_APP_PORT || 8192); relies on process.env.VCAP_APP_PORT variable which is automatically initialized by the BlueMix environment to 80, a default port for HTTP traffic. Use of the || operator (logical or) on the same line ensures that if we run the main.js code on a local Node.js runtime (i.e. outside of the Bluemix environment), the port number will default to 8192 since VCAP_APP_PORT would be null. Line 2 uses similar logic to set the hostname of the application. Finally, an empty JSON object in the package.json file is needed for the Node.js runtime used on BlueMix. Regardless of whether you have cloned the git repository for the application or copied and pasted the source code from the snippets above, at this point you should have a directory containing main.js and package.json files.

Deploy and run the application on BlueMix

Both BlueMix and Cloud Foundry rely on a Go based command line tool (called cf) to script common application management tasks. The getting started[5] section of the BlueMix documentation provides references on how to install cf the Cloud Foundry command line tool[7].  Once cf is installed, you can use it to connect to IBM Bluemix[8] as the deployment platform and then login to BlueMix[9] using your IBM ID (in line 1)

If the target and login commands completed successful you should be ready to deploy and run the application on BlueMix. Ensure that your working directory contains the application files main.js and package.json before running the cf push operation shown in the following snippet. Note that the -n argument specifies that hello-world-node-js-0123456789 should be used as the subdomain(hostname) for the application, -m tells BlueMix to allocate 128 megabytes of memory to the application, while the -c argument tells the Node.js runtime in BlueMix to run the code in the main.js file.

Once started, the cf tool will show you information about the progress of the deployment. If the deployment processes finishes correctly, you should see the “App started” message and your browser should show the Hello World message when visiting https://hello-world-node-js-0123456789.ng.bluemix.net An example of a push session is shown below. Note that parts with your <Your IBM ID> will be different for your session.

Since multiple BlueMix users may try to deploy an application to the same subdomain you may see the ‘host is taken’ error message (example below) during the deployment process. To work around this problem, simply change the subdomain to a different, unique value.

Simplify deployment with manifest.yml

A better alternative to having the deployment parameters in the command line is to create a Cloud Foundry manifest.yml file. As shown in following snippet the manifest file contains the values that can be specified during the push process. By convention the file starts with three dashes followed by YAML[11] based markup. A detailed description of the manifest file specification is available from the Cloud Foundry website[3]. The next time you will execute cf push, the cf tool will automatically use the contents of the manifest to deploy the application to Bluemix.



[1] https://www.ng.bluemix.net/docs/index.jsp
[2] http://www.nodejs.org
[3] http://docs.cloudfoundry.com
[4] https://help.github.com/articles/set-up-git
[5] https://www.ng.bluemix.net/docs/QuickStart.jsp
[6] http://docs.cloudfoundry.com/docs/common/install_ruby.html
[7] http://www.ng.bluemix.net/docs/BuildingWeb.jsp#install-cf
[8] http://www.ng.bluemix.net/docs/BuildingWeb.jsp#connect-bluemix
[9] http://www.ng.bluemix.net/docs/BuildingWeb.jsp#login-bluemix
[10] http://docs.cloudfoundry.com/docs/using/deploying-apps/manifest.html#attributes
[11] http://www.yaml.org

Register for an IBM ID Account

IBM ID provides access to IBM applications, services, communities, support, on-line purchasing, and more. IBM ID credentials are easy to create and your profile information is centralized so you can maintain it in a convenient and secure location.

To obtain an IBM ID:

  1. Visit https://www.ibm.com/account/profile/us?page=reg
  2. Complete the 2 step registration form
  3. In case you have questions while filling out the form, check out the IBM ID registration FAQ here: https://www.ibm.com/account/profile/us?page=regfaqhelp

To verify that your IBM ID is working, visit IBM BlueMix https://ace.ng.bluemix.net/ and sign in with the IBM ID by using the Login button on the upper right.

If you need more help, you can check out this video from IBM Support https://www.youtube.com/watch?v=afQmxE4wjIg

There is also a Worldwide IBM Registration Helpdesk. You can learn more about it here: https://www.ibm.com/account/profile/us?page=helpdesk