Jon Greaves’ Blog

Exciting times - launch of 2nd Generation InstantOn™

Over the past few weeks, I’ve described the features and functions “enterprise” clouds have been required to provide.  This isn’t purely what we have dreamed up, but rather what our enterprise and federal customers have asked for over the past 8 months since the launch of Carpathia InstantOn - the first generation of our cloud platform.  Because we are great believers in agile service development, we’ve been constantly enhancing our platform with new capabilities in plain sight of our customers via Carpathia Labs.  And as a result, we’re pleased to present 2nd generation InstantOn.

Pulling back the covers, 2nd Gen InstantOn has some pretty big changes in play.  Probably the biggest is the work we have accomplished with Citrix on our hypervisor.  The first version of our platform was based on OpenSource Xen – a stellar performer that allowed us to cut our teeth and learn a lot about operating a cloud, providing services to customers, and building a backlog of requirements.

The 2nd Generation of our platform moves us to Citrix XenServer 5.5 as part of the Citrix C3 initiative. In doing so, we are able to unlock many soft benefits such as the excellent support from Citrix for the core virtualization technology, but more so for the ability to tap into many of the enterprise features XenServer provides.

A good example of this is the ability to take a live vm backup.  Sounds pretty straightforward right? Well it is if you deploy some form of centralized storage, or your cloud really is a traditional virtualization platform.  With XenServer, we have been able to implement this action using a distributed storage solution – a local disk with synchronous mirrors to other compute nodes. And to top it off, it’s orchestrated.  Another great addition to our platform is our ability to take virtual machines created for vmware and migrate them to our cloud.  There are several more things I could discuss here - the list goes on and on! Bottom line, we couldn’t be more excited about this partnership.

We have also extended Carpathia Cloud Orchestration™. Rather than having me describe the features, I encourage you to check out some examples in our behind the scenes TechXchange . Orchestration is pretty difficult to demo via a Web UI - API’s tend to be a little dry to explain! Instead, we are going to show our administration CLI that sits on top of our API - this should give a good flavor for the key concepts of our cloud.

2nd Gen InstantOn also enhances our usage of some of the existing components in our cloud. We have been extremely impressed by the performance and ease of integration of the Parascale Object storage we deployed for the first version.  In this second version, we fully integrate Parascale to our XenServer hypervisor to provide template and ISO storage.  Parascale excels with parallel workloads and those that are WORM in nature.  This is a perfect use case for template management and ISO storage.  Multiple compute nodes talk to multiple storage nodes each with their own version of the object (ISO, template, etc).   Our storage workloads migrate from the network to local disk and progress back to the network during the vm lifecycle.

Speaking of lifecycles … this was a highly requested feature in the first version of our cloud. 2nd Gen InstantOn allows a full vm lifecycle to be managed.  Customers can start with a generic Carpathia VM, instantiate the vm, install apps, add hardening per policy, promote the vm to a template (templates are private to customers) then redeploy.  We have versioning on these vm’s. Think of it as CVS for virtual machines.

Hopefully this has provided you with a good flavour of 2nd Gen InstantOn. We’ll have alot more to share in coming blog posts. In the meantime, if you’re hungry for more now, I encourage you to check out some of the videos or contact our sales team .

In the last blog entry I talked about the business, financial and contractual side of enterprise cloud. Lets change gears and discuss the technology.

Basically there aren’t a lot of differences and the premise is the same - use virtualization to provide a different ROI (not necessarily lower) for the computing needs of the enterprise.  If we take a look at enterprise workloads, we get a little more insight into the characteristics that become more important for an enterprise vs. a 2.0/developer customer.

Many enterprise cloud deployments are “project”-based — that is, the enterprise needs to add a new feature - e.g. CRM - needs it up quickly, wants to avoid capital expense, etc.  The need for extreme elasticity is rarely included in the workloads of an enterprise.  Sure, once a month having some extra horsepower to close the books is great, but it’s not an hour-by-hour swing of multiples of the environment we might see from a social media site.  There is also the need for short-term project based solutions – e.g. need a development environment for a few months to build out a new version of an app - do a trial with a new software solution, etc.

When enterprises do take the plunge, they need more than just a flat pool of virtual machines. They look for a higher-level construct, group, or application.  Grouping allows architecture to be defined, blueprints created, and run books produced.  They also look for “topology” in that virtual appliances - such as loadbalancers - need to connect to firewalls, and web servers via vlans/virtual switches.  They want the same degree of isolation they have with today’s dedicated infrastructure.  The good news is we are seeing a high degree of innovation in this space with the likes of Citrix, Vyatta, Nicira, and Altor really pushing virtualization in the network.

Another big difference is the applications themselves.  A couple of days ago I had a really interesting conversation with a prospective customer.  They have a very cool app that automates document processing using OCR, bar code reading and assisted processing.  When they onboard new customers, there is a large ramp to digitize the historic data.  This company created a very smart queue-based solution decoupling the collection of the raw data from the processing.  So far, so good – the perfect candidate for some form of capacity of demand.  But there is a catch - a big one - and one we see all too often when architecting enterprise solutions…the software license terms.

The OCR package is licensed on a per processor basis.  Traditional perpetual license.  Scaling up is no issue - just purchase more licenses – but scaling down isn’t an option with the vendor. Today, they optimize for the worse case scenario, the cost of the compute in this equation is tiny compared to the cost of the license. Until ISV’s offer license models that fit the infrastructure deployment models, we will have a cadence mismatch.  We spend a lot of time looking for creative options to solve this – e.g. rather than scaling out - scaling virtual machines up by adding dynamically more memory, if its a per server license adding more CPU’s.

One area we are seeing good traction in is the virtualization of disaster recovery .  Providing offsite data storage via the cloud, then applications on demand, pay per use is hitting the mark.  If we go back to the software licensing schemes for a second, this is a place where we have some degree of synchronization.  Many vendors provide a “DR” license for a limited amount of time per year.  Now we can finally show compliance, based on the actual number of hours a DR instance was active.

So, enterprise cloud is a little more than “enterprises using the cloud”.  To support mission critical workloads enterprises look for the right blend of people, process and technology.  The cloud isn’t one size fits all — if it was, the battle for the cloud would be over.  Thank goodness there is plenty of room for innovation.

I was catching up on Twitter posts while enjoying the Australian Open and came across this

eekygeeky RT @cloudbzz : Quick Poll - What makes something an "Enterprise Cloud?" <~ enterprises use it?

This is a topic I’ve spent much time discussing with our customers who certainly fall in the “enterprise” bucket.  So here is my take (sorry Twitterati, 140 characters didn’t cut it!)

Lets start at the beginning and talk about how Enterprises want to pay for cloud.  While individuals/developers/2.0’s are more than happy to swipe a credit card and buy virtual machines, this doesn’t help your average Enterprise CFO.  They require more documentation, control, and accountability for using the company’s financial resources.  It would be very interesting to see a public company go through its SOX review and discuss how its infrastructure maps to this line item on the IT Managers credit card.  Who gets the points?

Who will use the cloud?  Cloud services assume one person makes all the cloud provisioning and purchasing decisions.  That’s not the case in enterprises.  Checks and balances, functional responsibility, approval chains, and change control boards, etc. are the reality.  Having one person with an ecommerce account they purchased a book with, isn’t good enough.  What is required are roles that can have permissions assigned, and for bonus points, delegate permissions.  That’s not to say “pay per drink” models are not what enterprises want, they just want to have them delivered via familiar fiscal terms and contracts.  Programmatically managing a cloud doesn’t remove the burden of responsibility, it changes it.  API-centric approaches need to take this into account.

Speaking of contracts.  A click -through a developer accepts during provisioning or signup, isn’t sufficient for an enterprise’s IT department.  Therefore putting T&C’s in place to protect the enterprise is essential.  As such, you can expect redlines and tweaks to meet requirements for service even if its delivered virtually.

So who’s watching the farm?  Enterprises are used to hosting/outsourcing infrastructure, and when they do, they expect a certain level of service that includes being able to reach someone responsible for the infrastructure 24×7.  Along with being able to reach someone they also want SLA’s on the responsiveness to resolving issues.  Many of the enterprise RFP’s we respond to now want to go one step further and ask for the framework being used to manage the infrastructure.  We elected ITIL and most recently moved to its latest revision.

And then there are certifications.  SAS70 Type 2 is the “must-have” certification for all hosting companies.  It’s basically a definition of a set of controls and an auditor’s review and opinion on those controls.  A control could be as simple as “we lock the datacenter door” which an auditor could review and pass.  Sharing these controls is something enterprises often demand.  Without knowing what the controls are, it’s very hard to judge how effective the policy is.  Very few cloud providers offer this level of transparency.  More so, we are now seeing other standards to make inroads; a good example is ISO27001 which is really starting to get some attention with its security focus.

Finally, cloud on-boarding is something enterprises are coming to expect.  Having great API’s and web UI’s, etc. is fine but when it comes to moving applications and data to the cloud, enterprises are looking for consulting and professional services — helping understand what should move and what should remain on dedicated infrastructure, understanding the ROI, potential savings, building a migration strategy, developing a DR plan – there are all key items an enterprise looks for when embracing the cloud.

Next post… The technical differences…

Over the past couple of months we have seen tremendous interest in using cloud for disaster recovery solutions. Some of this has been fueled by CIO’s looking for a more cost-effective method of DR from traditional cold/warm/hot sites and others from cloud customers who have been impacted by some of the outages of late at Rackspace and Amazon.

If we pull back the hype, cloud is really a virtualized platform running in a geography (i.e., datacenter, availability zone, location, region…).  Most of the more mature cloud providers have multiple cloud pops with some form of interconnect.  For Carpathia Hosting, our cloud pops are northern Virginia and Arizona.  Within a geography, cloud technology affords a great degree of resiliency to component failure, but lose the geography due to something catastrophic - whether it be natural disasters, facility infrastructure failures or sys-admin “oops” – at which point, cloud is much like any kind of hosting solution.  Offline .   Resiliency does != DR.

People who have adopted the cloud gain a lot of technology advantages that help them solve the issue of portability between cloud pops.  In addition, as standards emerge between cloud providers, it will become that much more straightforward to have a copy of the “application” in a remote location.  With today’s cloud platforms this isn’t automatic.  Although great improvements have been made in the past six months, the technology still isn’t there.

So cloud hosting customers should consider DR in exactly the same way a traditionally-hosted infrastructure does — build a plan, test a plan, re-test a plan.  Simply being in the cloud (at least today’s versions) doesn’t provide a DR solution.

So how does Carpathia Hosting position cloud DR?  We have an approach that takes advantage of the per CPU hour/per GB to allow customers to:

• Replicate data from production to a remote cloud    nstance (i.e., data can be hosted on dedicated infrastructure or cloud/virtualized).
• Host golden images of “servers” in our cloud template library.
• Build an “application group” that can instantiate virtual servers, virtual firewalls, virtual load balancers, etc. with one click from the golden images tied to replicated data.  Application groups understand private networking/vlans, security policy, meta data about servers.  Everything to re-create an infrastructure.
• Deliver as a managed service, customer picks up the “bat phone” and calls a DR event.  We put the plan in place to meet your recovery time objectives.
• Tests with the customer every six months (or whatever the policy is), record results, iterate plan, etc..
• Pay for server resources when you use them (twice a year in DR test and in a real DR).
• Deliver bandwidth in a very cost-effective way, not  “bytes transferred”.

This kind of cloud DR solution costs about 25% of a traditional warm DR site.  Since data replication takes place in real time, recover point objectives can be very aggressive (and your data is offsite by default).

If you would like to learn more, drop me a note or send an email to sales@carpathiahost.com .

Today some very exciting news was announced by the Xen Project.  The Xen project team announced the launch of the Xen Cloud Platform.  As I’ve mentioned in previous posts Xen is a corner stone of our InstantOn platform, in fact all of the relevant cloud computing platforms have Xen as a foundation.

Over the past 9 months we have been working on extending Xen as a pure hypervisor into a platform that can deploy cloud services meaning  provisioning across machines, seamless access to storage, deploying in a multitenant model etc etc.  We also firmly believe that enterprises will embrace cloud by extending not replacing managed and dedicated infrastructure hence the development of Cloud Orchestration™ and its adoption by all our cloud customers.

So the announcement from the Xen project comes at a great time.  The plan is to extend the Xen project beyond a pure hypervisor with other technology already in flight to provide security availability and performance needed to deliver cloud solutions directly attacking the enterprise market.

Carpathia is very excited to be part of this movement and looking forward to a new degree of engagement with the Xen team to further the adoption of cloud in enterprise and federal customers.

Congratulations to the Xen team, this helps all of us create the solutions our customers are demanding.

It’s great to see all the attention on hybrid clouds over the past few weeks.  I read with interest the blog post over at zdnet interviewing Rackspace’s Chief Strategy Officer who talked about Rackspace’s roadmap to deliver hybrid solutions in the coming months.  This doesn’t happen often but I’m in complete agreement with his comments that hybrid clouds will be the way enterprises embrace cloud computing for the foreseeable future, at least until the applications that drive enterprises become cloud aware.

I had a similar conversation with analysts from Gartner and IDC following the launch of our hybrid cloud solution at the beginning of June.  One thing the zdnet article didn’t cover that the analysts jumped on right away, is the savings customers can achieve by adopting a hybrid approach vs. pure cloud based on today’s public cloud providers.

A simple analogy we have been using is that of the auto industry.

Lets assume you are traveling to a new city for a day and need a car.  Not a good move to purchase or lease a car since you are not making a long term commitment.  The reverse is true, if you need a car for the long term, renting from Avis is not a cost-effective way to gain the use of a car.

We like to consider;

- Colocation = purchase a car, maintain a car yourself

- Managed services = lease a car, lease holder maintains the car

- Cloud computing = rent a car by the hour/day

The analogy fits very well, especially when you consider the managed scenario’s servers typically depreciate over 36 months which just so happens to be the same as most lease lengths.

So here’s the punchline - as an IT consumer, you need access to all three options depending on the problem you are trying to solve and most often, all three inside the same solution.  Carpathia AlwaysOn/InstantOn™ has this today and in fact, we have been busy sharing customer success stories on http://www.whatisinstanton.com /.  No need to wait for Rackspace to execute on its roadmap, come talk to us http://www.carpathiahosting.com/live-chat .

Over the past few months we have been busy putting the finishing touches on our cloud computing platform.  Rather than taking the usual technology approach and announcing the platform when it was ready, we decided to take a more agile service development approach by engaging with existing customers and prospects and evolving the platform with their input.  So today, we are announcing both the general availability of our cloud platform, which we call Carpathia InstantOn™ and discussing the solutions we have built for real customers. We believe this approach has led to something rather special.

It should be no surprise to folks who have been reading my previous posts, that Carpathia Hosting believes that blending dedicated and cloud technology leads to the best solution for customers both in terms of capabilities and price.  Allowing dedicated infrastructure to request capacity from a cloud, or enabling synchronization between the cloud and dedicated is a non-trivial task. We developed Cloud Orchestration to make this possible.

So what is cloud orchestration?  It’s an interface that sits on top of both our cloud computing and dedicated infrastructure inside the Carpathia Services Platform (CSP™).  Its purpose is to blend resources based on a number of criteria such as SLA’s, predicted capacity spikes, CLI, API, disaster-recovery events, etc.  It’s also a series of Carpathia Hosting-developed virtual machine images that provide the glue to make this possible.  For example, we have a virtual machine that can monitor the performance of dedicated infrastructure.  When certain conditions are met, it automatically provisions more compute resources in the cloud and then removes them when the demand subsides.  We also have virtual machines that provide load balancing, layer3-7 switches, and firewalls with capabilities to automatically reconfigure to support more application virtual machines as they are provisioned.

In addition to virtual machines that respond to dynamic events, we also developed a number of virtual machines that allow dedicated infrastructure to make use of cloud storage.  Both at the object file store level, in our case it’s as simple as mounting a filesystem. The underlying OS on the dedicated hardware has no knowledge it’s talking to a cloud or the cloud is providing multiple copies of its data for both availability and performance reasons.  Our block storage solution is also very interesting, allowing for local storage for day-to-day operations for virtual machines.  We extended this capability by creating a virtual machine that republishes block storage in the form of a virtual NAS appliance to dedicated and cloud resources.

More importantly, the above techniques have been used to build some very unique solutions for real customers.  A couple of quick examples.

We built a transcoding cloud that takes video content in one form and encodes to another (i.e. mpeg to Flash). To simplify its use and integration into dedicated infrastructure, the customers simply drop a mpeg video clip into a directory called “incoming” which is mounted on their collection servers.  The cloud detects this, spawns a new VM which collects the file encodes and writes the resulting Flash-formatted video to a directory called “outgoing”.  The file systems both live inside our object-based storage solution.  The virtual machines that provide the encoding are optimized for cores/memory to provide the most efficient encoding solution.  As more files are dropped, more VM’s are instantiated by the orchestration layer to keep up with demand.
Another solution we engineered provides a load test solution powered by the cloud. We use apache jmeter built into a custom virtual machine.  As one virtual machine reaches capacity for load testing, the orchestration layer creates more.  This capacity management is fed into a single jmeter test console that can now generate thousands of concurrent connections.

We have many other examples of cloud orchestration that we will be talking about in the coming weeks.  If you would like to learn more please join me and some of our customers for a webcast Tuesday, June 16, 2009 at 11:30am EDT.  We’ll take a deep dive into what it takes to build a cloud computing platform and how to leverage it to deliver value to your customers.  Register today at http://whatisinstanton.com/

Over the holidays I sat down with Ron Gula , CEO of Tenable Network Security to talk about how security, privacy and compliance will be forced to evolve to meet the demands of cloud computing, dynamic/portable workloads and a very new kind of outsourcing that’s emerging from these architectures. Ron had some very interesting points. You can also read the next chapter of the “Datacenter of the Future” titled “Security, Privacy and Risk Management” at the Datacenter Journal or download as a PDF here.

Jon. How do you see the adoption of cloud computing impacting the way we think about security today?
Ron. As with any new technology, there are advantages and disadvantages. I got a good start in my career working for US Internetworking in the late 90s where they were able to get customers to outsource their critical applications like Peoplesoft and SAP. I would love to make the argument we were more secure than the customer and that this was the main reason they wanted to give us their business. However, each customer was different. Sometimes they were outsourcing because of the cost model (rent vs. own), sometimes it was a manpower issue (USi was 24×7, and they were not), sometimes it was Internet bandwidth and sometime it was security. Of course in the 90s, some customers were still impressed that we had firewalls.

Today, as we look into the 21st century, I feel cloud computing will be something used in every
organization mostly as a method to save on costs. My concern is that users of cloud computing will
wash their hands of the security issues surrounding cloud computing. Who runs these servers? How
secure are they? How reliable are they? Some organizations that are interested in these types of
technologies might not even know to ask these sorts of questions.

Jon. One of the challenges of any security system is the sheer volume of data that needs to be
processed and interpreted. Unified Threat Management was the solution to this in today’s
infrastructure solutions. What role do you see UTM providing in cloud computing environments?
Ron. I think one of the big problems with security today is that computers are too flexible. They can
have a variety of purposes, uses and configurations. This gives the rise to complexity which is
often said to be the enemy of security.

My hope that in a cloud computing environment, customers will make use of single purpose
applications. For example, consider a web farm that runs 1000s of web servers. I would expect that
they are all configured, secured, patched and hardened the same way. This save you money and time and also makes it easy to spot when something isn’t configured correctly. If you have single purpose servers that are used a certain way, when they break, become compromised or have some sort of error, they behave differently. And lastly, when you go to harden these single purpose applications, it is much easier to know how they will work so you can put appropriate security measures like firewalls and system security settings in place.

My point here is that if done right, outsourcing a single application to a cloud computing service
can be very efficient and secure. If you were to compare this with an organization which simple
provided Linux operating systems to you, and it was up to you to configure and run these your
selves, you might still be "in the cloud" but you don’t have any of the benefits of the single
purpose applications.

And finally, to get back to UTM, if you have a cloud computing environment which is single purpose
(like a bunch of similar configured web servers) your UTM should be looking for behavior indicative
of a compromise or error. These are deviations from "known good" behaviors. In a random or mixed
environment, the UTM will be looking for "known bad" behaviors such as virus outbreaks, attacks
detected via intrusion detection rules and so on. There has been much written on looking for known
good and known bad behaviors. I am very much in favor of looking for "known good" but I also
understand that enterprise networks can be complex, even if there is an attempt to keep things
simple. Either way, you need a UTM (SIM, Firewalls, logging, IDS, anti-virus, etc.) to watch your
network. I just feel you are much more effective when monitoring for "known good" than "known bad".

Jon. All good security solutions blend proactive and reactive security systems as a way provide a
holistic picture of an environment. How do you see these tools adapting in highly virtualized and
dynamic computing environments?
Ron. There are some very, very cool reactive network security technologies that have been produced over the past decade. Unfortunately, I see very few of these being deployed operationally. The issue is reliability.

For example, if you want to reconfigure a firewall after a network IDS sees an attack, the IDS
better be right more than %99.99999 of the time. The first time it is wrong and legitimate traffic
is blocked, you have both a technical issue of needing to fix this detection rule, as well as a
political issue of impacting legitimate traffic.

What I do see is that anytime an organization can combine hardening of their network to only allow
authorized services with automation, they usually have a well run network. Hardening a network means different things to different people, but through the use of firewalls, running minimal configurations per host and having minimal user accounts a network can reduce the amount of potential attack space that can be exploited by an insider or outsider. Automation makes things happen regardless if a user is there to run the test as well. For example, patch and configuration auditing can detect a vast majority of missing patches and configurations which are against policy.

In virtualized environments, this is no different. The fact that a system is virtualized does not
make it any less immune to an attack. If an organization does not have the proper approach to
looking for unauthorized activity, configurations and changes to their network systems, be they
virtualized or real servers, they will likely have many servers that are vulnerable to exploitation
of some sort.

Jon. What opportunities does Cloud Computing provide to security companies. Do you see the management of security itself becoming a cloud service?
Ron. As we move further into the 21st century, we will see the emergence of new types of business models as well as new types of technologies that enable new types of services. In the late 90s, the state of the art MSP could watch your firewall and do some automated vulnerability scanning. Today, you can get an MSP to run your SIM, gather all of your logs, perform brand protection and certify that your ecommerce system meets the standards of the credit card industry. You can also get services for almost every type of function that occurs in your network including authentication, secure email and SPAM filtering, secure web hosting, secure chat hosting, secure DNS, secure data storage, secure SQL databases and so on. Many of these service companies offer combinations of various types of services as well.

What this means for a security company is that they have options. If are running a security company
and want to deliver a service to your customers, you now need to calculate if running your own
infrastructure truly gives you any advantage over running your own. The advantage could be a cost
savings, a time to market savings, or even some sort of scalability that would be hard to do alone.

Lastly, if you are funding a security company and concerned about cash flows, sometimes it is
difficult to decide how much money you should invest in your infrastructure before going live or
making any profit at all. With cloud computing, you can focus on getting your service offering
correct and purchase what you need from a cloud computing vendor as you go.