http://news.oreilly.com/2008/08/luke-kanies-wants-to-modernize.html

Errors aside in the transcript, it’s great article.  Administration is no longer a task, but a process.  If you are still doing administration via SSH, it’s time to look at Puppet.

package { "openssh-server":
      ensure  => latest,
      notify  => Service["sshd"],
}
file { "sshd_config":
       name     => "/etc/ssh/sshd_config",
       checksum => md5,
       ensure   => present,
       owner    => 'root',
       group    => 'root',
       mode     => '0600',
       require  => Package["openssh-server"],
       notify   => Service["sshd"],
}
service { "sshd":
       name       => "sshd",
       ensure     => running,
       enable     => true,
       hasrestart => true,
       hasstatus  => true,
       require => Package["openssh-server"],
}

This 22 line recipe does all of the following:

1. Installs the openssh-server RPM via ‘yum’.
2. Automatically upgrades the openssh-server RPM, if a newer version is available.
3. Makes sure the sshd_config configuration file exists and has the proper permissions.
4. Ensure the sshd server starts at boot time.
5. Ensure the sshd server is currently running.
6. If either the RPM is upgraded OR the sshd_config file changes restart the sshd service.
7. If during any time puppet runs again and the server doesn’t match the recipe it will change it back to this state.
8. Perform this task on every server you specify.

While the above recipe hasn’t been tested on other Unix platforms, only minor changes would be required.  Previously to do this you needed to create custom shell scripts, use Cfengine, purchase an expensive software automation tool, or manually perform this on each server installation.  Most options are hacks and not as graceful as Puppet. Package managers, while moved Unix administration into the 21st century (instead of the medieval times of compiling software), still have some warts.  Specifically package managers lack:

1. a good updating procedure.  Installations are well covered.
2. passing your own configuration files specific to your needs/wants.
3. performing the tasks in a specific order, or making sure specific actions occur before an application is installed
4. a service is running and will run at boot time

Things like ‘yum’ on CentOS/RHEL addressed #1 and #3 somewhat, but didn’t address configuration files, and the state of the service.  Before Puppet, it required creating custom RPMs.  With custom RPMs, the issue then became when updates occurred from the distro provider.

Puppet makes system administration a programming task, rather then manual labor process. It’s still very common to see administrators use a SSH prompt to manage each server. Manually performing administration is a time consuming and error prone process.  Puppet allows us a transfer of our best practices, apply our administration experience to the server’s configuration, and allows us to make network-wide installations with ease.

I agree with the notion; “Operations: The New Secret Sauce” (article #1 , article #2).  Puppet makes deployment of new VPS instances quick and exact.   System administrators are skilled individuals, great with keeping operations running smoothly, but usually not good with automation.  Puppet allows to transfer an administrator’s knowledge into a repeatable process.  Automating system administration is the next advancement with Web 2.0, Cloud Computing, SaaS, or whatever the latest trend may be called.  When using our HostCube service, Puppet puts the power of a large operations center like Google in the hands of much smaller companies, Cloud computing, while may address the quick provisioning of hardware, it doesn’t address operations. The ability to automate the install, configure, patch, monitor and backup are important aspects and HostCube does for you automatically and seamlessly.  The bigger and much more complex problem is system administration, not hardware provisioning.

Puppet allows us to Malkovich a setup, over and over and over again.

Existing 10for10 or custom plans may want to migrate to the new HostASite.com plans. Please contact billing to discuss any changes. If unsure, your control panel will display your hosting plan. If your plan is one of the above entries, and not labeled “custom”, it has automatically been upgraded and no action is necessary.

If you’ve outgrown our HostASite.com plans, we suggest visiting our managed VPS service HostCube. Managed VPSes give you dedicated CPU, dedicated memory, dedicated disk space, additional security, customization, and the support you’ve grown to know and love.  It’s like getting a dedicated server but at the fraction of the cost. HostCube VPSes are perfect for the reseller, web developer, or designer who needs to scale quickly and easily and not perform time consuming system administration.

The primary reasons for using a VPS:

• Dedicated resources (Quality of Service).
• More control of installed software.
• More secure.
• Dedicated IP address.

With shared hosting the hosting provider has to make sure each customer’s hosting configuration performs well. The administrator might be able to perform proactive measures, but in many cases this isn’t possible. This is because each shared hosting account uses the same memory, CPU and disk space. This is similar to a noisy neighbor in a massive apartment building. All it takes is one bad tenant to affect the others. Each customer must share the same resources on a shared hosting server.

Dedicated Resources (Quality of Service)

With a VPS you are allocated a fixed amount of resources (just like a dedicated server) and these resources are dedicated to you. Each customer is separated at the operating system level and another customer cannot affect your VPS. This increases quality of service since you have a specific amount of memory, CPU and disk given to your account. This is also the main reason for the differences in price between VPS and shared hosting.

A common issue on many oversold shared hosting providers is that you are suspended once you use too much CPU, memory or disk space. On a VPS you will never get suspended for this reason. Also, shared hosting’s dirty little secret is that many dynamically generated web pages (i.e. blog, forum, CMS, E-commerce) are primarily CPU bound. On a massively oversold shared server there is only so much CPU to go around.  These providers put 500-600 accounts on each server and for this reason the performance of ALL clients on that server is affected.

Other Advantages of Choosing a VPS Solution Include:

More Control of Installed Software

With a VPS we can customize the software to the customer’s exact specifications. Since software such as apache, PHP, MySQL, etc are dedicated to just your account. Conversely, shared hosting is configured to please the majority and exceptions are not possible. In addition, if you need a service (otherwise known as a daemon) running or custom programming libraries, this is all possible with a VPS.

More Secure

Since each VPS is separated at the operating system level, each customer is running in its own memory, CPU and disk space. This prevents your account from getting compromised when another customer forgets to update to their blog software to fix security risks that have been discovered – if hackers get into their shared hosting account they can quickly move horizontally into your account if you’re on the same shared server.

Dedicated IP address

This is important for any service that uses the source IP address for reputation purposes. This is extremely important with outbound E-mail (SMTP) and significantly decreases the chance of blockage because another customer sent out spam. To put this in perspective, we’ve seen cases where one misguided salesperson sending out less than 200 emails in an email blast has caused IP-based blocking of an entire IP (including all the responsible senders on that IP).

The HostCube Advantage

There’s one advantage that can sometimes be at credited to shared hosting: the provider manages all the system administration. Fortunately, this service is also provided when using a service like HostCube Managed VPS. The components of system administration include:

• Backups
• Service monitoring
• Security monitoring
• Software updates
• Software configuration

There’s no need to worry if your VPS is secure, and is your site running as our managed VPSes give you the best of both worlds: the ability to work as if in a shared hosting environment with the performance of a dedicated server.

Using 777 permissions on a folder means ANYONE on that server can write to it. Hackers LOVE this type of setup. In addition, with mod_php you must have at least 644 perms on PHP files, which ALSO means your files can be read by anyone. This means your MySQL password, key to your merchant account, etc., can be read by any customer on that shared server! If you ask me, not a secure solution.

We use suPHP instead of the default apache/mod_php for shared hosting.

SuPHP
Pros:

• PHP runs as your user/group
• PHP files can have perms of 640 (hiding things like passwords from other accounts)
• Files/folders written by PHP are written as user/group (no apache or other global user)
• Custom php.ini file per site (can add/remove security options)
• Can run php4 and php5 at the same time (on even the same site!)

Cons:

• Slower
• many PHP .htaccess options do not work (since you can have your own php.ini file this make this point moot)

apache/mod_php
Pros:

• Faster (about 25-30%)

Cons

• PHP safe mode isn’t safe
• files written by PHP are saved as the apache process (usually apache/apache user/group)

For our Shared hosting servers it’s a no brainer to use suPHP instead of mod_php, even if we take a performance hit. PHP is the #1 method hackers gain access to customer accounts. So once an account is hacked on a shared server, they can do much more damage with a mod_php setup. SuPHP accounts are much more sandboxed. We’ve had many hacked accounts via suPHP, and none of them have affected our other customers. In the future are going to replace suPHP and use LiteSpeed’s web server instead. It offers the same performance as mod_php and yet the same security as suPHP.

Our VPSes and dedicated servers we give the customer the option to select which PHP setup they want.

IMHO they completely missed the point and some of the blog commentators caught this. Regardless if their service is open, what they failed to mention, you are the system administrator. With that you have the responsibility of installing software, proactive monitoring, patch management, security, hardening and backups. This is fine if you are a full time system administrator; bad if you are a developer.

Cloud computing is about abstracting the technical details of your SaaS (Software as a Service) or PaaS (Platform as a Service). My favorite statement, “It just works!” applies here. What they are calling “open” is really a myth. Regardless if you have root access or not you still are locked into a specific hosting provider, OS, and the software applications you choose. Anyone that has switched dedicated server providers can attest this isn’t a small task. In addition, with the large amounts of SSH, FTP, IMAP and POP3 attacks we see its obvious proper system administration on a large scale isn’t already happening. What makes them think giving root access will make these other issues better?

Developers, for the most part, care about their development environment. In some cases yes, OS flavor does matter, but in most cases it does not. Developers typically want an environment that works and don’t have to worry about how to install and configure software packages. What’s important then? The development language and the tools that aid in the development. Giving root access to each developer (each on their own mind you) install to and configure a software package wastes time. Wading through docs, wikis, forums, and other online info trying to get a package configured, can be time consuming and frustrating experience. In some cases this requires a lot of technical skill. There has to be a better way. The better way is to offer pre-built configurations of services, programming languages and applications. This is what we offer with our HostCube service. Why reinvent the wheel each time you need a LAMP stack installed? Tools like Puppet automate this process and make it consistent.

Let me sidetrack for a minute and discuss the differences between system administrators and programmers. I’ve worked on both sides of the fence and from my experience, most developers make poor system administrators, as do many system administrators (sysadmins) make poor programmers. The mindsets are completely different. Developers care about how quickly they can develop their code and bring it into production. Sysadmins care about the stability, reliability and security of the service they are responsible for. As you can see, these two mindsets are always at odds with each other.

To solve this conflict, I believe in the traditional three tier development methodology. They are:

• Development – code that’s in flux and to “try out” new things
• Staging – some state of code that is stable and in testing before production
• Production – live code that’s being used by users, customers, vendors, etc.

This tiered environment allows for the best of both worlds and is recommended when a customer wants root access to a production server. Unfortunately I have seen many developers perform all of these tiers on their production system! In all of but the smallest of projects, this can lead disaster on many levels.

Creating a VPS for development is where I personally believe developers should/could have root. Let them play in their sandbox, break things and test out new code. Staging (which should mirror production configuration) and production should be managed by system administrators. In my opinion, developers, at least with production, should not have root access. At HostCube the value added is we perform the software installs, proactively monitor, patch management, security, hardening and backups. That’s what we’re experts in and have invested many years developing tools to automate this process. We also do realize unmanaged VPSes serve a valuable niche.

How do these providers offer these plans? Bandwidth, even if purchased in large quantities from a tier 1 provider, costs at least $10.00/Mbit. If the raw resources (hardware, bandwidth, power, employees, etc.) cost more than the monthly fee they are offering, how do they do it then? Do these providers have a secret sauce to put on their hamburgers? Do they cast a voodoo spell so they can offer more services for less? Are we raking our customers over the coals so we can drive around in our Porsche? Hardly.

The answer is simple and comes down to a mixture of four possible situations:

1. Oversell – They hope you don’t use the resources they are offering. Overselling itself is not a bad thing and we do it at low levels. We are now seeing overselling (or better aptly named false advertising) where it’s actually IMPOSSIBLE to offer such services on a shared hosting environment. That’s right technically impossible. They know while the uneducated consumer gets excited over these outrageous features, in reality will never even come close to using them.
2. Contract Terms – Put terms in their contract to make it almost impossible to use. Usually it’s either CPU limits and/or can upload specific file types (i.e. no videos). CPU limits are a big issue with any dynamic based web site and may run into this issue with low budget hosts.
3. Prepay – I’ve seen some offer 5 years prepayment. Time = Money. More money up for the hosting provider helps with profit. Hope that company is still around and shared hosting still exists in its current form.
4. Quality of Service – Cut corners any aspect of service. Use servers with no RAID or backups. Outsource their support to a third party company. Resell someone else’s service. Not monitor their servers and services for outages. Not have spare hardware, etc..

Yes the hosting industry is on this kick “my plans are bigger than yours!” competing on disk space, bandwidth and amount of Email accounts. In reality, hosting isn’t about this.

We realize those quantities (disk, bandwidth and Email) are somewhat important, what really matters is quality! This blog posting will focus primarily on the #4 situation which seems to affect the general hosting public the most. Hosting is primarily a service operation. As much as I would LOVE our servers to do it, they do not maintain themselves, nor does software configure itself!

Quality of Service (QOS) is about:

• how fast my web site loads
• how much spam is in my Email box
• how often is the web/E-mail service down
• how quickly can technical support resolve an issue
• how secure is my site from hackers and has the server been hacked or ‘rooted’
• using all of the disk space and bandwidth you actually paid for

Our company’s focus has primarily been B-2-B, where downtime is an expensive situation, especially for an Ecommerce based web site. We understand time is definitely equal to money.

We’ve gotten many sales inquires complaining about their existing provider. Here are some actual issues:

• Web site has been down for two weeks and the provider doesn’t know when it will be back up (no joke)
• The provider didn’t have valid backups and we’ve lost everything
• We have been down for one day and we are an ecommerce site. We are loosing thousands of dollars.
• Our Email service is unavailable every few days
• The server we were on was hacked
• It takes the provider over 24 hours to respond to our support tickets.

All of these are real pains that you would think motivate anyone to change providers. The ironic thing is after complaining about their existing provider, with the same breath, and then complains why is our service more expensive. All of these issues are QOS issues and has nothing to do with how big the provider’s disk, bandwidth or email is.

For a few dollars additional per month to use our service (in some cases it means a few less Starbucks Mocha Frappuccinos), you can be with a more reliable hosting provider. Yet these are the same customers that bounce from provider to provider wondering why they can’t find a hosting provider they are happy with.

I’m not saying our service is perfect, nor do we ever have outages. My point is at the price level some providers are offering, something has to be cut, and most likely it is some aspect of service. Our goal is to be a transparent hosting cloud. “It just works!”, has always been our motto and be proactive with our service as much as possible. As an employee or business owner you have many other things to worry about than if your Email service isn’t working. While to the outsider it looks like we don’t do much, behind the scenes there is much more going. Many applications are getting updated, monitoring of server or account attacks, getting Nagios alerts on server loads and then the occasional customer support question. Part of the purpose of this blog is to help explain the value added services we offer.

If you get excited in chasing after the “my plan is bigger than yours” providers then in the end our hosting services are not for you. If you care about QOS, then Empowering Media’s shared hosting and VPS hosting are perfect for you. The disk space and bandwidth plans listed with us are more importantly actually usable.

There is a revolution in the hosting industry and will discuss more about this in future blog postings.

• We can quickly upgrade a VPS (memory, CPU and disk space) the next level plan. Downtime is usually only a few minutes.
• Hardware is virtualized. Upgrading to more powerful hardware is transparent to the VPS.
• Redundant hardware. Power supply, CPU, memory, and hard drives all have spares. Should a component fail in most cases it will not affect a VPS.
• Hardware RAID 10. Many more hard drive spindles are available to store data, which improves disk IO performance.
• Pay as you go/grow. You only pay for the resources you need. You can upgrade/downgrade at any time.
• Less power consumption. For the customers who are concerned about the environment and being “green”. VPSes use less power when compared to dedicated servers.
• Higher server density per rack unit. The amount of VPSes you can fit in 2 rack units (otherwise known as 2U) would normally take 16 – 20Us. Since we pay per rack unit, this allows us to offer VPSes plans at a cheaper rate.

The limitations of a dedicated server are:

• Adding additional hardware/resources can lead to long periods of downtime
• Operating system configuration is tied directly to hardware. This makes it harder to swap different hardware, especially with the Microsoft Windows operating system.
• Must configure hardware to plan for future growth or peak usage periods. This means more money and resources wasted
• In many cases other vendors use software based RAID. Your CPU must do the RAID processing, which leads to more overhead.
• Most dedicated servers do not have redundant hardware.

Our VPS Plans are designed be better than dedicated servers in every way but at a fraction of the cost. Keep in mind, not all providers use quality high-end hardware like we do. Unfortunately a VPS can be placed on any type of hardware and in some cases other providers use the same low end servers as VPS nodes. All of our VPS node hardware is designed for five 9′s uptime (99.999%) and achieve this with ease.

VPS technology has improved dramatically over the past few years. We believe they are going to replace most situations where a dedicated servers were once needed. This isn’t to say VPSes should replace all servers. There are complex hosting situations when you should stay with a dedicated server:

• High disk IO (i.e. SQL server)
• CPU bound processes (i.e. application layer server)

While VPSes can be used in low volume situations, the virtualization overhead doesn’t make these situations the most efficient use of the hardware. We’ve designed our VPS plans specifically to make it apparent when a dedicated server is needed. Once you outgrow any of our VPS plans, in most cases it’s time to move your site to either a dedicated server or multiple VPS configuration.