Help Us Help You

As Director of Network Compliance, one of my less enjoyable jobs is to explain to a customer whose server has just been hacked exactly what damage has been done and what data can and cannot be saved.

All competent webhosts should provide customers with hosting solutions that are secure out of the box. Managed hosting providers work hard to make sure that what we provided customers remains secure on an ongoing basis. But most people can’t make much use of a hosting solution without taking it and making it their own–adding what they need to make their business work. Unfortunately, start adding anything to the solution you’ve been provided and it changes the security profile of the box.

It is not always obvious when a server is hacked. A malicious piece of code may lie buried in a random directory for weeks or even months before it activates and begins doing harm to the server or to other machines.

Unfortunately, this means it is usually not possible to simply restore a customer server from backups. Though we keep a daily, weekly and monthly backup of every VPS customer server, there is no way of knowing if the corruption occurred before the earliest backup was made. All too often, this means a customer is left rebuilding his or her server from scratch. Thankfully, this is a rare occurrence. ServInt, as well as most reputable software providers take active steps to deter and prevent malicious attacks.

In the 1990s websites were largely static html pages. The bulk of the work was in designing the pages. Once they went live, they changed little and needed updating only as often as the owner wished to update the content. But two things have occurred over the last 15 years that have dramatically changed the way webmasters interact with their sites.

The first change has been the development and implementation of server-side software such as PHP, ASP, and even WordPress and Magento. Most websites are no longer simply pages of static text, they are highly interactive and highly customizable. These new software developments open up a world of new things you can do, but they also open up all kinds of security pitfalls that need to be carefully avoided.

The second change is that the hardware that hosts these sites has become far more powerful. Advances in technology have not only increased the processing power and memory of host machines, but they have brought the price of this technology down so far that these machines are available for even entry-level hosting packages.

The keys to the Ferrari.

What this all means in terms of customer experience is that where at one time signing up for a web hosting account meant getting to borrow a bicycle to ride down the block, now it means getting the keys to the Ferrari.

Over the past five years especially, this combination of increasingly complex software and more powerful hardware has led to a dramatic increase in hacked servers on the web. Good managed web hosts routinely monitor their clients’ servers looking for any suspicious spikes in usage that might indicate unauthorized access. Companies should—and many do—try to work with customers to ensure that their server is ‘hardened’ (a pretty loaded term) and when circumstances dictate, that they have firewalls in place. But even with these steps and many others—forgive me if I must be intentionally vague here—at some point there is little even the most proactive host can do to anticipate a hack.

This is where customers come in.

One of the single best ways to prevent hacked servers is to keep all server-side software up to date. Vendors are constantly learning about and correcting weaknesses in their software code, releasing free updates to their users.

It would be great if a hosting company could magically update all of the third-party software customers have installed on their servers, but with literally thousands of different pieces of software for web designers to choose from, this is impossible on a practical level. A managed host does its part by upgrading operating systems and kernels as needed, but without consulting each customer personally and maintaining extensive lists, there isn’t even a way to determine all the software that is running on a customer’s server, let alone individually updating each customer’s products.

So what can customers do to protect themselves? Here are a few steps:

First, only install the software you need. Each application installed on a server opens that server up to any security risks the software has. The fewer pieces of software running on your system, the lower the chance of our server security being compromised.

Second, keep track of your installed software so you know what you’ve set up. I can’t tell you how many times I have traced the source of a security compromise for a customer only to have them say, “I didn’t even know that was still on my server.”

Third, keep the software you are running on your server up to date. There are options you can enable in cPanel and some other control panels to inform you when any software you downloaded directly from your control panel has been updated.  Also, many places such as The Symantec Security Focus Bugtraq list allow you to sign up for emails that will send you information on software updates.

For all other software, there should be a page on the designer’s site which lists current versions and where to download updates. Keeping a folder of bookmarks of these sites can be a real life saver. Simply surf to the pages you have marked a couple times a month and check for software updates.

Fourth, ensure that the computer you are accessing your website from is properly protected. Keeping your server locked down against attacks and completely up to date is only so helpful if a piece of malware on your desktop tracks your keystrokes and finds out your server’s password when you log in. Having your server’s  root access compromised (getting “rooted”) makes for a very bad day.

Finally, it sounds simple, but it is very important. Change your password, and change it often.

A few simple steps can put the power of security in your hands and go a long way to ensuring your server doesn’t fall victim to attack. A good managed host will work tirelessly to make sure that your business always stays up. But if you keep a close eye on what you put on your server and keep it updated, it’ll go a long way in helping us help you.

Photo by jonworth-eu

Remembering the Macho Man

I was saddened to hear of Randall Mario Poffo’s untimely death earlier today.  Of course, everybody knew Mr. Poffo as ”Macho Man” Randy Savage — or for me, just  ”The Macho Man,” the professional wrestling character he made famous in the 70′s, 80′s, and 90′s.  If you were a wrestling fan back then, it was tough not to root for Randy.  He was brash, he was colorful, and he was the living embodiment of professional wrestling attitude.

I didn’t know him personally, of course, but in 2003 or thereabouts, when I was working in our Network Operations Center (pre-MST), the Macho Man was a client of ServInt’s.  You can imagine the buzz around the office the day he joined the network:  ”The real Macho Man?”, everyone would ask.  ”Yeah, the real Macho Man.”  I suppose to some there were “more serious” customers that one might use in a business or social setting to represent the quality and prestige of our company’s clientele.  But for me, at that time, the first name that always came to mind — our coolest customer, in other words — was The Macho Man.  And not surprisingly, people continued to ask, “the real Macho Man?”  – and my response, delivered with a grin, would always be the same:  ”yes, the real Macho Man.”  There was no need for me to go any further.  The name had been dropped.  My mission was accomplished.

The real joy in having Mr. Poffo’s server under our wing was being able to talk to him on the phone on rare occasions.  For us ’round-the-clock tech support guys, it was always a real treat when Mr. Poffo would call and ask us for help.  You’d feel lucky if you were the guy who got to answer the phone when Randy was on the other end of the line.  I remember him as a very curious and patient person, polite and professional.  I guess I shouldn’t have been surprised that he was just a regular guy, but I was, if just a little.  And if you’re curious… no, his every-day voice wasn’t the booming growl he used inside the ring, in TV commercials and back-stage interviews.  But it was unmistakable — there was no doubt that you were talkin’ shop with Randy Savage.

After every Macho Man call, for a good 30 minutes or more, we couldn’t help but talk amongst ourselves in pathetic imitations of the Macho Man.  It was impossible not to.  Technical jargon just sounds better when you say it like Randy Savage.  More fun, too.  Every conversation was rehashed, sentence for sentence, for everyone else in the room.  For some reason, it was incredibly important that each of us knew exactly why he called.

As much as I may have wanted to, I never had the courage — nor did I think it was appropriate — to ask him for a patented “Snap into a Slim Jim, oooh yeah”!  But I was just as satisfied with phrases like “what version of Perl do I got” and  ”let’s just reboot it.”

“Macho Man” Randy Savage was an icon of my childhood and my favorite client during my tech support years.  On behalf of ServInt, I’d like to extend my sincerest condolences to Mr. Poffo’s family and friends.  Thanks for all the memories, Macho Man.

Photo by goodrob13

ServInt Answers: What Does “Network Compliance” Mean?

Hi, I’m Mike Witty, ServInt’s new Director of Network Compliance.

To those outside of our industry, the words “Network Compliance” probably sound a bit cryptic – even bureaucratic.  But network compliance actually affects every single customer we have.  I’d like to take this opportunity to explain how, and why.

First, a little about me. I’ve been at ServInt for more than ten years now — roughly 100 years in Internet time. I first cut my teeth as a member of ServInt’s Managed Services Team and worked my way up to management from there. Over the years, I’ve seen our service offerings evolve dramatically, and it’s been an exciting ride.

When I started here in 2000, the vast majority of the content on the Internet was static – basically little strings of HTML.  These sites rarely broke on their own, if ever. Since then, there have been seismic shifts in web technology and there is a constant race to remain up-to-date and competitive.

But, as always, there’s a tradeoff.  For every internet app that changes our lives for the better, there’s a new security concern, and a new way for the bad guys to reach out and cause trouble.  As head of ServInt’s Network Compliance Team, my job is to protect and defend our customers, content creators, and ServInt as a whole, to the best of our ability. Here are just a few of the challenges my team and I face daily and some tips on how to avoid becoming a victim.
Read more

Re-Introducing ServInt’s Managed Services Team

We’ve just made our world-class support a little more world class.

The goal of this project was to further increase our capability to deliver fervent, quality support 24-hours-a-day, 365 days a year. We’ve leveraged the expertise of our existing staff and increased the size of our Managed Services Team (MST) by 40% this year alone. By spreading this wealth of new faces and new talent throughout the day — and night — we can improve support reliability and consistency around the clock.  Not only does that help our North and South American customers sleep more  soundly at night, it also gives our international client-base access to the same high-level of service that they need and expect when the sun rises in their neck of the woods.

It’s easy to go out and interview candidates with what, on paper, look like relevant skills.  But finding intelligent, motivated people that are eager to learn, love being part of a successful team, and relish the satisfaction of providing excellent service to people that need it are what we look for in our MST members. That’s the combination that has always worked best for us and we’ve been fortunate to be able to stick to that formula right up to the present day.

Over time, our support has focused less on simply fixing an issue, and more about educating the client on what went wrong, why it happened, and how we are going to fix it.  The days of our department just “fixing stuff” are fading fast.  While there will always be a need for that kind of service and support, and we’ll always be there for folks who need a quick “repair”, it is equally important to prove that we’re prepared to translate our knowledge to clients in a way that’s easily digested and deployed.

Alongside this new initiative, we’re developing a long requested documentation project that will act as a consistently up-to-date repository for best practices for those who want more input into their server experience. This will be a purely informational project and we will continue to focus the vast majority of our resources on live, in-house, 24-7 support.

So, at the end of the day, this is an upgrade and not a reinvention. Time and time again, we’ve been told that our support is the crux of our business and it’s why people stay with ServInt for so long.  When we took a long hard look at our support to see what we could improve, we came to the fairly obvious conclusion that the soul of the department was its people, and with our latest batch of brilliant team members we aim to keep it that way.