Its been a while for me, but there was a period of my life where I was working for a company that was in a constant state of outage. They had a mix of services, and over the course of 2 years, I was flown across the country and around the world apologizing for the (lack) of services that my company provided. While I love traveling and accumulating Air Miles, this was not my idea of a trip as you can imagine.
Well, simply put, the Customers stayed! As you can well imagine, when a business and its service is being impacted by a 3rd party the natural inclination of anyone is to pull the service and move to another vendor. When SLAs are constantly being missed and month on month, services are not improving this is even more likely.
Its easier to say than to do – especially when you don’t have any news or even worse when you have bad news (you expected it to be fixed in 1 day and it’s going to take 1 week!). As I mentioned earlier, I quickly became skilled at speaking to Customer’s face-to-face which happened with quite a few of our Tier1 customers. I also became skilled at sending out mass emails, posts on message boards and forums and phone calls. Setting a timeline for an update by any/all of these methods and then ensuring that I met that criteria were key.
From my point of view, it was 2 years of hell, but no one single customer was impacted for that total amount of time. How do you fix this though, because it is extremely draining on your staff regardless … well, Quality Control is useful. Make sure that any new product launches are properly tested and tested and tested again before being released into a live environment. Try to get your staff to break it if possible while it’s in the testing phase. Make sure your documentation, release notes, and training material are complete and accurate.
Ensure that Senior Management gets involved at the appropriate intervals based on your Escalation Matrix so that they are aware of the impact to the Customers … DO NOT be afraid of escalating. If you are ON CALL 24/7 so are they! The money will be released when the phone rings at 2am!