1. Never perform a major system upgrade on a Friday afternoon (or Friday, for that matter). If it goes wrong, it is unlikely (unless your SLA says otherwise) that your vendor will be able to provide a fix for you by the end of the weekend.
2. Know your support contract. Again, if your SLA (service level agreement) says support is only to be provided at specific hours, don't expect it otherwise.
3. If you're performing an upgrade, make sure you back up your configuration beforehand.
4. Never tell the support agent that you're "really good at [insert Unix system here]" if you didn't perform #3 above. You'll be put on mute as your support analyst goes to laugh his ass off.
5. Realize that if your upgrade fails and requires a hardware replacement, you will not get it until Tuesday because your vendor won't be able to ship it until Monday.
6. Your vendor's developers are people. If it's a gorgeous 75+ degree day in Seattle, they'll do whatever they can be out of the office at 3pm.
Yes, I ran into a customer yesterday who must've not been paying attention during his sysadmin course. He, as a result, will not have a working server until Tuesday at the earliest. Should the upgrade have failed? Of course not. Should he have been performing an upgrade on a Friday? Of course not. A bad situation all around.
I wish developers would build in a time check in their upgrade tools (that'd be run against a server in their timezone). It should prevent upgrades on Fridays after noon. It should only allow upgrades Monday thru Thursday, realistically...
So anyway, the first month and a half at the new job has been going well. I'm still working hard to learn the products, and I think I'm making progress. It doesn't hurt that I have a good background in Linux support, so there's been less of a learning curve. I'm feeling good so far. Yay me.
Labels: Rants, Work