Client call in panic mode. CRM software techs logged in and filled up the disk.
With 50GB free out of 100GB, I wondered how that's possible. They wanted to copy the entire directory to another for "safety".
Explained I could backup and restore if needed in seconds. But no, they had done as they said. Result? Disk filled, everything halted.
I'm out and about.
Proxmox doesn't let me modify virtual hardware through its Android app or limited mobile web page. Desktop site request doesn't help, layout's all messed up. In emergency mode, I set up an xrdp FreeBSD jail with Mate and Firefox, connected via VPN on my smartphone, accessed the client's Proxmox.
Added a virtual disk, extended the LVM, and the underlying XFS filesystem.
Total time? A blind 10 minutes.
Triumphantly, I call the client (who knew I was away from a computer) to report success.
The naive response: "Okay, but all this time just to expand a virtual disk?"

I decide it's better not to comment 😊

#TechSupport #SysAdmin #Proxmox #ClientStories #ITLife

@stefano They don't pay us for what we do, they pay us for what we know.

@sullybiker they don't pay us to solve someone else's mess 🙂

@sullybiker me too. But not in 10 minutes while all I have is my mobile phone.
😆

@stefano Be careful with that. Walk on water enough times, people will start thinking it's normal.

@sullybiker this. So much. Now you know the next time, solving the same problem will take no less than 4 hours of absolute downtime of all the systems, even those that are not involved.

If you don't do it for you, at the very least do it for them, so they can understand the underlying complexity of the issues at stake.

See .. this happens to every professional: if "you" can't make your work be respected and valued for what it is: "nobody" will.

@stefano

@gnemmi @sullybiker I will. In general, I have a trusting relationship with my clients. They know that I give my best, and when I say something is impossible or time-consuming, they know it's true and trust me. It works with some, with others... less :-)
Additionally, interventions of this kind do not incur any costs for them (they are covered by comprehensive agreements I provide), so whether it takes me 1 minute or 10 minutes, it makes no difference to them.

@stefano @sullybiker this is pretty subjective, as it derives from personal experience, but in my case I found out that even if the client is on a particular agreement, in the end, this kind of episodes have a negative impact as they tends to diminish the quality of your craft on the eye of the client, ultimately leading them to believe that whatever it is that you do has no value, or at least, is not as valuable as they initially thought it was. Clients have an inherent tendency to undervalue

@gnemmi @sullybiker True. Tomorrow morning I will have a (probably unpleasant) meeting with a client I've had a good relationship with for years. They have an inadequate server, not powerful enough, and it's starting to show signs of age (disks starting to give problems, moments of overload, etc.).
We have meetings, I explain to them that a new server (and a new type of setup) is necessary, they agree and say to proceed in a couple of months (now they can't as they're busy), in the meantime, we keep what we have running, as much as possible.
However, as soon as they experience the first slowdown, they impatiently call saying things are not working as they would like. In 8 years, they have spent little (compared to the service they have) and have never had a minute of downtime.
If trust has eroded, starting from February 1st, they will find someone else to handle this issue. I only work when there is mutual trust.
When they don't have problems, they get used to it and believe it's all normal, not realizing that there is work behind making it happen.

@stefano @sullybiker exactly!
They take it for granted!
Almost any given client will never acknowledge the professionalism on your craft. They will always dimish your work for the simple reason that "to their eyes" if you can do it so fast and in such a restrained environment, it wasn't due to your expertise: it was because it was easy.
Bottom line: providing high profile solutions to those who can't appreciate it hem, is digging your own grave.

Old saying: "Don't throw daisies to the pigs"

Follow

@gnemmi @stefano 9pm tonight, on Slack AND email "Sorry to bother you but we can't contact our GPU server". They ran it out of memory, and it came back on its own. I am not on call, but this is exactly what we're talking about.

· · Web · 0 · 1 · 2
Sign in to participate in the conversation
Mastodon

The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!