As was foretold, we've added advertisements to the forums! If you have questions, or if you encounter any bugs, please visit this thread: https://forums.penny-arcade.com/discussion/240191/forum-advertisement-faq-and-reports-thread/

[sysadmin] on-call schedule - Always you

1121314151618»

Posts

  • wunderbarwunderbar What Have I Done? Registered User regular
    we actually do have a 3rd party cloud monitoring tool (an all in one solution that handles our ticket system, remote access, monitoring, etc) that is supposed to send out emails when servers become unreachable. Except that... it didn't.

    So I have a support ticket open with the company to figure out why.

    XBL: thewunderbar PSN: thewunderbar NNID: thewunderbar Steam: wunderbar87 Twitter: wunderbar
  • GilgaronGilgaron Registered User regular
    I assume they'll suggest that your SMTP server was among the ones that were down

  • wunderbarwunderbar What Have I Done? Registered User regular
    Gilgaron wrote: »
    I assume they'll suggest that your SMTP server was among the ones that were down

    No, it's a totally external service, they send an email from them to us, and our email is Office 365, so not on prem.

    XBL: thewunderbar PSN: thewunderbar NNID: thewunderbar Steam: wunderbar87 Twitter: wunderbar
  • wunderbarwunderbar What Have I Done? Registered User regular
    So troubleshooting this with the company. Looks like we found what is either a bug, or just some broken configuration somewhere.

    we turned an agent off, which should have triggered an email, it didn't. unchecked the box for "get emails when servers are unresponsive" saved. checked the box for "get emails when servers are unresponsive." Saved settings. And email within a couple minutes of turning the agent off.


    our UPS is also configured to send out emails when running on battery, and it didn't. And I can't get to the UI to check. And I'm home sick, so that'll have to wait.

    XBL: thewunderbar PSN: thewunderbar NNID: thewunderbar Steam: wunderbar87 Twitter: wunderbar
  • wunderbarwunderbar What Have I Done? Registered User regular
    When the digicert support person goes "huh, never seen that before" you know you're having a good time.

    p.s. certificates still remain the worst.

    XBL: thewunderbar PSN: thewunderbar NNID: thewunderbar Steam: wunderbar87 Twitter: wunderbar
    NaphtaliLD50MugsleyzagdrobDrovekThawmusDisruptedCapitalistlwt1973
  • electricitylikesmeelectricitylikesme Registered User regular
    wunderbar wrote: »
    When the digicert support person goes "huh, never seen that before" you know you're having a good time.

    p.s. certificates still remain the worst.

    Point of order: incorrect, it's still DNS.

  • twmjrtwmjr Registered User regular
    wunderbar wrote: »
    When the digicert support person goes "huh, never seen that before" you know you're having a good time.

    p.s. certificates still remain the worst.

    Point of order: incorrect, it's still DNS.

    a certificate problem caused by a DNS problem is when the real fun starts. if the problem somehow involves a printer you're in the IT Problem venn diagram convergence point of pure chaotic evil.

    zagdrobAridhol
  • AngelHedgieAngelHedgie Registered User regular
    I'm going to leave this story of Elon Musk "decommissioning" a server farm for all of you to read in utter horror and terror.

    Seriously. This is just...unhinged shit.

    XBL: Nox Aeternum / PSN: NoxAeternum / NN:NoxAeternum / Steam: noxaeternum
    DrovekwunderbarThawmusInfidelelectricitylikesmeSaerisFeldornDisruptedCapitalistlwt1973BlackDragon480
  • taliosfalcontaliosfalcon Registered User regular
    edited September 12
    i've switched jobs again, now i'm apparently completely gone from the IT/infrastructure admin side to a "devops manager" have i ever dev ops or managed before? ehhh.... but let's see what happens. It's kind of wild as i never in a million years would have seen my career going this direction 12 years ago when i was a windows sysadmin

    taliosfalcon on
    steam xbox - adeptpenguin
    schussThawmuswunderbarInfidelFeldornDisruptedCapitalistBlackDragon480
  • lwt1973lwt1973 King of Thieves SyndicationRegistered User regular
    Please stop chewing your gum while I'm talking to you over the phone about an issue, user.

    "He's sulking in his tent like Achilles! It's the Iliad?...from Homer?! READ A BOOK!!" -Handy
    DrovekThawmus
  • wunderbarwunderbar What Have I Done? Registered User regular
    Exchange Online is currently broken so a bunch of external email into Exchange Online isn't coming through.

    In case anyone was wondering what was going on today.

    XBL: thewunderbar PSN: thewunderbar NNID: thewunderbar Steam: wunderbar87 Twitter: wunderbar
    Feldorn
  • wunderbarwunderbar What Have I Done? Registered User regular
    Our phone system is currently down. calls to direct lines are not connecting but calls out are working. This is with a vendor so I'm mostly on hold for the day.

    However, our ticket system is also down which is nice because that means no one else can submit tickets about the phone system being down.

    XBL: thewunderbar PSN: thewunderbar NNID: thewunderbar Steam: wunderbar87 Twitter: wunderbar
    ThawmusCarpyzagdrob
  • FeldornFeldorn Mediocre Registered User regular
    I'm tired of all these mother fracking CVEs in this mother fracking Netscaler ADC.

    That is all.

    wunderbar
  • wunderbarwunderbar What Have I Done? Registered User regular
    I'm just tired of CVE's in general.

    XBL: thewunderbar PSN: thewunderbar NNID: thewunderbar Steam: wunderbar87 Twitter: wunderbar
    LD50Feldorn
  • LD50LD50 Registered User regular
    wunderbar wrote: »
    I'm just tired of CVE's in general.

    Tell me about it. That and data leaks from vendors.

    I guess they might be related...

    Feldorn
  • MyiagrosMyiagros Registered User regular
    I know I'd be bored if I didn't work for a MSP, but sometimes there's just too much shit to do and I have to stick to what I know instead of getting time to play with something new.

    Client wants to implement something for workflows, I figure something set up through Power Automate with a small DB or Excel sheet to hold the data might do the trick.
    • I've not done something like this before
    • I'm not a developer, coder, scripter (only basic stuff), etc
    • I don't know someone who could do this for them
    • I don't know of software that could do this for them
    Seems like a fun little project, learn a bit more about Power Automate or other 365 apps that might be able to do this stuff, could come in use later down the road, etc. Only problem is, gotta charge them through this whole process(we're a per hour shop) and no idea on how long it would take or whether it will do what they want, plus having to support it in the future. But, then the reality of MSP work rears it's ugly head and I realize I'll never have the time to focus on something like this since I have another couple dozen clients constantly keeping me busy.

    iRevert wrote: »
    Because if you're going to attempt to squeeze that big black monster into your slot you will need to be able to take at least 12 inches or else you're going to have a bad time...
    Steam: MyiagrosX27
  • SiliconStewSiliconStew Registered User regular
    Myiagros wrote: »
    I know I'd be bored if I didn't work for a MSP, but sometimes there's just too much shit to do and I have to stick to what I know instead of getting time to play with something new.

    Client wants to implement something for workflows, I figure something set up through Power Automate with a small DB or Excel sheet to hold the data might do the trick.
    • I've not done something like this before
    • I'm not a developer, coder, scripter (only basic stuff), etc
    • I don't know someone who could do this for them
    • I don't know of software that could do this for them
    Seems like a fun little project, learn a bit more about Power Automate or other 365 apps that might be able to do this stuff, could come in use later down the road, etc. Only problem is, gotta charge them through this whole process(we're a per hour shop) and no idea on how long it would take or whether it will do what they want, plus having to support it in the future. But, then the reality of MSP work rears it's ugly head and I realize I'll never have the time to focus on something like this since I have another couple dozen clients constantly keeping me busy.

    And this is why I never have and never will work at an MSP.

    Just remember that half the people you meet are below average intelligence.
    Thawmuswunderbartaliosfalcon
  • FFFF Once Upon a Time In OaklandRegistered User regular
    Myiagros wrote: »
    I know I'd be bored if I didn't work for a MSP, but sometimes there's just too much shit to do and I have to stick to what I know instead of getting time to play with something new.

    Client wants to implement something for workflows, I figure something set up through Power Automate with a small DB or Excel sheet to hold the data might do the trick.
    • I've not done something like this before
    • I'm not a developer, coder, scripter (only basic stuff), etc
    • I don't know someone who could do this for them
    • I don't know of software that could do this for them
    Seems like a fun little project, learn a bit more about Power Automate or other 365 apps that might be able to do this stuff, could come in use later down the road, etc. Only problem is, gotta charge them through this whole process(we're a per hour shop) and no idea on how long it would take or whether it will do what they want, plus having to support it in the future. But, then the reality of MSP work rears it's ugly head and I realize I'll never have the time to focus on something like this since I have another couple dozen clients constantly keeping me busy.

    And this is why I never have and never will work at an MSP.

    I've dipped my toes in some Power Automate stuff. It's very interesting in how powerful it can truly be, especially if everything else you use is M365 based. For example, if you can use a SharePoint list instead of Excel as a source, reading data out of the source gets much easier.

    That being said, Power Automate is obtuse as fuck when it comes to documentation and all of it's little weird gotchas, so figuring stuff out can be a time-intensive, tedious task, which may not be preferable if you have to charge by the hour.

    Huh...
    Feldornwunderbar
  • schussschuss Registered User regular
    Except SharePoint has a bunch of weird tipping points where lists suddenly slow down to a crawl. Not to mention the "native" connectors in powerbi are slower than calling an excel in SharePoint via the excel connector.

    Feldorn
  • FFFF Once Upon a Time In OaklandRegistered User regular
    Interesting, I haven't noticed any appreciable slowdowns but I've only dipped my toes in it. As such I really don't have anything with a lot of data yet.

    I had been trying to do more complex stuff in Power BI but gave up, mostly because we're not going to be allowed budget enough to buy the number of licenses I'd need to be able to share anything. That being said, that I've been able to convince people to use SharePoint for anything is a minor miracle. The alternative being literally nothing written down, documented, recorded, or remembered.

    Huh...
  • wunderbarwunderbar What Have I Done? Registered User regular
    fun morning when a Google recaptcha just randomly decides to break despite there being no changes to anything.

    XBL: thewunderbar PSN: thewunderbar NNID: thewunderbar Steam: wunderbar87 Twitter: wunderbar
  • DrovekDrovek Registered User regular
    wunderbar wrote: »
    fun morning when a Google recaptcha just randomly decides to break despite there being no changes to anything.

    How do you know you weren't replaced by a robot without even noticing? Have you perhaps noticed superhuman strength or constantly activating the store entrance alarms lately?

    steam_sig.png( < . . .
  • electricitylikesmeelectricitylikesme Registered User regular
    Why is Solarwinds so popular? The interface is utter garbage, and loves to auto-reload - by refreshing the page and then slowly reloading all the subviews it wants to put in it. I have difficulty believing anyone determines anything from this dumpster fire, so much as you buy it because it gets sold to management.

  • Inquisitor77Inquisitor77 2 x Penny Arcade Fight Club Champion A fixed point in space and timeRegistered User regular
    Why is Solarwinds so popular? The interface is utter garbage, and loves to auto-reload - by refreshing the page and then slowly reloading all the subviews it wants to put in it. I have difficulty believing anyone determines anything from this dumpster fire, so much as you buy it because it gets sold to management.

    My company is forcing its biggest products to integrate Carbon for "IP protection purposes" even in cases where it makes absolutely no sense because there is literally no IP to protect, just because some senior leader was sold on it as some kind of magic thing that cures cancer.

    Never underestimate the ability of people at the top to make the absolute dumbest fucking decisions.

    electricitylikesmeBlackDragon480
  • FeldornFeldorn Mediocre Registered User regular
    edited November 20
    Why is Solarwinds so popular? The interface is utter garbage, and loves to auto-reload - by refreshing the page and then slowly reloading all the subviews it wants to put in it. I have difficulty believing anyone determines anything from this dumpster fire, so much as you buy it because it gets sold to management.

    Cost is one factor, I found their products to be on the low end of the cost spectrum for a supported, mostly turnkey solution.

    Marketing is another, of course.

    Feldorn on
    wunderbar
  • wunderbarwunderbar What Have I Done? Registered User regular
    Yeah you get a passable product at a reasonable price. Are there better products? Yes. Are they going to cost more? yes. Solarwinds is far from perfect but I worked at a place where that was the product/cost we could get approved, so that's what we went with.

    XBL: thewunderbar PSN: thewunderbar NNID: thewunderbar Steam: wunderbar87 Twitter: wunderbar
  • FFFF Once Upon a Time In OaklandRegistered User regular
    I just assumed we have Solarwinds "because Cisco". Also, the sole user/admin of it is...intractable (to put it mildly) when it comes to literally any decisions about our network.

    In other, worse news, I've been told that we have to drop head count again. This time, as opposed to last time, the decision is not being made by Exec for us.

    My team is already small, only 5 people + myself and we don't have enough bandwidth as it is, but I still have to determine whom of my team can be let go with the least amount of disruption for my boss and Exec to make a final decision. Even worse is it sounds like Exec wants us to drop 3 people total. I know who else I would suggest (not from my team) but I don't know if my boss is in a place to hear it so I'm keeping my mouth shut.

    Of course, they want an answer by Monday. Tuesday morning at the latest. Happy Thanksgiving indeed.

    Huh...
  • MugsleyMugsley DelawareRegistered User regular
    "This is a decision a manager needs to make. Don't you have the last round of yearly reviews?"

    Drovekwunderbar
  • FFFF Once Upon a Time In OaklandRegistered User regular
    Mugsley wrote: »
    "This is a decision a manager needs to make. Don't you have the last round of yearly reviews?"

    Unfortunately, I am the manager (my bad, I didn't make that clear). It's not a guarantee that they'll lay off who I decide on, but it's a pretty good bet that they will. I was told basically "decide as if you had no choice but to choose one". Looking at each member of my team, performance this past year, and more/less progression wrt new initiatives and projects, (I've only been managing the team for a year as of last week) I know who I'm going to decide on but it's gonna be ugly no matter what.

    As far as employee reviews, we've never had a process in the 15+ years I've been employed here, but HR is totally going to "Develop a formal process for employee performance evaluation"...by 2027. 🙄

    Huh...
Sign In or Register to comment.