As was foretold, we've added advertisements to the forums! If you have questions, or if you encounter any bugs, please visit this thread: https://forums.penny-arcade.com/discussion/240191/forum-advertisement-faq-and-reports-thread/

[Sysadmin] Routing to null

SeidkonaSeidkona Had an upgradeRegistered User regular
Below is a compilation of links we've found to be helpful for Sys Admins (capitalized because we're important, like a towel):

AntiVirus/AntiMalware Tools:
Networking Tools:
Miscellaneous Tools:
  • APC Power Estimator - estimates your power needs based on devices, which you can specify
  • Dependency Walker - scans modules and builds a hierarchical dependency tree
  • Greenshot -- Screenshot + utilies
  • HJSplit -- Split/Join files
  • Jing - Screenshot + utilities
  • Notepad++ -- Enhanced Notepad (e.g., color coding)
  • WinDirStat -- File space analysis
  • HTAccess Redirect Generator -- Generate Redirects
  • Hostsman Manage Multiple Hosts files from one utility.
  • PC Decrapifier inventories all the bloatware (HP/Dell Utilities, etc) on a PC based on user-driven feedback and recommendations and removes them sans-uninstallers. Great for cleaning useless shit off out-of-the-box PCs.
  • Recuva undelete software that restores deleted files, as well as files on damaged or freshly formatted drives. Paid corporate licensing but free for personal use wink wink nudge nudge.
  • NirSoft has utilities to do nearly anything you want to do from sniffing passwords out of FTP/HTTP/SMTP traffic on your network to editing Outlook NK2 Autocomplete files to retrieving your Windows/Office product key to a nice viewer for BSOD minidump files.
  • Screen Connect Free Use this to create a free tech support portal for yourself for personal use.
  • VM for learning Puppet
  • MobaXterm Super handy terminal program. Has local Cygwin shell and can spin up a lot of servers. Get it.
  • Check to see if it's Thursday. It may or may not be.

Printer Configuration tools:

0amLr1Ul.png

Ransomware/Cryptoware information sheet

Stress Relief Tools:

...more to come

last updated 5.25.2017

Mostly just huntin' monsters.
XBL:Phenyhelm - 3DS:Phenyhelm
Seidkona on
«13456799

Posts

  • wunderbarwunderbar What Have I Done? Registered User regular
    I'd vote for adding Visual Studio Code to the miscellaneous tools list. It's completely replaced notepad++, and a lot of other editors I used to use.

    XBL: thewunderbar PSN: thewunderbar NNID: thewunderbar Steam: wunderbar87 Twitter: wunderbar
  • RandomHajileRandomHajile Not actually a Snatcher The New KremlinRegistered User regular
    @ArcSyn I think you did exactly what you needed to do. Since the SonicWall needs to route traffic for clients, it needs to know how to do that. I think that was the tricky thing on ours when I changed the operating mode.

  • iTunesIsEviliTunesIsEvil Cornfield? Cornfield.Registered User regular
    @Entaru FYI: Microsoft shuttered CodePlex, so Terminals is on GitHub now I do believe: https://github.com/terminals-Origin/Terminals

  • FeralFeral MEMETICHARIZARD interior crocodile alligator ⇔ ǝɹʇɐǝɥʇ ǝᴉʌoɯ ʇǝloɹʌǝɥɔ ɐ ǝʌᴉɹp ᴉRegistered User regular
    wunderbar wrote: »
    Guys, I've decommissioned two old physical Server 2008 R2 boxes this week and it feels so good.

    And I just had to spin up a new 2008 R1 (SP...2?) x86 instance because "hey, it's better than Server 2003!" <accidentally breaks pen in half>

    2008 is end of life in December so I hope that instance is completely segregated from the Internet

    every person who doesn't like an acquired taste always seems to think everyone who likes it is faking it. it should be an official fallacy.

    the "no true scotch man" fallacy.
  • DrovekDrovek Registered User regular
    Feral wrote: »
    Drovek wrote: »
    Feral wrote: »
    God, I'm sick of the "what changed?" style of troubleshooting

    "The XYZ server stopped doing ABC thing. Did we change anything on it?"

    That made sense 20 years ago when software mostly ran locally and systems were less interdependent.

    Today, XYZ server might be actively communicating with a dozen other servers every minute; users access it through a browser that has it's own security settings and updates; not to mention the underlying network and virtualization infrastructure it is dependent on; and god forbid there's any cloud or hosted component to it.

    Things are always changing every hour of every day.

    I've got a better idea. Instead of vaguely reporting a problem to me and ask me "what changed?" how about you do your fucking job and read the goddamn error message, look in fucking Event Viewer or pull up the software's diagnostic logs, and actually do some fucking troubleshooting.

    Fuck.

    And this is why configuration management is as needed as code version control.

    I agree, but that enables a similar problem.

    "The CM system says something about disabling weak SSL ciphers and now the XYZ server stopped working. Can we roll back that change?"

    "What makes you think those two things have anything to do with each other?"

    "It was the last thing we changed before the server stopped working."

    Fast forward to me going into the damn server and seeing that it's out of HD space.

    There's no substitute for having IT staff who can actually do troubleshooting.

    In that case a good alerting system would've caught that. I mean, I don't disagree with your final statement, but each tool has a given purpose.

    In the end, my preferred way of doing things would be phoenix servers. :P

    steam_sig.png( < . . .
  • FeralFeral MEMETICHARIZARD interior crocodile alligator ⇔ ǝɹʇɐǝɥʇ ǝᴉʌoɯ ʇǝloɹʌǝɥɔ ɐ ǝʌᴉɹp ᴉRegistered User regular
    Drovek wrote: »
    Feral wrote: »
    Drovek wrote: »
    Feral wrote: »
    God, I'm sick of the "what changed?" style of troubleshooting

    "The XYZ server stopped doing ABC thing. Did we change anything on it?"

    That made sense 20 years ago when software mostly ran locally and systems were less interdependent.

    Today, XYZ server might be actively communicating with a dozen other servers every minute; users access it through a browser that has it's own security settings and updates; not to mention the underlying network and virtualization infrastructure it is dependent on; and god forbid there's any cloud or hosted component to it.

    Things are always changing every hour of every day.

    I've got a better idea. Instead of vaguely reporting a problem to me and ask me "what changed?" how about you do your fucking job and read the goddamn error message, look in fucking Event Viewer or pull up the software's diagnostic logs, and actually do some fucking troubleshooting.

    Fuck.

    And this is why configuration management is as needed as code version control.

    I agree, but that enables a similar problem.

    "The CM system says something about disabling weak SSL ciphers and now the XYZ server stopped working. Can we roll back that change?"

    "What makes you think those two things have anything to do with each other?"

    "It was the last thing we changed before the server stopped working."

    Fast forward to me going into the damn server and seeing that it's out of HD space.

    There's no substitute for having IT staff who can actually do troubleshooting.

    In that case a good alerting system would've caught that. I mean, I don't disagree with your final statement, but each tool has a given purpose.

    In the end, my preferred way of doing things would be phoenix servers. :P

    We have an alerting system. It alerts on low HD space. Nobody reads the alerts because of alert fatigue.

    every person who doesn't like an acquired taste always seems to think everyone who likes it is faking it. it should be an official fallacy.

    the "no true scotch man" fallacy.
  • FeralFeral MEMETICHARIZARD interior crocodile alligator ⇔ ǝɹʇɐǝɥʇ ǝᴉʌoɯ ʇǝloɹʌǝɥɔ ɐ ǝʌᴉɹp ᴉRegistered User regular
    When you've chronically understaffed your department for years, including hiring a lot of people at below market rate (which means you get lower than average performers) you end up with a lot of this hole-in-the-bucket technical debt.

    every person who doesn't like an acquired taste always seems to think everyone who likes it is faking it. it should be an official fallacy.

    the "no true scotch man" fallacy.
  • SeidkonaSeidkona Had an upgrade Registered User regular
    edited April 2019
    wunderbar wrote: »
    I'd vote for adding Visual Studio Code to the miscellaneous tools list. It's completely replaced notepad++, and a lot of other editors I used to use.

    I love it too. I actually have a section half written for it with recommended plugins.

    And a cm tools section too!

    Seidkona on
    Mostly just huntin' monsters.
    XBL:Phenyhelm - 3DS:Phenyhelm
  • FeralFeral MEMETICHARIZARD interior crocodile alligator ⇔ ǝɹʇɐǝɥʇ ǝᴉʌoɯ ʇǝloɹʌǝɥɔ ɐ ǝʌᴉɹp ᴉRegistered User regular
    Seriously the dude who set up our alerting system bought it, added every server in the environment (including testing/staging/dev VM) as monitored nodes, and then turned on the default alerts

    Then quit

    every person who doesn't like an acquired taste always seems to think everyone who likes it is faking it. it should be an official fallacy.

    the "no true scotch man" fallacy.
  • SiliconStewSiliconStew Registered User regular
    Feral wrote: »
    Seriously the dude who set up our alerting system bought it, added every server in the environment (including testing/staging/dev VM) as monitored nodes, and then turned on the default alerts

    Then quit

    Man, he must have hated you.

    Just remember that half the people you meet are below average intelligence.
  • SeidkonaSeidkona Had an upgrade Registered User regular
    Mostly just huntin' monsters.
    XBL:Phenyhelm - 3DS:Phenyhelm
  • FeralFeral MEMETICHARIZARD interior crocodile alligator ⇔ ǝɹʇɐǝɥʇ ǝᴉʌoɯ ʇǝloɹʌǝɥɔ ɐ ǝʌᴉɹp ᴉRegistered User regular
    Feral wrote: »
    Seriously the dude who set up our alerting system bought it, added every server in the environment (including testing/staging/dev VM) as monitored nodes, and then turned on the default alerts

    Then quit

    Man, he must have hated you.

    Oh, he was just following this company's culture

    Which is to turn on all the emails on everything everywhere

    every person who doesn't like an acquired taste always seems to think everyone who likes it is faking it. it should be an official fallacy.

    the "no true scotch man" fallacy.
  • That_GuyThat_Guy I don't wanna be that guy Registered User regular
    I mean, if you aren't getting 300 emails a day, what are you even doing?

  • TL DRTL DR Not at all confident in his reflexive opinions of thingsRegistered User regular
    Client crypto infection had me restoring file servers and building new DCs from scratch until nearly 11 PM the other night

    Got a call today from a project manager, who responded to none of my 'emergency update: guys? a little hand here? [#41/41]' emails but who had lots of opinions about how I could have done things more efficiently

    2 days in between fielding related support tickets, exacerbated by things like

    -the client's Group Policy is an absolute cluster. "Oh, someone isn't seeing the Admin drive and you fixed a typo in the Downtown Office Admin Drive Map GPO? Yeah, you need to be looking at New Downtown Office Admin Drive Map."

    -they're in the middle of (read: abandoned) a massive project to untangle their file shares and re-organize everything such that we don't get a ticket from a user saying they deleted a file on their P: drive, then have to spend ten minutes decyphering which P: drive is being referred to

    -their helpdesk guys, who I have routinely bagged on for a number of things, are not the best. One called out sick the day after the infection hit, which had us joking that he caught Crypto Flu

    HOWEVER, one of said helpdesk guys wrote a very thoughtful letter to our management team, singing my praises and that of one of my coworkers who has also been primarily tasked with dealing with this, and it almost makes it all worth it.

  • iTunesIsEviliTunesIsEvil Cornfield? Cornfield.Registered User regular
    Feral wrote: »
    wunderbar wrote: »
    Guys, I've decommissioned two old physical Server 2008 R2 boxes this week and it feels so good.

    And I just had to spin up a new 2008 R1 (SP...2?) x86 instance because "hey, it's better than Server 2003!" <accidentally breaks pen in half>

    2008 is end of life in December so I hope that instance is completely segregated from the Internet

    Oh, you. Of course it's not. Why listen to me and my hysterical warnings when you could save like a couple hundred bucks or so? <breaks another pen>

  • AiouaAioua Ora Occidens Ora OptimaRegistered User regular
    Feral wrote: »
    Feral wrote: »
    Seriously the dude who set up our alerting system bought it, added every server in the environment (including testing/staging/dev VM) as monitored nodes, and then turned on the default alerts

    Then quit

    Man, he must have hated you.

    Oh, he was just following this company's culture

    Which is to turn on all the emails on everything everywhere

    At my work the culture is that alerts cut tickets instead of sending emails.

    It at least forces people to look at them for a second or two.

    life's a game that you're bound to lose / like using a hammer to pound in screws
    fuck up once and you break your thumb / if you're happy at all then you're god damn dumb
    that's right we're on a fucked up cruise / God is dead but at least we have booze
    bad things happen, no one knows why / the sun burns out and everyone dies
  • LD50LD50 Registered User regular
    Feral wrote: »
    wunderbar wrote: »
    Guys, I've decommissioned two old physical Server 2008 R2 boxes this week and it feels so good.

    And I just had to spin up a new 2008 R1 (SP...2?) x86 instance because "hey, it's better than Server 2003!" <accidentally breaks pen in half>

    2008 is end of life in December so I hope that instance is completely segregated from the Internet

    Oh, you. Of course it's not. Why listen to me and my hysterical warnings when you could save like a couple hundred bucks or so? <breaks another pen>

    They're borrowing money from their future selves. I'm sure it will work out fine.

  • That_GuyThat_Guy I don't wanna be that guy Registered User regular
    Feral wrote: »
    wunderbar wrote: »
    Guys, I've decommissioned two old physical Server 2008 R2 boxes this week and it feels so good.

    And I just had to spin up a new 2008 R1 (SP...2?) x86 instance because "hey, it's better than Server 2003!" <accidentally breaks pen in half>

    2008 is end of life in December so I hope that instance is completely segregated from the Internet

    Oh, you. Of course it's not. Why listen to me and my hysterical warnings when you could save like a couple hundred bucks or so? <breaks another pen>

    Easy there, Tiger. Those pens don't grow on trees. That's just going to come out of your server budget.

  • ArcSynArcSyn Registered User regular
    I understand certificates are important for the security of the internet. I also hate dealing with them internally. I just can't wrap my head around getting them installed/configured easily.

    4dm3dwuxq302.png
  • bowenbowen How you doin'? Registered User regular
    ArcSyn wrote: »
    I understand certificates are important for the security of the internet. I also hate dealing with them internally. I just can't wrap my head around getting them installed/configured easily.

    lets encrypt has taken the headache out of it for our stuff, so that's nice

    not a doctor, not a lawyer, examples I use may not be fully researched so don't take out of context plz, don't @ me
  • wunderbarwunderbar What Have I Done? Registered User regular
    certificates are the bane of existence.

    XBL: thewunderbar PSN: thewunderbar NNID: thewunderbar Steam: wunderbar87 Twitter: wunderbar
  • SeidkonaSeidkona Had an upgrade Registered User regular
    edited April 2019
    Fuck this week.

    I am on putting out my third major dire this week and I am just done.

    Currently on a weird issue with a cert update breaking API calls and causing them to be intermittent.

    Seidkona on
    Mostly just huntin' monsters.
    XBL:Phenyhelm - 3DS:Phenyhelm
  • LD50LD50 Registered User regular
    I love getting tickets from end users that look like they might be valid COBOL programs.

  • iTunesIsEviliTunesIsEvil Cornfield? Cornfield.Registered User regular
    bowen wrote: »
    ArcSyn wrote: »
    I understand certificates are important for the security of the internet. I also hate dealing with them internally. I just can't wrap my head around getting them installed/configured easily.

    lets encrypt has taken the headache out of it for our stuff, so that's nice

    Let's Encrypt is awesome!

    Next up: code-signing certs. Though I don't know that they'll help much, since it appears that 1- everyone manages to let their certs get in bad actors' hands, and 2- half the people that do have a c-s cert only sign half their packages.

  • ArcSynArcSyn Registered User regular
    Ok, checking out whatever Let's Encrypt is. Thanks for mentioning it!

    4dm3dwuxq302.png
  • That_GuyThat_Guy I don't wanna be that guy Registered User regular
    Around here we use NameCheap a lot. It's not free but it's still super cheap.

  • mcpmcp Registered User regular
    On Cisco Meraki firewalls, the firmware upgrade function is labeled as beta.

    Of all the fucking things you don't want a beta version of, I'd say that's near the top.

  • a5ehrena5ehren AtlantaRegistered User regular
    edited April 2019
    mcp wrote: »
    On Cisco Meraki firewalls, the firmware upgrade function is labeled as beta.

    Of all the fucking things you don't want a beta version of, I'd say that's near the top.

    That tells me that the devs don't use the user-facing upgrade function for internal dev.

    I actually own the software upgrade subsystem on the thing I work on, and the basics of it are rock-fuckin'-solid because I make everyone use it to do upgrades. I know the workarounds for when shit breaks, but I don't tell people what they are.

    a5ehren on
  • LD50LD50 Registered User regular
    mcp wrote: »
    On Cisco Meraki firewalls, the firmware upgrade function is labeled as beta.

    Of all the fucking things you don't want a beta version of, I'd say that's near the top.

    I imagine a big game show floor with a Cisco exec spinning a giant wheel of fortune where 1/4 of the 'prizes' are "brick $500 of network equipment"

  • bowenbowen How you doin'? Registered User regular
    a5ehren wrote: »
    mcp wrote: »
    On Cisco Meraki firewalls, the firmware upgrade function is labeled as beta.

    Of all the fucking things you don't want a beta version of, I'd say that's near the top.

    That tells me that the devs don't use the user-facing upgrade function for internal dev.

    I actually own the software upgrade subsystem on the thing I work on, and the basics of it are rock-fuckin'-solid because I make everyone use it to do upgrades. I know the workarounds for when shit breaks, but I don't tell people what they are.

    But this is a solved problem that's been in most software for decades at this point, why is it beta?

    not a doctor, not a lawyer, examples I use may not be fully researched so don't take out of context plz, don't @ me
  • LD50LD50 Registered User regular
    bowen wrote: »
    a5ehren wrote: »
    mcp wrote: »
    On Cisco Meraki firewalls, the firmware upgrade function is labeled as beta.

    Of all the fucking things you don't want a beta version of, I'd say that's near the top.

    That tells me that the devs don't use the user-facing upgrade function for internal dev.

    I actually own the software upgrade subsystem on the thing I work on, and the basics of it are rock-fuckin'-solid because I make everyone use it to do upgrades. I know the workarounds for when shit breaks, but I don't tell people what they are.

    But this is a solved problem that's been in most software for decades at this point, why is it beta?

    So they don't have to shell out for replacement hardware when someone uses it and it breaks.

  • TL DRTL DR Not at all confident in his reflexive opinions of thingsRegistered User regular
    4:30 AM, a user calls our emergency after-hours support line and leaves a message saying he can't log in. I'm on call, so my phone explodes - I get out of bed, fire up the laptop, and log into their server, which appears fine. Call the user, who says "Oh, yeah, it started working again like ten minutes after I called you."

    Fuck this sleep-deprived earth.

  • bowenbowen How you doin'? Registered User regular
    that means they typed in their password wrong or didn't have internet

    not a doctor, not a lawyer, examples I use may not be fully researched so don't take out of context plz, don't @ me
  • TL DRTL DR Not at all confident in his reflexive opinions of thingsRegistered User regular
    Yep.

  • SeidkonaSeidkona Had an upgrade Registered User regular
    Second job interview went pretty well.

    Mostly just huntin' monsters.
    XBL:Phenyhelm - 3DS:Phenyhelm
  • SiliconStewSiliconStew Registered User regular
    mcp wrote: »
    On Cisco Meraki firewalls, the firmware upgrade function is labeled as beta.

    Of all the fucking things you don't want a beta version of, I'd say that's near the top.

    I think that's just for the "newer" organization-wide scheduling UI. The old style scheduling is still on the per-network settings pages if you want to use it.

    Just remember that half the people you meet are below average intelligence.
  • ThegreatcowThegreatcow Lord of All Bacons Washington State - It's Wet up here innit? Registered User regular
    TL DR wrote: »
    4:30 AM, a user calls our emergency after-hours support line and leaves a message saying he can't log in. I'm on call, so my phone explodes - I get out of bed, fire up the laptop, and log into their server, which appears fine. Call the user, who says "Oh, yeah, it started working again like ten minutes after I called you."

    Fuck this sleep-deprived earth.

    FFFFFUUUUUUU-This is why I don't miss my old IT position. Shit like this. Nothing will ever raise my blood pressure more than getting a call on my PERSONAL cell phone because the CEO's kid who's interning for the summer called me saturday morning at 7am saying he couldn't attach a powerpoint to an email. And he only got the phone number because I was listed as a backup emergency contact for the on-call rotation. I wasn't even on call that weekend ffs.

  • lwt1973lwt1973 King of Thieves SyndicationRegistered User regular
    bowen wrote: »
    that means they typed in their password wrong or didn't have internet

    Had that yesterday. Was working on a remote user's computer and then I couldn't get in. I asked about if they had internet and they swore they did and they never lose internet so I ran them through a bunch of things trying to get on. I gave up and told them to wait until tomorrow for me to fix it. Two hours later I get a call saying their internet did go out.

    "He's sulking in his tent like Achilles! It's the Iliad?...from Homer?! READ A BOOK!!" -Handy
  • bowenbowen How you doin'? Registered User regular
    I'm pretty sure we all have stories about internet being out when they said it was.

    Or the power being out when they were 100% adamant the computer was up and running.

    not a doctor, not a lawyer, examples I use may not be fully researched so don't take out of context plz, don't @ me
  • twmjrtwmjr Registered User regular
    bowen wrote: »
    I'm pretty sure we all have stories about internet being out when they said it was.

    Or the power being out when they were 100% adamant the computer was up and running.

    "Our lights are on."

    "Yes, but does the *equipment* have power?"

    "Well, our lights are on."

This discussion has been closed.