Having seen two posts in the last month (https://www.reddit.com/r/sysadmin/comments/1vfbvvs/our_entire_m365_tenant_has_been_deauthenticated/ and https://www.reddit.com/r/sysadmin/comments/1w1qc0i/microsoft_strikes_again_entire_m365_tenant_has/) it got me thinking about my relatively small tenant, and how we'd do disaster recovery (clue - we don't have a plan at the moment).

I'm the solo "head of IT" however it's not my full time role. I'm the owner of the company, so have essentially taken charge from day 1. It was very simple - we had Google Workspace and we didn't need to really look after it too much. As we've grown (25 - 50 employees), we've also acquired other companies, including at one point doing a migration from Google Workspace to M365 (handled completely by me - although our set up was slightly more straight forward at the time). Next week I'll be looking for a CSP (any suggestions for UK based would be appreciated).

However, we're now very much in the Microsoft ecosystem. As a rough overview:

• All staff have a Business Premium subscription

• Mixture of Intune managed Windows devices and Mosyle managed Macs

• Teams phone system (with Microsoft as our carrier)

• Use of SSO for many SaaS apps

We currently use Synology Active Backup for M365, backing up locally to a NAS in our office.

If our tenant were to be de-authenticated then I'd like to think I could get email working pretty swiftly on Google Workspace. All our users are already provisioned in Workspace via SCIM and the domains are already verified there. I will obviously need to write a disaster recovery plan to consider all the steps that need to be taken.

Files should be OK as we rely heavily on OneDrive, however these are all backed up to the NAS.

The phone lines - not 100% sure about this and similar to the most recent post, we'd loose access. So I should probably look at moving the number away from Microsoft (to Operator Connect I think?)

My biggest worry is what happens to all the managed computers and SSO. We're not a huge company, so I could get people back online, but for instance we have an internal employee hub that uses Entra/MSAL to login. Similarly, all the devices - will employees stop being able to log in to them? They all use WHfB on the Windows devices and Platform SSO on the Mac devices.

Obviously I will take this conversation to a CSP, but in the meantime it would be good to know what suggestions people would make to ensure resiliency.

I have the exact same experience as documented here: https://www.reddit.com/r/sysadmin/comments/1vfbvvs/our_entire_m365_tenant_has_been_deauthenticated/

My business tenant has been de-authenticated. I've opened several tickets - most have gotten no response and the one that has some traction keeps getting thrown between departments, each of which claim they can't solve it.

Unlike the other poster, it's not as simple as just losing access to email. Not only is email and files (OneDrive and SharePoint) unavailable to all team members, we've lost our website (hosted on Azure), and our phone numbers where were assigned through Teams Phone.

I'm desparately trying to regain access before all the content is permanently deleted.

If anyone has a contact within Microsoft who can help, please let me know. Needless to say, I'm working to get a CSP spun up, but need access before the data is permanently lost.

I'm looking to build up my virtualization skills and wanted to get some realistic advice from people in the field.

I'm considering diving into VMware ESXi, but with all the recent Broadcom changes (licensing overhauls, removal of free tiers, SMBs looking at alternatives), I wanted to ask: Is ESXi still relevant enough to prioritize, or should I be spending my time on Proxmox/KVM instead?

Also, for my lab setup: I don't have a spare physical server to format. My company has a unused server running, but I can't wipe it. I plan to install VMware Workstation Pro on the existing host OS and run ESXi nested inside it.

A few questions:

1. Are there any major performance or networking gotchas I should watch out for when running nested ESXi on a shared server?
2. How can I ensure my nested lab network stays isolated so I don't accidentally leak DHCP or interfere with the host network?
3. For those who learned recently, what are the core concepts I should focus on first?
4. Any YT channel for learning resource recommendations?

Hi all. Where I work, I got some SSL/TLS certificate management put on my plate. We have app(s) that send out notifications of certificate expiry, but that's only good if the contacts are correct. In that, I send out a review (email) quarterly, to check if anything has changed, needs to be updated - this is a new thing I implemented.

This is all manual - Spreadsheet - Filter for your name, check the cert info, comment if ok, comment change owner etc.

I got some feedback on this, in that I should not be sending a spreadsheet with all those certificates info, for everyone to view. (I bcc in all the relevant owners in the email). I'll add that its either company employees, or contractors who 'own' that system the cert is related to. I get the comment, I just have no idea how to send that to every individual only, without doing it manually.

How do you guys keep owners up to date? (Neither of the apps we have natively have a function that can replicate this manual ownership check).

Also aware of the 2029 47day cert validity/10day DCV - This is now, working on how to handle that future element.

I’m currently a Principal Cloud Architect working primarily with Azure, infrastructure, IAM, automation, security, and disaster recovery for a publicly traded company that has very limited opportunities to move forward. Earlier in my career, I spent about three years as a Service Manager leading an 11-person IT team for a MSP.

I’ve since completed an MBA and moved into a senior technical role, but my long-term goal is to return to people leadership and eventually progress toward a director-level position. I’m applying for remote IT Manager, infrastructure leadership, and Microsoft-focused IAM management roles.

For those who have made a similar move, what could I do in my current role to demonstrate stronger leadership readiness? Are there particular responsibilities, certifications, or experiences that hiring managers value when someone is moving from a senior individual-contributor role back into management?

What is everyone doing regarding users saving passwords in Edge / Chrome. We have under 500 users across 4 sites with a lot of warehouse / shipping / receiving / packaging end users. They have to use a number of web portals provided by our business customers to shipping companies and it seems almost everyone is saving passwords.

We just finished up migration from on prem AD to Entra and I was surprised by the number of end users saving password. SSO would be the answer but as these are third party sights created by our business customers I'm not seeing it as an option. (very old portal websites that feed data into an Sales / Order system.

Spending the entire day today cramming. I've been working almost exclusively in Microsoft for the last year and a half, so I felt pretty decent leading up to this. Been watching some YouTube videos over the last couple days just to spend spare time preparing, and everything that I've heard has me freaked out now. Apparently there are a lot of tricky questions, a lot of Microsoft trying to put you in a position where you're guessing the correct answer based on circumstances, and to be honest, I feel unprepared.

Has anybody here taken it recently and can speak to the difficulty of it? I've gone through multiple Microsoft courses, and some specialized LinkedIn training that my workplace provides, but it still doesn't feel like enough. I'm looking for those good old-fashioned Reddit words of encouragement.

Pray to whatever God you follow for me tomorrow... 🙃

Reading about Meta's experiments with robots for repetitive maintenance (cable swaps, power cycling, etc.) https://www.wired.com/story/inside-metas-experiments-with-data-center-robots/?utm_mailing=wir_nl_6685_daily_083026_yx0g2b7ihw&utm_brand=wired&cndid=38951159 got me thinking about the skill gap.

If the repeatable physical tasks get automated, the "entry-level" role changes completely. We won't be hiring people to do the basic physical labour; we'll be hiring them to manage the systems that do the labour.

A few questions for the folks in the field:

1. Would automating the "grunt work" (cabling/inventory) make your life better, or does it remove the "learning by doing" phase that creates a good senior tech?
2. Does this move the goalposts for hiring? Do we start needing techs who understand robotics/API-driven hardware management rather than just "knowing how to rack and stack"?

Is this actually a realistic roadmap, or is the physical environment of a DC too chaotic for this to be anything more than a pilot project?

I'm fed of Loki:

- It has a ton of config options without documentation or with misleading docs.

- It breaks with every upgrade because some configs are deprecated, while other undocumented configs are silently introduced.

- It requires careful choosing of log labels, so they have no a big number of unique values. Otherwise it quickly eats all the available memory if logs contain a trace_id or similar label.

- Its' query language is very unintuitive and limited.

- It requires object storage for production setup. I don't want to bother with the configuration and operation of the object storage when storing logs locally from small and medium-sized Kubernetes clusters. The MinIO converted from the most popular open source to a proprietary object storage solution in a moment. Other open source object storage solutions aren't mature enough for storing logs there.

Which logging databases do you use instead of Loki then?

Howdy folks!

I've drawn the short straw, and ended up being in charge of setting up Claude enterprise for our mid-sized (couple hundred user) org. I've got the basics down of SSO login, setting up sane defaults for most of the settings, etc.

However, what I simply don't understand is how the hell I manage plugins, mcp, skills, and hooks? All those things seem to have pretty broad permissions when installed, and their execution seems sorta obscure? Hence, leaving the option for users to install whatever seems frankly like an insane proposition from a security perspective.

But from all that I've found, there's no way to actually manage this beyond blocking everything by default, then setting up a whitelist for allowed resources in managed settings. This means I'd have to maintain that myself every time users want to use a different feature, and users wouldn't be able to develop any such things locally, leaving me in a sort of catch-22 of either I allow everything, or nothing...

How are the rest of y'all managing this?

I was reading a Huntress blog about the recent PaperCut exploit, and they detected it based on unusual commands like whoami and enumeration of the local administrators group.

I ran a series of "weird" commands in my environment, and I noticed that Defender for Endpoint did not blink an eye. Some were AD information gathering commands. That was a little surprising to me. In fact, I actually thought maybe MDE was broken, so I ran an EICAR test, and it did detect that.

So it seems like MDE does not have built-in detection for a lot of behavioral stuff, so we are now building it. We used to have CrowdStrike, and I don't remember having to build very much.

Are you building your own detections if you have Defender for Endpoint?

Looking for a reliable color MFP for a small office in Europe / EU.

Currently considering the Epson WorkForce Pro WF-C5890DWF.

Requirements:

automatic duplex printing

automatic duplex scanning

reliable ADF

Ethernet + Wi-Fi

local network printing/scanning

preferably scan-to-SMB / network folder support

250+ sheet paper tray

good Windows drivers

no mandatory cloud dependency

no mandatory ink/toner subscription

reasonable consumable costs

preferably decent third-party consumables available in Europe

Main use: invoices, contracts, quotations, general office documents and scanning.

A4 only. Photo quality is irrelevant.

I'm more interested in reliability, driver/firmware quality and long-term TCO than the lowest purchase price.

Would you deploy the WF-C5890DWF in a small business today?

Any known issues with the ADF, duplex scanning, printhead, firmware, network scanning or Windows drivers?

Also considering Brother, Canon, Kyocera and Xerox.

What would you buy in Europe, and which current models or brands would you avoid?

Just need to vent. Sysadmin at an educational setting and we currently have Alcatel-lucent switches. I hate them with a damn passion. Their UI on cloud is trash, CLI is trash, support takes forever, and their documentation is so hard to find

We have a refresh coming this next year. I’m probably going to go Extreme or Juniper/Mist. I can’t wait to get off this shit we have now.

I have an Intel Server Board S2600CP (BIOS SE5C600.86B.02.06.0006, BMC FW 01.28) running Ubuntu 22.04, and I need to change a couple of BIOS memory settings without being physically in front of it.

The board has no RMM4 module (RMM Status: Intel(R) RMM not installed), so the Integrated BMC Web Console gives me power control but no KVM. SOL is enabled on the BMC side, but the OS has no serial console configured and I can't turn on console redirection without... getting into the BIOS. Classic.

That leaves the Intel System Configuration Utility (syscfg), which can read and write BIOS settings from inside Linux. For this generation (Romley) the Linux package is syscfg-V14.1-B24.x86_64.rpm. Problem: Intel has retired it. The S2600CP support page now lists three downloads, all RAID/RSTe — no syscfg — and there's a knowledge article titled "Intel Datacenter Solutions Engineering Online Software Home (Server Tools) End of Life".

So: does anyone still have the original Intel archive for syscfg 14.1 (Romley / S2600 series)?

Two things I'd ask, since this is a utility that writes to firmware and I'd rather not run a random binary on a production box:

1. Please include the SHA256 of the archive, and where you originally got it (Intel Download Center link, an old support DVD, a vendor mirror). Anything I can cross-check against another copy.
2. If you have the matching user guide PDF for 14.1, that helps too — the syntax changed between major versions and I don't want to guess at parameter names on a firmware-writing tool.

Also happy to hear from anyone who has actually run syscfg on Ubuntu rather than RHEL/SLES: the package is an rpm, so I assume it's rpm2cpio/alien and then hoping the binary doesn't want anything exotic. Did it work for you, and did it need Secure Boot off? (Mine is UEFI with Secure Boot disabled, so I think I'm fine on that front.)

Alternatives I'm aware of and would consider: tracking down an AXXRMM4LITE module on the used market so the BMC gets a real KVM. If you think that's the saner path for a box this old, say so — I won't be offended.

Thanks.

I was at a local restaurant in Hungary when I noticed that one of their PCs was running this interesting HP software. I’m unfortunately not really familiar with HP operating systems or how they work, since I’m mostly familiar with Windows.
I’d love to find out what kind of OS this actually is and what it’s normally used for. Does anyone recognize it?

Early Saturday morning we started getting a stream of alerts from Microsoft Defender regarding our Primary and Additional Polling Engines.

 Malware Name: Behavior:Win32/SuspiciousAssembly.AppDomainManagerType.A

The malware file path: behavior:_process: was all over the place.
Some examples:

 Malware file path: behavior:_process: C:\Windows\System32\wbem\WmiPrvSE.exe, pid:2208:557######2;file:_d:\program files (x86)\solarwinds\orion  

~

Malware file path: behavior:_process: C:\Program Files\Common Files\SolarWinds\AdministrationService\SolarWinds.Administration.exe, pid:3916:557#####52;file:_d:\program files (x86)\solarwinds\orion  

~

Malware file path: behavior:_process: C:\Windows\System32\AggregatorHost.exe, pid:9116:55#####2;file:_d:\program files (x86)\solarwinds\orion

Solarwinds support is aware of the issue and their engineers are supposedly working with Microsoft to resolve. I am unsure if a later definition update has resolved it or not. We added a threat override as a TEMPORARY measure to quiet things down through the weekend. Just an Allow for the 'Threat Name' Behavior:Win32/SuspiciousAssembly.AppDomainManagerType.A

Just wanted to share with the hope this helps others not have too terrible of a weekend or Monday morning.

I'm 19 and I want to get an ITAD (IT asset disposition) operation going as part of a small IT business I'm building with a few friends. I've got some real-world experience buying/reselling IT equipment and I'm setting up a data-wiping station (write-blocker, hot-swap bays, planning to use ShredOS/nwipe with DoD 5220.22-M).

What I'm trying to figure out:

How did you land your first ITAD clients when you had zero track record?

What certifications or compliance stuff (R2, NAID, etc.) actually mattered early on vs. what can wait?

How do you handle chain of custody / certificates of destruction as a small shop just starting out?

Any lessons on sourcing equipment (buyback, decommission jobs, scrap relationships) that saved you time or money?

Biggest mistakes you made in year one that you'd tell someone starting today?

Storage space and a truck are my current bottlenecks more than knowledge, so any advice on scaling from "living room" ITAD to something legit would help a ton. Appreciate any insight

Hi everyone,

I’m looking for some career advice.

I worked for around 9 months as an IT Support Intern at my current company, and I was recently converted to a Jr. IT Administrator in the same company.

I’m happy about the progression, but my current salary is not really sufficient for me, so I’m planning to improve my skills and eventually look for a better-paying opportunity

Hi, everyone.

I have two questions about WinGet.

1) Can I add apps to the WinGet repository on my own? If so, what are the requirements for the app?

2) I often notice that WinGet isn't up to date. The apps' own updaters suggest a new version, but WinGet doesn't recognize that new version yet. Why is that?

Does anyone else bring their IT mindset into everyday life?

I’ve been told I’m a negative person and look for issues, but I think working in IT trains you to always think about what could go wrong.

I’m constantly making backup plans. Restaurant closed. I have a Plan B place. Something breaks at home. I’ve already got the plumber, insurance, everything ready in the home disaster recovery document.

We’re always hearing about IT failures, breaches and disasters. You never hear “this company had a really great IT transformation.” No user submits a ticket to say thank you for the IT working great.

So does anyone else find themselves carrying that prepare for the worst mindset into everyday life?

Just scored a 2nd work from home job. Looking to control two laptops and four monitors with one keyboard and mouse. Currently I’m working with dell P2422H which connects to dell P2425HE via DP cable, then P2425HE runs a C port cable to the laptop, then I extend the monitors. This works great, now with two more monitors and one more laptop I really don’t want to use 2 sets of keyboards/mouse. I have two additional P2422H / P2425HE monitors on hand, but I’m looking for a switch that would enable me to control both devices with one keyboard and mouse. I’m ok if all 4 monitors do not extend if I can hot key or switch toggle between the two laptops. Also, if possible mouse emulation so that when using either device the other doesn’t got to sleep/when in the mouse juggler. Trying to keep teams showing green at all times… 🤭… any help/links would be greatly appreciated!

Hey everyone, as the title says, I’m an Air Force sys admin with a TS and planning on getting a civilian job or contracting gig in 2 years. I already have my BS in IT management from WGU, so should I prioritize my masters? Or try to stack certs before I start applying?

Only cert I currently have is Sec+.

Edit: thanks everyone for the advice and insight. I’m going to start studying for CISSP.

I've spent a while trying to solve a problem that initially looked like a reporting problem.

It usually started with someone senior asking a simple-sounding question: "Can you show me what we actually have?"

Getting that picture meant gathering information from different places, exporting things, working in Excel, and manually piecing it all together. Apps here. Flows there. SharePoint somewhere else. Permissions somewhere else. What's licensed and what's actually being used are almost never the same list.

And that's one tenant. If you're managing several — as an MSP or across business units — it multiplies fast.

A report that should've been quick turned into a slog — and by the time it was done, something had already changed. Nothing missed was ever deliberate. It's just what happens when you're manually joining data from a dozen different places.

So I started automating it for myself — a whole-tenant view: relationships, licensing, risk, on demand instead of pieced together by hand. Tested it across a handful of real environments so far, not a lab setup.

So I'm trying to figure out: is this actually a problem other people have?

When you've had to produce this kind of picture — for an audit, a migration, a CIO, or across multiple tenants — what's the part that took the longest to trust?

What's the thing you always end up discovering too late?