VMWARE Workspace One, iPhone, iOS 17, Enterprise WIPI after Update. Issue known since 07.07.2023 to 27.09.2023 and still affects endusers

Well; looks like some things change too fast in the modern world workplace.

We wish to inform you of a current concern pertaining to Workspace ONE UEM-enrolled iOS devices that have undergone an upgrade to iOS 17. It has come to our attention that these devices are occasionally being incorrectly identified as compromised following the upgrade. The VMware development team is actively engaged in addressing this matter.

Symptoms:

Upon the upgrade to iOS 17, Workspace ONE UEM-enrolled iOS, mobile devices are intermittently detected as compromised by the Workspace ONE mobile SDK, leading to the execution of an enterprise wipe. This process results in the removal of all enterprise data from these devices and their disconnection from the system.

Resolution:

Our product team is diligently working towards resolving this issue. For devices operating on iOS versions lower than iOS 17, the fix is anticipated to be automatically applied and does not necessitate an SDK/Productivity Apps/UEM upgrade. We estimate that the issue will be resolved by the 29th of September 2023.

Possible Workaround:

To mitigate this problem, re-enrolling the affected devices should provide a solution. As a temporary measure, administrators can disable the 'Compromise Protection' in the SDK profiles within the UEM console. Please be aware that Compromise Protection is a vital security feature and should be re-enabled once the Workspace ONE team issues the fix for this issue.

Note: Enterprise Wipe and/or Device Wipe commands do not land on the device unless the device is unlocked or connected to a network.

Often, it is recommended to use enterprise wipe rather than deleting the device. When the Delete Device command has been completely processed, all commands associated with that DeviceID are purged from the AirWatch DB, including the Enterprise Wipe Command tied to that device's DeviceID.

Therefore, if the Delete Device operation is performed on a device that is turned off/disconnected from the network, it will not receive the Enterprise Wipe Command issued by the console immediately upon the Delete Device command. This is why it is recommended to make use of the Enterprise Wipe Command as this preserves the device records associated with this device's DeviceID.

https://kb.vmware.com/s/article/92815

https://support.apple.com/en-us/HT213781

https://kb.vmware.com/s/article/50103263

Azure Application Proxy (Die Eierlegende kostenlose Woll-Milch-Sau um on-premises Server extern erreichbar zu machen)

Es gibt die Kafeemaschine für CH 29.- mit 30 gratis Kapseln, aber eine weitere Kapsel für CHF 3.-.

Es gibt einen Foto Tintenstrahldrucker fuer CHF 19.- aber die Patronen kosten CHF 99.- pro Farbe.

Genau davon reden wir hier nur merkt es keiner.

Was hört man an M365/Azure Schulungen Schönes?

Wieso macht ihr alles so kompliziert? Publiziert doch Eure in-house Server ganz einfach und schnell über einen Azure Application Proxy? Dann braucht ihr all die teure Hardware nicht mehr. Es ist kostenlos bei den meisten MS licence Suites dabei. Zwei, drei Klicks und der interne Intranet-Server steht sicher im Web. Keine DMZ Vmware Cluster, kein DMZ Veeam Server, Keine Segmentation FW keine teuren RSA FMS Lizenzen bei uns ist alles dabei!

Zudem nach dem Fortinet Fortigate SSL VPN Exploit im Juli 2023 erkundigen sich einige wieder, was andere wie machen. Jahrelange sichere und stabile Konzepte sowie Strategien werden plötzlich in Frage gestellt. Dann gab es noch die Citrix Access Gateway Lücken im November 2022 und Juli 2023. Der einte oder andere fängt an zu Grübeln. Das gab es alles schon früher nur redete keiner davon und meist wussten auch niemand davon.

Das liest man über Azure Application Proxy:

• Einfach zu bedienen. Benutzer können auf die On-Premises-Anwendungen auf die gleiche Weise zugreifen, wie sie Microsoft 365 und andere SaaS-Anwendungen, die mit Azure AD integriert sind, nutzen.

• Sicher. On-Premises-Anwendungen können Azure's Autorisierungssteuerungen und Sicherheitsanalysen nutzen, einschließlich bedingtem Zugriff und MFA (Multi-Faktor-Authentifizierung). Außerdem erfordert Application Proxy nicht, dass Sie eingehende Verbindungen über Ihre Firewall oder VPN öffnen.

• Kosteneffizient. Um Application Proxy zu nutzen, müssen Sie die Netzwerkinfrastruktur nicht ändern oder zusätzliche Geräte in Ihrer On-Premises-Umgebung installieren.

Man muss nur folgendes beachten:

Best Practices für Azure Application Proxy:

1. Installieren Sie den Connector-Server physisch in der Nähe der Anwendungsserver, um die Leistung zwischen dem Connector und der Anwendung zu optimieren.

2. Der Connector-Server und die Webanwendungsserver sollten sich im selben Active Directory-Domäne befinden oder sich über vertrauenswürdige Domänen erstrecken.

3. Installieren Sie mehrere Connector-Server lokal, um einen einzelnen Ausfallpunkt zu vermeiden.

4. Konfigurieren Sie FQDN (vollqualifizierter Domänenname) und DNS-Einträge intern und verwenden Sie keine IP-Adressen.

5. Stellen Sie sicher, dass Ihre Connector-Server uneingeschränkten Internetzugang haben und das Microsoft-Netzwerk erreichen können.

6. Verwenden Sie bedingten Zugriff, um den Zugriff auf Ihre Umgebung abzusichern.

7. Konfigurieren Sie SSO (Single Sign-On), um die Benutzererfahrung zu verbessern.

MS Links:

https://learn.microsoft.com/en-us/azure/active-directory/app-proxy/application-proxy-network-topology

https://github.com/MicrosoftDocs/azure-docs/blob/main/articles/active-directory/app-proxy/application-proxy-page-load-speed-problem.md

Alles Cloudy? Alles Ersetzen?

 Was ich persönlich davon halte:

• Für gelegentlichen Zugriff auf KMU-On-Premises-Server, die ich ins Web stellen möchte, ohne komplexe DMZ-Architektur intern: OK Jederzeit (Ticket, Zugang zur Abacus Zeiterfassung, Service Portal, Intranet, irgendetwas Kleines: Super)
• Für Unternehmen mit über 1'000+ MA, die Hauptanwendungen nutzen, landen wir dort, wo grosse Unternehmen seit Jahren mit WAN zwischen Standorten zu kämpfen haben: LATENZ / SESSION TTL Einstellungen usw. In der Regel setzte man dann im Enterprise Bereich WAN-Optimizer ein auf beiden Seiten welche jeweils den Preis eines Autos hatten.

  https://www.gartner.com/reviews/market/wan-optimization (Fortinet/Cisco/Riverbed...)

  https://www.inside-it.ch/vector-capital-kauft-riverbed-20230713

   

• Azure Application Proxy > JA, man kann mehrere Connector lokal installieren und sie bieten internes Load Balancing. Failover funktioniert, aber wenn einer ausfällt und man ihn nicht abmeldet, dann erhöht sich die Latenz++, z.B. bei RDP weil die Logik dort immer wieder versucht zu connecten. (Man lese im Web von verzweifelten Endkunden welche Citrix Accessgateway ersetzen wollten un den Effekt hatten [Wir bleiben fair: Letzte grössere Lücke Citrix AG NOV 2022 und JUL 2023])
• Das Erste, was man dann bei verschiedenen Softwareherstellern zu diesem Thema findet, ist, dass man die Timeout-Zeit erhöhen muss (Appliance, Windows Server oder in der App, sofern möglich). Das ist denen noch so recht wenn es eine kostenlose Option gibt um Ihre Lösung WAN tauglich zu machen. Nur der neueste Release ist dann auch für das optimiert?
• Ist doch nichts kostenlos, oder? Ist doch keiner so naiv und glaubt, dass es bei Microsoft etwas umsonst gibt, ausser Systernals von Mark? Das Ziel ist, dass man alles zu Microsoft bringt und dann feststellt, dass es nicht kostenlos bleibt und Latenz hat? Aber dafür gibt es eine weitere Lösung und die heissst Express Route Direct
• Latenz? Kein Problem, sagen die Microsoft-Cloud-Sales-Evangelisten. Dafür gibt es ja den allseits bekannten Express Route Direct, für den man bis zu CHF 44'867.-/MONAT zahlen kann. Das ist ein "goldenes" Kabel zu den MS Rechenzentren mit SLA.

  https://azure.microsoft.com/en-us/pricing/details/expressroute/#pricing

• Für KMU hoffen wir, dass es besser wird, wenn man es CH > CH (Standort Cloud Schweiz) routet? Für Grosskunden mit mehreren internationalen Standorten bleibt dann nur eine Lösung. MS wird nicht blöde sein und sich das Express Route Milliarden-Business kaputt machen. (Peering Traffic muss irgendjemand zahlen, lernt man bei den ISPs?)
• Aus über 25 Jahren Vergangenheit: Nicht davon ausgehen, dass jeder On-Premises-APP-Server für WAN-Zugriff gebaut ist. (Timeout/Session)

Preise Express Route, Switzerland, 13.07.2023

https://azure.microsoft.com/en-us/pricing/details/expressroute/#pricing

Das meinen andere dazu. Wunderbarer Kommentar von Michael Loftus aus den Blog eines MS Mitarbeiters der uns erzählt wir brauchen kein VPN mehr.

Source Linkedin 12.07.2023

Eine Schnellsuche zum Thema Azure Application Proxy und Probleme bringt folgendes:

https://github.com/MicrosoftDocs/azure-docs/blob/main/articles/active-directory/app-proxy/application-proxy-page-load-speed-problem.md

https://github.com/MicrosoftDocs/azure-docs/blob/main/articles/active-directory/app-proxy/application-proxy-network-topology.md

Jemand intergiert in Enterprise Umgebung, probiert, Produktiv und am Schluss eine Frage im neuen FAQ/Learning bei MS.

Oder war es der Endkunde selber im M365 Admin Portal und die IT-Service Firma kann dann helfen?

Red Teams use Azure Application Proxy to publish their C2 Server

Nice writeup by Adam Chester. So we can asume that it's "Safe" when Hacker use it. Right? Sudan agrees?

https://blog.xpnsec.com/azure-application-proxy-c2/

Why do they mention ExpressRoute so many times?

Screenshot Auszug MS KB:

https://twitter.com/MSFT365Status/status

https://connectivity.office.com/status

Microsoft M36X has been scaling up its information system to address downtime, which is unsurprising considering the recent outages experienced in the past months.

Checking the Twitter Status M365 (https://twitter.com/MSFT365Status/status) in the morning is a good way to anticipate what to expect before opening your IT Support Ticket system. This allows you to make informed decisions, such as staying at home or securing the IT support door. You may also think in long term about studying art or other jobs beyond relying solely on the Cloud-O-Matic-Manager (COMM) duty. Maybe you find time to check SLA you have with your customers or with the company who runs your cloud.

Microsoft offers another tool that traces VPN-traffic or aM365 access worldwide, which can be used to identify existing cloud customers. Therefore, the next time your neighbour displays unusual Behavior, like throwing his BYOD laptop out of the window or being excessively loud in their home office, you can interactively investigate the reason using this tool

https://connectivity.office.com/status

So this enables you too see high traffic points. But there are several green points at private only location. Only private person rentals block as example.

You can also utilize a map provided by Microsoft, highlighting locations where a high volume of traffic is received from customers connected to Microsoft Teams, Exchange, SharePoint, and other services. This map serves as a useful resource to crosscheck whether an address corresponds to a private residence or a company complex. Furthermore, if a specific point on the map exhibits a large data volume, you can assume that the person or company:

1. Has implemented a split/non-split (FULL) VPN policy and may be downloading movies/warez from onedrive? He does not understand that everything he does private on his business laptop or his BYOD-enterprise-integrated-somehow may route all traffic to his employer (firm) and in this case straight through Microsoft Services somehow? On-premise you you took him down or took away his laptop toy. In the cloud you expect a really fat Express Route bill at the end of the month. You can even realtime see the Cloud cost in CLI/PS Microsoft told this week. That is a real neat tool you can see your Azure cost exploding in real time for your developers who profit from scalability as they always said.

b) Is using plain Microsoft Windows Defender ATP and lacks robust commercial enterprise endpoint protection such as McAfee/Trellix. In such cases, there is a possibility that the endpoint is currently infected and vulnerable due to the high traffic flowing through M365 services. While this level of paranoia might not be common, it's essential to ensure proper endpoint protection.

In the past, on-premise enterprise IT teams effectively managed similar challenges with great success. These aspects can be effectively addressed in on-premise enterprise IT environments, especially when supported by dedicated IT teams.

That guy at Ochsengartenweg in Switzerland, i know him. I am sure he does not know we can see when he works or not? Please note that respecting privacy and confidentiality is crucial when using tools and maps to gather information about individuals or companies.

Microsoft 365 network health status

This map shows the network health status of locations where Microsoft receives a high volume of traffic from customers connected to Microsoft Teams, Exchange, SharePoint, and other services.

A degraded or unhealthy status can be caused by various issues across the entire customer connection path, including issues with Microsoft’s network and availability, the customer’s network infrastructure, internet service provider availability, and third-party intermediary devices. Go to Service health in the Microsoft 365 admin center for more details on any issues we’ve detected, including the scope of impact.

Region Basel

Not too many M365 users where "Borat" lives? Does BORAT use Google or TUX? Does he have an Amiga?

Some actual alerts these days: 

Screenshot MS customer Portal 28.06.2023

22-28.06 from Twitter Microsoft 365 Status

VPN differences:

Full VPN, as opposed to split VPN, can often be a challenge when it comes to bandwidth. Here's why:

1. Routing all traffic through the VPN: With a full VPN configuration, all network traffic from the user's device is typically routed through the VPN tunnel. This means that even non-business-related internet traffic, such as browsing social media or streaming videos, will be redirected through the VPN server. As a result, it increases the overall bandwidth consumption.

2. Increased latency: The additional distance that data has to travel through the VPN server before reaching its destination can introduce latency or delay. This latency can impact real-time applications, such as video conferencing or online gaming, where responsiveness is crucial. The increased round-trip time can affect the user experience and degrade the overall performance.

3. VPN encryption overhead: VPNs use encryption to secure the data transmitted through the tunnel, which adds an additional layer of processing. This encryption process can increase the size of data packets, leading to larger bandwidth usage. The overhead caused by encryption can become more pronounced when dealing with high-volume data transfers or bandwidth-intensive activities.

4. Bottlenecks at the VPN server: Full VPN setups often involve funneling all traffic through a central VPN server or a limited number of servers. This concentration of traffic can lead to congestion and potential bottlenecks at the server level. As more users connect to the VPN and utilize the available bandwidth, it can impact the performance for all users connected to that server.

5. Limited VPN server capacity: VPN servers have finite resources, including processing power, memory, and network bandwidth. When multiple users connect to a VPN server simultaneously, especially during peak usage periods, the server's capacity may be strained. This can result in reduced bandwidth availability for each user, affecting overall performance.

It's important to note that while full VPNs can present challenges with bandwidth, they also provide enhanced security and privacy by encrypting all internet traffic. Organizations and individuals should consider the trade-offs between security and bandwidth requirements when choosing between full VPN and split VPN configurations.

PROBLEM: The M365 (cloud) users are unable to access free/busy info from the on-premise room mailbox and part two are unable to book it

In hybrid environment that combine on-premises and Microsoft 365 (M365) accounts, organizations may encounter difficulties when attempting to add conference rooms to meetings. This issue affects not only Outlook Web App (OWA) but also the Outlook desktop client (outlook.exe).

Problem Overview

The problem arises when attempting to book conference rooms from the room list in Outlook. In both OWA and Outlook desktop client, users may find that conference rooms are missing or unavailable for selection during the meeting scheduling process. This issue is particularly prevalent in hybrid environments where room mailboxes are synchronized between on-premises Exchange Server and M365.

Root Cause

The root cause of this problem lies in the configuration of the RoomList distribution group, which is responsible for organizing and displaying room mailboxes. In hybrid environments, the RoomList distribution group needs to be properly synchronized and maintained between on-premises and M365.

Inaccurate recipient TypeDetails

One potential cause is the incorrect recipientTypeDetails property associated with room mailboxes during synchronization. When on-premises room mailboxes are synchronized to M365, the recipientTypeDetails property may be set as "MailUser" instead of the expected value, "RoomMailbox". This discrepancy prevents the room mailboxes from being recognized as valid conference rooms in M365.

What is wrong again in detail:

In a Full hybrid Exchange <> M365 Setup with Room resources on on-premise side you see following effect.

Error #1:

The test.M365 users are unable to access free/busy info from the on-premise room mailbox as example Audi_A3_OST_179602@butschlab.ch.

Error#2.

The test.M365 users can see the calender and free/busy but they are unable to BOOK the room effective (No permission changed and users was in on-premise side before migration)

We have following users:

This is also valid for:

In a hybrid deployment of on-premises Microsoft Exchange Server and Exchange Online, assume that Exchange Online users select add room in a meeting request in Outlook Web App. In this situation, on-premises room mailboxes aren't displayed in the list as expected. Therefore, Exchange Online users can't add an on-premises conference room to a meeting request.

This is kind of unsupported solution as some people say at MS in Forum. On the other hand there is a KB which clearly explains part of the solution.

But only parts of it? Here is the full working solution:

Error #1:

The M365 users are unable to access free/busy info from the on-premise room mailbox.

1. You have to generate a ROOMLIST or lets say NEW Distribution List with the option –roomlist
2. In that room list you make all existing on-premise rooms member (Their object)
3. You already have OR newly generate a ROOM on the M365 Side m365.room@butschlab.ch (Make a DUMMY room on Cloud side)
4. You make the new m365.room ROOM also member of that Room List (Distribution group) < Most important STEP

Error#2.

The M365 users can see the calender and free/busy but they are unable to BOOK the room effective (No permission changed and users was in on-premise side before migration)

Here is how to see which rooms are affected:

Get-Mailbox -RecipientTypeDetails RoomMailbox | Get-CalendarProcessing | ?{!$_.ProcessExternalMeetingMessages}

You have to enable the flag -ProcessExternalMeetingMessages $true for the on-premise accounts. We never touched this in on-premise ONLY because it was rather dangerous if you did it wrong. If you did not follow compliance and enabled some things like INLCUDE/SHOW subject or BODY from meetings and ACTIVATE that wrong external people could you see conf. Data from inide the org.

GUI, notive that the user accounts of ROOM resources are always disabled. Keep them seperated so nobody (Azure Admin)

Does delete them when he has his cleaning day and want's to be a good employee.

Since you are in Hybrid Mode also check shortly both sides on-premise and M365 if you have the same settings there:

Set-OrganizationRelationship | fl

The KB was written for the OWA / WEAPP but we proof it's also valid for outlook.exe

Please notice that the KB does NOT contain the full solution as we provide in our Butsch KB.

https://learn.microsoft.com/en-US/exchange/troubleshoot/calendars/cannot-add-conference-rooms-to-meeting-in-owa#more-information

Maybe it's the TOKENS and AUTHENTICATION also so Check ANYTHING related to 4001 if you have OAUTH oder DAUTH:

http://www.butsch.ch/post/M365Exchange-Hybrid-OAuth-Testing-command-OAuth-Cert-out-of-sync-4001-IIS-VDIR-OAuth-wrong

EventID which may be related if you have OAUTH active

Event ID 1003: This event indicates successful OAuth authentication requests. It confirms that the authentication process between Microsoft 365 and on-premises Exchange was successful.

Event ID 1004: This event indicates failed OAuth authentication requests. It can provide details about the reason for the authentication failure, such as incorrect credentials or authorization issues.

Event ID 1022: This event is logged when an OAuth token is issued to a client application. It confirms that the token was successfully issued for the authentication process.

Please note that these specific event IDs are not guaranteed to be available in all versions of Exchange Server or in every configuration. The availability of events and their IDs can vary based on your Exchange Server version and the specific logging settings enabled.

Additionally, it's important to understand that the Application log on the Exchange server may not provide comprehensive logging specifically for OAuth access from Microsoft 365 to on-premises Exchange. As mentioned earlier, monitoring the client-side logs in Outlook and the logs in Microsoft 365 (such as audit logs and trace logs) will provide more detailed information for troubleshooting OAuth-related issues.

If you encounter specific error messages or need more specific guidance on troubleshooting OAuth access, I recommend referring to Microsoft's official documentation, reaching out to Microsoft Support, or consulting with an Exchange Server specialist for further assistance.

www.butsch.ch

Resolve and find OAuth problem in Exchange Hybrid Setup Enviroments

Short Understanding OAuth:

OAuth (Open Authorization) is an industry-standard protocol that enables secure authorization for third-party applications without the need to disclose user credentials. It allows users to grant limited access to their resources on one site to another site, without sharing their credentials. In the context of Exchange and Hybrid setups, OAuth provides a streamlined and secure way for users to access their emails, calendars, and other resources.

https://datatracker.ietf.org/doc/html/rfc6749

https://learn.microsoft.com/en-us/azure/active-directory/fundamentals/auth-OAuth2

Since end 2022 every M365 TENANT has by default cloud side OAuth enabled. There is DAUTH (Older MS own made) and there is

an OPEN Source Authentication method called OAuth.

https://learn.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/deprecation-of-basic-authentication-exchange-online

You may land here if:

a) You just want to test if OAuth works on both sides between Exchange<>M365

b) Or your IIS Virtualdirectory do not accept OAuth beside NTLM etc.

c) If you have done a Self Signed OAuth Cert renew (The one with 2H+ UTC/CET Local time offset delay bug)

We explain how to fix and how to check if both sides have the latest certificate UTC time corrected.

So this KB is on how to TEST OAuth Exchange onpremise<>m365 but also why you came here because maybe it does not work and reports FAILURE not SUCCESS.

First let us check if your Microsoft 365 side has OAUTH enabled?

M365 side: Get-OrganizationConfig | FL oau*

Is Oauth working incoming and outgoing?

Following are detailed steps on how to test OAuth from from both sides in your Exchange <> M365 Hybrid Setup.

We have following users:

Short without any help or explanation:

Test-OAuthConnectivity -Service EWS -TargetUri https://outlook.office365.com/ews/exchange.asmx -Mailbox test.onpremise@schweiz.ch -Verbose | fl

Test-OAuthConnectivity -Service AutoD -TargetUri https://autodiscover-s.outlook.com/autodiscover/autodiscover.svc -Mailbox test.onpremise@schweiz.ch -Verbose | fl

Test-OAuthConnectivity -Service EWS -TargetUri https://outlook.schweiz.ch/ews/metadata/json/1 -Mailbox test.m365@schweiz.ch -Verbose | fl

Test-OAuthConnectivity -Service AutoD -TargetUri https://autodiscover.schweiz.ch/autodiscover/autodiscover.svc -Mailbox test.m365@schweiz.ch -Verbose | fl

Reference for command Test-OAuthConnectivity:

https://learn.microsoft.com/en-us/powershell/module/exchange/test-OAuthconnectivity?view=exchange-ps

Default value for all TEST outgoing to M365: https://outlook.office365.com/ews/exchange.asmx

Default value for all TEST outgoing to M365: https://autodiscover-s.outlook.com/autodiscover/autodiscover.svc

Test on On-premise > M365 Exchange outgoing direction:

If succesfull:

Test M365 > on-premise incoming direction:

Yellow change to what you have in Exchange Virtual Directory:

Retrieve those from your on-premise:

On-premise: Get-WebServicesVirtualDirectory | fl Extern*

Example: https://outlook.schweiz.ch

On-premise: Get-ClientAccessServer | fl autodis*

Example: https://autodiscover.schweiz.ch

Test https://autodiscover.schweiz.ch from external:

Open UP in a web browser from EXTERNAL (Mobily other network) and make sure you can LOGON to the Autodiscover URL with the on-premise account.

https://autodiscover.schweiz.ch/autodiscover/autodiscover.svc

Test on M365 side:

If succesfull:

Interpreting the Test Results:

After executing the Test-OAuthConnectivity command for both local and Microsoft 365 users, you will receive the test results. Here are a few possible outcomes:

Success:

If the test is successful, it means that the OAuth connectivity for the specified user is functioning correctly, and they can access Exchange resources seamlessly.

Failure:

If the test fails, it indicates an issue with the OAuth connectivity. Check for any misconfigurations in the OAuth configuration, network connectivity problems, or any other relevant errors. Troubleshooting these issues will be crucial to ensure uninterrupted access for the user.

Failure may be because of......

Failure:

CHECK1: OAuth not Set on VirtualDirectory EWS and AUTODISCOVER on Exchange on-premise

When troubleshooting OAuth-related issues, the first step is to ensure that OAuth is correctly set on the VirtualDirectory configurations for EWS (Exchange Web Services) and Autodiscover in the on-premises Exchange environment. In some cases, OAuth might not be properly configured or enabled on these VirtualDirectories, resulting in authentication failures and connectivity issues.

To address this, it is essential to verify and configure OAuth settings correctly on the EWS and Autodiscover VirtualDirectories, ensuring that the necessary authentication methods are enabled and OAuth is correctly configured.

Check VirtualDirectory on-premise:

Does it show OAuth right side?

Or yourself:

Get-AutodiscoverVirtualDirectory | fl *auth*

Get-WebServicesVirtualDirectory | fl *auth*

Failure:

CHECK2: OAuth Set, but does not PULL > FLIP-FLOP effect IIS VirtualDirectory Settings for EWS and AUTODISCOVER on Exchange on-premise

IF FAILURE: FLIP-FLOP the IIS Virtualdirectory Authentication Settings to be fireproof.

Even if ALL seems fine sometimes their is a FLIP FLOP effect which evn MS can't explain. So turn it OFF and TURN it ON.

Failure:

CHECK3: You have renewed the OAuth Certificate on the Exchange on-premise but

FAILURE: You did not do the last step in the KB on how to renew OAuth cert, Event 4001 Exchange on-premise

If you have recently renewed the OAuth certificate on your Exchange on-premises server but encounter authentication failures or other issues afterward, it is crucial to ensure that you have completed the final step mentioned in the corresponding knowledge base (KB) article.

Renewing the OAuth certificate involves multiple steps, and failure to follow the complete process can result in incomplete certificate installation or incorrect configuration. By carefully reviewing the KB article and verifying that all required steps, including post-renewal configuration, have been executed, you can avoid common pitfalls and ensure a successful OAuth certificate renewal.

https://learn.microsoft.com/en-us/exchange/troubleshoot/administration/cannot-access-owa-or-ecp-if-OAuth-expired

Failure:

Check IF Azure HAS your RE-done(latest) OAuth Exchange Certificate active

Azure side:

Connect to Azure from the machine where have acccess to Azure or the Exchange with the HCW Wizard.

Connect-MsolService (Logon with your Tenant or Admin account)

Get-MsolServicePrincipalCredential -AppPrincipalId 00000002-0000-0ff1-ce00-000000000000 -ReturnKeyValues $true

Note the StartDate and Enddate from the latest you made.

On-premise:

Get-ExchangeCertificate (Get-AuthConfig).CurrentCertificateThumbprint |fl Not*

Or CHECK below Script to get the time in UTC direct on-premise

 Attention Local Time Zone and Azure Time Zone are different. Remember when you did renew the OAuth cert

It took 2 or 4 hours to get active (OWA and PS was not wokting if so...)

Now hopefully, when doing the renew of the OAuth cert onpremise, you did not start 10 times.

AZURE = UTC

YOU = Local Time Zone +/- 2-4hrs or whatever

Use below script to get the time as it looks with azure

Re-check all as explained in:

https://learn.microsoft.com/en-us/exchange/troubleshoot/administration/cannot-access-owa-or-ecp-if-OAuth-expired

Failure:

FAILURE, 401 Access Denied: You don't not have all internal URL from your Exchange listed with AZURE as SPN

(He will accept only those address which the Cloud has SPN listed to easy explain)

https://learn.microsoft.com/en-us/exchange/troubleshoot/administration/401-access-denied-error-when-running-test-OAuthconnectivity

On-premise: Check the OnPremisesDiscoveryEndPoint and OnPremisesWebServiceEndPoint from:

Get-IntraOrganizationConfiguration | Fl

On-premise: Should be the same as per:

Get-WebServicesVirtualDirectory | FL ExternalUrl

Check Azure side:

Connect to Azure from the machine where have acccess to Azure or the Exchange with the HCW Wizard.

Connect-MsolService (Logon with your Tenant or Admin account)

(Get-MsolServicePrincipal -ServicePrincipalName "00000002-0000-0ff1-ce00-000000000000").ServicePrincipalNames

Check that ALL URL which you have on-premise are listed also there.

https://outlook.schweiz.ch

https://autodiscover.schweiz.ch

THUS

00000002-0000-0ff1-ce00-000000000000/outlook.schweiz.ch

00000002-0000-0ff1-ce00-000000000000/autodiscover.schweiz.ch

And any further DOMAINS you may have on your on-premise Exchange Domains .EU .DE .COM don't forget those.

External Links mentioned:

RFC6749:

https://datatracker.ietf.org/doc/html/rfc6749

Microsoft Azure Active Directory OAuth 2.0 Fundamentals:

https://learn.microsoft.com/en-us/azure/active-directory/fundamentals/auth-OAuth2

Deprecation of Basic Authentication in Exchange Online:

https://learn.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/deprecation-of-basic-authentication-exchange-online

Test-OAuthConnectivity PowerShell cmdlet:

https://learn.microsoft.com/en-us/powershell/module/exchange/test-OAuthconnectivity?view=exchange-ps

Troubleshooting OAuth Expiration in Exchange:

https://learn.microsoft.com/en-us/exchange/troubleshoot/administration/cannot-access-owa-or-ecp-if-OAuth-expired

Configuring OAuth Authentication in Exchange 2013:

https://learn.microsoft.com/en-us/exchange/configure-OAuth-authentication-between-exchange-and-exchange-online-organizations-exchange

We just stumbeld again over the KILLBIT event 3028 warning on several Exchange Versions dated 06/2023.

It still does not sound good from the text and you would think that some third party plugin or som health check blocks the files.

Some say this is normal, some say you can safely ignore this. MS does not seem to know.

On this machine we found following versions of the file. Deletion of the FILE and then IISRESET or Recycle APP Pools does not solve it.

Solution: NONE, just wait and do not handle on SIEM J

Enhancing Outlook Debug Logging for Troubleshooting

Mike Butsch, www.butsch.ch

What we want to do and why

Outlook debug logging is a valuable tool for diagnosing and resolving issues within Microsoft Outlook. By enabling advanced logging, you gain deeper insights into the application's behavior, allowing for more effective troubleshooting. In this blog post, we will explore the process of enabling global and advanced logging for Outlook, along with additional steps to enhance the logging capabilities.

In conclusion, by enabling and leveraging Outlook debug logging, you can gain valuable insights into the behavior of Microsoft Outlook and efficiently troubleshoot any issues that may arise. Remember to exercise caution when modifying the Windows Registry and follow the necessary steps outlined in this blog post.

Some key points in this paper:

* Attention: Do not make user you analyzer Local Adminstrator

* How to make the KEY under another user Registry hive

* Your are admin or user how to generate the Registry Key

* How to find something or for what to search for in the Debug Logfiles you made:

* Analyze WPA and ETL files

* Outlook 2010 Debug Logfiles

* Outlook 2013 and Outlook 2016 Debug Logfiles

* What if you want to enable this for some computers automatic in your Windows Domain on-premise with GPO?

in addition to the steps mentioned in the documentation, there is an additional parameter that can be enabled to gather more detailed information. This parameter involves modifying a specific subkey in the Windows Registry. Please note that the following steps require careful attention to avoid unintended issues:

Attention: Do not make user you analyser Local Adminstrator, this will produce probleme related to Exchange, ActiveSync GPO and Azure sync (Infos not Synced back to local Domain from Azure for that user)

http://www.butsch.ch/post/Active-Directory-accounts-with-ADMINSholderadminCount-flag-%7C-No-syncback-from-Azure-ms-ds-consistencyGuid

Note: If the user experiencing the issue does not have sufficient permissions to create the key, it is important not to grant them local admin privileges without addressing the "ADMINSholder" flag attribute. Granting local admin privileges without correcting the "ADMINSholder" flag may lead to further complications. Refer to these resources for more information on the "ADMINSholder" flag:

Attention: https://www.butsch.ch/post/Activesync-with-Exchange-2013-does-not-work-ADMINSHOLDER-Flag-(an-old-bad-friend) or https://www.butsch.ch/post/Exchange-Activesync-1053-Event-4003-Error-2007201020132016-Adminsholder

Again a sample why you should not make the user Local Admin to explain: Also be aware that if you are in M365 Azure Hybrid Mode those account with ADMINS Holder Flag Set wan't sync back or to Azure.

Target what we need for the USER who has the OUTLOOK.EXE Problem. That may be a different user than the LOGGED on s non domain joined machine.

How to set key for another user?

It's important to note that the user encountering the "OUTLOOK.EXE" problem may not be the same as the one logged on to the non-domain joined machine. Therefore, we need to set the key for the relevant user. Here is a brief explanation of how to set the key for another user:

1. Create a CMD shortcut on the desktop.

2. Right-click on the shortcut while holding down the SHIFT key and choose "Run as different user."

3. A command prompt (CMD.exe) will open.

4. Launch the Registry Editor (regedit.exe) from the command prompt.

5. Locate the appropriate registry hive for the user you want to debug in Outlook.

INFO: Why not make him local Admin again because most don't understand impact if not corrected afterwards! Please read carefull above IF you have that idea.

How to make the KEY under another user Registry hive Explanation #1

Solution you need to run regedit.exe under a seperate LOCAL ADMIN or Service account to have the permission to CREATE that missing key structure per user. Also keep in MIND that existing POLICY/GPO will overwrite the settings within 15-X minutes if you may have those set in Domain enviroment.

Find the office version "SHORTY" the user has as exmaple "15.0", "16.0" etc.

Your are admin or user and how to generate the Registry Key Explanation #2

When troubleshooting Outlook issues for a specific user, there are two options to access the user's registry hive.

By following either of these options, you can access the user's registry hive and make necessary

modifications to troubleshoot the Outlook.exe problem. Remember to exercise caution and ensure you

have the appropriate permissions and authority to access the user's registry hive.

FastShutdownBehavior

Once you have made the necessary changes to the registry, log off and log back on to apply the settings. Launch Outlook, and you should now see the logging in action. This allows you to identify any problematic areas or errors that may be occurring.

Check the key again under the HKCU > OK

Start Outlook, yes loggin active and you see warning

Check the things that don't work to produce logs. Click around and open things and other Calendar from OAB adressbook as example.

Check the logs generates

Folders:

To gather relevant log files for troubleshooting purposes, you can focus on specific folders or actions that are not functioning as expected. These log files can be shared with Microsoft Support to assist in resolving the issue. To convert the log files (ETL format) into a more readable format, you can use the PowerShell command `tracerpt.exe FILENAME.ETL -lr`. You can then preview the converted files or open them using the Event Viewer (`eventvwr.exe`) to examine the events more closely.

Convert in Powershell to some TEXT/XML.

ETL > Event Viewer ;-) ???

You can then preview the converted files or open them using the Event Viewer (`eventvwr.exe`) to examine the events more closely.

Filter may help if there are any error?

How to find something or for what to search for in the Debug Logfiles you made:

When analyzing ETL or XML files for debug information related to free/busy functionality in Outlook, you can look for specific events and data related to free/busy operations. Here are some key points to consider:

1. Event ID: Look for events with specific event IDs that pertain to free/busy operations. These event IDs can vary depending on the version of Outlook and the specific scenario. Typically, you may find events with IDs such as 2000, 2007, or 2016, which are commonly associated with free/busy processing.

2. Provider Names: Look for events associated with providers or services related to free/busy functionality. These providers can include Autodiscover, Exchange Web Services (EWS), Availability Service, or any other components involved in free/busy data retrieval and processing.

3. Timestamps: Pay attention to the timestamps of the events and their sequencing. This can help you understand the flow and timing of free/busy operations. Look for patterns, delays, or any inconsistencies that might indicate potential issues.

4. Error Codes and Messages: Take note of any error codes or error messages associated with the free/busy events. These can provide valuable information about the nature of the problem or any specific errors encountered during free/busy processing.

5. Data Fields: Look for specific data fields or attributes that contain information related to free/busy functionality. These can include user identifiers, calendar data, availability status, or any other relevant data that helps in understanding the free/busy operations.

To locate this debug information within the ETL or XML file, search for the relevant event IDs, provider names, error codes, or specific data fields mentioned above. Use the search functionality in your text editor or XML viewer to locate and analyze the corresponding events. Additionally, you can filter the events based on the time range or other criteria to focus on the relevant information.

By carefully examining these events and data within the ETL or XML file, you can gain insights into the free/busy operations, identify any errors or delays, and troubleshoot any issues affecting the free/busy functionality in Outlook.

Analyze WPA and ETL files

Also read following KB on how to process the files further with WPA

https://learn.microsoft.com/en-us/windows-hardware/test/wpt/opening-and-analyzing-etl-files-in-wpa

https://learn.microsoft.com/en-us/message-analyzer/message-analyzer-tutorial

If you are specifically working with ETL files generated from Outlook.exe debug mode the files are binary files containing event traces, and parsing them requires specific tools and techniques.

To analyze ETL files generated from Outlook.exe debug mode, you can use Microsoft Message Analyzer (MMA) or Windows Performance Analyzer (WPA), which are powerful tools for analyzing event traces. These tools allow you to load the ETL file and apply filters to extract specific information or errors related to Outlook.exe.

Here's a high-level overview of the steps involved in using Microsoft Message Analyzer to analyze Outlook.exe ETL files:

1. Download and install Microsoft Message Analyzer from the Microsoft Download Center.

   https://learn.microsoft.com/en-us/message-analyzer/installing-and-upgrading-message-analyzer

2. Launch Microsoft Message Analyzer.

3. Go to the "File" menu and select "New Session" to create a new session.

4. In the "New Session" window, click on the "Live Trace" tab.

5. Select the "Microsoft-Windows-Diagnostics-Performance" provider from the list.

6. Click on the "Capture" button to start capturing events.

7. Reproduce the issue or scenario for which you want to capture the event trace.

8. Once the desired events have been captured, click on the "Stop" button to stop capturing.

9. Go to the "File" menu and select "Save As" to save the captured events as an ETL file.

10. Open the saved ETL file in Microsoft Message Analyzer.

11. Use the filtering and analysis capabilities of Microsoft Message Analyzer to locate the specific errors or information related to free/busy or any other aspect of Outlook.exe.

Please note that analyzing ETL files can be a complex task, and familiarity with Microsoft Message Analyzer or similar tools is recommended for effective analysis. The steps provided above are a general guideline, and the exact steps may vary based on the version of Microsoft Message Analyzer or other tools you are using.

Remember to always exercise caution when analyzing ETL files and ensure you have the necessary permissions to access and analyze the files.

What if you want to enable this for some computers automatic in your Windows Domain on-premise with GPO?

Be carefull with the GPO. The Debug Mode of Outlook will generate 50MB+ Files maybe per minute. That is

why we want to target that Policy to a ADS-Group with Computer accounts. If you are unsure hot wo do that leave it and do it manual.

Some links regarding this:

Outlook Debug Logging

https://support.microsoft.com/en-gb/topic/how-to-enable-global-and-advanced-logging-for-microsoft-outlook-15c74560-2aaa-befd-c256-7c8823b1aefa

MS Learn:

https://learn.microsoft.com/en-us/outlook/troubleshoot/performance/enable-and-collect-logs-for-profile-creation-issues

Logfiles reference for different Outlook version:

Note The log files are created in multiple folders. These folders vary, depending on the version of Outlook that you're running.

Outlook 2010 Debug Logfiles

Log files in the folder

Outlook 2013 and Outlook 2016 Debug Logfiles

Log files in the folder

Note You can sort by Date modified to find the files that were created most recently.

Ersetzt Azure die Active Directory Domain Controller und Active Directory Services?

Der eigentliche Zweck von Azure-AD ist was?

Azure Active Directory wurde entwickelt, um Microsofts Präsenz in der Cloud zu erweitern. Azure Active Directory sollte ursprünglich Benutzer mit Microsoft M365-Diensten verbinden und eine einfachere Alternative zu ADFS für Single Sign-On bieten.

Nun hat es sich jedoch zu einer Plattform für neue Abonnementdienste entwickelt, die sich an Unternehmenskunden richten und für Funktionen berechnet werden, die Active Directory on-premise ohne zusätzliche Kosten bereitgestellt hat.

Nachteil: LDAP + RADIUS-Integration

Azure Active Directory bietet jedoch nicht alle Funktionen des herkömmlichen Active Directory und unterstützt ohne zusätzliches Abonnement keine Authentifizierungsprotokolle wie LDAP und RADIUS.

Nachteil: Gruppenrichtlinie, GPO-Konvertierung zu Intune Policy nur 75% der Möglichkeiten und geht nur für Englisch richtig

Vertraute und wichtige Konzepte wie Gruppenrichtlinienobjekte (GPO) werden durch Intune und Microsoft Endpoint Manager ersetzt. Diese langjährig etablierten Konzepte ermöglichten es, das Betriebssystem, das von Endbenutzern verwendet wurde, in einen unternehmensfähigen Endpunkt umzuwandeln. Es ist jedoch wichtig zu beachten, dass derzeit nur etwa 75% der bekannten Gruppenrichtlinien konvertiert werden können.

Zudem geht dies meist nur zuverlässig mit englischen GPO-Templates. Sobald andere Sprachen im Einsatz sind, sinkt die success %-Rate.

MS: "Known issues: Currently, the Group Policy analytics tool only supports non-ADMX settings in the English language. If you import a GPO with settings in languages other than English, then your MDM Support percentage is inaccurate."

https://learn.microsoft.com/en-us/mem/intune/configuration/group-policy-analytics

Gruppenrichtlinien waren ein äusserst nützliches Werkzeug, mit dem IT-Abteilungen von kleinen und mittleren Unternehmen bis hin zu grossen Unternehmen die meisten Aspekte ihrer Systeme standardisieren und steuern konnten.

Nachteil: Die Gruppenverwaltung mit rollenbasierter Zugriffssteuerung (RBAC) ist nicht im kostenlosen Tarif von Azure Active Directory enthalten.

Für On-Premise-Kunden kaufte Microsoft vor einigen Jahren genau für diesen Punkt eine Firma auf und Enterprise Agreement-Grosskunden konnten dies fast kostenlos einsetzen. Es gab ja schon alles nur wussten es die Leute nicht die es nicht interessierte.

Nachteil: Statt OU neue Administrative Einheiten (Niemand weiss, warum dies geändert wurde...)

Sogar organisatorische Einheiten (OUs) werden durch ein anderes Modell namens Administrative Einheiten ersetzt, das sich stark von AD unterscheidet.

Nachteil: NPS-Server-Rolle, die bei Windows Server kostenlos dabei war, ist nicht in Azure enthalten

Um ähnliche Funktionen wie das lokale Active Directory und die NPS-Serverrollen zu erreichen, müssen Sie die entsprechenden Azure-Dienste erwerben oder die richtige Bestellnummer finden und bestellen.

https://learn.microsoft.com/en-us/azure/active-directory/fundamentals/auth-radius

Ein Vorteil von Azure Active Directory besteht darin, dass es auch Nicht-Microsoft-Identitäten verwalten kann. Allerdings fallen für die Multifaktor-Authentifizierung (MFA) zusätzliche Gebühren für monatlich aktive Benutzer an. Gastbenutzer werden basierend auf einem Verhältnis von 1:5 lizenziert.

Aber es gibt ja alles bei Azure?

Die Lizenzierung ist komplex, und ein gestaffeltes Modell hält nützliche Funktionen hinter einer kostenpflichtigen Schranke verborgen. Zum Beispiel ist die Gruppenverwaltung mit rollenbasierter Zugriffssteuerung (RBAC) nicht im kostenlosen Tarif von Azure Active Directory enthalten, obwohl sie normalerweise für die Implementierung der Azure-Plattform erforderlich ist.

Ein Microsoft-fokussiertes Modell?

Azure Active Directory bildet das Fundament für Microsofts Portfolio an Identitäts-, Compliance-, Geräteverwaltungs- und Sicherheitsprodukten, da es eine gemeinsame Identität für Intune, Azure, M365 und vieles mehr bietet. Die Vielfalt an Produkten und die Herausforderungen bei der Migration von Active Directory in die Cloud haben zu einer eigenen Beratungsbranche für Implementierung und Planung geführt.

Die Vielfalt an Konfigurationen und Optionen mag für Unternehmen mit umfangreichen Ressourcen (Manpower und cash) zur Unterstützung von Bereitstellungen geeignet sein.

Da jedoch selbst bewährte Methoden von Microsoft für Azure Active Directory Premium-Stufen erfordern, kann Azure Active Directory für kleine und mittlere Unternehmen mit grundlegenden Anforderungen möglicherweise nicht geeignet sein.

Im Gegensatz zu AD gibt es keine einheitliche Plattform für Azure Active Directory.

Es ist in verschiedene Ebenen unterteilt, und Dienste sind hinter kostenpflichtigen Schranken versteckt.

Die Kosten steigen, wenn eine tiefere Integration in die Azure-Plattform erforderlich ist oder eine Interoperabilität mit Verzeichnissen ausserhalb des Microsoft-Ökosystems gewährleistet werden muss.

Zum Beispiel fallen Gebühren an, wenn Azure Active Directory mit Nicht-Microsoft-Identitäten verbunden wird. Es müssen entsprechende Azure Active Directory Premium P2-Lizenzen für die Berechtigungsverwaltung erworben werden. Die Business-to-Business-Zusammenarbeit soll auch als Gateway für Apps dienen, die von Microsoft bereitgestellt oder in Azure gehostet werden, und Microsoft positioniert seine eigenen Produkte und Dienstleistungen als erstklassige Lösungen.

Stattdessen lenkt es KMUs zu Cloud-Diensten, die den Umfang und die Tiefe seiner bestehenden Produktfamilien erweitern und etablierten Kunden zusätzliche Leistungen verkaufen.

Viele Azure-Dienste sind jedoch für Unternehmenskunden konzipiert und können schwer bereitzustellen und zu erlernen sein.

Es ist keine umfassende Lösung wie das ursprüngliche Active Directory on-premise.

Beispielsweise können Endpunkte nicht richtig verwaltet werden, ohne auch Microsoft Intune zu abonnieren.

Die Verwaltung plattformübergreifender Endpunkte erfordert das teuerste Intune-Abonnement und bietet derzeit nur begrenzte Unterstützung für Linux oder andere OS als Microsoft.

Azure Active Directory kann jedoch on-premise Systeme, Nicht-Windows-Endpunkte und den Zugriff auf Netzwerkressourcen nicht verwalten, ohne mit einem Domänencontroller oder zusätzlichen Diensten integriert zu sein.

Es ist möglich, Intune in einem Unternehmen ohne Active Directory on-premise zu nutzen, aber viele Organisationen sind immer noch gezwungen, eine hybride Umgebung zu haben, um eine vollständige Kompatibilität mit Active Directory on-premise oder Active Directory Federation Services zu gewährleisten.

Zur Schluss:

Kann Azure AD Active Directory on-premise ersetzen?

Die kurze Antwort lautet nein oder warum auch? Sicher nicht, um Kosten zu sparen.

Abhängig von Ihrem Abonnementniveau und den Anforderungen, die eine hybride Bereitstellung zwischen Active Directory on-premise und Azure Active Directory erfordern.

Azure Active Directory ist kein direkter Ersatz für Active Directory.

Die Referenzarchitektur von Microsoft sieht sowohl Active Directory on-premise als auch Azure Active Directory in einer Umgebung vor.

What is this about keywords: M365, ADMINSholder/adminCount, Synchronization by Azure AD Connect, ImmutableId, WP;ms-ds-consistencyGuid

Event: ADSYNC, 6100

Error in Synchronisation Service Manager: Insufficient access rights to perform the operation, 8344, permission-issue

Problem:

Synchronisation Service Manager (Microsoft Azure AD Connect Synchronization Services) does not sync the attribute WP;ms-ds-consistencyGuid Back from Azure to Local Active Directory.

This is because a local normal employee user has the ADMINSholder/adminCount flags set to 1/true. (Which is not good in Domain Enviroments)

This is by purpose to protect sensitive Windows account of been written wrong by error.

When a user has been temporarily part of the "Local Admin group" or a member of specific ADS groups like Print Operator or Backup Operators, they receive a special FLAG (ADMINSholder/adminCount=1). The objective is to remove this flag if it is no longer necessary or remove the user from those groups altogether. If not done these accounts may encounter limitations when it comes to functionality with ActiveSync, GPO, and Azure/M365.

If you would like to gain a better understanding of what ADMINSholder/adminCount entails, we recommend reading our informative blog posts. They provide insights into the impact of this flag, especially concerning ActiveSync and GPO. Here are the steps to remove the flag:

https://www.butsch.ch/post/Activesync-with-Exchange-2013-does-not-work-ADMINSHOLDER-Flag-(an-old-bad-friend)

https://www.butsch.ch/post/Exchange-Activesync-1053-Event-4003-Error-2007201020132016-Adminsholder

Please be aware that ADMINSholder Attribute is here to PROTECT sensitive accounts like ADMINISTRATOR, Service, IIS-accounts and resets or overwrites them every 15 minutes to prevent catastrophic impact of wrong GPO/Policies or Powershell commands who manipulate such accounts.

Here is Powershell sample to find the users under your ROOT ADS with excluding certain OU Llike deactivated or service:

Get-ADUser -Filter {admincount -gt 0} -Properties adminCount -ResultSetSize $null -SearchBase "DC=yourdomain,DC=local" | Where-Object { $_.DistinguishedName -notlike "*OU=Benutzer_deaktiviert*" -and $_.DistinguishedName -notlike "*OU=SERVICE_users_IT*" -and $_.DistinguishedName -notlike "*CN=Users,DC=yourdomain,DC=local" } | Format-List DistinguishedName, Enabled, SamAccountName

Change the yellow field to your enviroment or what your OU-folders contain.

This will provide you with a list of on-premises users who have the ADMINSholder FLAG set and will not synchronize back from Azure/M365 to your local Active Directory.

This is because Microsoft wants to protect those special accounts. (AdminSholder refers to a special account.)

Synchronisation Service Manager, Microsoft Azure AD Connect Synchronization Services

Event, ADSYNC, 6100,

The error (Or not because it helps you to find error on-premise and protects you from mistakes) as you see it

There is a user named CLAUDIO whose ms-ds-consistencyGuid value does not sync back from Azure to the Local Active Directory.

This synchronization failure is due to the Sync tool recognizing that the local user has the ADMINSHolder/Admincount FLAG set to 1 (true). As a result, the synchronization process does not update the value in the Local Active Directory.

Links:

https://techcommunity.microsoft.com/t5/core-infrastructure-and-security/hybrid-identity-getting-users-aligned/ba-p/2274690

Primary goal would be to FIX those accounts who are regular users and have the ADMINSolder/admount=1 flag set. You can do this manual:

https://www.butsch.ch/post/Activesync-with-Exchange-2013-does-not-work-ADMINSHOLDER-Flag-(an-old-bad-friend)

https://www.butsch.ch/post/Exchange-Activesync-1053-Event-4003-Error-2007201020132016-Adminsholder

Or you use a scripted solution. Be carefull you don't touch any service accounts my mistake.

C7Solutions has an excellent write up of the issue and also scripts with which you can modify the permission.

https://c7solutions.com/2015/07/configuring-writeback-permissions-in-active-directory-for-azure-active-directory-sync

Also Microsoft has some good Blogs posts:

https://techcommunity.microsoft.com/t5/core-infrastructure-and-security/hybrid-identity-getting-users-aligned/ba-p/2274690

How we found this info beside Technet:

Error: Outlook 2016 verlangt neues Update (Aktuell: 16.0.4266.1001 / Erforderlich: 16.0.4600.1000)

There is a link in the warning which leads to the rather delicate info abour EOL of Office 2016/2019 with M365.

Fact:

If you don't want to update your Office 2016/2019, keep your Exchange on-premise DAG with KEMP and all is fine ;-)

If you run CITRIX and use Microsoft Office, you have a problem that you will have to solve by 2025.

News or not? Since 2017 in a blog published?

End of support for Office 2016 and 2019 for customers who have their email accounts in Microsoft Office M365 is set for October 2023.

We often warn customers that by moving certain things to the Microsoft Cloud, they open themselves up to new dependencies or problems of another kind. These can be plugins from third-party suppliers that don't support the latest Office version or simply that their users are not educated on the latest release.

Here is one issue that we often see customers falling into.

If your email Exchange account is not on-premises and you have an M365 account, you are FORCED to use a newer Microsoft Office version by October 2023. This was communicated in an April 2017 blog post from Microsoft. "We won't take any active measures to block older Office versions from connecting to Microsoft 365 services if they're in extended support and are kept up to date"

https://learn.microsoft.com/en-us/lifecycle/faq/extended-security-updates

Just like with Windows 7 or 2008 R2, enterprise customers had the option to BUY Post Service Support through ESU (meaning patching and getting updates for old OS). This is what is meant by extended support for the connection to M365.

On a full updated W10 with Outlook.exe 2016 VL:

We haven seen this on a TEST Eval machine we use for testing things at a customer site.

W10, Outlook 2016 VL. While adding an add M365 Outlook account to the existing

on-premise Exchange account we suddenly had a warning in Outlook we never did

see before in Domain Enviroment. All our productive and client machines are allways updated

7+ days past patchday so we never did see the link or warning i guess.

We downloaded last patch we could find with that number 16.0.4600.1000 info

If you want to receive the Outlook Update and are not DOMAIN JOINED and internal WSUS you

Should activate this option on W10 to gt Office 2016 Updates (Just a side note....). We don't

See that often because we all have WSUS for customers.

In the Outlook itself you will see this warning and LINK to this URL.

https://learn.microsoft.com/en-gb/deployoffice/endofsupport/microsoft-365-services-connectivity

Older Office versions not supported for connecting to Microsoft 365 services

Older Office versions might still be able to connect to Microsoft 365 services, but that connectivity isn't supported.

In practical terms, what this means is that these older Office versions might not be able to use all the latest functionality and features of Microsoft 365 services. In addition, over time, these older versions might encounter other unexpected performance or reliability issues while using Microsoft 365 services. That's because as we make improvements to Microsoft 365 services, we're not taking into account or testing with these older Office versions.

We won't take any active measures to block older Office versions from connecting to Microsoft 365 services if they're in extended support and are kept up to date. This includes Office 2019 and Office 2016 after October 10, 2023. Both of these versions are in extended support until October 14, 2025.

Therefore, to provide the best experience with using Microsoft 365 services, we strongly recommend that you move off older Office versions to versions supported for connecting to Microsoft 365 services.