Exchange 2016 CU20 Schema Update setup.exe /preparead fail because of case sensitivity of OWA APP Policy

ISO/PATCH: ExchangeServer2016-x64-cu20

Cumulative Update 20 for Exchange Server 2016 (microsoft.com)

 Problem:

Exchange 2016 CU20 Setup.exe /preparead (Version 15.1.2242.4 Fails) on Server 2016 (1607)

Step Configuring Microsoft Exchange Server Organization Preparation results FAILED

Exchange 2016 CU 20 need and fails to update Active Directory Schema to newer Version (setup.exe /prepareschema works setup.exe /Preparead fails) if you have renames Outlook Web App Policy Default/default/DEFAULT.

We had a case in a Mother / Child Domain setup where we had to update Active Directory of the Mother domain of the company with commandline to a new Schema Version. This was related to the second Exchange 2016 Breach/Hotfix and we wanted to uplift Exchange 2016 from CUMU 19 to 20 urgently.

Prepareschema worked but the second command preparead failed.

 

 Schema Versions

 

 ERROR you see during the setup.exe /preparead

 Error from Powershell

The following error was generated when "$error.Clear();

$policyDefault = Get-OwaMailboxPolicy -DomainController $RoleDomainController | where

{$_.Identity -eq "Default"};

 if($policyDefault -eq $null)

{

New-OwaMailboxPolicy -Name "Default" -DomainController $RoleDomainController

}

" was run:

"Microsoft.Exchange.Data.Directory.ADObjectAlreadyExistsException: Active

Directory operation failed on NOVCHVOLDCW1.novartis.com. The object

'CN=Default,CN=OWA Mailbox Policies,CN=migros,CN=Microsoft

Exchange,CN=Services,CN=Configuration,DC=migros,DC=net' already exists. --->

System.DirectoryServices.Protocols.DirectoryOperationException: The object exists.

at System.DirectoryServices.Protocols.LdapConnection.ConstructResponse(Int32

messageId, LdapOperation operation, ResultAll resultType, TimeSpan

   

   

Source of problem:

   

   

You can see the OWA APP Policy you have with following:

Get-owamailboxpolicy –Domaincontroller Butschdcw1 | Fl identity

Notice the case Sensitivity of the IDENTITY "Default/default/DEFAULT"

   

Error full:

Workaround:

Change the identity name of Outlook Web app Policy back to Default

  1. Go into Exchange 2016 GUI (Exchange Administrative Center)
  2. Permission / Berechtigung
  3. Outlook Web App-Policy/Outlook Web App-Richtlinien
  4. Mark the "Default/default/DEFAULT" and click the PENCIL/EDIT
  5. Change the name to Default (D large rest small chars)
  6. On DOS replicate the DC's with repadmin.exe /syncall

After that you can run setup.exe /preparead and update the Schema for Exchange 2016 CU

   

   

   

   

Check the Schema after replication with repadmin.exe /syncall

CHECK OBJECTVERSION:

$RootDSE= ([ADSI]"").distinguishedName

([ADSI]"LDAP://cn=swiss,cn=Microsoft Exchange,cn=Services,cn=Configuration,$RootDSE").objectVersion

CHECK RANGEUPPER:

$RootDSE= ([ADSI]"").distinguishedName

([ADSI]"LDAP://CN=ms-Exch-Schema-Version-Pt,CN=Schema,CN=Configuration,$RootDSE").rangeUpper

   

16220 > OBJECTVERSION

15333 > RANGEUPPER

   

   

Some further reading why this could have happened

https://devblogs.microsoft.com/scripting/weekend-scripter-unexpected-case-sensitivity-in-powershell/

https://superuser.com/questions/720037/powershell-if-statement-case-insensitive

 

Final note on this issue:

We have seen several other such related issues with 2016/2019 Exchange. Something does not update or install simply because something is case sensitive or some argument is missing or there where it should not be. Mainly in long history customer which where over 15 years on Exchange in several version.

We know how to fix but always say "And then? Next Update or when it runs same? Does it run?" And sometimes Tier 3 from Microsoft does nothing else. They compare what's different with the customer to their reference and then change the Attribute with ADSIEDIT and close the case. That's it, no explanation.

Still the above mentioned gives me some bad feeling. The patch was released ASAP and it was the second patch. If the tested the patch to death someone else would have come again and said why do they keep the patch back so long? (For IT > It was because they had to discuss so long with NSA on how to turn things back).

If you read the story about the FBI who could change your Exchange settings by court you know what happened if you are not a naive IT-world geek.
Cloud Office 365 was not affected because their NSA backdoor works in another way (Read more on Google or search MSDN TechNet

 

 

 

 

SRV 2016, Windows Patch, BSOD, STOP CODE, 0xc000021a or CRITICAL SERVICE FAILED, ROLLBACK, Blue screen how to

Hello,

This is a collection of some technical things we used to recover a SRV 2016 with blue screens. We assume the first crash was related to a too early SRV 2016 VL Release ISO and ESX 6.5 (From 2018) and a combination of a Windows Defender Update.

This may help you to recover a server 2016 in general. It's so rare with 2012R2/2016 that this may help because it's not daily and people tend to go back A Snapshot or restore from Veeam these days. Still we have seen this happen.

 

SRV 2016, Windows Patch, BSOD, STOP CODE, 0xc000021a or CRITICAL SERVICE FAILED, ROLLBACK, Blue screen how to

Windows Patch, BSOD, STOP CODE, 0xc000021a, ROLLBACK, Blue screen how to (Notice the PAGE FILE Partition where memory DUMP was done)

BSOD, Bluescreen on Server 2016

STOP CODE, 0xc000021a

STOP CODE, CRTITICAL SERVICE FAILED

 

Server 2016 problems patches 02/2021: KB4601318 fails to update, fails at 24% Windows Server 2016 - Microsoft Q&A

Customer did run following VMware setup for the cluster (Because they wanted to test the newer release for some days)

VMware ESX Versions:

6.5 Version 9298722

ESXi 6.5 U2C

ESXi650-201808001

8/14/2018

9298722

NA

6.5 Version 13635690

ESXi 6.5 EP 14

ESXi650-201905001

05/14/2019

13635690

N/A

 

Build numbers and versions of VMware ESXi/ESX (2143832)

 

Rollback of Updates that caused the Bluescreen if you installed Windows Update before.

 

  • Choose blue recovery console
  • Choose troubleshoot
  • Choose cmd.exe
  • Change KEYBOARD layout so you type the Local Admin password correct
  • Logon with Local Admin password

Rollback the last updates with: dism /image:D:\ /cleanup-image /revertpendingactions (D: is the drive where your Windows Server install is thus where c:\porgram files and c:\windows are (Search that Partition from C: to Z:))

 

 

Enter password (Hopefully)

 

Change KEYBOARD layout so you type the Local Admin password correct

Logon with Local Admin password

Search the windows Partition

 

Check with:

Sfc /scannow

 

 

Run: checkdisk if you think there is damage to file system or disk:

Chkdsk G: /f /r /x

 

Check the pending operations he should do or has done during the crash:

 

Remove C:\windows\winsxs\pending.xml.

Cd g:\windows\winsxs\

Rename pending.xml pending.old

 

Rollback the last updates with:

dism /image:D:\ /cleanup-image /revertpendingactions

D: is the drive where your Windows Server install is thus where c:\porgram files and c:\windows are (Search that Partition from C: to Z:)

 

 

Error: 0x800f082f

BAD: (Looks more worse now….)

 

GOOD: (Looks good until you try to reinstall the same patch in 1 week again…)

 

Weputil.exe reboot

OR

Type: EXIT

OR NOT

If you have "CRITICAL SERVICE FAILED" this could be related to a UNSIGNED DRIVER or something is wrong with Certificates (CODE SIGNING). Maybe ask

Firewall TEAM if they block CERTIFICATE REVOCATION and if they don't know what you talk about check yourself. Read more on blog.

Try these BOOT Option with F8

If the Server comes UP with "Disbale Driver Signature Enforcement" and you don't need that (Because it's not a high secure server you could disable it permanent). We do not recommend this on an Exchange Server as example or anything security related.

If it comes up run:

Sfc /scannow

 

Manually disable certificate signature validation:

open CMD.exe as an Administrator

bcdedit -set loadoptions DDISABLE_INTEGRITY_CHECKS

 

Re-enable the certificate validation

open CMD.exe as an Administrator

bcdedit -set loadoptions DENABLE_INTEGRITY_CHECKS

 

    

FAQ: How to remove Remove failed packages in Windows PE

 

Looking why the Server crashed with NIRSOFT tool Bluescreenview

 

 

Microsoft recommends the PAGE FILE Partition to be RAM + SOME XXXMB. This is what happens IF the Server ever has a blue screen so don't do it that way.

I am unsure if the Server would have picked the Partition if no more space was there (Like they recommend). Never like that Rekommandation.

Die Größe der Auslagerungsdatei sollte die Größe des physikalischen RAMs im System überschreiten (SharePoint Server) - SharePoint Server | Microsoft Docs

Ursache: Eine bewährte Methode für Windows ist es, die Größe der Auslagerungsdatei auf einen Wert größer als oder gleich der Gesamtmenge des verfügbaren physischen Speichers festzulegen. Für die automatische Wiederherstellung von Heapspeicher funktioniert die Speicherbereinigung in der Regel effektiver, wenn die Größe des verwalteten Heapspeichers sich der Auslagerungsgröße nähert. Unterschreitet die Größe der Auslagerungsdatei die RAM-Größe, werden neue Zuordnungen von verwaltetem Speicher erteilt, wodurch die Speicherbereinigung aufwändiger wird und die CPU-Beanspruchung steigt.

 

 

 

February 02/2021 Windows Updates Deinstall Adobe Flash on Server and Clients W10 – Attention VMware vCenter/ESX Admins

February 02/2021 Windows Updates Deinstall Adobe Flash on Server and Clients W10 – Attention VMware vCenter/ESX Admins

Mit den Februar 2021 Windows Updates wird Adobe Flash (MS) de-installiert. Von Hand installierte Adobe Flash Binary bleiben auf den Systemen. Bei Teils Kunden brauchen wir ja noch FLASH fuer den Web Zugriff auf vCenter/ESX.

 

Wenn man nicht via HTML5 auf das VMware vCenter drauf kann dann einfach nochmals eine letzte Adobe Flash Version manuell installieren. (https://get.adobe.com/de/flashplayer/about/)

 

Zugriff vCenter 6.5:

https://blogs.vmware.com/vsphere/2016/12/new-vcenter-management-clients-vsphere-6-5.html

 

Vmware Produkte welche noch Flash brauchen:

https://www.virtuallyghetto.com/2020/10/adobe-flash-is-going-away-is-your-vmware-environment-and-it-organization-ready-for-it.html

 

 

 

 

2020 WSUS-Server content Drive suddenly no space over 300GB *.ESD Upgrade files

Windows Update Server filling since a few months over the 350GB max. Value you know from WSUS-Server which runs over years

  • You checked the internal WSUS GUI Command to clean (That does not free space often…)
  • You cleaned the WSUS maybe even if free or commercial scripts like Adamj Clean-WSUS
  • Still you don't get under 350GB for the WSUS content drive
  • You are at a point where the SQL Cleanup stales, Your SQL Management Studio crash
  • You would have to use sqlcmd.exe to clean the WSUS because no space left

Source:

The Source is mostly ESD Windows Distribution Files (*.ESD) or updating from Windows 10 to other W10 versions. These exploded that last few months. Maybe you did one update like a 1903 to 1909 and now you have the full range coming in. This is around 120 to 160GB on Data.

This add. to the 350GB you normally have with running a certain range of products from like 2010-2016 office and W7/W10.

Quick and Dirty Workaround:

When you can't approve new updates and they are urgent and you can't expand the Disk temporary because it's a VM or the storage team refuses to do so (Because they like to save money for the customer [Who understands why?])

  1. Make sure nobody in your SBS or Enterprise does need those updates
  2. Just delete them from the \WSUSCONTENT\ drive recursive with del *.esd /s
  3. Find the person who turned the category on without thinking in advance ;-)
  4. Cancel the Download in the WSUS-GUI and also DENY them if there still NON APPROVED

Check other WSUS category from us:

http://www.butsch.ch/category/WSUS.aspx

 

Afterwards choose "cancel download" and "DENY" them.

 

 

 

09/2020 Patchday, KB4577015, breaks MMC (wsecedit.dll ) console for local security and GPO SRV 2016

 

ERROR: wsecedit.dll, MMC, Local Security Policy, Security Options > "MMC has detected and error in a snap-in"

Update 2020-09 Cumulative Update (KB4577015) bug mit GPO/MMC.

"Next steps: We are working on a resolution and will provide an update in an upcoming release."

Macht ein bug bei Server 2016 z.B. MMC-Konsole. Ich würde daher DC oder IT-MANAGEMENT Server 28.09.2020 nicht weiter patchen.

DC GPO nicht mehr verwaltbar auf SRV 2016 direkt selber.

 

https://docs.microsoft.com/en-us/answers/questions/92345/gpmc-error-for-34security-options34-after-updates.html

 

Workaround:

 

  1. RSAT Tools auf W10 installieren und von dort managen
  2. Unschöner fix unten:

 

reg delete "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\SecEdit\Reg Values\MACHINE/Software/Microsoft/Windows/CurrentVersion/Policies/System/DontDisplayLockedUserId"