ESX: VM’s with wrong CPUID mask show bluescreen after 5.X > 6.0

This is due the NX/CD flag CPUID mask set on machines. Esp. we had seen this on Server 2012R2 which were installed on ESX5.0/5.X and the flag had to be set so machines where running. Be sure to capture this in advance or in time because the SRV2012 will start in Recover Mode at some point.

2008R2 > Bluescreen

2012R2 > Boots into Recovery Console

https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2014835

To resolve this issue, reset the CPUID Mask settings on the affected virtual machine.

To reset the CPUID Mask settings:

  1. Using the vSphere Client, connect to vCenter Server and locate the affected virtual machine.
  2. Power off the virtual machine.
  3. Right-click the virtual machine and click Edit Settings > Options > CPUID Mask > Advanced.
  4. Click Reset All to Default to reset the CPUID Mask.
  5. Click OK > OK, then power on the virtual machine.
  6. The virtual machine now shows the correct EVC mode.

Note: If these steps do not resolve the issue, upgrade the virtual machine's virtual hardware to the latest version. For more information, see Upgrading a virtual machine to the latest hardware version (1010675).

Here is the relevant Link where the Flag was set:

http://www.butsch.ch/post/Server-2012R2-VMware-50X-Blue-screenBSOD-Event-41-CPUID-Mask-Flag.aspx

Here is a script to report the flags on all machines:

http://www.lucd.info/2010/05/13/nxxd-flag-setting-report/

 

VMWARE / VDI malware Protection Symantec, Trend and Mcafee

Symantec Endpoint Protection still has no Agentless Virus scan version like Trend or Mcafee with Move. Those use VSHIELD API from VMware and need no direct Software running in the VM. (http://www.vmware.com/pdf/vshield_55_admin.pdf)

BUT test have shown that even with the Agent in the VM/VDI Symantec SEP 12.X is faster in daily tracking, stable status, scanning but only slow if the machine does Virus pattern updates once a day.

Keep in mind that most virus producers only update the main definitions once a day (mcafee 17:00 CET) and the rest is GTI/0-day releases on all three.

So even with the Agent in VDI machines you over the thumb get more or even performance.

Also keep in mind that Virus API like the one from Microsoft has been sources for a lot of trouble, false events and fights the last few years. You can decide if you want that between:

  1. your antivirus producer and MS

    OR

  2. Between your antivirus producer and VMWARE

To mention on that part would be a solution with Hypervisor which mixes up things again.

The problem in general may be not so actual since Netapp and all the new companies who come out with Flash/SSD Storage try to solve it on the other side.

Gartner Magic Quadrant

http://blogs.antivirussales.ca/en/blog/gartner-magic-quadrant-for-endpoint-protection-platforms/

Mentioned products in terms of VM in those articles:

MCAFEE:

McAfee's Management for Optimized Virtual Environments (MOVE) has offered optimized anti-malware scanning in virtualized environments for two years, and now MOVE 2.5 offers agentless anti-malware scanning in VMware environments using native vShield API integration.

Symantec:

Symantec does not yet offer an "agentless" version for optimizing anti-malware scanning in virtualized environments (although its shared Insight cache feature can be used to improve performance).

2012 Symantec SEP 12.1 and Mcafee MOVE under VMware 5.X

http://www.acmehk.net/report_download/Tolly212130SymantecSEP12dot1VMwareAVPerformance.pdf

2012 Symantec SEP 12.1 and Trend

http://www.symantec.com/connect/sites/default/files/Tolly212117SymantecSEP12_TRendDS8_VMwareAVPerformance.pdf

Back in 2011 Trend was faster

2011 Symantec SEP 11, Trend and Mcafee

http://www.trendmicro.com/cloud-content/us/pdfs/business/reports/rpt_test_deep-security-7.5-vs-mcafee-and-symantec_tolly.pdf

 

Server 2012R2, VMware 5.0.X Blue screen/BSOD, Event 41, CPUID Mask Flag

 

Problem:

You have a blue screen and Event 41 under ESX 5.0.0 when you run Server 2012 R2 because you're VMWARE Team it should run

(You ask > why does it not show up under Supported OS then. They say > VMWARE says it's supported it only shows it's not)

Well not in a validated environment for sure!

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2060019

http://support.microsoft.com/kb/2902739

Event 41

The system has rebooted without cleanly shutting down first. This error could be caused if the system stopped responding, crashed, or lost power unexpectedly.

The computer has rebooted from a bugcheck. The bugcheck was: 0x00000109 (0xa3a01f59c38d1747, 0xb3b72be0160d1792, 0x00000000c0000103, 0x0000000000000007).

A dump was saved in: C:\Windows\MEMORY.DMP. Report Id: 011915-11203-01.

 

Download NITSOFT BLUESCREEN and install on the Server (Just unpack and run exe) to analyses the memory dump if you

Want to be sure.

http://www.nirsoft.net/utils/blue_screen_view.html

 

Two ways to solve this:

  1. UPGRADE VMware 5.0.0 to 5.0.1 U-something and all behind > Project 2 days
  2. Change a CPUID Mask Flag

Here is what to change if you're HAVE AMD Servers:

  1. Shutdown Server
  2. Edit Settings
  3. Options
  4. Advanced

 

Before change on AMD (Attention different settings for AMD and INTEL CPU)

INTEL:

----:0---:----:----:----:----:----:----

AMD:

----0---------------------------

Screenshot: This is what to change for AMD

Vmware KB 2060019

To work around this issue, manually create a CPUID mask for the affected virtual machines:

 

To manually create a CPUID mask for the affected virtual machines:

  • Power down the virtual machine.
  • Right-click the virtual machine and click Edit Settings.
  • Click the Options tab.
  • Select CPUID Mask under Advanced.
  • Click Advanced.
    • For Intel:
      • Under the Register column, locate the edx register under Level 80000001.
      • Paste this into the value field:

        ----:0---:----:----:----:----:----:----

        For example,
        cpuid.80000001.edx = ----:0---:----:----:----:----:----:----

       

    • For AMD:
      • Select the AMD Override tab.
      • Change cpuid.80000001.edx.amd = -----------H-------------------- to
        cpuid.80000001.edx.amd = ----0---------------------------
  • Click OK to close the virtual machine properties.

Remark 12.02.2015

Under ESX 5.5.0 2068190 BOTH the INTEL and AMD Server show the FLAG AS as

"-----------H--------------------"

without the : (Doublodots). As exmaple on an INTEL XEON X5675, 3.07HGz CPU

On a ESX Server where we could select "Server 2012 X64" as new machine.