Symantec Endpoint Protection Client causing a Windows BSOD – an extreme approach to Intrusion Prevention

Imagine just finishing your morning iced coffee (yes iced, don’t judge), and hearing reports from multiple users that either upon login attempts OR after 15-20 minutes after logging in, the system experiences a BSoD and reports a stop code of BAD_POOL_CALLER with indication of bugcheck for IDSvia64.sys:

BAD_POOL_CALLER = No productivity for you today!

Even the most skilled/trained engineers or support specialists get tripped up on where to start with these types of error reports. Should I check the Event Logs if I can? Should I try to see if I can stay logged in enough to run NirSoft BlueScreenView to review dump or minidump files for additional investigation? Should I begin review of the affected machines in any inventory/configuration management systems? The list goes on…

Fortunately, we live in a time where simply searching for the answer online is more advantageous – remember, social networks have an unreal ability to index and surface results in real-time. Searching on Twitter for latest “gripes” regarding bad pool caller resulted in this winning article as the FIRST result:

https://support.symantec.com/us/en/article.TECH256643.html

Details in the article identify that the LiveUpdate downloaded Intrusion Prevention signature sequence 2019/10/14 r61 was to blame, and that the r62 release should be used as a replacement. Funny story, Symantec – if you can’t keep the machine RUNNING long enough to get a new LiveUpdate, this becomes terribly difficult to resolve, and so does the recommended backdating of definitions approach you recommend.

In the “your mileage may vary” audience, it was quickly identified that the following repeatable process could be completed to resurrect the faulty definitions updates:

Confirm that your SEP Management server has the latest definitions synchronized, and/or trust that the LiveUpdate servers are now delivering the latest (and stable) definitions to endpoint clients.
Boot the machine into Safe Mode using the approach of choice.
1. Use the SHIFT + Restart option on the Windows 10 sign-in Screen
  - While on the sign-in screen, press and hold the SHIFT key, then select the Power button and finally select Restart.
  - Upon entering the Recovery Console, choose Troubleshoot.
  - Select Advanced Options on this screen.
  - Select Startup Settings on the next screen.
  - On the Start-up Settings screen, select the Restart Button.
  - Finally, select the number key (or function/Fn key) that corresponds to the entry for “Enable Safe Mode”.
2. Wait for three boot failures – There’s no place like Automatic Repair!
  1. You may be presented with a choice of a user account. Select an administrative account that you know the passcode for, and provide the credentials when prompted to do so.
  2. You may also be asked for your BitLocker recovery key, if enabled, so make sure to have that handy as well.
  3. On the “Automatic repair” options screen, select Advanced Options.
  4. At this point, the steps you have to take are the same as those shown above. Follow the path “Advanced options -> Startup Settings -> Restart.” Then, press the 4 (or the F4 key) on your keyboard to boot into minimal Safe Mode.
If possible, log into the affected device as a Local Administrator. Hopefully you are using LAPS or some other form of lateral escalation attack management approach in your environment(s)!
Navigate to %ProgramData%\Symantec\Symantec Endpoint Protection\CurrentVersion\Data\Definitions\IPSDefs.
Select all files within the IPSDefs folder, and SHIFT-Delete to remove. Do NOT remove the IPSDefs folder itself!
Reboot the device to restore functionality. There may be a warning that definitions are out-of-date, but this can be temporarily ignored as the client will check in and force a LiveUpdate check after a certain amount of time.

The great news is that in all honesty, it took a lot less time to identify the root cause and issue a resolution to the devices than it took to write this article. Thanks to others who shared their trials and troubleshooting efforts online over the past two days, so that we could streamline a process even further to resolve an inconceivable Symantec fubar…