Hung threads stream process terminated
Website Visitors:Issue: copying OS via pvs imaging wizard in pvs base machine, would show a green status bar with the progress. It completes till the end, you wont see Done button. It will be in green progress bar forever. You might see an error in pvs event log:
Event id: 11
Detected one or more hung threads. Please send Stream.log to support.
In other event id 11 entry, you would see, Terminating StreamProcess.
This is because of hung threads and streamprocess is terminated.
What are Hung Threads and why is the StreamProcess terminating?
On PVS Servers StreamProcess Hung Threads can lead to poor performance, constant target re-connections and even full outages. These Hung Threads are usually detected by looking at the Windows Application Event logs.
The following Events will be recorded:
Level | Source | Id | Text |
---|---|---|---|
Error | StreamProcess | 11 | Detected one or more hung threads. |
Error | StreamProcess | 11 | Terminating StreamProcess. |
Information | StreamProcess | 10 | Stream Process Stopped |
At this point all the Target Devices connected to the affected PVS Server will need to reconnect to other servers in the Site.
What are StreamProcess Threads?
The StreamProcess executable handles all communication between the PVS Server and the Target Devices.
According to the “Threads per Port” configuration on the PVS Server, multiple Threads are created from the StreamProcess executable. These threads handle the communication with the Target Devices.
What exactly are Hung Threads?
A Hung Thread is a thread of the StreamProcess executable that is waiting on a resource to be available to continue its work with the Target Device There are two resources that, when not available, will hang the thread
vDisk access: If there are any kind of Storage issues, such as communication with storage, permissions, Operating System problems, etc…, the Thread will hang waiting for the vDisk data to be available. Database access: Problems with the SQL Server or Network communication with the SQL Server will cause the Thread to hang while waiting for Database access.
Why and how is the StreamProcess terminated?
The StreamProcess Threads routinely notify of their health and what actions they are performing. If StreamProcess detects that a Thread has not notified of its status for a long time, that Thread is deemed as possibly hung. When StreamProcess detects there are a high number of Threads hung, the StreamProcess will terminate itself to force the Target Devices to reconnect to other PVS Servers.
It’s important to note that this action is performed to actually protect the stability of the Target Devices. If StreamProcess is not terminated, and the Target Devices are not forced to reconnect to other PVS Servers, then the Target Devices will potentially hang causing an interruption in service.
The StreamService will then detect that the StreamProcess has terminated and it will proceed to start the StreamProcess again.
What to do if you encounter Events in the Windows Event Logs identifying Hung Threads
There’s basically only two reasons for a StreamProcess Thread to become hung.
- Inability to access the vDisk or the Storage
- Inability to access the PVS Database
Engage the Storage, Network and DBA teams to check the health of those system and resource availability Confirm all the Anti Virus Exclusions are configured as per the following Citrix Article: https://support.citrix.com/article/CTX124185 Check if there’s a pattern to the occurrence of the Events in the Event Logs
For example if there are Hung Threads every night at the same time, check if there are Backup processes that might impact Storage Access or Maintenance procedures
To eliminate the possibility of issues with the Operating System, reboot the PVS Servers If the issue persists contact Citrix Support to capture diagnostic data and further narrow down the actual root cause of the issue.
Note: Hung threads can be caused by a number of issues, most of the time it is a database or storage issue. Restart your PVS server and check it again. If it still occurs again, restart citrix pvs and your database services. This worked for me.
Want to learn more on Citrix Automations and solutions???
Subscribe to get our latest content by email.