Ram's Virtual Tech Site

All about Virtualization -Personal Blog

Category: Citrix

Local Host Cache Reintroduction– Long Awaited Feature

Local Host Cache (LHC) & Evolution

Local Host Cache was a core feature of the Independent Management Architecture (IMA) that was introduced with Citrix Metaframe XP 1.0 in 2001, and was still used until Citrix XenApp 6.5 and now reintroduced in XenApp/Desktop 7.12

Technically, the LHC is a simple Access database where it stores a subset of the data store in each Presentation (XenApp) server. The IMA service running on each Presentation(XenApp) Server downloads the information for every 30 mins or whenever a configuration change is made in farm.

LHC primary functions are permits a server to function in the absence of a connection to the data store & improves performance by caching information of applications.

LHC contain the information of servers, published applications, Domain & Licensing. LHC evolved a lot over the years and allowed SQL downtimes for an indefinite period in its last release with XenApp 6.5.

If the data store is unreachable, the LHC contains enough information about the farm to allow normal operations for an indefinite period, if necessary. However, no new static information can be published, or added to the farm, until the farm data store is reachable and operational again.

The disappearance of LHC

With the release of the awful version 7.0 of XenApp in 2013 and the move to XenDesktop FlexCast Management Architecture (FMA), Citrix decided to remove the Local Host Cache feature–and many others–without offering any other alternative. To be fair, Citrix converged XenApp into XenDesktop, which was already using the FMA design since the version 5 and without Local Cache Host equivalent.  This decision immediately made the SQL infrastructure a critical piece of any XenApp implementation. Any downtime on the SQL infrastructure would immediately cause a downtime for new sessions on the XenApp infrastructure as well. It could also have some side effects with the old Citrix Web Interface.

Citrix recommends having a highly available SQL infrastructure to host XenApp and XenDesktop databases. While you can successfully implement HA for your SQL infrastructure, it does not necessarily mean that you will avoid downtimes, as many components are to be considered.

The pseudo rebirth of LHC with Connection Leasing (CL)

Facing a storm of complaints, Citrix also started–finally!–to listen to its customers and released XenDesktop 7.6 in Sept 2014 with the Connection Leasing (CL) feature enabled by default.

Unfortunately, CL was not full replacement of LHC and it is alternative option provided in placement of LHC, limited to frequently used and assigned applications/desktops (up to 2 weeks by default). For users not using Citrix frequently or using pooled desktops, CL is completely useless and did not resolve anything. There are also many limitations: load management, workspace control, power actions are not supported.

The reintroduction of LHC

Citrix came up with a milestone achievement with its new idea as part of the XenDesktop 7.12 release in Dec 2016. This time, they claimed to bring back all the Local Host Cache (LHC) features from XenApp 6.5, even adding few improvements to make it more reliable. LHC feature is offered for Cloud and On Premises implementations along Connection Leasing in 7.12, but is considered the primary mechanism to allow connection-brokering operations when database connectivity to the site database is disrupted. Surprisingly, Local Host Cache feature is disabled by default. Let us expect Citrix to enable that feature by default in the next version.

When installing XenDesktop 7.12 and up, a SQL Express instance(Local DB) will be installed locally on each Delivery Controller to store the Local Host Cache. Config Synchronizer Service (CSS) takes care of the synchronization between the remote database and the Local Host Cache (Local DB). The Secondary Brokering Service (Citrix High Availability Service) takes over from the Principal Broker when an outage is detected and does all registration and brokering operations.

There are many limitations to consider with this version of LHC

  • Local DB, which is a runtime version of SQL Server with a specific licensing that limits the usage of four cores.
  • No support for Pooled desktops, which is a huge downside.
  • No change can be made to the farm (assignments, publications, power actions, etc.), you cannot even open the consoles (Director & Studio) and PowerShell
  • No control over the LHC election process and only a single Delivery Controller will take care of all VDA registrations and broker sessions for the whole zone during an outage which limits  5,000 VDAs per zone (not enforced)
  • Most importantly, it is only a one-way communication between the LHC and the remote SQL database
  • New version of the Local Host Cache would not assure you zero downtime. There is also a delay before users can actually connect .When the remote database goes down, VDAs still have to re-register to the newly and ONLY elected Delivered Controller. It can result in users not having icons in StoreFront or users not able to start new sessions for a short period.

In conclusion, it took Citrix almost 4 years to deliver a somewhat equivalent of the good old Local Host Cache for XenDesktop 7.x. The database is not a single point of failure anymore in a XenDesktop/XenApp deployment. However, customers with large deployments are not supported with this version of the Local Host Cache and some of the -HUGE- limitations can discourage you from using that feature

Ref:

 

PVS Streaming Service Abrupt Termination – Cache Change Mode Procedure for production vdisk

Issue:

PVS stream service abrupt termination  intermittently (approx. once in month) which causing user sessions freeze and user unable to launch HSD’s.

Environment :

2 Citrix PVS Servers (VM’s) with version 7.6
2000-3000 concurrent  Users
86 HSD’s & 6 Golden Images
Microsoft Hypervisor 2012R2 ( 15 Node) – CICSO UCS

Observations:

  • Issue occurring once or twice in a month and there is no common pattern in days or hours,issue recurring in both PVS servers at a time
  • No changes in environment
  • Onsite engineer informed that issue existed since 3 months and issue getting resolved post restart of PVS servers.
  • One day,  same issue repeated but issue not sorted out post restarting of PVS servers -> Issue escalated to support team (Me)
  • Observed  Event Id 11 :”Detected one or more hung threads , DbAccess error: <Record was not found> <-31754> (in ServerStatusSetDeviceCount() called from SSProtocolLogin.cpp:2903” -> Indicates “Thread hangs under the stream service” & DB Access errors
  • Observed multiple vDisk retries on the problematic target devices. 11 at boot time and approximately 611 per hour during session
  • Observed recommended MacAfee exclusions are not in place -> Stopped MacAfee service and restarted PVS server -> PVS Streaming service stable for some time on one PVS server  and again terminated ->Due to time constraint, logged a call with vendor(Citrix).
  • After 2 hrs, Citrix support joined the call and started collecting CDF races and procdump collection for the terminating stream service
  • After few hours , issue resolved automatically and Citrix support unable to find root cause with collected logs
  • In 2 months , issue repeated 2 times and customer frustrated as root cause was not found for abrupt streaming service termination intermittently.
  • Support Team (Myself) analyzed the environment and observed the Cache mode is configured as “ Cache on Server”  which is not recommended for Production environment , Best practice to use “Cache on RAM overflow to HDD”  which is a best practice to reduce load on PVS server & optimal performance ->Taken the same observation Citrix support and requested their observations

Explained to customer that missing of best practices will lead to these type of intermittent issues , since  there is no root cause found  and it is not a best practice to keep cache on server in production environment , prepared a plan to change cache configuration to” Cache on RAM overflow to HDD”.

Current PVS Storage configuration for cache as below

PVS1 (VM)->1700 GB allocated  through Virtual HBA ( Total golden Image Sizes is 440 Gb & Remaining for Write Cache)

PVS2 (VM) -> 1700 GB allocated through Virtual HBA ( Total golden Image Sizes is 440 Gb & Remaining for Write Cache)

Proposed Storage change Configuration as below:

Post referring multiple blogs, Write Cache proposed to all images(profiles) is 20 GB -> Therefore , for 86 HSD, 1820 GB required and it should present to complete Hyper-v cluster as HSD hosted on cluster.

Citrix-XenApp-XenDesktop-XenServer Servicing Options

Citrix provides servicing options to give greater flexibility and choice in how to adopt new XenApp, XenDesktop, and XenServer functionality while giving greater predictability for maintaining and managing the support of your environment

Last year, Citrix introduced two new XenApp / XenDesktop servicing options, the LTSR, which stands for Long Term Service Release and the CR a.k.a. Current Release., In 2016, Citrix announced first LTSR of XenApp and XenDesktop 7.6 and in 2017 first LTSR for XenServer 7.1 that is available for download on Citrix.com.

What is LTSR?

As a benefit of Software Maintenance, Long Term Service Releases (LTSR) of XenApp ,XenDesktop,XenServer enable enterprises to retain a particular release for an extended period of time while receiving minor updates that provide fixes, typically void of new functionality. Long Term Service Releases (LTSR) is ideal for large enterprise production environments where you would prefer to retain the same base version for an extended period

A Long Term Service Release guarantees 5 years of mainstream support and an optional 5 years of extended support (needs to purchased separately). This includes cumulative updates every 4 to 6 months, a new LTSR version of XenApp / XenDesktop every 12 to 24 months and any potential (hot) fixes

A valid Software Maintenance (SM) contract is needed to make use of the LTSR or CR servicing option.

Ideal customer environment for a LTSR is for the customers who typically follow a 3-5 year version upgrade cycle

Long Term Service Releases will have a regular cadence of Cumulative Updates that will typically contain only fixes

What is Current Release?

Any new release of XenApp/XenDesktop/XenServer will be labeled a Current Release. With the CR servicing option you can always make use of (install) the most recent XenApp and/or XenDesktop versions including all the latest enhancements and additions that come with it.

Its release cycles are much shorter with a new version release being announced every three to nine months in general.

Citrix recommends that large enterprise customers have a combination of Current Release and Long Term Service Release environments.

Switching from a LTSR to a CR servicing, and vice versa, is always optional as well

All initial releases of XenApp/XenDesktop/XenServer will be a Current Release. There will likely be multiple Current Releases of a major XenApp/XenDesktop/XenServer version (i.e. 7.6, 7.6 FP1, 7.6 FP2, 7.6 FP3, 7.7, 7.8 ,7.9,7.11,7.13,7.14); however, there will likely only be one LTSR release of that version after that release is considered customer-tested and industry-proven (i.e. 7.6 FP3).

How will the customer know if their environment is Long Term Service Release compliant?

Citrix support and engineering have developed the LTSR Assistant tool which will scan your environment and compare your environment with the necessary LTSR components to determine if you are compliant. The tool provides a report that will outline the necessary updates to achieve compliance. The LTSR Assistant tool is available for download athttp://support.citrix.com/article/CTX209577.

Will a customer running an LTSR compliant environment be supported if they also have non- compliant components?

Citrix does not recommend mixing non-compliant components. For example, if a customer decides to implement Provisioning Services 7.7, which is not compliant with the current 7.6 LTSR environment and the customer has an issue with Provisioning Services 7.7 the customer may be asked to move to the latest Provisioning Services Current Release to receive public fixes

How often will Citrix release a Long Term Service Release of XenApp and XenDesktop or XenServer?

Citrix will release a Long Term Service Release of XenApp and XenDesktop or XenServer based on the number of features, implementations, customer support cases and general feedback. However, as very general guidance it can be expected that Citrix will release a new Long Term Service Release every 12-24 months; however, Citrix reserves the full rights to alter those timelines.

Is Citrix discontinuing the process of providing Hotfix Rollup Packs (HRP) for XenApp and XenDesktop?

With LTSR, Cumulative Updates will replace Hotfix Rollup Packs (HRP). Hotfix Rollup Packs (HRP) will still be made available for XenApp 6.5.

Will 7.6 LTSR support XenApp for Windows Server 2008 R2 for 10 years?

Windows Server 2008 R2 will not be eligible for extended support. Citrix will continue to monitor Windows 2008 R2 lifecycle dates for future determination of lifecycle milestones.

 

 

Delivery Controller vs Data Collector

 

Delivery Controller

Data Collector

No LHC LHC(Local Host Cache)
Connection Leasing No Connection Leasing
Pulls all information, static as well as dynamic from the central Site database Has static as well dynamic(run-time) information cached locally
There is no direct communication between delivery controllers.
No scheduled communication between the VDA’s and/or Site databases, only when needed.
Communicates with the IMA store, Peer  Data Collectors and its session Hosts(within its own zone) on a scheduled interval, or when a Farm configuration change has been made.
Is responsible for brokering and maintaining new and existing user session only. Often hosts user session, but can be configured as a dedicated data collector as well.
Can have a different operating system installed then the server and desktop VDA’s Need to have the same operating system as all other session hosts and DC’s within same Farm.
Core services installed only. The HDX stack is part of VDA software Has all the XenApp 6.5 or earlier bits and bytes fully installed.
Zones are optional. When configured they do need at least one Delivery controller present. Each Zone has one Data Collector. Having multiple data collector means multiple zones.
Election does not apply.
Deploy multiple, at least 2 Delivery controllers per site /zone (again one per zone is the minimum)
Can, and sometimes need to be elected. Configure at least one other Session Host per zone that can be elected as a Data Collector when needed.
When Central Site DB is down, no site wide configuration changes are possible. By default, Connection Leasing will kick in, enabling users to launch sessions which are assigned at least once during last 2 weeks prior to DB going offline. when IMA Db is down, no Farm wide configuration changes are possible. Everything else continues to work as expected due to the LHC present on the Data Collectors and Session Hosts in each Zone.
A Delivery controller can have a direct connection(API) with a Hypervisor or cloud platform of choice. Does not have any direct connection(API) with a Hypervisor or cloud platform management capabilities.
Almost all the communication directly flows through a Delivery Controller to Central Site DB. Session Hosts as well as Data Collectors directly communicate with IMA database.
VDA’s needs to successfully register themselves with a Delivery controller. When a XenApp server boots it needs a IMA service but it will not register itself anywhere.

 

Citrix IMA vs. FMA…

IMA back then…

FMA as it is today…

IMA – Independent Management Architecture. FMA – Flexcast Management Architecture
Farm. Site
Worker Group. Machine Catalog / Delivery Group.
Worker / Session Host / XenApp server. Virtual Delivery Agent (VDA). There is a
desktop OS VDA as well as a server OS
VDA, including Linux.
Data Collector (one per zone). Delivery Controller (multiple per Site).
Zones. Zones (as of version7.7)
Local Host Cache (LHC). Conenction Leasing
Delivery Services Console / App center. Citrix Studio (including StoreFront) and
Director.
EdgeSight monitoring (optional). Partly built into Director
Application folders. Application folders (new feature in 7.6) and
Tags (all 7.x versions).
IMA Data store. Central Site database (SQL only).
Load evaluators. Load management policies.
IMA protocol and service. Virtual Delivery Agents / TCP.
Farm Administrators. Delegated Site Administration using roles
and scopes, which are configurable as well.
Citrix Receiver. Citrix X1 Receiver. It will provide one
interface for both XenApp / XenDesktop as
well as XenMobile.
Smart Auditor. Session Recording.
Shadowing users. Microsoft Remote Assistance, launched
from Director.
USB 2.0. USB 3.0. Support
Session Pre-launch and Session Lingering. Session Pre-launch and Session Lingering.
Both have been re-introduced.
Power and capacity management. Basic power management from the GUI- advanced via powershell
Webinterface / StoreFront. Webinterface / StoreFront.
Single Sign-on for all or most of applications There is no separate Single Sign-On
component available for XenApp 7.x. This is
now configured using a combination of StoreFront, Receiver and policies.
Installed hotfixes inventory Installed hotfixes inventory from studio
Support for Windows Server 2003 and
2008R2.
FMA 7.x supports Windows Server 2008
R2 ,Server 2012 R2,2016

 

Issues encountered post deployment of Netscaler VPX 10.5

Issues encountered post deployment of Netscaler 10.5

Requirement:

Customer imported NetScaler 10.5 VPX to Hyper-v and requested us to configure further configurations

Issue 1:Netscaler URL is not opening over internet

Observations & changes done:

Netscaler has 3 Interfaces ( DMZ, LAN Zone & Loopback)

Netscaler Interface

 

Netscaler IP’s as below

Netscaler IP

  • 172.16.8.X is DMZ Virtual IP. It should be properly natted to public IP 192.X.X.X, then only Netscaler Access gateway web page will open over internet.
  • Network Team will do internal routes from 172.16.8.X to core switches so that it will reach to Citrix infra servers
  • Note that ,172.16.8.x is the virtual IP which you will configure in Gateway virtual server
  • Make sure that 80(STA Port),443(STA Port) ,1494 & 2598 ports opened bidirectional from Netscaler Virtual IP(172.16.8.X) to Citrix infrastructure servers

After above configurations, netscaler web page opening over internet but observed certificate errors and Authentication issue

Issue 2:

User getting error that the credentials are incorrect when logging to Netscaler

Resolution:

The LDAP configuration was not as per the article http://support.citrix.com/article/CTX108876 correcting which rectified the behavior of incorrect username password.

Netscaler LDAP-1

 

Netscaler LDAP-2

 

Netscaler LDAP-3

Issue 3: Certificates errors on Netscaler.

Observations & changes done:

  • Observed intermediate & root certificates are missing in NetScaler which creating authentication issues too..
  • From Client end they are able to get authenticating prompt but not able to get establishing the full session
  • Using the openssl command we have verified that the certificate chain is complete and linked on the VPN virtual server on Netscaler Gateway.
    • # /usr/bin/openssl s_client -connect <ip:port> -showcerts

As per article http://support.citrix.com/article/CTX114146

Issue 4:

VDI launching is working with internal URL and not working externally, throwing VDI error

Observations & Changes done

  • Observed session polices were incorrectly configured, created 2 session policies (Web & Receiver Policy)

Using the article http://support.citrix.com/article/CTX139963

Netscaler Session-1

Netscaler Session-2

Netscaler Session-3

Netscaler Session-4

For Receiver, need to configure account services address (Similar to Xenapp Services URL)

Netscaler Session-5

Issue 5:

Error: Cannot complete request, before log into Netscaler webpage and issue is same from internal URL too.

Observations:

  • Load balancing Virtual name(VDIDesktopxx.locaL) is configured in Session profile but these load balancing VIP (SF1+Sf2) were hosted on separate load balancer and there was some issue with load balancing VIP
  • Customer removed Storefront load balancing IP configuration , informing us to point one storefront(SF1) only in Netscaler.
  • Post Load balancing configuration removal, we got the error “Cannot complete request” as netscaler is unable to find the load balance IP

Changes done:

  • Certificate was binded with local load balancing virtual name(VDIDesktopxx.locaL) hence to maintain the same , we created alias entry for SF1 server so that same URL will be accessed internally and the same reachable from netscaler
  • Observed XML was set to false in DDC, recommended to make it true so ran the command set-brokersite -TrustRequestsSentToTheXmlServicePort $true

After doing all above changes, Users are able to launch VDI externally and internally without any issues

 

Anitivirus Exclusion List – Hyper-v/Citrix

The following antivirus exclusions should be applied to all Citrix infrastructure servers:

  • Set real-time scanning to scan local drives only and not network drives
  • Disable scan on boot
  • Remove any unnecessary antivirus related entries from the Run key
  • Exclude the pagefile(s) from being scanned
  • Exclude IIS log files from being scanned
  • Exclude Windows event logs from being scanned

Hyper-V

Exception Item Description
VHD,VSV,ISO,AVHD,VFD,XML,BIN,VHDX,AVHDX,HRL Extensions
C:\ProgramData\Microsoft\Windows\Hyper-V\*.* Directory & Subdirectories
C:\Users\Public\Documents\Hyper-V\Virtual Hard Disks\*.* Directory & Subdirectories
C:\ProgramData\Microsoft\Windows\Hyper-V\Snapshots Directory
C:\ClusterStorage\*.* Directory & Subdirectories
Vmms.exe Process exclusions
Vmwp.exe Process exclusions
Clussvc.exe Process exclusions

 

Citrix (XenApp/Xen Desktop)

 

Component Exclusion List
Citrix Director & StoreFront Director and StoreFront:
\inetpub\temp\IIS Temporary Compressed Files
\Windows\system32\inetsrv\w3wp.exe
\Windows\SysWOW64\inetsrv\w3wp.exe StoreFront:
\Program Files\Citrix\Receiver StoreFront\Services\SubscriptionsStoreService
Citrix Profile Manager Agent:
Do not scan on open or status-check operations
UserProfileManager.exe
EdgeSight Agent:
<AllUsersProfile>\Application Data\Citrix\System Monitoring\Data
\ProgramFiles\Citrix\System Monitoring\Agent\Core\rscorsvc.exe
\ProgramFiles\Citrix\System Monitoring\Agent\Core\Firebird\bin\fbserver.exe Server:
\CommonProgramFiles\Citrix\System Monitoring\Server\RSSH
\ProgramFiles\Citrix\System Monitoring\Server\EdgeSight\scripts\rssh
\ProgramFiles\Citrix\System Monitoring\Server\EdgeSight\Pages
\ProgramFiles\Microsoft SQL Server\MSSQL\Reporting Services
\ProgramFiles\Microsoft SQL Server\MSSQL\Data
\SystemRoot\SYSTEM32\Logfiles
Provisioning Services – For Server
Note: An even easier approach would be to
 exclude the complete Provisioning services folder
For PVS Server
\Windows\System32\drivers\CvhdBusP6.sys (Windows Server 2008)
\Windows\System32\drivers\CVhdMp.sys (Windows Server 2012)
\Windows\System32\drivers\CfsDep2.sys
\Program Files\Citrix\Provisioning Services\BNTFTP.EXE
\ProgramData\Citrix\Provisioning Services\Tftpboot\ARDBP32.BIN
\Program Files\Citrix\Provisioning Services\StreamService.exe
\Program Files\Citrix\Provisioning Services\StreamProcess.exe
\Program Files\Citrix\Provisioning Services\soapserver.exe
C:\Windows\System32\drivers\CVhdBusP6.sys => (PVS 6.1)
C:\Windows\System32\drivers\CVhdBus2.sys => (PVS 5.6)
C:\Windows\System32\drivers\CFsDep2.sys => (PVS 5.6 and PVS 6.1)
C:\Program Files\Citrix\Provisioning Services\BNTFTP.EXE => (PVS 5.6 and PVS 6.1)
C:\ProgramData\Citrix\Provisioning Services\Tftpboot\ARDBP32.BIN => (PVS 5.6 and PVS 6.1)
D:\Store => ( i.e. local vdisk store) ->Exclude scanning of Local vDisk Store
Provisioning Services – For Target Devices
Note: An even easier approach would be to
 exclude the complete Provisioning services folder
Target Devices:
\Program Files\Citrix\Provisioning Services\BNDevice.exe
\Windows\System32\drivers\bnistack6.sys
\Program Files\Citrix\Provisioning Services\TargetOSOptimizer.exe
\Windows\System32\drivers\CfsDep2.sys
\Windows\System32\drivers\CVhdBusP6.sys
\Program Files\Citrix\Personal vDisk\BIN\WIN7\
C:\Windows\System32\drivers\bnistack.sys => (Only targets, Win2003/XP)
C:\Windows\System32\drivers\bnistack6.sys => (Only targets, 2008/Win7)
C:\Windows\System32\drivers\BNNF.sys => (Only targets, Win2003/XP)
C:\Windows\System32\drivers\BNNS.sys => (Only targets, Win2003/XP)
C:\Windows\System32\drivers\BNNS6.sys => (Doesn’t exist anymore with PVS6.1 Agent)
C:\Windows\System32\drivers\BNPort.sys => (Only targets, Win2003/XP)
C:\Windows\System32\drivers\CFsDep2.sys => (Win2003/XP & 2008/Win7)
C:\Windows\System32\drivers\CVhdBusP52.sys => (Only targets, Win2003/XP)
C:\Windows\System32\drivers\CVhdBusP6.sys => (2008/Win7)
C:\Program Files\Citrix\Provisioning Services\BNDevice.exe => (Only targets, 2008/Win7)
C:\Program Files\Citrix\Provisioning Services\TargetOSOptimizer.exe => (Only targets, 2008/Win7)
Target – Personal vDisk:
CTXPVD.exe
CTXPVDSVC.exe
Exclude scanning of Write Cache
Xen App -Session Controller(Controller) Controller:

\Windows\system32\csrss.exe
\Windows\system32\winlogon.exe
\Windows\system32\userinit.exe
\Windows\system32\smss.exe
\Program Files\Citrix\Group Policy\Client-Side Extension\CitrixCseEngine.exe
\Program Files (x86)\Citrix\System32\wfshell.exe
\Program Files (x86)\Citrix\system32\ctxxmlss.exe
\Program Files (x86)\Citrix\System32\CtxSvcHost.exe
\Program Files (x86)\Citrix\system32\mfcom.exe
\Program Files (x86)\Citrix\System32\Citrix\Ima\ImaSrv.exe
\Program Files (x86)\Citrix\System32\Citrix\Ima\IMAAdvanceSrv.exe
\Program Files (x86)\Citrix\HealthMon\HCAService.exe
\Program Files (x86)\Citrix\Streaming Client\RadeSvc.exe
\Program Files (x86)\Citrix\Streaming Client\RadeHlprSvc.exe
\Program Files (x86)\Citrix\Independent Management Architecture\RadeOffline.mdb
\Program Files (x86)\Citrix\Independent Management Architecture\imalhc.mdb

Xen App -Session Host Session Host:

\Windows\system32\spoolsv.exe
\Windows\system32\csrss.exe
\Windows\system32\winlogon.exe
\Windows\system32\userinit.exe
\Windows\system32\smss.exe
\Program Files\Citrix\Group Policy\Client-Side Extension\CitrixCseEngine.exe
\Program Files (x86)\Citrix\System32\wfshell.exe
\Program Files (x86)\Citrix\system32\CpSvc.exe
\Program Files (x86)\Citrix\System32\CtxSvcHost.exe
\Program Files (x86)\Citrix\system32\mfcom.exe
\Program Files (x86)\Citrix\System32\Citrix\Ima\ImaSrv.exe
\Program Files (x86)\Citrix\System32\Citrix\Ima\IMAAdvanceSrv.exe
\Program Files (x86)\Citrix\HealthMon\HCAService.exe
\Program Files (x86)\Citrix\Streaming Client\RadeSvc.exe
\Program Files (x86)\Citrix\Streaming Client\RadeHlprSvc.exe
\Program Files (x86)\Citrix\XTE\bin\XTE.exe
\Program Files (x86)\Citrix\Independent Management Architecture\RadeOffline.mdb
%AppData%\ICAClient\Cache (if using pass-through authentication)

XenDesktop – Controller Controller:

\Windows\system32\csrss.exe
\Windows\system32\winlogon.exe
\Windows\system32\userinit.exe
\Windows\system32\smss.exe

Controller – pre-XenDesktop 7.x:

\Program Files\Citrix\Group Policy\Client-Side Extension\CitrixCseEngine.exe
\Program Files (x86)\Citrix\System32\wfshell.exe
\Program Files (x86)\Citrix\system32\ctxxmlss.exe
\Program Files (x86)\Citrix\System32\CtxSvcHost.exe
\Program Files (x86)\Citrix\system32\mfcom.exe

Windows Server OS Machines – XenDesktop 7.x:

\Windows\system32\spoolsv.exe
\Windows\system32\csrss.exe
\Windows\system32\winlogon.exe
\Windows\system32\userinit.exe
\Windows\system32\smss.exe
\Program Files\Citrix\Group Policy\Client-Side Extension\CitrixCseEngine.exe
\Program Files (x86)\Citrix\System32\wfshell.exe
\Program Files (x86)\Citrix\system32\CpSvc.exe
\Program Files (x86)\Citrix\System32\CtxSvcHost.exe

Ref:

 

Recovering VM’s from Failed Pool Member -XenServer

Few months back, I had an issue with one of the xen server, we had 2 servers (Server1 & Server2) in a pool. server1 is down and unable to bring it up due to hardware issue.

Troubleshooting’s:

  • Not able to connect SERVER2 with Xen center
  • Login to -> Putty to SERVER2 , issued the command to su –   To change shell to super user mode
  • Issue the command xe vm-list & xe host-list -> Check server is displaying VM’s or host , if not execute below commands
  • Issue the command xe-tool-stack-restart  -> To display the VM’s & host, by issuing this command VM’s & server will not have any impact
  • After issuing above command , all xen server commands are working and able to see host & VM list
  • Currently SERVER2 is a slave , issue the below command to force SERVER2 as a master
    • xe pool-emergency-transition-to-master
  • Now the SERVER2 is a master, issue the below command to know the which xen servers are false or live  and  to know UID’s

xe host-list params=uuid,name-label,host-metrics-live

Xen1

  • Collected the Xen Server UID from above screenshot and issue the below command to know which VM’s are running on the failed server

xe vm-list is-control-domain=false resident-on=fdd15cbc-42ce-4154-8731-9a469ce04976  (Format : xe vm-list is-control-domain=false resident-on=UUID_of_failed_server)

Xen2

  • It displayed all VM;’s which are running on failed server , collect the VM UID and issue the VM power reset  command as below

Format :  xe vm-reset-powerstate uuid=<UUID_of_the_VM_to_recover> –force

  • xe vm-reset-powerstate uuid= 26177b43-eb3e-5d49-ee09-a93bfdca1bbc–force
  • xe vm-reset-powerstate uuid= cb3f18ac-846c-ae46-1bbe-3c6e2ea6560a–force
  • By issuing above commands, VM’s registered in Xencenter and it is visible, powered on each VM.

Xen3

Note:
  • Above POST created based on personal experience & knowledge for personal reuse.
  • In case, if you wish you to use the above article, use the above steps upon proper testing and reader will be responsible for any outcome.
error: Content is protected !!