Category: Xen Desktop

Step by Step to install Linux Virtual Delivery Agent Installation for RHEL

April 26, 2022 / Ram Prasad

Unfortunately, citrix has not provided detailed step by step procedure to install VDA on Linux distributions. There are so many articles published to install VDA on windows machines whereas not for Linux distributions.

In one of my recent project, I spent lot of time to install VDA on Red Hat Enterprise Linux due to lack of detailed documentation hence I am publishing below step by step procedure to install VDA on Red Hat Enterprise Linux.

This articles covers below components installation & configuration

• RHEL 8.2 Linux Workstation VM Installation on Nutanix AHV
• XenDesktop VDA 2203 Agent Installation & configuration
• NVIDIA Graphics Drivers Installation
• NVDIA vGPU licensing

The Linux Virtual Delivery Agent (VDA) enables access to the Linux virtual apps and desktops anywhere from any device where Citrix Workspace app is installed. Install the VDA software on your Linux virtual machines, configure the Delivery Controller, and then use Citrix Studio to make the apps and desktops available to users.

Supported Linux distributions
System requirements | Linux Virtual Delivery Agent 2203 LTSR (citrix.com)

RHEL 8.2 Linux Workstation VM Installation

Mount the RHEL 8.2 ISO and boot the VM with ISO

Select “Install Red Hat Enterprise Linux 8.2”

In above options, modify each installation setting based on requirement

As my requirement is Linux “Workstation”, I chosen “Base Environment” , choose options based on your requirement

For VDA, choosing GNOME Applications & GRAPHICS Administration tools is sufficient. Choose based on your requirement

Select Automatic unless you have a specific requirements of partitions

Enable Network to get IP from DHCP and Provide Hostname correctly at this page

Post Selection of all options, you will see below output

Reboot VM post installation

Note:

My Citrix VDI setup do not have proxy connectivity hence to install packages and dependencies, I need to mount YUM repository from locally mounted ISO which avoids internet connectivity for any RPM installations. Ignore YUM repository steps if VM able to connect internet

Steps to mount YUM repository locally

Mount the RHEL installation ISO to a directory like /mnt/disc, e.g.:
mkdir -p /mnt/disc
mount /dev/sr0 /mnt/disc

Create below directories to copy repositories

mkdir /home/BaseOS
mkdir /home/AppStream

Copy the media.repo file from the root of the mounted directory

cp -ivr /mnt/disc/BaseOS/* /home/BaseOS/

cp -ivr /mnt/disc/AppStream/* /home/AppStream/

Unmount /mnt/disc -> do after copying files

Delete files if any found under directory /etc/yum.repos.d/ and create the file vi /etc/yum.repos.d/local.repo

Note: you should have only local.repo file under /etc/yum.repos.d/
Copy and Paste the below text in local.repo

[BaseOS]
name = BaseOS
enabled = 1
gpgcheck = 0
baseurl = file:///home/BaseOS/
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
[AppStream]
name = AppStream
enabled = 1
gpgcheck = 0
baseurl = file:///home/AppStream/
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release

Save the changes by typing wq!

Clear the cache and check whether you can get the packages list from the DVD repo with below commands

yum clean all
yum repolist enabled

You should get below output

To disable Red Hat Subscription Management warning , edit below file and change enabled value to 0
vi /etc/yum/pluginconf.d/subscription-manager.conf

Install & Configure Linux Virtual Delivery Agent

Prerequisite for VDA: Dotnet runtime is required for VDA installation

URL:
https://dotnet.microsoft.com/en-us/download/dotnet/thank-you/runtime-aspnetcore-6.0.3-linux-x64-binaries

Upload aspnetcore-runtime file to /root directory and execute below command to extract dotnet runtime
cd /root
mkdir dotnet
chmod 777 dotnet ( Note : command to change directory permissions)

Extract runtime to /root/dotnet with below comand

tar -xf aspnetcore-runtime-6.0.3-linux-x64.tar.gz -C /root/dotnet

Download “XenDesktopVDA-22.03.0.18-1.el8_x.x86_64” and upload XenDesktop VDA file to /tmp directory
Note: The version used is Linux Virtual Delivery Agent 2203 an DDC version is 2112

Run below command to install VDA

sudo yum -y localinstall XenDesktopVDA-22.03.0.18-1.el8_x.x86_64.rpm

Post installation of VDA, configure the VDA with below command

/opt/Citrix/VDA/sbin/ctxinstall.sh

Input below values during configuration
• Host name of the machine on which the Linux VDA is to be installed
• IP address of DNA
• IP address of NTP Server
• Domain name – The NetBIOS name of the domain
• Realm name – The Kerberos realm name (Note: Name should be in CAPITAL LETTERS)
• Fully Qualified Domain Name (FQDN) of the domain controller

While configuring VDA, it prompts for dotnet core runtime path, set the dotnet Core Runtime path as “/root/dotnet”

Note

• As my requirement is for graphics (GPU), here I selected “Y” HDX 3D pro. Select “N” if VDI do not require Graphics
• Linux multisession does not support “HDX 3D PRO” select “NO” if requirement is for multi session and no graphics card required.
• If HDX3D pro enabled then mandatorily NIVIDIA graphics drivers are to be installed else, while launching VDA grey screen appears because Linux by default search for graphics drivers

Cross verify the below Inputs before proceed further

Post VDA Installation and configuration. Need to check the below 2 services status
systemctl status ctxvda
systemctl status ctxhdx

NVIDIA Graphics Drivers installation & vGPU license assignment

Follow below steps, if VDI has requirement to enable Graphics else ignore the steps

Prerequisite for NVIDIA Guest drivers installation: Assign GPU or vGPU to VM

Upload the NVIDIA drivers (highlighted) to /root folder

Note: Upload 2 NVIDIA guest drivers(highlighted above) into /root folder

Install the below dependencies one by one before install of NVIDIA drivers
yum install elfutils-libelf-devel libglvnd-devel
yum install gcc
yum install gcc kernel-devel
yum install gcc kernel-headers

Once successfully installed. Install NVIDIA Package with below command

bash ./NVIDIA-Linux-x86_64-450.102.04-grid.run

Note: Without NVIDIA license server assignment, VDI functionality will be minimum hence map NVIDIA license server for complete functionality.

vGPU License Server Assignment on Linux

run below command to check license status

nvidia-smi –q

Ouptut before License server assignment

To provide NVIDIA License Server on Linux workstation, edit the file gridd.conf with below steps

cd /etc/nvidia/

Create new file “gridd.conf ” by using “gridd.conf.template” file
cp gridd.conf.template gridd.conf

Edit the /etc/nvidia/gridd.conf by vi editor with below highlighted values

ServerAddress=IP Address
ServerPort=7070
FeatureType=0
EnableUI=TRUE ( uncomment by removing #)

Stop & start nvidia service for license reflection

sudo nvidia-gridd stop
sudo nvidia-gridd start

Run command nvidia-smi –q to check license status and verify logs /var/logs/messages
If license not applied then restart VM.

Post License server assignment, you should get below output

Note: My use case is to create Machine Catalog with Unmanaged hence I created catalog and assigned VM’s directly to catalog. If your use case is to create Master Image and rollout VDI with MCS, follow below Citrix KB.

URL:https://docs.citrix.com/en-us/linux-virtual-delivery-agent/current-release/installation-overview/use-mcs-to-create-linux-vms.html

KB References

Quick installation by using easy install
XD Collect Traces
Streaming Linux target devices

XenApp/XenDesktop/Netscaler Gateway Communication Workflow

December 31, 2019 / Ram Prasad

SSL Connection

This is the first step when user type the NetScaler Gateway vServer’s address into browser. We need to focus on the SSL handshake between client and server if any issue happens.

Authentication

Commonly, customer uses LDAP domain authentication. In this article, I will use dual factor authentication as an example (LDAP+Radius).

Get the App/Desktop List.

Get the ica file.

ClienLaunch App/Desktop

Ref: https://support.citrix.com/article/CTX227054

PVS Streaming Service Abrupt Termination – Cache Change Mode Procedure for production vdisk

July 23, 2017 / Ram Prasad

Issue:

PVS stream service abrupt termination intermittently (approx. once in month) which causing user sessions freeze and user unable to launch HSD’s.

Environment :

2 Citrix PVS Servers (VM’s) with version 7.6

2000-3000 concurrent Users

86 HSD’s & 6 Golden Images

Microsoft Hypervisor 2012R2 ( 15 Node) – CICSO UCS

Observations:

Issue occurring once or twice in a month and there is no common pattern in days or hours,issue recurring in both PVS servers at a time
No changes in environment
Onsite engineer informed that issue existed since 3 months and issue getting resolved post restart of PVS servers.
One day, same issue repeated but issue not sorted out post restarting of PVS servers -> Issue escalated to support team (Me)
Observed Event Id 11 :”Detected one or more hung threads , DbAccess error: <Record was not found> <-31754> (in ServerStatusSetDeviceCount() called from SSProtocolLogin.cpp:2903” -> Indicates “Thread hangs under the stream service” & DB Access errors
Observed multiple vDisk retries on the problematic target devices. 11 at boot time and approximately 611 per hour during session
Observed recommended MacAfee exclusions are not in place -> Stopped MacAfee service and restarted PVS server -> PVS Streaming service stable for some time on one PVS server and again terminated ->Due to time constraint, logged a call with vendor(Citrix).
After 2 hrs, Citrix support joined the call and started collecting CDF races and procdump collection for the terminating stream service
After few hours , issue resolved automatically and Citrix support unable to find root cause with collected logs
In 2 months , issue repeated 2 times and customer frustrated as root cause was not found for abrupt streaming service termination intermittently.
Support Team (Myself) analyzed the environment and observed the Cache mode is configured as “ Cache on Server” which is not recommended for Production environment , Best practice to use “Cache on RAM overflow to HDD” which is a best practice to reduce load on PVS server & optimal performance ->Taken the same observation Citrix support and requested their observations

Explained to customer that missing of best practices will lead to these type of intermittent issues , since there is no root cause found and it is not a best practice to keep cache on server in production environment , prepared a plan to change cache configuration to” Cache on RAM overflow to HDD”.

Current PVS Storage configuration for cache as below

PVS1 (VM)->1700 GB allocated through Virtual HBA ( Total golden Image Sizes is 440 Gb & Remaining for Write Cache)

PVS2 (VM) -> 1700 GB allocated through Virtual HBA ( Total golden Image Sizes is 440 Gb & Remaining for Write Cache)

Proposed Storage change Configuration as below:

Post referring multiple blogs, Write Cache proposed to all images(profiles) is 20 GB -> Therefore , for 86 HSD, 1820 GB required and it should present to complete Hyper-v cluster as HSD hosted on cluster.

1820 GB -> To allocate Hyper-v Cluster ( To Create 20 GB Write Cache for each VM)
New LUN of 800 GB to PVS 1 (VM) -> To store Golden Images ( Taken new LUN for easy migration from old LUN and extra space taken for future requirement)
New LUN of 800 GB to PVS 2 (VM) -> To store Golden Images ( Taken new LUN for easy migration from old LUN and extra space taken for future requirement)
Post Cache change and migration, planned to release old LUNs’ ( 1700 & 1700 ) to storage team

How to change Cache Mode for existing production Vdisk:

Get new storage drive as explained above and assign the same drive letter in PVS 1 & PVS2 as it is under Load Balance.
Log into PVS Server 1 ( Initially do only changes in one PVS server).
Copy the Golden Image(.VHD & PVP files) to new drive.
Defragment the vdisk by right clicking the VHD and Mount -> This step is doing as per best practice to achieve optimal performance.
Create New Store in PVS Server and map the path to New drive.
Import the VdIsk to New Store.
Change vdisk to “Private Mode”.
Create new device or change existing device (To boot vdisk in Private Image mode).
Go to Device Properties in device collection, change the vdisk path ( to newly imported vdisk) ->Make sure to change the boot from Network to vdisk.
Go to VM Settings ->Remove IDE Controller-> DVD Drive ( As Write cache drive need to map as “D drive”).
Under IDE Controller -> Create one HDD 20 GB with Fixed drive to assign to HSD (To Write Cache drive to VM ).
Power on & Login to VM , format the drive with MBR ->Assign the drive letter “D” drive (Give Volume name as “Write Cache” & make sure your vdisk in Private mode by accessing vdisk tray icon).
Restart VM.
As an optional , it is best practice to redirect the Page File to Write Cache Drive ( Page File planned to keep as 4 GB post referring few blogs).
Post restart of VM, Go to Page files settings and configure page file(4GB) only to “D drive”.
Go to PVS Server -> vdisk pool -> Change the vdisk from Private to Standard (Note: Cache options will visible only in Standard Image Mode).
Change Vdisk Cache to “Cache on RAM with overflow to HDD” and assign 4 GB RAM in cache ( 4 GB cache is decided post referring few blogs).
Again boot HSD and observe the Cache status (It will be visible as Cache on RAM overflow to HDD and vdisk as Read-only).
Assign this vdisk to any Test VM(device) and ask users to test.
If Image is fine then shut down the devices attached to vdisk -> And copy VHD & PVP (Don’t copy LOK files) to second PVS server -> Make sure replication should be green

To replicate to other devices(VM’s)

Follow same steps 11th to 15^th for rest of the VM’s.
Reboot HSD.
Test the HSD accessibility & Cache configuration.

Conclusions

Since exact root cause was not found, vendor provided & self-analyzed would be the cause for abrupt streaming service termination

When each Target Device boots up the OS is not aware of the Write Cache and writes to the logical disk that it is presented (the vDisk). The PVS driver then redirects this data at the block level to the write cache which in your environment is held on the PVS server. When BNIStack driver ( transport stack for communicating with the PVS server) is then initialized, it will pull down chunks of the vDisk as and when they are needed from your vDisk store which is on your PVS server also. The BNIStack driver will also access the write cache directly when it needs to issue additional writes from the target device. The communication in relation to the above is carried out between the BNIStack driver on the target device and the Stream Service on the PVS server. The case notes state that environment have approximately 2000 users which I suspect is putting a large overhead on the Stream Service responsible for servicing all Write Cache requests.

The Target Device BNIStack driver is also responsible for retries (because UDP does not). The base timeout for a packet timeout is 10 seconds. If the server responds quickly, this value is reduced by half all the way down to 1 second. Correspondingly, if the server responds slowly the timeout will double all the way up to 10 seconds.

A retry timeout of 1 second or less may cause excessive I/O retries, leading to slow response and hanging of target devices which will then ultimately lead to a stream service failure. Since we know we are seeing issues with the stream service and subsequent hanging/crashing of this service and considering we are aware of the large write cache overheads incurred on the stream service already, I suspect that the load we are putting on the stream service is too great, culminating in it grinding to a halt, this will then present itself with the symptoms we see at the target device e.g. slow logging in, sluggishness inside the session.

As explained above, the recommendation to mitigate this is to move the Write Cache away from the PVS Server itself and place the processing overhead at the target device instead.

Recommendations & Reference

We have been observing the environment for about 2 months now and the issue has not occurred since we had disabled the antivirus services.
As a best practice, upgrade all environmental components (servers and target device software) to a required level, either PVS 7.6 LTSR CU3 OR PVS CR 7.13.
FYI: The recommendation for desktop operating systems start with 256-512MB and for server operating systems start with 2-4GB.
PVS RAM Cache consideration -> https://www.citrix.com/blogs/2015/01/19/size-matters-pvs-ram-cache-overflow-sizing/.
PVS Antivirus best practice -> https://support.citrix.com/article/CTX124185.

XenDesktop Controller Hotfix Update Procedure

July 6, 2016 / Ram Prasad

Implementation Plan

Take a Full backup of Citrix Databases on server locally and tapes.
Take a snapshot of DDC01 (Controller 1)
Download and Install Hotfix update 1(CTX135207) on DDC01(Controller 1)
Reboot DDC01
Test VDI by stopping the services in DDC02 so that session will be established to DDC01.
Take a snapshot of DDC02 (Controller 2)
Install Hotfix update 1 on DDC02(Controller 2) – Similar procedure of DDC01
Reboot DDC02
Test VDI by stopping the services in DDC01 so that session will be established to DDC02
Observe for 1 week and remove snapshots.

Roll Back

Uninstall the component from ARP/Programs and Features.
Restore the data store as described in Knowledge Center article CTX135207.
Install the desired level of the component (base or other hotfix).
Restart the Controller even if not prompted to do so

Revert the snapshot which was taken before installation

Find the document with screenshot in attachment

Document – Xen Desktop7.5 Hotfix Update Installation Procedure

Additional Information

Hotfix Name Changes for XenApp/XenDesktop 7.5

June 25, 2016 / Ram Prasad

Information

This article explains the changes for Citrix hotfix naming conventions in XenDesktop 7.1/7.5 with the introduction of XenApp 7.5.

Hotfix Name Changes

With the reintroduction of XenApp in version 7.5, the same underlying components are used for XenApp and XenDesktop. As a result, the ‘XA’ and ‘XD’ designation will not appear in the hotfix name. Instead, the component name is prepended to the hotfix name.

Example 1

A hotfix previously named XD750DStudioWX86001 will now be DStudio750WX86001.

Hotfix Version Number Association

The following components did not change between version 7.1 and 7.5. From now on, updates to these components will only contain the 7.5 association in the name. The hotfixes will be available and compatible with both the 7.1 and 7.5 component versions.

Broker Agent
Desktop OS VDA
Director VDA Plugin
Enhanced Desktop Experience
Personalization AppV – Studio
Personalization AppV – VDA
Server OS VDA
StoreFront Privilege Service
Universal Print Client
Universal Print Server
WMI Proxy Plugin

Example 2

A hotfix previously named XD710ICAWSWX86006 will now be ICAWS750WX86006.

VDA Core Services Hotfixes and Machine Type Association

For the VDA core services hotfixes, the OS type is designated in the hotfix name. ‘TS’ for Terminal Server, the hotfix would apply to a Windows Server operating system. ‘WS’ for workstation, the hotfix would apply to a desktop Windows operating system. There is no correlation between the hotfix numbering for the Server OS (TS) and Desktop OS (WS) hotfixes. The ICATS hotfix ending in 007 might not have the same fixed issues as the ICAWS hotfix ending in 007.

Example 3

A hotfix named ICAWS750WX86007 is a VDA core services hotfix for a Windows 7, 8, or 8.1 operating system (32-bit).

A hotfix named ICATS750WX64007 is a VDA core services hotfix for a Windows Server 2008R2 , 2012, or 2012R2 operating system (64-bit).

Note: The previously released version 7.1 hotfixes will not be rebuilt with the new version identification, but the hotfix readme documents will reflect their support for versions 7.1 and 7.5.

Ref: http://support.citrix.com/article/CTX200156

Error: You cannot access this session because no licenses are available -Xen Desktop

June 25, 2016 / Ram Prasad

Symptoms or Error

A XenDesktop session fails to start, with the following error: “You cannot access this session because no licenses are available.” Event ID: 9027, Event ID: 1163

Event Source: Citrix ICA Service
Error 0 received while obtaining a license for a Citrix XenApp client connection.
The license request has been rejected.
- Note: The license error “flex code -18” shown in the DDC log stands for “License server system does not support this feature”.

The Broker log might contain the following errors:
- Controller:EventLogManager decided to log event CDS_EVENT_LICENSE_NONE_CHECKED_OUT of type Warning with arguments:
  - This is based on event log groups LicensingCheckout
- Licensing:MFLic_GetLicense result Success, request result Rejected
- Licensing:License request rejected, flex code -18

Solution

Resolution 1 – Event ID: 9027

Check the configuration of the site: XenDesktop License edition and model using PowerShell command “Get-BrokerSite” which shows license configuration for the site.

For Virtual Machine hosted applications

AppLicenseEdition
ApplicationLicenseModel

For XenDesktop sessions

DesktopLicenseEdition
DesktopLicenseModel

Compare the configuration with the licenses used in the environment from license server. Check the licenses in the license server. Change the site configuration using the Set-BrokerSite PowerShell command.

Examples

To configure the site to use the Platinum edition, use the following command:

Set-BrokerSite -DesktopLicenseEdition PLT

To set up Virtual Machine hosted applications to use Platinum edition:

Set-BrokerSite -AppLicenseEdition PLT

Resolution 2:

Download and install the most recent version of the licensing server.
On the DDC, restart the Citrix services: (This can be done on a live system and will not affect the users)
- Citrix AD Identity Service
- Citrix Broker Service
- Citrix Configuration Service
- Citrix Diagnostic Facility COM Service
- Citrix Host Service
- Citrix Machine Creation Service
- Citrix Machine Identity Service
On the License Server, restart Citrix services.
- Citrix Licensing
- Citrix Licensing Config Service
- Citrix Licensing Support Service
Access Desktop Studio > Configuration > Licensing > Change Licensing Server > Verify.
If the issue persists, restart all the DDCs in the farm one by one.

Resolution 3 – Event ID: 1163

In this case, upgrade the database as prompted by XenDesktop Studio.

Restart the Desktop Studio Server, error should no longer appear.

Resolution 4

Confirm all licenses are visible in License admin console.
Verify end date of subscription advantage of licenses are relevant to version of product.
Confirm start up server is showing in License admin console and hosts are communicating.
Import startup.lic or reinstall license admin console, which should create a new startup.lic.
1. Issue occurs mainly due to either corrupt startup.lic or no startup.lic under C:\Program Files\Citrix\Licensing\MyFiles
Restart Citrix licensing service.
Relaunch application, error should no longer appear.

Problem Cause

Event ID: 9027 Licensing is not set up properly in XenDesktop Site; licenses are not checked out on the license server when the user tries to start a new XenDesktop session.
License Model: Could be User/Device or Concurrent License Edition: Could be Platinum (PLT), Enterprise (ENT) or Advanced (ADV).
Event ID: 1163 XenDesktop Upgrade did not complete successfully and the following error was observed in the XenDesktop Studio: “A database upgrade is available. Learn more about this upgrade”
Startup license file is missing from the My Files folder.
corrupt startup.lic or no startup.lic under C:\Program Files\Citrix\Licensing\MyFiles

Additional Resources

http://support.citrix.com/article/CTX136266

Find master image used by machine catalog

May 27, 2016 / Ram Prasad

To load Citrix snappins , you need to add the Citrix powershell snapins. To do so, type : Add-PSSnapin citrix* or asnp citrix*

Get-ProvScheme is the powershell command provides the details of provisioning scheme & Master Image

Note: The above details can also be fetched by logging to SQL configuration database(Table: DesktopUpdateManagerSchema.ProvisioningScheme))

Syntax:

MasterImageVM : XDHyp:\HostingUnits\<Hosting Resource Connection Name>\<Master Image Name>.vm\Citrix_XD_<Catalogue Name>.snapshot

Note:

Above POST created based on personal experience & knowledge for personal reuse.
In case, if you wish you to use the above article, use the above steps upon proper testing and reader will be responsible for any outcome.

Latest News