Netscaler Split tunnel hostname redirection

 After a Netscaler upgrade from 12.1.63.23 to 13.0.88.14 we started to see that users are not able to access a few IP segments specially 172.16.0.0/16 . Interestingly this is working if the client VPN plugin version is 22.2.1.103.

We raised a case with Citrix and after back and forth conversation it is realized that they have changed their design to use this segment for their intranet applications that are defined in split tunneling. This has been confirmed by the support that for some reason the VPN client is intentionally using 172.16.0.0 / 255.255.0.0 as it's Intranet Application hostname interception range. This means the client is pointing that range, on it's own, to itself to be used for hostnames that you list as Intranet Applications in the gateway config.

We will be able to see this clearly if we try to debug the traffic from the client plugin. The fix for this is configure a custom spoofed IP range so that this does not overlap the said I segment range. Below is how we do that. 

VPN Session Profile > Client Experience > Advanced Settings > "Spoofed IP Addresses for FQDN Based Tunneling". We used 169.254.0.0 / 255.255.0.0 as per Citrix support and now we can access 172.16.xxx.xxx on our VPN.

Big3d Timeout on F5

  After the F5 GTM upgrade to 14.1.4.6 it is observed that all the VIP on the GTM are flapping. We observed that this was matching multiple errors. Here the VIP will show a log saying it turned green to red because of big3d timeout. 

Basically, this is a timeout from GTM to LTM

Below were the articles that we saw matching our situation where we thought it is the problem is the resources on the LTM. Since it is pegging up we increased the resources on the LTM.

https://my.f5.com/manage/s/article/K35326235

We tried the recommended action as in below.  But this did not work. 

Recommended Actions

One or more actions can be taken on the impacted BIG-IP device to free up resources so that big3d can respond in a timely manner:

1. Add additional CPU cores to VM or vCMP guest
2. Reduce the size and/or complexity of the BIG-IP configuration

If the above actions do not work or are not feasible, then you may increase the BIG-IP monitor timeout value on BIG-IP DNS:

1. Create a new BIG-IP monitor and increase the probe timeout from 90 seconds (default) to a new value.  Increase in 5 second increments and test to see if the timeout messages stop.
2. Select the new BIG-IP monitor for the impacted BIG-IP server Health Monitor.
3. Click Update.

Timer Change :

Then we tried changing the timer. We created a monitor as in the below settings and configured it to all GTM. What i did not do is we have to apply the same monitor to all the LTM on the server. 

Further, there was a recommendation to update the timers. F5 recommends that the timeout value should be equal to 3 times the frequency ( interval) +1 . Example is below. Also they recommend that the interval value should be shorter. Below is the optimum settings that they suggested.