Last night I had an opportunity to be reminded of how powerful and dangerous our BlueCoat ProxySG is to wield. Yesterday when I got home I tried to connect to my workstation via Citrix when I was greeted with an error: “Your App is not available. Try again later.” Strange, it had been so solid for many months. I was technically off the clock (as I am hourly) and after I used VPN to connect in I saw other sessions were still active, so I chalked it up to “I’ll fix that tomorrow.” As the evening progressed, it nagged me a bit so I tried a few more connections before deciding it would be best if the users weren’t up in arms in the morning in case they had trouble connecting as well, so I got to work.
The first placed I look was web interface – both servers – to see if there was an error logged. To my surprise, there was nothing recent in web interface. The event logs on the XenApp servers themselves were also equally clean, and the vserver on netscaler was up with both STA’s functioning. What could be wrong? One thing I did not consider was my own client – it had operated without error for months. So at this point I began cycling various components of the infrastructure. The vserver, ZDC, app servers all got restarts but still, nothing. At that point I tried connecting through AGEE, internally, through the same site (bypassing the firewall). To my surprise, it worked! So what was different about it?
It was then that a co-worker messaged me and asked what was up with remote access. He mentioned that the remote website was presenting a certificate from our proxy (as is normal if SSL interception was happening). I verified that this was indeed the case. So what had happened?
Our ProxySG is setup to SSL intercept based on the category classification of the destination. Interception is used for a number of reasons – authentication, monitoring, and content filtering. I asked the tech if he had added any categories to our list of categories for interception late in the day after I had left work. The category, “Education” matched our own organization and the problem was clear. Our Proxy sits in-line with our firewall transparently, so the traffic to and from AGEE was being intercepted and the certificate was replaced. Citrix receiver aborted the connection because the session wasn’t valid. I added an access rule that bypasses interception for our domain, refreshed the page and verified the certificate was once again the public cert. Success! My workstation hadn’t presented me with a certificate error because it was issued from a trusted root CA as a result of being on our domain.
This situation goes to show that if you’re having trouble connecting through Access Gateway, be certain to verify your certificate!