Issue: We had to reboot our ADC. We were not able to save the configuration both from GUI and command line in ADC. We did a force reboot. When powering on, we observed lot of license related errors like “The license info cannot be fetched” ” No licenses were found” etc.. But after logging in, we were able to see the license file. After netscaler is powered on, we were not able to login through GUI and ssh. In GUI it was “Connection refused” error and in ssh it was “Access denied” error. After long time we were able to login to the netscaler but immediately we got, “Appliance license expired” error after logging in.
Troubleshooting: If your netscaler shows license errors, check cat /var/log/license.log. If the license log contains, “lmgrd will now shut down all vendor daemons” means web portal, and no other option will work immediately. You have to wait for long time for it to work and licenses wont work.
license.log errors:
Error getting status: Cannot find license file. (-1,359:2 “No such file or directory”)
(CITRIX) TCP_NODELAY NOT enabled
(lmgrd) lmgrd will now shut down all the vendor daemons
(lmgrd) Shutting down CITRIX pid=1111 because of signal 15
(lmgrd) Can’t connect to the license server system. Shutdown CITRIX failed.
(lmgrd) No socket connection to license server manager. (-7,10015:36 “Operation now in progress”)
(CITRIX) daemon shutdown requested – shutting down
(lmgrd) CITRIX exited with status 46 (lmgrd requested vendor daemon down)
(lmgrd) EXITING DUE TO SIGNAL 15
These messages basically indicate that the ADC was not able to find the location of the license file as the license server daemon was not starting. When we looked at /etc/hosts file, it was empty. But normally it should be like Hostname=”ADC-Fqdn”. Same value should be shown in /etc/rc.conf file and /nsconfig/rc.conf. There were lot of garbage values in these files which caused huge delay in reboot process.
Resolution:
- From working ADC, copied /etc/hosts file to the non working ADC and changed the hostname in it.
- Deleted existing /etc/rc.conf and /nsconfig/rc.conf files. From working ADC, copied the rc.conf file into /etc/ folder and changed the hostname in it. We did not copy rc.conf file into /nsconfig folder.
With this, license errors were removed and reboot was very quick as expected.
