Ok, first the details. Environment is Sun Solaris, Oracle, SM7.1 sp7, EM 9.1. latest SMD agents. EM has been installed and running for several months, this is a new installation. Solman running for years, SP7 for a few months now.
We can call up Introscope Webview and log in with standard ID, interface works. Enterprise Manager starts and runs with no issues.
Problem - for some reason last week EM disconnected itself from SolMan and now we cannot get Solman to see the running instance of EM again. Managed System config fails now because EM is not seen, can't finish configuring some systems. EWA reports are coming in gray because of missing Introscope metrics. Currently have 16 systems configured through Managed Systems Config. All were reporting in quite nicely till noon Friday. No system outages occurred that day, had restarted Solman 2 days prior to bump up Shared Memory setting to 300m to avoid short dumps because of monitoring activity. Xmx and Xms set to 2048 in lax file (were 1024, I increased it to see if the error would go away. It did not).
Stopped/started EM several times, stopped/started SMD agents, short of rebooting Solman (production system, not easy to get time slice to do that), I'm not sure where to look any more.
Not really seeing any errors in EM logs. As far as they're concerned, it's running fine. There are SOME metrics being reported in to Introscope, not sure why some get there and others not. EM Self Monitoring screen in Introscope shows 137 agents connected (about right), 2,626 metrics (seems low). So the agents ARE getting there, but something is still blocking the connection.
Seeing "failed to bind to server socket... Address already in use" in the Introscope log, checked those port and nothing else seems to be using it (8081:6001). When I stop EM, the 6001 entry goes away.
Also seeing "Error accessing to Enterprise Manager (socketTest) ... java.net.ConnectException: Connection refused" on the app server that EM is running on, but that's a symptom of the real problem and not really useful (to me, anyway).
I've gone through a lot of the posts here already, tried various things, getting close to reboot time. Any suggestions to try before I have to go that route?
Thanks.
Bernie