Category Archives: JBoss

Looking for JBoss Maven Repository?

JBoss has decommissioned their Maven 2 repository (about an year ago according to their site) which was available at http://repository.jboss.org/maven2. But many resources out there still refer to this repository, and many people face the following error when they try to use this repository.

Access denied to: http://repository.jboss.org/maven2

This is because JBoss has deactivated this repository and setup a 403 (HTTP Forbidden) error on this URL. After googling for a while, reading through JIRA entries etc., I found this page which pointed to a new repository from JBoss that contains most (if not all) of the artifacts from the previous one. The new repository URL is http://repository.jboss.org/nexus/content/groups/public-jboss/.

It could have been better if JBoss could have given a hint about this in their old repository URL, instead of sending a 403, which gives no clues.

In fact, as the URL indicates, this is a Nexus Maven Repository instance. You can access the Nexus Repository Manager from http://repository.jboss.org/nexus/ which lists all repositories hosted in it.

JBoss JMX Console Vulnerability – Standard Security Is Not Enough !

On 20th October 2011 JBoss released a Security Alert, informing about the existence of a worm which makes use of a security loophole in JBoss JMX Console to attack servers out there in the web. According to this notice, users running unsecured JMX consoles were vulnerable to this attack.

I’ve been running several JBoss Application Server instances exposed to the web, but I always ensure that JMX Console and other management features of JBoss are secured before exposing it to the web. I usually use the standard Username/Password login module for authentication for these JBoss services (I know it’s not very secure, but that was sufficient). Initially when I was setting this up, I referred to this article from JBoss : http://community.jboss.org/wiki/SecureTheJmxConsole [Note: Now it is updated to include the additional steps to protect against this threat].

According to the security alert, password protected JMX consoles were safe from this threat. Since it was password protected, I thought I was secure against this threat. I couldn’t have been more wrong, and I had to learn it the hard way.
Continue reading

JBoss – Changing RMI Remote Client Callback Address

Recently during a JBoss production deployment (4.2.3.GA) that I had to carry on, I came across a problem with RMI Remoting (EJB3), which gave an exception when remote EJBs are invoked. The exception I got was ‘java.lang.IllegalArgumentException: port out of range:-1’, whenever a Remote EJB call was made.

12:57:58,354 WARN  [ServiceExceptionTranslatorAspect] Unable to Translate Exception : org.jboss.remoting.CannotConnectException
12:57:58,402 ERROR [[default]] Servlet.service() for servlet default threw exception
java.lang.IllegalArgumentException: port out of range:-1
at java.net.InetSocketAddress.<init>(InetSocketAddress.java:118)
at org.jboss.remoting.transport.socket.SocketClientInvoker.createSocket(SocketClientInvoker.java:183)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.getConnection(MicroSocketClientInvoker.java:827)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:569)
at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)
at org.jboss.remoting.Client.invoke(Client.java:1634)
at org.jboss.remoting.Client.invoke(Client.java:548)
at org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:62)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
at org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:67)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
at org.jboss.aspects.security.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:53)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
at org.jboss.ejb3.remoting.IsLocalInterceptor.invoke(IsLocalInterceptor.java:74)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
at org.jboss.ejb3.stateless.StatelessRemoteProxy.invoke(StatelessRemoteProxy.java:107)

The application was working fine in our local environments, and this problem happened only in production. This deployment consisted of two JBoss AS instances running two applications, which communicates with each other via EJB Remoting.

Googling did not give me any positive lead, and I was stuck on this problem for a while. So I tried out the only option I was left with, trying to go to detailed log output to see what’s happening under the hood. After changing the log level for CONSOLE and org.jboss category to DEBUG level (if you are new to JBoss, you can change the logging mechanism by modifying jboss-log4j.xml in conf directory of server profile).

This gave me the following output.

12:21:41,049 DEBUG [AuthenticationContextInterceptor] AuthenticationContextInterceptor Intercepting EJB Invocation...
12:21:41,049 DEBUG [AuthenticationContextInterceptor] Setting the authenticated user details in Authentication Context: XXXX
12:21:41,061 DEBUG [AuthenticationContextInterceptor] AuthenticationContextInterceptor Intercepting EJB Invocation...
12:21:41,061 DEBUG [AuthenticationContextInterceptor] Setting the authenticated user details in Authentication Context: XXXX
12:21:43,994 DEBUG [MicroSocketClientInvoker] SocketClientInvoker[b874d2, socket://192.168.12.4:-1] constructed
12:21:43,994 DEBUG [MicroRemoteClientInvoker] SocketClientInvoker[b874d2, socket://192.168.12.4:-1] connecting
12:21:43,994 DEBUG [MicroSocketClientInvoker] Creating semaphore with size 50
12:21:43,994 DEBUG [MicroRemoteClientInvoker] SocketClientInvoker[b874d2, socket://192.168.12.4:-1] connected
12:21:43,995 DEBUG [InvokerRegistry] removed SocketClientInvoker[b874d2, socket://192.168.12.4:-1] from registry
12:21:43,995 DEBUG [MicroSocketClientInvoker] SocketClientInvoker[b874d2, socket://192.168.12.4:-1] disconnecting ...
12:21:44,946 ERROR [[default]] Servlet.service() for servlet default threw exception
java.lang.IllegalArgumentException: port out of range:-1

As the highlighted line shows, JBoss was trying to communicate to the remote EJB via port -1, which is of course invalid. Also, an interesting observation was that the IP address which JBoss tries to use is not the one I expected it to use. This, was actually due to the network setup of the production environment. The production server in question had two network cards, each having local IPs  192.168.12.4 and 192.168.12.5. The 192.168.12.4 NIC was used to expose the server to Internet via NAT. 192.168.12.5 was used to internally connect to the machine via VPN for maintenance, etc. Since 192.168.12.4 was used for NAT, it was restricted to HTTP traffic on port 8080 only.

So the problem that I was facing was that JBoss was using the 192.168.12.4 to refer to the remote EJB, where as I expected it to use 192.168.12.5 (note that we have to start JBoss bound to all addresses using -b 0.0.0.0 because we access it via multiple NICs). Since JBoss could not get a free port on 192.168.12.4, it was falling back to port -1.

So I wanted to find out a way to force JBoss to use the IP address I wanted for remote EJB calls (without binding it specifically to one IP using -b). I was already referring to the remote server’s JNDI Registry via the IP I expected (192.168.12.5). After trying out various options, finally I found the following in the deploy folder of JBoss.

File: <JBOSS_HOME>/server/xxxx/deploy/ejb3.deployer/META-INF/jboss-service.xml

<mbean code="org.jboss.remoting.transport.Connector"
name="jboss.remoting:type=Connector,name=DefaultEjb3Connector,handler=ejb3">
<depends>jboss.aop:service=AspectDeployer</depends>
<attribute name="InvokerLocator">socket://${jboss.bind.address}:13873</attribute>
<attribute name="Configuration">
<handlers>
<handler subsystem="AOP">org.jboss.aspects.remoting.AOPRemotingInvocationHandler</handler>
</handlers>
</attribute>
</mbean>

I changed it to the following, so that instead of dynamically resolving the IP address, JBoss will use the IP I wanted when remote client callbacks are created.

<mbean code="org.jboss.remoting.transport.Connector"
name="jboss.remoting:type=Connector,name=DefaultEjb3Connector,handler=ejb3">
<depends>jboss.aop:service=AspectDeployer</depends>
<attribute name="InvokerLocator">socket://192.168.12.5:13873</attribute>
<attribute name="Configuration">
<handlers>
<handler subsystem="AOP">org.jboss.aspects.remoting.AOPRemotingInvocationHandler</handler>
</handlers>
</attribute>
</mbean>

With this in place, the client was able to invoke the remote EJBs without any issues. This is a rare situation, but if anyone else face the same issue, I guess this post would help to get it sorted out.