I must say, I am quite fortunate to have got some good opportunity to try and run our j2ee application on IBM AIX OS.

I deployed the application and started the application. All went well until I hit upon the very famous “Core Dump – Dump Handler is Processing Signal 11”. I had to spend almost 2 to 3 hours weaving thru lots of newsgroups on the internet and figured out that the FileDescriptor Upper Limit was not set(which means its set to unlimited) there is a problem in AIX in performing a malloc() to obtain a memory segment.

This is what I learnt

This is due to an AIX issue when WLS attempts to perform a malloc() to obtain a memory segment. The WLS Posix Socket Muxer code (NativeIO and stuff) does a malloc based on the FDLimit. If it’s unlimited, AIX 4 and 5 tend to throw a malloc exception. WebLogic Server WLS made it 1025, which the user can set higher if desired.

Check out this URL for more details…