The full error message looks like:

java -version
Error occurred during initialization of VM
Could not reserve enough space for object heap
Could not create the Java virtual machine.

This appears to be similar to http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6245770 (which is supposedly fixed in 1.5.0_07 but apparently isn't, since the issue is still present in 1.5.0_10, and also has been reported with 1.6). In other words, Java checks the total physical memory (but not the per-user limits ulimit), then tries to grab a large chunk of virtual memory (1GB or so) on startup in case it later might want to do something useful with this. The amount it tries to get varies with the physical memory in the machine.

On shared machines such as LX64SLC4 (where machines have between 2GB and 8GB), per-user limits on virtual memory usage (ulimit -v) prevent it from getting this memory. And Java then falls over... and we cannot do anything about this, since this is still a binary-only product. Not having per-user memory limits on these shared machines is not an option either, since this would allow users to monopolize the machine.

A wider discussion is at http://forum.java.sun.com/thread.jspa?threadID=651637 and http://forum.java.sun.com/thread.jspa?threadID=632411

Workarounds involve explicitly limiting memory usage from the command line, via combinations of parameters (XX:ReservedCodeCacheSize, XX:InitialCodeCacheSize, Xms, Xmx) For java, use one of

 java -XX:ReservedCodeCacheSize=128m -XX:InitialCodeCacheSize=48m -version
 java  -Xms64m -Xmx384m -version

 javac -J-Xmx32m -J-XX:ReservedCodeCacheSize=128m

To our knowledge, this cannot be configured on a per-machine basis.

(Reminder: JAVA is not supported by CERN IT)