Hi,
My application fails all of a sudden with the java.lang.OutOfMemoryError: GC overhead limit exceeded.
I am using an EC2 server with instance type m5.xlarge (16GB RAM), and using an external MySql S3 database. Sometimes it goes down with the same error and starts automatically after some time. Do I need to increase the heap size of the application? If so how as I am using the application’s tomcat server and deployed it separately?
Or is there any memory leak that I need to look into?
Any inputs would be much appreciated.
Thanks,
Rak
Adding to the above, I see that the issue occurs when I try to see the content in the CMS.
I am using the same database which was created in the previous production cycle and a fresh hippo project. Do I need to modify something apart from context.xml to connect the new project with the old database.
Note: Apart from some ftl, css changes and different content(test content), nothing much has been updated in the fresh project
how much heap memory have you assigned to your tomcat application?
You can take a heap dump on OOM errors and analyze it:
http://www.oracle.com/technetwork/java/javase/clopts-139448.html#gbzrr
/m
I am using the default tomcat server that Hippo provides.
$ ps aux | grep java
ec2-user 573 0.0 1.0 11772840 333548 ? Sl 12:53 0:16 /usr/lib/jvm/java/bin/java -classpath /usr/share/apache-maven/boot/plexus-classworlds-2.5.2.jar -Dclassworlds.conf=/usr/share/apache-maven/bin/m2.conf -Dmaven.home=/usr/share/apache-maven -Dlibrary.jansi.path=/usr/share/apache-maven/lib/jansi-native -Dmaven.multiModuleProjectDirectory=/home/ec2-user/IBLJ/ib-dev org.codehaus.plexus.classworlds.launcher.Launcher -P cargo.run
ec2-user 1075 0.0 1.1 11772840 360588 ? Sl 13:43 0:17 /usr/lib/jvm/java/bin/java -classpath /usr/share/apache-maven/boot/plexus-classworlds-2.5.2.jar -Dclassworlds.conf=/usr/share/apache-maven/bin/m2.conf -Dmaven.home=/usr/share/apache-maven -Dlibrary.jansi.path=/usr/share/apache-maven/lib/jansi-native -Dmaven.multiModuleProjectDirectory=/home/ec2-user/IBLJ/ib-dev org.codehaus.plexus.classworlds.launcher.Launcher -P cargo.run
ec2-user 1145 24.5 3.6 6453356 1158764 ? Sl 13:43 79:35 /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.171-8.b10.38.amzn1.x86_64/jre/bin/java -agentlib:jdwp=transport=dt_socket,address=8000,server=y,suspend=n -noverify -Xms128m -Xmx512m -Dproject.basedir=/home/ec2-user/IBLJ/ib-dev -Drepo.path=/home/ec2-user/IBLJ/ib-dev/target/storage -Dsend.usage.statistics.to.hippo=true -Drepo.autoexport.allowed=true -Dlog4j.configurationFile=file:///home/ec2-user/IBLJ/ib-dev/conf/log4j2-dev.xml -DLog4jContextSelector=org.apache.logging.log4j.core.selector.BasicContextSelector -Drepo.base.path=/home/ec2-user/IBLJ/ib-dev -Drepo.config=file:///home/ec2-user/IBLJ/ib-dev/conf/repository.xml -Drepo.bootstrap=true -Dcatalina.home=/home/ec2-user/IBLJ/ib-dev/target/cargo/installs/apache-tomcat-8.5.23/apache-tomcat-8.5.23 -Dcatalina.base=/home/ec2-user/IBLJ/ib-dev/target/tomcat8x -Djava.io.tmpdir=/home/ec2-user/IBLJ/ib-dev/target/tomcat8x/temp -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.util.logging.config.file=/home/ec2-user/IBLJ/ib-dev/target/tomcat8x/conf/logging.properties -classpath /home/ec2-user/IBLJ/ib-dev/target/cargo/installs/apache-tomcat-8.5.23/apache-tomcat-8.5.23/bin/tomcat-juli.jar:/home/ec2-user/IBLJ/ib-dev/target/cargo/installs/apache-tomcat-8.5.23/apache-tomcat-8.5.23/bin/bootstrap.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.171-8.b10.38.amzn1.x86_64/lib/tools.jar org.apache.catalina.startup.Bootstrap start
ec2-user 3194 0.0 1.2 11772840 410596 ? Sl Jul08 0:37 /usr/lib/jvm/java/bin/java -classpath /usr/share/apache-maven/boot/plexus-classworlds-2.5.2.jar -Dclassworlds.conf=/usr/share/apache-maven/bin/m2.conf -Dmaven.home=/usr/share/apache-maven -Dlibrary.jansi.path=/usr/share/apache-maven/lib/jansi-native -Dmaven.multiModuleProjectDirectory=/home/ec2-user/IBLJ/ib-dev org.codehaus.plexus.classworlds.launcher.Launcher -P cargo.run
we don’t provide any tomcat server, unless you are talking about cargo profile, which is meant for development only, which seems to be the case.
see:
https://www.onehippo.org/library/enterprise/installation-and-configuration/linux-installation-manual.html
https://www.onehippo.org/library/deployment/configuring/configuring-hippo-for-mysql.html
Your current heap is 512MB, that might not be enough for your application (Xms128m -Xmx512m )
/m
Yes, I was talking about cargo. And ya I am currently working on setting up Tomcat on my server. What should be the value for the below fields in context.xml?
Parameter name=“repository-address” value=“rmi://127.0.0.1:1099/hipporepository” override=“false”/>
Parameter name=“repository-directory” value="${catalina.base}/…/repository" override=“false”/>*
Parameter name=“check-username” value=“liveuser” override=“false”/>*
You can give extra memory using cargo.jvmargs in the cargo maven profile. In your top level pom find the cargo profile. In the properties element you can do:
<cargo.jvmargs>
-agentlib:jdwp=transport=dt_socket,address=9000,server=y,suspend=${cargo.debug.suspend} -noverify ${javaagent} ${cargo.jvm.args}</cargo.jvmargs>
These are the arguments that cargo will get and use for tomcat. This is for local development only of course.