Missing Dependencies in Tomcat Deployment

Experience Manager (PaaS/OnPrem)
1

Hello beautiful people,

Recently while helping a client troubleshoot their 11.2 to 13.2 upgrade, I discovered that the CMS Updater Editor was not loading on their hosted environments (ie, Dev), but it did work locally. When I checked the logs, I found this was due to a NoClassDefFoundError for org.slf4j.event.Level. I interpreted this as a sign that a dependency was missing, so I decided to check the libs added to Tomcat, since the libs included in the project wars would be consistent across all environments.

I made a list of all jars in my local tomcat9x (cargo.run) common/libs and shared/libs folders, and compared it to the list of jars in Tomcat’s libs folder on the Dev environment. I found 11 jars on my local environment which were missing on the client’s Dev environment:

< hippo-cms7-commons-13.2.0.jar
< hippo-repository-api-13.2.0.jar
< hippo-repository-builtin-13.2.0.jar
< hippo-services-13.2.0.jar
< hippo-services-autoreload-13.2.0.jar
< hst-api-13.2.0.jar
< jcl-over-slf4j-1.7.25.jar
< log4j-api-2.11.1.jar
< log4j-core-2.11.1.jar
< log4j-slf4j-impl-2.11.1.jar
< slf4j-api-1.7.25.jar

The missing autoreload jar makes sense, since that was unwanted on prod-like environments. I was, however, surprised by the other missing hippo-* jars, as well as the missing logging jars. Not unsurprisingly, the culprit behind the NoClassDefFoundError was the missing slf4j-api-1.7.25.jar. Upon copying this jar to the Dev environment and restarting Tomcat, the Updater Editor worked as expected. Unfortunately, adding that jar also broke brXM logging. The hippo-* logs only had a small number of items appended after that jar was added to the Tomcat libs folder.

So this leads me to two questions (or groups of questions, as it may be):

  1. Why might the logging have broken when the SLF4J jar was added to Tomcat? Perhaps that jar was already contributed from somewhere else, with a log4j/slf4j configuration relative to the pre-existing jar. But if this is the case, why was slf4j’s Error class not found by the CMS application, and why was the logging on the Dev environment seemingly the same as my local environment (which definitely had the SLF4J jar)?

  2. Why are these 11 jars added to Tomcat directly (locally, this is part of the cargo.run profile configured in hippo-cms7-project), instead of included in the assembled war files for the applicable brXM webapps? I suspect that there is a need to share these jars across the multiple applications, or perhaps embedding these libs in multiple webapps (ie, cms and site) in the same container would cause conflicts? In any case, the cargo.run maven profile is not typically used to run brXM on non-local environments, so where can I find the necessary brXM documentation which specifies the necessary logging jars for a non-local environment?

Thanks in advance!
Dave

2

check if cms/site war contain logging jars and make sure those get excluded. Only shared directory should contain those jars

3

They did not. The only log-related jars in the Tomcat folder (which includes the exploded wars under ./webapps/) were those in the ./libs/ directory.

4

Check out this link which explains the “dist” profile. Prepare the First Project Release - Bloomreach Experience Manager (PaaS/Self-Hosted) - The Fast and Flexible Headless CMS
I recommend creating a fresh project with the brXM version you are working with, and run mvn clean verify && mvn -Pdist . Then inspect the contents of distribution .tar.gz that produces

5

Hello Dave,

as you mentioned that this is in the context of an upgrade effort from v11 to v13, I would also advise to have a look here. Double check the required changes both in the project (dependencies, configuration xml structure, assembly files etc) but as well the changes required in running instances. More specifically look at the sections where setenv.sh is being mentioned.

HTH and regards,
Lef

6

Thanks @bcanvural!

That link was about Blue/Green cloud deployments and was somewhat helpful. However, it lead me to another page, which others may find even more helpful (it explains the separate shared and common classloaders and is likely the cause of my logging issue):

Thanks again!

7

Thanks @Lef_Karamoulas!

At least part of that log4j upgrade had been performed, but upon further review, it looks as though it was incomplete. That was definitely a helpful resource.

Thanks again!