Channel Manager issues in Kubernetes

Experience Manager (PaaS/OnPrem)
1

We are migrating our site/cms (13.4.x) into Kubernetes, which is, for the most part, going very well, but one nagging issue is the Channel Manager/Preview, which shows no channels. I’m curious if anyone has experienced anything similar. Note that everything works fine in our more traditional Apache + Tomcat setup.

The errors I’m getting are:

06.11.2020 21:20:06 WARN  http-nio-8080-exec-1 [ChannelStore.loadChannels:470] Cannot get channels
java.lang.IllegalStateException: Channel Manager is not available since there is not HstRequestContext
	at org.onehippo.cms7.channelmanager.HstUtil.getHostGroup(HstUtil.java:31) ~[hippo-addon-channel-manager-frontend-13.4.3.jar:13.4.3]
	at org.onehippo.cms7.channelmanager.channels.ChannelStore.loadChannels(ChannelStore.java:467) [hippo-addon-channel-manager-frontend-13.4.3.jar:13.4.3]
	at org.onehippo.cms7.channelmanager.channels.ChannelStore.update(ChannelStore.java:447) [hippo-addon-channel-manager-frontend-13.4.3.jar:13.4.3]
	at org.onehippo.cms7.channelmanager.RootPanel.render(RootPanel.java:150) [hippo-addon-channel-manager-frontend-13.4.3.jar:13.4.3]
	at org.onehippo.cms7.channelmanager.ChannelManagerPerspective.render(ChannelManagerPerspective.java:82) [hippo-addon-channel-manager-frontend-13.4.3.jar:13.4.3]
	at org.hippoecm.frontend.plugins.standards.tabs.TabsPlugin.render(TabsPlugin.java:202) [hippo-cms-api-13.4.3.jar:13.4.3]
	at org.hippoecm.frontend.plugins.cms.root.RootPlugin.render(RootPlugin.java:225) [hippo-cms-perspectives-13.4.3.jar:13.4.3]
	at org.hippoecm.frontend.PluginPage.render(PluginPage.java:286) [hippo-cms-engine-13.4.3.jar:13.4.3]
	at org.hippoecm.frontend.PluginRequestTarget.respond(PluginRequestTarget.java:148) [hippo-cms-api-13.4.3.jar:13.4.3]
	at org.apache.wicket.request.cycle.RequestCycle$HandlerExecutor.respond(RequestCycle.java:895) [wicket-core-7.11.0.jar:7.11.0]
	at org.apache.wicket.request.RequestHandlerStack.execute(RequestHandlerStack.java:64) [wicket-request-7.11.0.jar:7.11.0]
	at org.apache.wicket.request.RequestHandlerStack.execute(RequestHandlerStack.java:97) [wicket-request-7.11.0.jar:7.11.0]
	at org.apache.wicket.request.cycle.RequestCycle.execute(RequestCycle.java:265) [wicket-core-7.11.0.jar:7.11.0]
	at org.apache.wicket.request.cycle.RequestCycle.processRequest(RequestCycle.java:222) [wicket-core-7.11.0.jar:7.11.0]
	at org.apache.wicket.request.cycle.RequestCycle.processRequestAndDetach(RequestCycle.java:293) [wicket-core-7.11.0.jar:7.11.0]
	at org.apache.wicket.protocol.http.WicketFilter.processRequestCycle(WicketFilter.java:261) [wicket-core-7.11.0.jar:7.11.0]
	at org.apache.wicket.protocol.http.WicketFilter.processRequest(WicketFilter.java:203) [wicket-core-7.11.0.jar:7.11.0]
	at org.apache.wicket.protocol.http.WicketFilter.doFilter(WicketFilter.java:284) [wicket-core-7.11.0.jar:7.11.0]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:9.0.38]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:9.0.38]
	at org.hippoecm.frontend.plugins.login.ConcurrentLoginFilter.doFilter(ConcurrentLoginFilter.java:54) [hippo-cms-login-13.4.3.jar:13.4.3]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:9.0.38]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:9.0.38]
	at org.hippoecm.hst.container.HstDelegateeFilterBean.doFilter(HstDelegateeFilterBean.java:269) [hst-core-13.4.3.jar:13.4.3]
	at org.hippoecm.hst.container.DelegatingFilter.doFilter(DelegatingFilter.java:68) [hst-commons-13.4.3.jar:13.4.3]
	at org.hippoecm.hst.container.HstFilter.doFilter(HstFilter.java:51) [hst-commons-13.4.3.jar:13.4.3]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:9.0.38]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:9.0.38]
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) [catalina.jar:9.0.38]
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) [catalina.jar:9.0.38]
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) [catalina.jar:9.0.38]
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143) [catalina.jar:9.0.38]
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) [catalina.jar:9.0.38]
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690) [catalina.jar:9.0.38]
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) [catalina.jar:9.0.38]
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) [catalina.jar:9.0.38]
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:374) [tomcat-coyote.jar:9.0.38]
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) [tomcat-coyote.jar:9.0.38]
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) [tomcat-coyote.jar:9.0.38]
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1590) [tomcat-coyote.jar:9.0.38]
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-coyote.jar:9.0.38]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_265]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_265]
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-util.jar:9.0.38]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_265]

And I’m seeing lots of this sort of thing in the logs:

06.11.2020 18:58:48 WARN  http-nio-8080-exec-4 [HstDelegateeFilterBean.doFilter:267] 'Request{ method='GET', scheme='https', host='path-to.cms', requestURI='/cms/', queryString='1-1.IBehaviorListener.0-root-pinger&_=1604689107638'}' can not be matched to a host. Skip HST Filter and request processing.

I’ve confirmed our cms web.xml contains the proper HstFilter and here is a representation of our hosts setup:

  • hst:hst
    • hst:hosts
      • test-k8s
        • path-to
          • site
            • hst:root
  • hst:platform
    • hst:hosts
      • test-k8s
        • path-to
          • cms
            • hst:root
              • hst:ismapped = false
              • hst:namedpipeline = WebApplicationInvokingPipeline

As an experiment, I did create a mount for our CMS under hst:hst, and all expected channels appeared (though preview still didn’t work). It seems as if the mount under hst:platform is being ignored even though the host looks to be coming through properly in the logs.

I’m just trying to think of what else I might be missing.

2

To confirm, you are accessing the cms at url: http://cms.path-to/ right? and the site is at http://site.path-to/ ? (Mind the reverse hierarchical order)
The error looks like host configuration issue. I assume you already saw https://documentation.bloomreach.com/13/library/concepts/hst-configuration-model/hosts-configuration.html

3

also check your filter order in cms web.xml,
see:

4

Thanks so much.

To be clear–

We’re accessing our cms at a URL such as cms-test.our.host.edu and our sites themselves using pattern like www-test.our.host.edu. We have lots of channels, all of which follow the same general rule. For example, archive-test.our.host.edu and database-test.our.host.edu.

At the top of our cms web.xml (just under the context-param), we have:

  <listener>
    <listener-class>org.hippoecm.hst.platform.container.HstCmsContextLoaderListener</listener-class>
  </listener>

  <filter>
    <filter-name>HstFilter</filter-name>
    <filter-class>org.hippoecm.hst.container.HstFilter</filter-class>
  </filter>

  <filter-mapping>
    <filter-name>HstFilter</filter-name>
    <url-pattern>/*</url-pattern>
  </filter-mapping>

As a test, at one point, I copied in the web.xml from GoGreen but encountered the same issue.

I do believe everything should be consistent with:

It does seem like I have a configuration problem somewhere, but I’ve spent hours staring at our hst:hosts configuration under hst:platform, and I just don’t see it.

Is there any additional logging I could turn on that would enable me to see what’s going on when matching virtualhost groups between hst:platform and hst:hosts?

5

Hstdelegateefilterbean logs like you enabled should log information about host matching. But just to be clear, I see that in that log you have

06.11.2020 18:58:48 WARN  http-nio-8080-exec-4 [HstDelegateeFilterBean.doFilter:267] 'Request{ method='GET', scheme='https', host='path-to.cms', requestURI='/cms/', queryString='1-1.IBehaviorListener.0-root-pinger&_=1604689107638'}' can not be matched to a host. Skip HST Filter and request processing.

This log tells me you are trying to access the cms url at literally http://path-to.cms/ . for http://cms-test.our.host.edu/ to work you need 4 nested virtualhost nodes, with “edu” being the parent of it all and the last child being cms-test. do you have this in your host configuration?

I.e. did you just replace the actual cms urls to hide it in this post, or is that host configuration your actual host configuration?

6

Probably makes sense for me just to include a screenshot of our QA env config.

qa-config
qa-config1548×1096 205 KB

7

I’m just hiding the CMS/site urls. We have nested nodes for each part of the URL. I’ll take another look at Hstdelegateefilterbean logging and see if I notice anything new.

8

Just bringing this to a close–

One of our developers discovered that we had a hst-config.properties being deployed to our cms webapp, which included the line

hst.configuration.rootPath = /hst:hst

With this, hst:platform was being ignored.

In our case, we were coming from a separate deployment model (with site and cms on different servers), which we’re collapsing into a single deployment on Kubernetes. I didn’t stop to think that including hst-config.properties in the CMS webapp might be unnecessary or even destructive.

Thanks so much for everyone’s help on this.

9

Nice find, thank for sharing