-
Task
-
Resolution: Unresolved
-
Normal
-
None
-
None
-
None
-
None
In order to load the new backup format introduced in Solr 8.9, Solr has to run in SolrCloud mode for mirrors.
The default command for Solr Docker image is "solr-foreground". It only takes the option "--cloud" to run in SolrCloud mode, but then it isn't initializing cores/collections anymore.
References:
[SEARCH-733] Initialize locally with Solr 9 in SolrCloud mode
Using "precreate-core" is just adding a new duplicate core:
docker compose down --volumes
docker compose run --rm solr bash -c 'precreate-core annotation && /opt/solr/docker/scripts/docker-entrypoint.sh solr-foreground --cloud'
Log excerpts, focused on the core "annotation" for an example:
INFO (main) [c: s: r: x: t:] o.a.s.c.CorePropertiesLocator Found 17 core definitions underneath /var/solr/data INFO (main) [c: s: r: x: t:] o.a.s.c.CorePropertiesLocator Cores are: [annotation, editor, recording, release-group, label, annotation, instrument, series, place, work, tag, release, artist, cdstub, area, url, event] ERROR (main) [c: s: r: x: t:] o.a.s.s.CoreContainerProvider Could not start Solr. Check solr/home property and the logs => org.apache.solr.common.SolrException: Found multiple cores with the name [annotation], with instancedirs [/var/solr/data/annotation] and [/var/solr/data/mycores/mbsssss/annotation] at org.apache.solr.core.CoreContainer.checkForDuplicateCoreNames(CoreContainer.java:1254) org.apache.solr.common.SolrException: Found multiple cores with the name [annotation], with instancedirs [/var/solr/data/annotation] and [/var/solr/data/mycores/mbsssss/annotation] at org.apache.solr.core.CoreContainer.checkForDuplicateCoreNames(CoreContainer.java:1254) ~[?:?] at org.apache.solr.core.CoreContainer.loadInternal(CoreContainer.java:1064) ~[?:?] at org.apache.solr.core.CoreContainer.load(CoreContainer.java:788) ~[?:?] at org.apache.solr.servlet.CoreContainerProvider.createCoreContainer(CoreContainerProvider.java:408) ~[?:?] at org.apache.solr.servlet.CoreContainerProvider.init(CoreContainerProvider.java:231) ~[?:?] at org.apache.solr.servlet.CoreContainerProvider.contextInitialized(CoreContainerProvider.java:108) ~[?:?] at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1049) ~[jetty-server-10.0.22.jar:10.0.22] at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:624) ~[jetty-servlet-10.0.22.jar:10.0.22] at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:984) ~[jetty-server-10.0.22.jar:10.0.22] at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:740) ~[jetty-servlet-10.0.22.jar:10.0.22] at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:392) ~[jetty-servlet-10.0.22.jar:10.0.22] at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1304) ~[jetty-webapp-10.0.22.jar:10.0.22] at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:901) ~[jetty-server-10.0.22.jar:10.0.22] at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:306) ~[jetty-servlet-10.0.22.jar:10.0.22] at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:532) ~[jetty-webapp-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:121) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:121) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.22.jar:10.0.22] at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:221) ~[jetty-server-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.server.Server.start(Server.java:470) ~[jetty-server-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.22.jar:10.0.22] at org.eclipse.jetty.server.Server.doStart(Server.java:415) ~[jetty-server-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1919) ~[jetty-xml-10.0.22.jar:10.0.22] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:?] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?] at java.base/java.lang.reflect.Method.invoke(Unknown Source) ~[?:?] at org.eclipse.jetty.start.Main.invokeMain(Main.java:229) ~[start.jar:10.0.22] at org.eclipse.jetty.start.Main.start(Main.java:528) ~[start.jar:10.0.22] at org.eclipse.jetty.start.Main.main(Main.java:76) ~[start.jar:10.0.22] ERROR (main) [c: s: r: x: t:] o.a.s.s.CoreContainerProvider Error processing the request. CoreContainer has shut down. ERROR (main) [c: s: r: x: t:] o.a.s.s.SolrDispatchFilter Could not start Dispatch Filter. => javax.servlet.UnavailableException: Error processing the request. CoreContainer has shut down. at org.apache.solr.servlet.CoreContainerProvider.checkReady(CoreContainerProvider.java:141) javax.servlet.UnavailableException: Error processing the request. CoreContainer has shut down. at org.apache.solr.servlet.CoreContainerProvider.checkReady(CoreContainerProvider.java:141) ~[?:?] at org.apache.solr.servlet.CoreContainerProvider.getCoreContainer(CoreContainerProvider.java:122) ~[?:?] at org.apache.solr.servlet.SolrDispatchFilter.getCores(SolrDispatchFilter.java:163) ~[?:?] at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:141) ~[?:?] at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:133) ~[jetty-servlet-10.0.22.jar:10.0.22] at org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$2(ServletHandler.java:725) ~[jetty-servlet-10.0.22.jar:10.0.22] at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source) ~[?:?] at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Unknown Source) ~[?:?] at java.base/java.util.stream.ReferencePipeline$Head.forEach(Unknown Source) ~[?:?] at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:749) ~[jetty-servlet-10.0.22.jar:10.0.22] at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:392) ~[jetty-servlet-10.0.22.jar:10.0.22] at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1304) ~[jetty-webapp-10.0.22.jar:10.0.22] at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:901) ~[jetty-server-10.0.22.jar:10.0.22] at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:306) ~[jetty-servlet-10.0.22.jar:10.0.22] at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:532) ~[jetty-webapp-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:121) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:121) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.22.jar:10.0.22] at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:221) ~[jetty-server-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.server.Server.start(Server.java:470) ~[jetty-server-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.22.jar:10.0.22] at org.eclipse.jetty.server.Server.doStart(Server.java:415) ~[jetty-server-10.0.22.jar:10.0.22] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.22.jar:10.0.22] at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1919) ~[jetty-xml-10.0.22.jar:10.0.22] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:?] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?] at java.base/java.lang.reflect.Method.invoke(Unknown Source) ~[?:?] at org.eclipse.jetty.start.Main.invokeMain(Main.java:229) ~[start.jar:10.0.22] at org.eclipse.jetty.start.Main.start(Main.java:528) ~[start.jar:10.0.22] at org.eclipse.jetty.start.Main.main(Main.java:76) ~[start.jar:10.0.22]
The branch yvanzo/mb-solr@solr-9.7.0 runs smoothly in standalone mode:
docker-compose down --volumes docker-compose up
Log excerpts, focused on the core "annotation" for an example:
INFO (main) [c: s: r: x: t:] o.a.s.c.CorePropertiesLocator Found 16 core definitions underneath /var/solr/data INFO (main) [c: s: r: x: t:] o.a.s.c.CorePropertiesLocator Cores are: [editor, recording, release-group, label, annotation, instrument, series, place, work, tag, release, artist, cdstub, area, url, event] INFO (coreLoadExecutor-13-thread-1) [c: s: r: x:annotation t:] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0 INFO (coreLoadExecutor-13-thread-1) [c: s: r: x:annotation t:] o.a.s.s.IndexSchema Schema name=annotation INFO (coreLoadExecutor-13-thread-1) [c: s: r: x:annotation t:] o.a.s.s.IndexSchema Loaded schema annotation/1.5 with uniqueid field id INFO (coreLoadExecutor-13-thread-1) [c: s: r: x:annotation t:] o.a.s.c.CoreContainer Creating SolrCore 'annotation' using configuration from instancedir /var/solr/data/mycores/mbsssss/annotation, trusted=true INFO (coreLoadExecutor-13-thread-1) [c: s: r: x:annotation t:] o.a.s.c.SolrCore Opening new SolrCore at [/var/solr/data/mycores/mbsssss/annotation], dataDir=[/var/solr/data/data/annotation/] INFO (coreLoadExecutor-13-thread-1) [c: s: r: x:annotation t:] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer INFO (coreLoadExecutor-13-thread-1) [c: s: r: x:annotation t:] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog INFO (coreLoadExecutor-13-thread-1) [c: s: r: x:annotation t:] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=/var/solr/data/data/annotation defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 INFO (coreLoadExecutor-13-thread-1) [c: s: r: x:annotation t:] o.a.s.u.CommitTracker Hard AutoCommit: if uncommitted for 60000ms; INFO (coreLoadExecutor-13-thread-1) [c: s: r: x:annotation t:] o.a.s.u.CommitTracker Soft AutoCommit: if uncommitted for 10000ms; INFO (coreLoadExecutor-13-thread-1) [c: s: r: x:annotation t:] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /var/solr/data/mycores/mbsssss/annotation/conf INFO (searcherExecutor-34-thread-1-processing-annotation) [c: s: r: x:annotation t:] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms
However, it doesn’t initialize correctly in SolrCloud mode:
diff --git a/docker-compose.yml b/docker-compose.yml index 9d18de6..ae6ba2a 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -12,6 +12,7 @@ services: - "127.0.0.1:8983:8983" volumes: - solr-data:/var/solr/data/data + command: solr-foreground --cloud volumes: solr-data:
docker-compose down --volumes docker-compose up
Log excerpts, still focused on the core "annotation" for an example:
INFO (main) [c: s: r: x: t:] o.a.s.c.CorePropertiesLocator Found 16 core definitions underneath /var/solr/data INFO (main) [c: s: r: x: t:] o.a.s.c.CorePropertiesLocator Cores are: [editor, recording, release-group, label, annotation, instrument, series, place, work, tag, release, artist, cdstub, area, url, event] ERROR (coreLoadExecutor-21-thread-1-processing-172.23.0.2:8983_solr) [c:annotation s: r: x:annotation t:] o.a.s.c.ZkController => org.apache.solr.common.SolrException: No coreNodeName for CoreDescriptor[name=annotation;instanceDir=/var/solr/data/mycores/mbsssss/annotation] at org.apache.solr.cloud.ZkController.checkStateInZk(ZkController.java:2011) org.apache.solr.common.SolrException: No coreNodeName for CoreDescriptor[name=annotation;instanceDir=/var/solr/data/mycores/mbsssss/annotation] at org.apache.solr.cloud.ZkController.checkStateInZk(ZkController.java:2011) ~[?:?] at org.apache.solr.cloud.ZkController.preRegister(ZkController.java:1949) ~[?:?] at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1741) ~[?:?] at org.apache.solr.core.CoreContainer.lambda$loadInternal$13(CoreContainer.java:1083) ~[?:?] at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:212) ~[metrics-core-4.2.26.jar:4.2.26] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[?:?] at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ~[?:?] at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$1(ExecutorUtil.java:449) ~[?:?] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?] at java.base/java.lang.Thread.run(Unknown Source) [?:?]
For the record, "coreNodeName" is an optional entry of "core.properties" files which is otherwise generated automatically. Setting it to an arbitrary value in "core.properties" files is mainly generating new errors. In any case, there seems to be no reason to set it manually.
This suggests that we should just create collections through a custom set-up script.