Project

General

Profile

bug #8783

Massive Heap consumption in server instance opening the Description-Matrix editor

Added by Andreas Kohlbecker almost 2 years ago. Updated 7 days ago.

Status:
Closed
Priority:
New
Assignee:
Category:
taxeditor
Target version:
Start date:
12/18/2019
Due date:
% Done:

0%

Severity:
critical
Found in Version:
Tags:

Description

This was observed with a locally running cdmlib-remote-webapp instance configured with 700 MB of Heap:

Left pane shows the cdmlib-remote-webapp instance*, right pane the Taxeditor.

[caryophyllales_spp] 14:34:40,792 WARN [qtp759156157-23][or.sp.re.su.RemoteInvocationTraceInterceptor] - Processing of HttpInvokerServiceExporter remote call resulted in fatal exception: eu.etaxonomy.cdm.api.service.IService.load
java.lang.OutOfMemoryError: GC overhead limit exceeded
    at java.util.HashMap.newNode(HashMap.java:1742)
    at java.util.HashMap.putVal(HashMap.java:630)
    at java.util.HashMap.put(HashMap.java:611)
    at java.util.HashSet.add(HashSet.java:219)
    at java.util.AbstractCollection.addAll(AbstractCollection.java:344)
    at org.hibernate.collection.internal.PersistentSet.endRead(PersistentSet.java:327)
    at org.hibernate.engine.loading.internal.CollectionLoadContext.endLoadingCollection(CollectionLoadContext.java:234)
    at org.hibernate.engine.loading.internal.CollectionLoadContext.endLoadingCollections(CollectionLoadContext.java:221)
    at org.hibernate.engine.loading.internal.CollectionLoadContext.endLoadingCollections(CollectionLoadContext.java:194)
    at org.hibernate.loader.Loader.endCollectionLoad(Loader.java:1179)
    at org.hibernate.loader.Loader.initializeEntitiesAndCollections(Loader.java:1143)
    at org.hibernate.loader.Loader.processResultSet(Loader.java:992)
    at org.hibernate.loader.Loader.doQuery(Loader.java:930)
    at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:336)
    at org.hibernate.loader.Loader.doList(Loader.java:2611)
    at org.hibernate.loader.Loader.doList(Loader.java:2594)
    at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2423)
    at org.hibernate.loader.Loader.list(Loader.java:2418)
    at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:501)
    at org.hibernate.hql.internal.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:371)
    at org.hibernate.engine.query.spi.HQLQueryPlan.performList(HQLQueryPlan.java:216)
    at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1326)
    at org.hibernate.internal.QueryImpl.list(QueryImpl.java:87)
    at eu.etaxonomy.cdm.persistence.dao.initializer.AdvancedBeanInitializer.bulkLoadLazyCollections(AdvancedBeanInitializer.java:533)
    at eu.etaxonomy.cdm.persistence.dao.initializer.AdvancedBeanInitializer.bulkLoadLazies(AdvancedBeanInitializer.java:435)
    at eu.etaxonomy.cdm.persistence.dao.initializer.AdvancedBeanInitializer.initializeNodeNoWildcard(AdvancedBeanInitializer.java:244)
    at eu.etaxonomy.cdm.persistence.dao.initializer.AdvancedBeanInitializer.initializeNode(AdvancedBeanInitializer.java:125)
    at eu.etaxonomy.cdm.persistence.dao.initializer.AdvancedBeanInitializer.initializeNodeRecursive(AdvancedBeanInitializer.java:105)
    at eu.etaxonomy.cdm.persistence.dao.initializer.AdvancedBeanInitializer.initializeNodeRecursive(AdvancedBeanInitializer.java:107)
    at eu.etaxonomy.cdm.persistence.dao.initializer.AdvancedBeanInitializer.initializeNodeRecursive(AdvancedBeanInitializer.java:107)
    at eu.etaxonomy.cdm.persistence.dao.initializer.AdvancedBeanInitializer.initializeAll(AdvancedBeanInitializer.java:85)
    at eu.etaxonomy.cdm.persistence.dao.initializer.AdvancedBeanInitializer.initialize(AdvancedBeanInitializer.java:57)

At the test-server it takes ~700MB to load the matrix data without garbage collection

picture321-1.png View (64.2 KB) Andreas Kohlbecker, 12/18/2019 02:43 PM

picture521-1.png View (9.92 KB) Andreas Kohlbecker, 12/18/2019 03:03 PM

picture071-1.png View (54.7 KB) Katja Luther, 05/26/2020 01:49 PM


Related issues

Related to Edit - bug #8785: Move functionality of buttons to context menu of the character matrix Resolved 12/19/2019
Related to Edit - bug #9029: Adapt character matrix to using dtos instead of whole cdm entities Closed 05/26/2020
Related to Edit - bug #9820: OptionalDataException when saving character matrix New 10/19/2021

History

#1 Updated by Andreas Kohlbecker almost 2 years ago

#2 Updated by Andreas Müller almost 2 years ago

  • Target version changed from Unassigned CDM tickets to Release 5.13

As this is a critical issue I suggest to put it into 5.13. (5.12 is probably not possible as the task is difficult). We may then discuss who will care for it.

#3 Updated by Andreas Müller over 1 year ago

  • Target version changed from Release 5.13 to Release 5.14

#4 Updated by Katja Luther over 1 year ago

  • Target version changed from Release 5.14 to Release 5.15

#5 Updated by Katja Luther over 1 year ago

  • File picture701-1.png added
  • Status changed from New to In Progress

The implementation for the rowWrappers changed, having a look at the consumption shows a smaller heap (opening the character matrix two times):

#6 Updated by Katja Luther over 1 year ago

  • File deleted (picture701-1.png)

#7 Updated by Katja Luther over 1 year ago

#8 Updated by Katja Luther over 1 year ago

  • Status changed from In Progress to Resolved
  • Assignee changed from Katja Luther to Andreas Kohlbecker

This seems to be fixed.

#9 Updated by Andreas Kohlbecker over 1 year ago

from the diagrams it really looks like this problem is fixed. Can you please link the rowWrappers-Ticket with this one?

#10 Updated by Andreas Kohlbecker over 1 year ago

  • Assignee changed from Andreas Kohlbecker to Katja Luther

please see the last comment

#11 Updated by Katja Luther over 1 year ago

  • Related to bug #8785: Move functionality of buttons to context menu of the character matrix added

#12 Updated by Katja Luther over 1 year ago

  • Related to bug #9029: Adapt character matrix to using dtos instead of whole cdm entities added

#13 Updated by Katja Luther over 1 year ago

  • Assignee changed from Katja Luther to Andreas Kohlbecker

I added the relevant tickets

#14 Updated by Andreas Kohlbecker over 1 year ago

  • Assignee changed from Andreas Kohlbecker to Katja Luther

Thank you, I think we can close this ticket now.

#15 Updated by Katja Luther over 1 year ago

  • Status changed from Resolved to Closed

#16 Updated by Katja Luther 8 days ago

  • Status changed from Closed to In Progress

#17 Updated by Andreas Müller 8 days ago

If we reopen then we should adapt the milestone. But maybe it is better to keep this closed and create a new ticket as the problem is probably not as big as before anymore (if it exists at all, which is not sure).

#18 Updated by Andreas Kohlbecker 8 days ago

  • Status changed from In Progress to Feedback

Was comment 16 ("changed from Closed to In Progress") a mistake, it completely misses any explanation?
I am assuming that this ticket is fully solved.

#19 Updated by Katja Luther 8 days ago

  • Status changed from Feedback to Closed

Actually maybe this was the wrong ticket because the exception coming up again was not an OutOfMemoryException but an OptionalDataException. I think we should open a new ticket for this.

#20 Updated by Andreas Müller 7 days ago

Katja Luther wrote:

Actually maybe this was the wrong ticket because the exception coming up again was not an OutOfMemoryException but an OptionalDataException. I think we should open a new ticket for this.

Did you open a new ticket? I can't see any.

Also OptionalDataException is probably related to OutOfMemoryException as you (see #9358) so there is a good chance that there is the same reason for both problems. Please relate new ticket to both this ticket and #9358.

#21 Updated by Katja Luther 7 days ago

  • Related to bug #9820: OptionalDataException when saving character matrix added

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 40 MB)