Project

General

Profile

bug #8754

DescriptionElements are not deleted with the Descriptive Data Set

Added by Andreas Kohlbecker 8 months ago. Updated about 2 months ago.

Status:
Resolved
Priority:
New
Category:
taxeditor
Target version:
Start date:
12/06/2019
Due date:
% Done:

30%

Severity:
normal
Found in Version:
Tags:

Description

Based on the dump and situation in #8752:

  • deleted both descriptive datasets
  • delete the Character trees (for problems during this step see the ticket from which this one has been copied #8753)
  • try to delete a Character term

This fails since the terms are still being used in description elements (QuatitativeData in case of bract length)

picture499-1.png View (8.29 KB) Andreas Müller, 06/19/2020 05:12 PM

picture985-1.png View (4.12 KB) Andreas Müller, 06/19/2020 06:51 PM

picture985-2.png View (1.63 KB) Andreas Müller, 06/19/2020 06:51 PM

picture985-3.png View (3.05 KB) Andreas Müller, 06/19/2020 06:51 PM


Related issues

Blocked by Edit - bug #9053: CCE when selecting Character Tree for descriptive dataset Closed 06/04/2020
Copied from Edit - bug #8753: EagerLoadingException on opening Character Tree Editor after deleting all Desciptive Data Sets Closed 12/06/2019

Associated revisions

Revision af46e1a1 (diff)
Added by Katja Luther 3 months ago

ref #8754: implement delete method for descriptiveDataSet

Revision e423ad05 (diff)
Added by Katja Luther about 2 months ago

ref #8754: add deleteConfig for descriptiveDataset

Revision 08a36bdf (diff)
Added by Katja Luther about 2 months ago

ref #8754: adapt deleteHandling to configuration

Revision 97a2bbb9 (diff)
Added by Katja Luther about 2 months ago

ref #8754: adapt description for delete descriptions

History

#1 Updated by Andreas Kohlbecker 8 months ago

  • Copied from bug #8753: EagerLoadingException on opening Character Tree Editor after deleting all Desciptive Data Sets added

#2 Updated by Andreas Kohlbecker 8 months ago

  • Description updated (diff)

#3 Updated by Katja Luther 7 months ago

  • Target version changed from Release 5.12 to Release 5.13

#4 Updated by Andreas Müller 5 months ago

  • Target version changed from Release 5.13 to Release 5.14

#5 Updated by Andreas Müller 4 months ago

  • Target version changed from Release 5.14 to Release 5.15

#6 Updated by Katja Luther 3 months ago

Deleting the descriptive dataset calls the delete method of IdentifiableServiceBase, but should be an extra delete method to handle the deletion of the descriptions.

#7 Updated by Katja Luther 3 months ago

  • Status changed from New to Resolved
  • Assignee changed from Katja Luther to Andreas Müller

A new method for deletion of descriptive datasets are implemented, which uses DescriptionService.deleteDescription() for the deletion of descriptions and when the description is deleted, the description elements are deleted by cascading.

please review.

#8 Updated by Katja Luther 3 months ago

  • Status changed from Resolved to In Progress

the deletion of larger datasets takes some time, therefore it is implemented as a long running task now. Need to test, so I take the ticket back to me.

#9 Updated by Katja Luther 3 months ago

  • Assignee changed from Andreas Müller to Katja Luther

#10 Updated by Katja Luther 3 months ago

  • Status changed from In Progress to Resolved
  • Assignee changed from Katja Luther to Andreas Müller

please review.

#11 Updated by Andreas Müller 2 months ago

  • Description updated (diff)

#12 Updated by Andreas Müller 2 months ago

  • Description updated (diff)

#13 Updated by Andreas Müller 2 months ago

  • Blocked by bug #9053: CCE when selecting Character Tree for descriptive dataset added

#14 Updated by Andreas Müller 2 months ago

When trying to delete a descriptive dataset I got the following exception. Though it seems as if the delete was executed (data did not exist anymore).

last remote method : http://test.e-taxonomy.eu:80/cdmserver/rem_conf_am/remoting/progressmonitor.service
last remote request client time : 2020-06-04T09:28:55.063
last remote request response header time : Thu, 04 Jun 2020 09:28:55 GMT
client error time : 2020-06-04T09:28:55.11
login : admin
editor version : 5.15.0.202006032253
server : test.e-taxonomy.eu (edit-test) / rem_conf_am
schema version : 5.15.0.0.20200510
os : Windows Server 2012 R2 6.3 amd64
java : 1.8.0_121
org.springframework.dao.InvalidDataAccessResourceUsageException: could not execute statement; SQL [n/a]; nested exception is org.hibernate.exception.SQLGrammarException: could not execute statement
    at org.springframework.orm.hibernate5.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:147)
    at org.springframework.orm.hibernate5.HibernateTransactionManager.convertHibernateAccessException(HibernateTransactionManager.java:741)
    at org.springframework.orm.hibernate5.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:589)
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:761)
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:730)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:485)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:291)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
    at com.sun.proxy.$Proxy649.delete(Unknown Source)
    at eu.etaxonomy.cdm.api.service.longrunningService.LongRunningTasksService$5.doRun(LongRunningTasksService.java:155)
    at eu.etaxonomy.cdm.common.monitor.RemotingProgressMonitorThread.run(RemotingProgressMonitorThread.java:42)
Caused by: org.hibernate.exception.SQLGrammarException: could not execute statement
    at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:106)
    at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42)
    at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:109)
    at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:95)
    at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:207)
    at org.hibernate.engine.jdbc.batch.internal.NonBatchingBatch.addToBatch(NonBatchingBatch.java:45)
    at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2886)
    at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3386)
    at org.hibernate.action.internal.EntityInsertAction.execute(EntityInsertAction.java:89)
    at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:560)
    at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:434)
    at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:337)
    at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
    at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1282)
    at org.hibernate.envers.internal.synchronization.AuditProcess.doBeforeTransactionCompletion(AuditProcess.java:156)
    at org.hibernate.envers.internal.synchronization.AuditProcessManager$1.doBeforeTransactionCompletion(AuditProcessManager.java:46)
    at org.hibernate.engine.spi.ActionQueue$BeforeTransactionCompletionProcessQueue.beforeTransactionCompletion(ActionQueue.java:899)
    at org.hibernate.engine.spi.ActionQueue.beforeTransactionCompletion(ActionQueue.java:481)
    at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:2340)
    at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:485)
    at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:147)
    at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.access$100(JdbcResourceLocalTransactionCoordinatorImpl.java:38)
    at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:231)
    at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:65)
    at org.springframework.orm.hibernate5.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:581)
    ... 10 more
Caused by: java.sql.SQLException: No value specified for parameter 6
    at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:129)
    at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
    at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
    at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1132)
    at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1057)
    at com.mysql.cj.jdbc.ClientPreparedStatement.executeLargeUpdate(ClientPreparedStatement.java:1377)
    at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdate(ClientPreparedStatement.java:1042)
    at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeUpdate(NewProxyPreparedStatement.java:384)
    at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:204)
    ... 30 more

#15 Updated by Andreas Müller 2 months ago

  • Status changed from Resolved to Feedback
  • Assignee changed from Andreas Müller to Katja Luther
  • % Done changed from 0 to 20

Generally I have problems to understand this ticket.

Do I understand it correctly that the expectation is that deleting the DDS will also delete all specimen and taxon descriptions including their description elements?

This is critical because descriptions are not primarily related to DDSs but to their specimen or taxon. So only deleting the specimen or the taxon is expected to automatically delete the description.

When deleting a DDS we need a configurator asking if 1) all specimen descriptions should not only be removed from the DDS but also be deleted completely, 2) all "literature" descriptions should be deleted completely, 3) all "default" descriptions should be deleted completely, 4) all aggregated descriptions should be deleted completely (and maybe in future 5. all keys should be deleted completely).

For 4) this is the expected behavior but for all others it is not necessarily the case.

This issue is maybe currently not yet so clear because at the moment it is not yet possible to use existing descriptions for a DDS. But this should be possible in future especially to make it possible to reuse import result but also in other contexts.

#16 Updated by Andreas Müller about 2 months ago

Andreas Müller wrote:

Generally I have problems to understand this ticket.

Do I understand it correctly that the expectation is that deleting the DDS will also delete all specimen and taxon descriptions including their description elements?

This is critical because descriptions are not primarily related to DDSs but to their specimen or taxon. So only deleting the specimen or the taxon is expected to automatically delete the description.

When deleting a DDS we need a configurator asking if 1) all specimen descriptions should not only removed from the DDS but also deleted completely, 2) all "literature" descriptions should be deleted completely, 3) all "default" descriptions should be deleted completely, 4) all aggregated descriptions should be deleted completely (and maybe in future 5. all keys should be deleted completely).

For 4) this is the expected behavior but for all others it is not necessarily the case.

This issue is maybe currently not yet so clear because at the moment it is not yet possible to use existing descriptions for a DDS. But this should be possible in future especially to make it possible to reuse import result but also in other contexts.

Katja, Andreas K., can you please have a look on my comment?

#17 Updated by Katja Luther about 2 months ago

Andreas Müller wrote:

Andreas Müller wrote:

Generally I have problems to understand this ticket.

Do I understand it correctly that the expectation is that deleting the DDS will also delete all specimen and taxon descriptions including their description elements?

This is critical because descriptions are not primarily related to DDSs but to their specimen or taxon. So only deleting the specimen or the taxon is expected to automatically delete the description.

When deleting a DDS we need a configurator asking if 1) all specimen descriptions should not only removed from the DDS but also deleted completely, 2) all "literature" descriptions should be deleted completely, 3) all "default" descriptions should be deleted completely, 4) all aggregated descriptions should be deleted completely (and maybe in future 5. all keys should be deleted completely).

For 4) this is the expected behavior but for all others it is not necessarily the case.

This issue is maybe currently not yet so clear because at the moment it is not yet possible to use existing descriptions for a DDS. But this should be possible in future especially to make it possible to reuse import result but also in other contexts.

Katja, Andreas K., can you please have a look on my comment?

I think you are right and I implement the delete method right now with a configurator.

#18 Updated by Andreas Kohlbecker about 2 months ago

I don't exactly remember the situation out of which I wrote this tickt, but from my current point of view the solution suggested by you Andreas and implemented by Katja seems to be the correct way to transparently handle this.

#19 Updated by Katja Luther about 2 months ago

  • Status changed from Feedback to Resolved
  • Assignee changed from Katja Luther to Andreas Müller

now the configuration is available. please review

#20 Updated by Andreas Müller about 2 months ago

  • Status changed from Resolved to Feedback
  • Assignee changed from Andreas Müller to Katja Luther

Katja Luther wrote:

now the configuration is available. please review

Something still is wrong here. What I did:

Use AM_test_dataset in rem_conf_am and look at Nepenthes morganiana.

  • At beginning 2 specimen descriptions were there for specimens "Germany, outside" and "no information" (which is a pure Derived Unit without FieldUnit)
  • when selecting delete, no delete warning with configuration possibilities appears but only the record is deleted from the matrix without warning. Sometimes even both records were deleted though the focus was only at 1 of them; I expected exactly 1 record to be deleted and a dialog with "do you really want ..." and with the possibility to configure if the record should be delete from the specimen, too.

#21 Updated by Katja Luther about 2 months ago

Andreas Müller wrote:

Katja Luther wrote:

now the configuration is available. please review

Something still is wrong here. What I did:

Use AM_test_dataset in rem_conf_am and look at Nepenthes morganiana.

  • At beginning 2 specimen descriptions were there for specimens "Germany, outside" and "no information" (which is a pure Derived Unit without FieldUnit)
  • when selecting delete, no delete warning with configuration possibilities appears but only the record is deleted from the matrix without warning. Sometimes even both records were deleted though the focus was only at 1 of them; I expected exactly 1 record to be deleted and a dialog with "do you really want ..." and with the possibility to configure if the record should be delete from the specimen, too.

Did you try to delete the descriptive dataset? This is what the ticket is about.
For removing descriptions from the matrix there is the ticket #8785.

#22 Updated by Andreas Müller about 2 months ago

  • % Done changed from 20 to 30

There is also another strange behavior. When I do "reattache" the specimen to the matrix the old record seems to be reattached (which might be a good idea but needs to be discussed for details). But the problem seems to be that if in the meanwhile the record has been changed this seem not be reflected in the UI or in the record itself. It looks like the old version is somewhere cached and attached as it was instead of reloading it from the DB.

... hmm, I tested now for a relatively long time with Matrix Editor open and deleting and reattaching Specimen Descriptions and at the same time the Specimen Tree Editor was open and sometimes I edited data there sometimes I only used it for checking referencing objects and ids. I sometimes only closed these editor sometimes closed the TaxEditor application and restartet. I was working on rem_conf_am / Am test dataset with specimen attached to Nepenthes morganiana.

However the behavior to me seems mostly unpredictable. Sometimes specimen descriptions are removed from the matrix which I did never remove. Sometimes other descriptions reappear after restart though I had deleted them. Sometimes measurement values that were deleted in Specimen Tree Editor reappear after reattaching a specimen to the matrix, ... I am a bit confused how to describe this because nothing is really reproducable but very often does not work as expected.

The main issue I originally wanted to test was how the deleting and adding of specimens in the matrix reacts if a possible SpecimenDescription candidate already exists. Is this reused or there always a new record created, or something else. But I couldn't really find.
However, for such a case we probably need some warning saying: there is a specimen description available already, do you want to use it? Also for imports this might be needed.

But first of all, could you please test if you can reproduce the above strange behvior (with my data or with other data)?

#23 Updated by Andreas Müller about 2 months ago

Katja Luther wrote:

Andreas Müller wrote:

Katja Luther wrote:

now the configuration is available. please review

Something still is wrong here. What I did:

Use AM_test_dataset in rem_conf_am and look at Nepenthes morganiana.

  • At beginning 2 specimen descriptions were there for specimens "Germany, outside" and "no information" (which is a pure Derived Unit without FieldUnit)
  • when selecting delete, no delete warning with configuration possibilities appears but only the record is deleted from the matrix without warning. Sometimes even both records were deleted though the focus was only at 1 of them; I expected exactly 1 record to be deleted and a dialog with "do you really want ..." and with the possibility to configure if the record should be delete from the specimen, too.

Did you try to delete the descriptive dataset? This is what the ticket is about.
For removing descriptions from the matrix there is the ticket #8785.

Upps, sorry, yes you are right. I was mixing things up here.

#24 Updated by Katja Luther about 2 months ago

Andreas Müller wrote:

There is also another strange behavior. When I do "reattache" the specimen to the matrix the old record seems to be reattached (which might be a good idea but needs to be discussed for details). But the problem seems to be that if in the meanwhile the record has been changed this seem not be reflected in the UI or in the record itself. It looks like the old version is somewhere cached and attached as it was instead of reloading it from the DB.

... hmm, I tested now for a relatively long time with Matrix Editor open and deleting and reattaching Specimen Descriptions and at the same time the Specimen Tree Editor was open and sometimes I edited data there sometimes I only used it for checking referencing objects and ids. I sometimes only closed these editor sometimes closed the TaxEditor application and restartet. I was working on rem_conf_am / Am test dataset with specimen attached to Nepenthes morganiana.

However the behavior to me seems mostly unpredictable. Sometimes specimen descriptions are removed from the matrix which I did never remove. Sometimes other descriptions reappear after restart though I had deleted them. Sometimes measurement values that were deleted in Specimen Tree Editor reappear after reattaching a specimen to the matrix, ... I am a bit confused how to describe this because nothing is really reproducable but very often does not work as expected.

The main issue I originally wanted to test was how the deleting and adding of specimens in the matrix reacts if a possible SpecimenDescription candidate already exists. Is this reused or there always a new record created, or something else. But I couldn't really find.
However, for such a case we probably need some warning saying: there is a specimen description available already, do you want to use it? Also for imports this might be needed.

But first of all, could you please test if you can reproduce the above strange behvior (with my data or with other data)?

I could not test it, yet, but maybe this is a problem of synchronisation between the two editors. I will have a look.

#25 Updated by Andreas Müller about 2 months ago

Katja Luther wrote:

I could not test it, yet, but maybe this is a problem of synchronisation between the two editors. I will have a look.

OK, thank. Of course this issue also belongs more to #8785 then to this ticket. I mixed up these 2 tickets. I will move my comments to the other ticket or link from there to here.

#26 Updated by Andreas Müller about 2 months ago

So back to original issue of this ticket.

The configurator now looks like

so the user does not know what the configuration is for.

Please change the labels and explanations in a way that it becomes clear what the expected result is.

E.g. Fully delete the following related descriptions (unchecked descriptions stay in the database attached to their taxa/specimens):

#27 Updated by Andreas Müller about 2 months ago

I moved #note-22 to #8785 as it belongs to this ticket (I added it accidentely here)

#28 Updated by Andreas Müller about 2 months ago

I watched another strange behavior about the selected taxon. I created a new DDS in rem_conf_am and added the taxon node for "Danthella test Node" which is a grand-child of Facchinia:

In the DDS editor this shows up correctly:

But in the matrix the whole tree up to Facchinia is shown:

#29 Updated by Katja Luther about 2 months ago

Andreas Müller wrote:

I watched another strange behavior about the selected taxon. I created a new DDS in rem_conf_am and added the taxon node for "Danthella test Node" which is a grand-child of Facchinia:

In the DDS editor this shows up correctly:

But in the matrix the whole tree up to Facchinia is shown:

This depends on the rank defined in the descriptive dataset. If you do not define a max rank, the whole tree is shown.

#30 Updated by Andreas Müller about 2 months ago

Katja Luther wrote:

This depends on the rank defined in the descriptive dataset. If you do not define a max rank, the whole tree is shown.

Upps, you are right. But I wonder if this makes sense? We need to discuss this with users (Norbert). Maybe there was a discussion on this during additivity, I somehow remember, so we may have a look to the wiki. To me it doesn't sound reasonable. If a subtree filter is defined why should we also want to show the higher taxa? Maybe the common parent for a set of subtrees still makes sense.
Hmm, maybe it was for the default descriptions to be able to add default descriptions on a higher level.
This needs to be discussed but it is definetly a new ticket as this is not a regression (it was like this before).

#31 Updated by Katja Luther about 2 months ago

Andreas Müller wrote:

So back to original issue of this ticket.

The configurator now looks like

so the user does not know what the configuration is for.

Please change the labels and explanations in a way that it becomes clear what the expected result is.

E.g. Fully delete the following related descriptions (unchecked descriptions stay in the database attached to their taxa/specimens):

The description of the dialog is adapted.

#32 Updated by Katja Luther about 2 months ago

  • Assignee changed from Katja Luther to Andreas Müller

#33 Updated by Andreas Müller about 2 months ago

  • Status changed from Feedback to Resolved

#34 Updated by Katja Luther about 2 months ago

Katja Luther wrote:

Andreas Müller wrote:

There is also another strange behavior. When I do "reattache" the specimen to the matrix the old record seems to be reattached (which might be a good idea but needs to be discussed for details). But the problem seems to be that if in the meanwhile the record has been changed this seem not be reflected in the UI or in the record itself. It looks like the old version is somewhere cached and attached as it was instead of reloading it from the DB.

... hmm, I tested now for a relatively long time with Matrix Editor open and deleting and reattaching Specimen Descriptions and at the same time the Specimen Tree Editor was open and sometimes I edited data there sometimes I only used it for checking referencing objects and ids. I sometimes only closed these editor sometimes closed the TaxEditor application and restartet. I was working on rem_conf_am / Am test dataset with specimen attached to Nepenthes morganiana.

However the behavior to me seems mostly unpredictable. Sometimes specimen descriptions are removed from the matrix which I did never remove. Sometimes other descriptions reappear after restart though I had deleted them. Sometimes measurement values that were deleted in Specimen Tree Editor reappear after reattaching a specimen to the matrix, ... I am a bit confused how to describe this because nothing is really reproducable but very often does not work as expected.

The main issue I originally wanted to test was how the deleting and adding of specimens in the matrix reacts if a possible SpecimenDescription candidate already exists. Is this reused or there always a new record created, or something else. But I couldn't really find.
However, for such a case we probably need some warning saying: there is a specimen description available already, do you want to use it? Also for imports this might be needed.

But first of all, could you please test if you can reproduce the above strange behvior (with my data or with other data)?

I could not test it, yet, but maybe this is a problem of synchronisation between the two editors. I will have a look.

this is fixed.

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 40 MB)