Project

General

Profile

Actions

feature request #8096

closed

Implement CacheUpdater as long running task with correct progress monitoring

Added by Andreas Müller almost 4 years ago. Updated almost 4 years ago.

Status:
Closed
Priority:
Priority14
Assignee:
Category:
cdmlib-remote
Target version:
Start date:
Due date:
% Done:

100%

Estimated time:
Severity:
normal

Files

picture638-1.png (30.4 KB) picture638-1.png Patrick Plitzner, 04/03/2019 01:41 PM
picture686-1.png (4.43 KB) picture686-1.png Andreas Müller, 04/04/2019 11:13 AM
picture787-1.png (1.63 KB) picture787-1.png Andreas Müller, 04/04/2019 11:17 AM
picture455-1.png (21 KB) picture455-1.png Katja Luther, 04/05/2019 10:09 AM
picture115-1.png (2.55 KB) picture115-1.png Andreas Müller, 04/08/2019 05:48 PM

Related issues

Related to EDIT - feature request #8030: Cache updater service methods externalized and base class specificClosedAndreas Müller

Actions
Related to EDIT - feature request #8232: Show updated objects of cache updater separatelyNewKatja Luther

Actions
Copied to EDIT - bug #8237: Cache updating sometimes does not work as expected for references and namesNewAndreas Müller

Actions
Actions #1

Updated by Andreas Müller almost 4 years ago

  • Related to feature request #8030: Cache updater service methods externalized and base class specific added
Actions #2

Updated by Katja Luther almost 4 years ago

  • Status changed from New to In Progress
  • Priority changed from New to Priority14
Actions #3

Updated by Katja Luther almost 4 years ago

  • Status changed from In Progress to Resolved
  • Assignee changed from Katja Luther to Andreas Müller

please review

Actions #4

Updated by Katja Luther almost 4 years ago

  • % Done changed from 0 to 60
Actions #5

Updated by Katja Luther almost 4 years ago

  • Assignee changed from Andreas Müller to Patrick Plitzner
Actions #6

Updated by Patrick Plitzner almost 4 years ago

I tested updating caches with all options on on the lates nightly. The progress bar shows the current work done and the progress increases over time. I did not check if the caches were really updated as this functionality was already implemented.

One minor issue:

It still says "No subtask name" if you do not expand the details. This text should also be set. Can be the same as the text in the details.

Actions #7

Updated by Katja Luther almost 4 years ago

  • Status changed from Feedback to Resolved
  • Assignee changed from Katja Luther to Andreas Müller

Patrick Plitzner wrote:

I tested updating caches with all options on on the lates nightly. The progress bar shows the current work done and the progress increases over time. I did not check if the caches were really updated as this functionality was already implemented.

One minor issue:

It still says "No subtask name" if you do not expand the details. This text should also be set. Can be the same as the text in the details.

this is fixed.

Actions #8

Updated by Andreas Müller almost 4 years ago

I get a strange duplication of task and progress now:

Actions #9

Updated by Andreas Müller almost 4 years ago

When running on references (in rem_conf_am) I only get no task shown:

Actions #10

Updated by Andreas Müller almost 4 years ago

Also when rerunning on references I always get the same result (updated objects 2669), second time I would expect 0.

Can you check if there is still an error in the algorithm?

The same happens for taxon names (always 6 updated objects).

Please also check for taxa and specimens. I havn't tested yet.

Actions #11

Updated by Katja Luther almost 4 years ago

Andreas Müller wrote:

When running on references (in rem_conf_am) I only get no task shown:

now it looks like this:

Actions #12

Updated by Katja Luther almost 4 years ago

Andreas Müller wrote:

Also when rerunning on references I always get the same result (updated objects 2669), second time I would expect 0.

Can you check if there is still an error in the algorithm?

The same happens for taxon names (always 6 updated objects).

Please also check for taxa and specimens. I havn't tested yet.

Maybe it is the same issue as for teams we found a few weeks ago. I will have a look.

Actions #13

Updated by Katja Luther almost 4 years ago

Katja Luther wrote:

Andreas Müller wrote:

Also when rerunning on references I always get the same result (updated objects 2669), second time I would expect 0.

Can you check if there is still an error in the algorithm?

The same happens for taxon names (always 6 updated objects).

Please also check for taxa and specimens. I havn't tested yet.

Maybe it is the same issue as for teams we found a few weeks ago. I will have a look.

It is a problem of the cache update:

if (this.protectedAbbrevTitleCache == false){
           String oldAbbrevTitleCache = this.abbrevTitleCache;

           String newAbbrevTitleCache = cacheStrategy.getFullAbbrevTitleString(this);

           if ( oldAbbrevTitleCache == null   || ! oldAbbrevTitleCache.equals(newAbbrevTitleCache) ){
                this.setAbbrevTitleCache(null, false);
                String newCache = this.getAbbrevTitleCache();

                if (newCache == null){
                    logger.warn("New abbrevCache should never be null");
                }
                if (oldAbbrevTitleCache == null){
                    logger.info("oldAbbrevTitleCache was illegaly null and has been fixed");
                }
                result = true;
            }
        }
        return result;

first newAbbrevTitleCache is created by cacheStrategy.getFullAbbrevTitleString(this) and compared to the abbrevTitleCache of the reference and then the new cache is created by this.getAbbrevTitleCache(); and the strings are not always the same.

Actions #14

Updated by Katja Luther almost 4 years ago

the problem for references is that if cacheStrategy.getFullAbbrevTitleString(this) returns null, the abbrevTitleCache is set to titleCache and then the comparison fails because the old abbrevTitleCache is the titleCache and the computed abbrevTitleCache is null.

Actions #15

Updated by Katja Luther almost 4 years ago

  • Status changed from Feedback to Resolved

there are problems with objects having a null titleCache because they get a new titlecache while saving.
I think we should keep it like it is, because this can show the user dirty data because objects without titlecache should not exist.

Actions #16

Updated by Katja Luther almost 4 years ago

  • Assignee changed from Katja Luther to Andreas Müller
Actions #17

Updated by Andreas Müller almost 4 years ago

Katja Luther wrote:

Andreas Müller wrote:

When running on references (in rem_conf_am) I only get no task shown:

now it looks like this:

Much better now, but I get 2 colons in status list now:

(not serious)

Actions #18

Updated by Andreas Müller almost 4 years ago

  • Status changed from Resolved to Feedback
  • Assignee changed from Andreas Müller to Katja Luther

If running on multiple classes the update result returns the total number of updated objects. Better we had them separated by classes, e.g. 10 taxon names updated, 5 authors/teams updated, ...

But this may go into a new ticket if not easy to fix.

Actions #19

Updated by Andreas Müller almost 4 years ago

Katja Luther wrote:

the problem for references is that if cacheStrategy.getFullAbbrevTitleString(this) returns null, the abbrevTitleCache is set to titleCache and then the comparison fails because the old abbrevTitleCache is the titleCache and the computed abbrevTitleCache is null.

Hm, in current rem_conf_am are 19 references with empty abbrevTitleCache but 40 objects (references) are updated according to the report.

I think we should further investigate here how to best handle this issue. However, this is another ticket. Can you create a new ticket for this? If the 2 above minor issues are fixed or moved to a "remaining" issue we can close this ticket.

Actions #20

Updated by Katja Luther almost 4 years ago

Andreas Müller wrote:

Katja Luther wrote:

the problem for references is that if cacheStrategy.getFullAbbrevTitleString(this) returns null, the abbrevTitleCache is set to titleCache and then the comparison fails because the old abbrevTitleCache is the titleCache and the computed abbrevTitleCache is null.

Hm, in current rem_conf_am are 19 references with empty abbrevTitleCache but 40 objects (references) are updated according to the report.

I think we should further investigate here how to best handle this issue. However, this is another ticket. Can you create a new ticket for this? If the 2 above minor issues are fixed or moved to a "remaining" issue we can close this ticket.

The other references are references where the titleCache itself is null and during save/merge the titleCache is created by the CdmPreDataChangeListener

if (identifiableEntity.hasEmptyTitleCache()){
                    identifiableEntity.setTitleCache(identifiableEntity.toString(), false);
                }
Actions #21

Updated by Katja Luther almost 4 years ago

Andreas Müller wrote:

If running on multiple classes the update result returns the total number of updated objects. Better we had them separated by classes, e.g. 10 taxon names updated, 5 authors/teams updated, ...

But this may go into a new ticket if not easy to fix.

This needs a separation of the different kind of objects because actually all updated objects are in one list. If we want to release soon, I would create a new ticket for this issue.(-> #8232)

Actions #22

Updated by Katja Luther almost 4 years ago

  • Status changed from Feedback to Resolved
  • Assignee changed from Katja Luther to Andreas Müller

the two colons are fixed and for a better result there is a new ticket.

Actions #23

Updated by Andreas Müller almost 4 years ago

Actions #24

Updated by Andreas Müller almost 4 years ago

  • Status changed from Resolved to Closed
  • Assignee changed from Andreas Müller to Katja Luther

Moved null/empty cache issues to new ticket #8237.

All the rest works or has new ticket so we can close this one.

Actions #25

Updated by Andreas Müller almost 4 years ago

  • % Done changed from 60 to 100
Actions #26

Updated by Andreas Müller almost 4 years ago

  • Copied to bug #8237: Cache updating sometimes does not work as expected for references and names added
Actions

Also available in: Atom PDF