Project

General

Profile

task #8679

Further unify description aggregation methods

Added by Andreas Müller 8 months ago. Updated 4 months ago.

Status:
Closed
Priority:
New
Category:
cdmlib
Target version:
Start date:
11/13/2019
Due date:
% Done:

100%

Severity:
normal

Description

Common base classes already exist. Further work might be:

  • use the same calls for "aggregateWithinTaxon", "aggregateToHigherRank"
  • use TaxonNodeFilter in structured description agg. similar to distribution aggr.
  • use memory handling like in distribution agg.
  • merge source handling
  • do not create new objects if object exists already from previous aggregation
  • discuss dataset handling for distribution aggregation (DONE for DescriptionAggregation)
  • tests for includeUnpublished
  • tests for sourceType
  • fully delete empty aggregations
  • ...

Related issues

Related to Edit - feature request #8677: Add distribution aggregation to set subtree menu Closed 11/12/2019
Related to Edit - feature request #8713: Improve structured descriptive data aggregation Resolved 11/21/2019
Related to Edit - bug #8788: Cloned descriptions do not show up in dataportal anymore Feedback 12/20/2019
Follows Edit - task #8651: Unify description aggregation methods (distribution and structured descriptive data) Closed 11/05/2019
Copied to Edit - task #8871: Remaining issues to unify description aggregation methods New 03/02/2020

Associated revisions

Revision 9235bde5 (diff)
Added by Andreas Müller 8 months ago

ref #8679 fix rerun DistributionAggregation with changed sources and deleted distributions

Revision 008371ae (diff)
Added by Andreas Müller 8 months ago

ref #8679 move AggregationMode to ConfiguratorBase

Revision 7800d0d0 (diff)
Added by Andreas Müller 8 months ago

ref #8679 generic source handling with SourceModes

Revision a8496b29 (diff)
Added by Andreas Müller 8 months ago

ref #8679 improve tests

Revision 2acab840 (diff)
Added by Andreas Müller 8 months ago

ref #8679 improve monitoring

Revision b37495b9 (diff)
Added by Andreas Müller 8 months ago

ref #8679 move DistributionAggregation functionality to base class

Revision f6fda820 (diff)
Added by Andreas Müller 7 months ago

ref #8679 remove old COMPUTE marker and move ignore lists to configurator and some cleanup

Revision ad8460be (diff)
Added by Andreas Müller 7 months ago

ref #8679 preliminary fix failing test for rank aggregation with absent status

Revision d22d2ab0 (diff)
Added by Andreas Müller 7 months ago

ref #8679 add tests to StructuredDescriptionAggregation and fix toGrandParent issue and allow Feature

Revision 3177d9fb (diff)
Added by Andreas Müller 7 months ago

ref #8679 adapt TaxEditor to changes in DescriptionAggregation dataset handling

Revision 9af9bac7 (diff)
Added by Andreas Müller 7 months ago

ref #8679 adapt TaxEditor to changes in DescriptionAggregation dataset handling

Revision 4bf5063f (diff)
Added by pplitzner 7 months ago

ref #8679 Add specimen to sibling taxon

Revision 6b6ebf81 (diff)
Added by pplitzner 7 months ago

ref #8679 Refactoring

Revision eace364b (diff)
Added by pplitzner 7 months ago

ref #8679 Add leaf color

Revision cf6340d0 (diff)
Added by Patrick Plitzner 7 months ago

ref #8679 Test leaf color

Revision 1f0292f8 (diff)
Added by Patrick Plitzner 7 months ago

ref #8679 Convert to base class aggregation architecture

  • convert aggregation of categorical data within taxon

Revision 54ed2cf2 (diff)
Added by Patrick Plitzner 7 months ago

ref #8679 Add apache commons math to pom

Revision 185e63ee (diff)
Added by Patrick Plitzner 7 months ago

ref #8679 Convert to base class aggregation architecture

  • convert aggregation of quantitative data within taxon

Revision 93c52359 (diff)
Added by Patrick Plitzner 7 months ago

ref #8679 Convert to base class aggregation architecture

  • init dataset in preAccumulate
  • filter SpecimenDescriptions by data set

Revision 5ef244fc (diff)
Added by Patrick Plitzner 7 months ago

ref #8679 Convert to base class aggregation architecture

  • convert aggregation of quantitative data to parent
  • convert aggregation of categorical data to parent

Revision 7903d041 (diff)
Added by Patrick Plitzner 7 months ago

ref #8679 PropertyPath and code cleanup

Revision 33373464 (diff)
Added by Patrick Plitzner 7 months ago

ref #8679 Remove usage of apache commons math

Revision 2b226eee (diff)
Added by Patrick Plitzner 7 months ago

ref #8679 Add checks and repairs for valid quantitative data

Revision 46ae6d5a (diff)
Added by Patrick Plitzner 7 months ago

ref #8679 Set taxon node filter and aggregation mode

Revision 91f2c632 (diff)
Added by Patrick Plitzner 7 months ago

ref #8679 Load dataset in each session

Revision 45ca2f16 (diff)
Added by Patrick Plitzner 7 months ago

ref #8679 Replace existing description elements in re-aggregation

-add test for re-aggregation

Revision 9a2d6065 (diff)
Added by Patrick Plitzner 7 months ago

ref #8679 Set batchMinFreeHeap in test setup

Revision 34726589 (diff)
Added by Patrick Plitzner 7 months ago

ref #8679 Add initTransaction() super method

  • hook for initializing object when a new transaction starts

Revision 07b9b244 (diff)
Added by Patrick Plitzner 7 months ago

ref #8679 Set taxon node filter and no batch size adaption

Revision a4ca6fce (diff)
Added by Patrick Plitzner 7 months ago

ref #8679 Add sources to aggregated description

  • within taxon: specimen description
  • parent taxon: aggregated taxon description

Revision 5ee7595c (diff)
Added by Andreas Müller 7 months ago

ref #8679 move aggregation modes to own class file

Revision de4502ef (diff)
Added by Patrick Plitzner 7 months ago

ref #8679 Adapt to cdmlib changes

Revision 47992818 (diff)
Added by Andreas Müller 7 months ago

ref #8679 let AggregationMode and AggregationSourceMode implement IKeyTerm and implement some list methods

Revision 19739293 (diff)
Added by Andreas Müller 7 months ago

ref #8679 rename supports fields and make public

Revision 80c4e54e (diff)
Added by Patrick Plitzner 7 months ago

ref #8679 Filter out cloned specimen descriptions

Revision 2a71dd1c (diff)
Added by Patrick Plitzner 7 months ago

ref #8679 Add cancel points for aggregation progress monitor

Revision c6883acf (diff)
Added by pplitzner 7 months ago

ref #8679 Add test for incomplete categorical data

Revision 851664fd (diff)
Added by Andreas Müller 7 months ago

ref #8679 try to fix progress tasks and percentage not showing up

Revision 3c162754 (diff)
Added by Andreas Müller 7 months ago

ref #8679 top level error handling and renaming aggregation methods

Revision ff19d87a (diff)
Added by Andreas Müller 7 months ago

ref #8679 add done() to monitoring if unexpected exception occurred.

Revision 151c9262 (diff)
Added by Andreas Müller 7 months ago

ref #8679 fix structured desc aggregation for missing values and min/max only values

Revision e153cc8e (diff)
Added by Andreas Müller 7 months ago

ref #8679 handle null values in missing values correctly

History

#1 Updated by Andreas Müller 8 months ago

  • Due date set to 11/06/2019
  • Start date changed from 11/13/2019 to 11/06/2019
  • Follows task #8651: Unify description aggregation methods (distribution and structured descriptive data) added

#2 Updated by Andreas Müller 8 months ago

  • Description updated (diff)
  • Due date deleted (11/06/2019)
  • Start date changed from 11/06/2019 to 11/13/2019

#3 Updated by Andreas Müller 8 months ago

  • Status changed from New to In Progress

#4 Updated by Andreas Müller 8 months ago

#5 Updated by Andreas Müller 8 months ago

  • Description updated (diff)

#6 Updated by Andreas Müller 8 months ago

  • Description updated (diff)

#7 Updated by Andreas Müller 8 months ago

#8 Updated by Andreas Müller 7 months ago

  • Description updated (diff)

#9 Updated by Andreas Müller 7 months ago

  • Description updated (diff)
  • Assignee changed from Andreas Müller to Patrick Plitzner
  • % Done changed from 0 to 30

#10 Updated by Patrick Plitzner 7 months ago

  • Description updated (diff)

#11 Updated by Andreas Müller 7 months ago

  • Assignee changed from Patrick Plitzner to Andreas Müller

#12 Updated by Andreas Müller 7 months ago

  • Related to bug #8788: Cloned descriptions do not show up in dataportal anymore added

#13 Updated by Andreas Müller 6 months ago

  • Target version changed from Release 5.12 to Release 5.13

#14 Updated by Andreas Müller 4 months ago

  • Copied to task #8871: Remaining issues to unify description aggregation methods added

#15 Updated by Andreas Müller 4 months ago

  • Status changed from In Progress to Closed
  • Target version changed from Release 5.13 to Release 5.12
  • % Done changed from 30 to 100

Open issues moved to new ticket #8871

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 40 MB)