Project

General

Profile

Actions

task #10222

closed

Use DTOs for portal taxon page - first implementation for E+M

Added by Andreas Müller about 1 year ago. Updated 9 months ago.

Status:
Closed
Priority:
Highest
Category:
cdm-dataportal
Target version:
Start date:
Due date:
% Done:

70%

Estimated time:
Severity:
major

Description

... to improve performance and simplify dataportal rendering. If possible formatting should take place server side.

Open issues: how to pass configuration?

  • huge number of parameters
  • json configuration

Discussion about data, contained in DTO(s):

  • taxonName: should everything be handled in typedLabel or in own DTO? => typedLabel or taggedText should be enough
    • if we have an own dto, we can add rank and rankorder index, but maybe we only need the rank string if all ordering is done serverside => ordering is done serverside
    • nomenclatural source is missing
    • to create a link to the name page the uuid is needed. => added
  • name relationships => implemented -> ruleConsidered is missing => added
  • name facts => added
  • protologue links => added
  • currently the facts are merged into the feature tree (see _mergeFeatureTreeDescriptions in cdm_api.module of the dataportal code), we should handle this similar when creating the dto => implemented, see #note-27
  • the distribution facts are handled separately in dataportal, maybe we should handle them in a separate item in the dto as well.
  • error in distribution string (#note-52) => fixed when passing the statusOrderPreference param to the service
  • include description sources in description element sources
  • fallback areas not yet handled correctly in long text distributions, e.g. Caucasia (Ab+...) in Crepis => seems to be fixed in the meanwhile
  • IUCN status missing in DTO (for A. alba) => seems to be fixed in the meanwhile

Issues on portal side:

  • bibliographie
  • breadcrumbs do not work
  • open classification at current taxon position does not work
  • namerelationships (Abies nobilis A. Dietr., Fl. Berlin: 793. 1824, nom. illeg. [is earlier homonym of Abies nobilis (D. Don) Lindl. 1833] in synonymy of Abies alba)
  • adapt portal code that the default feature tree does not has a default uuid.
  • name facts
  • add setting to switch between dto usage and standard pages.
  • unnecessary additional webservice calls
  • combining misapplied name records which refer to the same name (example in Abies alba)
    • open new issue: biblio footnote for second sensu reference
  • IUCN "distributions" not shown correctly
  • correct handling of textual distributions
  • annotatins on names: https://test.e-taxonomy.eu/dataportal/preview/euromed/cdm_dataportal/taxon/a17ab06c-07ba-4743-9851-ce692e15f91b (but this is the only one in E+M)

issues not relevant for E+M handled in #10322

https://test.e-taxonomy.eu/dataportal/preview/euromed/cdm_dataportal/newtaxon/6a8e85bd-5e52-4ae2-9444-99128c87a672 (Abies alba)


Files

clipboard-202302201350-qnda3.png (25.3 KB) clipboard-202302201350-qnda3.png Andreas Müller, 02/20/2023 01:50 PM
clipboard-202305112039-ssit3.png (36.6 KB) clipboard-202305112039-ssit3.png Andreas Müller, 05/11/2023 08:39 PM
clipboard-202305112041-wxgfh.png (25.1 KB) clipboard-202305112041-wxgfh.png Andreas Müller, 05/11/2023 08:41 PM

Related issues

Related to EDIT - feature request #9271: Improve E+M dataportal performanceNewAndreas Müller

Actions
Related to EDIT - task #6208: Implement synonymy service responding with DTOIn ProgressAndreas Müller

Actions
Related to EDIT - feature request #8285: Handle taxonomically valueless taxa in data portalNewKatja Luther

Actions
Related to EDIT - bug #9396: nomenclatural status missing in typedesignations shown in synonymy NewAndreas Müller

Actions
Related to EDIT - feature request #5807: Implement sub-distribution area inclusion in TaxonService.findTaxaAndNamesByFullText()NewAndreas Müller

Actions
Related to EDIT - feature request #10127: Add full reference information to sec. references in dataportalClosedAndreas Müller

Actions
Related to EDIT - feature request #10260: Add resolve mechanism to (external) identifiersClosedAndreas Müller

Actions
Related to EDIT - task #4911: improve performance of distributionInfoFor serviceIn ProgressAndreas Müller

Actions
Related to EDIT - feature request #9503: Handle term tree of areas and distribution status for distributionInfo in dataportalClosedKatja Luther

Actions
Related to EDIT - feature request #10336: Allow defining the "et al." position in name formatters for data portalClosedKatja Luther

Actions
Related to EDIT - bug #7716: Apply description sources (references) for a DescriptionBase to all included description elementsIn ProgressAndreas Müller

Actions
Related to EDIT - bug #10360: Correct Name in Source display in portalClosedKatja Luther

Actions
Related to EDIT - bug #7781: original spelling is missing for accepted taxon in taxon pageNewKatja Luther

Actions
Copied to EDIT - feature request #10322: Use DTOs for portal taxon page (cont.)NewAndreas Müller

Actions
Actions #1

Updated by Andreas Müller about 1 year ago

Actions #2

Updated by Andreas Müller about 1 year ago

  • Status changed from New to In Progress
Actions #4

Updated by Katja Luther about 1 year ago

  • Description updated (diff)
Actions #5

Updated by Katja Luther about 1 year ago

  • Description updated (diff)
Actions #6

Updated by Katja Luther about 1 year ago

  • Description updated (diff)
Actions #7

Updated by Andreas Müller about 1 year ago

  • taxonName: should everything be handled in typedLabel or in own DTO?
    • if we have an own dto, we can add rank and rankorder index, but maybe we only need the rank string if all ordering is done serverside

Is there done any ordering via ranks in the current taxon pages?

Actions #8

Updated by Andreas Müller about 1 year ago

  • specimen: could we reuse or reorganize the already existing derivedUnit/fieldunit dtos? They are already implemented for the use in dataportal
    • there are already methods implemented to get the specimen subtree for one taxon to get the data for the specimen tree view, they already return dtos

Can you leave a note which methods are used there?

Actions #9

Updated by Katja Luther about 1 year ago

  • Description updated (diff)
Actions #11

Updated by Andreas Müller about 1 year ago

  • Description updated (diff)
Actions #12

Updated by Andreas Müller about 1 year ago

  • % Done changed from 0 to 10

taxonNode agent relationships should be implemented now. Please review. Do we need more fields (I tried to use only the absolute minimum).

Actions #13

Updated by Katja Luther about 1 year ago

  • Description updated (diff)
Actions #14

Updated by Andreas Müller about 1 year ago

  • Description updated (diff)
Actions #15

Updated by Andreas Müller about 1 year ago

Actions #16

Updated by Katja Luther about 1 year ago

  • Description updated (diff)
Actions #17

Updated by Katja Luther about 1 year ago

  • Description updated (diff)
Actions #18

Updated by Katja Luther about 1 year ago

  • Description updated (diff)
Actions #19

Updated by Katja Luther about 1 year ago

  • Description updated (diff)
Actions #20

Updated by Andreas Müller about 1 year ago

  • Related to bug #9396: nomenclatural status missing in typedesignations shown in synonymy added
Actions #21

Updated by Andreas Müller about 1 year ago

  • Related to feature request #5807: Implement sub-distribution area inclusion in TaxonService.findTaxaAndNamesByFullText() added
Actions #22

Updated by Katja Luther about 1 year ago

There is a first implementation for the dataportal available, you can have a look if you exchange the taxon with newtaxon in the url, for example https://test.e-taxonomy.eu/dataportal/preview/cichorieae/cdm_dataportal/newtaxon/b335ceee-d6c1-4c93-841d-3b4bd279d855

Actions #23

Updated by Andreas Müller about 1 year ago

  • % Done changed from 10 to 20

Katja Luther wrote:

  • the distribution facts are handled separately in dataportal, maybe we should handle them in a separate item in the dto as well.

Distribution data are now handled within the feature tree, using the same datastructure for urlmapping and condensed distribtion string as before and using a DTO representation for tree that is similar (but not equal) to the one used in the previous version.
Please leave a not if something is missing.

Actions #24

Updated by Andreas Müller about 1 year ago

  • Description updated (diff)
Actions #25

Updated by Andreas Müller about 1 year ago

  • Description updated (diff)
Actions #26

Updated by Andreas Müller about 1 year ago

  • Description updated (diff)

Katja Luther wrote:

  • currently the facts are merged into the feature tree (see _mergeFeatureTreeDescriptions in cdm_api.module of the dataportal code), we should handle this similar when creating the dto

The webservice allows passing a parameter "featureTree". This will be used to order and filter the returned factual data accordingly. (implementing soon)

Actions #27

Updated by Andreas Müller about 1 year ago

Andreas Müller wrote in #note-26:

Katja Luther wrote:

  • currently the facts are merged into the feature tree (see _mergeFeatureTreeDescriptions in cdm_api.module of the dataportal code), we should handle this similar when creating the dto

The webservice allows passing a parameter "featureTree". This will be used to order and filter the returned factual data accordingly. (implementing soon)

This is implemented now.

Actions #28

Updated by Andreas Müller about 1 year ago

  • Description updated (diff)
Actions #29

Updated by Andreas Müller about 1 year ago

  • Description updated (diff)
Actions #30

Updated by Andreas Müller about 1 year ago

  • Description updated (diff)
Actions #31

Updated by Andreas Müller about 1 year ago

  • Description updated (diff)
Actions #32

Updated by Katja Luther about 1 year ago

The dto for common name dtos are different to the other facts, therefore it is more difficult to handle it in a generic way. It is necessary to know where to find the type of the fact otherwise it is difficult to decide how to render the content.

For example for the common name there is no typedLabel.

Actions #33

Updated by Andreas Müller about 1 year ago

I implemented basic error handling. So the message field can be evaluated now and empty results should not happen anymore.

Actions #34

Updated by Andreas Müller about 1 year ago

  • Description updated (diff)
Actions #35

Updated by Katja Luther about 1 year ago

To create the same html code like before it is necessary to provide the feature uuid for the descriptions.

Actions #36

Updated by Katja Luther about 1 year ago

The order of the (homotypic?) synonyms is not correct, for example for Aizoon hispanicum L.(Cyprus) the order of the synonyms should be

  • Aizoanthemum hispanicum (L.) H. E. K. Hartmann
  • Aizoanthemopsis hispanica (L.) Klak

but is the other way round.

Actions #37

Updated by Andreas Müller about 1 year ago

Katja Luther wrote in #note-35:

To create the same html code like before it is necessary to provide the feature uuid for the descriptions.

I do not fully understand what you mean here. What is "the feature uuid for the description"? A description does not have feature. Do you mean fact/description element? Why can't you take this form the factual data tree which is ordered first by feature (including uuid) and then by it's attached factual data. So each facts has a feature uuid attached via it's "parent feature".
If this does not answer your request please add an example html to make clear how the feature uuid is used.

Actions #38

Updated by Katja Luther about 1 year ago

Andreas Müller wrote in #note-37:

Katja Luther wrote in #note-35:

To create the same html code like before it is necessary to provide the feature uuid for the descriptions.

I do not fully understand what you mean here. What is "the feature uuid for the description"? A description does not have feature. Do you mean fact/description element? Why can't you take this form the factual data tree which is ordered first by feature (including uuid) and then by it's attached factual data. So each facts has a feature uuid attached via it's "parent feature".
If this does not answer your request please add an example html to make clear how the feature uuid is used.

Sorry that was the wrong term, I meant the uuid of the feature of the facts. This should be added to the object containing the facts of one feature, actually only the label of the feature is contained.

Actions #39

Updated by Andreas Müller about 1 year ago

Katja Luther wrote in #note-38:

Andreas Müller wrote in #note-37:

Katja Luther wrote in #note-35:

To create the same html code like before it is necessary to provide the feature uuid for the descriptions.

I do not fully understand what you mean here. What is "the feature uuid for the description"? A description does not have feature. Do you mean fact/description element? Why can't you take this form the factual data tree which is ordered first by feature (including uuid) and then by it's attached factual data. So each facts has a feature uuid attached via it's "parent feature".
If this does not answer your request please add an example html to make clear how the feature uuid is used.

Sorry that was the wrong term, I meant the uuid of the feature of the facts. This should be added to the object containing the facts of one feature, actually only the label of the feature is contained.

I still do not understand why you say that it is not there. It is available via the tree. E.g.

Example from https://test.e-taxonomy.eu/cdmserver/cyprus/portal/taxon/a4637006-8ea3-443b-b4e6-b6f66884e7bc/page

Repeating it in the fact itself is possbile but redundant which I would like to avoid in general. If there is an important reason to have this redundancy please explain. If really needed we can add it ofcourse.

Actions #40

Updated by Andreas Müller about 1 year ago

Katja Luther wrote in #note-36:

The order of the (homotypic?) synonyms is not correct, for example for Aizoon hispanicum L.(Cyprus) the order of the synonyms should be

  • Aizoanthemum hispanicum (L.) H. E. K. Hartmann
  • Aizoanthemopsis hispanica (L.) Klak

but is the other way round.

Hmm, this is strange. The PortalDtoLoader uses a comparator (eu.etaxonomy.cdm.compare.taxon.TaxonComparator). This comparator results in the "newtaxon" ordering. Which in IMO is correct as the only difference between the 2 synonyms is the alphabetic string which should have Aizoanthemopsis before Aizoanthemum (no difference in date, rank or status). However, I can see that on the old taxon page AND in the taxEditor it is ordered the other way round.
Do you know which comparator is used e.g. in taxEditor or for the old pages? And if it is the TaxonComparator class which configuration it uses (with rank/without rank)? And could you debug the ordering in taxEditor to see why exactly it orderes the names the other way round?

Actions #41

Updated by Katja Luther about 1 year ago

  • Description updated (diff)
Actions #42

Updated by Andreas Müller about 1 year ago

  • Description updated (diff)

Andreas Müller wrote:

  • name relationships => implemented -> ruleConsidered is missing

added

Actions #43

Updated by Andreas Müller about 1 year ago

Actions #44

Updated by Andreas Müller about 1 year ago

  • Description updated (diff)
Actions #45

Updated by Katja Luther about 1 year ago

  • Description updated (diff)
Actions #46

Updated by Andreas Müller about 1 year ago

  • Description updated (diff)
Actions #47

Updated by Andreas Müller 12 months ago

  • Description updated (diff)
Actions #48

Updated by Andreas Müller 12 months ago

  • Description updated (diff)
Actions #49

Updated by Andreas Müller 12 months ago

  • Description updated (diff)
Actions #50

Updated by Andreas Müller 12 months ago

Actions #51

Updated by Katja Luther 11 months ago

Andreas Müller wrote in #note-26:

Katja Luther wrote:

  • currently the facts are merged into the feature tree (see _mergeFeatureTreeDescriptions in cdm_api.module of the dataportal code), we should handle this similar when creating the dto

The webservice allows passing a parameter "featureTree". This will be used to order and filter the returned factual data accordingly. (implementing so

Andreas Müller wrote in #note-27:

Andreas Müller wrote in #note-26:

Katja Luther wrote:

  • currently the facts are merged into the feature tree (see _mergeFeatureTreeDescriptions in cdm_api.module of the dataportal code), we should handle this similar when creating the dto

The webservice allows passing a parameter "featureTree". This will be used to order and filter the returned factual data accordingly. (implementing soon)

This is implemented now.

Also on portal side the usage of the feature tree parameter is implemented.

Actions #52

Updated by Andreas Müller 10 months ago

The status preference in condensed distribution string does not seem to work yet. For abies alba it shows cGa(F) while in production it is Ga(F). There is a native and a cultivated record for Ga(F)

Actions #53

Updated by Andreas Müller 10 months ago

  • Description updated (diff)
Actions #54

Updated by Andreas Müller 10 months ago

  • Description updated (diff)
Actions #55

Updated by Andreas Müller 10 months ago

Andreas Müller wrote in #note-40:

Katja Luther wrote in #note-36:

The order of the (homotypic?) synonyms is not correct, for example for Aizoon hispanicum L.(Cyprus) the order of the synonyms should be

  • Aizoanthemum hispanicum (L.) H. E. K. Hartmann
  • Aizoanthemopsis hispanica (L.) Klak

but is the other way round.

Hmm, this is strange. The PortalDtoLoader uses a comparator (eu.etaxonomy.cdm.compare.taxon.TaxonComparator). This comparator results in the "newtaxon" ordering. Which in IMO is correct as the only difference between the 2 synonyms is the alphabetic string which should have Aizoanthemopsis before Aizoanthemum (no difference in date, rank or status). However, I can see that on the old taxon page AND in the taxEditor it is ordered the other way round.
Do you know which comparator is used e.g. in taxEditor or for the old pages? And if it is the TaxonComparator class which configuration it uses (with rank/without rank)? And could you debug the ordering in taxEditor to see why exactly it orderes the names the other way round?

KL: Das scheint behoben zu sein, ich habe zumindest bei dem im Ticket angeführten Fall und noch einem anderen Taxon keine falsche Reihenfolge mehr gesehen.

Actions #56

Updated by Andreas Müller 10 months ago

  • Description updated (diff)
Actions #57

Updated by Andreas Müller 10 months ago

  • Description updated (diff)
  • % Done changed from 20 to 30
Actions #58

Updated by Andreas Müller 10 months ago

  • Description updated (diff)
Actions #59

Updated by Andreas Müller 10 months ago

  • Description updated (diff)
Actions #60

Updated by Katja Luther 10 months ago

  • Description updated (diff)
Actions #61

Updated by Katja Luther 10 months ago

mail AM:

ich habe dem Service jetzt noch einige wichtige Parameter spendiert, die meisten von distributionInfoFor kopiert. Könntest du versuchen, die entsprechend portalseitig zu übergeben.
Lediglich „hiddenAreaMarkerTypes“ habe ich in „fallbackArea….“ Umgenannt, da das inzwischen besser passt. Ist aber der gleiche Parameter.

Actions #62

Updated by Andreas Müller 10 months ago

  • Description updated (diff)
Actions #63

Updated by Katja Luther 10 months ago

  • Description updated (diff)
Actions #64

Updated by Katja Luther 10 months ago

  • Description updated (diff)
Actions #65

Updated by Katja Luther 10 months ago

  • Description updated (diff)
Actions #66

Updated by Katja Luther 10 months ago

  • Description updated (diff)
Actions #67

Updated by Andreas Müller 10 months ago

  • Description updated (diff)
Actions #68

Updated by Andreas Müller 10 months ago

  • Description updated (diff)
Actions #69

Updated by Katja Luther 10 months ago

  • Description updated (diff)
Actions #70

Updated by Katja Luther 10 months ago

  • Description updated (diff)
Actions #71

Updated by Katja Luther 10 months ago

  • Description updated (diff)
Actions #72

Updated by Andreas Müller 10 months ago

nom. illeg. is now shown twice:

Actions #73

Updated by Andreas Müller 10 months ago

Name in source is now also shown if it is equal to the accepted name. That differs from the old version. I have not checked if this configurable or always the case.

Actions #74

Updated by Andreas Müller 10 months ago

  • Description updated (diff)

Open classification at current taxon position does not yet work

Actions #75

Updated by Andreas Müller 10 months ago

  • Description updated (diff)
Actions #76

Updated by Andreas Müller 10 months ago

  • Description updated (diff)
Actions #77

Updated by Andreas Müller 10 months ago

  • Description updated (diff)
Actions #78

Updated by Katja Luther 10 months ago

Andreas Müller wrote in #note-73:

Name in source is now also shown if it is equal to the accepted name. That differs from the old version. I have not checked if this configurable or always the case.

this should be fixed

Actions #79

Updated by Katja Luther 10 months ago

Andreas Müller wrote in #note-72:

nom. illeg. is now shown twice:

this should be fixed

Actions #80

Updated by Katja Luther 10 months ago

  • Description updated (diff)
Actions #81

Updated by Katja Luther 10 months ago

  • Description updated (diff)
Actions #82

Updated by Katja Luther 10 months ago

  • Description updated (diff)
Actions #83

Updated by Andreas Müller 10 months ago

  • Subject changed from Use DTOs for portal taxon page to Use DTOs for portal taxon page - first implementation for E+M
  • Description updated (diff)
Actions #84

Updated by Andreas Müller 10 months ago

  • Tags changed from performance to performance, euro+med
  • Description updated (diff)
Actions #85

Updated by Andreas Müller 10 months ago

  • Description updated (diff)
Actions #86

Updated by Katja Luther 10 months ago

  • Description updated (diff)
Actions #87

Updated by Andreas Müller 10 months ago

  • Description updated (diff)
Actions #88

Updated by Andreas Müller 10 months ago

  • Description updated (diff)
Actions #89

Updated by Katja Luther 9 months ago

  • Description updated (diff)
Actions #90

Updated by Katja Luther 9 months ago

  • Description updated (diff)

With the current implementation the textual distribution

Actions #91

Updated by Katja Luther 9 months ago

  • Description updated (diff)
Actions #92

Updated by Andreas Müller 9 months ago

  • Description updated (diff)
Actions #93

Updated by Andreas Müller 9 months ago

  • Description updated (diff)
Actions #94

Updated by Andreas Müller 9 months ago

  • Description updated (diff)
Actions #95

Updated by Andreas Müller 9 months ago

Actions #96

Updated by Andreas Müller 9 months ago

  • Status changed from In Progress to Resolved
  • Target version changed from Release 5.44 to Release 5.38
  • % Done changed from 30 to 70
Actions #97

Updated by Andreas Müller 9 months ago

  • Related to task #4911: improve performance of distributionInfoFor service added
Actions #98

Updated by Andreas Müller 9 months ago

  • Status changed from Resolved to In Progress
  • % Done changed from 70 to 60

Looks like the name relationship filter is not yet implemented, neither on portal side when calling the webservice nor on server side

Actions #99

Updated by Andreas Müller 9 months ago

  • Related to feature request #9503: Handle term tree of areas and distribution status for distributionInfo in dataportal added
Actions #100

Updated by Andreas Müller 9 months ago

Andreas Müller wrote in #note-98:

Looks like the name relationship filter is not yet implemented, neither on portal side when calling the webservice nor on server side

This was partly a mistake. The name relationships filter exists on dataportal side (php). Only the possible prefiltering serverside is not implemented yet. But this is a minor issue.

Actions #101

Updated by Andreas Müller 9 months ago

  • Assignee changed from Andreas Müller to Katja Luther

However, taxon annotations are not yet shown but they were in old version. Maybe I was wrong when I was saying recently that they are also not implemented in the old version. So they should still be implemented before closing this ticket.

Actions #102

Updated by Andreas Müller 9 months ago

  • Description updated (diff)
Actions #103

Updated by Andreas Müller 9 months ago

  • Related to feature request #10336: Allow defining the "et al." position in name formatters for data portal added
Actions #104

Updated by Andreas Müller 9 months ago

  • Related to bug #7716: Apply description sources (references) for a DescriptionBase to all included description elements added
Actions #105

Updated by Andreas Müller 9 months ago

  • Description updated (diff)
  • Status changed from In Progress to Resolved
  • % Done changed from 60 to 90

All issues relevant for E+M seem to be fixed. From my side we can close this ticket. Please do the final review.

Actions #106

Updated by Andreas Müller 9 months ago

  • % Done changed from 90 to 70
Actions #107

Updated by Katja Luther 9 months ago

Andreas Müller wrote in #note-101:

However, taxon annotations are not yet shown but they were in old version. Maybe I was wrong when I was saying recently that they are also not implemented in the old version. So they should still be implemented before closing this ticket.

This is fixed.

Actions #108

Updated by Katja Luther 9 months ago

  • Status changed from Resolved to Closed
  • Assignee changed from Katja Luther to Andreas Müller

I tested some random pages of euro+med and I could not find any error messages or problems, I also tested some pages of cyprus, cichorieae and flora of cuba to find potential regressions in the current implementation. I think we can close this ticket.

Actions #109

Updated by Andreas Müller 8 months ago

  • Related to bug #10360: Correct Name in Source display in portal added
Actions #110

Updated by Andreas Müller about 2 months ago

  • Related to bug #7781: original spelling is missing for accepted taxon in taxon page added
Actions

Also available in: Atom PDF