Project

General

Profile

bug #8858

Ordering of distribution status 'undefined' is not correct in E+M

Added by Andreas Müller over 1 year ago. Updated 8 months ago.

Status:
Closed
Priority:
Highest
Category:
cdmlib
Target version:
Start date:
02/24/2020
Due date:
% Done:

100%

Severity:
normal
Found in Version:
Tags:

Description

AM

guter Hinweis. Grundsätzlich hat das nicht mit der Transmission-Engine zu tun, sondern mit der on-the-fly Berechnung, welcher Status angezeigt werden soll (erstere ist primär für die Aggregierung der Daten von einem Rang zum anderen zuständig).
Das Problem hier scheint zu sein, dass der Status „undefined“ nicht aus dem sortierten Standardvokabular kommt sondern aus einem separaten „User Defined“ Vokabular, da der Term für E+M extra angelegt wurde. Derzeit werden Terme aus verschiedenen Vokabularen deterministisch aber undefiniert sortiert. Daher läuft das hier wohl noch falsch.
Datenmodellseitig sind inzwischen die Grundlagen gelegt, dass das auch anders geht, so dass wir uns da mal hinsetzen sollten um das umzustellen. Das sollte vor dem offiziellen Release passieren.

Ich lege ein Ticket an für den E+M Fall, auch wenn das sowieso auf der Liste steht.

===

ERS:

ich frage mich gerade, weshalb bei einer Verbreitungsangabe mit unterschiedlichen Status für dasselbe Areal der "schlechtere" Status für die Ausgabe im Portal ausgewählt wird, und nicht derjenige, der laut unserer festgelegten Hierarchie den anderen "überschreiben" soll. Das sieht doch so aus, als sei der Tansmission Engine eben noch nicht gelaufen (dazu gab es unterschiedliche Aussagen, als ich letzten Donnerstag mit Walter darüber gesprochen hatte).

Beispiel:

Für Centaurea chalchidicaea und Griechenland
https://europlusmed.org/cdm_dataportal/taxon/60d8dc3b-ac65-4346-9869-85898518fc2a

wird im Portal der Status "undefined" mit einer uralten Quelle ausgegeben.

Es gibt aber einen record für dasselbe Areal mit dem Status "native" und vier neueren Quellen, der stattdessen agezeigt werden sollte (und im BerlinModell auch korrekt ausgegeben wird),

Wie kann das im CDM verwirklicht werden? Das ist eine entscheidende Voraussetzung für das Release des Portals.


Related issues

Related to Edit - feature request #3816: Create "Undefined" status and choose it for NULL status Closed 01/15/2014
Related to Edit - bug #8297: Fix condensed distribution string for E+M Closed 05/23/2019
Related to Edit - bug #9477: New ordered terms do not have correct order index Closed 02/25/2021
Copied to Edit - feature request #9500: Allow removing certain distribution status from distribution publication In Progress 03/02/2021

Associated revisions

Revision b553ef04 (diff)
Added by Andreas Müller 8 months ago

ref #8858 ordering of terms from different vocabularies use voc.uuid as vocabulary order attribute (fix ordering of "undefined" status in E+M)

Revision 8b7c16f0 (diff)
Added by Andreas Müller 8 months ago

ref #3816, ref #8858 add "undefined" to PAT vocabulary

Revision 4dccf2d6 (diff)
Added by Andreas Müller 8 months ago

fix #8858 Improve presence/absence handling for PresenceAbsenceTerm.compareTo

History

#2 Updated by Andreas Müller 8 months ago

  • Status changed from New to Resolved
  • Target version changed from Euro+Med Portal Release to Release 5.21
  • % Done changed from 0 to 40

Fixed by adding ordering by vocabulary uuid to term ordering. This is a preliminary solution until ordering via ordered termtree is not yet possible in the webservice.

We should open a new ticket to fully fix this. Also further data cleaning should be done on distributions with status undefined.

#3 Updated by Andreas Müller 8 months ago

#4 Updated by Andreas Müller 8 months ago

Another solution is implementing #3816

#5 Updated by Andreas Müller 8 months ago

  • Related to bug #8297: Fix condensed distribution string for E+M added

#6 Updated by Andreas Müller 8 months ago

  • Related to bug #9477: New ordered terms do not have correct order index added

#7 Updated by Andreas Müller 8 months ago

  • Status changed from Resolved to Closed
  • % Done changed from 40 to 100

Andreas Müller wrote:

Another solution is implementing #3816

This solution finally has been implemented. The status priority seems to work now.

#8 Updated by Andreas Müller 8 months ago

  • Related to bug #4409: fallback areas are not adressable in the shape file added

#9 Updated by Andreas Müller 8 months ago

  • Related to deleted (bug #4409: fallback areas are not adressable in the shape file)

#11 Updated by Andreas Müller 8 months ago

  • Status changed from Closed to Resolved

The reason why the first fix didn't work is that PresenceAbsenceTerm had it's own compareTo method, including an isPresent/isAbsent check. Present terms were generally preferred to absence terms which for "undefined" did not work as "undefined" is "present" as isAbsent is not a three-state variable and the state is simply not known.
Therefore even after including "undefined" into the CDM vocabulary the comparison did not work for comparing undefined with absent terms.

This is fixed now the way that the presence/absence check is only done if the vocabularies differ (the assumption that presence terms should win still seems to be more reasonable then the assumption that the vocabulary with the lower uuid should win).

Reset to review for checking if the result is as expected.
Example data can be found for Talinum paniculatum/Madeira.

#12 Updated by Andreas Müller 8 months ago

  • % Done changed from 100 to 50

#13 Updated by Andreas Müller 8 months ago

  • Copied to feature request #9500: Allow removing certain distribution status from distribution publication added

#14 Updated by Andreas Müller 8 months ago

  • Status changed from Resolved to Closed
  • % Done changed from 50 to 100

The new implementation works as expected. The result is for Talinum paniculatum on test is "-● -Md(??M) [aAu(A) pAz(aF aP) aHu -Md]" http://test.e-taxonomy.eu/dataportal/preview/euromed/cdm_dataportal/taxon/3f28aea3-829f-46dc-957a-9991c30050f4

However, further research shows that the "undefined" distributions were not shown at all in the Berlin Model system. This is because they were put into brackets, which means that they are handled as false or unreliable information.
E.g.: http://ww2.bgbm.org/EuroPlusMed/PTaxonDetailOccurrence.asp?NameId=48608&PTRefFk=7300000

So we probably need functionality to fully remove "undefined" distributions from any publication output (condensed distribution string, textual distribution and map).

This will be handled in a new ticket: #9500

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 40 MB)