Project

General

Profile

Actions

task #7446

open

Check if all extensions are needed

Added by Andreas Müller almost 6 years ago. Updated over 5 years ago.

Status:
New
Priority:
Priority14
Category:
cdmadapter
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
5:00 h
Severity:
normal
Tags:

Description

about 560.000 Extensions are currently imported.

We need to check if they are all needed or if some of them are duplicates. Also check if they should better run into alternative identifiers.

Once done we need to check if they should be shown on the dataportal.

SELECT  e.type_id, et.titleCache, ab.DTYPE, r.refType,  dtb.DTYPE, count(*) as n
FROM Extension e INNER JOIN DefinedTermBase et ON et.id = e.type_id
LEFT OUTER JOIN AgentBase_Extension abMN ON abMN.extensions_id = e.id
LEFT OUTER JOIN AgentBase ab ON ab.id = abMN.AgentBase_id
LEFT OUTER JOIN DefinedTermBase_Extension dtbMN ON dtbMN.extensions_id = e.id
LEFT OUTER JOIN DefinedTermBase dtb ON dtb.id = dtbMN.DefinedTermBase_id
LEFT OUTER JOIN Reference_Extension rMN ON rMN.extensions_id = e.id
LEFT OUTER JOIN Reference r ON r.id = rMN.Reference_id

GROUP BY type_id, et.titleCache,  ab.DTYPE, r.refType,  dtb.DTYPE
ORDER BY et.titleCache, ab.DTYPE, r.refType,  dtb.DTYPE, n
SELECT  e.type_id, e.value, et.titleCache, tb.DTYPE, r.refType, r.id, r.titleCache,  dtb.DTYPE
FROM Extension e INNER JOIN DefinedTermBase et ON et.id = e.type_id
LEFT OUTER JOIN AgentBase_Extension abMN ON abMN.extensions_id = e.id
LEFT OUTER JOIN AgentBase ab ON ab.id = abMN.AgentBase_id
LEFT OUTER JOIN DefinedTermBase_Extension dtbMN ON dtbMN.extensions_id = e.id
LEFT OUTER JOIN DefinedTermBase dtb ON dtb.id = dtbMN.DefinedTermBase_id
LEFT OUTER JOIN Reference_Extension rMN ON rMN.extensions_id = e.id
LEFT OUTER JOIN Reference r ON r.id = rMN.Reference_id
LEFT OUTER JOIN TaxonBase_Extension tbMN ON tbMN.extensions_id = e.id
LEFT OUTER JOIN TaxonBase tb ON tb.id = tbMN.TaxonBase_id
LEFT OUTER JOIN TaxonName_Extension nMN ON nMN.extensions_id = e.id
LEFT OUTER JOIN TaxonName n ON n.id = nMN.TaxonName_id
WHERE et.titleCache like '%Nomenclatural Standard%'
ORDER BY e.value, r.id

Done:

  • common name reference language deduplicated

TODO:

  • remove experts and species experts, check if same as sec ref now and adapt PESI import to use sec in case of E+M
  • Berlin Model IdInSource currently only for Synonyms, where do they come from, can they be moved to alternative identifiers, why do accepted taxa not have this id, is the semantics always the same, aren't there similar fields in other BM tables?
  • very few references have DateString extensions, this should be unified with similar information in Reference.notes, RefDetail notes, and other fields with similar information
  • Nomenclatural Standard can often be handled as BPH or TL/2 (alternative) identifier. Check how "-" should be handled (remove or does it mean no BPH or TL/2 entry exists)? what to do with BPH/S? Few exceptions exist.
  • Check what Source_Acc means for names (74.702x, 4x for taxa)
  • Handle IsoCode and TDWG code extension as term relationship, once this relationship exists
Actions #2

Updated by Andreas Müller almost 6 years ago

  • Description updated (diff)
  • Estimated time set to 5:00 h
Actions #3

Updated by Andreas Müller almost 6 years ago

  • Description updated (diff)
Actions #4

Updated by Andreas Müller almost 6 years ago

  • Description updated (diff)
Actions #5

Updated by Andreas Müller almost 6 years ago

  • Description updated (diff)
Actions #6

Updated by Andreas Müller almost 6 years ago

  • Description updated (diff)
Actions #7

Updated by Andreas Müller almost 6 years ago

  • Description updated (diff)
Actions #8

Updated by Andreas Müller over 5 years ago

  • Target version changed from Euro+Med Portal Release to Euro+Med Migration
Actions

Also available in: Atom PDF