Project

General

Profile

Actions

task #9132

open

Update cyprus images to Scaler default API and add thumbnails

Added by Andreas Müller over 3 years ago. Updated over 2 years ago.

Status:
In Progress
Priority:
Highest
Category:
data
Target version:
Start date:
Due date:
% Done:

90%

Estimated time:
Severity:
normal

Description

Currently only 1 representation exists (for the full image) using the Scaler IIF API with links like:

https://pictures.bgbm.org/digilib/Scaler/IIIF/Cyprus!Melilotus_italicus_B1.jpg/full/full/0/default.jpg

We better use the Scaler default API using links like

https://pictures.bgbm.org/digilib/Scaler?fn=Cyprus/Arum_rupicola_B2.jpg&mo=file (for full image)

or

https://pictures.bgbm.org/digilib/Scaler?fn=Cyprus/Arum_rupicola_B2.jpg&mo=fit&dw=400&dh=400 (for scaled images)

To implement this the script from #5744 needs to be adapted.
The API is available at: https://robcast.github.io/digilib/scaler-api.html


Files

uv-fist-image-determines-aspectratio.png (362 KB) uv-fist-image-determines-aspectratio.png Andreas Kohlbecker, 08/18/2020 03:32 PM
manifest-canvas-for-thumbs.json (4.02 KB) manifest-canvas-for-thumbs.json Andreas Kohlbecker, 08/18/2020 04:57 PM
manifest.json (4.02 KB) manifest.json Andreas Kohlbecker, 08/18/2020 04:57 PM

Related issues

Related to EDIT - feature request #9134: Rule based creation of additional media representations "on the fly"ClosedAndreas Kohlbecker

Actions
Related to EDIT - feature request #9198: option in iiif manifest creation to set canvas size to thumbnail sizeClosedAndreas Kohlbecker

Actions
Related to EDIT - task #9852: Use rulebased creation of thumbnail links for Salvador imagesNewAndreas Müller

Actions
Actions #2

Updated by Andreas Müller over 3 years ago

Keyword has not correct format and can not be parsed: showing abscission zone for file Bromus_fasciculatus_s_delilei_B3.jpg
Keyword has not correct format and can not be parsed: showing lemma and callus for file Bromus_fasciculatus_s_delilei_B4.jpg
More than 1 taxon found for media: Carex egorovae . Will now try to use only taxon with taxon node.
More than 1 taxon found for media: Carex egorovae . Will now try to use only taxon with taxon node.
More than 1 taxon found for media: Carex egorovae . Will now try to use only taxon with taxon node.

Person could not be parsed: G, N. Hadjikyriakou for file Euphorbia_falcata_s_falcata_A1.jpg
Person could not be parsed: G, N. Hadjikyriakou for file Euphorbia_falcata_s_falcata_D1.jpg
Keyword has not correct format and can not be parsed: Date: 16.03.2015 Photographer: G. N. Hadjikyriakou for file Euphorbia_helioscopia_s_helioscopioides_A1.jpg
Keyword has not correct format and can not be parsed: Date: 16.03.2015 Photographer: G. N. Hadjikyriakou for file Euphorbia_helioscopia_s_helioscopioides_B1.jpg
Keyword has not correct format and can not be parsed: Date: 16.03.2015 Photographer: G. N. Hadjikyriakou for file Euphorbia_helioscopia_s_helioscopioides_B2.jpg
More than 1 taxon found for media: Ranunculus neapolitanus . Will now try to use only taxon with taxon node.
More than 1 taxon found for media: Sambucus nigra . Will now try to use only taxon with taxon node.
Incorrect filename:silene_conoidea_D1.jpg

Actions #3

Updated by Andreas Müller over 3 years ago

Problem (Could not read block (block start: 14468, block length: 15217440, data length: 65527).) when reading metadata from file: Micromeria_chionistrae_C1.jpg
Problem (Could not read block (block start: 13908, block length: 15217440, data length: 65527).) when reading metadata from file: Micromeria_chionistrae_C2.jpg
Problem (Could not read block (block start: 14572, block length: 15217440, data length: 65527).) when reading metadata from file: Moricandia_arvensis_C3.jpg
Problem (Could not read block (block start: 12164, block length: 15217440, data length: 65527).) when reading metadata from file: Moricandia_arvensis_C4.jpg
Problem (Could not read block (block start: 15136, block length: 15217440, data length: 65527).) when reading metadata from file: Nigella_ciliaris_C7.jpg
Problem (Could not read block (block start: 13546, block length: 15217440, data length: 65527).) when reading metadata from file: Nigella_ciliaris_C8.jpg
Problem (Could not read block (block start: 13584, block length: 15217440, data length: 65527).) when reading metadata from file: Nigella_ciliaris_C9.jpg
Problem (Could not read block (block start: 12388, block length: 15217440, data length: 65527).) when reading metadata from file: Teucrium_karpasiticum_C1.jpg
Problem (Could not read block (block start: 14838, block length: 15217440, data length: 65527).) when reading metadata from file: Teucrium_karpasiticum_C2.jpg
Problem (Could not read block (block start: 12228, block length: 15217440, data length: 65527).) when reading metadata from file: Teucrium_salaminium_C1.jpg
Problem (Could not read block (block start: 15484, block length: 15217440, data length: 65527).) when reading metadata from file: Teucrium_salaminium_C2.jpg
Problem (Could not read block (block start: 14760, block length: 15217440, data length: 65527).) when reading metadata from file: Veronica_bozakmanii_C2.jpg
Problem (Could not read block (block start: 12416, block length: 15217440, data length: 65527).) when reading metadata from file: Veronica_bozakmanii_C3.jpg

Actions #4

Updated by Andreas Müller over 3 years ago

Taxon not found: Ophrys kotschyi-sphegodes

Actions #6

Updated by Andreas Müller over 3 years ago

AM:

noch ein „kleiner“ Nachtrag. Es gibt im CDM inzwischen >100 „Bilder“, die in als Datei nicht mehr existieren. Dies führt zu Unschönheiten bei der Anzeige. Ich denke, wir sollten die löschen, wollte das aber nicht machen ohne es vorher mit dir abgesprochen zu haben. Kannst du das mal überprüfen?
"Sedum_caespitosum_A3.jpg"
"Beta_vulgaris_C1.jpg"
"Podospermum_alpigenum_A1.jpg"
"Astragalus_echinus_A1.jpg"
"Crypsis_factorovskyi_B1.jpg"
"Crypsis_factorovskyi_B2.jpg"
"Pinguicula_crystallina_B1.jpg"
"Euphorbia_helioscopia_B1.jpg"
"Iris_albicans_A1.jpg"
"Saccharum_ravennae_A1.jpg"
"Sinapis_alba_E1.jpg"
"Lactuca_triquetra_E1.jpg"
"Ononis_variegata_B1.jpg"
"FIlago_pyramidata_B1.jpg"
"Sorbus_graeca_C1.jpg"
"Linum_trigynum_A1.jpg"
"Linum_usitatissimum_B1.jpg"
"Ononis_serrata_A1.jpg"
"Cytinus_hypocistis_B1.jpg"
"Urtica_dioica_s_cypria_B1.jpg"
"Geranium_lucidum_B1.jpg"
"Lindbergella_sintenisii_B1.jpg"
"Matthiola_longipetala_A1.jpg"
"Hypopitys_monotropa_A1.jpg"
"Beta_vulgaris_s_maritima_B1.jpg"
"Podospermum_laciniatum_D2.jpg"
"Helianthemum_sanguineum_A1.jpg"
"Helianthemum_sanguineum_A1.jpg"
"Helianthemum_sanguineum_A1.jpg"
"Cyperus_fuscus_B1.jpg"
"Securigera_cretica_B1.jpg"
"Euphorbia_falcata_s_macrostegia_D1.jpg"
"Carex_flacca_s_serrulata_A1.jpg"
"Epipactis_microphylla_A1.jpg"
"Epilobium_tetragonum_B1.jpg"
"Lapsana_communis_s_communis_D1.jpg"
"Lepidium_latifolium_D1.jpg"
"Malvella_sherardiana_C1.jpg"
"Medicago_lupulina_C1.jpg"
"Nigella_ciliaris_C1.jpg"
"Osyris_alba_A2.jpg"
"Orobanche_orientalis_B2.jpg"
"Silene_muscipula_B2.jpg"
"Vicia_lathyroides_B2.jpg"
"Orobanche_orientalis_B2.jpg"
"Melilotus_elegans_C2.jpg"
"Cynoglossum_natolicum_C2.jpg"
"Ranunculus_bulbosus_s_bulbosus_A1.jpg"
"Symphyotrichum_squamatum_B1.jpg"
"Ranunculus_peltatus_s_peltatus_A1.jpg"
"Astragalus_echinus_A2.jpg"
"Astragalus_echinus_A4.jpg"
"Astragalus_echinus_B2.jpg"
"Astragalus_echinus_B3.jpg"
"Beta_vulgaris_s_maritima_D2.jpg"
"Chaenorhinum_rubrifolium_B1.jpg"
"Cyperus_fuscus_B2.jpg"
"Cyperus_fuscus_B3.jpg"
"Euphorbia_falcata_s_macrostegia_A1.jpg"
"Euphorbia_falcata_s_macrostegia_B1.jpg"
"Euphorbia_falcata_s_macrostegia_D2.jpg"
"Euphorbia_helioscopia_A1.jpg"
"Euphorbia_helioscopia_A2.jpg"
"Euphorbia_helioscopia_A3.jpg"
"Euphorbia_helioscopia_A4.jpg"
"Euphorbia_helioscopia_A5.jpg"
"Euphorbia_helioscopia_D1.jpg"
"Euphorbia_helioscopia_D2.jpg"
"Helianthemum_sanguineum_A1.jpg"
"Iris_albicans_B1.jpg"
"Iris_albicans_B2.jpg"
"Lactuca_triquetra_A1.jpg"
"Lactuca_triquetra_B1.jpg"
"Lactuca_triquetra_B2.jpg"
"Lactuca_triquetra_B3.jpg"
"Lactuca_triquetra_B4.jpg"
"Lactuca_triquetra_C1.jpg"
"Lactuca_triquetra_C2.jpg"
"Lindbergella_sintenisii_B2.jpg"
"Matthiola_longipetala_A2.jpg"
"Matthiola_longipetala_A3.jpg"
"Podospermum_alpigenum_A2.jpg"
"Podospermum_alpigenum_A3.jpg"
"Podospermum_alpigenum_A4.jpg"
"Podospermum_alpigenum_A5.jpg"
"Podospermum_alpigenum_A6.jpg"
"Podospermum_laciniatum_A1.jpg"
"Podospermum_laciniatum_C1.jpg"
"Podospermum_laciniatum_C2.jpg"
"Podospermum_laciniatum_D1.jpg"
"Ranunculus_peltatus_s_peltatus_A1.JPG"
"Ranunculus_peltatus_s_peltatus_A2.JPG"
"Saccharum_ravennae_A3.jpg"
"Saccharum_ravennae_B2.jpg"
"Securigera_securidaca_A1.jpg"
"Securigera_securidaca_B1.jpg"
"Securigera_securidaca_B2.jpg"
"Securigera_securidaca_B4.jpg"
"Securigera_securidaca_D1.jpg"
"Veronica_persica_A1.jpg"
"Veronica_persica_B4.jpg"
"Veronica_persica_C1.jpg"
"Veronica_persica_D1.jpg"
"Podospermum_laciniatum_A2.jpg"
"Podospermum_laciniatum_B1.jpg"
"Podospermum_laciniatum_B2.jpg"

Actions #7

Updated by Andreas Müller over 3 years ago

  • Status changed from New to In Progress
  • % Done changed from 0 to 40
SELECT Replace(Replace(uri, 'https://pictures.bgbm.org/digilib/Scaler/IIIF/Cyprus!', ''), '/full/full/0/default.jpg','')
FROM MediaRepresentationPart p
WHERE uri  NOT LIKE 'https://pictures.bgbm.org/digilib/Scaler?fn=Cyprus/%'
Actions #8

Updated by Andreas Kohlbecker over 3 years ago

The Universalviewer appends ?t={some-hash} to the image URI. By this the links to the scaler api become invalid and the last parameter it not parsed correctly.

Invalid URI with appended parameter t (last parameter as parsed in bold letters, white space after schema part for formatting ):

https: //pictures.bgbm.org/digilib/Scaler?fn=Cyprus/Arum_rupicola_B2.jpg&mo=fit&dw=400&dh=400?t=1594725873250

adding an additional dummy parameter (uvfix for "universalviewer fix") to the end of the query part help parse the parameters:

https: //pictures.bgbm.org/digilib/Scaler?fn=Cyprus/Arum_rupicola_B2.jpg&mo=fit&dw=400&dh=400&uvfix=1?t=1594725873250

==> adapt the thumbnail URIs accordingly

Actions #10

Updated by Andreas Kohlbecker over 3 years ago

  • Related to feature request #9134: Rule based creation of additional media representations "on the fly" added
Actions #11

Updated by Andreas Müller over 3 years ago

I updated data in cyprus

UPDATE MediaRepresentationPart p
SET uri = Concat(uri, '&uvfix=1')
WHERE p.uri LIKE '%&mo=fit&dw=400&dh=400' OR p.uri LIKE '%&mo=fit&dw=200&dh=200'

and adapted the import code

Actions #12

Updated by Andreas Müller over 3 years ago

  • Status changed from In Progress to Resolved
  • Assignee changed from Andreas Müller to Andreas Kohlbecker
  • % Done changed from 40 to 50

The above problems are all solved and discussed with Ralf so I guess we can close this ticket.

Actions #13

Updated by Andreas Kohlbecker over 3 years ago

Andreas Müller wrote:

The above problems are all solved and discussed with Ralf so I guess we can close this ticket.

Let's wait for Ralfs answer regarding the decision if mo=fit or mo=crop should be used.

Actions #14

Updated by Andreas Kohlbecker over 3 years ago

  • Assignee changed from Andreas Kohlbecker to Andreas Müller
  • % Done changed from 50 to 40

Ralf voted for cropping the images (Email from 16.07.20 17:58)

Actions #15

Updated by Andreas Kohlbecker over 3 years ago

I modified the small thumbnails images with following sql statement:

UPDATE  MediaRepresentationPart p set p.uri = replace(uri, '&mo=fit&dw=200&dh=200', '&mo=crop&dw=200&dh=147') , height=147, width=200 where p.uri LIKE '%&mo=fit&dw=200&dh=200%'

in the test-server.

This does not work for the UV as hoped. The UV takes the aspect ration of the first full resolution image as default, for all thumbnail images. So using the crop function is not the complete solution.

see http://test.e-taxonomy.eu/dataportal/preview/cyprus/cdm_dataportal/taxon/e9925386-b197-4d81-a4fc-137ea96e6b4f/images

This is task still requires more research.

Actions #16

Updated by Andreas Müller over 3 years ago

  • Status changed from Resolved to Feedback
  • Assignee changed from Andreas Müller to Andreas Kohlbecker

Can you explain this a bit more? I do not yet understand what the exact problem is.

Actions #17

Updated by Andreas Kohlbecker over 3 years ago

  • Status changed from Feedback to Resolved

This is a complicated issue which required quite some time for a comprehensive explanation comment 15 contains all information which is required to fully understand the issue. Since I am the one to solve this ticket I would prefer not to spend too much time for further explanations.

Actions #18

Updated by Andreas Müller over 3 years ago

OK. It was assigned to me before therefore I asked.
So is the status "Resolved" correct or is there still work to do?

Actions #19

Updated by Andreas Kohlbecker over 3 years ago

adding picture to illustrate the above mentioned problem

Actions #20

Updated by Andreas Kohlbecker over 3 years ago

I a filed a bug report for the UV developers: https://github.com/UniversalViewer/universalviewer/issues/743

Actions #21

Updated by Andreas Kohlbecker over 3 years ago

I found a workaround for my case which at least works with universalviewer@v3.0.1:

The thumbnail dimensions are corrected if I set width and height of the thumbnails for the canvas size. You can see the change by comparing the attached manifest files: original= manifest.json, tweaked= manifest-canvas-for-thumbs.json

I am not sure if this solution conforms to the IIIF Presentation API 3.0 specification or if I should fear that coming changes to UV will behave differently with respect to my workaround.

Actions #22

Updated by Andreas Kohlbecker over 3 years ago

  • Related to feature request #9198: option in iiif manifest creation to set canvas size to thumbnail size added
Actions #23

Updated by Andreas Kohlbecker over 3 years ago

implementation ticket for above workaround #9198

once there is a better solution ( https://github.com/UniversalViewer/universalviewer/issues/743 ) we might want to remove the workaround again.

Actions #24

Updated by Andreas Kohlbecker over 3 years ago

  • Assignee changed from Andreas Kohlbecker to Andreas Müller
  • % Done changed from 40 to 50

I think we can close this ticket now.

Actions #25

Updated by Andreas Müller over 3 years ago

I think we should keep it for removing the thumbnails again after the release and also for updating the cichorieae data, too.

Actions #26

Updated by Andreas Müller over 3 years ago

  • Status changed from Resolved to In Progress
  • Target version changed from Release 5.18 to Release 5.19
Actions #27

Updated by Andreas Müller over 3 years ago

  • Target version changed from Release 5.19 to Release 5.18
Actions #28

Updated by Andreas Kohlbecker over 3 years ago

I updated the cyprus thumbnail representations in the production system to be cropped as documented in comment 15

Actions #29

Updated by Andreas Müller over 3 years ago

Removing thumbnails:

SELECT *
FROM MediaRepresentation mr
INNER JOIN MediaRepresentationPart mrp ON mrp.representation_id = mr.id
WHERE EXISTS (
   SELECT * FROM MediaRepresentation mr2 INNER JOIN MediaRepresentationPart mrp2 ON mrp2.representation_id = mr2.id 
    WHERE mr.media_id = mr2.media_id AND mr.id <> mr2.id    
    AND mrp2.uri LIKE '%&mo=file'
    )
AND mrp.uri LIKE '%dw=200&dh=147&uvfix=1'
ORDER BY mr.media_id
Actions #30

Updated by Andreas Müller over 3 years ago

  • % Done changed from 50 to 80

Thumbnails have been removed from cyprus DB (also from AUD tables).

So the only open issue is now to update links in cichorieae database if needed.

Actions #31

Updated by Andreas Müller about 3 years ago

  • % Done changed from 80 to 90

Links in the Cichorieae database have all be updated to use the scaler API.

Only open issue is now to reload metadata and maybe to find images which exist on the imageserver but are still missing in CDM for Cichorieae.

Actions #32

Updated by Andreas Müller about 3 years ago

Hi Andreas,

ich bin ja gerade dabei #9132 auch für Cichorieae zu machen. Die Links habe ich bereits alle umgestellt. Wie sieht es denn da mit der Erstellung der Thumbnails aus. Ist das Mapping schon so definiert wie es sein soll. Derzeit laden die Seiten oft noch recht langsam, aber das liegt vermutlich primär daran, dass die Bilder derzeit ja vom Manifestserver noch alle geladen werden, oder? Offen ist von meiner Seite noch, dass die Metadaten alle nochmal aktualisiert werden, wobei ich nicht genau weiss, ob das derzeit überhaupt notwendig ist, wenn die Metadaten sowies serverseitig nochmal geladen werden. Was meinst Du? Auch muss ich mit Norbert nochmal abklären, ob alle Bilder erfasst sind.

Also ich habe mir das gerade nochmal angeschaut für ein Beispiel https://pictures.bgbm.org/digilib/Scaler?fn=Cichorieae/Melanoseris_bracteata_Bc_04.jpg&mo=file . Dort sind auf jeden Fall Metadaten drin als Keywords: Taxon: Melanoseris bracteata Locality: China, Tibet, Cuo-na, Gong-ri Village Coordinates: c. 27°51'N, 91°45'E Date: 7.9.2012 Photographer: Hu Guo-Xiong Collection: Hu Guo-Xiong & al. 1209024 (KUN) Description: Auf der CDM Seite wird aber keins der Keywords angezeigt http://cichorieae.e-taxonomy.net/portal/cdm_dataportal/taxon/d64581d8-fd8f-4356-9f34-2fee92370eaf/images . Ein Auslesen dieser Metadaten ist also entweder noch nicht implementiert (obwohl ich dachte das wäre inzwischen in cdmlib integriert) oder funktioniert noch nicht.

Actions #33

Updated by Andreas Müller about 3 years ago

  • Target version changed from Release 5.18 to Release 5.19
Actions #34

Updated by Andreas Müller about 3 years ago

  • Target version changed from Release 5.19 to Release 5.21
Actions #35

Updated by Andreas Müller about 3 years ago

  • Target version changed from Release 5.21 to Release 5.22
Actions #36

Updated by Andreas Müller almost 3 years ago

  • Target version changed from Release 5.22 to Release 5.25
Actions #37

Updated by Andreas Müller over 2 years ago

  • Target version changed from Release 5.25 to Release 5.45
Actions #38

Updated by Andreas Müller over 2 years ago

  • Related to task #9852: Use rulebased creation of thumbnail links for Salvador images added
Actions

Also available in: Atom PDF