Project

General

Profile

Actions

task #3560

open

feature request #7491: [MASTER] Withheld unpublished taxa from webservice results and in DAOs

Withheld unpublished taxa from webservice used in E+M dataportal

Added by Andreas Müller over 10 years ago. Updated over 4 years ago.

Status:
Resolved
Priority:
Highest
Category:
cdmlib-remote
Target version:
Start date:
Due date:
% Done:

70%

Estimated time:
2:00 h
Severity:
critical

Description

see also #1780, #3740, #3739

the following services have been secured.

AbstractIdentifiableController:

  • open:
    • doGetTitleCache (not E+M)

BaseController:

  • doGet
  • open:
    • doGetMethod

TaxonController:

  • doGetClassifications
  • doGetDescriptionElementsByType
  • doGetDescriptions
  • doGetFieldUnits
  • doGetTaggedName (not yet correct)
  • doGetTaxonNodes (not yet correct)
  • open:
    • doGetIncludedTaxa
    • doGetTaxonNodeAgentRelations
    • doListSpecimensOrObservations
    • getAcceptedFor

TaxonPortalController:

  • doGetSynonymy
  • doGetTaxonNodes
  • doGetTaxonRelations
  • doGetMedia
  • doGetSubtreeMedia
  • doGetFromNameRelations
  • doGetToNameRelations

TaxonListController

  • doFind
  • dofindByEverythingFullText
  • doFindByFullText
  • doSearch
  • open:
    • doFindBestMatchingTaxon
    • dofindByDescriptionElementFullText
    • doFindByIdentifier
    • doFindByMarker
    • doFindByNameParts

TaxonPortalListController

  • no services

TaxonNodeController

  • doPageChildNodes
  • open:
    • getParent()

ClassificationController

  • getChildNodes
  • getChildNodesAtRank
  • getTaxonInContext
  • open:
    • getGroupedTaxaByHigherTaxon (not E+M)
    • getGroupedTaxaByMarkedParents (not E+M)

ClassificationPortalController

  • not relevant:
    • getClassificationRootNode

ClassificationPortalListController

  • getChildNodes
  • getChildNodesAtRank
  • getChildNodesOfTaxon
  • getPathFromTaxon
  • getPathFromTaxonToRank
  • getSiblingsOfTaxon

ClassificationListController

  • no services

DwcA-Controller:

  • doDwcaTaxExport

General:

  • open:
    • check property paths in all controllers if they include taxa
    • web-services from other controllers that return taxa as part of there result

Files

ws-endpoint-usage.ods (23.2 KB) ws-endpoint-usage.ods Andreas Müller, 08/15/2018 06:13 PM

Related issues

Related to EDIT - feature request #3709: [E+M][Editor] sufficient rights management for E+M workflowResolvedAndreas Müller

Actions
Related to EDIT - feature request #3740: Allow recursively setting the publish bit for TaxonBaseClosedAndreas Müller

Actions
Related to EDIT - bug #7745: Subtree filter for dataportal servicesClosedAndreas Kohlbecker

Actions
Related to EDIT - bug #7756: org.hibernate.QueryException in portal/taxon/{uuid}/subtree/media.json webservice related to relation direction param in prepareTaxonRelationshipQuery()FeedbackAndreas Müller

Actions
Related to EDIT - task #8424: evaluate publish flags in the full derivative pathNewAndreas Kohlbecker

Actions
Related to EDIT - feature request #4484: Evaluate publish flag for derivates in data portalNewAndreas Müller

Actions
Related to EDIT - feature request #8875: Filter data portal by taxonomic rankNewAndreas Müller

Actions
Related to EDIT - bug #9559: OccurrenceServiceImpl.findRootUnitDTOs() and related methods suppress unpublished unitsNewAndreas Kohlbecker

Actions
Related to EDIT - feature request #10334: Evaluate DescriptionBase.publish in webservicesClosedAndreas Müller

Actions
Precedes EDIT - feature request #7457: E+M: Allow filtering for only published in searchClosedKatja Luther

Actions
Precedes EDIT - feature request #7492: Withheld unpublished taxa from webservice used in other webservicesNewAndreas Müller

Actions
Actions #1

Updated by Andreas Kohlbecker about 10 years ago

  • Category changed from cdmserver to cdmlib-remote
Actions #2

Updated by Andreas Kohlbecker about 10 years ago

  • Severity changed from critical to blocker
Actions #3

Updated by Andreas Kohlbecker about 10 years ago

  • Priority changed from Priority08 to Highest

This must be implemented at the persistence level

  • TaxonDao

  • SpecimenDao

  • Search functions

can we use Hibernate filtered collections for this?

Actions #4

Updated by Andreas Kohlbecker about 10 years ago

  • Assignee changed from Andreas Kohlbecker to Andreas Müller
  • Category changed from cdmlib-remote to cdmlib
  • Subject changed from Withheld unpublished taxa and specimen from webservice results to Withheld unpublished taxa and specimen from webservice results and in DAOs
Actions #5

Updated by Andreas Kohlbecker about 10 years ago

  • Severity changed from blocker to critical

for now we will use a second database for publishing, the entities marked as unpublished can be filtered out at the database level, so this issue is no longer a blocker.

Actions #6

Updated by Andreas Müller almost 8 years ago

  • Target version changed from CDM UML 3.3/3.4 - Postprocessing to Unassigned CDM tickets
Actions #7

Updated by Andreas Müller over 7 years ago

  • Target version changed from Unassigned CDM tickets to Euro+Med Portal Release
Actions #8

Updated by Andreas Kohlbecker about 7 years ago

  • Tags set to phycobank, Euro+Med
  • Description updated (diff)
  • Due date set to 03/31/2016

this is crucial for the phycobank.

Actions #9

Updated by Andreas Kohlbecker over 6 years ago

Actions #10

Updated by Andreas Müller over 6 years ago

Actions #11

Updated by Andreas Müller over 5 years ago

  • Priority changed from Highest to Priority14
  • Estimated time set to 15:00 h
Actions #12

Updated by Andreas Müller over 5 years ago

  • Due date deleted (03/31/2016)
  • Status changed from New to In Progress
  • Priority changed from Priority14 to Highest
Actions #13

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)
Actions #14

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)
Actions #15

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)
  • % Done changed from 0 to 10
Actions #16

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)
Actions #17

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)
  • % Done changed from 10 to 20
Actions #18

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)
Actions #20

Updated by Andreas Müller over 5 years ago

  • Subject changed from Withheld unpublished taxa and specimen from webservice results and in DAOs to Withheld unpublished taxa from webservice results and in DAOs
Actions #21

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)
Actions #22

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)
Actions #23

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)
Actions #24

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)
Actions #25

Updated by Andreas Müller over 5 years ago

Actions #26

Updated by Andreas Müller over 5 years ago

  • Tags changed from phycobank, euro+med to phycobank, euro+med, search
Actions #27

Updated by Andreas Müller over 5 years ago

  • Target version changed from Euro+Med Portal Release to Release 5.1
Actions #28

Updated by Andreas Müller over 5 years ago

  • Parent task set to #7491
Actions #29

Updated by Andreas Müller over 5 years ago

  • Subject changed from Withheld unpublished taxa from webservice results and in DAOs to Withheld unpublished taxa from webservice used in E+M dataportal
Actions #30

Updated by Andreas Müller over 5 years ago

  • % Done changed from 20 to 60
Actions #31

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)
Actions #32

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)
Actions #33

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)
Actions #34

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)
Actions #35

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)
Actions #36

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)
Actions #37

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)
Actions #38

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)
Actions #39

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)
Actions #40

Updated by Andreas Müller over 5 years ago

  • Precedes feature request #7492: Withheld unpublished taxa from webservice used in other webservices added
Actions #41

Updated by Andreas Müller over 5 years ago

  • Category changed from cdmlib to cdmlib-remote
Actions #42

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)
Actions #43

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)
Actions #44

Updated by Andreas Müller over 5 years ago

  • % Done changed from 60 to 70
  • Estimated time changed from 15:00 h to 2:00 h
Actions #45

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)
  • Status changed from In Progress to Resolved
  • Assignee changed from Andreas Müller to Andreas Kohlbecker

I still want to unify the handling of unpublished a bit, but maybe you can already start testing if there some services that urgently need to be protected to not show unpublished data in the data portal.

The general usecase to protect ALL webservices to not show unpublished data is handled in #7492 and is not part of this ticket.

Actions #46

Updated by Andreas Kohlbecker over 5 years ago

Hi Andreas,

first of all I analyzed the logfile on the production server and created a statistics on all webservice calls that are send to the cdm-server.
By this we now have an overview on the webservice endpoints that are really used together with a usage count. The count corresponds to the number of calls to this endpoint as seen in the logfile.

I quickly went over the logfile and marked all webservice endpoints which potentially need filtering by the publish flag. Endpoints marked yellow needs further investigation, grey ones are ok. Other colors are explained in the sheet.

ws-endpoint-usage.ods

Actions #47

Updated by Andreas Kohlbecker over 5 years ago

  • File ws-endpoint-usage.ods added
Actions #48

Updated by Andreas Müller over 5 years ago

Andreas Kohlbecker wrote:

first of all I analyzed the logfile on the production server and created a statistics on all webservice calls that are send to the cdm-server.
By this we now have an overview on the webservice endpoints that are really used together with a usage count. The count corresponds to the number of calls to this endpoint as seen in the logfile.

I quickly went over the logfile and marked all webservice endpoints which potentially need filtering by the publish flag. Endpoints marked yellow needs further investigation, grey ones are ok. Other colors are explained in the sheet.

How do we want report further results/improvements? Handling in ods is not comfortable if 2 people work on it.

Some of the yellow services are already checked. See the overview in the ticket description.

Actions #49

Updated by Andreas Kohlbecker over 5 years ago

  • File ws-endpoint-usage.ods added

I updated the ws-endpoint-usage.ods spread sheet by the information from the issue description.

There are now only few controller methods left which need to be checked.

Andreas

Actions #50

Updated by Andreas Kohlbecker over 5 years ago

  • File deleted (ws-endpoint-usage.ods)
Actions #51

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)
Actions #52

Updated by Andreas Kohlbecker over 5 years ago

  • File ws-endpoint-usage.ods added
Actions #53

Updated by Andreas Kohlbecker over 5 years ago

  • File deleted (ws-endpoint-usage.ods)
Actions #54

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)

secured TaxonPortalController.doGetMedia and doGetSubtreeMedia

Actions #55

Updated by Andreas Müller over 5 years ago

  • Description updated (diff)

secured TaxonPortalController.doGetFromNameRelations and doGetToNameRelations

Actions #56

Updated by Andreas Müller over 5 years ago

  • Private changed from Yes to No
Actions #57

Updated by Andreas Müller over 5 years ago

  • Assignee changed from Andreas Kohlbecker to Andreas Müller
Actions #59

Updated by Andreas Müller over 5 years ago

  • File deleted (ws-endpoint-usage.ods)
Actions #60

Updated by Andreas Müller over 5 years ago

  • Related to bug #7745: Subtree filter for dataportal services added
Actions #61

Updated by Andreas Kohlbecker over 5 years ago

  • Related to bug #7756: org.hibernate.QueryException in portal/taxon/{uuid}/subtree/media.json webservice related to relation direction param in prepareTaxonRelationshipQuery() added
Actions #62

Updated by Andreas Kohlbecker about 5 years ago

  • Tags changed from phycobank, euro+med, search to phycobank, euro+med, search, security, permission
Actions #63

Updated by Andreas Kohlbecker over 4 years ago

  • Description updated (diff)
Actions #64

Updated by Andreas Kohlbecker over 4 years ago

  • Description updated (diff)
Actions #65

Updated by Andreas Müller over 4 years ago

Can you please adapt the new information on doPageByRestrictions to the logic of the rest of the list? In general we follow the logic that the list shows the already secured web-services. Explicitly open issues are listed in an open: list. Same for not relevant:

Actions #66

Updated by Andreas Müller over 4 years ago

Also this ticket is only about E+M data portal relevant webservices. For other webservices please use #7492.

Actions #67

Updated by Andreas Müller over 4 years ago

Be aware that this ticket has already status resolved in 5.1. It's task description should not be enlarged anymore if not absolutely needed.

Actions #68

Updated by Andreas Kohlbecker over 4 years ago

  • Description updated (diff)
Actions #69

Updated by Andreas Kohlbecker over 4 years ago

I cleaned the ticket up as requested

Actions #72

Updated by Andreas Müller over 4 years ago

  • Related to task #8424: evaluate publish flags in the full derivative path added
Actions #73

Updated by Andreas Müller over 4 years ago

Actions #74

Updated by Andreas Müller almost 4 years ago

Actions #75

Updated by Andreas Müller almost 3 years ago

  • Related to bug #9559: OccurrenceServiceImpl.findRootUnitDTOs() and related methods suppress unpublished units added
Actions #79

Updated by Andreas Müller 9 months ago

Actions

Also available in: Atom PDF