task #3560
feature request #7491: [MASTER] Withheld unpublished taxa from webservice results and in DAOs
Withheld unpublished taxa from webservice used in E+M dataportal
70%
Description
see also #1780, #3740, #3739
the following services have been secured.
AbstractIdentifiableController:
- open:
- doGetTitleCache (not E+M)
BaseController:
- doGet
- open:
- doGetMethod
- doGetMethod
TaxonController:
- doGetClassifications
- doGetDescriptionElementsByType
- doGetDescriptions
- doGetFieldUnits
- doGetTaggedName (not yet correct)
- doGetTaxonNodes (not yet correct)
- open:
- doGetIncludedTaxa
- doGetTaxonNodeAgentRelations
- doListSpecimensOrObservations
- getAcceptedFor
TaxonPortalController:
- doGetSynonymy
- doGetTaxonNodes
- doGetTaxonRelations
- doGetMedia
- doGetSubtreeMedia
- doGetFromNameRelations
- doGetToNameRelations
TaxonListController
- doFind
- dofindByEverythingFullText
- doFindByFullText
- doSearch
- open:
- doFindBestMatchingTaxon
- dofindByDescriptionElementFullText
- doFindByIdentifier
- doFindByMarker
- doFindByNameParts
TaxonPortalListController
- no services
TaxonNodeController
- doPageChildNodes
- open:
- getParent()
ClassificationController
- getChildNodes
- getChildNodesAtRank
- getTaxonInContext
- open:
- getGroupedTaxaByHigherTaxon (not E+M)
- getGroupedTaxaByMarkedParents (not E+M)
ClassificationPortalController
- not relevant:
- getClassificationRootNode
ClassificationPortalListController
- getChildNodes
- getChildNodesAtRank
- getChildNodesOfTaxon
- getPathFromTaxon
- getPathFromTaxonToRank
- getSiblingsOfTaxon
ClassificationListController
- no services
DwcA-Controller:
- doDwcaTaxExport
General:
- open:
- check property paths in all controllers if they include taxa
- web-services from other controllers that return taxa as part of there result
Related issues
Associated revisions
ref #3560 implement publish filter for TaxonNodeDao.listChildrenOf and .countChildrenOf
ref #3560 implement publish filter for ClassificationDao.listChildrenOf and .countChildrenOf
ref #3560 add includeUnpublished for loadChildNodesOfTaxonNode in TaxEditor
ref #3560 add includeUnpublished for loadChildNodesOfTaxonNode in Vaadin
ref #3560 use global "unpublished" constant
ref #3560 implement unpublished for load and find in service layer
ref #3560 unpublished for doGetTaxonNodes and doGetTaggedName and doGetSubtreeMedia
ref #3560 implement unpublished for taxon collection retrieval in occurrence service
ref #3560 implement unpublished for rankSpecificRootNode in service and remote (no tests yet)
ref #3560 implement unpublished for listSiblings in service and remote (no tests yet)
ref #3560 implement unpublished in BaseController.doGet
ref #3560 tests for unpublished listRankSpecificRootNodes
ref #3560 implement publish in taxon service by name search
ref #3560 implement publish in taxon service by name search
ref #3560 fix test in TaxEditor for findMatchingXXX
ref #3560 set includeUnpublished as default for FindTaxaAndNames configurator
ref #3560 search also for unpublished as default in TaxEditor
ref #3560 implement loadTreeBranchToTaxon (pathTo) for unpublished
ref #3560 add exception package to TaxEdtior
ref #3560 add publish flag to lucene index
ref #3560 add publish flag to lucene search
ref #3560 fix pathFrom in classification service impl
ref #3560 further publish flag issues for lucene search and first tests on lucene search
ref #3560 try to add publish to synonym search in findTaxaAndNamesByFullText (probably not working yet)
ref #3560 further improve full text taxonAndName search for publish flag, including tests
ref #3560 remove unpublished synonyms from taxon.synonymy
ref #3560 handle access denied for unpublished data in some TaxonController methods
ref #3560 remove unpublished synonyms from taxon.synonymy (fix)
ref #3560 handle access denied for unpublished data in BaseController.doGet
ref #3560 remove unpublished synonyms from taxon.synonymy (handle homotypic syns correctly)
ref #3560 fix wrong method call in BaseController
ref #3560 add includeUnpublished for taxon relationships
ref #3560 fix includeUnpublished for taxon relationships and correct includePublished->includeUnpublished
ref #3560 secured TaxonPortalController.doGetMedia and doGetSubtreeMedia
ref #3560 secured TaxonPortalController.doGetFromNameRelations and doGetToNameRelations
ref #3560 unpublished exception handling for TaxonController.doGetAcceptedFor
ref #3560 unpublished exception handling for TaxonController.getPathFromTaxon(ToRank)
ref #3560 fix checkPublished in service base class
ref #3560
History
#1 Updated by Andreas Kohlbecker about 7 years ago
- Category changed from cdmserver to cdmlib-remote
#2 Updated by Andreas Kohlbecker about 7 years ago
- Severity changed from critical to blocker
#3 Updated by Andreas Kohlbecker about 7 years ago
- Priority changed from Priority08 to Highest
This must be implemented at the persistence level
TaxonDao
SpecimenDao
Search functions
can we use Hibernate filtered collections for this?
#4 Updated by Andreas Kohlbecker about 7 years ago
- Assignee changed from Andreas Kohlbecker to Andreas Müller
- Category changed from cdmlib-remote to cdmlib
- Subject changed from Withheld unpublished taxa and specimen from webservice results to Withheld unpublished taxa and specimen from webservice results and in DAOs
#5 Updated by Andreas Kohlbecker about 7 years ago
- Severity changed from blocker to critical
for now we will use a second database for publishing, the entities marked as unpublished can be filtered out at the database level, so this issue is no longer a blocker.
#6 Updated by Andreas Müller almost 5 years ago
- Target version changed from CDM UML 3.3/3.4 - Postprocessing to Unassigned CDM tickets
#7 Updated by Andreas Müller over 4 years ago
- Target version changed from Unassigned CDM tickets to Euro+Med Portal Release
#8 Updated by Andreas Kohlbecker about 4 years ago
- Tags set to phycobank, Euro+Med
- Description updated (diff)
- Due date set to 03/31/2016
this is crucial for the phycobank.
#9 Updated by Andreas Kohlbecker over 3 years ago
- Related to feature request #3709: [E+M][Editor] sufficient rights management for E+M workflow added
#10 Updated by Andreas Müller over 3 years ago
- Related to feature request #3740: Allow recursively setting the publish bit for TaxonBase added
#11 Updated by Andreas Müller almost 3 years ago
- Priority changed from Highest to Priority14
- Estimated time set to 15.00 h
#12 Updated by Andreas Müller over 2 years ago
- Due date deleted (
03/31/2016) - Status changed from New to In Progress
- Priority changed from Priority14 to Highest
#13 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
#14 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
#15 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
- % Done changed from 0 to 10
#16 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
#17 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
- % Done changed from 10 to 20
#18 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
#20 Updated by Andreas Müller over 2 years ago
- Subject changed from Withheld unpublished taxa and specimen from webservice results and in DAOs to Withheld unpublished taxa from webservice results and in DAOs
#21 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
#22 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
#23 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
#24 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
#25 Updated by Andreas Müller over 2 years ago
- Precedes feature request #7457: E+M: Allow filtering for only published in search added
#26 Updated by Andreas Müller over 2 years ago
- Tags changed from phycobank, euro+med to phycobank, euro+med, search
#27 Updated by Andreas Müller over 2 years ago
- Target version changed from Euro+Med Portal Release to Release 5.1
#28 Updated by Andreas Müller over 2 years ago
- Parent task set to #7491
#29 Updated by Andreas Müller over 2 years ago
- Subject changed from Withheld unpublished taxa from webservice results and in DAOs to Withheld unpublished taxa from webservice used in E+M dataportal
#30 Updated by Andreas Müller over 2 years ago
- % Done changed from 20 to 60
#31 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
#32 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
#33 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
#34 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
#35 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
#36 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
#37 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
#38 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
#39 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
#40 Updated by Andreas Müller over 2 years ago
- Precedes feature request #7492: Withheld unpublished taxa from webservice used in other webservices added
#41 Updated by Andreas Müller over 2 years ago
- Category changed from cdmlib to cdmlib-remote
#42 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
#43 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
#44 Updated by Andreas Müller over 2 years ago
- % Done changed from 60 to 70
- Estimated time changed from 15.00 h to 2.00 h
#45 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
- Status changed from In Progress to Resolved
- Assignee changed from Andreas Müller to Andreas Kohlbecker
I still want to unify the handling of unpublished a bit, but maybe you can already start testing if there some services that urgently need to be protected to not show unpublished data in the data portal.
The general usecase to protect ALL webservices to not show unpublished data is handled in #7492 and is not part of this ticket.
#46 Updated by Andreas Kohlbecker over 2 years ago
Hi Andreas,
first of all I analyzed the logfile on the production server and created a statistics on all webservice calls that are send to the cdm-server.
By this we now have an overview on the webservice endpoints that are really used together with a usage count. The count corresponds to the number of calls to this endpoint as seen in the logfile.
I quickly went over the logfile and marked all webservice endpoints which potentially need filtering by the publish flag. Endpoints marked yellow needs further investigation, grey ones are ok. Other colors are explained in the sheet.
#47 Updated by Andreas Kohlbecker over 2 years ago
- File ws-endpoint-usage.ods added
#48 Updated by Andreas Müller over 2 years ago
Andreas Kohlbecker wrote:
first of all I analyzed the logfile on the production server and created a statistics on all webservice calls that are send to the cdm-server.
By this we now have an overview on the webservice endpoints that are really used together with a usage count. The count corresponds to the number of calls to this endpoint as seen in the logfile.I quickly went over the logfile and marked all webservice endpoints which potentially need filtering by the publish flag. Endpoints marked yellow needs further investigation, grey ones are ok. Other colors are explained in the sheet.
How do we want report further results/improvements? Handling in ods is not comfortable if 2 people work on it.
Some of the yellow services are already checked. See the overview in the ticket description.
#49 Updated by Andreas Kohlbecker over 2 years ago
- File ws-endpoint-usage.ods added
I updated the ws-endpoint-usage.ods spread sheet by the information from the issue description.
There are now only few controller methods left which need to be checked.
Andreas
#50 Updated by Andreas Kohlbecker over 2 years ago
- File deleted (
ws-endpoint-usage.ods)
#51 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
#52 Updated by Andreas Kohlbecker over 2 years ago
- File ws-endpoint-usage.ods added
#53 Updated by Andreas Kohlbecker over 2 years ago
- File deleted (
ws-endpoint-usage.ods)
#54 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
secured TaxonPortalController.doGetMedia and doGetSubtreeMedia
#55 Updated by Andreas Müller over 2 years ago
- Description updated (diff)
secured TaxonPortalController.doGetFromNameRelations and doGetToNameRelations
#56 Updated by Andreas Müller over 2 years ago
- Private changed from Yes to No
#57 Updated by Andreas Müller over 2 years ago
- Assignee changed from Andreas Kohlbecker to Andreas Müller
#58 Updated by Andreas Müller over 2 years ago
- File ws-endpoint-usage.ods added
#59 Updated by Andreas Müller over 2 years ago
- File deleted (
ws-endpoint-usage.ods)
#60 Updated by Andreas Müller over 2 years ago
- Related to bug #7745: Subtree filter for dataportal services added
#61 Updated by Andreas Kohlbecker over 2 years ago
- Related to bug #7756: org.hibernate.QueryException in portal/taxon/{uuid}/subtree/media.json webservice related to relation direction param in prepareTaxonRelationshipQuery() added
#62 Updated by Andreas Kohlbecker about 2 years ago
- Tags changed from phycobank, euro+med, search to phycobank, euro+med, search, security, permission
#63 Updated by Andreas Kohlbecker over 1 year ago
- Description updated (diff)
#64 Updated by Andreas Kohlbecker over 1 year ago
- Description updated (diff)
#65 Updated by Andreas Müller over 1 year ago
Can you please adapt the new information on doPageByRestrictions to the logic of the rest of the list? In general we follow the logic that the list shows the already secured web-services. Explicitly open issues are listed in an open: list. Same for not relevant:
#66 Updated by Andreas Müller over 1 year ago
Also this ticket is only about E+M data portal relevant webservices. For other webservices please use #7492.
#67 Updated by Andreas Müller over 1 year ago
Be aware that this ticket has already status resolved in 5.1. It's task description should not be enlarged anymore if not absolutely needed.
#68 Updated by Andreas Kohlbecker over 1 year ago
- Description updated (diff)
#69 Updated by Andreas Kohlbecker over 1 year ago
I cleaned the ticket up as requested
#72 Updated by Andreas Müller over 1 year ago
- Related to task #8424: evaluate publish flags in the full derivative path added
#73 Updated by Andreas Müller over 1 year ago
- Related to feature request #4484: Evaluate publish flag for derivates in data portal added
#74 Updated by Andreas Müller 12 months ago
- Related to feature request #8875: Filter data portal by taxonomic rank added