Project

General

Profile

bug #6296

Simple search for common names returns duplicates

Added by Andreas Kohlbecker almost 2 years ago. Updated 6 months ago.

Status:
Resolved
Priority:
Highest
Assignee:
Category:
cdmlib
Target version:
Start date:
12/22/2016
Due date:
% Done:

100%

Severity:
critical
Found in Version:

Description

This is a bug the cdmlib-services

For example:

http -b http://api.cybertaxonomy.org/salvador/portal/taxon/find.json\?doTaxaByCommonNames\=1\&doMisappliedNames\=0\&doSynonyms\=0\&doTaxa\=0\&pageNumber\=0\&pageSize\=25\&query\=amate\&tree\=b010c84d-6049-45f4-9f13-c065101eaa26 | jq '.records[].titleCache'

NOTE: The http command is from httpie

Broken queries:

TODO:

  1. Implement tests for theses search mode combinations. DONE: cdmlib|cc95089
  2. fix the bug

Associated revisions

Revision 8311a989 (diff)
Added by Katja Luther almost 2 years ago

fix #6296: include the common name search into the search query of taxa and synonyms

Revision 7304d345 (diff)
Added by Katja Luther almost 2 years ago

fixed copy paste error for common name search

Revision c4f7a205 (diff)
Added by Katja Luther almost 2 years ago

fix #6296: fix problems for simple search

Revision cc95089b (diff)
Added by Andreas Kohlbecker almost 2 years ago

ref #6296 more tests for findByTaxaAndName

Revision 177a283a (diff)
Added by Katja Luther almost 2 years ago

fix #6296: fix the parameter settings in search queries

Revision 4e6f5b53 (diff)
Added by Andreas Kohlbecker almost 2 years ago

ref #6296 confirming that modified expectations are correct, removing FIXMEs

History

#1 Updated by Andreas Müller almost 2 years ago

  • Assignee changed from Andreas Müller to Katja Luther
  • Priority changed from New to Highest
  • Target version changed from Unassigned CDM tickets to Release 4.5

#2 Updated by Katja Luther almost 2 years ago

This is a problem in the lucene search, if a taxon has two common names starting with "amate" the lucene search returns the taxon two times. We need to add a duplicate filter.

#3 Updated by Katja Luther almost 2 years ago

tested findByEverythingFullText(), but this works fine also with more than one common name starting with the same word.
@AK which service method is called for the simple search?

#4 Updated by Andreas Kohlbecker almost 2 years ago

The method in question is

Pager<IdentifiableEntity> eu.etaxonomy.cdm.api.service.TaxonServiceImpl.findTaxaAndNames(IFindTaxaAndNamesConfigurator configurator)

so this issue has nothing to do with the free text search.

The results of the common names search is simply added to the result set in findTaxaAndNames after the search by accepted name, synonym, misapplied name has been done. The first search process already returns a pager object of the desired size. Adding the results of the common name search completely breaks the paging mechanism. The search for common names thus must be integrated into the preceding search step.

#5 Updated by Katja Luther almost 2 years ago

The commonNameSearch is now integrated into the taxa and synonym search query.

#6 Updated by Katja Luther almost 2 years ago

  • Status changed from New to Resolved
  • % Done changed from 0 to 50

#7 Updated by Andreas Kohlbecker almost 2 years ago

  • Status changed from Resolved to In Progress

With the test case mentioned in #6317 an exception is thrown:

2017-01-09 15:26:32.283:WARN:oejs.ServletHandler:/portal/taxon/find.json
org.hibernate.hql.internal.ast.QuerySyntaxException: unexpected token: f near line 1, column 196 [select t.id from eu.etaxonomy.cdm.model.taxon.Taxon t  join t.taxonNodes as tn join t.descriptions d join d.descriptionElements e join e.feature f  WHERE  tn.classification = :classificationAND  f.supportsCommonTaxonName = true and e.name LIKE :queryString]
    at org.hibernate.hql.internal.ast.QuerySyntaxException.convert(QuerySyntaxException.java:74)
    at org.hibernate.hql.internal.ast.ErrorCounter.throwQueryException(ErrorCounter.java:91)
    at org.hibernate.hql.internal.ast.QueryTranslatorImpl.parse(QueryTranslatorImpl.java:288)
    at org.hibernate.hql.internal.ast.QueryTranslatorImpl.doCompile(QueryTranslatorImpl.java:187)
    at org.hibernate.hql.internal.ast.QueryTranslatorImpl.compile(QueryTranslatorImpl.java:142)
    at org.hibernate.engine.query.spi.HQLQueryPlan.<init>(HQLQueryPlan.java:115)
    at org.hibernate.engine.query.spi.HQLQueryPlan.<init>(HQLQueryPlan.java:76)
    at org.hibernate.engine.query.spi.QueryPlanCache.getHQLQueryPlan(QueryPlanCache.java:150)
    at org.hibernate.internal.AbstractSessionImpl.getHQLQueryPlan(AbstractSessionImpl.java:302)
    at org.hibernate.internal.AbstractSessionImpl.createQuery(AbstractSessionImpl.java:240)
    at org.hibernate.internal.SessionImpl.createQuery(SessionImpl.java:1894)
    at eu.etaxonomy.cdm.persistence.dao.hibernate.taxon.TaxonDaoHibernateImpl.prepareQuery(TaxonDaoHibernateImpl.java:398)
    at eu.etaxonomy.cdm.persistence.dao.hibernate.taxon.TaxonDaoHibernateImpl.prepareTaxaByName(TaxonDaoHibernateImpl.java:601)
    at eu.etaxonomy.cdm.persistence.dao.hibernate.taxon.TaxonDaoHibernateImpl.countTaxaByName(TaxonDaoHibernateImpl.java:658)

#8 Updated by Katja Luther almost 2 years ago

  • Status changed from In Progress to Resolved
  • Assignee changed from Katja Luther to Andreas Kohlbecker

#9 Updated by Andreas Kohlbecker almost 2 years ago

  • Status changed from Resolved to In Progress
  • Assignee changed from Andreas Kohlbecker to Katja Luther
  • Target version changed from Release 4.5 to Release 4.6

This works now in general but searching only for common names causes an exception:

http://api.cybertaxonomy.org/salvador/portal/taxon/find.json?doTaxaByCommonNames=1&pageNumber=0&pageSize=25&query=amate&tree=b010c84d-6049-45f4-9f13-c065101eaa26

(By the way, this is exactly the example request given in the issue description)

please implement a test for this case

#11 Updated by Katja Luther almost 2 years ago

  • Status changed from In Progress to Resolved

#12 Updated by Katja Luther almost 2 years ago

  • Status changed from Resolved to In Progress

bugs are fixed and tests are added

#13 Updated by Andreas Kohlbecker almost 2 years ago

There are at least two other problems. I extended the according service tests. You will find in the cdmlib|cc95089 some FIXME sections which are commented which contain the failing test cases.

#14 Updated by Katja Luther almost 2 years ago

  • Status changed from In Progress to Resolved

#15 Updated by Andreas Kohlbecker almost 2 years ago

  • Description updated (diff)

#16 Updated by Katja Luther almost 2 years ago

  • Target version changed from Release 4.6 to Release 4.5

#17 Updated by Andreas Müller over 1 year ago

  • Assignee changed from Katja Luther to Andreas Kohlbecker

Andreas K, could you please review if this is fully fixed now?

#18 Updated by Andreas Müller over 1 year ago

if not, please open new ticket with remaining issues

#19 Updated by Andreas Kohlbecker 6 months ago

  • Assignee changed from Andreas Kohlbecker to Katja Luther
  • % Done changed from 50 to 100

duplicates are no longer returned, so this issue can in principle be closed, however, as last step cleaning up commented lines in the committed code would be great!
@Katja, please close this issue once the code cleaning is done.

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 40 MB)