Project

General

Profile

Actions
History

WikiStart » DataModelsAndFormats » CommonDataModel »

CdmVersionTwoDiscussion » History » Revision 51

« Previous | Revision 51/77 (diff) | Next »
Helene Fradin, 03/19/2009 11:08 AM

CDM v2.0 Discussion

This is a site to discuss possible changes to the CDM v1.4 to go into CDM v2.0

See also Component C5.80 - Review of CDM v.1 and model for descriptive data in CDM v.2

DESCRIPTIVE DATA - PROPOSED REVISIONS

1. MAJOR - Character/Descriptor/Feature concept

Impacted objects: Feature

The Feature class is described in the class comments by: "The class for individual properties (also designed as character, type or category) of observed phenomena able to be described or measured."

a. Issues

It is very interesting that the object Feature is not typed such as Characters in SDD (Categorical, Quantitative, etc.) or many other models. However, if the information is needed as to what kind of data is supported by a certain Feature, it is not clearly stated how to understand and use the different attributes. Moreover, there are a dozen categories of Features (Additional Publication, Image, Cultivation, Description, ...) that are rich but difficult to interpret in the case of the import.

As a reminder, below is the list of the Feature class attributes:

  • supportsTextData -> feature can be described with TextData objects

  • supportsQuantitativeData -> feature can be described with QuantitativeData objects

  • supportsDistribution -> feature can be described with Distribution objects (geographical)

  • supportsIndividualAssociation > feature can be described with IndividualsAssociation objects (between the described specimen and a second one - for instance a host, only for SpecimenDescription)

  • supportsTaxonInteraction > feature can be described with TaxonInteraction objects (between the described Taxon and a second one - for instance a parasite, a prey or a hybrid parent, only for TaxonDescription)

  • supportsCommonTaxonName -> feature can be described with CommonTaxonName objects

  • recommendedModifierEnumeration -> set of TermVocabulary containing the Modifier objects recommended to be used for DescriptionElementBase elements

  • recommendedStatisticalMeasures -> set of StatisticalMeasure recommended to be used in case of QuantitativeData

  • supportedCategoricalEnumerations -> set of TermVocabulary containing the list of possible State to be used in CategoricalData

The flexibility of the Feature class is not a problem for the import of SDD descriptive data: for each character, a new DESCRIPTION Feature instance is created:

  • for SDD CategoricalCharacter, supportedCategoricalEnumerations is set with the states defined in SDD in the elements StateDefinition

  • for SDD QuantitativeCharacter, supportsQuantitativeData is set to true.

  • for SDD TextCharacter, support supportsTextData is set to true.

  • SDD SequenceCharacter: so far, this data are not imported and I don't have an SDD example of this element being used. I guess it should be imported in a Sequence object?

However, exporting SDD data raises questions about the object Feature. I can see 3 different problems:

  1. There is no safeguard to ensure that DescriptionElementBase objects used for a description tally with the way the corresponding Feature has been described (for example, a DescriptionElementBase associated with a Feature that has only information on supportedCategoricalEnumerations, could be of the type QuantitativeData).

  2. The SDD standard and most descriptive models require the definition of a descriptive system (list of characters, potential states, potential measures) before expressing the strutured descriptions through this descriptive system. It is difficult to export properly this descriptive system to SDD: I can either export all the Feature (but most of them will be non relevant to the exported descriptions), or I can create the descriptive system by scanning all descriptions to extract only characters that are effectively used in the concerned descriptions (loss of efficiency).

  3. In SDD, categorical states do not have to be defined at the Character level, they can be defined at a more general level and shared. Therefore, the supportedCategoricalEnumerations could well be empty: how do we know then that it supports StateData?

b. Example

If we consider the feature (character/descriptor in other models) "Leaf length". Below are examples corresponding to each problem described above:

  1. A new Feature Instance names "Leaf length" is created with the attribute supportsQuantitativeData set to true and supportedCategoricalEnumerations set to null. It is still possible to create a DescriptionElementBase of type CategoricalData with the attribute feature set to "Leaf length" feature, and for example, the attribute states set to a list of StateData containing one item {"small"}. -> A feature described as a quantitative feature is used as a categorical feature.

  2. Exporting 2 descriptions from the CDM, which contain only 1 DescriptionElementBase, such as:

Viola hederacea -> Leaf Length (mm) -> {Min = 2.3, Mean = 5.1, Max = 7.9, SD = 1.3, N = 20}

Viola betonicifolia -> Leaf Length (mm) -> {Min = 2.9, Mean = 5.3, Max = 7.4, SD = 1.3, N = 21}

There might be other Feature instances stored in the CDM ("Leaf complexity", "Body shape", "Flattening of body", ...) related or not to the descriptions of such plants.

Therefore, when exporting the descriptive system, either there will be a majority of non-used features exported, if all feature are exported, or descriptions will have to be scanned one by one to detect only effectively used ones. For the last solution, it is ok with this simple example, but if with potentially hundreds of descriptions and hundreds of characters, the complexity increases quickly.

  1. The states "small", "medium", "large" could be defined as DescriptiveConcept elements in SDD and the CategoricalCharacter "Leaf length" could contain no StateDefinition elements, using the stated defined more generally in CodedDescriptions. In this case, when the character "LeafLength" is imported, a Feature with no supportedCategoricalEnumerations is created. This Feature type is undefined while it supports CategoricalData.

c. Current solution

For now, all Feature instances are exported.

d. Proposed change (NOT IMPLEMENTED)

I think there should be a distinction within Feature attributes, between the type of data supported by the Feature (supportsTextData, supportsQuantitativeData, etc.) and the domain of possible values or frame of reference (recommendedStatisticalMeasures, supportedCategoricalEnumerations).

In practical terms:

  • I would add a boolean to the attribute: 'supportsCategoricalData' (IMPLEMENTED),

  • I would remove the domain of possible values (recommendedModifierEnumeration, recommendedStatisticalMeasures, supportedCategoricalEnumerations) and create a new class that we could call for example PossibleValues or RecommendedValues from which would inherit RecommendedModifiers, RecommendedStates, and RecommendedStatisticalMeasures.

  • I would add an attribute (e.g. PossibleValuesDomains) that would be a Set).

It doesn't prevent problem 1 from happening but at least it clarifies the typing of Feature objects: it is set only through the boolean attributes 'supports...'.

It doesn't resolve problem 2. I would suggest to attach an DescriptiveSystem object to a DescriptionBase object (see item 6).

It resolves problem 3. The typing of the Feature will only depend on the boolean attributes.

Hagedorn - 27/02/2009 One comment on PossibleStatisticalMeasures: at this point both SDD and CDM take the position that all statistical measures known to the system are in principle valid data and thus allowed. At the same time, the designer of a matrix has a valid interest to make a choice of preferred measures. This is the reason why we speak of "recommendedStatisticalMeasures". Example: Leaf Length, Kurtosis = 2.3 is just as valid a statement (although highly unlikely) as Leaf Length, mean = 12.3. However: Flower color = Long is simply wrong. Thus the strict enforcement of possible states.

The base class seems reasonable, I would, however, recommend renaming it from PossibleStates to AvailableStates.

Müller - 27/02/2009 The PossibleValues class seems reasonable to me but instead of having subclasses all having the same structure we could use Java generics instead

Class PossibleValues{

Set supportedValues;

}

and/or something similar for the Vocabulary based supported values and IPossibleValue implemented by all relevant classes like MeasurementUnit and StatisticalMeasure

2. HIGHLY CRITICAL - Mixed properties associated with mixed objects

Impacted objects: all objects inheriting from VersionableEntity

a. Issue

[Gregor I find your observation about the limitation that "essential general properties (title, description, media and original sources) are available only for a restricted number of objects" very interesting. I had some discussions with Markus, trying to get him on erring on the side of allowing sometimes a property which is only necessary under very special use cases, rather than custom tailoring properties to the currently perceived needs. I can understand that Markus wanted to have a clean model, but since in SDD we started doing this, and in the end found that more and more things are shared, we at some point decided to move quite a bit (I am not claiming the fully correct bit) into the abstract base classes.

The "precision" aimed at, is also in my view responsible to the deep class hierarchy, which hinders a ready understanding of the model. From the UML it is difficult to derive which properties some derived classes have, because all inheritance layers contribute.

d. Proposed change (NOT IMPLEMENTED)

I think these properties should be made generic, therefore available at a higher level.

The specific attributes I am thinking of are: representations (Set), media (Set), sources (Set).

To implement this, I can see 2 solutions: a drastic one and a less drastic one.

  • drastic (directly inspired from the use of the SDD Representation element) : the problem is that it would impact the CDM at a high level so I am probably overlooking important issues raised by this.

It consists in having these attributes at the level of the VersionableEntity object. However, as the Representation, Media and OriginalSource classes all inherit from VersionableEntity, they should be removed from this hierarchy of objects and defined independantly.

The new VersionableEntity attribute would be:

representations: Set

and the Representation object, defined independantly, would contain media and sources as attributes.

In parallel, redundant attributes in lower classes could be removed.

Therefore, any CDM object inheriting from VersionabeEntity could be represented in the same way: a title and a description (possibly available in several languages), one or several images attached to the object, and one or several sources.

  • less drastic: to make available these properties largely, they could be put back up in the hierarchy.

I would suggest:

 > adding to TermBase: sources + media

 > adding to Media: representations

 > adding to ReferencedEntityBase: media

 > adding to IdentifiableEntity: representations + media

 > adding to FeatureNode: representations + media + sources

 > removing media from DefinedTermBase

 > removing media from DescriptionElementBase

 > removing media from IdentifiableMediaEntity

Clark suggested that we could make a TermBase an IdentifiableEntity - IdentifiableEntities do have a collection of OriginalSources, and space for the IdInSource.

3. MAJOR - Creation of a defined set of descriptions

Impacted objects: new object

a. Issue

Cf. mail exchanges between Gregor Hagedorn, Ben Clark and Helene Fradin in December 2009 "Keys and descriptions in the CDM".

There is no equivalent way of representing a SDD Dataset into the CDM and multi-access keys.

d. Proposed change (NOT IMPLEMENTED)

The solution proposed by Ben was a delimited set of taxa and their description. It would certainly be helpful for the import/export between SDD and CDM.

[Gregor] Perhaps to generalize this, a working set of taxa and a default character tree (to optionally create a subset of all taxa) could be provided? Such a working set could then carry a flag that it is suitably revised to serve as a multi-access key.

public class WorkingSet {

private Map matrix;

private DescriptiveSystem descriptiveSystem;

private boolean multiAccessKey;

private Language defaultLanguage;

...

}

4. MAJOR - Mapping use and rederential objects

5. MAJOR - Problem how CDM handles the link between description and scientific taxonomic name

a. Issue

The fact that structured descriptions (DescriptionBase objects) cannot always be linked with a scientific taxonomic name raises problems for regrouping related descriptions. If the only possibility to regroup descriptions is by using the association with an existing taxonomic hierarchy, it limits the possibility of extracting sets of descriptions from the CDM. In addition, when importing data into the CDM, the information on potential connections between descriptions other than taxonomic is lost if not structured identically (e.g. use of the Scope class). A model such as SDD uses a Dataset object which contains a set of descriptions that can be tagged with a name, a description and media objects.

6. MINOR - Descriptive system

Impacted objects: DescriptionBase

a. Issue

There is no possibility of associating a set of features/characters/descriptors to a description, or a set of descriptions.

d. Proposed change (IMPLEMENTED as an attribute of DescriptionBase)

To create a new object called DescriptiveSystem which contains at least a set of Feature objects possibly associated with domain of values.

public class DescriptiveSystem {

private Set features;

// OR private Set>;

}

CURRENT INTERMEDIARY IMPLEMENTATION: http://dev.e-taxonomy.eu/trac/attachment/wiki/CdmVersionTwoDiscussion/DescriptionBase.gatcl.PNG

7. MINOR - How to express uncertainty or inapplicability ?

8. MINOR - Handling of multiple languages

9. MINOR - Media properties and associations

IMPLEMENTED

10. MINOR - A default measurement unit for Feature

11. MINOR - Ordering of TermVocabulary for supportedCategoricalEnumerations in Feature

12. MINOR - Why is the setParent function not public in FeatureNode ?

13. MINOR - How to distinguish between characters and groups as they are both Feature objects ?

Should the 'partOf' attribute be used?

14. MINOR - How to export and reimport multi-types characters/features/descriptors between CDM and SDD ?

Updated by Helene Fradin about 15 years ago · 51 revisions