Project

General

Profile

Actions

bug #6389

closed

Returning a Map in cdmlib leads to data loss

Added by Patrick Plitzner almost 7 years ago. Updated over 5 years ago.

Status:
Closed
Priority:
Highest
Assignee:
Patrick Plitzner
Category:
taxeditor
Target version:
Start date:
Due date:
% Done:

100%

Estimated time:
Severity:
critical
Found in Version:

Description

I added the following method to IService/ServiceBase. I then invoked it on the taxeditor side in different places. The returned map on the taxeditor side never has the expected size of i but only contains the first half of the entries. On the cdmlib side it is completely filled.

    public Map returnMap() {
        Map map = new HashMap<>();
        for(int i=0;i<10;i++){
            map.put(i, i);
        }
        return map;
    }

In a quick search in the service methods I only found save methods also returning maps but I am not sure if the return values are used on the taxeditor side.

In Campanula production DB search for "India, Sikkim*" in the specimen editor. The type icon for the DerivedUnits seem to be placed randomly every time you search.
After a little investigation I found out that it was due to IOccurrenceService.listTypeDesignations(Collection specimens). The returned collection of SpecimenTypeDesignation is not equal to the one that you could create by iterating through all DerivedUnits and adding the result of DerivedUnit.getSpecimenTypeDesignations().
As this is nevertheless a very resource intensive call just for showing icons I replaced it.
But anyway this should be analyzed.

Actions #1

Updated by Andreas Müller almost 7 years ago

Check if this method is used elsewhere. If not make it deprecated with reason that it currently gives not the expected results but may be fixed in future and link to the ticket and set to Reviewed with priority 11.

Actions #2

Updated by Andreas Müller almost 7 years ago

  • Priority changed from New to Highest
  • Target version changed from Unassigned CDM tickets to Release 4.6
Actions #3

Updated by Andreas Müller over 6 years ago

  • Target version changed from Release 4.6 to Release 4.7
Actions #4

Updated by Andreas Müller over 6 years ago

  • Target version changed from Release 4.7 to Release 4.8
Actions #5

Updated by Patrick Plitzner over 6 years ago

  • Target version changed from Release 4.8 to Release 4.9
Actions #6

Updated by Andreas Müller over 6 years ago

  • Target version changed from Release 4.9 to Release 4.10
Actions #7

Updated by Andreas Müller about 6 years ago

  • Target version changed from Release 4.10 to Release 4.11
Actions #8

Updated by Andreas Müller about 6 years ago

  • Target version changed from Release 4.11 to Release 4.12
Actions #9

Updated by Andreas Müller about 6 years ago

  • Target version changed from Release 4.12 to Release 4.13
Actions #10

Updated by Andreas Müller almost 6 years ago

  • Target version changed from Release 4.13 to Release 4.14
Actions #11

Updated by Andreas Müller almost 6 years ago

  • Target version changed from Release 4.14 to Release 5.0
Actions #12

Updated by Andreas Müller almost 6 years ago

  • Description updated (diff)
Actions #13

Updated by Andreas Müller almost 6 years ago

  • Description updated (diff)
Actions #14

Updated by Patrick Plitzner almost 6 years ago

  • Subject changed from listTypeDesignations() returns undeterministic results in specimen editor to Returning a Map in cdmlib leads to data loss
  • Severity changed from normal to critical
Actions #15

Updated by Patrick Plitzner almost 6 years ago

  • Description updated (diff)
Actions #16

Updated by Patrick Plitzner almost 6 years ago

  • Description updated (diff)
Actions #17

Updated by Andreas Kohlbecker almost 6 years ago

  • Status changed from New to Feedback

To me it looks like this problem is related to Spring HTTPInvoker and the serilazization/de-serialization of maps.

For serialization and de-serialization of Maps the equals() and hashCode() methods are crucial and that the types of value and key are implementing Serializable. In your test code you are using int as key and value. But maps hold Objects, so internally java will need to convert these into Objects by un-/boxing.

First of all I would rewrite the test method, so that the map is type save and so that un-/boxing is not needed:


    public Map<Integer, Integer> returnMap() {
        Map<Integer, Integer> map = new HashMap<>();
        for(int i=0;i<10;i++){
            map.put(Integer.valueOf(i), Integer.valueOf(i));
        }
        return map;
    }

see also e.g.:

https://stackoverflow.com/questions/2747819/serializing-and-deserializing-a-map-with-key-as-string#2747911

Actions #18

Updated by Patrick Plitzner almost 6 years ago

There is no change with using Integer.valueOf(i)

Actions #19

Updated by Andreas Kohlbecker almost 6 years ago

Patrick Plitzner wrote:

There is no change with using Integer.valueOf(i)

Ok, this is good news!
Maybe it helps to take a look at the serialization and deserialization of Objects in HttpInvoker.

BTW: You could also test if the serialization and deserialization with jdk is working, see ObjectStreams.

Actions #20

Updated by Patrick Plitzner almost 6 years ago

Tested serialization and deserialization for plain java with the following:

    public static void main(String args[]) {

        HashMap<Integer, Integer> map = new HashMap<>();
        for(int i=0;i<100;i++){
            map.put(Integer.valueOf(i),Integer.valueOf(i));
        }
        try{
            FileOutputStream fos =
                    new FileOutputStream("map.file");
            ObjectOutputStream oos = new ObjectOutputStream(fos);
            oos.writeObject(map);
            oos.close();
            fos.close();
            System.out.println("Serialized HashMap data is saved in map.file");

            map = null;
            FileInputStream fis = new FileInputStream("map.file");
            ObjectInputStream ois = new ObjectInputStream(fis);
            map = (HashMap) ois.readObject();
            ois.close();
            fis.close();
        }catch(Exception e) {
            e.printStackTrace();
            return;
        }
        System.out.println("Deserialized HashMap");
        map.forEach((key, value)->System.out.println("key: "+ key + ",  value: "+value));
    }

This works fine. So it must have something to do with the serialization process of spring.

Actions #21

Updated by Patrick Plitzner almost 6 years ago

The problem has nothing to do with the deserialization but with CacheLoader.load(Map map, List alreadyVisitedEntities, boolean update) where the Map is not reconstructed correctly.

Actions #22

Updated by Patrick Plitzner almost 6 years ago

  • Status changed from Feedback to Resolved
  • % Done changed from 0 to 50
Actions #23

Updated by Patrick Plitzner almost 6 years ago

  • Assignee changed from Patrick Plitzner to Andreas Kohlbecker
  • % Done changed from 50 to 0

In the CacheLoader class the map was deconstructed into an array of twice the size containing keys and values alternating. Then the map was re-built but the iteration was only over half the size of the array. So the resulting map was always half the size and only contained the first half of entries.
I checked for cdmlib service methods that have a Map as return value and there are also some, but few, methods, other than save methods, that use Maps and that are also used in the taxeditor. So if there were any bugs related to these methods they could also be solved with this fix.

Actions #24

Updated by Patrick Plitzner almost 6 years ago

  • Description updated (diff)
Actions #25

Updated by Andreas Kohlbecker over 5 years ago

  • Status changed from Resolved to Closed
  • Assignee changed from Andreas Kohlbecker to Patrick Plitzner
  • % Done changed from 0 to 100

looks ok and I applied your changeset 98baf5f onto cdmlib-cache

Actions

Also available in: Atom PDF