XStream
  1. XStream
  2. XSTR-767

Serializing lambda expressions fails with XStream and JDK 1.8u40

    Details

    • Type: Bug Bug
    • Status: Closed Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.7
    • Fix Version/s: 1.4.8
    • Component/s: Core
    • Labels:
      None

      Description

      It seems we're not able to deserialize Lambda expressions with XStream 1.4.7

      The code in question looks roughly like this:

      @FunctionalInterface
      public interface Visible extends Serializable

      { public boolean isVisible(); }

      public interface IMenuItem extends Serializable

      { // ... stuff omitted ... }

      public abstract class AbstractMenuItem implements IMenuItem

      { private final Visible visibilityFunction; // ... stuff omitted ... }

      The method that is invoked with lambda expressions (that later get stored in the 'visibilityFunction' field of an AbstractMenuItem subclass instance) looks like this:

      public <T extends Page> void addEntry(String resourceLabel , Class<T> pageClazz,Visible visibilityFunction);

      Since the Visible interface extends Serializable , the compiler generates code for a serializable Lambda (and in fact it does, I checked the generated .class file with javap).

      The code used to setup XStream looks like this:

      private XStream createXStream()
      {
      final XStream xstream = new XStream( new StaxDriver() ) {
      @Override
      protected MapperWrapper wrapMapper(MapperWrapper next)

      { return new WicketProxyMapper( new CGLIBMapper( new HibernateMapper(next) ) ); // }

      };

      // override the built-in DynamicProxyConverter that chokes on Apache Wicket proxies
      xstream.registerConverter( new WicketProxyConverter() , XStream.PRIORITY_VERY_HIGH );

      // register Hibernate-specific converters
      xstream.registerConverter(new HibernateProxyConverter());
      xstream.registerConverter(new HibernatePersistentCollectionConverter(xstream.getMapper()));
      xstream.registerConverter(new HibernatePersistentMapConverter(xstream.getMapper()));
      xstream.registerConverter(new HibernatePersistentSortedMapConverter(xstream.getMapper()));
      xstream.registerConverter(new HibernatePersistentSortedSetConverter(xstream.getMapper()));

      // register CGLLib converters
      xstream.registerConverter(new CGLIBEnhancedConverter(xstream.getMapper(), xstream.getReflectionProvider() , getClass().getClassLoader() ) );

      return xstream;
      }

        Activity

        Hide
        Tobias Gierke added a comment -

        generated XML

        Show
        Tobias Gierke added a comment - generated XML
        Tobias Gierke made changes -
        Field Original Value New Value
        Attachment xstream.xml.gz [ 67034 ]
        Hide
        Tobias Gierke added a comment -

        Here's the accompanying stacktrace:

        com.thoughtworks.xstream.converters.ConversionException: com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$$Lambda$130/1880580028 : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$$Lambda$130/1880580028
        ---- Debugging information ----
        message : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$$Lambda$130/1880580028
        cause-exception : com.thoughtworks.xstream.mapper.CannotResolveClassException
        cause-message : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$$Lambda$130/1880580028
        class : com.vodecc.voipmng.boundary.wicket.general.PageWithHeaderAndFooter$11
        required-type : com.vodecc.voipmng.boundary.wicket.general.PageWithHeaderAndFooter$11
        converter-type : com.thoughtworks.xstream.converters.reflection.ReflectionConverter
        path : /com.vodecc.voipmng.boundary.wicket.qosmng.ServiceQualityPage/children/com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$1/children/com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$1$1[2]/children/com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$SubMenuList/children/children/org.apache.wicket.markup.html.list.ListItem/children/children/com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$NestedListEntry$1/children/org.apache.wicket.markup.html.list.ListItem[3]/children/children/val$visibilityFunction
        line number : 1
        class[1] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$SimpleListEntry
        class[2] : org.apache.wicket.markup.html.list.ListItem
        class[3] : [Ljava.lang.Object;
        converter-type[1] : com.thoughtworks.xstream.converters.collections.ArrayConverter
        class[4] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$NestedListEntry$1
        class[5] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$NestedListEntry
        class[6] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$SubMenuList$1
        class[7] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$SubMenuList
        class[8] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$1$1
        class[9] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$1
        class[10] : com.vodecc.voipmng.boundary.wicket.qosmng.ServiceQualityPage
        version : not available
        -------------------------------
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:79)
        at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:65)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66)
        at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.unmarshallField(AbstractReflectionConverter.java:474)
        at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.doUnmarshal(AbstractReflectionConverter.java:406)
        at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.unmarshal(AbstractReflectionConverter.java:257)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72)
        at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:65)
        ...
        ... STACKTRACE ABBREVIATED
        ...
        at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32)
        at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1185)
        at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1169)
        at com.thoughtworks.xstream.XStream.fromXML(XStream.java:1049)

        Show
        Tobias Gierke added a comment - Here's the accompanying stacktrace: com.thoughtworks.xstream.converters.ConversionException: com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$$Lambda$130/1880580028 : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$$Lambda$130/1880580028 ---- Debugging information ---- message : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$$Lambda$130/1880580028 cause-exception : com.thoughtworks.xstream.mapper.CannotResolveClassException cause-message : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$$Lambda$130/1880580028 class : com.vodecc.voipmng.boundary.wicket.general.PageWithHeaderAndFooter$11 required-type : com.vodecc.voipmng.boundary.wicket.general.PageWithHeaderAndFooter$11 converter-type : com.thoughtworks.xstream.converters.reflection.ReflectionConverter path : /com.vodecc.voipmng.boundary.wicket.qosmng.ServiceQualityPage/children/com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$1/children/com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$1$1 [2] /children/com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$SubMenuList/children/children/org.apache.wicket.markup.html.list.ListItem/children/children/com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$NestedListEntry$1/children/org.apache.wicket.markup.html.list.ListItem [3] /children/children/val$visibilityFunction line number : 1 class [1] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$SimpleListEntry class [2] : org.apache.wicket.markup.html.list.ListItem class [3] : [Ljava.lang.Object; converter-type [1] : com.thoughtworks.xstream.converters.collections.ArrayConverter class [4] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$NestedListEntry$1 class [5] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$NestedListEntry class [6] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$SubMenuList$1 class [7] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$SubMenuList class [8] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$1$1 class [9] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$1 class [10] : com.vodecc.voipmng.boundary.wicket.qosmng.ServiceQualityPage version : not available ------------------------------- at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:79) at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:65) at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66) at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.unmarshallField(AbstractReflectionConverter.java:474) at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.doUnmarshal(AbstractReflectionConverter.java:406) at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.unmarshal(AbstractReflectionConverter.java:257) at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72) at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:65) ... ... STACKTRACE ABBREVIATED ... at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32) at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1185) at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1169) at com.thoughtworks.xstream.XStream.fromXML(XStream.java:1049)
        Hide
        Jörg Schaible added a comment - - edited

        Hi Tobias,

        some general remarks about lambda expressions first:

        1. XStream will never be able to unmarshal XML generated for unserializable lambda expressions, since the object (and therefore the XML) lacks any useful information to recreate such an instance.
        2. A serialized lambda expression uses a replacement object of type java.lang.invoke.SerializedLambda. XStream is already capable of marshalling and unmarshalling such serializable lambda expressions.
        3. The class name of the recreated lambda expression is different, it seems those names are kind of temporarily unique.
        4. The information of the SerializedLambda seems compiler specific. For me the value "implMethodName" name differs depending on the compiler. The internal Eclipse compiler has generated a name like "lambda$0" while my Oracle JDK 8 (and 9) compiler generates a name like "lamdba$theMethodName$hex$1". As result one of my lambda serialization test fails in Eclipse, when I used the original compiler to generate the class file or in Maven's surefire, when Eclipse created it (NPE in SerializedLambda#readResolve). Might be an Eclipse compiler bug, but I've recognized that your XML contains names like generated from Eclipse compiled classes.

        However, you have a more specific problem. The exception provides the XPath to the problematic XML element. If you format the provided XML (you gzipped the attachment twice), it is in line 1840:

        <val_-visibilityFunction class="com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$$Lambda$130/1880580028" reference="../../../../../../../../../../../../../../../data/com.vodecc.voipmng.boundary.wicket.general.menu.MenuBar/children/list/com.vodecc.voipmng.boundary.wicket.general.menu.Menu_-WicketMenu[2]/children/list/com.vodecc.voipmng.boundary.wicket.general.menu.Menu_-WicketMenu/children/list/com.vodecc.voipmng.boundary.wicket.general.menu.MenuEntry_-2[3]/val_-visibilityFunction"/>
        

        While you marshal serializable lambda expressions, you try to use the same instance in multiple places and XStream therefore references it. However, since class names of lambda expressions cannot be loaded, it fails here. I'll have to investigate further.

        Show
        Jörg Schaible added a comment - - edited Hi Tobias, some general remarks about lambda expressions first: XStream will never be able to unmarshal XML generated for unserializable lambda expressions, since the object (and therefore the XML) lacks any useful information to recreate such an instance. A serialized lambda expression uses a replacement object of type java.lang.invoke.SerializedLambda. XStream is already capable of marshalling and unmarshalling such serializable lambda expressions. The class name of the recreated lambda expression is different, it seems those names are kind of temporarily unique. The information of the SerializedLambda seems compiler specific. For me the value "implMethodName" name differs depending on the compiler. The internal Eclipse compiler has generated a name like "lambda$0" while my Oracle JDK 8 (and 9) compiler generates a name like "lamdba$theMethodName$hex$1". As result one of my lambda serialization test fails in Eclipse, when I used the original compiler to generate the class file or in Maven's surefire, when Eclipse created it (NPE in SerializedLambda#readResolve). Might be an Eclipse compiler bug, but I've recognized that your XML contains names like generated from Eclipse compiled classes. However, you have a more specific problem. The exception provides the XPath to the problematic XML element. If you format the provided XML (you gzipped the attachment twice), it is in line 1840: <val_-visibilityFunction class= "com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$$Lambda$130/1880580028" reference= "../../../../../../../../../../../../../../../data/com.vodecc.voipmng.boundary.wicket.general.menu.MenuBar/children/list/com.vodecc.voipmng.boundary.wicket.general.menu.Menu_-WicketMenu[2]/children/list/com.vodecc.voipmng.boundary.wicket.general.menu.Menu_-WicketMenu/children/list/com.vodecc.voipmng.boundary.wicket.general.menu.MenuEntry_-2[3]/val_-visibilityFunction" /> While you marshal serializable lambda expressions, you try to use the same instance in multiple places and XStream therefore references it. However, since class names of lambda expressions cannot be loaded, it fails here. I'll have to investigate further.
        Hide
        Jörg Schaible added a comment -

        I've added a new acceptance test to HEAD (LambdaTest) where I use a custom mapper to support the referencing case. That should also work for the 1.4.x line.

        Show
        Jörg Schaible added a comment - I've added a new acceptance test to HEAD (LambdaTest) where I use a custom mapper to support the referencing case. That should also work for the 1.4.x line.
        Hide
        Jörg Schaible added a comment -

        General support for lambda expressions now available in trunk and 1.4.x branch covering several other cases also.

        Show
        Jörg Schaible added a comment - General support for lambda expressions now available in trunk and 1.4.x branch covering several other cases also.
        Jörg Schaible made changes -
        Resolution Fixed [ 1 ]
        Fix Version/s 1.4.x Maintenance [ 19602 ]
        Status Open [ 1 ] Resolved [ 5 ]
        Jörg Schaible made changes -
        Fix Version/s 1.4.x Maintenance [ 19602 ]
        Fix Version/s 1.4.8 [ 20992 ]
        Jörg Schaible made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Jörg Schaible
            Reporter:
            Tobias Gierke
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: