[XSTR-767] Serializing lambda expressions fails with XStream and JDK 1.8u40

Details

Type: Bug
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: 1.4.7
Fix Version/s: 1.4.8
Component/s: Core
Labels:
None

Description

It seems we're not able to deserialize Lambda expressions with XStream 1.4.7

The code in question looks roughly like this:

@FunctionalInterface
public interface Visible extends Serializable

{ public boolean isVisible(); }

public interface IMenuItem extends Serializable

{ // ... stuff omitted ... }

public abstract class AbstractMenuItem implements IMenuItem

{ private final Visible visibilityFunction; // ... stuff omitted ... }

The method that is invoked with lambda expressions (that later get stored in the 'visibilityFunction' field of an AbstractMenuItem subclass instance) looks like this:

public <T extends Page> void addEntry(String resourceLabel , Class<T> pageClazz,Visible visibilityFunction);

Since the Visible interface extends Serializable , the compiler generates code for a serializable Lambda (and in fact it does, I checked the generated .class file with javap).

The code used to setup XStream looks like this:

private XStream createXStream()
{
final XStream xstream = new XStream( new StaxDriver() ) {
@Override
protected MapperWrapper wrapMapper(MapperWrapper next)

{ return new WicketProxyMapper( new CGLIBMapper( new HibernateMapper(next) ) ); // }

};

// override the built-in DynamicProxyConverter that chokes on Apache Wicket proxies
xstream.registerConverter( new WicketProxyConverter() , XStream.PRIORITY_VERY_HIGH );

// register Hibernate-specific converters
xstream.registerConverter(new HibernateProxyConverter());
xstream.registerConverter(new HibernatePersistentCollectionConverter(xstream.getMapper()));
xstream.registerConverter(new HibernatePersistentMapConverter(xstream.getMapper()));
xstream.registerConverter(new HibernatePersistentSortedMapConverter(xstream.getMapper()));
xstream.registerConverter(new HibernatePersistentSortedSetConverter(xstream.getMapper()));

// register CGLLib converters
xstream.registerConverter(new CGLIBEnhancedConverter(xstream.getMapper(), xstream.getReflectionProvider() , getClass().getClassLoader() ) );

return xstream;
}

Options
- Sort By Name
- Sort By Date
- Ascending
- Descending
- Download All

Attachments

xstream.xml.gz

07/Jan/15 4:24 AM

206 kB

Tobias Gierke

Activity

Hide

Permalink

Tobias Gierke added a comment - 07/Jan/15 4:24 AM

generated XML

Show

Tobias Gierke added a comment - 07/Jan/15 4:24 AM generated XML

Tobias Gierke made changes - 07/Jan/15 4:24 AM

Field	Original Value	New Value
Attachment		xstream.xml.gz [ 67034 ]

Hide

Permalink

Tobias Gierke added a comment - 07/Jan/15 4:27 AM

Here's the accompanying stacktrace:

com.thoughtworks.xstream.converters.ConversionException: com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$$Lambda$130/1880580028 : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$$Lambda$130/1880580028
---- Debugging information ----
message : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$$Lambda$130/1880580028
cause-exception : com.thoughtworks.xstream.mapper.CannotResolveClassException
cause-message : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$$Lambda$130/1880580028
class : com.vodecc.voipmng.boundary.wicket.general.PageWithHeaderAndFooter$11
required-type : com.vodecc.voipmng.boundary.wicket.general.PageWithHeaderAndFooter$11
converter-type : com.thoughtworks.xstream.converters.reflection.ReflectionConverter
path : /com.vodecc.voipmng.boundary.wicket.qosmng.ServiceQualityPage/children/com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$1/children/com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$1$1[2]/children/com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$SubMenuList/children/children/org.apache.wicket.markup.html.list.ListItem/children/children/com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$NestedListEntry$1/children/org.apache.wicket.markup.html.list.ListItem[3]/children/children/val$visibilityFunction
line number : 1
class[1] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$SimpleListEntry
class[2] : org.apache.wicket.markup.html.list.ListItem
class[3] : [Ljava.lang.Object;
converter-type[1] : com.thoughtworks.xstream.converters.collections.ArrayConverter
class[4] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$NestedListEntry$1
class[5] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$NestedListEntry
class[6] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$SubMenuList$1
class[7] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$SubMenuList
class[8] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$1$1
class[9] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$1
class[10] : com.vodecc.voipmng.boundary.wicket.qosmng.ServiceQualityPage
version : not available
-------------------------------
at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:79)
at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:65)
at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66)
at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.unmarshallField(AbstractReflectionConverter.java:474)
at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.doUnmarshal(AbstractReflectionConverter.java:406)
at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.unmarshal(AbstractReflectionConverter.java:257)
at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72)
at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:65)
...
... STACKTRACE ABBREVIATED
...
at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32)
at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1185)
at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1169)
at com.thoughtworks.xstream.XStream.fromXML(XStream.java:1049)

Show

Tobias Gierke added a comment - 07/Jan/15 4:27 AM Here's the accompanying stacktrace: com.thoughtworks.xstream.converters.ConversionException: com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$$Lambda$130/1880580028 : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$$Lambda$130/1880580028 ---- Debugging information ---- message : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$$Lambda$130/1880580028 cause-exception : com.thoughtworks.xstream.mapper.CannotResolveClassException cause-message : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$$Lambda$130/1880580028 class : com.vodecc.voipmng.boundary.wicket.general.PageWithHeaderAndFooter$11 required-type : com.vodecc.voipmng.boundary.wicket.general.PageWithHeaderAndFooter$11 converter-type : com.thoughtworks.xstream.converters.reflection.ReflectionConverter path : /com.vodecc.voipmng.boundary.wicket.qosmng.ServiceQualityPage/children/com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$1/children/com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$1$1 [2] /children/com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$SubMenuList/children/children/org.apache.wicket.markup.html.list.ListItem/children/children/com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$NestedListEntry$1/children/org.apache.wicket.markup.html.list.ListItem [3] /children/children/val$visibilityFunction line number : 1 class [1] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$SimpleListEntry class [2] : org.apache.wicket.markup.html.list.ListItem class [3] : [Ljava.lang.Object; converter-type [1] : com.thoughtworks.xstream.converters.collections.ArrayConverter class [4] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$NestedListEntry$1 class [5] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$NestedListEntry class [6] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$SubMenuList$1 class [7] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$SubMenuList class [8] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$1$1 class [9] : com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$1 class [10] : com.vodecc.voipmng.boundary.wicket.qosmng.ServiceQualityPage version : not available ------------------------------- at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:79) at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:65) at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66) at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.unmarshallField(AbstractReflectionConverter.java:474) at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.doUnmarshal(AbstractReflectionConverter.java:406) at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.unmarshal(AbstractReflectionConverter.java:257) at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72) at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:65) ... ... STACKTRACE ABBREVIATED ... at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32) at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1185) at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1169) at com.thoughtworks.xstream.XStream.fromXML(XStream.java:1049)

Hide

Permalink

Jörg Schaible added a comment - 10/Jan/15 9:01 AM - edited

Hi Tobias,

some general remarks about lambda expressions first:

XStream will never be able to unmarshal XML generated for unserializable lambda expressions, since the object (and therefore the XML) lacks any useful information to recreate such an instance.
A serialized lambda expression uses a replacement object of type java.lang.invoke.SerializedLambda. XStream is already capable of marshalling and unmarshalling such serializable lambda expressions.
The class name of the recreated lambda expression is different, it seems those names are kind of temporarily unique.
The information of the SerializedLambda seems compiler specific. For me the value "implMethodName" name differs depending on the compiler. The internal Eclipse compiler has generated a name like "lambda$0" while my Oracle JDK 8 (and 9) compiler generates a name like "lamdba$theMethodName$hex$1". As result one of my lambda serialization test fails in Eclipse, when I used the original compiler to generate the class file or in Maven's surefire, when Eclipse created it (NPE in SerializedLambda#readResolve). Might be an Eclipse compiler bug, but I've recognized that your XML contains names like generated from Eclipse compiled classes.

However, you have a more specific problem. The exception provides the XPath to the problematic XML element. If you format the provided XML (you gzipped the attachment twice), it is in line 1840:

<val_-visibilityFunction class="com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$$Lambda$130/1880580028" reference="../../../../../../../../../../../../../../../data/com.vodecc.voipmng.boundary.wicket.general.menu.MenuBar/children/list/com.vodecc.voipmng.boundary.wicket.general.menu.Menu_-WicketMenu[2]/children/list/com.vodecc.voipmng.boundary.wicket.general.menu.Menu_-WicketMenu/children/list/com.vodecc.voipmng.boundary.wicket.general.menu.MenuEntry_-2[3]/val_-visibilityFunction"/>

While you marshal serializable lambda expressions, you try to use the same instance in multiple places and XStream therefore references it. However, since class names of lambda expressions cannot be loaded, it fails here. I'll have to investigate further.

Show

Jörg Schaible added a comment - 10/Jan/15 9:01 AM - edited Hi Tobias, some general remarks about lambda expressions first: XStream will never be able to unmarshal XML generated for unserializable lambda expressions, since the object (and therefore the XML) lacks any useful information to recreate such an instance. A serialized lambda expression uses a replacement object of type java.lang.invoke.SerializedLambda. XStream is already capable of marshalling and unmarshalling such serializable lambda expressions. The class name of the recreated lambda expression is different, it seems those names are kind of temporarily unique. The information of the SerializedLambda seems compiler specific. For me the value "implMethodName" name differs depending on the compiler. The internal Eclipse compiler has generated a name like "lambda$0" while my Oracle JDK 8 (and 9) compiler generates a name like "lamdba$theMethodName$hex$1". As result one of my lambda serialization test fails in Eclipse, when I used the original compiler to generate the class file or in Maven's surefire, when Eclipse created it (NPE in SerializedLambda#readResolve). Might be an Eclipse compiler bug, but I've recognized that your XML contains names like generated from Eclipse compiled classes. However, you have a more specific problem. The exception provides the XPath to the problematic XML element. If you format the provided XML (you gzipped the attachment twice), it is in line 1840: <val_-visibilityFunction class= "com.vodecc.voipmng.boundary.wicket.general.PageWithMenu$$Lambda$130/1880580028" reference= "../../../../../../../../../../../../../../../data/com.vodecc.voipmng.boundary.wicket.general.menu.MenuBar/children/list/com.vodecc.voipmng.boundary.wicket.general.menu.Menu_-WicketMenu[2]/children/list/com.vodecc.voipmng.boundary.wicket.general.menu.Menu_-WicketMenu/children/list/com.vodecc.voipmng.boundary.wicket.general.menu.MenuEntry_-2[3]/val_-visibilityFunction" /> While you marshal serializable lambda expressions, you try to use the same instance in multiple places and XStream therefore references it. However, since class names of lambda expressions cannot be loaded, it fails here. I'll have to investigate further.

Hide

Permalink

Jörg Schaible added a comment - 10/Jan/15 6:16 PM

I've added a new acceptance test to HEAD (LambdaTest) where I use a custom mapper to support the referencing case. That should also work for the 1.4.x line.

Show

Jörg Schaible added a comment - 10/Jan/15 6:16 PM I've added a new acceptance test to HEAD (LambdaTest) where I use a custom mapper to support the referencing case. That should also work for the 1.4.x line.

Hide

Permalink

Jörg Schaible added a comment - 28/Jan/15 5:42 PM

General support for lambda expressions now available in trunk and 1.4.x branch covering several other cases also.

Show

Jörg Schaible added a comment - 28/Jan/15 5:42 PM General support for lambda expressions now available in trunk and 1.4.x branch covering several other cases also.

Jörg Schaible made changes - 28/Jan/15 5:42 PM

Resolution		Fixed [ 1 ]
Fix Version/s		1.4.x Maintenance [ 19602 ]
Status	Open [ 1 ]	Resolved [ 5 ]

Jörg Schaible made changes - 18/Feb/15 12:28 PM

Fix Version/s	1.4.x Maintenance [ 19602 ]
Fix Version/s		1.4.8 [ 20992 ]

Jörg Schaible made changes - 18/Feb/15 12:29 PM

Status

Resolved [ 5 ]

Closed [ 6 ]

People

Assignee:

Jörg Schaible

Reporter:

Tobias Gierke

Votes:

0 Vote for this issue

Watchers:

2 Start watching this issue

Dates

Created:

07/Jan/15 3:04 AM

Updated:

18/Feb/15 12:29 PM

Resolved:

28/Jan/15 5:42 PM