XStream generates XML that is not well-formed (according to the XML specification) by writing illegal characters in CHARACTER sequences

If you look at the XML code above, you will probably not see the XML entities & x 1 ; but the real character 0, this is because your browser is replacing the XML entity. The XML is really looking like this:

<xstreamEncodingTest.TestClass>
<chars>& #x1;& #x2;& #x3;</chars>
<s>& #x4;& #x5;& #x6;</s>
</xstreamEncodingTest.TestClass>

(I've inserted spaces to prevent the browser of replacing the entities!)

The behavior and what you can do about it is documented in the FAQ:
http://xstream.codehaus.org/faq.html#XML_null_char
http://xstream.codehaus.org/faq.html#XML_control_char

I disagree on that. Your answer in the FAQ is not a solution for the issue. XStream seams to be inconsistent with the XML version it generates.

• The Dom4JWriter already generates an header with version="1.0", but writes out control sequences any way. This is a bug.
• The PrettyPrintWriter does not write out an XML header. So in theory it is possible to add it by hand, but this is not a good solution.
• And finally the StaxWriter fails completely to write out the data. Obviously it does not support XML 1.1 at all. And then there is also the problem that you don't necessarily know which implementation of the Stax API will be used, because XMLOutputFactory.newInstance() is used to find an implementation, which looks into the Java property "javax.xml.stream.XMLOutputFactory", the file $JAVA_HOME/lib/stax.properties, the file $JAVA_HOME/lib/jaxp.properties, tries to use a JAR service provider and finally falls back to the Sun default implementation "com.sun.xml.internal.stream.XMLOutputFactoryImpl"

So you cannot always just add an XML 1.1 header and even if you can, there is no easy way to ensure, that the writer in use can generate XML 1.1.

I think XStream should be able to either ensure that XML 1.1 will always be used (which means to control what writers are used) or ensure that only (well-formed) XML 1.0 will be generated. Otherwise XStream is unable to serialize all instances of such a simple object as a String.

I would prefer to generate XML 1.0 for several reasons:

• It's hard to control which writer will be used and to check if it is able to generate XML 1.1
• Even if you can ensure it, you won't be able to serialize the String "\0", which is illegal in XML 1.1, too.
• I think that it is easy to implement an StringConverter that uses Java literals as serialized form of the String (converting all characters that are not allowed into \xx or \uxxxx)
• The CharArrayConverter could just call context.convertAnother(new String(characterArray));
• And I don't think that there are so many places in XStream where an arbitrary user defined String is serialized. (Probably the two mentioned converter are the only places that need to be changed)

So I don't want to be annoying but I really think that this is a bug. Therefore I reopened the issue.

regards

I'm going to write such a XML 1.0 compliant StringConverter and provide it here as attachment.

I disagree on that. Your answer in the FAQ is not a solution for the issue. XStream seams to be inconsistent with the XML version it generates.

It has a documented behaviour.

The Dom4JWriter already generates an header with version="1.0", but writes out control sequences any way. This is a bug.

Since Dom4JWriter creates a DOM strcuture, I assume you mean the Dom4JXmlWriter. However, that one is using DOM4J's XmlWriter for the poutput, that has to be setup in this case.

The PrettyPrintWriter does not write out an XML header. So in theory it is possible to add it by hand, but this is not a good solution.

This is by design, because it writes XML snippets, i.e. you should be able to contat the output of XStream into a larger XML file. This is also explained in FAQ.

And finally the StaxWriter fails completely to write out the data. Obviously it does not support XML 1.1 at all.

Here's the same case as with Dom4J, you have to setup the StAX writer. XStream itself is not wrapping the setup of the different XML writer implementations.

And then there is also the problem that you don't necessarily know which implementation of the Stax API will be used, because XMLOutputFactory.newInstance() is used to find an implementation, which looks into the Java property "javax.xml.stream.XMLOutputFactory", the file $JAVA_HOME/lib/stax.properties, the file $JAVA_HOME/lib/jaxp.properties, tries to use a JAR service provider and finally falls back to the Sun default implementation "com.sun.xml.internal.stream.XMLOutputFactoryImpl"

So you cannot always just add an XML 1.1 header and even if you can, there is no easy way to ensure, that the writer in use can generate XML 1.1.

With the upcoming version 1.4 you can either rely on the Java SPI or select the StAX implementation explicitly.

I think XStream should be able to either ensure that XML 1.1 will always be used (which means to control what writers are used) or ensure that only (well-formed) XML 1.0 will be generated. Otherwise XStream is unable to serialize all instances of such a simple object as a String.

That's true, but there is no generic solution. See below.

I would prefer to generate XML 1.0 for several reasons:

It's hard to control which writer will be used and to check if it is able to generate XML 1.1

Even if you can ensure it, you won't be able to serialize the String "\0", which is illegal in XML 1.1, too.

I think that it is easy to implement an StringConverter that uses Java literals as serialized form of the String (converting all characters that are not allowed into \xx or \uxxxx)

You imply here

1. that each consumer of the XML is again a Java client or at least a client that understands this notation and can decode it and
2. that the result of the StringConverter is actually used within an XML format

Both assupmtions are not valid. But you are always free to register such a converter.

The CharArrayConverter could just call context.convertAnother(new String(characterArray));

As said, the converter may not make any assumptions about the target format at all.

And I don't think that there are so many places in XStream where an arbitrary user defined String is serialized. (Probably the two mentioned converter are the only places that need to be changed)

So I don't want to be annoying but I really think that this is a bug. Therefore I reopened the issue.

Believe me, this has been come up before and we went through some circles with it in the last years. The current approach is the most pragmatic. If you know that your Strings in a member may contain such stuff, you may register a local converter that encodes the String and if you care about the XML header and well-formed documents you can also setup the PPW to produce it.