Benchmarks

Introduction

Benchmark results are always dependent on a very individual setup. Normally it is not useful to generalize such results for every use case, but it can give you a hint. However, if you're really in the need of maximum performance, you should probably create an own benchmark with your objects or even use a profiler to detect the real hot spots in your application.

XStream uses the Java Microbenchmark Harness (JMH) of the JDK Tools as benchmark framework starting with version 1.4.9. As result it contains a ZIP file (xstream-jmh-<version>-app.zip) as new artifact containing anything required to run the benchmarks. Unpack the file and use the scripts in the bin directory to execute the benchmarks. Use option -h to look at the options provided by JMH. You may exchange the libraries in the lib directory with other versions of XStream or the individual parsers or you may even add new JMH benchmarks to the default ones of XStream.

All benchmark values below measure the average throughput in nanosecond per operation. JMH provides additional measurement options, see online help. The maximum deviation for each benchmark is recorded in the reference files of the distributed ZIP file. The benchmark is executed on Linux 4.14.65 Gentoo 64-bit system with an Intel Core i7 CPU 920 of 2.67 GHz. Note again, that these values are no replacement for real profiler results and they may vary from run to run (see reference files) due to this machine's background processes. However, it can give you some idea of what you can expect using different parser technologies.

Parser Benchmark

The values represent the average throughput of 15 runs with a single thread. The benchmarks emphasis the parser efficiency for different structures.

Parser Text Array Nested
W3C DOM (Oracle JDK 1.8.0_181) 9710278.896 53954694.751 5148010.389
JDOM (1.1.3) 6387821.035 6898339.792 14111857.552
JDOM 2 (2.0.5) 5921949.583 8723291.385 10579620.188
DOM4J (1.6.1) 7452345.867 93099746.029 5533035.930
XOM (1.1) 8204769.944 42486494.920 8086714.065
StAX (BEA 1.2.0) 2879876.995 666062.149 572549.126
StAX (Woodstox 3.2.7) 1845769.311 632147.776 604788.852
StAX (Oracle JDK 1.8.0_131) 7444273.102 706572.092 617636.917
XPP (Xpp3 min 1.1.4c) 2155470.575 661082.180 12444913.194
XPP (kXML2 min 2.3.0) 3663477.841 854613.418 34372562.351
Binary (XStream 1.4.10) 1129812.942 383203.739 260233.635
Jettison (1.2) 3016232.225 555908.503 619961.028
Text
A single element with a text of 100.000 characters.
Array
A single element with 1.000 child elements.
Nested
Nested elements in 1000 levels (since version 1.4.10).

Converter Type Benchmark

The values represent the average throughput of 16 runs with four threads using the Xpp3 parser for a structure with 1.000 elements. The benchmarks demonstrate the different converter types that can be used for a standard Java class.

Converter Type Throughput
Custom 9176744.283
Java Bean 18353984.976
Reflection 23371721.858
Custom
A converter especially written for the Java type to convert.
Java Bean
Usage of the generic JavaBeanConverter, since the Java type respects the Java Bean contract.
Reflection
Usage of the generic converter based on reflection.

String Converter Benchmark

The values represent the average throughput of 16 runs with four threads using the Xpp3 parser for a structure with 10.000 string elements of various sizes and duplicates. The benchmarks demonstrate different implementations and configurations of the StringConverter.

StringConverter Strategy Throughput
No Cache 9422597.717
Intern 12576002.757
ConcurrentMap (length limit) 10411028.373
ConcurrentMap (unlimited) 10666492.267
Sync'd WeakCache (length limit) 10948390.386
Sync'd WeakCache (unlimited) 11917404.787
No Cache
An implementation that does not cache deserialized String values with the consequence that repeated values will always allocate separate memory.
Intern
An implementation that uses String.intern() to cache the individual values. The memory pool used for the values is dependent on the JDK version. Up to Java 7 this was the permanent generation space i.e. the memory has to be shared with all loaded classes. It is up to the garbage collection when these string values are freed again.
ConcurrentMap (length limit)
An implementation that uses a ConcurrentHashMap as cache for strings of limited length (38 characters). The lifetime of the cache is equivalent with the lifetime of the XStream instance.
ConcurrentMap (unlimited)
An implementation that uses a ConcurrentHashMap as cache for all strings. The lifetime of the cache is equivalent with the lifetime of the XStream instance.
Sync'd WeakCache (length limit)
An implementation that uses a WeakCache for strings of limited length (38 characters). This cache uses weak references for its keys and values. An entry is therefore only kept as long as the deserialized object structure is referencing it. This is XStream's default strategy.
Syn'd WeakCache (unlimited)
An implementation that uses a WeakCache for all strings. This cache uses weak references for its keys and values. An entry is therefore ony kept as long as the deserialized object structure is referencing it.

Name Coder Benchmark

The values represent the average throughput of 25 runs with four threads using the Xpp3 parser for a structure with 250 nested elements using names invalid for XML elements. The benchmarks demonstrate different implementation strategies for a NameCoder to create valid tag names in XML.

NameCoder Implementation Throughput
No Coding 4190972.243
Dollar Coding 4621025.135
Escaped Underscore Coding 5896886.514
Cached Escaped Underscore Coding 4350643.046
Xml Friendly Coding 4938586.549
No Coding
An implementation that does not encode the names of XML elements. It relies on the fact that the object graph does not contain elements with invalid XML names, because the name of the class types and members are either conforming or have been aliased.
Dollar Coding
An implementation that uses String.replace to replace any dollar sign with '·' (middle dot), a valid character normally not used for Java identifiers. In typical Java code there are no other invalid characters used for Java identifiers, however, Java allows identifiers to contain a wide range of UTF-8 characters and the JVM has even less restrictions.
Escaped Underscore Coding
An implementation that uses a StringBuilder to create the XML name by replacing any dollar sign with '_-' and escapes every plain underscore with two ones. The implementation will therefore only use characters in the standard ASCII range. It is not possible to use a simple minus sign as replacement because it is not a valid first character for XML names. However, the comments about Java identifiers and JVM identifiers from the Dollar Coding still apply.
Cached Escaped Underscore Coding
An implementation that implements a cache for the NameCoder that escapes the underscores.
Xml Friendly Coding
The default implementation of XStream using a StringBuilder and a cache, encoding any character that is invalid for XML names. It implements also the underscore escaping for compatibility reasons with XML created by earlier versions of XStream.