java - OutOfMemory with JMH and Mode.AverageTime -

i writing micro-benchmark compare string concatenation using + operator vs stringbuilder. aim, created jmh benchmark class based on openjdk example uses batchsize parameter:

@state(scope.thread) @benchmarkmode(mode.averagetime) @measurement(batchsize = 10000, iterations = 10) @warmup(batchsize = 10000, iterations = 10) @fork(1) public class stringconcatenationbenchmark {      private string string;      private stringbuilder stringbuilder;      @setup(level.iteration)     public void setup() {         string = "";         stringbuilder = new stringbuilder();     }      @benchmark     public void stringconcatenation() {         string += "some more data";     }      @benchmark     public void stringbuilderconcatenation() {         stringbuilder.append("some more data");     }  } 

when run benchmark following error stringbuilderconcatenation method:

java.lang.outofmemoryerror: java heap space     @ java.util.arrays.copyof(arrays.java:3332)     @ java.lang.abstractstringbuilder.expandcapacity(abstractstringbuilder.java:137)     @ java.lang.abstractstringbuilder.ensurecapacityinternal(abstractstringbuilder.java:121)     @ java.lang.abstractstringbuilder.append(abstractstringbuilder.java:421)     @ java.lang.stringbuilder.append(stringbuilder.java:136)     @ link.pellegrino.string_concatenation.stringconcatenationbenchmark.stringbuilderconcatenation(stringconcatenationbenchmark.java:29)     @ link.pellegrino.string_concatenation.generated.stringconcatenationbenchmark_stringbuilderconcatenation.stringbuilderconcatenation_avgt_jmhstub(stringconcatenationbenchmark_stringbuilderconcatenation.java:165)     @ link.pellegrino.string_concatenation.generated.stringconcatenationbenchmark_stringbuilderconcatenation.stringbuilderconcatenation_averagetime(stringconcatenationbenchmark_stringbuilderconcatenation.java:130)     @ sun.reflect.nativemethodaccessorimpl.invoke0(native method)     @ sun.reflect.nativemethodaccessorimpl.invoke(nativemethodaccessorimpl.java:62)     @ sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl.java:43)     @ java.lang.reflect.method.invoke(method.java:497)     @ org.openjdk.jmh.runner.benchmarkhandler$benchmarktask.call(benchmarkhandler.java:430)     @ org.openjdk.jmh.runner.benchmarkhandler$benchmarktask.call(benchmarkhandler.java:412)     @ java.util.concurrent.futuretask.run(futuretask.java:266)     @ java.util.concurrent.threadpoolexecutor.runworker(threadpoolexecutor.java:1142)     @ java.util.concurrent.threadpoolexecutor$worker.run(threadpoolexecutor.java:617)     @ java.lang.thread.run(thread.java:745) 

i thinking default jvm heap size has increased, tried allow 10gb using -xmx10g value -jvmargs option provided jmh. unfortunately, still error.

consequently, tried reduce value batchsize parameter 1 still outofmemoryerror.

the workaround have found set benchmark mode mode.singleshottime. since mode seems consider batch single shot (even if s/op displayed in units column), seems metric want: average time perform set of batch operations. however, still don't understand why not working mode.averagetime.

please note benchmarks method stringconcatenation work expected whatever benchmark mode used. issue occurs stringbuilderconcatenation method makes use of stringbuilder.

any understand why previous example not working benchmark mode set mode.averagetime welcome.

jmh version used 1.10.4.

you're right mode.singleshottime need: measures time single batch. when using mode.averagetime iteration still works until iteration time finishes (which 1 second default). measures time per executing single batch (only batches finished during execution time counted), final results differ, execution time same.

another problem @setup(level.iteration) forces setup executed before every iteration, not before every batch. strings not limited batch size. string version not cause outofmemoryerror because it's slower stringbuilder, during 1 second it's capable build shorter string.

not beautiful way fix benchmark (while still using average time mode , batchsize parameter) reset string/stringbuilder manually:

@state(scope.thread) @benchmarkmode(mode.averagetime) @outputtimeunit(timeunit.microseconds) @measurement(batchsize = 10000, iterations = 10) @warmup(batchsize = 10000, iterations = 10) @fork(1) public class stringconcatenationbenchmark {     private static final string s = "some more data";     private static final int maxlen = s.length()*10000;      private string string;      private stringbuilder stringbuilder;      @setup(level.iteration)     public void setup() {         string = "";         stringbuilder = new stringbuilder();     }      @benchmark     public void stringconcatenation() {         if(string.length() >= maxlen) string = "";         string += s;     }      @benchmark     public void stringbuilderconcatenation() {         if(stringbuilder.length() >= maxlen) stringbuilder = new stringbuilder();         stringbuilder.append(s);     } } 

here's results on box (i5 3340, 4gb ram, 64bit win7, jdk 1.8.0_45):

benchmark                   mode  cnt       score       error  units stringbuilderconcatenation  avgt   10     145.997 ±     2.301  us/op stringconcatenation         avgt   10  324878.341 ± 39824.738  us/op 

so can see 3 batches fit second stringconcatenation (1e6/324878) while stringbuilderconcatenation thousands of batches can executed resulting in enormous string leading outofmemoryerror.

i don't know why adding more memory doesn't work you, me -xmx4g enough run stringbuilder test of original benchmark. box faster, resulting string longer. note big string can hit array size limit (2 billion of elements) if have enough memory. check exception stacktrace after adding memory: same? if hit array size limit, still outofmemoryerror, stacktrace different little bit. anyways enough memory results benchmark incorrect (both string , stringbuilder).