Criterium 0.4.3 API documentation

Fraction of excution time allowed for final cleanup before a warning is issued.

Maximum number of attempts to run finalisers and gc.

Number of executions required

Target elapsed time for execution for a single measurement.

Time period used to let the code run so that jit compiler can do its work.

Provides a mutable place

Interrogation of differences in a state.

Convenience macro for benchmarking an expression, expr. Results are reported to out in human readable format. Options for report format are: :os, :runtime, and :verbose.

Benchmark an expression. This tries its best to eliminate sources of error. This also means that it runs for a while. It will typically take 70s for a fast test expression (less than 1s run time) or 10s plus 60 run times for longer running expressions.

Benchmark a function. This tries its best to eliminate sources of error. This also means that it runs for a while. It will typically take 70s for a fast test expression (less than 1s run time) or 10s plus 60 run times for longer running expressions.

Bootstrap a statistic. Statistic can produce multiple statistics as a vector so you can use juxt to pass multiple statistics. http://en.wikipedia.org/wiki/Bootstrapping_(statistics)

Bootstrap a statistic. Statistic can produce multiple statistics as a vector so you can use juxt to pass multiple statistics. http://en.wikipedia.org/wiki/Bootstrapping_(statistics)

Sets the estimated overhead.

Estimate the number of executions required in order to have at least the specified execution period, check for the jvm to have constant class loader and compilation state.

Calculate a conservative estimate of the timing loop overhead.

Time the execution of n invocations of f. See execute-expr-core-timed-part.

Performs the part of execute-expr where we actually measure the elapsed run time. Evaluates (f) n times, each time saving the return value as an Object in mutable-place.

The idea is that except for the call to (f), the only things done during each iteration are a few arithmetic operations and comparisons to 0 on primitive longs, and the storage of the return value.

The JVM is not free to optimize away the calls to f because the return values are saved in mutable-place.

This array is at most +max-obj-array-size+ elements long, to save memory. An artificially intelligent JVM might be able to determine that if n is larger than +max-obj-array-size+, some of the return values are overwritten and thus those calls need not be made. I doubt we will see that kind of optimization any time soon, and perhaps some JVM rules even prohibit doing so since the writes to ret-vals-arr could potentially be read by a different thread.

Extract reporting options from the given options vector. Returns a two element vector containing the reporting options followed by the non-reporting options

Time a final clean up of JVM memory. If this time is significant compared to the runtime, then the runtime should maybe include this time.

Force garbage collection and finalisers so that execution time associated with this is not incurred later. Up to max-attempts are made.

Get mutable field value.

Report a (inconsistent) snapshot of the heap memory used.

Returns the total compilation time for the JVM instance.

Returns the name of the JIT compiler.

Report a (inconsistent) snapshot of the memory situation.

Return the operating system details as a hash.

Return a keyword describing the effect of outliers on the estimate of mean runtime.

Find the significance of outliers given boostrapped mean and variance estimates. See http://www.ellipticgroup.com/misc/article_supplement.pdf, p17.

Find the outliers in the data using a boxplot technique.

Convenience macro for benchmarking an expression, expr. Results are reported to out in human readable format. Options for report format are: :os, :runtime, and :verbose.

Benchmark an expression. Less rigorous benchmark (higher uncertainty).

Benchmark an expression. Less rigorous benchmark (higher uncertainty).

Print format output

Benchmark an expression. This tries its best to eliminate sources of error. This also means that it runs for a while. It will typically take 70s for a quick test expression (less than 1s run time) or 10s plus 60 run times for longer running expressions.

Benchmark multiple expressions in a 'round robin' fashion. Very similar to run-benchmark, except it takes multiple expressions in a sequence instead of only one (each element of the sequence should be a map with keys :f and :expr-string). It runs the following steps in sequence:

1. Execute each expr once

2. Run expression 1 for at least warmup-jit-period nanoseconds so the JIT has an opportunity to optimize it. Then do the same for each of the other expressions.

3. Run expression 1 many times to estimate how many times it must be executed to take a total of target-execution-time nanoseconds. The result is a number of iterations n-exec1 for expression 1. Do the same for each of the other expressions, each with the same target-execution-time, each resulting in its own independent number of executions.

4. Run expression 1 n-exec1 times, measuring the total elapsed time. Do the same for the rest of the expressions.

5. Repeat step 4 a total of sample-count times.

Return the runtime details as a hash.

Determine a scale factor and unit for displaying a time.

Set mutable field to value.

Check to see if a state delta represents no change

Return a state object for the difference between two states

Return the operating system details.

Returns a vector containing execution time and result of specified function.

Returns a vector containing execution time, change in loaded and unloaded class counts, change in compilation time and result of specified function.

Obtain a timestamp

Obtain a timestamp, possibly using MXBean.

Run expression for the given amount of time to enable JIT compilation.

Warn if the JIT options are suspicious looking.

Macro to enable progress reporting during the benchmark.