NAF Benchmark Setup

Test machine

CPU: dual Xeon E5-2643v3 (3.4GHz, 6 cores) workstation. Hyperthreading is off. RAM: 128 GB DDR4-2133 ECC Registered. Storage: 4 x 2 TB SSD, in RAID 0. OS: Ubuntu 18.04.1 LTS.

Test setup

Only one compression or decompression task was running at each time. All compression tasks were complete first, then all decompression tasks were executed.

For compression, first the genome was loaded into disk cache (by running 'cat'). So all subsequent compression tasks used cached input file.

For decompression, for each compressed file, first an idle decompression run was performed and not timed. This idle run serves two purposes: 1. Load the compressed file(s) into disk cache. 2. The decompressed output is verified for identity with the original file (using md5sum).

For each compression or decompression task (combination of compressor and parametes), if the first run took under 30 seconds, the same run was repeated 10 times and smallest time was used. This allows to suppress measurement noise for fast runs, while still completing the slow runs in reasonable time.

Each compressor was used with the range of its available parameters. The best parameter was selected for each of the metrics used in comparison below.

Compressors used

Format	Compressor	Version	Options	Levels
fasta	cat	8.22	cat	none
sz	snzip	1.0.4	snzip -c	none
lz4	lz4	1.6	lz4 -c	-1 to -9
gz	gzip	1.6	gzip -c	-1 to -9
bz2	bzip2	1.0.6	bzip2 -cz	-1 to -9
br	brotli	1.0.7	brotli -c	-q 0 to -q 11
zstd	zstd	1.3.8	zstd -c	-1 to --ultra -22
xz	xz	5.2.2	xz -zkc	-0 to -e9
zpaq	zpaq1	7.15	zpaq a	-m1 to -m4
blast	blast1,2	2.6.0+	makeblastdb -dbtype nucl -title ''	none
2bit	faToTwoBit1,3	?	faToTwoBit -long	none
mfc	mfcompress1	1.01	MFCompressC	-1 to -3
dlim	deliminate1	1.3c	delim a	none
naf	ennaf	1.0.0	ennaf	-1 to -22

¹ Operates via wrapper due to inability to pipe input or output or other deficiencies.

² Wrapper stores mask separately.

³ Wrapper stores ambiguous IUPAC nucleotides separately.