綾小路龍之介の素人思考

[linux] xz、bzip2、gzip の圧縮率、圧縮展開速度の比較

xz は圧縮時間より圧縮率、圧縮展開時間比が重要な場合 (アーカイブを多数に配布する場合等) に適し、bzip2 は圧縮展開時間比より圧縮率、圧縮時間が重要な場合 (圧縮展開を繰り返す場合) に適し、gzip は圧縮率より、圧縮展開時間比が重要な場合 (xz と同様) に適すということが言える。

テストに使ったファイルは 1 つ 4.5MB 程度のテキストファイルを 22 個アーカイブした 100MB の tar ファイル。詳細は以下。

$ tar tvf log.tar
-rw-r--r-- hoge/hoge  4478655 2009-05-04 23:59 2009-05-04.log
-rw-r--r-- hoge/hoge  4544114 2009-05-05 23:59 2009-05-05.log
-rw-r--r-- hoge/hoge  4631386 2009-05-06 23:59 2009-05-06.log
-rw-r--r-- hoge/hoge  4596132 2009-05-07 23:59 2009-05-07.log
-rw-r--r-- hoge/hoge  4621793 2009-05-08 23:59 2009-05-08.log
-rw-r--r-- hoge/hoge  4415000 2009-05-09 23:59 2009-05-09.log
-rw-r--r-- hoge/hoge  4569259 2009-05-10 23:59 2009-05-10.log
-rw-r--r-- hoge/hoge  4466073 2009-05-11 23:59 2009-05-11.log
-rw-r--r-- hoge/hoge  4783604 2009-05-12 23:59 2009-05-12.log
-rw-r--r-- hoge/hoge  4636846 2009-05-13 23:59 2009-05-13.log
-rw-r--r-- hoge/hoge  4605847 2009-05-14 23:59 2009-05-14.log
-rw-r--r-- hoge/hoge  4533941 2009-05-15 23:59 2009-05-15.log
-rw-r--r-- hoge/hoge  4393997 2009-05-16 23:59 2009-05-16.log
-rw-r--r-- hoge/hoge  4879404 2009-05-17 23:59 2009-05-17.log
-rw-r--r-- hoge/hoge  4715859 2009-05-18 23:59 2009-05-18.log
-rw-r--r-- hoge/hoge  4416494 2009-05-19 23:59 2009-05-19.log
-rw-r--r-- hoge/hoge  4594489 2009-05-20 23:59 2009-05-20.log
-rw-r--r-- hoge/hoge  4469918 2009-05-21 23:59 2009-05-21.log
-rw-r--r-- hoge/hoge  4707008 2009-05-22 23:59 2009-05-22.log
-rw-r--r-- hoge/hoge  4572997 2009-05-23 23:59 2009-05-23.log
-rw-r--r-- hoge/hoge  5089547 2009-05-24 23:59 2009-05-24.log
-rw-r--r-- hoge/hoge  4631351 2009-05-25 23:59 2009-05-25.log
$ ls -la log.tar*
-rw-r--r-- 1 hoge hoge 101376000 May 28 17:52 log.tar

xz による圧縮と展開。これを含めて以降は圧縮率は最高のものを使用。

$ time xz -9 log.tar

real    497m31.607s
user    5m53.382s
sys     81m34.590s
$ ls -la log.tar*
-rw-r--r-- 1 hoge hoge  23211216 May 28 17:54 log.tar.xz
$ time xz -d log.tar.xz

real    0m8.739s
user    0m3.876s
sys     0m0.432s

bzip2 による圧縮と展開。

$ time bzip2 -9 log.tar

real    0m42.859s
user    0m21.101s
sys     0m0.280s
$ ls -la log.tar*
-rw-r--r-- 1 hoge hoge  27761964 May 28 17:54 log.tar.bz2
$ time bzip2 -d log.tar.bz2

real    0m17.876s
user    0m8.317s
sys     0m0.580s

gzip による圧縮と展開。

$ time gzip -9 log.tar

real    0m28.624s
user    0m13.961s
sys     0m0.336s
$ ls -la log.tar*
-rw-r--r-- 1 hoge hoge  41040119 May 28 17:54 log.tar.gz
$ time gzip -d log.tar.gz
-rw-r--r-- 1 hoge hoge 101376000 May 28 17:52 log.tar

xz による圧縮と展開。これを含めて以降は圧縮率が最高となるオプションを使用。

$ time xz -9 log.tar

real    497m31.607s
user    5m53.382s
sys     81m34.590s
$ ls -la log.tar*
-rw-r--r-- 1 hoge hoge  23211216 May 28 17:54 log.tar.xz
$ time xz -d log.tar.xz

real    0m8.739s
user    0m3.876s
sys     0m0.432s

bzip2 による圧縮と展開。

$ time bzip2 -9 log.tar

real    0m42.859s
user    0m21.101s
sys     0m0.280s
$ ls -la log.tar*
-rw-r--r-- 1 hoge hoge  27761964 May 28 17:54 log.tar.bz2
$ time bzip2 -d log.tar.bz2

real    0m17.876s
user    0m8.317s
sys     0m0.580s

gzip による圧縮と展開。

$ time gzip -9 log.tar

real    0m28.624s
user    0m13.961s
sys     0m0.336s
$ ls -la log.tar*
-rw-r--r-- 1 hoge hoge  41040119 May 28 17:54 log.tar.gz
$ time gzip -d log.tar.gz

real    0m3.544s
user    0m1.324s
sys     0m0.196s

まとめると以下のようになる。time コマンドの出力では user 部分がコマンドにより CPU が使用された時間。圧縮速度については xz は bzip2 と gzip に比べて 1 桁遅い。展開速度については xz は gzip と同程度、bzip2 はこれらより 1 桁遅い。xz は圧縮に CPU 時間を消費するが、圧縮ファイルを展開する回数が多いほど効率が良くなる。bzip2 は xz に圧縮時間で 1 桁短く、圧縮率で迫り、圧縮展開時間比で劣る。gzip は xz と同様の特徴を持つが、xz に比べて圧縮速度が 1 桁早く、圧縮率が悪い。

圧縮アルゴリズムの比較
コマンド サイズ (bytes) 圧縮率 (%) 圧縮時間 (sec) 圧縮展開時間比 展開時間 (sec) 圧縮速度 (bytes/sec) 展開速度 (bytes/sec)
xz -9 23211216 22.89 353.382 3.876 91.17 -221190.62 20166352.94
bzip2 -9 27761964 27.38 21.101 8.317 2.53 -3488651.53 8851032.34
gzip -9 41040119 40.48 13.961 1.324 10.54 -4321744.93 45570907.09

ここで圧縮速度 = (圧縮後ファイルサイズ - 圧縮前ファイルサイズ) / 圧縮時間。展開速度 = (展開後ファイルサイズ - 展開前ファイルサイズ) / 展開時間。

今回、xz, bzip2, gzip 圧縮で効率の比較を行った。その結果、xz は圧縮時間より圧縮率、圧縮展開時間比が重要な場合 (アーカイブを多数に配布する場合等) に適し、bzip2 は圧縮展開時間比より圧縮率、圧縮時間が重要な場合 (圧縮展開を繰り返す場合) に適し、gzip は圧縮率より、圧縮展開時間比が重要な場合 (xz と同様) に適すということが言える。

ダミーファイルの場合

圧縮元ファイルを圧縮に適さないランダムな内容のファイルにすることで、処理速度自体を比較できないかなぁ。

/dev/urandom をソースにして 1GB のダミーファイルを作成

$ dd if=/dev/urandom of=./dummy_file.dat count=1000 bs=1000000
1000+0 records in
1000+0 records out
1000000000 bytes (1.0 GB) copied, 129.265 s, 7.7 MB/s

gzip で圧縮展開。時間は time で計測。圧縮展開の結果は /dev/null に捨てる

$ time --verbose gzip --verbose --stdout --best ./dummy_file.dat > /dev/null
./dummy_file.dat:        -0.0%
        Command being timed: "gzip --verbose --stdout --best ./dummy_file.dat"
        User time (seconds): 79.57
        System time (seconds): 1.24
        Percent of CPU this job got: 97%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 1:22.47
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 1860
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 1
        Minor (reclaiming a frame) page faults: 137
        Voluntary context switches: 319
        Involuntary context switches: 518
        Swaps: 0
        File system inputs: 1945608
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
$ time --verbose gzip --verbose --stdout --best ./dummy_file.dat > ./dummy_file.dat.gz
./dummy_file.dat:        -0.0%
        Command being timed: "gzip --verbose --stdout --best ./dummy_file.dat"
        User time (seconds): 80.76
        System time (seconds): 14.42
        Percent of CPU this job got: 92%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 1:42.79
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 1860
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 1
        Minor (reclaiming a frame) page faults: 137
        Voluntary context switches: 602
        Involuntary context switches: 372
        Swaps: 0
        File system inputs: 1945160
        File system outputs: 1953448
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
$ time --verbose gzip --verbose --stdout --decompress ./dummy_file.dat.gz > /dev/null
./dummy_file.dat.gz:     -0.0%
        Command being timed: "gzip --verbose --stdout --decompress ./dummy_file.dat.gz"
        User time (seconds): 12.38
        System time (seconds): 1.20
        Percent of CPU this job got: 62%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:21.71
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 1748
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 1
        Minor (reclaiming a frame) page faults: 97
        Voluntary context switches: 1739
        Involuntary context switches: 47
        Swaps: 0
        File system inputs: 1953456
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
$ ls --time-style=+%H:%M --all --format=long ./dummy_file.dat ./dummy_file.dat.gz
-rw-r--r-- 1 u00 u00 1000000000 14:08 ./dummy_file.dat
-rw-r--r-- 1 u00 u00 1000161997 14:11 ./dummy_file.dat.gz

bzip2 で圧縮展開。時間は time で計測。圧縮展開の結果は /dev/null に捨てる

$ time --verbose bzip2 --verbose --stdout --best ./dummy_file.dat > /dev/null
  ./dummy_file.dat:  0.996:1,  8.035 bits/byte, -0.44% saved, 1000000000 in, 1004435202 out.
        Command being timed: "bzip2 --verbose --stdout --best ./dummy_file.dat"
        User time (seconds): 474.61
        System time (seconds): 1.30
        Percent of CPU this job got: 98%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 8:04.99
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 7596
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 1695
        Voluntary context switches: 1368
        Involuntary context switches: 2434
        Swaps: 0
        File system inputs: 1945096
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
$ time --verbose bzip2 --verbose --stdout --best ./dummy_file.dat > ./dummy_file.dat.bz
  ./dummy_file.dat:  0.996:1,  8.035 bits/byte, -0.44% saved, 1000000000 in, 1004435202 out.
        Command being timed: "bzip2 --verbose --stdout --best ./dummy_file.dat"
        User time (seconds): 472.55
        System time (seconds): 16.34
        Percent of CPU this job got: 97%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 8:23.94
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 7624
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 1698
        Voluntary context switches: 2298
        Involuntary context switches: 1135
        Swaps: 0
        File system inputs: 1945152
        File system outputs: 1961792
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
$ time --verbose bzip2 --verbose --stdout --decompress ./dummy_file.dat.bz > /dev/null
  ./dummy_file.dat.bz: done
        Command being timed: "bzip2 --verbose --stdout --decompress ./dummy_file.dat.bz"
        User time (seconds): 173.11
        System time (seconds): 1.88
        Percent of CPU this job got: 99%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 2:56.65
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 4684
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 970
        Voluntary context switches: 174
        Involuntary context switches: 669
        Swaps: 0
        File system inputs: 1961792
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
$ ls --time-style=+%H:%M --all --format=long ./dummy_file.dat ./dummy_file.dat.bz
-rw-r--r-- 1 u00 u00 1000000000 14:08 ./dummy_file.dat
-rw-r--r-- 1 u00 u00 1004435202 14:28 ./dummy_file.dat.bz

xz で圧縮展開。時間は time で計測。圧縮展開の結果は /dev/null に捨てる

$ time --verbose xz --verbose --stdout --best ./dummy_file.dat > /dev/null
./dummy_file.dat (1/1)
  100 %       953.7 MiB / 953.7 MiB = 1.000   1.0 MiB/s      15:32
        Command being timed: "xz --verbose --stdout --best ./dummy_file.dat"
        User time (seconds): 930.47
        System time (seconds): 2.10
        Percent of CPU this job got: 99%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 15:32.80
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 691096
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 172372
        Voluntary context switches: 23
        Involuntary context switches: 3258
        Swaps: 0
        File system inputs: 1945136
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
$ time --verbose xz --verbose --stdout --best ./dummy_file.dat > ./dummy_file.dat.xz
./dummy_file.dat (1/1)
  100 %       953.7 MiB / 953.7 MiB = 1.000   925 KiB/s      17:35
        Command being timed: "xz --verbose --stdout --best ./dummy_file.dat"
        User time (seconds): 1042.30
        System time (seconds): 12.19
        Percent of CPU this job got: 99%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 17:35.57
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 690900
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 15
        Minor (reclaiming a frame) page faults: 172356
        Voluntary context switches: 209
        Involuntary context switches: 3756
        Swaps: 0
        File system inputs: 1946456
        File system outputs: 1953224
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
$ time --verbose xz --verbose --stdout --decompress ./dummy_file.dat.xz > /dev/null
./dummy_file.dat.xz (1/1)
  100 %       953.7 MiB / 953.7 MiB = 1.000    48 MiB/s       0:19
        Command being timed: "xz --verbose --stdout --decompress ./dummy_file.dat.xz"
        User time (seconds): 3.61
        System time (seconds): 1.42
        Percent of CPU this job got: 25%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:19.82
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 67684
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 15
        Minor (reclaiming a frame) page faults: 16486
        Voluntary context switches: 2684
        Involuntary context switches: 50
        Swaps: 0
        File system inputs: 1954184
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
$ ls --time-style=+%H:%M --all --format=long ./dummy_file.dat ./dummy_file.dat.xz
-rw-r--r-- 1 u00 u00 1000000000 14:08 ./dummy_file.dat
-rw-r--r-- 1 u00 u00 1000049572 15:04 ./dummy_file.dat.xz

元ファイルの内容が十分にランダムなので、圧縮されることはないわけだけど、生成されたファイルは以下。

$ ls --time-style=+%H:%M --all --format=long ./dummy_file.dat ./dummy_file.dat.bz ./dummy_file.dat.gz ./dummy_file.dat.xz
-rw-r--r-- 1 u00 u00 1000000000 14:08 ./dummy_file.dat
-rw-r--r-- 1 u00 u00 1004435202 14:28 ./dummy_file.dat.bz
-rw-r--r-- 1 u00 u00 1000161997 14:11 ./dummy_file.dat.gz
-rw-r--r-- 1 u00 u00 1000049572 15:04 ./dummy_file.dat.xz
コマンド サイズ (bytes) 圧縮 展開 圧縮展開時間比
率 (%) 時間 (sec) 速度 (bytes/sec) 時間 (sec) 速度 (bytes/sec)
xz --best 1000049572 100.00 930.47 53.27 3.61 13731.85 257.74
bzip2 --best 1004435202 100.44 474.61 9344.94 173.11 25620.71 2.74
gzip --best 1000161997 100.01 79.57 2035.90 12.38 13085.37 6.42

cvs ファイルの場合

適当に同じような内容が連続するファイルとして CSV ファイルをテストに使ってみた。1GB のダミーファイルを作成

$ dd if=/usr/share/mecab/dic/unidic/lex.csv of=./dummy_file.dat.tmp count=100 bs=1000000
100+0 records in
100+0 records out
100000000 bytes (100 MB) copied, 1.99075 s, 50.2 MB/s
$ cat ./dummy_file.dat.tmp > ./dummy_file.dat
$ cat ./dummy_file.dat.tmp >> ./dummy_file.dat
$ cat ./dummy_file.dat.tmp >> ./dummy_file.dat
$ cat ./dummy_file.dat.tmp >> ./dummy_file.dat
$ cat ./dummy_file.dat.tmp >> ./dummy_file.dat
$ cat ./dummy_file.dat.tmp >> ./dummy_file.dat
$ cat ./dummy_file.dat.tmp >> ./dummy_file.dat
$ cat ./dummy_file.dat.tmp >> ./dummy_file.dat
$ cat ./dummy_file.dat.tmp >> ./dummy_file.dat
$ cat ./dummy_file.dat.tmp >> ./dummy_file.dat

gzip で圧縮展開。時間は time で計測。圧縮展開の結果は /dev/null に捨てる

$ time --verbose gzip --verbose --stdout --best ./dummy_file.dat > /dev/null
./dummy_file.dat:        91.0%
        Command being timed: "gzip --verbose --stdout --best ./dummy_file.dat"
        User time (seconds): 143.88
        System time (seconds): 1.36
        Percent of CPU this job got: 99%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 2:26.58
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 1844
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 1
        Minor (reclaiming a frame) page faults: 148
        Voluntary context switches: 149
        Involuntary context switches: 690
        Swaps: 0
        File system inputs: 1898208
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
$ time --verbose gzip --verbose --stdout --best ./dummy_file.dat > ./dummy_file.dat.gz
./dummy_file.dat:        91.0%
        Command being timed: "gzip --verbose --stdout --best ./dummy_file.dat"
        User time (seconds): 134.52
        System time (seconds): 1.69
        Percent of CPU this job got: 97%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 2:20.34
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 1848
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 1
        Minor (reclaiming a frame) page faults: 148
        Voluntary context switches: 181
        Involuntary context switches: 520
        Swaps: 0
        File system inputs: 1898208
        File system outputs: 175840
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
$ time --verbose gzip --verbose --stdout --decompress ./dummy_file.dat.gz > /dev/null
./dummy_file.dat.gz:     91.0%
        Command being timed: "gzip --verbose --stdout --decompress ./dummy_file.dat.gz"
        User time (seconds): 9.49
        System time (seconds): 0.10
        Percent of CPU this job got: 99%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:09.69
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 1756
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 1
        Minor (reclaiming a frame) page faults: 99
        Voluntary context switches: 14
        Involuntary context switches: 74
        Swaps: 0
        File system inputs: 79872
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
+ ls --time-style=+%H:%M --all --format=long ./dummy_file.dat ./dummy_file.dat.gz
-rw-r--r-- 1 u00 u00 1000000000 16:45 ./dummy_file.dat
-rw-r--r-- 1 u00 u00   90027408 16:50 ./dummy_file.dat.gz

bzip2 で圧縮展開。時間は time で計測。圧縮展開の結果は /dev/null に捨てる

$ time --verbose bzip2 --verbose --stdout --best ./dummy_file.dat > /dev/null
  ./dummy_file.dat: 13.852:1,  0.578 bits/byte, 92.78% saved, 1000000000 in, 72192757 out.
        Command being timed: "bzip2 --verbose --stdout --best ./dummy_file.dat"
        User time (seconds): 250.11
        System time (seconds): 1.43
        Percent of CPU this job got: 95%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 4:23.11
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 7668
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 2
        Minor (reclaiming a frame) page faults: 1692
        Voluntary context switches: 1890
        Involuntary context switches: 1807
        Swaps: 0
        File system inputs: 1898384
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
$ time --verbose bzip2 --verbose --stdout --best ./dummy_file.dat > ./dummy_file.dat.bz
  ./dummy_file.dat: 13.852:1,  0.578 bits/byte, 92.78% saved, 1000000000 in, 72192757 out.
        Command being timed: "bzip2 --verbose --stdout --best ./dummy_file.dat"
        User time (seconds): 259.72
        System time (seconds): 1.88
        Percent of CPU this job got: 96%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 4:31.72
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 7624
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 2
        Minor (reclaiming a frame) page faults: 1692
        Voluntary context switches: 1787
        Involuntary context switches: 1385
        Swaps: 0
        File system inputs: 1898232
        File system outputs: 141008
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
$ time --verbose bzip2 --verbose --stdout --decompress ./dummy_file.dat.bz > /dev.null
  ./dummy_file.dat.bz: done
        Command being timed: "bzip2 --verbose --stdout --decompress ./dummy_file.dat.bz"
        User time (seconds): 37.68
        System time (seconds): 0.26
        Percent of CPU this job got: 99%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:37.98
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 4668
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 2
        Minor (reclaiming a frame) page faults: 966
        Voluntary context switches: 4
        Involuntary context switches: 158
        Swaps: 0
        File system inputs: 71968
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
$ ls --time-style=+%H:%M --all --format=long ./dummy_file.dat ./dummy_file.dat.bz
-rw-r--r-- 1 u00 u00 1000000000 16:45 ./dummy_file.dat
-rw-r--r-- 1 u00 u00   72192757 16:59 ./dummy_file.dat.bz

xz で圧縮展開。時間は time で計測。圧縮展開の結果は /dev/null に捨てる

$ time --verbose xz --verbose --stdout --best ./dummy_file.dat > /dev/null
./dummy_file.dat (1/1)
  100 %        59.6 MiB / 953.7 MiB = 0.062   870 KiB/s      18:42
        Command being timed: "xz --verbose --stdout --best ./dummy_file.dat"
        User time (seconds): 1117.20
        System time (seconds): 2.90
        Percent of CPU this job got: 99%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 18:42.47
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 691008
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 172413
        Voluntary context switches: 227
        Involuntary context switches: 7043
        Swaps: 0
        File system inputs: 1954928
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
$ time --verbose xz --verbose --stdout --best ./dummy_file.dat > ./dummy_file.dat.xz
./dummy_file.dat (1/1)
  100 %        59.6 MiB / 953.7 MiB = 0.062   981 KiB/s      16:35
        Command being timed: "xz --verbose --stdout --best ./dummy_file.dat"
        User time (seconds): 992.17
        System time (seconds): 3.07
        Percent of CPU this job got: 99%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 16:35.98
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 691200
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 3
        Minor (reclaiming a frame) page faults: 172411
        Voluntary context switches: 81
        Involuntary context switches: 3605
        Swaps: 0
        File system inputs: 1953200
        File system outputs: 121992
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
$ time --verbose xz --verbose --stdout --decompress ./dummy_file.dat.xz > /dev/null
./dummy_file.dat.xz (1/1)
  100 %        59.6 MiB / 953.7 MiB = 0.062    80 MiB/s       0:11
        Command being timed: "xz --verbose --stdout --decompress ./dummy_file.dat.xz"
        User time (seconds): 11.64
        System time (seconds): 0.23
        Percent of CPU this job got: 99%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:11.92
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 67632
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 3
        Minor (reclaiming a frame) page faults: 16502
        Voluntary context switches: 6
        Involuntary context switches: 38
        Swaps: 0
        File system inputs: 113992
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
$ ls --time-style=+%H:%M --all --format=long ./dummy_file.dat ./dummy_file.dat.xz
-rw-r--r-- 1 u00 u00 1000000000 16:45 ./dummy_file.dat
-rw-r--r-- 1 u00 u00   62456252 17:35 ./dummy_file.dat.xz
$ ls --time-style=+%H:%M --all --format=long ./dummy_file.dat ./dummy_file.dat.bz ./dummy_file.dat.gz ./dummy_file.dat.tmp ./dummy_file.dat.xz
-rw-r--r-- 1 u00 u00 1000000000 16:45 ./dummy_file.dat
-rw-r--r-- 1 u00 u00   72192757 16:59 ./dummy_file.dat.bz
-rw-r--r-- 1 u00 u00   90027408 16:50 ./dummy_file.dat.gz
-rw-r--r-- 1 u00 u00  100000000 16:45 ./dummy_file.dat.tmp
-rw-r--r-- 1 u00 u00   62456252 17:35 ./dummy_file.dat.xz
コマンド サイズ 圧縮 展開 圧縮展開時間比
時間 速度 時間 速度
(bytes) (%) /gzip (-) (sec) /gzip (-) (bytes/sec) /gzip (-) (sec) /gzip (-) (bytes/sec) /gzip (-) (-) /gzip (-)
xz --best 62456252 6.24 0.69 1117.20 7.76 -839190.60 0.13 11.64 1.22 80544995.53 0.83 95.97 6.33
bzip2 --best 72192757 7.21 0.80 250.11 1.73 -3709596.74 0.58 37.68 3.97 24623334.47 0.25 6.63 0.43
gzip --best 90027408 9.00 1.00 143.88 1.00 -6324524.54 1.00 9.49 1.00 95887522.86 1.00 15.16 1.00

リファレンス

  1. user sys real time - Google 検索
  2. 【 time 】 指定したコマンドの実行時間を表示する - Linuxコマンド集:ITpro
  3. unix - What do 'real', 'user' and 'sys' mean in the output of time(1)? - Stack Overflow
  4. どうすれば本当の'、'ユーザー'と'sysは'時間の出力(1)意味ですか?
  5. UNIXの部屋 コマンド検索:time (*BSD/Linux)
  6. 404 Blog Not Found:tips - xzってどうよ!?
  7. 千年の風に吹かれて : gzip 、 bzip2 、 xz
  8. A Quick Benchmark: Gzip vs. Bzip2 vs. LZMA
  9. xz bzip2 gzip - Google 検索

ソーシャルブックマーク

  1. はてなブックマーク
  2. Google Bookmarks
  3. del.icio.us

ChangeLog

  1. Posted: 2010-01-01T13:18:18+09:00
  2. Modified: 2010-01-01T13:18:18+09:00
  3. Generated: 2017-02-19T23:09:36+09:00