mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ilya Pronin <ipro...@twopensource.com>
Subject Review Request 68089: Added LevelDB compaction after replicated log truncation.
Date Fri, 27 Jul 2018 20:12:04 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68089/
-----------------------------------------------------------

Review request for mesos, Benjamin Mahler and Jie Yu.


Bugs: MESOS-184
    https://issues.apache.org/jira/browse/MESOS-184


Repository: mesos


Description
-------

LevelDB compaction algorithm doesn't seem to work well with out key
usage pattern and background compaction doesn't get triggered:
https://github.com/google/leveldb/issues/603. Because of that the
replicated log storage grows over time until it consumes all disk space
on the partition. As a workaround this change adds manual
leveldb::DB::CompactRange() invocation. Compaction is triggered after
every truncation action persistence that involved removal of stored
keys. By default this workaround is disabled and has to be turned on
with autoCompact parameter of the Log constructor.


Diffs
-----

  include/mesos/log/log.hpp ba355331d682a7f2bdc534b4eb4402e2347528d8 
  src/log/leveldb.hpp 1d5842a13cfe99fa3ea2ac73eabb2a0b55e15239 
  src/log/leveldb.cpp 72eeffb17e2de39594de284bc01063445f82ab2c 
  src/log/log.hpp 5612a1a2c4f0d3bba3d946b365f34616180eb426 
  src/log/log.cpp 6a7080ab5115c777bd5039acb0f830563866781f 
  src/log/replica.hpp 082c81cd92e1d46f8014cd23953878f9e23a19d7 
  src/log/replica.cpp 25c128029c9d54f92145e2294f8f59d11b27da2a 
  src/tests/log_tests.cpp a8980e3676f51d14087f56338ff45de2927ea992 


Diff: https://reviews.apache.org/r/68089/diff/1/


Testing
-------

Updated `LogStorageTest` cases to run with and without compaction enabled. Ran `make check`.


Thanks,

Ilya Pronin


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message