mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benjamin Hindman" <b...@berkeley.edu>
Subject Re: Review Request 36627: Fixed cgroups oom killer and memory pressure tests on Ubuntu 14.04.
Date Sat, 01 Aug 2015 22:30:41 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36627/#review93849
-----------------------------------------------------------

Ship it!



src/tests/containerizer/cgroups_tests.cpp (lines 1143 - 1146)
<https://reviews.apache.org/r/36627/#comment148258>

    But doesn't this mean we might stay in an infinite loop forever? I'm assuming that somehow
you figured out that things are just delayed but it eventually converges correctly, is that
the case? Can you leave a comment on why not incrementing 'i' won't just make this an infinite
loop?



src/tests/containerizer/memory_test_helper.cpp (line 93)
<https://reviews.apache.org/r/36627/#comment148259>

    Any specific reason to memset it to 0 instead of 1?


- Benjamin Hindman


On July 29, 2015, 11:14 p.m., Artem Harutyunyan wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36627/
> -----------------------------------------------------------
> 
> (Updated July 29, 2015, 11:14 p.m.)
> 
> 
> Review request for mesos, Benjamin Hindman and Joris Van Remoortere.
> 
> 
> Bugs: MESOS-2660
>     https://issues.apache.org/jira/browse/MESOS-2660
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> See summary.
> 
> 
> Diffs
> -----
> 
>   src/tests/containerizer/cgroups_tests.cpp caecd5dfa3fef33dba35cfc1b5934a11e2cc961a

>   src/tests/containerizer/memory_test_helper.cpp 48a35632786963f484f66642b5c67afd4f7a89cc

> 
> Diff: https://reviews.apache.org/r/36627/diff/
> 
> 
> Testing
> -------
> 
> It seems there is still one more cgroups memory test failing on more test failing on
my box. I'd like to fix that too and commit it together with this one. 
> 
> sudo make check
> 
> Verified that the process actually gets killed by oom-killer:
> 
> ```
> # tail -f /var/log/syslog
> 
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052405] lt-memory-test- invoked
oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052408] lt-memory-test- cpuset=/
mems_allowed=0
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052411] CPU: 7 PID: 76599
Comm: lt-memory-test- Tainted: G           OE 3.16.0-41-generic #57~14.04.1-Ubuntu
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052413] Hardware name: VMware,
Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/20/2014
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052413]  ffff88022efc1000
ffff8801fd2efc30 ffffffff81765721 ffff880231f10a30
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052415]  ffff8801fd2efcb8
ffffffff8175f2d5 ffff8802366f30c0 ffff8801e9405b00
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052416]  ffff8801fd2efc80
ffffffff81165067 ffff880231f10ee8 ffff880231f10a30
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052418] Call Trace:
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052424]  [<ffffffff81765721>]
dump_stack+0x45/0x56
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052426]  [<ffffffff8175f2d5>]
dump_header+0x7f/0x1f1
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052430]  [<ffffffff81165067>]
? find_lock_task_mm+0x47/0xa0
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052432]  [<ffffffff811654e5>]
oom_kill_process+0x205/0x360
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052434]  [<ffffffff812eb975>]
? security_capable_noaudit+0x15/0x20
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052437]  [<ffffffff811ca2e1>]
mem_cgroup_oom_synchronize+0x581/0x5e0
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052439]  [<ffffffff811c97c0>]
? mem_cgroup_try_charge_mm+0xa0/0xa0
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052440]  [<ffffffff81165ce4>]
pagefault_out_of_memory+0x14/0x80
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052442]  [<ffffffff8175d97f>]
mm_fault_error+0x67/0x140
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052445]  [<ffffffff8105b28c>]
__do_page_fault+0x4ec/0x560
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052449]  [<ffffffff810a6208>]
? __enqueue_entity+0x78/0x80
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052450]  [<ffffffff810a7f35>]
? set_next_entity+0x95/0xb0
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052452]  [<ffffffff81011627>]
? __switch_to+0x167/0x580
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052454]  [<ffffffff8105b331>]
do_page_fault+0x31/0x70
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052456]  [<ffffffff8176fe68>]
page_fault+0x28/0x30
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052457] Task in /mesos_test
killed as a result of limit of /mesos_test
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052459] memory: usage 65536kB,
limit 65536kB, failcnt 24
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052460] memory+swap: usage
0kB, limit 18014398509481983kB, failcnt 0
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052461] kmem: usage 0kB, limit
18014398509481983kB, failcnt 0
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052461] Memory cgroup stats
for /mesos_test: cache:0KB rss:65536KB rss_huge:63488KB mapped_file:0KB writeback:0KB inactive_anon:0KB
active_anon:0KB inactive_file:0KB active_file:0KB unevictable:65536KB
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052467] [ pid ]   uid  tgid
total_vm      rss nr_ptes swapents oom_score_adj name
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052574] [76599]     0 76599
   82452    31766     122        1             0 lt-memory-test-
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052575] Memory cgroup out
of memory: Kill process 76599 (lt-memory-test-) score 14 or sacrifice child
> Jul 22 14:56:00 harutyunyan-virtual-machine kernel: [17440.052577] Killed process 76599
(lt-memory-test-) total-vm:329808kB, anon-rss:67988kB, file-rss:59076kB
> 
> 
> ```
> 
> 
> Thanks,
> 
> Artem Harutyunyan
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message