mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mesos Reviewbot Windows <revi...@mesos.apache.org>
Subject Re: Review Request 68158: Fixed the iptables deadlock in CNI port mapper plugin.
Date Thu, 02 Aug 2018 07:07:09 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68158/#review206785
-----------------------------------------------------------



FAIL: Some of the unit tests failed. Please check the relevant logs.

Reviews applied: `['68158']`

Failed command: `Start-MesosCITesting`

All the build artifacts available at: http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2027/mesos-review-68158

Relevant logs:

- [mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2027/mesos-review-68158/logs/mesos-tests-stdout.log):

```
[----------] 9 tests from Endpoint/SlaveEndpointTest (982 ms total)

[----------] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest
[ RUN      ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0
[       OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/0 (31 ms)
[ RUN      ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1
[       OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1 (35 ms)
[----------] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest (73 ms total)

[----------] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN      ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[       OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (1201 ms)
[----------] 1 test from IsolationFlag/CpuIsolatorTest (1222 ms total)

[----------] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN      ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[       OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (1389 ms)
[----------] 1 test from IsolationFlag/MemoryIsolatorTest (1411 ms total)

[----------] Global test environment tear-down
[==========] 1013 tests from 98 test cases ran. (700445 ms total)
[  PASSED  ] 1011 tests.
[  FAILED  ] 2 tests, listed below:
[  FAILED  ] DockerTest.ROOT_DOCKER_interface
[  FAILED  ] DockerTest.ROOT_DOCKER_kill

 2 FAILED TESTS
  YOU HAVE 222 DISABLED TESTS

```

- [mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2027/mesos-review-68158/logs/mesos-tests-stderr.log):

```
I0802 07:06:49.881471 12444 slave.cpp:3939] Shutting down framework 8297884e-f31b-4b26-9c8c-ccfb744f6787-0000
I0802 07:06:49.881471 11216 master.cpp:10963] Updating the state of task 532adba1-32d4-44be-ad63-aba7a2b057a5
of framework 8297884e-f31b-4b26-9c8c-ccfb744f6787-0000 (latest state: TASK_KILLED, statI0802
07:06:49.581436 12220 exec.cpp:162] Version: 1.7.0
I0802 07:06:49.606456  6056 exec.cpp:236] Executor registered on agent 8297884e-f31b-4b26-9c8c-ccfb744f6787-S0
I0802 07:06:49.609436  7708 executor.cpp:182] Received SUBSCRIBED event
I0802 07:06:49.614432  7708 executor.cpp:186] Subscribed executor on windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net
I0802 07:06:49.614432  7708 executor.cpp:182] Received LAUNCH event
I0802 07:06:49.619463  7708 executor.cpp:679] Starting task 532adba1-32d4-44be-ad63-aba7a2b057a5
I0802 07:06:49.698460  7708 executor.cpp:499] Running 'D:\DCOS\mesos\src\mesos-containerizer.exe
launch <POSSIBLY-SENSITIVE-DATA>'
I0802 07:06:49.853458  7708 executor.cpp:693] Forked command at 12168
I0802 07:06:49.883463  4152 exec.cpp:445] Executor asked to shutdown
I0802 07:06:49.884474 10908 executor.cpp:182] Received SHUTDOWN event
I0802 07:06:49.884474 10908 executor.cpp:796] Shutting down
I0802 07:06:49.884474 10908 executor.cpp:909] Sending SIGTERM to process tree at pid 12us
update state: TASK_KILLED)
I0802 07:06:49.881471 12444 slave.cpp:6658] Shutting down executor '532adba1-32d4-44be-ad63-aba7a2b057a5'
of framework 8297884e-f31b-4b26-9c8c-ccfb744f6787-0000 at executor(1)@192.10.1.6:60867
I0802 07:06:49.883463 12444 slave.cpp:931] Agent terminating
W0802 07:06:49.883463 12444 slave.cpp:3935] Ignoring shutdown framework 8297884e-f31b-4b26-9c8c-ccfb744f6787-0000
because it is terminating
I0802 07:06:49.883463 11216 master.cpp:11061] Removing task 532adba1-32d4-44be-ad63-aba7a2b057a5
with resources cpus(allocated: *):4; mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated:
*):[31000-32000] of framework 8297884e-f31b-4b26-9c8c-ccfb744f6787-0000 on agent 8297884e-f31b-4b26-9c8c-ccfb744f6787-S0
at slave(462)@192.10.1.6:59164 (windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0802 07:06:49.887471 11216 master.cpp:1338] Agent 8297884e-f31b-4b26-9c8c-ccfb744f6787-S0
at slave(462)@192.10.1.6:59164 (windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
disconnected
I0802 07:06:49.887471 11216 master.cpp:3354] Disconnecting agent 8297884e-f31b-4b26-9c8c-ccfb744f6787-S0
at slave(462)@192.10.1.6:59164 (windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0802 07:06:49.888473 11216 master.cpp:3373] Deactivating agent 8297884e-f31b-4b26-9c8c-ccfb744f6787-S0
at slave(462)@192.10.1.6:59164 (windows-02.enofukwu14ruplxn0gs3yzmsgf.xx.internal.cloudapp.net)
I0802 07:06:49.888473  7676 hierarchical.cpp:359] Removed framework 8297884e-f31b-4b26-9c8c-ccfb744f6787-0000
I0802 07:06:49.888473  7676 hierarchical.cpp:795] Agent 8297884e-f31b-4b26-9c8c-ccfb744f6787-S0
deactivated
I0802 07:06:49.888473 11432 containerizer.cpp:2407] Destroying container baf4bdb7-236e-4299-9fae-7b32648fc207
in RUNNING state
I0802 07:06:49.888473 11432 containerizer.cpp:3021] Transitioning the state of container baf4bdb7-236e-4299-9fae-7b32648fc207
from RUNNING to DESTROYING
I0802 07:06:49.889477 11432 launcher.cpp:155] Asked to destroy container baf4bdb7-236e-4299-9fae-7b32648fc207
I0802 07:06:49.961460  7676 containerizer.cpp:2860] Container baf4bdb7-236e-4299-9fae-7b32648fc207
has exited
I0802 07:06:49.989460 11024 master.cpp:1180] Master terminating
I0802 07:06:49.991472 11432 hierarchical.cpp:637] Removed agent 8297884e-f31b-4b26-9c8c-ccfb744f6787-S0
I0802 07:06:50.301430  5192 process.cpp:926] Stopped the socket accept loop
```

- Mesos Reviewbot Windows


On Aug. 2, 2018, 5:16 a.m., Jie Yu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68158/
> -----------------------------------------------------------
> 
> (Updated Aug. 2, 2018, 5:16 a.m.)
> 
> 
> Review request for mesos, Avinash sridharan, Chun-Hung Hsiao, and Greg Mann.
> 
> 
> Bugs: MESOS-9127
>     https://issues.apache.org/jira/browse/MESOS-9127
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> It is possible that the port mapping cleanup command will cause iptables
> to deadlock if there are a lot of entires in the iptables, because the
> `sed` won't process the next line while executing `iptables -w -t nat -D
> ...`. But the executing of `iptables -w -t nat -D ... ` might get stuck
> if the first command `iptables -w -t nat -S %s` didn't finish (because
> the xtables lock is not released). The first command might not finish if
> it has a lot of output, filling the pipe that `sed` hasn't had a chance
> to process yet. See more details in MESOS-9127.
> 
> This patch fixed the issue by writing the commands to a file and then
> executing them.
> 
> 
> Diffs
> -----
> 
>   src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.cpp
c40b57f78193520f9f0b901201b5c4c855cde8b3 
> 
> 
> Diff: https://reviews.apache.org/r/68158/diff/1/
> 
> 
> Testing
> -------
> 
> sudo make check
> ```
> [       OK ] CniIsolatorPortMapperTest.ROOT_INTERNET_CURL_PortMapper (8827 ms)
> ```
> 
> 
> Thanks,
> 
> Jie Yu
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message