mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cong Wang" <>
Subject Re: Review Request 31505: Add flow classifiers for fq_codel on egress
Date Mon, 01 Jun 2015 22:45:31 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated June 1, 2015, 10:45 p.m.)

Review request for mesos, Chi Zhang, Ian Downes, and Jie Yu.


Rebase the patch per dicussion with Jie.

Bugs: MESOS-2422

Repository: mesos


Currently we do nothing on the host egress side. By default, kernel uses its own hash function
to classify the packets to different TX queues (if the hardware interface supports multiqueue).
So packets coming out of different containers could end up queueing in the same TX queue,
in this case we saw buffer bloat on some TX queue caused packet drops.

We need to isolation the egress traffic so that containers will not have interference with
each other. The number of hardware TX queues is limited by hardware interface, usually not
enough to map our container in 1:1 way, therefore we need some software solution. We choose
fq_codel and use tc filters to classify packets coming out of different containers to different
fq_codel flows, and the codel algorithm on each flow could also help us to reduce the buffer
bloat. Note when the packets leave fq_codel, they still share the physical TX queue(s), this
is however (almost) beyond what we can control, we have to rely on the kernel behavior.

TODO: get some performance numbers

Diffs (updated)

  src/linux/routing/filter/basic.cpp 4ce8acb7040f2ed8ec3834dd7702a20f5316c20f 
  src/linux/routing/filter/filter.hpp aaca57fbe80e3ffa3dd2c2bbed93849013b7a382 
  src/linux/routing/filter/icmp.cpp 76877fb94a1c1e79133b8975419d2ea7d0300650 
  src/linux/routing/filter/ip.hpp 9645f9488938c55fec253b36d9fa30eae72a8ca2 
  src/linux/routing/filter/ip.cpp 0f3b856bd04f6a881e35452a24399715cd8a174f 
  src/slave/containerizer/isolators/network/port_mapping.hpp 6579286b779882bec493a8e0f10486adc316dc6e

  src/slave/containerizer/isolators/network/port_mapping.cpp 843e52d6f9923d0ee0a0297cd5c464b8b72f5de3



Manually start two mesos containers with netperf running side.


Cong Wang

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message