[Bug 1521958] Re: rabbit: starvation of connections for reply

[Corey Bryant]

Hi Hui,

It looks like we're going to need this in liberty and kilo as well.
Liberty is at 2.5.0 and we don't do stable point releases for oslo
libraries.  Can you provide patches for liberty and kilo too?

Thanks,
Corey

-- 
You received this bug notification because you are a member of Ubuntu
Sponsors Team, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1521958

Title:
  rabbit: starvation of connections for reply

Status in Ubuntu Cloud Archive:
  New
Status in Ubuntu Cloud Archive juno series:
  New
Status in oslo.messaging:
  Fix Released
Status in oslo.messaging package in Ubuntu:
  New
Status in oslo.messaging source package in Trusty:
  New
Status in oslo.messaging source package in Vivid:
  Fix Released

Bug description:
  Hi,

  When a client died/restart/stop but was waiting more replies that the rpc_connection_pool size,
  the server will hold all connections from the pool during the retry logic in case of the client come back with the same reply_queue_id (that occurs only is rabbit is restart, not the client).

  Cheers,

  
  ---------------------------

  [Impact]

  This patch addresses an issue when multiple clients lost
  networks/died/restart/stop, the server will hold all connections from
  the pool(rpc_connection_pool size) during the retry logic in case of
  the client come back with the same reply_queue_id (that occurs only is
  rabbit is restart, not the client), and which cause nova-conductor
  infinitely reconnects to rabbit if large nova-compute hosts are
  deployment until all of the connections of the old reply messages are
  expired, for the quite large scaled cloud, high availability is
  broken.

  [Test Case]

  Note steps are for trusty-icehouse, including latest oslo.messaging
  library (1.3.0-0ubuntu1.2 at the time of this writing).

  Deploy an OpenStack cloud w/ multiple rabbit nodes and multiple nova
  compute hosts then cut off the network between OpenStack services and
  RabbitMQ. Observe that the nova-conductor is infinitely reconnecting
  to rabbit nodes.

  [Regression Potential]

  None.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1521958/+subscriptions