Rabbitmq Tcp_listen_options Binary

Hello, fellow RabbitMQ users!

I am very new RabbitMQ user (using version three.v.1) and struggeling to
understand the memory usage with my simple application. I am using Stomp
plugin and have 100 clients connected (creating effectually 500 queues). In that location
is not much activity there from the clients, the organization is mainly idle.
What i do non sympathize is the high retention usage of my Rabbit case.
Namely why so much retentivity is gone to binaries section and why the "Binary
references" section does not testify the utilise of that memory. Thanks for whatever
clues that would help me to minimize Rabbit memory use in this example.

Picture from Rabbit Web Panel:

<Loading Image... >

Output of "rabbitmqctrl status":

Status of node '***@pk-pc' ...
[{pid,2194},
{running_applications,
[{rabbitmq_management,"RabbitMQ Management Console","3.5.one"},
{rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.five.one"},
{webmachine,"webmachine","1.x.3-rmq3.five.one-gite9359c7"},
{mochiweb,"MochiMedia Spider web Server","2.seven.0-rmq3.5.1-git680dba8"},
{rabbitmq_stomp,"Embedded Rabbit Stomp Adapter","3.5.ane"},
{rabbitmq_management_agent,"RabbitMQ Direction Agent","3.5.one"},
{rabbit,"RabbitMQ","3.5.1"},
{os_mon,"CPO CXC 138 46","2.2.14"},
{mnesia,"MNESIA CXC 138 12","4.11"},
{amqp_client,"RabbitMQ AMQP Client","iii.5.1"},
{sockjs,"SockJS","0.three.4-rmq3.5.1-git3132eb9"},
{inets,"INETS CXC 138 49","5.ix.7"},
{xmerl,"XML parser","i.3.5"},
{sasl,"SASL CXC 138 11","ii.3.four"},
{stdlib,"ERTS CXC 138 10","ane.19.4"},
{kernel,"ERTS CXC 138 10","2.16.4"}]},
{os,{unix,linux}},
{erlang_version,
"Erlang R16B03 (erts-5.10.4) [source] [64-fleck] [smp:iv:4]
[async-threads:30] [kernel-poll:true]\n"},
{retentiveness,
[{full,372755696},
{connection_readers,0},
{connection_writers,0},
{connection_channels,0},
{connection_other,22698048},
{queue_procs,23044248},
{queue_slave_procs,0},
{plugins,11556504},
{other_proc,11947144},
{mnesia,1911816},
{mgmt_db,6136048},
{msg_index,2087400},
{other_ets,1717856},
{binary,265533264},
{lawmaking,20184143},
{atom,703377},
{other_system,5235848}]},
{alarms,[]},
{listeners,[{clustering,25672,"::"},{amqp,5672,"::"},{stomp,61613,"::"}]},
{vm_memory_high_watermark,0.4},
{vm_memory_limit,6681539379},
{disk_free_limit,50000000},
{disk_free,97045086208},
{file_descriptors,
[{total_limit,924},
{total_used,105},
{sockets_limit,829},
{sockets_used,103}]},
{processes,[{limit,131072},{used,2840}]},
{run_queue,0},
{uptime,4122}]

Thank you for any clues.

BR,

Priit.

--
Y'all received this bulletin because y'all are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and terminate receiving emails from it, send an electronic mail to rabbitmq-users+***@googlegroups.com.
To post to this group, send an email to rabbitmq-***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

About of that retentivity is TCP buffers. You can configure their size to 8-16K to get significant decrease in RAM use for depression activeness connections.

MK

Post by Priit Karu
How-do-you-do, boyfriend RabbitMQ users!
I am very new RabbitMQ user (using version 3.5.1) and struggeling to understand the retention usage with my unproblematic application. I am using Stomp plugin and accept 100 clients connected (creating effectually 500 queues). There is not much activeness there from the clients, the organisation is mainly idle. What i do not understand is the high memory usage of my Rabbit instance. Namely why so much memory is gone to binaries department and why the "Binary references" section does not show the utilize of that memory. Thanks for whatever clues that would help me to minimize Rabbit retentiveness use in this instance.
[{pid,2194},
{running_applications,
[{rabbitmq_management,"RabbitMQ Direction Console","3.5.one"},
{rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","iii.5.1"},
{webmachine,"webmachine","1.10.3-rmq3.5.one-gite9359c7"},
{mochiweb,"MochiMedia Spider web Server","2.seven.0-rmq3.5.1-git680dba8"},
{rabbitmq_stomp,"Embedded Rabbit Stomp Adapter","3.5.1"},
{rabbitmq_management_agent,"RabbitMQ Management Agent","3.5.i"},
{rabbit,"RabbitMQ","3.5.1"},
{os_mon,"CPO CXC 138 46","two.2.14"},
{mnesia,"MNESIA CXC 138 12","4.eleven"},
{amqp_client,"RabbitMQ AMQP Client","3.5.1"},
{sockjs,"SockJS","0.3.iv-rmq3.five.1-git3132eb9"},
{inets,"INETS CXC 138 49","5.nine.7"},
{xmerl,"XML parser","i.3.5"},
{sasl,"SASL CXC 138 11","2.3.four"},
{stdlib,"ERTS CXC 138 10","ane.nineteen.iv"},
{kernel,"ERTS CXC 138 x","2.16.four"}]},
{os,{unix,linux}},
{erlang_version,
"Erlang R16B03 (erts-v.10.iv) [source] [64-bit] [smp:4:4] [async-threads:30] [kernel-poll:true]\n"},
{retentiveness,
[{total,372755696},
{connection_readers,0},
{connection_writers,0},
{connection_channels,0},
{connection_other,22698048},
{queue_procs,23044248},
{queue_slave_procs,0},
{plugins,11556504},
{other_proc,11947144},
{mnesia,1911816},
{mgmt_db,6136048},
{msg_index,2087400},
{other_ets,1717856},
{binary,265533264},
{code,20184143},
{atom,703377},
{other_system,5235848}]},
{alarms,[]},
{listeners,[{clustering,25672,"::"},{amqp,5672,"::"},{stomp,61613,"::"}]},
{vm_memory_high_watermark,0.four},
{vm_memory_limit,6681539379},
{disk_free_limit,50000000},
{disk_free,97045086208},
{file_descriptors,
[{total_limit,924},
{total_used,105},
{sockets_limit,829},
{sockets_used,103}]},
{processes,[{limit,131072},{used,2840}]},
{run_queue,0},
{uptime,4122}]
Thank you for any clues.
BR,
Priit.
--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from information technology, ship an electronic mail to rabbitmq-users+***@googlegroups.com.
To mail to this grouping, send an email to rabbitmq-***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Thanks, Michael for quick response.

I accept already changed the tcp_listen_options for my rabbit case just
have not got whatsoever change to the binary size. My ubuntu had default buffer
for both tcp_rmem and tcp_wmem set to 87380:

true cat /proc/sys/net/ipv4/tcp_rmem
4096 87380 6291456

Here is my rabbitmq.conf.

[{rabbit,
[
{tcp_listen_options, [binary,
{sndbuf, 4096},
{recbuf, 4096},
{buffer, 4096},
{packet, raw},
{reuseaddr, true},
{backlog, 128},
{nodelay, true},
{linger, {true, 0}},
{exit_on_close, false}]}]}
].

Am i doing something however wrong with this configuration as information technology does not
seem to change the "binary" memory size at all.

Mail past Michael Klishin
Virtually of that memory is TCP buffers. You can configure their size to eight-16K
to become meaning decrease in RAM utilize for low activity connections.
MK
Hello, beau RabbitMQ users!
I am very new RabbitMQ user (using version 3.five.1) and struggeling to
sympathise the retention usage with my simple application. I am using Stomp
plugin and have 100 clients connected (creating around 500 queues). In that location
is non much action at that place from the clients, the system is mainly idle.
What i do not understand is the high memory usage of my Rabbit case.
Namely why so much memory is gone to binaries section and why the "Binary
references" department does not prove the use of that retentivity. Thanks for any
clues that would help me to minimize Rabbit retentivity use in this case.
<https://lh3.googleusercontent.com/-fF7N9gWTDWA/VUof1htTusI/AAAAAAAACoQ/VAS7c7aKTfo/s1600/rabbit_memory.png>
[{pid,2194},
{running_applications,
[{rabbitmq_management,"RabbitMQ Management Console","iii.5.ane"},
{rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","iii.five.1"},
{webmachine,"webmachine","ane.10.iii-rmq3.v.1-gite9359c7"},
{mochiweb,"MochiMedia Web Server","2.7.0-rmq3.5.ane-git680dba8"},
{rabbitmq_stomp,"Embedded Rabbit Stomp Adapter","3.5.one"},
{rabbitmq_management_agent,"RabbitMQ Management Agent","3.5.one"},
{rabbit,"RabbitMQ","three.v.1"},
{os_mon,"CPO CXC 138 46","2.2.14"},
{mnesia,"MNESIA CXC 138 12","4.11"},
{amqp_client,"RabbitMQ AMQP Client","3.5.1"},
{sockjs,"SockJS","0.3.4-rmq3.v.i-git3132eb9"},
{inets,"INETS CXC 138 49","5.nine.vii"},
{xmerl,"XML parser","i.3.5"},
{sasl,"SASL CXC 138 11","2.three.iv"},
{stdlib,"ERTS CXC 138 10","1.19.4"},
{kernel,"ERTS CXC 138 10","ii.xvi.4"}]},
{os,{unix,linux}},
{erlang_version,
"Erlang R16B03 (erts-5.10.iv) [source] [64-chip] [smp:4:4]
[async-threads:xxx] [kernel-poll:truthful]\northward"},
{memory,
[{total,372755696},
{connection_readers,0},
{connection_writers,0},
{connection_channels,0},
{connection_other,22698048},
{queue_procs,23044248},
{queue_slave_procs,0},
{plugins,11556504},
{other_proc,11947144},
{mnesia,1911816},
{mgmt_db,6136048},
{msg_index,2087400},
{other_ets,1717856},
{binary,265533264},
{code,20184143},
{atom,703377},
{other_system,5235848}]},
{alarms,[]},
{listeners,[{clustering,25672,"::"},{amqp,5672,"::"},{stomp,61613,"::"}]},
{vm_memory_high_watermark,0.four},
{vm_memory_limit,6681539379},
{disk_free_limit,50000000},
{disk_free,97045086208},
{file_descriptors,
[{total_limit,924},
{total_used,105},
{sockets_limit,829},
{sockets_used,103}]},
{processes,[{limit,131072},{used,2840}]},
{run_queue,0},
{uptime,4122}]
Cheers for whatever clues.
BR,
Priit.
--
Y'all received this message because you are subscribed to the Google Groups
"rabbitmq-users" group.
To unsubscribe from this grouping and stop receiving emails from it, send an
<javascript:>.
For more options, visit https://groups.google.com/d/optout.

--
You lot received this message considering you are subscribed to the Google Groups "rabbitmq-users" grouping.
To unsubscribe from this grouping and stop receiving emails from it, send an email to rabbitmq-users+***@googlegroups.com.
To postal service to this group, send an e-mail to rabbitmq-***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Do yous observe a growing number of connections? There was a bug fix in STOMP after 3.5.1 which makes sure nosotros don't leave connections with terminated due to socket exceptions backside.

Your agreement of TCP options seems to be correct and indeed autotuning defaults to about 87K on Linux.

MK

Post by Priit Karu
Thanks, Michael for quick response.
cat /proc/sys/net/ipv4/tcp_rmem
4096 87380 6291456
Here is my rabbitmq.conf.
[{rabbit,
[
{tcp_listen_options, [binary,
{sndbuf, 4096},
{recbuf, 4096},
{buffer, 4096},
{bundle, raw},
{reuseaddr, truthful},
{backlog, 128},
{nodelay, true},
{linger, {truthful, 0}},
{exit_on_close, false}]}]}
].
Am i doing something still incorrect with this configuration as it does non seem to modify the "binary" memory size at all.

Mail service by Michael Klishin
Most of that memory is TCP buffers. You can configure their size to viii-16K to get significant decrease in RAM utilize for low activity connections.
MK

Post by Priit Karu
Hello, swain RabbitMQ users!
I am very new RabbitMQ user (using version 3.5.1) and struggeling to empathize the memory usage with my elementary awarding. I am using Stomp plugin and accept 100 clients continued (creating effectually 500 queues). There is non much activity there from the clients, the arrangement is mainly idle. What i exercise non understand is the loftier memory usage of my Rabbit example. Namely why so much memory is gone to binaries department and why the "Binary references" section does non evidence the use of that memory. Thanks for any clues that would assistance me to minimize Rabbit memory use in this case.
[{pid,2194},
{running_applications,
[{rabbitmq_management,"RabbitMQ Management Panel","3.5.i"},
{rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.5.i"},
{webmachine,"webmachine","1.10.3-rmq3.5.i-gite9359c7"},
{mochiweb,"MochiMedia Spider web Server","two.seven.0-rmq3.5.1-git680dba8"},
{rabbitmq_stomp,"Embedded Rabbit Stomp Adapter","three.5.1"},
{rabbitmq_management_agent,"RabbitMQ Management Agent","iii.v.1"},
{rabbit,"RabbitMQ","3.5.1"},
{os_mon,"CPO CXC 138 46","2.two.14"},
{mnesia,"MNESIA CXC 138 12","iv.11"},
{amqp_client,"RabbitMQ AMQP Client","three.five.1"},
{sockjs,"SockJS","0.iii.4-rmq3.5.one-git3132eb9"},
{inets,"INETS CXC 138 49","v.9.7"},
{xmerl,"XML parser","1.three.five"},
{sasl,"SASL CXC 138 11","2.3.4"},
{stdlib,"ERTS CXC 138 10","1.xix.4"},
{kernel,"ERTS CXC 138 10","ii.16.4"}]},
{os,{unix,linux}},
{erlang_version,
"Erlang R16B03 (erts-v.10.four) [source] [64-bit] [smp:4:iv] [async-threads:30] [kernel-poll:true]\n"},
{memory,
[{total,372755696},
{connection_readers,0},
{connection_writers,0},
{connection_channels,0},
{connection_other,22698048},
{queue_procs,23044248},
{queue_slave_procs,0},
{plugins,11556504},
{other_proc,11947144},
{mnesia,1911816},
{mgmt_db,6136048},
{msg_index,2087400},
{other_ets,1717856},
{binary,265533264},
{code,20184143},
{atom,703377},
{other_system,5235848}]},
{alarms,[]},
{listeners,[{clustering,25672,"::"},{amqp,5672,"::"},{stomp,61613,"::"}]},
{vm_memory_high_watermark,0.4},
{vm_memory_limit,6681539379},
{disk_free_limit,50000000},
{disk_free,97045086208},
{file_descriptors,
[{total_limit,924},
{total_used,105},
{sockets_limit,829},
{sockets_used,103}]},
{processes,[{limit,131072},{used,2840}]},
{run_queue,0},
{uptime,4122}]
Thanks for whatsoever clues.
BR,
Priit.
--
You received this bulletin because you are subscribed to the Google Groups "rabbitmq-users" group.
For more options, visit https://groups.google.com/d/optout.

--
Y'all received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
For more than options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this grouping and stop receiving emails from information technology, send an email to rabbitmq-users+***@googlegroups.com.
To post to this group, send an email to rabbitmq-***@googlegroups.com.
For more than options, visit https://groups.google.com/d/optout.

Rabbitmq Tcp_listen_options Binary,

Source: https://rabbitmq-users.narkive.com/QgAHrxDS/rabbitmq-binaries-memory

Posted by: victorywongeste.blogspot.com

Share this post