Post

2 followers Follow
0
Avatar

Who are the AsyncSocketCloser threads?

I had an issue once when in my multi node deployment one of the nodes was heavily performing garbage collection and eventually crashed OOM.
This was due to bug in business logic, however looking at the crash heapdump I saw large amount of threads named "AsyncSocketCloser for <IP>(name)<v12>:PORT". Just curious who do these threads are.

Hovhannes Antonyan

Please sign in to leave a comment.

10 comments

0
Avatar

An interestingly when this happened (only one of the nodes was under GC), the gemfire client's weren't able to establish connection with the system.

Hovhannes Antonyan 0 votes
0
Avatar

So there were around 500 AsyncSocketCloser threads and also around 2400 P2P handshake reader threads.
I am trying to understand what was the reason the gemfire clients weren't connecting to other nodes.
Is there any limit on amount of P2P handshake reader threads?

Hovhannes Antonyan 0 votes
0
Avatar

There is an attribute "max-connections" which define Maximum number of client connections for the server. When the maximum is reached, the server refuses additional client connections. by default its value is 800

http://gemfire81.docs.pivotal.io/latest/userguide/index.html#reference/topics/cache_xml.html

For p2p reader thread depends on conserve-
sockets parameter. If it is set to true, there is one sender and one receiver between members. If it set to false, there is an unlimited number of senders and receivers.
please have a look into below document
http://gemfire81.docs.pivotal.io/latest/userguide/managing/monitor_tune/socket_communication_have_enough_sockets.htm

Shuvro Das 0 votes
0
Avatar

conserve-sockets is disabled, so for internal communication there are unlimited connections and unlimited threads?
Does the "max-connections" affect amount of "ServerConnection" threads? If max-connections has default value, does this mean there can be no more than 800 ServerConnection threads on a single node?

Hovhannes Antonyan 0 votes
0
Avatar

Yea, I understand. What I am saying is the amount of client connections reflected by the server connection threads?
If there are 800 connections, should I also see 800 server connection threads?

Hovhannes Antonyan 0 votes
0
Avatar

When do server connection threads (also p2p hanshake reader, p2p message reader) threads get destroyed, what is their keep alive time? Is this bound to socket lease time or they have different settings/configurations?

Hovhannes Antonyan 0 votes
0
Avatar

for client/server it depends on idle-timeout
http://gemfire81.docs.pivotal.io/latest/userguide/reference/topics/cache_xml.html
http://gemfire81.docs.pivotal.io/latest/userguide/topologies_and_comm/topology_concepts/how_the_pool_manages_connections.html

For p2p connection there is no keep alive once process finished it close the thread.
"socket-lease-time" is for a thread, that can have exclusive access to a socket it is not actively using.

http://gemfire81.docs.pivotal.io/latest/userguide/index.html#reference/topics/gemfire_properties.html

Shuvro Das 0 votes