An interestingly when this happened (only one of the nodes was under GC), the gemfire client's weren't able to establish connection with the system.
Who are the AsyncSocketCloser threads?
I had an issue once when in my multi node deployment one of the nodes was heavily performing garbage collection and eventually crashed OOM.
This was due to bug in business logic, however looking at the crash heapdump I saw large amount of threads named "AsyncSocketCloser for <IP>(name)<v12>:PORT". Just curious who do these threads are.
Please sign in to leave a comment.
So there were around 500 AsyncSocketCloser threads and also around 2400 P2P handshake reader threads.
I am trying to understand what was the reason the gemfire clients weren't connecting to other nodes.
Is there any limit on amount of P2P handshake reader threads?
Is there any limit on ServerConnection threads?
There is an attribute "max-connections" which define Maximum number of client connections for the server. When the maximum is reached, the server refuses additional client connections. by default its value is 800
For p2p reader thread depends on conserve-
sockets parameter. If it is set to true, there is one sender and one receiver between members. If it set to false, there is an unlimited number of senders and receivers.
please have a look into below document
conserve-sockets is disabled, so for internal communication there are unlimited connections and unlimited threads?
Does the "max-connections" affect amount of "ServerConnection" threads? If max-connections has default value, does this mean there can be no more than 800 ServerConnection threads on a single node?
Yes, if numbers of client connection reached 800, server refuse new client connection
Yea, I understand. What I am saying is the amount of client connections reflected by the server connection threads?
If there are 800 connections, should I also see 800 server connection threads?
When do server connection threads (also p2p hanshake reader, p2p message reader) threads get destroyed, what is their keep alive time? Is this bound to socket lease time or they have different settings/configurations?
Yes, I think you should see 800 server connection thread.
for client/server it depends on idle-timeout
For p2p connection there is no keep alive once process finished it close the thread.
"socket-lease-time" is for a thread, that can have exclusive access to a socket it is not actively using.