8 Replies Latest reply on Jun 23, 2014 8:00 AM by 36672

    Tcp connection establishes connection to server but then goes to time wait and a new connection is established with port incrementation

           Application establishes connection to Marketo server on port 443 (SSL) Some data is returned from the server to the application After about a second this tcp connection goes to TIME_WAIT. Application then establishes a new tcp connection to server with the port number incremented. Some data is returned from the server to the application.  After about a second this tcp connection goes to TIME_WAIT ...This repeats about every second ...  We are a data integration company with a partnership with Marketo for context.

           Any ideas why this is occurring ??!0EM50000000RwLY

        • Re: Tcp connection establishes connection to server but then goes to time wait and a new connection is established with port incrementation
          Kenny Elkington
          Hey Peter,

          It might help if you could provide a little more context here.  What resource are you trying to access, and what are you trying to accomplish?
          • Re: Tcp connection establishes connection to server but then goes to time wait and a new connection is established with port incrementation
            For context we are a partner company (Scribe Software) with Marketo that offers a replication service that allows for Marketo mObjects such as Leads to replicate down to a SQL Server such as all fields and all records.  We also see the same above behavior when we are integrating into Marketo say from Salesforce to Marketo and a user wants to bring in Opportunities into Marketo.

            We are using the endpoint via a SOAP call to : https://827-UDX-588.mktoapi.com/soap/mktows/2_3

            with a user of :  
            scribedev31_33160868515D904099DF03  This is just one user account we are using as we have about four accounts for our dev and test work.

            So during the execution of the replication and/or integration we are seeing the above behavior with respect to TCP connection establishment, pass some data, have that connection go to time_wait and then having to repeat this process over and over while incrementing the port number on our local machine agent.  This appears to be very inefficient and resource intensive.

            We believe that this establishment, termination, re-establishment with incremental local port number is occurring on Marketo's side and we are just trying to identify the reason for this.

            Thanks for the reply!

            Pete
            • Re: Tcp connection establishes connection to server but then goes to time wait and a new connection is established with port incrementation
              Hi Pete,

              This might be a server load issue.

              Just to clarify, you are saying this is only happening on one of your Marketo instances, and the other three are performing normally? 

              Also, is there a specific time of day this is happening? 
              • Re: Tcp connection establishes connection to server but then goes to time wait and a new connection is established with port incrementation
                This is occurring on all of the Marketo instances we have been provisioned from Marketo for our integration dev/testing work.  And I believe it will occur at any time of day.  I believe there is something that your firewall/web server/ IIS if used is sensing a large api hit and it is attempting to prevent data from being sent from a single source port on our side and therefore breaks the connection and then it is re-established on our side?

                Snippet below (You can see the multiple handshakes occurring over TLS):

                The time stamps from this wireshark are UTC and the Marketo server or load balancer IP is : 199.15.215.110 .  Would it be possible to investigate the web server logs to see what the server is doing at this time period?

                     "No.","Time","Source","Destination","Protocol","Length","Info"
                     "9","2014-06-09 15:14:46.525234000","192.168.11.131","199.15.215.110","TCP","54","49918 > https [FIN, ACK] Seq=1 Ack=1126 Win=62932 Len=0"
                     "12","2014-06-09 15:14:46.525822000","192.168.11.131","199.15.215.110","TCP","54","49918 > https [ACK] Seq=2 Ack=1127 Win=62932 Len=0"
                     "13","2014-06-09 15:14:46.527974000","192.168.11.131","199.15.215.110","TCP","66","49920 > https [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1"
                     "32","2014-06-09 15:14:46.623180000","192.168.11.131","199.15.215.110","TCP","54","49920 > https [ACK] Seq=1 Ack=1 Win=64240 Len=0"
                     "33","2014-06-09 15:14:46.623604000","192.168.11.131","199.15.215.110","TLSv1","213","Client Hello"
                     "40","2014-06-09 15:14:46.721647000","192.168.11.131","199.15.215.110","TLSv1","390","Change Cipher Spec, Encrypted Handshake Message, Application Data"
                     "50","2014-06-09 15:14:46.817345000","192.168.11.131","199.15.215.110","TLSv1","879","Application Data"
                     "77","2014-06-09 15:14:47.164627000","192.168.11.131","199.15.215.110","TCP","54","49920 > https [FIN, ACK] Seq=1321 Ack=1309 Win=62932 Len=0"
                     "80","2014-06-09 15:14:47.165238000","192.168.11.131","199.15.215.110","TCP","54","49920 > https [ACK] Seq=1322 Ack=1310 Win=62932 Len=0"
                     "81","2014-06-09 15:14:47.167070000","192.168.11.131","199.15.215.110","TCP","66","49922 > https [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1"
                     "90","2014-06-09 15:14:47.262287000","192.168.11.131","199.15.215.110","TCP","54","49922 > https [ACK] Seq=1 Ack=1 Win=64240 Len=0"
                     "91","2014-06-09 15:14:47.262718000","192.168.11.131","199.15.215.110","TLSv1","213","Client Hello"
                     "96","2014-06-09 15:14:47.356043000","192.168.11.131","199.15.215.110","TLSv1","390","Change Cipher Spec, Encrypted Handshake Message, Application Data"
                     "99","2014-06-09 15:14:47.498028000","192.168.11.131","199.15.215.110","TLSv1","879","Application Data"
                     "103","2014-06-09 15:14:47.765506000","192.168.11.131","199.15.215.110","TCP","54","49922 > https [ACK] Seq=1321 Ack=1310 Win=62932 Len=0"
                     "104","2014-06-09 15:14:47.775173000","192.168.11.131","199.15.215.110","TCP","54","49922 > https [FIN, ACK] Seq=1321 Ack=1310 Win=62932 Len=0"
                     "106","2014-06-09 15:14:47.778395000","192.168.11.131","199.15.215.110","TCP","66","49923 > https [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1"
                     "109","2014-06-09 15:14:47.873145000","192.168.11.131","199.15.215.110","TCP","54","49923 > https [ACK] Seq=1 Ack=1 Win=64240 Len=0"
                     "110","2014-06-09 15:14:47.873507000","192.168.11.131","199.15.215.110","TLSv1","213","Client Hello"
                     "114","2014-06-09 15:14:47.969577000","192.168.11.131","199.15.215.110","TLSv1","390","Change Cipher Spec, Encrypted Handshake Message, Application Data"
                     "117","2014-06-09 15:14:48.066142000","192.168.11.131","199.15.215.110","TLSv1","879","Application Data"
                     "120","2014-06-09 15:14:48.351290000","192.168.11.131","199.15.215.110","TCP","54","49923 > https [FIN, ACK] Seq=1321 Ack=1309 Win=62932 Len=0"
                     "123","2014-06-09 15:14:48.351988000","192.168.11.131","199.15.215.110","TCP","54","49923 > https [ACK] Seq=1322 Ack=1310 Win=62932 Len=0"
                     "124","2014-06-09 15:14:48.353822000","192.168.11.131","199.15.215.110","TCP","66","49924 > https [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1"
                     "126","2014-06-09 15:14:48.446117000","192.168.11.131","199.15.215.110","TCP","54","49924 > https [ACK] Seq=1 Ack=1 Win=64240 Len=0"
                     "127","2014-06-09 15:14:48.446506000","192.168.11.131","199.15.215.110","TLSv1","213","Client Hello"
                     "130","2014-06-09 15:14:48.541917000","192.168.11.131","199.15.215.110","TLSv1","390","Change Cipher Spec, Encrypted Handshake Message, Application Data"
                     "133","2014-06-09 15:14:48.643662000","192.168.11.131","199.15.215.110","TLSv1","879","Application Data"
                     "137","2014-06-09 15:14:48.851992000","192.168.11.131","199.15.215.110","TCP","54","49924 > https [FIN, ACK] Seq=1321 Ack=1309 Win=62932 Len=0"
                     "140","2014-06-09 15:14:48.853938000","192.168.11.131","199.15.215.110","TCP","54","49924 > https [ACK] Seq=1322 Ack=1310 Win=62932 Len=0"
                     "141","2014-06-09 15:14:48.857453000","192.168.11.131","199.15.215.110","TCP","66","49925 > https [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1"
                     "144","2014-06-09 15:14:48.950265000","192.168.11.131","199.15.215.110","TCP","54","49925 > https [ACK] Seq=1 Ack=1 Win=64240 Len=0"
                     "145","2014-06-09 15:14:48.950692000","192.168.11.131","199.15.215.110","TLSv1","213","Client Hello"
                     "148","2014-06-09 15:14:49.046273000","192.168.11.131","199.15.215.110","TLSv1","390","Change Cipher Spec, Encrypted Handshake Message, Application Data"
                     "151","2014-06-09 15:14:49.195993000","192.168.11.131","199.15.215.110","TLSv1","879","Application Data"
                     "155","2014-06-09 15:14:49.581664000","192.168.11.131","199.15.215.110","TCP","54","49925 > https [ACK] Seq=1321 Ack=1310 Win=62932 Len=0"
                     "156","2014-06-09 15:14:49.581852000","192.168.11.131","199.15.215.110","TCP","54","49925 > https [FIN, ACK] Seq=1321 Ack=1310 Win=62932 Len=0"
                     "158","2014-06-09 15:14:49.584547000","192.168.11.131","199.15.215.110","TCP","66","49926 > https [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1"
                     "160","2014-06-09 15:14:49.682786000","192.168.11.131","199.15.215.110","TCP","54","49926 > https [ACK] Seq=1 Ack=1 Win=64240 Len=0"
                     "161","2014-06-09 15:14:49.683269000","192.168.11.131","199.15.215.110","TLSv1","213","Client Hello"
                     "164","2014-06-09 15:14:49.786609000","192.168.11.131","199.15.215.110","TLSv1","390","Change Cipher Spec, Encrypted Handshake Message, Application Data"
                     "174","2014-06-09 15:14:49.877587000","192.168.11.131","199.15.215.110","TLSv1","879","Application Data"
                     "177","2014-06-09 15:14:50.116840000","192.168.11.131","199.15.215.110","TCP","54","49926 > https [FIN, ACK] Seq=1321 Ack=1309 Win=62932 Len=0"
                     "180","2014-06-09 15:14:50.117582000","192.168.11.131","199.15.215.110","TCP","54","49926 > https [ACK] Seq=1322 Ack=1310 Win=62932 Len=0"
                     "181","2014-06-09 15:14:50.119626000","192.168.11.131","199.15.215.110","TCP","66","49927 > https [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1"
                     "183","2014-06-09 15:14:50.218126000","192.168.11.131","199.15.215.110","TCP","54","49927 > https [ACK] Seq=1 Ack=1 Win=64240 Len=0"
                • Re: Tcp connection establishes connection to server but then goes to time wait and a new connection is established with port incrementation
                  I will forward this to our operations team and get back to you with a response.
                  • Re: Tcp connection establishes connection to server but then goes to time wait and a new connection is established with port incrementation
                    Just wanted to inquire if there is  any update to this question.  Thanks!