Fix eCONNECT_SYN timeout incorrectly transitioning to eCLOSE_WAIT (#1301)

[94xhn]

Fixes #1301.

Problem

Per RFC 793, a connection that fails during establishment (SYN-SENT state) must transition to CLOSED, not CLOSE-WAIT. CLOSE-WAIT is only valid for a connection that was previously ESTABLISHED and is now waiting for the local application to close it after the remote peer initiated a close.

prvTCPSendPacket() in FreeRTOS_TCP_Transmission.c violated this: when a socket in eCONNECT_SYN gives up after 3 SYN retries with no response, it called vTCPStateChange(pxSocket, eCLOSE_WAIT) instead of eCLOSED.

This isn't just a naming/spec-compliance issue - it has a concrete functional consequence. vTCPStateChange()'s "connecting phase failed" branch in FreeRTOS_TCP_IP.c only matched eCLOSE_WAIT as the target state:

if( ( ( xPreviousState == eCONNECT_SYN ) || ... ) &&
    ( eTCPState == eCLOSE_WAIT ) )
{
    /* forces bBefore = pdTRUE so the state-change-notification logic below runs */
}

Since the SYN-retry-exhausted path already used eCLOSE_WAIT (matching this branch), the immediate symptom is masked for that one call site. However, this same "connecting phase failed" branch is what must also fire for the other legitimate way a connecting socket gets closed - e.g. receiving an RST while in eCONNECT_SYN, which correctly calls vTCPStateChange(pxSocket, eCLOSED) elsewhere (see xProcessReceivedTCPPacket()'s RST handling). Because eCLOSED was never part of this branch's condition, that path does not set xEventBits |= eSOCKET_CLOSED (the bAfter == pdFALSE branch a few lines below).

FreeRTOS_connect() (FreeRTOS_Sockets.c) waits specifically on eSOCKET_CONNECT | eSOCKET_CLOSED via xEventGroupWaitBits() to detect a failed connection attempt promptly:

uxEvents = xEventGroupWaitBits( pxSocket->xEventGroup,
                                eSOCKET_CONNECT | eSOCKET_CLOSED,
                                ... , xRemainingTime );
if( ( uxEvents & eSOCKET_CLOSED ) != 0U )
{
    xResult = -pdFREERTOS_ERRNO_ENOTCONN;
    ...
}

Without eSOCKET_CLOSED ever being set for this transition, FreeRTOS_connect() cannot detect the failure early and instead just sits until its own xRemainingTime timeout expires, returning ETIMEDOUT instead of promptly returning ENOTCONN.

Fix

Two changes, matching exactly what maintainer @htibosch proposed and tested on real hardware in the issue thread, in a prior PR #1302 that was independently approved by two maintainers but never merged (its CI was never triggered for the fork PR, and the original submitter closed it after ~2.5 months of inactivity - not due to any code problem):

1. FreeRTOS_TCP_Transmission.c: change the SYN-retry-exhausted call from vTCPStateChange(pxSocket, eCLOSE_WAIT) to vTCPStateChange(pxSocket, eCLOSED), matching RFC 793.
2. FreeRTOS_TCP_IP.c: extend the "connecting phase failed" branch's condition to also match eCLOSED (not just eCLOSE_WAIT), so that any connecting-phase failure - whether via SYN-retry timeout or RST - consistently sets eSOCKET_CLOSED and lets FreeRTOS_connect() return promptly.

Verification

I don't have a Ruby/Ceedling environment fully set up in this checkout to run the complete unit-test suite (got as far as installing Ruby/CMock, but the test build also requires the unifdefall tool which isn't readily available on this machine), so I verified in two ways:

1. Diffed my change against PR Fix incorrect state transition #1302's source changes (already reviewed and approved by two maintainers) - identical.
2. Wrote a standalone C reproduction of the exact bBefore/bAfter/eSOCKET_CLOSED logic from vTCPStateChange(), confirming:
   • eCONNECT_SYN -> eCLOSED: eSOCKET_CLOSED is NOT set before the fix, IS set after.
   • eCONNECT_SYN -> eCLOSE_WAIT (the pre-existing, already-working path used by e.g. keep-alive timeout from eESTABLISHED): behavior unchanged before/after, confirming no regression.
   • eSYN_FIRST/eSYN_RECEIVED -> eCLOSED: same fix applies symmetrically (these share the same condition).

Added test_vTCPStateChange_ClosedState_PrvStateSyn to FreeRTOS_TCP_IP_utest.c, following the existing test_vTCPStateChange_ClosedWaitState_PrvStateSyn as a template, to cover the newly-fixed eCONNECT_SYN -> eCLOSED path and assert eSOCKET_CLOSED is correctly set in xEventBits.

Disclosure

Generative AI (Claude) was used to help investigate this issue, implement, and verify this fix. All changes were reviewed by me before submission.

[htibosch]

Thank you @94xhn for reminding of this issue.
At this moment I can not reaach1302. But I do have a copy of my proposed changes.
Mentioned PR page gives me an error. I will try again tomorrow.
Hein

[94xhn]

Thanks for checking, Hein. No problem - please take your time. I?ll leave #1355 open while you verify the reference change; the relevant changes are self-contained in this PR, and #1302 is only included for comparison.

[htibosch]

/bot run checks