Solution for "Failed to secure tcp" timeout

[YuZhiYuanDev]

Problem Description

When using the RustDesk client, if both key and token are provided, the client will call secure_tcp to attempt establishing an encrypted connection with the server. However, the server does not respond to this request, causing the client to time out.

Relevant code: src/client.rs

if !key.is_empty() && !token.is_empty() {
    // mainly for the security of token
    secure_tcp(&mut socket, &key)
        .await
        .map_err(|e| anyhow!("Failed to secure tcp: {}", e))?;
}

If not logged into the API account, token is empty, so secure_tcp is not triggered and no timeout occurs.
Even in non-logged-in state, encrypted connection is still possible; you could have secure_tcp return directly (skip the actual encryption handshake).

---

Solutions

1. Use our forked server (recommended)

If you have high requirements for encryption or want to avoid timeout while keeping the client logged in, it is recommended to use the following fork:

• databk/rustdesk-server
  • Docker image: databk/rustdesk-server
  • Docker image: databk/rustdesk-server-s6

This fork optimizes the encryption handshake process, is compatible with scenarios where both key and token are provided, and additionally supports WebSocket connections and disabling UDP functionality.

2. Specify the same key on the server side

When starting hbbs and hbbr, set the same custom key using the -k parameter:

hbbs -r <relay-server-ip[:port]> -k <key>
hbbr -k <key>

This solution is suitable for scenarios with low encryption requirements.

3. Log out of the client before connecting

If the server uses a system-generated key (or a custom key pair), a logged-in client may time out or fail to connect.
Solution: Log out of the client before connecting.

4. Modify the client source code and compile it yourself

Fork the official code, modify secure_tcp to return directly (skip the encryption handshake), then compile via GitHub Actions.
Reference: Official build documentation

---

Recommendation

• Solution 1 (using our fork) is highly recommended: No need to modify the client, keep logged-in state, good compatibility, and supports enhanced features like WebSocket and disabling UDP.
• If encryption requirements are low, choose Solution 2 (specify key on server).
• If you are okay with logging out, choose Solution 3.
• If you are willing to compile yourself, choose Solution 4.

[vasmarfas]

Hi, I'm not quite sure how the backend at https://github.com/databk/rustdesk-console solves this problem. I just installed your project in place of https://github.com/lantongxue/rustdesk-api-server-pro, and I'm still getting the error “Failed to secure tcp: deadline has elapsed.”

[OCMan2K]

Solution 2 is not working for me, I specified the same key in the docker command ( as in the rustdesk client configuration )

same problem ith secure Tcp
I have also renamed the key files
mv id_ed25519 id_ed25519.bak
mv id_ed25519.pub id_ed25519.pub.ba
and restarted the docker but same problem....

I have replaced my official buit with your forked version and restored my key files but the problem remains, tcp error....

In fact with your fork its working... but my superior doesn't want me to use unofficial packages...

[YuZhiYuanDev]

@OCMan2K I fully understand your concerns about the security of third-party server implementations. However, if you are using the official client and server, this issue cannot be resolved—because the official client includes logic that automatically encrypts the connection test request whenever a custom server is used, while open-source servers do not support this feature. This may be an intentional restriction, designed to encourage users to purchase the paid version.

[YuZhiYuanDev]

@vasmarfas Could you please try using the solution I mentioned above? That should resolve the error.

[MiraMaX166]

I tried to download a custom server, I tried to launch a standard server with "-k" options, but nothing helps, it does not establish a connection while there is a login to the server api.

[MiraMaX166]

hey, does this thing work completely for anyone? If yes, please share the recipe!

[YuZhiYuanDev]

@MiraMaX166 Sorry to hear that, have you tried our forked server? It doesn't need -k and works for me.

[MiraMaX166]

Yes, thank you. I did everything, everything works. I would like to contribute and translate it into Russian.

[OCMan2K]

For me I can make French translation :)

[YuZhiYuanDev]

@MiraMaX166 @OCMan2K Thank you, I truly appreciate you both.