fix: harden LinkContentFetcher against SSRF

[simpleqt]

Validate every request target before the request is made, including each
redirect hop: redirects are now followed manually with per-hop re-validation
and a max_redirects cap (default 5). Add an optional allowed_hosts domain
suffix whitelist and always reject hosts resolving to private, loopback,
link-local, multicast, reserved, unspecified, unique-local, or shared
address space ranges. Stream response bodies with a max_response_bytes cap
(default 50 MB) and raise a clear error when it is exceeded.

Security context: LinkContentFetcher performed no target validation on user-supplied URLs (private/link-local/cloud-metadata ranges reachable, cross-host redirects followed without re-validation, unbounded response bodies), which turns any pipeline that feeds it untrusted URLs into an internal HTTP client with the response body landing in Document.content.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[vercel]

@simpleqt is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

[github-actions]

Hi @simpleqt, thanks for your interest in contributing to Haystack! 🙏

⚠️ You currently have 3 open pull requests in this repository (#12453, #12439 and this one). Our review capacity is limited, so please hold off opening more PRs until we've had a chance to review your first 2 open PRs. This helps us give each contribution the attention it deserves. Thank you!

This is an automated message to help us keep the review queue healthy.

[davidsbatista]

https://github.com/deepset-ai/haystack/blob/main/SECURITY.md#ssrf-via-url-fetching-components-eg-linkcontentfetcher

[simpleqt]

Understood — thank you for the pointer. The SECURITY.md position is clear: URL-fetching components intentionally do not gate hosts and SSRF prevention belongs to the application/network layer, so the opt-in allowlist approach in this PR was out of scope. Closing is the right call; no concerns from our side. (The max_runs_per_component validation in #12453 is separate and untouched by this policy — happy to iterate there.)