Twitter extractor will retrieve "/home" instead of a tweet URL

Twitter is now a fully logged-on website: you cannot access a tweet without an account.
So, the extractor will try to request the twitter.com/xxx/status/xxx but will follow location on /home (with the login screen) and will attempt to call the oembed API with /home URI.

The only fix that I found to disable this behavior is to disable the "follow redirection" behavior on cURL.

$client = new CurlClient();
$client->setSettings([
    'follow_location' => false
]);

$embed = new Embed(new Crawler($client));

We may use the cookie to inject the auth_token cookie, but I'm not sure the token won't change after a few hours/days.

This may impact other embeds, so if you have a better solution, please let me know!

Thank you for the fix, that worked for me as well. I only use that for twitter.com URLs.

Thanks, the 'follow_location' also helped here....

Here's the patch how I added it, being used from the Drupal url_embed module, https://www.drupal.org/project/url_embed/issues/3435840

--- src/Http/Crawler.php.orig   2024-03-27 13:33:31.547671482 +0100
+++ src/Http/Crawler.php        2024-03-27 13:34:14.180154682 +0100
@@ -23,6 +23,9 @@
     public function __construct(ClientInterface $client = null, RequestFactoryInterface $requestFactory = null, UriFactoryInterface $uriFactory = null)
     {
         $this->client = $client ?: new CurlClient();
+        $this->client->setSettings([
+                'follow_location' => false
+        ]);
         $this->requestFactory = $requestFactory ?: FactoryDiscovery::getRequestFactory();
         $this->uriFactory = $uriFactory ?: FactoryDiscovery::getUriFactory();
     }

Prevent EmbedParser to follow infinite redirect for Twitter links

@Divi

Hi @Divi, PR #589 should fix this: the oEmbed request now always goes to publish.twitter.com with the original tweet URL, even when the page redirects to x.com/home or the login wall.

You can test it with composer require embed/embed:dev-fix/4.5.5. We will leave the PR open for a few days before releasing 4.5.5, so any feedback is very welcome!

Navigation Menu

Repository navigation

Twitter extractor will retrieve "/home" instead of a tweet URL #520

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Twitter extractor will retrieve "/home" instead of a tweet URL #520

Description

Activity

stevecoug commented on Jul 1, 2023

helmo commented on Mar 27, 2024

Vitorinox commented on Oct 31, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions