Curl error thrown for http POST to /session #1110

Capn-Precious · 2024-08-19T13:46:39Z

Capn-Precious
Aug 19, 2024

The following errors are happening in a production & staging environment when we attempt to screenshot with panther, set size for the window, or do anything that utilizes panther for web scraping. It seems like the connection drops or is lost between panther and php-webdriver.

Errors:

Facebook\WebDriver\Exception\Internal\WebDriverCurlException: Curl error thrown for http POST to /session with params: 
{
    "capabilities": {
        "firstMatch": [
            {
                "browserName": "chrome",
                "goog:chromeOptions": {
                    "args": [
                        "--no-sandbox",
                        "--headless",
                        "--disable-dev-shm-usage",
                        "--hide-scrollbars"
                    ]
                }
            }
        ]
    },
    "desiredCapabilities": {
        "browserName": "chrome",
        "platform": "ANY",
        "goog:chromeOptions": {
            "args": [
                "--no-sandbox",
                "--headless",
                "--disable-dev-shm-usage",
                "--hide-scrollbars"
            ]
        }
    }
}
...
Operation timed out after 180000 milliseconds with 0 bytes received in /app/vendor/php-webdriver/webdriver/lib/Exception/Internal/WebDriverCurlException.php:20

Current Workarounds:

implemented exponential backoff when the WebDriverCurlException is thrown
implementing longer timeouts for connection, request, & read.

Versions::

php: 8.3
symfony: 6.4
symfony/panther: 2.1.1
php-webdriver/webdriver: 1.15.1
chrome: latest
chromedriver: latest, stable for chrome

uuf6429 · 2024-11-19T19:56:52Z

uuf6429
Nov 19, 2024

Could it help to use the "new" headless mode? https://stackoverflow.com/a/73840130/314056

0 replies

Capn-Precious · 2024-11-19T20:20:16Z

Capn-Precious
Nov 19, 2024
Author

I did try using the new headless mode but ran into issues with the window size which is a known bug for chrome. The workaround that was utilized was switching to an older version of chromedriver defined in our dockerfile which helps for now.

...
#version: 124.0.6367.78-r0)
RUN echo "https://dl-cdn.alpinelinux.org/alpine/v3.19/main/" >> /etc/apk/repositories;
RUN echo "https://dl-cdn.alpinelinux.org/alpine/v3.19/community/" >> /etc/apk/repositories;
...

2 replies

uuf6429 Nov 19, 2024

issues with the window size which is a known bug for chrome

Do you have any more info about that? I'm curious to know more.

Also, then essentially your last answers your question, right? Too bad it's not the answer we'd have hoped for.

Capn-Precious Nov 19, 2024
Author

Yes, chrome pretty much ignores the window size when you try to set it which is a bit annoying as it would then capture then entire page.

Here's some links to the error:
https://issues.chromium.org/issues/362522328
https://issues.chromium.org/issues/40275252

I mean there is a plan to move the old headless into it's own binary so that it works like before but that has not happened yet. Also, yes the reverting did resolve it for now but we can only hope the issues are resolved in future versions.

Google on chrome headless: https://developer.chrome.com/docs/chromium/headless

Navigation Menu

Curl error thrown for http POST to /session #1110

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 2 comments 2 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Curl error thrown for http POST to /session #1110

Uh oh!

Uh oh!

Capn-Precious Aug 19, 2024

Replies: 2 comments · 2 replies

Uh oh!

uuf6429 Nov 19, 2024

Uh oh!

Capn-Precious Nov 19, 2024 Author

Uh oh!

Uh oh!

uuf6429 Nov 19, 2024

Uh oh!

Capn-Precious Nov 19, 2024 Author

Capn-Precious
Aug 19, 2024

Replies: 2 comments 2 replies

uuf6429
Nov 19, 2024

Capn-Precious
Nov 19, 2024
Author

Capn-Precious Nov 19, 2024
Author