metarefresh incompatible with SSP 2.5.0: Unsupported option "http" passed to "Symfony\Component\HttpClient\CurlHttpClient"

[msalle]

With the full source tarball for SSP 2.5.0 I get an error

metarefresh: Unsupported option "http" passed to "Symfony\Component\HttpClient\CurlHttpClient", did you mean "http_version"?

My guess is this comes from
metarefresh's src/MetaLoader.php line 403
which needs adapting for the new Symfony version 7.4 included with 2.5.0, along the lines of simplesamlphp/simplesamlphp#2574 ?

[tvdijen]

Thanks @msalle ! Would you be able to run a test with the suggested fix in #63 ?
You can install it by running bin/composer simplesamlphp/simplesamlphp-module-metarefresh#dev-bugfix/http-client

[msalle]

Thanks for the quick response!

Thanks @msalle ! Would you be able to run a test with the suggested fix in #63 ? You can install it by running bin/composer simplesamlphp/simplesamlphp-module-metarefresh#dev-bugfix/http-client

With the #63 as patch I now get a new error:

15:35:20.678Z metarefresh: Unsupported option "User-Agent" passed to "Symfony\Component\HttpClient\CurlHttpClient", did you mean "auth_basic", "auth_bearer", "query", "headers", "body", "json", "user_data", "max_redirects", "http_version", "base_uri", "buffer", "on_progress", "resolve", "proxy", "no_proxy", "timeout", "max_duration", "bindto", "verify_peer", "verify_host", "cafile", "capath", "local_cert", "local_pk", "passphrase", "ciphers", "peer_fingerprint", "capture_peer_cert_chain", "crypto_method", "extra", "auth_ntlm"?

[tvdijen]

Try again? I've pushed a one-liner fix

[msalle]

Try again? I've pushed a one-liner fix

Now I get a stacktrace:

0 /project/srv/www/site/simplesamlphp-2.5.0-prod/public/module.php:17 (N/A)
SimpleSAML\Error\Error: UNHANDLEDEXCEPTION
Backtrace:
2 /project/srv/www/site/simplesamlphp-2.5.0-prod/src/SimpleSAML/Error/ExceptionHandler.php:47 (SimpleSAML\Error\ExceptionHandler::customExceptionHandler)
1 /project/srv/www/site/simplesamlphp-2.5.0-prod/vendor/symfony/error-handler/ErrorHandler.php:538 (Symfony\Component\ErrorHandler\ErrorHandler::handleException)
0 [builtin] (N/A)
Caused by: TypeError: preg_match(): Argument #2 ($subject) must be of type string, null given
Backtrace:
8 /project/srv/www/site/simplesamlphp-2.5.0-prod/modules/metarefresh/src/MetaLoader.php:126 (preg_match)
7 /project/srv/www/site/simplesamlphp-2.5.0-prod/modules/metarefresh/src/MetaLoader.php:126 (SimpleSAML\Module\metarefresh\MetaLoader::loadSource)
6 /project/srv/www/site/simplesamlphp-2.5.0-prod/modules/metarefresh/src/MetaRefresh.php:129 (SimpleSAML\Module\metarefresh\MetaRefresh::runRefresh)
5 /project/srv/www/site/simplesamlphp-2.5.0-prod/modules/metarefresh/src/Controller/MetaRefresh.php:84 (SimpleSAML\Module\metarefresh\Controller\MetaRefresh::main)
4 /project/srv/www/site/simplesamlphp-2.5.0-prod/vendor/symfony/http-kernel/HttpKernel.php:183 (Symfony\Component\HttpKernel\HttpKernel::handleRaw)
3 /project/srv/www/site/simplesamlphp-2.5.0-prod/vendor/symfony/http-kernel/HttpKernel.php:76 (Symfony\Component\HttpKernel\HttpKernel::handle)
2 /project/srv/www/site/simplesamlphp-2.5.0-prod/vendor/symfony/http-kernel/Kernel.php:193 (Symfony\Component\HttpKernel\Kernel::handle)
1 /project/srv/www/site/simplesamlphp-2.5.0-prod/src/SimpleSAML/Module.php:240 (SimpleSAML\Module::process)
0 /project/srv/www/site/simplesamlphp-2.5.0-prod/public/module.php:17 (N/A)
Error report with id 38adb9a8 generated.

I'm just verifying if that's due to a patch I have on MetaLoader.php... No it's not, it's https://github.com/simplesamlphp/simplesamlphp-module-metarefresh/blob/master/src/MetaLoader.php#L126

[msalle]

And more clearly from the commandline:

./bin/metarefresh.php -s https://metadata.surfconext.nl/edugain-downstream-idp.xml
PHP Warning:  Undefined array key 0 in /project/srv/www/site/simplesamlphp-2.5.0-prod/modules/metarefresh/src/MetaLoader.php on line 126
PHP Fatal error:  Uncaught TypeError: preg_match(): Argument #2 ($subject) must be of type string, null given in /project/srv/www/site/simplesamlphp-2.5.0-prod/modules/metarefresh/src/MetaLoader.php:126
Stack trace:
#0 /project/srv/www/site/simplesamlphp-2.5.0-prod/modules/metarefresh/src/MetaLoader.php(126): preg_match()
#1 /project/srv/www/site/simplesamlphp-2.5.0-prod/modules/metarefresh/bin/metarefresh.php(120): SimpleSAML\Module\metarefresh\MetaLoader->loadSource()
#2 {main}
  thrown in /project/srv/www/site/simplesamlphp-2.5.0-prod/modules/metarefresh/src/MetaLoader.php on line 126

The array has named indices of all the response headers, not 0 etc. (checked with a print_r).

[tvdijen]

We're getting there slowly. Can you give it another try with the updated PR?

[msalle]

We're getting there slowly. Can you give it another try with the updated PR?

not yet there...:

[root@ metarefresh]# ./bin/metarefresh.php -s http://metadata.surfconext.nl/edugain-downstream-idp.xml
PHP Fatal error:  Uncaught Error: Call to a member function getStatusCode() on array in /project/srv/www/site/simplesamlphp-2.5.0-prod/modules/metarefresh/src/MetaLoader.php:126
Stack trace:
#0 /project/srv/www/site/simplesamlphp-2.5.0-prod/modules/metarefresh/bin/metarefresh.php(120): SimpleSAML\Module\metarefresh\MetaLoader->loadSource()
#1 {main}
  thrown in /project/srv/www/site/simplesamlphp-2.5.0-prod/modules/metarefresh/src/MetaLoader.php on line 126

[tvdijen]

I think I got it this time.

[msalle]

Yes! The metarefresh cronjob works fine now!

I experimented also a little with the commandline tool, which still produces a stacktrace, but only after succesfully downloading:

./bin/metarefresh.php -s http://metadata.surfconext.nl/edugain-downstream-idp.xml > edugain.php
PHP Warning:  Undefined array key "entityid" in /project/srv/www/site/simplesamlphp-2.5.0-prod/modules/metarefresh/src/MetaLoader.php on line 505
PHP Fatal error:  Uncaught TypeError: addslashes(): Argument #1 ($string) must be of type string, null given in /project/srv/www/site/simplesamlphp-2.5.0-prod/modules/metarefresh/src/MetaLoader.php:509
Stack trace:
#0 /project/srv/www/site/simplesamlphp-2.5.0-prod/modules/metarefresh/src/MetaLoader.php(509): addslashes()
#1 /project/srv/www/site/simplesamlphp-2.5.0-prod/modules/metarefresh/bin/metarefresh.php(124): SimpleSAML\Module\metarefresh\MetaLoader->dumpMetadataStdOut()
#2 {main}
  thrown in /project/srv/www/site/simplesamlphp-2.5.0-prod/modules/metarefresh/src/MetaLoader.php on line 509

If I bork the URL and get a 404 it looks like I more or less get the right stacktrace:

# ./bin/metarefresh.php -s http://metadata.surfconext.nl/edugain-downstream-idp.xm
PHP Fatal error:  Uncaught Symfony\Component\HttpClient\Exception\ClientException: HTTP/2 404  returned for "https://metadata.surfconext.nl/edugain-downstream-idp.xm". in /project/srv/www/site/simplesamlphp-2.5.0-prod/vendor/symfony/http-client/Response/CommonResponseTrait.php:170
Stack trace:
#0 /project/srv/www/site/simplesamlphp-2.5.0-prod/vendor/symfony/http-client/Response/TransportResponseTrait.php(67): Symfony\Component\HttpClient\Response\CurlResponse->checkStatusCode()
#1 /project/srv/www/site/simplesamlphp-2.5.0-prod/modules/metarefresh/src/MetaLoader.php(116): Symfony\Component\HttpClient\Response\CurlResponse->getHeaders()
#2 /project/srv/www/site/simplesamlphp-2.5.0-prod/modules/metarefresh/bin/metarefresh.php(120): SimpleSAML\Module\metarefresh\MetaLoader->loadSource()
#3 {main}
  thrown in /project/srv/www/site/simplesamlphp-2.5.0-prod/vendor/symfony/http-client/Response/CommonResponseTrait.php on line 170

although it probably should be caught?

And likewise for illegal hostname:

# ./bin/metarefresh.php -s http://metadata.surfconext.n/
PHP Fatal error:  Uncaught Symfony\Component\HttpClient\Exception\TransportException: Could not resolve host: metadata.surfconext.n for "http://metadata.surfconext.n/". in /project/srv/www/site/simplesamlphp-2.5.0-prod/vendor/symfony/http-client/Response/CurlResponse.php:337
Stack trace:
#0 /project/srv/www/site/simplesamlphp-2.5.0-prod/vendor/symfony/http-client/Response/TransportResponseTrait.php(173): Symfony\Component\HttpClient\Response\CurlResponse::perform()
#1 /project/srv/www/site/simplesamlphp-2.5.0-prod/vendor/symfony/http-client/Response/CommonResponseTrait.php(145): Symfony\Component\HttpClient\Response\CurlResponse::stream()
#2 /project/srv/www/site/simplesamlphp-2.5.0-prod/vendor/symfony/http-client/Response/TransportResponseTrait.php(54): Symfony\Component\HttpClient\Response\CurlResponse::initialize()
#3 /project/srv/www/site/simplesamlphp-2.5.0-prod/modules/metarefresh/src/MetaLoader.php(115): Symfony\Component\HttpClient\Response\CurlResponse->getStatusCode()
#4 /project/srv/www/site/simplesamlphp-2.5.0-prod/modules/metarefresh/bin/metarefresh.php(120): SimpleSAML\Module\metarefresh\MetaLoader->loadSource()
#5 {main}

Next Symfony\Component\HttpClient\Exception\TransportException: Could not resolve host: metadata.surfconext.n for "http://metadata.surfconext.n/". in /project/srv/www/site/simplesamlphp-2.5.0-prod/vendor/symfony/http-client/Chunk/ErrorChunk.php:55
Stack trace:
#0 /project/srv/www/site/simplesamlphp-2.5.0-prod/vendor/symfony/http-client/Response/CommonResponseTrait.php(146): Symfony\Component\HttpClient\Chunk\ErrorChunk->isFirst()
#1 /project/srv/www/site/simplesamlphp-2.5.0-prod/vendor/symfony/http-client/Response/TransportResponseTrait.php(54): Symfony\Component\HttpClient\Response\CurlResponse::initialize()
#2 /project/srv/www/site/simplesamlphp-2.5.0-prod/modules/metarefresh/src/MetaLoader.php(115): Symfony\Component\HttpClient\Response\CurlResponse->getStatusCode()
#3 /project/srv/www/site/simplesamlphp-2.5.0-prod/modules/metarefresh/bin/metarefresh.php(120): SimpleSAML\Module\metarefresh\MetaLoader->loadSource()
#4 {main}
  thrown in /project/srv/www/site/simplesamlphp-2.5.0-prod/vendor/symfony/http-client/Chunk/ErrorChunk.php on line 55

[tvdijen]

It's hard for me to test, because I've never used this module and I think I need to set it up before I can use it like that. The command-line gives me no output at all..
I think the exceptions from the HTTP-client should be caught now with my latest change.

Can you debug the contents of the metadata-array at line 509 for me? It's very strange that the entityid is not there...

[madsi1m]

#63 seems to fix things for me.
I had to also add this change #64 and now I can see a list of IDPs without errors/warnings.

[tvdijen]

Thanks @madsi1m ! But I'd much rather figure out where this difference is coming from. (and how this relates to the http-client update, because it seems unrelated).

[madsi1m]

With just #63 i get the list, with #64 it gets rid of the errors/warnings, it seems for some reason some IDPs are an array of single item [0] which then has everything inside which cause the issues, so i do a check.

I can confirm i do not have this list problem on SSP 2.4.4 but do on 2.5.0