add proxy argument and improve dns cache usage (#2209)

* add proxy argument and improve dns cache usage

add proxy argument that useing the -x and --proxy argument. add it to
the static curl config struct, command usage and help outputs of the
cli.

parse these argument together with the environment variables like
http_proxy before setting the CURLOPT_PROXY in the curl configuration
option. this is required, as there is no easy way to ascertain/get what
the CURLOPT_PROXY that libcurl will use. by the point it is set by
libcurl, we have no control over it anymore, and need it for the other
steps in the configuration.

if the CURLOPT_PROXY is set, skip the DNS cache population which would
set the CURLOPT_RESOLVE. this is currently not perfect however. if a
proxy is set with socks4 or socks5 scheme, the host should be resolving
the hostname.

* codespell, clang-format and hints fixes

* add curl version and ssl enabelement macro checks

might fix rocky linux 8 compilation issues.

* add proxy_resolves_hostname, determined by proxy scheme

leave the functions that print out an curl_easyoption, but dont use it. organize the code slightly, print out the final CURLOPT_PROXY and proxy_resolves_hostname flag on verbose mode, add comments

* remove unused handle_curl_easyoption and format_curl_easyoption functions

* fix typo in the proxy argument

* fix typo with proxy scheme socks5a->socks5h

* improve proxy environment parsing

add another argument: --no-proxy , which is used when setting
CURL_NOPROXY

additionally parse all_proxy, ALL_PROXY, no_proxy and NO_PROXY
environment variables in the correct order.

set the curlopt_proxy and curlopt_noproxy of libcurl, and additionally
save them in check_curl_working_state.

add function determine_hostname_resolver, uses the working state and
static config. it can tokenize the no_proxy variable and check for exact
matches, but cannot determine subnet matches for ip addresses yet.

* document proxy cli arguments

clarify and add more examples of proxy environment variables and their
behavior when multiple are specified, overriden etc.

add single wildcard '*' checking for no_proxy to
determine_hostname_resolver, special case per curlopt_noproxy
documentation

* check curlopt_noproxy before accessing it

* switch argument from --no-proxy to --noproxy like curl cli

* check if host name is a subdomain of an noproxy item

* use strdup where destination working_state.curlopt_proxy may be NULL

* add disclaimer about uppercase HTTP_PROXY

* add subdomain checks for each item in the no_proxy, if the target host is a subdomain proxy wont resolve it

add function ip_addr_inside_cidr, use it for checking possible cidr ranges  given in the no_proxy

* wip tests that work on local perl http/https server

* wip tests that work on the live debian image

* fix subnet definition

* make apache2 listen on [::1] for ipv6 tests

* remove squid certificate

* rewrite ip_addr_inside_cidr, split ipv4 and ipv6 parsing path and copy them to a shared buffer later on for prefix check

* Adapt tests for the squid sever, disable checking return code for socks 4/5 proxies. Squid does not support it, and we do not install a capable proxy for these schemes.

* specify localhost acl and allow it through the proxy. used in check_curl tests

* typo in comment

* move function comments to header

* fix failing tests

* handle case where proxy is set as empty string

* removed duplicate tests, corrected wrong comments

* corrected some annotations

* move docker apache subdomain setup files to /tools/subdomain1

* add a newline before dying in handle_curl_option_return_code

* fix the -ssl better, now does not segfault on empty --ssl argument as well.


---------

Co-authored-by: Ahmet Oeztuerk <Ahmet.Oeztuerk@consol.de>

1 files changed, 72 insertions, 19 deletions

diff --git a/plugins/check_curl.c b/plugins/check_curl.c
index d7d68de5..4a1fb647 100644
--- a/plugins/check_curl.c
+++ b/plugins/check_curl.c
@@ -874,7 +874,8 @@ check_curl_config_wrapper process_arguments(int argc, char **argv) {
                 COOKIE_JAR,
                 HAPROXY_PROTOCOL,
                 STATE_REGEX,
-                OUTPUT_FORMAT
+                OUTPUT_FORMAT,
+                NO_PROXY,
         };
 
         static struct option longopts[] = {
@@ -889,6 +890,8 @@ check_curl_config_wrapper process_arguments(int argc, char **argv) {
                 {"url", required_argument, 0, 'u'},
                 {"port", required_argument, 0, 'p'},
                 {"authorization", required_argument, 0, 'a'},
+                {"proxy", required_argument, 0, 'x'},
+                {"noproxy", required_argument, 0, NO_PROXY},
                 {"proxy-authorization", required_argument, 0, 'b'},
                 {"header-string", required_argument, 0, 'd'},
                 {"string", required_argument, 0, 's'},
@@ -961,7 +964,7 @@ check_curl_config_wrapper process_arguments(int argc, char **argv) {
 
         while (true) {
                 int option_index = getopt_long(
-                        argc, argv, "Vvh46t:c:w:A:k:H:P:j:T:I:a:b:d:e:p:s:R:r:u:f:C:J:K:DnlLS::m:M:NEB",
+                        argc, argv, "Vvh46t:c:w:A:k:H:P:j:T:I:a:x:b:d:e:p:s:R:r:u:f:C:J:K:DnlLS::m:M:NEB",
                         longopts, &option);
                 if (option_index == -1 || option_index == EOF || option_index == 1) {
                         break;
@@ -1049,6 +1052,10 @@ check_curl_config_wrapper process_arguments(int argc, char **argv) {
                         strncpy(result.config.curl_config.user_auth, optarg, MAX_INPUT_BUFFER - 1);
                         result.config.curl_config.user_auth[MAX_INPUT_BUFFER - 1] = 0;
                         break;
+                case 'x': /* proxy info */
+                        strncpy(result.config.curl_config.proxy, optarg, DEFAULT_BUFFER_SIZE - 1);
+                        result.config.curl_config.proxy[DEFAULT_BUFFER_SIZE - 1] = 0;
+                        break;
                 case 'b': /* proxy-authorization info */
                         strncpy(result.config.curl_config.proxy_auth, optarg, MAX_INPUT_BUFFER - 1);
                         result.config.curl_config.proxy_auth[MAX_INPUT_BUFFER - 1] = 0;
@@ -1344,6 +1351,10 @@ check_curl_config_wrapper process_arguments(int argc, char **argv) {
                 case HAPROXY_PROTOCOL:
                         result.config.curl_config.haproxy_protocol = true;
                         break;
+                case NO_PROXY:
+                        strncpy(result.config.curl_config.no_proxy, optarg, DEFAULT_BUFFER_SIZE - 1);
+                        result.config.curl_config.no_proxy[DEFAULT_BUFFER_SIZE - 1] = 0;
+                        break;
                 case '?':
                         /* print short usage statement if args not parsable */
                         usage5();
@@ -1371,35 +1382,35 @@ check_curl_config_wrapper process_arguments(int argc, char **argv) {
                  * parameters, like -S and -C combinations */
                 result.config.curl_config.ssl_version = CURL_SSLVERSION_DEFAULT;
                 if (tls_option_optarg != NULL) {
-                        char *plus_ptr = strchr(optarg, '+');
+                        char *plus_ptr = strchr(tls_option_optarg, '+');
                         if (plus_ptr) {
                                 got_plus = true;
                                 *plus_ptr = '\0';
                         }
 
-                        if (optarg[0] == '2') {
+                        if (tls_option_optarg[0] == '2') {
                                 result.config.curl_config.ssl_version = CURL_SSLVERSION_SSLv2;
-                        } else if (optarg[0] == '3') {
+                        } else if (tls_option_optarg[0] == '3') {
                                 result.config.curl_config.ssl_version = CURL_SSLVERSION_SSLv3;
-                        } else if (!strcmp(optarg, "1") || !strcmp(optarg, "1.0")) {
+                        } else if (!strcmp(tls_option_optarg, "1") || !strcmp(tls_option_optarg, "1.0")) {
 #if LIBCURL_VERSION_NUM >= MAKE_LIBCURL_VERSION(7, 34, 0)
                                 result.config.curl_config.ssl_version = CURL_SSLVERSION_TLSv1_0;
 #else
                                 result.config.ssl_version = CURL_SSLVERSION_DEFAULT;
 #endif /* LIBCURL_VERSION_NUM >= MAKE_LIBCURL_VERSION(7, 34, 0) */
-                        } else if (!strcmp(optarg, "1.1")) {
+                        } else if (!strcmp(tls_option_optarg, "1.1")) {
 #if LIBCURL_VERSION_NUM >= MAKE_LIBCURL_VERSION(7, 34, 0)
                                 result.config.curl_config.ssl_version = CURL_SSLVERSION_TLSv1_1;
 #else
                                 result.config.ssl_version = CURL_SSLVERSION_DEFAULT;
 #endif /* LIBCURL_VERSION_NUM >= MAKE_LIBCURL_VERSION(7, 34, 0) */
-                        } else if (!strcmp(optarg, "1.2")) {
+                        } else if (!strcmp(tls_option_optarg, "1.2")) {
 #if LIBCURL_VERSION_NUM >= MAKE_LIBCURL_VERSION(7, 34, 0)
                                 result.config.curl_config.ssl_version = CURL_SSLVERSION_TLSv1_2;
 #else
                                 result.config.ssl_version = CURL_SSLVERSION_DEFAULT;
 #endif /* LIBCURL_VERSION_NUM >= MAKE_LIBCURL_VERSION(7, 34, 0) */
-                        } else if (!strcmp(optarg, "1.3")) {
+                        } else if (!strcmp(tls_option_optarg, "1.3")) {
 #if LIBCURL_VERSION_NUM >= MAKE_LIBCURL_VERSION(7, 52, 0)
                                 result.config.curl_config.ssl_version = CURL_SSLVERSION_TLSv1_3;
 #else
@@ -1522,8 +1533,8 @@ void print_help(void) {
         printf(" %s\n", "-I, --IP-address=ADDRESS");
         printf("    %s\n",
                    "IP address or name (use numeric address if possible to bypass DNS lookup).");
-        printf("    %s\n", "This overwrites the network address of the target while leaving everything "
-                                           "else (HTTP headers) as they are");
+        printf("    %s\n",
+                     "This overwrites the network address of the target while leaving everything else (HTTP headers) as they are");
         printf(" %s\n", "-p, --port=INTEGER");
         printf("    %s", _("Port number (default: "));
         printf("%d)\n", HTTP_PORT);
@@ -1587,8 +1598,7 @@ void print_help(void) {
         printf("    %s\n", _("String to expect in the content"));
         printf(" %s\n", "-u, --url=PATH");
         printf("    %s\n", _("URL to GET or POST (default: /)"));
-        printf("    %s\n", _("This is the part after the address in a URL, so for "
-                                                 "\"https://example.com/index.html\" it would be '-u /index.html'"));
+        printf("    %s\n", _("This is the part after the address in a URL, so for \"https://example.com/index.html\" it would be '-u /index.html'"));
         printf(" %s\n", "-P, --post=STRING");
         printf("    %s\n", _("URL decoded http POST data"));
         printf(" %s\n",
@@ -1614,6 +1624,18 @@ void print_help(void) {
         printf(" %s\n", "--state-regex=STATE");
         printf("    %s\n", _("Return STATE if regex is found, OK if not. STATE can be one of "
                                                  "\"critical\",\"warning\""));
+        printf(" %s\n", "-x, --proxy=PROXY_SERVER");
+        printf("    %s\n", _("Specify the proxy in form of <scheme>://<host(name)>:<port>"));
+        printf("    %s\n", _("Available schemes are http, https, socks4, socks4a, socks5, socks5h"));
+        printf("    %s\n", _("If port is not specified, libcurl defaults to 1080"));
+        printf("    %s\n", _("This value will be set as CURLOPT_PROXY"));
+        printf(" %s\n", "--noproxy=COMMA_SEPARATED_LIST");
+        printf("    %s\n", _("Specify hostnames, addresses and subnets where proxy should not be used"));
+        printf("    %s\n", _("Example usage: \"example.com,::1,1.1.1.1,localhost,192.168.0.0/16\""));
+        printf("    %s\n", _("Do not use brackets when specifying IPv6 addresses"));
+        printf("    %s\n", _("Special case when an item is '*' : matches all hosts/addresses "
+                "and effectively disables proxy."));
+        printf("    %s\n", _("This value will be set as CURLOPT_NOPROXY"));
         printf(" %s\n", "-a, --authorization=AUTH_PAIR");
         printf("    %s\n", _("Username:password on sites with basic authentication"));
         printf(" %s\n", "-b, --proxy-authorization=AUTH_PAIR");
@@ -1722,10 +1744,39 @@ void print_help(void) {
 #endif
 
         printf("\n %s\n", "CHECK WEBSERVER CONTENT VIA PROXY:");
-        printf(" %s\n", _("It is recommended to use an environment proxy like:"));
-        printf(" %s\n",
-                   _("http_proxy=http://192.168.100.35:3128 ./check_curl -H www.monitoring-plugins.org"));
-        printf(" %s\n", _("legacy proxy requests in check_http style still work:"));
+        printf(" %s\n", _("Proxies are specified or disabled for certain hosts/addresses using environment variables"
+                " or -x/--proxy and --noproxy arguments:"));
+        printf(" %s\n", _("Checked environment variables: all_proxy, http_proxy, https_proxy, no_proxy"));
+        printf(" %s\n", _("Environment variables can also be given in uppercase, but the lowercase ones will "
+                                          "take predence if both are defined."));
+        printf(" %s\n", _("The environment variables are overwritten by -x/--proxy and --noproxy arguments:"));
+        printf(" %s\n", _("all_proxy/ALL_PROXY environment variables are read first, but protocol "
+                                        "specific environment variables override them."));
+        printf(" %s\n", _("If SSL is enabled and used, https_proxy/HTTPS_PROXY will be checked and overwrite "
+                        "http_proxy/HTTPS_PROXY."));
+        printf(" %s\n", _("Curl accepts proxies using http, https, socks4, socks4a, socks5 and socks5h schemes."));
+        printf(" %s\n", _("http_proxy=http://192.168.100.35:3128 ./check_curl -H www.monitoring-plugins.org"));
+        printf(" %s\n", _("http_proxy=http://used.proxy.com HTTP_PROXY=http://ignored.proxy.com ./check_curl -H www.monitoring-plugins.org"));
+        printf(" %s\n", _("  Lowercase http_proxy takes predence over uppercase HTTP_PROXY"));
+        printf(" %s\n", _("./check_curl -H www.monitoring-plugins.org -x http://192.168.100.35:3128"));
+        printf(" %s\n", _("http_proxy=http://unused.proxy1.com HTTP_PROXY=http://unused.proxy2.com ./check_curl "
+                        "-H www.monitoring-plugins.org --proxy http://used.proxy"));
+        printf(" %s\n", _("  Proxy specified by --proxy overrides any proxy specified by environment variable."));
+        printf(" %s\n", _("  Curl uses port 1080 by default as port is not specified"));
+        printf(" %s\n", _("HTTPS_PROXY=http://192.168.100.35:3128 ./check_curl -H www.monitoring-plugins.org --ssl"));
+        printf(" %s\n", _("  HTTPS_PROXY is read as --ssl is toggled"));
+        printf(" %s\n", _("./check_curl -H www.monitoring-plugins.org --proxy socks5h://192.168.122.21"));
+        printf(" %s\n", _("./check_curl -H www.monitoring-plugins.org -x http://unused.proxy.com --noproxy '*'"));
+        printf(" %s\n", _("  Disabled proxy for all hosts by using '*' in no_proxy ."));
+        printf(" %s\n", _("NO_PROXY=www.monitoring-plugins.org ./check_curl -H www.monitoring-plugins.org -x http://unused.proxy.com"));
+        printf(" %s\n", _("  Exact matches with the hostname/address work."));
+        printf(" %s\n", _("no_proxy=192.168.178.0/24 ./check_curl -I 192.168.178.10 -x http://proxy.acme.org"));
+        printf(" %s\n", _("no_proxy=acme.org ./check_curl -H nonpublic.internalwebapp.acme.org -x http://proxy.acme.org"));
+        printf(" %s\n", _("  Do not use proxy when accessing internal domains/addresses, but use a default proxy when accessing public web."));
+        printf(" %s\n", _("  IMPORTANT: Check_curl can not always determine whether itself or the proxy will "
+                "resolve a hostname before sending a request and getting an answer."
+                "This can lead to DNS resolvation issues if hostname is only resolvable over proxy."));
+        printf(" %s\n", _("Legacy proxy requests in check_http style still work:"));
         printf(" %s\n", _("check_curl -I 192.168.100.35 -p 3128 -u http://www.monitoring-plugins.org/ "
                                           "-H www.monitoring-plugins.org"));
 
@@ -1756,13 +1807,15 @@ void print_usage(void) {
         printf(" %s -H <vhost> | -I <IP-address> [-u <uri>] [-p <port>]\n", progname);
         printf("       [-J <client certificate file>] [-K <private key>] [--ca-cert <CA certificate "
                    "file>] [-D]\n");
-        printf("       [-w <warn time>] [-c <critical time>] [-t <timeout>] [-L] [-E] [-a auth]\n");
-        printf("       [-b proxy_auth] [-f <ok|warning|critical|follow|sticky|stickyport|curl>]\n");
+        printf("       [-w <warn time>] [-c <critical time>] [-t <timeout>] [-L] [-E] [-x <proxy>]\n");
+        printf("       [-a auth] [-b proxy_auth] [-f "
+                   "<ok|warning|critical|follow|sticky|stickyport|curl>]\n");
         printf("       [-e <expect>] [-d string] [-s string] [-l] [-r <regex> | -R <case-insensitive "
                    "regex>]\n");
         printf("       [-P string] [-m <min_pg_size>:<max_pg_size>] [-4|-6] [-N] [-M <age>]\n");
         printf("       [-A string] [-k string] [-S <version>] [--sni] [--haproxy-protocol]\n");
         printf("       [-T <content-type>] [-j method]\n");
+        printf("       [--noproxy=<comma separated list of hosts, IP addresses, IP CIDR subnets>\n");
         printf("       [--http-version=<version>] [--enable-automatic-decompression]\n");
         printf("       [--cookie-jar=<cookie jar file>\n");
         printf(" %s -H <vhost> | -I <IP-address> -C <warn_age>[,<crit_age>]\n", progname);