Using Curl to access web pages

Links: Security/Web , Linux/OpenSSL

Curl with selfsigned cert to ip

export CURL_IP="10.1.1.5"
export CURL_HOST="myhost.co.nz"
export CURL_CA_FILE="ca_my_selfsigned_cert.pem"

curl -i --cacert ${CURL_CA_FILE} --resolve ${CURL_HOST}:443:${CURL_IP} "https://${CURL_HOST}/"

Set headers

Check Headers e.g. CORS (access-control-allow-origin:) headers

$ curl -k -sD - -H "Origin: https://wiki.vigor.nz" -o /dev/null https://wiki.vigor.nz/

curl -H "Origin: https://mydomain.nz" -H "Access-Control-Request-Method: GET" -H "Access-Control-Request-Headers: X-Requested-With"   -X OPTIONS --verbose https://vigor.nz 2>&1 | grep -i "access\|cors"

Use Curl to measure web page response

From: https://stackoverflow.com/questions/18215389/how-do-i-measure-request-and-response-times-at-once-using-curl

Create a new file, curl-format.txt, and paste in:

     time_namelookup:  %{time_namelookup}s\n
        time_connect:  %{time_connect}s\n
     time_appconnect:  %{time_appconnect}s\n
    time_pretransfer:  %{time_pretransfer}s\n
       time_redirect:  %{time_redirect}s\n
  time_starttransfer:  %{time_starttransfer}s\n
                     ----------\n
          time_total:  %{time_total}s\n

Make a request:

 curl -w "@curl-format.txt" -o /dev/null -s "http://wordpress.com/"

Or on Windows, it's...

 curl -w "@curl-format.txt" -o NUL -s "http://wordpress.com/"

All in one, using ?[1-10] for repeat and add --parallel

curl -w "%{time_namelookup}s\n time_connect:  %{time_connect}s\n time_appconnect:  %{time_appconnect}s\n time_pretransfer:  %{time_pretransfer}s\n time_redirect:  %{time_redirect}s\n time_starttransfer:  %{time_starttransfer}s\n" -o /dev/null -s "https://wiki.vigor.nz/?[1-10]" --parallel

Curl with http proxy

curl -x myproxy.com:8080 https://reqbin.com/echo

Curl and set useragent

curl -i --user-agent "Googlebot" https://www.google.com/

Curl with host header/SNI

 curl -i -H "Host: test-public.example.com" https://test.example.com/HealthCheck

CategoryLinux