How Make HTTP Requests With Curl Command In Bash

When working in a command-line environment, if you need to interact with web services and APIs. One powerful tool for making HTTP requests in Bash is the curl command. curl is a widely-used command-line tool for transferring data with URLs.

In this article, we’ll explore how to use curl to make HTTP requests and perform various tasks like GET, POST, PUT, DELETE and more.

What is Curl Command?

Curl stands for Client URL, and it was created by Daniel Stenberg. It is an open-source software project with a rich history dating back to the late 1990s. Over the years, curl has become a fundamental tool for developers, system administrators, and anyone who needs to work with web resources programmatically.

At its core, curl is designed to make HTTP requests. It supports various HTTP methods, including GET, POST, PUT, DELETE, and more. Whether you need to retrieve data from a website, send data to a remote server, or perform other HTTP-related actions, curl provides the necessary functionality.

One of the key advantages of curl is its cross-platform compatibility. It is available for most major operating systems, including Linux, macOS, and Windows. Additionally, it can handle various protocols beyond HTTP, such as FTP, SCP, LDAP, and more.

Prerequisites

Before we dive into using curl, ensure that you have it installed on your system. Most Linux distributions come with curl pre-installed. You can check if it’s installed by running the following command:

curl --version

If you don’t have curl installed, you can easily install it using your package manager. For example, on Debian-based systems like Ubuntu, you can use apt:

sudo apt-get update
sudo apt-get install curl

On Red Hat-based systems, you can use yum:

sudo yum install curl

Basic Usage

The basic syntax of the curl command is as follows:

curl [options] [URL]

Here, [options] are various command-line flags that modify the behavior of the curl command, and [URL] is the Uniform Resource Locator, the address of the resource you want to interact with.

Some commonly used options are:

-H or --header: Used to specify headers in the request.
-X or --request: Used to specify the HTTP method (GET, POST, PUT, etc.).
-d or --data: Used to send data in the request body (used with POST or PUT requests).
-o or --output: Used to specify the output file for saving the response.
-i or --include: Used to include the response headers in the output.
-s or --silent: Used to silence the progress meter and error messages.

Making GET Requests

The simplest and most common use case for curl is to make GET requests to retrieve data from a URL. To make a GET request, use the following syntax:

curl https://www.example.com

This will make a GET request to https://www.example.com and display the HTML content of the specified URL in your terminal.

Saving the Output to a File

You can also save the output of a curl request to a file using the -o option. For example, to save the content of a website to a file named output.html:

curl -o output.html https://www.example.com

Now, the content of the website will be saved in the output.html file in your current directory.

Making POST Requests

To send data to a server using a POST request, you can use the -d option followed by the data you want to send. For example, to send a JSON payload:

curl -X POST -d '{"key": "value"}' https://example.com/api

In this example, we use the -X option to specify the HTTP method as POST and -d to provide the JSON data.

Adding Headers

You can include custom headers in your HTTP requests using the -H option. For instance, to set the Authorization header:

curl -H "Authorization: Bearer YOUR_TOKEN" https://example.com/api/resource

Replace YOUR_TOKEN with your actual authentication token.

Handling HTTPS and SSL

By default, curl supports secure HTTPS connections. You can use the -k option to bypass SSL certificate validation if needed, but this is generally not recommended as it exposes you to potential security risks.

curl -k https://www.example.com

Following Redirects

If a web page returns an HTTP redirect response (e.g., a 301 or 302 status code), curl will not follow the redirect by default. To make curl follow redirects, use the -L option:

curl -L https://example.com/redirecting-page

This is useful when you want to retrieve the content of the final destination after following the redirects.

Customizing User Agents

Some websites may block or restrict access to requests coming from certain user agents. You can set a custom user agent using the -A option to mimic a web browser or other client:

curl -A "Mozilla/5.0" https://example.com

In this example, we set the user agent to mimic Mozilla Firefox 5.0.

Timeout and Retry

You can specify a timeout for your curl requests using the --max-time option, which is useful for preventing long-running requests from hanging indefinitely:

curl --max-time 10 https://example.com

In this example, curl will timeout if it doesn’t receive a response within 10 seconds.

To retry a request in case of failure, you can use the --retry option:

curl --retry 3 https://example.com

This will retry the request up to 3 times in case of a failure.

Using Proxy Servers

If you need to route your requests through a proxy server, you can use the -x option:

curl -x http://proxy.example.com:8080 https://example.com

Replace http://proxy.example.com:8080 with the address and port of your proxy server.

Uploading Files

curl can also be used to upload files to a server using the -F option:

curl -F "file=@/path/to/file.txt" https://example.com/upload

This is useful for interacting with APIs that accept file uploads.

Sending Form Data

To send form data in a POST request, you can use the --data-urlencode option:

curl -X POST --data-urlencode "name=John Doe" --data-urlencode "email=johndoe@example.com" https://example.com/submit

This will encode the form data and send it in the request body.

Verbose Mode

For debugging purposes, you can use the -v or --verbose option to get detailed information about the request and response:

curl -v https://example.com

This can help you troubleshoot issues with your requests.

Conclusion

The curl command is a powerful tool for making HTTP requests in the Bash shell. With its wide range of options and flexibility, you can interact with web services, APIs, and websites directly from your terminal.

Whether you’re fetching data, posting content, or customizing headers, curl provides a versatile solution for handling HTTP requests in a command-line environment.

In this article, we’ve covered various aspects of using curl to make HTTP requests, including GET and POST requests, custom headers, handling HTTPS, following redirects, and more.

If you want to discover more about Making HTTP Requests With Client, then you can check the following articles:

Thank you for reading