Curl download array of files

This makes it ideal for automation if you trust the source. Today we'll be walking you through the basics of using curl to download a file. Our sample file will be the readme for blcli, BitLaunch's command-line interface , which is hosted on GitHub. In the case of our readme, the complete command would like this:. So what if we want to use cURL to save a file to our server?

For that, we must use the -O option:. Sign Up. DigitalOcean home. Community Control Panel. Hacktoberfest Contribute to Open Source. Launch an Interactive Terminal! About the authors. Brian Hogan. Still looking for an answer? If the target server responded with the HTTP redirection code 3xx for the requested file, the local file you downloaded would be empty. In this case, you have to add the -L or --location option to tell cURL to follow the redirects.

In this example, after 10 attempts, cURL will throw an error and abort the download process. The number of bytes can be abbreviated by appending a suffix: kilobytes k or K , megabytes m or M , and gigabytes g or G. Do not! Show 10 more comments. A very simple solution would be the following: If you have a file 'file.

So if you have control over your input-file-format, maybe this is the simplest solution for you! Dirk Dirk 1 1 gold badge 10 10 silver badges 13 13 bronze badges. Will this use HTTP keep-alive? FullDecent It reuses the connection this way — Allan Deamon. Or you could just do this: cat urls. SamB 8, 5 5 gold badges 44 44 silver badges 53 53 bronze badges. Probably downvoted because it's wrong.

The -o option for curl specifies an output file as its argument. Other answers recommend -O , which tells curl to determine the local name based on the remote name of the file. Just don't set -P too high or your RAM may be overwhelmed. GNU parallel can achieve similar results. The downside of those methods is that they don't use a single connection for all files, which what curl does if you pass multiple URLs to it at once as in: curl -O out1.

Community Bot 1 1 1 silver badge. Stefan Gruenwald Stefan Gruenwald 2, 23 23 silver badges 26 26 bronze badges. You should quote your variable references. What if someone planted a file with a special character in your text file?

This is a horrible solution; it not only spawns a separate process for each download, but it also has to re-establish the TCP connection every single time, wasting a lot of time on even medium-latency networks. As others have rightly mentioned: -cat urls.