Using scp to copy files and folders

SCP (secure copy command) allows secure transferring of files between two hosts( does not matter if its local to local, local to remote, remote to local or remote to remote).

SCP uses the same authentication and security as the SSH protocol on which it is based. Since SCP relies on SSH (Secure Shell) to transfer files, all you need is the username and password for the source and target systems. 

LOCAL TO REMOTE

Copy a single file from the local machine to a remote machine:

The scp command needs a source and destination to copy files from one location to another location.

scp localmachine/path_to_the_file username@server_ip:/path_to_remote_directory

It will ask you to provide the password for that user, and then copy the file securely. 

Copy a local directory to a remote server:

If you want to copy the entire local directory to the server, then you use the -r (recursive) flag

scp -r localmachine/path_to_the_directory username@server_ip:/path_to_remote_directory/

Make sure that the source directory doesn’t have a forward slash at the end of the path, at the same time the destination path *must* have a forward slash.

Copy all files in a local directory to a remote directory

If you want to copy only the files inside a local directory to a remote directory, just add a forward slash(/) and * at the end of the source directory and give the path of destination directory. You will use the -r flag

scp -r localmachine/path_to_the_directory/* username@server_ip:/path_to_remote_directory/

 

REMOTE TO LOCAL

If you want to copy a single file, a directory or all files on a remote server to your local machine,the syntax's are the same as those shown above

Copy a single file:

scp username@server_ip:/path_to_remote_directory local_machine/path_to_the_file 

Copy a remote directory to a local machine:

scp -r username@server_ip:/path_to_remote_directory local-machine/path_to_the_directory/

Copy all files in a remote directory to a local directory:

scp -r username@server_ip:/path_to_remote_directory/* local-machine/path_to_the_directory/ 

 

REMOTE TO ANOTHER REMOTE

Copy a single file:

scp username@server1_ip:/path_to_the_remote_file username@server2_ip:/
  path_to_destination_directory/

Copy a directory from one location on a remote server to different location on the same server:

scp username@server1_ip:/path_to_the_remote_file username@server2_ip:/
  path_to_destination_directory/

Copy all files in a remote directory to a local directory

scp -r username@server1_ip:/path_to_source_directory/* username@server2_ip:/
  path_to_the_destination_directory/ 

REMOTE TO THE SAME REMOTE SERVER

Copy a single file:

scp username@server_ip:/path_to_the_remote_file username@server_ip:/
  path_to_destination_directory/

Copy a directory from one location on remote server to different location on the same server:

scp username@server_ip:/path_to_the_remote_file username@server_ip:/
  path_to_destination_directory/

Copy all files in a remote directory to a local directory

scp -r username@server_ip:/path_to_source_directory/* usdername@server_ip:/
  path_to_the_destination_directory/ 

Using curl to fetch from a URL which outputs a dynamically generated Excel file

We often come across this case where another company provides us with a url which we can  use via a browser to get an excel or pdf report. 

Usually it also accepts one or more parameters, which are used by the backend script in the report generation process.

How does one call this via CURL to allow our application to fetch the reports automatically  based on some pre defined schedule, or in a bulk fashion for a set of running parameters.

Its possible and to do it  one can use the delimiters -o and -j

Lets say our url looks something like this:

http://sample.example.com/xyz?type=xlsx&o=1&rt=1&id=11887&user=1

 Then we can use the call below to achieve this.

curl -o -j "http://sample.example.com/xyz?type=xlsx&o=1&rt=1&id=11887&user=1"