File Transferring In Linux system - LinuxHunt
193
 

File Transferring In Linux system

File Transferring In Linux system

In Linux operating system we can transfer or copy the files by two ways . If you are using GNOME (GNOME :- it is graphically user interface , in this user can see the desktop as like a windows desktop , that contain some icons) as show in picture

And the second way is by using terminal. In terminal we can transfer files by using commands, Terminal is faster then the GNOME . The terminal is a command interface.

and

So command that are used for

files transferring form one place to another, we use variety of commands.

1. If we transferring files within the system we use command like :-
“ cp “ +path+filename /+path of destination

Example:-
Copy the tsp.text file from path /home/tomcat/webapps/tsp.text to home

cp /home/tomcat/webapps/tsp.text /home

cp –R /home/tomcat/webapps/tsp.text /home

and we can also use other option with cp command
these can see by using command “man cp “ the output is

[root]

#man cp
By default, sparse SOURCE files are detected by a crude heuristic and the corresponding DEST file is made sparse as well. That is the behavior selected by

–sparse=auto. Specify –sparse=always to create a sparse DEST file whenever the SOURCE file contains a long enough sequence of zero bytes. Use
–sparse=never to inhibit creation of sparse files.

When –reflink[=always] is specified, perform a lightweight copy, where the data blocks are copied only when modified. If this is not possible the copy
fails, or if –reflink=auto is specified, fall back to a standard copy.

The backup suffix is ‘~’, unless set with –suffix or SIMPLE_BACKUP_SUFFIX. The version control method may be selected via the –backup option or through

the VERSION_CONTROL environment variable. Here are the values:

none, off
never make backups (even if –backup is given)

numbered, t
make numbered backups

existing, nil
numbered if numbered backups exist, simple otherwise

simple, never
always make simple backups

As a special case, cp makes a backup of SOURCE when the force and backup options are given and SOURCE and DEST are the same name for an existing, regular

file.

GNU coreutils online help: <http://www.gnu.org/software/coreutils/> Report cp translation bugs to <http://translationproject.org/team/>

AUTHOR
Written by Torbjorn Granlund, David MacKenzie, and Jim Meyering.

COPYRIGHT
Copyright © 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.

SEE ALSO
The full documentation for cp is maintained as a Texinfo manual. If the info and cp programs are properly installed at your site, the command

info coreutils ‘cp invocation’

should give you access to the complete manual.

2. If we transferring files from one linux server to another we use command with the ip address and server username with server ssh port no.
At this end we can use two types of commands one is “SCP” and other is “rsync” scp commands is mostly using command . When we have to transfer larger amount of data then we use “rsync” command.

Secure copy protocol (SCP) is a means of securely transferring computer files between a local host and a remote host or between two remote hosts. It is based on the Secure Shell (SSH) protocol. “SCP” commonly refers to both the Secure Copy Protocol and the program itself.

rsync is a widely-used utility to keep copies of a file on two computer systems the same. It is commonly found on Unix-like systems and functions as both a file synchronization and file transfer program. The rsyncalgorithm, a type of delta encoding, is used to minimize network usage.

Sytanx:-
scp –r –P(ssh port no.) username@server ip :/ path of file /path of file to copy
scp -r –P22 root@127.0.0.1:/usr/dbbackup /home/backup_all/

(Note :- This command is write on main server there you want to copy the file , And also check firewall sometime firewall block this command )

[root]

#man scp

NAME
scp — secure copy (remote file copy program)

SYNOPSIS
scp [-12346BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file] [-l limit] [-o ssh_option] [-P port] [-S program] [[user@]host1:]file1 …
[[user@]host2:]file2

DESCRIPTION
scp copies files between hosts on a network. It uses ssh(1) for data transfer, and uses the same authentication and provides the same security as ssh(1).
scp will ask for passwords or passphrases if they are needed for authentication.

File names may contain a user and host specification to indicate that the file is to be copied to/from that host. Local file names can be made explicit using
absolute or relative pathnames to avoid scp treating file names containing ‘:’ as host specifiers. Copies between two remote hosts are also permitted.

The options are as follows:

-1 Forces scp to use protocol 1.

-2 Forces scp to use protocol 2.

-3 Copies between two remote hosts are transferred through the local host. Without this option the data is copied directly between the two remote hosts.

Note that this option disables the progress meter.

-4 Forces scp to use IPv4 addresses only.

-6 Forces scp to use IPv6 addresses only.

-B Selects batch mode (prevents asking for passwords or passphrases).

-C Compression enable. Passes the -C flag to ssh(1) to enable compression.

-c cipher
Selects the cipher to use for encrypting the data transfer. This option is directly passed to ssh(1).

-F ssh_config
Specifies an alternative per-user configuration file for ssh. This option is directly passed to ssh(1).

-i identity_file
Selects the file from which the identity (private key) for public key authentication is read. This option is directly passed to ssh(1).
-l limit
Limits the used bandwidth, specified in Kbit/s.
-P port
Specifies the port to connect to on the remote host. Note that this option is written with a capital ‘P’, because -p is already reserved for preserv‐
ing the times and modes of the file.

-p Preserves modification times, access times, and modes from the original file.

-q Quiet mode: disables the progress meter as well as warning and diagnostic messages from ssh(1).

-r Recursively copy entire directories. Note that scp follows symbolic links encountered in the tree traversal.

-S program
Name of program to use for the encrypted connection. The program must understand ssh(1) options.

-v Verbose mode. Causes scp and ssh(1) to print debugging messages about their progress. This is helpful in debugging connection, authentication, and
configuration problems.

For rsync :-

rsync -hrtplu –progress -e “ssh -p 22” username@server ip :/ path of file /path of file to copy

rsync -hrtplu –progress -e “ssh -p 22” root@172.0.0.1:/ home/backup/dbbackup.tar.gz /home/backup_172/

[root]

#man rsync

NAME
rsync – a fast, versatile, remote (and local) file-copying tool

SYNOPSIS
Local: rsync [OPTION…] SRC… [DEST]

Access via remote shell:
Pull: rsync [OPTION…] [USER@]HOST:SRC… [DEST]
Push: rsync [OPTION…] SRC… [USER@]HOST:DEST

Access via rsync daemon:
Pull: rsync [OPTION…] [USER@]HOST::SRC… [DEST]
rsync [OPTION…] rsync://[USER@]HOST[:PORT]/SRC… [DEST]
Push: rsync [OPTION…] SRC… [USER@]HOST::DEST
rsync [OPTION…] SRC… rsync://[USER@]HOST[:PORT]/DEST

Usages with just one SRC arg and no DEST arg will list the source files instead of copying.

DESCRIPTION
Rsync is a fast and extraordinarily versatile file copying tool. It can copy locally, to/from another host over any remote shell, or to/from a remote rsync
daemon. It offers a large number of options that control every aspect of its behavior and permit very flexible specification of the set of files to be
copied. It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the
source files and the existing files in the destination. Rsync is widely used for backups and mirroring and as an improved copy command for everyday use.

Rsync finds files that need to be transferred using a “quick check” algorithm (by default) that looks for files that have changed in size or in last-modi‐
fied time. Any changes in the other preserved attributes (as requested by options) are made on the destination file directly when the quick check indicates
that the file’s data does not need to be updated.

Some of the additional features of rsync are:

o support for copying links, devices, owners, groups, and permissions

o exclude and exclude-from options similar to GNU tar

o a CVS exclude mode for ignoring the same files that CVS would ignore

o can use any transparent remote shell, including ssh or rsh

o does not require super-user privileges
o pipelining of file transfers to minimize latency costs

o support for anonymous or authenticated rsync daemons (ideal for mirroring)

GENERAL
Rsync copies files either to or from a remote host, or locally on the current host (it does not support copying files between two remote hosts).

There are two different ways for rsync to contact a remote system: using a remote-shell program as the transport (such as ssh or rsh) or contacting an rsync
daemon directly via TCP. The remote-shell transport is used whenever the source or destination path contains a single colon (:) separator after a host
specification. Contacting an rsync daemon directly happens when the source or destination path contains a double colon (::) separator after a host specifi‐
cation, OR when an rsync:// URL is specified (see also the “USING RSYNC-DAEMON FEATURES VIA A REMOTE-SHELL CONNECTION” section for an exception to this lat‐
ter rule).

As a special case, if a single source arg is specified without a destination, the files are listed in an output format similar to “ls -l”.

As expected, if neither the source or destination path specify a remote host, the copy occurs locally (see also the –list-only option).

Rsync refers to the local side as the “client” and the remote side as the “server”. Don’t confuse “server” with an rsync daemon — a daemon is always a
server, but a server can be either a daemon or a remote-shell spawned process.

SETUP
See the file README for installation instructions.

Once installed, you can use rsync to any machine that you can access via a remote shell (as well as some that you can access using the rsync daemon-mode
protocol). For remote transfers, a modern rsync uses ssh for its communications, but it may have been configured to use a different remote shell by
default, such as rsh or remsh.

You can also specify any remote shell you like, either by using the -e command line option, or by setting the RSYNC_RSH environment variable.

Note that rsync must be installed on both the source and destination machines.

USAGE
You use rsync in the same way you use rcp. You must specify a source and a destination, one of which may be remote.

Perhaps the best way to explain the syntax is with some examples:

rsync -t *.c foo:src/
GENERAL
Rsync copies files either to or from a remote host, or locally on the current host (it does not support copying files between two remote hosts).

There are two different ways for rsync to contact a remote system: using a remote-shell program as the transport (such as ssh or rsh) or contacting an rsync
daemon directly via TCP. The remote-shell transport is used whenever the source or destination path contains a single colon (:) separator after a host
specification. Contacting an rsync daemon directly happens when the source or destination path contains a double colon (::) separator after a host specifi‐
cation, OR when an rsync:// URL is specified (see also the “USING RSYNC-DAEMON FEATURES VIA A REMOTE-SHELL CONNECTION” section for an exception to this lat‐
ter rule).

As a special case, if a single source arg is specified without a destination, the files are listed in an output format similar to “ls -l”.

As expected, if neither the source or destination path specify a remote host, the copy occurs locally (see also the –list-only option).

Rsync refers to the local side as the “client” and the remote side as the “server”. Don’t confuse “server” with an rsync daemon — a daemon is always a
server, but a server can be either a daemon or a remote-shell spawned process.

SETUP
See the file README for installation instructions.

Once installed, you can use rsync to any machine that you can access via a remote shell (as well as some that you can access using the rsync daemon-mode
protocol). For remote transfers, a modern rsync uses ssh for its communications, but it may have been configured to use a different remote shell by
default, such as rsh or remsh.

You can also specify any remote shell you like, either by using the -e command line option, or by setting the RSYNC_RSH environment variable.

Note that rsync must be installed on both the source and destination machines.

USAGE
You use rsync in the same way you use rcp. You must specify a source and a destination, one of which may be remote.

Perhaps the best way to explain the syntax is with some examples:

rsync -t *.c foo:src/

This would transfer all files matching the pattern *.c from the current directory to the directory src on the machine foo. If any of the files already exist
on the remote system then the rsync remote-update protocol is used to update the file by sending only the differences in the data. Note that the expansion
of wildcards on the commandline (*.c) into a list of files is handled by the shell before it runs rsync and not by rsync itself (exactly the same as all
other posix-style programs).

rsync -avz foo:src/bar /data/tmp

This would recursively transfer all files from the directory src/bar on the machine foo into the /data/tmp/bar directory on the local machine. The files are
transferred in “archive” mode, which ensures that symbolic links, devices, attributes, permissions, ownerships, etc. are preserved in the transfer. Addi‐
tionally, compression will be used to reduce the size of data portions of the transfer.

rsync -avz foo:src/bar/ /data/tmp

A trailing slash on the source changes this behavior to avoid creating an additional directory level at the destination. You can think of a trailing / on a
source as meaning “copy the contents of this directory” as opposed to “copy the directory by name”, but in both cases the attributes of the containing
directory are transferred to the containing directory on the destination. In other words, each of the following commands copies the files in the same way,
including their setting of the attributes of /dest/foo:

rsync -av /src/foo /dest
rsync -av /src/foo/ /dest/foo

Note also that host and module references don’t require a trailing slash to copy the contents of the default directory. For example, both of these copy the
remote directory’s contents into “/dest”:

rsync -av host: /dest
rsync -av host::module /dest

You can also use rsync in local-only mode, where both the source and destination don’t have a ’:’ in the name. In this case it behaves like an improved copy
command.

Finally, you can list all the (listable) modules available from a particular rsync daemon by leaving off the module name:

rsync somehost.mydomain.com::

See the following section for more details.

ADVANCED USAGE
The syntax for requesting multiple files from a remote host is done by specifying additional remote-host args in the same style as the first, or with the
hostname omitted. For instance, all these work:

rsync -av host:file1 :file2 host:file{3,4} /dest/
rsync -av host::modname/file{1,2} host::modname/file3 /dest/
rsync -av host::modname/file1 ::modname/file{3,4}

Older versions of rsync required using quoted spaces in the SRC, like these examples:

rsync -av host:’dir1/file1 dir2/file2′ /dest
You can also use rsync in local-only mode, where both the source and destination don’t have a ’:’ in the name. In this case it behaves like an improved copy
command.

Finally, you can list all the (listable) modules available from a particular rsync daemon by leaving off the module name:

rsync somehost.mydomain.com::

See the following section for more details.

ADVANCED USAGE
The syntax for requesting multiple files from a remote host is done by specifying additional remote-host args in the same style as the first, or with the
hostname omitted. For instance, all these work:

rsync -av host:file1 :file2 host:file{3,4} /dest/
rsync -av host::modname/file{1,2} host::modname/file3 /dest/
rsync -av host::modname/file1 ::modname/file{3,4}

Older versions of rsync required using quoted spaces in the SRC, like these examples:

rsync -av host:’dir1/file1 dir2/file2′ /dest
rsync host::’modname/dir1/file1 modname/dir2/file2′ /dest

This word-splitting still works (by default) in the latest rsync, but is not as easy to use as the first method.

If you need to transfer a filename that contains whitespace, you can either specify the –protect-args (-s) option, or you’ll need to escape the whitespace
in a way that the remote shell will understand. For instance:

rsync -av host:’file\ name\ with\ spaces’ /dest

CONNECTING TO AN RSYNC DAEMON
It is also possible to use rsync without a remote shell as the transport. In this case you will directly connect to a remote rsync daemon, typically using
TCP port 873. (This obviously requires the daemon to be running on the remote system, so refer to the STARTING AN RSYNC DAEMON TO ACCEPT CONNECTIONS sec‐
tion below for information on that.)

Using rsync in this way is the same as using it with a remote shell except that:

o you either use a double colon :: instead of a single colon to separate the hostname from the path, or you use an rsync:// URL.
o you either use a double colon :: instead of a single colon to separate the hostname from the path, or you use an rsync:// URL.

o the first word of the “path” is actually a module name.

o the remote daemon may print a message of the day when you connect.

o if you specify no path name on the remote daemon then the list of accessible paths on the daemon will be shown.

o if you specify no local destination then a listing of the specified files on the remote daemon is provided.

o you must not specify the –rsh (-e) option.

An example that copies all the files in a remote module named “src”:

rsync -av host::src /dest

Some modules on the remote daemon may require authentication. If so, you will receive a password prompt when you connect. You can avoid the password prompt
by setting the environment variable RSYNC_PASSWORD to the password you want to use or using the –password-file option. This may be useful when scripting
rsync.

WARNING: On some systems environment variables are visible to all users. On those systems using –password-file is recommended.

You may establish the connection via a web proxy by setting the environment variable RSYNC_PROXY to a hostname:port pair pointing to your web proxy. Note
that your web proxy’s configuration must support proxy connections to port 873.

You may also establish a daemon connection using a program as a proxy by setting the environment variable RSYNC_CONNECT_PROG to the commands you wish to run
in place of making a direct socket connection. The string may contain the escape “%H” to represent the hostname specified in the rsync command (so use “%%”
if you need a single “%” in your string). For example:

export RSYNC_CONNECT_PROG=’ssh proxyhost nc %H 873′
rsync -av targethost1::module/src/ /dest/
rsync -av rsync:://targethost2/module/src/ /dest/

The command specified above uses ssh to run nc (netcat) on a proxyhost, which forwards all data to port 873 (the rsync daemon) on the targethost (%H).

USING RSYNC-DAEMON FEATURES VIA A REMOTE-SHELL CONNECTION
It is sometimes useful to use various features of an rsync daemon (such as named modules) without actually allowing any new socket connections into a system
(other than what is already required to allow remote-shell access). Rsync supports connecting to a host using a remote shell and then spawning a single-use
“daemon” server that expects to read its config file in the home dir of the remote user. This can be useful if you want to encrypt a daemon-style trans‐
fer’s data, but since the daemon is started up fresh by the remote user, you may not be able to use features such as chroot or change the uid used by the
daemon. (For another way to encrypt a daemon transfer, consider using ssh to tunnel a local port to a remote machine and configure a normal rsync daemon on
EXAMPLES
Here are some examples of how I use rsync.

To backup my wife’s home directory, which consists of large MS Word files and mail folders, I use a cron job that runs

rsync -Cavz . arvidsjaur:backup

each night over a PPP connection to a duplicate directory on my machine “arvidsjaur”.

To synchronize my samba source trees I use the following Makefile targets:

get:
rsync -avuzb –exclude ‘*~’ samba:samba/ .
put:
rsync -Cavuzb . samba:samba/
sync: get put

this allows me to sync with a CVS directory at the other end of the connection. I then do CVS operations on the remote machine, which saves a lot of time as
the remote CVS protocol isn’t very efficient.

I mirror a directory between my “old” and “new” ftp sites with the command:

rsync -az -e ssh –delete ~ftp/pub/samba nimbus:”~ftp/pub/tridge”

This is launched from cron every few hours.
OPTIONS SUMMARY
Here is a short summary of the options available in rsync. Please refer to the detailed description below for a complete description.

-v, –verbose increase verbosity
–info=FLAGS fine-grained informational verbosity
–debug=FLAGS fine-grained debug verbosity
–msgs2stderr special output handling for debugging
-q, –quiet suppress non-error messages
–no-motd suppress daemon-mode MOTD (see caveat)
-c, –checksum skip based on checksum, not mod-time & size
-a, –archive archive mode; equals -rlptgoD (no -H,-A,-X)
–no-OPTION turn off an implied OPTION (e.g. –no-D)
-r, –recursive recurse into directories
-R, –relative use relative path names
–no-implied-dirs don’t send implied dirs with –relative
-b, –backup make backups (see –suffix & –backup-dir)
–backup-dir=DIR make backups into hierarchy based in DIR
–suffix=SUFFIX backup suffix (default ~ w/o –backup-dir)
-u, –update skip files that are newer on the receiver
–inplace update destination files in-place
–append append data onto shorter files
–append-verify –append w/old data in file checksum
-d, –dirs transfer directories without recursing
-l, –links copy symlinks as symlinks
-L, –copy-links transform symlink into referent file/dir
–copy-unsafe-links only “unsafe” symlinks are transformed
–safe-links ignore symlinks that point outside the tree
–munge-links munge symlinks to make them safer
-k, –copy-dirlinks transform symlink to dir into referent dir
-K, –keep-dirlinks treat symlinked dir on receiver as dir
-H, –hard-links preserve hard links
-p, –perms preserve permissions
-E, –executability preserve executability
–chmod=CHMOD affect file and/or directory permissions
-A, –acls preserve ACLs (implies -p)
-X, –xattrs preserve extended attributes
-o, –owner preserve owner (super-user only)
-g, –group preserve group
–devices preserve device files (super-user only)
–copy-devices copy device contents as regular file
–specials preserve special files
–contimeout=SECONDS set daemon connection timeout in seconds
-I, –ignore-times don’t skip files that match size and time
–size-only skip files that match in size
–modify-window=NUM compare mod-times with reduced accuracy
-T, –temp-dir=DIR create temporary files in directory DIR
-y, –fuzzy find similar file for basis if no dest file
–compare-dest=DIR also compare received files relative to DIR
–copy-dest=DIR … and include copies of unchanged files
–link-dest=DIR hardlink to files in DIR when unchanged
-z, –compress compress file data during the transfer
–compress-level=NUM explicitly set compression level
–skip-compress=LIST skip compressing files with suffix in LIST
-C, –cvs-exclude auto-ignore files in the same way CVS does
-f, –filter=RULE add a file-filtering RULE
-F same as –filter=’dir-merge /.rsync-filter’
repeated: –filter=’- .rsync-filter’
–exclude=PATTERN exclude files matching PATTERN
–exclude-from=FILE read exclude patterns from FILE
–include=PATTERN don’t exclude files matching PATTERN
–include-from=FILE read include patterns from FILE
–files-from=FILE read list of source-file names from FILE
-0, –from0 all *from/filter files are delimited by 0s
-s, –protect-args no space-splitting; wildcard chars only
–address=ADDRESS bind address for outgoing socket to daemon
–port=PORT specify double-colon alternate port number
–sockopts=OPTIONS specify custom TCP options
–blocking-io use blocking I/O for the remote shell
–outbuf=N|L|B set out buffering to None, Line, or Block
–stats give some file-transfer stats
-8, –8-bit-output leave high-bit chars unescaped in output
-h, –human-readable output numbers in a human-readable format
–progress show progress during transfer
-P same as –partial –progress
-i, –itemize-changes output a change-summary for all updates
-M, –remote-option=OPTION send OPTION to the remote side only
–out-format=FORMAT output updates using the specified FORMAT
–log-file=FILE log what we’re doing to the specified FILE
–log-file-format=FMT log updates using the specified FMT
–password-file=FILE read daemon-access password from FILE……………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

For every command you want to know type on terminal “ man + command “ & then enter you can see all the description of that command.

1 Comment

Post A Comment

Get notified automatically by email when there is new content!
For Questions or Comments, Contact us: editor @ linuxhunt.com

You have successfully subscribed to the newsletter

There was an error while trying to send your request. Please try again.

LinuxHunt will use the information you provide on this form to be in touch with you and to provide updates.