Examples how to use fast data transfer application

1. To send one file called "local.data" from the local system directory to another computer in the "/home/remoteuser/destiantionDir" folder, with default parameters, there are two options:

   • Client/Server mode

     First, the FDT server needs to be started on the remote system. ( The default settings will be used, which implies the default ports,control 54321 and transfer 43210 on both the client and the server ). -S is used to disable the standalone mode, which means that the server will stop after the session will finish

     [remote computer]$ java -jar fdt.jar -S
     

     Then, the client will be started on the local system specifying the source file, the remote address (or hostname) where the server was started in the previous step and the destination directory

     [local computer]$ java -jar fdt.jar -c <remote_address> -d /home/remoteuser/destinationDir /home/localuser/local.data
     

     OR

     [local computer]$ java -jar fdt.jar -c <remote_address> -d destinationDir ./local.data
     

   • Secure Copy (SCP) mode

     In this mode the server will be started on the remote system automatically by the local FDT client using SSH.

     [local computer]$ java -jar fdt.jar /home/localuser/local.data 
     remoteuser@<remote_address>:/home/remoteuser/destinationDir
     

     OR

     [local computer]$ java -jar fdt.jar ./local.data remoteuser@<remote_address>:destinationDir
     

     If the remoteuser parameter is not specified the local user, running the fdt command, will be used to login on the remote system

2. To get the content of an entire folder and all its children, located in the user's home directory, the -r ( recursive mode ) flag will be specified and also -pull to sink the data from the server. In the Client/Server mode the access to the server will be restricted to the local IP addresses only ( with -f flag ).

   • Client/Server mode

     Multiple addresses may be specfied using the -f flag using ':'. If the client's IP address(es) is not specified in the allowed IP addresses the connection will be closed. In the following command the server is started in standalone mode, which means that will continue to run after the session will finish. The transfer rate for every client sessions will be limited to 4 MBytes/s

     [remote computer]$ java -jar fdt.jar -f allowedIP1:allowedIP2 -limit 4M
     

     OR

     [remote computer]$ java -jar fdt.jar -f allowedIP1:allowedIP2  -limit 4096K
     

     The command for the local client will be.

     [local computer]$ java -jar fdt.jar -pull -r -c <remote_address> -d /home/localuser/localDir /home/remoteuser/remoteDir
     

     OR

     [local computer]$ java -jar fdt.jar -pull -r -c <remote_address> -d localDir remoteDir 
     

   • SCP mode

     In this mode only the order of the parameters will be changed, and -r is the only argument that must be added ( -pull is implicit ). Same authentication policies apply as in the first example

     [local computer]$ java -jar fdt.jar -r remoteuser@<remote_address>:/home/remoteuser/remoteDir /home/localuser/localDir
     

     OR

     [local computer]$ java -jar fdt.jar -r remoteuser@<remote_address>:remoteDir localDir
     

3. To test the network connectivity a transfer here is an example which transfers data from /dev/zero to /dev/null using 10 streams in blocking mode, for both the server and the client with 8 MBytes buffers. The server will stop after the test is finished

   • Client/Server mode

     [remote computer]$ java -jar fdt.jar -bio -bs 8M -f allowedIP -S
     [local computer]$ java -jar fdt.jar -c <remote_address> -bio -P 10 -d /dev/null /dev/zero 
     

   • SCP mode

     [local computer]$ java -jar fdt.jar -bio -P 10 /dev/zero remoteAddress:/dev/null
     

4. The user can also define a list of files ( a filename per line ) to be transfered. FDT will detect if the files are located on multiple devices and will use a dedicated thread for each device.

     [remote computer]$ java -jar fdt.jar -S
     [local computer]$ java -jar fdt.jar -fl ./file_list.txt -c <remote_address> -d /home/remoteuser/destDir
   

5. To test the local read/write performance of the local disk the DDCopy may be used.

   • The following command will copy the entire partition /dev/dsk/c0d1p1 to /dev/null reporting every 2 seconds ( the default ) the I/O speed

     [local computer]$ java -cp fdt.jar lia.util.net.common.DDCopy if=/dev/dsk/c0d1p1 of=/dev/null
     

   • To test the write speed of the file system using a 1GB file read from /dev/zero the following command may be used. The operating system will sync() the data to the disk. The data will be read/write using 10MB buffers

     [local computer]$ java -cp fdt.jar lia.util.net.common.DDCopy if=/dev/zero of=/home/user/1GBTestFile bs=10M count=100 flags=NOSYNC
     

     OR

     [local computer]$ java -cp fdt.jar lia.util.net.common.DDCopy if=/dev/zero of=/home/user/1GBTestFile bs=1M bn=10 count=100 flags=NOSYNC
     

6. Network test mode. The source and destination (-d) options can be omitted. If present, will be ignored! FDT will act only as a network benchmark tool. If the server is started with this flag ALL network transfers will be considered benchmark transfers!

   First, the FDT server needs to be started on the remote system. ( The default settings will be used, which implies the default ports,control 54321 and transfer 43210, on both the client and the server ).

    [remote computer]$ java -jar fdt.jar
   

   Then, the client will be started on the local system specifying the network testing mode argument, the remote address (or hostname) where the server was started.

    [local computer]$ java -jar fdt.jar -c <remote_address> -nettest
   

7. To copy all files from i.e. /tmp/fdt directory contents to destination directory /home/user/destination. First, the FDT server needs to be started on the remote system. ( The default settings will be used, which implies the default ports,control 54321 and transfer 43210 on both the client and the server ).

    [remote computer]$ java -jar fdt.jar
   

   Then, the client will be started on the local system specifying the network testing mode argument, the remote address (or hostname) where the server was started.

    [local computer]$ java -jar fdt.jar -c <remote_address> -pull -r -d /home/user/destination /tmp/fdt/
   

8. To copy files from remote machine to other remote machine using coordinator mode first you need to launch FDT on source and destination servers. First, the FDT server needs to be started on the remote system. ( The default settings will be used, which implies the default ports,control 54321 and transfer 43210 on both the client and the server ).

    [remote source computer]$ java -jar fdt.jar -agent
   
    [remote destination computer]$ java -jar fdt.jar -agent
   

   Then, the coordinator will be started on the local system specifying destination ip, port, directory and source ip with the file list on coordinator machine.

    [local computer]$ java -jar fdt.jar -c <source-host> -p <source-port> -dIP <destination-ip> -dp <destination-port> -sIP <source-ip> -d /home/user/destination -fl /file/list/on/coordinator/files10.txt -coord
   

   After executing this command coordinator will send job message to source agent, source agent then will ask for destination agent for transfer port (open socked for transfers) and then source agent will begin copying

   [NOTE] FDT will fail if we will start to transfer file which size has changed after generating file transfer jobs in FDT