Scenario:
We have pdf files on server2 named by ticket number, i.e. 2543675.pdf, due to users making mistakes with renaming the files correctly and causing imports to get stuck, I have been tasked with renaming these files automatically via a web interface with just one button click to initiate the renaming of files to format ticketnumber_customerNumber_customerName.pdf. In order for me to do that, I need to move/rsync the files from server2 to server1. Extract the base name from pdf file which is the customer number, run a query against customer ticket table in some database and rename the files with the data I get from the tickets table. Once done, I move/rsync the files back to server1 with rsync where the files are then uploaded/imported by an automated system that expects them to be named properly or the process will fail and halt the work flow.
Test Environment using Virtualbox VMs:
server1 192.168.1.2 Centos 7 - Hosting web interface for rsync
server2 192.168.1.3 Ubunutu 10 - Hosting SMB share with pdf
workstation 192.168.1.100 Workstation invoking php script via web page
What I've tried:
Server1
1. Generated ssh keys and copied them to remote server server2 for automatic authentication via key. Used ssh-copy-id
2. Copied ssh keys to /opt/www-files directory on server1
3. Changed owner to apache:apache for /opt/www-files on server1
4. Current access to directory www-files 755, files id_rsa 600, id_rsa.pub 644
5. Ran web interface readFiles2.php and it didn't work when running script. I get an error in apache log:
[Fri Aug 03 12:54:16.432825 2018] [:error] [pid 1670] [client 192.168.1.100:40808] PHP Warning: ssh2_connect(): Unable to connect to 192.168.1.3 on port 22 in /var/www/html/readFiles2.php on line 20
[Fri Aug 03 12:54:16.432886 2018] [:error] [pid 1670] [client 192.168.1.100:40808] PHP Warning: ssh2_connect(): Unable to connect to 192.168.1.3 in /var/www/html/readFiles2.php on line 20
[Fri Aug 03 12:54:16.432910 2018] [:error] [pid 1670] [client 192.168.1.100:40808] PHP Warning: ssh2_fingerprint() expects parameter 1 to be resource, boolean given in /var/www/html/readFiles2.php on line 21
[Fri Aug 03 12:54:16.432920 2018] [:error] [pid 1670] [client 192.168.1.100:40808] PHP Warning: ssh2_auth_pubkey_file() expects parameter 1 to be resource, boolean given in /var/www/html/readFiles2.php on line 22
[Fri Aug 03 12:54:16.432960 2018] [:error] [pid 1670] [client 192.168.1.100:40808] PHP Warning: ssh2_exec() expects parameter 1 to be resource, boolean given in /var/www/html/readFiles2.php on line 27
[Fri Aug 03 12:54:16.432970 2018] [:error] [pid 1670] [client 192.168.1.100:40808] PHP Warning: ssh2_fetch_stream() expects parameter 1 to be resource, null given in /var/www/html/readFiles2.php on line 28
[Fri Aug 03 12:54:16.432986 2018] [:error] [pid 1670] [client 192.168.1.100:40808] PHP Warning: stream_set_blocking() expects parameter 1 to be resource, null given in /var/www/html/readFiles2.php on line 30
[Fri Aug 03 12:54:16.432997 2018] [:error] [pid 1670] [client 192.168.1.100:40808] PHP Warning: stream_set_blocking() expects parameter 1 to be resource, null given in /var/www/html/readFiles2.php on line 31
[Fri Aug 03 12:54:16.433010 2018] [:error] [pid 1670] [client 192.168.1.100:40808] PHP Warning: stream_get_contents() expects parameter 1 to be resource, null given in /var/www/html/readFiles2.php on line 34
[Fri Aug 03 12:54:16.433019 2018] [:error] [pid 1670] [client 192.168.1.100:40808] PHP Warning: stream_get_contents() expects parameter 1 to be resource, null given in /var/www/html/readFiles2.php on line 36
[Fri Aug 03 12:54:16.433043 2018] [:error] [pid 1670] [client 192.168.1.100:40808] PHP Warning: fclose() expects parameter 1 to be resource, null given in /var/www/html/readFiles2.php on line 42
[Fri Aug 03 12:54:16.433051 2018] [:error] [pid 1670] [client 192.168.1.100:40808] PHP Warning: fclose() expects parameter 1 to be resource, null given in /var/www/html/readFiles2.php on line 43
-
if I run readFiles1.php from web interface a I get an unexplained error in apache but if I run this script from the console as root, it works fine. Files are transferred back and forth depending on which command I run within the script.
Apache Error:
ssh: connect to host 192.168.1.3 port 22: Permission denied
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.2]
Server's File Layout
Server1
/
├── opt
│ └── www-files
│ ├── id_rsa
│ └── id_rsa.pub
│
├── var
│ └── www
│ └── html
│ ├── files
│ ├── readFiles1.php
│ └── readFiles2.php
Server2
/
├── srv
│ └── pdfs
│ ├── 2534566.pdf
│ ├── 2456345.dpf
│ ├── 2643123.pdf
│ ├── 2456256.pdf
│ ├── 2789023.pdf
│ ├── 2567454.pdf
│ ├── 2876430.pdf
. .
. .
. .
Script: readFiles1.php
<?php
function readFiles($dir){
if(is_dir($dir)){
if($dh=opendir($dir)){
while(($file = readdir($dh)) != false){
echo "Filename: ".$file."<br>";
}
closedir($dh);
}
}
}
//$cmd = 'rsync -av --ignore-existing --remove-source-files /var/www/html/files/* root@192.168.1.3:/srv/pdfs/';
$cmd = "rsync -av --ignore-existing --remove-source-files root@192.168.1.3:/srv/pdfs/ /var/www/html/files/";
shell_exec($cmd);
$dir_source = "./files/";
readFiles($dir_source);
?>
Script: readFiles2.php Found this script on the web, had to install some dependencies.
<?php
include('/usr/share/pear/Net/SSH2.php');
function readFiles($dir){
if(is_dir($dir)){
if($dh=opendir($dir)){
while(($file = readdir($dh)) != false){
echo "Filename: ".$file."<br>";
}
closedir($dh);
}
}
}
function runRemoteSSH($command){
$connection = ssh2_connect('192.168.1.3');
$ssh_key = ssh2_fingerprint($connection);
ssh2_auth_pubkey_file($connection, 'root','/opt/www-files/id_rsa.pub','/opt/www-files/id_rsa');
$log = array();
$log[] = "Sending command: ".$command;
$log[] = "--------------------";
$stream = ssh2_exec($connection, $command);
$streamError = ssh2_fetch_stream($stream, SSH2_STREAM_STDERR);
stream_set_blocking($streamError, true);
stream_set_blocking($stream, true);
$log[] = "output of command: ";
$log[] = stream_get_contents($stream);
$log[] = "------------------";
$error = stream_get_contents($stream);
if(strlen($error) > 0 ){
$log[] = 'Error occured:';
$log[] = $error;
$log[] = "--------";
}
fclose($streamError);
fclose($stream);
}
//$cmd = 'rsync -av --ignore-existing --remove-source-files /var/www/html/files/* root@192.168.1.3:/srv/pdfs/';
$cmd = "rsync -av --ignore-existing --remove-source-files root@192.168.1.3:/srv/pdfs/ /var/www/html/files/";
runRemoteSSH($cmd);
$dir_source = "./files/";
readFiles($dir_source);
?>
Script 3(readFiles3.php): Gets files from local SMB share, works fine here since no ssh connection is needed.
<?php
function readFiles($dir){
if(is_dir($dir)){
if($dh=opendir($dir)){
while(($file = readdir($dh)) != false){
if($file != "." && $file != ".."){
$list = explode(".",$file);
$source_file = "./files/".$file;
$target_file = "./files/".$list[0]."_custno_custname.".$list[1];
echo "Renaming file[$source_file] to [$target_file]<br>";
rename($source_file, $target_file);
}
}
closedir($dh);
}
}
}
$cmd = "rsync -av --ignore-existing --remove-source-files /srv/smb/tickets/ /var/www/html/files";
echo "Getting files....<br>";
shell_exec($cmd);
$dir_source = "./files/";
readFiles($dir_source);
echo "Moving files back....<br>";
$cmd = "rsync -av --ignore-existing --remove-source-files /var/www/html/files/ /srv/smb/tickets/ ";
shell_exec($cmd);
?>
