使用exec进行长时间运行的脚本

Question

I want to get some data from and API and save for that user in database, this actions takes random times depending on the time and sometimes it takes even 4 hours, I am executing the script using exec and & in the background in php,

My question is that is exec safe for long running jobs, I dont know much about fork,and linux processes etc so I dont know what happened internally on CPU cores,

Here is something I found that confused me,

http://symcbean.blogspot.com/2010/02/php-and-long-running-processes.html

Can somebody tell me if I am going in right direction with exec?

will the process be killed itself after script completion?

Thanks

Answer 1

Well, that article is talking about process "trees" and how a child process depends of it spawning parent.

The PHP instance starts a child process (through exec or similar). If it doesn't wait for the process output, the PHP script ends (and the response is sent to the browser, for instance), but the process will sit idling, waiting for it's child process to finish.

The problem with this is that the child process (the long running 4 hours process) is not guaranteed to finish its job, before apache decides to kill its parent process (because you have too many idle processes) and, effectively, killing its children.

The article's author then gives the suggestion of using a daemon and separate the child process from the parent process.

Edit:

Answering the question you left in the comments, here's a quick explanation of the command he uses in the article

echo /usr/bin/php -q longThing.php | at now

Starting from left to right.

• echo prints to Standard Output (STDOUT) the stuff you put in front of it so...
• echo /usr/bin/php -q longThing.php will print to the shell /usr/bin/php -q longThing.php
• | (pipeline) feeds directly the STDOUT of a previous command to the standard input (STDIN) of the next command.
• at reads commands from STDIN and executes them at a specified time. at now means the command will be executed immediately.

So basically this is the same thing as running the following sequence in the shell:

1. at now - Opens the at prompt
2. /usr/bin/php -q longThing.php - The command we want to run
3. ^D (by pressing Control+D) - To save the job

---

So, regarding your questions:

Will the child process be immediately killed after the PARENT PHP script ends?

No.

Will the child process be killed at all, in some future moment?

Yes. Apache takes care of that for you.

Will the child process finish its job before being killed?

Maybe. Maybe not. Apache might kill it before its done. Odds of that happening increase with the number of idle processes and with the time the process takes to finish.

---

Sidenote:

I think this article does point in the right direction but I dislike the idea of spawning processes directly from PHP. In fact, PHP does not have the appropriate tools for running (long and/or intensive) bg work. With PHP alone, you have little to no control over it.

I can, however, give you the solution we found for a similar problem I faced a while ago. We created a small program that would accept and queue data processing requests (about 5 mins long) and report back when the request was finished. That way we could control how many processes could be running at the same time, memory usage, number of requests by the same user, etc...

The program was actually hosted in another LAN server, which prevented memory usage spikes slowing down the webserver.

At the front-end, the user would be informed when the request was completed through long polling,