I'm using php curl with nginx as a proxy. here is my code:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_PROXY, $proxy);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$curl_scraped_page = curl_exec($ch);
curl_close($ch);
echo $curl_scraped_page;
after sometime that this running the nginx load is extreamly slow and sometime it returns error 500.
the log says
failed (24: Too many open files),
some more details:
root@proxy-s2:~# ulimit -Hn
4096
root@proxy-s2:~# ulimit -Sn
1024
There is nothing else running on the server, and no other script is using this proxy.
Is it nginx bug? how to resolve it?
or
What else could it be? how can it be resolved?
- I didn't change the default nginx configuration
- Nginx restart solved the problem (temporarily I guess)
here is my nginx.conf
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
gzip on;
server {
listen 8080;
location / {
resolver 8.8.8.8;
proxy_pass http://$http_host$uri$is_args$args;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}
top
top - 09:23:55 up 21:51, 1 user, load average: 0.09, 0.13, 0.08
KiB Mem: 496164 total, 444328 used, 51836 free, 12300 buffers
KiB Swap: 0 total, 0 used, 0 free. 336228 cached Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
8 root 20 0 0 0 0 S 0.0 0.0 4:57.56 rcuos/0
4904 nobody 20 0 97796 14128 1012 R 0.0 2.8 4:19.82 nginx
7 root 20 0 0 0 0 S 0.0 0.0 2:11.35 rcu_sched
3 root 20 0 0 0 0 S 0.0 0.0 0:18.50 ksoftirqd/0
832 root 20 0 139208 6808 172 S 0.0 1.4 0:13.11 nova-agent
45 root 20 0 0 0 0 S 0.0 0.0 0:06.21 xenbus
74 root 20 0 0 0 0 S 0.0 0.0 0:03.03 kworker/u30:1
155 root 20 0 0 0 0 S 0.0 0.0 0:02.73 jbd2/xvda1-8
46 root 20 0 0 0 0 R 0.0 0.0 0:02.39 kworker/0:1
57 root 20 0 0 0 0 S 0.0 0.0 0:01.91 kswapd0
1 root 20 0 33448 2404 1136 S 0.0 0.5 0:01.47 init
391 root 20 0 18048 1336 996 S 0.0 0.3 0:00.97 xe-daemon
1034 syslog 20 0 255840 2632 784 S 0.0 0.5 0:00.90 rsyslogd
1107 root 20 0 61364 3048 2364 S 0.0 0.6 0:00.73 sshd
40 root rt 0 0 0 0 S 0.0 0.0 0:00.29 watchdog/0
316 root 20 0 19472 456 252 S 0.0 0.1 0:00.12 upstart-udev-br
6 root 20 0 0 0 0 S 0.0 0.0 0:00.11 kworker/u30:0
1098 root 20 0 23652 1036 784 S 0.0 0.2 0:00.08 cron
7935 root 20 0 105632 4272 3284 S 0.0 0.9 0:00.07 sshd
330 root 20 0 51328 1348 696 S 0.0 0.3 0:00.06 systemd-udevd
7953 root 20 0 22548 3428 1680 S 0.0 0.7 0:00.05 bash
678 root 20 0 15256 524 268 S 0.0 0.1 0:00.04 upstart-socket-
8647 root 20 0 25064 1532 1076 R 0.0 0.3 0:00.03 top
mpstat
root@proxy-s2:~# mpstat
Linux 3.13.0-55-generic (proxy-s2) 07/09/2015 _x86_64_ (1 CPU)
09:22:17 AM CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle
09:22:17 AM all 0.94 0.00 1.63 0.16 0.00 2.16 0.92 0.00 0.00 94.20
iostat
root@proxy-s2:~# iostat
Linux 3.13.0-55-generic (proxy-s2) 07/09/2015 _x86_64_ (1 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
0.94 0.00 3.80 0.16 0.92 94.19
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
xvdc 0.01 0.02 0.00 1710 0
xvda 3.16 4.19 88.56 322833 6815612
