weixin_39844880
2020-11-26 18:10 阅读 13

redis-server crashes after loading module

I was following this guide.

Everything was fine, until I added the loadmodule directive in redis.conf

After adding it and restarting the redis-server, it get's hung up until timeout:


Job for redis-server.service failed because a timeout was exceeded.
See "systemctl status redis-server.service" and "journalctl -xe" for details.

$ sudo service redis-server status


● redis-server.service - Advanced key-value store
   Loaded: loaded (/lib/systemd/system/redis-server.service; enabled; vendor preset: enabled)
   Active: activating (start) since Sat 2019-01-19 05:51:00 UTC; 13s ago
     Docs: http://redis.io/documentation,
           man:redis-server(1)
  Process: 23037 ExecStop=/bin/kill -s TERM $MAINPID (code=exited, status=0/SUCCESS)
  Process: 24593 ExecStart=/usr/bin/redis-server /etc/redis/redis.conf (code=exited, status=0/SUCCESS)
 Main PID: 22625 (code=exited, status=0/SUCCESS)
    Tasks: 0 (limit: 1152)
   CGroup: /system.slice/redis-server.service

Jan 19 05:51:00 XXXXXX systemd[1]: redis-server.service: Service hold-off time over, scheduling restart.
Jan 19 05:51:00 XXXXXX systemd[1]: redis-server.service: Scheduled restart job, restart counter is at 3.
Jan 19 05:51:00 XXXXXX systemd[1]: Stopped Advanced key-value store.
Jan 19 05:51:00 XXXXXX systemd[1]: Starting Advanced key-value store...
Jan 19 05:51:00 XXXXXX systemd[1]: redis-server.service: Can't open PID file /var/run/redis/redis-server.pid (yet?) after start: No such file or directory

$ redis-cli


Could not connect to Redis at 127.0.0.1:6379: Connection refused
Could not connect to Redis at 127.0.0.1:6379: Connection refused
not connected> 

$ sudo tail /var/log/redis/redis-server.log


      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

24737:M 19 Jan 05:55:31.911 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
24737:M 19 Jan 05:55:31.911 # Server initialized
24737:M 19 Jan 05:55:31.911 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
24737:M 19 Jan 05:55:31.911 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
24737:M 19 Jan 05:55:31.911 # Module /home/XXXXXX/rejson/rejson/src/rejson.so failed to load: /home/XXXXXX/rejson/rejson/src/rejson.so: cannot open shared object file: Permission denied
24737:M 19 Jan 05:55:31.911 # Can't load module from /home/XXXXXX/rejson/rejson/src/rejson.so: server aborting

If I comment out the loadmodule for rejson on redis.conf and restart, then everything works fine.

OS: Ubuntu 18.04 Redis server v=4.0.9 sha=00000000:0 malloc=jemalloc-3.6.0 bits=64 build=76095d16786fbcba

Thanks!

该提问来源于开源项目:RedisJSON/RedisJSON

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享

9条回答 默认 最新

  • weixin_39792049 weixin_39792049 2020-11-26 18:10

    Hello

    24737:M 19 Jan 05:55:31.911 # Module /home/XXXXXX/rejson/rejson/src/rejson.so failed to load: /home/XXXXXX/rejson/rejson/src/rejson.so: cannot open shared object file: Permission denied

    It looks like this is a permissions issue - your redis-server process can access the module's shared library. You can chmod and chown the file to resolve it.

    点赞 评论 复制链接分享
  • weixin_39844880 weixin_39844880 2020-11-26 18:10

    Yes I also thought so, but the issue persist even if I set the permissions to 0777 as for the owner should it be my user? root? (I tried both with same luck) or some other user? and does the user really matter if I just allow everything with given permissions?

    Thank you

    点赞 评论 复制链接分享
  • weixin_39844880 weixin_39844880 2020-11-26 18:10

    By the way I understand that we are talking about the rejson.so file, correct?

    Thanks!

    点赞 评论 复制链接分享
  • weixin_39792049 weixin_39792049 2020-11-26 18:10

    Yep - rejson.so. It has to be accessible to the user running redis-server.

    Perhaps the path is wrong (double rejson in it)? What does ls -al /home/XXXXXX/rejson/rejson/src/rejson.so yield?

    点赞 评论 复制链接分享
  • weixin_39792049 weixin_39792049 2020-11-26 18:10

    I'd hate to think there's black voodoo involved here so perhaps the path itself is not accessible to the user running the server... try moving the .so to /tmp and giving it 777 - if that doesn't work there's another (i.e. not permissions-related) issue. Pretty confident this should work though.

    点赞 评论 复制链接分享
  • weixin_39844880 weixin_39844880 2020-11-26 18:10

    I just tried what you say and sadly still not working, but now it complains it doesn't exist.

    $ sudo tail /var/log/redis/redis-server.log

    
          `-._    `-.__.-'    _.-'                                       
              `-._        _.-'                                           
                  `-.__.-'                                               
    
    22738:M 20 Jan 20:56:40.665 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
    22738:M 20 Jan 20:56:40.665 # Server initialized
    22738:M 20 Jan 20:56:40.665 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
    22738:M 20 Jan 20:56:40.665 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
    22738:M 20 Jan 20:56:40.665 # Module /tmp/rejson.so failed to load: /tmp/rejson.so: cannot open shared object file: No such file or directory
    22738:M 20 Jan 20:56:40.665 # Can't load module from /tmp/rejson.so: server aborting
    

    $ ls -la /tmp/rejson.so -rwxrwxrwx 1 redis redis 480208 Jan 19 05:45 /tmp/rejson.so

    Thank you!

    点赞 评论 复制链接分享
  • weixin_39562998 weixin_39562998 2020-11-26 18:10

    Maybe it shouldn't be in /tmp... Maybe it's mounted noexec, in which case, executable binaries will not be executable unless explicitly moved to a non-tmp location?

    点赞 评论 复制链接分享
  • weixin_39562998 weixin_39562998 2020-11-26 18:10

    also, if you are using something like selinux, those things are known to mess with permissions and whatnot.

    点赞 评论 复制链接分享
  • weixin_39844880 weixin_39844880 2020-11-26 18:10

    Hell thanks!

    I moved it to /etc/redis and it worked!

    As for the /tmp I'm not sure but I think it was fine $ df /tmp

    
    Filesystem     1K-blocks    Used Available Use% Mounted on
    /dev/vda1       25226960 2286064  22924512  10% /
    

    $ mount | grep /dev/vda1 /dev/vda1 on / type ext4 (rw,relatime,data=ordered)

    As for the selinux I have no idea, if you tell me how, I'll check it just in case it can help anyone else.

    Anyways, thanks again! : )

    点赞 评论 复制链接分享

相关推荐