weixin_39624816
weixin_39624816
2021-01-10 07:44

DBZ-2637 Enhance Postgres validation checks

DBZ-2637 Enhance Postgres validation checks + add wal_level config check + add LOGIN and REPLICATION roles check + add replication slot is already in use check

closes https://issues.redhat.com/browse/DBZ-2637

该提问来源于开源项目:debezium/debezium

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享
  • 邀请回答

13条回答

  • weixin_39624816 weixin_39624816 4月前

    Can you have a look at #2037 and also at the tasks https://issues.redhat.com/browse/DBZ-2893 and https://issues.redhat.com/browse/DBZ-2894. This should fix the issue with AWS, and also with other environments later in a more generic way.b WDYT?

    点赞 评论 复制链接分享
  • weixin_39964869 weixin_39964869 4月前

    Applied, thanks!

    点赞 评论 复制链接分享
  • weixin_39695954 weixin_39695954 4月前

    Hi, I've noticed that this change seems to break Debezium on AWS RDS where some roles have non-standard names e.g. rds_replication instead of replication

    点赞 评论 复制链接分享
  • weixin_39964869 weixin_39964869 4月前

    Could you please raise a Jira issue?

    点赞 评论 复制链接分享
  • weixin_39624816 weixin_39624816 4月前

    thank you ! I will create the Jira issue for it.

    See: https://issues.redhat.com/browse/DBZ-2800

    点赞 评论 复制链接分享
  • weixin_39624816 weixin_39624816 4月前

    I checked the query on Amazon RDS Postgres and it looks like it's working. Can you maybe review my comment in the task and validate it's the same for you? https://issues.redhat.com/browse/DBZ-2800

    点赞 评论 复制链接分享
  • weixin_39695954 weixin_39695954 4月前

    Thank you for checking it. I am not sure about classic RDS but for Aurora the master user does not have replication role:

    
    postgres=> SELECT rolcanlogin, rolreplication FROM pg_roles WHERE rolname = current_user;
     rolcanlogin | rolreplication
    -------------+----------------
     t           | f
    (1 row)
    

    master is however a member of rds_replication:

    
    postgres=> \du
                                                                      List of roles
           Role name       |                         Attributes                         |                          Member of
    -----------------------+------------------------------------------------------------+-------------------------------------------------------------
     full_sch_udm_data     | Cannot login                                               | {}
     full_sch_udm_data_pii | Cannot login                                               | {}
     master                | Create role, Create DB                                    +| {rds_superuser,rds_replication}
                           | Password valid until infinity                              |
     rds_ad                | Cannot login                                               | {}
     rds_iam               | Cannot login                                               | {}
     rds_password          | Cannot login                                               | {}
     rds_replication       | Cannot login                                               | {}
    

    The Postgres version I am using is 11.6

    点赞 评论 复制链接分享
  • weixin_39821189 weixin_39821189 4月前

    I have ran into the same issue on classic RDS, running PostgreSQL 12.4-R1:

    
    SELECT rolcanlogin, rolreplication FROM pg_roles WHERE rolname = current_user;
     rolcanlogin | rolreplication
    -------------+----------------
     t           | f
    (1 row)
    

    and the master is a member of rds_replication:

    
    \du postgres
                                    List of roles
     Role name |          Attributes           |            Member of
    -----------+-------------------------------+---------------------------------
     postgres  | Create role, Create DB       +| {rds_superuser,rds_replication}
               | Password valid until infinity |
    

    Oddly enough, it looks like rds_replication itself doesn't have rolreplication=true

    
    SELECT rolcanlogin, rolreplication FROM pg_roles WHERE rolname='rds_replication';
     rolcanlogin | rolreplication
    -------------+----------------
     f           | f
    (1 row)
    
    点赞 评论 复制链接分享
  • weixin_39821189 weixin_39821189 4月前

    I've downgraded to 1.3.1 and can confirm that the connector works given my current setup described above ^

    点赞 评论 复制链接分享
  • weixin_39624816 weixin_39624816 4月前

    Thank you for checking. And sorry to bother you again. Can you share the output of SELECT * FROM pg_roles WHERE rolname = current_user; for the master user.

    When I checked on a fresh AWS RDS Postgres I got (for master user):

    
    postgres=> SELECT rolcanlogin, rolreplication FROM pg_roles WHERE rolname = current_user;
    
     rolcanlogin | rolreplication  
    -------------+----------------
    
     t           | t
    
    
    点赞 评论 复制链接分享
  • weixin_39624816 weixin_39624816 4月前

    Happy new year ! And sorry to bother you again again.

    Can you share the output of SELECT * FROM pg_roles WHERE rolname = current_user; for the master user so I can see the complete list of available roles for the current user. Thx again!

    点赞 评论 复制链接分享
  • weixin_39814369 weixin_39814369 4月前

    See DBZ-2800 for what we'll do for 1.4.0.Final: add the RDS specific role name to the check, and also make it more lenient (i.e. log the error about missing roles, but don't make it an actual validation error).

    点赞 评论 复制链接分享
  • weixin_39624816 weixin_39624816 4月前

    Thank you !

    I took a closer look at a AWS RDS Postgres installations today. When I understand it correctly, the debezium connector user needs to have one of these roles assigned: rds_superuser, rdsadmin or rdsrepladmin in order to have the rolreplication rights that allow starting a logical replication. Correct?

    When I get it right, that's also why we currently recommend to use the RDS master user as debezium user. Can you confirm my findings?

    点赞 评论 复制链接分享

相关推荐