weixin_39930748
weixin_39930748
2020-11-22 11:21

Openstack egress SG ICMP rule drops all traffic

Expected Behavior

Egress SG rule created by felix allowing ICMP traffic

Current Behavior

It seems that Openstack does not pass a port_range_min SG parameter with egress ICMP rules as it does with ingress ones. So when an egress rule is added, felix throws an error that it cannot validate the rule and drops all traffic.

Felix generates the following log output:


Nov 05 14:16:13 lsn-mb1021 calico-felix[1556296]: WARNING validation_filter.go 57: Validation failed; treating as missing error=error with the following fields:
                                                  -  icmp_code = '0'
                                                  -  icmp_type = '0'
                                                  -  icmp_code = '0'
                                                   key=ProfileRules(name=openstack-sg-2ba2f6c8-00a9-4a06-b4ff-4a65b4c0360b) value=&model.ProfileRules{InboundRules:[]model.Rule{model.Rule{Action:"", IPVersion:(*int)(0xc422d20e88), Protocol:(*numorstring.Protocol)(0xc422ff41e0), NotProtocol:(*numorstring.Protocol)(nil), ICMPType:(*int)(0xc422d20ef8), ICMPCode:(*int)(0xc422d20ee8), NotICMPType:(*int)(nil), NotICMPCode:(*int)(nil), SrcTag:"", SrcNet:(*net.IPNet)(0xc4220e4ea0), SrcNets:[]*net.IPNet(nil), SrcSelector:"", SrcPorts:[]numorstring.Port(nil), DstTag:"", DstSelector:"", DstNet:(*net.IPNet)(nil), DstNets:[]*net.IPNet(nil), DstPorts:[]numorstring.Port(nil), NotSrcTag:"", NotSrcNet:(*net.IPNet)(nil), NotSrcNets:[]*net.IPNet(nil), NotSrcSelector:"", NotSrcPorts:[]numorstring.Port(nil), NotDstTag:"", NotDstSelector:"", NotDstNet:(*net.IPNet)(nil), NotDstNets:[]*net.IPNet(nil), NotDstPorts:[]numorstring.Port(nil), LogPrefix:""}}, OutboundRules:[]model.Rule{model.Rule{Action:"", IPVersion:(*int)(0xc422d20d00), Protocol:(*numorstring.Protocol)(nil), NotProtocol:(*numorstring.Protocol)(nil), ICMPType:(*int)(nil), ICMPCode:(*int)(nil), NotICMPType:(*int)(nil), NotICMPCode:(*int)(nil), SrcTag:"", SrcNet:(*net.IPNet)(nil), SrcNets:[]*net.IPNet(nil), SrcSelector:"", SrcPorts:[]numorstring.Port(nil), DstTag:"", DstSelector:"", DstNet:(*net.IPNet)(0xc4220e4840), DstNets:[]*net.IPNet(nil), DstPorts:[]numorstring.Port(nil), NotSrcTag:"", NotSrcNet:(*net.IPNet)(nil), NotSrcNets:[]*net.IPNet(nil), NotSrcSelector:"", NotSrcPorts:[]numorstring.Port(nil), NotDstTag:"", NotDstSelector:"", NotDstNet:(*net.IPNet)(nil), NotDstNets:[]*net.IPNet(nil), NotDstPorts:[]numorstring.Port(nil), LogPrefix:""}, model.Rule{Action:"", IPVersion:(*int)(0xc422d20d50), Protocol:(*numorstring.Protocol)(nil), NotProtocol:(*numorstring.Protocol)(nil), ICMPType:(*int)(nil), ICMPCode:(*int)(nil), NotICMPType:(*int)(nil), NotICMPCode:(*int)(nil), SrcTag:"", SrcNet:(*net.IPNet)(nil), SrcNets:[]*net.IPNet(nil), SrcSelector:"", SrcPorts:[]numorstring.Port(nil), DstTag:"", DstSelector:"", DstNet:(*net.IPNet)(0xc4220e49f0), DstNets:[]*net.IPNet(nil), DstPorts:[]numorstring.Port(nil), NotSrcTag:"", NotSrcNet:(*net.IPNet)(nil), NotSrcNets:[]*net.IPNet(nil), NotSrcSelector:"", NotSrcPorts:[]numorstring.Port(nil), NotDstTag:"", NotDstSelector:"", NotDstNet:(*net.IPNet)(nil), NotDstNets:[]*net.IPNet(nil), NotDstPorts:[]numorstring.Port(nil), LogPrefix:""}, model.Rule{Action:"", IPVersion:(*int)(0xc422d20da8), Protocol:(*numorstring.Protocol)(0xc422ff4160), NotProtocol:(*numorstring.Protocol)(nil), ICMPType:(*int)(0xc422d20e48), ICMPCode:(*int)(0xc422d20db8), NotICMPType:(*int)(nil), NotICMPCode:(*int)(nil), SrcTag:"", SrcNet:(*net.IPNet)(nil), SrcNets:[]*net.IPNet(nil), SrcSelector:"", SrcPorts:[]numorstring.Port(nil), DstTag:"", DstSelector:"", DstNet:(*net.IPNet)(0xc4220e4c30), DstNets:[]*net.IPNet(nil), DstPorts:[]numorstring.Port(nil), NotSrcTag:"", NotSrcNet:(*net.IPNet)(nil), NotSrcNets:[]*net.IPNet(nil), NotSrcSelector:"", NotSrcPorts:[]numorstring.Port(nil), NotDstTag:"", NotDstSelector:"", NotDstNet:(*net.IPNet)(nil), NotDstNets:[]*net.IPNet(nil), NotDstPorts:[]numorstring.Port(nil), LogPrefix:""}}}
Nov 05 14:16:13 lsn-mb1021 calico-felix[1556296]: 2017-11-05 14:16:13.481 [WARNING][1556296] active_rules_calculator.go 290: Profile not known or invalid, generating dummy profile that drops all traffic. profileID="openstack-sg-2ba2f6c8-00a9-4a06-b4ff-4a65b4c0360b"
Nov 05 14:16:13 lsn-mb1021 calico-felix[1556296]: 2017-11-05 14:16:13.481 [INFO][1556296] int_dataplane.go 574: Received *proto.ActiveProfileUpdate update from calculation graph msg=id:<name:> profile:<inbound_rules: rule_id:> outbound_rules:<action: rule_id:> >
Nov 05 14:16:13 lsn-mb1021 calico-felix[1556296]: 2017-11-05 14:16:13.481 [INFO][1556296] table.go 398: Queueing update of chain. chainName="cali-pri-_R8P12ttgKKbXurrjDk" ipVersion=0x4 table="filter"
</action:></inbound_rules:></name:>

Steps to Reproduce (for bugs)

Create any one of these security group rules and apply the group to an instance:


$ openstack security group rule create --remote-ip 0.0.0.0/0 --protocol icmp --egress 2ba2f6c8-00a9-4a06-b4ff-4a65b4c0360b
+-------------------+--------------------------------------+
| Field             | Value                                |
+-------------------+--------------------------------------+
| created_at        | 2017-11-05T14:24:14Z                 |
| description       |                                      |
| direction         | egress                               |
| ether_type        | IPv4                                 |
| id                | d8d4152a-4637-4da4-b50f-8b513ebc1f5d |
| name              | None                                 |
| port_range_max    | None                                 |
| port_range_min    | None                                 |
| project_id        | 7414bd929e6b467aa955da9d30b16621     |
| protocol          | icmp                                 |
| remote_group_id   | None                                 |
| remote_ip_prefix  | 0.0.0.0/0                            |
| revision_number   | 1                                    |
| security_group_id | 2ba2f6c8-00a9-4a06-b4ff-4a65b4c0360b |
| updated_at        | 2017-11-05T14:24:14Z                 |
+-------------------+--------------------------------------+

$ openstack security group rule create --remote-ip 0.0.0.0/0 --protocol icmp --icmp-type 0 --egress 2ba2f6c8-00a9-4a06-b4ff-4a65b4c0360b
+-------------------+--------------------------------------+
| Field             | Value                                |
+-------------------+--------------------------------------+
| created_at        | 2017-11-05T14:26:53Z                 |
| description       |                                      |
| direction         | egress                               |
| ether_type        | IPv4                                 |
| id                | b53e499b-ce22-4c0e-8249-fdf5e28e4366 |
| name              | None                                 |
| port_range_max    | None                                 |
| port_range_min    | None                                 |
| project_id        | 7414bd929e6b467aa955da9d30b16621     |
| protocol          | icmp                                 |
| remote_group_id   | None                                 |
| remote_ip_prefix  | 0.0.0.0/0                            |
| revision_number   | 1                                    |
| security_group_id | 2ba2f6c8-00a9-4a06-b4ff-4a65b4c0360b |
| updated_at        | 2017-11-05T14:26:53Z                 |
+-------------------+--------------------------------------+

$ openstack security group rule create --remote-ip 0.0.0.0/0 --protocol icmp --icmp-type 0 --icmp-code 0 --egress 2ba2f6c8-00a9-4a06-b4ff-4a65b4c0360b
+-------------------+--------------------------------------+
| Field             | Value                                |
+-------------------+--------------------------------------+
| created_at        | 2017-11-05T14:28:56Z                 |
| description       |                                      |
| direction         | egress                               |
| ether_type        | IPv4                                 |
| id                | efc4b30e-b085-47dc-9ab9-8fe132cc93fe |
| name              | None                                 |
| port_range_max    | None                                 |
| port_range_min    | None                                 |
| project_id        | 7414bd929e6b467aa955da9d30b16621     |
| protocol          | icmp                                 |
| remote_group_id   | None                                 |
| remote_ip_prefix  | 0.0.0.0/0                            |
| revision_number   | 1                                    |
| security_group_id | 2ba2f6c8-00a9-4a06-b4ff-4a65b4c0360b |
| updated_at        | 2017-11-05T14:28:56Z                 |
+-------------------+--------------------------------------+

Context

This behavior did not occur on 1.x. I have tested Felix 2.3 and 2.6, both show the same output.

Your Environment

  • Calico Felix 2.6
  • etcd 2.2.5
  • Ubuntu 16.04
  • OpenStack Ocata

该提问来源于开源项目:projectcalico/felix

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享
  • 邀请回答

9条回答

  • weixin_39547158 weixin_39547158 5月前

    were you able to retest with my fix in place i.e. v2.6.3+ or v3.0.0+?

    点赞 评论 复制链接分享
  • weixin_39765697 weixin_39765697 5月前

    ping

    点赞 评论 复制链接分享
  • weixin_39859052 weixin_39859052 5月前

    Given the age on this, and the fact we think we've got a fix, I'm going to close this.

    Please respond on here if you see this again and we can re-open!

    点赞 评论 复制链接分享
  • weixin_39547158 weixin_39547158 5月前

    Hmm, 0 should be a valid value for the type/code so Felix shouldn't be barfing on it. The fact that OpenStack is converting 0 -> None looks like a bug (in OpenStack) to me.

    Looks like the validation condition is wrong here: https://github.com/projectcalico/libcalico-go/blob/master/lib/backend/model/rule.go#L34 0 should be allowed, 255 should be disallowed.

    点赞 评论 复制链接分享
  • weixin_39547158 weixin_39547158 5月前

    I've put up a fix for the validation error but it's not clear that OpenStack doesn't have a bug (at least in the command output above). I think it should be distinguishing between 0 and None in the output above. Presumably protocol=icmp port_range_min=None would mean "match all ICMP traffic" but protocol=icmp port_range_min=0 should mean "Match ICMP traffic with type=0"

    点赞 评论 复制链接分享
  • weixin_39634884 weixin_39634884 5月前

    Any thoughts on why this would be different for egress than ingress?

    Does the ICMP code/type range, when specified, normally show up in the stdout from openstack security group rule create? I think is assuming that it normally shows up in the port_range fields (and hence is being misrepresented here as None) - but perhaps it just doesn't appear at all?

    点赞 评论 复制链接分享
  • weixin_39547158 weixin_39547158 5月前

    Yes, indeed, I was assuming it'd be echoed back. It'd be good to know what _neutron_rule_to_etcd_rule is getting as input but I don't think we log that even at DEBUG.

    点赞 评论 复制链接分享
  • weixin_39634884 weixin_39634884 5月前

    For reference, the relevant networking-calico code: http://git.openstack.org/cgit/openstack/networking-calico/tree/networking_calico/plugins/ml2/drivers/calico/t_etcd.py#n988

    点赞 评论 复制链接分享
  • weixin_39930748 weixin_39930748 5月前

    Yeah it does show the type/code when specified as non-zero values.

    
    ubuntu:~$ openstack security group create test
    +-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    | Field           | Value                                                                                                                                                                      |
    +-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    | created_at      | 2017-11-07T02:27:44Z                                                                                                                                                       |
    | description     | test                                                                                                                                                                       |
    | id              | a0b7b671-2f9d-471a-bc7c-12f023aee23b                                                                                                                                       |
    | name            | test                                                                                                                                                                       |
    | project_id      | 4450a82f9deb45668f35ce712fc74074                                                                                                                                           |
    | revision_number | 1                                                                                                                                                                          |
    | rules           | created_at='2017-11-07T02:27:44Z', direction='egress', ethertype='IPv4', id='3324ce59-1d52-4de0-bf91-749557b7a568', revision_number='1', updated_at='2017-11-07T02:27:44Z' |
    |                 | created_at='2017-11-07T02:27:44Z', direction='egress', ethertype='IPv6', id='d705eb44-66ea-4127-a462-2d133f61b4a4', revision_number='1', updated_at='2017-11-07T02:27:44Z' |
    | updated_at      | 2017-11-07T02:27:44Z                                                                                                                                                       |
    +-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    ubuntu:~$ openstack security group rule create --remote-ip 0.0.0.0/0 --protocol icmp --icmp-type 3 --icmp-code 1 --egress a0b7b671-2f9d-471a-bc7c-12f023aee23b
    +-------------------+--------------------------------------+
    | Field             | Value                                |
    +-------------------+--------------------------------------+
    | created_at        | 2017-11-07T02:28:30Z                 |
    | description       |                                      |
    | direction         | egress                               |
    | ether_type        | IPv4                                 |
    | id                | 0de57f5f-754f-4158-8f4a-15fdb7bfaed6 |
    | name              | None                                 |
    | port_range_max    | 1                                    |
    | port_range_min    | 3                                    |
    | project_id        | 4450a82f9deb45668f35ce712fc74074     |
    | protocol          | icmp                                 |
    | remote_group_id   | None                                 |
    | remote_ip_prefix  | 0.0.0.0/0                            |
    | revision_number   | 1                                    |
    | security_group_id | a0b7b671-2f9d-471a-bc7c-12f023aee23b |
    | updated_at        | 2017-11-07T02:28:30Z                 |
    +-------------------+--------------------------------------+
    
    点赞 评论 复制链接分享

相关推荐