weixin_39562340
2021-01-12 12:32 阅读 9

Get Private IP of instance launched by autoscaling group

This issue was originally opened by as hashicorp/terraform#11713. It was migrated here as part of the provider split. The original body of the issue is below.

ASG resource "aws_autoscaling_group" "REDIS_ASG" { name = "${var.environment}-REDIS_ASG" launch_configuration = "${aws_launch_configuration.redis_launch_conf.name}" #availability_zones = ["us-west-2a", "us-west-2b", "us-west-2c"] vpc_zone_identifier = ["${data.aws_subnet.PrivateDBSubnetAZ1.id}","${data.aws_subnet.PrivateDBSubnetAZ2.id}","${data.aws_subnet.PrivateDBSubnetAZ3.id}"] min_size = 1 max_size = 1 desired_capacity = 1 health_check_grace_period = 600 health_check_type = "EC2"
force_delete = "false" termination_policies = ["OldestInstance"] tag { key = "Name" value = "${var.environment}-int-redis" propagate_at_launch = true } }

i am trying to create a A Record resource "aws_route53_record" "redis" { zone_id = "${data.aws_route53_zone.dns.zone_id}" name = "redis-${var.environment}.${data.aws_route53_zone.dns.name}" type = "A" ttl = "60" records = ["${aws_autoscaling_group.REDIS_ASG.private_ip}"] }

Output- * Resource 'aws_autoscaling_group.REDIS_ASG' does not have attribute 'private_ip' for variable 'aws_autoscaling_group.REDIS_ASG.private_ip'

该提问来源于开源项目:hashicorp/terraform-provider-aws

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享

50条回答 默认 最新

  • weixin_39734493 weixin_39734493 2021-01-12 12:32

    I just had a need for this today. I would love for this to be a real thing.

    点赞 评论 复制链接分享
  • weixin_39980711 weixin_39980711 2021-01-12 12:32

    I ran into a need for this today. I'd love this to exist. Any updates on the request?

    点赞 评论 复制链接分享
  • weixin_39672443 weixin_39672443 2021-01-12 12:32

    Hello – Can you describe the use case here? I'm curious why you would want the direct IP of any instance that's in an autoscaling group, as opposed to connecting via a Load Balancer.

    If I'm not mistaken a specific advantage of an ASG is ensuring you have a certain number of instances available and running, not that any given one of them exists at a certain IP.

    If the ASG resource were to export a private IP, and on AWS side the ASG were to change instances for any reason, that IP could become stale until your next plan and apply, right? In which case there could be a window of time when your A record routes you to a bad IP, correct? Where as if the A record pointed to an IP that's assigned to a load balance, the roll over would happen automatically.

    I'm curious how exposing this would be used in a reliable, stable way. Thanks!

    点赞 评论 复制链接分享
  • weixin_39734493 weixin_39734493 2021-01-12 12:32

    You are correct in saying that IP would be stale and Terraform would need to run in order to update the A record attached to the Route53 record.

    We have use case that we have a set of servers that run different jobs. Everything is controlled in puppet and Terraform. Right now we have a module that builds off a count (normally). What we wanted to do was put them into an ASG in order to have self healing in case a node would fail. However we cant move forward with testing due to there is no output of private ips or a node list.

    We started an effect sometime ago to remove IP Address from configs and swap to DNS names. It would be icing on the cake if we can build a way to self heal these servers in case of aws instance failure using ASG.

    点赞 评论 复制链接分享
  • weixin_39798031 weixin_39798031 2021-01-12 12:32

    +1

    点赞 评论 复制链接分享
  • weixin_39528467 weixin_39528467 2021-01-12 12:32

    Another use case https://github.com/terraform-community-modules/tf_aws_bastion_s3_keys

    This sets up a bastion in an autoscaling group min=max=desired=1. The reasoning behind this is to autorelaunch the bastion if it goes down. W/o this however I have to do a check after to see what ip the bastion actually got.

    点赞 评论 复制链接分享
  • weixin_39774491 weixin_39774491 2021-01-12 12:32

    It would be very useful to have this feature. Users can run types of one off workloads with a list of known IPs for the ASG within a null_resource. +1 for me as well.

    点赞 评论 复制链接分享
  • weixin_39824529 weixin_39824529 2021-01-12 12:32

    Look at my comment in the closed issue migrated here: https://github.com/hashicorp/terraform/issues/11713

    To repeat:

    Very useful, will provide same functionality as CloudFormation does. We can simply wait for ASG to finish creating all instances and report back with private ip's, instance id's etc. so we can have further actions follow in terraform like creating dns records for the instances etc.

    Also no one says that ASG has to be related to an ELB, in our case we have the instances behind HAProxy so we need to know their IP's.

    This has been opened since February, is it really that difficult to resolve?

    点赞 评论 复制链接分享
  • weixin_39824529 weixin_39824529 2021-01-12 12:32

    Another example, I want to create Route53 DNS records and health checks after launching instances:

    
    # Host-specific A records for both load-balancers
    resource "aws_route53_record" "service-record" {
       zone_id = "<my-zone-id>"
       count = 2
       name = "service-lb-${format("%03d", count.index + 1)}.mydomain.com"
       type = "A"
       ttl = "60"
       records = ["${element(aws_instance.lb-service.*.public_ip, count.index)}"]
    }
    
    # Health checks for each of the load balancers
    resource "aws_route53_health_check" "service-healthcheck" {
      ip_address = "${element(aws_instance.lb-service.*.public_ip, count.index)}"
      count = 2
      port = 50000
      type = "HTTP"
      resource_path = "/health"
      failure_threshold = "5"
      request_interval = "30"
      tags = {
        Name = "service-${format("%03d", count.index + 1)}.production"
      }
    }
    
    # Group consisting of 2 alias records to the host records, with associated health checks, having weighted routing
    resource "aws_route53_record" "service-group" {
       zone_id = "<my-zone-id>"
       count = 2
       name = "service.mydomain.com"
       type = "A"
       weighted_routing_policy = {
          weight = "50"
       }
       health_check_id = "${element(aws_route53_health_check.service-healthcheck.*.id, count.index)}"
       set_identifier = "service${format("%03d", count.index + 1)}"
       alias {
         name = "${element(aws_route53_record.service-record.*.fqdn, count.index)}"
         zone_id = "<my-zone-id>"
         evaluate_target_health = true
       }
    }
    </my-zone-id></my-zone-id></my-zone-id>

    When instances are launched via ASG there is no way to do this. We are forced to move this code into user-data instead which is big PITA.

    点赞 评论 复制链接分享
  • weixin_39829501 weixin_39829501 2021-01-12 12:32

    +1

    点赞 评论 复制链接分享
  • weixin_39824529 weixin_39824529 2021-01-12 12:32

    Any update on this?

    点赞 评论 复制链接分享
  • weixin_39672443 weixin_39672443 2021-01-12 12:32

    I'm not opposed to this feature in general but I do have opinions on how it should be implemented. I'll also be upfront and say that we (HashiCorp) are unlikely to get to this feature soon, but gladly welcome contributions if someone can pick it up.

    As for how to implement:

    • The DescribeAutoScalingGroups endpoint returns a list of ASGs (we query by the specific name, so we get just the one, if it exists)
    • The AutoScalingGroup has a attribute that lists it's Instances
    • That list contains Instance information for each instance, but it is incomplete information. Instance IP Addresses (public or private) are not included here

    So the problem here in short is that the IP Addresses are not returned by default and you need to do a handful of additional API calls to get them. For large setups, these extra API calls add up quickly, so by default, we should not include Instance IP Addresses in the state of AutoScaling Groups. I recognize it's a useful feature but also think it will not be an often used one, so by default we shouldn't consume all these API calls.

    That said, I believe/propose we can support this functionality with a data source:

    • data.aws_autoscaling_group.group (singular) not to be confused with data.aws_autoscaling_groups (plural)
    • this data source should provide more information about the specific ASG (as opposed to data.aws_autoscaling_groups which just returns names)
    • should provide and optional, though default false, attribute like get_instance_properties to instruct the data source to look up instance attributes:
    • information on Instances we get from DescribeAutoScalingGroup > Instances
    • information returned from DescribeInstances

    Alternatively we could expand the data.aws_autoscaling_groups data source, but I think I'd prefer the separation.

    To get the IP address(es) for the Instance(s) we need to call DescribeInstances for each one. Fortunately we can use an EC2Filter and filter by requester-id, in this case, the AutoScaling Group Id, so we don't have to group up the instance-ids from the DescribeASG output and then feed that into the DescribeInstances call.

    I believe this new data source would keep this instance information in a TypeSet of Instances. Unfortunately you can't directly reference the value in a set with things like element(set, count). So if you want a list of ip addresses, we may want to take the results of the DescribeInstances call and aggregate all the public ips into a calculated field public_ips , and private ips into private_ips.

    So in the end the out put of this data source would be like this:

    • asg info
    • set of instances
    • list of public ips
    • list of private ips

    How does this sound?

    ```hcl resource "autoscaling_group" "example" { [...] }

    data "autoscaling_group" "example_ips" { autoscaling_id = "${aws_autoscaling_group.example.id}" # Fetch Instance information # Retrieves instance information # Retrieves public_ip's # Retrieves private_ip's get_instance_properties = true }

    resource "aws_route53_record" "service-record" { zone_id = "" count = 2 name = "service-lb-${format("%03d", count.index + 1)}.mydomain.com" type = "A" ttl = "60" records = ["${element(data.aws_autoscaling_group.example_ips.private_ips, count.index)}"] }

    点赞 评论 复制链接分享
  • weixin_39734493 weixin_39734493 2021-01-12 12:32

    I love this and it would help a lot.

    点赞 评论 复制链接分享
  • weixin_39824529 weixin_39824529 2021-01-12 12:32

    beautiful!

    点赞 评论 复制链接分享
  • weixin_39527163 weixin_39527163 2021-01-12 12:32

    any update on this, in real need of this solution.

    点赞 评论 复制链接分享
  • weixin_39789206 weixin_39789206 2021-01-12 12:32

    Has this been looked at? a complete stopper on what im required to do.

    点赞 评论 复制链接分享
  • weixin_39626180 weixin_39626180 2021-01-12 12:32

    any update on this?

    点赞 评论 复制链接分享
  • weixin_39948824 weixin_39948824 2021-01-12 12:32

    This would be amazing to have!

    点赞 评论 复制链接分享
  • weixin_39590453 weixin_39590453 2021-01-12 12:32

    This would be glorious! Since Lambda triggered by CloudWatchEvent is a mess!

    点赞 评论 复制链接分享
  • weixin_39958248 weixin_39958248 2021-01-12 12:32

    Need this one too

    点赞 评论 复制链接分享
  • weixin_39636609 weixin_39636609 2021-01-12 12:32

    Anyone find workaround or solution for this issue? This is completely show stopper for what I need to do.

    点赞 评论 复制链接分享
  • weixin_39628247 weixin_39628247 2021-01-12 12:32

    +1

    点赞 评论 复制链接分享
  • weixin_39612297 weixin_39612297 2021-01-12 12:32

    +1

    点赞 评论 复制链接分享
  • weixin_39761558 weixin_39761558 2021-01-12 12:32

    +1

    点赞 评论 复制链接分享
  • weixin_39722070 weixin_39722070 2021-01-12 12:32

    +1

    点赞 评论 复制链接分享
  • weixin_39600447 weixin_39600447 2021-01-12 12:32

    +1

    点赞 评论 复制链接分享
  • weixin_39760689 weixin_39760689 2021-01-12 12:32

    +1

    点赞 评论 复制链接分享
  • weixin_39927378 weixin_39927378 2021-01-12 12:32

    +1

    点赞 评论 复制链接分享
  • weixin_39775910 weixin_39775910 2021-01-12 12:32

    +1

    点赞 评论 复制链接分享
  • weixin_39700063 weixin_39700063 2021-01-12 12:32

    Here are some solutions that I implement:

    1. Use the module terraform-aws-modules/autoscaling/aws to launch my ASG's Tag the ASG and then use the aws_instance data source to search for the instances based on the tags. Then do what ever I need to do with the data

    2. When using an ASG for a bastion host. I create an iam_instance_profile for the ASG that allows the instance itself to update its own route53 record when it boots up using a launch config script.

    Hope this helps

    点赞 评论 复制链接分享
  • weixin_39747577 weixin_39747577 2021-01-12 12:32

    +1

    点赞 评论 复制链接分享
  • weixin_39769228 weixin_39769228 2021-01-12 12:32

    +1

    点赞 评论 复制链接分享
  • weixin_39925413 weixin_39925413 2021-01-12 12:32

    Is there currently any workaround available?

    点赞 评论 复制链接分享
  • weixin_39925413 weixin_39925413 2021-01-12 12:32

    This way allowed me to get ips of worker nodes:

    
    data "aws_instances" "workers" {
      instance_tags {
        Name = "lexsys-eks-asg"
      }
    }
    
    output "private-ips" {
      value = "${data.aws_instances.workers.private_ips}"
    }
    
    output "public-ips" {
      value = "${data.aws_instances.workers.public_ips}"
    }
    
    
    Outputs:
    
    private-ips = [
        10.0.0.75,
        10.0.1.87
    ]
    public-ips = [
        52.44.215.211,
        54.153.70.110
    ]
    

    Sure, this workaround has the limitations, but worked for me.

    点赞 评论 复制链接分享
  • weixin_39789979 weixin_39789979 2021-01-12 12:32

    One use case is for cluster setup. In such a scenario we want to be able to only provide the SSH keys of the instances to people who want to test a cluster like kafka, zookeeper, elasticsearch etc without having to give them access to the AWS console. If terraform could simply output the private ip address then the troubleshooters would have a list of IP addresses to log into simply with their SSH keys and without requiring access to AWS Console.

    点赞 评论 复制链接分享
  • weixin_39916758 weixin_39916758 2021-01-12 12:32

    +1

    点赞 评论 复制链接分享
  • weixin_39710951 weixin_39710951 2021-01-12 12:32

    Yeah another use case is when making a kubernetes cluster through an asg. you need the master's private ip to feed into the workers. Is there someone who knows a work around for this?

    点赞 评论 复制链接分享
  • weixin_39758032 weixin_39758032 2021-01-12 12:32

    +1

    点赞 评论 复制链接分享
  • weixin_39868592 weixin_39868592 2021-01-12 12:32

    +1

    点赞 评论 复制链接分享
  • weixin_39868663 weixin_39868663 2021-01-12 12:32

    +1. My use case involves Hashicorp Vault, which needs to be initialised before being recognised as a healthy node by the load balancer.

    点赞 评论 复制链接分享
  • weixin_39528289 weixin_39528289 2021-01-12 12:32

    +1

    点赞 评论 复制链接分享
  • weixin_39609770 weixin_39609770 2021-01-12 12:32

    Any workarounds for Google Cloud Platform? It has the same problem with google_compute_instance_group_manager.

    点赞 评论 复制链接分享
  • weixin_39824529 weixin_39824529 2021-01-12 12:32

    the question is specific for when instances are created via aws_autoscaling_group

    I'm afraid you'll need to wait patiently as the rest of us have (since Feb 2017) for this to be included in terraform, my prognoses would be around version 2.27 ... probably :-)

    点赞 评论 复制链接分享
  • weixin_39925413 weixin_39925413 2021-01-12 12:32

    I do use aws_autoscaling_group to create instances.

    I tag the instances in the template and use this tag to retrieve IPs in the data section.

    点赞 评论 复制链接分享
  • weixin_39824529 weixin_39824529 2021-01-12 12:32

    I see was not aware of that feature of data module.

    点赞 评论 复制链接分享
  • weixin_39572409 weixin_39572409 2021-01-12 12:32

    https://github.com/terraform-providers/terraform-provider-aws/issues/511#issuecomment-401362499 is a workaround but be carefull doc page

    Note: It's strongly discouraged to use this data source for querying ephemeral instances (e.g. managed via autoscaling group), as the output may change at any time and you'd need to re-run apply every time an instance comes up or dies.

    This note is other reason to implement this new data source

    点赞 评论 复制链接分享
  • weixin_39637614 weixin_39637614 2021-01-12 12:32

    you can pull all instance data with a round about lookup.

    ( example assuming some asg named aws_autoscaling_group.one is already defined )

    
    data "aws_instances" "nodes" {
      depends_on = [ "aws_autoscaling_group.one" ]
    
      instance_tags {
        Name = "some_unique_tag_in_your_asg"
      }
    }
    
    data "aws_instance" "asg-one-instances" {
      count = "${ var.asg_one_count }"
      depends_on = ["data.aws_instances.nodes"]
      instance_id = "${data.aws_instances.nodes.ids[count.index]}"
    }
    
    output "private-ips" {
      value = "${ data.aws_instance.asg-one-instances.*.private_ip }"
    }
    
    output "public-ips" {
      value = "${ data.aws_instance.asg-one-instances.*.public_ip }"
    }
    
    output "private-dnsnames" {
      value = "${ data.aws_instance.asg-one-instances.*.private_dns }"
    }
    
    output "public-dnsnames" {
      value = "${ data.aws_instance.asg-one-instances.*.public_dns }"
    }
    

    but I would much prefer to have the instance data directly available in the asg attributes.

    点赞 评论 复制链接分享
  • weixin_39824529 weixin_39824529 2021-01-12 12:32

    the problem I see with this is the asg_one_count variable. It is a static value which will/may not match the actual state of the ASG on consecutive runs, i.e. teh ASG has grown or shrunk in the mean time.

    点赞 评论 复制链接分享
  • weixin_39517054 weixin_39517054 2021-01-12 12:32

    it works to get the endpoints and private ips when you launch, which is what is mostly needed.

    Also, just set count = "${ var.asg_one_count }" to count = ${aws_autoscaling_group.one.desired_capacity}

    If people truly have need of dynamic ips then they should write it up in python, bash, go, etc and call it from terraform then use that instead of the desired capacity but I doubt it will really be necessary or desired in the long run.

    点赞 评论 复制链接分享
  • weixin_39802814 weixin_39802814 2021-01-12 12:32

    I started off with 's config and ended with a simpler workaround:

    hcl
    data "aws_instances" "ecs_instances_meta" {
      instance_tags = {
        # Use whatever name you have given to your instances
        Name = var.ecs_cluster_name
      }
    }
    
    output "ecs-private-ips" {
      value = data.aws_instances.ecs_instances_meta.private_ips
    }
    
    点赞 评论 复制链接分享

相关推荐