[Bug 1915345] [NEW] eic_harvest_hostkeys fails in local zones

[Robert C Jennings]

Public bug reported:

ec2-instance-connect breaks during host key harvesting for instances
launched in local zones[1].  Here are is the relevant debug data:

$ systemctl is-system-running 
degraded

$ systemctl list-units --failed
  UNIT                         LOAD   ACTIVE SUB    DESCRIPTION                             
● ec2-instance-connect.service loaded failed failed EC2 Instance Connect Host Key Harvesting


$ journalctl --unit ec2-instance-connect
-- Logs begin at Wed 2021-02-10 22:47:47 UTC, end at Wed 2021-02-10 22:55:46 UTC. --
Feb 10 22:48:16 ip-172-31-51-82 systemd[1]: Starting EC2 Instance Connect Host Key Harvesting...
Feb 10 22:48:16 ip-172-31-51-82 systemd[1]: ec2-instance-connect.service: Main process exited, code=exited, status=255/EXCEPTION
Feb 10 22:48:16 ip-172-31-51-82 systemd[1]: ec2-instance-connect.service: Failed with result 'exit-code'.
Feb 10 22:48:16 ip-172-31-51-82 systemd[1]: Failed to start EC2 Instance Connect Host Key Harvesting.

$ dpkg-query -l ec2-instance-connect
ii  ec2-instance-connect 1.1.13-0ubuntu1 all          Configures ssh daemon to accept EC2 Instance Connect ssh keys


$ lsb_release -c
Codename:	hirsute

$ cat /etc/cloud/build.info 
build_name: server
serial: 20210208

$ ec2metadata --availability-zone --ami-id
us-west-2-lax-1a
ami-098f71a7a25a0f1f2


$ bash -x /usr/share/ec2-instance-connect/eic_harvest_hostkeys
...
++ /usr/bin/curl -s -f -m 1 -H 'X-aws-ec2-metadata-token: AQAEAEvStI0Ugwz1C3GQh7oubFTah7bXQllCmFU6BtMI6b6l5zMkVQ==' http://169.254.169.254/latest/meta-data/placement/availability-zone/
+ zone=us-west-2-lax-1a
+ zone_exit=0
+ '[' 0 -ne 0 ']'
+ /bin/echo us-west-2-lax-1a
+ /bin/grep -Eq '^([a-z]+-){2,3}[0-9][a-z]$'
+ /usr/bin/head -n 1
+ exit 255

[1] https://aws.amazon.com/about-aws/global-infrastructure/localzones/

** Affects: ec2-instance-connect (Ubuntu)
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to ec2-instance-connect in Ubuntu.
https://bugs.launchpad.net/bugs/1915345

Title:
  eic_harvest_hostkeys fails in local zones

Status in ec2-instance-connect package in Ubuntu:
  New

Bug description:
  ec2-instance-connect breaks during host key harvesting for instances
  launched in local zones[1].  Here are is the relevant debug data:

  $ systemctl is-system-running 
  degraded

  $ systemctl list-units --failed
    UNIT                         LOAD   ACTIVE SUB    DESCRIPTION                             
  ● ec2-instance-connect.service loaded failed failed EC2 Instance Connect Host Key Harvesting

  
  $ journalctl --unit ec2-instance-connect
  -- Logs begin at Wed 2021-02-10 22:47:47 UTC, end at Wed 2021-02-10 22:55:46 UTC. --
  Feb 10 22:48:16 ip-172-31-51-82 systemd[1]: Starting EC2 Instance Connect Host Key Harvesting...
  Feb 10 22:48:16 ip-172-31-51-82 systemd[1]: ec2-instance-connect.service: Main process exited, code=exited, status=255/EXCEPTION
  Feb 10 22:48:16 ip-172-31-51-82 systemd[1]: ec2-instance-connect.service: Failed with result 'exit-code'.
  Feb 10 22:48:16 ip-172-31-51-82 systemd[1]: Failed to start EC2 Instance Connect Host Key Harvesting.

  $ dpkg-query -l ec2-instance-connect
  ii  ec2-instance-connect 1.1.13-0ubuntu1 all          Configures ssh daemon to accept EC2 Instance Connect ssh keys

  
  $ lsb_release -c
  Codename:	hirsute

  $ cat /etc/cloud/build.info 
  build_name: server
  serial: 20210208

  $ ec2metadata --availability-zone --ami-id
  us-west-2-lax-1a
  ami-098f71a7a25a0f1f2

  
  $ bash -x /usr/share/ec2-instance-connect/eic_harvest_hostkeys
  ...
  ++ /usr/bin/curl -s -f -m 1 -H 'X-aws-ec2-metadata-token: AQAEAEvStI0Ugwz1C3GQh7oubFTah7bXQllCmFU6BtMI6b6l5zMkVQ==' http://169.254.169.254/latest/meta-data/placement/availability-zone/
  + zone=us-west-2-lax-1a
  + zone_exit=0
  + '[' 0 -ne 0 ']'
  + /bin/echo us-west-2-lax-1a
  + /bin/grep -Eq '^([a-z]+-){2,3}[0-9][a-z]$'
  + /usr/bin/head -n 1
  + exit 255

  [1] https://aws.amazon.com/about-aws/global-infrastructure/localzones/

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ec2-instance-connect/+bug/1915345/+subscriptions