Messed up copying ssh pub key to server?

[Robert Moskowitz]

Oops....

I found the guide to do:

ssh-copy-id -i ~/.ssh/id_rsa.pub username at ubuntu_server_ip

in my .ssh I have:

$ ls .ssh/
config      id_ed25519.pub  id_rsa.pub   known_hosts.old
id_ed25519  id_rsa          known_hosts

so I did:

ssh-copy-id -i ~/.ssh/id_ed25519.pub me at host.htt-consult.com

I then tried

ssh host

I get:

Enter passphrase for key '/home/me/.ssh/id_ed25519':

I press enter and then get

me at host's password:

I enter my password and get it.  How do I get rid of that "Enter 
passphrase"  And why is it still asking for my password?