Transplanting Ubuntu 24.04.1 server to new hardware, best approach?

Mon Jan 20 21:10:43 UTC 2025

LONG reply below....

On Mon, 20 Jan 2025 15:40:11 +0000, Liam Proven <lproven at gmail.com> wrote:

>On Mon, 20 Jan 2025 at 12:55, Bo Berglund <bo.berglund at gmail.com> wrote:
>
>> The upgrade process seems to have worked but has not solved my problems with the
>> grub boot menu, which does not update reflecting the new preferred kernel so it
>> starts with an older kernel all the time. :-(
>
>I've watched your struggles with this but I do not claim to remember
>all the steps.

Well I was on this at the end of November 2024, but in beginning of Dec there
was a power outage at the company that killed a RAID server system hosting our
VMWARE ESX virtual servers, and I have been struggling with recovering the
lost data from among others the server I am discussing here, which acts like
amirror to the company Subversion server.
And there was other data that were lost and had to be recreated too. :(

It took past XMas for the IT guys to restore to the state on Dec 5 2024....
And during that time I had to keep watch on the server I want to move.

>> The grub problem comes probably from the fact that this box came with Windows
>> and the grub was created when I initially installed Ubuntu Desktop on it as
>> multi-boot with Windows (which I don't ever use) and I guess there is a problem
>> there.
>
>Not enough info to say.
>I do not remember all your troubleshooting.
>I think it's a remote box and you can't see the GRUB menu, correct?

No, it is a Lenovo IdeaCenter mini-tower under my desk....

>Is it UEFI?

Yes, there is an UEFI partition and Windows is involved with that and the BIOS
access
too, which has bothered me a lot over the years.
For instance I want to make the Lenovo to start up as soon as power is applied
in
order to avoid what happens when there is a power outage and someone has to
press
the power button to start it again.

But to reach BIOS I have to boot up and log onto Windows and go from there into 
some hoops to finally get the BIOS settings...
I haven't dared doing this after the last power outage.

>I think that the copy of Ubuntu you are connecting to and working with
>is not the copy of Ubuntu that is in charge of the boot loader.

I believe that the Desktop Ubuntu, which is still there but now no longer on the
grub start menu, is the one that originally managed the grub dual boot system.

>Can you post a full list of all disks, their mount points, and their
>UUIDs, please?
>
>So the output from these commands:
>
>sudo -s
>sfdisk -l
>blkid
>lsblk
>mount

Here you go, but I exclude all of these loop# items:

# sfdisk -l
(---Loop output removed---)
Disk /dev/nvme0n1: 476.94 GiB, 512110190592 bytes, 1000215216 sectors
Disk model: SAMSUNG MZALQ512HALU-000L1
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: A4DBAC89-9420-4410-8278-EA159A472A94

Device              Start        End   Sectors  Size Type
/dev/nvme0n1p1       2048     534527    532480  260M EFI System
/dev/nvme0n1p2     534528     567295     32768   16M Microsoft reserved
/dev/nvme0n1p3     567296  126396415 125829120   60G Microsoft basic data
/dev/nvme0n1p4  998166528 1000214527   2048000 1000M Windows recovery
environment
/dev/nvme0n1p5  126396416  189310975  62914560   30G Linux filesystem
/dev/nvme0n1p6  189310976  252225535  62914560   30G Linux filesystem
/dev/nvme0n1p7  252225536  419997695 167772160   80G Linux filesystem
/dev/nvme0n1p8  419997696  973668351 553670656  264G Linux filesystem
/dev/nvme0n1p9  973668352  977334271   3665920  1.7G Linux swap
/dev/nvme0n1p10 977334272  979431423   2097152    1G Linux filesystem
(---Loop output removed---)
Partition table entries are not in disk order.

# blkid
/dev/nvme0n1p9: UUID="a77b40db-2377-4d25-b304-8d233664d1ca" TYPE="swap"
PARTUUID="73b9b0f7-a718-4d61-a4e1-82a7b721f3f1"
/dev/nvme0n1p7: LABEL="dataspace" UUID="fb4a08b7-378e-42eb-9b7a-2c7b4f85cd06"
BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="d750c587-3325-4198-aa6a-faf96cf6e8de"
/dev/nvme0n1p5: LABEL="ubuntu20.04" UUID="3d88bb41-a2e7-43ae-acba-24c06cd7230a"
BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="312eff35-570c-4bf9-af35-d0bde35e1329"
/dev/nvme0n1p3: LABEL="Windows" BLOCK_SIZE="512" UUID="3810C0E610C0AC66"
TYPE="ntfs" PARTLABEL="Basic data partition"
PARTUUID="eead212a-1a29-473b-8dab-06c8b428f229"
/dev/nvme0n1p1: LABEL_FATBOOT="SYSTEM" LABEL="SYSTEM" UUID="42BE-49BB"
BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="EFI system partition"
PARTUUID="e4c0c063-a423-40de-b039-3a95ac2bf958"
/dev/nvme0n1p8: LABEL="home" UUID="389c6db9-3256-47ed-b67f-c11c76eed17f"
BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="c1c73f4e-3059-42fd-879b-a1c8a55f3bf3"
/dev/nvme0n1p10: LABEL="GPartedLive" UUID="d6b92d15-bdd8-492a-9391-002f20c842ce"
BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="09839fa2-1034-42a4-beca-84e5c7b57ccd"
/dev/nvme0n1p6: LABEL="server20.04.3"
UUID="ec0e8708-8a6a-4bbf-93ba-0a09b1e2ddc1" BLOCK_SIZE="4096" TYPE="ext4"
PARTUUID="5049c993-2498-43a1-aabc-b55afd4592e8"
/dev/nvme0n1p4: LABEL="WinRE_DRV" BLOCK_SIZE="512" UUID="DA8EC18E8EC1641B"
TYPE="ntfs" PARTLABEL="Basic data partition"
PARTUUID="c34d579d-3fdb-4de5-83ba-bf7043ef587d"
/dev/loop1: BLOCK_SIZE="131072" TYPE="squashfs"
/dev/nvme0n1p2: PARTLABEL="Microsoft reserved partition"
PARTUUID="cc4279d3-6656-4e29-9b8e-71e791ff376e"
(---Loop output removed---)

# lsblk
NAME         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
(---Loop output removed---)
nvme0n1      259:0    0 476.9G  0 disk
+-nvme0n1p1  259:1    0   260M  0 part /boot/efi
+-nvme0n1p2  259:2    0    16M  0 part
+-nvme0n1p3  259:3    0    60G  0 part
+-nvme0n1p4  259:4    0  1000M  0 part
+-nvme0n1p5  259:5    0    30G  0 part
+-nvme0n1p6  259:6    0    30G  0 part /
+-nvme0n1p7  259:7    0    80G  0 part /var/lib/svn
+-nvme0n1p8  259:8    0   264G  0 part /home
+-nvme0n1p9  259:9    0   1.7G  0 part [SWAP]
+-nvme0n1p10 259:10   0     1G  0 part

#mount    (----This command spews out a *lot* of text---)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs
(rw,nosuid,relatime,size=7080536k,nr_inodes=1770134,mode=755,inode64)
devpts on /dev/pts type devpts
(rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs
(rw,nosuid,nodev,noexec,relatime,size=1423752k,mode=755,inode64)
efivarfs on /sys/firmware/efi/efivars type efivarfs
(rw,nosuid,nodev,noexec,relatime)
/dev/nvme0n1p6 on / type ext4 (rw,relatime,errors=remount-ro)
securityfs on /sys/kernel/security type securityfs
(rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,inode64)
tmpfs on /run/lock type tmpfs
(rw,nosuid,nodev,noexec,relatime,size=5120k,inode64)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs
(rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17295)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs
(rw,nosuid,nodev,relatime,pagesize=2M)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime)
tracefs on /sys/kernel/tracing type tracefs (rw,nosuid,nodev,noexec,relatime)
nfsd on /proc/fs/nfsd type nfsd (rw,relatime)
fusectl on /sys/fs/fuse/connections type fusectl
(rw,nosuid,nodev,noexec,relatime)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
(---snaps output removed---)
/dev/nvme0n1p8 on /home type ext4 (rw,nosuid,nodev,relatime)
/dev/nvme0n1p1 on /boot/efi type vfat
(rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro)
/dev/nvme0n1p7 on /var/lib/svn type ext4 (rw,nosuid,nodev,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc
(rw,nosuid,nodev,noexec,relatime)
sunrpc on /run/rpc_pipefs type rpc_pipefs (rw,relatime)
lxcfs on /var/lib/lxcfs type fuse.lxcfs
(rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
(---snaps output removed---)
(---my external network mounts removed
tmpfs on /run/user/1000 type tmpfs
(rw,nosuid,nodev,relatime,size=1423748k,nr_inodes=355937,mode=700,uid=1000,gid=1000,inode64)

>> Once that was done I could update grub using the desktop ubuntu and then boot to
>> the server which is the main system for it. At that time both Ubuntus and
>> Windows appeared in the grub menu.
>
>Key question missing here: whose GRUB?

There seems to be one single grub menu which is showed when the device
reboots...
It changes depending on what kernel grub finds when doing the config.
It seems to look everywhere on the disk to find what it is after....

>There is 1 copy of GRUB per copy of Linux. (Setting aside distros that
>use other bootloaders and things.)
>
>You can run `update-grub` as often as you like but if a different copy
>of Ubuntu is in charge of the boot sector then that GRUB never runs.

That is exactly my observation and I am wondering what the update-grub 
really does on the other operating systems then?
It seems like it can list all of the bootable partitions and also read
what kernels are available there....

>If it is an UEFI machine then GRUB is in the ESP.
>That is the UEFI System Partition.

Yes, and that is nvme0n1p1 in this case.
It is mounted to /boot so I have been able to look into it, but I don't
know enough to do anything but list/read files etc.

>Is there any chance you have more than 1 ESP?

No, there is one single such partition

>There should only be 1 on the main boot drive.
>It should be mounted at `/boot/efi/`

Like it is....

>You need the copy of GRUB that's in your server OS to be in control of
>the boot menu.
>
>I think the old copy of GRUB in the desktop OS is the current one.

At one time in my previous life (before Dec 6 2024) I rebooted and had a monitor
and keyboard attached so I could interact. Then I selected the desktop OS to 
boot and it started doing that but somehow the desktop never appeared.
Instead I got a black screen only.
At that time I tried to use PuTTY to SSH into it and that worked after
I selected the PuTTY SSH entry for the desktop.
And while I was connected there I could do an update-grub, which showed
that it could see the server Ubuntu.

After I next rebooted I got back to the server and it no longer used the
outdated kernel.

But I could no longer find the Desktop on the boot menu when I tried to get 
back there.

>There are reinstall instructions here:
>
>https://askubuntu.com/questions/831216/how-can-i-reinstall-grub-to-the-efi-partition
>
>Basically, it is the same as ever:
>
>DO NOT DO THIS YET. READ IT ALL FIRST.
>
>To install GRUB you do:
>
>````
>sudo -s
>grub-install /dev/sdX
>````
>
>where sdX is the main device node of your boot drive.
>
>*BUT* with UEFI you _must_ have the ESP mounted first, and some other
>stuff. It's not as easy as in the old BIOS days. That's why you and we
>_must_ know if it's BIOS or UEFI and which ESP and which OS.

It is definitely UEFI.

ESP is on partition nvme0n1p1 named FATBOOT

Operating systems are on:

nvme0n1p3 ("Windows")        Never used except to get to BIOS
nvme0n1p5 ("ubuntu20.04")    Desktop 20.04
nvme0n1p6 ("server20.04.3")  Server, now at 24.04.1 distupgraded twice
nvme0n1p10 ("GPartedLive")   Gparted for use from the boot menu

See the partitions and mount points:
$ sudo lsblk -o UUID,NAME,FSTYPE,SIZE,MOUNTPOINT,LABEL
UUID                     NAME         FSTYPE   SIZE MOUNTPOINT   LABEL
42BE-49BB                +-nvme0n1p1  vfat     260M /boot/efi    SYSTEM
                         +-nvme0n1p2            16M
3810C0E610C0AC66         +-nvme0n1p3  ntfs      60G              Windows
DA8EC18E8EC1641B         +-nvme0n1p4  ntfs    1000M              WinRE_DRV
3d88bb41-a2e7-43ae-acba- +-nvme0n1p5  ext4      30G              ubuntu20.04
ec0e8708-8a6a-4bbf-93ba- +-nvme0n1p6  ext4      30G /            server20.04.3
fb4a08b7-378e-42eb-9b7a- +-nvme0n1p7  ext4      80G /var/lib/svn dataspace
389c6db9-3256-47ed-b67f- +-nvme0n1p8  ext4     264G /home        home
a77b40db-2377-4d25-b304- +-nvme0n1p9  swap     1.7G [SWAP]
d6b92d15-bdd8-492a-9391- +-nvme0n1p10 ext4       1G              GPartedLive

I can mount the Ubuntu Desktop drive nvme0n1p5 and have a look around....

$ sudo ls -la  /mnt/ubuntudsk/boot/efi/
total 24
drwx------ 6 root root 4096 Jan  1  1970  .
drwxr-xr-x 5 root root 4096 Dec 26 15:17  ..
drwx------ 2 root root 4096 Jan 23  2021 '$RECYCLE.BIN'
drwx------ 2 root root 4096 Jan 23  2021  BOOT
drwx------ 5 root root 4096 Oct 19  2021  EFI
drwx------ 2 root root 4096 Jan 23  2021 'System Volume Information'
Looks like Microsoft was there (System Volume Information)

And:

$ sudo ls -la  /mnt/ubuntudsk/boot/efi/EFI/
total 20
drwx------ 5 root root 4096 Oct 19  2021 .
drwx------ 6 root root 4096 Jan  1  1970 ..
drwx------ 2 root root 4096 Oct 19  2021 Boot
drwx------ 4 root root 4096 Jan 23  2021 Microsoft
drwx------ 2 root root 4096 Oct 19  2021 ubuntu

etc....
So it is accessible even if it cannot boot.

>> It seems like the only OS that can properly update grub is the Desktop Ubuntu,
>> but it is no longer appearing in the boot menu so it can not be started...
>
>I can't imagine how but the world is strange and I often encounter
>things I can't imagine. :-)
>
>> Now some years later I am planning to repeat the transplant onto yet another
>> hardware platform, this time an ASUS NUC 13 PRO with Core i7 processor and SSD
>> drives etc. No OS at all right now.
>
>Uhoh.
>
>Bad plan. You don't know what is wrong.
>
>I _STRONGLY_ advise fixing this FIRST!
>
>Recommendation:
>
>1. Find what the boot problem is.
>2. Fix the boot problem.
>3. Get booting under control of the desired OS.
>4. Learn how to install _and reinstall_ and modify GRUB.
>5. Copy the OS onto blank new machine.

If I do #5 using GParted then what happens on boot?
Will the single operating system partition start up or is as I believe a grub
start menu required?
And a populated EFI partition etc?
That is why I thought of installing a desktop version first to get the EFI &
boot handled by that installer.

>6. Install fresh copy of GRUB.
>
>Don't mess around with desktop versions or unneeded dual booting.

OK, I thought that I *needed* to install fresh using an Ubuntu in order for it
to fix the start menu on the totally empty disk by creating the EFI partition
and populating the same..

I don't need a desktop system on this device anyway, it was in the plan just
because of the creation of a boot menu where the server could be added.

So:
If I copy just the partitions used by the server over to the new drive will 
it then be able to boot or do I have to create an EFI partition and how is
that populated in that single-boot case?

My end state for this is to have my existing server running on new hardware
preferably all alone on the drive except for the data partitions holding
the Subversion repositories and video library etc.

>Liam Proven ~ Profile: https://about.me/liamproven
>Email: lproven at cix.co.uk ~ gMail/gTalk/FB: lproven at gmail.com
>Twitter/LinkedIn: lproven ~ Skype: liamproven
>IoM: (+44) 7624 227612: UK: (+44) 7939-087884
>Czech [+ WhatsApp/Telegram/Signal]: (+420) 702-829-053

-- 
Bo Berglund
Developer in Sweden