[Bug 1882986] Re: open-iscsi is slowing down the boot process

Mon Jun 15 21:08:55 UTC 2020

Many thanks for the explanations Christian!

Here are some more clarification on my setup.

Indeed, I have done a "discover" for my NAS, and I have a node configured.
The other element is that the NAS is generally off, which means if open-iscsci would try to communicate with the NAS at startup, that could timeout.

What I don't yet fully understand with isci, is that when I need my
NAS's LUN, since I have once and for all "discovered" it, I just run

$ sudo iscsiadm -m node -l

First, this works with or without the 3 services at startup... not sure
it should, and then what is the use of those services (at least in my
case!)

But also unlike the same open-iscsi the command line does NOT return...
which does not prevent the remote mount to work perfectly fine.

When I'm done with the mount, I just unmount it, do CTRL-C on the
command line (which was not necessary in 16.04), and do

$ sudo iscsiadm -m node -u

What I also don't get from reading the man is the difference in what you explain:

$ sudo iscsiadm -m node -l
Logging in to [iface: default, target: iqn.2000-01.com.synology:diskstation.blocks, portal: 192.168.0.100,3260] (multiple)

And the command you quote from open-iscsi says:

$ sudo iscsiadm -m node --loginall=automatic
iscsiadm: No records found

Is it because my nodes configuration has a "manual" somewhere:

$ sudo cat /etc/iscsi/nodes/iqn.2000-01.com.synology:diskstation.blocks/192.168.0.100,3260,0/default
# BEGIN RECORD 2.0-874
node.name = iqn.2000-01.com.synology:diskstation.blocks
node.tpgt = 0
node.startup = manual
....

I didn't knowingly put it there, it is apparently the default value when
issuing the "discovery" command:

$ sudo iscsiadm --mode discovery --op update --type sendtargets --portal
192.168.0.100

Now from your explanations, I tried with only the two parts related to
iscsi (without open-isci) and I have the same behaviour.

I guess my thinking was right. The logic you explain is that iscsi
believes nodes are needed for the startup of the machine (when he finds
some) and then waits for the network to become ready (at least) and
possibly more to ping the nodes (?)

I don't think iscsi by itself takes a lot of time, or even that there is
a timeout with my NAS that is not powered on, it is just because you
need to wait for the network that all the "graphical" process is
delayed.

Two proofs of that.
I have a fuse mount of my own (1fichierfs : https://gitlab.com/BylonAkila/astreamfs) that runs at session start, since you want to run fuse mounts for the user, and not as root.
For the 20.04, and also for Raspberry OS that does the same "trick", I now introduced an optional "wait for network" feature in the mount itself.

Here is what I get on the log
[1fichierfs     0.000] NOTICE: started: Monday 15 June 2020 at 22:22:36
[1fichierfs     0.000] INFO: successfuly parsed arguments.
[1fichierfs     0.000] INFO: log level is 7.
[1fichierfs     0.000] INFO: user_agent=1fichierfs/1.7.1.1
[1fichierfs     0.008] INFO: <<< API(in) (iReq:1) folder/ls.cgi POST={"folder_id":0,"files":1} name=/
[1fichierfs     8.071] NOTICE: Waited 8 seconds for network at startup.

You see, it said it waited 8 seconds, after when "programs at session start" are kicked in.
It can also be less, sometimes 6 seconds, sometimes 2.

Second exhibit is the SVG graph with and without iscsi (only the 2 iscsi related parts).

What you want to look at is the red line saying "Plymouth-quit-
wait.service" which is, as it says, when plymouth exits and the user
starts seeing the desktop.

As you can see after the "plymouth-quit-wait", just after gdm, is a
predecessor to the user-1000.slice which I guess starts what we commonly
call the user session.

You can observe in the first graph that the "network-online" target
occurs about 7 to 8 seconds after the start of user-1000.slice, which is
coherent with my 1fichierfs log.

But at the point where the network is finally online, almost all is
ready apart thinks like openvpn which obviously need the network to be
online to start clients, and some others.

In the second graph "with iscsi", you can see that after unattended-upgraded and snapd service, nothing happens, and we simply sleep waiting for the network, because:
- iscsi wants it
- and (probably) iscsi said (I couldn't find how) that it is needed before starting users, and now you see that gdm + plymouth-quit-wait are started AFTER iscsi, which is here after the network.

So there we also waited for about 7 to 8 seconds, but a lot of services that could be started in the "normal" run without waiting for the network are now "not started yet", which means with have lost more due to not taking the opportunity to run tasks in parallel.

Of course, if this is desktop machine with "auto-login". Would you not be in "auto-login", you probably barely notice a difference unless you a very quick to type your password.

** Attachment added: "Startup Graph with NO iscsi"
   https://bugs.launchpad.net/ubuntu/+source/open-iscsi/+bug/1882986/+attachment/5384159/+files/startup_noiscsi.svg

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to open-iscsi in Ubuntu.
https://bugs.launchpad.net/bugs/1882986

Title:
  open-iscsi is slowing down the boot process

Status in open-iscsi package in Ubuntu:
  Triaged

Bug description:
  This is not a bug, but rather an "optimisation" request.

  (Probably set this as "enhancement request" + Low)

  Apparently, the package is assuming the user will need some iscsi
  mounts for his session, and is putting dependencies in the systemd
  services/targets which effects are to delay "graphical.target" to
  after the point when the network is online.

  A great job has been done by Ubuntu so that the O.S. appears to be
  "snappy" from the boot, and when the session is in "auto-login", it
  really makes a great difference and a good feeling of the system being
  very quick.

  This assumption of open-iscsi sort of ruins that effort.

  As an example, on my PC the graphical target is delayed 10 seconds
  more (was 22 seconds and is now 32). The impression is not as good and
  the system feels "slow again" (although it is just a feeling!)

  Step to reproduced (you don't even need to have iscsi LUNs to to so, just install the package!)
  - Start from a clean 20.04, boot up and issue: systemd-analyze
  - Now install open-iscsi, reboot and issue again: systemd-analyze

  The result will probably be a big impact on "graphical target",
  although total time does not change a lot.

  My usage is not needing iscsi targets for my session.
  I have a NAS with iscsi LUNs, and when I need those mounts, I just start them with a command.

  sudo iscsiadm -m node -l

  Then Gnome recognises a new disk has been inserted and does an "auto mount".
  This command works whether the service was started or not.

  This wrong assumption is easily fixed in my case with this command:

  sudo systemctl disable iscsid.socket iscsid.service open-iscsi.service

  Then, at the next reboot the graphical target is snappy again, and does not have to wait for network-online and remote-fs targets.

  I don't know what can be done to cope with both situations : those who
  need an iscsi target mounted for their session, and those who don't...
  but I guess the philosophy now should be to assume the user does not
  need such targets, and don't put dependencies that delay the snappy
  boot process.

  For those who need those mounted remote fs for their session, detailed
  help on how to enable iscsi services at startup should be provided.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/open-iscsi/+bug/1882986/+subscriptions