[Bug 2069125] Re: [SRU] Manila driver error with ONTAP SVM-scoped user
Mauricio Faria de Oliveira
2069125 at bugs.launchpad.net
Tue Oct 8 21:42:37 UTC 2024
Hey Rodrigo o/
Thanks for the adjusted debdiffs, I'll take a look. However, for now:
Comment #32:
> About the test steps, I mentioned in item (1) that numerous contributors
> and driver maintainers have validated the fix internally and confirmed
> that it worked without issues. Unfortunately that's the best we have.
Yes, I was aware about item 1 in [Test Plan] when I wrote comment #28,
but I didn't think that was the end of options thus suggested an idea:
Comment #28:
> For `Test Steps`, I can appreciate the limitations.
> However, we'd need at least a minimal way to verify the code change,
> even if that is synthetically or with a mocked up call
> (AFAICT/IIUIC the code change does look correct and safe).
"""
I also appreciate the existing validation by contributors/drive maintainers
is 'the best we have', but is that really 'the best we _could_ have'?
Apparently, we _could_ do something.
Is it _not possible_ to do at least _a mocked up test_ for this code path?
That seems to be standard practice in test suites of openstack components.
>From what I can look at the patch, one could mock `send_request` to return
normally or trigger the expected exception (with/without the right message),
and check that the caller returns or reraises as expected in the 3 cases.
Right? If that is not possible, please clarify the reason, as I understand
this SRU is needed, so we'd need to figure tests out so it can be released.
It's difficult to SRU a change that can't be _minimally_ tested in _any way_,
or even regression tested in indirect ways (to at least check it does not
break other things -- that is a valid test too, when direct tests are hard).
I'll continue to review and hopefully upload the changes, and it might even
be accepted into -proposed depending on the reviewer/discussion, but without
some way to test this, I'd imagine it would be gated at release to -updates.
Thanks again for all your work on this.
** Changed in: manila (Ubuntu Noble)
Status: New => Incomplete
** Changed in: manila (Ubuntu Jammy)
Status: New => Incomplete
** Changed in: cloud-archive/bobcat
Status: New => Incomplete
--
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/2069125
Title:
[SRU] Manila driver error with ONTAP SVM-scoped user
Status in Ubuntu Cloud Archive:
New
Status in Ubuntu Cloud Archive antelope series:
Fix Released
Status in Ubuntu Cloud Archive bobcat series:
Incomplete
Status in Ubuntu Cloud Archive caracal series:
New
Status in Ubuntu Cloud Archive yoga series:
Fix Committed
Status in Ubuntu Cloud Archive zed series:
Won't Fix
Status in OpenStack Shared File Systems Service (Manila):
Fix Released
Status in manila package in Ubuntu:
Fix Released
Status in manila source package in Jammy:
Incomplete
Status in manila source package in Mantic:
Won't Fix
Status in manila source package in Noble:
Incomplete
Bug description:
************** SRU DESCRIPTION AT THE BOTTOM *************
With the same NetApp stanza in the manila.conf file which was used without any issue in the Zed release was used in the Bobcat release. In the Bobcat release, the share creation worked normally, but adding access rule was not worked and couldn't delete the share. Below is the error log that occurs when adding a rule.
I set all the roles indicated in NetApp's OpenStack operation guide in storage side(https://netapp-openstack-dev.github.io/openstack-docs/bobcat/manila/configuration/ontap_configuration/section_ontap-config.html#ontap-prerequisites).
########### manila-share.log ############
2024-05-27 15:43:14.708 19 INFO oslo.messaging.notification.share.create.end [None req-4b46bc06-9332-40f3-9ef0-57895519228c c2e47ee4c8295d950db5757f73dfe9b5149947ccf5dc4e4ba3370c210217bcc4 76a637a88d624e3ea80b261a4c66dc2a - - - -] {"message_id": "d9fcc12a-5449-437c-85a0-eb5bdddab553", "publisher_id": "share.dc1-infra-rnd-stack-ctrl-01 at c400", "event_type": "share.create.end", "priority": "INFO", "payload": {"share_id": "68e79de3-5e22-472b-a895-c79e0b677b01", "user_id": "c2e47ee4c8295d950db5757f73dfe9b5149947ccf5dc4e4ba3370c210217bcc4", "project_id": "76a637a88d624e3ea80b261a4c66dc2a", "snapshot_id": null, "share_group_id": null, "size": 20, "name": "asdasd", "description": null, "proto": "NFS", "is_public": true, "availability_zone": null, "host": "dc1-infra-rnd-stack-ctrl-01 at c400#N1_Data", "status": "creating", "share_type_id": "40cdd81c-1fa8-4fc6-8f5e-288d0b9f5430", "share_type": "NFS_VOLUME"}, "timestamp": "2024-05-27 06:43:14.708153"}
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server [None req-be5d1bf0-c013-47ac-94bc-2de599a3862f c2e47ee4c8295d950db5757f73dfe9b5149947ccf5dc4e4ba3370c210217bcc4 76a637a88d624e3ea80b261a4c66dc2a - - - -] Exception during message handling: manila.share.drivers.netapp.dataontap.client.api.NaApiError: NetApp API failed. Reason - 15661:entry doesn't exist
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/manila/share/manager.py", line 236, in wrapped
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server return f(self, *args, **kwargs)
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/manila/utils.py", line 481, in wrapper
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server return func(self, *args, **kwargs)
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/manila/share/manager.py", line 4177, in update_access
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server self.update_access_for_instances(context, [share_instance_id],
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/manila/share/manager.py", line 4191, in update_access_for_instances
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server self.access_helper.update_access_rules(
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/manila/share/access.py", line 299, in update_access_rules
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server self._update_access_rules(context, share_instance_id,
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/manila/share/access.py", line 336, in _update_access_rules
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server driver_rule_updates = self._update_rules_through_share_driver(
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/manila/share/access.py", line 401, in _update_rules_through_share_driver
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server driver_rule_updates = self.driver.update_access(
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/manila/share/drivers/netapp/dataontap/cluster_mode/drv_single_svm.py", line 103, in update_access
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server self.library.update_access(context, share, access_rules, add_rules,
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/manila/share/drivers/netapp/utils.py", line 115, in trace_wrapper
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server result = f(self, *args, **kwargs)
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/manila/share/drivers/netapp/dataontap/cluster_mode/lib_base.py", line 2355, in update_access
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server helper.update_access(share, share_name, access_rules)
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/manila/share/drivers/netapp/utils.py", line 115, in trace_wrapper
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server result = f(self, *args, **kwargs)
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/manila/share/drivers/netapp/dataontap/protocols/base.py", line 34, in wrapped_func
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server return source_func(self, *args, **kwargs)
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_concurrency/lockutils.py", line 414, in inner
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server return f(*args, **kwargs)
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/manila/share/drivers/netapp/dataontap/protocols/base.py", line 32, in source_func
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server return f(self, *args, **kwargs)
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/manila/share/drivers/netapp/dataontap/protocols/nfs_cmode.py", line 114, in update_access
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server auth_methods = self._get_auth_methods()
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/manila/share/drivers/netapp/utils.py", line 115, in trace_wrapper
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server result = f(self, *args, **kwargs)
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/manila/share/drivers/netapp/dataontap/protocols/nfs_cmode.py", line 221, in _get_auth_methods
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server kerberos_enabled = self._client.is_kerberos_enabled()
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/manila/share/drivers/netapp/utils.py", line 115, in trace_wrapper
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server result = f(self, *args, **kwargs)
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/manila/share/drivers/netapp/dataontap/client/client_cmode.py", line 2042, in is_kerberos_enabled
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server result = self.send_request('kerberos-config-get', api_args)
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/manila/share/drivers/netapp/dataontap/client/client_base.py", line 89, in send_request
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server return self.connection.invoke_successfully(
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/manila/share/drivers/netapp/dataontap/client/api.py", line 717, in invoke_successfully
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server return self.get_client(use_zapi=use_zapi).invoke_successfully(
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib64/python3.9/site-packages/manila/share/drivers/netapp/dataontap/client/api.py", line 388, in invoke_successfully
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server raise NaApiError(code, msg)
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server manila.share.drivers.netapp.dataontap.client.api.NaApiError: NetApp API failed. Reason - 15661:entry doesn't exist
2024-05-27 15:43:57.077 19 ERROR oslo_messaging.rpc.server
2024-05-27 15:44:08.487 19 INFO manila.share.manager [None req-4bfe58a9-a794-497d-8b75-7ee098ea0e11 - - - - - -] Updating share status
===============
SRU DESCRIPTION
===============
[Impact]
The NetApp driver kerberos-config-get fails when using a SVM-scoped
user because it does have not enough privileges to perform that check.
This failure causes the entire stack to fail, thus preventing access
rules from being added to shares. The fix addresses this by capturing
the exception and not reraising it, allowing the operation to
continue.
[Test case]
Testing around this is limited because:
1) The NetApp CI upstream is broken at this time. The fix was validated internally by contributors and NetApp driver maintainers.
2) We do not have a NetApp box in our lab to verify the SRU for this scenario.
3) Running the Manila tempest suite is useless because the change is limited in scope to the NetApp driver, that is only operational when using NetApp storage.
[Regression Potential]
Given that the change is limited to the NetApp driver, it is small and
was peer-validated, we consider the regression potential minimal.
[Other Info]
None.
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/2069125/+subscriptions
More information about the Ubuntu-openstack-bugs
mailing list