NACK/Cmnt: [PATCH] Add test script and yaml for dpll
William Tu
witu at nvidia.com
Thu Feb 29 16:06:13 UTC 2024
My mistake! Will resend.
William
On 2/29/24, 12:33 AM, "Stefan Bader" <stefan.bader at canonical.com> wrote:
On 28.02.24 15:43, William Tu wrote:
> From: Tony Duan <yifeid at nvidia.com <mailto:yifeid at nvidia.com>>
>
> BugLink: https://bugs.launchpad.net/bugs/2053155 <https://bugs.launchpad.net/bugs/2053155>
>
> Add a script and yaml to verify dpll is supported from kernel. The
> script and yaml file were supported in upstream linux but was deleted
> in this repo. So copy them directly from latest upsteam now.
>
> Signed-off-by: Tony Duan <yifeid at nvidia.com <mailto:yifeid at nvidia.com>>
> Signed-off-by: William Tu <witu at nvidia.com <mailto:witu at nvidia.com>>
> ---
Rejected for the following reasons:
- This patch has no indication in the subject wrt what it is for.
> Documentation/netlink/genetlink.yaml | 330 ++++++++++
> tools/net/ynl/cli.py | 77 +++
> tools/net/ynl/lib/__init__.py | 8 +
> tools/net/ynl/lib/nlspec.py | 607 +++++++++++++++++++
> tools/net/ynl/lib/ynl.py | 873 +++++++++++++++++++++++++++
> 5 files changed, 1895 insertions(+)
> create mode 100644 Documentation/netlink/genetlink.yaml
> create mode 100755 tools/net/ynl/cli.py
> create mode 100644 tools/net/ynl/lib/__init__.py
> create mode 100644 tools/net/ynl/lib/nlspec.py
> create mode 100644 tools/net/ynl/lib/ynl.py
>
> diff --git a/Documentation/netlink/genetlink.yaml b/Documentation/netlink/genetlink.yaml
> new file mode 100644
> index 000000000000..3283bf458ff1
> --- /dev/null
> +++ b/Documentation/netlink/genetlink.yaml
> @@ -0,0 +1,330 @@
> +# SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause)
> +%YAML 1.2
> +---
> +$id: http://kernel.org/schemas/netlink/genetlink-legacy.yaml# <http://kernel.org/schemas/netlink/genetlink-legacy.yaml>
> +$schema: https://json-schema.org/draft-07/schema <https://json-schema.org/draft-07/schema>
> +
> +# Common defines
> +$defs:
> + uint:
> + type: integer
> + minimum: 0
> + len-or-define:
> + type: [ string, integer ]
> + pattern: ^[0-9A-Za-z_]+( - 1)?$
> + minimum: 0
> + len-or-limit:
> + # literal int or limit based on fixed-width type e.g. u8-min, u16-max, etc.
> + type: [ string, integer ]
> + pattern: ^[su](8|16|32|64)-(min|max)$
> + minimum: 0
> +
> +# Schema for specs
> +title: Protocol
> +description: Specification of a genetlink protocol
> +type: object
> +required: [ name, doc, attribute-sets, operations ]
> +additionalProperties: False
> +properties:
> + name:
> + description: Name of the genetlink family.
> + type: string
> + doc:
> + type: string
> + protocol:
> + description: Schema compatibility level. Default is "genetlink".
> + enum: [ genetlink ]
> + uapi-header:
> + description: Path to the uAPI header, default is linux/${family-name}.h
> + type: string
> +
> + definitions:
> + description: List of type and constant definitions (enums, flags, defines).
> + type: array
> + items:
> + type: object
> + required: [ type, name ]
> + additionalProperties: False
> + properties:
> + name:
> + type: string
> + header:
> + description: For C-compatible languages, header which already defines this value.
> + type: string
> + type:
> + enum: [ const, enum, flags ]
> + doc:
> + type: string
> + # For const
> + value:
> + description: For const - the value.
> + type: [ string, integer ]
> + # For enum and flags
> + value-start:
> + description: For enum or flags the literal initializer for the first value.
> + type: [ string, integer ]
> + entries:
> + description: For enum or flags array of values.
> + type: array
> + items:
> + oneOf:
> + - type: string
> + - type: object
> + required: [ name ]
> + additionalProperties: False
> + properties:
> + name:
> + type: string
> + value:
> + type: integer
> + doc:
> + type: string
> + render-max:
> + description: Render the max members for this enum.
> + type: boolean
> +
> + attribute-sets:
> + description: Definition of attribute spaces for this family.
> + type: array
> + items:
> + description: Definition of a single attribute space.
> + type: object
> + required: [ name, attributes ]
> + additionalProperties: False
> + properties:
> + name:
> + description: |
> + Name used when referring to this space in other definitions, not used outside of the spec.
> + type: string
> + name-prefix:
> + description: |
> + Prefix for the C enum name of the attributes. Default family[name]-set[name]-a-
> + type: string
> + enum-name:
> + description: Name for the enum type of the attribute.
> + type: string
> + doc:
> + description: Documentation of the space.
> + type: string
> + subset-of:
> + description: |
> + Name of another space which this is a logical part of. Sub-spaces can be used to define
> + a limited group of attributes which are used in a nest.
> + type: string
> + attributes:
> + description: List of attributes in the space.
> + type: array
> + items:
> + type: object
> + required: [ name ]
> + additionalProperties: False
> + properties:
> + name:
> + type: string
> + type: &attr-type
> + enum: [ unused, pad, flag, binary,
> + uint, sint, u8, u16, u32, u64, s32, s64,
> + string, nest, array-nest, nest-type-value ]
> + doc:
> + description: Documentation of the attribute.
> + type: string
> + value:
> + description: Value for the enum item representing this attribute in the uAPI.
> + $ref: '#/$defs/uint'
> + type-value:
> + description: Name of the value extracted from the type of a nest-type-value attribute.
> + type: array
> + items:
> + type: string
> + byte-order:
> + enum: [ little-endian, big-endian ]
> + multi-attr:
> + type: boolean
> + nested-attributes:
> + description: Name of the space (sub-space) used inside the attribute.
> + type: string
> + enum:
> + description: Name of the enum type used for the attribute.
> + type: string
> + enum-as-flags:
> + description: |
> + Treat the enum as flags. In most cases enum is either used as flags or as values.
> + Sometimes, however, both forms are necessary, in which case header contains the enum
> + form while specific attributes may request to convert the values into a bitfield.
> + type: boolean
> + checks:
> + description: Kernel input validation.
> + type: object
> + additionalProperties: False
> + properties:
> + flags-mask:
> + description: Name of the flags constant on which to base mask (unsigned scalar types only).
> + type: string
> + min:
> + description: Min value for an integer attribute.
> + $ref: '#/$defs/len-or-limit'
> + max:
> + description: Max value for an integer attribute.
> + $ref: '#/$defs/len-or-limit'
> + min-len:
> + description: Min length for a binary attribute.
> + $ref: '#/$defs/len-or-define'
> + max-len:
> + description: Max length for a string or a binary attribute.
> + $ref: '#/$defs/len-or-define'
> + exact-len:
> + description: Exact length for a string or a binary attribute.
> + $ref: '#/$defs/len-or-define'
> + sub-type: *attr-type
> + display-hint: &display-hint
> + description: |
> + Optional format indicator that is intended only for choosing
> + the right formatting mechanism when displaying values of this
> + type.
> + enum: [ hex, mac, fddi, ipv4, ipv6, uuid ]
> +
> + # Make sure name-prefix does not appear in subsets (subsets inherit naming)
> + dependencies:
> + name-prefix:
> + not:
> + required: [ subset-of ]
> + subset-of:
> + not:
> + required: [ name-prefix ]
> +
> + # type property is only required if not in subset definition
> + if:
> + properties:
> + subset-of:
> + not:
> + type: string
> + then:
> + properties:
> + attributes:
> + items:
> + required: [ type ]
> +
> + operations:
> + description: Operations supported by the protocol.
> + type: object
> + required: [ list ]
> + additionalProperties: False
> + properties:
> + enum-model:
> + description: |
> + The model of assigning values to the operations.
> + "unified" is the recommended model where all message types belong
> + to a single enum.
> + "directional" has the messages sent to the kernel and from the kernel
> + enumerated separately.
> + enum: [ unified ]
> + name-prefix:
> + description: |
> + Prefix for the C enum name of the command. The name is formed by concatenating
> + the prefix with the upper case name of the command, with dashes replaced by underscores.
> + type: string
> + enum-name:
> + description: Name for the enum type with commands.
> + type: string
> + async-prefix:
> + description: Same as name-prefix but used to render notifications and events to separate enum.
> + type: string
> + async-enum:
> + description: Name for the enum type with notifications/events.
> + type: string
> + list:
> + description: List of commands
> + type: array
> + items:
> + type: object
> + additionalProperties: False
> + required: [ name, doc ]
> + properties:
> + name:
> + description: Name of the operation, also defining its C enum value in uAPI.
> + type: string
> + doc:
> + description: Documentation for the command.
> + type: string
> + value:
> + description: Value for the enum in the uAPI.
> + $ref: '#/$defs/uint'
> + attribute-set:
> + description: |
> + Attribute space from which attributes directly in the requests and replies
> + to this command are defined.
> + type: string
> + flags: &cmd_flags
> + description: Command flags.
> + type: array
> + items:
> + enum: [ admin-perm ]
> + dont-validate:
> + description: Kernel attribute validation flags.
> + type: array
> + items:
> + enum: [ strict, dump, dump-strict ]
> + config-cond:
> + description: |
> + Name of the kernel config option gating the presence of
> + the operation, without the 'CONFIG_' prefix.
> + type: string
> + do: &subop-type
> + description: Main command handler.
> + type: object
> + additionalProperties: False
> + properties:
> + request: &subop-attr-list
> + description: Definition of the request message for a given command.
> + type: object
> + additionalProperties: False
> + properties:
> + attributes:
> + description: |
> + Names of attributes from the attribute-set (not full attribute
> + definitions, just names).
> + type: array
> + items:
> + type: string
> + reply: *subop-attr-list
> + pre:
> + description: Hook for a function to run before the main callback (pre_doit or start).
> + type: string
> + post:
> + description: Hook for a function to run after the main callback (post_doit or done).
> + type: string
> + dump: *subop-type
> + notify:
> + description: Name of the command sharing the reply type with this notification.
> + type: string
> + event:
> + type: object
> + additionalProperties: False
> + properties:
> + attributes:
> + description: Explicit list of the attributes for the notification.
> + type: array
> + items:
> + type: string
> + mcgrp:
> + description: Name of the multicast group generating given notification.
> + type: string
> + mcast-groups:
> + description: List of multicast groups.
> + type: object
> + required: [ list ]
> + additionalProperties: False
> + properties:
> + list:
> + description: List of groups.
> + type: array
> + items:
> + type: object
> + required: [ name ]
> + additionalProperties: False
> + properties:
> + name:
> + description: |
> + The name for the group, used to form the define and the value of the define.
> + type: string
> + flags: *cmd_flags
> diff --git a/tools/net/ynl/cli.py b/tools/net/ynl/cli.py
> new file mode 100755
> index 000000000000..0f8239979670
> --- /dev/null
> +++ b/tools/net/ynl/cli.py
> @@ -0,0 +1,77 @@
> +#!/usr/bin/env python3
> +# SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
> +
> +import argparse
> +import json
> +import pprint
> +import time
> +
> +from lib import YnlFamily, Netlink
> +
> +
> +class YnlEncoder(json.JSONEncoder):
> + def default(self, obj):
> + if isinstance(obj, bytes):
> + return bytes.hex(obj)
> + if isinstance(obj, set):
> + return list(obj)
> + return json.JSONEncoder.default(self, obj)
> +
> +
> +def main():
> + parser = argparse.ArgumentParser(description='YNL CLI sample')
> + parser.add_argument('--spec', dest='spec', type=str, required=True)
> + parser.add_argument('--schema', dest='schema', type=str)
> + parser.add_argument('--no-schema', action='store_true')
> + parser.add_argument('--json', dest='json_text', type=str)
> + parser.add_argument('--do', dest='do', type=str)
> + parser.add_argument('--dump', dest='dump', type=str)
> + parser.add_argument('--sleep', dest='sleep', type=int)
> + parser.add_argument('--subscribe', dest='ntf', type=str)
> + parser.add_argument('--replace', dest='flags', action='append_const',
> + const=Netlink.NLM_F_REPLACE)
> + parser.add_argument('--excl', dest='flags', action='append_const',
> + const=Netlink.NLM_F_EXCL)
> + parser.add_argument('--create', dest='flags', action='append_const',
> + const=Netlink.NLM_F_CREATE)
> + parser.add_argument('--append', dest='flags', action='append_const',
> + const=Netlink.NLM_F_APPEND)
> + parser.add_argument('--process-unknown', action=argparse.BooleanOptionalAction)
> + parser.add_argument('--output-json', action='store_true')
> + args = parser.parse_args()
> +
> + def output(msg):
> + if args.output_json:
> + print(json.dumps(msg, cls=YnlEncoder))
> + else:
> + pprint.PrettyPrinter().pprint(msg)
> +
> + if args.no_schema:
> + args.schema = ''
> +
> + attrs = {}
> + if args.json_text:
> + attrs = json.loads(args.json_text)
> +
> + ynl = YnlFamily(args.spec, args.schema, args.process_unknown)
> +
> + if args.ntf:
> + ynl.ntf_subscribe(args.ntf)
> +
> + if args.sleep:
> + time.sleep(args.sleep)
> +
> + if args.do:
> + reply = ynl.do(args.do, attrs, args.flags)
> + output(reply)
> + if args.dump:
> + reply = ynl.dump(args.dump, attrs)
> + output(reply)
> +
> + if args.ntf:
> + ynl.check_ntf()
> + output(ynl.async_msg_queue)
> +
> +
> +if __name__ == "__main__":
> + main()
> diff --git a/tools/net/ynl/lib/__init__.py b/tools/net/ynl/lib/__init__.py
> new file mode 100644
> index 000000000000..f7eaa07783e7
> --- /dev/null
> +++ b/tools/net/ynl/lib/__init__.py
> @@ -0,0 +1,8 @@
> +# SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
> +
> +from .nlspec import SpecAttr, SpecAttrSet, SpecEnumEntry, SpecEnumSet, \
> + SpecFamily, SpecOperation
> +from .ynl import YnlFamily, Netlink
> +
> +__all__ = ["SpecAttr", "SpecAttrSet", "SpecEnumEntry", "SpecEnumSet",
> + "SpecFamily", "SpecOperation", "YnlFamily", "Netlink"]
> diff --git a/tools/net/ynl/lib/nlspec.py b/tools/net/ynl/lib/nlspec.py
> new file mode 100644
> index 000000000000..fbce52395b3b
> --- /dev/null
> +++ b/tools/net/ynl/lib/nlspec.py
> @@ -0,0 +1,607 @@
> +# SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
> +
> +import collections
> +import importlib
> +import os
> +import yaml
> +
> +
> +# To be loaded dynamically as needed
> +jsonschema = None
> +
> +
> +class SpecElement:
> + """Netlink spec element.
> +
> + Abstract element of the Netlink spec. Implements the dictionary interface
> + for access to the raw spec. Supports iterative resolution of dependencies
> + across elements and class inheritance levels. The elements of the spec
> + may refer to each other, and although loops should be very rare, having
> + to maintain correct ordering of instantiation is painful, so the resolve()
> + method should be used to perform parts of init which require access to
> + other parts of the spec.
> +
> + Attributes:
> + yaml raw spec as loaded from the spec file
> + family back reference to the full family
> +
> + name name of the entity as listed in the spec (optional)
> + ident_name name which can be safely used as identifier in code (optional)
> + """
> + def __init__(self, family, yaml):
> + self.yaml = yaml
> + self.family = family
> +
> + if 'name' in self.yaml:
> + self.name = self.yaml['name']
> + self.ident_name = self.name.replace('-', '_')
> +
> + self._super_resolved = False
> + family.add_unresolved(self)
> +
> + def __getitem__(self, key):
> + return self.yaml[key]
> +
> + def __contains__(self, key):
> + return key in self.yaml
> +
> + def get(self, key, default=None):
> + return self.yaml.get(key, default)
> +
> + def resolve_up(self, up):
> + if not self._super_resolved:
> + up.resolve()
> + self._super_resolved = True
> +
> + def resolve(self):
> + pass
> +
> +
> +class SpecEnumEntry(SpecElement):
> + """ Entry within an enum declared in the Netlink spec.
> +
> + Attributes:
> + doc documentation string
> + enum_set back reference to the enum
> + value numerical value of this enum (use accessors in most situations!)
> +
> + Methods:
> + raw_value raw value, i.e. the id in the enum, unlike user value which is a mask for flags
> + user_value user value, same as raw value for enums, for flags it's the mask
> + """
> + def __init__(self, enum_set, yaml, prev, value_start):
> + if isinstance(yaml, str):
> + yaml = {'name': yaml}
> + super().__init__(enum_set.family, yaml)
> +
> + self.doc = yaml.get('doc', '')
> + self.enum_set = enum_set
> +
> + if 'value' in yaml:
> + self.value = yaml['value']
> + elif prev:
> + self.value = prev.value + 1
> + else:
> + self.value = value_start
> +
> + def has_doc(self):
> + return bool(self.doc)
> +
> + def raw_value(self):
> + return self.value
> +
> + def user_value(self, as_flags=None):
> + if self.enum_set['type'] == 'flags' or as_flags:
> + return 1 << self.value
> + else:
> + return self.value
> +
> +
> +class SpecEnumSet(SpecElement):
> + """ Enum type
> +
> + Represents an enumeration (list of numerical constants)
> + as declared in the "definitions" section of the spec.
> +
> + Attributes:
> + type enum or flags
> + entries entries by name
> + entries_by_val entries by value
> + Methods:
> + get_mask for flags compute the mask of all defined values
> + """
> + def __init__(self, family, yaml):
> + super().__init__(family, yaml)
> +
> + self.type = yaml['type']
> +
> + prev_entry = None
> + value_start = self.yaml.get('value-start', 0)
> + self.entries = dict()
> + self.entries_by_val = dict()
> + for entry in self.yaml['entries']:
> + e = self.new_entry(entry, prev_entry, value_start)
> + self.entries[e.name] = e
> + self.entries_by_val[e.raw_value()] = e
> + prev_entry = e
> +
> + def new_entry(self, entry, prev_entry, value_start):
> + return SpecEnumEntry(self, entry, prev_entry, value_start)
> +
> + def has_doc(self):
> + if 'doc' in self.yaml:
> + return True
> + for entry in self.entries.values():
> + if entry.has_doc():
> + return True
> + return False
> +
> + def get_mask(self, as_flags=None):
> + mask = 0
> + for e in self.entries.values():
> + mask += e.user_value(as_flags)
> + return mask
> +
> +
> +class SpecAttr(SpecElement):
> + """ Single Netlink attribute type
> +
> + Represents a single attribute type within an attr space.
> +
> + Attributes:
> + type string, attribute type
> + value numerical ID when serialized
> + attr_set Attribute Set containing this attr
> + is_multi bool, attr may repeat multiple times
> + struct_name string, name of struct definition
> + sub_type string, name of sub type
> + len integer, optional byte length of binary types
> + display_hint string, hint to help choose format specifier
> + when displaying the value
> + sub_message string, name of sub message type
> + selector string, name of attribute used to select
> + sub-message type
> +
> + is_auto_scalar bool, attr is a variable-size scalar
> + """
> + def __init__(self, family, attr_set, yaml, value):
> + super().__init__(family, yaml)
> +
> + self.type = yaml['type']
> + self.value = value
> + self.attr_set = attr_set
> + self.is_multi = yaml.get('multi-attr', False)
> + self.struct_name = yaml.get('struct')
> + self.sub_type = yaml.get('sub-type')
> + self.byte_order = yaml.get('byte-order')
> + self.len = yaml.get('len')
> + self.display_hint = yaml.get('display-hint')
> + self.sub_message = yaml.get('sub-message')
> + self.selector = yaml.get('selector')
> +
> + self.is_auto_scalar = self.type == "sint" or self.type == "uint"
> +
> +
> +class SpecAttrSet(SpecElement):
> + """ Netlink Attribute Set class.
> +
> + Represents a ID space of attributes within Netlink.
> +
> + Note that unlike other elements, which expose contents of the raw spec
> + via the dictionary interface Attribute Set exposes attributes by name.
> +
> + Attributes:
> + attrs ordered dict of all attributes (indexed by name)
> + attrs_by_val ordered dict of all attributes (indexed by value)
> + subset_of parent set if this is a subset, otherwise None
> + """
> + def __init__(self, family, yaml):
> + super().__init__(family, yaml)
> +
> + self.subset_of = self.yaml.get('subset-of', None)
> +
> + self.attrs = collections.OrderedDict()
> + self.attrs_by_val = collections.OrderedDict()
> +
> + if self.subset_of is None:
> + val = 1
> + for elem in self.yaml['attributes']:
> + if 'value' in elem:
> + val = elem['value']
> +
> + attr = self.new_attr(elem, val)
> + self.attrs[attr.name] = attr
> + self.attrs_by_val[attr.value] = attr
> + val += 1
> + else:
> + real_set = family.attr_sets[self.subset_of]
> + for elem in self.yaml['attributes']:
> + attr = real_set[elem['name']]
> + self.attrs[attr.name] = attr
> + self.attrs_by_val[attr.value] = attr
> +
> + def new_attr(self, elem, value):
> + return SpecAttr(self.family, self, elem, value)
> +
> + def __getitem__(self, key):
> + return self.attrs[key]
> +
> + def __contains__(self, key):
> + return key in self.attrs
> +
> + def __iter__(self):
> + yield from self.attrs
> +
> + def items(self):
> + return self.attrs.items()
> +
> +
> +class SpecStructMember(SpecElement):
> + """Struct member attribute
> +
> + Represents a single struct member attribute.
> +
> + Attributes:
> + type string, type of the member attribute
> + byte_order string or None for native byte order
> + enum string, name of the enum definition
> + len integer, optional byte length of binary types
> + display_hint string, hint to help choose format specifier
> + when displaying the value
> + struct string, name of nested struct type
> + """
> + def __init__(self, family, yaml):
> + super().__init__(family, yaml)
> + self.type = yaml['type']
> + self.byte_order = yaml.get('byte-order')
> + self.enum = yaml.get('enum')
> + self.len = yaml.get('len')
> + self.display_hint = yaml.get('display-hint')
> + self.struct = yaml.get('struct')
> +
> +
> +class SpecStruct(SpecElement):
> + """Netlink struct type
> +
> + Represents a C struct definition.
> +
> + Attributes:
> + members ordered list of struct members
> + """
> + def __init__(self, family, yaml):
> + super().__init__(family, yaml)
> +
> + self.members = []
> + for member in yaml.get('members', []):
> + self.members.append(self.new_member(family, member))
> +
> + def new_member(self, family, elem):
> + return SpecStructMember(family, elem)
> +
> + def __iter__(self):
> + yield from self.members
> +
> + def items(self):
> + return self.members.items()
> +
> +
> +class SpecSubMessage(SpecElement):
> + """ Netlink sub-message definition
> +
> + Represents a set of sub-message formats for polymorphic nlattrs
> + that contain type-specific sub messages.
> +
> + Attributes:
> + name string, name of sub-message definition
> + formats dict of sub-message formats indexed by match value
> + """
> + def __init__(self, family, yaml):
> + super().__init__(family, yaml)
> +
> + self.formats = collections.OrderedDict()
> + for elem in self.yaml['formats']:
> + format = self.new_format(family, elem)
> + self.formats[format.value] = format
> +
> + def new_format(self, family, format):
> + return SpecSubMessageFormat(family, format)
> +
> +
> +class SpecSubMessageFormat(SpecElement):
> + """ Netlink sub-message format definition
> +
> + Represents a single format for a sub-message.
> +
> + Attributes:
> + value attribute value to match against type selector
> + fixed_header string, name of fixed header, or None
> + attr_set string, name of attribute set, or None
> + """
> + def __init__(self, family, yaml):
> + super().__init__(family, yaml)
> +
> + self.value = yaml.get('value')
> + self.fixed_header = yaml.get('fixed-header')
> + self.attr_set = yaml.get('attribute-set')
> +
> +
> +class SpecOperation(SpecElement):
> + """Netlink Operation
> +
> + Information about a single Netlink operation.
> +
> + Attributes:
> + value numerical ID when serialized, None if req/rsp values differ
> +
> + req_value numerical ID when serialized, user -> kernel
> + rsp_value numerical ID when serialized, user <- kernel
> + is_call bool, whether the operation is a call
> + is_async bool, whether the operation is a notification
> + is_resv bool, whether the operation does not exist (it's just a reserved ID)
> + attr_set attribute set name
> + fixed_header string, optional name of fixed header struct
> +
> + yaml raw spec as loaded from the spec file
> + """
> + def __init__(self, family, yaml, req_value, rsp_value):
> + super().__init__(family, yaml)
> +
> + self.value = req_value if req_value == rsp_value else None
> + self.req_value = req_value
> + self.rsp_value = rsp_value
> +
> + self.is_call = 'do' in yaml or 'dump' in yaml
> + self.is_async = 'notify' in yaml or 'event' in yaml
> + self.is_resv = not self.is_async and not self.is_call
> + self.fixed_header = self.yaml.get('fixed-header', family.fixed_header)
> +
> + # Added by resolve:
> + self.attr_set = None
> + delattr(self, "attr_set")
> +
> + def resolve(self):
> + self.resolve_up(super())
> +
> + if 'attribute-set' in self.yaml:
> + attr_set_name = self.yaml['attribute-set']
> + elif 'notify' in self.yaml:
> + msg = self.family.msgs[self.yaml['notify']]
> + attr_set_name = msg['attribute-set']
> + elif self.is_resv:
> + attr_set_name = ''
> + else:
> + raise Exception(f"Can't resolve attribute set for op '{self.name}'")
> + if attr_set_name:
> + self.attr_set = self.family.attr_sets[attr_set_name]
> +
> +
> +class SpecMcastGroup(SpecElement):
> + """Netlink Multicast Group
> +
> + Information about a multicast group.
> +
> + Value is only used for classic netlink families that use the
> + netlink-raw schema. Genetlink families use dynamic ID allocation
> + where the ids of multicast groups get resolved at runtime. Value
> + will be None for genetlink families.
> +
> + Attributes:
> + name name of the mulitcast group
> + value integer id of this multicast group for netlink-raw or None
> + yaml raw spec as loaded from the spec file
> + """
> + def __init__(self, family, yaml):
> + super().__init__(family, yaml)
> + self.value = self.yaml.get('value')
> +
> +
> +class SpecFamily(SpecElement):
> + """ Netlink Family Spec class.
> +
> + Netlink family information loaded from a spec (e.g. in YAML).
> + Takes care of unfolding implicit information which can be skipped
> + in the spec itself for brevity.
> +
> + The class can be used like a dictionary to access the raw spec
> + elements but that's usually a bad idea.
> +
> + Attributes:
> + proto protocol type (e.g. genetlink)
> + msg_id_model enum-model for operations (unified, directional etc.)
> + license spec license (loaded from an SPDX tag on the spec)
> +
> + attr_sets dict of attribute sets
> + msgs dict of all messages (index by name)
> + sub_msgs dict of all sub messages (index by name)
> + ops dict of all valid requests / responses
> + ntfs dict of all async events
> + consts dict of all constants/enums
> + fixed_header string, optional name of family default fixed header struct
> + mcast_groups dict of all multicast groups (index by name)
> + """
> + def __init__(self, spec_path, schema_path=None, exclude_ops=None):
> + with open(spec_path, "r") as stream:
> + prefix = '# SPDX-License-Identifier: '
> + first = stream.readline().strip()
> + if not first.startswith(prefix):
> + raise Exception('SPDX license tag required in the spec')
> + self.license = first[len(prefix):]
> +
> + stream.seek(0)
> + spec = yaml.safe_load(stream)
> +
> + self._resolution_list = []
> +
> + super().__init__(self, spec)
> +
> + self._exclude_ops = exclude_ops if exclude_ops else []
> +
> + self.proto = self.yaml.get('protocol', 'genetlink')
> + self.msg_id_model = self.yaml['operations'].get('enum-model', 'unified')
> +
> + if schema_path is None:
> + schema_path = os.path.dirname(os.path.dirname(spec_path)) + f'/{self.proto}.yaml'
> + if schema_path:
> + global jsonschema
> +
> + with open(schema_path, "r") as stream:
> + schema = yaml.safe_load(stream)
> +
> + if jsonschema is None:
> + jsonschema = importlib.import_module("jsonschema")
> +
> + jsonschema.validate(self.yaml, schema)
> +
> + self.attr_sets = collections.OrderedDict()
> + self.sub_msgs = collections.OrderedDict()
> + self.msgs = collections.OrderedDict()
> + self.req_by_value = collections.OrderedDict()
> + self.rsp_by_value = collections.OrderedDict()
> + self.ops = collections.OrderedDict()
> + self.ntfs = collections.OrderedDict()
> + self.consts = collections.OrderedDict()
> + self.mcast_groups = collections.OrderedDict()
> +
> + last_exception = None
> + while len(self._resolution_list) > 0:
> + resolved = []
> + unresolved = self._resolution_list
> + self._resolution_list = []
> +
> + for elem in unresolved:
> + try:
> + elem.resolve()
> + except (KeyError, AttributeError) as e:
> + self._resolution_list.append(elem)
> + last_exception = e
> + continue
> +
> + resolved.append(elem)
> +
> + if len(resolved) == 0:
> + raise last_exception
> +
> + def new_enum(self, elem):
> + return SpecEnumSet(self, elem)
> +
> + def new_attr_set(self, elem):
> + return SpecAttrSet(self, elem)
> +
> + def new_struct(self, elem):
> + return SpecStruct(self, elem)
> +
> + def new_sub_message(self, elem):
> + return SpecSubMessage(self, elem);
> +
> + def new_operation(self, elem, req_val, rsp_val):
> + return SpecOperation(self, elem, req_val, rsp_val)
> +
> + def new_mcast_group(self, elem):
> + return SpecMcastGroup(self, elem)
> +
> + def add_unresolved(self, elem):
> + self._resolution_list.append(elem)
> +
> + def _dictify_ops_unified(self):
> + self.fixed_header = self.yaml['operations'].get('fixed-header')
> + val = 1
> + for elem in self.yaml['operations']['list']:
> + if 'value' in elem:
> + val = elem['value']
> +
> + op = self.new_operation(elem, val, val)
> + val += 1
> +
> + self.msgs[op.name] = op
> +
> + def _dictify_ops_directional(self):
> + self.fixed_header = self.yaml['operations'].get('fixed-header')
> + req_val = rsp_val = 1
> + for elem in self.yaml['operations']['list']:
> + if 'notify' in elem or 'event' in elem:
> + if 'value' in elem:
> + rsp_val = elem['value']
> + req_val_next = req_val
> + rsp_val_next = rsp_val + 1
> + req_val = None
> + elif 'do' in elem or 'dump' in elem:
> + mode = elem['do'] if 'do' in elem else elem['dump']
> +
> + v = mode.get('request', {}).get('value', None)
> + if v:
> + req_val = v
> + v = mode.get('reply', {}).get('value', None)
> + if v:
> + rsp_val = v
> +
> + rsp_inc = 1 if 'reply' in mode else 0
> + req_val_next = req_val + 1
> + rsp_val_next = rsp_val + rsp_inc
> + else:
> + raise Exception("Can't parse directional ops")
> +
> + if req_val == req_val_next:
> + req_val = None
> + if rsp_val == rsp_val_next:
> + rsp_val = None
> +
> + skip = False
> + for exclude in self._exclude_ops:
> + skip |= bool(exclude.match(elem['name']))
> + if not skip:
> + op = self.new_operation(elem, req_val, rsp_val)
> +
> + req_val = req_val_next
> + rsp_val = rsp_val_next
> +
> + self.msgs[op.name] = op
> +
> + def find_operation(self, name):
> + """
> + For a given operation name, find and return operation spec.
> + """
> + for op in self.yaml['operations']['list']:
> + if name == op['name']:
> + return op
> + return None
> +
> + def resolve(self):
> + self.resolve_up(super())
> +
> + definitions = self.yaml.get('definitions', [])
> + for elem in definitions:
> + if elem['type'] == 'enum' or elem['type'] == 'flags':
> + self.consts[elem['name']] = self.new_enum(elem)
> + elif elem['type'] == 'struct':
> + self.consts[elem['name']] = self.new_struct(elem)
> + else:
> + self.consts[elem['name']] = elem
> +
> + for elem in self.yaml['attribute-sets']:
> + attr_set = self.new_attr_set(elem)
> + self.attr_sets[elem['name']] = attr_set
> +
> + for elem in self.yaml.get('sub-messages', []):
> + sub_message = self.new_sub_message(elem)
> + self.sub_msgs[sub_message.name] = sub_message
> +
> + if self.msg_id_model == 'unified':
> + self._dictify_ops_unified()
> + elif self.msg_id_model == 'directional':
> + self._dictify_ops_directional()
> +
> + for op in self.msgs.values():
> + if op.req_value is not None:
> + self.req_by_value[op.req_value] = op
> + if op.rsp_value is not None:
> + self.rsp_by_value[op.rsp_value] = op
> + if not op.is_async and 'attribute-set' in op:
> + self.ops[op.name] = op
> + elif op.is_async:
> + self.ntfs[op.name] = op
> +
> + mcgs = self.yaml.get('mcast-groups')
> + if mcgs:
> + for elem in mcgs['list']:
> + mcg = self.new_mcast_group(elem)
> + self.mcast_groups[elem['name']] = mcg
> diff --git a/tools/net/ynl/lib/ynl.py b/tools/net/ynl/lib/ynl.py
> new file mode 100644
> index 000000000000..03c7ca6aaae9
> --- /dev/null
> +++ b/tools/net/ynl/lib/ynl.py
> @@ -0,0 +1,873 @@
> +# SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
> +
> +from collections import namedtuple
> +import functools
> +import os
> +import random
> +import socket
> +import struct
> +from struct import Struct
> +import yaml
> +import ipaddress
> +import uuid
> +
> +from .nlspec import SpecFamily
> +
> +#
> +# Generic Netlink code which should really be in some library, but I can't quickly find one.
> +#
> +
> +
> +class Netlink:
> + # Netlink socket
> + SOL_NETLINK = 270
> +
> + NETLINK_ADD_MEMBERSHIP = 1
> + NETLINK_CAP_ACK = 10
> + NETLINK_EXT_ACK = 11
> + NETLINK_GET_STRICT_CHK = 12
> +
> + # Netlink message
> + NLMSG_ERROR = 2
> + NLMSG_DONE = 3
> +
> + NLM_F_REQUEST = 1
> + NLM_F_ACK = 4
> + NLM_F_ROOT = 0x100
> + NLM_F_MATCH = 0x200
> +
> + NLM_F_REPLACE = 0x100
> + NLM_F_EXCL = 0x200
> + NLM_F_CREATE = 0x400
> + NLM_F_APPEND = 0x800
> +
> + NLM_F_CAPPED = 0x100
> + NLM_F_ACK_TLVS = 0x200
> +
> + NLM_F_DUMP = NLM_F_ROOT | NLM_F_MATCH
> +
> + NLA_F_NESTED = 0x8000
> + NLA_F_NET_BYTEORDER = 0x4000
> +
> + NLA_TYPE_MASK = NLA_F_NESTED | NLA_F_NET_BYTEORDER
> +
> + # Genetlink defines
> + NETLINK_GENERIC = 16
> +
> + GENL_ID_CTRL = 0x10
> +
> + # nlctrl
> + CTRL_CMD_GETFAMILY = 3
> +
> + CTRL_ATTR_FAMILY_ID = 1
> + CTRL_ATTR_FAMILY_NAME = 2
> + CTRL_ATTR_MAXATTR = 5
> + CTRL_ATTR_MCAST_GROUPS = 7
> +
> + CTRL_ATTR_MCAST_GRP_NAME = 1
> + CTRL_ATTR_MCAST_GRP_ID = 2
> +
> + # Extack types
> + NLMSGERR_ATTR_MSG = 1
> + NLMSGERR_ATTR_OFFS = 2
> + NLMSGERR_ATTR_COOKIE = 3
> + NLMSGERR_ATTR_POLICY = 4
> + NLMSGERR_ATTR_MISS_TYPE = 5
> + NLMSGERR_ATTR_MISS_NEST = 6
> +
> +
> +class NlError(Exception):
> + def __init__(self, nl_msg):
> + self.nl_msg = nl_msg
> +
> + def __str__(self):
> + return f"Netlink error: {os.strerror(-self.nl_msg.error)}\n{self.nl_msg}"
> +
> +
> +class NlAttr:
> + ScalarFormat = namedtuple('ScalarFormat', ['native', 'big', 'little'])
> + type_formats = {
> + 'u8' : ScalarFormat(Struct('B'), Struct("B"), Struct("B")),
> + 's8' : ScalarFormat(Struct('b'), Struct("b"), Struct("b")),
> + 'u16': ScalarFormat(Struct('H'), Struct(">H"), Struct("<H")),
> + 's16': ScalarFormat(Struct('h'), Struct(">h"), Struct("<h")),
> + 'u32': ScalarFormat(Struct('I'), Struct(">I"), Struct("<I")),
> + 's32': ScalarFormat(Struct('i'), Struct(">i"), Struct("<i")),
> + 'u64': ScalarFormat(Struct('Q'), Struct(">Q"), Struct("<Q")),
> + 's64': ScalarFormat(Struct('q'), Struct(">q"), Struct("<q"))
> + }
> +
> + def __init__(self, raw, offset):
> + self._len, self._type = struct.unpack("HH", raw[offset : offset + 4])
> + self.type = self._type & ~Netlink.NLA_TYPE_MASK
> + self.is_nest = self._type & Netlink.NLA_F_NESTED
> + self.payload_len = self._len
> + self.full_len = (self.payload_len + 3) & ~3
> + self.raw = raw[offset + 4 : offset + self.payload_len]
> +
> + @classmethod
> + def get_format(cls, attr_type, byte_order=None):
> + format = cls.type_formats[attr_type]
> + if byte_order:
> + return format.big if byte_order == "big-endian" \
> + else format.little
> + return format.native
> +
> + def as_scalar(self, attr_type, byte_order=None):
> + format = self.get_format(attr_type, byte_order)
> + return format.unpack(self.raw)[0]
> +
> + def as_auto_scalar(self, attr_type, byte_order=None):
> + if len(self.raw) != 4 and len(self.raw) != 8:
> + raise Exception(f"Auto-scalar len payload be 4 or 8 bytes, got {len(self.raw)}")
> + real_type = attr_type[0] + str(len(self.raw) * 8)
> + format = self.get_format(real_type, byte_order)
> + return format.unpack(self.raw)[0]
> +
> + def as_strz(self):
> + return self.raw.decode('ascii')[:-1]
> +
> + def as_bin(self):
> + return self.raw
> +
> + def as_c_array(self, type):
> + format = self.get_format(type)
> + return [ x[0] for x in format.iter_unpack(self.raw) ]
> +
> + def __repr__(self):
> + return f"[type:{self.type} len:{self._len}] {self.raw}"
> +
> +
> +class NlAttrs:
> + def __init__(self, msg, offset=0):
> + self.attrs = []
> +
> + while offset < len(msg):
> + attr = NlAttr(msg, offset)
> + offset += attr.full_len
> + self.attrs.append(attr)
> +
> + def __iter__(self):
> + yield from self.attrs
> +
> + def __repr__(self):
> + msg = ''
> + for a in self.attrs:
> + if msg:
> + msg += '\n'
> + msg += repr(a)
> + return msg
> +
> +
> +class NlMsg:
> + def __init__(self, msg, offset, attr_space=None):
> + self.hdr = msg[offset : offset + 16]
> +
> + self.nl_len, self.nl_type, self.nl_flags, self.nl_seq, self.nl_portid = \
> + struct.unpack("IHHII", self.hdr)
> +
> + self.raw = msg[offset + 16 : offset + self.nl_len]
> +
> + self.error = 0
> + self.done = 0
> +
> + extack_off = None
> + if self.nl_type == Netlink.NLMSG_ERROR:
> + self.error = struct.unpack("i", self.raw[0:4])[0]
> + self.done = 1
> + extack_off = 20
> + elif self.nl_type == Netlink.NLMSG_DONE:
> + self.done = 1
> + extack_off = 4
> +
> + self.extack = None
> + if self.nl_flags & Netlink.NLM_F_ACK_TLVS and extack_off:
> + self.extack = dict()
> + extack_attrs = NlAttrs(self.raw[extack_off:])
> + for extack in extack_attrs:
> + if extack.type == Netlink.NLMSGERR_ATTR_MSG:
> + self.extack['msg'] = extack.as_strz()
> + elif extack.type == Netlink.NLMSGERR_ATTR_MISS_TYPE:
> + self.extack['miss-type'] = extack.as_scalar('u32')
> + elif extack.type == Netlink.NLMSGERR_ATTR_MISS_NEST:
> + self.extack['miss-nest'] = extack.as_scalar('u32')
> + elif extack.type == Netlink.NLMSGERR_ATTR_OFFS:
> + self.extack['bad-attr-offs'] = extack.as_scalar('u32')
> + else:
> + if 'unknown' not in self.extack:
> + self.extack['unknown'] = []
> + self.extack['unknown'].append(extack)
> +
> + if attr_space:
> + # We don't have the ability to parse nests yet, so only do global
> + if 'miss-type' in self.extack and 'miss-nest' not in self.extack:
> + miss_type = self.extack['miss-type']
> + if miss_type in attr_space.attrs_by_val:
> + spec = attr_space.attrs_by_val[miss_type]
> + desc = spec['name']
> + if 'doc' in spec:
> + desc += f" ({spec['doc']})"
> + self.extack['miss-type'] = desc
> +
> + def cmd(self):
> + return self.nl_type
> +
> + def __repr__(self):
> + msg = f"nl_len = {self.nl_len} ({len(self.raw)}) nl_flags = 0x{self.nl_flags:x} nl_type = {self.nl_type}\n"
> + if self.error:
> + msg += '\terror: ' + str(self.error)
> + if self.extack:
> + msg += '\textack: ' + repr(self.extack)
> + return msg
> +
> +
> +class NlMsgs:
> + def __init__(self, data, attr_space=None):
> + self.msgs = []
> +
> + offset = 0
> + while offset < len(data):
> + msg = NlMsg(data, offset, attr_space=attr_space)
> + offset += msg.nl_len
> + self.msgs.append(msg)
> +
> + def __iter__(self):
> + yield from self.msgs
> +
> +
> +genl_family_name_to_id = None
> +
> +
> +def _genl_msg(nl_type, nl_flags, genl_cmd, genl_version, seq=None):
> + # we prepend length in _genl_msg_finalize()
> + if seq is None:
> + seq = random.randint(1, 1024)
> + nlmsg = struct.pack("HHII", nl_type, nl_flags, seq, 0)
> + genlmsg = struct.pack("BBH", genl_cmd, genl_version, 0)
> + return nlmsg + genlmsg
> +
> +
> +def _genl_msg_finalize(msg):
> + return struct.pack("I", len(msg) + 4) + msg
> +
> +
> +def _genl_load_families():
> + with socket.socket(socket.AF_NETLINK, socket.SOCK_RAW, Netlink.NETLINK_GENERIC) as sock:
> + sock.setsockopt(Netlink.SOL_NETLINK, Netlink.NETLINK_CAP_ACK, 1)
> +
> + msg = _genl_msg(Netlink.GENL_ID_CTRL,
> + Netlink.NLM_F_REQUEST | Netlink.NLM_F_ACK | Netlink.NLM_F_DUMP,
> + Netlink.CTRL_CMD_GETFAMILY, 1)
> + msg = _genl_msg_finalize(msg)
> +
> + sock.send(msg, 0)
> +
> + global genl_family_name_to_id
> + genl_family_name_to_id = dict()
> +
> + while True:
> + reply = sock.recv(128 * 1024)
> + nms = NlMsgs(reply)
> + for nl_msg in nms:
> + if nl_msg.error:
> + print("Netlink error:", nl_msg.error)
> + return
> + if nl_msg.done:
> + return
> +
> + gm = GenlMsg(nl_msg)
> + fam = dict()
> + for attr in NlAttrs(gm.raw):
> + if attr.type == Netlink.CTRL_ATTR_FAMILY_ID:
> + fam['id'] = attr.as_scalar('u16')
> + elif attr.type == Netlink.CTRL_ATTR_FAMILY_NAME:
> + fam['name'] = attr.as_strz()
> + elif attr.type == Netlink.CTRL_ATTR_MAXATTR:
> + fam['maxattr'] = attr.as_scalar('u32')
> + elif attr.type == Netlink.CTRL_ATTR_MCAST_GROUPS:
> + fam['mcast'] = dict()
> + for entry in NlAttrs(attr.raw):
> + mcast_name = None
> + mcast_id = None
> + for entry_attr in NlAttrs(entry.raw):
> + if entry_attr.type == Netlink.CTRL_ATTR_MCAST_GRP_NAME:
> + mcast_name = entry_attr.as_strz()
> + elif entry_attr.type == Netlink.CTRL_ATTR_MCAST_GRP_ID:
> + mcast_id = entry_attr.as_scalar('u32')
> + if mcast_name and mcast_id is not None:
> + fam['mcast'][mcast_name] = mcast_id
> + if 'name' in fam and 'id' in fam:
> + genl_family_name_to_id[fam['name']] = fam
> +
> +
> +class GenlMsg:
> + def __init__(self, nl_msg):
> + self.nl = nl_msg
> + self.genl_cmd, self.genl_version, _ = struct.unpack_from("BBH", nl_msg.raw, 0)
> + self.raw = nl_msg.raw[4:]
> +
> + def cmd(self):
> + return self.genl_cmd
> +
> + def __repr__(self):
> + msg = repr(self.nl)
> + msg += f"\tgenl_cmd = {self.genl_cmd} genl_ver = {self.genl_version}\n"
> + for a in self.raw_attrs:
> + msg += '\t\t' + repr(a) + '\n'
> + return msg
> +
> +
> +class NetlinkProtocol:
> + def __init__(self, family_name, proto_num):
> + self.family_name = family_name
> + self.proto_num = proto_num
> +
> + def _message(self, nl_type, nl_flags, seq=None):
> + if seq is None:
> + seq = random.randint(1, 1024)
> + nlmsg = struct.pack("HHII", nl_type, nl_flags, seq, 0)
> + return nlmsg
> +
> + def message(self, flags, command, version, seq=None):
> + return self._message(command, flags, seq)
> +
> + def _decode(self, nl_msg):
> + return nl_msg
> +
> + def decode(self, ynl, nl_msg):
> + msg = self._decode(nl_msg)
> + fixed_header_size = 0
> + if ynl:
> + op = ynl.rsp_by_value[msg.cmd()]
> + fixed_header_size = ynl._struct_size(op.fixed_header)
> + msg.raw_attrs = NlAttrs(msg.raw, fixed_header_size)
> + return msg
> +
> + def get_mcast_id(self, mcast_name, mcast_groups):
> + if mcast_name not in mcast_groups:
> + raise Exception(f'Multicast group "{mcast_name}" not present in the spec')
> + return mcast_groups[mcast_name].value
> +
> +
> +class GenlProtocol(NetlinkProtocol):
> + def __init__(self, family_name):
> + super().__init__(family_name, Netlink.NETLINK_GENERIC)
> +
> + global genl_family_name_to_id
> + if genl_family_name_to_id is None:
> + _genl_load_families()
> +
> + self.genl_family = genl_family_name_to_id[family_name]
> + self.family_id = genl_family_name_to_id[family_name]['id']
> +
> + def message(self, flags, command, version, seq=None):
> + nlmsg = self._message(self.family_id, flags, seq)
> + genlmsg = struct.pack("BBH", command, version, 0)
> + return nlmsg + genlmsg
> +
> + def _decode(self, nl_msg):
> + return GenlMsg(nl_msg)
> +
> + def get_mcast_id(self, mcast_name, mcast_groups):
> + if mcast_name not in self.genl_family['mcast']:
> + raise Exception(f'Multicast group "{mcast_name}" not present in the family')
> + return self.genl_family['mcast'][mcast_name]
> +
> +
> +
> +class SpaceAttrs:
> + SpecValuesPair = namedtuple('SpecValuesPair', ['spec', 'values'])
> +
> + def __init__(self, attr_space, attrs, outer = None):
> + outer_scopes = outer.scopes if outer else []
> + inner_scope = self.SpecValuesPair(attr_space, attrs)
> + self.scopes = [inner_scope] + outer_scopes
> +
> + def lookup(self, name):
> + for scope in self.scopes:
> + if name in scope.spec:
> + if name in scope.values:
> + return scope.values[name]
> + spec_name = scope.spec.yaml['name']
> + raise Exception(
> + f"No value for '{name}' in attribute space '{spec_name}'")
> + raise Exception(f"Attribute '{name}' not defined in any attribute-set")
> +
> +
> +#
> +# YNL implementation details.
> +#
> +
> +
> +class YnlFamily(SpecFamily):
> + def __init__(self, def_path, schema=None, process_unknown=False):
> + super().__init__(def_path, schema)
> +
> + self.include_raw = False
> + self.process_unknown = process_unknown
> +
> + try:
> + if self.proto == "netlink-raw":
> + self.nlproto = NetlinkProtocol(self.yaml['name'],
> + self.yaml['protonum'])
> + else:
> + self.nlproto = GenlProtocol(self.yaml['name'])
> + except KeyError:
> + raise Exception(f"Family '{self.yaml['name']}' not supported by the kernel")
> +
> + self.sock = socket.socket(socket.AF_NETLINK, socket.SOCK_RAW, self.nlproto.proto_num)
> + self.sock.setsockopt(Netlink.SOL_NETLINK, Netlink.NETLINK_CAP_ACK, 1)
> + self.sock.setsockopt(Netlink.SOL_NETLINK, Netlink.NETLINK_EXT_ACK, 1)
> + self.sock.setsockopt(Netlink.SOL_NETLINK, Netlink.NETLINK_GET_STRICT_CHK, 1)
> +
> + self.async_msg_ids = set()
> + self.async_msg_queue = []
> +
> + for msg in self.msgs.values():
> + if msg.is_async:
> + self.async_msg_ids.add(msg.rsp_value)
> +
> + for op_name, op in self.ops.items():
> + bound_f = functools.partial(self._op, op_name)
> + setattr(self, op.ident_name, bound_f)
> +
> +
> + def ntf_subscribe(self, mcast_name):
> + mcast_id = self.nlproto.get_mcast_id(mcast_name, self.mcast_groups)
> + self.sock.bind((0, 0))
> + self.sock.setsockopt(Netlink.SOL_NETLINK, Netlink.NETLINK_ADD_MEMBERSHIP,
> + mcast_id)
> +
> + def _add_attr(self, space, name, value, search_attrs):
> + try:
> + attr = self.attr_sets[space][name]
> + except KeyError:
> + raise Exception(f"Space '{space}' has no attribute '{name}'")
> + nl_type = attr.value
> +
> + if attr.is_multi and isinstance(value, list):
> + attr_payload = b''
> + for subvalue in value:
> + attr_payload += self._add_attr(space, name, subvalue, search_attrs)
> + return attr_payload
> +
> + if attr["type"] == 'nest':
> + nl_type |= Netlink.NLA_F_NESTED
> + attr_payload = b''
> + sub_attrs = SpaceAttrs(self.attr_sets[space], value, search_attrs)
> + for subname, subvalue in value.items():
> + attr_payload += self._add_attr(attr['nested-attributes'],
> + subname, subvalue, sub_attrs)
> + elif attr["type"] == 'flag':
> + attr_payload = b''
> + elif attr["type"] == 'string':
> + attr_payload = str(value).encode('ascii') + b'\x00'
> + elif attr["type"] == 'binary':
> + if isinstance(value, bytes):
> + attr_payload = value
> + elif isinstance(value, str):
> + attr_payload = bytes.fromhex(value)
> + elif isinstance(value, dict) and attr.struct_name:
> + attr_payload = self._encode_struct(attr.struct_name, value)
> + else:
> + raise Exception(f'Unknown type for binary attribute, value: {value}')
> + elif attr.is_auto_scalar:
> + scalar = int(value)
> + real_type = attr["type"][0] + ('32' if scalar.bit_length() <= 32 else '64')
> + format = NlAttr.get_format(real_type, attr.byte_order)
> + attr_payload = format.pack(int(value))
> + elif attr['type'] in NlAttr.type_formats:
> + format = NlAttr.get_format(attr['type'], attr.byte_order)
> + attr_payload = format.pack(int(value))
> + elif attr['type'] in "bitfield32":
> + attr_payload = struct.pack("II", int(value["value"]), int(value["selector"]))
> + elif attr['type'] == 'sub-message':
> + msg_format = self._resolve_selector(attr, search_attrs)
> + attr_payload = b''
> + if msg_format.fixed_header:
> + attr_payload += self._encode_struct(msg_format.fixed_header, value)
> + if msg_format.attr_set:
> + if msg_format.attr_set in self.attr_sets:
> + nl_type |= Netlink.NLA_F_NESTED
> + sub_attrs = SpaceAttrs(msg_format.attr_set, value, search_attrs)
> + for subname, subvalue in value.items():
> + attr_payload += self._add_attr(msg_format.attr_set,
> + subname, subvalue, sub_attrs)
> + else:
> + raise Exception(f"Unknown attribute-set '{msg_format.attr_set}'")
> + else:
> + raise Exception(f'Unknown type at {space} {name} {value} {attr["type"]}')
> +
> + pad = b'\x00' * ((4 - len(attr_payload) % 4) % 4)
> + return struct.pack('HH', len(attr_payload) + 4, nl_type) + attr_payload + pad
> +
> + def _decode_enum(self, raw, attr_spec):
> + enum = self.consts[attr_spec['enum']]
> + if enum.type == 'flags' or attr_spec.get('enum-as-flags', False):
> + i = 0
> + value = set()
> + while raw:
> + if raw & 1:
> + value.add(enum.entries_by_val[i].name)
> + raw >>= 1
> + i += 1
> + else:
> + value = enum.entries_by_val[raw].name
> + return value
> +
> + def _decode_binary(self, attr, attr_spec):
> + if attr_spec.struct_name:
> + decoded = self._decode_struct(attr.raw, attr_spec.struct_name)
> + elif attr_spec.sub_type:
> + decoded = attr.as_c_array(attr_spec.sub_type)
> + else:
> + decoded = attr.as_bin()
> + if attr_spec.display_hint:
> + decoded = self._formatted_string(decoded, attr_spec.display_hint)
> + return decoded
> +
> + def _decode_array_nest(self, attr, attr_spec):
> + decoded = []
> + offset = 0
> + while offset < len(attr.raw):
> + item = NlAttr(attr.raw, offset)
> + offset += item.full_len
> +
> + subattrs = self._decode(NlAttrs(item.raw), attr_spec['nested-attributes'])
> + decoded.append({ item.type: subattrs })
> + return decoded
> +
> + def _decode_unknown(self, attr):
> + if attr.is_nest:
> + return self._decode(NlAttrs(attr.raw), None)
> + else:
> + return attr.as_bin()
> +
> + def _rsp_add(self, rsp, name, is_multi, decoded):
> + if is_multi == None:
> + if name in rsp and type(rsp[name]) is not list:
> + rsp[name] = [rsp[name]]
> + is_multi = True
> + else:
> + is_multi = False
> +
> + if not is_multi:
> + rsp[name] = decoded
> + elif name in rsp:
> + rsp[name].append(decoded)
> + else:
> + rsp[name] = [decoded]
> +
> + def _resolve_selector(self, attr_spec, search_attrs):
> + sub_msg = attr_spec.sub_message
> + if sub_msg not in self.sub_msgs:
> + raise Exception(f"No sub-message spec named {sub_msg} for {attr_spec.name}")
> + sub_msg_spec = self.sub_msgs[sub_msg]
> +
> + selector = attr_spec.selector
> + value = search_attrs.lookup(selector)
> + if value not in sub_msg_spec.formats:
> + raise Exception(f"No message format for '{value}' in sub-message spec '{sub_msg}'")
> +
> + spec = sub_msg_spec.formats[value]
> + return spec
> +
> + def _decode_sub_msg(self, attr, attr_spec, search_attrs):
> + msg_format = self._resolve_selector(attr_spec, search_attrs)
> + decoded = {}
> + offset = 0
> + if msg_format.fixed_header:
> + decoded.update(self._decode_struct(attr.raw, msg_format.fixed_header));
> + offset = self._struct_size(msg_format.fixed_header)
> + if msg_format.attr_set:
> + if msg_format.attr_set in self.attr_sets:
> + subdict = self._decode(NlAttrs(attr.raw, offset), msg_format.attr_set)
> + decoded.update(subdict)
> + else:
> + raise Exception(f"Unknown attribute-set '{attr_space}' when decoding '{attr_spec.name}'")
> + return decoded
> +
> + def _decode(self, attrs, space, outer_attrs = None):
> + if space:
> + attr_space = self.attr_sets[space]
> + rsp = dict()
> + search_attrs = SpaceAttrs(attr_space, rsp, outer_attrs)
> +
> + for attr in attrs:
> + try:
> + attr_spec = attr_space.attrs_by_val[attr.type]
> + except (KeyError, UnboundLocalError):
> + if not self.process_unknown:
> + raise Exception(f"Space '{space}' has no attribute with value '{attr.type}'")
> + attr_name = f"UnknownAttr({attr.type})"
> + self._rsp_add(rsp, attr_name, None, self._decode_unknown(attr))
> + continue
> +
> + if attr_spec["type"] == 'nest':
> + subdict = self._decode(NlAttrs(attr.raw), attr_spec['nested-attributes'], search_attrs)
> + decoded = subdict
> + elif attr_spec["type"] == 'string':
> + decoded = attr.as_strz()
> + elif attr_spec["type"] == 'binary':
> + decoded = self._decode_binary(attr, attr_spec)
> + elif attr_spec["type"] == 'flag':
> + decoded = True
> + elif attr_spec.is_auto_scalar:
> + decoded = attr.as_auto_scalar(attr_spec['type'], attr_spec.byte_order)
> + elif attr_spec["type"] in NlAttr.type_formats:
> + decoded = attr.as_scalar(attr_spec['type'], attr_spec.byte_order)
> + if 'enum' in attr_spec:
> + decoded = self._decode_enum(decoded, attr_spec)
> + elif attr_spec["type"] == 'array-nest':
> + decoded = self._decode_array_nest(attr, attr_spec)
> + elif attr_spec["type"] == 'bitfield32':
> + value, selector = struct.unpack("II", attr.raw)
> + if 'enum' in attr_spec:
> + value = self._decode_enum(value, attr_spec)
> + selector = self._decode_enum(selector, attr_spec)
> + decoded = {"value": value, "selector": selector}
> + elif attr_spec["type"] == 'sub-message':
> + decoded = self._decode_sub_msg(attr, attr_spec, search_attrs)
> + else:
> + if not self.process_unknown:
> + raise Exception(f'Unknown {attr_spec["type"]} with name {attr_spec["name"]}')
> + decoded = self._decode_unknown(attr)
> +
> + self._rsp_add(rsp, attr_spec["name"], attr_spec.is_multi, decoded)
> +
> + return rsp
> +
> + def _decode_extack_path(self, attrs, attr_set, offset, target):
> + for attr in attrs:
> + try:
> + attr_spec = attr_set.attrs_by_val[attr.type]
> + except KeyError:
> + raise Exception(f"Space '{attr_set.name}' has no attribute with value '{attr.type}'")
> + if offset > target:
> + break
> + if offset == target:
> + return '.' + attr_spec.name
> +
> + if offset + attr.full_len <= target:
> + offset += attr.full_len
> + continue
> + if attr_spec['type'] != 'nest':
> + raise Exception(f"Can't dive into {attr.type} ({attr_spec['name']}) for extack")
> + offset += 4
> + subpath = self._decode_extack_path(NlAttrs(attr.raw),
> + self.attr_sets[attr_spec['nested-attributes']],
> + offset, target)
> + if subpath is None:
> + return None
> + return '.' + attr_spec.name + subpath
> +
> + return None
> +
> + def _decode_extack(self, request, op, extack):
> + if 'bad-attr-offs' not in extack:
> + return
> +
> + msg = self.nlproto.decode(self, NlMsg(request, 0, op.attr_set))
> + offset = 20 + self._struct_size(op.fixed_header)
> + path = self._decode_extack_path(msg.raw_attrs, op.attr_set, offset,
> + extack['bad-attr-offs'])
> + if path:
> + del extack['bad-attr-offs']
> + extack['bad-attr'] = path
> +
> + def _struct_size(self, name):
> + if name:
> + members = self.consts[name].members
> + size = 0
> + for m in members:
> + if m.type in ['pad', 'binary']:
> + if m.struct:
> + size += self._struct_size(m.struct)
> + else:
> + size += m.len
> + else:
> + format = NlAttr.get_format(m.type, m.byte_order)
> + size += format.size
> + return size
> + else:
> + return 0
> +
> + def _decode_struct(self, data, name):
> + members = self.consts[name].members
> + attrs = dict()
> + offset = 0
> + for m in members:
> + value = None
> + if m.type == 'pad':
> + offset += m.len
> + elif m.type == 'binary':
> + if m.struct:
> + len = self._struct_size(m.struct)
> + value = self._decode_struct(data[offset : offset + len],
> + m.struct)
> + offset += len
> + else:
> + value = data[offset : offset + m.len]
> + offset += m.len
> + else:
> + format = NlAttr.get_format(m.type, m.byte_order)
> + [ value ] = format.unpack_from(data, offset)
> + offset += format.size
> + if value is not None:
> + if m.enum:
> + value = self._decode_enum(value, m)
> + elif m.display_hint:
> + value = self._formatted_string(value, m.display_hint)
> + attrs[m.name] = value
> + return attrs
> +
> + def _encode_struct(self, name, vals):
> + members = self.consts[name].members
> + attr_payload = b''
> + for m in members:
> + value = vals.pop(m.name) if m.name in vals else None
> + if m.type == 'pad':
> + attr_payload += bytearray(m.len)
> + elif m.type == 'binary':
> + if m.struct:
> + if value is None:
> + value = dict()
> + attr_payload += self._encode_struct(m.struct, value)
> + else:
> + if value is None:
> + attr_payload += bytearray(m.len)
> + else:
> + attr_payload += bytes.fromhex(value)
> + else:
> + if value is None:
> + value = 0
> + format = NlAttr.get_format(m.type, m.byte_order)
> + attr_payload += format.pack(value)
> + return attr_payload
> +
> + def _formatted_string(self, raw, display_hint):
> + if display_hint == 'mac':
> + formatted = ':'.join('%02x' % b for b in raw)
> + elif display_hint == 'hex':
> + formatted = bytes.hex(raw, ' ')
> + elif display_hint in [ 'ipv4', 'ipv6' ]:
> + formatted = format(ipaddress.ip_address(raw))
> + elif display_hint == 'uuid':
> + formatted = str(uuid.UUID(bytes=raw))
> + else:
> + formatted = raw
> + return formatted
> +
> + def handle_ntf(self, decoded):
> + msg = dict()
> + if self.include_raw:
> + msg['raw'] = decoded
> + op = self.rsp_by_value[decoded.cmd()]
> + attrs = self._decode(decoded.raw_attrs, op.attr_set.name)
> + if op.fixed_header:
> + attrs.update(self._decode_struct(decoded.raw, op.fixed_header))
> +
> + msg['name'] = op['name']
> + msg['msg'] = attrs
> + self.async_msg_queue.append(msg)
> +
> + def check_ntf(self):
> + while True:
> + try:
> + reply = self.sock.recv(128 * 1024, socket.MSG_DONTWAIT)
> + except BlockingIOError:
> + return
> +
> + nms = NlMsgs(reply)
> + for nl_msg in nms:
> + if nl_msg.error:
> + print("Netlink error in ntf!?", os.strerror(-nl_msg.error))
> + print(nl_msg)
> + continue
> + if nl_msg.done:
> + print("Netlink done while checking for ntf!?")
> + continue
> +
> + decoded = self.nlproto.decode(self, nl_msg)
> + if decoded.cmd() not in self.async_msg_ids:
> + print("Unexpected msg id done while checking for ntf", decoded)
> + continue
> +
> + self.handle_ntf(decoded)
> +
> + def operation_do_attributes(self, name):
> + """
> + For a given operation name, find and return a supported
> + set of attributes (as a dict).
> + """
> + op = self.find_operation(name)
> + if not op:
> + return None
> +
> + return op['do']['request']['attributes'].copy()
> +
> + def _op(self, method, vals, flags=None, dump=False):
> + op = self.ops[method]
> +
> + nl_flags = Netlink.NLM_F_REQUEST | Netlink.NLM_F_ACK
> + for flag in flags or []:
> + nl_flags |= flag
> + if dump:
> + nl_flags |= Netlink.NLM_F_DUMP
> +
> + req_seq = random.randint(1024, 65535)
> + msg = self.nlproto.message(nl_flags, op.req_value, 1, req_seq)
> + if op.fixed_header:
> + msg += self._encode_struct(op.fixed_header, vals)
> + search_attrs = SpaceAttrs(op.attr_set, vals)
> + for name, value in vals.items():
> + msg += self._add_attr(op.attr_set.name, name, value, search_attrs)
> + msg = _genl_msg_finalize(msg)
> +
> + self.sock.send(msg, 0)
> +
> + done = False
> + rsp = []
> + while not done:
> + reply = self.sock.recv(128 * 1024)
> + nms = NlMsgs(reply, attr_space=op.attr_set)
> + for nl_msg in nms:
> + if nl_msg.extack:
> + self._decode_extack(msg, op, nl_msg.extack)
> +
> + if nl_msg.error:
> + raise NlError(nl_msg)
> + if nl_msg.done:
> + if nl_msg.extack:
> + print("Netlink warning:")
> + print(nl_msg)
> + done = True
> + break
> +
> + decoded = self.nlproto.decode(self, nl_msg)
> +
> + # Check if this is a reply to our request
> + if nl_msg.nl_seq != req_seq or decoded.cmd() != op.rsp_value:
> + if decoded.cmd() in self.async_msg_ids:
> + self.handle_ntf(decoded)
> + continue
> + else:
> + print('Unexpected message: ' + repr(decoded))
> + continue
> +
> + rsp_msg = self._decode(decoded.raw_attrs, op.attr_set.name)
> + if op.fixed_header:
> + rsp_msg.update(self._decode_struct(decoded.raw, op.fixed_header))
> + rsp.append(rsp_msg)
> +
> + if not rsp:
> + return None
> + if not dump and len(rsp) == 1:
> + return rsp[0]
> + return rsp
> +
> + def do(self, method, vals, flags=None):
> + return self._op(method, vals, flags)
> +
> + def dump(self, method, vals):
> + return self._op(method, vals, [], dump=True)
--
- Stefan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/kernel-team/attachments/20240229/1d0d9631/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 9718 bytes
Desc: not available
URL: <https://lists.ubuntu.com/archives/kernel-team/attachments/20240229/1d0d9631/attachment-0001.bin>
More information about the kernel-team
mailing list