[Bug 1628761] Re: [2.1] netaddr assumes MAC OUI is ascii

[Gavin Panella]

FWIW, http://paste.ubuntu.com/23275209/ (a short Python 3 script) makes
me fairly sure that everything in netaddr's oui.txt is UTF-8. The script
tries to guess the encoding of each entry using chardet, ignores those
guessed as ASCII or UTF-8, then prints each entry after being decoded
using the guessed encoding and again using UTF-8. The UTF-8 result
always appears to be the better choice:
http://paste.ubuntu.com/23275206/.

I mention this because it means that an upstream fix is probably easy,
and because we may be able to monkeypatch something in the meantime. I
do think the defensive approach you've taken in lp:~mpontillo/maas
/netaddr-assumes-mac-oui-ascii--bug-1628761 is the right one for now
though.

Of course, the example in the bug description is an exception to the
rule: I can't find anything to decode that and produce something
reasonable, though UTF-8 does at least offer a non-crashing option. I
suspect it's been mangled, maybe by copy-and-paste at some indeterminate
time in the past, and the original is now as good as unrecoverable from
oui.txt. Similar corruption appears in the oui.txt that the ieee-data
package installs.

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to python-netaddr in Ubuntu.
https://bugs.launchpad.net/bugs/1628761

Title:
  [2.1] netaddr assumes MAC OUI is ascii

Status in MAAS:
  Triaged
Status in python-netaddr package in Ubuntu:
  New

Bug description:
  This causes a traceback when we try to get the MAC organization, as
  follows:

      --- <exception caught here> ---
        File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 246, in inContext
          result = inContext.theWork()
        File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 262, in <lambda>
          inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
        File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 118, in callWithContext
          return self.currentContext().callWithContext(ctx, func, *args, **kw)
        File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 81, in callWithContext
          return func(*args,**kw)
        File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 827, in callInContext
          return func(*args, **kwargs)
        File "/usr/lib/python3/dist-packages/maasserver/utils/orm.py", line 603, in call_within_transaction
          return func_outside_txn(*args, **kwargs)
        File "/usr/lib/python3/dist-packages/maasserver/utils/orm.py", line 422, in retrier
          return func(*args, **kwargs)
        File "/usr/lib/python3.5/contextlib.py", line 30, in inner
          return func(*args, **kwds)
        File "/usr/lib/python3/dist-packages/maasserver/websockets/base.py", line 356, in list
          for obj in objs
        File "/usr/lib/python3/dist-packages/maasserver/websockets/base.py", line 356, in <listcomp>
          for obj in objs
        File "/usr/lib/python3/dist-packages/maasserver/websockets/base.py", line 205, in full_dehydrate
          return self.dehydrate(obj, data, for_list=for_list)
        File "/usr/lib/python3/dist-packages/maasserver/websockets/handlers/discovery.py", line 38, in dehydrate
          data["mac_organization"] = obj.mac_organization
        File "/usr/lib/python3/dist-packages/maasserver/models/discovery.py", line 228, in mac_organization
          return get_mac_organization(str(self.mac_address))
        File "/usr/lib/python3/dist-packages/provisioningserver/utils/network.py", line 817, in get_mac_organization
          return get_eui_organization(EUI(mac))
        File "/usr/lib/python3/dist-packages/provisioningserver/utils/network.py", line 802, in get_eui_organization
          registration = eui.oui.registration()
        File "/usr/lib/python3/dist-packages/netaddr/eui/__init__.py", line 478, in oui
          return OUI(self.value >> 24)
        File "/usr/lib/python3/dist-packages/netaddr/eui/__init__.py", line 97, in __init__
          data = fh.read(size)
        File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
          return codecs.ascii_decode(input, self.errors)[0]
      builtins.UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2420: ordinal not in range(128)

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1628761/+subscriptions