[apparmor] backtrace from logparser.py prefetch_next_log_line

[Kshitij Gupta]

Hello,

On Sun, Sep 14, 2014 at 1:55 AM, Christian Boltz <apparmor at cboltz.de> wrote:
> Hello,
>
> while playing around with aa-logprof, I got the following backtrace.
>
> # python3 aa-logprof
> Reading log entries from /var/log/messages.
> Updating AppArmor profiles in /etc/apparmor.d.
> Traceback (most recent call last):
>   File "aa-logprof", line 52, in <module>
>     apparmor.do_logprof_pass(logmark)
>   File "/home/cb/apparmor/HEAD-CLEAN/utils/apparmor/aa.py", line 2275, in do_logprof_pass
>     log = log_reader.read_log(logmark)
>   File "/home/cb/apparmor/HEAD-CLEAN/utils/apparmor/logparser.py", line 339, in read_log
>     line = self.get_next_log_entry()
>   File "/home/cb/apparmor/HEAD-CLEAN/utils/apparmor/logparser.py", line 71, in get_next_log_entry
>     self.prefetch_next_log_entry()
>   File "/home/cb/apparmor/HEAD-CLEAN/utils/apparmor/logparser.py", line 64, in prefetch_next_log_entry
>     self.next_log_entry = self.LOG.readline()
>   File "/usr/lib64/python3.4/codecs.py", line 695, in readline
>     return self.reader.readline(size)
>   File "/usr/lib64/python3.4/codecs.py", line 548, in readline
>     data = self.read(readsize, firstline=True)
>   File "/usr/lib64/python3.4/codecs.py", line 494, in read
>     newchars, decodedbytes = self.decode(data, self.errors)
> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position 123: invalid start byte
>
>
Encodings cant let one live in peace.

> Maybe it would be a good idea not to assume any encoding and use raw
> bytes instead? (Patches welcome ;-)
>
> You can use   aa-logprof -f testfile   (attached) to reproduce this bug.
>

Thanks for the test file. :-)

I've tried a tiny hack-ish fix (based on your open('testfile', 'rb')
patch). The strings given in byte can be typecasted to string due to
which,

b'2014-07-12T19:44:19.023487+02:00 geeko
org.gtk.Private.GoaVolumeMonitor[9603]: g_dbus_connection_real_closed:
Remote peer vanished with error: Fehler beim Empfang der Nachricht:
Die Verbindung wurde vom Kommunikationspartner zur\xfcckgesetzt
(g-io-error-quark, 0). Exiting.\r\n'

becomes:

"b'2014-07-12T19:44:19.023487+02:00 geeko
org.gtk.Private.GoaVolumeMonitor[9603]: g_dbus_connection_real_closed:
Remote peer vanished with error: Fehler beim Empfang der Nachricht:
Die Verbindung wurde vom Kommunikationspartner zur\\xfcckgesetzt
(g-io-error-quark, 0). Exiting.\\r\\n'"

notice the prefixed b' and suffixed ' . This happens only in Python3
for Python2 no prefix or suffix are added.
After typecasting to string the regex's can be applied and hopefully
they can ignore the prefix and suffix part and find the relevant
things or maybe we can update the regex to do so?

Opinions?

Regards,

Kshitij Gupta


=== modified file 'utils/apparmor/logparser.py'
--- utils/apparmor/logparser.py    2014-08-20 22:55:44 +0000
+++ utils/apparmor/logparser.py    2014-09-14 14:43:37 +0000
@@ -59,7 +59,7 @@
     def prefetch_next_log_entry(self):
         if self.next_log_entry:
             sys.stderr.out('A log entry already present: %s' %
self.next_log_entry)
-        self.next_log_entry = self.LOG.readline()
+        self.next_log_entry = str(self.LOG.readline())
         while not self.RE_LOG_v2_6_syslog.search(self.next_log_entry)
and not self.RE_LOG_v2_6_audit.search(self.next_log_entry) and not
(self.logmark and self.logmark in self.next_log_entry):
             self.next_log_entry = self.LOG.readline()
             if not self.next_log_entry:
@@ -330,7 +330,7 @@
         #event_type = None
         try:
             #print(self.filename)
-            self.LOG = open_file_read(self.filename)
+            self.LOG = open(self.filename, 'rb')
         except IOError:
             raise AppArmorException('Can not read AppArmor logfile: '
+ self.filename)
         #LOG = open_file_read(log_open)


>
> Regards,
>
> Christian Boltz
> --
> Yes, I know how much devs hate writing documentation... I was a dev.
> [Carlos E. R. in opensuse-factory]
>
> --
> AppArmor mailing list
> AppArmor at lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/apparmor
>