[Bug 1914481] Re: use the size of the data when determining the server response

[Brian Murray]

Focal recreation of the failure:

root at clean-focal-amd64:~# test/submit-crash test-crashes/hirsute/amd64/_usr_bin_apport-cli.2001.crash
Submitting test-crashes/hirsute/amd64/_usr_bin_apport-cli.2001.crash ...
NULL, Ubuntu 21.04, /usr/bin/apport-cli
root at clean-focal-amd64:~# apt-cache policy whoopsie
whoopsie:
  Installed: 0.2.69ubuntu0.2
...
[11:04:01] Parsing /var/crash/_usr_bin_apport-cli.2001.crash.
[11:04:01] Uploading /var/crash/_usr_bin_apport-cli.2001.crash.
[11:04:01] Sent; server replied with: No error
[11:04:01] Response code: 200
[11:04:01] Got command: OOPSID
0


Verification of the fix:
root at clean-focal-amd64:~# apt-cache policy whoopsie
whoopsie:
  Installed: 0.2.69ubuntu0.3
root at clean-focal-amd64:~# test/submit-crash test-crashes/hirsute/amd64/_usr_bin_apport-cli.2001.crash
Submitting test-crashes/hirsute/amd64/_usr_bin_apport-cli.2001.crash ...
e603db9e-6b09-11eb-aff9-00163ed65bda, Ubuntu 21.04, /usr/bin/apport-cli
[11:06:22] Parsing /var/crash/_usr_bin_apport-cli.2001.crash.
[11:06:22] Uploading /var/crash/_usr_bin_apport-cli.2001.crash.
[11:06:22] Sent; server replied with: No error
[11:06:22] Response code: 200
[11:06:22] Reported OOPS ID e603db9e-6b09-11eb-aff9-00163ed65bda

Additional test using production error tracker:
bdmurray at clean-focal-amd64:~$ sudo CRASH_DB_URL=https://daisy.ubuntu.com whoopsie -f
[11:07:37] Using lock path: /var/lock/whoopsie/lock
[11:07:37] The default IPv4 route is: /org/freedesktop/NetworkManager/ActiveConnection/1
[11:07:37] Not a paid data plan: /org/freedesktop/NetworkManager/ActiveConnection/1
[11:07:37] Found usable connection: /org/freedesktop/NetworkManager/ActiveConnection/1
[11:07:41] Parsing /var/crash/_usr_bin_apport-cli.2001.crash.
[11:07:41] Uploading /var/crash/_usr_bin_apport-cli.2001.crash.
[11:07:55] Sent; server replied with: No error
[11:07:55] Response code: 200
[11:07:55] Reported OOPS ID 1b83dec2-6b0a-11eb-8f66-fa163e6cac46


** Tags removed: verification-needed-focal
** Tags added: verification-done-focal

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to whoopsie in Ubuntu.
https://bugs.launchpad.net/bugs/1914481

Title:
  use the size of the data when determining the server response

Status in whoopsie package in Ubuntu:
  Fix Released
Status in whoopsie source package in Focal:
  Fix Committed
Status in whoopsie source package in Groovy:
  Fix Committed
Status in whoopsie source package in Hirsute:
  Fix Released

Bug description:
  [Note]
  This SRU has been built in a security-only PPA and should be released to both -updates and -security.

  [Impact]
  whoopsie's server_response code is using "g_string_append" instead of "g_string_append_len" which has the knock on effect of sending too much data to its "handle_response". This ends up being a problem if the daisy servers are running on Ubuntu 18.04 instead of Ubuntu 16.04.

  Here's an example when using whoopsie on groovy to send a crash to a
  bionic daisy server:

  [15:35:30] Sent; server replied with: No error
  [15:35:30] Response code: 200
  [15:35:30] Initial response data is: 2bbb776e-64e6-11eb-a8d6-00163eddedf4 OOPSID
  0

  [15:35:30] Got command: OOPSID

  We can see a fair number of extra characters (\n0\n\n) after the OOSID
  command. This becomes more problematic when daisy requests a core dump
  from the client as the CORE command won't match and the client will
  never send the core dump.

  [Test Case]
  Setup a Bionic version of the Error Tracker:
  0) modify /etc/hosts so daisy.staging.ubuntu.com points to the IP of the apache server for daisy
  1) sudo service stop whoopsie
  2) sudo CRASH_DB_URL=https://daisy.staging.ubuntu.com whoopsie -f
  3) Run test/submit-crash test-crashes/hirsute/amd64/_bin_cat.2001.crash
  4) check the whoopsie log file for "Got command: OOPSID" and extra data.

  With the version of whoopsie from -proposed this will not happen.
  Additionally, a regression test should be run against the staging
  version of the error tracker by removing the entry from /etc/hosts for
  the daisy.staging server. After confirming that one test crash works
  one should also send a python crash, and an end of life release crash
  as those all generate different response codes from the server.

  [Regression Potential]
  The code being changed is clearly wrong and doesn't confirm to the curl API https://curl.se/libcurl/c/CURLOPT_WRITEFUNCTION.html. Additionallly, this is similar to the code before r707 of daisy which introduced this change so there is little chance of regression. That being said we are running a regression test to ensure whoopsie works with servers running Ubuntu 16.04.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/whoopsie/+bug/1914481/+subscriptions