[Bug 1861440] Re: [upstream regression] wget does not honor dot-prefixed domains in no_proxy env variable
Bug Watch Updater
1861440 at bugs.launchpad.net
Mon Sep 21 20:14:31 UTC 2020
Launchpad has imported 7 comments from the remote bug at
https://bugzilla.redhat.com/show_bug.cgi?id=1763702.
If you reply to an imported comment from within Launchpad, your comment
will be sent to the remote bug automatically. Read more about
Launchpad's inter-bugtracker facilities at
https://help.launchpad.net/InterBugTracking.
------------------------------------------------------------------------
On 2019-10-21T11:43:39+00:00 fperalta wrote:
Description of problem:
Version-Release number of selected component (if applicable):
How reproducible:
Steps to Reproduce:
1.
2.
3.
Actual results:
Expected results:
Additional info:
Reply at:
https://bugs.launchpad.net/ubuntu/+source/wget/+bug/1861440/comments/0
------------------------------------------------------------------------
On 2019-10-21T11:52:43+00:00 fperalta wrote:
Sorry I hit enter to fast..
(In reply to Francisco Peralta from comment #0)
> Description of problem:
wget does not correctly use the no_proxy variable in RHEL 8
> Version-Release number of selected component (if applicable):
wget-1.19.5-7.el8_0.1.x86_64
> How reproducible:
Always.
If using a newer wget 1.20.3 or if using older wget from RHEL 7 I do not reproduce the issue.
> Steps to Reproduce:
1. $ export http_proxy=http://www.notexisting.com:8080
2. $ export no_proxy=localhost,.redhat.com
3. $ wget www.redhat.com
> Actual results:
--2019-10-21 13:09:46-- http://www.redhat.com/
Resolving www.nonexisting.com (www.nonexisting.com)... 192.249.111.222
Connecting to www.nonexisting.com (www.nonexsisting.com)|192.249.111.222|:8080... ^C
> Expected results:
--2019-10-21 13:50:42-- http://www.redhat.com/
Resolving www.redhat.com (www.redhat.com)... 2a02:26f0:97:181::d44, 2a02:26f0:97:19d::d44, 23.2.233.53
Connecting to www.redhat.com (www.redhat.com)|2a02:26f0:97:181::d44|:80... failed: Network is unreachable.
Connecting to www.redhat.com (www.redhat.com)|2a02:26f0:97:19d::d44|:80... failed: Network is unreachable.
Connecting to www.redhat.com (www.redhat.com)|23.2.233.53|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://www.redhat.com/en [following]
--2019-10-21 13:50:42-- https://www.redhat.com/en
Connecting to www.redhat.com (www.redhat.com)|23.2.233.53|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘index.html’
index.html [ <=>
] 71.39K --.-KB/s in 0.04s
2019-10-21 13:50:42 (1.90 MB/s) - ‘index.html’ saved [73101]
Converting links in index.html... 20-131
Converted links in 1 files in 0.004 seconds.
> Additional info:
The issue must have been fixed in the upstream community, I could not
find the exact place nor the root cause of introducing it, but I think
it's a matter of updating the RHEL 8 version of wget.
Reply at:
https://bugs.launchpad.net/ubuntu/+source/wget/+bug/1861440/comments/1
------------------------------------------------------------------------
On 2019-11-07T16:44:29+00:00 thozza wrote:
During writing a test for the issue, I found a corner case which is
still not fixed in upstream. I'll wait for their response first.
For more information, please see https://lists.gnu.org/archive/html/bug-
wget/2019-11/msg00011.html
Reply at:
https://bugs.launchpad.net/ubuntu/+source/wget/+bug/1861440/comments/2
------------------------------------------------------------------------
On 2019-11-13T21:23:01+00:00 shawn wrote:
The GNU Emacs manual covers this corner case in at least a little more
detail. See
https://www.gnu.org/software/emacs/manual/html_node/url/Proxies.html
As per your post, correlated with the EMACS manual, if you want the
"host" mit.edu to not use the proxy, you would need to add it to
no_proxy (so no_proxy=.mit.edu,mit.edu).
I'd really like to see this bug fixed as our local proxy server actually
denies requests to proxy internally, so without changing proxy vars
(that we've been using for over a decade), wget to our internal websites
just doesn't work.
Reply at:
https://bugs.launchpad.net/ubuntu/+source/wget/+bug/1861440/comments/3
------------------------------------------------------------------------
On 2019-11-15T14:46:15+00:00 shawn wrote:
Just a few links for extra documentation.
This regression in wget appears to have gone into changes in src/host.c with commit 55d25fc20c0141cb7cb8bd0a6964b81aa0b50124 on 2018-01-07 and was released with wget 1.19.3.
http://git.savannah.gnu.org/cgit/wget.git/commit/?id=fd85ac9cc623847e9d94d9f9241ab34e2c146cbf
This was reported upstream in #53622
(https://savannah.gnu.org/bugs/?53622).
Although not acknowledged in the upstream issue tracker, this appears fixed in upstream commit fd85ac9cc623847e9d94d9f9241ab34e2c146cbf on 2018-10-25.
http://git.savannah.gnu.org/cgit/wget.git/commit/?id=fd85ac9cc623847e9d94d9f9241ab34e2c146cbf
According to git tags in the wget repo, 1.19.5 was release 2018-05-06,
so this fix wasn't included until the 1.20 release on 2018-11-13.
Reply at:
https://bugs.launchpad.net/ubuntu/+source/wget/+bug/1861440/comments/4
------------------------------------------------------------------------
On 2019-11-21T15:48:37+00:00 thozza wrote:
So the no_proxy correct behavior in wget is for a longer discussion and
since there is no standard, it is impossible to get it completely right.
For this reason I decided to continue the discussion and possible
changes in upstream, but will backport the current current upstream wget
behavior to RHEL-8 in order to solve this pressing customer issue as
soon as possible.
Reply at:
https://bugs.launchpad.net/ubuntu/+source/wget/+bug/1861440/comments/5
------------------------------------------------------------------------
On 2019-11-22T10:09:51+00:00 fperalta wrote:
Thank you Tomás,
Yes, I think the most important thing now is to make sure the behaviour of no_proxy is again consistent with previous versions.
Then discussions about what is the standard to use and agree on it can be taken and if then eventually those will be different, it will be announced in release notes in advance and proper way.
Kind Regards,
Cisco.
Reply at:
https://bugs.launchpad.net/ubuntu/+source/wget/+bug/1861440/comments/6
** Changed in: wget (CentOS)
Importance: Unknown => Critical
** Bug watch added: GNU Savannah Bug Tracker #53622
http://savannah.gnu.org/bugs/?53622
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to wget in Ubuntu.
https://bugs.launchpad.net/bugs/1861440
Title:
[upstream regression] wget does not honor dot-prefixed domains in
no_proxy env variable
Status in wget package in Ubuntu:
New
Status in wget package in CentOS:
Unknown
Bug description:
Traditionally (AFAIK for at least the last decade), tools that support
the no_proxy environment variable support specifying an entire
subdomain by prefixing it with a "dot". For example, to exclude any
website under example.com from using the proxy, you would set no_proxy
to .example.com (export no_proxy=.example.com).
A regression in wget 1.19 changed this behavior to expect non-prefixed
domains (example.com vs .example.com). This regression was ultimately
fixed and released with the 1.20 release of wget. bionic includes the
regressed behavior version of wget.
The regression was apparently introduced in wget 1.19.3. This bug
should not effect other Ubuntu releases (xenial contains 1.17.1 and
both disco and eoan contain 1.20.x versions that have the upstream
fix).
For more details, see references below and my additional comments on
the RHEL8 bug filed for this issue (RH bug 1763702 linked below).
What happens:
no_proxy=.example.com in bionic sends requests to the proxy server for URLs like http://www.example.com/ despite requesting proxy exception via no_proxy.
What should happen:
Request should bypass the proxy and go directly to the web server. (works in xenial, disco and eoan as expected).
System/software information:
$ lsb_release -rd
Description: Ubuntu 18.04.3 LTS
Release: 18.04
$ apt-cache policy wget
wget:
Installed: 1.19.4-1ubuntu2.2
Candidate: 1.19.4-1ubuntu2.2
Version table:
*** 1.19.4-1ubuntu2.2 500
500 http://div6mirrors.llan.ll.mit.edu:80/ubuntu bionic-updates/main amd64 Packages
500 http://div6mirrors.llan.ll.mit.edu:80/ubuntu bionic-security/main amd64 Packages
100 /var/lib/dpkg/status
1.19.4-1ubuntu2 500
500 http://div6mirrors.llan.ll.mit.edu:80/ubuntu bionic/main amd64 Packages
References:
* Upstream wget bug report:
GNU Wget - Bugs: bug #53622 wget no_proxy leading dot on (sub)domains not working contradicting man page
https://savannah.gnu.org/bugs/?53622
* Upstream commit reference that introduces the regression
http://git.savannah.gnu.org/cgit/wget.git/commit/?id=fd85ac9cc623847e9d94d9f9241ab34e2c146cbf
* Upstream commit reference that introduces the fix
http://git.savannah.gnu.org/cgit/wget.git/commit/?id=fd85ac9cc623847e9d94d9f9241ab34e2c146cbf
* Expected behavior of no_proxy as documented in the GNU Emacs manual:
https://www.gnu.org/software/emacs/manual/html_node/url/Proxies.html
* Red Hat Bugzilla entry for this issue (Reported against RHEL8.1)
Bug 1763702 - wget is ignoring no_proxy environment variable
https://bugzilla.redhat.com/show_bug.cgi?id=1763702
* Red Hat Bugzilla entry tracking the (now released) errata package for RHEL8.1
Bug 1772821 - wget is ignoring no_proxy environment variable [rhel-8.1.0.z]
https://bugzilla.redhat.com/show_bug.cgi?id=1772821
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/wget/+bug/1861440/+subscriptions
More information about the foundations-bugs
mailing list