[Bug 1618288] [NEW] wget using "if-modified-since" is not idempotent and corrupts downloaded copy of website on second use

[Launchpad Bug Tracker]

You have been subscribed to a public bug:

I use wget to copy a web site from one server to another, adjusting file
suffixes and paths.

Since updating to 16.04 LTS from 14.04 the command that I used
previously  has begun corrupting the destination site on second and
subsequent invocations.

The options relevant to the problem seem to be -N (use timestamping), -k
(convert links) and -E (adjust extensions). The problem arises with
linked files whose names do not end in .html. On the first invocation
everything is good: file foo.txt is downloaded and linked as foo.txt. On
the second invocation the wget log (option -v) suggests that it has
examined foo.txt on the server, but then it reports "File
'<copylocation>/foo.txt.html' not modified on server. Omitting
download." and then it changes the link in the referring file to
foo.txt.html.

I think this is a bug. Do others have an opinion?

Workaround: include the option "--no-if-modified-since" which seems to
restore the old, correct behavior.

Thanks.

P.S. The full command that misbehaves is: wget -nH -r -E -k -N -x -l inf
-P <destination for copy> "http://<source web site>"

** Affects: wget (Ubuntu)
     Importance: Undecided
         Status: New

-- 
wget using "if-modified-since" is not idempotent and corrupts downloaded copy of website on second use
https://bugs.launchpad.net/bugs/1618288
You received this bug notification because you are a member of Ubuntu Foundations Bugs, which is subscribed to wget in Ubuntu.