Bug#947915: release-notes: Suggest cleaning up leftover *.dpkg-old etc. config files

[Karl O. Pinc]

Package: release-notes
Severity: normal
Tags: patch

Hello,

Attached is a patch which suggests cleaning up unused config files
leftover from prior upgrades, the foo.dpkg-old and similar files,
before starting the new upgrade.

Regards,
Karl

-- System Information:
Debian Release: 10.2
APT prefers stable
APT policy: (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 4.19.0-6-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

[Justin B Rye]

Post by Karl O. Pinc
Attached is a patch which suggests cleaning up unused config files
leftover from prior upgrades, the foo.dpkg-old and similar files,
before starting the new upgrade.
--- a/en/upgrading.dbk
+++ b/en/upgrading.dbk
@@ -301,6 +301,17 @@ $ apt-forktracer | sort
It is a good idea to <link linkend="obsolete">remove obsolete
packages</link> from your system before upgrading.
</para>
+ <para>
+ A previous upgrade can have left unused copies of configuration

I'd recommend may have

Post by Karl O. Pinc
+ files; <link linkend="configuration-changes">old versions</link>
+ of configuration files, versions supplied by the package
+ maintainers, etc. Removing leftover files from previous upgrades,
+ before performing another upgrade, can avoid confusion.

The commas in this sentence seem subtly wrong.

Post by Karl O. Pinc
+ Find such
+ </para>
+ <screen>
+# find /etc -name '*.dpkg-*'
+ </screen>

But in fact if you're going to do this sweep (and haven't previously
been in the habit of doing it), doesn't it make more sense to learn it
as something to do *after* every (dist- or package-)upgrade? What
kinds of confusion does it risk if you do it then?

It might fit best in 4.7. "Preparing for the next release".

(Personally I have a cronjob that nags me if it sees any *.dpkg-new,
*.dpkg-old, *.dpkg-save, *.pam-old or *ucf-old files lying about.)

[Karl O. Pinc]

On Thu, 2 Jan 2020 01:45:16 +0000

Post by Justin B Rye
(Personally I have a cronjob that nags me if it sees any *.dpkg-new,
*.dpkg-old, *.dpkg-save, *.pam-old or *ucf-old files lying about.)

I noticed with the Buster upgrade that there are some packages
that want to do a "three way merge", which, when it fails,
uses some other long suffix for the files it leaves laying
around. But I forget the name.

Using locate I also notice a few *.dpkg-bak files.

Regards,

Karl <***@karlpinc.com>
Free Software: "You don't pay back, you pay forward."
-- Robert A. Heinlein

[Karl O. Pinc]

On Wed, 1 Jan 2020 22:56:13 -0600

Post by Karl O. Pinc
On Thu, 2 Jan 2020 01:45:16 +0000

Post by Justin B Rye
(Personally I have a cronjob that nags me if it sees any *.dpkg-new,
*.dpkg-old, *.dpkg-save, *.pam-old or *ucf-old files lying about.)

I noticed with the Buster upgrade that there are some packages
that want to do a "three way merge", which, when it fails,
uses some other long suffix for the files it leaves laying
around. But I forget the name.

Finally found them: *.ucf-dist *.merge-error *.ucf-old

Post by Karl O. Pinc
Using locate I also notice a few *.dpkg-bak files.

That makes the complete list: *.dpkg-new *.dpkg-old *.dpkg-save,
*.dpkg-bak, *.pam-old, *.ucf-old, *.ucf-dist, *.merge-error

Regards,

Karl <***@karlpinc.com>
Free Software: "You don't pay back, you pay forward."
-- Robert A. Heinlein

[Karl O. Pinc]

On Thu, 2 Jan 2020 01:45:16 +0000

Post by Justin B Rye

Post by Karl O. Pinc
Attached is a patch which suggests cleaning up unused config files
leftover from prior upgrades, the foo.dpkg-old and similar files,
before starting the new upgrade.
--- a/en/upgrading.dbk
+++ b/en/upgrading.dbk
@@ -301,6 +301,17 @@ $ apt-forktracer | sort
It is a good idea to <link linkend="obsolete">remove obsolete
packages</link> from your system before upgrading.
</para>
+ <para>
+ A previous upgrade can have left unused copies of
configuration

I'd recommend may have

Agreed.

Post by Justin B Rye

Post by Karl O. Pinc
+ files; <link linkend="configuration-changes">old
versions</link>
+ of configuration files, versions supplied by the package
+ maintainers, etc. Removing leftover files from previous upgrades,
+ before performing another upgrade, can avoid confusion.

The commas in this sentence seem subtly wrong.

I think the commas are right, as a parenthetical-ish without
parenthesis, but the sentence structure is strange. But
I want the forgettable stuff in the middle and the important
stuff on the ends of the sentence where the reader pays attention.

Post by Justin B Rye

Post by Karl O. Pinc
+ Find such
+ </para>
+ <screen>
+# find /etc -name '*.dpkg-*'
+ </screen>

But in fact if you're going to do this sweep (and haven't previously
been in the habit of doing it), doesn't it make more sense to learn it
as something to do *after* every (dist- or package-)upgrade? What
kinds of confusion does it risk if you do it then?
It might fit best in 4.7. "Preparing for the next release".

I like to leave the leftover files laying around for a while after
upgrade just in case they shed light on some problem that I don't
notice until later. So I clean them up (or not, usually) before
doing the next upgrade because I forget they exist.

It is also theoretically possible a dist-upgrade will be required
for some installed package in the middle of the release cycle.
That could create another *.dist-* file.

Regards,

Karl <***@karlpinc.com>
Free Software: "You don't pay back, you pay forward."
-- Robert A. Heinlein

[Justin B Rye]

Post by Karl O. Pinc

Post by Justin B Rye

Post by Karl O. Pinc
+ files; <link linkend="configuration-changes">old versions</link>
+ of configuration files, versions supplied by the package
+ maintainers, etc. Removing leftover files from previous upgrades,
+ before performing another upgrade, can avoid confusion.

The commas in this sentence seem subtly wrong.

I think the commas are right, as a parenthetical-ish without
parenthesis, but the sentence structure is strange. But
I want the forgettable stuff in the middle and the important
stuff on the ends of the sentence where the reader pays attention.

The trouble is, it isn't purely parenthetical; it's talking about
removing the leftover files under particular circumstances, so it's
rather like a defining relative clause (which doesn't take commas).
The competing misinterpretation is that you can avoid confusion by
removing these files and subsequently performing another upgrade
(that you otherwise wouldn't have performed). Given that you're
proposing this text as pre-upgrade advice, but that it's also true for
any other time, we might as well just say

Removing files left over from previous upgrades
can avoid confusion.

Post by Karl O. Pinc

Post by Justin B Rye
It might fit best in 4.7. "Preparing for the next release".

I like to leave the leftover files laying around for a while after
upgrade just in case they shed light on some problem that I don't
notice until later. So I clean them up (or not, usually) before
doing the next upgrade because I forget they exist.

It seems to me that users ought to do this within a few days of a
dist-upgrade while they remember the circumstances, not months or
years later. But it's quite possible that it only seems that way to
me because of the habits I've got into thanks to my nagging cronjobs
(and associated system configuration backups).

[...]

Post by Karl O. Pinc
That makes the complete list: *.dpkg-new *.dpkg-old *.dpkg-save,
*.dpkg-bak, *.pam-old, *.ucf-old, *.ucf-dist, *.merge-error

The ucf manpage also says that .ucf-new can occur as a symptom of an
interrupted run (like .dpkg-new); and in theory there's .dpkg-inst, so
maybe the search should use globs like "*.dpkg-*" and "*.ucf-*"
instead of trying to itemise them all.

[Karl O. Pinc]

On Thu, 2 Jan 2020 17:28:07 +0000

Post by Justin B Rye

Post by Karl O. Pinc

Post by Justin B Rye

Post by Karl O. Pinc
+ files; <link linkend="configuration-changes">old
versions</link>
+ of configuration files, versions supplied by the package
+ maintainers, etc. Removing leftover files from previous upgrades,
+ before performing another upgrade, can avoid confusion.

Given that you're
proposing this text as pre-upgrade advice, but that it's also true for
any other time, we might as well just say
Removing files left over from previous upgrades
can avoid confusion.

Agreed. Shorter is better.

Post by Justin B Rye

Post by Karl O. Pinc

Post by Justin B Rye
It might fit best in 4.7. "Preparing for the next release".

I like to leave the leftover files laying around for a while after
upgrade just in case they shed light on some problem that I don't
notice until later. So I clean them up (or not, usually) before
doing the next upgrade because I forget they exist.

It seems to me that users ought to do this within a few days of a
dist-upgrade while they remember the circumstances, not months or
years later. But it's quite possible that it only seems that way to
me because of the habits I've got into thanks to my nagging cronjobs
(and associated system configuration backups).

Users should certainly deal with making sure their configuration files
are working immediately after an upgrade. I'm sure I'm catering to
my retentive streak in keeping them around for some time. The
good part about forgetting to deal with them is that when you come
back years later to clean up you can be pretty confident that you
don't need them any more and can delete them without having to
think. :-) Especially because by that point in the upgrade
you should already have a backup.

---
On an unrelated note (FYI): The bulk of what's between
the title of section 4.2. Checking APT configuration status
and 4.2.1 involves cleanup around ensuring that the system
is "pure Debian stable", as does a lot of the whole section.
I think the section title could be improved but a) I don't
know where to discuss this, and b) I don't have a suggestion
ready. ("Remove complicating factors" is too mysterious, yes?)

A better title might inspire breaking up what's between 4.2 and
4.2.1 into separate sections. (I'd also move the paragraph
starting "Direct upgrades from Debian releases older than 10 (buster)
are not supported." to the top.)

Apologies for being offtopic.
---

Post by Justin B Rye

Post by Karl O. Pinc
That makes the complete list: *.dpkg-new *.dpkg-old *.dpkg-save,
*.dpkg-bak, *.pam-old, *.ucf-old, *.ucf-dist, *.merge-error

The ucf manpage also says that .ucf-new can occur as a symptom of an
interrupted run (like .dpkg-new); and in theory there's .dpkg-inst, so
maybe the search should use globs like "*.dpkg-*" and "*.ucf-*"
instead of trying to itemise them all.

Agreed. Especially because we're not putting a command in that
just deletes them. The user has to decide what to do.

Regards,

Karl <***@karlpinc.com>
Free Software: "You don't pay back, you pay forward."
-- Robert A. Heinlein

[Karl O. Pinc]

On Thu, 2 Jan 2020 12:41:14 -0600

Post by Karl O. Pinc
(I'd also move the paragraph
starting "Direct upgrades from Debian releases older than 10 (buster)
are not supported." to the top.)

Oops. I take that back.

Regards,

Karl <***@karlpinc.com>
Free Software: "You don't pay back, you pay forward."
-- Robert A. Heinlein

[Karl O. Pinc]

Hi,

Attached are 2 patches:

cleanup_v2.patch
This should incorporate all changes discussed so far.

section_v1.patch
This applies on top of the cleanup patch. It re-titles
the 4.2 section and adds sub-sections. If you want this
in a separate bug report, discussed elsewhere, etc.,
please let me know. (I had it in my brain and wanted
to get it out.)

Regards,

Karl <***@karlpinc.com>
Free Software: "You don't pay back, you pay forward."
-- Robert A. Heinlein

[Karl O. Pinc]

On Thu, 2 Jan 2020 13:27:19 -0600

Post by Karl O. Pinc
cleanup_v2.patch
This should incorporate all changes discussed so far.
section_v1.patch
This applies on top of the cleanup patch. It re-titles
the 4.2 section and adds sub-sections. If you want this
in a separate bug report, discussed elsewhere, etc.,
please let me know. (I had it in my brain and wanted
to get it out.)

Attached is section_v2.patch, to be applied on top of
cleanup_v2.patch. (Replaces section_v1.patch.)

Regards,

Karl <***@karlpinc.com>
Free Software: "You don't pay back, you pay forward."
-- Robert A. Heinlein

[Karl O. Pinc]

On Thu, 2 Jan 2020 16:56:10 -0600

On Thu, 2 Jan 2020 13:27:19 -0600

Post by Karl O. Pinc
cleanup_v2.patch
This should incorporate all changes discussed so far.
section_v1.patch
This applies on top of the cleanup patch. It re-titles
the 4.2 section and adds sub-sections. If you want this
in a separate bug report, discussed elsewhere, etc.,
please let me know. (I had it in my brain and wanted
to get it out.)

Attached is section_v2.patch, to be applied on top of
cleanup_v2.patch. (Replaces section_v1.patch.)

Attached is a cleanup_v3.patch.

Regards,

Karl <***@karlpinc.com>
Free Software: "You don't pay back, you pay forward."
-- Robert A. Heinlein

[Justin B Rye]

+ <section id="cleanup-revised-configuration-files">
+ <title>Cleanup revised configuration files</title>

"Cleanup" is the noun; you want the verb, "Clean up". And I'm not
sure you want "revised"; I would recommend "leftover", or you could
just omit it.

[Karl O. Pinc]

On Sun, 5 Jan 2020 16:34:04 +0000

Post by Justin B Rye

+ <section id="cleanup-revised-configuration-files">
+ <title>Cleanup revised configuration files</title>

"Cleanup" is the noun; you want the verb, "Clean up". And I'm not
sure you want "revised"; I would recommend "leftover", or you could
just omit it.

Done. (Used "leftover".)

Patch sections_v4.patch attached. This patch should
apply directly to HEAD and incorporates the old
cleanup.patch series.

In the FYI catagory, there are errors building
HEAD on master:

Writing ch-whats-new.en.html for chapter(ch-whats-new)
ERROR: xref linking to secure-boot has no generated link text.
Error: no ID for constraint linkend: "secure-boot".
Writing ch-installing.en.html for chapter(ch-installing)
ERROR: xref linking to before-first-reboot has no generated link text.
Error: no ID for constraint linkend: "before-first-reboot".
ERROR: xref linking to noteworthy-obsolete-packages has no generated link text.
Error: no ID for constraint linkend: "noteworthy-obsolete-packages".
Writing ch-upgrading.en.html for chapter(ch-upgrading)

Regards,

Karl <***@karlpinc.com>
Free Software: "You don't pay back, you pay forward."
-- Robert A. Heinlein