wrong FNAME in tarfile if tgz extension is used

[maciejmmmisiak]

BPO	44495
Nosy	@veaviticus

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-06-23.06:55:14.378>
labels = ['type-bug', 'library', '3.9', '3.10', '3.11']
title = 'wrong FNAME in tarfile if tgz extension is used'
updated_at = <Date 2022-01-16.17:58:03.979>
user = 'https://bugs.python.org/maciejmmmisiak'

bugs.python.org fields:

activity = <Date 2022-01-16.17:58:03.979>
actor = 'iritkatriel'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2021-06-23.06:55:14.378>
creator = 'maciej.mm.misiak'
dependencies = []
files = []
hgrepos = []
issue_num = 44495
keywords = []
message_count = 2.0
messages = ['396382', '405468']
nosy_count = 2.0
nosy_names = ['veaviticus', 'maciej.mm.misiak']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue44495'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

Linked PRs

• gh-88661: Derive the gzip FNAME field like gunzip does #158125

[maciejmmmisiak]

This code is incomplete:

def _init_write_gz(self):
...
    if self.name.endswith(".gz"):
        self.name = self.name[:-3]
    # RFC1952 says we must use ISO-8859-1 for the FNAME field.
    self.__write(self.name.encode("iso-8859-1", "replace") + NUL)

If it is used in following way '.gz' is stripped properly and FNAME='somefile.tar':
tarfile.open('somefile.tar.gz', 'w:gz')
but with
tarfile.open('somefile.tgz', 'w:gz')

FNAME is incorrectly prepared as somefile.tgz

[veaviticus]

The code referenced in the previous comment only hits for tarfiles built from Streams.

The same (incorrect) code exists in the gzip.py library as well, and hits the more common usecase of building a tar.gz from a set of files on disk.

def _write_gzip_header(self, compresslevel):
    self.fileobj.write(b'\037\213')             # magic header
    self.fileobj.write(b'\010')                 # compression method
    try:
        # RFC 1952 requires the FNAME field to be Latin-1. Do not
        # include filenames that cannot be represented that way.
        fname = os.path.basename(self.name)
        if not isinstance(fname, bytes):
            fname = fname.encode('latin-1')
        if fname.endswith(b'.gz'):
            fname = fname[:-3]

This effects decompressing the file with 7zip, who respects the FNAME value, and thus attempts to create a new file with the same name as the file its currently decompressing. Or if you extract to another directory, it creates a tar file that is named "foo.tgz", which is confusing to users who are expecting a tar.

You can very easily reproduce this:

import tarfile
f = tarfile.open("test.tgz", mode="w:gz")
f.close()

and then "extract" the file with 7zip

[abadger]

I wonder if _write_gzip_header() (and associated functions) need to be restructured so that it takes fname as a parameter. It doesn't seem like _write_gzip_header() as currently written has enough information to decide that the file is definitlely a gzipped tarfile and therefore should turn into BASENAME.tar when it is uncompressed.