Skip to content

Local range download leaves an output file after source-read failure despite delete_on_failure=True #2223

Open

Description

@JunaidAhamed-7777

On trunk 8a0ef626, the local driver's range-download method creates a destination file but does not clean it up when opening the source fails, even with delete_on_failure=True.

This reproducer is sequential: obtain the object metadata, explicitly remove the stored file, then try to download. It does not depend on winning a filesystem race.

import os
import tempfile
from libcloud.storage.drivers.local import LocalStorageDriver

with tempfile.TemporaryDirectory() as root:
    driver = LocalStorageDriver(root)
    container = driver.create_container("bucket")
    obj = driver.upload_object_via_stream(iter([b"payload"]), container, "object")
    os.unlink(driver.get_object_cdn_url(obj))
    destination = os.path.join(root, "download")
    try:
        driver.download_object_range(
            obj, destination, start_bytes=0, delete_on_failure=True
        )
    except FileNotFoundError:
        pass
    print(os.path.exists(destination), os.path.getsize(destination))
    # True 0

The public range-download contract describes delete_on_failure as deleting a partially downloaded file when the download was unsuccessful, with hash mismatch/file size given as examples. I expected the newly created empty output to be removed for a source-read failure as well. Otherwise a failed operation leaves a file that can be mistaken for a valid empty download.

The local implementation accepts the flag but never reads it. It opens the destination before consuming the source iterator and has no failure cleanup around that loop.

Checks with unchanged source, Python 3.12/Linux:

  • Range download with delete_on_failure=True: raises FileNotFoundError, leaves a zero-byte output.
  • Range download with delete_on_failure=False: same result.
  • Full download_object with the same missing source: returns False, leaves no output.
  • Successful range [0, 3) before removing the source: returns True, writes b"pay".

No code change yet. This report is limited to failure cleanup for local range downloads; it does not claim a concurrent-write reproduction or a full test-suite pass.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions