Chapter 5

Paths with pathlib

Every filename so far has been a bare string, and Python looked for it "wherever the script is running from." That's fine for a demo and a liability in real code. This chapter is pathlib, the standard-library way to build and inspect the things that point at files. It replaces string joins, the older os.path functions, and a fair amount of guesswork.

From here on, every path in the book is a Path.

Path

from pathlib import Path

notes = Path("notes")
piranesi = notes / "piranesi.txt"

print(piranesi.as_posix())

notes/piranesi.txt

Path("notes") makes a path object. It doesn't touch the disk; it's just a value that represents a location. The / operator joins paths, and it uses the right separator for the platform you're on. On Windows that last line would be notes\piranesi.txt; .as_posix() forces forward slashes, which is what this book prints so the output reads the same for everyone.

Turning each book title from the log into the path where its note would live looks like this:

from pathlib import Path

def note_path(title):
    stem = title.lower().replace(" ", "-")
    return Path("notes") / f"{stem}.txt"

print(note_path("Piranesi").as_posix())
print(note_path("One Hundred Years of Solitude").as_posix())

notes/piranesi.txt
notes/one-hundred-years-of-solitude.txt

That stem rule is crude. A title with a comma or an apostrophe in it would need more care. The point is that you build the path by joining pieces, never by gluing strings with + and hoping the slashes land right.

Inspecting a path

A Path can take itself apart:

p = Path("notes/one-hundred-years-of-solitude.txt")

print("name:  ", p.name)
print("stem:  ", p.stem)
print("suffix:", p.suffix)
print("parent:", p.parent.as_posix())
print("parts: ", p.parts)

name:   one-hundred-years-of-solitude.txt
stem:   one-hundred-years-of-solitude
suffix: .txt
parent: notes
parts:  ('notes', 'one-hundred-years-of-solitude.txt')

.name is the last component, .stem is the name without its extension, .suffix is the extension, and .parent is everything above it. These are just string manipulation done correctly.

It can also ask the disk about itself:

p = Path("notes/piranesi.txt")

print("exists: ", p.exists())
print("is_file:", p.is_file())
print("is_dir: ", p.is_dir())
print("missing:", (Path("notes") / "dune.txt").exists())

exists:  True
is_file: True
is_dir:  False
missing: False

read_text and write_text

For a small file you want in one gulp, Path has shortcuts that open, read or write, and close for you:

note = Path("notes/piranesi.txt").read_text(encoding="utf-8")
print(note.splitlines()[0])

Path("scratch.txt").write_text("a quick note\n", encoding="utf-8")
print(Path("scratch.txt").read_text(encoding="utf-8"), end="")

Piranesi
a quick note

There's a .read_bytes() and .write_bytes() pair for binary. Pass encoding to the text versions the same as you would to open, for the reasons in Chapter 4.

Use these for one-shot reads and writes of small files. The moment you want to loop over lines, or the file might be large, go back to with open(...) from Chapter 2.

Relative and absolute

Path("notes/piranesi.txt") is a relative path. It means "starting from wherever the program is running," and that folder is the working directory:

from pathlib import Path

p = Path("notes/piranesi.txt")
print("relative?", not p.is_absolute())

full = p.resolve()
print("absolute?", full.is_absolute())
print("last two parts:", full.parts[-2:])

relative? True
absolute? True
last two parts: ('notes', 'piranesi.txt')

.resolve() turns a relative path into an absolute path, one that starts from the root of the filesystem and means the same thing no matter where you run from.

This matters because the working directory is not always what you expect. Run python report.py from your project folder and Path("notes") points at your project's notes. Run the same script from your home folder and it points at a notes folder there, which probably doesn't exist. The script "works on my machine" and fails in a scheduled job, and the reason is the working directory.

For a script that needs its own data files, build the paths from the script's location instead of the working directory:

HERE = Path(__file__).parent
notes = HERE / "notes"

__file__ is the path to the running script, so HERE is the folder it lives in, whatever directory you launched it from.

Common mistakes

Gluing paths with +. "notes" + "/" + name gives you a wrong separator on Windows and a double slash if name already starts with one. Path("notes") / name handles both.

A leading slash on the right of /. Path("notes") / "/piranesi.txt" returns /piranesi.txt, not notes/piranesi.txt. An absolute path on the right throws away everything on the left. Keep the right-hand side a bare name.

Trusting .exists(). It answers about this instant. The file can be deleted between your check and your open. If you're about to open the file anyway, skip the check and catch FileNotFoundError (Chapter 1).

Practice

Try each of these before you read the solution under it.

1. Write note_path(title) that returns the Path to a title's note, and has_note(title) that returns whether that file exists.
2. Loop over reading-log.csv and, for each book, print its title and whether a note file exists for it.
3. Write backup_path(path) that takes a Path to a file and returns a sibling path with the same stem and a .bak suffix.

Solutions

1. has_note builds on note_path rather than repeating the logic.

from pathlib import Path

def note_path(title):
    stem = title.lower().replace(" ", "-")
    return Path("notes") / f"{stem}.txt"

def has_note(title):
    return note_path(title).exists()

print(note_path("Piranesi").as_posix())
print(has_note("Piranesi"))
print(has_note("Dune"))

notes/piranesi.txt
True
False

2. Read the log line by line, take the title, check for its note.

from pathlib import Path

def note_path(title):
    stem = title.lower().replace(" ", "-")
    return Path("notes") / f"{stem}.txt"

with open("reading-log.csv", encoding="utf-8") as f:
    next(f)  # header
    for line in f:
        title = line.split(",")[0]
        mark = "note" if note_path(title).exists() else "--"
        print(f"{mark:5} {title}")

--    The Left Hand of Darkness
note  Project Hail Mary
--    "Rendezvous with Rama
note  One Hundred Years of Solitude
--    The Peripheral
note  Klara and the Sun
note  Piranesi
--    Recursion
--    A Memory Called Empire
--    The Three-Body Problem

The quoted title is still split wrong, and finding notes properly is the next chapter's job. Four of ten books have a note.

3. Path.with_suffix swaps the extension.

from pathlib import Path

def backup_path(path):
    return path.with_suffix(".bak")

print(backup_path(Path("notes/piranesi.txt")).as_posix())
print(backup_path(Path("settings.json")).as_posix())

notes/piranesi.bak
settings.bak

Where this leaves you

You build paths by joining Path pieces, you can pull a path apart into its name, stem, suffix, and parent, you can ask whether it exists, and you know the difference between a relative path and an absolute one and why the working directory can bite you. Chapter 6 uses all of this to answer the most-asked file question there is: what's in this folder, and how do I find the file I want?