Files, Paths and Text I/O

1. Introduction to Data Persistence and File Systems

Welcome to Module 3 of our Python Mastery Series: Persistent Data, File Systems and Input/Output! Up to this point, all data structures, variables, lists, dictionaries, and state objects we constructed resided exclusively in volatile System RAM (Random Access Memory). The moment a script finishes execution or the terminal closes, volatile memory is wiped clean, and all computed results vanish permanently.

Production applications require Data Persistence—storing state permanently on non-volatile secondary storage (Solid State Drives or Hard Disk Drives). Whether writing application log files, saving customer records, storing configuration settings, or loading analytical reports, reading and writing files is a fundamental software engineering requirement.

In this lesson, we will master Python's built-in file handling mechanisms using `open()`, safe resource cleanup via Context Managers (`with` blocks), cross-platform path management using the modern `pathlib` module, character encodings (`utf-8`), and routing file-system operations cleanly with Structural Pattern Matching (`match-case`).

2. Modern Cross-Platform Path Manipulation with `pathlib`

Historically, developers manipulated file paths as raw text strings or used legacy os.path string-joining functions. However, hardcoding path strings creates severe cross-platform bugs because Windows uses backslashes (C:\Users\Main) while Linux and macOS use forward slashes (/home/user/main).

Python 3.4+ solved this permanently by introducing the standard `pathlib` module. The Path object provides an object-oriented, cross-platform interface that automatically uses the correct path separators for the host Operating System.

1. Core `pathlib.Path` Operations

• Current Working Directory
• Path.cwd()
• Returns absolute path of directory where Python process was launched.
• Path Join Operator
• base_dir / "logs" / "app.log"
• Uses slash operator (/) to concatenate path segments safely.
• Exists Check
• path_obj.exists()
• Returns True if target file or folder exists on disk.
• Is File / Is Directory?
• path_obj.is_file() / path_obj.is_dir()
• Evaluates whether target path is a file or a folder.
• Directory Creation
• path_obj.mkdir(parents=True, exist_ok=True)
• Creates directory tree safely without crashing if folder already exists.
• Path Metadata Attributes
• path.name, path.stem, path.suffix
• Extracts filename, base name without extension, or file extension (e.g., .txt).

from pathlib import Path

# Instantiating base working directory
working_dir = Path.cwd()
data_folder = working_dir / "system_storage"

# Safely creating folder hierarchy on disk
data_folder.mkdir(parents=True, exist_ok=True)

# Creating target file path object
log_file_path = data_folder / "audit_event.log"

print("=== Cross-Platform Path Inspection ===")
print("Absolute Folder Path :", data_folder)
print("Target File Path     :", log_file_path)
print("File Base Name (stem):", log_file_path.stem)
print("File Extension       :", log_file_path.suffix)
print("Does Folder Exist?   :", data_folder.exists())

=== Cross-Platform Path Inspection ===
Absolute Folder Path : /workspace/python_mastery_course/system_storage
Target File Path     : /workspace/python_mastery_course/system_storage/audit_event.log
File Base Name (stem): audit_event
File Extension       : .log
Does Folder Exist?   : True

3. The `open()` Function and File Access Modes

To interact with files on disk, Python provides the built-in open(file, mode='r', encoding='utf-8') function. It establishes a stream buffer connection between Python and the underlying Operating System file system.

1. Comprehensive File Opening Modes Matrix

• 'r'
• Read (Default)
• Pointer placed at **start of file** for reading.
• Raises FileNotFoundError
• 'w'
• Write (Overwrite)
• Truncates existing content to 0 bytes! Overwrites completely.
• Creates new empty file
• 'a'
• Append
• Pointer placed at **end of file**. Preserves prior content.
• Creates new empty file
• 'x'
• Exclusive Creation
• Opens stream strictly for creation and writing.
• Raises FileExistsError if file exists
• 'r+' / 'w+'
• Read + Write
• Opens bidirectional stream for simultaneous reading and writing.
• Depends on r or w rule

4. Safe Resource Management: Context Managers (`with` Blocks)

When you open a file stream, the Operating System allocates system resources (file descriptors). If a program opens hundreds of files without closing them, system memory leaks occur, file locks persist, and data written to buffers might not flush to physical disk storage.

1. Why Un-Managed `open()` is Dangerous

# DANGEROUS / UNPYTHONIC ANTI-PATTERN:
file_handle = open("data.txt", "w")
file_handle.write("Sample content")
# If an exception occurs right here, file_handle.close() NEVER RUNS!
file_handle.close()
    

2. The Pythonic Solution: The `with` Statement

A Context Manager (introduced by the with keyword) guarantees that Python automatically closes the file stream when execution exits the indented block—even if runtime exceptions or unexpected errors occur inside the block!

from pathlib import Path

target_path = Path.cwd() / "sample_output.txt"

# 1. Writing Text using Context Manager ('w' mode)
with open(target_path, mode="w", encoding="utf-8") as file:
    file.write("Header: System Configuration File\n")
    file.write("Version: 1.0.0\n")
    file.write("Status: Operational\n")

# File is automatically closed here!

# 2. Appending Text using Context Manager ('a' mode)
with open(target_path, mode="a", encoding="utf-8") as file:
    file.write("Log Entry: Maintenance completed successfully.\n")

print(f"File successfully written to: {target_path.name}")
print(f"File Stream Closed? : {file.closed}")  # Proves automatic cleanup!

File successfully written to: sample_output.txt
File Stream Closed? : True

5. Reading Strategies: `.read()`, `.readline()`, and Iteration

Python provides multiple methods to consume text content from an open file stream, each tailored to different file sizes:

1. Reading Full Content (`.read()`)

Reads the entire file into a single string. Useful for small text files, but dangerous for huge multi-gigabyte log files as it loads everything into RAM at once.

2. Line-by-Line List Extraction (`.readlines()`)

Reads all lines and returns a list of string lines (retaining trailing \n newline characters).

3. High-Performance Stream Iteration (`for line in file:`)

The standard Pythonic way to read large files! It streams one line at a time into memory lazily with minimal RAM consumption.

from pathlib import Path

target_path = Path.cwd() / "sample_output.txt"

print("--- Strategy 1: Iterating Line-by-Line (Memory Efficient) ---")
with open(target_path, mode="r", encoding="utf-8") as file:
    for line_no, line_content in enumerate(file, start=1):
        # Stripping trailing newline for clean console printing
        clean_line = line_content.rstrip("\n")
        print(f"Line #{line_no:02d} | {clean_line}")

print("\n--- Strategy 2: Extracting Lines into a List ---")
with open(target_path, mode="r", encoding="utf-8") as file:
    lines_list = [line.strip() for line in file if "Log Entry" in line]

print("Filtered Matching Lines:", lines_list)

--- Strategy 1: Iterating Line-by-Line (Memory Efficient) ---
Line #01 | Header: System Configuration File
Line #02 | Version: 1.0.0
Line #03 | Status: Operational
Line #04 | Log Entry: Maintenance completed successfully.

--- Strategy 2: Extracting Lines into a List ---
Filtered Matching Lines: ['Log Entry: Maintenance completed successfully.']

6. Robust File I/O Exception Handling

File system operations are prone to runtime environment errors (e.g., missing files, insufficient disk permissions, full disks, or corrupted encodings). Wrap file operations in try-except blocks to handle errors gracefully.

from pathlib import Path

non_existent_file = Path.cwd() / "missing_config.json"

try:
    with open(non_existent_file, mode="r", encoding="utf-8") as f:
        content = f.read()
except FileNotFoundError as err:
    print(f"IO_ERROR: Target file does not exist! details: {err.filename}")
except PermissionError:
    print("SECURITY_ERROR: Access denied due to file permissions!")
except Exception as gen_err:
    print(f"UNKNOWN_ERROR: Unexpected error occurred -> {gen_err}")

IO_ERROR: Target file does not exist! details: missing_config.json

7. Integrating File Operations with `match-case` Pattern Matching

Combining pathlib.Path operations and context managers with match-case structural pattern matching creates clean, robust CLI file-management utilities.

from pathlib import Path

def execute_file_command(command_payload: list):
    """
    Executes file system actions using Structural Pattern Matching.
    Demonstrates handling file paths and text I/O dynamically.
    """
    match command_payload:
        case ["WRITE", filename, *lines] if len(lines) > 0:
            file_path = Path.cwd() / filename
            with open(file_path, mode="w", encoding="utf-8") as f:
                for line in lines:
                    f.write(f"{line}\n")
            return f"SUCCESS: Wrote {len(lines)} lines to '{filename}'."
            
        case ["READ", filename]:
            file_path = Path.cwd() / filename
            if not file_path.exists():
                return f"ERROR: File '{filename}' not found!"
            with open(file_path, mode="r", encoding="utf-8") as f:
                content = f.read().strip()
            return f"=== CONTENTS OF '{filename}' ===\n{content}"
            
        case ["APPEND", filename, new_content]:
            file_path = Path.cwd() / filename
            with open(file_path, mode="a", encoding="utf-8") as f:
                f.write(f"{new_content}\n")
            return f"SUCCESS: Appended entry to '{filename}'."
            
        case ["DELETE", filename]:
            file_path = Path.cwd() / filename
            if file_path.exists():
                file_path.unlink()  # Deletes file from disk
                return f"SUCCESS: Deleted file '{filename}' from disk."
            return f"WARNING: Cannot delete missing file '{filename}'."
            
        case _:
            return "ERROR: Unrecognized file system command."

# Testing the file dispatcher
print(execute_file_command(["WRITE", "test_log.txt", "Line A: System Init", "Line B: Server Active"]))
print(execute_file_command(["APPEND", "test_log.txt", "Line C: Shutdown Request"]))
print(execute_file_command(["READ", "test_log.txt"]))
print(execute_file_command(["DELETE", "test_log.txt"]))

SUCCESS: Wrote 2 lines to 'test_log.txt'.
SUCCESS: Appended entry to 'test_log.txt'.
=== CONTENTS OF 'test_log.txt' ===
Line A: System Init
Line B: Server Active
Line C: Shutdown Request
SUCCESS: Deleted file 'test_log.txt' from disk.

8. Frequently Asked Interview Questions with Answers

9. Homework & Practical Assignments

10. File & Workspace Directory Structure

python_mastery_course/
│
├── lesson_01/ ... lesson_26/
│
└── lesson_27/
    ├── pathlib_basics_demo.py
    ├── context_manager_demo.py
    ├── reading_strategies_demo.py
    ├── file_io_exceptions.py
    ├── match_case_file_dispatcher.py
    ├── log_stream_task.py          <-- (Task 1)
    ├── file_command_task.py        <-- (Task 2)
    └── audit_logger_os.py          <-- (Task 3: Master Review Capstone)
        

11. What We Will Learn Next