~ephes

Building things with Django and coding agents. Breaking things in my home lab. Writing about both.

How to Add Extra Data to a Python Package

Today, I learned how to include external files in a Python package using uv and the hatchling build backend. My goal was to add a directory containing sample fixture data to the package. Here’s the resulting pyproject.toml file:

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.build.targets.sdist]
packages = ["src/mypackage"]

[tool.hatch.build.targets.sdist.force-include]
"data/fixtures" = "mypackage/fixtures"  # Include the fixtures directory in the sdist

[tool.hatch.build.targets.wheel]
packages = ["src/mypackage"]

[tool.hatch.build.targets.wheel.force-include]
"data/fixtures" = "mypackage/fixtures"  # Include the fixtures directory in the wheel

This turned out to be surprisingly tricky to get right, and the toml file doesn’t make it immediately clear which paths are the source and which are the target in the sdist and wheel. So, here’s an overview of the directory structure:

project_root/
├── src/
│   └── mypackage/
│       ├── __init__.py
│       ├── module1.py
│       └── fixtures/           # Included as mypackage/fixtures by pyproject.toml
│           ├── fixture1.json
│           └── fixture2.json
├── data/
│   └── fixtures/
│       ├── fixture1.json
│       └── fixture2.json
├── pyproject.toml
└── README.md
Read more →

Weeknotes 2024-12-02

When I die please mix my ashes with concrete, then turn me into a brick and throw me at an Amazon Warehouse --Erik Uden

Yep, it’s been a stressful week, just as expected. Somehow, we managed to record a podcast episode, but now I need to find the time to get it published.

Fediverse

Read more →

Weeknotes 2024-11-25

Don't use a big word when a singularly unloquacious and diminutive linguistic expression will satisfactorily accomplish the contemporary necessity. --Dgar

This week brought some improvements and a new release for podcast-transcripts. Overall, though, it was more work and less play than usual—a pattern I suspect will continue through the end of the year.

Out of Context Images

Read more →

🎙️ Introducing podcast-transcript: Audio Transcription Made Simple

Hey folks! I recently built a little command-line tool called podcast-transcript that turns audio into text. While it started as a podcast transcription project during the PyDDF autumn sprint, it works great for any speech audio. The coolest part? It can transcribe a 2-hour podcast in about 90 seconds!

Quick Start 🚀

pip install podcast-transcript  # or use pipx or uvx
transcribe https://d2mmy4gxasde9x.cloudfront.net/cast_audio/pp_53.mp3

Why Groq?

After trying different approaches, I landed on using the Groq API for transcription. Here's why:

  • It's blazing fast
  • Getting an API key is free and API usage is free (with reasonable limits: 8 hours of audio per day, 2 hours per hour)
  • The Whisper large-v3 model handles multiple languages well (especially noticeable for German content)

Technical Bits

The tool handles some interesting challenges under the hood:

  • Automatically resamples audio to 16kHz mono before upload (if you don't do it before, Groq will after upload)
  • Splits files larger than 25MB into chunks and stitches the transcripts back together
  • Uses httpx for direct API calls to get detailed JSON responses inspired by Simon Willison’s approach
  • Outputs in multiple formats: DOTe JSON, Podlove JSON, WebVTT, and plain text

Future Plans

I'm planning to add support for local transcription using the OpenAI Whisper model. While Whisper v2 works well enough for English content, v3 shows notable improvements for other languages (especially German). I initially skipped local processing because of the PyTorch dependency, but it's on the roadmap! I also plan to add multitrack support for handling audio files with separate speaker tracks.

The code is open source and contributions are welcome. Let me know if you try it out!

Read more →

Weeknotes 2024-11-18

I learned all my HTML<br>
From a code camp held yearly in hell<br>
We started on `<s>`<br>
Then striked through the rest<br>
Now I have a career in DevRel --Heydon Pickering


Published a new podcast episode discussing Python 3.13. I’ve been using Jupyter notebooks to generate transcripts for the episodes and have started consolidating that workflow into a dedicated Python command-line application.

Out of Context Images

Read more →