merge refactors pr #364 from @michilyy

* Save to DB only unique trackhashes

* Add check if track already exists in playlist

* replace all paths with `pathlib.Path`

* `architecture.md`:
* add config folder layout

`config.py`:
* fix bug where `pathlib.Path` cannot be serialized

`files.py`:
* remove unused imports
* update path concatenation to `pathlib.Path`
* add config-folder creation

`imgserver.py`:
* fix serialisation bug

`playlistlib.py`:
* update path concatenation to `pathlib.Path`

* update all `settings.Paths` usages to new singleton `Paths` class.

* update all usages of `settings.Paths`

* `files.py`:
* rework assets copy function.
* remove unused loop and unused `shutil.copy2` function

`settings.py`
* fix recursion exception in `Paths`

* `settings.py`:
* remove Singleton and `@property` todos from `Paths`

* `__init__.py`:
* remove now unused function `create_config_dir()`

`setup.files`:
* remove because merged into `settings.Paths()`
  for more central and clear flow how the base path gets decided

`settings.py`:
* add `copy_assets` function

`start_swingmusic.py`:
* add configurable settings.Paths class

`__main__.py`:
* update click to used correct default path

* remove wrong commited egg files

* remove change in the wrong branch

* add forgotten `property` decorator
update `get_files_and_dirs` to use pathlib where possible

`config.py`:
* update type annotation

`folders.py`:
* convert `pathlib` to posix path where needed for sub-functions

`folderlib.py`:
* rework `get_files_and_dirs` to use `pathlib` where possible

`settings.py`:
* add forgotten `@property`

`start_swingmusic.py`:
* remove second `log_startup_info()`

* `artistlib.py`:
* fix calling property
`tagger.py`:
* fix comparing elements in `pathlib.Path`

* add support for repeating lyrics.

* rework lyrics api and lib

* update most path functions.
add type-hint pathlib where needed

* for serialization paths are converted to posix path

* use `open` instead of `os.open`
update `metaclass` with constant

* fix initial config exception if empty file existed

* update `userConfig` with `InitVar` to be excluded from `asdict`

* remove `is_windows_slash()`
rework path function to use pathlib

* convert `pathlib.Path` to `str` for serialization

* fixing bug with str + pathlib

* `__main__.py`:
* update click to use package version
* remove now unused function `print_version`

`filesystem.py`:
* rework `CWD` to use importlib

`pyproject.toml`:
* disable namespace for `importlib.resources` to work correctly

* update `lyrics.py`:
* remove unused functions
* simplify functions

* fix bug where assets get created on root

* remove unused code

* update lyrics for clearer structure.

* add support for unsynced lyrics

* fix wrong return type in unsynced lyrics

* update `/check` to use `send_lyrics`

* prefer tags to duplicates

* `lyrics.py`:
* add docs to a function group

* `logger.py`:
* add logging config dict.
* combine Logging into one file
* add socket logger
* add debug mode to logger
* add JSONL formater

* `logger.py`:
* update config to directly use the formater.
resolves circular import exception

`__main__.py`:
* add logger setup to main

`start_swingmusic.py`:
* add debug option to cli

* `lyrics.py`:
* add offset support

* add `setuptools-scm` to get version from git

* add support for docker build with scm

* add support for docker build with scm
need someone who can test the changes workflow

* update all usage of `version.txt` to `metadata.version()`

* 2x update all usage of `version.txt` to `metadata.version()`

* update to no local_scheme version

* provide fix for #331.
convert `sql.Row` and `TrackTable` to dict before converting to dataclass.

* fix `__main__.py`:
* wrong import and uncommited changes
* add debug and base_path parameter

* fix logger pathlib

* add client build workflow

* set name

* split client from build

* try fixing builds

* try another fix

* try also another fix

* try again something new

* try again something new

* change runner

* fix failed run because of malformed runner

* add wheel builds

* remove systems from pure python build

* add isolated pyinstaller build

* artifacts with names

* wrong wheel path

* try fetch-depth for tag fetch

* disable fail-fast.
add wheel installation

* add install system packages

* add debug

* fix wheel install
fix pyinstaller spec file

* try fix for pyinstaller

* try another fix

* build on release

* add concrete release types

* only run on released or pre-released

* try release upload

* reformat upload

* fix needs tag

* identifiable pyinstaller builds

* compress client folder before uploading

* update to src build

* remove no more needed aarch64 build script
rename pyinstaller assets to lowercase

* remove unneeded code

* fix: save to DB only unique track hashes

* replace click with argparse

* set concrete types in argparse

* replace manuall path usages with pathlib

* remove unused `configs.py` file

* reformat `start_swingmusic.py`

* fix empty set startup exception

* optimizing static files serve function

* fixing bug in optimisation of static files serve function

* fix folder view bug

* colorlib.py:
* fix wrong type exception
* remove singe use Index_everything class
* update logging of populate.py

* cleanup files

* fix settings.py Paths copy function.
Created folder on file.

* add exist check to folder

* remove unused `INFO` class

* fix multiprocessing bug on windows

* potential icon fix for pyinstaller
fix multiple logging bug

* fix argparse config path bug
add jobs file

* cleanup code fragments
fix logging issue
add notes to function

* note that concurrent creates own sys.modules

* refactor some lyrics plugin condition
remove unused import from hashing

* refactor taglib.py

* update import statements to be static

* playlistlib.py:
* refactoring and more doc strings
populate.py:
* add poc bugfix
settings.py:
* add typehint

* possible bugfix for multitreading globals

* folder.py:
* add check if provided path is absolute
populate.py:
* add bug note
settings.py:
* add possible error from Singleton implementation
start_swingmusic.py:
* correct spelling
* pass resolved path to Paths
tagger.py:
* add logging

* trying out fixes for multithreading

* only upload results not metadata

* fix build action again

* folder.py:
* strictly use pathlib where possible

folderlib.py:
* add missing docstring to function, who really need it.

track.py:
* refactor some code

folder.py:
* refactor some more code

* Merge DBPath class and Paths class.
Update all usages of DBPath

folderslib.py:
* fix bug with logging

taglib.py:
* add missing docstring

settings.py:
* merge classes
* refactor

* network.py:
* add more docstring

config.py:
* update pathlib usage

tools.py:
* refactor
* add docstrings

* colorlib.py:
* add docstring

Refactor App builder into grouped config settings.

* update assets access for migration

* Update FUNDING.yml

* Update FUNDING.yml

* upgrade tinytag in requirements.txt

* update readme

* update license

* update readme

* Update README.md

* Update README.md

* cleanup requirements.txt
remove unused import in audio_segment.py
add entrypoint.sh for appimage support
update pyproject.toml for optional dependencies
add appimage to github workflow

* fix invalid workflow file

* AppImage build needs more research.
Commenting for now

* testing a new build workflow

* add libev installation

* update workflow to new optional dependencies

* trying again another fix

* finally fix all optional deps installation correctly

* remove AppImage poc

* albumslib.py:
* add docstring

folder.py:
* add unix path fix

update logger name to `__name__`

* update build with docker
update Dockerfile with git
fix typo in lyrics.py
add dynamic deps back

* add log for static folder

* add missing import

* add some more todos

* add support for AppImages even when it's not perfect.

* quick bugfix for wrong appimage config path

* fix uploading not finding AppImages builds aka wrong pattern

* optimise docker build by using artifacts.
Add client path option.
change docstring to sphinx format

* add todos

* Now support AppImages for real:
manually build AppImage as we are building a complex project.

* fix missing dep in AppImage build

* add full AppImage metadata

* add missing image file.

* only update swingmusic appimage not tool

* add todo and fix AppImage build again.

* Try fixing some path mixup in AppImage build

* add debug tag to action

* correct path to appimage folder

* do not download tool before checkout

* Another fix for path in appimage build

* extend config files with more information

* default client dir is now inside the config dir.
TODOs updated.

* default client dir is now inside the config dir.
TODOs updated.
Add priority todos.

* Auto download client when client not found.
Respects user provided dir.

* rename `requests` submodule to `request`

* poc for arm AppImage builds

* try out another fix

* fix typo in build.yml

* add missing arch tag

* fix uploading double names

* unique naming

* enable fallback version for project.

* do not download client into readonly dir.

* fix relative client download path. Client was resolved into parent of config.

* remove client backup path as client is now downloadable

* `Paths` checks if config folder exists and creates it if necessary.
logger no more creates the config folder.
`app_builder.py`: static route no more with '/client'

* path are only created in MainProcess.
fix gz file not found.

* move assets into src and update usages accordingly

* remove solved todos

* Only upload artefacts if not draft/master aka only on tag

* wrong type in assets copy

* update log with correct priority

* add debug statements and logging to Paths

* remove debugging statement

* remove double version tag from docker build

* fork save release protection

* fix typo

* add fallback client dir for static builds.

* update argparse to new param

* add missing import pathlib

* add sparse checkout as we do not need everything downloaded

* add assets copy check

* init logger bevor Paths

* remove unused import

* check if logdir exists and create if not

* only add exec info to file

* remove exception log from cli

* move logging into main.
Allows tools support again.

* UserConfig now correctly uses _finished key.
Bug where _finished was never written

* double save serverId.
update root_dir to trow no exception on init.
remove debug param

* clean up TODOs

---------

Co-authored-by: skilletfun <skilletfun.laptew.sergey@yandex.ru>
Co-authored-by: Mungai Njoroge <geoffreymungai45@gmail.com>
This commit is contained in:
michily
2025-08-28 09:28:11 +00:00
committed by GitHub
parent b4b0a6e11f
commit e770606567
197 changed files with 2961 additions and 2150 deletions
+3
View File
@@ -0,0 +1,3 @@
"""
This module contains all the data processing and non-API libraries
"""
+30
View File
@@ -0,0 +1,30 @@
"""
Contains methods relating to albums.
"""
from swingmusic.models.track import Track
def remove_duplicate_on_merge_versions(tracks: list[Track]):
"""
Removes duplicate tracks when merging versions of the same album.
"""
# TODO!
pass
def sort_by_track_no(tracks: list[Track]) -> list[Track]:
"""
Sort tracks by track number.
Track numbers cannot be longer than three positions.
:param tracks: List of Tracks
:return: Sorted list of Tracks
"""
for t in tracks:
track = str(t.track).zfill(3)
t._pos = int(f"{t.disc}{track}")
tracks = sorted(tracks, key=lambda t: t._pos)
return tracks
+187
View File
@@ -0,0 +1,187 @@
import os
import time
import random
import urllib
import requests
from io import BytesIO
from pathlib import Path
from concurrent.futures import ProcessPoolExecutor
from PIL import Image, PngImagePlugin, UnidentifiedImageError
from requests.exceptions import ConnectionError as RequestConnectionError
from requests.exceptions import ReadTimeout
from swingmusic import settings
from swingmusic.models.artist import Artist
from swingmusic.store.artists import ArtistStore
# from swingmusic.db.libdata import ArtistTable
# from swingmusic.store import artists as artist_store
# from swingmusic.store.tracks import TrackStore
from swingmusic.utils.hashing import create_hash
from swingmusic.utils.progressbar import tqdm
LARGE_ENOUGH_NUMBER = 100
PngImagePlugin.MAX_TEXT_CHUNK = LARGE_ENOUGH_NUMBER * (1024**2)
# https://stackoverflow.com/a/61466412
def get_artist_image_link(artist: str):
"""
Returns an artist image url.
"""
response: requests.Response | None = None
def make_request():
query = urllib.parse.quote(artist) # type: ignore
url = f"https://api.deezer.com/search/artist?q={query}"
user_agents = [
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0 Safari/605.1.15",
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0",
"Mozilla/5.0 (iPhone; CPU iPhone OS 14_6 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0 Mobile/15E148 Safari/604.1",
]
headers = {
"User-Agent": random.choice(user_agents),
"Accept": "application/json, text/plain, */*",
"Accept-Language": "en-US,en;q=0.9",
"Referer": "https://www.deezer.com/",
"Origin": "https://www.deezer.com",
}
return requests.get(url, headers=headers, timeout=30)
for attempt in range(5):
try:
response = make_request()
try:
data = response.json()
except requests.exceptions.JSONDecodeError:
return None
for res in data["data"]:
res_hash = create_hash(res["name"], decode=True)
artist_hash = create_hash(artist, decode=True)
if res_hash == artist_hash:
return str(res["picture_big"])
return None
except (RequestConnectionError, ReadTimeout, IndexError, KeyError):
if attempt == 4:
print("Failed to get artist image link ")
if attempt <= 4:
time.sleep(10)
else:
return None
# except (IndexError, KeyError):
# print(f"Encountered index/key error in attempt {attempt}")
# if response is not None:
# print(response.headers)
# return None
# TODO: Move network calls to utils/network.py
class DownloadImage:
def __init__(self, url: str, name: str) -> None:
img = self.download(url)
if img is None:
return
sm_path = settings.Paths().sm_artist_img_path / name
lg_path = settings.Paths().lg_artist_img_path / name
md_path = settings.Paths().md_artist_img_path / name
entries = [
(lg_path, None), # save in the original size
(sm_path, settings.Defaults.SM_ARTIST_IMG_SIZE),
(md_path, settings.Defaults.MD_ARTIST_IMG_SIZE),
]
self.save_img(img, entries)
@staticmethod
def download(url: str) -> Image.Image | None:
"""
Downloads the image from the url.
Retries after 10 seconds on a connection error.
"""
for attempt in range(2):
try:
response = requests.get(url, timeout=10)
return Image.open(BytesIO(response.content))
except (RequestConnectionError, requests.Timeout, ReadTimeout):
if attempt == 0:
time.sleep(10)
else:
return None
except UnidentifiedImageError:
return None
@staticmethod
def save_img(img: Image.Image, entries: list[tuple[Path, int | None]]):
"""
Saves the image to the destinations.
"""
ratio = img.width / img.height
for entry in entries:
path, size = entry
if size is None:
img.save(path, format="webp")
continue
img.resize((size, int(size / ratio)), Image.Resampling.LANCZOS).save(
path, format="webp"
)
class CheckArtistImages:
def __init__(self):
# read all files in the artist image folder
storeArtists = ArtistStore.get_flat_list()
path = settings.Paths().sm_artist_img_path
processed = set(i.replace(".webp", "") for i in os.listdir(path))
unprocessed = (
artist for artist in storeArtists if artist.artisthash not in processed
)
num_workers = max(1, (os.cpu_count() or 1) // 2)
with ProcessPoolExecutor(max_workers=num_workers) as executor:
res = list(
tqdm(
executor.map(self.download_image, unprocessed),
total=len(storeArtists) - len(processed),
desc="Downloading missing artist images",
)
)
list(res)
@staticmethod
def download_image(artist: Artist):
"""
Checks if an artist image exists and downloads it if not.
:param artist: The artist name
"""
img_path = (
settings.Paths().sm_artist_img_path / f"{artist.artisthash}.webp"
)
if img_path.exists():
return
url = get_artist_image_link(artist.name)
if url is not None:
return DownloadImage(url, name=f"{artist.artisthash}.webp")
+286
View File
@@ -0,0 +1,286 @@
"""
Contains everything that deals with image colour extraction.
"""
import logging
import os
import pathlib
import colorgram
from pathlib import Path
from typing import Generator
from swingmusic.utils.progressbar import tqdm
from concurrent.futures import ProcessPoolExecutor, as_completed
from swingmusic import settings
from swingmusic.store.albums import AlbumStore
from swingmusic.db.userdata import LibDataTable
from swingmusic.store.artists import ArtistStore
log = logging.getLogger(__name__)
def get_image_colors(image: pathlib.Path, count=1) -> list[str]:
"""
Extracts ``count`` numbers of the most dominant colours from an image.
:params image: Path to image.
:params count: How many colours should be extracted?
:returns: List["rgb(red, green, blue)", ...]
"""
if image.exists():
colors = sorted(colorgram.extract(image, count), key=lambda c: c.hsl.h)
else:
return []
formatted_colors = []
for color in colors:
color = f"rgb({color.rgb.r}, {color.rgb.g}, {color.rgb.b})"
formatted_colors.append(color)
return formatted_colors
def process_color(item_hash: str, is_album=True) -> list[str]:
"""
Parse colours from images associated with song
:param item_hash: hash of item for colour calculation
:param is_album: if item is an album
:return: list with colour strings
"""
if is_album:
path = settings.Paths().sm_thumb_path
else:
path = settings.Paths().sm_artist_img_path
path = path / (item_hash + ".webp")
if not path.exists():
return []
return get_image_colors(path)
def extract_color_worker(item_data: dict) -> dict:
"""
Generic worker function for extracting colours in parallel.
Returns data to main process for batch database operations.
Works for both albums and artists based on item_data configuration.
"""
hash_field: str = item_data["hash_field"]
path_func: Path = item_data["path_func"]
item_hash: str = item_data[hash_field]
path = path_func / (item_hash + ".webp")
if not path.exists():
return {hash_field: item_hash, "color": None, "error": "Image not found"}
colors = get_image_colors(path)
if not colors:
return {
hash_field: item_hash,
"color": None,
"error": "Color extraction failed",
}
return {hash_field: item_hash, "color": colors[0], "error": None}
class ColorProcessor:
"""
Generic color processor for extracting dominant colors from images.
Uses multiprocessing for parallel color extraction and batch database operations.
"""
def __init__(
self,
item_type: str,
store: AlbumStore | ArtistStore,
path_func: Path,
hash_field: str,
):
"""
Initialize the color processor.
Args:
item_type: Type of item ("album" or "artist")
store: Store object (AlbumStore or ArtistStore)
path_func: Function to get the image path
hash_field: Name of the hash field ("albumhash" or "artisthash")
"""
self.item_type = item_type
self.store = store
self.path_func = path_func
self.hash_field = hash_field
# Read existing colors from database to filter out already processed items
existing_colors = set()
for color_data in LibDataTable.get_all_colors(item_type):
if color_data["color"]:
existing_colors.add(color_data["itemhash"])
# Filter items that need color processing
items_needing_colors = self._get_items_needing_colors(existing_colors)
if not items_needing_colors:
return
self._process_colors_parallel(items_needing_colors)
def _get_items_needing_colors(
self, existing_colors: set
) -> Generator[dict, None, None]:
"""
Generator that yields items needing color processing.
"""
for item in self.store.get_flat_list():
# Skip if item already has color in memory store
if item.color:
continue
# Skip if item already has color in database
item_hash = getattr(item, self.hash_field)
if item_hash in existing_colors:
continue
yield {
self.hash_field: item_hash,
"item_type": self.item_type,
"path_func": self.path_func,
"hash_field": self.hash_field,
}
def _process_colors_parallel(self, items: Generator[dict, None, None]) -> None:
"""
Process colors using multiprocessing and batch database operations.
"""
items_list = list(items)
if not items_list:
return
cpus = max(1, (os.cpu_count() or 1) // 2)
batch_size = 20 # Process results in batches
with ProcessPoolExecutor(max_workers=cpus) as executor:
# Submit all jobs
future_to_item = {
executor.submit(extract_color_worker, item): item for item in items_list
}
batch = []
processed_count = 0
# Process results as they complete
progress_bar = tqdm(
as_completed(future_to_item),
total=len(items_list),
desc=f"Processing {self.item_type} colors",
)
for future in progress_bar:
try:
result = future.result()
if result["color"] is not None:
batch.append(result)
# Process batch when it reaches batch_size or we're done
if len(batch) >= batch_size or processed_count + 1 >= len(
items_list
):
if batch:
self._process_batch(batch)
batch = []
processed_count += 1
except Exception as e:
item_data = future_to_item[future]
item_hash = item_data[self.hash_field]
log.error(f"Error processing {self.item_type} {item_hash}: {e}")
def _process_batch(self, batch: list[dict]) -> None:
"""
Process a batch of color results - update database and memory stores.
"""
if not batch:
return
# Prepare database records
db_inserts = []
db_updates = []
for result in batch:
item_hash = result[self.hash_field]
color = result["color"]
# Check if record exists in database
existing_record = LibDataTable.find_one(item_hash, type=self.item_type)
if existing_record is None:
db_inserts.append(
{
"itemhash": self.item_type + item_hash,
"color": color,
"itemtype": self.item_type,
}
)
else:
db_updates.append(
{"itemhash": self.item_type + item_hash, "color": color}
)
# Batch database operations
if db_inserts:
LibDataTable.insert_many(db_inserts)
if db_updates:
for update_data in db_updates:
clean_hash = update_data["itemhash"].replace(self.item_type, "")
LibDataTable.update_one(clean_hash, {"color": update_data["color"]})
# Update in-memory store
store_map = getattr(self.store, f"{self.item_type}map")
for result in batch:
item_hash = result[self.hash_field]
color = result["color"]
item = store_map.get(item_hash)
if item:
item.set_color(color)
class ProcessAlbumColors:
"""
Extracts the most dominant color from the album art and saves it to the database.
Uses multiprocessing for parallel color extraction and batch database operations.
"""
def __init__(self) -> None:
ColorProcessor(
item_type="album",
store=AlbumStore,
path_func=settings.Paths().sm_thumb_path,
hash_field="albumhash",
)
class ProcessArtistColors:
"""
Extracts the most dominant colour from the artist art and saves it to the database.
Uses multiprocessing for parallel colour extraction and batch database operations.
"""
def __init__(self) -> None:
ColorProcessor(
item_type="artist",
store=ArtistStore,
path_func=settings.Paths().sm_artist_img_path,
hash_field="artisthash",
)
View File
+37
View File
@@ -0,0 +1,37 @@
from typing import Any
from swingmusic.store.albums import AlbumStore
from swingmusic.store.artists import ArtistStore
from swingmusic.store.tracks import TrackStore
def get_extra_info(hash: str, type: str):
"""
Generates extra info for a track, album or artist, which will be stored
in the database (in favorites, playlists and scrobble data) for backup and restore.
The extra info contains all the fields needed to reconstruct the itemhash. The track contains an additional filepath field which can be used to locate the file when restoring.
"""
extra: dict[str, Any] = {}
if type == "track":
trackentry = TrackStore.trackhashmap.get(hash)
if trackentry is not None:
track = trackentry.get_best()
extra["filepath"] = track.filepath
extra["title"] = track.title
extra["artists"] = [a["name"] for a in track.artists]
extra["album"] = track.albumhash
elif type == "album":
album = AlbumStore.get_album_by_hash(hash)
if album is not None:
extra["albumartists"] = [a["name"] for a in album.albumartists]
extra["title"] = album.title
elif type == "artist":
artist = ArtistStore.get_artist_by_hash(hash)
if artist is not None:
extra["name"] = artist.name
return extra
+156
View File
@@ -0,0 +1,156 @@
import pathlib
from pathlib import Path
import logging
from swingmusic.lib.sortlib import sort_folders, sort_tracks
from swingmusic.models import Folder
from swingmusic.serializers.track import serialize_tracks
from swingmusic.utils.filesystem import SUPPORTED_FILES
from swingmusic.store.folder import FolderStore
log = logging.getLogger(__name__)
def create_folder(path: str, trackcount=0) -> Folder:
"""
Creates a folder object from a path.
"""
folder = Path(path)
return Folder(
name=folder.name,
path=folder.as_posix() + "/",
is_sym=folder.is_symlink(),
trackcount=trackcount,
)
def get_folders(paths: list[str]):
"""
Filters out folders that don't have any tracks and
returns a list of folder objects.
"""
folders = FolderStore.count_tracks_containing_paths(paths)
return [
create_folder(f["path"], f["trackcount"])
for f in folders
if f["trackcount"] > 0
]
def get_files_and_dirs(
path: pathlib.Path,
start: int,
limit: int,
tracksortby: str,
foldersortby: str,
tracksort_reverse: bool,
foldersort_reverse: bool,
tracks_only: bool = False,
skip_empty_folders=True,
) -> dict[str: list|int|str]:
"""
Scan folder for files and folders.
Will only return files in `swingmusic.utils.filesystem.SUPPORTED_FILES`.
If `skip_empty_folders` is True
:param path:
:param start:
:param limit:
:param tracksortby:
:param foldersortby:
:param tracksort_reverse:
:param foldersort_reverse:
:param tracks_only: If True, will only return tracks with no folders
:param skip_empty_folders: If True, will call recursively and skip empty folders until >0 supported file found.
:returns: List of tracks and folders in that immediate path.
"""
path = pathlib.Path(path)
# if file or non-existent
if not path.exists() or not path.is_dir():
return {
"path": path.as_posix(),
"tracks": [],
"folders": [],
"total": 0
}
# iter through all folders
# add files with supported suffix
# ignore hidden folder
dirs, files = [], []
for entry in path.iterdir():
ext = entry.suffix.lower()
if entry.is_dir() and not entry.stem.startswith("."):
dirs.append((entry / "").as_posix())
# only append as posix for FolderStore and sort_folder function
# TODO: rework everything to support pathlib
# add a trailing slash to the folder path
# to avoid matching a folder starting with the same name as the root path
# eg. .../Music and .../Music VideosI
elif entry.is_file() and ext in SUPPORTED_FILES:
files.append(entry)
"""
# sort files by most recent
# TODO: rework if realy needed.
files_with_mtime = []
for file in files:
try:
files_with_mtime.append(
{
"path": file.as_posix(),
"time": file.lstat().st_mtime,
}
)
except OSError as e:
log.error(e)
files_with_mtime.sort(key=lambda f: f["time"])
files = [f["path"] for f in files_with_mtime]
"""
# if supported files were found
# convert files to tracks
tracks = []
if len(files) > 0:
if limit == -1:
limit = len(files)
# only return tracks already indexed by us
tracks = list(FolderStore.get_tracks_by_filepaths(files))
tracks = sort_tracks(tracks, tracksortby, tracksort_reverse)
tracks = tracks[start : start + limit]
folders = []
if not tracks_only:
folders = get_folders(dirs)
folders = sort_folders(folders, foldersortby, foldersort_reverse)
if skip_empty_folders and len(folders) == 1 and len(tracks) == 0:
# INFO: When we only have one folder and no tracks,
# skip through empty folders.
# Call recursively with the first folder in the list.
return get_files_and_dirs(
folders[0].path,
start=start,
limit=limit,
tracksortby=tracksortby,
foldersortby=foldersortby,
tracksort_reverse=tracksort_reverse,
foldersort_reverse=foldersort_reverse,
tracks_only=tracks_only,
skip_empty_folders=True,
)
return {
"path": path.as_posix(),
"tracks": serialize_tracks(tracks),
"folders": folders,
"total": len(files),
}
+30
View File
@@ -0,0 +1,30 @@
from swingmusic.db.userdata import MixTable
from swingmusic.plugins.mixes import MixesPlugin
def find_mix(mixid: str, sourcehash: str):
"""
Find a mix in the homepage store or the db.
"""
from swingmusic.store.homepage import HomepageStore
mixtype = "custom_mixes" if mixid[0] == "t" else "artist_mixes"
# INFO: Try getting the mix from the homepage store
mix = HomepageStore.get_mix(mixtype, mixid)
if mix and mix["sourcehash"] == sourcehash:
return mix
# INFO: Get the mix from the db
mix = MixTable.get_by_sourcehash(sourcehash)
if not mix:
return None
if mixtype == "custom_mixes":
mix = MixesPlugin.get_track_mix(mix)
if not mix:
return None
return mix.to_dict()
+170
View File
@@ -0,0 +1,170 @@
import os
import pathlib
from swingmusic.db.userdata import PlaylistTable
from swingmusic.lib.home import find_mix
from swingmusic.lib.home.recentlyadded import get_recently_added_playlist
from swingmusic.lib.home.recentlyplayed import get_recently_played_playlist
from swingmusic.models.logger import TrackLog
from swingmusic.store.albums import AlbumStore
from swingmusic.store.artists import ArtistStore
from swingmusic.store.tracks import TrackStore
def create_items(entries: list[TrackLog], limit: int):
"""
TODO: rework so that returns a dict with
{
"recently_played": ...,
"artist_mixes_for_you": ...
}
also keep in mind that the web-ui is beeing translated.
"""
custom_playlists = [
{"name": "recentlyadded", "handler": get_recently_added_playlist},
{"name": "recentlyplayed", "handler": get_recently_played_playlist},
]
items = []
added = set()
for entry in entries:
if len(items) >= limit:
break
if entry.source in added:
continue
added.add(entry.source)
if entry.type == "mix":
if not entry.type_src:
continue
splits = entry.type_src.split(".")
try:
mixid = splits[0]
sourcehash = splits[1]
except IndexError:
continue
# INFO: Get mix from homepage store
mix = find_mix(mixid, sourcehash)
if not mix:
continue
items.append(
{
"type": "mix",
"hash": entry.type_src,
"timestamp": entry.timestamp,
}
)
continue
if entry.type == "album":
album = AlbumStore.albummap.get(entry.type_src)
if album is None:
continue
item = {
"type": "album",
"hash": entry.type_src,
"timestamp": entry.timestamp,
}
items.append(item)
continue
if entry.type == "artist":
artist = ArtistStore.artistmap.get(entry.type_src)
if artist is None:
continue
items.append(
{
"type": "artist",
"hash": entry.type_src,
"timestamp": entry.timestamp,
}
)
continue
if entry.type == "folder":
folder = entry.type_src
if not folder:
continue
if not folder.endswith("/"):
folder += "/"
is_home_dir = entry.type_src == "$home"
if is_home_dir:
folder = os.path.expanduser("~")
if not pathlib.Path(folder).exists():
continue
item = {
"type": "folder",
"hash": folder,
"timestamp": entry.timestamp,
}
items.append(item)
continue
if entry.type == "playlist":
is_custom = entry.type_src in [i["name"] for i in custom_playlists]
if is_custom:
items.append(
{
"type": "playlist",
"hash": entry.type_src,
"timestamp": entry.timestamp,
"is_custom": True,
}
)
continue
playlist = PlaylistTable.get_by_id(entry.type_src)
if playlist is None:
continue
item = {
"type": "playlist",
"hash": entry.type_src,
"timestamp": entry.timestamp,
}
items.append(item)
continue
if entry.type == "favorite":
items.append(
{
"type": "favorite",
"timestamp": entry.timestamp,
}
)
continue
t = TrackStore.trackhashmap.get(entry.trackhash)
if t is None:
continue
item = {
"type": "track",
"hash": entry.trackhash,
"timestamp": entry.timestamp,
}
items.append(item)
return items
@@ -0,0 +1,42 @@
from swingmusic.db.userdata import ScrobbleTable
from swingmusic.lib.home.create_items import create_items
from swingmusic.models.logger import TrackLog
def get_recently_played(
limit: int, userid: int | None = None, _entries: list[TrackLog] = []
):
"""
Get the recently played items for the homepage.
Pass a list of track log entries to use a subset of the scrobble table.
"""
# TODO: Paginate this
items = []
BATCH_SIZE = 200
current_index = 0
if len(_entries):
entries = _entries
limit = 1
else:
entries = ScrobbleTable.get_all(0, BATCH_SIZE, userid=userid)
max_iterations = 20
iterations = 0
while len(items) < limit and iterations < max_iterations:
items.extend(create_items(entries, limit))
current_index += BATCH_SIZE
if len(items) < limit:
entries = ScrobbleTable.get_all(
start=current_index + 1, limit=BATCH_SIZE, userid=userid
)
if not entries:
break
iterations += 1
return items
+217
View File
@@ -0,0 +1,217 @@
import pathlib
from datetime import datetime
from swingmusic.lib.playlistlib import get_first_4_images
from swingmusic.models.playlist import Playlist
from swingmusic.models.track import Track
from swingmusic.store.tracks import TrackStore
from swingmusic.store.albums import AlbumStore
from swingmusic.store.artists import ArtistStore
from itertools import groupby
from swingmusic.utils import flatten
from swingmusic.utils.dates import (
create_new_date,
date_string_to_time_passed,
)
older_albums = set()
older_artists = set()
def calc_based_on_percent(items: list[str], total: int):
"""
Checks if items is more than 85% of total items. Returns a boolean and the most common item.
"""
most_common = max(items, key=items.count)
most_common_count = items.count(most_common)
return most_common_count / total >= 0.7, most_common, most_common_count
def check_is_album_folder(tracks: list[Track]):
albumhashes = [t.albumhash for t in tracks]
return calc_based_on_percent(albumhashes, len(tracks))
def check_is_artist_folder(tracks: list[Track]):
# INFO: flatten artist hashes using "-" as a separator
artisthashes = flatten([t.artisthashes for t in tracks])
return calc_based_on_percent(artisthashes, len(tracks))
def check_is_track_folder(tracks: list[Track]):
# INFO: is more of a playlist
if len(tracks) >= 3:
return False
return [create_track(t) for t in tracks]
def create_track(t: Track):
"""
Creates a recently added track entry.
"""
return {
"type": "track",
"hash": t.trackhash,
"timestamp": t.last_mod,
"help_text": "NEW TRACK",
}
# INFO: Keys: folder, tracks, time (timestamp)
# group_type = dict[str, str | list[Track] | float]
def check_folder_type(group_: dict):
# check if all tracks in group have the same albumhash
# if so, return "album"
key: str = group_["folder"]
tracks: list[Track] = group_["tracks"]
time: float = group_["time"]
existing_artist_hashes: set[str] = set(ArtistStore.artistmap.keys())
existing_album_hashes: set[str] = set(AlbumStore.albummap.keys())
if len(tracks) == 1:
entry = create_track(tracks[0])
entry["timestamp"] = time
return entry
is_album, albumhash, _ = check_is_album_folder(tracks)
if is_album:
# album = AlbumTable.get_album_by_albumhash(albumhash)
entry = AlbumStore.albummap.get(albumhash)
if entry is None:
return None
return {
"type": "album",
"hash": albumhash,
"timestamp": time,
"help_text": (
"NEW ALBUM" if albumhash in existing_album_hashes else "NEW TRACKS"
),
}
is_artist, artisthash, trackcount = check_is_artist_folder(tracks)
if is_artist:
entry = ArtistStore.artistmap.get(artisthash)
if entry is None:
return None
return {
"type": "artist",
"hash": artisthash,
"timestamp": time,
"help_text": (
"NEW ARTIST" if artisthash not in existing_artist_hashes else "NEW MUSIC"
),
}
is_track_folder = check_is_track_folder(tracks)
return (
is_track_folder
if is_track_folder
else {
"type": "folder",
"hash": key,
"timestamp": time,
"help_text": "NEW MUSIC",
}
)
def group_track_by_folders(tracks: list[Track], groups: dict[str, list[Track]]):
"""
Groups tracks by folder and returns a list of groups sorted by last modified date.
Uses generator expressions to avoid creating intermediate lists.
"""
# INFO: sort tracks by folder name, then group by folder name
tracks = sorted(tracks, key=lambda t: t.folder)
thisgroup = groupby(tracks, lambda t: t.folder)
for folder, thistracks in thisgroup:
groups.setdefault(folder, []).extend(thistracks)
return groups
def get_recently_added_items(limit: int = 7):
tracks = get_recently_added_tracks(start=0, limit=None)
groups = group_track_by_folders(tracks, {})
grouplist = []
# INFO: sort tracks by last modified date in descending order to get the most recent last modified date
for folder, trackgroup in groups.items():
if not pathlib.Path(folder).exists():
continue
trackgroup.sort(key=lambda t: t.last_mod, reverse=True)
grouplist.append(
{
"folder": folder,
"len": len(trackgroup),
"tracks": trackgroup,
"time": trackgroup[0].last_mod,
}
)
# sort groups by last modified date
grouplist = sorted(grouplist, key=lambda group: group["time"], reverse=True)
recent_items = []
for group in grouplist:
item = check_folder_type(group)
if item not in recent_items:
if not item:
continue
(
recent_items.append(item)
if type(item) == dict
else recent_items.extend(item)
)
if len(recent_items) >= limit:
break
return recent_items
def get_recently_added_playlist(limit: int = 100):
playlist = Playlist(
id="recentlyadded",
name="Recently Added",
image=None,
last_updated="Now",
settings={},
trackhashes=[],
)
tracks = get_recently_added_tracks(limit=limit)
try:
# Create date to show as last updated
date = datetime.fromtimestamp(tracks[0].last_mod)
except IndexError:
return playlist, []
playlist._last_updated = date_string_to_time_passed(create_new_date(date))
images = get_first_4_images(tracks=tracks)
playlist.images = images
playlist.duration = sum(t.duration for t in tracks)
playlist.count = len(tracks)
return playlist, tracks
def get_recently_added_tracks(start: int = 0, limit: int | None = 100):
return TrackStore.get_recently_added(start, limit)
+35
View File
@@ -0,0 +1,35 @@
from datetime import datetime
from swingmusic.db.userdata import ScrobbleTable
from swingmusic.models.playlist import Playlist
from swingmusic.lib.playlistlib import get_first_4_images
from swingmusic.utils.dates import (
create_new_date,
date_string_to_time_passed,
)
from swingmusic.store.tracks import TrackStore
def get_recently_played_playlist(limit: int = 100):
playlist = Playlist(
id="recentlyplayed",
name="Recently Played",
image=None,
last_updated="Now",
settings={},
trackhashes=[],
)
scrobbles = ScrobbleTable.get_all(None, 100)
tracks = TrackStore.get_tracks_by_trackhashes(
[scrobble.trackhash for scrobble in scrobbles]
)
date = datetime.fromtimestamp(tracks[0].lastplayed)
playlist._last_updated = date_string_to_time_passed(create_new_date(date))
images = get_first_4_images(tracks=tracks)
playlist.images = images
return playlist, tracks
+161
View File
@@ -0,0 +1,161 @@
from swingmusic.db.userdata import FavoritesTable, PlaylistTable
from swingmusic.lib.home import find_mix
from swingmusic.lib.home.recentlyadded import get_recently_added_playlist
from swingmusic.lib.home.recentlyplayed import get_recently_played_playlist
from swingmusic.lib.playlistlib import get_first_4_images
from swingmusic.serializers.album import album_serializer
from swingmusic.serializers.artist import serialize_for_card
from swingmusic.serializers.playlist import serialize_for_card as serialize_playlist
from swingmusic.serializers.track import serialize_track
from swingmusic.store.albums import AlbumStore
from swingmusic.store.artists import ArtistStore
from swingmusic.store.folder import FolderStore
from swingmusic.store.tracks import TrackStore
from swingmusic.utils.dates import timestamp_to_time_passed
def recover_items(items: list[dict]):
custom_playlists = [
{"name": "recentlyadded", "handler": get_recently_added_playlist},
{"name": "recentlyplayed", "handler": get_recently_played_playlist},
]
recovered = []
for item in items:
recovered_item = None
if item["type"] == "album":
album = AlbumStore.get_album_by_hash(item["hash"])
if album is None:
continue
album = album_serializer(
album,
to_remove={
"genres",
"date",
"count",
"duration",
"albumartists_hashes",
"og_title",
},
)
recovered_item = {
"type": "album",
"item": album,
}
elif item["type"] == "artist":
artist = ArtistStore.get_artist_by_hash(item["hash"])
if artist is None:
continue
recovered_item = {
"type": "artist",
"item": serialize_for_card(artist),
}
elif item["type"] == "folder":
count = FolderStore.count_tracks_containing_paths([item["hash"]])
recovered_item = {
"type": "folder",
"item": {
"path": item["hash"],
"count": count[0]["trackcount"],
},
}
elif item["type"] == "playlist":
if item.get("is_custom"):
playlist, _ = next(
i["handler"]()
for i in custom_playlists
if i["name"] == item["hash"]
)
playlist.images = [i["image"] for i in playlist.images]
playlist = serialize_playlist(
playlist, to_remove={"settings", "duration"}
)
recovered_item = {
"type": "playlist",
"item": playlist,
}
else:
playlist = PlaylistTable.get_by_id(item["hash"])
if playlist is None:
continue
tracks = TrackStore.get_tracks_by_trackhashes(playlist.trackhashes)
playlist.clear_lists()
if not playlist.has_image:
images = get_first_4_images(tracks)
images = [i["image"] for i in images]
playlist.images = images
recovered_item = {
"type": "playlist",
"item": serialize_playlist(playlist),
}
elif item["type"] == "favorite":
image = None
last_trackhash = FavoritesTable.get_last_trackhash()
if last_trackhash:
trackhash = last_trackhash.replace("track_", "")
entry = TrackStore.trackhashmap.get(trackhash)
if entry:
image = entry.tracks[0].image
recovered_item = {
"type": "favorite",
"item": {
"count": FavoritesTable.count_tracks(),
"image": image,
},
}
elif item["type"] == "track":
track = TrackStore.trackhashmap.get(item["hash"])
if track is None:
continue
recovered_item = {
"type": "track",
"item": serialize_track(track.get_best()),
}
elif item["type"] == "mix":
try:
splits = item["hash"].split(".")
mixid = splits[0]
sourcehash = splits[1]
except IndexError:
continue
mix = find_mix(mixid, sourcehash)
if mix is None:
continue
recovered_item = {
"type": "mix",
"item": mix,
}
if recovered_item is not None:
helptext = item.get("help_text") or item.get("type")
secondary_text = item.get("secondary_text")
if "secondary_text" in item:
secondary_text = item["secondary_text"]
elif "timestamp" in item:
secondary_text = timestamp_to_time_passed(item["timestamp"])
if helptext:
recovered_item["item"]["help_text"] = helptext
if secondary_text:
recovered_item["item"]["time"] = secondary_text
recovered.append(recovered_item)
return recovered
+43
View File
@@ -0,0 +1,43 @@
import gc
import logging
from time import time
from swingmusic.lib.mapstuff import (
map_album_colors,
map_artist_colors,
map_favorites,
map_scrobble_data,
)
from swingmusic.lib.populate import CordinateMedia
from swingmusic.lib.recipes.recents import RecentlyAdded
from swingmusic.lib.tagger import IndexTracks
from swingmusic.store.albums import AlbumStore
from swingmusic.store.artists import ArtistStore
from swingmusic.store.folder import FolderStore
from swingmusic.store.tracks import TrackStore
from swingmusic.utils.threading import background
log = logging.getLogger(__name__)
@background
def index_everything():
IndexTracks()
key = str(time())
TrackStore.load_all_tracks(key)
AlbumStore.load_albums(key)
ArtistStore.load_artists(key)
FolderStore.load_filepaths()
# NOTE: Rebuild recently added items on the homepage store
RecentlyAdded()
# map colors
map_album_colors()
map_artist_colors()
map_scrobble_data()
map_favorites()
CordinateMedia(instance_key=str(time()))
gc.collect()
log.info("Indexing completed")
+343
View File
@@ -0,0 +1,343 @@
import datetime
import pathlib
from pathlib import Path
from swingmusic.store.tracks import TrackStore
# # # # # # # # # # # # # # # # # # # #
# Functions for parsing lyrics lines #
# # # # # # # # # # # # # # # # # # # #
def parse_lyrics_lines(lyrics:str) -> list[dict]:
"""
Split lyrics into lines and determine there tag type.
Parses the tag if the following format is present: [tag]*[tags] <body>
else tag_type is unknown
tag-type and tags are lists combined by their index
:param lyrics: Full lyrics body
:return: {'tag_types', 'body', 'tags'}
"""
entries = []
for line in lyrics.splitlines():
data = {
"tag_types": [],
"tags": []
}
if line.startswith("["):
# loop until all tags are parsed in line
while True:
if "[" in line and "]" in line: # second tag
bracket_content, after_content = line.split("]", 1)
bracket_content = bracket_content.removeprefix("[")
data["tags"].append(bracket_content)
data["body"] = after_content
line = after_content
# check which tag type it is
if bracket_content[0].isnumeric():
data["tag_types"].append( "time" )
elif bracket_content[0].isalpha():
data["tag_types"].append( "meta" )
else:
# if no brackets inside the line, there is also no tag.
break
elif line.startswith("#"):
data["tag_types"].append("comment")
data["tags"] = ""
data["body"] = line
else:
data["tag_types"].append("unknown")
data["tags"] = "unknown"
data["body"] = line
entries.append(data)
return entries
def filter_parse_lyrics_lines(lines:list[dict], tag_types:list|str) -> list[dict]:
"""
filter all lyrics line to only contain given tags
:param lines: list returned by `parse_lyrics_lines`
:param tag_types: list or string of tags return should contain
"""
if isinstance(tag_types, str):
tag_types = [tag_types]
found_tags = []
# line = {"tags", "body", "tag_types"}
for line in lines:
group = {
"tag_types": [],
"tags": []
}
for (tag, tag_type) in zip(line["tags"], line["tag_types"]):
if tag_type in tag_types:
group["tag_types"].append(tag_type)
group["tags"].append(tag)
group["body"] = line["body"]
# filter out no match
if len(group["tags"]) > 0:
found_tags.append(group)
return found_tags
def parse_time_tag(lines:list[dict]) -> list[dict]:
"""
Filter time-tags from lines and parse them.
"""
# filter tag-type time
# format into dict with timestamps
parsed_times = []
time_tags = filter_parse_lyrics_lines(lines, "time")
# line = {"tags", "body", "tag_types"}
for line in time_tags:
for (tag, tag_type) in zip(line["tags"], line["tag_types"]):
minute, seconds = tag.split(":", 1)
parsed_times.append({
"minute": minute,
"seconds": seconds,
"body": line["body"],
})
return parsed_times
# # # # # # # # # # # # # # # # # # # #
# Lyrics class for simplified usage #
# # # # # # # # # # # # # # # # # # # #
class Lyrics:
SUPPORTED_METATAGS = {
"ti": "title",
"ar": "artist",
"al": "album",
"au": "author",
"lr": "lyricist",
"length": "length",
"by": "lrc_author",
"offset": "offset",
"re": "recorder",
"tool": "tool",
"ve": "version"
}
lyrics:str
parsed_lyrics:list[dict]
meta:dict = {}
is_synced:bool = False
def __init__(self, lyrics:str=""):
"""
:param lyrics: entire lyrics body
"""
if lyrics is None:
raise ValueError("Lyrics can not be None")
if isinstance(lyrics, list):
lyrics = lyrics[0]
lyrics = lyrics.replace("engdesc", "")
self.lyrics = lyrics
parsed = parse_lyrics_lines(lyrics)
# translate meta tags
meta = filter_parse_lyrics_lines(parsed, "meta")
for line in meta:
for tag in line["tags"]:
name, body = tag.split(":", 1)
name = name.lower()
dict_name = self.SUPPORTED_METATAGS.get(name, name)
self.meta[dict_name] = body
# check if synced or not.
# not fail-save:
# If even just one time tag in the entire lyrics gets flagged as synced
if len(filter_parse_lyrics_lines(parsed, "time")) > 0:
self.is_synced = True
self.parsed_lyrics = filter_parse_lyrics_lines(parsed, "time")
else:
self.is_synced = False
self.parsed_lyrics = filter_parse_lyrics_lines(parsed, "unknown")
# TODO: add support for multilanguage lyrics
def format_synced_lyrics(self):
"""
Formats synced lyrics into a list of dicts
"""
if not self.is_synced:
raise ValueError("Cannot format synced lyrics if no synced lyrics exist for track.\nPlease use `format_unsynced_lyrics()`")
lyrics = []
time_tags = parse_time_tag(self.parsed_lyrics)
for entry in time_tags:
minutes = entry["minute"]
if "." in entry["seconds"]:
seconds = entry["seconds"].split(".")[0]
milli = entry["seconds"].split(".")[-1]
else:
seconds = entry["seconds"]
milli = "0"
minutes = int(minutes)
seconds = int(seconds)
milli = int(milli)
seconds = datetime.timedelta(minutes=minutes, seconds=seconds, milliseconds=milli).total_seconds()
offset = 0
if "offset" in self.meta:
offset = int(self.meta["offset"]) # offset in milliseconds
milliseconds = seconds * 1000 - offset
lyrics.append({"time": milliseconds, "text": entry["body"]})
return lyrics
def format_unsynced_lyrics(self) -> list[str]:
"""
return unsynced lyrics.
If no lyrics provided return empty string.
"""
lyrics = [item["body"] for item in self.parsed_lyrics]
return lyrics
def __bool__(self):
"""
return True if contains anything
"""
return bool(self.parsed_lyrics)
# # # # # # # # # # # # # # # # # # # # # # # # # # #
# Path and parse function to get lyrics from track #
# # # # # # # # # # # # # # # # # # # # # # # # # # #
def get_lyrics_file(track_path: str|pathlib.Path) -> Lyrics:
"""
Try to get lyrics from a relative lrc file.
:param track_path: path of track
"""
track_path = Path(track_path)
lyrics_path = track_path.with_suffix(".lrc")
extended_path = track_path.with_suffix(".rlrc")
# check paths
if lyrics_path.exists():
lyrics = Lyrics(lyrics_path.read_text())
return lyrics
elif extended_path.exists():
lyrics = Lyrics(extended_path.read_text())
return lyrics
else:
return Lyrics()
def get_lyrics_from_duplicates(track_path: str, trackhash: str) -> Lyrics:
"""
Finds the lyrics from other duplicate tracks
:param track_path: path of track
:param trackhash: Track-hash value
"""
entry = TrackStore.trackhashmap.get(trackhash, None)
if entry is None:
return Lyrics()
for track in entry.tracks:
if track.trackhash == trackhash and track.filepath != track_path:
lyrics = get_lyrics_file(track.filepath)
if lyrics:
return lyrics
return Lyrics()
def get_lyrics_from_tags(trackhash: str) -> Lyrics:
"""
Gets the lyrics from the tags of the track
:param trackhash:
"""
entry = TrackStore.trackhashmap.get(trackhash, None)
if entry is None:
return Lyrics()
for track in entry.tracks:
if "lyrics" in track.extra:
lyrics = track.extra["lyrics"]
if lyrics:
return Lyrics(lyrics)
return Lyrics("")
def check_lyrics_file(filepath: str, trackhash: str):
"""
Checks if the lyrics file exists for a track
"""
lyrics_file = Path(filepath).with_suffix(".lrc")
if lyrics_file.exists:
return True
entry = TrackStore.trackhashmap.get(trackhash, None)
if entry is None:
return False
for track in entry.tracks:
if track.trackhash == trackhash and track.filepath != filepath:
lyrics_file = Path(track.filepath).with_suffix(".lrc")
if lyrics_file.exists():
return True
return False
+94
View File
@@ -0,0 +1,94 @@
from swingmusic.db.userdata import LibDataTable, FavoritesTable, ScrobbleTable
from swingmusic.store.albums import AlbumStore
from swingmusic.store.artists import ArtistStore
from swingmusic.store.tracks import TrackStore
from typing import Any
def map_scrobble_data():
"""
Maps scrobble data to the in-memory stores.
The scrobble data is loaded from the database and grouped by trackhash.
The album and artist scrobble data (for those tracks) are then incremented based on the data.
"""
records = ScrobbleTable.get_all(0, None)
# group records by trackhash
grouped: dict[str, dict[str, Any]] = {}
for record in records:
# aggregate playcount, playduration and lastplayed
item = grouped.setdefault(record.trackhash, {})
item["playcount"] = item.get("playcount", 0) + 1
item["playduration"] = item.get("playduration", 0) + record.duration
item["lastplayed"] = max(item.get("lastplayed", 0), record.timestamp)
# increment playcount, playduration and lastplayed for albums and artists
for trackhash, data in grouped.items():
track = TrackStore.trackhashmap.get(trackhash)
if track is None:
continue
track.increment_playcount(
data["playduration"], data["lastplayed"], data["playcount"]
)
album = AlbumStore.albummap.get(track.tracks[0].albumhash)
if album:
album.increment_playcount(
data["playduration"], data["lastplayed"], data["playcount"]
)
for artisthash in track.tracks[0].artisthashes:
artist = ArtistStore.artistmap.get(artisthash)
if artist:
artist.increment_playcount(
data["playduration"], data["lastplayed"], data["playcount"]
)
def map_favorites():
"""
Maps favorites data to the in-memory stores.
"""
favorites = FavoritesTable.get_all()
for entry in favorites:
if entry.type == "album":
album = AlbumStore.albummap.get(entry.hash)
if album:
album.toggle_favorite_user(entry.userid)
elif entry.type == "artist":
artist = ArtistStore.artistmap.get(entry.hash)
if artist:
artist.toggle_favorite_user(entry.userid)
elif entry.type == "track":
track = TrackStore.trackhashmap.get(entry.hash)
if track:
track.toggle_favorite_user(entry.userid)
def map_artist_colors():
colors = LibDataTable.get_all_colors(type="artist")
for color in colors:
artist = ArtistStore.artistmap.get(color["itemhash"])
if artist:
artist.set_color(color["color"])
def map_album_colors():
colors = LibDataTable.get_all_colors(type="album")
for color in colors:
album = AlbumStore.albummap.get(color["itemhash"])
if album:
album.set_color(color["color"])
+76
View File
@@ -0,0 +1,76 @@
import json
from typing import Any
from swingmusic.serializers.album import serialize_for_card
from swingmusic.serializers.artist import serialize_for_card as serialize_artist
from swingmusic.store.albums import AlbumStore
from swingmusic.store.artists import ArtistStore
from swingmusic.utils.hashing import create_hash
def validate_page_items(items: list[dict[str, str]], existing: list[dict[str, str]]):
"""
Validate the items in a page before adding them to the database.
"""
validated: list[dict[str, str]] = []
indexed = set(create_hash(json.dumps(item)) for item in existing)
for item in items:
if create_hash(json.dumps(item)) in indexed:
continue
if item["type"] == "album":
album = AlbumStore.albummap.get(item["hash"])
if album is not None:
validated.append(item)
elif item["type"] == "artist":
artist = ArtistStore.artistmap.get(item["hash"])
if artist is not None:
validated.append(item)
else:
raise ValueError(f"Invalid item type: {item['type']}")
return validated
def remove_page_items(existing: list[dict[str, str]], item: dict[str, str]):
return [
i
for i in existing
if create_hash(json.dumps(i)) != create_hash(json.dumps(item))
]
def recover_page_items(items: list[dict[str, str]], for_homepage: bool = False):
"""
Recover the items in a page.
"""
recovered: list[dict[str, Any]] = []
for item in items:
if item["type"] == "album":
album = AlbumStore.albummap.get(item["hash"])
if album is not None:
item = serialize_for_card(album.album)
if for_homepage:
del item["type"]
item = {"item": item, "type": "album"}
recovered.append(item)
elif item["type"] == "artist":
artist = ArtistStore.artistmap.get(item["hash"])
if artist is not None:
item = serialize_artist(artist.artist)
if for_homepage:
del item["type"]
item = {"item": item, "type": "artist"}
recovered.append(item)
recovered.reverse()
return recovered
+171
View File
@@ -0,0 +1,171 @@
"""
This library contains all the functions related to playlists.
"""
import random
import string
import logging
from PIL import Image, ImageSequence
from swingmusic import settings
from swingmusic.models.track import Track
from swingmusic.store.albums import AlbumStore
from swingmusic.store.tracks import TrackStore
logger = logging.getLogger(__name__)
def create_thumbnail(image: Image, img_name: str) -> str:
"""
Creates a 250 px high thumbnail from the Image.
It will keep the aspect ratio.
Images are saved in the playlist-img path
:param image: Image object.
:param img_name: Name of image.
:return: Filename of image.
"""
aspect_ratio = image.width / image.height
new_w = round(250 * aspect_ratio)
thumb = image.resize((new_w, 250), Image.Resampling.LANCZOS)
thumb_filename = "thumb_" + img_name
thumb_path = settings.Paths().playlist_img_path / thumb_filename
thumb.save(thumb_path, "webp")
return thumb_filename
def create_gif_thumbnail(image: Image, img_name: str):
"""
Creates a 250 px high thumbnail from the provided GIF.
Keeps the aspect ratio.
Images are saved in the playlist-img path
:param image: Image object.
:param img_name: Name of image.
:return: Filename of image.
"""
thumb_name = "thumb_" + img_name
thumb_path = settings.Paths().playlist_img_path / thumb_name
frames = []
for frame in ImageSequence.Iterator(image):
aspect_ratio = frame.width / frame.height
new_w = round(250 * aspect_ratio)
thumb = frame.resize((new_w, 250), Image.Resampling.LANCZOS)
frames.append(thumb)
frames[0].save(thumb_path, save_all=True, append_images=frames[1:])
return thumb_name
def save_p_image(img: Image, pid: int, content_type: str = None, filename: str = None) -> str:
"""
Saves a playlist banner image and returns the filepath.
"""
# img = Image.open(file)
random_str = "".join(random.choices(string.ascii_letters + string.digits, k=5))
if not filename:
filename = str(pid) + str(random_str) + ".webp"
full_img_path = settings.Paths().playlist_img_path / filename
if content_type == "image/gif":
frames = []
for frame in ImageSequence.Iterator(img):
frames.append(frame.copy())
frames[0].save(full_img_path, save_all=True, append_images=frames[1:])
create_gif_thumbnail(img, img_path=filename)
return filename
img.save(full_img_path, "webp")
create_thumbnail(img, img_name=filename)
return filename
def duplicate_images(images: list):
if len(images) == 1:
images *= 4
elif len(images) == 2:
images += list(reversed(images))
elif len(images) == 3:
images = images + images[:1]
return images
# TODO: mutable var in param.
def get_first_4_images(
tracks: list[Track] = [],
trackhashes: list[str] = []
) -> list[dict["str", str]]:
"""
Returns images of the first 4 albums that appear in the track list.
When tracks are not passed, trackhashes need to be passed.
Tracks are then resolved from the store.
"""
if len(trackhashes) > 0:
tracks = TrackStore.get_tracks_by_trackhashes(trackhashes)
albums = []
for track in tracks:
if track.albumhash not in albums:
albums.append(track.albumhash)
if len(albums) == 4:
break
albums = AlbumStore.get_albums_by_hashes(albums)
images = [
{
"image": album.image,
"color": album.color,
}
for album in albums
]
if len(images) == 4:
return images
return duplicate_images(images)
def cleanup_playlist_images() -> None:
"""
Deletes all unlinked files in playlist-img folder.
All files not present in the PlaylistTable will get deleted
"""
# Import here to avoid circular import
from swingmusic.db.userdata import PlaylistTable
playlists = PlaylistTable.get_all()
linked_images = {p.image for p in playlists if p.image and p.image != "None"}
playlist_dir = settings.Paths().playlist_img_path
# Find unlinked images (including thumbnails)
for file in playlist_dir.iterdir():
if not file.isfile:
continue
name = file.name # not stem. PlaylistTable saves with extension
if file not in linked_images:
if name.removeprefix("thumb_") not in linked_images:
continue
try:
file.unlink(missing_ok=True)
except OSError as e:
logger.exception("could not delete file", exc_info=e)
+176
View File
@@ -0,0 +1,176 @@
import functools
import os
from dataclasses import asdict
import multiprocessing as mp
from requests import ReadTimeout
from concurrent.futures import ProcessPoolExecutor
from requests import ConnectionError as RequestConnectionError
import logging
from swingmusic import settings
from swingmusic.lib.artistlib import CheckArtistImages
from swingmusic.lib.taglib import extract_thumb
from swingmusic.models import Album, Artist
from swingmusic.models.lastfm import SimilarArtist
from swingmusic.models.track import Track
from swingmusic.store.albums import AlbumStore
from swingmusic.store.artists import ArtistStore
from swingmusic.utils.network import has_connection
from swingmusic.utils.progressbar import tqdm
from swingmusic.request.artists import fetch_similar_artists
from swingmusic.lib.colorlib import ProcessAlbumColors, ProcessArtistColors
from swingmusic.db.userdata import SimilarArtistTable
log = logging.getLogger(__name__)
class CordinateMedia:
"""
Cordinates the extracting of thumbnails
"""
def __init__(self, instance_key: str):
ProcessTrackThumbnails()
ProcessAlbumColors()
ProcessArtistColors()
tried_to_download_new_images = False
if has_connection():
tried_to_download_new_images = True
try:
CheckArtistImages()
except (RequestConnectionError, ReadTimeout) as e:
log.error(
"Internet connection lost. Downloading artist images suspended."
)
log.error(e) # REVIEW More informations = good
else:
log.warning("No internet connection. Downloading artist images suspended!")
# Re-process the new artist images.
if tried_to_download_new_images:
ProcessArtistColors()
if has_connection():
print("Attempting to download similar artists...")
FetchSimilarArtistsLastFM()
def get_image(tracks: list[Track], paths=None):
"""
The function retrieves an image from a list of tracks by extracting the thumbnail from the first track that has one.
:param tracks: A list of Track objects to extract the image from.
:type tracks: list[Track]
:return: None
"""
for track in tracks:
extracted = extract_thumb(track.filepath, track.albumhash + ".webp", paths)
if extracted:
return
class ProcessTrackThumbnails:
"""
Extracts the album art from all albums in album store.
"""
def extract(self, albums: list[Album]):
"""
Extracts the album art with platform-specific logic.
"""
cpus = max(1, os.cpu_count() // 2)
albumsMap = ( AlbumStore.get_album_tracks(album.albumhash) for album in albums )
# Create process pool with worker function
with mp.Pool(processes=cpus) as pool:
worker = functools.partial(get_image, paths=settings.Paths())
# Process files and track progress
results = list(
tqdm(
pool.imap_unordered(worker, albumsMap),
total=len(albums),
desc="Extracting track images",
)
)
list(results)
def __init__(self) -> None:
"""
Filters out albums that already have thumbnails and
extracts the thumbnail for the other albums.
"""
path = settings.Paths().sm_thumb_path
# read all the files in the thumbnail directory
processed = set(file.stem for file in path.iterdir())
# filter out albums that already have thumbnails
albums = filter(
lambda album: album.albumhash not in processed,
AlbumStore.get_flat_list(),
)
albums = list(albums)
self.extract(albums)
def save_similar_artists(artist: Artist):
"""
Downloads and saves similar artists to the database.
"""
if SimilarArtistTable.exists(artist.artisthash):
return
artists = fetch_similar_artists(artist.name)
# INFO: Nones mean there was a connection error
if artists is None:
return
artist_ = SimilarArtist(artist.artisthash, artists)
SimilarArtistTable.insert_one(asdict(artist_))
class FetchSimilarArtistsLastFM:
"""
Fetches similar artists from LastFM using a thread pool.
"""
def __init__(self) -> None:
# read all artists from db
storeArtists = ArtistStore.get_flat_list()
processed = set(a.artisthash for a in SimilarArtistTable.get_all())
# filter out artists that already have similar artists using generator
def artist_generator():
for artist in storeArtists:
if artist.artisthash in processed:
yield artist
artists = list(artist_generator())
cpus = max(1, os.cpu_count() // 2)
with ProcessPoolExecutor(max_workers=cpus) as executor:
try:
# TODO: fix negative total length
results = list(
tqdm(
executor.map(save_similar_artists, artist_generator()),
total=len(artists),
desc="Fetching similar artists",
)
)
list(results)
# any exception that can be raised by the pool
except Exception as e:
log.warning(e)
return
+17
View File
@@ -0,0 +1,17 @@
### Steps to reproduce
### Expected behavior
Tell us what should happen
### Actual behavior
Tell us what happens instead
### Your System configuration
- Python version:
- Pydub version:
- ffmpeg or avlib?:
- ffmpeg/avlib version:
### Is there an audio file you can include to help us reproduce?
You can include the audio file in this issue - just put it in a zip file and drag/drop the zip file into the github issue.
+19
View File
@@ -0,0 +1,19 @@
os: linux
dist: bionic # focal
language: python
before_install:
- sudo apt-get update --fix-missing
install:
- sudo apt-get install -y ffmpeg libopus-dev python-scipy python3-scipy
python:
- "2.7"
- "3.6"
- "3.7"
- "3.8"
- "3.9"
- "pypy2"
- "pypy3"
script:
- python test/test.py
after_script:
- pip install pylama && python -m pylama -i W,E501 pydub/ || true
+693
View File
@@ -0,0 +1,693 @@
# API Documentation
This document is a work in progress.
If you're looking for some functionality in particular, it's a good idea to take a look at the [source code](https://github.com/jiaaro/pydub). Core functionality is mostly in `pydub/audio_segment.py` a number of `AudioSegment` methods are in the `pydub/effects.py` module, and added to `AudioSegment` via the effect registration process (the `register_pydub_effect()` decorator function)
Currently Undocumented:
- Playback (`pydub.playback`)
- Signal Processing (compression, EQ, normalize, speed change - `pydub.effects`, `pydub.scipy_effects`)
- Signal generators (Sine, Square, Sawtooth, Whitenoise, etc - `pydub.generators`)
- Effect registration system (basically the `pydub.utils.register_pydub_effect` decorator)
## AudioSegment()
`AudioSegment` objects are immutable, and support a number of operators.
```python
from pydub import AudioSegment
sound1 = AudioSegment.from_file("/path/to/sound.wav", format="wav")
sound2 = AudioSegment.from_file("/path/to/another_sound.wav", format="wav")
# sound1 6 dB louder, then 3.5 dB quieter
louder = sound1 + 6
quieter = sound1 - 3.5
# sound1, with sound2 appended
combined = sound1 + sound2
# sound1 repeated 3 times
repeated = sound1 * 3
# duration
duration_in_milliseconds = len(sound1)
# first 5 seconds of sound1
beginning = sound1[:5000]
# last 5 seconds of sound1
end = sound1[-5000:]
# split sound1 in 5-second slices
slices = sound1[::5000]
# Advanced usage, if you have raw audio data:
sound = AudioSegment(
# raw audio data (bytes)
data=b'',
# 2 byte (16 bit) samples
sample_width=2,
# 44.1 kHz frame rate
frame_rate=44100,
# stereo
channels=2
)
```
Any operations that combine multiple `AudioSegment` objects in *any* way will first ensure that they have the same number of channels, frame rate, sample rate, bit depth, etc. When these things do not match, the lower quality sound is modified to match the quality of the higher quality sound so that quality is not lost: mono is converted to stereo, bit depth and frame rate/sample rate are increased as needed. If you do not want this behavior, you may explicitly reduce the number of channels, bits, etc using the appropriate `AudioSegment` methods.
### AudioSegment(…).from_file()
Open an audio file as an `AudioSegment` instance and return it. there are also a number of wrappers provided for convenience, but you should probably just use this directly.
```python
from pydub import AudioSegment
# wave and raw dont use ffmpeg
wav_audio = AudioSegment.from_file("/path/to/sound.wav", format="wav")
raw_audio = AudioSegment.from_file("/path/to/sound.raw", format="raw",
frame_rate=44100, channels=2, sample_width=2)
# all other formats use ffmpeg
mp3_audio = AudioSegment.from_file("/path/to/sound.mp3", format="mp3")
# use a file you've already opened (advanced …ish)
with open("/path/to/sound.wav", "rb") as wav_file:
audio_segment = AudioSegment.from_file(wav_file, format="wav")
# also supports the os.PathLike protocol for python >= 3.6
from pathlib import Path
wav_path = Path("path/to/sound.wav")
wav_audio = AudioSegment.from_file(wav_path)
```
The first argument is the path (as a string) of the file to read, **or** a file handle to read from.
**Supported keyword arguments**:
- `format` | example: `"aif"` | default: autodetected
Format of the output file. Supports `"wav"` and `"raw"` natively, requires ffmpeg for all other formats. `"raw"` files require 3 additional keyword arguments, `sample_width`, `frame_rate`, and `channels`, denoted below with: **`raw` only**. This extra info is required because raw audio files do not have headers to include this info in the file itself like wav files do.
- `sample_width` | example: `2`
**`raw` only** — Use `1` for 8-bit audio `2` for 16-bit (CD quality) and `4` for 32-bit. Its the number of bytes per sample.
- `channels` | example: `1`
**`raw` only** — `1` for mono, `2` for stereo.
- `frame_rate` | example: `2`
**`raw` only** — Also known as sample rate, common values are `44100` (44.1kHz - CD audio), and `48000` (48kHz - DVD audio)
- `start_second` | example: `2.0` | default: `None`
Offset (in seconds) to start loading the audio file. If `None`, the audio will start loading from the beginning.
- `duration` | example: `2.5` | default: `None`
Number of seconds to be loaded. If `None`, full audio will be loaded.
### AudioSegment(…).export()
Write the `AudioSegment` object to a file returns a file handle of the output file (you don't have to do anything with it, though).
```python
from pydub import AudioSegment
sound = AudioSegment.from_file("/path/to/sound.wav", format="wav")
# simple export
file_handle = sound.export("/path/to/output.mp3", format="mp3")
# more complex export
file_handle = sound.export("/path/to/output.mp3",
format="mp3",
bitrate="192k",
tags={"album": "The Bends", "artist": "Radiohead"},
cover="/path/to/albumcovers/radioheadthebends.jpg")
# split sound in 5-second slices and export
for i, chunk in enumerate(sound[::5000]):
with open("sound-%s.mp3" % i, "wb") as f:
chunk.export(f, format="mp3")
```
The first argument is the location (as a string) to write the output, **or** a file handle to write to. If you do not pass an output file or path, a temporary file is generated.
**Supported keyword arguments**:
- `format` | example: `"aif"` | default: `"mp3"`
Format of the output file. Supports `"wav"` and `"raw"` natively, requires ffmpeg for all other formats.
- `codec` | example: `"libvorbis"`
For formats that may contain content encoded with different codecs, you can specify the codec you'd like the encoder to use. For example, the "ogg" format is often used with the "libvorbis" codec. (requires ffmpeg)
- `bitrate` | example: `"128k"`
For compressed formats, you can pass the bitrate you'd like the encoder to use (requires ffmpeg). Each codec accepts different bitrate arguments so take a look at the [ffmpeg documentation](https://www.ffmpeg.org/ffmpeg-codecs.html#Audio-Encoders) for details (bitrate usually shown as `-b`, `-ba` or `-a:b`).
- `tags` | example: `{"album": "1989", "artist": "Taylor Swift"}`
Allows you to supply media info tags for the encoder (requires ffmpeg). Not all formats can receive tags (mp3 can).
- `parameters` | example: `["-ac", "2"]`
Pass additional [command line parameters](https://www.ffmpeg.org/ffmpeg.html) to the ffmpeg call. These are added to the end of the call (in the output file section).
- `id3v2_version` | example: `"3"` | default: `"4"`
Set the ID3v2 version used by ffmpeg to add tags to the output file. If you want Windows Exlorer to display tags, use `"3"` here ([source](http://superuser.com/a/453133)).
- `cover` | example: `"/path/to/imgfile.png"`
Allows you to supply a cover image (path to the image file). Currently, only MP3 files allow this keyword argument. Cover image must be a jpeg, png, bmp, or tiff file.
### AudioSegment.empty()
Creates a zero-duration `AudioSegment`.
```python
from pydub import AudioSegment
empty = AudioSegment.empty()
len(empty) == 0
```
This is useful for aggregation loops:
```python
from pydub import AudioSegment
sounds = [
AudioSegment.from_wav("sound1.wav"),
AudioSegment.from_wav("sound2.wav"),
AudioSegment.from_wav("sound3.wav"),
]
playlist = AudioSegment.empty()
for sound in sounds:
playlist += sound
```
### AudioSegment.silent()
Creates a silent audiosegment, which can be used as a placeholder, spacer, or as a canvas to overlay other sounds on top of.
```python
from pydub import AudioSegment
ten_second_silence = AudioSegment.silent(duration=10000)
```
**Supported keyword arguments**:
- `duration` | example: `3000` | default: `1000` (1 second)
Length of the silent `AudioSegment`, in milliseconds
- `frame_rate` | example `44100` | default: `11025` (11.025 kHz)
Frame rate (i.e., sample rate) of the silent `AudioSegment` in Hz
### AudioSegment.from_mono_audiosegments()
Creates a multi-channel audiosegment out of multiple mono audiosegments (two or more). Each mono audiosegment passed in should be exactly the same length, down to the frame count.
```python
from pydub import AudioSegment
left_channel = AudioSegment.from_wav("sound1.wav")
right_channel = AudioSegment.from_wav("sound1.wav")
stereo_sound = AudioSegment.from_mono_audiosegments(left_channel, right_channel)
```
### AudioSegment(…).dBFS
Returns the loudness of the `AudioSegment` in dBFS (db relative to the maximum possible loudness). A Square wave at maximum amplitude will be roughly 0 dBFS (maximum loudness), whereas a Sine Wave at maximum amplitude will be roughly -3 dBFS.
```python
from pydub import AudioSegment
sound = AudioSegment.from_file("sound1.wav")
loudness = sound.dBFS
```
### AudioSegment(…).channels
Number of channels in this audio segment (1 means mono, 2 means stereo)
```python
from pydub import AudioSegment
sound = AudioSegment.from_file("sound1.wav")
channel_count = sound.channels
```
### AudioSegment(…).sample_width
Number of bytes in each sample (1 means 8 bit, 2 means 16 bit, etc). CD Audio is 16 bit, (sample width of 2 bytes).
```python
from pydub import AudioSegment
sound = AudioSegment.from_file("sound1.wav")
bytes_per_sample = sound.sample_width
```
### AudioSegment(…).frame_rate
CD Audio has a 44.1kHz sample rate, which means `frame_rate` will be `44100` (same as sample rate, see `frame_width`). Common values are `44100` (CD), `48000` (DVD), `22050`, `24000`, `12000` and `11025`.
```python
from pydub import AudioSegment
sound = AudioSegment.from_file("sound1.wav")
frames_per_second = sound.frame_rate
```
### AudioSegment(…).frame_width
Number of bytes for each "frame". A frame contains a sample for each channel (so for stereo you have 2 samples per frame, which are played simultaneously). `frame_width` is equal to `channels * sample_width`. For CD Audio it'll be `4` (2 channels times 2 bytes per sample).
```python
from pydub import AudioSegment
sound = AudioSegment.from_file("sound1.wav")
bytes_per_frame = sound.frame_width
```
### AudioSegment(…).rms
A measure of loudness. Used to compute dBFS, which is what you should use in most cases. Loudness is logarithmic (rms is not), which makes dB a much more natural scale.
```python
from pydub import AudioSegment
sound = AudioSegment.from_file("sound1.wav")
loudness = sound.rms
```
### AudioSegment(…).max
The highest amplitude of any sample in the `AudioSegment`. Useful for things like normalization (which is provided in `pydub.effects.normalize`).
```python
from pydub import AudioSegment
sound = AudioSegment.from_file("sound1.wav")
peak_amplitude = sound.max
```
### AudioSegment(…).max_dBFS
The highest amplitude of any sample in the `AudioSegment`, in dBFS (relative to the highest possible amplitude value). Useful for things like normalization (which is provided in `pydub.effects.normalize`).
```python
from pydub import AudioSegment
sound = AudioSegment.from_file("sound1.wav")
normalized_sound = sound.apply_gain(-sound.max_dBFS)
```
### AudioSegment(…).duration_seconds
Returns the duration of the `AudioSegment` in seconds (`len(sound)` returns milliseconds). This is provided for convenience; it calls `len()` internally.
```python
from pydub import AudioSegment
sound = AudioSegment.from_file("sound1.wav")
assert sound.duration_seconds == (len(sound) / 1000.0)
```
### AudioSegment(…).raw_data
The raw audio data of the AudioSegment. Useful for interacting with other audio libraries or weird APIs that want audio data in the form of a bytestring. Also comes in handy if youre implementing effects or other direct signal processing.
You probably dont need this, but if you do… youll know.
```python
from pydub import AudioSegment
sound = AudioSegment.from_file("sound1.wav")
raw_audio_data = sound.raw_data
```
### AudioSegment(…).frame_count()
Returns the number of frames in the `AudioSegment`. Optionally you may pass in a `ms` keywork argument to retrieve the number of frames in that number of milliseconds of audio in the `AudioSegment` (useful for slicing, etc).
```python
from pydub import AudioSegment
sound = AudioSegment.from_file("sound1.wav")
number_of_frames_in_sound = sound.frame_count()
number_of_frames_in_200ms_of_sound = sound.frame_count(ms=200)
```
**Supported keyword arguments**:
- `ms` | example: `3000` | default: `None` (entire duration of `AudioSegment`)
When specified, method returns number of frames in X milliseconds of the `AudioSegment`
### AudioSegment(…).append()
Returns a new `AudioSegment`, created by appending another `AudioSegment` to this one (i.e., adding it to the end), Optionally using a crossfade. `AudioSegment(…).append()` is used internally when adding `AudioSegment` objects together with the `+` operator.
By default a 100ms (0.1 second) crossfade is used to eliminate pops and crackles.
```python
from pydub import AudioSegment
sound1 = AudioSegment.from_file("sound1.wav")
sound2 = AudioSegment.from_file("sound2.wav")
# default 100 ms crossfade
combined = sound1.append(sound2)
# 5000 ms crossfade
combined_with_5_sec_crossfade = sound1.append(sound2, crossfade=5000)
# no crossfade
no_crossfade1 = sound1.append(sound2, crossfade=0)
# no crossfade
no_crossfade2 = sound1 + sound2
```
**Supported keyword arguments**:
- `crossfade` | example: `3000` | default: `100` (entire duration of `AudioSegment`)
When specified, method returns number of frames in X milliseconds of the `AudioSegment`
### AudioSegment(…).overlay()
Overlays an `AudioSegment` onto this one. In the resulting `AudioSegment` they will play simultaneously. If the overlaid `AudioSegment` is longer than this one, the result will be truncated (so the end of the overlaid sound will be cut off). The result is always the same length as this `AudioSegment` even when using the `loop`, and `times` keyword arguments.
Since `AudioSegment` objects are immutable, you can get around this by overlaying the shorter sound on the longer one, or by creating a silent `AudioSegment` with the appropriate duration, and overlaying both sounds on to that one.
```python
from pydub import AudioSegment
sound1 = AudioSegment.from_file("sound1.wav")
sound2 = AudioSegment.from_file("sound2.wav")
played_togther = sound1.overlay(sound2)
sound2_starts_after_delay = sound1.overlay(sound2, position=5000)
volume_of_sound1_reduced_during_overlay = sound1.overlay(sound2, gain_during_overlay=-8)
sound2_repeats_until_sound1_ends = sound1.overlay(sound2, loop=true)
sound2_plays_twice = sound1.overlay(sound2, times=2)
# assume sound1 is 30 sec long and sound2 is 5 sec long:
sound2_plays_a_lot = sound1.overlay(sound2, times=10000)
len(sound1) == len(sound2_plays_a_lot)
```
**Supported keyword arguments**:
- `position` | example: `3000` | default: `0` (beginning of this `AudioSegment`)
The overlaid `AudioSegment` will not begin until X milliseconds have passed
- `loop` | example: `True` | default: `False` (entire duration of `AudioSegment`)
The overlaid `AudioSegment` will repeat (starting at `position`) until the end of this `AudioSegment`
- `times` | example: `4` | default: `1` (entire duration of `AudioSegment`)
The overlaid `AudioSegment` will repeat X times (starting at `position`) but will still be truncated to the length of this `AudioSegment`
- `gain_during_overlay` | example: `-6.0` | default: `0` (no change in volume during overlay)
Change the original audio by this many dB while overlaying audio. This can be used to make the original audio quieter while the overlaid audio plays.
### AudioSegment(…).apply_gain(`gain`)
Change the amplitude (generally, loudness) of the `AudioSegment`. Gain is specified in dB. This method is used internally by the `+` operator.
```python
from pydub import AudioSegment
sound1 = AudioSegment.from_file("sound1.wav")
# make sound1 louder by 3.5 dB
louder_via_method = sound1.apply_gain(+3.5)
louder_via_operator = sound1 + 3.5
# make sound1 quieter by 5.7 dB
quieter_via_method = sound1.apply_gain(-5.7)
quieter_via_operator = sound1 - 5.7
```
### AudioSegment(…).fade()
A more general (more flexible) fade method. You may specify `start` and `end`, or one of the two along with duration (e.g., `start` and `duration`).
```python
from pydub import AudioSegment
sound1 = AudioSegment.from_file("sound1.wav")
fade_louder_for_3_seconds_in_middle = sound1.fade(to_gain=+6.0, start=7500, duration=3000)
fade_quieter_beteen_2_and_3_seconds = sound1.fade(to_gain=-3.5, start=2000, end=3000)
# easy way is to use the .fade_in() convenience method. note: -120dB is basically silent.
fade_in_the_hard_way = sound1.fade(from_gain=-120.0, start=0, duration=5000)
fade_out_the_hard_way = sound1.fade(to_gain=-120.0, end=0, duration=5000)
```
**Supported keyword arguments**:
- `to_gain` | example: `-3.0` | default: `0` (0dB, no change)
Resulting change at the end of the fade. `-6.0` means fade will be be from 0dB (no change) to -6dB, and everything after the fade will be -6dB.
- `from_gain` | example: `-3.0` | default: `0` (0dB, no change)
Change at the beginning of the fade. `-6.0` means fade (and all audio before it) will be be at -6dB will fade up to 0dB the rest of the audio after the fade will be at 0dB (i.e., unchanged).
- `start` | example: `7500` | NO DEFAULT
Position to begin fading (in milliseconds). `5500` means fade will begin after 5.5 seconds.
- `end` | example: `4` | NO DEFAULT
The overlaid `AudioSegment` will repeat X times (starting at `position`) but will still be truncated to the length of this `AudioSegment`
- `duration` | example: `4` | NO DEFAULT
You can use `start` or `end` with duration, instead of specifying both - provided as a convenience.
### AudioSegment(…).fade_out()
Fade out (to silent) the end of this `AudioSegment`. Uses `.fade()` internally.
**Supported keyword arguments**:
- `duration` | example: `5000` | NO DEFAULT
How long (in milliseconds) the fade should last. Passed directly to `.fade()` internally
### AudioSegment(…).fade_in()
Fade in (from silent) the beginning of this `AudioSegment`. Uses `.fade()` internally.
**Supported keyword arguments**:
- `duration` | example: `5000` | NO DEFAULT
How long (in milliseconds) the fade should last. Passed directly to `.fade()` internally
### AudioSegment(…).reverse()
Make a copy of this `AudioSegment` that plays backwards. Useful for Pink Floyd, screwing around, and some audio processing algorithms.
### AudioSegment(…).set_sample_width()
Creates an equivalent version of this `AudioSegment` with the specified sample width (in bytes). Increasing this value does not generally cause a reduction in quality. Reducing it *definitely* does cause a loss in quality. Higher Sample width means more dynamic range.
### AudioSegment(…).set_frame_rate()
Creates an equivalent version of this `AudioSegment` with the specified frame rate (in Hz). Increasing this value does not generally cause a reduction in quality. Reducing it *definitely does* cause a loss in quality. Higher frame rate means larger frequency response (higher frequencies can be represented).
### AudioSegment(…).set_channels()
Creates an equivalent version of this `AudioSegment` with the specified number of channels (1 is Mono, 2 is Stereo). Converting from mono to stereo does not cause any audible change. Converting from stereo to mono may result in loss of quality (but only if the left and right chanels differ).
### AudioSegment(…).split_to_mono()
Splits a stereo `AudioSegment` into two, one for each channel (Left/Right). Returns a list with the new `AudioSegment` objects with the left channel at index 0 and the right channel at index 1.
### AudioSegment(…).apply_gain_stereo()
```python
from pydub import AudioSegment
sound1 = AudioSegment.from_file("sound1.wav")
# make left channel 6dB quieter and right channe 2dB louder
stereo_balance_adjusted = sound1.apply_gain_stereo(-6, +2)
```
Apply gain to the left and right channel of a stereo `AudioSegment`. If the `AudioSegment` is mono, it will be converted to stereo before applying the gain.
Both gain arguments are specified in dB.
### AudioSegment(…).pan()
```python
from pydub import AudioSegment
sound1 = AudioSegment.from_file("sound1.wav")
# pan the sound 15% to the right
panned_right = sound1.pan(+0.15)
# pan the sound 50% to the left
panned_left = sound1.pan(-0.50)
```
Takes one positional argument, *pan amount*, which should be between -1.0 (100% left) and +1.0 (100% right)
When pan_amount == 0.0 the left/right balance is not changed.
Panning does not alter the *perceived* loundness, but since loudness
is decreasing on one side, the other side needs to get louder to
compensate. When panned hard left, the left channel will be 3dB louder and
the right channel will be silent (and vice versa).
### AudioSegment(…).get_array_of_samples()
Returns the raw audio data as an array of (numeric) samples. Note: if the audio has multiple channels, the samples for each channel will be serialized  for example, stereo audio would look like `[sample_1_L, sample_1_R, sample_2_L, sample_2_R, …]`.
This method is mainly for use in implementing effects, and other processing.
```python
from pydub import AudioSegment
sound = AudioSegment.from_file(sound1.wav)
samples = sound.get_array_of_samples()
# then modify samples...
new_sound = sound._spawn(samples)
```
note that when using numpy or scipy you will need to convert back to an array before you spawn:
```python
import array
import numpy as np
from pydub import AudioSegment
sound = AudioSegment.from_file(sound1.wav)
samples = sound.get_array_of_samples()
# Example operation on audio data
shifted_samples = np.right_shift(samples, 1)
# now you have to convert back to an array.array
shifted_samples_array = array.array(sound.array_type, shifted_samples)
new_sound = sound._spawn(shifted_samples_array)
```
Here's how to convert to a numpy float32 array:
```python
import numpy as np
from pydub import AudioSegment
sound = AudioSegment.from_file("sound1.wav")
sound = sound.set_frame_rate(16000)
channel_sounds = sound.split_to_mono()
samples = [s.get_array_of_samples() for s in channel_sounds]
fp_arr = np.array(samples).T.astype(np.float32)
fp_arr /= np.iinfo(samples[0].typecode).max
```
And how to convert it back to an AudioSegment:
```python
import io
import scipy.io.wavfile
wav_io = io.BytesIO()
scipy.io.wavfile.write(wav_io, 16000, fp_arr)
wav_io.seek(0)
sound = pydub.AudioSegment.from_wav(wav_io)
```
### AudioSegment(…).get_dc_offset()
Returns a value between -1.0 and 1.0 representing the DC offset of a channel. This is calculated using `audioop.avg()` and normalizing the result by samples max value.
**Supported keyword arguments**:
- `channel` | example: `2` | default: `1`
Selects left (1) or right (2) channel to calculate DC offset. If segment is mono, this value is ignored.
### AudioSegment(…).remove_dc_offset()
Removes DC offset from channel(s). This is done by using `audioop.bias()`, so watch out for overflows.
**Supported keyword arguments**:
- `channel` | example: `2` | default: None
Selects left (1) or right (2) channel remove DC offset. If value if None, removes from all available channels. If segment is mono, this value is ignored.
- `offset` | example: `-0.1` | default: None
Offset to be removed from channel(s). Calculates offset if it's None. Offset values must be between -1.0 and 1.0.
## Effects
Collection of DSP effects that are implemented by `AudioSegment` objects.
### AudioSegment(…).invert_phase()
Make a copy of this `AudioSegment` and inverts the phase of the signal. Can generate anti-phase waves for noise suppression or cancellation.
## Silence
Various functions for finding/manipulating silence in AudioSegments. For creating silent AudioSegments, see AudioSegment.silent().
### silence.detect_silence()
Returns a list of all silent sections [start, end] in milliseconds of audio_segment. Inverse of detect_nonsilent(). Can be very slow since it has to iterate over the whole segment.
```python
from pydub import AudioSegment, silence
print(silence.detect_silence(AudioSegment.silent(2000)))
# [[0, 2000]]
```
**Supported keyword arguments**:
- `min_silence_len` | example: `500` | default: 1000
The minimum length for silent sections in milliseconds. If it is greater than the length of the audio segment an empty list will be returned.
- `silence_thresh` | example: `-20` | default: -16
The upper bound for how quiet is silent in dBFS.
- `seek_step` | example: `5` | default: 1
Size of the step for checking for silence in milliseconds. Smaller is more precise. Must be a positive whole number.
### silence.detect_nonsilent()
Returns a list of all silent sections [start, end] in milliseconds of audio_segment. Inverse of detect_silence() and has all the same arguments. Can be very slow since it has to iterate over the whole segment.
**Supported keyword arguments**:
- `min_silence_len` | example: `500` | default: 1000
The minimum length for silent sections in milliseconds. If it is greater than the length of the audio segment an empty list will be returned.
- `silence_thresh` | example: `-20` | default: -16
The upper bound for how quiet is silent in dBFS.
- `seek_step` | example: `5` | default: 1
Size of the step for checking for silence in milliseconds. Smaller is more precise. Must be a positive whole number.
### silence.split_on_silence()
Returns list of audio segments from splitting audio_segment on silent sections.
**Supported keyword arguments**:
- `min_silence_len` | example: `500` | default: 1000
The minimum length for silent sections in milliseconds. If it is greater than the length of the audio segment an empty list will be returned.
- `silence_thresh` | example: `-20` | default: -16
The upper bound for how quiet is silent in dBFS.
- `seek_step` | example: `5` | default: 1
Size of the step for checking for silence in milliseconds. Smaller is more precise. Must be a positive whole number.
- `keep_silence` ~ example: True | default: 100
How much silence to keep in ms or a bool. leave some silence at the beginning and end of the chunks. Keeps the sound from sounding like it is abruptly cut off.
When the length of the silence is less than the keep_silence duration it is split evenly between the preceding and following non-silent segments.
If True is specified, all the silence is kept, if False none is kept.
### silence.detect_leading_silence()
Returns the millisecond/index that the leading silence ends. If there is no end it will return the length of the audio_segment.
```python
from pydub import AudioSegment, silence
print(silence.detect_silence(AudioSegment.silent(2000)))
# 2000
```
**Supported keyword arguments**:
- `silence_thresh` | example: `-20` | default: -50
The upper bound for how quiet is silent in dBFS.
- `chunk_size` | example: `5` | default: 10
Size of the step for checking for silence in milliseconds. Smaller is more precise. Must be a positive whole number.
+101
View File
@@ -0,0 +1,101 @@
James Robert
github: jiaaro
twitter: @jiaaro
web: jiaaro.com
email: pydub@jiaaro.com
Marc Webbie
github: marcwebbie
Jean-philippe Serafin
github: jeanphix
Anurag Ramdasan
github: AnuragRamdasan
Choongmin Lee
github: clee704
Patrick Pittman
github: ptpittman
Hunter Lang
github: hunterlang
Alexey
github: nihisil
Jaymz Campbell
github: jaymzcd
Ross McFarland
github: ross
John McMellen
github: jmcmellen
Johan Lövgren
github: dashj
Joachim Krüger
github: jkrgr
Shichao An
github: shichao-an
Michael Bortnyck
github: mbortnyck
André Cloete
github: aj-cloete
David Acacio
github: dacacioa
Thiago Abdnur
github: bolaum
Aurélien Ooms
github: aureooms
Mike Mattozzi
github: mmattozzi
Marcio Mazza
github: marciomazza
Sungsu Lim
github: proflim
Evandro Myller
github: emyller
Sérgio Agostinho
github: SergioRAgostinho
Antonio Larrosa
github: antlarr
Aaron Craig
github: craigthelinguist
Carlos del Castillo
github: greyalien502
Yudong Sun
github: sunjerry019
Jorge Perianez
github: JPery
Chendi Luo
github: Creonalia
Daniel Lefevre
gitHub: dplefevre
Grzegorz Kotfis
github: gkotfis
Pål Orby
github: orby
+168
View File
@@ -0,0 +1,168 @@
# v0.25.1
- Fix crashing bug in new scipy-powered EQ effects
# v0.25.0
- Don't show a runtime warning about the optional ffplay dependency being missing until someone trys to use it
- Documentation improvements
- Python 3.9 support
- Improved efficiency of loading wave files with `pydub.AudioSegment.from_file()`
- Ensure `pydub.AudioSegment().export()` always retuns files with a seek position at the beginning of the file
- Added more EQ effects to `pydub.scipy_effects` (requires scipy to be installed)
- Fix a packaging bug where the LICENSE file was not included in the source distribution
- Add a way to instantiate a `pydub.AudioSegment()` with a portion of an audio file via `pydub.AudioSegment().from_file()`
# v0.24.1
- Fix bug where ffmpeg errors in Python 3 are illegible
- Fix bug where `split_on_silence` fails when there are one or fewer nonsilent segments
- Fix bug in fallback audioop implementation
# v0.24.0
- Fix inconsistent handling of 8-bit audio
- Fix bug where certain files will fail to parse
- Fix bug where pyaudio stream is not closed on error
- Allow codecs and parameters in wav and raw export
- Fix bug in `pydub.AudioSegment.from_file` where supplied codec is ignored
- Allow `pydub.silence.split_on_silence` to take a boolean for `keep_silence`
- Fix bug where `pydub.silence.split_on_silence` sometimes adds non-silence from adjacent segments
- Fix bug where `pydub.AudioSegment.extract_wav_headers` fails on empty wav files
- Add new function `pydub.silence.detect_leading_silence`
- Support conversion between an arbitrary number of channels and mono in `pydub.AudioSegment.set_channels`
- Fix several issues related to reading from filelike objects
# v0.23.1
- Fix bug in passing ffmpeg/avconv parameters for `pydub.AudioSegment.from_mp3()`, `pydub.AudioSegment.from_flv()`, `pydub.AudioSegment.from_ogg()`, and `pydub.AudioSegment.from_wav()`
- Fix logic bug in `pydub.effects.strip_silence()`
# v0.23.0
- Add support for playback via simpleaudio
- Allow users to override the type in `pydub.AudioSegment().get_array_of_samples()` (PR #313)
- Fix a bug where the wrong codec was used for 8-bit audio (PR #309 - issue #308)
# v0.22.1
- Fix `pydub.utils.mediainfo_json()` to work with newer, backwards-incompatible versions of ffprobe/avprobe
# v0.22.0
- Adds support for audio with frame rates (sample rates) of 48k and higher (requires scipy) (PR #262, fixes #134, #237, #209)
- Adds support for PEP 519 File Path protocol (PR #252)
- Fixes a few places where handles to temporary files are kept open (PR #280)
- Add the license file to the python package to aid other packaging projects (PR #279, fixes #274)
- Big fix for `pydub.silence.detect_silence()` (PR #263)
# v0.21.0
- NOTE: Semi-counterintuitive change: using the a stride when slicing AudioSegment instances (for example, `sound[::5000]`) will return chunks of 5000ms (not 1ms chunks every 5000ms) (#222)
- Debug output from ffmpeg/avlib is no longer printed to the console unless you set up logging (see README for how to set up logging for your converter) (#223)
- All pydub exceptions are now subclasses of `pydub.exceptions.PydubException` (PR #244)
- The utilities in `pydub.silence` now accept a `seek_step`argument which can optionally be passed to improve the performance of silence detection (#211)
- Fix to `pydub.silence` utilities which allow you to detect perfect silence (#233)
- Fix a bug where threaded code screws up your terminal session due to ffmpeg inheriting the stdin from the parent process. (#231)
- Fix a bug where a crashing programs using pydub would leave behind their temporary files (#206)
# v0.20.0
- Add new parameter `gain_during_overlay` to `pydub.AudioSegment.overlay` which allows users to adjust the volume of the target AudioSegment during the portion of the segment which is overlaid with the additional AudioSegment.
- `pydub.playback.play()` No longer displays the (very verbose) playback "banner" when using ffplay
- Fix a confusing error message when using invalid crossfade durations (issue #193)
# v0.19.0
- Allow codec and ffmpeg/avconv parameters to be set in the `pydub.AudioSegment.from_file()` for more control while decoding audio files
- Allow `AudioSegment` objects with more than two channels to be split using `pydub.AudioSegment().split_to_mono()`
- Add support for inverting the phase of only one channel in a multi-channel `pydub.AudioSegment` object
- Fix a bug with the latest avprobe that broke `pydub.utils.mediainfo()`
- Add tests for webm encoding/decoding
# v0.18.0
- Add a new constructor: `pydub.AudioSegment.from_mono_audiosegments()` which allows users to create a multi-channel audiosegment out of multiple mono ones.
- Refactor `pydub.AudioSegment._sync()` to support an arbitrary number of audiosegment arguments.
# v0.17.0
- Add the ability to add a cover image to MP3 exports via the `cover` keyword argument to `pydub.AudioSegment().export()`
- Add `pydub.AudioSegment().get_dc_offset()` and `pydub.AudioSegment().remove_dc_offset()` which allow detection and removal of DC offset in audio files.
- Minor fixes for windows users
# v0.16.7
- Make `pydub.AudioSegment()._spawn()` accept array.array instances containing audio samples
# v0.16.6
- Make `pydub.AudioSegment()` objects playable inline in ipython notebooks.
- Add scipy powered high pass, low pass, and band pass filters, which can be high order filters (they take `order` as a keyword argument). They are used for `pydub.AudioSegment().high_pass_filter()`, `pydub.AudioSegment().low_pass_filter()`, `pydub.AudioSegment().band_pass_filter()` when the `pydub.scipy_effects` module is imported.
- Fix minor bug in `pydub.silence.detect_silence()`
# v0.16.5
- Update `pydub.AudioSegment()._spawn()` method to allow user subclassing of `pydub.AudioSegment`
- Add a workaround for incorrect duration reporting of some mp3 files on macOS
# v0.16.4
- Add support for radd (basically, allow `sum()` to operate on an iterable of `pydub.AudioSegment()` objects)
- Fix bug in 24-bit wav support (understatement. It didn't work right at all the first time)
# v0.16.3
- Add support for python 3.5 (overstatement. We just added python 3.5 to CI and it worked 😄)
- Add native support for 24-bit wav files (ffmpeg/avconv not required)
# v0.16.2
- Fix bug where you couldn't directly instantiate `pydub.AudioSegment` with `bytes` data in python 3
# v0.16.1
- pydub will use any ffmpeg/avconv binary that's in the current directory (as reported by `os.getcwd()`) before searching for a system install
# v0.16.0
- Make it easier to instantiate `pydub.AudioSegment()` directly when creating audio segments from raw audio data (without having to write it to a file first)
- Add `pydub.AudioSegment().get_array_of_samples()` method which returns the samples which make up an audio segment (you should usually prefer this over `pydub.AudioSegment().raw_data`)
- Add `pydub.AudioSegment().raw_data` property which returns the raw audio data for an audio segment as a bytes (python 3) or a bytestring (python 3)
- Allow users to specify frame rate in `pydub.AudioSegment.silent()` constructor
# v0.15.0
- Add support for RAW audio (basically WAV format, but without wave headers)
- Add a new exception `pydub.exceptions.CouldntDecodeError` to indicate a failure of ffmpeg/avconv to decode a file (as indicated by ffmpeg/avconv exit code)
# v0.14.2
- Fix a bug in python 3.4 which failed to read wave files with no audio data (should have been audio segments with a duration of 0 ms)
# v0.14.1
- Fix a bug in `pydub.utils.mediainfo()` that caused inputs containing unescaped characters to raise a runtime error (inputs are not supposed to require escaping)
# v0.14.0
- Rename `pydub.AudioSegment().set_gain()` to `pydub.AudioSegment().apply_gain_stereo()` to better reflect it's place in the world (as a counterpart to `pydub.AudioSegment().apply_gain()`)
# v0.13.0
- Add `pydub.AudioSegment().pan()` which returns a new stereo audio segment panned left/right as specified.
# v0.12.0
- Add a logger, `"pydub.converter"` which logs the ffmpeg commands being run by pydub.
- Add `pydub.AudioSegment().split_to_mono()` method which returns a list of mono audio segments. One for each channel in the original audio segment.
- Fix a bug in `pydub.silence.detect_silence()` which caused the function to break when a silent audio segment was equal in length to the minimum silence length. It should report a single span of silence covering the whole silent audio segment. Now it does.
- Fix a bug where uncommon wav formats (those not supported by the stdlib wave module) would throw an exception rather than converting to a more common format via ffmpeg/avconv
# v0.11.0
- Add `pydub.AudioSegment().max_dBFS` which reports the loudness (in dBFS) of the loudest point (i.e., highest amplitude sample) of an audio segment
# v0.10.0
- Overhaul Documentation
- Improve performance of `pydub.AudioSegment().overlay()`
- Add `pydub.AudioSegment().invert_phase()` which (shocker) inverts the phase of an audio segment
- Fix a type error in `pydub.AudioSegment.get_sample_slice()`
# v0.9.5
- Add `pydub.generators` module containing simple signal generation functions (white noise, sine, square wave, etc)
- Add a `loops` keyword argument to `pydub.AudioSegment().overlay()` which allows users to specify that the overlaid audio should be repeated (i.e., looped) a certain number of times, or indefinitely
# 0.9.4
- Fix a bug in db_to_float() where all values were off by a factor of 2
# 0.9.3
- Allow users to set the location of their converter by setting `pydub.AudioSegment.converter = "/path/to/ffmpeg"` and added a shim to support the old method of assigning to `pydub.AudioSegment.ffmpeg` (which is deprecated now that we support avconv)
# v0.9.2
- Add support for Python 3.4
- Audio files opened with format "wave" are treated as "wav" and "m4a" are treated as "mp4"
- Add `pydub.silence` module with simple utilities for detecting and removing silence.
- Fix a bug affecting auto-detection of ffmpeg/avconv on windows.
- Fix a bug that caused pydub to only work when ffmpeg/avconv is present (it should be able to work with WAV data without any dependencies)
# v0.9.1
- Add a runtime warning when ffmpeg/avconv cannot be found to aid debugging
# v0.9.0
- Added support for pypy (by reimplementing audioop in python). Also, we contributed our implementation to the pypy project, so that's 💯
- Add support for avconv as an alternative to ffmpeg
- Add a new helper module `pydub.playback` which allows you to quickly listen to an audio segment using ffplay (or avplay)
- Add new function `pydub.utils.mediainfo('/path/to/audio/file.ext')` which reports back the results of ffprobe (or avprobe) including codec, bitrate, channels, etc
+45
View File
@@ -0,0 +1,45 @@
Pydub loves user contributions.
We are happy to merge Pull Requests for features and bug fixes, of course. But, also spelling corrections, PEP 8 conformity, and platform-specific fixes.
Don't be shy!
### How to contribute:
1. Fork [pydub on github](https://github.com/jiaaro/pydub)
2. Commit changes
3. Send a Pull Request
you did it!
don't forget to append your name to the AUTHORS file ;)
There _are_ a few things that will make your Pull Request more likely to be merged:
1. Maintain backward compatibility
2. Avoid new dependencies
3. Include tests (and make sure they pass)
4. Write a short description of **what** is changed and **why**
5. Keep your Pull Request small, and focused on fixing one thing.
Smaller is easier to review, and easier to understand.
If you want to fix spelling and PEP 8 violations, send two pull requests :)
### Want to pitch in?
Take a look at our issue tracker for anything tagged [`bug`][bugs] or [`todo`][todos] - these are goals of the project and your improvements are _very_ likely to be merged!
That being said, there are many possible contributions we haven't thought of already. Those are welcome too!
Here are some general topics of interest for future development:
- Make it easier to get started with pydub
- More/better audio effects
- Support more audio formats
- Improve handling of large audio files
- Make things faster and use less memory.
[bugs]: https://github.com/jiaaro/pydub/issues?q=is%3Aissue+is%3Aopen+label%3Abug
[todos]: https://github.com/jiaaro/pydub/issues?q=is%3Aissue+is%3Aopen+label%3Atodo
+20
View File
@@ -0,0 +1,20 @@
Copyright (c) 2011 James Robert, http://jiaaro.com
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+1
View File
@@ -0,0 +1 @@
include LICENSE
+333
View File
@@ -0,0 +1,333 @@
# Pydub [![Build Status](https://travis-ci.org/jiaaro/pydub.svg?branch=master)](https://travis-ci.org/jiaaro/pydub) [![Build status](https://ci.appveyor.com/api/projects/status/gy1ucp9o5khq7fqi/branch/master?svg=true)](https://ci.appveyor.com/project/jiaaro/pydub/branch/master)
Pydub lets you do stuff to audio in a way that isn't stupid.
**Stuff you might be looking for**:
- [Installing Pydub](https://github.com/jiaaro/pydub#installation)
- [API Documentation](https://github.com/jiaaro/pydub/blob/master/API.markdown)
- [Dependencies](https://github.com/jiaaro/pydub#dependencies)
- [Playback](https://github.com/jiaaro/pydub#playback)
- [Setting up ffmpeg](https://github.com/jiaaro/pydub#getting-ffmpeg-set-up)
- [Questions/Bugs](https://github.com/jiaaro/pydub#bugs--questions)
## Quickstart
Open a WAV file
```python
from pydub import AudioSegment
song = AudioSegment.from_wav("never_gonna_give_you_up.wav")
```
...or a mp3
```python
song = AudioSegment.from_mp3("never_gonna_give_you_up.mp3")
```
... or an ogg, or flv, or [anything else ffmpeg supports](http://www.ffmpeg.org/general.html#File-Formats)
```python
ogg_version = AudioSegment.from_ogg("never_gonna_give_you_up.ogg")
flv_version = AudioSegment.from_flv("never_gonna_give_you_up.flv")
mp4_version = AudioSegment.from_file("never_gonna_give_you_up.mp4", "mp4")
wma_version = AudioSegment.from_file("never_gonna_give_you_up.wma", "wma")
aac_version = AudioSegment.from_file("never_gonna_give_you_up.aiff", "aac")
```
Slice audio:
```python
# pydub does things in milliseconds
ten_seconds = 10 * 1000
first_10_seconds = song[:ten_seconds]
last_5_seconds = song[-5000:]
```
Make the beginning louder and the end quieter
```python
# boost volume by 6dB
beginning = first_10_seconds + 6
# reduce volume by 3dB
end = last_5_seconds - 3
```
Concatenate audio (add one file to the end of another)
```python
without_the_middle = beginning + end
```
How long is it?
```python
without_the_middle.duration_seconds == 15.0
```
AudioSegments are immutable
```python
# song is not modified
backwards = song.reverse()
```
Crossfade (again, beginning and end are not modified)
```python
# 1.5 second crossfade
with_style = beginning.append(end, crossfade=1500)
```
Repeat
```python
# repeat the clip twice
do_it_over = with_style * 2
```
Fade (note that you can chain operations because everything returns
an AudioSegment)
```python
# 2 sec fade in, 3 sec fade out
awesome = do_it_over.fade_in(2000).fade_out(3000)
```
Save the results (again whatever ffmpeg supports)
```python
awesome.export("mashup.mp3", format="mp3")
```
Save the results with tags (metadata)
```python
awesome.export("mashup.mp3", format="mp3", tags={'artist': 'Various artists', 'album': 'Best of 2011', 'comments': 'This album is awesome!'})
```
You can pass an optional bitrate argument to export using any syntax ffmpeg
supports.
```python
awesome.export("mashup.mp3", format="mp3", bitrate="192k")
```
Any further arguments supported by ffmpeg can be passed as a list in a
'parameters' argument, with switch first, argument second. Note that no
validation takes place on these parameters, and you may be limited by what
your particular build of ffmpeg/avlib supports.
```python
# Use preset mp3 quality 0 (equivalent to lame V0)
awesome.export("mashup.mp3", format="mp3", parameters=["-q:a", "0"])
# Mix down to two channels and set hard output volume
awesome.export("mashup.mp3", format="mp3", parameters=["-ac", "2", "-vol", "150"])
```
## Debugging
Most issues people run into are related to converting between formats using
ffmpeg/avlib. Pydub provides a logger that outputs the subprocess calls to
help you track down issues:
```python
>>> import logging
>>> l = logging.getLogger("pydub.converter")
>>> l.setLevel(logging.DEBUG)
>>> l.addHandler(logging.StreamHandler())
>>> AudioSegment.from_file("./test/data/test1.mp3")
subprocess.call(['ffmpeg', '-y', '-i', '/var/folders/71/42k8g72x4pq09tfp920d033r0000gn/T/tmpeZTgMy', '-vn', '-f', 'wav', '/var/folders/71/42k8g72x4pq09tfp920d033r0000gn/T/tmpK5aLcZ'])
<pydub.audio_segment.AudioSegment object at 0x101b43e10>
```
Don't worry about the temporary files used in the conversion. They're cleaned up
automatically.
## Bugs & Questions
You can file bugs in our [github issues tracker](https://github.com/jiaaro/pydub/issues),
and ask any technical questions on
[Stack Overflow using the pydub tag](http://stackoverflow.com/questions/ask?tags=pydub).
We keep an eye on both.
## Installation
Installing pydub is easy, but don't forget to install ffmpeg/avlib (the next section in this doc)
pip install pydub
Or install the latest dev version from github (or replace `@master` with a [release version like `@v0.12.0`](https://github.com/jiaaro/pydub/releases))…
pip install git+https://github.com/jiaaro/pydub.git@master
-OR-
git clone https://github.com/jiaaro/pydub.git
-OR-
Copy the pydub directory into your python path. Zip
[here](https://github.com/jiaaro/pydub/zipball/master)
## Dependencies
You can open and save WAV files with pure python. For opening and saving non-wav
files like mp3 you'll need [ffmpeg](http://www.ffmpeg.org/) or
[libav](http://libav.org/).
### Playback
You can play audio if you have one of these installed (simpleaudio _strongly_ recommended, even if you are installing ffmpeg/libav):
- [simpleaudio](https://simpleaudio.readthedocs.io/en/latest/)
- [pyaudio](https://people.csail.mit.edu/hubert/pyaudio/docs/#)
- ffplay (usually bundled with ffmpeg, see the next section)
- avplay (usually bundled with libav, see the next section)
```python
from pydub import AudioSegment
from pydub.playback import play
sound = AudioSegment.from_file("mysound.wav", format="wav")
play(sound)
```
## Getting ffmpeg set up
You may use **libav or ffmpeg**.
Mac (using [homebrew](http://brew.sh)):
```bash
# libav
brew install libav
#### OR #####
# ffmpeg
brew install ffmpeg
```
Linux (using aptitude):
```bash
# libav
apt-get install libav-tools libavcodec-extra
#### OR #####
# ffmpeg
apt-get install ffmpeg libavcodec-extra
```
Windows:
1. Download and extract libav from [Windows binaries provided here](http://builds.libav.org/windows/).
2. Add the libav `/bin` folder to your PATH envvar
3. `pip install pydub`
## Important Notes
`AudioSegment` objects are [immutable](http://www.devshed.com/c/a/Python/String-and-List-Python-Object-Types/1/)
### Ogg exporting and default codecs
The Ogg specification ([http://tools.ietf.org/html/rfc5334](rfc5334)) does not specify
the codec to use, this choice is left up to the user. Vorbis and Theora are just
some of a number of potential codecs (see page 3 of the rfc) that can be used for the
encapsulated data.
When no codec is specified exporting to `ogg` will _default_ to using `vorbis`
as a convenience. That is:
```python
from pydub import AudioSegment
song = AudioSegment.from_mp3("test/data/test1.mp3")
song.export("out.ogg", format="ogg") # Is the same as:
song.export("out.ogg", format="ogg", codec="libvorbis")
```
## Example Use
Suppose you have a directory filled with *mp4* and *flv* videos and you want to convert all of them to *mp3* so you can listen to them on your mp3 player.
```python
import os
import glob
from pydub import AudioSegment
video_dir = '/home/johndoe/downloaded_videos/' # Path where the videos are located
extension_list = ('*.mp4', '*.flv')
os.chdir(video_dir)
for extension in extension_list:
for video in glob.glob(extension):
mp3_filename = os.path.splitext(os.path.basename(video))[0] + '.mp3'
AudioSegment.from_file(video).export(mp3_filename, format='mp3')
```
### How about another example?
```python
from glob import glob
from pydub import AudioSegment
playlist_songs = [AudioSegment.from_mp3(mp3_file) for mp3_file in glob("*.mp3")]
first_song = playlist_songs.pop(0)
# let's just include the first 30 seconds of the first song (slicing
# is done by milliseconds)
beginning_of_song = first_song[:30*1000]
playlist = beginning_of_song
for song in playlist_songs:
# We don't want an abrupt stop at the end, so let's do a 10 second crossfades
playlist = playlist.append(song, crossfade=(10 * 1000))
# let's fade out the end of the last song
playlist = playlist.fade_out(30)
# hmm I wonder how long it is... ( len(audio_segment) returns milliseconds )
playlist_length = len(playlist) / (1000*60)
# lets save it!
with open("%s_minute_playlist.mp3" % playlist_length, 'wb') as out_f:
playlist.export(out_f, format='mp3')
```
## License ([MIT License](http://opensource.org/licenses/mit-license.php))
Copyright © 2011 James Robert, http://jiaaro.com
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+30
View File
@@ -0,0 +1,30 @@
build: false
environment:
matrix:
- PYTHON: "C:/Python27"
FFMPEG: "4.2.3"
- PYTHON: "C:/Python34"
FFMPEG: "4.2.3"
- PYTHON: "C:/Python35"
FFMPEG: "4.2.3"
- PYTHON: "C:/Python36"
FFMPEG: "4.2.3"
- PYTHON: "C:/Python36"
FFMPEG: "latest"
matrix:
allow_failures:
- FFMPEG: "latest"
init:
- "ECHO %PYTHON%"
- ps: "ls C:/Python*"
install:
- "%PYTHON%/python.exe -m pip install wheel"
- "%PYTHON%/python.exe -m pip install -e ."
# Install ffmpeg
- ps: Start-FileDownload ('https://github.com/advancedfx/ffmpeg.zeranoe.com-builds-mirror/releases/download/20200915/ffmpeg-' + $env:FFMPEG + '-win64-shared.zip') ffmpeg-shared.zip
- 7z x ffmpeg-shared.zip > NULL
- "SET PATH=%cd%\\ffmpeg-%FFMPEG%-win64-shared\\bin;%PATH%"
# check ffmpeg installation (also shows version)
- "ffmpeg.exe -version"
test_script:
- "%PYTHON%/python.exe test/test.py"
@@ -0,0 +1 @@
from .audio_segment import AudioSegment
File diff suppressed because it is too large Load Diff
+341
View File
@@ -0,0 +1,341 @@
import sys
import math
import array
from .utils import (
db_to_float,
ratio_to_db,
register_pydub_effect,
make_chunks,
audioop,
get_min_max_value
)
from .silence import split_on_silence
from .exceptions import TooManyMissingFrames, InvalidDuration
if sys.version_info >= (3, 0):
xrange = range
@register_pydub_effect
def apply_mono_filter_to_each_channel(seg, filter_fn):
n_channels = seg.channels
channel_segs = seg.split_to_mono()
channel_segs = [filter_fn(channel_seg) for channel_seg in channel_segs]
out_data = seg.get_array_of_samples()
for channel_i, channel_seg in enumerate(channel_segs):
for sample_i, sample in enumerate(channel_seg.get_array_of_samples()):
index = (sample_i * n_channels) + channel_i
out_data[index] = sample
return seg._spawn(out_data)
@register_pydub_effect
def normalize(seg, headroom=0.1):
"""
headroom is how close to the maximum volume to boost the signal up to (specified in dB)
"""
peak_sample_val = seg.max
# if the max is 0, this audio segment is silent, and can't be normalized
if peak_sample_val == 0:
return seg
target_peak = seg.max_possible_amplitude * db_to_float(-headroom)
needed_boost = ratio_to_db(target_peak / peak_sample_val)
return seg.apply_gain(needed_boost)
@register_pydub_effect
def speedup(seg, playback_speed=1.5, chunk_size=150, crossfade=25):
# we will keep audio in 150ms chunks since one waveform at 20Hz is 50ms long
# (20 Hz is the lowest frequency audible to humans)
# portion of AUDIO TO KEEP. if playback speed is 1.25 we keep 80% (0.8) and
# discard 20% (0.2)
atk = 1.0 / playback_speed
if playback_speed < 2.0:
# throwing out more than half the audio - keep 50ms chunks
ms_to_remove_per_chunk = int(chunk_size * (1 - atk) / atk)
else:
# throwing out less than half the audio - throw out 50ms chunks
ms_to_remove_per_chunk = int(chunk_size)
chunk_size = int(atk * chunk_size / (1 - atk))
# the crossfade cannot be longer than the amount of audio we're removing
crossfade = min(crossfade, ms_to_remove_per_chunk - 1)
# DEBUG
#print("chunk: {0}, rm: {1}".format(chunk_size, ms_to_remove_per_chunk))
chunks = make_chunks(seg, chunk_size + ms_to_remove_per_chunk)
if len(chunks) < 2:
raise Exception("Could not speed up AudioSegment, it was too short {2:0.2f}s for the current settings:\n{0}ms chunks at {1:0.1f}x speedup".format(
chunk_size, playback_speed, seg.duration_seconds))
# we'll actually truncate a bit less than we calculated to make up for the
# crossfade between chunks
ms_to_remove_per_chunk -= crossfade
# we don't want to truncate the last chunk since it is not guaranteed to be
# the full chunk length
last_chunk = chunks[-1]
chunks = [chunk[:-ms_to_remove_per_chunk] for chunk in chunks[:-1]]
out = chunks[0]
for chunk in chunks[1:]:
out = out.append(chunk, crossfade=crossfade)
out += last_chunk
return out
@register_pydub_effect
def strip_silence(seg, silence_len=1000, silence_thresh=-16, padding=100):
if padding > silence_len:
raise InvalidDuration("padding cannot be longer than silence_len")
chunks = split_on_silence(seg, silence_len, silence_thresh, padding)
crossfade = padding / 2
if not len(chunks):
return seg[0:0]
seg = chunks[0]
for chunk in chunks[1:]:
seg = seg.append(chunk, crossfade=crossfade)
return seg
@register_pydub_effect
def compress_dynamic_range(seg, threshold=-20.0, ratio=4.0, attack=5.0, release=50.0):
"""
Keyword Arguments:
threshold - default: -20.0
Threshold in dBFS. default of -20.0 means -20dB relative to the
maximum possible volume. 0dBFS is the maximum possible value so
all values for this argument sould be negative.
ratio - default: 4.0
Compression ratio. Audio louder than the threshold will be
reduced to 1/ratio the volume. A ratio of 4.0 is equivalent to
a setting of 4:1 in a pro-audio compressor like the Waves C1.
attack - default: 5.0
Attack in milliseconds. How long it should take for the compressor
to kick in once the audio has exceeded the threshold.
release - default: 50.0
Release in milliseconds. How long it should take for the compressor
to stop compressing after the audio has falled below the threshold.
For an overview of Dynamic Range Compression, and more detailed explanation
of the related terminology, see:
http://en.wikipedia.org/wiki/Dynamic_range_compression
"""
thresh_rms = seg.max_possible_amplitude * db_to_float(threshold)
look_frames = int(seg.frame_count(ms=attack))
def rms_at(frame_i):
return seg.get_sample_slice(frame_i - look_frames, frame_i).rms
def db_over_threshold(rms):
if rms == 0: return 0.0
db = ratio_to_db(rms / thresh_rms)
return max(db, 0)
output = []
# amount to reduce the volume of the audio by (in dB)
attenuation = 0.0
attack_frames = seg.frame_count(ms=attack)
release_frames = seg.frame_count(ms=release)
for i in xrange(int(seg.frame_count())):
rms_now = rms_at(i)
# with a ratio of 4.0 this means the volume will exceed the threshold by
# 1/4 the amount (of dB) that it would otherwise
max_attenuation = (1 - (1.0 / ratio)) * db_over_threshold(rms_now)
attenuation_inc = max_attenuation / attack_frames
attenuation_dec = max_attenuation / release_frames
if rms_now > thresh_rms and attenuation <= max_attenuation:
attenuation += attenuation_inc
attenuation = min(attenuation, max_attenuation)
else:
attenuation -= attenuation_dec
attenuation = max(attenuation, 0)
frame = seg.get_frame(i)
if attenuation != 0.0:
frame = audioop.mul(frame,
seg.sample_width,
db_to_float(-attenuation))
output.append(frame)
return seg._spawn(data=b''.join(output))
# Invert the phase of the signal.
@register_pydub_effect
def invert_phase(seg, channels=(1, 1)):
"""
channels- specifies which channel (left or right) to reverse the phase of.
Note that mono AudioSegments will become stereo.
"""
if channels == (1, 1):
inverted = audioop.mul(seg._data, seg.sample_width, -1.0)
return seg._spawn(data=inverted)
else:
if seg.channels == 2:
left, right = seg.split_to_mono()
else:
raise Exception("Can't implicitly convert an AudioSegment with " + str(seg.channels) + " channels to stereo.")
if channels == (1, 0):
left = left.invert_phase()
else:
right = right.invert_phase()
return seg.from_mono_audiosegments(left, right)
# High and low pass filters based on implementation found on Stack Overflow:
# http://stackoverflow.com/questions/13882038/implementing-simple-high-and-low-pass-filters-in-c
@register_pydub_effect
def low_pass_filter(seg, cutoff):
"""
cutoff - Frequency (in Hz) where higher frequency signal will begin to
be reduced by 6dB per octave (doubling in frequency) above this point
"""
RC = 1.0 / (cutoff * 2 * math.pi)
dt = 1.0 / seg.frame_rate
alpha = dt / (RC + dt)
original = seg.get_array_of_samples()
filteredArray = array.array(seg.array_type, original)
frame_count = int(seg.frame_count())
last_val = [0] * seg.channels
for i in range(seg.channels):
last_val[i] = filteredArray[i] = original[i]
for i in range(1, frame_count):
for j in range(seg.channels):
offset = (i * seg.channels) + j
last_val[j] = last_val[j] + (alpha * (original[offset] - last_val[j]))
filteredArray[offset] = int(last_val[j])
return seg._spawn(data=filteredArray)
@register_pydub_effect
def high_pass_filter(seg, cutoff):
"""
cutoff - Frequency (in Hz) where lower frequency signal will begin to
be reduced by 6dB per octave (doubling in frequency) below this point
"""
RC = 1.0 / (cutoff * 2 * math.pi)
dt = 1.0 / seg.frame_rate
alpha = RC / (RC + dt)
minval, maxval = get_min_max_value(seg.sample_width * 8)
original = seg.get_array_of_samples()
filteredArray = array.array(seg.array_type, original)
frame_count = int(seg.frame_count())
last_val = [0] * seg.channels
for i in range(seg.channels):
last_val[i] = filteredArray[i] = original[i]
for i in range(1, frame_count):
for j in range(seg.channels):
offset = (i * seg.channels) + j
offset_minus_1 = ((i-1) * seg.channels) + j
last_val[j] = alpha * (last_val[j] + original[offset] - original[offset_minus_1])
filteredArray[offset] = int(min(max(last_val[j], minval), maxval))
return seg._spawn(data=filteredArray)
@register_pydub_effect
def pan(seg, pan_amount):
"""
pan_amount should be between -1.0 (100% left) and +1.0 (100% right)
When pan_amount == 0.0 the left/right balance is not changed.
Panning does not alter the *perceived* loundness, but since loudness
is decreasing on one side, the other side needs to get louder to
compensate. When panned hard left, the left channel will be 3dB louder.
"""
if not -1.0 <= pan_amount <= 1.0:
raise ValueError("pan_amount should be between -1.0 (100% left) and +1.0 (100% right)")
max_boost_db = ratio_to_db(2.0)
boost_db = abs(pan_amount) * max_boost_db
boost_factor = db_to_float(boost_db)
reduce_factor = db_to_float(max_boost_db) - boost_factor
reduce_db = ratio_to_db(reduce_factor)
# Cut boost in half (max boost== 3dB) - in reality 2 speakers
# do not sum to a full 6 dB.
boost_db = boost_db / 2.0
if pan_amount < 0:
return seg.apply_gain_stereo(boost_db, reduce_db)
else:
return seg.apply_gain_stereo(reduce_db, boost_db)
@register_pydub_effect
def apply_gain_stereo(seg, left_gain=0.0, right_gain=0.0):
"""
left_gain - amount of gain to apply to the left channel (in dB)
right_gain - amount of gain to apply to the right channel (in dB)
note: mono audio segments will be converted to stereo
"""
if seg.channels == 1:
left = right = seg
elif seg.channels == 2:
left, right = seg.split_to_mono()
l_mult_factor = db_to_float(left_gain)
r_mult_factor = db_to_float(right_gain)
left_data = audioop.mul(left._data, left.sample_width, l_mult_factor)
left_data = audioop.tostereo(left_data, left.sample_width, 1, 0)
right_data = audioop.mul(right._data, right.sample_width, r_mult_factor)
right_data = audioop.tostereo(right_data, right.sample_width, 0, 1)
output = audioop.add(left_data, right_data, seg.sample_width)
return seg._spawn(data=output,
overrides={'channels': 2,
'frame_width': 2 * seg.sample_width})
@@ -0,0 +1,32 @@
class PydubException(Exception):
"""
Base class for any Pydub exception
"""
class TooManyMissingFrames(PydubException):
pass
class InvalidDuration(PydubException):
pass
class InvalidTag(PydubException):
pass
class InvalidID3TagVersion(PydubException):
pass
class CouldntDecodeError(PydubException):
pass
class CouldntEncodeError(PydubException):
pass
class MissingAudioParameter(PydubException):
pass
@@ -0,0 +1,142 @@
"""
Each generator will return float samples from -1.0 to 1.0, which can be
converted to actual audio with 8, 16, 24, or 32 bit depth using the
SiganlGenerator.to_audio_segment() method (on any of it's subclasses).
See Wikipedia's "waveform" page for info on some of the generators included
here: http://en.wikipedia.org/wiki/Waveform
"""
import math
import array
import itertools
import random
from .audio_segment import AudioSegment
from .utils import (
db_to_float,
get_frame_width,
get_array_type,
get_min_max_value
)
class SignalGenerator(object):
def __init__(self, sample_rate=44100, bit_depth=16):
self.sample_rate = sample_rate
self.bit_depth = bit_depth
def to_audio_segment(self, duration=1000.0, volume=0.0):
"""
Duration in milliseconds
(default: 1 second)
Volume in DB relative to maximum amplitude
(default 0.0 dBFS, which is the maximum value)
"""
minval, maxval = get_min_max_value(self.bit_depth)
sample_width = get_frame_width(self.bit_depth)
array_type = get_array_type(self.bit_depth)
gain = db_to_float(volume)
sample_count = int(self.sample_rate * (duration / 1000.0))
sample_data = (int(val * maxval * gain) for val in self.generate())
sample_data = itertools.islice(sample_data, 0, sample_count)
data = array.array(array_type, sample_data)
try:
data = data.tobytes()
except:
data = data.tostring()
return AudioSegment(data=data, metadata={
"channels": 1,
"sample_width": sample_width,
"frame_rate": self.sample_rate,
"frame_width": sample_width,
})
def generate(self):
raise NotImplementedError("SignalGenerator subclasses must implement the generate() method, and *should not* call the superclass implementation.")
class Sine(SignalGenerator):
def __init__(self, freq, **kwargs):
super(Sine, self).__init__(**kwargs)
self.freq = freq
def generate(self):
sine_of = (self.freq * 2 * math.pi) / self.sample_rate
sample_n = 0
while True:
yield math.sin(sine_of * sample_n)
sample_n += 1
class Pulse(SignalGenerator):
def __init__(self, freq, duty_cycle=0.5, **kwargs):
super(Pulse, self).__init__(**kwargs)
self.freq = freq
self.duty_cycle = duty_cycle
def generate(self):
sample_n = 0
# in samples
cycle_length = self.sample_rate / float(self.freq)
pulse_length = cycle_length * self.duty_cycle
while True:
if (sample_n % cycle_length) < pulse_length:
yield 1.0
else:
yield -1.0
sample_n += 1
class Square(Pulse):
def __init__(self, freq, **kwargs):
kwargs['duty_cycle'] = 0.5
super(Square, self).__init__(freq, **kwargs)
class Sawtooth(SignalGenerator):
def __init__(self, freq, duty_cycle=1.0, **kwargs):
super(Sawtooth, self).__init__(**kwargs)
self.freq = freq
self.duty_cycle = duty_cycle
def generate(self):
sample_n = 0
# in samples
cycle_length = self.sample_rate / float(self.freq)
midpoint = cycle_length * self.duty_cycle
ascend_length = midpoint
descend_length = cycle_length - ascend_length
while True:
cycle_position = sample_n % cycle_length
if cycle_position < midpoint:
yield (2 * cycle_position / ascend_length) - 1.0
else:
yield 1.0 - (2 * (cycle_position - midpoint) / descend_length)
sample_n += 1
class Triangle(Sawtooth):
def __init__(self, freq, **kwargs):
kwargs['duty_cycle'] = 0.5
super(Triangle, self).__init__(freq, **kwargs)
class WhiteNoise(SignalGenerator):
def generate(self):
while True:
yield (random.random() * 2) - 1.0
@@ -0,0 +1,14 @@
"""
"""
import logging
converter_logger = logging.getLogger("swingmusic.pydub.converter")
def log_conversion(conversion_command):
converter_logger.debug("subprocess.call(%s)", repr(conversion_command))
def log_subprocess_output(output):
if output:
for line in output.rstrip().splitlines():
converter_logger.debug('subprocess output: %s', line.rstrip())
@@ -0,0 +1,71 @@
"""
Support for playing AudioSegments. Pyaudio will be used if it's installed,
otherwise will fallback to ffplay. Pyaudio is a *much* nicer solution, but
is tricky to install. See my notes on installing pyaudio in a virtualenv (on
OSX 10.10): https://gist.github.com/jiaaro/9767512210a1d80a8a0d
"""
import subprocess
from tempfile import NamedTemporaryFile
from .utils import get_player_name, make_chunks
def _play_with_ffplay(seg):
PLAYER = get_player_name()
with NamedTemporaryFile("w+b", suffix=".wav") as f:
seg.export(f.name, "wav")
subprocess.call([PLAYER, "-nodisp", "-autoexit", "-hide_banner", f.name])
def _play_with_pyaudio(seg):
import pyaudio
p = pyaudio.PyAudio()
stream = p.open(format=p.get_format_from_width(seg.sample_width),
channels=seg.channels,
rate=seg.frame_rate,
output=True)
# Just in case there were any exceptions/interrupts, we release the resource
# So as not to raise OSError: Device Unavailable should play() be used again
try:
# break audio into half-second chunks (to allows keyboard interrupts)
for chunk in make_chunks(seg, 500):
stream.write(chunk._data)
finally:
stream.stop_stream()
stream.close()
p.terminate()
def _play_with_simpleaudio(seg):
import simpleaudio
return simpleaudio.play_buffer(
seg.raw_data,
num_channels=seg.channels,
bytes_per_sample=seg.sample_width,
sample_rate=seg.frame_rate
)
def play(audio_segment):
try:
playback = _play_with_simpleaudio(audio_segment)
try:
playback.wait_done()
except KeyboardInterrupt:
playback.stop()
except ImportError:
pass
else:
return
try:
_play_with_pyaudio(audio_segment)
return
except ImportError:
pass
else:
return
_play_with_ffplay(audio_segment)
+553
View File
@@ -0,0 +1,553 @@
try:
from __builtin__ import max as builtin_max
from __builtin__ import min as builtin_min
except ImportError:
from builtins import max as builtin_max
from builtins import min as builtin_min
import math
import struct
try:
from fractions import gcd
except ImportError: # Python 3.9+
from math import gcd
from ctypes import create_string_buffer
class error(Exception):
pass
def _check_size(size):
if size != 1 and size != 2 and size != 4:
raise error("Size should be 1, 2 or 4")
def _check_params(length, size):
_check_size(size)
if length % size != 0:
raise error("not a whole number of frames")
def _sample_count(cp, size):
return len(cp) / size
def _get_samples(cp, size, signed=True):
for i in range(_sample_count(cp, size)):
yield _get_sample(cp, size, i, signed)
def _struct_format(size, signed):
if size == 1:
return "b" if signed else "B"
elif size == 2:
return "h" if signed else "H"
elif size == 4:
return "i" if signed else "I"
def _get_sample(cp, size, i, signed=True):
fmt = _struct_format(size, signed)
start = i * size
end = start + size
return struct.unpack_from(fmt, buffer(cp)[start:end])[0]
def _put_sample(cp, size, i, val, signed=True):
fmt = _struct_format(size, signed)
struct.pack_into(fmt, cp, i * size, val)
def _get_maxval(size, signed=True):
if signed and size == 1:
return 0x7f
elif size == 1:
return 0xff
elif signed and size == 2:
return 0x7fff
elif size == 2:
return 0xffff
elif signed and size == 4:
return 0x7fffffff
elif size == 4:
return 0xffffffff
def _get_minval(size, signed=True):
if not signed:
return 0
elif size == 1:
return -0x80
elif size == 2:
return -0x8000
elif size == 4:
return -0x80000000
def _get_clipfn(size, signed=True):
maxval = _get_maxval(size, signed)
minval = _get_minval(size, signed)
return lambda val: builtin_max(min(val, maxval), minval)
def _overflow(val, size, signed=True):
minval = _get_minval(size, signed)
maxval = _get_maxval(size, signed)
if minval <= val <= maxval:
return val
bits = size * 8
if signed:
offset = 2**(bits-1)
return ((val + offset) % (2**bits)) - offset
else:
return val % (2**bits)
def getsample(cp, size, i):
_check_params(len(cp), size)
if not (0 <= i < len(cp) / size):
raise error("Index out of range")
return _get_sample(cp, size, i)
def max(cp, size):
_check_params(len(cp), size)
if len(cp) == 0:
return 0
return builtin_max(abs(sample) for sample in _get_samples(cp, size))
def minmax(cp, size):
_check_params(len(cp), size)
max_sample, min_sample = 0, 0
for sample in _get_samples(cp, size):
max_sample = builtin_max(sample, max_sample)
min_sample = builtin_min(sample, min_sample)
return min_sample, max_sample
def avg(cp, size):
_check_params(len(cp), size)
sample_count = _sample_count(cp, size)
if sample_count == 0:
return 0
return sum(_get_samples(cp, size)) / sample_count
def rms(cp, size):
_check_params(len(cp), size)
sample_count = _sample_count(cp, size)
if sample_count == 0:
return 0
sum_squares = sum(sample**2 for sample in _get_samples(cp, size))
return int(math.sqrt(sum_squares / sample_count))
def _sum2(cp1, cp2, length):
size = 2
total = 0
for i in range(length):
total += getsample(cp1, size, i) * getsample(cp2, size, i)
return total
def findfit(cp1, cp2):
size = 2
if len(cp1) % 2 != 0 or len(cp2) % 2 != 0:
raise error("Strings should be even-sized")
if len(cp1) < len(cp2):
raise error("First sample should be longer")
len1 = _sample_count(cp1, size)
len2 = _sample_count(cp2, size)
sum_ri_2 = _sum2(cp2, cp2, len2)
sum_aij_2 = _sum2(cp1, cp1, len2)
sum_aij_ri = _sum2(cp1, cp2, len2)
result = (sum_ri_2 * sum_aij_2 - sum_aij_ri * sum_aij_ri) / sum_aij_2
best_result = result
best_i = 0
for i in range(1, len1 - len2 + 1):
aj_m1 = _get_sample(cp1, size, i - 1)
aj_lm1 = _get_sample(cp1, size, i + len2 - 1)
sum_aij_2 += aj_lm1**2 - aj_m1**2
sum_aij_ri = _sum2(buffer(cp1)[i*size:], cp2, len2)
result = (sum_ri_2 * sum_aij_2 - sum_aij_ri * sum_aij_ri) / sum_aij_2
if result < best_result:
best_result = result
best_i = i
factor = _sum2(buffer(cp1)[best_i*size:], cp2, len2) / sum_ri_2
return best_i, factor
def findfactor(cp1, cp2):
size = 2
if len(cp1) % 2 != 0:
raise error("Strings should be even-sized")
if len(cp1) != len(cp2):
raise error("Samples should be same size")
sample_count = _sample_count(cp1, size)
sum_ri_2 = _sum2(cp2, cp2, sample_count)
sum_aij_ri = _sum2(cp1, cp2, sample_count)
return sum_aij_ri / sum_ri_2
def findmax(cp, len2):
size = 2
sample_count = _sample_count(cp, size)
if len(cp) % 2 != 0:
raise error("Strings should be even-sized")
if len2 < 0 or sample_count < len2:
raise error("Input sample should be longer")
if sample_count == 0:
return 0
result = _sum2(cp, cp, len2)
best_result = result
best_i = 0
for i in range(1, sample_count - len2 + 1):
sample_leaving_window = getsample(cp, size, i - 1)
sample_entering_window = getsample(cp, size, i + len2 - 1)
result -= sample_leaving_window**2
result += sample_entering_window**2
if result > best_result:
best_result = result
best_i = i
return best_i
def avgpp(cp, size):
_check_params(len(cp), size)
sample_count = _sample_count(cp, size)
prevextremevalid = False
prevextreme = None
avg = 0
nextreme = 0
prevval = getsample(cp, size, 0)
val = getsample(cp, size, 1)
prevdiff = val - prevval
for i in range(1, sample_count):
val = getsample(cp, size, i)
diff = val - prevval
if diff * prevdiff < 0:
if prevextremevalid:
avg += abs(prevval - prevextreme)
nextreme += 1
prevextremevalid = True
prevextreme = prevval
prevval = val
if diff != 0:
prevdiff = diff
if nextreme == 0:
return 0
return avg / nextreme
def maxpp(cp, size):
_check_params(len(cp), size)
sample_count = _sample_count(cp, size)
prevextremevalid = False
prevextreme = None
max = 0
prevval = getsample(cp, size, 0)
val = getsample(cp, size, 1)
prevdiff = val - prevval
for i in range(1, sample_count):
val = getsample(cp, size, i)
diff = val - prevval
if diff * prevdiff < 0:
if prevextremevalid:
extremediff = abs(prevval - prevextreme)
if extremediff > max:
max = extremediff
prevextremevalid = True
prevextreme = prevval
prevval = val
if diff != 0:
prevdiff = diff
return max
def cross(cp, size):
_check_params(len(cp), size)
crossings = 0
last_sample = 0
for sample in _get_samples(cp, size):
if sample <= 0 < last_sample or sample >= 0 > last_sample:
crossings += 1
last_sample = sample
return crossings
def mul(cp, size, factor):
_check_params(len(cp), size)
clip = _get_clipfn(size)
result = create_string_buffer(len(cp))
for i, sample in enumerate(_get_samples(cp, size)):
sample = clip(int(sample * factor))
_put_sample(result, size, i, sample)
return result.raw
def tomono(cp, size, fac1, fac2):
_check_params(len(cp), size)
clip = _get_clipfn(size)
sample_count = _sample_count(cp, size)
result = create_string_buffer(len(cp) / 2)
for i in range(0, sample_count, 2):
l_sample = getsample(cp, size, i)
r_sample = getsample(cp, size, i + 1)
sample = (l_sample * fac1) + (r_sample * fac2)
sample = clip(sample)
_put_sample(result, size, i / 2, sample)
return result.raw
def tostereo(cp, size, fac1, fac2):
_check_params(len(cp), size)
sample_count = _sample_count(cp, size)
result = create_string_buffer(len(cp) * 2)
clip = _get_clipfn(size)
for i in range(sample_count):
sample = _get_sample(cp, size, i)
l_sample = clip(sample * fac1)
r_sample = clip(sample * fac2)
_put_sample(result, size, i * 2, l_sample)
_put_sample(result, size, i * 2 + 1, r_sample)
return result.raw
def add(cp1, cp2, size):
_check_params(len(cp1), size)
if len(cp1) != len(cp2):
raise error("Lengths should be the same")
clip = _get_clipfn(size)
sample_count = _sample_count(cp1, size)
result = create_string_buffer(len(cp1))
for i in range(sample_count):
sample1 = getsample(cp1, size, i)
sample2 = getsample(cp2, size, i)
sample = clip(sample1 + sample2)
_put_sample(result, size, i, sample)
return result.raw
def bias(cp, size, bias):
_check_params(len(cp), size)
result = create_string_buffer(len(cp))
for i, sample in enumerate(_get_samples(cp, size)):
sample = _overflow(sample + bias, size)
_put_sample(result, size, i, sample)
return result.raw
def reverse(cp, size):
_check_params(len(cp), size)
sample_count = _sample_count(cp, size)
result = create_string_buffer(len(cp))
for i, sample in enumerate(_get_samples(cp, size)):
_put_sample(result, size, sample_count - i - 1, sample)
return result.raw
def lin2lin(cp, size, size2):
_check_params(len(cp), size)
_check_size(size2)
if size == size2:
return cp
new_len = (len(cp) / size) * size2
result = create_string_buffer(new_len)
for i in range(_sample_count(cp, size)):
sample = _get_sample(cp, size, i)
if size < size2:
sample = sample << (4 * size2 / size)
elif size > size2:
sample = sample >> (4 * size / size2)
sample = _overflow(sample, size2)
_put_sample(result, size2, i, sample)
return result.raw
def ratecv(cp, size, nchannels, inrate, outrate, state, weightA=1, weightB=0):
_check_params(len(cp), size)
if nchannels < 1:
raise error("# of channels should be >= 1")
bytes_per_frame = size * nchannels
frame_count = len(cp) / bytes_per_frame
if bytes_per_frame / nchannels != size:
raise OverflowError("width * nchannels too big for a C int")
if weightA < 1 or weightB < 0:
raise error("weightA should be >= 1, weightB should be >= 0")
if len(cp) % bytes_per_frame != 0:
raise error("not a whole number of frames")
if inrate <= 0 or outrate <= 0:
raise error("sampling rate not > 0")
d = gcd(inrate, outrate)
inrate /= d
outrate /= d
prev_i = [0] * nchannels
cur_i = [0] * nchannels
if state is None:
d = -outrate
else:
d, samps = state
if len(samps) != nchannels:
raise error("illegal state argument")
prev_i, cur_i = zip(*samps)
prev_i, cur_i = list(prev_i), list(cur_i)
q = frame_count / inrate
ceiling = (q + 1) * outrate
nbytes = ceiling * bytes_per_frame
result = create_string_buffer(nbytes)
samples = _get_samples(cp, size)
out_i = 0
while True:
while d < 0:
if frame_count == 0:
samps = zip(prev_i, cur_i)
retval = result.raw
# slice off extra bytes
trim_index = (out_i * bytes_per_frame) - len(retval)
retval = buffer(retval)[:trim_index]
return (retval, (d, tuple(samps)))
for chan in range(nchannels):
prev_i[chan] = cur_i[chan]
cur_i[chan] = samples.next()
cur_i[chan] = (
(weightA * cur_i[chan] + weightB * prev_i[chan])
/ (weightA + weightB)
)
frame_count -= 1
d += outrate
while d >= 0:
for chan in range(nchannels):
cur_o = (
(prev_i[chan] * d + cur_i[chan] * (outrate - d))
/ outrate
)
_put_sample(result, size, out_i, _overflow(cur_o, size))
out_i += 1
d -= inrate
def lin2ulaw(cp, size):
raise NotImplementedError()
def ulaw2lin(cp, size):
raise NotImplementedError()
def lin2alaw(cp, size):
raise NotImplementedError()
def alaw2lin(cp, size):
raise NotImplementedError()
def lin2adpcm(cp, size, state):
raise NotImplementedError()
def adpcm2lin(cp, size, state):
raise NotImplementedError()
@@ -0,0 +1,175 @@
"""
This module provides scipy versions of high_pass_filter, and low_pass_filter
as well as an additional band_pass_filter.
Of course, you will need to install scipy for these to work.
When this module is imported the high and low pass filters from this module
will be used when calling audio_segment.high_pass_filter() and
audio_segment.high_pass_filter() instead of the slower, less powerful versions
provided by pydub.effects.
"""
from scipy.signal import butter, sosfilt
from .utils import (register_pydub_effect,stereo_to_ms,ms_to_stereo)
def _mk_butter_filter(freq, type, order):
"""
Args:
freq: The cutoff frequency for highpass and lowpass filters. For
band filters, a list of [low_cutoff, high_cutoff]
type: "lowpass", "highpass", or "band"
order: nth order butterworth filter (default: 5th order). The
attenuation is -6dB/octave beyond the cutoff frequency (for 1st
order). A Higher order filter will have more attenuation, each level
adding an additional -6dB (so a 3rd order butterworth filter would
be -18dB/octave).
Returns:
function which can filter a mono audio segment
"""
def filter_fn(seg):
assert seg.channels == 1
nyq = 0.5 * seg.frame_rate
try:
freqs = [f / nyq for f in freq]
except TypeError:
freqs = freq / nyq
sos = butter(order, freqs, btype=type, output='sos')
y = sosfilt(sos, seg.get_array_of_samples())
return seg._spawn(y.astype(seg.array_type))
return filter_fn
@register_pydub_effect
def band_pass_filter(seg, low_cutoff_freq, high_cutoff_freq, order=5):
filter_fn = _mk_butter_filter([low_cutoff_freq, high_cutoff_freq], 'band', order=order)
return seg.apply_mono_filter_to_each_channel(filter_fn)
@register_pydub_effect
def high_pass_filter(seg, cutoff_freq, order=5):
filter_fn = _mk_butter_filter(cutoff_freq, 'highpass', order=order)
return seg.apply_mono_filter_to_each_channel(filter_fn)
@register_pydub_effect
def low_pass_filter(seg, cutoff_freq, order=5):
filter_fn = _mk_butter_filter(cutoff_freq, 'lowpass', order=order)
return seg.apply_mono_filter_to_each_channel(filter_fn)
@register_pydub_effect
def _eq(seg, focus_freq, bandwidth=100, mode="peak", gain_dB=0, order=2):
"""
Args:
focus_freq - middle frequency or known frequency of band (in Hz)
bandwidth - range of the equalizer band
mode - Mode of Equalization(Peak/Notch(Bell Curve),High Shelf, Low Shelf)
order - Rolloff factor(1 - 6dB/Octave 2 - 12dB/Octave)
Returns:
Equalized/Filtered AudioSegment
"""
filt_mode = ["peak", "low_shelf", "high_shelf"]
if mode not in filt_mode:
raise ValueError("Incorrect Mode Selection")
if gain_dB >= 0:
if mode == "peak":
sec = band_pass_filter(seg, focus_freq - bandwidth/2, focus_freq + bandwidth/2, order = order)
seg = seg.overlay(sec - (3 - gain_dB))
return seg
if mode == "low_shelf":
sec = low_pass_filter(seg, focus_freq, order=order)
seg = seg.overlay(sec - (3 - gain_dB))
return seg
if mode == "high_shelf":
sec = high_pass_filter(seg, focus_freq, order=order)
seg = seg.overlay(sec - (3 - gain_dB))
return seg
if gain_dB < 0:
if mode == "peak":
sec = high_pass_filter(seg, focus_freq - bandwidth/2, order=order)
seg = seg.overlay(sec - (3 + gain_dB)) + gain_dB
sec = low_pass_filter(seg, focus_freq + bandwidth/2, order=order)
seg = seg.overlay(sec - (3 + gain_dB)) + gain_dB
return seg
if mode == "low_shelf":
sec = high_pass_filter(seg, focus_freq, order=order)
seg = seg.overlay(sec - (3 + gain_dB)) + gain_dB
return seg
if mode=="high_shelf":
sec=low_pass_filter(seg, focus_freq, order=order)
seg=seg.overlay(sec - (3 + gain_dB)) +gain_dB
return seg
@register_pydub_effect
def eq(seg, focus_freq, bandwidth=100, channel_mode="L+R", filter_mode="peak", gain_dB=0, order=2):
"""
Args:
focus_freq - middle frequency or known frequency of band (in Hz)
bandwidth - range of the equalizer band
channel_mode - Select Channels to be affected by the filter.
L+R - Standard Stereo Filter
L - Only Left Channel is Filtered
R - Only Right Channel is Filtered
M+S - Blumlien Stereo Filter(Mid-Side)
M - Only Mid Channel is Filtered
S - Only Side Channel is Filtered
Mono Audio Segments are completely filtered.
filter_mode - Mode of Equalization(Peak/Notch(Bell Curve),High Shelf, Low Shelf)
order - Rolloff factor(1 - 6dB/Octave 2 - 12dB/Octave)
Returns:
Equalized/Filtered AudioSegment
"""
channel_modes = ["L+R", "M+S", "L", "R", "M", "S"]
if channel_mode not in channel_modes:
raise ValueError("Incorrect Channel Mode Selection")
if seg.channels == 1:
return _eq(seg, focus_freq, bandwidth, filter_mode, gain_dB, order)
if channel_mode == "L+R":
return _eq(seg, focus_freq, bandwidth, filter_mode, gain_dB, order)
if channel_mode == "L":
seg = seg.split_to_mono()
seg = [_eq(seg[0], focus_freq, bandwidth, filter_mode, gain_dB, order), seg[1]]
return AudioSegment.from_mono_audio_segements(seg[0], seg[1])
if channel_mode == "R":
seg = seg.split_to_mono()
seg = [seg[0], _eq(seg[1], focus_freq, bandwidth, filter_mode, gain_dB, order)]
return AudioSegment.from_mono_audio_segements(seg[0], seg[1])
if channel_mode == "M+S":
seg = stereo_to_ms(seg)
seg = _eq(seg, focus_freq, bandwidth, filter_mode, gain_dB, order)
return ms_to_stereo(seg)
if channel_mode == "M":
seg = stereo_to_ms(seg).split_to_mono()
seg = [_eq(seg[0], focus_freq, bandwidth, filter_mode, gain_dB, order), seg[1]]
seg = AudioSegment.from_mono_audio_segements(seg[0], seg[1])
return ms_to_stereo(seg)
if channel_mode == "S":
seg = stereo_to_ms(seg).split_to_mono()
seg = [seg[0], _eq(seg[1], focus_freq, bandwidth, filter_mode, gain_dB, order)]
seg = AudioSegment.from_mono_audio_segements(seg[0], seg[1])
return ms_to_stereo(seg)
+182
View File
@@ -0,0 +1,182 @@
"""
Various functions for finding/manipulating silence in AudioSegments
"""
import itertools
from .utils import db_to_float
def detect_silence(audio_segment, min_silence_len=1000, silence_thresh=-16, seek_step=1):
"""
Returns a list of all silent sections [start, end] in milliseconds of audio_segment.
Inverse of detect_nonsilent()
audio_segment - the segment to find silence in
min_silence_len - the minimum length for any silent section
silence_thresh - the upper bound for how quiet is silent in dFBS
seek_step - step size for interating over the segment in ms
"""
seg_len = len(audio_segment)
# you can't have a silent portion of a sound that is longer than the sound
if seg_len < min_silence_len:
return []
# convert silence threshold to a float value (so we can compare it to rms)
silence_thresh = db_to_float(silence_thresh) * audio_segment.max_possible_amplitude
# find silence and add start and end indicies to the to_cut list
silence_starts = []
# check successive (1 sec by default) chunk of sound for silence
# try a chunk at every "seek step" (or every chunk for a seek step == 1)
last_slice_start = seg_len - min_silence_len
slice_starts = range(0, last_slice_start + 1, seek_step)
# guarantee last_slice_start is included in the range
# to make sure the last portion of the audio is searched
if last_slice_start % seek_step:
slice_starts = itertools.chain(slice_starts, [last_slice_start])
for i in slice_starts:
audio_slice = audio_segment[i:i + min_silence_len]
if audio_slice.rms <= silence_thresh:
silence_starts.append(i)
# short circuit when there is no silence
if not silence_starts:
return []
# combine the silence we detected into ranges (start ms - end ms)
silent_ranges = []
prev_i = silence_starts.pop(0)
current_range_start = prev_i
for silence_start_i in silence_starts:
continuous = (silence_start_i == prev_i + seek_step)
# sometimes two small blips are enough for one particular slice to be
# non-silent, despite the silence all running together. Just combine
# the two overlapping silent ranges.
silence_has_gap = silence_start_i > (prev_i + min_silence_len)
if not continuous and silence_has_gap:
silent_ranges.append([current_range_start,
prev_i + min_silence_len])
current_range_start = silence_start_i
prev_i = silence_start_i
silent_ranges.append([current_range_start,
prev_i + min_silence_len])
return silent_ranges
def detect_nonsilent(audio_segment, min_silence_len=1000, silence_thresh=-16, seek_step=1):
"""
Returns a list of all nonsilent sections [start, end] in milliseconds of audio_segment.
Inverse of detect_silent()
audio_segment - the segment to find silence in
min_silence_len - the minimum length for any silent section
silence_thresh - the upper bound for how quiet is silent in dFBS
seek_step - step size for interating over the segment in ms
"""
silent_ranges = detect_silence(audio_segment, min_silence_len, silence_thresh, seek_step)
len_seg = len(audio_segment)
# if there is no silence, the whole thing is nonsilent
if not silent_ranges:
return [[0, len_seg]]
# short circuit when the whole audio segment is silent
if silent_ranges[0][0] == 0 and silent_ranges[0][1] == len_seg:
return []
prev_end_i = 0
nonsilent_ranges = []
for start_i, end_i in silent_ranges:
nonsilent_ranges.append([prev_end_i, start_i])
prev_end_i = end_i
if end_i != len_seg:
nonsilent_ranges.append([prev_end_i, len_seg])
if nonsilent_ranges[0] == [0, 0]:
nonsilent_ranges.pop(0)
return nonsilent_ranges
def split_on_silence(audio_segment, min_silence_len=1000, silence_thresh=-16, keep_silence=100,
seek_step=1):
"""
Returns list of audio segments from splitting audio_segment on silent sections
audio_segment - original pydub.AudioSegment() object
min_silence_len - (in ms) minimum length of a silence to be used for
a split. default: 1000ms
silence_thresh - (in dBFS) anything quieter than this will be
considered silence. default: -16dBFS
keep_silence - (in ms or True/False) leave some silence at the beginning
and end of the chunks. Keeps the sound from sounding like it
is abruptly cut off.
When the length of the silence is less than the keep_silence duration
it is split evenly between the preceding and following non-silent
segments.
If True is specified, all the silence is kept, if False none is kept.
default: 100ms
seek_step - step size for interating over the segment in ms
"""
# from the itertools documentation
def pairwise(iterable):
"s -> (s0,s1), (s1,s2), (s2, s3), ..."
a, b = itertools.tee(iterable)
next(b, None)
return zip(a, b)
if isinstance(keep_silence, bool):
keep_silence = len(audio_segment) if keep_silence else 0
output_ranges = [
[ start - keep_silence, end + keep_silence ]
for (start,end)
in detect_nonsilent(audio_segment, min_silence_len, silence_thresh, seek_step)
]
for range_i, range_ii in pairwise(output_ranges):
last_end = range_i[1]
next_start = range_ii[0]
if next_start < last_end:
range_i[1] = (last_end+next_start)//2
range_ii[0] = range_i[1]
return [
audio_segment[ max(start,0) : min(end,len(audio_segment)) ]
for start,end in output_ranges
]
def detect_leading_silence(sound, silence_threshold=-50.0, chunk_size=10):
"""
Returns the millisecond/index that the leading silence ends.
audio_segment - the segment to find silence in
silence_threshold - the upper bound for how quiet is silent in dFBS
chunk_size - chunk size for interating over the segment in ms
"""
trim_ms = 0 # ms
assert chunk_size > 0 # to avoid infinite loop
while sound[trim_ms:trim_ms+chunk_size].dBFS < silence_threshold and trim_ms < len(sound):
trim_ms += chunk_size
# if there is no end it should return the length of the segment
return min(trim_ms, len(sound))
+440
View File
@@ -0,0 +1,440 @@
from __future__ import division
from io import BufferedReader
import json
import os
import re
import sys
from subprocess import Popen, PIPE
from math import log, ceil
from tempfile import TemporaryFile
from warnings import warn
from functools import wraps
try:
import audioop
except ImportError:
import pyaudioop as audioop
if sys.version_info >= (3, 0):
basestring = str
FRAME_WIDTHS = {
8: 1,
16: 2,
32: 4,
}
ARRAY_TYPES = {
8: "b",
16: "h",
32: "i",
}
ARRAY_RANGES = {
8: (-0x80, 0x7f),
16: (-0x8000, 0x7fff),
32: (-0x80000000, 0x7fffffff),
}
def get_frame_width(bit_depth):
return FRAME_WIDTHS[bit_depth]
def get_array_type(bit_depth, signed=True):
t = ARRAY_TYPES[bit_depth]
if not signed:
t = t.upper()
return t
def get_min_max_value(bit_depth):
return ARRAY_RANGES[bit_depth]
def _fd_or_path_or_tempfile(fd, mode='w+b', tempfile=True):
close_fd = False
if fd is None and tempfile:
fd = TemporaryFile(mode=mode)
close_fd = True
if isinstance(fd, basestring):
fd = open(fd, mode=mode)
close_fd = True
if isinstance(fd, BufferedReader):
close_fd = True
try:
if isinstance(fd, os.PathLike):
fd = open(fd, mode=mode)
close_fd = True
except AttributeError:
# module os has no attribute PathLike, so we're on python < 3.6.
# The protocol we're trying to support doesn't exist, so just pass.
pass
return fd, close_fd
def db_to_float(db, using_amplitude=True):
"""
Converts the input db to a float, which represents the equivalent
ratio in power.
"""
db = float(db)
if using_amplitude:
return 10 ** (db / 20)
else: # using power
return 10 ** (db / 10)
def ratio_to_db(ratio, val2=None, using_amplitude=True):
"""
Converts the input float to db, which represents the equivalent
to the ratio in power represented by the multiplier passed in.
"""
ratio = float(ratio)
# accept 2 values and use the ratio of val1 to val2
if val2 is not None:
ratio = ratio / val2
# special case for multiply-by-zero (convert to silence)
if ratio == 0:
return -float('inf')
if using_amplitude:
return 20 * log(ratio, 10)
else: # using power
return 10 * log(ratio, 10)
def register_pydub_effect(fn, name=None):
"""
decorator for adding pydub effects to the AudioSegment objects.
example use:
@register_pydub_effect
def normalize(audio_segment):
...
or you can specify a name:
@register_pydub_effect("normalize")
def normalize_audio_segment(audio_segment):
...
"""
if isinstance(fn, basestring):
name = fn
return lambda fn: register_pydub_effect(fn, name)
if name is None:
name = fn.__name__
from .audio_segment import AudioSegment
setattr(AudioSegment, name, fn)
return fn
def make_chunks(audio_segment, chunk_length):
"""
Breaks an AudioSegment into chunks that are <chunk_length> milliseconds
long.
if chunk_length is 50 then you'll get a list of 50 millisecond long audio
segments back (except the last one, which can be shorter)
"""
number_of_chunks = ceil(len(audio_segment) / float(chunk_length))
return [audio_segment[i * chunk_length:(i + 1) * chunk_length]
for i in range(int(number_of_chunks))]
def which(program):
"""
Mimics behavior of UNIX which command.
"""
# Add .exe program extension for windows support
if os.name == "nt" and not program.endswith(".exe"):
program += ".exe"
envdir_list = [os.curdir] + os.environ["PATH"].split(os.pathsep)
for envdir in envdir_list:
program_path = os.path.join(envdir, program)
if os.path.isfile(program_path) and os.access(program_path, os.X_OK):
return program_path
def get_encoder_name():
"""
Return enconder default application for system, either avconv or ffmpeg
"""
if which("avconv"):
return "avconv"
elif which("ffmpeg"):
return "ffmpeg"
else:
# should raise exception
warn("Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work", RuntimeWarning)
return "ffmpeg"
def get_player_name():
"""
Return enconder default application for system, either avconv or ffmpeg
"""
if which("avplay"):
return "avplay"
elif which("ffplay"):
return "ffplay"
else:
# should raise exception
warn("Couldn't find ffplay or avplay - defaulting to ffplay, but may not work", RuntimeWarning)
return "ffplay"
def get_prober_name():
"""
Return probe application, either avconv or ffmpeg
"""
if which("avprobe"):
return "avprobe"
elif which("ffprobe"):
return "ffprobe"
else:
# should raise exception
warn("Couldn't find ffprobe or avprobe - defaulting to ffprobe, but may not work", RuntimeWarning)
return "ffprobe"
def fsdecode(filename):
"""Wrapper for os.fsdecode which was introduced in python 3.2 ."""
if sys.version_info >= (3, 2):
PathLikeTypes = (basestring, bytes)
if sys.version_info >= (3, 6):
PathLikeTypes += (os.PathLike,)
if isinstance(filename, PathLikeTypes):
return os.fsdecode(filename)
else:
if isinstance(filename, bytes):
return filename.decode(sys.getfilesystemencoding())
if isinstance(filename, basestring):
return filename
raise TypeError("type {0} not accepted by fsdecode".format(type(filename)))
def get_extra_info(stderr):
"""
avprobe sometimes gives more information on stderr than
on the json output. The information has to be extracted
from stderr of the format of:
' Stream #0:0: Audio: flac, 88200 Hz, stereo, s32 (24 bit)'
or (macOS version):
' Stream #0:0: Audio: vorbis'
' 44100 Hz, stereo, fltp, 320 kb/s'
:type stderr: str
:rtype: list of dict
"""
extra_info = {}
re_stream = r'(?P<space_start> +)Stream #0[:\.](?P<stream_id>([0-9]+))(?P<content_0>.+)\n?(?! *Stream)((?P<space_end> +)(?P<content_1>.+))?'
for i in re.finditer(re_stream, stderr):
if i.group('space_end') is not None and len(i.group('space_start')) <= len(
i.group('space_end')):
content_line = ','.join([i.group('content_0'), i.group('content_1')])
else:
content_line = i.group('content_0')
tokens = [x.strip() for x in re.split('[:,]', content_line) if x]
extra_info[int(i.group('stream_id'))] = tokens
return extra_info
def mediainfo_json(filepath, read_ahead_limit=-1):
"""Return json dictionary with media info(codec, duration, size, bitrate...) from filepath
"""
prober = get_prober_name()
command_args = [
"-v", "info",
"-show_format",
"-show_streams",
]
try:
command_args += [fsdecode(filepath)]
stdin_parameter = None
stdin_data = None
except TypeError:
if prober == 'ffprobe':
command_args += ["-read_ahead_limit", str(read_ahead_limit),
"cache:pipe:0"]
else:
command_args += ["-"]
stdin_parameter = PIPE
file, close_file = _fd_or_path_or_tempfile(filepath, 'rb', tempfile=False)
file.seek(0)
stdin_data = file.read()
if close_file:
file.close()
command = [prober, '-of', 'json'] + command_args
res = Popen(command, stdin=stdin_parameter, stdout=PIPE, stderr=PIPE)
output, stderr = res.communicate(input=stdin_data)
output = output.decode("utf-8", 'ignore')
stderr = stderr.decode("utf-8", 'ignore')
try:
info = json.loads(output)
except json.decoder.JSONDecodeError:
# If ffprobe didn't give any information, just return it
# (for example, because the file doesn't exist)
return None
if not info:
return info
extra_info = get_extra_info(stderr)
audio_streams = [x for x in info['streams'] if x['codec_type'] == 'audio']
if len(audio_streams) == 0:
return info
# We just operate on the first audio stream in case there are more
stream = audio_streams[0]
def set_property(stream, prop, value):
if prop not in stream or stream[prop] == 0:
stream[prop] = value
for token in extra_info[stream['index']]:
m = re.match(r'([su]([0-9]{1,2})p?) \(([0-9]{1,2}) bit\)$', token)
m2 = re.match(r'([su]([0-9]{1,2})p?)( \(default\))?$', token)
if m:
set_property(stream, 'sample_fmt', m.group(1))
set_property(stream, 'bits_per_sample', int(m.group(2)))
set_property(stream, 'bits_per_raw_sample', int(m.group(3)))
elif m2:
set_property(stream, 'sample_fmt', m2.group(1))
set_property(stream, 'bits_per_sample', int(m2.group(2)))
set_property(stream, 'bits_per_raw_sample', int(m2.group(2)))
elif re.match(r'(flt)p?( \(default\))?$', token):
set_property(stream, 'sample_fmt', token)
set_property(stream, 'bits_per_sample', 32)
set_property(stream, 'bits_per_raw_sample', 32)
elif re.match(r'(dbl)p?( \(default\))?$', token):
set_property(stream, 'sample_fmt', token)
set_property(stream, 'bits_per_sample', 64)
set_property(stream, 'bits_per_raw_sample', 64)
return info
def mediainfo(filepath):
"""Return dictionary with media info(codec, duration, size, bitrate...) from filepath
"""
prober = get_prober_name()
command_args = [
"-v", "quiet",
"-show_format",
"-show_streams",
filepath
]
command = [prober, '-of', 'old'] + command_args
res = Popen(command, stdout=PIPE)
output = res.communicate()[0].decode("utf-8")
if res.returncode != 0:
command = [prober] + command_args
output = Popen(command, stdout=PIPE).communicate()[0].decode("utf-8")
rgx = re.compile(r"(?:(?P<inner_dict>.*?):)?(?P<key>.*?)\=(?P<value>.*?)$")
info = {}
if sys.platform == 'win32':
output = output.replace("\r", "")
for line in output.split("\n"):
# print(line)
mobj = rgx.match(line)
if mobj:
# print(mobj.groups())
inner_dict, key, value = mobj.groups()
if inner_dict:
try:
info[inner_dict]
except KeyError:
info[inner_dict] = {}
info[inner_dict][key] = value
else:
info[key] = value
return info
def cache_codecs(function):
cache = {}
@wraps(function)
def wrapper():
try:
return cache[0]
except:
cache[0] = function()
return cache[0]
return wrapper
@cache_codecs
def get_supported_codecs():
encoder = get_encoder_name()
command = [encoder, "-codecs"]
res = Popen(command, stdout=PIPE, stderr=PIPE)
output = res.communicate()[0].decode("utf-8")
if res.returncode != 0:
return []
if sys.platform == 'win32':
output = output.replace("\r", "")
rgx = re.compile(r"^([D.][E.][AVS.][I.][L.][S.]) (\w*) +(.*)")
decoders = set()
encoders = set()
for line in output.split('\n'):
match = rgx.match(line.strip())
if not match:
continue
flags, codec, name = match.groups()
if flags[0] == 'D':
decoders.add(codec)
if flags[1] == 'E':
encoders.add(codec)
return (decoders, encoders)
def get_supported_decoders():
return get_supported_codecs()[0]
def get_supported_encoders():
return get_supported_codecs()[1]
def stereo_to_ms(audio_segment):
'''
Left-Right -> Mid-Side
'''
channel = audio_segment.split_to_mono()
channel = [channel[0].overlay(channel[1]), channel[0].overlay(channel[1].invert_phase())]
return AudioSegment.from_mono_audiosegments(channel[0], channel[1])
def ms_to_stereo(audio_segment):
'''
Mid-Side -> Left-Right
'''
channel = audio_segment.split_to_mono()
channel = [channel[0].overlay(channel[1]) - 3, channel[0].overlay(channel[1].invert_phase()) - 3]
return AudioSegment.from_mono_audiosegments(channel[0], channel[1])
+5
View File
@@ -0,0 +1,5 @@
[wheel]
universal = 1
[pep8]
max-line-length = 100
+42
View File
@@ -0,0 +1,42 @@
__doc__ = """
Manipulate audio with an simple and easy high level interface.
See the README file for details, usage info, and a list of gotchas.
"""
from setuptools import setup
setup(
name='pydub',
version='0.25.1',
author='James Robert',
author_email='jiaaro@gmail.com',
description='Manipulate audio with an simple and easy high level interface',
license='MIT',
keywords='audio sound high-level',
url='http://pydub.com',
packages=['pydub'],
long_description=__doc__,
classifiers=[
'Development Status :: 5 - Production/Stable',
'License :: OSI Approved :: MIT License',
'Programming Language :: Python',
'Programming Language :: Python :: 2',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Intended Audience :: Developers',
'Operating System :: OS Independent',
"Topic :: Multimedia :: Sound/Audio",
"Topic :: Multimedia :: Sound/Audio :: Analysis",
"Topic :: Multimedia :: Sound/Audio :: Conversion",
"Topic :: Multimedia :: Sound/Audio :: Editors",
"Topic :: Multimedia :: Sound/Audio :: Mixers",
"Topic :: Software Development :: Libraries",
'Topic :: Utilities',
]
)
+29
View File
@@ -0,0 +1,29 @@
"""
Recipes are a way to create mixes.
"""
from abc import ABC, abstractmethod
from typing import Any, List
class HomepageRoutine(ABC):
"""
A routine creates a row of homepage items.
"""
@property
@abstractmethod
def is_valid(self) -> bool: ...
def __init__(self) -> None:
if not self.is_valid:
return
self.run()
@abstractmethod
def run(self) -> List[Any]:
"""
Creates the homepage items and saves them to the
homepage store if self.is_valid is true.
"""
...
+38
View File
@@ -0,0 +1,38 @@
from swingmusic.db.userdata import UserTable
from swingmusic.lib.recipes import HomepageRoutine
from swingmusic.plugins.mixes import MixesPlugin
from swingmusic.store.homepage import HomepageStore
class ArtistMixes(HomepageRoutine):
store_key = "artist_mixes"
@property
def is_valid(self):
return MixesPlugin().enabled
def run(self):
users = UserTable.get_all()
for user in users:
mix = MixesPlugin()
mixes = mix.create_artist_mixes(user.id)
if not mixes:
continue
HomepageStore.set_mixes(mixes, entrykey=self.store_key, userid=user.id)
custom_mixes = []
for _mix in mixes:
custom_mix = MixesPlugin.get_track_mix(_mix)
if custom_mix:
custom_mixes.append(custom_mix)
HomepageStore.set_mixes(
custom_mixes, entrykey="custom_mixes", userid=user.id
)
def __init__(self) -> None:
super().__init__()
+40
View File
@@ -0,0 +1,40 @@
from pprint import pprint
from swingmusic.db.userdata import UserTable
from swingmusic.lib.recipes import HomepageRoutine
from swingmusic.lib.recipes.artistmixes import ArtistMixes
from swingmusic.models.mix import Mix
from swingmusic.plugins.mixes import MixesPlugin
from swingmusic.store.homepage import HomepageStore
class BecauseYouListened(HomepageRoutine):
store_keys = ["because_you_listened_to_artist", "artists_you_might_like"]
@property
def is_valid(self):
return MixesPlugin().enabled
def run(self):
users = UserTable.get_all()
for user in users:
entry: dict[str, Mix] = HomepageStore.entries.get(
ArtistMixes.store_key
).items.get(user.id) # type: ignore
if not entry:
continue
because_you_listened_to_artist, artists_you_might_like = (
MixesPlugin().get_because_items(list(entry.values()))
)
if not because_you_listened_to_artist or not artists_you_might_like:
continue
HomepageStore.entries[self.store_keys[0]].items[
user.id
] = because_you_listened_to_artist
HomepageStore.entries[self.store_keys[1]].items[
user.id
] = artists_you_might_like
+96
View File
@@ -0,0 +1,96 @@
from swingmusic.db.userdata import ScrobbleTable, UserTable
from swingmusic.lib.home.recentlyadded import get_recently_added_items
from swingmusic.lib.home.get_recently_played import get_recently_played
from swingmusic.lib.recipes import HomepageRoutine
from swingmusic.store.homepage import HomepageStore
class RecentlyPlayed(HomepageRoutine):
ITEM_LIMIT = 15
store_key = "recently_played"
def __init__(self, userid: int | None = None) -> None:
"""
The userid is provided when we are running this routine
outside a cron job. ie. when a user records a new scrobble.
"""
self.userids = [userid] if userid else [user.id for user in UserTable.get_all()]
# NOTE: When the userid is provided
# we need to update the store for that userid only
# using the last scrobble entry.
self.update_only = userid is not None
super().__init__()
@property
def is_valid(self):
return True
def run(self):
if self.update_only:
last_entry = ScrobbleTable.get_last_entry(self.userids[0])
if last_entry:
items = get_recently_played(
limit=self.ITEM_LIMIT, userid=self.userids[0], _entries=[last_entry]
)
try:
item = items[0]
store_entry = HomepageStore.entries[self.store_key].items[
self.userids[0]
][0]
except IndexError:
store_entry = None
item = None
if (
store_entry
and item
and store_entry.get("type", "") + store_entry.get("hash", "")
== item.get("type", "") + item.get("hash", "")
):
# If the item is the same as the one in the store
# only update the timestamp
HomepageStore.entries[self.store_key].items[self.userids[0]][0][
"timestamp"
] = item["timestamp"]
else:
# Otherwise, insert the new item
# and remove the oldest item if there are more than 15 items
HomepageStore.entries[self.store_key].items[self.userids[0]].insert(
0, item
)
if (
len(
HomepageStore.entries[self.store_key].items[self.userids[0]]
)
> self.ITEM_LIMIT
):
HomepageStore.entries[self.store_key].items[
self.userids[0]
].pop()
for userid in self.userids:
items = get_recently_played(limit=self.ITEM_LIMIT, userid=userid)
HomepageStore.entries[self.store_key].items[userid] = items
class RecentlyAdded(HomepageRoutine):
ITEM_LIMIT = 15
store_key = "recently_added"
@property
def is_valid(self):
return True
def __init__(self):
super().__init__()
def run(self):
items = get_recently_added_items(limit=self.ITEM_LIMIT)
# NOTE: Recently added is a global entry
# So we don't need a userid
HomepageStore.entries[self.store_key].items[0] = items
+83
View File
@@ -0,0 +1,83 @@
from gettext import ngettext
import pendulum
from swingmusic.crons.cron import CronJob
from swingmusic.db.userdata import UserTable
from swingmusic.lib.recipes import HomepageRoutine
from swingmusic.store.homepage import HomepageStore
from swingmusic.utils.dates import get_date_range, seconds_to_time_string
from swingmusic.utils.stats import get_artists_in_period
class TopArtists(CronJob, HomepageRoutine):
"""
A routine to populate the top streamed artists/albums in the last week or month
"""
hours = 1
ITEM_LIMIT = 15
@property
def is_valid(self):
"""
Only valid if it's the middle or last 2 days of this month.
When the duration is "week", it's valid on the weekend.
"""
if self.duration == "month":
now = pendulum.now()
middle_day = now.days_in_month // 2
return (
now.day in range(middle_day, middle_day + 2)
or now.day > now.days_in_month - 2
)
if self.duration == "week":
return pendulum.now().isoweekday() in (5, 6, 7)
return False
def __init__(self, duration: str = "month") -> None:
super().__init__()
self.duration = duration
if not self.is_valid:
return
def run(self):
if not self.is_valid:
self.destroy()
return
self.userids = [user.id for user in UserTable.get_all()]
for userid in self.userids:
date_range = get_date_range(self.duration)
artists = get_artists_in_period(date_range[0], date_range[1], userid)[
: self.ITEM_LIMIT
]
artists = [
{
"type": "artist",
"hash": artist["artisthash"],
"help_text": seconds_to_time_string(artist["playduration"]),
"secondary_text": str(artist["playcount"])
+ " "
+ ngettext("play", "plays", artist["playcount"]),
}
for artist in artists
]
HomepageStore.entries[f"top_streamed_{self.duration}ly_artists"].items[
userid
] = artists
def destroy(self):
"""
Clear the top streamed entry from the homepage store.
"""
keys = [f"top_streamed_{self.duration}ly_artists"]
for key in keys:
HomepageStore.entries[key].items = {}
+319
View File
@@ -0,0 +1,319 @@
"""
This library contains all the functions related to the search functionality.
"""
from rapidfuzz import process, utils, fuzz
from unidecode import unidecode
from swingmusic import models
from swingmusic.models.album import Album
from swingmusic.models.artist import Artist
from swingmusic.models.playlist import Playlist
from swingmusic.models.track import Track
from swingmusic.serializers.album import serialize_for_card as serialize_album
from swingmusic.serializers.album import serialize_for_card_many as serialize_albums
from swingmusic.serializers.artist import serialize_for_card, serialize_for_cards
from swingmusic.serializers.track import serialize_track, serialize_tracks
from swingmusic.store.albums import AlbumStore
from swingmusic.store.artists import ArtistStore
from swingmusic.store.tracks import TrackStore
from swingmusic.utils.remove_duplicates import remove_duplicates
# ratio = fuzz.ratio
# wratio = fuzz.WRatio
class Cutoff:
"""
Holds all the default cutoff values.
"""
tracks: int = 50
albums: int = 50
artists: int = 50
playlists: int = 50
class Limit:
"""
Holds all the default limit values.
"""
tracks: int = 150
albums: int = 150
artists: int = 150
playlists: int = 150
class SearchTracks:
def __init__(self, query: str) -> None:
self.query = query
self.tracks = TrackStore.get_flat_list()
def __call__(self, limit: int = Limit.tracks) -> list[models.Track]:
"""
Gets all songs with a given title.
"""
track_titles = [unidecode(track.title).lower() for track in self.tracks]
results = process.extract(
self.query,
track_titles,
score_cutoff=Cutoff.tracks,
limit=limit,
processor=utils.default_process,
scorer=fuzz.WRatio,
)
tracks: list[Track] = []
for item in results:
track = self.tracks[item[2]]
track._score = item[1]
tracks.append(track)
return remove_duplicates(tracks)
class SearchArtists:
def __init__(self, query: str) -> None:
self.query = query
self.artists = ArtistStore.get_flat_list()
def __call__(self, limit: int = Limit.artists):
"""
Gets all artists with a given name.
"""
choices = [unidecode(a.name).lower() for a in self.artists]
results = process.extract(
self.query,
choices,
score_cutoff=Cutoff.artists,
limit=limit,
processor=utils.default_process,
scorer=fuzz.WRatio,
)
artists: list[Artist] = []
for item in results:
artist = self.artists[item[2]]
artist._score = item[1]
artists.append(artist)
return artists
class SearchAlbums:
def __init__(self, query: str) -> None:
self.query = query
self.albums = AlbumStore.get_flat_list()
def __call__(self, limit: int = Limit.albums):
"""
Gets all albums with a given title.
"""
choices = [unidecode(a.title).lower() for a in self.albums]
results = process.extract(
self.query,
choices,
score_cutoff=Cutoff.albums,
limit=limit,
processor=utils.default_process,
scorer=fuzz.token_sort_ratio,
)
albums: list[Album] = []
for item in results:
album = self.albums[item[2]]
album._score = item[1]
albums.append(album)
return albums
class SearchPlaylists:
def __init__(self, playlists: list[models.Playlist], query: str) -> None:
self.playlists = playlists
self.query = query
def __call__(self, limit: int = Limit.playlists):
choices = [p.name for p in self.playlists]
results = process.extract(
self.query,
choices,
score_cutoff=Cutoff.playlists,
limit=limit,
processor=utils.default_process,
scorer=fuzz.WRatio,
)
playlists: list[Playlist] = []
for item in results:
playlist = self.playlists[item[2]]
playlist._score = item[1]
playlists.append(playlist)
return playlists
_type = models.Track | models.Album | models.Artist
def get_titles(items: list[_type]):
for item in items:
if isinstance(item, models.Track):
text = item.og_title
elif isinstance(item, models.Album):
text = item.title
elif isinstance(item, models.Artist):
text = item.name
else:
text = None
yield text
class TopResults:
"""
Joins all tracks, albums and artists
then fuzzy searches them as a single unit.
"""
@staticmethod
def collect_all():
all_items: list[_type] = []
all_items.extend(ArtistStore.get_flat_list())
all_items.extend(TrackStore.get_flat_list())
all_items.extend(AlbumStore.get_flat_list())
return all_items, get_titles(all_items)
@staticmethod
def get_track_items(item: Track | Album | Artist, limit=5):
tracks: list[Track] = []
# INFO: If the item is a track, return empty list
# to be filled by the results from the top search
if isinstance(item, Track):
return tracks
# INFO: If the item is an album, get the tracks from the album
if isinstance(item, Album):
tracks = TrackStore.get_tracks_by_albumhash(item.albumhash)[:limit]
tracks.sort(key=lambda x: x.playduration, reverse=True)
return tracks
# INFO: If the item is an artist, get the tracks from the artist
if isinstance(item, Artist):
tracks = TrackStore.get_tracks_by_artisthash(item.artisthash)[:limit]
tracks.sort(key=lambda x: x.playduration, reverse=True)
return tracks
@staticmethod
def get_album_items(item: Track | Album | Artist, limit=6):
albums: list[Album] = []
# INFO: If the item is a track or album, search for albums
if isinstance(item, Track) or isinstance(item, Album):
return albums
# INFO: If the item is an artist, get the albums from the artist
if isinstance(item, Artist):
albums = AlbumStore.get_albums_by_artisthash(item.artisthash)[:limit]
return albums
@staticmethod
def search(
query: str,
limit: int = None,
albums_only=False,
tracks_only=False,
):
tracks_limit = Limit.tracks if tracks_only else 4
albums_limit = Limit.albums if albums_only else limit
artists_limit = limit
# INFO: Individually search all stores as each type has a different scorer
tracks = SearchTracks(query)(limit=tracks_limit) if not albums_only else []
albums = SearchAlbums(query)(limit=albums_limit)
artists = SearchArtists(query)(limit=artists_limit)
# INFO: Combine all results and sort them by score
all_results = artists + tracks + albums
all_results = sorted(all_results, key=lambda x: int(x._score), reverse=True)
# INFO: Get the top result
top_result = all_results[0]
top_tracks = []
if not albums_only:
top_tracks = TopResults.get_track_items(top_result, limit=tracks_limit)
# INFO: If there are not enough tracks, fill with search results
if len(top_tracks) < tracks_limit:
found_tracks_set = {track.trackhash for track in top_tracks}
for track in tracks:
if track.trackhash not in found_tracks_set:
top_tracks.append(track)
if len(top_tracks) >= tracks_limit:
break
top_tracks = serialize_tracks(top_tracks)
if tracks_only:
return top_tracks
top_albums = TopResults.get_album_items(top_result, limit=albums_limit)
# INFO: If there are not enough albums, fill with search results
if len(top_albums) < albums_limit:
found_albums_set = {album.albumhash for album in top_albums}
for album in albums:
if album.albumhash not in found_albums_set:
top_albums.append(album)
if len(top_albums) >= albums_limit:
break
top_albums = serialize_albums(top_albums)
if albums_only:
return top_albums
artists = serialize_for_cards(artists)
if isinstance(top_result, Track):
top_result = serialize_track(top_result)
top_result["type"] = "track"
if isinstance(top_result, Album):
top_result = serialize_album(top_result)
top_result["type"] = "album"
if isinstance(top_result, Artist):
top_result = serialize_for_card(
top_result, include={"albumcount", "trackcount"}
)
top_result["type"] = "artist"
return {
"top_result": top_result,
"tracks": top_tracks,
"artists": artists,
"albums": top_albums,
}
+55
View File
@@ -0,0 +1,55 @@
from itertools import groupby
import os
from typing import Callable
from swingmusic.lib.albumslib import sort_by_track_no
from swingmusic.models.folder import Folder
from swingmusic.models.track import Track
from swingmusic.utils import flatten
def sort_tracks(tracks: list[Track], key: str, reverse: bool = False):
"""
Sorts a list of tracks by a key.
"""
if key == "default":
return tracks
sortfunc: Callable[[Track], str] = lambda track: getattr(track, key)
if key == "artists" or key == "albumartists":
sortfunc = lambda track: getattr(track, key)[0]["name"]
if key == "disc":
# INFO: Group tracks into albums, then sort them by disc number.
tracks = sorted(tracks, key=lambda x: x.album.casefold())
groups = groupby(tracks, lambda x: x.albumhash)
return flatten([sort_by_track_no(list(g)) for k, g in groups])
# INFO: sort tracks by title for a fallback value
tracks = sorted(tracks, key=lambda t: t.title.casefold())
if key == "title" and not reverse:
return tracks
return sorted(
tracks,
key=lambda track: sortfunc(track).casefold()
if isinstance(sortfunc(track), str)
else sortfunc(track),
reverse=reverse,
)
def sort_folders(folders: list[Folder], key: str, reverse: bool = False):
"""
Sorts a list of folders by a key.
"""
if key == "default":
return folders
sortfunc: Callable[[Folder], str | float] = lambda folder: getattr(folder, key)
if key == "lastmod":
sortfunc = lambda folder: os.path.getmtime(folder.path)
return sorted(folders, key=sortfunc, reverse=reverse)
+332
View File
@@ -0,0 +1,332 @@
import os
from functools import partial
from multiprocessing import Pool, cpu_count
from swingmusic import settings
from swingmusic.config import UserConfig
from swingmusic.db.libdata import TrackTable
from swingmusic.lib.taglib import extract_thumb, get_tags
from swingmusic.models.album import Album
from swingmusic.models.artist import Artist
from swingmusic.models.track import Track
from swingmusic.store.folder import FolderStore
from swingmusic.store.tracks import TrackStore
from swingmusic.utils import flatten
from swingmusic.utils.filesystem import run_fast_scandir
from swingmusic.utils.parsers import get_base_album_title
from swingmusic.utils.progressbar import tqdm
from swingmusic.utils.remove_duplicates import remove_duplicates
from logging import getLogger
log = getLogger(__name__)
def parse_file_tags(file: str, config: UserConfig) -> dict | None:
"""Worker function to process individual files"""
try:
return get_tags(file, config=config)
except Exception as e:
log.warning(f"Failed to process file {file}: {e}")
return None
class IndexTracks:
def __init__(self) -> None:
"""
Indexes all tracks in the database.
An instance key is used to prevent multiple instances of the
same class from running at the same time.
"""
dirs_to_scan = UserConfig().rootDirs
if len(dirs_to_scan) == 0:
log.warning(
(
"The root directory is not configured. "
+ "Open the app in your webbrowser to configure."
)
)
return
try:
if dirs_to_scan[0] == "$home":
dirs_to_scan = [settings.Paths().USER_HOME_DIR.as_posix()]
except IndexError:
pass
files = set()
for _dir in dirs_to_scan:
files = files.union(run_fast_scandir(_dir, full=True)[1])
unmodified, modified_tracks = self.filter_modded()
untagged = files - unmodified
self.tag_untagged(untagged)
self.extract_thumb_with_overwrite(modified_tracks)
@staticmethod
def extract_thumb_with_overwrite(tracks: list[dict[str, str]]):
"""
Extracts the thumbnail from a list of filepaths,
overwriting the existing thumbnail if it exists,
for modified files.
"""
for track in tracks:
try:
extract_thumb(
track["filepath"], track["albumhash"] + ".webp", overwrite=True, paths=settings.Paths()
)
except FileNotFoundError:
continue
@staticmethod
def filter_modded():
"""
Removes tracks from the database that have been modified
since they were indexed.
Returns a tuple of unmodified paths and modified tracks.
Unmodified paths are indexed and the modified tracks are
"""
unmodified_paths = set()
modified_tracks: list[dict[str, str]] = []
to_remove = set()
for track in TrackTable.get_all():
try:
if track.last_mod == round(os.path.getmtime(track.filepath)):
unmodified_paths.add(track.filepath)
continue
except (FileNotFoundError, OSError) as e:
log.warning(e) # REVIEW More informations = good
to_remove.add(track.filepath)
modified_tracks.append(
{
"filepath": track.filepath,
"albumhash": track.albumhash,
}
)
to_remove = to_remove.union(set(t["filepath"] for t in modified_tracks))
TrackTable.remove_tracks_by_filepaths(to_remove)
# REVIEW: Remove after testing!
track = TrackTable.get_tracks_by_filepaths(list(to_remove)[:1])
if track:
raise Exception("Track not removed")
# =============================================================
return unmodified_paths, modified_tracks
def tag_untagged(self, files: set[str]):
config = UserConfig()
# Create process pool with worker function
with Pool(processes=max(1, cpu_count() // 2)) as pool:
worker = partial(parse_file_tags, config=config)
# Process files and track progress
results = []
for result in tqdm(
pool.imap_unordered(worker, files),
total=len(files),
desc="Reading files",
):
if result is not None:
results.append(result)
# Bulk insert results
for tags in results:
TrackTable.insert_one(tags)
FolderStore.filepaths.add(tags["filepath"])
print(f"{len(results)} new files indexed")
print("Done")
#
# Create functions
#
def create_albums(_trackhashes: list[str] = []) -> list[tuple[Album, set[str]]]:
"""
Creates album objects using the indexed tracks. Takes in an optional
list of trackhashes to create the albums from. If no list is provided,
all tracks are used.
The trackhashes are passed when creating albums from the watchdogg module.
Returns a list of tuples containing the album and the trackhashes in the album.
ie:
>>> list[tuple[Album, set[str]]]
"""
albums = dict()
if _trackhashes:
all_tracks: list[Track] = TrackStore.get_tracks_by_trackhashes(_trackhashes)
else:
all_tracks: list[Track] = TrackStore.get_flat_list()
all_tracks = remove_duplicates(all_tracks)
for track in all_tracks:
if track.albumhash not in albums:
albums[track.albumhash] = {
"albumartists": track.albumartists,
"artisthashes": [a["artisthash"] for a in track.albumartists],
"albumhash": track.albumhash,
"base_title": None,
"color": None,
"created_date": track.last_mod,
"date": track.date,
"duration": track.duration,
"genres": [*track.genres] if track.genres else [],
"og_title": track.og_album,
"lastplayed": track.lastplayed,
"playcount": track.playcount,
"playduration": track.playduration,
"title": track.album,
"tracks": {track.trackhash},
"pathhash": track.pathhash,
"extra": {},
}
else:
album = albums[track.albumhash]
album["tracks"].add(track.trackhash)
album["playcount"] += track.playcount
album["playduration"] += track.playduration
album["lastplayed"] = max(album["lastplayed"], track.lastplayed)
album["duration"] += track.duration
album["date"] = min(album["date"], track.date)
album["created_date"] = min(album["created_date"], track.last_mod)
if track.genres:
album["genres"].extend(track.genres)
for album in albums.values():
genres = []
for genre in album["genres"]:
if genre not in genres:
genres.append(genre)
album["genres"] = genres
album["genrehashes"] = " ".join([g["genrehash"] for g in genres])
album["base_title"], _ = get_base_album_title(album["og_title"])
del genres
trackhashes = album.pop("tracks")
album["trackcount"] = len(trackhashes)
albums[album["albumhash"]] = (Album(**album), trackhashes)
return list(albums.values())
def create_artists( artisthashes: list[str]) -> list[tuple[Artist, set[str], set[str]]]:
"""
Creates artist objects using the indexed tracks. Takes in an optional
list of artisthashes to create the artists from. If no list is provided,
all tracks are used.
Returns a list of tuples containing the artist, the trackhashes for the artist
and the albumhashes for the artist.
ie:
>>> list[tuple[Artist, set[str], set[str]]]
"""
if artisthashes:
all_tracks: list[Track] = flatten(
[TrackStore.get_tracks_by_artisthash(hash) for hash in artisthashes]
)
else:
all_tracks: list[Track] = TrackStore.get_flat_list()
all_tracks = remove_duplicates(all_tracks)
artists = dict()
for track in all_tracks:
this_artists = [*track.artists]
for a in track.albumartists:
if a not in this_artists:
a["in_track"] = False
this_artists.append(a)
for thisartist in this_artists:
if thisartist["artisthash"] not in artists:
artists[thisartist["artisthash"]] = {
"albumcount": None,
"albums": {track.albumhash},
"artisthash": thisartist["artisthash"],
"created_date": track.last_mod,
"date": track.date,
"duration": track.duration,
"genres": track.genres if track.genres else [],
"name": None,
"names": {thisartist["name"]},
"lastplayed": track.lastplayed,
"playcount": track.playcount,
"playduration": track.playduration,
"trackcount": None,
"tracks": (
{track.trackhash} if thisartist.get("in_track", True) else set()
),
"extra": {},
}
else:
artist: dict = artists[thisartist["artisthash"]]
artist["duration"] += track.duration
artist["playcount"] += track.playcount
artist["playduration"] += track.playduration
artist["albums"].add(track.albumhash)
artist["date"] = min(artist["date"], track.date)
artist["lastplayed"] = max(artist["lastplayed"], track.lastplayed)
artist["created_date"] = min(artist["created_date"], track.last_mod)
artist["names"].add(thisartist["name"])
artist.setdefault("albums", set())
if thisartist.get("in_track", True):
artist["tracks"].add(track.trackhash)
if track.genres:
artist["genres"].extend(track.genres)
for artist in artists.values():
artist["albumcount"] = len(artist["albums"])
artist["trackcount"] = len(artist["tracks"])
genres = []
for genre in artist["genres"]:
if genre not in genres:
genres.append(genre)
artist["genres"] = genres
artist["genrehashes"] = " ".join([g["genrehash"] for g in genres])
artist["name"] = sorted(artist["names"])[0]
# INFO: Delete temporary keys
del artist["names"]
tracks = artist.pop("tracks")
albums = artist.pop("albums")
# INFO: Delete local variables
del genres
artists[artist["artisthash"]] = (Artist(**artist), tracks, albums)
return list(artists.values())
+314
View File
@@ -0,0 +1,314 @@
import pathlib
from dataclasses import dataclass
import os
from io import BytesIO
from pathlib import Path
import re
from typing import Any
import pendulum
from PIL import Image, UnidentifiedImageError
from tinytag import TinyTag
from swingmusic.config import UserConfig
from swingmusic.settings import Defaults, Paths
from swingmusic.utils.hashing import create_hash
from swingmusic.utils.parsers import split_artists
def parse_album_art(filepath: str):
"""
Returns the album art for a given audio file.
:params filepath: Path to file
:returns: `Pil.Image` if available else None
"""
tags = TinyTag.get(filepath, image=True)
image = tags.images.any
if image:
return image.data
return None
def extract_thumb(filepath: str, webp_path: str, overwrite=False, paths:Paths=None) -> bool:
"""
Extracts the thumbnail from an audio file.
Returns the path to the thumbnail.
"""
# this function will be run multithreaded.
# Modules are not cached in concurrent runs.
# If Paths is tried to be imported
if paths is None:
paths = Paths()
lg_img_path = paths.lg_thumb_path / webp_path
sm_img_path = paths.sm_thumb_path / webp_path
xms_img_path = paths.xsm_thumb_path / webp_path
md_img_path = paths.md_thumb_path / webp_path
images = [
(lg_img_path, Defaults.LG_THUMB_SIZE),
(sm_img_path, Defaults.SM_THUMB_SIZE),
(xms_img_path, Defaults.XSM_THUMB_SIZE),
(md_img_path, Defaults.MD_THUMB_SIZE),
]
def save_image(img: Image.Image):
width, height = img.size
ratio = width / height
for path, size in images:
img.resize((size, int(size / ratio)), Image.LANCZOS).save(path, "webp")
del img
if not overwrite and sm_img_path.exists():
img_size = os.path.getsize(sm_img_path)
if img_size > 0:
return True
album_art = parse_album_art(filepath)
if album_art is not None:
try:
img = Image.open(BytesIO(album_art))
except (UnidentifiedImageError, OSError):
return False
try:
save_image(img)
except OSError:
try:
png = img.convert("RGB")
save_image(png)
except: # pylint: disable=bare-except
return False
return True
return False
def parse_date(date_str: str) -> int | None:
"""
Extracts the date from a string and returns a timestamp.
"""
try:
date = pendulum.parse(date_str, strict=False)
return int(date.timestamp())
except Exception as e:
return None
def clean_filename(filename: str):
if "official" in filename.lower():
return re.sub(r"\s*\([^)]*official[^)]*\)", "", filename, flags=re.IGNORECASE)
return filename
@dataclass
class ParseData:
artist: str
title: str
config: UserConfig
def __post_init__(self):
self.artist = split_artists(self.artist, self.config)
def extract_artist_title(filename: str, config: UserConfig):
"""
extract data from filename with specified separators
:params filename: filename
:params config: UserConfig for user separators
"""
path = Path(filename).with_suffix("")
path = clean_filename(str(path))
split_result = path.split(" - ")
split_result = [x.strip() for x in split_result]
if len(split_result) == 1:
return ParseData(
"",
split_result[0],
config,
)
if len(split_result) > 2:
try:
int(split_result[0])
return ParseData(
split_result[1],
" - ".join(split_result[2:]),
config,
)
except ValueError:
pass
artist = split_result[0]
title = split_result[1]
return ParseData(artist, title, config)
def get_tags(filepath: str, config: UserConfig) -> dict:
"""
Parse tags from an audio file.
If tag entries are missing, try getting them from the file name
:param filepath: Path to file.
:param config: UserConfig for ``split`` and ``splitignore`` config
:return: Metadata dict
:raise FileNotFoundError: If filepath is invalid
"""
filepath = pathlib.Path(filepath)
filename = filepath.stem
if not filepath.exists():
raise FileNotFoundError(filepath)
last_mod = round(filepath.stat().st_mtime)
tags = TinyTag.get(filepath)
if hasattr(tags, "other"):
other = tags.other
else:
other = {}
metadata: dict[str, Any] = {
"album": tags.album,
"albumartists": tags.albumartist,
"artists": tags.artist,
"title": tags.title,
"last_mod": last_mod,
"filepath": filepath.as_posix(),
"folder": filepath.parent.as_posix(),
"bitrate": tags.bitrate,
"duration": tags.duration,
"track": tags.track,
"disc": tags.disc,
"genres": tags.genre,
"copyright": " ".join(other.get("copyright", [])), # INFO: Extract copyright from extra data
"extra": {},
"date": parse_date(tags.year or "") or int(last_mod)
}
# check the necessary tags and set them
no_albumartist: bool = (tags.albumartist == "") or (tags.albumartist is None)
no_artist: bool = (tags.artist == "") or (tags.artist is None)
if no_albumartist and not no_artist:
# INFO: If no albumartist, use the artist
metadata["albumartists"] = tags.artist
if no_artist and not no_albumartist:
# INFO: If no artist, use the albumartist
metadata["artists"] = tags.albumartist
parse_data = None
# INFO: If title or album is empty, extract the album and title from the filename
to_filename = ["title", "album"]
for tag in to_filename:
p = metadata[tag]
if p == "" or p is None:
parse_data = extract_artist_title(filename, config)
title = parse_data.title.replace("_", " ")
metadata[tag] = title
# INFO: If artist or albumartist is empty
# extract the artist and albumartist from the filename
parse = ["artists", "albumartists"]
for tag in parse:
p = metadata[tag]
if p == "" or p is None:
if not parse_data:
parse_data = extract_artist_title(filename, config)
artist = parse_data.artist
if artist:
metadata[tag] = ", ".join(artist)
else:
metadata[tag] = "Unknown"
# make values beautiful
# INFO: If these are empty, set to "Unknown"
to_check = ["album", "albumartists"]
for prop in to_check:
if not metadata[prop]:
metadata[prop] = "Unknown"
# INFO: Round the bitrate and duration
to_round = ["bitrate", "duration"]
for prop in to_round:
try:
metadata[prop] = int(getattr(tags, prop))
except TypeError:
metadata[prop] = 0
# INFO: Convert these to int
to_int = ["track", "disc"]
for prop in to_int:
try:
metadata[prop] = int(getattr(tags, prop))
except (ValueError, TypeError):
metadata[prop] = 1
# generate hash
# create albumhash using og_album
metadata["albumhash"] = create_hash(
tags.album or "", metadata.get("albumartists", "")
)
metadata["trackhash"] = create_hash(
metadata.get("artists", ""),
metadata.get("album", ""),
metadata.get("title", ""),
)
# extract extra information not already in tags
extra: dict[str, Any] = {
k: v for k, v in tags.as_dict().items() if not k in metadata
}
extra["hashinfo"] = {
"algo": "sha1",
"format": "[:5]+[-5:]", # first 5 + last 5 chars
}
# REMOVE EMPTY VALUES
to_pop = ["filename", "artists", "albumartist", "year"]
for key, value in extra.items():
# None --bool--> False --not--> True
# [] --bool--> False --not--> True
# "" --bool--> False --not--> True
# [""] --bool--> True --not--> False
if isinstance(value, list) and not "".join(value):
to_pop.append(key)
continue
if not value:
to_pop.append(key)
for key in to_pop:
extra.pop(key, None)
metadata["extra"] = extra
return metadata
+84
View File
@@ -0,0 +1,84 @@
"""
This library contains all the functions related to tracks.
"""
import os
import pathlib
from swingmusic.lib.pydub.pydub import AudioSegment
from swingmusic.lib.pydub.pydub.silence import detect_leading_silence, detect_silence
from swingmusic.utils.threading import ProcessWithReturnValue
def get_leading_silence_end(filepath: pathlib.Path):
"""
Returns the leading silence of a track.
"""
format = filepath.suffix.replace(".", "")
try:
audio = AudioSegment.from_file(filepath, format=format)
silence = detect_leading_silence(audio, silence_threshold=-40.0, chunk_size=10)
except Exception as e:
return 0
return silence if silence > 1000 else 0
def get_trailing_silence_start(filepath: str):
"""
Returns the trailing silence of a track.
"""
format = filepath.suffix.replace(".", "")
try:
audio = AudioSegment.from_file(filepath, format=format)
duration = len(audio)
except Exception as e:
return None
audio = audio[-30000:] if len(audio) > 30000 else audio
silence_groups = detect_silence(audio, silence_thresh=-40.0, seek_step=10)
if len(silence_groups) == 0:
return duration
silence_group = silence_groups[-1]
is_ok = silence_group[1] == len(audio)
if is_ok:
return duration - (silence_group[1] - silence_group[0])
return duration
def get_silence_paddings(ending_file: str, starting_file: str):
"""
Returns the ending silence of a track and the starting silence of the next.
"""
starting_file = pathlib.Path(starting_file)
ending_file = pathlib.Path(ending_file)
silence = {"starting_file": 0, "ending_file": 0}
ending_thread = None
starting_thread = None
if ending_file.exists():
ending_thread = ProcessWithReturnValue(
target=get_trailing_silence_start, args=(ending_file,)
)
ending_thread.start()
if os.path.exists(starting_file):
starting_thread = ProcessWithReturnValue(
target=get_leading_silence_end, args=(starting_file,)
)
starting_thread.start()
if ending_thread:
silence["ending_file"] = ending_thread.join()
if starting_thread:
silence["starting_file"] = starting_thread.join()
return silence
+75
View File
@@ -0,0 +1,75 @@
from swingmusic.utils.threading import background
import subprocess
@background
def start_transcoding(
input_path: str, output_path: str, bitrate: str, container_args: list[str], compression_level: int = 12
):
"""
Starts a background transcoding process for an audio file.
This function uses FFmpeg to transcode an audio file from one format to another,
with specified bitrate and container format. It runs as a background task.
Args:
input_path (str): The path to the input audio file.
output_path (str): The path where the transcoded file will be saved.
bitrate (str): The desired bitrate for the output file (e.g., "128k").
container_args (list[str]): FFmpeg arguments specific to the output container format.
compression_level (int): Compression level (0-9, default: 6).
Returns:
None
Note:
This function is decorated with @background, which means it runs asynchronously.
The actual transcoding process is handled by FFmpeg in a subprocess.
The function will print status messages about the transcoding process.
"""
# Base command
command = [
"ffmpeg",
"-i",
input_path,
"-map_metadata", "0", # Add this line to copy metadata
"-b:a",
bitrate,
"-vn",
"-compression_level",
str(compression_level),
# REVIEW: Idk what any flag below this point does!
"-movflags", "faststart+frag_keyframe+empty_moov", # TODO. specify fragment size
"-write_xing", "0", # ffmpeg.org/ffmpeg-formats.html
"-fflags", "+bitexact", #
]
# Add format-specific parameters
command.extend(container_args)
# Add output path and overwrite flag
command.extend([output_path, "-y"])
process = subprocess.Popen(
command, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL
)
print(f"Started transcoding process with PID: {process.pid}")
try:
# Wait for the process to complete
process.wait()
print(f"Transcoding process (PID: {process.pid}) completed")
except KeyboardInterrupt:
print(f"Transcoding interrupted. Terminating process (PID: {process.pid})")
finally:
# Ensure the process is terminated
try:
process.terminate()
process.wait(timeout=5) # Wait up to 5 seconds for graceful termination
except subprocess.TimeoutExpired:
print(
f"Process (PID: {process.pid}) did not terminate gracefully. Killing..."
)
process.kill()
+372
View File
@@ -0,0 +1,372 @@
"""
This library contains the classes and functions related to the watchdog file watcher.
"""
import json
import os
import sqlite3
import time
from watchdog.events import PatternMatchingEventHandler
from watchdog.observers.api import BaseObserverSubclassCallable
from watchdog.observers import Observer
from swingmusic import settings
from swingmusic.config import UserConfig
from swingmusic.db.libdata import TrackTable
from swingmusic.db.userdata import LibDataTable
from swingmusic.lib.colorlib import process_color
from swingmusic.lib.tagger import create_albums, create_artists
from swingmusic.lib.taglib import extract_thumb, get_tags
from swingmusic.logger import log
from swingmusic.models import Artist, Track
from swingmusic.store.albums import AlbumStore
from swingmusic.store.artists import ArtistMapEntry, ArtistStore
from swingmusic.store.tracks import TrackStore
class Watcher:
"""
Contains the methods for initializing and starting watchdog.
"""
observers: list[BaseObserverSubclassCallable] = []
def __init__(self):
self.observer = Observer()
def run(self):
"""
Starts watchers for each dir in root_dirs
"""
trials = 0
while trials < 10:
try:
# dirs = sdb.get_root_dirs()
dirs = UserConfig().rootDirs
dirs = [rf"{d}" for d in dirs]
dir_map = [
{"original": d, "realpath": os.path.realpath(d)} for d in dirs
]
break
except sqlite3.OperationalError:
trials += 1
time.sleep(1)
else:
log.error(
"WatchDogError: Failed to start Watchdog. Waiting for database timed out!"
)
return
if len(dirs) == 0:
log.warning(
"WatchDogInfo: No root directories configured. Watchdog not started."
)
return
dir_map = [d for d in dir_map if d["realpath"] != d["original"]]
# if len(dirs) > 0 and dirs[0] == "$home":
# dirs = [settings.USER_HOME_DIR]
if any([d == "$home" for d in dirs]):
dirs = [settings.Paths().USER_HOME_DIR]
event_handler = Handler(root_dirs=dirs, dir_map=dir_map)
for _dir in dirs:
exists = os.path.exists(_dir)
if not exists:
log.error("WatchdogError: Directory not found: %s", _dir)
for _dir in dirs:
self.observer.schedule(
event_handler, os.path.realpath(_dir), recursive=True
)
self.observers.append(self.observer)
try:
self.observer.start()
log.info("Started watchdog")
except (FileNotFoundError, PermissionError):
log.error(
"WatchdogError: Failed to start watchdog, root directories could not be resolved."
)
return
except OSError as e:
log.error("Failed to start watchdog. %s", e)
return
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
self.stop_all()
for obsv in self.observers:
obsv.join()
def stop_all(self):
"""
Unschedules and stops all existing watchers.
"""
log.info("Stopping all watchdog observers")
for obsv in self.observers:
obsv.unschedule_all()
obsv.stop()
def restart(self):
"""
Stops all existing watchers, refetches root_dirs from the db
and restarts the watchers.
"""
log.info("🔃 Restarting watchdog")
self.stop_all()
self.run()
def handle_color(albumhash: str):
entry = LibDataTable.find_one(albumhash, "album")
if entry and entry.color:
return
colors = process_color(albumhash, is_album=True)
if colors:
return
if entry is None:
LibDataTable.insert_one(
{"itemhash": albumhash, "color": colors[0], "itemtype": "album"}
)
else:
LibDataTable.update_one(albumhash, {"color": colors[0]})
return colors
def add_track(filepath: str) -> None:
"""
Processes the audio tags for a given file ands add them to the database and store.
Then creates the folder, album and artist objects for the added track and adds them to the store.
"""
TrackStore.remove_track_by_filepath(filepath)
config = UserConfig()
tags = get_tags(filepath, config)
# if the track is somehow invalid, return
if tags is None or tags["bitrate"] == 0 or tags["duration"] == 0:
return
TrackTable.insert_one(tags)
extract_thumb(filepath, tags["albumhash"] + ".webp", overwrite=True)
colors = handle_color(tags["albumhash"])
track = Track(**tags)
TrackStore.add_track(track)
# SECTION: Index album
albumentry = AlbumStore.albummap.get(track.albumhash)
if albumentry is None:
album, trackhashes = create_albums([track.trackhash])[0]
AlbumStore.index_new_album(album, trackhashes)
else:
trackhash_exists = track.trackhash in albumentry.trackhashes
if not trackhash_exists:
albumentry.trackhashes.add(track.trackhash)
albumentry.album.trackcount += 1
albumentry.set_color(colors[0]) if colors else None
# SECTION: Index artist
artists = create_artists(track.artisthashes)
for artist in artists:
ArtistStore.artistmap[artist[0].artisthash] = ArtistMapEntry(
artist=artist[0],
albumhashes=artist[1],
trackhashes=artist[2],
)
def remove_track(filepath: str) -> None:
"""
Removes a track from the music dict.
"""
try:
track = TrackStore.get_tracks_by_filepaths([filepath])[0]
except IndexError:
return
db.remove_tracks_by_filepaths(filepath)
TrackStore.remove_track_by_filepath(filepath)
empty_album = TrackStore.count_tracks_by_trackhash(track.albumhash) > 0
if empty_album:
AlbumStore.remove_album_by_hash(track.albumhash)
artists: list[Artist] = track.artists + track.albumartists # type: ignore
for artist in artists:
empty_artist = not ArtistStore.artist_has_tracks(artist.artisthash)
if empty_artist:
ArtistStore.remove_artist_by_hash(artist.artisthash)
class Handler(PatternMatchingEventHandler):
files_to_process = []
files_to_process_windows = []
file_sizes = {}
root_dirs = []
dir_map = []
def __init__(self, root_dirs: list[str], dir_map: dict[str:str]):
self.root_dirs = root_dirs
self.dir_map = dir_map
patterns = [f"*{f}" for f in settings.SUPPORTED_FILES]
PatternMatchingEventHandler.__init__(
self,
patterns=patterns,
ignore_directories=True,
)
def get_abs_path(self, path: str):
"""
Convert a realpath to a path relative to the matching root directory.
"""
for d in self.dir_map:
if d["realpath"] in path:
return path.replace(d["realpath"], d["original"])
return path
def on_created(self, event):
"""
Fired when a supported file is created.
"""
try:
self.file_sizes[event.src_path] = os.path.getsize(event.src_path)
except FileNotFoundError:
return
self.files_to_process.append(event.src_path)
self.files_to_process_windows.append(event.src_path)
def on_deleted(self, event):
"""
Fired when a delete event occurs on a supported file.
"""
path = self.get_abs_path(event.src_path)
remove_track(path)
def on_moved(self, event):
"""
Fired when a move event occurs on a supported file.
"""
trash = "share/Trash"
if trash in event.dest_path:
path = self.get_abs_path(event.src_path)
remove_track(path)
elif trash in event.src_path:
path = self.get_abs_path(event.dest_path)
add_track(path)
elif trash not in event.dest_path and trash not in event.src_path:
dest_path = self.get_abs_path(event.dest_path)
src_path = self.get_abs_path(event.src_path)
add_track(dest_path)
remove_track(src_path)
def on_closed(self, event):
"""
Fired when a created file is closed.
NOT FIRED IN WINDOWS
"""
try:
# Get initial file size
initial_size = os.path.getsize(event.src_path)
# Wait for 10 seconds
time.sleep(10)
# Check if file size has changed
current_size = os.path.getsize(event.src_path)
if current_size > 0 and current_size == initial_size:
path = self.get_abs_path(event.src_path)
add_track(path)
# Remove from processing list only after successful processing
self.files_to_process.remove(event.src_path)
else:
# File is still being modified or has been deleted
log.info(
f"File {event.src_path} is still being modified. Skipping processing for now."
)
except FileNotFoundError:
# file was closed and deleted.
log.info(f"File {event.src_path} was closed and deleted before processing.")
except ValueError:
# file was already removed from the list by another event handler.
log.info(
f"File {event.src_path} was already removed from the processing list."
)
def on_modified(self, event):
# this event handler is triggered twice on windows
# for copy events. We need to test how this behaves in
# Linux.
if event.src_path not in self.files_to_process_windows:
return
# Check if file write operation is complete
try:
current_size = os.path.getsize(event.src_path)
except FileNotFoundError:
# File was deleted or moved
return
previous_size = self.file_sizes.get(event.src_path, -1)
if current_size == previous_size:
# Wait for a short duration to ensure the file write operation is complete
time.sleep(10)
# Check the file size again
try:
current_size = os.path.getsize(event.src_path)
except FileNotFoundError:
# File was deleted or moved
return
if current_size == previous_size:
try:
os.rename(event.src_path, event.src_path)
path = self.get_abs_path(event.src_path)
remove_track(path)
add_track(path)
self.files_to_process_windows.remove(event.src_path)
del self.file_sizes[event.src_path]
except OSError:
# File is locked
pass
return
# Update the file size for the next iteration
self.file_sizes[event.src_path] = current_size