merge refactors pr #364 from @michilyy

* Save to DB only unique trackhashes * Add check if track already exists in playlist * replace all paths with `pathlib.Path` * `architecture.md`: * add config folder layout `config.py`: * fix bug where `pathlib.Path` cannot be serialized `files.py`: * remove unused imports * update path concatenation to `pathlib.Path` * add config-folder creation `imgserver.py`: * fix serialisation bug `playlistlib.py`: * update path concatenation to `pathlib.Path` * update all `settings.Paths` usages to new singleton `Paths` class. * update all usages of `settings.Paths` * `files.py`: * rework assets copy function. * remove unused loop and unused `shutil.copy2` function `settings.py` * fix recursion exception in `Paths` * `settings.py`: * remove Singleton and `@property` todos from `Paths` * `__init__.py`: * remove now unused function `create_config_dir()` `setup.files`: * remove because merged into `settings.Paths()` for more central and clear flow how the base path gets decided `settings.py`: * add `copy_assets` function `start_swingmusic.py`: * add configurable settings.Paths class `__main__.py`: * update click to used correct default path * remove wrong commited egg files * remove change in the wrong branch * add forgotten `property` decorator update `get_files_and_dirs` to use pathlib where possible `config.py`: * update type annotation `folders.py`: * convert `pathlib` to posix path where needed for sub-functions `folderlib.py`: * rework `get_files_and_dirs` to use `pathlib` where possible `settings.py`: * add forgotten `@property` `start_swingmusic.py`: * remove second `log_startup_info()` * `artistlib.py`: * fix calling property `tagger.py`: * fix comparing elements in `pathlib.Path` * add support for repeating lyrics. * rework lyrics api and lib * update most path functions. add type-hint pathlib where needed * for serialization paths are converted to posix path * use `open` instead of `os.open` update `metaclass` with constant * fix initial config exception if empty file existed * update `userConfig` with `InitVar` to be excluded from `asdict` * remove `is_windows_slash()` rework path function to use pathlib * convert `pathlib.Path` to `str` for serialization * fixing bug with str + pathlib * `__main__.py`: * update click to use package version * remove now unused function `print_version` `filesystem.py`: * rework `CWD` to use importlib `pyproject.toml`: * disable namespace for `importlib.resources` to work correctly * update `lyrics.py`: * remove unused functions * simplify functions * fix bug where assets get created on root * remove unused code * update lyrics for clearer structure. * add support for unsynced lyrics * fix wrong return type in unsynced lyrics * update `/check` to use `send_lyrics` * prefer tags to duplicates * `lyrics.py`: * add docs to a function group * `logger.py`: * add logging config dict. * combine Logging into one file * add socket logger * add debug mode to logger * add JSONL formater * `logger.py`: * update config to directly use the formater. resolves circular import exception `__main__.py`: * add logger setup to main `start_swingmusic.py`: * add debug option to cli * `lyrics.py`: * add offset support * add `setuptools-scm` to get version from git * add support for docker build with scm * add support for docker build with scm need someone who can test the changes workflow * update all usage of `version.txt` to `metadata.version()` * 2x update all usage of `version.txt` to `metadata.version()` * update to no local_scheme version * provide fix for #331. convert `sql.Row` and `TrackTable` to dict before converting to dataclass. * fix `__main__.py`: * wrong import and uncommited changes * add debug and base_path parameter * fix logger pathlib * add client build workflow * set name * split client from build * try fixing builds * try another fix * try also another fix * try again something new * try again something new * change runner * fix failed run because of malformed runner * add wheel builds * remove systems from pure python build * add isolated pyinstaller build * artifacts with names * wrong wheel path * try fetch-depth for tag fetch * disable fail-fast. add wheel installation * add install system packages * add debug * fix wheel install fix pyinstaller spec file * try fix for pyinstaller * try another fix * build on release * add concrete release types * only run on released or pre-released * try release upload * reformat upload * fix needs tag * identifiable pyinstaller builds * compress client folder before uploading * update to src build * remove no more needed aarch64 build script rename pyinstaller assets to lowercase * remove unneeded code * fix: save to DB only unique track hashes * replace click with argparse * set concrete types in argparse * replace manuall path usages with pathlib * remove unused `configs.py` file * reformat `start_swingmusic.py` * fix empty set startup exception * optimizing static files serve function * fixing bug in optimisation of static files serve function * fix folder view bug * colorlib.py: * fix wrong type exception * remove singe use Index_everything class * update logging of populate.py * cleanup files * fix settings.py Paths copy function. Created folder on file. * add exist check to folder * remove unused `INFO` class * fix multiprocessing bug on windows * potential icon fix for pyinstaller fix multiple logging bug * fix argparse config path bug add jobs file * cleanup code fragments fix logging issue add notes to function * note that concurrent creates own sys.modules * refactor some lyrics plugin condition remove unused import from hashing * refactor taglib.py * update import statements to be static * playlistlib.py: * refactoring and more doc strings populate.py: * add poc bugfix settings.py: * add typehint * possible bugfix for multitreading globals * folder.py: * add check if provided path is absolute populate.py: * add bug note settings.py: * add possible error from Singleton implementation start_swingmusic.py: * correct spelling * pass resolved path to Paths tagger.py: * add logging * trying out fixes for multithreading * only upload results not metadata * fix build action again * folder.py: * strictly use pathlib where possible folderlib.py: * add missing docstring to function, who really need it. track.py: * refactor some code folder.py: * refactor some more code * Merge DBPath class and Paths class. Update all usages of DBPath folderslib.py: * fix bug with logging taglib.py: * add missing docstring settings.py: * merge classes * refactor * network.py: * add more docstring config.py: * update pathlib usage tools.py: * refactor * add docstrings * colorlib.py: * add docstring Refactor App builder into grouped config settings. * update assets access for migration * Update FUNDING.yml * Update FUNDING.yml * upgrade tinytag in requirements.txt * update readme * update license * update readme * Update README.md * Update README.md * cleanup requirements.txt remove unused import in audio_segment.py add entrypoint.sh for appimage support update pyproject.toml for optional dependencies add appimage to github workflow * fix invalid workflow file * AppImage build needs more research. Commenting for now * testing a new build workflow * add libev installation * update workflow to new optional dependencies * trying again another fix * finally fix all optional deps installation correctly * remove AppImage poc * albumslib.py: * add docstring folder.py: * add unix path fix update logger name to `__name__` * update build with docker update Dockerfile with git fix typo in lyrics.py add dynamic deps back * add log for static folder * add missing import * add some more todos * add support for AppImages even when it's not perfect. * quick bugfix for wrong appimage config path * fix uploading not finding AppImages builds aka wrong pattern * optimise docker build by using artifacts. Add client path option. change docstring to sphinx format * add todos * Now support AppImages for real: manually build AppImage as we are building a complex project. * fix missing dep in AppImage build * add full AppImage metadata * add missing image file. * only update swingmusic appimage not tool * add todo and fix AppImage build again. * Try fixing some path mixup in AppImage build * add debug tag to action * correct path to appimage folder * do not download tool before checkout * Another fix for path in appimage build * extend config files with more information * default client dir is now inside the config dir. TODOs updated. * default client dir is now inside the config dir. TODOs updated. Add priority todos. * Auto download client when client not found. Respects user provided dir. * rename `requests` submodule to `request` * poc for arm AppImage builds * try out another fix * fix typo in build.yml * add missing arch tag * fix uploading double names * unique naming * enable fallback version for project. * do not download client into readonly dir. * fix relative client download path. Client was resolved into parent of config. * remove client backup path as client is now downloadable * `Paths` checks if config folder exists and creates it if necessary. logger no more creates the config folder. `app_builder.py`: static route no more with '/client' * path are only created in MainProcess. fix gz file not found. * move assets into src and update usages accordingly * remove solved todos * Only upload artefacts if not draft/master aka only on tag * wrong type in assets copy * update log with correct priority * add debug statements and logging to Paths * remove debugging statement * remove double version tag from docker build * fork save release protection * fix typo * add fallback client dir for static builds. * update argparse to new param * add missing import pathlib * add sparse checkout as we do not need everything downloaded * add assets copy check * init logger bevor Paths * remove unused import * check if logdir exists and create if not * only add exec info to file * remove exception log from cli * move logging into main. Allows tools support again. * UserConfig now correctly uses _finished key. Bug where _finished was never written * double save serverId. update root_dir to trow no exception on init. remove debug param * clean up TODOs --------- Co-authored-by: skilletfun <skilletfun.laptew.sergey@yandex.ru> Co-authored-by: Mungai Njoroge <geoffreymungai45@gmail.com>
2026-06-05 04:53:01 +00:00 · 2025-08-28 09:28:11 +00:00
parent b4b0a6e11f
commit e770606567
197 changed files with 2961 additions and 2150 deletions
@@ -0,0 +1,332 @@
+# from tqdm import tqdm
+
+import itertools
+import json
+from typing import Callable, Iterable
+from swingmusic.db.libdata import TrackTable
+
+from swingmusic.models import Track
+from swingmusic.utils.auth import get_current_userid
+from swingmusic.utils.remove_duplicates import remove_duplicates
+
+TRACKS_LOAD_KEY = ""
+
+
+class TrackGroup:
+    """
+    Tracks grouped under the same trackhash.
+    """
+
+    def __init__(self, tracks: list[Track]):
+        self.tracks = tracks
+
+    def append(self, track: Track):
+        """
+        Adds a track to the group.
+        """
+        self.tracks.append(track)
+
+    def remove(self, track: Track):
+        """
+        Removes a track from the group.
+        """
+        self.tracks.remove(track)
+
+    def increment_playcount(self, duration: int, timestamp: int, playcount: int = 1):
+        """
+        Increments the playcount of all tracks in the group.
+        """
+        for track in self.tracks:
+            track.playcount += playcount
+            track.lastplayed = timestamp
+            track.playduration += duration
+
+    def toggle_favorite_user(self, userid: int | None = None):
+        """
+        Adds or removes a user from the list of users who have favorited the track.
+        """
+        if userid is None:
+            userid = get_current_userid()
+
+        for track in self.tracks:
+            track.toggle_favorite_user(userid)
+
+    def get_best(self):
+        """
+        Returns the track with higest bitrate.
+        """
+        return max(self.tracks, key=lambda x: x.bitrate)
+
+    def __len__(self):
+        return len(self.tracks)
+
+
+class classproperty(property):
+    """
+    A class property decorator.
+    """
+
+    def __get__(self, owner_self, owner_cls):
+        if self.fget:
+            return self.fget(owner_cls)
+
+
+class TrackStore:
+    # {'trackhash': Track[]}
+    trackhashmap: dict[str, TrackGroup] = dict()
+
+    @classproperty
+    def tracks(cls) -> list[Track]:
+        return cls.get_flat_list()
+
+    @classmethod
+    def get_flat_list(cls):
+        """
+        Returns a flat list of all tracks.
+        """
+        return list(
+            itertools.chain.from_iterable(
+                [group.tracks for group in cls.trackhashmap.values()]
+            )
+        )
+
+    @classmethod
+    def load_all_tracks(cls, instance_key: str):
+        """
+        Loads all tracks from the database into the store.
+        """
+
+        print("Loading tracks... ", end="")
+        global TRACKS_LOAD_KEY
+        TRACKS_LOAD_KEY = instance_key
+
+        cls.trackhashmap = dict()
+        tracks = TrackTable.get_all()
+
+        # INFO: Load all tracks into the dict store
+        for track in tracks:
+            if instance_key != TRACKS_LOAD_KEY:
+                return
+
+            exists = cls.trackhashmap.get(track.trackhash, None)
+            if not exists:
+                cls.trackhashmap[track.trackhash] = TrackGroup([track])
+            else:
+                cls.trackhashmap[track.trackhash].append(track)
+
+        print("Done!")
+
+    @classmethod
+    def add_track(cls, track: Track):
+        """
+        Adds a single track to the store.
+        """
+        group = cls.trackhashmap.get(track.trackhash, None)
+
+        if group:
+            return group.append(track)
+
+        cls.trackhashmap[track.trackhash] = TrackGroup([track])
+
+    @classmethod
+    def add_tracks(cls, tracks: list[Track]):
+        """
+        Adds multiple tracks to the store.
+        """
+
+        for track in tracks:
+            cls.add_track(track)
+
+    @classmethod
+    def remove_track(cls, track: Track):
+        """
+        Removes a single track from the store.
+        """
+        group = cls.trackhashmap.get(track.trackhash, None)
+
+        if group:
+            group.remove(track)
+
+            if len(group) == 0:
+                del cls.trackhashmap[track.trackhash]
+
+    @classmethod
+    def remove_track_by_filepath(cls, filepath: str):
+        """
+        Removes a track from the store by its filepath.
+        """
+
+        return cls.remove_tracks_by_filepaths({filepath})
+
+    @classmethod
+    def remove_tracks_by_filepaths(cls, filepaths: set[str]):
+        """
+        Removes multiple tracks from the store by their filepaths.
+        """
+
+        filecount = len(filepaths)
+
+        for trackhash in cls.trackhashmap:
+            group = cls.trackhashmap[trackhash]
+
+            for track in group.tracks:
+                if track.filepath in filepaths:
+                    group.remove(track)
+
+                    if len(group) == 0:
+                        del cls.trackhashmap[trackhash]
+
+                    filecount -= 1
+
+                if filecount == 0:
+                    break
+
+    @classmethod
+    def count_tracks_by_trackhash(cls, trackhash: str) -> int:
+        """
+        Counts the number of tracks with a specific trackhash.
+        """
+        return len(cls.trackhashmap.get(trackhash, []))
+
+    # ================================================
+    # ================== GETTERS =====================
+    # ================================================
+
+    @classmethod
+    def get_tracks_by_trackhashes(cls, trackhashes: Iterable[str]) -> list[Track]:
+        """
+        Returns a list of tracks by their hashes.
+        """
+        hash_set = set(trackhashes)
+        tracks: list[Track] = []
+
+        for trackhash in hash_set:
+            group = cls.trackhashmap.get(trackhash, None)
+
+            if group:
+                track = group.get_best()
+                tracks.append(track)
+
+        # sort the tracks in the order of the given trackhashes
+        if type(trackhashes) is list:
+            tracks.sort(key=lambda t: trackhashes.index(t.trackhash))
+
+        return tracks
+
+    @classmethod
+    def get_tracks_by_filepaths(cls, paths: list[str]) -> list[Track]:
+        """
+        Returns all tracks matching the given paths.
+        """
+        # tracks = sorted(cls.trackhashmap, key=lambda x: x.filepath)
+        # tracks = use_bisection(tracks, "filepath", paths)
+        # return [track for track in tracks if track is not None]
+        # return cls.find_tracks_by(key="filepath", value=paths)
+
+        tracks: list[Track] = []
+
+        for trackhash in cls.trackhashmap:
+            group = cls.trackhashmap.get(trackhash)
+
+            if not group:
+                continue
+
+            for track in group.tracks:
+                if track.filepath in paths:
+                    tracks.append(track)
+
+        return tracks
+
+    @classmethod
+    def find_tracks_by(
+        cls,
+        key: str,
+        value: str,
+        predicate: Callable = lambda prop_value, value: prop_value == value,
+        including_duplicates: bool = False,
+    ):
+        """
+        Find all tracks by a specific key.
+        """
+        tracks: list[Track] = []
+
+        for trackhash in cls.trackhashmap:
+            group = cls.trackhashmap.get(trackhash, None)
+
+            if not group:
+                continue
+
+            for track in group.tracks:
+                prop_value = getattr(track, key)
+                if predicate(prop_value, value):
+                    tracks.append(track)
+
+        if including_duplicates:
+            return tracks
+
+        return remove_duplicates(tracks)
+
+    @classmethod
+    def get_tracks_by_albumhash(cls, album_hash: str) -> list[Track]:
+        """
+        Returns all tracks matching the given album hash.
+        """
+        return cls.find_tracks_by(key="albumhash", value=album_hash)
+
+    @classmethod
+    def get_tracks_by_artisthash(cls, artisthash: str):
+        """
+        Returns all tracks matching the given artist. Duplicate tracks are removed.
+        """
+        predicate = lambda artisthashes, artisthash: artisthash in artisthashes
+        return cls.find_tracks_by(
+            key="artisthashes", value=artisthash, predicate=predicate
+        )
+
+    @classmethod
+    def get_tracks_in_path(cls, path: str):
+        """
+        Returns all tracks in the given path.
+        """
+        predicate: Callable[[str, str], bool] = (
+            lambda track_folder, path: track_folder.startswith(path)
+        )
+
+        return cls.find_tracks_by(
+            key="folder",
+            value=path,
+            predicate=predicate,
+            including_duplicates=True,
+        )
+
+    @classmethod
+    def get_recently_added(cls, start: int, limit: int | None):
+        """
+        Returns the most recently added tracks.
+        """
+        tracks = cls.get_flat_list()
+
+        if limit is None:
+            return sorted(tracks, key=lambda x: x.last_mod, reverse=True)[start:]
+
+        return sorted(tracks, key=lambda x: x.last_mod, reverse=True)[start:limit]
+
+    @classmethod
+    def get_recently_played(cls, limit: int):
+        tracks = cls.get_flat_list()
+        return sorted(tracks, key=lambda x: x.lastplayed, reverse=True)[:limit]
+
+    @classmethod
+    def export(cls):
+        path = "tracks.json"
+
+        with open(path, "w") as f:
+            data = [
+                {
+                    "title": t.title,
+                    "album": t.album,
+                    "artists": [a["name"] for a in t.artists],
+                }
+                for t in cls.get_flat_list()
+            ]
+            json.dump(data, f)