merge refactors pr #364 from @michilyy

* Save to DB only unique trackhashes

* Add check if track already exists in playlist

* replace all paths with `pathlib.Path`

* `architecture.md`:
* add config folder layout

`config.py`:
* fix bug where `pathlib.Path` cannot be serialized

`files.py`:
* remove unused imports
* update path concatenation to `pathlib.Path`
* add config-folder creation

`imgserver.py`:
* fix serialisation bug

`playlistlib.py`:
* update path concatenation to `pathlib.Path`

* update all `settings.Paths` usages to new singleton `Paths` class.

* update all usages of `settings.Paths`

* `files.py`:
* rework assets copy function.
* remove unused loop and unused `shutil.copy2` function

`settings.py`
* fix recursion exception in `Paths`

* `settings.py`:
* remove Singleton and `@property` todos from `Paths`

* `__init__.py`:
* remove now unused function `create_config_dir()`

`setup.files`:
* remove because merged into `settings.Paths()`
  for more central and clear flow how the base path gets decided

`settings.py`:
* add `copy_assets` function

`start_swingmusic.py`:
* add configurable settings.Paths class

`__main__.py`:
* update click to used correct default path

* remove wrong commited egg files

* remove change in the wrong branch

* add forgotten `property` decorator
update `get_files_and_dirs` to use pathlib where possible

`config.py`:
* update type annotation

`folders.py`:
* convert `pathlib` to posix path where needed for sub-functions

`folderlib.py`:
* rework `get_files_and_dirs` to use `pathlib` where possible

`settings.py`:
* add forgotten `@property`

`start_swingmusic.py`:
* remove second `log_startup_info()`

* `artistlib.py`:
* fix calling property
`tagger.py`:
* fix comparing elements in `pathlib.Path`

* add support for repeating lyrics.

* rework lyrics api and lib

* update most path functions.
add type-hint pathlib where needed

* for serialization paths are converted to posix path

* use `open` instead of `os.open`
update `metaclass` with constant

* fix initial config exception if empty file existed

* update `userConfig` with `InitVar` to be excluded from `asdict`

* remove `is_windows_slash()`
rework path function to use pathlib

* convert `pathlib.Path` to `str` for serialization

* fixing bug with str + pathlib

* `__main__.py`:
* update click to use package version
* remove now unused function `print_version`

`filesystem.py`:
* rework `CWD` to use importlib

`pyproject.toml`:
* disable namespace for `importlib.resources` to work correctly

* update `lyrics.py`:
* remove unused functions
* simplify functions

* fix bug where assets get created on root

* remove unused code

* update lyrics for clearer structure.

* add support for unsynced lyrics

* fix wrong return type in unsynced lyrics

* update `/check` to use `send_lyrics`

* prefer tags to duplicates

* `lyrics.py`:
* add docs to a function group

* `logger.py`:
* add logging config dict.
* combine Logging into one file
* add socket logger
* add debug mode to logger
* add JSONL formater

* `logger.py`:
* update config to directly use the formater.
resolves circular import exception

`__main__.py`:
* add logger setup to main

`start_swingmusic.py`:
* add debug option to cli

* `lyrics.py`:
* add offset support

* add `setuptools-scm` to get version from git

* add support for docker build with scm

* add support for docker build with scm
need someone who can test the changes workflow

* update all usage of `version.txt` to `metadata.version()`

* 2x update all usage of `version.txt` to `metadata.version()`

* update to no local_scheme version

* provide fix for #331.
convert `sql.Row` and `TrackTable` to dict before converting to dataclass.

* fix `__main__.py`:
* wrong import and uncommited changes
* add debug and base_path parameter

* fix logger pathlib

* add client build workflow

* set name

* split client from build

* try fixing builds

* try another fix

* try also another fix

* try again something new

* try again something new

* change runner

* fix failed run because of malformed runner

* add wheel builds

* remove systems from pure python build

* add isolated pyinstaller build

* artifacts with names

* wrong wheel path

* try fetch-depth for tag fetch

* disable fail-fast.
add wheel installation

* add install system packages

* add debug

* fix wheel install
fix pyinstaller spec file

* try fix for pyinstaller

* try another fix

* build on release

* add concrete release types

* only run on released or pre-released

* try release upload

* reformat upload

* fix needs tag

* identifiable pyinstaller builds

* compress client folder before uploading

* update to src build

* remove no more needed aarch64 build script
rename pyinstaller assets to lowercase

* remove unneeded code

* fix: save to DB only unique track hashes

* replace click with argparse

* set concrete types in argparse

* replace manuall path usages with pathlib

* remove unused `configs.py` file

* reformat `start_swingmusic.py`

* fix empty set startup exception

* optimizing static files serve function

* fixing bug in optimisation of static files serve function

* fix folder view bug

* colorlib.py:
* fix wrong type exception
* remove singe use Index_everything class
* update logging of populate.py

* cleanup files

* fix settings.py Paths copy function.
Created folder on file.

* add exist check to folder

* remove unused `INFO` class

* fix multiprocessing bug on windows

* potential icon fix for pyinstaller
fix multiple logging bug

* fix argparse config path bug
add jobs file

* cleanup code fragments
fix logging issue
add notes to function

* note that concurrent creates own sys.modules

* refactor some lyrics plugin condition
remove unused import from hashing

* refactor taglib.py

* update import statements to be static

* playlistlib.py:
* refactoring and more doc strings
populate.py:
* add poc bugfix
settings.py:
* add typehint

* possible bugfix for multitreading globals

* folder.py:
* add check if provided path is absolute
populate.py:
* add bug note
settings.py:
* add possible error from Singleton implementation
start_swingmusic.py:
* correct spelling
* pass resolved path to Paths
tagger.py:
* add logging

* trying out fixes for multithreading

* only upload results not metadata

* fix build action again

* folder.py:
* strictly use pathlib where possible

folderlib.py:
* add missing docstring to function, who really need it.

track.py:
* refactor some code

folder.py:
* refactor some more code

* Merge DBPath class and Paths class.
Update all usages of DBPath

folderslib.py:
* fix bug with logging

taglib.py:
* add missing docstring

settings.py:
* merge classes
* refactor

* network.py:
* add more docstring

config.py:
* update pathlib usage

tools.py:
* refactor
* add docstrings

* colorlib.py:
* add docstring

Refactor App builder into grouped config settings.

* update assets access for migration

* Update FUNDING.yml

* Update FUNDING.yml

* upgrade tinytag in requirements.txt

* update readme

* update license

* update readme

* Update README.md

* Update README.md

* cleanup requirements.txt
remove unused import in audio_segment.py
add entrypoint.sh for appimage support
update pyproject.toml for optional dependencies
add appimage to github workflow

* fix invalid workflow file

* AppImage build needs more research.
Commenting for now

* testing a new build workflow

* add libev installation

* update workflow to new optional dependencies

* trying again another fix

* finally fix all optional deps installation correctly

* remove AppImage poc

* albumslib.py:
* add docstring

folder.py:
* add unix path fix

update logger name to `__name__`

* update build with docker
update Dockerfile with git
fix typo in lyrics.py
add dynamic deps back

* add log for static folder

* add missing import

* add some more todos

* add support for AppImages even when it's not perfect.

* quick bugfix for wrong appimage config path

* fix uploading not finding AppImages builds aka wrong pattern

* optimise docker build by using artifacts.
Add client path option.
change docstring to sphinx format

* add todos

* Now support AppImages for real:
manually build AppImage as we are building a complex project.

* fix missing dep in AppImage build

* add full AppImage metadata

* add missing image file.

* only update swingmusic appimage not tool

* add todo and fix AppImage build again.

* Try fixing some path mixup in AppImage build

* add debug tag to action

* correct path to appimage folder

* do not download tool before checkout

* Another fix for path in appimage build

* extend config files with more information

* default client dir is now inside the config dir.
TODOs updated.

* default client dir is now inside the config dir.
TODOs updated.
Add priority todos.

* Auto download client when client not found.
Respects user provided dir.

* rename `requests` submodule to `request`

* poc for arm AppImage builds

* try out another fix

* fix typo in build.yml

* add missing arch tag

* fix uploading double names

* unique naming

* enable fallback version for project.

* do not download client into readonly dir.

* fix relative client download path. Client was resolved into parent of config.

* remove client backup path as client is now downloadable

* `Paths` checks if config folder exists and creates it if necessary.
logger no more creates the config folder.
`app_builder.py`: static route no more with '/client'

* path are only created in MainProcess.
fix gz file not found.

* move assets into src and update usages accordingly

* remove solved todos

* Only upload artefacts if not draft/master aka only on tag

* wrong type in assets copy

* update log with correct priority

* add debug statements and logging to Paths

* remove debugging statement

* remove double version tag from docker build

* fork save release protection

* fix typo

* add fallback client dir for static builds.

* update argparse to new param

* add missing import pathlib

* add sparse checkout as we do not need everything downloaded

* add assets copy check

* init logger bevor Paths

* remove unused import

* check if logdir exists and create if not

* only add exec info to file

* remove exception log from cli

* move logging into main.
Allows tools support again.

* UserConfig now correctly uses _finished key.
Bug where _finished was never written

* double save serverId.
update root_dir to trow no exception on init.
remove debug param

* clean up TODOs

---------

Co-authored-by: skilletfun <skilletfun.laptew.sergey@yandex.ru>
Co-authored-by: Mungai Njoroge <geoffreymungai45@gmail.com>

src/swingmusic/api/scrobble/__init__.py

@@ -0,0 +1,380 @@
+from gettext import ngettext
+from flask_openapi3 import Tag
+from flask_openapi3 import APIBlueprint
+import pendulum
+from pydantic import Field, BaseModel
+from swingmusic.api.apischemas import TrackHashSchema
+from typing import Literal
+import locale
+
+from swingmusic.db.userdata import FavoritesTable, ScrobbleTable
+from swingmusic.lib.extras import get_extra_info
+from swingmusic.lib.recipes.recents import RecentlyPlayed
+from swingmusic.models.album import Album
+from swingmusic.models.stats import StatItem
+from swingmusic.models.track import Track
+from swingmusic.plugins.lastfm import LastFmPlugin
+from swingmusic.serializers.artist import serialize_for_card
+from swingmusic.serializers.album import serialize_for_card as serialize_for_album_card
+from swingmusic.serializers.track import serialize_track, serialize_tracks
+from swingmusic.settings import Defaults
+from swingmusic.store.albums import AlbumStore
+from swingmusic.store.artists import ArtistStore
+from swingmusic.store.tracks import TrackStore
+from swingmusic.utils.dates import (
+    get_date_range,
+    get_duration_in_seconds,
+    seconds_to_time_string,
+)
+from swingmusic.utils.stats import (
+    calculate_album_trend,
+    calculate_artist_trend,
+    calculate_new_albums,
+    calculate_new_artists,
+    calculate_scrobble_trend,
+    calculate_track_trend,
+    get_albums_in_period,
+    get_artists_in_period,
+    get_tracks_in_period,
+)
+
+bp_tag = Tag(name="Logger", description="Log item plays")
+api = APIBlueprint("logger", __name__, url_prefix="/logger", abp_tags=[bp_tag])
+
+
+class LogTrackBody(TrackHashSchema):
+    timestamp: int = Field(description="The timestamp of the track")
+    duration: int = Field(description="The duration of the track in seconds")
+    source: str = Field(
+        description="The play source of the track",
+        json_schema_extra={
+            "examples": [
+                f"al:{Defaults.API_ALBUMHASH}",
+                f"tr:{Defaults.API_TRACKHASH}",
+                f"ar:{Defaults.API_ARTISTHASH}",
+            ]
+        },
+    )
+
+
+def format_date(start: float, end: float):
+    return f"{pendulum.from_timestamp(start).format('MMM D, YYYY')} - {pendulum.from_timestamp(end).format('MMM D, YYYY')}"
+
+
+@api.post("/track/log")
+def log_track(body: LogTrackBody):
+    """
+    Log a track play to the database.
+    """
+    timestamp = body.timestamp
+    duration = body.duration
+
+    if not timestamp or duration < 5:
+        return {"msg": "Invalid entry."}, 400
+
+    trackentry = TrackStore.trackhashmap.get(body.trackhash)
+    if trackentry is None:
+        return {"msg": "Track not found."}, 404
+
+    scrobble_data = dict(body)
+    # REVIEW: Do we need to store the extra info in the database?
+    # OR .... can we just write it to the backup file on demand?
+    scrobble_data["extra"] = get_extra_info(body.trackhash, "track")
+    ScrobbleTable.add(scrobble_data)
+
+    # NOTE: Update the recently played homepage for this userid
+    RecentlyPlayed(userid=scrobble_data["userid"])
+
+    # Update play data on the in-memory stores
+    track = trackentry.tracks[0]
+    album = AlbumStore.albummap.get(track.albumhash)
+
+    if album:
+        album.increment_playcount(duration, timestamp)
+
+    for hash in track.artisthashes:
+        artist = ArtistStore.artistmap.get(hash)
+
+        if artist:
+            artist.increment_playcount(duration, timestamp)
+
+    trackentry.increment_playcount(duration, timestamp)
+    track = trackentry.tracks[0]
+
+    lastfm = LastFmPlugin()
+
+    if (
+        lastfm.enabled
+        and track.duration > 30
+        and body.duration >= min(track.duration / 2, 240)
+        # SEE: https://www.last.fm/api/scrobbling#when-is-a-scrobble-a-scrobble
+    ):
+        lastfm.scrobble(trackentry.tracks[0], timestamp)
+
+    return {"msg": "recorded"}, 201
+
+
+class ChartItemsQuery(BaseModel):
+    duration: Literal["week", "month", "year", "alltime"] = Field(
+        "year",
+        description="Duration to fetch data for",
+    )
+    limit: int = Field(10, description="Number of top tracks to return")
+    order_by: Literal["playcount", "playduration"] = Field(
+        "playduration", description="Property to order by"
+    )
+
+
+# SECTION: STATS
+
+
+def get_help_text(
+    playcount: int, playduration: int, order_by: Literal["playcount", "playduration"]
+):
+    """
+    Get the help text given the playcount and playduration.
+    """
+    if order_by == "playcount":
+        if playcount == 0:
+            return "unplayed"
+
+        return f"{playcount} play{'' if playcount == 1 else 's'}"
+    if order_by == "playduration":
+        return seconds_to_time_string(playduration)
+
+
+# DISCLAIMER: Code beyond this point was partially written by Claude 3.5 Sonnet in Cursor.
+# TODO: Refactor, group and clean up
+
+
+@api.get("/top-tracks")
+def get_top_tracks(query: ChartItemsQuery):
+    """
+    Get the top N tracks played within a given duration.
+    """
+    start_time, end_time = get_date_range(query.duration)
+    previous_start_time = start_time - get_duration_in_seconds(query.duration)
+
+    current_period_tracks, current_period_scrobbles, duration = get_tracks_in_period(
+        start_time, end_time
+    )
+    previous_period_tracks, previous_period_scrobbles, _ = get_tracks_in_period(
+        previous_start_time, start_time
+    )
+    scrobble_trend = (
+        "rising"
+        if current_period_scrobbles > previous_period_scrobbles
+        else (
+            "falling"
+            if current_period_scrobbles < previous_period_scrobbles
+            else "stable"
+        )
+    )
+
+    sorted_tracks = sort_tracks(current_period_tracks, query.order_by)
+    top_tracks = sorted_tracks[: query.limit]
+
+    response = []
+    for track in top_tracks:
+        trend = calculate_track_trend(
+            track, current_period_tracks, previous_period_tracks
+        )
+        track = {
+            **serialize_track(track),
+            "trend": trend,
+            "help_text": get_help_text(
+                track.playcount, track.playduration, query.order_by
+            ),
+        }
+
+        response.append(track)
+
+    return {
+        "tracks": response,
+        "scrobbles": {
+            "text": f"{current_period_scrobbles} total play{'' if current_period_scrobbles == 1 else 's'} ({seconds_to_time_string(duration)})",
+            "trend": scrobble_trend,
+            "dates": format_date(start_time, end_time),
+        },
+    }, 200
+
+
+def sort_tracks(tracks: list[Track], order_by: Literal["playcount", "playduration"]):
+    return sorted(tracks, key=lambda x: getattr(x, order_by), reverse=True)
+
+
+@api.get("/top-artists")
+def get_top_artists(query: ChartItemsQuery):
+    """
+    Get the top N artists played within a given duration.
+    """
+    start_time, end_time = get_date_range(query.duration)
+    previous_start_time = start_time - get_duration_in_seconds(query.duration)
+
+    current_period_artists = get_artists_in_period(start_time, end_time)
+    previous_period_artists = get_artists_in_period(previous_start_time, start_time)
+
+    new_artists = calculate_new_artists(current_period_artists, start_time)
+    scrobble_trend = calculate_scrobble_trend(
+        len(current_period_artists), len(previous_period_artists)
+    )
+
+    sorted_artists = sort_artists(current_period_artists, query.order_by)
+    top_artists = sorted_artists[: query.limit]
+
+    response = []
+    for artist in top_artists:
+        trend = calculate_artist_trend(
+            artist, current_period_artists, previous_period_artists
+        )
+        db_artist = ArtistStore.get_artist_by_hash(artist["artisthash"])
+
+        if db_artist is None:
+            continue
+
+        artist = {
+            **serialize_for_card(db_artist),
+            "trend": trend,
+            "help_text": get_help_text(
+                artist["playcount"], artist["playduration"], query.order_by
+            ),
+            "extra": {
+                "playcount": artist["playcount"],
+            },
+        }
+        response.append(artist)
+
+    return {
+        "artists": response,
+        "scrobbles": {
+            "text": f"{new_artists} {'new' if query.duration != 'alltime' else ''} {ngettext('artist', 'artists', new_artists)}",
+            "trend": scrobble_trend,
+            "dates": format_date(start_time, end_time),
+        },
+    }, 200
+
+
+def sort_artists(artists, order_by):
+    return sorted(artists, key=lambda x: x[order_by], reverse=True)
+
+
+@api.get("/top-albums")
+def get_top_albums(query: ChartItemsQuery):
+    """
+    Get the top N albums played within a given duration.
+    """
+    start_time, end_time = get_date_range(query.duration)
+    previous_start_time = start_time - get_duration_in_seconds(query.duration)
+
+    current_period_albums = get_albums_in_period(start_time, end_time)
+    previous_period_albums = get_albums_in_period(previous_start_time, start_time)
+
+    new_albums = calculate_new_albums(current_period_albums, previous_period_albums)
+    scrobble_trend = calculate_scrobble_trend(
+        len(current_period_albums), len(previous_period_albums)
+    )
+
+    sorted_albums = sort_albums(current_period_albums, query.order_by)
+    top_albums = sorted_albums[: query.limit]
+
+    response = []
+    for album in top_albums:
+        trend = calculate_album_trend(
+            album, current_period_albums, previous_period_albums
+        )
+        album = {
+            **serialize_for_album_card(album),
+            "trend": trend,
+            "help_text": get_help_text(
+                album.playcount, album.playduration, query.order_by
+            ),
+        }
+        response.append(album)
+
+    return {
+        "albums": response,
+        "scrobbles": {
+            "text": f"{new_albums} new album{'' if new_albums == 1 else 's'} played",
+            "trend": scrobble_trend,
+            "dates": format_date(start_time, end_time),
+        },
+    }, 200
+
+
+def sort_albums(albums: list[Album], order_by: Literal["playcount", "playduration"]):
+    return sorted(albums, key=lambda x: getattr(x, order_by), reverse=True)
+
+
+@api.get("/stats")
+def get_stats():
+    """
+    Get the stats for the user.
+    """
+    period = "week"
+    start_time, end_time = get_date_range(period)
+
+    said_period = period
+    match period:
+        case "week":
+            said_period = "this week"
+        case "month":
+            said_period = "this month"
+        case "year":
+            said_period = "this year"
+        case "alltime":
+            said_period = "all time"
+
+    count = len(TrackStore.get_flat_list())
+    total_tracks = StatItem(
+        "trackcount",
+        "in your library",
+        locale.format_string("%d", count, grouping=True)
+        + " "
+        + ngettext("track", "tracks", count),
+    )
+
+    tracks, playcount, playduration = get_tracks_in_period(start_time, end_time)
+
+    playcount = StatItem(
+        "streams",
+        said_period,
+        f"{playcount} track {ngettext('play', 'plays', playcount)}",
+    )
+
+    playduration = StatItem(
+        "playtime",
+        said_period,
+        f"{seconds_to_time_string(playduration)} listened",
+    )
+
+    tracks = sorted(tracks, key=lambda t: t.playduration, reverse=True)
+
+    # Find the top track from the last 7 days
+    top_track = StatItem(
+        "toptrack",
+        f"Top track {said_period}",
+        (
+            tracks[0].title + " - " + tracks[0].artists[0]["name"]
+            if len(tracks) > 0
+            else "—"
+        ),
+        (tracks[0].image if len(tracks) > 0 else None),
+    )
+
+    fav_count = FavoritesTable.count_favs_in_period(start_time, end_time)
+    favorites = StatItem(
+        "favorites",
+        said_period,
+        f"{fav_count} {'new' if period != 'alltime' else ''} favorite{'' if fav_count == 1 else 's'}",
+    )
+
+    return {
+        "stats": [
+            top_track,
+            playcount,
+            playduration,
+            favorites,
+            total_tracks,
+        ],
+        "dates": format_date(start_time, end_time),
+    }, 200