merge refactors pr #364 from @michilyy

* Save to DB only unique trackhashes

* Add check if track already exists in playlist

* replace all paths with `pathlib.Path`

* `architecture.md`:
* add config folder layout

`config.py`:
* fix bug where `pathlib.Path` cannot be serialized

`files.py`:
* remove unused imports
* update path concatenation to `pathlib.Path`
* add config-folder creation

`imgserver.py`:
* fix serialisation bug

`playlistlib.py`:
* update path concatenation to `pathlib.Path`

* update all `settings.Paths` usages to new singleton `Paths` class.

* update all usages of `settings.Paths`

* `files.py`:
* rework assets copy function.
* remove unused loop and unused `shutil.copy2` function

`settings.py`
* fix recursion exception in `Paths`

* `settings.py`:
* remove Singleton and `@property` todos from `Paths`

* `__init__.py`:
* remove now unused function `create_config_dir()`

`setup.files`:
* remove because merged into `settings.Paths()`
  for more central and clear flow how the base path gets decided

`settings.py`:
* add `copy_assets` function

`start_swingmusic.py`:
* add configurable settings.Paths class

`__main__.py`:
* update click to used correct default path

* remove wrong commited egg files

* remove change in the wrong branch

* add forgotten `property` decorator
update `get_files_and_dirs` to use pathlib where possible

`config.py`:
* update type annotation

`folders.py`:
* convert `pathlib` to posix path where needed for sub-functions

`folderlib.py`:
* rework `get_files_and_dirs` to use `pathlib` where possible

`settings.py`:
* add forgotten `@property`

`start_swingmusic.py`:
* remove second `log_startup_info()`

* `artistlib.py`:
* fix calling property
`tagger.py`:
* fix comparing elements in `pathlib.Path`

* add support for repeating lyrics.

* rework lyrics api and lib

* update most path functions.
add type-hint pathlib where needed

* for serialization paths are converted to posix path

* use `open` instead of `os.open`
update `metaclass` with constant

* fix initial config exception if empty file existed

* update `userConfig` with `InitVar` to be excluded from `asdict`

* remove `is_windows_slash()`
rework path function to use pathlib

* convert `pathlib.Path` to `str` for serialization

* fixing bug with str + pathlib

* `__main__.py`:
* update click to use package version
* remove now unused function `print_version`

`filesystem.py`:
* rework `CWD` to use importlib

`pyproject.toml`:
* disable namespace for `importlib.resources` to work correctly

* update `lyrics.py`:
* remove unused functions
* simplify functions

* fix bug where assets get created on root

* remove unused code

* update lyrics for clearer structure.

* add support for unsynced lyrics

* fix wrong return type in unsynced lyrics

* update `/check` to use `send_lyrics`

* prefer tags to duplicates

* `lyrics.py`:
* add docs to a function group

* `logger.py`:
* add logging config dict.
* combine Logging into one file
* add socket logger
* add debug mode to logger
* add JSONL formater

* `logger.py`:
* update config to directly use the formater.
resolves circular import exception

`__main__.py`:
* add logger setup to main

`start_swingmusic.py`:
* add debug option to cli

* `lyrics.py`:
* add offset support

* add `setuptools-scm` to get version from git

* add support for docker build with scm

* add support for docker build with scm
need someone who can test the changes workflow

* update all usage of `version.txt` to `metadata.version()`

* 2x update all usage of `version.txt` to `metadata.version()`

* update to no local_scheme version

* provide fix for #331.
convert `sql.Row` and `TrackTable` to dict before converting to dataclass.

* fix `__main__.py`:
* wrong import and uncommited changes
* add debug and base_path parameter

* fix logger pathlib

* add client build workflow

* set name

* split client from build

* try fixing builds

* try another fix

* try also another fix

* try again something new

* try again something new

* change runner

* fix failed run because of malformed runner

* add wheel builds

* remove systems from pure python build

* add isolated pyinstaller build

* artifacts with names

* wrong wheel path

* try fetch-depth for tag fetch

* disable fail-fast.
add wheel installation

* add install system packages

* add debug

* fix wheel install
fix pyinstaller spec file

* try fix for pyinstaller

* try another fix

* build on release

* add concrete release types

* only run on released or pre-released

* try release upload

* reformat upload

* fix needs tag

* identifiable pyinstaller builds

* compress client folder before uploading

* update to src build

* remove no more needed aarch64 build script
rename pyinstaller assets to lowercase

* remove unneeded code

* fix: save to DB only unique track hashes

* replace click with argparse

* set concrete types in argparse

* replace manuall path usages with pathlib

* remove unused `configs.py` file

* reformat `start_swingmusic.py`

* fix empty set startup exception

* optimizing static files serve function

* fixing bug in optimisation of static files serve function

* fix folder view bug

* colorlib.py:
* fix wrong type exception
* remove singe use Index_everything class
* update logging of populate.py

* cleanup files

* fix settings.py Paths copy function.
Created folder on file.

* add exist check to folder

* remove unused `INFO` class

* fix multiprocessing bug on windows

* potential icon fix for pyinstaller
fix multiple logging bug

* fix argparse config path bug
add jobs file

* cleanup code fragments
fix logging issue
add notes to function

* note that concurrent creates own sys.modules

* refactor some lyrics plugin condition
remove unused import from hashing

* refactor taglib.py

* update import statements to be static

* playlistlib.py:
* refactoring and more doc strings
populate.py:
* add poc bugfix
settings.py:
* add typehint

* possible bugfix for multitreading globals

* folder.py:
* add check if provided path is absolute
populate.py:
* add bug note
settings.py:
* add possible error from Singleton implementation
start_swingmusic.py:
* correct spelling
* pass resolved path to Paths
tagger.py:
* add logging

* trying out fixes for multithreading

* only upload results not metadata

* fix build action again

* folder.py:
* strictly use pathlib where possible

folderlib.py:
* add missing docstring to function, who really need it.

track.py:
* refactor some code

folder.py:
* refactor some more code

* Merge DBPath class and Paths class.
Update all usages of DBPath

folderslib.py:
* fix bug with logging

taglib.py:
* add missing docstring

settings.py:
* merge classes
* refactor

* network.py:
* add more docstring

config.py:
* update pathlib usage

tools.py:
* refactor
* add docstrings

* colorlib.py:
* add docstring

Refactor App builder into grouped config settings.

* update assets access for migration

* Update FUNDING.yml

* Update FUNDING.yml

* upgrade tinytag in requirements.txt

* update readme

* update license

* update readme

* Update README.md

* Update README.md

* cleanup requirements.txt
remove unused import in audio_segment.py
add entrypoint.sh for appimage support
update pyproject.toml for optional dependencies
add appimage to github workflow

* fix invalid workflow file

* AppImage build needs more research.
Commenting for now

* testing a new build workflow

* add libev installation

* update workflow to new optional dependencies

* trying again another fix

* finally fix all optional deps installation correctly

* remove AppImage poc

* albumslib.py:
* add docstring

folder.py:
* add unix path fix

update logger name to `__name__`

* update build with docker
update Dockerfile with git
fix typo in lyrics.py
add dynamic deps back

* add log for static folder

* add missing import

* add some more todos

* add support for AppImages even when it's not perfect.

* quick bugfix for wrong appimage config path

* fix uploading not finding AppImages builds aka wrong pattern

* optimise docker build by using artifacts.
Add client path option.
change docstring to sphinx format

* add todos

* Now support AppImages for real:
manually build AppImage as we are building a complex project.

* fix missing dep in AppImage build

* add full AppImage metadata

* add missing image file.

* only update swingmusic appimage not tool

* add todo and fix AppImage build again.

* Try fixing some path mixup in AppImage build

* add debug tag to action

* correct path to appimage folder

* do not download tool before checkout

* Another fix for path in appimage build

* extend config files with more information

* default client dir is now inside the config dir.
TODOs updated.

* default client dir is now inside the config dir.
TODOs updated.
Add priority todos.

* Auto download client when client not found.
Respects user provided dir.

* rename `requests` submodule to `request`

* poc for arm AppImage builds

* try out another fix

* fix typo in build.yml

* add missing arch tag

* fix uploading double names

* unique naming

* enable fallback version for project.

* do not download client into readonly dir.

* fix relative client download path. Client was resolved into parent of config.

* remove client backup path as client is now downloadable

* `Paths` checks if config folder exists and creates it if necessary.
logger no more creates the config folder.
`app_builder.py`: static route no more with '/client'

* path are only created in MainProcess.
fix gz file not found.

* move assets into src and update usages accordingly

* remove solved todos

* Only upload artefacts if not draft/master aka only on tag

* wrong type in assets copy

* update log with correct priority

* add debug statements and logging to Paths

* remove debugging statement

* remove double version tag from docker build

* fork save release protection

* fix typo

* add fallback client dir for static builds.

* update argparse to new param

* add missing import pathlib

* add sparse checkout as we do not need everything downloaded

* add assets copy check

* init logger bevor Paths

* remove unused import

* check if logdir exists and create if not

* only add exec info to file

* remove exception log from cli

* move logging into main.
Allows tools support again.

* UserConfig now correctly uses _finished key.
Bug where _finished was never written

* double save serverId.
update root_dir to trow no exception on init.
remove debug param

* clean up TODOs

---------

Co-authored-by: skilletfun <skilletfun.laptew.sergey@yandex.ru>
Co-authored-by: Mungai Njoroge <geoffreymungai45@gmail.com>

src/swingmusic/api/folder.py

@@ -0,0 +1,303 @@
+"""
+Contains all the folder routes.
+"""
+import pathlib
+from datetime import datetime
+import os
+from pathlib import Path
+
+import psutil
+from pydantic import BaseModel, Field
+from flask_openapi3 import Tag
+from flask_openapi3 import APIBlueprint
+from showinfm import show_in_file_manager
+
+from swingmusic import settings
+from swingmusic.config import UserConfig
+from swingmusic.db.libdata import TrackTable
+from swingmusic.db.userdata import FavoritesTable, PlaylistTable
+from swingmusic.lib.folderslib import get_files_and_dirs, get_folders
+from swingmusic.serializers.track import serialize_track, serialize_tracks
+from swingmusic.store.tracks import TrackStore
+from swingmusic.utils.wintools import is_windows
+
+tag = Tag(name="Folders", description="Get folders and tracks in a directory")
+api = APIBlueprint("folder", __name__, url_prefix="/folder", abp_tags=[tag])
+
+
+class FolderTree(BaseModel):
+    folder: str = Field("$home", description="The folder to things from")
+    sorttracksby: str = Field(
+        "default",
+        description="""The field to sort tracks by. Options: [
+            "default",
+            "album",
+            "albumartists",
+            "artists",
+            "bitrate",
+            "date",
+            "disc",
+            "duration",
+            "last_mod",
+            "lastplayed",
+            "playduration",
+            "playcount",
+            "title",
+        ]""",
+    )
+    tracksort_reverse: bool = Field(
+        False,
+        description="Whether to reverse the sort order of the tracks",
+    )
+    sortfoldersby: str = Field(
+        "lastmod",
+        description="""The field to sort folders by.
+        Options: [
+            "default",
+            "name",
+            "lastmod",
+            "trackcount",
+        ]
+        """,
+    )
+    foldersort_reverse: bool = Field(
+        False,
+        description="Whether to reverse the sort order of the folders",
+    )
+    start: int = Field(0, description="The start index")
+    limit: int = Field(50, description="The max number of items to return")
+    tracks_only: bool = Field(False, description="Whether to only get tracks")
+
+
+@api.post("")
+def get_folder_tree(body: FolderTree):
+    """
+    Get folder
+
+    Returns a list of all the folders and tracks in the given folder.
+    """
+    og_req_dir = body.folder
+    req_dir = body.folder
+    tracks_only = body.tracks_only
+
+    config = UserConfig()
+    root_dirs = config.rootDirs
+
+    if req_dir == "$home" and "$home" in root_dirs:
+        req_dir = settings.Paths().USER_HOME_DIR.as_posix()
+
+    if req_dir == "$home":
+            folders = get_folders(root_dirs)
+
+            return {
+                "folders": folders,
+                "tracks": [],
+            }
+
+    if req_dir.startswith("$playlist"):
+        splits = req_dir.split("/")
+
+        if len(splits) == 2:
+            pid = splits[1]
+            playlist = PlaylistTable.get_by_id(int(pid))
+            tracks = TrackStore.get_tracks_by_trackhashes(
+                playlist.trackhashes[
+                    body.start : body.start + body.limit if body.limit != -1 else None
+                ]
+            )
+
+            return {
+                "path": req_dir,
+                "folders": [],
+                "tracks": serialize_tracks(tracks),
+            }
+
+        playlists = PlaylistTable.get_all()
+        playlists = sorted(
+            playlists,
+            key=lambda p: datetime.strptime(p.last_updated, "%Y-%m-%d %H:%M:%S"),
+            reverse=True,
+        )
+
+        return {
+            "path": req_dir,
+            "folders": [
+                {
+                    "name": p.name,
+                    "path": f"$playlist/{p.id}",
+                    "trackcount": p.count,
+                }
+                for p in playlists
+            ],
+            "tracks": [],
+        }
+
+    if req_dir == "$favorites":
+        tracks, total = FavoritesTable.get_fav_tracks(body.start, body.limit)
+        tracks = TrackStore.get_tracks_by_trackhashes([t.hash for t in tracks])
+
+        return {
+            "tracks": serialize_tracks(tracks),
+            "folders": [],
+            "path": req_dir,
+        }
+
+    # TODO: currently only fixed on unix. Windows/Mac still pending.
+    # note
+
+    if not pathlib.Path(req_dir).exists():
+        req_dir = "/" + req_dir
+
+    results = get_files_and_dirs(
+        pathlib.Path(req_dir),
+        start=body.start,
+        limit=body.limit,
+        tracks_only=tracks_only,
+        tracksortby=body.sorttracksby,
+        foldersortby=body.sortfoldersby,
+        tracksort_reverse=body.tracksort_reverse,
+        foldersort_reverse=body.foldersort_reverse,
+    )
+
+    if og_req_dir == "$home" and config.showPlaylistsInFolderView:
+        # Get all playlists and return them as a list of folders
+        playlists_item = {
+            "name": "Playlists",
+            "path": "$playlists",
+            "trackcount": sum(p.count for p in PlaylistTable.get_all()),
+        }
+
+        favorites_item = {
+            "name": "Favorites",
+            "path": "$favorites",
+            "trackcount": FavoritesTable.get_fav_tracks(0, -1)[1],
+        }
+
+        results["folders"].insert(0, playlists_item)
+        results["folders"].insert(0, favorites_item)
+
+    return results
+
+
+def get_all_drives(is_win: bool = False):
+    """
+    Returns a list of all the drives on a Windows machine.
+    """
+    drives_ = psutil.disk_partitions(all=True)
+    drives = [Path(d.mountpoint).as_posix() for d in drives_]
+
+    if is_win:
+        return drives
+    else:
+        remove = (
+            "/boot",
+            "/tmp",
+            "/snap",
+            "/var",
+            "/sys",
+            "/proc",
+            "/etc",
+            "/run",
+            "/dev",
+        )
+        drives = [d for d in drives if not d.startswith(remove)]
+
+    return drives
+
+
+class DirBrowserBody(BaseModel):
+    folder: str = Field(
+        "$root",
+        description="The folder to list directories from",
+    )
+
+
+@api.post("/dir-browser")
+def list_folders(body: DirBrowserBody):
+    """
+    List folders
+
+    Returns a list of all the folders in the given folder.
+    Used when selecting root dirs.
+    """
+    req_dir = body.folder
+    is_win = is_windows()
+
+    if req_dir == "$root":
+        return {
+            "folders": [{"name": d, "path": d} for d in get_all_drives(is_win=is_win)]
+        }
+
+
+    req_dir = pathlib.Path(req_dir)
+
+    if not req_dir.exists():
+        req_dir = "/" / req_dir
+
+    try:
+        entries = os.scandir(req_dir)
+    except PermissionError:
+        return {"folders": []}
+
+    # only get dirs and remove hidden dirs
+    dirs = []
+    for entry in entries:
+        entry = pathlib.Path(entry)
+        name = entry.name
+
+        if name.startswith("$"): # ignore windows system folder
+            continue
+
+        if name.startswith("."): # ignore unix hidden folder
+            continue
+
+        if entry.is_dir():  # lastly, check if is dir
+            dirs.append({
+                "name": name,
+                "path": entry.as_posix()
+            })
+
+    return {
+        "folders": sorted(dirs, key=lambda i: i["name"]),
+    }
+
+
+class FolderOpenInFileManagerQuery(BaseModel):
+    path: str = Field(
+        description="The path to open in the file manager",
+    )
+
+
+@api.get("/show-in-files")
+def open_in_file_manager(query: FolderOpenInFileManagerQuery):
+    """
+    Open in file manager
+
+    Opens the given path in the file manager on the host machine.
+    """
+    show_in_file_manager(query.path)
+
+    return {"success": True}
+
+
+class GetTracksInPathQuery(BaseModel):
+    path: str = Field(
+        description="The path to get tracks from",
+    )
+
+
+@api.get("/tracks/all")
+def get_tracks_in_path(query: GetTracksInPathQuery):
+    """
+    Get tracks in path
+
+    Gets all (or a max of 300) tracks from the given path and its subdirectories.
+
+    Used when adding tracks to the queue.
+    """
+    tracks = TrackTable.get_tracks_in_path(query.path)
+    tracks = (serialize_track(t) for t in tracks if Path(t.filepath).exists())
+
+    return {
+        "tracks": list(tracks)[:300],
+    }