TDvorak/swingmusic-extended
1
0
Fork 0
mirror of https://github.com/Dvorinka/swingmusic-extended.git synced 2026-06-03 20:13:02 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge pull request #147 from swing-opensource/v1.3.0

Browse Source 
New release: v1.3.0
This commit is contained in:
Mungai Njoroge
2023-10-11 14:34:40 -07:00
committed by GitHub
parent 1fc340cb95 629b3fcde3
commit c179d09d3b
118 changed files with 4802 additions and 2055 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/CODE_OF_CONDUCT.md
+13
View File
@@ -0,0 +1,13 @@
# Contributor Code of Conduct
As contributors and maintainers of this project, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities.
We are committed to making participation in this project a harassment-free experience for everyone, regardless of the level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, age, or religion.
Examples of unacceptable behavior by participants include the use of sexual language or imagery, derogatory comments or personal attacks, trolling, public or private harassment, insults, or other unprofessional conduct.
Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct. Project maintainers who do not follow the Code of Conduct may be removed from the project team.
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by opening an issue or contacting one or more of the project maintainers.
This Code of Conduct is adapted from the [Contributor Covenant](http://contributor-covenant.org), version 1.0.0, available at [http://contributor-covenant.org/version/1/0/0/](http://contributor-covenant.org/version/1/0/0/)
.github/FUNDING.yml
+9
View File
@@ -0,0 +1,9 @@
# These are supported funding model platforms
# github: [yyx990803, posva]
# patreon: evanyou
# open_collective: vuejs
# ko_fi: # Replace with a single Ko-fi username
# tidelift: npm/vue
# custom: # Replace with a single custom sponsorship URL
custom: https://swingmusic.vercel.app/support-us
.github/ISSUE_TEMPLATE/config.yml
+8
View File
@@ -0,0 +1,8 @@
blank_issues_enabled: true
contact_links:
- name: Create new issue
url: https://github.com/swing-opensource/swingmusic/issues/new
about: Please use the following link to create a new issue.
- name: Support Swing Music
url: https://swingmusic.vercel.app/support-us
about: Love Swing Music? Please consider supporting this project.
.github/PULL_REQUEST_TEMPLATE.md
+32
View File
@@ -0,0 +1,32 @@
<!--
Please make sure to read the Pull Request Guidelines:
https://github.com/swing-opensource/swingmusic/.github/CONTRIBUTING.md#pull-request-guidelines
-->
<!-- PULL REQUEST TEMPLATE -->
<!-- (Update "[ ]" to "[x]" to check a box) -->
**What kind of change does this PR introduce?** (check at least one)
- [ ] Bugfix
- [ ] Feature
- [ ] Code style update
- [ ] Refactor
- [ ] Build-related changes
- [ ] Other, please describe:
**Does this PR introduce a breaking change?** (check one)
- [ ] Yes
- [ ] No
If yes, please describe the impact and migration path for existing applications:
**The PR fulfills these requirements:**
- [ ] When resolving a specific issue, it's referenced in the PR's title (e.g. `fix #xxx[,#xxx]`, where "xxx" is the issue number)
If adding a **new feature**, the PR's description includes:
- [ ] A convincing reason for adding this feature (it's best to open a suggestion issue first and wait for approval before working on it)
**Other information:**
.github/contributing.md
+94
View File
@@ -0,0 +1,94 @@
# Swing Music Contributing Guide
Hi! We're really excited that you are interested in contributing to Swing Music. This project uses Python, [Flask](https://flask.palletsprojects.com/en/2.3.x/), Sqlite, [Poetry](https://python-poetry.org/), and [Vue](https://vuejs.org/).
If you are interested in making a code contribution take a moment to read through the following guidelines:
- [Code of Conduct](./CODE_OF_CONDUCT.md)
- [Pull Request Guidelines](#pull-request-guidelines)
## Pull Request Guidelines
- Checkout a topic branch from the relevant branch, e.g. `master`, and merge back against that branch.
- If adding a new feature:
- Provide a convincing reason to add this feature. Ideally, you should open a suggestion issue first and have it approved before working on it.
- If fixing bug:
- Provide a detailed description of the bug in the PR.
## Development Setup
This project is broken down into 2 parts. The server (this repo) and the client (which lives [here](https://github.com/swing-opensource/swingmusic-client)).
To contribute to the server development, you need to install [Poetry package manager](https://python-poetry.org/docs).
Fork this repo, git clone and install the dependencies:
```sh
git clone https://github.com/swing-opensource/swingmusic.git
# or with ssh
git clone git@github.com:swing-opensource/swingmusic.git
cd swingmusic
poetry install
```
You need a LastFM API key which you can get on the [API accounts page](https://www.last.fm/api/accounts). Then set it as an environment variable under the name: `LASTFM_API_KEY`.
Finally, run the server. You can use a different port if you have another Swing Music instance running on port `1970`.
```sh
poetry run python manage.py --port 1980
```
After that, checkout into a new branch and make your changes.
```sh
git checkout <branch_name>
```
Finally, commit your changes and open a pull request.
## Contributing to the client
You need to have [yarn](https://yarnpkg.com/) installed in your machine. See their [install guide](https://yarnpkg.com/getting-started/install).
Fork the repo, git clone and install the dependencies:
```sh
git clone https://github.com/swing-opensource/swingmusic-client.git
# or with ssh
git clone git@github.com:swing-opensource/swingmusic-client.git
cd swingmusic-client
yarn install
```
You can now run the client.
```sh
yarn dev
```
You can see the client at http://localhost:5173.
> The client is hardcoded to hook into the server on port `1980` (to allow the another server instance to be running on the default port). You can follow the instructions above to set up the server in that port, or you can change the port in `swingmusic-client/config.ts`. Don't forget to change it back when in the PR.
## Where can I go for help?
If you need help, you can email me at: geoffreymungai45@gmail.com
## What does the Code of Conduct mean for me?
Our Code of Conduct means that you are responsible for treating everyone on the project with respect and courtesy regardless of their identity. If you are the victim of any inappropriate behavior or comments as described in our Code of Conduct, we are here for you and will do the best to ensure that the abuser is reprimanded appropriately, per our code.
See you around?
.github/workflows/release.yml
+107
View File
@@ -0,0 +1,107 @@
name: Release Swing Music
on:
push:
branches:
- main
paths:
- 'client/**'
- 'server/**'
workflow_dispatch:
env:
NODE_VERSION: 18
PYTHON_VERSION: 3.10
jobs:
release-linux:
runs-on: ubuntu-20.04
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Setup Node.js
uses: actions/setup-node@v2
with:
node-version: ${{ env.NODE_VERSION }}
- name: Install client dependencies
run: yarn install
working-directory: ./client
- name: Build client
run: yarn build --outDir ../swingmusic/client
working-directory: ./client
- name: Setup Python
uses: actions/setup-python@v2
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Install server dependencies
run: poetry install
working-directory: ./server
- name: Build server
run: poetry run python manage.py --build
working-directory: ./server
- name: Create Release
id: create_release
uses: actions/create-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: ${{ github.event.head_commit.message.replace('release', '').trim() }}
release_name: ${{ github.event.head_commit.message.replace('release', '').trim() }}
draft: true
prerelease: false
- name: Upload Release Asset
id: upload-release-asset
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ./server/dist/swingmusic
asset_name: swingmusic
asset_content_type: application/octet-stream
release-windows:
runs-on: windows-latest
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Setup Node.js
uses: actions/setup-node@v2
with:
node-version: ${{ env.NODE_VERSION }}
- name: Install client dependencies
run: yarn install
working-directory: ./client
- name: Build client
run: yarn build --outDir ../swingmusic/client
working-directory: ./client
- name: Setup Python
uses: actions/setup-python@v2
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Install server dependencies
run: poetry install
working-directory: ./server
- name: Build server
run: poetry run python manage.py --build
working-directory: ./server
- name: Create Release
id: create_release
uses: actions/create-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: ${{ github.event.head_commit.message.replace('release', '').trim() }}
release_name: ${{ github.event.head_commit.message.replace('release', '').trim() }}
draft: true
prerelease: false
- name: Upload Release Asset
id: upload-release-asset
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ./server/dist/swingmusic.exe
asset_name: swingmusic.exe
asset_content_type: application/octet-stream
.gitignore
+7
View File
@@ -22,3 +22,10 @@ encoderx.py
dist
build
client
.gitignore
logs.txt
*.spec
TODO.md
testdata.py
Dockerfile
+4 -24
View File
@@ -1,22 +1,8 @@
FROM node:latest AS CLIENT
FROM ubuntu:latest
RUN git clone https://github.com/swing-opensource/swingmusic-client.git client
WORKDIR /
WORKDIR /client
RUN git checkout $(git describe --tags $(git rev-list --tags --max-count=1))
RUN yarn install
RUN yarn build
FROM python:latest
WORKDIR /app/swingmusic
COPY . .
COPY --from=CLIENT /client/dist/ client
COPY ./dist/swingmusic /swingmusic
EXPOSE 1970/tcp
@@ -24,10 +10,4 @@ VOLUME /music
VOLUME /config
RUN pip install poetry
RUN poetry config virtualenvs.create false
RUN poetry install
ENTRYPOINT ["poetry", "run", "python", "manage.py", "--host", "0.0.0.0", "--config", "/config"]
ENTRYPOINT ["/swingmusic", "--host", "0.0.0.0", "--config", "/config"]
Dockerfile.og
+35
View File
@@ -0,0 +1,35 @@
FROM node:latest AS CLIENT
RUN git clone https://github.com/swing-opensource/swingmusic-client.git client
WORKDIR /client
RUN git checkout $(git describe --tags $(git rev-list --tags --max-count=1))
RUN yarn install
RUN yarn build
FROM python:latest
WORKDIR /app/swingmusic
COPY . .
COPY --from=CLIENT /client/dist/ client
EXPOSE 1970/tcp
VOLUME /music
VOLUME /config
RUN pip install poetry
RUN poetry config virtualenvs.create false
RUN poetry install
ENV LASTFM_API_KEY="45c6776a1029a280fabd6a2c8158023d"
ENTRYPOINT ["poetry", "run", "python", "manage.py", "--host", "0.0.0.0", "--config", "/config"]
README.md
+40 -88
View File
@@ -1,24 +1,20 @@
# Swing music
![SWING MUSIC PLAYER BANNER IMAGE](screenshots/readme-artist.webp)
Swing Music is a beautiful, self-hosted music player for your local audio files. Like a cooler Spotify ... but bring your own music. Just run the app and enjoy your music library in a web browser.
![SWING MUSIC PLAYER BANNER IMAGE](screenshots/readme-album.webp)
<a href="https://swingmusic.vercel.app/support-us.html" target="_blank"><img src="screenshots/supportus.png" alt="Buy Me A Coffee" style="height: 60px !important;width: auto !important;" ></a>
![SWING MUSIC PLAYER BANNER IMAGE](screenshots/readme-playlist.webp)
| ![SWING MUSIC PLAYER BANNER IMAGE](screenshots/artist.webp) | ![SWING MUSIC PLAYER BANNER IMAGE](screenshots/album.webp) |
| ------------------------------------------------------------ | ----------------------------------------------------------- |
| ![SWING MUSIC PLAYER BANNER IMAGE](screenshots/artist2.webp) | ![SWING MUSIC PLAYER BANNER IMAGE](screenshots/album2.webp) |
---
### Make listening to your local music fun again.
`Swing` is a music player for local audio files built with both visual coolness and functionality in mind. Just run the app and enjoy your music library in a web browser.
### For screenshots, see the [screenshots page on the website](https://swingmusic.vercel.app/screenshots.html).
> Note: This project is in the early stages of development. Many features are missing but will be added with time.
For full size screenshots, visit the [website](https://swingmusic.vercel.app).
### Setup
Download the latest release from the [release page](https://github.com/geoffrey45/swingmusic/releases) and launch it. For Linux, you need to make the file executable first.
Swing Music is available as pre-compiled binaries for Windows and Linux. Just download the latest release from
the [release page](https://github.com/geoffrey45/swingmusic/releases) and launch it.
For Linux, you need to make the file executable first.
```bash
chmod a+x ./swingmusic
@@ -26,23 +22,26 @@ chmod a+x ./swingmusic
./swingmusic
```
The app should start at <http://localhost:1970> by default.
The app should start at <http://localhost:1970> by default. You can change the default port or host by using
the `--host` and `--port` flags.
```
Usage: swingmusic [options]
Options:
--build: Build the application (in development)
--host: Set the host
--port: Set the port
--show-feat, -sf: Do not extract featured artists from the song title
--show-prod, -sp: Do not hide producers in the song title
--help, -h: Show this help message
--version, -v: Show the app version
```
### Options
| Option | Short | Description |
| -------------------- | ------ | ----------------------------------------------------------------------------- |
| `--help` | `-h` | Show help message |
| `--version` | `-v` | Show the app version |
| `--host` | | Set the host |
| `--port` | | Set the port |
| `--config` | | Set the config path |
| `--no-periodic-scan` | `-nps` | Disable periodic scan |
| `--scan-interval` | `-psi` | Set the periodic scan interval in seconds. Default is 300 seconds (5 minutes) |
| `--build` | | Build the application (in development) |
To stream your music across your local network, use the `--host` flag to run the app in all ports. Like this:
```sh
@@ -53,11 +52,13 @@ The link to access the app will be printed on your terminal. Copy it and open it
### Docker
You can run Swing in a Docker container. To do so, clone the repository and build the image:
You can run Swing Music in a Docker container. To do so, clone the repository and build the image:
git clone https://github.com/swing-opensource/swingmusic.git --depth 1
cd swingmusic
docker build . -t swingmusic
```bash
git clone https://github.com/swing-opensource/swingmusic.git --depth 1
cd swingmusic
docker build . -t swingmusic
```
Then create the container. Here are some example snippets to help you get started creating a container.
@@ -74,7 +75,7 @@ services:
- /path/to/music:/music
- /path/to/config:/config
ports:
- 1970:1970
- "1970:1970"
restart: unless-stopped
```
@@ -92,71 +93,22 @@ docker run -d \
#### Parameters
Container images are configured using parameters passed at runtime (such as those above). These parameters are separated by a colon and indicate `<external>:<internal>` respectively. For example, `-p 8080:80` would expose port `80` from inside the container to be accessible from the host's IP on port `8080` outside the container.
Container images are configured using parameters passed at runtime (such as those above). These parameters are separated
by a colon and indicate `<external>:<internal>` respectively. For example, `-p 8080:80` would expose port `80` from
inside the container to be accessible from the host's IP on port `8080` outside the container.
| Parameter | Function |
| :----: | --- |
| `-p 1970` | WebUI |
| `-v /music` | Recommended directory to store your music collection. You can bind other folder if you wish. |
| `-v /config` | Configuration files. |
### Development
This project is broken down into 2. The client and the server. The client comprises of the user interface code. This part is written in Typescript, Vue 3 and SCSS. To setup the client, checkout the [swing client repo ](https://github.com/geoffrey45/swing-client) on GitHub.
The second part of this project is the server. This is the main part of the app that runs on your machine, interacts with audio files and send data to the client. It's written in Python 3.
The following instructions will guide you on how to setup the **server**.
---
The project uses [Python poetry](https://python-poetry.org) as the virtual environment manager. Follow the instructions in [their docs](https://python-poetry.org/docs/) to install it in your machine.
> It is assumed that you have `Python 3.10` or newer installed in your machine. This project uses type hinting features so older version of Python will not work. If you don't have Python installed in your machine, get it from the [python website](https://www.python.org/downloads/).
Clone this repo locally in your machine. Then install the project dependencies and start the app.
```sh
git clone https://github.com/geoffrey45/swingmusic.git
cd swingmusic
# install dependencies using poetry
poetry install
# start the app
poetry run python manage.py
```
| Parameter | Function |
| :----------: | -------------------------------------------------------------------------------------------- |
| `-p 1970` | WebUI |
| `-v /music` | Recommended directory to store your music collection. You can bind other folder if you wish. |
| `-v /config` | Configuration files. |
### Contributing
If you want to contribute to this project, feel free to open an issue or a pull request on Github. Your contributions are highly valued and appreciated. Feature suggestions, bug reports and code contribution are welcome.
See [contributing guidelines](.github/contributing.md).
### License
This software is provided to you with terms stated in the MIT License. Read the full text in the `LICENSE` file located at the root of this repository.
---
### A brain dump ...
I started working on this project on dec 2021. Why? I like listening and exploring music and I like it more when I can enjoy it (like really really). I'd been searching for cute music players for linux that allow me to manage my ever growing music library. Some of the main features I was looking for were:
- A simple and beautiful user interface (main reason)
- Creating automated daily mixes based on my listening activity.
- Ability to move files around without breaking my playlists and mixes.
- Something that can bring together all the audio files scattered all over my disks into a single place.
- Browsing related artists and albums.
- Reading albums & artists biographies and getting insights on song lyrics (kinda Genium.com-ish).
- Web browser based user interface.
- a lot more ... but I can't remember them at the moment
I've been working to make sure that most (if not all) of the features listed above are built. Some of them are done, but most are not even touched yet. A lot of work is needed and I know that it will take a lot of time to build and perfect them.
I've been keeping a small 🤥 list of a few cool features that I'd like to build in future. Some of the features listed there are outright stupid but some are cool. You can check it out in [this notion page](https://rhetorical-othnielia-565.notion.site/Cool-features-1a0cd5b797904da687bec441e7c7aa19). https://rhetorical-othnielia-565.notion.site/Cool-features-1a0cd5b797904da687bec441e7c7aa19
I have been working on this project solo, so it’s very hard to push things fast. If you have programming knowledge in Python or Vue, feel free to contribute to the project. Your contributions are highly appreciated.
---
**[MIT License](https://opensource.org/licenses/MIT) | Copyright (c) 2023 Mungai Njoroge**
app/api/__init__.py
+21 -5
View File
@@ -3,23 +3,39 @@ This module combines all API blueprints into a single Flask app instance.
"""
from flask import Flask
from flask_compress import Compress
from flask_cors import CORS
from app.api import (album, artist, favorites, folder, imgserver, playlist,
search, settings, track, colors)
from app.api import (
album,
artist,
colors,
favorites,
folder,
imgserver,
playlist,
search,
send_file,
settings,
)
def create_api():
"""
Creates the Flask instance, registers modules and registers all the API blueprints.
"""
app = Flask(__name__, static_url_path="")
CORS(app)
app = Flask(__name__)
CORS(app, origins="*")
Compress(app)
app.config["COMPRESS_MIMETYPES"] = [
"application/json",
]
with app.app_context():
app.register_blueprint(album.api)
app.register_blueprint(artist.api)
app.register_blueprint(track.api)
app.register_blueprint(send_file.api)
app.register_blueprint(search.api)
app.register_blueprint(folder.api)
app.register_blueprint(playlist.api)
app/api/album.py
+100 -20
View File
@@ -2,17 +2,20 @@
Contains all the album routes.
"""
from dataclasses import asdict
import random
from flask import Blueprint, request
from app.db.sqlite.albums import SQLiteAlbumMethods as adb
from app.db.sqlite.albumcolors import SQLiteAlbumMethods as adb
from app.db.sqlite.favorite import SQLiteFavoriteMethods as favdb
from app.db.sqlite.lastfm.similar_artists import SQLiteLastFMSimilarArtists as lastfmdb
from app.lib.albumslib import sort_by_track_no
from app.models import FavType, Track
from app.utils.remove_duplicates import remove_duplicates
from app.store.tracks import TrackStore
from app.serializers.album import serialize_for_card
from app.serializers.track import serialize_track
from app.store.albums import AlbumStore
from app.store.tracks import TrackStore
from app.utils.hashing import create_hash
get_albums_by_albumartist = adb.get_albums_by_albumartist
check_is_fav = favdb.check_is_favorite
@@ -33,7 +36,7 @@ def get_album_tracks_and_info():
return error_msg, 400
try:
albumhash = data["hash"]
albumhash: str = data["albumhash"]
except KeyError:
return error_msg, 400
@@ -61,13 +64,12 @@ def get_album_tracks_and_info():
return list(genres)
album.genres = get_album_genres(tracks)
tracks = remove_duplicates(tracks)
album.count = len(tracks)
album.get_date_from_tracks(tracks)
try:
album.duration = sum((t.duration for t in tracks))
album.duration = sum(t.duration for t in tracks)
except AttributeError:
album.duration = 0
@@ -78,7 +80,10 @@ def get_album_tracks_and_info():
album.is_favorite = check_is_fav(albumhash, FavType.album)
return {"tracks": tracks, "info": album}
return {
"tracks": [serialize_track(t, remove_disc=False) for t in tracks],
"info": album,
}
@api.route("/album/<albumhash>/tracks", methods=["GET"])
@@ -87,13 +92,7 @@ def get_album_tracks(albumhash: str):
Returns all the tracks in the given album, sorted by disc and track number.
"""
tracks = TrackStore.get_tracks_by_albumhash(albumhash)
tracks = [asdict(t) for t in tracks]
for t in tracks:
track = str(t["track"]).zfill(3)
t["pos"] = int(f"{t['disc']}{track}")
tracks = sorted(tracks, key=lambda t: t["pos"])
tracks = sort_by_track_no(tracks)
return {"tracks": tracks}
@@ -107,18 +106,99 @@ def get_artist_albums():
albumartists: str = data["albumartists"]
limit: int = data.get("limit")
exclude: str = data.get("exclude")
base_title: str = data.get("base_title")
albumartists: list[str] = albumartists.split(",")
albums = [
{
"artisthash": a,
"albums": AlbumStore.get_albums_by_albumartist(a, limit, exclude=exclude),
"albums": AlbumStore.get_albums_by_albumartist(
a, limit, exclude=base_title
),
}
for a in albumartists
]
albums = [a for a in albums if len(a["albums"]) > 0]
albums = [
{
"artisthash": a["artisthash"],
"albums": [serialize_for_card(a_) for a_ in (a["albums"])],
}
for a in albums
if len(a["albums"]) > 0
]
return {"data": albums}
@api.route("/album/versions", methods=["POST"])
def get_album_versions():
"""
Returns other versions of the given album.
"""
data = request.get_json()
if data is None:
return {"msg": "No albumartist provided"}
og_album_title: str = data["og_album_title"]
base_title: str = data["base_title"]
artisthash: str = data["artisthash"]
albums = AlbumStore.get_albums_by_artisthash(artisthash)
albums = [
a
for a in albums
if create_hash(a.base_title) == create_hash(base_title)
and create_hash(og_album_title) != create_hash(a.og_title)
]
for a in albums:
tracks = TrackStore.get_tracks_by_albumhash(a.albumhash)
a.get_date_from_tracks(tracks)
return {"data": albums}
@api.route("/album/similar", methods=["GET"])
def get_similar_albums():
"""
Returns similar albums to the given album.
"""
data = request.args
if data is None:
return {"msg": "No artisthash provided"}
artisthash: str = data["artisthash"]
limit: int = data.get("limit")
if limit is None:
limit = 6
limit = int(limit)
similar_artists = lastfmdb.get_similar_artists_for(artisthash)
if similar_artists is None:
return {"albums": []}
artisthashes = similar_artists.get_artist_hash_set()
if len(artisthashes) == 0:
return {"albums": []}
albums = [AlbumStore.get_albums_by_artisthash(a) for a in artisthashes]
albums = [a for sublist in albums for a in sublist]
albums = list({a.albumhash: a for a in albums}.values())
try:
albums = random.sample(albums, min(len(albums), limit))
except ValueError:
pass
return {"albums": [serialize_for_card(a) for a in albums[:limit]]}
app/api/artist.py
+129 -207
View File
@@ -2,164 +2,34 @@
Contains all the artist(s) routes.
"""
import random
from collections import deque
import math
from datetime import datetime
from flask import Blueprint, request
from app.db.sqlite.favorite import SQLiteFavoriteMethods as favdb
from app.models import Album, FavType, Track
from app.utils.remove_duplicates import remove_duplicates
from app.requests.artists import fetch_similar_artists
from app.db.sqlite.lastfm.similar_artists import SQLiteLastFMSimilarArtists as fmdb
from app.models import Album, FavType
from app.serializers.album import serialize_for_card_many
from app.serializers.track import serialize_tracks
from app.store.albums import AlbumStore
from app.store.tracks import TrackStore
from app.store.artists import ArtistStore
from app.store.tracks import TrackStore
from app.telemetry import Telemetry
from app.utils.threading import background
api = Blueprint("artist", __name__, url_prefix="/")
class CacheEntry:
"""
The cache entry class for the artists cache.
"""
def __init__(
self, artisthash: str, albumhashes: set[str], tracks: list[Track]
) -> None:
self.albums: list[Album] = []
self.tracks: list[Track] = []
self.artisthash: str = artisthash
self.albumhashes: set[str] = albumhashes
if len(tracks) > 0:
self.tracks: list[Track] = tracks
self.type_checked = False
self.albums_fetched = False
ARTIST_VISIT_COUNT = 0
class ArtistsCache:
"""
Holds artist page cache.
"""
@background
def send_event():
global ARTIST_VISIT_COUNT
ARTIST_VISIT_COUNT += 1
artists: deque[CacheEntry] = deque(maxlen=1)
@classmethod
def get_albums_by_artisthash(cls, artisthash: str):
"""
Returns the cached albums for the given artisthash.
"""
for (index, albums) in enumerate(cls.artists):
if albums.artisthash == artisthash:
return albums.albums, index
return [], -1
@classmethod
def albums_cached(cls, artisthash: str) -> bool:
"""
Returns True if the artist is in the cache.
"""
for entry in cls.artists:
if entry.artisthash == artisthash and len(entry.albums) > 0:
return True
return False
@classmethod
def albums_fetched(cls, artisthash: str):
"""
Checks if the albums have been fetched for the given artisthash.
"""
for entry in cls.artists:
if entry.artisthash == artisthash:
return entry.albums_fetched
@classmethod
def tracks_cached(cls, artisthash: str) -> bool:
"""
Checks if the tracks have been cached for the given artisthash.
"""
for entry in cls.artists:
if entry.artisthash == artisthash and len(entry.tracks) > 0:
return True
return False
@classmethod
def add_entry(cls, artisthash: str, albumhashes: set[str], tracks: list[Track]):
"""
Adds a new entry to the cache.
"""
cls.artists.append(CacheEntry(artisthash, albumhashes, tracks))
@classmethod
def get_tracks(cls, artisthash: str):
"""
Returns the cached tracks for the given artisthash.
"""
entry = [a for a in cls.artists if a.artisthash == artisthash][0]
return entry.tracks
@classmethod
def get_albums(cls, artisthash: str):
"""
Returns the cached albums for the given artisthash.
"""
entry = [a for a in cls.artists if a.artisthash == artisthash][0]
albums = [AlbumStore.get_album_by_hash(h) for h in entry.albumhashes]
entry.albums = [album for album in albums if album is not None]
store_albums = AlbumStore.get_albums_by_artisthash(artisthash)
all_albums_hash = "-".join([a.albumhash for a in entry.albums])
for album in store_albums:
if album.albumhash not in all_albums_hash:
entry.albums.append(album)
entry.albums_fetched = True
@classmethod
def process_album_type(cls, artisthash: str):
"""
Checks the cached albums type for the given artisthash.
"""
entry = [a for a in cls.artists if a.artisthash == artisthash][0]
for album in entry.albums:
album.check_type()
album_tracks = TrackStore.get_tracks_by_albumhash(album.albumhash)
album_tracks = remove_duplicates(album_tracks)
album.get_date_from_tracks(album_tracks)
album.check_is_single(album_tracks)
entry.type_checked = True
def add_albums_to_cache(artisthash: str):
"""
Fetches albums and adds them to the cache.
"""
tracks = TrackStore.get_tracks_by_artist(artisthash)
if len(tracks) == 0:
return False
albumhashes = set(t.albumhash for t in tracks)
ArtistsCache.add_entry(artisthash, albumhashes, [])
return True
# =======================================================
# ===================== ROUTES ==========================
# =======================================================
if ARTIST_VISIT_COUNT % 5 == 0:
Telemetry.send_artist_visited()
@api.route("/artist/<artisthash>", methods=["GET"])
@@ -167,6 +37,7 @@ def get_artist(artisthash: str):
"""
Get artist data.
"""
send_event()
limit = request.args.get("limit")
if limit is None:
@@ -179,20 +50,7 @@ def get_artist(artisthash: str):
if artist is None:
return {"error": "Artist not found"}, 404
tracks_cached = ArtistsCache.tracks_cached(artisthash)
if tracks_cached:
tracks = ArtistsCache.get_tracks(artisthash)
else:
tracks = TrackStore.get_tracks_by_artist(artisthash)
albumhashes = set(t.albumhash for t in tracks)
hashes_from_albums = set(
a.albumhash for a in AlbumStore.get_albums_by_artisthash(artisthash)
)
albumhashes = albumhashes.union(hashes_from_albums)
ArtistsCache.add_entry(artisthash, albumhashes, tracks)
tracks = TrackStore.get_tracks_by_artisthash(artisthash)
tcount = len(tracks)
acount = AlbumStore.count_albums_by_artisthash(artisthash)
@@ -205,7 +63,34 @@ def get_artist(artisthash: str):
artist.is_favorite = favdb.check_is_favorite(artisthash, FavType.artist)
return {"artist": artist, "tracks": tracks[:limit]}
genres = set()
for t in tracks:
if t.genre is not None:
genres = genres.union(t.genre)
genres = list(genres)
try:
min_stamp = min(t.date for t in tracks)
year = datetime.fromtimestamp(min_stamp).year
except ValueError:
year = 0
decade = None
if year:
decade = math.floor(year / 10) * 10
decade = str(decade)[2:] + "s"
if decade:
genres.insert(0, decade)
return {
"artist": artist,
"tracks": serialize_tracks(tracks[:limit]),
"genres": genres,
}
@api.route("/artist/<artisthash>/albums", methods=["GET"])
@@ -219,28 +104,53 @@ def get_artist_albums(artisthash: str):
limit = int(limit)
is_cached = ArtistsCache.albums_cached(artisthash)
all_albums = AlbumStore.get_albums_by_artisthash(artisthash)
if not is_cached:
add_albums_to_cache(artisthash)
# start: check for missing albums. ie. compilations and features
all_tracks = TrackStore.get_tracks_by_artisthash(artisthash)
albums_fetched = ArtistsCache.albums_fetched(artisthash)
track_albums = set(t.albumhash for t in all_tracks)
missing_album_hashes = track_albums.difference(set(a.albumhash for a in all_albums))
if not albums_fetched:
ArtistsCache.get_albums(artisthash)
if len(missing_album_hashes) > 0:
missing_albums = AlbumStore.get_albums_by_hashes(list(missing_album_hashes))
all_albums.extend(missing_albums)
all_albums, index = ArtistsCache.get_albums_by_artisthash(artisthash)
# end check
if not ArtistsCache.artists[index].type_checked:
ArtistsCache.process_album_type(artisthash)
def get_album_tracks(albumhash: str):
tracks = [t for t in all_tracks if t.albumhash == albumhash]
if len(tracks) > 0:
return tracks
return TrackStore.get_tracks_by_albumhash(albumhash)
for a in all_albums:
a.check_type()
album_tracks = get_album_tracks(a.albumhash)
if len(album_tracks) == 0:
continue
a.get_date_from_tracks(album_tracks)
if a.date == 0:
AlbumStore.remove_album_by_hash(a.albumhash)
continue
a.check_is_single(album_tracks)
all_albums = sorted(all_albums, key=lambda a: str(a.date), reverse=True)
singles = [a for a in all_albums if a.is_single]
eps = [a for a in all_albums if a.is_EP]
def remove_EPs_and_singles(albums: list[Album]):
albums = [a for a in albums if not a.is_EP]
albums = [a for a in albums if not a.is_single]
return albums
def remove_EPs_and_singles(albums_: list[Album]):
albums_ = [a for a in albums_ if not a.is_single]
albums_ = [a for a in albums_ if not a.is_EP]
return albums_
albums = filter(lambda a: artisthash in a.albumartists_hashes, all_albums)
albums = list(albums)
@@ -257,16 +167,20 @@ def get_artist_albums(artisthash: str):
artist = ArtistStore.get_artist_by_hash(artisthash)
if return_all is not None:
if artist is None:
return {"error": "Artist not found"}, 404
if return_all is not None and return_all == "true":
limit = len(all_albums)
singles_and_eps = singles + eps
return {
"artistname": artist.name,
"albums": albums[:limit],
"singles": singles[:limit],
"eps": eps[:limit],
"appearances": appearances[:limit],
"compilations": compilations[:limit]
"albums": serialize_for_card_many(albums[:limit]),
"singles_and_eps": serialize_for_card_many(singles_and_eps[:limit]),
"appearances": serialize_for_card_many(appearances[:limit]),
"compilations": serialize_for_card_many(compilations[:limit]),
}
@@ -275,32 +189,40 @@ def get_all_artist_tracks(artisthash: str):
"""
Returns all artists by a given artist.
"""
tracks = TrackStore.get_tracks_by_artist(artisthash)
tracks = TrackStore.get_tracks_by_artisthash(artisthash)
return {"tracks": tracks}
return {"tracks": serialize_tracks(tracks)}
#
# @api.route("/artist/<artisthash>/similar", methods=["GET"])
# def get_similar_artists(artisthash: str):
# """
# Returns similar artists.
# """
# limit = request.args.get("limit")
#
# if limit is None:
# limit = 6
#
# limit = int(limit)
#
# artist = ArtistStore.get_artist_by_hash(artisthash)
#
# if artist is None:
# return {"error": "Artist not found"}, 404
#
# similar_hashes = fetch_similar_artists(artist.name)
# similar = ArtistStore.get_artists_by_hashes(similar_hashes)
#
# if len(similar) > limit:
# similar = random.sample(similar, limit)
#
# return {"similar": similar[:limit]}
@api.route("/artist/<artisthash>/similar", methods=["GET"])
def get_similar_artists(artisthash: str):
"""
Returns similar artists.
"""
limit = request.args.get("limit")
if limit is None:
limit = 6
limit = int(limit)
artist = ArtistStore.get_artist_by_hash(artisthash)
if artist is None:
return {"error": "Artist not found"}, 404
# result = LastFMStore.get_similar_artists_for(artist.artisthash)
result = fmdb.get_similar_artists_for(artist.artisthash)
if result is None:
return {"artists": []}
similar = ArtistStore.get_artists_by_hashes(result.get_artist_hash_set())
if len(similar) > limit:
similar = random.sample(similar, limit)
return {"artists": similar[:limit]}
# TODO: Rewrite this file using generators where possible
app/api/colors.py
+5 -7
View File
@@ -8,11 +8,9 @@ api = Blueprint("colors", __name__, url_prefix="/colors")
def get_album_color(albumhash: str):
album = Store.get_album_by_hash(albumhash)
if len(album.colors) > 0:
return {
"color": album.colors[0]
}
msg = {"color": ""}
return {
"color": ""
}
if album is None or len(album.colors) == 0:
return msg, 404
return {"color": album.colors[0]}
app/api/favorites.py
+44 -21
View File
@@ -2,11 +2,17 @@ from flask import Blueprint, request
from app.db.sqlite.favorite import SQLiteFavoriteMethods as favdb
from app.models import FavType
from app.serializers.album import serialize_for_card_many
from app.serializers.track import serialize_tracks
from app.utils.bisection import UseBisection
from app.store.artists import ArtistStore
from app.store.albums import AlbumStore
from app.store.tracks import TrackStore
from app.serializers.favorites_serializer import (
recent_fav_album_serializer,
recent_fav_artist_serializer,
)
api = Blueprint("favorite", __name__, url_prefix="/")
@@ -78,7 +84,7 @@ def get_favorite_albums():
if limit == 0:
limit = len(albums)
return {"albums": fav_albums[:limit]}
return {"albums": serialize_for_card_many(fav_albums[:limit])}
@api.route("/tracks/favorite")
@@ -101,7 +107,7 @@ def get_favorite_tracks():
if limit == 0:
limit = len(tracks)
return {"tracks": tracks[:limit]}
return {"tracks": serialize_tracks(tracks[:limit])}
@api.route("/artists/favorite")
@@ -149,52 +155,69 @@ def get_all_favorites():
track_limit = int(track_limit)
album_limit = int(album_limit)
artist_limit = int(artist_limit)
largest = max(track_limit, album_limit, artist_limit)
favs = favdb.get_all()
favs.reverse()
favs = [fav for fav in favs if fav[1] != ""]
tracks = []
albums = []
artists = []
for fav in favs:
if (
len(tracks) >= track_limit
and len(albums) >= album_limit
and len(artists) >= artist_limit
):
break
if not len(tracks) >= track_limit:
if not len(tracks) >= largest:
if fav[2] == FavType.track:
tracks.append(fav[1])
if not len(albums) >= album_limit:
if fav[2] == FavType.album:
albums.append(fav[1])
if not len(artists) >= artist_limit:
if not len(artists) >= largest:
if fav[2] == FavType.artist:
artists.append(fav[1])
if fav[2] == FavType.album:
albums.append(fav[1])
src_tracks = sorted(TrackStore.tracks, key=lambda x: x.trackhash)
src_albums = sorted(AlbumStore.albums, key=lambda x: x.albumhash)
src_artists = sorted(ArtistStore.artists, key=lambda x: x.artisthash)
tracks = UseBisection(src_tracks, "trackhash", tracks)()
albums = UseBisection(src_albums, "albumhash", albums)()
albums = UseBisection(src_albums, "albumhash", albums, limit=album_limit)()
artists = UseBisection(src_artists, "artisthash", artists)()
tracks = remove_none(tracks)
albums = remove_none(albums)
artists = remove_none(artists)
recents = []
# first_n = favs
for fav in favs:
if len(recents) >= largest:
break
if fav[2] == FavType.album:
try:
album = [a for a in albums if a.albumhash == fav[1]][0]
recents.append(
{"type": "album", "item": recent_fav_album_serializer(album)}
)
except IndexError:
continue
if fav[2] == FavType.artist:
try:
artist = [a for a in artists if a.artisthash == fav[1]][0]
recents.append(
{"type": "artist", "item": recent_fav_artist_serializer(artist)}
)
except IndexError:
continue
return {
"tracks": tracks,
"albums": albums,
"artists": artists,
"recents": recents[:album_limit],
"tracks": tracks[:track_limit],
"albums": albums[:album_limit],
"artists": artists[:artist_limit],
}
app/api/folder.py
+36 -5
View File
@@ -2,17 +2,20 @@
Contains all the folder routes.
"""
import os
import psutil
from pathlib import Path
import psutil
from flask import Blueprint, request
from showinfm import show_in_file_manager
from app import settings
from app.lib.folderslib import GetFilesAndDirs, get_folders
from app.db.sqlite.settings import SettingsSQLMethods as db
from app.utils.wintools import win_replace_slash, is_windows
from app.lib.folderslib import GetFilesAndDirs, get_folders
from app.serializers.track import serialize_track
from app.store.tracks import TrackStore as store
from app.utils.wintools import is_windows, win_replace_slash
api = Blueprint("folder", __name__, url_prefix="/")
api = Blueprint("folder", __name__, url_prefix="")
@api.route("/folder", methods=["POST"])
@@ -116,3 +119,31 @@ def list_folders():
return {
"folders": sorted(dirs, key=lambda i: i["name"]),
}
@api.route("/folder/show-in-files")
def open_in_file_manager():
path = request.args.get("path")
if path is None:
return {"error": "No path provided."}, 400
show_in_file_manager(path)
return {"success": True}
@api.route("/folder/tracks/all")
def get_tracks_in_path():
path = request.args.get("path")
if path is None:
return {"error": "No path provided."}, 400
tracks = store.get_tracks_in_path(path)
tracks = sorted(tracks, key=lambda i: i.last_mod)
tracks = (serialize_track(t) for t in tracks if Path(t.filepath).exists())
return {
"tracks": list(tracks)[:300],
}
app/api/imgserver.py
+11
View File
@@ -22,6 +22,17 @@ def send_fallback_img(filename: str = "default.webp"):
return send_from_directory(path, filename)
@api.route("/t/o/<imgpath>")
def send_original_thumbnail(imgpath: str):
path = Paths.get_original_thumb_path()
fpath = Path(path) / imgpath
if fpath.exists():
return send_from_directory(path, imgpath)
return send_fallback_img()
@api.route("/t/<imgpath>")
def send_lg_thumbnail(imgpath: str):
path = Paths.get_lg_thumb_path()
app/api/playlist.py
+252 -70
View File
@@ -3,35 +3,27 @@ All playlist-related routes.
"""
import json
from datetime import datetime
import pathlib
from flask import Blueprint, request
from PIL import UnidentifiedImageError
from PIL import UnidentifiedImageError, Image
from app import models
from app.db.sqlite.playlists import SQLitePlaylistMethods
from app.lib import playlistlib
from app.utils.dates import date_string_to_time_passed, create_new_date
from app.utils.remove_duplicates import remove_duplicates
from app.store.tracks import TrackStore
from app.lib.albumslib import sort_by_track_no
from app.models.track import Track
from app.store.albums import AlbumStore
from app.store.tracks import TrackStore
from app.utils.dates import create_new_date, date_string_to_time_passed
from app.utils.remove_duplicates import remove_duplicates
from app.settings import Paths
api = Blueprint("playlist", __name__, url_prefix="/")
PL = SQLitePlaylistMethods
insert_one_playlist = PL.insert_one_playlist
get_playlist_by_name = PL.get_playlist_by_name
count_playlist_by_name = PL.count_playlist_by_name
get_all_playlists = PL.get_all_playlists
get_playlist_by_id = PL.get_playlist_by_id
tracks_to_playlist = PL.add_tracks_to_playlist
add_artist_to_playlist = PL.add_artist_to_playlist
update_playlist = PL.update_playlist
delete_playlist = PL.delete_playlist
# get_tracks_by_trackhashes = SQLiteTrackMethods.get_tracks_by_trackhashes
def duplicate_images(images: list):
if len(images) == 1:
images *= 4
@@ -43,24 +35,33 @@ def duplicate_images(images: list):
return images
def get_first_4_images(trackhashes: list[str]) -> list[dict['str', str]]:
tracks = TrackStore.get_tracks_by_trackhashes(trackhashes)
def get_first_4_images(
tracks: list[Track] = [], trackhashes: list[str] = []
) -> list[dict["str", str]]:
if len(trackhashes) > 0:
tracks = TrackStore.get_tracks_by_trackhashes(trackhashes)
albums = []
for track in tracks:
if track.albumhash not in albums:
albums.append(track.albumhash)
if len(albums) == 4:
break
albums = AlbumStore.get_albums_by_hashes(albums)
images = [
{
'image': album.image,
'color': ''.join(album.colors),
"image": album.image,
"color": "".join(album.colors),
}
for album in albums
]
if len(images) == 4:
return images
return duplicate_images(images)
@@ -69,16 +70,15 @@ def send_all_playlists():
"""
Gets all the playlists.
"""
# get the no_images query param
no_images = request.args.get("no_images", False)
playlists = get_all_playlists()
playlists = PL.get_all_playlists()
playlists = list(playlists)
for playlist in playlists:
if not no_images:
playlist.images = get_first_4_images(playlist.trackhashes)
playlist.images = [img['image'] for img in playlist.images]
playlist.images = get_first_4_images(trackhashes=playlist.trackhashes)
playlist.images = [img["image"] for img in playlist.images]
playlist.clear_lists()
@@ -90,6 +90,25 @@ def send_all_playlists():
return {"data": playlists}
def insert_playlist(name: str, image: str = None):
playlist = {
"image": image,
"last_updated": create_new_date(),
"name": name,
"trackhashes": json.dumps([]),
"settings": json.dumps(
{
"has_gif": False,
"banner_pos": 50,
"square_img": True if image else False,
"pinned": False,
}
),
}
return PL.insert_one_playlist(playlist)
@api.route("/playlist/new", methods=["POST"])
def create_playlist():
"""
@@ -100,22 +119,12 @@ def create_playlist():
if data is None:
return {"error": "Playlist name not provided"}, 400
existing_playlist_count = count_playlist_by_name(data["name"])
existing_playlist_count = PL.count_playlist_by_name(data["name"])
if existing_playlist_count > 0:
return {"error": "Playlist already exists"}, 409
playlist = {
"artisthashes": json.dumps([]),
"banner_pos": 50,
"has_gif": 0,
"image": None,
"last_updated": create_new_date(),
"name": data["name"],
"trackhashes": json.dumps([]),
}
playlist = insert_one_playlist(playlist)
playlist = insert_playlist(data["name"])
if playlist is None:
return {"error": "Playlist could not be created"}, 500
@@ -123,8 +132,35 @@ def create_playlist():
return {"playlist": playlist}, 201
def get_path_trackhashes(path: str):
"""
Returns a list of trackhashes in a folder.
"""
tracks = TrackStore.get_tracks_in_path(path)
tracks = sorted(tracks, key=lambda t: t.last_mod)
return [t.trackhash for t in tracks]
def get_album_trackhashes(albumhash: str):
"""
Returns a list of trackhashes in an album.
"""
tracks = TrackStore.get_tracks_by_albumhash(albumhash)
tracks = sort_by_track_no(tracks)
return [t["trackhash"] for t in tracks]
def get_artist_trackhashes(artisthash: str):
"""
Returns a list of trackhashes for an artist.
"""
tracks = TrackStore.get_tracks_by_artisthash(artisthash)
return [t.trackhash for t in tracks]
@api.route("/playlist/<playlist_id>/add", methods=["POST"])
def add_track_to_playlist(playlist_id: str):
def add_item_to_playlist(playlist_id: str):
"""
Takes a playlist ID and a track hash, and adds the track to the playlist
"""
@@ -133,15 +169,31 @@ def add_track_to_playlist(playlist_id: str):
if data is None:
return {"error": "Track hash not provided"}, 400
trackhash = data["track"]
try:
itemtype = data["itemtype"]
except KeyError:
itemtype = None
insert_count = tracks_to_playlist(int(playlist_id), [trackhash])
try:
itemhash: str = data["itemhash"]
except KeyError:
itemhash = None
if itemtype == "tracks":
trackhashes = itemhash.split(",")
elif itemtype == "folder":
trackhashes = get_path_trackhashes(itemhash)
elif itemtype == "album":
trackhashes = get_album_trackhashes(itemhash)
elif itemtype == "artist":
trackhashes = get_artist_trackhashes(itemhash)
else:
trackhashes = []
insert_count = PL.add_tracks_to_playlist(int(playlist_id), trackhashes)
if insert_count == 0:
return {"error": "Track already exists in playlist"}, 409
add_artist_to_playlist(int(playlist_id), trackhash)
PL.update_last_updated(int(playlist_id))
return {"error": "Item already exists in playlist"}, 409
return {"msg": "Done"}, 200
@@ -151,7 +203,10 @@ def get_playlist(playlistid: str):
"""
Gets a playlist by id, and if it exists, it gets all the tracks in the playlist and returns them.
"""
playlist = get_playlist_by_id(int(playlistid))
no_tracks = request.args.get("no_tracks", False)
no_tracks = no_tracks == "true"
playlist = PL.get_playlist_by_id(int(playlistid))
if playlist is None:
return {"msg": "Playlist not found"}, 404
@@ -163,17 +218,14 @@ def get_playlist(playlistid: str):
playlist.last_updated = date_string_to_time_passed(playlist.last_updated)
playlist.set_duration(duration)
playlist.set_count(len(tracks))
if not playlist.has_image:
playlist.images = get_first_4_images(playlist.trackhashes)
if len(playlist.images) > 2:
# swap 3rd image with first (3rd image is the visible image in UI)
playlist.images[2], playlist.images[0] = playlist.images[0], playlist.images[2]
playlist.images = get_first_4_images(tracks)
playlist.clear_lists()
return {"info": playlist, "tracks": tracks}
return {"info": playlist, "tracks": tracks if not no_tracks else []}
@api.route("/playlist/<playlistid>/update", methods=["PUT"])
@@ -181,7 +233,7 @@ def update_playlist_info(playlistid: str):
if playlistid is None:
return {"error": "Playlist ID not provided"}, 400
db_playlist = get_playlist_by_id(int(playlistid))
db_playlist = PL.get_playlist_by_id(int(playlistid))
if db_playlist is None:
return {"error": "Playlist not found"}, 404
@@ -193,34 +245,36 @@ def update_playlist_info(playlistid: str):
data = request.form
settings = json.loads(data.get("settings"))
settings["has_gif"] = False
playlist = {
"id": int(playlistid),
"artisthashes": json.dumps([]),
"banner_pos": db_playlist.banner_pos,
"has_gif": 0,
"image": db_playlist.image,
"last_updated": create_new_date(),
"name": str(data.get("name")).strip(),
"settings": settings,
"trackhashes": json.dumps([]),
}
if image:
try:
playlist["image"] = playlistlib.save_p_image(image, playlistid)
pil_image = Image.open(image)
content_type = image.content_type
playlist["image"] = playlistlib.save_p_image(
pil_image, playlistid, content_type
)
if image.content_type == "image/gif":
playlist["has_gif"] = 1
# reset banner position to center.
playlist["banner_pos"] = 50
PL.update_banner_pos(int(playlistid), 50)
playlist["settings"]["has_gif"] = True
except UnidentifiedImageError:
return {"error": "Failed: Invalid image"}, 400
p_tuple = (*playlist.values(),)
update_playlist(int(playlistid), playlist)
PL.update_playlist(int(playlistid), playlist)
playlist = models.Playlist(*p_tuple)
playlist.last_updated = date_string_to_time_passed(playlist.last_updated)
@@ -230,6 +284,52 @@ def update_playlist_info(playlistid: str):
}
@api.route("/playlist/<playlistid>/pin_unpin", methods=["GET"])
def pin_unpin_playlist(playlistid: str):
"""
Pins or unpins a playlist.
"""
playlist = PL.get_playlist_by_id(int(playlistid))
if playlist is None:
return {"error": "Playlist not found"}, 404
settings = playlist.settings
try:
settings["pinned"] = not settings["pinned"]
except KeyError:
settings["pinned"] = True
PL.update_settings(int(playlistid), settings)
return {"msg": "Done"}, 200
@api.route("/playlist/<playlistid>/remove-img", methods=["GET"])
def remove_playlist_image(playlistid: str):
"""
Removes the playlist image.
"""
pid = int(playlistid)
playlist = PL.get_playlist_by_id(pid)
if playlist is None:
return {"error": "Playlist not found"}, 404
PL.remove_banner(pid)
playlist.image = None
playlist.thumb = None
playlist.settings["has_gif"] = False
playlist.has_image = False
playlist.images = get_first_4_images(trackhashes=playlist.trackhashes)
playlist.last_updated = date_string_to_time_passed(playlist.last_updated)
return {"playlist": playlist}, 200
@api.route("/playlist/delete", methods=["POST"])
def remove_playlist():
"""
@@ -246,24 +346,106 @@ def remove_playlist():
except KeyError:
return message, 400
delete_playlist(pid)
PL.delete_playlist(pid)
return {"msg": "Done"}, 200
@api.route("/playlist/<pid>/set-image-pos", methods=["POST"])
def update_image_position(pid: int):
@api.route("/playlist/<pid>/remove-tracks", methods=["POST"])
def remove_tracks_from_playlist(pid: int):
data = request.get_json()
message = {"msg": "No data provided"}
if data is None:
return message, 400
return {"error": "Track index not provided"}, 400
# {
# trackhash: str;
# index: int;
# }
tracks = data["tracks"]
PL.remove_tracks_from_playlist(pid, tracks)
return {"msg": "Done"}, 200
def playlist_exists(name: str) -> bool:
return PL.count_playlist_by_name(name) > 0
@api.route("/playlist/save-item", methods=["POST"])
def save_item_as_playlist():
data = request.get_json()
msg = {"error": "'itemtype', 'playlist_name' and 'itemhash' not provided"}, 400
if data is None:
return msg
try:
pos = data["pos"]
playlist_name = data["playlist_name"]
except KeyError:
return message, 400
playlist_name = None
PL.update_banner_pos(pid, pos)
if playlist_exists(playlist_name):
return {"error": "Playlist already exists"}, 409
return {"msg": "Image position saved"}, 200
try:
itemtype = data["itemtype"]
except KeyError:
itemtype = None
try:
itemhash: str = data["itemhash"]
except KeyError:
itemhash = None
if itemtype is None or playlist_name is None or itemhash is None:
return msg
if itemtype == "tracks":
trackhashes = itemhash.split(",")
elif itemtype == "folder":
trackhashes = get_path_trackhashes(itemhash)
elif itemtype == "album":
trackhashes = get_album_trackhashes(itemhash)
elif itemtype == "artist":
trackhashes = get_artist_trackhashes(itemhash)
else:
trackhashes = []
if len(trackhashes) == 0:
return {"error": "No tracks founds"}, 404
image = (
itemhash + ".webp" if itemtype != "folder" and itemtype != "tracks" else None
)
playlist = insert_playlist(playlist_name, image)
if playlist is None:
return {"error": "Playlist could not be created"}, 500
# save image
if itemtype != "folder" and itemtype != "tracks":
filename = itemhash + ".webp"
base_path = (
Paths.get_artist_img_lg_path()
if itemtype == "artist"
else Paths.get_lg_thumb_path()
)
img_path = pathlib.Path(base_path + "/" + filename)
if img_path.exists():
img = Image.open(img_path)
playlistlib.save_p_image(
img, str(playlist.id), "image/webp", filename=filename
)
PL.add_tracks_to_playlist(playlist.id, trackhashes)
playlist.set_count(len(trackhashes))
images = get_first_4_images(trackhashes=trackhashes)
playlist.images = [img["image"] for img in images]
return {"playlist": playlist}, 201
app/api/search.py
+50 -87
View File
@@ -2,13 +2,11 @@
Contains all the search routes.
"""
from unidecode import unidecode
from flask import Blueprint, request
from unidecode import unidecode
from app import models
from app.lib import searchlib
from app.store.tracks import TrackStore
api = Blueprint("search", __name__, url_prefix="/")
@@ -17,74 +15,52 @@ SEARCH_COUNT = 12
"""The max amount of items to return per request"""
class SearchResults:
def query_in_quotes(query: str) -> bool:
"""
Holds all the search results.
Returns True if the query is in quotes
"""
query: str = ""
tracks: list[models.Track] = []
albums: list[models.Album] = []
playlists: list[models.Playlist] = []
artists: list[models.Artist] = []
try:
return query.startswith('"') and query.endswith('"')
except AttributeError:
return False
class Search:
def __init__(self, query: str) -> None:
self.tracks: list[models.Track] = []
self.query = unidecode(query)
SearchResults.query = self.query
def search_tracks(self):
def search_tracks(self, in_quotes=False):
"""
Calls :class:`SearchTracks` which returns the tracks that fuzzily match
the search terms. Then adds them to the `SearchResults` store.
"""
self.tracks = TrackStore.tracks
tracks = searchlib.SearchTracks(self.query)()
SearchResults.tracks = tracks
return tracks
return searchlib.TopResults().search(
self.query, tracks_only=True, in_quotes=in_quotes
)
def search_artists(self):
"""Calls :class:`SearchArtists` which returns the artists that fuzzily match
the search term. Then adds them to the `SearchResults` store.
"""
artists = searchlib.SearchArtists(self.query)()
SearchResults.artists = artists
return artists
return searchlib.SearchArtists(self.query)()
def search_albums(self):
def search_albums(self, in_quotes=False):
"""Calls :class:`SearchAlbums` which returns the albums that fuzzily match
the search term. Then adds them to the `SearchResults` store.
"""
albums = searchlib.SearchAlbums(self.query)()
SearchResults.albums = albums
return searchlib.TopResults().search(
self.query, albums_only=True, in_quotes=in_quotes
)
return albums
# def search_playlists(self):
# """Calls :class:`SearchPlaylists` which returns the playlists that fuzzily match
# the search term. Then adds them to the `SearchResults` store.
# """
# playlists = utils.Get.get_all_playlists()
# playlists = [serializer.Playlist(playlist) for playlist in playlists]
# playlists = searchlib.SearchPlaylists(playlists, self.query)()
# SearchResults.playlists = playlists
# return playlists
def get_top_results(self):
finder = searchlib.SearchAll()
return finder.search(self.query)
def search_all(self):
"""Calls all the search methods."""
self.search_tracks()
self.search_albums()
self.search_artists()
# self.search_playlists()
def get_top_results(
self,
limit: int,
in_quotes=False,
):
finder = searchlib.TopResults()
return finder.search(self.query, in_quotes=in_quotes, limit=limit)
@api.route("/search/tracks", methods=["GET"])
@@ -94,10 +70,12 @@ def search_tracks():
"""
query = request.args.get("q")
in_quotes = query_in_quotes(query)
if not query:
return {"error": "No query provided"}, 400
tracks = Search(query).search_tracks()
tracks = Search(query).search_tracks(in_quotes)
return {
"tracks": tracks[:SEARCH_COUNT],
@@ -112,14 +90,16 @@ def search_albums():
"""
query = request.args.get("q")
in_quotes = query_in_quotes(query)
if not query:
return {"error": "No query provided"}, 400
tracks = Search(query).search_albums()
albums = Search(query).search_albums(in_quotes)
return {
"albums": tracks[:SEARCH_COUNT],
"more": len(tracks) > SEARCH_COUNT,
"albums": albums[:SEARCH_COUNT],
"more": len(albums) > SEARCH_COUNT,
}
@@ -130,6 +110,7 @@ def search_artists():
"""
query = request.args.get("q")
if not query:
return {"error": "No query provided"}, 400
@@ -141,24 +122,6 @@ def search_artists():
}
# @searchbp.route("/search/playlists", methods=["GET"])
# def search_playlists():
# """
# Searches for playlists.
# """
# query = request.args.get("q")
# if not query:
# return {"error": "No query provided"}, 400
# playlists = DoSearch(query).search_playlists()
# return {
# "playlists": playlists[:SEARCH_COUNT],
# "more": len(playlists) > SEARCH_COUNT,
# }
@api.route("/search/top", methods=["GET"])
def get_top_results():
"""
@@ -166,21 +129,15 @@ def get_top_results():
"""
query = request.args.get("q")
limit = request.args.get("limit", "6")
limit = int(limit)
in_quotes = query_in_quotes(query)
if not query:
return {"error": "No query provided"}, 400
results = Search(query).get_top_results()
# max_results = 2
# return {
# "tracks": SearchResults.tracks[:max_results],
# "albums": SearchResults.albums[:max_results],
# "artists": SearchResults.artists[:max_results],
# "playlists": SearchResults.playlists[:max_results],
# }
return {
"results": results
}
return Search(query).get_top_results(in_quotes=in_quotes, limit=limit)
@api.route("/search/loadmore")
@@ -188,26 +145,32 @@ def search_load_more():
"""
Returns more songs, albums or artists from a search query.
"""
query = request.args.get("q")
in_quotes = query_in_quotes(query)
s_type = request.args.get("type")
index = int(request.args.get("index") or 0)
if s_type == "tracks":
t = SearchResults.tracks
t = Search(query).search_tracks(in_quotes)
return {
"tracks": t[index: index + SEARCH_COUNT],
"tracks": t[index : index + SEARCH_COUNT],
"more": len(t) > index + SEARCH_COUNT,
}
elif s_type == "albums":
a = SearchResults.albums
a = Search(query).search_albums(in_quotes)
return {
"albums": a[index: index + SEARCH_COUNT],
"albums": a[index : index + SEARCH_COUNT],
"more": len(a) > index + SEARCH_COUNT,
}
elif s_type == "artists":
a = SearchResults.artists
a = Search(query).search_artists()
return {
"artists": a[index: index + SEARCH_COUNT],
"artists": a[index : index + SEARCH_COUNT],
"more": len(a) > index + SEARCH_COUNT,
}
# TODO: Rewrite this file using generators where possible
app/api/track.py → app/api/send_file.py
+12 -4
View File
@@ -24,9 +24,17 @@ def send_track_file(trackhash: str):
filepath = request.args.get("filepath")
if filepath is not None and os.path.exists(filepath):
audio_type = get_mime(filepath)
return send_file(filepath, mimetype=audio_type)
if filepath is not None:
try:
track = TrackStore.get_tracks_by_filepaths([filepath])[0]
except IndexError:
track = None
track_exists = track is not None and os.path.exists(track.filepath)
if track_exists:
audio_type = get_mime(filepath)
return send_file(filepath, mimetype=audio_type)
if trackhash is None:
return msg, 404
@@ -41,7 +49,7 @@ def send_track_file(trackhash: str):
try:
return send_file(track.filepath, mimetype=audio_type)
except FileNotFoundError:
except (FileNotFoundError, OSError) as e:
return msg, 404
return msg, 404
app/api/settings.py
+128 -16
View File
@@ -1,17 +1,16 @@
from flask import Blueprint, request
from app import settings
from app.logger import log
from app.db.sqlite.settings import SettingsSQLMethods as sdb
from app.lib import populate
from app.lib.watchdogg import Watcher as WatchDog
from app.db.sqlite.settings import SettingsSQLMethods as sdb
from app.logger import log
from app.settings import Paths, SessionVarKeys, set_flag
from app.store.albums import AlbumStore
from app.store.artists import ArtistStore
from app.store.tracks import TrackStore
from app.utils.generators import get_random_str
from app.utils.threading import background
from app.store.albums import AlbumStore
from app.store.tracks import TrackStore
from app.store.artists import ArtistStore
api = Blueprint("settings", __name__, url_prefix="/")
@@ -21,13 +20,24 @@ def get_child_dirs(parent: str, children: list[str]):
return [_dir for _dir in children if _dir.startswith(parent) and _dir != parent]
def reload_everything():
def reload_everything(instance_key: str):
"""
Reloads all stores using the current database items
"""
TrackStore.load_all_tracks()
AlbumStore.load_albums()
ArtistStore.load_artists()
try:
TrackStore.load_all_tracks(instance_key)
except Exception as e:
log.error(e)
try:
AlbumStore.load_albums(instance_key=instance_key)
except Exception as e:
log.error(e)
try:
ArtistStore.load_artists(instance_key)
except Exception as e:
log.error(e)
@background
@@ -35,15 +45,16 @@ def rebuild_store(db_dirs: list[str]):
"""
Restarts the watchdog and rebuilds the music library.
"""
instance_key = get_random_str()
log.info("Rebuilding library...")
TrackStore.remove_tracks_by_dir_except(db_dirs)
reload_everything()
reload_everything(instance_key)
key = get_random_str()
try:
populate.Populate(key=key)
populate.Populate(instance_key=instance_key)
except populate.PopulateCancelledError:
reload_everything()
reload_everything(instance_key)
return
WatchDog().restart()
@@ -51,6 +62,7 @@ def rebuild_store(db_dirs: list[str]):
log.info("Rebuilding library... ✅")
# I freaking don't know what this function does anymore
def finalize(new_: list[str], removed_: list[str], db_dirs_: list[str]):
"""
Params:
@@ -96,7 +108,7 @@ def add_root_dirs():
sdb.remove_root_dirs(db_dirs)
if incoming_home:
finalize([_h], [], [settings.Paths.USER_HOME_DIR])
finalize([_h], [], [Paths.USER_HOME_DIR])
return {"root_dirs": [_h]}
# ---
@@ -127,3 +139,103 @@ def get_root_dirs():
dirs = sdb.get_root_dirs()
return {"dirs": dirs}
# maps settings to their parser flags
mapp = {
"artist_separators": SessionVarKeys.ARTIST_SEPARATORS,
"extract_feat": SessionVarKeys.EXTRACT_FEAT,
"remove_prod": SessionVarKeys.REMOVE_PROD,
"clean_album_title": SessionVarKeys.CLEAN_ALBUM_TITLE,
"remove_remaster": SessionVarKeys.REMOVE_REMASTER_FROM_TRACK,
"merge_albums": SessionVarKeys.MERGE_ALBUM_VERSIONS,
"show_albums_as_singles": SessionVarKeys.SHOW_ALBUMS_AS_SINGLES,
}
@api.route("/settings/", methods=["GET"])
def get_all_settings():
"""
Get all settings from the database.
"""
settings = sdb.get_all_settings()
key_list = list(mapp.keys())
s = {}
for key in key_list:
val_index = key_list.index(key)
try:
s[key] = settings[val_index]
if type(s[key]) == int:
s[key] = bool(s[key])
if type(s[key]) == str:
s[key] = str(s[key]).split(",")
except IndexError:
s[key] = None
root_dirs = sdb.get_root_dirs()
s["root_dirs"] = root_dirs
return {
"settings": s,
}
@background
def reload_all_for_set_setting():
reload_everything(get_random_str())
@api.route("/settings/set", methods=["POST"])
def set_setting():
key = request.get_json().get("key")
value = request.get_json().get("value")
if key is None or value is None or key == "root_dirs":
return {"msg": "Invalid arguments!"}, 400
root_dir = sdb.get_root_dirs()
if not root_dir:
return {"msg": "No root directories set!"}, 400
if key not in mapp:
return {"msg": "Invalid key!"}, 400
sdb.set_setting(key, value)
flag = mapp[key]
if key == "artist_separators":
value = str(value).split(",")
value = set(value)
set_flag(flag, value)
reload_all_for_set_setting()
# if value is a set, convert it to a string
# (artist_separators)
if type(value) == set:
value = ",".join(value)
return {"result": value}
@background
def run_populate():
populate.Populate(instance_key=get_random_str())
@api.route("/settings/trigger-scan", methods=["GET"])
def trigger_scan():
"""
Triggers a scan.
"""
run_populate()
return {"msg": "Scan triggered!"}
app/arg_handler.py
+68 -27
View File
@@ -4,19 +4,13 @@ Handles arguments passed to the program.
import os.path
import sys
from configparser import ConfigParser
import PyInstaller.__main__ as bundler
from app import settings
from app.print_help import HELP_MESSAGE
from app.utils.wintools import is_windows
from app.logger import log
from app.print_help import HELP_MESSAGE
from app.utils.xdg_utils import get_xdg_config_dir
# from app.api.imgserver import set_app_dir
config = ConfigParser()
config.read("pyinstaller.config.ini")
from app.utils.wintools import is_windows
ALLARGS = settings.ALLARGS
ARGS = sys.argv[1:]
@@ -28,8 +22,10 @@ class HandleArgs:
self.handle_host()
self.handle_port()
self.handle_config_path()
self.handle_no_feat()
self.handle_remove_prod()
self.handle_periodic_scan()
self.handle_periodic_scan_interval()
self.handle_help()
self.handle_version()
@@ -38,10 +34,33 @@ class HandleArgs:
"""
Runs Pyinstaller.
"""
if ALLARGS.build in ARGS:
with open("pyinstaller.config.ini", "w", encoding="utf-8") as file:
config["DEFAULT"]["BUILD"] = "True"
config.write(file)
if ALLARGS.build not in ARGS:
return
if settings.IS_BUILD:
print("Do the cha cha slide instead!")
print("https://www.youtube.com/watch?v=wZv62ShoStY")
sys.exit(0)
lastfm_key = settings.Keys.LASTFM_API
posthog_key = settings.Keys.POSTHOG_API_KEY
if not lastfm_key:
log.error("ERROR: LASTFM_API_KEY not set in environment")
sys.exit(0)
if not posthog_key:
log.error("ERROR: POSTHOG_API_KEY not set in environment")
sys.exit(0)
try:
with open("./app/configs.py", "w", encoding="utf-8") as file:
# copy the api keys to the config file
line1 = f'LASTFM_API_KEY = "{lastfm_key}"\n'
line2 = f'POSTHOG_API_KEY = "{posthog_key}"\n'
file.write(line1)
file.write(line2)
_s = ";" if is_windows() else ":"
@@ -54,14 +73,17 @@ class HandleArgs:
"--clean",
f"--add-data=assets{_s}assets",
f"--add-data=client{_s}client",
f"--add-data=pyinstaller.config.ini{_s}.",
f"--icon=assets/logo-fill.ico",
"-y",
]
)
with open("pyinstaller.config.ini", "w", encoding="utf-8") as file:
config["DEFAULT"]["BUILD"] = "False"
config.write(file)
finally:
# revert and remove the api keys for dev mode
with open("./app/configs.py", "w", encoding="utf-8") as file:
line1 = "LASTFM_API_KEY = ''\n"
line2 = "POSTHOG_API_KEY = ''\n"
file.write(line1)
file.write(line2)
sys.exit(0)
@@ -92,7 +114,7 @@ class HandleArgs:
print("ERROR: Host not specified")
sys.exit(0)
settings.FLASKVARS.FLASK_HOST = host # type: ignore
settings.FLASKVARS.set_flask_host(host) # type: ignore
@staticmethod
def handle_config_path():
@@ -117,15 +139,34 @@ class HandleArgs:
settings.Paths.set_config_dir(get_xdg_config_dir())
@staticmethod
def handle_no_feat():
# if ArgsEnum.no_feat in ARGS:
if any((a in ARGS for a in ALLARGS.show_feat)):
settings.FromFlags.EXTRACT_FEAT = False
def handle_periodic_scan():
if any((a in ARGS for a in ALLARGS.no_periodic_scan)):
settings.SessionVars.DO_PERIODIC_SCANS = False
@staticmethod
def handle_remove_prod():
if any((a in ARGS for a in ALLARGS.show_prod)):
settings.FromFlags.REMOVE_PROD = False
def handle_periodic_scan_interval():
if any((a in ARGS for a in ALLARGS.periodic_scan_interval)):
index = [
ARGS.index(a) for a in ALLARGS.periodic_scan_interval if a in ARGS
][0]
try:
interval = ARGS[index + 1]
except IndexError:
print("ERROR: Interval not specified")
sys.exit(0)
try:
psi = int(interval)
except ValueError:
print("ERROR: Interval should be a number")
sys.exit(0)
if psi < 0:
print("WADAFUCK ARE YOU TRYING?")
sys.exit(0)
settings.SessionVars.PERIODIC_SCAN_INTERVAL = psi
@staticmethod
def handle_help():
app/config.py
-61
View File
@@ -1,61 +0,0 @@
"""
Module for managing the JSON config file.
"""
import json
from enum import Enum
from typing import Type
from app.settings import Db
class ConfigKeys(Enum):
ROOT_DIRS = ("root_dirs", list[str])
PLAYLIST_DIRS = ("playlist_dirs", list[str])
USE_ART_COLORS = ("use_art_colors", bool)
DEFAULT_ART_COLOR = ("default_art_color", str)
SHUFFLE_MODE = ("shuffle_mode", str)
REPEAT_MODE = ("repeat_mode", str)
AUTOPLAY_ON_START = ("autoplay_on_start", bool)
VOLUME = ("volume", int)
def __init__(self, key_name: str, data_type: Type):
self.key_name = key_name
self.data_type = data_type
def get_data_type(self) -> Type:
return self.data_type
class ConfigManager:
def __init__(self, config_file_path: str):
self.config_file_path = config_file_path
def read_config(self):
try:
with open(self.config_file_path) as f:
return json.load(f)
except FileNotFoundError:
return {}
# in case of errors, return an empty dict
def write_config(self, config_data):
with open(self.config_file_path, "w") as f:
json.dump(config_data, f, indent=4)
def get_value(self, key: ConfigKeys):
config_data = self.read_config()
value = config_data.get(key.key_name)
if value is not None:
return key.get_data_type()(value)
def set_value(self, key: ConfigKeys, value):
config_data = self.read_config()
config_data[key.key_name] = value
self.write_config(config_data)
settings = ConfigManager(Db.get_json_config_path())
a = settings.get_value(ConfigKeys.ROOT_DIRS)
app/configs.py
+2
View File
@@ -0,0 +1,2 @@
LASTFM_API_KEY = ''
POSTHOG_API_KEY = ''
app/db/sqlite/__init__.py
-2
View File
@@ -6,8 +6,6 @@ import sqlite3
from pathlib import Path
from sqlite3 import Connection as SqlConn
from app.settings import Db
def create_connection(db_file: str) -> SqlConn:
"""
app/db/sqlite/albums.py → app/db/sqlite/albumcolors.py
+25 -1
View File
@@ -17,13 +17,16 @@ class SQLiteAlbumMethods:
"""
cur.execute(sql, (albumhash, colors))
return cur.lastrowid
lastrowid = cur.lastrowid
return lastrowid
@classmethod
def get_all_albums(cls):
with SQLiteManager() as cur:
cur.execute("SELECT * FROM albums")
albums = cur.fetchall()
cur.close()
if albums is not None:
return albums
@@ -35,8 +38,29 @@ class SQLiteAlbumMethods:
with SQLiteManager() as cur:
cur.execute("SELECT * FROM albums WHERE albumartist=?", (albumartist,))
albums = cur.fetchall()
cur.close()
if albums is not None:
return tuples_to_albums(albums)
return []
@staticmethod
def exists(albumhash: str, cur: Cursor = None):
"""
Checks if an album exists in the database.
"""
sql = "SELECT COUNT(1) FROM albums WHERE albumhash = ?"
def _exists(cur: Cursor):
cur.execute(sql, (albumhash,))
count = cur.fetchone()[0]
return count != 0
if cur:
return _exists(cur)
with SQLiteManager() as cur:
return _exists(cur)
app/db/sqlite/artists.py → app/db/sqlite/artistcolors.py
+33 -5
View File
@@ -10,7 +10,7 @@ from .utils import SQLiteManager
class SQLiteArtistMethods:
@staticmethod
def insert_one_artist(cur: Cursor, artisthash: str, colors: str | list[str]):
def insert_one_artist(cur: Cursor, artisthash: str, colors: list[str]):
"""
Inserts a single artist into the database.
"""
@@ -23,14 +23,42 @@ class SQLiteArtistMethods:
cur.execute(sql, (artisthash, colors))
@staticmethod
def get_all_artists():
def get_all_artists(cur_: Cursor = None):
"""
Get all artists from the database and return a generator of Artist objects
"""
sql = """SELECT * FROM artists"""
with SQLiteManager() as cur:
cur.execute(sql)
if not cur_:
with SQLiteManager() as cur:
cur.execute(sql)
for artist in cur.fetchall():
for artist in cur.fetchall():
yield artist
cur.close()
else:
cur_.execute(sql)
for artist in cur_.fetchall():
yield artist
@staticmethod
def exists(artisthash: str, cur: Cursor = None):
"""
Checks if an artist exists in the database.
"""
sql = "SELECT COUNT(1) FROM artists WHERE artisthash = ?"
def _exists(cur: Cursor):
cur.execute(sql, (artisthash,))
count = cur.fetchone()[0]
return count != 0
if cur:
return _exists(cur)
with SQLiteManager() as cur:
return _exists(cur)
app/db/sqlite/favorite.py
+9 -2
View File
@@ -14,6 +14,7 @@ class SQLiteFavoriteMethods:
with SQLiteManager(userdata_db=True) as cur:
cur.execute(sql, (itemhash, fav_type))
items = cur.fetchall()
cur.close()
return len(items) > 0
@classmethod
@@ -28,6 +29,7 @@ class SQLiteFavoriteMethods:
sql = """INSERT INTO favorites(type, hash) VALUES(?,?)"""
with SQLiteManager(userdata_db=True) as cur:
cur.execute(sql, (fav_type, fav_hash))
cur.close()
@classmethod
def get_all(cls) -> list[tuple]:
@@ -37,7 +39,9 @@ class SQLiteFavoriteMethods:
sql = """SELECT * FROM favorites"""
with SQLiteManager(userdata_db=True) as cur:
cur.execute(sql)
return cur.fetchall()
favs = cur.fetchall()
cur.close()
return [fav for fav in favs if fav[1] != ""]
@classmethod
def get_favorites(cls, fav_type: str) -> list[tuple]:
@@ -47,7 +51,9 @@ class SQLiteFavoriteMethods:
sql = """SELECT * FROM favorites WHERE type = ?"""
with SQLiteManager(userdata_db=True) as cur:
cur.execute(sql, (fav_type,))
return cur.fetchall()
all_favs = cur.fetchall()
cur.close()
return all_favs
@classmethod
def get_fav_tracks(cls) -> list[tuple]:
@@ -79,3 +85,4 @@ class SQLiteFavoriteMethods:
with SQLiteManager(userdata_db=True) as cur:
cur.execute(sql, (fav_hash, fav_type))
cur.close()
app/db/sqlite/lastfm/__init__.py
View File
app/db/sqlite/lastfm/similar_artists.py
+62
View File
@@ -0,0 +1,62 @@
from app.models.lastfm import SimilarArtist
from ..utils import SQLiteManager
class SQLiteLastFMSimilarArtists:
"""
This class contains methods for interacting with the lastfm_similar_artists table.
"""
@classmethod
def insert_one(cls, artist: SimilarArtist):
"""
Inserts a single artist into the database.
"""
sql = """INSERT OR REPLACE INTO lastfm_similar_artists(artisthash, similar_artists) VALUES(?,?)"""
with SQLiteManager(userdata_db=True) as cur:
cur.execute(sql, (artist.artisthash, artist.similar_artist_hashes))
cur.close()
@classmethod
def get_similar_artists_for(cls, artisthash: str):
"""
Returns a list of similar artists.
"""
sql = """SELECT * FROM lastfm_similar_artists WHERE artisthash = ?"""
with SQLiteManager(userdata_db=True) as cur:
cur.execute(sql, (artisthash,))
similar_artists = cur.fetchone()
cur.close()
if similar_artists is None:
return None
return SimilarArtist(artisthash, similar_artists[2])
@classmethod
def get_all(cls):
"""
Returns a list of all similar artists.
"""
sql = """SELECT * FROM lastfm_similar_artists"""
with SQLiteManager(userdata_db=True) as cur:
cur.execute(sql)
similar_artists = cur.fetchall()
cur.close()
for a in similar_artists:
yield SimilarArtist(a[1], a[2])
@classmethod
def exists(cls, artisthash: str):
"""
Checks if an artist exists in the database by counting the number of rows
"""
sql = """SELECT COUNT(*) FROM lastfm_similar_artists WHERE artisthash = ?"""
with SQLiteManager(userdata_db=True) as cur:
cur.execute(sql, (artisthash,))
count = cur.fetchone()[0]
cur.close()
return count > 0
app/db/sqlite/migrations.py
+13 -49
View File
@@ -6,62 +6,26 @@ from app.db.sqlite.utils import SQLiteManager
class MigrationManager:
all_get_sql = "SELECT * FROM migrations"
_base = "UPDATE migrations SET"
_end = "= ? WHERE id = 1"
pre_init_set_sql = f"{_base} pre_init_version {_end}"
post_init_set_sql = f"{_base} post_init_version {_end}"
@classmethod
def get_preinit_version(cls) -> int:
@staticmethod
def get_version() -> int:
"""
Returns the latest userdata pre-init database version.
Returns the latest userdata database version.
"""
sql = "SELECT * FROM dbmigrations"
with SQLiteManager() as cur:
cur.execute(cls.all_get_sql)
return int(cur.fetchone()[1])
cur.execute(sql)
ver = int(cur.fetchone()[1])
cur.close()
@classmethod
def get_maindb_postinit_version(cls) -> int:
"""
Returns the latest maindb post-init database version.
"""
with SQLiteManager() as cur:
cur.execute(cls.all_get_sql)
return int(cur.fetchone()[2])
@classmethod
def get_userdatadb_postinit_version(cls) -> int:
"""
Returns the latest userdata post-init database version.
"""
with SQLiteManager(userdata_db=True) as cur:
cur.execute(cls.all_get_sql)
return cur.fetchone()[2]
return ver
# 👇 Setters 👇
@classmethod
def set_preinit_version(cls, version: int):
@staticmethod
def set_version(version: int):
"""
Sets the userdata pre-init database version.
"""
sql = "UPDATE dbmigrations SET version = ? WHERE id = 1"
with SQLiteManager() as cur:
cur.execute(cls.pre_init_set_sql, (version,))
@classmethod
def set_maindb_postinit_version(cls, version: int):
"""
Sets the maindb post-init database version.
"""
with SQLiteManager() as cur:
cur.execute(cls.post_init_set_sql, (version,))
@classmethod
def set_userdatadb_postinit_version(cls, version: int):
"""
Sets the userdata post-init database version.
"""
with SQLiteManager(userdata_db=True) as cur:
cur.execute(cls.post_init_set_sql, (version,))
cur.execute(sql, (version,))
cur.close()
app/db/sqlite/playlists.py
+73 -33
View File
@@ -13,17 +13,25 @@ class SQLitePlaylistMethods:
This class contains methods for interacting with the playlists table.
"""
@staticmethod
def update_last_updated(playlist_id: int):
"""Updates the last updated date of a playlist."""
sql = """UPDATE playlists SET last_updated = ? WHERE id = ?"""
with SQLiteManager(userdata_db=True) as cur:
cur.execute(sql, (create_new_date(), playlist_id))
@staticmethod
def insert_one_playlist(playlist: dict):
# banner_pos,
# has_gif,
sql = """INSERT INTO playlists(
artisthashes,
banner_pos,
has_gif,
image,
last_updated,
name,
settings,
trackhashes
) VALUES(:artisthashes, :banner_pos, :has_gif, :image, :last_updated, :name, :trackhashes)
) VALUES(:image, :last_updated, :name, :settings, :trackhashes)
"""
playlist = OrderedDict(sorted(playlist.items()))
@@ -31,6 +39,7 @@ class SQLitePlaylistMethods:
with SQLiteManager(userdata_db=True) as cur:
cur.execute(sql, playlist)
pid = cur.lastrowid
cur.close()
p_tuple = (pid, *playlist.values())
return tuple_to_playlist(p_tuple)
@@ -43,6 +52,7 @@ class SQLitePlaylistMethods:
cur.execute(sql, (name,))
data = cur.fetchone()
cur.close()
if data is not None:
return tuple_to_playlist(data)
@@ -57,6 +67,7 @@ class SQLitePlaylistMethods:
cur.execute(sql, (name,))
data = cur.fetchone()
cur.close()
return int(data[0])
@@ -65,6 +76,7 @@ class SQLitePlaylistMethods:
with SQLiteManager(userdata_db=True) as cur:
cur.execute("SELECT * FROM playlists")
playlists = cur.fetchall()
cur.close()
if playlists is not None:
return tuples_to_playlists(playlists)
@@ -79,6 +91,7 @@ class SQLitePlaylistMethods:
cur.execute(sql, (playlist_id,))
data = cur.fetchone()
cur.close()
if data is not None:
return tuple_to_playlist(data)
@@ -87,12 +100,11 @@ class SQLitePlaylistMethods:
# FIXME: Extract the "add_track_to_playlist" method to use it for both the artisthash and trackhash lists.
@staticmethod
def add_item_to_json_list(playlist_id: int, field: str, items: list[str]):
@classmethod
def add_item_to_json_list(cls, playlist_id: int, field: str, items: set[str]):
"""
Adds a string item to a json dumped list using a playlist id and field name.
Takes the playlist ID, a field name,
an item to add to the field, and an error to raise if the item is already in the field.
Takes the playlist ID, a field name, an item to add to the field.
"""
sql = f"SELECT {field} FROM playlists WHERE id = ?"
@@ -103,6 +115,7 @@ class SQLitePlaylistMethods:
if data is not None:
db_items: list[str] = json.loads(data[0])
# Remove duplicates, without changing the order.
for item in items:
if item in db_items:
items.remove(item)
@@ -113,36 +126,28 @@ class SQLitePlaylistMethods:
cur.execute(sql, (json.dumps(db_items), playlist_id))
return len(items)
cls.update_last_updated(playlist_id)
@classmethod
def add_tracks_to_playlist(cls, playlist_id: int, trackhashes: list[str]):
"""
Adds trackhashes to a playlist
"""
return cls.add_item_to_json_list(playlist_id, "trackhashes", trackhashes)
@classmethod
@background
def add_artist_to_playlist(cls, playlist_id: int, trackhash: str):
track = SQLiteTrackMethods.get_track_by_trackhash(trackhash)
if track is None:
return
artists: list[Artist] = track.artist # type: ignore
artisthashes = [a.artisthash for a in artists]
cls.add_item_to_json_list(playlist_id, "artisthashes", artisthashes)
@staticmethod
def update_playlist(playlist_id: int, playlist: dict):
def update_playlist(cls, playlist_id: int, playlist: dict):
sql = """UPDATE playlists SET
has_gif = ?,
image = ?,
last_updated = ?,
name = ?
name = ?,
settings = ?
WHERE id = ?
"""
del playlist["id"]
del playlist["trackhashes"]
del playlist["artisthashes"]
del playlist["banner_pos"]
playlist["settings"] = json.dumps(playlist["settings"])
playlist = OrderedDict(sorted(playlist.items()))
params = (*playlist.values(), playlist_id)
@@ -150,13 +155,16 @@ class SQLitePlaylistMethods:
with SQLiteManager(userdata_db=True) as cur:
cur.execute(sql, params)
@staticmethod
def update_last_updated(playlist_id: int):
"""Updates the last updated date of a playlist."""
sql = """UPDATE playlists SET last_updated = ? WHERE id = ?"""
cls.update_last_updated(playlist_id)
@classmethod
def update_settings(cls, playlist_id: int, settings: dict):
sql = """UPDATE playlists SET settings = ? WHERE id = ?"""
with SQLiteManager(userdata_db=True) as cur:
cur.execute(sql, (create_new_date(), playlist_id))
cur.execute(sql, (json.dumps(settings), playlist_id))
cls.update_last_updated(playlist_id)
@staticmethod
def delete_playlist(pid: str):
@@ -166,8 +174,40 @@ class SQLitePlaylistMethods:
cur.execute(sql, (pid,))
@staticmethod
def update_banner_pos(playlistid: int, pos: int):
sql = """UPDATE playlists SET banner_pos = ? WHERE id = ?"""
def remove_banner(playlistid: int):
sql = """UPDATE playlists SET image = NULL WHERE id = ?"""
with SQLiteManager(userdata_db=True) as cur:
cur.execute(sql, (pos, playlistid))
cur.execute(sql, (playlistid,))
@classmethod
def remove_tracks_from_playlist(cls, playlistid: int, tracks: list[dict[str, int]]):
"""
Removes tracks from a playlist by trackhash and position.
"""
sql = """UPDATE playlists SET trackhashes = ? WHERE id = ?"""
with SQLiteManager(userdata_db=True) as cur:
cur.execute("SELECT trackhashes FROM playlists WHERE id = ?", (playlistid,))
data = cur.fetchone()
if data is None:
return
trackhashes: list[str] = json.loads(data[0])
for track in tracks:
# {
# trackhash: str;
# index: int;
# }
index = trackhashes.index(track["trackhash"])
if index == track["index"]:
trackhashes.remove(track["trackhash"])
cur.execute(sql, (json.dumps(trackhashes), playlistid))
cls.update_last_updated(playlistid)
app/db/sqlite/queries.py
+30 -14
View File
@@ -3,15 +3,16 @@ This file contains the SQL queries to create the database tables.
"""
# banner_pos integer NOT NULL,
# has_gif integer,
CREATE_USERDATA_TABLES = """
CREATE TABLE IF NOT EXISTS playlists (
id integer PRIMARY KEY,
artisthashes text,
banner_pos integer NOT NULL,
has_gif integer,
image text,
last_updated text not null,
name text not null,
settings text,
trackhashes text
);
@@ -24,8 +25,22 @@ CREATE TABLE IF NOT EXISTS favorites (
CREATE TABLE IF NOT EXISTS settings (
id integer PRIMARY KEY,
root_dirs text NOT NULL,
exclude_dirs text
)
exclude_dirs text,
artist_separators text NOT NULL default '/,;',
extract_feat integer NOT NULL DEFAULT 1,
remove_prod integer NOT NULL DEFAULT 1,
clean_album_title integer NOT NULL DEFAULT 1,
remove_remaster integer NOT NULL DEFAULT 1,
merge_albums integer NOT NULL DEFAULT 0,
show_albums_as_singles integer NOT NULL DEFAULT 0
);
CREATE TABLE IF NOT EXISTS lastfm_similar_artists (
id integer PRIMARY KEY,
artisthash text NOT NULL,
similar_artists text NOT NULL,
UNIQUE (artisthash)
);
"""
CREATE_APPDB_TABLES = """
@@ -37,7 +52,7 @@ CREATE TABLE IF NOT EXISTS tracks (
artist text NOT NULL,
bitrate integer NOT NULL,
copyright text,
date text NOT NULL,
date integer NOT NULL,
disc integer NOT NULL,
duration integer NOT NULL,
filepath text NOT NULL,
@@ -46,6 +61,7 @@ CREATE TABLE IF NOT EXISTS tracks (
title text NOT NULL,
track integer NOT NULL,
trackhash text NOT NULL,
last_mod float NOT NULL,
UNIQUE (filepath)
);
@@ -56,8 +72,6 @@ CREATE TABLE IF NOT EXISTS albums (
UNIQUE (albumhash)
);
CREATE TABLE IF NOT EXISTS artists (
id integer PRIMARY KEY,
artisthash text NOT NULL,
@@ -73,14 +87,16 @@ CREATE TABLE IF NOT EXISTS folders (
);
"""
# changed from migrations to dbmigrations in v1.3.0
# to avoid conflicts with the previous migrations.
CREATE_MIGRATIONS_TABLE = """
CREATE TABLE IF NOT EXISTS migrations (
CREATE TABLE IF NOT EXISTS dbmigrations (
id integer PRIMARY KEY,
pre_init_version integer NOT NULL DEFAULT 0,
post_init_version integer NOT NULL DEFAULT 0
version integer NOT NULL DEFAULT 0
);
INSERT INTO migrations (pre_init_version, post_init_version)
SELECT 0, 0
WHERE NOT EXISTS (SELECT 1 FROM migrations);
INSERT INTO dbmigrations (version)
SELECT -1
WHERE NOT EXISTS (SELECT 1 FROM dbmigrations);
"""
app/db/sqlite/settings.py
+61
View File
@@ -1,4 +1,8 @@
from pprint import pprint
from typing import Any
from app.db.sqlite.utils import SQLiteManager
from app.settings import SessionVars
from app.utils.wintools import win_replace_slash
@@ -7,6 +11,26 @@ class SettingsSQLMethods:
Methods for interacting with the settings table.
"""
@staticmethod
def get_all_settings():
"""
Gets all settings from the database.
"""
sql = "SELECT * FROM settings WHERE id = 1"
with SQLiteManager(userdata_db=True) as cur:
cur.execute(sql)
settings = cur.fetchone()
cur.close()
# if root_dirs not set
if settings is None:
return []
# omit id, root_dirs, and exclude_dirs
return settings[3:]
@staticmethod
def get_root_dirs() -> list[str]:
"""
@@ -18,6 +42,7 @@ class SettingsSQLMethods:
with SQLiteManager(userdata_db=True) as cur:
cur.execute(sql)
dirs = cur.fetchall()
cur.close()
dirs = [_dir[0] for _dir in dirs]
return [win_replace_slash(d) for d in dirs]
@@ -87,3 +112,39 @@ class SettingsSQLMethods:
cur.execute(sql)
dirs = cur.fetchall()
return [_dir[0] for _dir in dirs]
@staticmethod
def get_settings() -> dict[str, Any]:
pass
@staticmethod
def set_setting(key: str, value: Any):
sql = f"UPDATE settings SET {key} = :value WHERE id = 1"
if type(value) == bool:
value = str(int(value))
with SQLiteManager(userdata_db=True) as cur:
cur.execute(sql, {"value": value})
def load_settings():
s = SettingsSQLMethods.get_all_settings()
try:
db_separators: str = s[0]
db_separators = db_separators.replace(" ", "")
separators = db_separators.split(",")
separators = set(separators)
except IndexError:
separators = {";", "/"}
SessionVars.ARTIST_SEPARATORS = separators
# boolean settings
SessionVars.EXTRACT_FEAT = bool(s[1])
SessionVars.REMOVE_PROD = bool(s[2])
SessionVars.CLEAN_ALBUM_TITLE = bool(s[3])
SessionVars.REMOVE_REMASTER_FROM_TRACK = bool(s[4])
SessionVars.MERGE_ALBUM_VERSIONS = bool(s[5])
SessionVars.SHOW_ALBUMS_AS_SINGLES = bool(s[6])
app/db/sqlite/tracks.py
+27 -6
View File
@@ -9,6 +9,7 @@ from sqlite3 import Cursor
from app.db.sqlite.utils import tuple_to_track, tuples_to_tracks
from .utils import SQLiteManager
from app.utils.unicode import handle_unicode
class SQLiteTrackMethods:
@@ -21,7 +22,7 @@ class SQLiteTrackMethods:
"""
Inserts a single track into the database.
"""
sql = """INSERT INTO tracks(
sql = """INSERT OR REPLACE INTO tracks(
album,
albumartist,
albumhash,
@@ -34,21 +35,37 @@ class SQLiteTrackMethods:
filepath,
folder,
genre,
last_mod,
title,
track,
trackhash
) VALUES(:album, :albumartist, :albumhash, :artist, :bitrate, :copyright,
:date, :disc, :duration, :filepath, :folder, :genre, :title, :track, :trackhash)
:date, :disc, :duration, :filepath, :folder, :genre, :last_mod, :title, :track, :trackhash)
"""
track = OrderedDict(sorted(track.items()))
cur.execute(sql, track)
track["artist"] = track["artists"]
track["albumartist"] = track["albumartists"]
del track["artists"]
del track["albumartists"]
try:
cur.execute(sql, track)
except UnicodeEncodeError:
# for each of the values in the track, call handle_unicode on it
for key, value in track.items():
track[key] = handle_unicode(value)
cur.execute(sql, track)
@classmethod
def insert_many_tracks(cls, tracks: list[dict]):
"""
Inserts a list of tracks into the database.
"""
with SQLiteManager() as cur:
for track in tracks:
cls.insert_one_track(track, cur)
@@ -83,12 +100,16 @@ class SQLiteTrackMethods:
return None
@staticmethod
def remove_track_by_filepath(filepath: str):
def remove_tracks_by_filepaths(filepaths: str | set[str]):
"""
Removes a track from the database using its filepath.
Removes a track or tracks from the database using their filepaths.
"""
if isinstance(filepaths, str):
filepaths = {filepaths}
with SQLiteManager() as cur:
cur.execute("DELETE FROM tracks WHERE filepath=?", (filepath,))
for filepath in filepaths:
cur.execute("DELETE FROM tracks WHERE filepath=?", (filepath,))
@staticmethod
def remove_tracks_by_folders(folders: set[str]):
app/db/sqlite/utils.py
+26 -17
View File
@@ -5,9 +5,10 @@ Helper functions for use with the SQLite database.
import sqlite3
from sqlite3 import Connection, Cursor
import time
from typing import Optional
from app.models import Album, Playlist, Track
from app.settings import Db
from app import settings
def tuple_to_track(track: tuple):
@@ -61,14 +62,20 @@ class SQLiteManager:
for you. It also commits and closes the connection when you're done.
"""
def __init__(self, conn: Connection | None = None, userdata_db=False) -> None:
def __init__(
self,
conn: Optional[Connection] = None,
userdata_db=False,
test_db_path: str = None,
) -> None:
"""
When a connection is passed in, don't close the connection, because it's
a connection to the search database [in memory db].
"""
self.conn: Connection | None = conn
self.conn = conn
self.CLOSE_CONN = True
self.userdata_db = userdata_db
self.test_db_path = test_db_path
if conn:
self.conn = conn
@@ -78,10 +85,13 @@ class SQLiteManager:
if self.conn is not None:
return self.conn.cursor()
db_path = Db.get_app_db_path()
if self.test_db_path:
db_path = self.test_db_path
else:
db_path = settings.Db.get_app_db_path()
if self.userdata_db:
db_path = Db.get_userdata_db_path()
db_path = settings.Db.get_userdata_db_path()
self.conn = sqlite3.connect(
db_path,
@@ -90,19 +100,18 @@ class SQLiteManager:
return self.conn.cursor()
def __exit__(self, exc_type, exc_value, exc_traceback):
if self.conn:
trial_count = 0
trial_count = 0
while trial_count < 10:
try:
self.conn.commit()
while trial_count < 10:
try:
self.conn.commit()
if self.CLOSE_CONN:
self.conn.close()
if self.CLOSE_CONN:
self.conn.close()
return
except sqlite3.OperationalError:
trial_count += 1
time.sleep(3)
return
except sqlite3.OperationalError:
trial_count += 1
time.sleep(3)
self.conn.close()
self.conn.close()
app/enums/album_versions.py
+62
View File
@@ -0,0 +1,62 @@
from enum import Enum
class AlbumVersionEnum(Enum):
"""
Enum for album versions.
"""
Explicit = ("explicit",)
ANNIVERSARY_EDITION = ("anniversary",)
DIAMOND_EDITION = ("diamond",)
Centennial_EDITION = ("centennial",)
GOLDEN_EDITION = ("gold",)
PLATINUM_EDITION = ("platinum",)
SILVER_EDITION = ("silver",)
ULTIMATE_EDITION = ("ultimate",)
EXPANDED = ("expanded",)
EXTENDED = ("extended",)
DELUXE = ("deluxe",)
SUPER_DELUXE = ("super deluxe",)
COMPLETE = ("complete",)
LEGACY_EDITION = ("legacy",)
SPECIAL_EDITION = ("special",)
COLLECTORS_EDITION = ("collector",)
ARCHIVE_EDITION = ("archive",)
Acoustic = ("acoustic",)
instrumental = ("instrumental",)
DOUBLE_DISC = ("double disc", "double disk")
SUMMER_EDITION = ("summer",)
WINTER_EDITION = ("winter",)
SPRING_EDITION = ("spring",)
FALL_EDITION = ("fall",)
BONUS_EDITION = ("bonus",)
BONUS_TRACK = ("bonus track",)
ORIGINAL = ("original",)
INTL_VERSION = ("international",)
UK_VERSION = ("uk version",)
US_VERSION = ("us version",)
PARENTAL_ADVISORY = ("PA version",)
Limited_EDITION = ("limited",)
MONO = ("mono",)
STEREO = ("stereo",)
HI_RES = ("Hi-Res",)
RE_MIX = ("re-mix",)
RE_RECORDED = ("re-recorded", "rerecorded")
REISSUE = ("reissue",)
REMASTERED = ("remaster",)
def get_all_keywords():
return "|".join("|".join(i.value) for i in AlbumVersionEnum)
app/functions.py
-40
View File
@@ -1,40 +0,0 @@
"""
This module contains functions for the server
"""
import time
from requests import ConnectionError as RequestConnectionError
from requests import ReadTimeout
from app.lib.artistlib import CheckArtistImages
from app.lib.populate import Populate, PopulateCancelledError
from app.lib.trackslib import validate_tracks
from app.logger import log
from app.utils.generators import get_random_str
from app.utils.network import Ping
from app.utils.threading import background
@background
def run_periodic_checks():
"""
Checks for new songs every N minutes.
"""
# ValidateAlbumThumbs()
# ValidatePlaylistThumbs()
validate_tracks()
while True:
try:
Populate(key=get_random_str())
except PopulateCancelledError:
pass
if Ping()():
try:
CheckArtistImages()
except (RequestConnectionError, ReadTimeout):
log.error(
"Internet connection lost. Downloading artist images stopped."
)
time.sleep(300)
app/lib/albumslib.py
+45 -1
View File
@@ -1,3 +1,47 @@
"""
Contains methods relating to albums.
"""
"""
from dataclasses import asdict
from typing import Any
from app.logger import log
from app.models.track import Track
from app.store.albums import AlbumStore
from app.store.tracks import TrackStore
def validate_albums():
"""
Removes albums that have no tracks.
Probably albums that were added from incompletely written files.
"""
album_hashes = {t.albumhash for t in TrackStore.tracks}
albums = AlbumStore.albums
for album in albums:
if album.albumhash not in album_hashes:
AlbumStore.remove_album(album)
def remove_duplicate_on_merge_versions(tracks: list[Track]) -> list[Track]:
"""
Removes duplicate tracks when merging versions of the same album.
"""
pass
def sort_by_track_no(tracks: list[Track]) -> list[dict[str, Any]]:
tracks = [asdict(t) for t in tracks]
for t in tracks:
track = str(t["track"]).zfill(3)
t["_pos"] = int(f"{t['disc']}{track}")
tracks = sorted(tracks, key=lambda t: t["_pos"])
return tracks
app/lib/artistlib.py
+62 -29
View File
@@ -1,19 +1,22 @@
from concurrent.futures import ThreadPoolExecutor
from pathlib import Path
from io import BytesIO
from PIL import Image, UnidentifiedImageError
import requests
import os
import urllib
from concurrent.futures import ThreadPoolExecutor
from io import BytesIO
from pathlib import Path
from tqdm import tqdm
from requests.exceptions import ConnectionError as ReqConnError, ReadTimeout
import requests
from PIL import Image, UnidentifiedImageError
from requests.exceptions import ConnectionError as RequestConnectionError
from requests.exceptions import ReadTimeout
from app import settings
from app.models import Artist, Track, Album
from app.utils.hashing import create_hash
from app.models import Album, Artist, Track
from app.store import artists as artist_store
from app.utils.hashing import create_hash
from app.utils.progressbar import tqdm
CHECK_ARTIST_IMAGES_KEY = ""
def get_artist_image_link(artist: str):
@@ -36,7 +39,7 @@ def get_artist_image_link(artist: str):
return res["picture_big"]
return None
except (ReqConnError, ReadTimeout, IndexError, KeyError):
except (RequestConnectionError, ReadTimeout, IndexError, KeyError):
return None
@@ -73,24 +76,49 @@ class DownloadImage:
class CheckArtistImages:
def __init__(self):
with ThreadPoolExecutor() as pool:
list(
def __init__(self, instance_key: str):
global CHECK_ARTIST_IMAGES_KEY
CHECK_ARTIST_IMAGES_KEY = instance_key
# read all files in the artist image folder
path = settings.Paths.get_artist_img_sm_path()
processed = "".join(os.listdir(path)).replace("webp", "")
# filter out artists that already have an image
artists = filter(
lambda a: a.artisthash not in processed, artist_store.ArtistStore.artists
)
artists = list(artists)
# process the rest
key_artist_map = ((instance_key, artist) for artist in artists)
with ThreadPoolExecutor(max_workers=4) as executor:
res = list(
tqdm(
pool.map(self.download_image, artist_store.ArtistStore.artists),
total=len(artist_store.ArtistStore.artists),
desc="Downloading artist images",
executor.map(self.download_image, key_artist_map),
total=len(artists),
desc="Downloading missing artist images",
)
)
list(res)
@staticmethod
def download_image(artist: Artist):
def download_image(_map: tuple[str, Artist]):
"""
Checks if an artist image exists and downloads it if not.
:param artist: The artist name
"""
img_path = Path(settings.Paths.get_artist_img_sm_path()) / f"{artist.artisthash}.webp"
instance_key, artist = _map
if CHECK_ARTIST_IMAGES_KEY != instance_key:
return
img_path = (
Path(settings.Paths.get_artist_img_sm_path()) / f"{artist.artisthash}.webp"
)
if img_path.exists():
return
@@ -138,7 +166,7 @@ def get_artists_from_tracks(tracks: list[Track]) -> set[str]:
"""
artists = set()
master_artist_list = [[x.name for x in t.artist] for t in tracks]
master_artist_list = [[x.name for x in t.artists] for t in tracks]
artists = artists.union(*master_artist_list)
return artists
@@ -155,17 +183,22 @@ def get_albumartists(albums: list[Album]) -> set[str]:
return artists
def get_all_artists(
tracks: list[Track], albums: list[Album]
) -> list[Artist]:
def get_all_artists(tracks: list[Track], albums: list[Album]) -> list[Artist]:
artists_from_tracks = get_artists_from_tracks(tracks=tracks)
artist_from_albums = get_albumartists(albums=albums)
artists = list(artists_from_tracks.union(artist_from_albums))
artists = sorted(artists)
artists.sort()
lower_artists = set(a.lower().strip() for a in artists)
indices = [[ar.lower().strip() for ar in artists].index(a) for a in lower_artists]
artists = [artists[i] for i in indices]
# Remove duplicates
artists_dup_free = set()
artist_hashes = set()
return [Artist(a) for a in artists]
for artist in artists:
artist_hash = create_hash(artist, decode=True)
if artist_hash not in artist_hashes:
artists_dup_free.add(artist)
artist_hashes.add(artist_hash)
return [Artist(a) for a in artists_dup_free]
app/lib/colorlib.py
+65 -30
View File
@@ -6,15 +6,20 @@ import json
from pathlib import Path
import colorgram
from tqdm import tqdm
from app import settings
from app.db.sqlite.albums import SQLiteAlbumMethods as db
from app.db.sqlite.artists import SQLiteArtistMethods as adb
from app.db.sqlite.albumcolors import SQLiteAlbumMethods as aldb
from app.db.sqlite.artistcolors import SQLiteArtistMethods as adb
from app.db.sqlite.utils import SQLiteManager
from app.store.artists import ArtistStore
from app.store.albums import AlbumStore
from app.logger import log
from app.lib.errors import PopulateCancelledError
from app.utils.progressbar import tqdm
PROCESS_ALBUM_COLORS_KEY = ""
PROCESS_ARTIST_COLORS_KEY = ""
def get_image_colors(image: str, count=1) -> list[str]:
@@ -34,7 +39,11 @@ def get_image_colors(image: str, count=1) -> list[str]:
def process_color(item_hash: str, is_album=True):
path = settings.Paths.get_sm_thumb_path() if is_album else settings.Paths.get_artist_img_sm_path()
path = (
settings.Paths.get_sm_thumb_path()
if is_album
else settings.Paths.get_artist_img_sm_path()
)
path = Path(path) / (item_hash + ".webp")
if not path.exists():
@@ -48,26 +57,42 @@ class ProcessAlbumColors:
Extracts the most dominant color from the album art and saves it to the database.
"""
def __init__(self) -> None:
albums = [a for a in AlbumStore.albums if len(a.colors) == 0]
def __init__(self, instance_key: str) -> None:
global PROCESS_ALBUM_COLORS_KEY
PROCESS_ALBUM_COLORS_KEY = instance_key
albums = [
a
for a in AlbumStore.albums
if a is not None and a.colors is not None and len(a.colors) == 0
]
with SQLiteManager() as cur:
for album in tqdm(albums, desc="Processing missing album colors"):
sql = "SELECT COUNT(1) FROM albums WHERE albumhash = ?"
cur.execute(sql, (album.albumhash,))
count = cur.fetchone()[0]
try:
for album in tqdm(albums, desc="Processing missing album colors"):
if PROCESS_ALBUM_COLORS_KEY != instance_key:
raise PopulateCancelledError(
"A newer 'ProcessAlbumColors' instance is running. Stopping this one."
)
if count != 0:
continue
# TODO: Stop hitting the database for every album.
# Instead, fetch all the data from the database and
# check from memory.
colors = process_color(album.albumhash)
exists = aldb.exists(album.albumhash, cur=cur)
if exists:
continue
if colors is None:
continue
colors = process_color(album.albumhash)
album.set_colors(colors)
color_str = json.dumps(colors)
db.insert_one_album(cur, album.albumhash, color_str)
if colors is None:
continue
album.set_colors(colors)
color_str = json.dumps(colors)
aldb.insert_one_album(cur, album.albumhash, color_str)
finally:
cur.close()
class ProcessArtistColors:
@@ -75,23 +100,33 @@ class ProcessArtistColors:
Extracts the most dominant color from the artist art and saves it to the database.
"""
def __init__(self) -> None:
def __init__(self, instance_key: str) -> None:
all_artists = [a for a in ArtistStore.artists if len(a.colors) == 0]
global PROCESS_ARTIST_COLORS_KEY
PROCESS_ARTIST_COLORS_KEY = instance_key
with SQLiteManager() as cur:
for artist in tqdm(all_artists, desc="Processing missing artist colors"):
sql = "SELECT COUNT(1) FROM artists WHERE artisthash = ?"
try:
for artist in tqdm(
all_artists, desc="Processing missing artist colors"
):
if PROCESS_ARTIST_COLORS_KEY != instance_key:
raise PopulateCancelledError(
"A newer 'ProcessArtistColors' instance is running. Stopping this one."
)
cur.execute(sql, (artist.artisthash,))
count = cur.fetchone()[0]
exists = adb.exists(artist.artisthash, cur=cur)
if count != 0:
continue
if exists:
continue
colors = process_color(artist.artisthash, is_album=False)
colors = process_color(artist.artisthash, is_album=False)
if colors is None:
continue
if colors is None:
continue
artist.set_colors(colors)
adb.insert_one_artist(cur, artist.artisthash, colors)
artist.set_colors(colors)
adb.insert_one_artist(cur, artist.artisthash, colors)
finally:
cur.close()
app/lib/errors.py
+7
View File
@@ -0,0 +1,7 @@
class PopulateCancelledError(Exception):
"""
Raised when the instance key of a looping function called
inside Populate is changed.
"""
pass
app/lib/folderslib.py
+3 -3
View File
@@ -19,7 +19,7 @@ def create_folder(path: str, count=0) -> Folder:
name=folder.name,
path=win_replace_slash(str(folder)),
is_sym=folder.is_symlink(),
count=count
count=count,
)
@@ -32,11 +32,11 @@ def get_folders(paths: list[str]):
for track in TrackStore.tracks:
for path in paths:
if track.filepath.startswith(path):
if track.folder.startswith(path):
count_dict[path] += 1
folders = [{"path": path, "count": count_dict[path]} for path in paths]
return [create_folder(f['path'], f['count']) for f in folders if f['count'] > 0]
return [create_folder(f["path"], f["count"]) for f in folders if f["count"] > 0]
class GetFilesAndDirs:
app/lib/playlistlib.py
+14 -6
View File
@@ -16,7 +16,9 @@ def create_thumbnail(image: Any, img_path: str) -> str:
Creates a 250 x 250 thumbnail from a playlist image
"""
thumb_path = "thumb_" + img_path
full_thumb_path = os.path.join(settings.Paths.get_app_dir(), "images", "playlists", thumb_path)
full_thumb_path = os.path.join(
settings.Paths.get_app_dir(), "images", "playlists", thumb_path
)
aspect_ratio = image.width / image.height
@@ -33,7 +35,9 @@ def create_gif_thumbnail(image: Any, img_path: str):
Creates a 250 x 250 thumbnail from a playlist image
"""
thumb_path = "thumb_" + img_path
full_thumb_path = os.path.join(settings.Paths.get_app_dir(), "images", "playlists", thumb_path)
full_thumb_path = os.path.join(
settings.Paths.get_app_dir(), "images", "playlists", thumb_path
)
frames = []
@@ -50,19 +54,22 @@ def create_gif_thumbnail(image: Any, img_path: str):
return thumb_path
def save_p_image(file, pid: str):
def save_p_image(
img: Image, pid: str, content_type: str = None, filename: str = None
) -> str:
"""
Saves a playlist banner image and returns the filepath.
"""
img = Image.open(file)
# img = Image.open(file)
random_str = "".join(random.choices(string.ascii_letters + string.digits, k=5))
filename = pid + str(random_str) + ".webp"
if not filename:
filename = pid + str(random_str) + ".webp"
full_img_path = os.path.join(settings.Paths.get_playlist_img_path(), filename)
if file.content_type == "image/gif":
if content_type == "image/gif":
frames = []
for frame in ImageSequence.Iterator(img):
@@ -78,6 +85,7 @@ def save_p_image(file, pid: str):
return filename
#
# class ValidatePlaylistThumbs:
# """
app/lib/populate.py
+235 -41
View File
@@ -1,31 +1,41 @@
import os
from collections import deque
from concurrent.futures import ThreadPoolExecutor
from tqdm import tqdm
from typing import Generator
from requests import ConnectionError as RequestConnectionError
from requests import ReadTimeout
from app import settings
from app.db.sqlite.tracks import SQLiteTrackMethods
from app.db.sqlite.settings import SettingsSQLMethods as sdb
from app.db.sqlite.favorite import SQLiteFavoriteMethods as favdb
from app.db.sqlite.lastfm.similar_artists import \
SQLiteLastFMSimilarArtists as lastfmdb
from app.db.sqlite.settings import SettingsSQLMethods as sdb
from app.db.sqlite.tracks import SQLiteTrackMethods
from app.lib.albumslib import validate_albums
from app.lib.artistlib import CheckArtistImages
from app.lib.colorlib import ProcessAlbumColors, ProcessArtistColors
from app.lib.errors import PopulateCancelledError
from app.lib.taglib import extract_thumb, get_tags
from app.lib.trackslib import validate_tracks
from app.logger import log
from app.models import Album, Artist, Track
from app.utils.filesystem import run_fast_scandir
from app.models.lastfm import SimilarArtist
from app.requests.artists import fetch_similar_artists
from app.store.albums import AlbumStore
from app.store.tracks import TrackStore
from app.store.artists import ArtistStore
from app.store.tracks import TrackStore
from app.utils.filesystem import run_fast_scandir
from app.utils.network import has_connection
from app.utils.progressbar import tqdm
get_all_tracks = SQLiteTrackMethods.get_all_tracks
insert_many_tracks = SQLiteTrackMethods.insert_many_tracks
remove_tracks_by_filepaths = SQLiteTrackMethods.remove_tracks_by_filepaths
POPULATE_KEY = ""
class PopulateCancelledError(Exception):
pass
class Populate:
"""
Populates the database with all songs in the music directory
@@ -34,20 +44,22 @@ class Populate:
also checks if the album art exists in the image path, if not tries to extract it.
"""
def __init__(self, key: str) -> None:
def __init__(self, instance_key: str) -> None:
global POPULATE_KEY
POPULATE_KEY = key
POPULATE_KEY = instance_key
validate_tracks()
validate_albums()
tracks = get_all_tracks()
tracks = list(tracks)
dirs_to_scan = sdb.get_root_dirs()
if len(dirs_to_scan) == 0:
log.warning(
(
"The root directory is not configured. "
+ "Open the app in your webbrowser to configure."
"The root directory is not configured. "
+ "Open the app in your webbrowser to configure."
)
)
return
@@ -58,32 +70,85 @@ class Populate:
except IndexError:
pass
files = []
files = set()
for _dir in dirs_to_scan:
files.extend(run_fast_scandir(_dir, full=True)[1])
files = files.union(run_fast_scandir(_dir, full=True)[1])
untagged = self.filter_untagged(tracks, files)
unmodified, modified_tracks = self.remove_modified(tracks)
untagged = files - unmodified
if len(untagged) == 0:
log.info("All clear, no unread files.")
if len(untagged) != 0:
self.tag_untagged(untagged, instance_key)
self.extract_thumb_with_overwrite(modified_tracks)
try:
ProcessTrackThumbnails(instance_key)
ProcessAlbumColors(instance_key)
ProcessArtistColors(instance_key)
except PopulateCancelledError as e:
log.warn(e)
return
self.tag_untagged(untagged, key)
tried_to_download_new_images = False
ProcessTrackThumbnails()
ProcessAlbumColors()
ProcessArtistColors()
if has_connection():
tried_to_download_new_images = True
try:
CheckArtistImages(instance_key)
except (RequestConnectionError, ReadTimeout) as e:
log.error(
"Internet connection lost. Downloading artist images stopped."
)
else:
log.warning(f"No internet connection. Downloading artist images stopped!")
# Re-process the new artist images.
if tried_to_download_new_images:
ProcessArtistColors(instance_key=instance_key)
if has_connection():
try:
FetchSimilarArtistsLastFM(instance_key)
except PopulateCancelledError as e:
log.warn(e)
return
ArtistStore.load_artists(instance_key)
@staticmethod
def filter_untagged(tracks: list[Track], files: list[str]):
tagged_files = [t.filepath for t in tracks]
return set(files) - set(tagged_files)
def remove_modified(tracks: Generator[Track, None, None]):
"""
Removes tracks from the database that have been modified
since they were added to the database.
"""
unmodified_paths = set()
modified_tracks: list[Track] = []
modified_paths = set()
for track in tracks:
try:
if track.last_mod == round(os.path.getmtime(track.filepath)):
unmodified_paths.add(track.filepath)
continue
except (FileNotFoundError, OSError) as e:
TrackStore.remove_track_obj(track)
remove_tracks_by_filepaths(track.filepath)
modified_paths.add(track.filepath)
modified_tracks.append(track)
TrackStore.remove_tracks_by_filepaths(modified_paths)
remove_tracks_by_filepaths(modified_paths)
return unmodified_paths, modified_tracks
@staticmethod
def tag_untagged(untagged: set[str], key: str):
log.info("Found %s new tracks", len(untagged))
tagged_tracks: list[dict] = []
tagged_tracks: deque[dict] = deque()
tagged_count = 0
fav_tracks = favdb.get_fav_tracks()
@@ -91,7 +156,8 @@ class Populate:
for file in tqdm(untagged, desc="Reading files"):
if POPULATE_KEY != key:
raise PopulateCancelledError("Populate key changed")
log.warning("'Populate.tag_untagged': Populate key changed")
return
tags = get_tags(file)
@@ -105,11 +171,11 @@ class Populate:
if not AlbumStore.album_exists(track.albumhash):
AlbumStore.add_album(AlbumStore.create_album(track))
for artist in track.artist:
for artist in track.artists:
if not ArtistStore.artist_exists(artist.artisthash):
ArtistStore.add_artist(Artist(artist.name))
for artist in track.albumartist:
for artist in track.albumartists:
if not ArtistStore.artist_exists(artist.artisthash):
ArtistStore.add_artist(Artist(artist.name))
@@ -118,27 +184,155 @@ class Populate:
log.warning("Could not read file: %s", file)
if len(tagged_tracks) > 0:
log.info("Adding %s tracks to database", len(tagged_tracks))
insert_many_tracks(tagged_tracks)
log.info("Added %s/%s tracks", tagged_count, len(untagged))
@staticmethod
def extract_thumb_with_overwrite(tracks: list[Track]):
"""
Extracts the thumbnail from a list of filepaths,
overwriting the existing thumbnail if it exists,
for modified files.
"""
for track in tracks:
try:
extract_thumb(track.filepath, track.image, overwrite=True)
except FileNotFoundError:
continue
def get_image(album: Album):
for track in TrackStore.tracks:
if track.albumhash == album.albumhash:
extract_thumb(track.filepath, track.image)
break
def get_image(_map: tuple[str, Album]):
"""
The function retrieves an image from an album by iterating through its tracks and extracting the thumbnail from the first track that has one.
:param album: An instance of the `Album` class representing the album to retrieve the image from.
:type album: Album
:return: None
"""
instance_key, album = _map
if POPULATE_KEY != instance_key:
raise PopulateCancelledError("'ProcessTrackThumbnails': Populate key changed")
matching_tracks = filter(
lambda t: t.albumhash == album.albumhash, TrackStore.tracks
)
try:
track = next(matching_tracks)
extracted = extract_thumb(track.filepath, track.image)
while not extracted:
try:
track = next(matching_tracks)
extracted = extract_thumb(track.filepath, track.image)
except StopIteration:
break
return
except StopIteration:
pass
CPU_COUNT = os.cpu_count() // 2
class ProcessTrackThumbnails:
def __init__(self) -> None:
with ThreadPoolExecutor(max_workers=4) as pool:
"""
Extracts the album art from all albums in album store.
"""
def __init__(self, instance_key: str) -> None:
"""
Filters out albums that already have thumbnails and
extracts the thumbnail for the other albums.
"""
path = settings.Paths.get_sm_thumb_path()
# read all the files in the thumbnail directory
processed = "".join(os.listdir(path)).replace("webp", "")
# filter out albums that already have thumbnails
albums = filter(
lambda album: album.albumhash not in processed, AlbumStore.albums
)
albums = list(albums)
# process the rest
key_album_map = ((instance_key, album) for album in albums)
with ThreadPoolExecutor(max_workers=CPU_COUNT) as executor:
results = list(
tqdm(
pool.map(get_image, AlbumStore.albums),
total=len(AlbumStore.albums),
executor.map(get_image, key_album_map),
total=len(albums),
desc="Extracting track images",
)
)
list(results)
def save_similar_artists(_map: tuple[str, Artist]):
"""
Downloads and saves similar artists to the database.
"""
instance_key, artist = _map
if POPULATE_KEY != instance_key:
raise PopulateCancelledError(
"'FetchSimilarArtistsLastFM': Populate key changed"
)
if lastfmdb.exists(artist.artisthash):
return
artist_hashes = fetch_similar_artists(artist.name)
artist_ = SimilarArtist(artist.artisthash, "~".join(artist_hashes))
if len(artist_.similar_artist_hashes) == 0:
return
lastfmdb.insert_one(artist_)
class FetchSimilarArtistsLastFM:
"""
Fetches similar artists from LastFM using a thread pool.
"""
def __init__(self, instance_key: str) -> None:
# read all artists from db
processed = lastfmdb.get_all()
processed = ".".join(a.artisthash for a in processed)
# filter out artists that already have similar artists
artists = filter(lambda a: a.artisthash not in processed, ArtistStore.artists)
artists = list(artists)
# process the rest
key_artist_map = ((instance_key, artist) for artist in artists)
with ThreadPoolExecutor(max_workers=CPU_COUNT) as executor:
try:
results = list(
tqdm(
executor.map(save_similar_artists, key_artist_map),
total=len(artists),
desc="Fetching similar artists",
)
)
list(results)
except PopulateCancelledError as e:
raise e
# any exception that can be raised by the pool
except Exception as e:
log.warn(e)
return
app/lib/searchlib.py
+177 -47
View File
@@ -1,21 +1,26 @@
"""
This library contains all the functions related to the search functionality.
"""
from typing import List, Generator, TypeVar, Any
import itertools
from typing import Any, Generator, List, TypeVar
from rapidfuzz import fuzz, process
from rapidfuzz import process, utils
from unidecode import unidecode
from app import models
from app.utils.remove_duplicates import remove_duplicates
from app.db.sqlite.favorite import SQLiteFavoriteMethods as favdb
from app.models.enums import FavType
from app.models.track import Track
from app.serializers.album import serialize_for_card as serialize_album
from app.serializers.album import serialize_for_card_many as serialize_albums
from app.serializers.artist import serialize_for_cards
from app.serializers.track import serialize_track, serialize_tracks
from app.store.albums import AlbumStore
from app.store.artists import ArtistStore
from app.store.tracks import TrackStore
from app.utils.remove_duplicates import remove_duplicates
ratio = fuzz.ratio
wratio = fuzz.WRatio
# ratio = fuzz.ratio
# wratio = fuzz.WRatio
class Cutoff:
@@ -56,6 +61,7 @@ class SearchTracks:
track_titles,
score_cutoff=Cutoff.tracks,
limit=Limit.tracks,
processor=utils.default_process,
)
tracks = [self.tracks[i[2]] for i in results]
@@ -67,7 +73,7 @@ class SearchArtists:
self.query = query
self.artists = ArtistStore.artists
def __call__(self) -> list:
def __call__(self):
"""
Gets all artists with a given name.
"""
@@ -78,6 +84,7 @@ class SearchArtists:
artists,
score_cutoff=Cutoff.artists,
limit=Limit.artists,
processor=utils.default_process,
)
return [self.artists[i[2]] for i in results]
@@ -100,17 +107,11 @@ class SearchAlbums:
albums,
score_cutoff=Cutoff.albums,
limit=Limit.albums,
processor=utils.default_process,
)
return [self.albums[i[2]] for i in results]
# get all artists that matched the query
# for get all albums from the artists
# get all albums that matched the query
# return [**artist_albums **albums]
# recheck next and previous artist on play next or add to playlist
class SearchPlaylists:
def __init__(self, playlists: List[models.Playlist], query: str) -> None:
@@ -124,12 +125,13 @@ class SearchPlaylists:
playlists,
score_cutoff=Cutoff.playlists,
limit=Limit.playlists,
processor=utils.default_process,
)
return [self.playlists[i[2]] for i in results]
_type = List[models.Track | models.Album | models.Artist]
_type = models.Track | models.Album | models.Artist
_S2 = TypeVar("_S2")
_ResultType = int | float
@@ -140,7 +142,6 @@ def get_titles(items: _type):
text = item.og_title
elif isinstance(item, models.Album):
text = item.title
# print(text)
elif isinstance(item, models.Artist):
text = item.name
else:
@@ -149,7 +150,7 @@ def get_titles(items: _type):
yield text
class SearchAll:
class TopResults:
"""
Joins all tracks, albums and artists
then fuzzy searches them as a single unit.
@@ -157,11 +158,11 @@ class SearchAll:
@staticmethod
def collect_all():
all_items: _type = []
all_items: list[_type] = []
all_items.extend(ArtistStore.artists)
all_items.extend(TrackStore.tracks)
all_items.extend(AlbumStore.albums)
all_items.extend(ArtistStore.artists)
return all_items, get_titles(all_items)
@@ -170,44 +171,173 @@ class SearchAll:
items = list(items)
results = process.extract(
query=query,
choices=items,
score_cutoff=Cutoff.tracks,
limit=20
query=query, choices=items, score_cutoff=Cutoff.tracks, limit=1
)
return results
@staticmethod
def sort_results(items: _type):
def map_with_type(item: _type):
"""
Separates results into differrent lists using itertools.groupby.
Map the results to their respective types.
"""
mapped_items = [
{"type": "track", "item": item} if isinstance(item, models.Track) else
{"type": "album", "item": item} if isinstance(item, models.Album) else
{"type": "artist", "item": item} if isinstance(item, models.Artist) else
{"type": "Unknown", "item": item} for item in items
]
if isinstance(item, models.Track):
return {"type": "track", "item": item}
mapped_items.sort(key=lambda x: x["type"])
if isinstance(item, models.Album):
tracks = TrackStore.get_tracks_by_albumhash(item.albumhash)
tracks = remove_duplicates(tracks)
groups = [
list(group) for key, group in
itertools.groupby(mapped_items, lambda x: x["type"])
]
item.get_date_from_tracks(tracks)
try:
item.duration = sum((t.duration for t in tracks))
except AttributeError:
item.duration = 0
# merge items of a group into a dict that looks like: {"albums": [album1, ...]}
groups = [
{f"{group[0]['type']}s": [i['item'] for i in group]} for group in groups
]
item.check_is_single(tracks)
return groups
if not item.is_single:
item.check_type()
item.is_favorite = favdb.check_is_favorite(
item.albumhash, fav_type=FavType.album
)
return {"type": "album", "item": item}
if isinstance(item, models.Artist):
track_count = 0
duration = 0
for track in TrackStore.get_tracks_by_artisthash(item.artisthash):
track_count += 1
duration += track.duration
album_count = AlbumStore.count_albums_by_artisthash(item.artisthash)
item.set_trackcount(track_count)
item.set_albumcount(album_count)
item.set_duration(duration)
return {"type": "artist", "item": item}
@staticmethod
def search(query: str):
items, titles = SearchAll.collect_all()
results = SearchAll.get_results(titles, query)
results = [items[i[2]] for i in results]
def get_track_items(item: dict[str, _type], query: str, limit=5):
tracks: list[Track] = []
return SearchAll.sort_results(results)
if item["type"] == "track":
tracks.extend(SearchTracks(query)())
if item["type"] == "album":
t = TrackStore.get_tracks_by_albumhash(item["item"].albumhash)
t.sort(key=lambda x: x.last_mod)
# if there are less than the limit, get more tracks
if len(t) < limit:
remainder = limit - len(t)
more_tracks = SearchTracks(query)()
t.extend(more_tracks[:remainder])
tracks.extend(t)
if item["type"] == "artist":
t = TrackStore.get_tracks_by_artisthash(item["item"].artisthash)
# if there are less than the limit, get more tracks
if len(t) < limit:
remainder = limit - len(t)
more_tracks = SearchTracks(query)()
t.extend(more_tracks[:remainder])
tracks.extend(t)
return tracks[:limit]
@staticmethod
def get_album_items(item: dict[str, _type], query: str, limit=6):
if item["type"] == "track":
return SearchAlbums(query)()[:limit]
if item["type"] == "album":
return SearchAlbums(query)()[:limit]
if item["type"] == "artist":
albums = AlbumStore.get_albums_by_artisthash(item["item"].artisthash)
# if there are less than the limit, get more albums
if len(albums) < limit:
remainder = limit - len(albums)
more_albums = SearchAlbums(query)()
albums.extend(more_albums[:remainder])
return albums[:limit]
@staticmethod
def search(
query: str,
limit: int = None,
albums_only=False,
tracks_only=False,
in_quotes=False,
):
items, titles = TopResults.collect_all()
results = TopResults.get_results(titles, query)
tracks_limit = Limit.tracks if tracks_only else 4
albums_limit = Limit.albums if albums_only else limit
artists_limit = limit
# map results to their respective items
try:
result = [items[i[2]] for i in results][0]
except IndexError:
if tracks_only:
return []
if albums_only:
return []
return {
"top_result": None,
"tracks": [],
"artists": [],
"albums": [],
}
result = TopResults.map_with_type(result)
if in_quotes:
top_tracks = SearchTracks(query)()[:tracks_limit]
else:
top_tracks = TopResults.get_track_items(result, query, limit=tracks_limit)
top_tracks = serialize_tracks(top_tracks)
if tracks_only:
return top_tracks
if in_quotes:
albums = SearchAlbums(query)()[:albums_limit]
else:
albums = TopResults.get_album_items(result, query, limit=albums_limit)
albums = serialize_albums(albums)
if albums_only:
return albums
artists = SearchArtists(query)()[:artists_limit]
artists = serialize_for_cards(artists)
if result["type"] == "track":
result["item"] = serialize_track(result["item"])
if result["type"] == "album":
result["item"] = serialize_album(result["item"])
return {
"top_result": result,
"tracks": top_tracks,
"artists": artists,
"albums": albums,
}
app/lib/taglib.py
+48 -15
View File
@@ -1,13 +1,13 @@
import datetime
import os
from io import BytesIO
import pendulum
from PIL import Image, UnidentifiedImageError
from tinytag import TinyTag
from app.settings import Defaults, Paths
from app.utils.hashing import create_hash
from app.utils.parsers import parse_title_from_filename, parse_artist_from_filename
from app.utils.parsers import parse_artist_from_filename, parse_title_from_filename
from app.utils.wintools import win_replace_slash
@@ -23,22 +23,32 @@ def parse_album_art(filepath: str):
return None
def extract_thumb(filepath: str, webp_path: str) -> bool:
def extract_thumb(filepath: str, webp_path: str, overwrite=False) -> bool:
"""
Extracts the thumbnail from an audio file. Returns the path to the thumbnail.
Extracts the thumbnail from an audio file.
Returns the path to the thumbnail.
"""
img_path = os.path.join(Paths.get_lg_thumb_path(), webp_path)
original_img_path = os.path.join(Paths.get_original_thumb_path(), webp_path)
lg_img_path = os.path.join(Paths.get_lg_thumb_path(), webp_path)
sm_img_path = os.path.join(Paths.get_sm_thumb_path(), webp_path)
tsize = Defaults.THUMB_SIZE
sm_tsize = Defaults.SM_THUMB_SIZE
def save_image(img: Image.Image):
img.resize((sm_tsize, sm_tsize), Image.ANTIALIAS).save(sm_img_path, "webp")
img.resize((tsize, tsize), Image.ANTIALIAS).save(img_path, "webp")
width, height = img.size
ratio = width / height
if os.path.exists(img_path):
img_size = os.path.getsize(img_path)
img.save(original_img_path, "webp")
img.resize((tsize, int(tsize / ratio)), Image.ANTIALIAS).save(
lg_img_path, "webp"
)
img.resize((sm_tsize, int(sm_tsize / ratio)), Image.ANTIALIAS).save(
sm_img_path, "webp"
)
if not overwrite and os.path.exists(sm_img_path):
img_size = os.path.getsize(sm_img_path)
if img_size > 0:
return True
@@ -64,18 +74,30 @@ def extract_thumb(filepath: str, webp_path: str) -> bool:
return False
def extract_date(date_str: str | None, filepath: str) -> int:
def parse_date(date_str: str | None) -> int | None:
"""
Extracts the date from a string and returns a timestamp.
"""
try:
return int(date_str.split("-")[0])
except: # pylint: disable=bare-except
# TODO: USE FILEPATH LAST-MOD DATE instead of current date
return datetime.date.today().today().year
date = pendulum.parse(date_str, strict=False)
return int(date.timestamp())
except Exception as e:
return None
def get_tags(filepath: str):
"""
Returns the tags for a given audio file.
"""
filetype = filepath.split(".")[-1]
filename = (filepath.split("/")[-1]).replace(f".{filetype}", "")
try:
last_mod = round(os.path.getmtime(filepath))
except FileNotFoundError:
return None
try:
tags = TinyTag.get(filepath)
except: # noqa: E722
@@ -141,9 +163,17 @@ def get_tags(filepath: str):
tags.image = f"{tags.albumhash}.webp"
tags.folder = win_replace_slash(os.path.dirname(filepath))
tags.date = extract_date(tags.year, filepath)
tags.date = parse_date(tags.year) or int(last_mod)
tags.filepath = win_replace_slash(filepath)
tags.filetype = filetype
tags.last_mod = last_mod
tags.artists = tags.artist
tags.albumartists = tags.albumartist
# sub underscore with space
tags.title = tags.title.replace("_", " ")
tags.album = tags.album.replace("_", " ")
tags = tags.__dict__
@@ -163,6 +193,9 @@ def get_tags(filepath: str):
"samplerate",
"track_total",
"year",
"bitdepth",
"artist",
"albumartist",
]
for tag in to_delete:
app/lib/trackslib.py
+5 -7
View File
@@ -3,18 +3,16 @@ This library contains all the functions related to tracks.
"""
import os
from tqdm import tqdm
from app.db.sqlite.tracks import SQLiteTrackMethods as tdb
from app.store.tracks import TrackStore
from app.utils.progressbar import tqdm
def validate_tracks() -> None:
"""
Gets all songs under the ~/ directory.
Removes track records whose files no longer exist.
"""
for track in tqdm(TrackStore.tracks, desc="Removing deleted tracks"):
for track in tqdm(TrackStore.tracks, desc="Validating tracks"):
if not os.path.exists(track.filepath):
print(f"Removing {track.filepath}")
TrackStore.tracks.remove(track)
tdb.remove_track_by_filepath(track.filepath)
TrackStore.remove_track_obj(track)
tdb.remove_tracks_by_filepaths(track.filepath)
app/lib/watchdogg.py
+89 -28
View File
@@ -1,6 +1,7 @@
"""
This library contains the classes and functions related to the watchdog file watcher.
"""
import json
import os
import sqlite3
import time
@@ -8,18 +9,18 @@ import time
from watchdog.events import PatternMatchingEventHandler
from watchdog.observers import Observer
from app.logger import log
from app.lib.taglib import get_tags
from app.models import Artist, Track
from app import settings
from app.db.sqlite.albumcolors import SQLiteAlbumMethods as aldb
from app.db.sqlite.settings import SettingsSQLMethods as sdb
from app.db.sqlite.tracks import SQLiteManager
from app.db.sqlite.tracks import SQLiteTrackMethods as db
from app.db.sqlite.settings import SettingsSQLMethods as sdb
from app.store.tracks import TrackStore
from app.lib.colorlib import process_color
from app.lib.taglib import extract_thumb, get_tags
from app.logger import log
from app.models import Artist, Track
from app.store.albums import AlbumStore
from app.store.artists import ArtistStore
from app.store.tracks import TrackStore
class Watcher:
@@ -94,7 +95,7 @@ class Watcher:
)
return
except OSError as e:
log.error('Failed to start watchdog. %s', e)
log.error("Failed to start watchdog. %s", e)
return
try:
@@ -125,34 +126,64 @@ class Watcher:
self.run()
def handle_colors(cur: sqlite3.Cursor, albumhash: str):
exists = aldb.exists(albumhash, cur)
if exists:
return
colors = process_color(albumhash, is_album=True)
if colors is None:
return
aldb.insert_one_album(cur=cur, albumhash=albumhash, colors=json.dumps(colors))
return colors
def add_track(filepath: str) -> None:
"""
Processes the audio tags for a given file ands add them to the database and store.
Then creates the folder, album and artist objects for the added track and adds them to the store.
"""
TrackStore.remove_track_by_filepath(filepath)
tags = get_tags(filepath)
if tags is None:
# if the track is somehow invalid, return
if tags is None or tags["bitrate"] == 0 or tags["duration"] == 0:
return
colors = None
with SQLiteManager() as cur:
db.remove_track_by_filepath(tags["filepath"])
db.insert_one_track(tags, cur)
extracted = extract_thumb(filepath, tags["albumhash"] + ".webp")
if not extracted:
return
colors = handle_colors(cur, tags["albumhash"])
track = Track(**tags)
TrackStore.add_track(track)
if not AlbumStore.album_exists(track.albumhash):
album = AlbumStore.create_album(track)
album.set_colors(colors)
AlbumStore.add_album(album)
artists: list[Artist] = track.artist + track.albumartist # type: ignore
artists: list[Artist] = track.artists + track.albumartists # type: ignore
for artist in artists:
if not ArtistStore.artist_exists(artist.artisthash):
ArtistStore.add_artist(Artist(artist.name))
extract_thumb(filepath, track.image, overwrite=True)
def remove_track(filepath: str) -> None:
"""
@@ -163,15 +194,15 @@ def remove_track(filepath: str) -> None:
except IndexError:
return
db.remove_track_by_filepath(filepath)
db.remove_tracks_by_filepaths(filepath)
TrackStore.remove_track_by_filepath(filepath)
empty_album = TrackStore.count_tracks_by_hash(track.albumhash) > 0
empty_album = TrackStore.count_tracks_by_trackhash(track.albumhash) > 0
if empty_album:
AlbumStore.remove_album_by_hash(track.albumhash)
artists: list[Artist] = track.artist + track.albumartist # type: ignore
artists: list[Artist] = track.artists + track.albumartists # type: ignore
for artist in artists:
empty_artist = not ArtistStore.artist_has_tracks(artist.artisthash)
@@ -183,6 +214,7 @@ def remove_track(filepath: str) -> None:
class Handler(PatternMatchingEventHandler):
files_to_process = []
files_to_process_windows = []
file_sizes = {}
root_dirs = []
dir_map = []
@@ -212,6 +244,11 @@ class Handler(PatternMatchingEventHandler):
"""
Fired when a supported file is created.
"""
try:
self.file_sizes[event.src_path] = os.path.getsize(event.src_path)
except FileNotFoundError:
return
self.files_to_process.append(event.src_path)
self.files_to_process_windows.append(event.src_path)
@@ -253,7 +290,11 @@ class Handler(PatternMatchingEventHandler):
if os.path.getsize(event.src_path) > 0:
path = self.get_abs_path(event.src_path)
add_track(path)
except FileNotFoundError:
# file was closed and deleted.
pass
except ValueError:
# file was removed from the list by another event handler.
pass
def on_modified(self, event):
@@ -264,18 +305,38 @@ class Handler(PatternMatchingEventHandler):
if event.src_path not in self.files_to_process_windows:
return
file_size = -1
while file_size != os.path.getsize(event.src_path):
file_size = os.path.getsize(event.src_path)
time.sleep(0.1)
# Check if file write operation is complete
try:
os.rename(event.src_path, event.src_path)
path = self.get_abs_path(event.src_path)
remove_track(path)
add_track(path)
self.files_to_process_windows.remove(event.src_path)
except OSError:
# File is locked, skipping
pass
current_size = os.path.getsize(event.src_path)
except FileNotFoundError:
# File was deleted or moved
return
previous_size = self.file_sizes.get(event.src_path, -1)
if current_size == previous_size:
# Wait for a short duration to ensure the file write operation is complete
time.sleep(5)
# Check the file size again
try:
current_size = os.path.getsize(event.src_path)
except FileNotFoundError:
# File was deleted or moved
return
if current_size == previous_size:
try:
os.rename(event.src_path, event.src_path)
path = self.get_abs_path(event.src_path)
remove_track(path)
add_track(path)
self.files_to_process_windows.remove(event.src_path)
del self.file_sizes[event.src_path]
except OSError:
# File is locked
pass
return
# Update the file size for the next iteration
self.file_sizes[event.src_path] = current_size
app/logger.py
+1
View File
@@ -2,6 +2,7 @@
Logger module
"""
from app.settings import IS_BUILD
import logging
app/migrations/__init__.py
+28 -21
View File
@@ -6,14 +6,28 @@ Reads and applies the latest database migrations.
PLEASE NOTE: OLDER MIGRATIONS CAN NEVER BE DELETED.
ONLY MODIFY OLD MIGRATIONS FOR BUG FIXES OR ENHANCEMENTS ONLY
[TRY NOT TO MODIFY BEHAVIOR, UNLESS YOU KNOW WHAT YOU'RE DOING].
PS: Fuck that! Do what you want.
"""
from app.db.sqlite.migrations import MigrationManager
from app.logger import log
from app.migrations import v1_3_0
from app.migrations.base import Migration
from .main import main_db_migrations
from .userdata import userdata_db_migrations
migrations: list[list[Migration]] = [
[
# v1.3.0
v1_3_0.RemoveSmallThumbnailFolder,
v1_3_0.RemovePlaylistArtistHashes,
v1_3_0.AddSettingsToPlaylistTable,
v1_3_0.AddLastUpdatedToTrackTable,
v1_3_0.MovePlaylistsAndFavoritesTo10BitHashes,
v1_3_0.RemoveAllTracks,
v1_3_0.UpdateAppSettingsTable,
]
]
def apply_migrations():
@@ -21,24 +35,17 @@ def apply_migrations():
Applies the latest database migrations.
"""
userdb_version = MigrationManager.get_userdatadb_postinit_version()
maindb_version = MigrationManager.get_maindb_postinit_version()
version = MigrationManager.get_version()
for migration in main_db_migrations:
if migration.version > maindb_version:
log.info("Running new MAIN-DB post-init migration: %s", migration.name)
migration.migrate()
if version != len(migrations):
# run migrations after the previous migration version
for migration in migrations[(version - 1) :]:
for m in migration:
try:
m.migrate()
log.info("Applied migration: %s", m.__name__)
except:
log.error("Failed to run migration: %s", m.__name__)
for migration in userdata_db_migrations:
if migration.version > userdb_version:
log.info("Running new USERDATA-DB post-init migration: %s", migration.name)
migration.migrate()
def set_postinit_migration_versions():
"""
Sets the post-init migration versions.
"""
# TODO: Don't forget to remove the zeros below when you add a valid migration 👇.
MigrationManager.set_maindb_postinit_version(0)
MigrationManager.set_userdatadb_postinit_version(0)
# bump migration version
MigrationManager.set_version(len(migrations))
app/migrations/__preinit/__init__.py
-41
View File
@@ -1,41 +0,0 @@
"""
Pre-init migrations are executed before the database is created.
Useful when you need to move files or folders before the database is created.
`Example use cases: Moving files around, dropping tables, etc.`
PLEASE NOTE: OLDER MIGRATIONS CAN NEVER BE DELETED.
ONLY MODIFY OLD MIGRATIONS FOR BUG FIXES OR ENHANCEMENTS ONLY.
[TRY NOT TO MODIFY BEHAVIOR, UNLESS YOU KNOW WHAT YOU'RE DOING].
"""
from sqlite3 import OperationalError
from app.db.sqlite.migrations import MigrationManager
from app.logger import log
from .drop_artist_and_album_color_tables import DropArtistAndAlbumColorTables
from .move_to_xdg_folder import MoveToXdgFolder
all_preinits = [MoveToXdgFolder, DropArtistAndAlbumColorTables]
def run_preinit_migrations():
"""
Runs all pre-init migrations.
"""
try:
userdb_version = MigrationManager.get_preinit_version()
except OperationalError:
userdb_version = 0
for migration in all_preinits:
if migration.version > userdb_version:
log.warn("Running new pre-init migration: %s", migration.name)
migration.migrate()
def set_preinit_migration_versions():
"""
Sets the migration versions.
"""
MigrationManager.set_preinit_version(all_preinits[-1].version)
app/migrations/__preinit/drop_artist_and_album_color_tables.py
-24
View File
@@ -1,24 +0,0 @@
"""
Another shot at attempting to fix duplicate album and artist color entries.
This release should finally fix the issue. The migration script will now remove
the album and artist color tables and recreate them.
"""
from app.db.sqlite.utils import SQLiteManager
from app.logger import log
class DropArtistAndAlbumColorTables:
version = 2
name = "DropArtistAndAlbumColorTables"
@staticmethod
def migrate():
with SQLiteManager() as cur:
tables = ["artists", "albums"]
for table in tables:
cur.execute(f"DROP TABLE IF EXISTS {table}")
cur.execute("VACUUM")
log.info("Deleted artist and album color data to fix a few bugs. ✅")
app/migrations/__preinit/move_to_xdg_folder.py
-49
View File
@@ -1,49 +0,0 @@
"""
This migration handles moving the config folder to the XDG standard location.
It also handles moving the userdata and the downloaded artist images to the new location.
"""
import os
import shutil
from app.settings import Paths
from app.logger import log
class MoveToXdgFolder:
version = 1
name = "MoveToXdgFolder"
@staticmethod
def migrate():
old_config_dir = os.path.join(Paths.USER_HOME_DIR, ".swing")
new_config_dir = Paths.get_app_dir()
if not os.path.exists(old_config_dir):
log.info("No old config folder found. Skipping migration.")
return
log.info("Found old config folder: %s", old_config_dir)
old_imgs_dir = os.path.join(old_config_dir, "images")
# move images to new location
if os.path.exists(old_imgs_dir):
shutil.copytree(
old_imgs_dir,
os.path.join(new_config_dir, "images"),
copy_function=shutil.copy2,
dirs_exist_ok=True,
)
log.warn("Moved artist images to: %s", new_config_dir)
# move userdata.db to new location
userdata_db = os.path.join(old_config_dir, "userdata.db")
if os.path.exists(userdata_db):
shutil.copy2(userdata_db, new_config_dir)
log.warn("Moved userdata.db to: %s", new_config_dir)
log.warn("Migration complete. ✅")
# swing.db is not moved because the new code fixes bugs which require
# the whole database to be recreated anyway. (ie. the bug which caused duplicate album and artist color entries)
app/migrations/base.py
+11
View File
@@ -0,0 +1,11 @@
class Migration:
"""
Base migration class.
"""
@staticmethod
def migrate():
"""
Code to run when migrating, override this method.
"""
pass
app/migrations/main/__init__.py
-10
View File
@@ -1,10 +0,0 @@
"""
Migrations for the main database.
PLEASE NOTE: OLDER MIGRATIONS CAN NEVER BE DELETED.
ONLY MODIFY OLD MIGRATIONS FOR BUG FIXES OR ENHANCEMENTS ONLY
[TRY NOT TO MODIFY BEHAVIOR, UNLESS YOU KNOW WHAT YOU'RE DOING].
"""
main_db_migrations = []
app/migrations/userdata/__init__.py
-10
View File
@@ -1,10 +0,0 @@
"""
Migrations for the userdata database.
PLEASE NOTE: OLDER MIGRATIONS CAN NEVER BE DELETED.
ONLY MODIFY OLD MIGRATIONS FOR BUG FIXES OR ENHANCEMENTS ONLY
[TRY NOT TO MODIFY BEHAVIOR, UNLESS YOU KNOW WHAT YOU'RE DOING].
"""
userdata_db_migrations = []
app/migrations/v1_3_0/__init__.py
+314
View File
@@ -0,0 +1,314 @@
import json
import os
import shutil
import time
from collections import OrderedDict
from sqlite3 import OperationalError
from typing import Generator
from app.db.sqlite.utils import SQLiteManager
from app.migrations.base import Migration
from app.settings import Paths
from app.utils.decorators import coroutine
from app.utils.hashing import create_hash
from app.telemetry import Telemetry
from app.utils.threading import background
# playlists table
# ---------------
# 0: id
# 1: banner_pos
# 2: has_gif
# 3: image
# 4: last_updated
# 5: name
# 6: trackhashes
@background
def send_telemetry():
Telemetry.send_app_installed()
class RemoveSmallThumbnailFolder(Migration):
"""
Removes the small thumbnail folder.
Because we are added a new folder "original" in the same directory, and the small thumbs folder is used to check if an album's thumbnail is already extracted.
So we need to remove it, to force the app to extract thumbnails for all albums.
"""
@staticmethod
def migrate():
send_telemetry()
thumbs_sm_path = Paths.get_sm_thumb_path()
thumbs_lg_path = Paths.get_lg_thumb_path()
for path in [thumbs_sm_path, thumbs_lg_path]:
if os.path.exists(path):
shutil.rmtree(path)
for path in [thumbs_sm_path, thumbs_lg_path]:
os.makedirs(path, exist_ok=True)
class RemovePlaylistArtistHashes(Migration):
"""
removes the artisthashes column from the playlists table.
"""
@staticmethod
def migrate():
# remove artisthashes column
sql = "ALTER TABLE playlists DROP COLUMN artisthashes"
with SQLiteManager(userdata_db=True) as cur:
try:
cur.execute(sql)
except OperationalError:
pass
cur.close()
class AddSettingsToPlaylistTable(Migration):
"""
adds the settings column and removes the banner_pos and has_gif columns
to the playlists table.
"""
@staticmethod
def migrate():
select_playlists_sql = "SELECT * FROM playlists"
with SQLiteManager(userdata_db=True) as cur:
create_playlist_table_sql = """CREATE TABLE IF NOT EXISTS playlists (
id integer PRIMARY KEY,
image text,
last_updated text not null,
name text not null,
settings text,
trackhashes text
);"""
insert_playlist_sql = """INSERT INTO playlists(
image,
last_updated,
name,
settings,
trackhashes
) VALUES(:image, :last_updated, :name, :settings, :trackhashes)
"""
cur.execute(select_playlists_sql)
# load all playlists
playlists = cur.fetchall()
# drop old playlists table
cur.execute("DROP TABLE playlists")
# create new playlists table
cur.execute(create_playlist_table_sql)
def transform_playlists(pipeline: Generator, playlists: tuple):
for playlist in playlists:
# create dict that matches the new schema
p = {
"id": playlist[0],
"name": playlist[5],
"image": playlist[3],
"trackhashes": playlist[6],
"last_updated": playlist[4],
"settings": json.dumps(
{
"has_gif": False,
"banner_pos": playlist[1],
"square_img": False,
"pinned": False,
}
),
}
pipeline.send(p)
@coroutine
def insert_playlist():
while True:
playlist = yield
p = OrderedDict(sorted(playlist.items()))
cur.execute(insert_playlist_sql, p)
# insert playlists using a coroutine
# (my first coroutine)
pipeline = insert_playlist()
transform_playlists(pipeline, playlists)
pipeline.close()
cur.close()
class AddLastUpdatedToTrackTable(Migration):
"""
adds the last modified column to the tracks table.
"""
@staticmethod
def migrate():
# add last_mod column and default to current timestamp
timestamp = time.time()
sql = f"ALTER TABLE tracks ADD COLUMN last_mod text not null DEFAULT '{timestamp}'"
with SQLiteManager() as cur:
try:
cur.execute(sql)
except OperationalError:
pass
cur.close()
class MovePlaylistsAndFavoritesTo10BitHashes(Migration):
"""
moves the playlists and favorites to 10 bit hashes.
"""
@staticmethod
def migrate():
def get_track_data_by_hash(trackhash: str, tracks: list[tuple]) -> tuple:
for track in tracks:
# trackhash is the 15th bit hash
if track[15] == trackhash:
# return artist, album, title
return track[4], track[1], track[13]
def get_track_by_albumhash(albumhash: str, tracks: list[tuple]) -> tuple:
for track in tracks:
# albumhash is the 3rd bit hash
if track[3] == albumhash:
# return album, albumartist
return track[1], track[2]
_base = "SELECT * FROM"
fetch_playlists_sql = f"{_base} playlists"
fetch_tracks_sql = f"{_base} tracks"
update_playlist_hashes_sql = (
"UPDATE playlists SET trackhashes = :trackhashes WHERE id = :id"
)
fetch_favorites_sql = f"{_base} favorites"
update_fav_sql = "UPDATE favorites SET hash = :hash WHERE id = :id"
remove_fav_sql = "DELETE FROM favorites WHERE id = :id"
db_tracks = []
# read tracks from db
with SQLiteManager() as cur:
cur.execute(fetch_tracks_sql)
db_tracks.extend(cur.fetchall())
cur.close()
# update playlists
with SQLiteManager(userdata_db=True) as cur:
cur.execute(fetch_playlists_sql)
playlists = cur.fetchall()
# for each playlist
for p in playlists:
pid = p[0]
# load trackhashes
trackhashes: list[str] = json.loads(p[5])
for index, t in enumerate(trackhashes):
(artist, album, title) = get_track_data_by_hash(t, db_tracks)
# create new hash
new_hash = create_hash(artist, album, title, decode=True, limit=10)
trackhashes[index] = new_hash
# convert to string
trackhashes = json.dumps(trackhashes)
# save to db
cur.execute(
update_playlist_hashes_sql, {"trackhashes": trackhashes, "id": pid}
)
cur.close()
# update favorites
with SQLiteManager(userdata_db=True) as cur:
cur.execute(fetch_favorites_sql)
favorites = cur.fetchall()
# for each favorite
for f in favorites:
fid = f[0]
fhash: str = f[1]
ftype: str = f[2] # "track" || "album"
if ftype == "album":
(album, albumartist) = get_track_by_albumhash(fhash, db_tracks)
# create new hash
new_hash = create_hash(album, albumartist, decode=True, limit=10)
# save to db
cur.execute(update_fav_sql, {"hash": new_hash, "id": fid})
continue
if ftype == "track":
(artist, album, title) = get_track_data_by_hash(fhash, db_tracks)
# create new hash
new_hash = create_hash(artist, album, title, decode=True, limit=10)
# save to db
cur.execute(update_fav_sql, {"hash": new_hash, "id": fid})
continue
# remove favorites that are not track or album. ie. artists
cur.execute(remove_fav_sql, {"id": fid})
cur.close()
class RemoveAllTracks(Migration):
"""
removes all tracks from the tracks table.
"""
@staticmethod
def migrate():
sql = "DELETE FROM tracks"
with SQLiteManager() as cur:
cur.execute(sql)
cur.close()
class UpdateAppSettingsTable(Migration):
@staticmethod
def migrate():
drop_table_sql = "DROP TABLE settings"
create_table_sql = """
CREATE TABLE IF NOT EXISTS settings (
id integer PRIMARY KEY,
root_dirs text NOT NULL,
exclude_dirs text,
artist_separators text NOT NULL default '/,;',
extract_feat integer NOT NULL DEFAULT 1,
remove_prod integer NOT NULL DEFAULT 1,
clean_album_title integer NOT NULL DEFAULT 1,
remove_remaster integer NOT NULL DEFAULT 1,
merge_albums integer NOT NULL DEFAULT 0,
show_albums_as_singles integer NOT NULL DEFAULT 0
);
"""
with SQLiteManager(userdata_db=True) as cur:
cur.execute(drop_table_sql)
cur.execute(create_table_sql)
app/models/album.py
+79 -25
View File
@@ -1,12 +1,13 @@
import dataclasses
import datetime
from dataclasses import dataclass
from .track import Track
from .artist import Artist
from ..utils.hashing import create_hash
from ..utils.parsers import parse_feat_from_title
from app.settings import SessionVarKeys, get_flag
from app.settings import FromFlags
from ..utils.hashing import create_hash
from ..utils.parsers import get_base_title_and_versions, parse_feat_from_title
from .artist import Artist
from .track import Track
@dataclass(slots=True)
@@ -27,28 +28,56 @@ class Album:
date: str = ""
og_title: str = ""
base_title: str = ""
is_soundtrack: bool = False
is_compilation: bool = False
is_single: bool = False
is_EP: bool = False
is_favorite: bool = False
is_live: bool = False
genres: list[str] = dataclasses.field(default_factory=list)
versions: list[str] = dataclasses.field(default_factory=list)
def __post_init__(self):
self.og_title = self.title
self.image = self.albumhash + ".webp"
if FromFlags.EXTRACT_FEAT:
# Fetch album artists from title
if get_flag(SessionVarKeys.EXTRACT_FEAT):
featured, self.title = parse_feat_from_title(self.title)
if len(featured) > 0:
original_lower = "-".join([a.name.lower() for a in self.albumartists])
self.albumartists.extend([Artist(a) for a in featured if a.lower() not in original_lower])
self.albumartists.extend(
[Artist(a) for a in featured if a.lower() not in original_lower]
)
from ..store.tracks import TrackStore
TrackStore.append_track_artists(self.albumhash, featured, self.title)
# Handle album version data
if get_flag(SessionVarKeys.CLEAN_ALBUM_TITLE):
get_versions = not get_flag(SessionVarKeys.MERGE_ALBUM_VERSIONS)
self.title, self.versions = get_base_title_and_versions(
self.title, get_versions=get_versions
)
self.base_title = self.title
if "super_deluxe" in self.versions:
self.versions.remove("deluxe")
if "original" in self.versions and self.check_is_soundtrack():
self.versions.remove("original")
self.versions = [v.replace("_", " ") for v in self.versions]
else:
self.base_title = get_base_title_and_versions(
self.title, get_versions=False
)[0]
self.albumartists_hashes = "-".join(a.artisthash for a in self.albumartists)
def set_colors(self, colors: list[str]):
@@ -78,7 +107,7 @@ class Album:
"""
keywords = ["motion picture", "soundtrack"]
for keyword in keywords:
if keyword in self.title.lower():
if keyword in self.og_title.lower():
return True
return False
@@ -93,10 +122,19 @@ class Album:
if "various artists" in artists:
return True
substrings = [
"the essential", "best of", "greatest hits", "#1 hits", "number ones", "super hits",
"ultimate collection", "anthology", "great hits", "biggest hits", "the hits"
]
substrings = {
"the essential",
"best of",
"greatest hits",
"#1 hits",
"number ones",
"super hits",
"ultimate collection",
"anthology",
"great hits",
"biggest hits",
"the hits",
}
for substring in substrings:
if substring in self.title.lower():
@@ -108,9 +146,9 @@ class Album:
"""
Checks if the album is a live album.
"""
keywords = ["live from", "live at", "live in"]
keywords = ["live from", "live at", "live in", "live on", "mtv unplugged"]
for keyword in keywords:
if keyword in self.title.lower():
if keyword in self.og_title.lower():
return True
return False
@@ -122,27 +160,43 @@ class Album:
return self.title.strip().endswith(" EP")
def check_is_single(self, tracks: list[Track]):
"""
Checks if the album is a single.
"""
keywords = ["single version", "- single"]
show_albums_as_singles = get_flag(SessionVarKeys.SHOW_ALBUMS_AS_SINGLES)
for keyword in keywords:
if keyword in self.title.lower():
if keyword in self.og_title.lower():
self.is_single = True
return
if show_albums_as_singles and len(tracks) == 1:
self.is_single = True
return
if (
len(tracks) == 1
and create_hash(tracks[0].title) == create_hash(self.title) # if they have the same title
# and tracks[0].track == 1
# and tracks[0].disc == 1
# TODO: Review -> Are the above commented checks necessary?
len(tracks) == 1
and (
create_hash(tracks[0].title) == create_hash(self.title)
or create_hash(tracks[0].title) == create_hash(self.og_title)
) # if they have the same title
# and tracks[0].track == 1
# and tracks[0].disc == 1
# TODO: Review -> Are the above commented checks necessary?
):
self.is_single = True
def get_date_from_tracks(self, tracks: list[Track]):
for track in tracks:
if track.date != "Unknown":
self.date = track.date
break
"""
Gets the date of the album its tracks.
Args:
tracks (list[Track]): The tracks of the album.
"""
if self.date:
return
dates = (int(t.date) for t in tracks if t.date)
self.date = datetime.datetime.fromtimestamp(min(dates)).year
app/models/artist.py
+4 -1
View File
@@ -1,4 +1,3 @@
import json
import dataclasses
from dataclasses import dataclass
@@ -20,6 +19,10 @@ class ArtistMinimal:
self.artisthash = create_hash(self.name, decode=True)
self.image = self.artisthash + ".webp"
# hack to override all the variations from unreleased files (sorry guys!)
if self.artisthash == "5a37d5315e":
self.name = "Juice WRLD"
@dataclass(slots=True)
class Artist(ArtistMinimal):
app/models/lastfm.py
+13
View File
@@ -0,0 +1,13 @@
from dataclasses import dataclass
@dataclass
class SimilarArtist:
artisthash: str
similar_artist_hashes: str
def get_artist_hash_set(self) -> set[str]:
"""
Returns a set of similar artists.
"""
return set(self.similar_artist_hashes.split("~"))
app/models/playlist.py
+13 -10
View File
@@ -11,12 +11,10 @@ class Playlist:
"""Creates playlist objects"""
id: int
artisthashes: str | list[str]
banner_pos: int
has_gif: str | bool
image: str
last_updated: str
name: str
settings: str | dict
trackhashes: str | list[str]
thumb: str = ""
@@ -24,16 +22,19 @@ class Playlist:
duration: int = 0
has_image: bool = False
images: list[str] = dataclasses.field(default_factory=list)
pinned: bool = False
def __post_init__(self):
self.trackhashes = json.loads(str(self.trackhashes))
# self.artisthashes = json.loads(str(self.artisthashes))
# commentted until we need it 👆
self.artisthashes = []
self.count = len(self.trackhashes)
self.has_gif = bool(int(self.has_gif))
self.has_image = (Path(settings.Paths.get_playlist_img_path()) / str(self.image)).exists()
if isinstance(self.settings, str):
self.settings = dict(json.loads(self.settings))
self.pinned = self.settings.get("pinned", False)
self.has_image = (
Path(settings.Paths.get_playlist_img_path()) / str(self.image)
).exists()
if self.image is not None:
self.thumb = "thumb_" + self.image
@@ -44,10 +45,12 @@ class Playlist:
def set_duration(self, duration: int):
self.duration = duration
def set_count(self, count: int):
self.count = count
def clear_lists(self):
"""
Removes data from lists to make it lighter for sending
over the API.
"""
self.trackhashes = []
self.artisthashes = []
app/models/track.py
+88 -27
View File
@@ -1,10 +1,16 @@
import dataclasses
from dataclasses import dataclass
from app.settings import FromFlags
from .artist import ArtistMinimal
from app.settings import SessionVarKeys, get_flag
from app.utils.hashing import create_hash
from app.utils.parsers import split_artists, remove_prod, parse_feat_from_title
from app.utils.parsers import (
clean_title,
get_base_title_and_versions,
parse_feat_from_title,
remove_prod,
split_artists,
)
from .artist import ArtistMinimal
@dataclass(slots=True)
@@ -14,12 +20,12 @@ class Track:
"""
album: str
albumartist: str | list[ArtistMinimal]
albumartists: str | list[ArtistMinimal]
albumhash: str
artist: str | list[ArtistMinimal]
artists: str | list[ArtistMinimal]
bitrate: int
copyright: str
date: str
date: int
disc: int
duration: int
filepath: str
@@ -28,46 +34,86 @@ class Track:
title: str
track: int
trackhash: str
last_mod: str | int
filetype: str = ""
image: str = ""
artist_hashes: str = ""
is_favorite: bool = False
# temporary attributes
_pos: int = 0 # for sorting tracks by disc and track number
_ati: str = "" # (album track identifier) for removing duplicates when merging album versions
og_title: str = ""
og_album: str = ""
def __post_init__(self):
self.og_title = self.title
self.og_album = self.album
self.last_mod = int(self.last_mod)
self.date = int(self.date)
if self.artist is not None:
artists = split_artists(self.artist)
if self.artists is not None:
artists = split_artists(self.artists)
new_title = self.title
if FromFlags.EXTRACT_FEAT:
if get_flag(SessionVarKeys.EXTRACT_FEAT):
featured, new_title = parse_feat_from_title(self.title)
original_lower = "-".join([a.lower() for a in artists])
artists.extend([a for a in featured if a.lower() not in original_lower])
original_lower = "-".join([create_hash(a) for a in artists])
artists.extend(
a for a in featured if create_hash(a) not in original_lower
)
if FromFlags.REMOVE_PROD:
self.artist_hashes = "-".join(create_hash(a, decode=True) for a in artists)
self.artists = [ArtistMinimal(a) for a in artists]
albumartists = split_artists(self.albumartists)
if not albumartists:
self.albumartists = self.artists[:1]
else:
self.albumartists = [ArtistMinimal(a) for a in albumartists]
if get_flag(SessionVarKeys.REMOVE_PROD):
new_title = remove_prod(new_title)
# if track is a single
if self.og_title == self.album:
self.album = new_title
self.rename_album(new_title)
if get_flag(SessionVarKeys.REMOVE_REMASTER_FROM_TRACK):
new_title = clean_title(new_title)
self.title = new_title
self.artist_hashes = "-".join(create_hash(a, decode=True) for a in artists)
self.artist = [ArtistMinimal(a) for a in artists]
if get_flag(SessionVarKeys.CLEAN_ALBUM_TITLE):
self.album, _ = get_base_title_and_versions(
self.album, get_versions=False
)
albumartists = split_artists(self.albumartist)
self.albumartist = [ArtistMinimal(a) for a in albumartists]
if get_flag(SessionVarKeys.MERGE_ALBUM_VERSIONS):
self.recreate_albumhash()
self.filetype = self.filepath.rsplit(".", maxsplit=1)[-1]
self.image = self.albumhash + ".webp"
if self.genre is not None:
self.genre = str(self.genre).replace("/", ",").replace(";", ",")
self.genre = str(self.genre).lower().split(",")
if self.genre is not None and self.genre != "":
self.genre = self.genre.lower()
separators = {"/", ";", "&"}
contains_rnb = "r&b" in self.genre
contains_rock = "rock & roll" in self.genre
if contains_rnb:
self.genre = self.genre.replace("r&b", "RnB")
if contains_rock:
self.genre = self.genre.replace("rock & roll", "rock")
for s in separators:
self.genre: str = self.genre.replace(s, ",")
self.genre = self.genre.split(",")
self.genre = [g.strip() for g in self.genre]
self.recreate_hash()
@@ -77,21 +123,36 @@ class Track:
Recreates a track hash if the track title was altered
to prevent duplicate tracks having different hashes.
"""
if self.og_title == self.title:
if self.og_title == self.title and self.og_album == self.album:
return
self.trackhash = create_hash(", ".join([a.name for a in self.artist]), self.album, self.title)
self.trackhash = create_hash(
", ".join(a.name for a in self.artists), self.og_album, self.title
)
def recreate_artists_hash(self):
self.artist_hashes = "-".join(a.artisthash for a in self.artist)
"""
Recreates a track's artist hashes if the artist list was altered
"""
self.artist_hashes = "-".join(a.artisthash for a in self.artists)
def recreate_albumhash(self):
"""
Recreates an albumhash of a track to merge all versions of an album.
"""
albumartists = (a.name for a in self.albumartists)
self.albumhash = create_hash(self.album, *albumartists)
def rename_album(self, new_album: str):
"""
Renames an album
"""
self.album = new_album
def add_artists(self, artists: list[str], new_album_title: str):
for artist in artists:
if create_hash(artist) not in self.artist_hashes:
self.artist.append(ArtistMinimal(artist))
if create_hash(artist, decode=True) not in self.artist_hashes:
self.artists.append(ArtistMinimal(artist))
self.recreate_artists_hash()
self.rename_album(new_album_title)
app/periodic_scan.py
+32
View File
@@ -0,0 +1,32 @@
"""
This module contains functions for the server
"""
import time
from app.lib.populate import Populate, PopulateCancelledError
from app.settings import SessionVarKeys, get_flag, get_scan_sleep_time
from app.utils.generators import get_random_str
from app.utils.threading import background
from app.logger import log
@background
def run_periodic_scans():
"""
Runs periodic scans.
"""
# ValidateAlbumThumbs()
# ValidatePlaylistThumbs()
run_periodic_scan = True
while run_periodic_scan:
run_periodic_scan = get_flag(SessionVarKeys.DO_PERIODIC_SCANS)
try:
Populate(instance_key=get_random_str())
except PopulateCancelledError:
log.error("'run_periodic_scans': Periodic scan cancelled.")
pass
sleep_time = get_scan_sleep_time()
time.sleep(sleep_time)
app/print_help.py
+24 -11
View File
@@ -1,19 +1,32 @@
from app.settings import ALLARGS
from tabulate import tabulate
args = ALLARGS
help_args_list = [
["--help", "-h", "Show this help message"],
["--version", "-v", "Show the app version"],
["--host", "", "Set the host"],
["--port", "", "Set the port"],
["--config", "", "Set the config path"],
["--no-periodic-scan", "-nps", "Disable periodic scan"],
[
"--scan-interval",
"-psi",
"Set the scan interval in seconds. Default 600s (10 minutes)",
],
[
"--build",
"",
"Build the application (in development)",
],
]
HELP_MESSAGE = f"""
Usage: swingmusic [options]
Swing Music is a beautiful, self-hosted music player for your
local audio files. Like a cooler Spotify ... but bring your own music.
Options:
{args.build}: Build the application (in development)
{args.host}: Set the host
{args.port}: Set the port
{args.config}: Set the config path
{', '.join(args.show_feat)}: Do not extract featured artists from the song title
{', '.join(args.show_prod)}: Do not hide producers in the song title
Usage: swingmusic [options] [args]
{', '.join(args.help)}: Show this help message
{', '.join(args.version)}: Show the app version
{tabulate(help_args_list, headers=["Option", "Short", "Description"], tablefmt="simple_grid", maxcolwidths=[None, None, 40])}
"""
app/requests/artists.py
+13 -5
View File
@@ -5,20 +5,28 @@ import requests
from app import settings
from app.utils.hashing import create_hash
from requests import ConnectionError, HTTPError, ReadTimeout
import urllib.parse
def fetch_similar_artists(name: str):
"""
Fetches similar artists from Last.fm
"""
url = f"https://ws.audioscrobbler.com/2.0/?method=artist.getsimilar&artist={name}&api_key=" \
f"{settings.Keys.LASTFM_API}&format=json&limit=250"
url = f"https://ws.audioscrobbler.com/2.0/?method=artist.getsimilar&artist={urllib.parse.quote_plus(name, safe='')}&api_key={settings.Keys.LASTFM_API}&format=json&limit=250"
response = requests.get(url, timeout=10)
response.raise_for_status()
try:
response = requests.get(url, timeout=10)
response.raise_for_status()
except (ConnectionError, ReadTimeout, HTTPError):
return []
data = response.json()
artists = data["similarartists"]["artist"]
try:
artists = data["similarartists"]["artist"]
except KeyError:
return []
for artist in artists:
yield create_hash(artist["name"])
app/serializers/__init__.py
View File
app/serializers/album.py
+33
View File
@@ -0,0 +1,33 @@
from dataclasses import asdict
from app.models import Album
def album_serializer(album: Album, to_remove: set[str]) -> dict:
album_dict = asdict(album)
to_remove.update(key for key in album_dict.keys() if key.startswith("is_"))
for key in to_remove:
album_dict.pop(key, None)
# remove artist images
for artist in album_dict["albumartists"]:
artist.pop("image", None)
return album_dict
def serialize_for_card(album: Album):
props_to_remove = {
"duration",
"count",
"albumartists_hashes",
"og_title",
"base_title",
"genres",
}
return album_serializer(album, props_to_remove)
def serialize_for_card_many(albums: list[Album]):
return [serialize_for_card(a) for a in albums]
app/serializers/artist.py
+23
View File
@@ -0,0 +1,23 @@
from dataclasses import asdict
from app.models.artist import Artist
def serialize_for_card(artist: Artist):
artist_dict = asdict(artist)
props_to_remove = {
"is_favorite",
"trackcount",
"duration",
"albumcount",
}
for key in props_to_remove:
artist_dict.pop(key, None)
return artist_dict
def serialize_for_cards(artists: list[Artist]):
return [serialize_for_card(a) for a in artists]
app/serializers/favorites_serializer.py
+40
View File
@@ -0,0 +1,40 @@
from app.models import Album, Artist, Track
def recent_fav_track_serializer(track: Track) -> dict:
"""
Simplifies a track object into a dictionary to remove unused
properties on the client.
"""
return {
"image": track.image,
"title": track.title,
"trackhash": track.trackhash,
"filepath": track.filepath,
}
def recent_fav_album_serializer(album: Album) -> dict:
"""
Simplifies an album object into a dictionary to remove unused
properties on the client.
"""
return {
"image": album.image,
"title": album.og_title,
"albumhash": album.albumhash,
"artist": album.albumartists[0].name,
"colors": album.colors,
}
def recent_fav_artist_serializer(artist: Artist) -> dict:
"""
Simplifies an artist object into a dictionary to remove unused
properties on the client.
"""
return {
"image": artist.image,
"name": artist.name,
"artisthash": artist.artisthash,
}
app/serializers/track.py
+41
View File
@@ -0,0 +1,41 @@
from dataclasses import asdict
from app.models.track import Track
def serialize_track(track: Track, to_remove: set = {}, remove_disc=True) -> dict:
album_dict = asdict(track)
props = {
"date",
"genre",
"last_mod",
"og_title",
"og_album",
"copyright",
"disc",
"track",
"artist_hashes",
}.union(to_remove)
if not remove_disc:
props.remove("disc")
props.remove("track")
props.update(key for key in album_dict.keys() if key.startswith(("is_", "_")))
props.remove("is_favorite")
for key in props:
album_dict.pop(key, None)
to_remove_images = ["artists", "albumartists"]
for key in to_remove_images:
for artist in album_dict[key]:
artist.pop("image", None)
return album_dict
def serialize_tracks(
tracks: list[Track], _remove: set = {}, remove_disc=True
) -> list[dict]:
return [serialize_track(t, _remove, remove_disc) for t in tracks]
app/settings.py
+102 -8
View File
@@ -2,12 +2,21 @@
Contains default configs
"""
import os
import sys
from typing import Any
from app import configs
join = os.path.join
if getattr(sys, "frozen", False) and hasattr(sys, "_MEIPASS"):
IS_BUILD = True
else:
IS_BUILD = False
class Release:
APP_VERSION = "v1.2.1"
APP_VERSION = "1.3.0"
class Paths:
@@ -26,7 +35,9 @@ class Paths:
@classmethod
def get_config_folder(cls):
return "swingmusic" if cls.get_config_dir() != cls.USER_HOME_DIR else ".swingmusic"
return (
"swingmusic" if cls.get_config_dir() != cls.USER_HOME_DIR else ".swingmusic"
)
@classmethod
def get_app_dir(cls):
@@ -64,6 +75,10 @@ class Paths:
def get_lg_thumb_path(cls):
return join(cls.get_thumbs_path(), "large")
@classmethod
def get_original_thumb_path(cls):
return join(cls.get_thumbs_path(), "original")
@classmethod
def get_assets_path(cls):
return join(Paths.get_app_dir(), "assets")
@@ -71,7 +86,7 @@ class Paths:
# defaults
class Defaults:
THUMB_SIZE = 400
THUMB_SIZE = 500
SM_THUMB_SIZE = 64
SM_ARTIST_IMG_SIZE = 64
"""
@@ -105,6 +120,22 @@ class FLASKVARS:
FLASK_PORT = 1970
FLASK_HOST = "localhost"
@classmethod
def get_flask_port(cls):
return cls.FLASK_PORT
@classmethod
def get_flask_host(cls):
return cls.FLASK_HOST
@classmethod
def set_flask_port(cls, port):
cls.FLASK_PORT = port
@classmethod
def set_flask_host(cls, host):
cls.FLASK_HOST = host
class ALLARGS:
"""
@@ -115,13 +146,23 @@ class ALLARGS:
port = "--port"
host = "--host"
config = "--config"
show_feat = ["--show-feat", "-sf"]
show_prod = ["--show-prod", "-sp"]
help = ["--help", "-h"]
version = ["--version", "-v"]
show_feat = ("--show-feat", "-sf")
show_prod = ("--show-prod", "-sp")
dont_clean_albums = ("--no-clean-albums", "-nca")
dont_clean_tracks = ("--no-clean-tracks", "-nct")
no_periodic_scan = ("--no-periodic-scan", "-nps")
periodic_scan_interval = ("--scan-interval", "-psi")
help = ("--help", "-h")
version = ("--version", "-v")
class FromFlags:
class SessionVars:
"""
Variables that can be altered per session.
"""
EXTRACT_FEAT = True
"""
Whether to extract the featured artists from the song title.
@@ -132,6 +173,44 @@ class FromFlags:
Whether to remove the producers from the song title.
"""
CLEAN_ALBUM_TITLE = True
REMOVE_REMASTER_FROM_TRACK = True
DO_PERIODIC_SCANS = True
PERIODIC_SCAN_INTERVAL = 600 # 10 minutes
"""
The interval between periodic scans in seconds.
"""
MERGE_ALBUM_VERSIONS = False
ARTIST_SEPARATORS = set()
SHOW_ALBUMS_AS_SINGLES = False
# TODO: Find a way to eliminate this class without breaking typings
class SessionVarKeys:
EXTRACT_FEAT = "EXTRACT_FEAT"
REMOVE_PROD = "REMOVE_PROD"
CLEAN_ALBUM_TITLE = "CLEAN_ALBUM_TITLE"
REMOVE_REMASTER_FROM_TRACK = "REMOVE_REMASTER_FROM_TRACK"
DO_PERIODIC_SCANS = "DO_PERIODIC_SCANS"
PERIODIC_SCAN_INTERVAL = "PERIODIC_SCAN_INTERVAL"
MERGE_ALBUM_VERSIONS = "MERGE_ALBUM_VERSIONS"
ARTIST_SEPARATORS = "ARTIST_SEPARATORS"
SHOW_ALBUMS_AS_SINGLES = "SHOW_ALBUMS_AS_SINGLES"
def get_flag(key: SessionVarKeys) -> bool:
return getattr(SessionVars, key)
def set_flag(key: SessionVarKeys, value: Any):
setattr(SessionVars, key, value)
def get_scan_sleep_time() -> int:
return SessionVars.PERIODIC_SCAN_INTERVAL
class TCOLOR:
"""
@@ -153,3 +232,18 @@ class TCOLOR:
class Keys:
# get last fm api key from os environment
LASTFM_API = os.environ.get("LASTFM_API_KEY")
POSTHOG_API_KEY = os.environ.get("POSTHOG_API_KEY")
@classmethod
def load(cls):
if IS_BUILD:
cls.LASTFM_API = configs.LASTFM_API_KEY
cls.POSTHOG_API_KEY = configs.POSTHOG_API_KEY
cls.verify_keys()
@classmethod
def verify_keys(cls):
if not cls.LASTFM_API:
print("ERROR: LASTFM_API_KEY not set in environment")
sys.exit(0)
app/setup/__init__.py
+15 -6
View File
@@ -1,12 +1,13 @@
"""
Prepares the server for use.
"""
from app.db.sqlite.settings import load_settings
from app.setup.files import create_config_dir
from app.setup.sqlite import setup_sqlite, run_migrations
from app.setup.sqlite import run_migrations, setup_sqlite
from app.store.albums import AlbumStore
from app.store.tracks import TrackStore
from app.store.artists import ArtistStore
from app.store.tracks import TrackStore
from app.utils.generators import get_random_str
def run_setup():
@@ -14,6 +15,14 @@ def run_setup():
setup_sqlite()
run_migrations()
TrackStore.load_all_tracks()
AlbumStore.load_albums()
ArtistStore.load_artists()
try:
load_settings()
except IndexError:
# settings table is empty
pass
instance_key = get_random_str()
TrackStore.load_all_tracks(instance_key)
AlbumStore.load_albums(instance_key)
ArtistStore.load_artists(instance_key)
app/setup/files.py
+6 -13
View File
@@ -5,29 +5,20 @@ create the config directory and copy the assets to the app directory.
import os
import shutil
from configparser import ConfigParser
from app import settings
from app.utils.filesystem import get_home_res_path
config = ConfigParser()
config_path = get_home_res_path("pyinstaller.config.ini")
config.read(config_path)
try:
IS_BUILD = config["DEFAULT"]["BUILD"] == "True"
except KeyError:
# If the key doesn't exist, it means that the app is being executed in dev mode.
IS_BUILD = False
class CopyFiles:
"""Copies assets to the app directory."""
"""
Copies assets to the app directory.
"""
def __init__(self) -> None:
assets_dir = "assets"
if IS_BUILD:
if settings.IS_BUILD:
assets_dir = get_home_res_path("assets")
files = [
@@ -63,6 +54,7 @@ def create_config_dir() -> None:
thumb_path = os.path.join("images", "thumbnails")
small_thumb_path = os.path.join(thumb_path, "small")
large_thumb_path = os.path.join(thumb_path, "large")
original_thumb_path = os.path.join(thumb_path, "original")
artist_img_path = os.path.join("images", "artists")
small_artist_img_path = os.path.join(artist_img_path, "small")
@@ -76,6 +68,7 @@ def create_config_dir() -> None:
thumb_path,
small_thumb_path,
large_thumb_path,
original_thumb_path,
artist_img_path,
small_artist_img_path,
large_artist_img_path,
app/setup/sqlite.py
+11 -19
View File
@@ -4,12 +4,17 @@ Applies migrations.
"""
from app.db.sqlite import create_connection, create_tables, queries
from app.migrations import apply_migrations, set_postinit_migration_versions
from app.migrations.__preinit import run_preinit_migrations, set_preinit_migration_versions
from app.migrations import apply_migrations
from app.settings import Db
def run_migrations():
"""
Run migrations and updates migration version.
"""
apply_migrations()
def setup_sqlite():
"""
Create Sqlite databases and tables.
@@ -17,25 +22,12 @@ def setup_sqlite():
# if os.path.exists(DB_PATH):
# os.remove(DB_PATH)
run_preinit_migrations()
app_db_conn = create_connection(Db.get_app_db_path())
playlist_db_conn = create_connection(Db.get_userdata_db_path())
user_db_conn = create_connection(Db.get_userdata_db_path())
create_tables(app_db_conn, queries.CREATE_APPDB_TABLES)
create_tables(playlist_db_conn, queries.CREATE_USERDATA_TABLES)
create_tables(user_db_conn, queries.CREATE_USERDATA_TABLES)
create_tables(app_db_conn, queries.CREATE_MIGRATIONS_TABLE)
create_tables(playlist_db_conn, queries.CREATE_MIGRATIONS_TABLE)
app_db_conn.close()
playlist_db_conn.close()
def run_migrations():
"""
Run migrations and updates migration version.
"""
apply_migrations()
set_preinit_migration_versions()
set_postinit_migration_versions()
user_db_conn.close()
app/start_info_logger.py
+6 -25
View File
@@ -1,6 +1,6 @@
import os
from app.settings import TCOLOR, Release, FLASKVARS, Paths, FromFlags
from app.settings import FLASKVARS, TCOLOR, Paths, Release
from app.utils.network import get_ip
@@ -12,39 +12,20 @@ def log_startup_info():
print(lines)
print(f"{TCOLOR.HEADER}SwingMusic {Release.APP_VERSION} {TCOLOR.ENDC}")
adresses = [FLASKVARS.FLASK_HOST]
adresses = [FLASKVARS.get_flask_host()]
if FLASKVARS.FLASK_HOST == "0.0.0.0":
if FLASKVARS.get_flask_host() == "0.0.0.0":
adresses = ["localhost", get_ip()]
print("Started app on:")
for address in adresses:
# noinspection HttpUrlsUsage
print(
f"➤ {TCOLOR.OKGREEN}http://{address}:{FLASKVARS.FLASK_PORT}{TCOLOR.ENDC}"
f"➤ {TCOLOR.OKGREEN}http://{address}:{FLASKVARS.get_flask_port()}{TCOLOR.ENDC}"
)
print(lines)
print("\n")
print(lines+"\n")
to_print = [
[
"Extract featured artists from titles",
FromFlags.EXTRACT_FEAT
],
[
"Remove prod. from titles",
FromFlags.REMOVE_PROD
]
]
for item in to_print:
print(
f"{item[0]}: {TCOLOR.FAIL}{item[1]}{TCOLOR.ENDC}"
)
print(
f"{TCOLOR.YELLOW}Data folder: {Paths.get_app_dir()}{TCOLOR.ENDC}"
)
print(f"{TCOLOR.YELLOW}Data folder: {Paths.get_app_dir()}{TCOLOR.ENDC}")
print("\n")
app/store/albums.py
+41 -23
View File
@@ -1,11 +1,15 @@
import json
import random
from tqdm import tqdm
from app.db.sqlite.albumcolors import SQLiteAlbumMethods as aldb
from app.models import Album, Track
from app.db.sqlite.albums import SQLiteAlbumMethods as aldb
from ..utils.hashing import create_hash
from .tracks import TrackStore
from app.utils.progressbar import tqdm
ALBUM_LOAD_KEY = ""
class AlbumStore:
@@ -18,21 +22,26 @@ class AlbumStore:
"""
return Album(
albumhash=track.albumhash,
albumartists=track.albumartist, # type: ignore
title=track.album,
albumartists=track.albumartists, # type: ignore
title=track.og_album,
)
@classmethod
def load_albums(cls):
def load_albums(cls, instance_key: str):
"""
Loads all albums from the database into the store.
"""
global ALBUM_LOAD_KEY
ALBUM_LOAD_KEY = instance_key
cls.albums = []
albumhashes = set(t.albumhash for t in TrackStore.tracks)
for albumhash in tqdm(albumhashes, desc="Loading albums"):
for albumhash in tqdm(albumhashes, desc=f"Loading albums"):
if instance_key != ALBUM_LOAD_KEY:
return
for track in TrackStore.tracks:
if track.albumhash == albumhash:
cls.albums.append(cls.create_album(track))
@@ -40,7 +49,7 @@ class AlbumStore:
db_albums: list[tuple] = aldb.get_all_albums()
for album in tqdm(db_albums, desc="Mapping album colors"):
for album in db_albums:
albumhash = album[1]
colors = json.loads(album[2])
@@ -65,15 +74,21 @@ class AlbumStore:
@classmethod
def get_albums_by_albumartist(
cls, artisthash: str, limit: int, exclude: str
cls, artisthash: str, limit: int, exclude: str
) -> list[Album]:
"""
Returns N albums by the given albumartist, excluding the specified album.
"""
albums = [album for album in cls.albums if artisthash in album.albumartists_hashes]
albums = [
album for album in cls.albums if artisthash in album.albumartists_hashes
]
albums = [album for album in albums if album.albumhash != exclude]
albums = [
album
for album in albums
if create_hash(album.base_title) != create_hash(exclude)
]
if len(albums) > limit:
random.shuffle(albums)
@@ -86,10 +101,11 @@ class AlbumStore:
"""
Returns an album by its hash.
"""
try:
return [a for a in cls.albums if a.albumhash == albumhash][0]
except IndexError:
return None
for album in cls.albums:
if album.albumhash == albumhash:
return album
return None
@classmethod
def get_albums_by_hashes(cls, albumhashes: list[str]) -> list[Album]:
@@ -108,21 +124,16 @@ class AlbumStore:
"""
Returns all albums by the given artist.
"""
return [album for album in cls.albums if artisthash in album.albumartists_hashes]
return [
album for album in cls.albums if artisthash in album.albumartists_hashes
]
@classmethod
def count_albums_by_artisthash(cls, artisthash: str):
"""
Count albums for the given artisthash.
"""
albumartists = [a.albumartists for a in cls.albums]
artisthashes = []
for artist in albumartists:
artisthashes.extend([a.artisthash for a in artist]) # type: ignore
master_string = "-".join(artisthashes)
master_string = "-".join(a.albumartists_hashes for a in cls.albums)
return master_string.count(artisthash)
@classmethod
@@ -132,6 +143,13 @@ class AlbumStore:
"""
return albumhash in "-".join([a.albumhash for a in cls.albums])
@classmethod
def remove_album(cls, album: Album):
"""
Removes an album from the store.
"""
cls.albums.remove(album)
@classmethod
def remove_album_by_hash(cls, albumhash: str):
"""
app/store/artists.py
+20 -11
View File
@@ -1,29 +1,35 @@
import json
from tqdm import tqdm
from app.db.sqlite.artists import SQLiteArtistMethods as ardb
from app.db.sqlite.artistcolors import SQLiteArtistMethods as ardb
from app.lib.artistlib import get_all_artists
from app.models import Artist
from app.utils.bisection import UseBisection
from .tracks import TrackStore
from app.utils.progressbar import tqdm
from .albums import AlbumStore
from .tracks import TrackStore
ARTIST_LOAD_KEY = ""
class ArtistStore:
artists: list[Artist] = []
@classmethod
def load_artists(cls):
def load_artists(cls, instance_key: str):
"""
Loads all artists from the database into the store.
"""
global ARTIST_LOAD_KEY
ARTIST_LOAD_KEY = instance_key
cls.artists = get_all_artists(TrackStore.tracks, AlbumStore.albums)
db_artists: list[tuple] = list(ardb.get_all_artists())
for artist in ardb.get_all_artists():
if instance_key != ARTIST_LOAD_KEY:
return
for art in tqdm(db_artists, desc="Loading artists"):
cls.map_artist_color(art)
cls.map_artist_color(artist)
@classmethod
def map_artist_color(cls, artist_tuple: tuple):
@@ -61,8 +67,11 @@ class ArtistStore:
Returns an artist by its hash.P
"""
artists = sorted(cls.artists, key=lambda x: x.artisthash)
artist = UseBisection(artists, "artisthash", [artisthash])()[0]
return artist
try:
artist = UseBisection(artists, "artisthash", [artisthash])()[0]
return artist
except IndexError:
return None
@classmethod
def get_artists_by_hashes(cls, artisthashes: list[str]) -> list[Artist]:
@@ -89,7 +98,7 @@ class ArtistStore:
for track in TrackStore.tracks:
artists.update(track.artist_hashes)
album_artists: list[str] = [a.artisthash for a in track.albumartist]
album_artists: list[str] = [a.artisthash for a in track.albumartists]
artists.update(album_artists)
master_hash = "-".join(artists)
app/store/tracks.py
+51 -18
View File
@@ -1,27 +1,36 @@
from tqdm import tqdm
# from tqdm import tqdm
from app.models import Track
from app.db.sqlite.favorite import SQLiteFavoriteMethods as favdb
from app.db.sqlite.tracks import SQLiteTrackMethods as tdb
from app.models import Track
from app.utils.bisection import UseBisection
from app.utils.remove_duplicates import remove_duplicates
from app.utils.progressbar import tqdm
TRACKS_LOAD_KEY = ""
class TrackStore:
tracks: list[Track] = []
@classmethod
def load_all_tracks(cls):
def load_all_tracks(cls, instance_key: str):
"""
Loads all tracks from the database into the store.
"""
global TRACKS_LOAD_KEY
TRACKS_LOAD_KEY = instance_key
cls.tracks = list(tdb.get_all_tracks())
fav_hashes = favdb.get_fav_tracks()
fav_hashes = " ".join([t[1] for t in fav_hashes])
print("\n") # adds space between progress bars and startup info
for track in tqdm(cls.tracks, desc="Loading tracks"):
if instance_key != TRACKS_LOAD_KEY:
return
if track.trackhash in fav_hashes:
track.is_favorite = True
@@ -41,6 +50,16 @@ class TrackStore:
cls.tracks.extend(tracks)
@classmethod
def remove_track_obj(cls, track: Track):
"""
Removes a single track from the store.
"""
try:
cls.tracks.remove(track)
except ValueError:
pass
@classmethod
def remove_track_by_filepath(cls, filepath: str):
"""
@@ -49,9 +68,19 @@ class TrackStore:
for track in cls.tracks:
if track.filepath == filepath:
cls.tracks.remove(track)
cls.remove_track_obj(track)
break
@classmethod
def remove_tracks_by_filepaths(cls, filepaths: set[str]):
"""
Removes multiple tracks from the store by their filepaths.
"""
for track in cls.tracks:
if track.filepath in filepaths:
cls.remove_track_obj(track)
@classmethod
def remove_tracks_by_dir_except(cls, dirs: list[str]):
"""Removes all tracks not in the root directories."""
@@ -64,18 +93,11 @@ class TrackStore:
tdb.remove_tracks_by_folders(to_remove)
@classmethod
def count_tracks_by_hash(cls, trackhash: str) -> int:
def count_tracks_by_trackhash(cls, trackhash: str) -> int:
"""
Counts the number of tracks with a specific hash.
Counts the number of tracks with a specific trackhash.
"""
count = 0
for track in cls.tracks:
if track.trackhash == trackhash:
count += 1
return count
return sum(1 for track in cls.tracks if track.trackhash == trackhash)
@classmethod
def make_track_fav(cls, trackhash: str):
@@ -98,7 +120,9 @@ class TrackStore:
track.is_favorite = False
@classmethod
def append_track_artists(cls, albumhash: str, artists: list[str], new_album_title:str):
def append_track_artists(
cls, albumhash: str, artists: list[str], new_album_title: str
):
tracks = cls.get_tracks_by_albumhash(albumhash)
for track in tracks:
@@ -135,12 +159,21 @@ class TrackStore:
Returns all tracks matching the given album hash.
"""
tracks = [t for t in cls.tracks if t.albumhash == album_hash]
return remove_duplicates(tracks)
return remove_duplicates(tracks, is_album_tracks=True)
@classmethod
def get_tracks_by_artist(cls, artisthash: str) -> list[Track]:
def get_tracks_by_artisthash(cls, artisthash: str):
"""
Returns all tracks matching the given artist. Duplicate tracks are removed.
"""
tracks = [t for t in cls.tracks if artisthash in t.artist_hashes]
return remove_duplicates(tracks)
tracks = remove_duplicates(tracks)
tracks.sort(key=lambda x: x.last_mod)
return tracks
@classmethod
def get_tracks_in_path(cls, path: str):
"""
Returns all tracks in the given path.
"""
return (t for t in cls.tracks if t.folder.startswith(path))
app/telemetry.py
+76
View File
@@ -0,0 +1,76 @@
import uuid as UUID
from posthog import Posthog
from app.logger import log
from app.settings import Keys, Paths, Release
from app.utils.hashing import create_hash
from app.utils.network import has_connection
class Telemetry:
"""
Handles sending telemetry data to posthog.
"""
user_id = ""
off = False
@classmethod
def init(cls) -> None:
try:
cls.posthog = Posthog(
project_api_key=Keys.POSTHOG_API_KEY,
host="https://app.posthog.com",
disable_geoip=False,
)
cls.create_userid()
except AssertionError:
cls.disable_telemetry()
@classmethod
def create_userid(cls):
"""
Creates a unique user id for the user and saves it to a file.
"""
uuid_path = Paths.get_app_dir() + "/userid.txt"
try:
with open(uuid_path, "r") as f:
cls.user_id = f.read().strip()
except FileNotFoundError:
uuid = str(UUID.uuid4())
cls.user_id = "user_" + create_hash(uuid, limit=15)
with open(uuid_path, "w") as f:
f.write(cls.user_id)
@classmethod
def disable_telemetry(cls):
cls.off = True
@classmethod
def send_event(cls, event: str):
"""
Sends an event to posthog.
"""
if cls.off:
return
if has_connection():
cls.posthog.capture(cls.user_id, event=f"v{Release.APP_VERSION}-{event}")
@classmethod
def send_app_installed(cls):
"""
Sends an event to posthog when the app is installed.
"""
cls.send_event("app-installed")
@classmethod
def send_artist_visited(cls):
"""
Sends an event to posthog when an artist page is visited.
"""
cls.send_event("artist-page-visited")
app/utils/bisection.py
+18 -23
View File
@@ -1,3 +1,6 @@
from app.models.track import Track
class UseBisection:
"""
Uses bisection to find a list of items in another list.
@@ -6,10 +9,13 @@ class UseBisection:
items.
"""
def __init__(self, source: list, search_from: str, queries: list[str]) -> None:
def __init__(
self, source: list, search_from: str, queries: list[str], limit=-1
) -> None:
self.source_list = source
self.queries_list = queries
self.attr = search_from
self.limit = limit
def find(self, query: str):
left = 0
@@ -26,32 +32,21 @@ class UseBisection:
return None
def __call__(self) -> list:
def __call__(self):
if len(self.source_list) == 0:
return [None]
return []
return [self.find(query) for query in self.queries_list]
results: list[Track] = []
for query in self.queries_list:
res = self.find(query)
def bisection_search_string(strings: list[str], target: str) -> str | None:
"""
Finds a string in a list of strings using bisection search.
"""
if not strings:
return None
if res is None:
continue
strings = sorted(strings)
results.append(res)
left = 0
right = len(strings) - 1
while left <= right:
middle = (left + right) // 2
if strings[middle] == target:
return strings[middle]
if self.limit != -1 and len(results) >= self.limit:
break
if strings[middle] < target:
left = middle + 1
else:
right = middle - 1
return None
return results
app/utils/dates.py
+3 -39
View File
@@ -1,3 +1,4 @@
import pendulum
from datetime import datetime
_format = "%Y-%m-%d %H:%M:%S"
@@ -21,44 +22,7 @@ def date_string_to_time_passed(prev_date: str) -> str:
diff = now - then
seconds = diff.seconds
print(seconds)
if seconds < 0:
return "from the future 🛸"
now = pendulum.now()
return now.subtract(seconds=seconds).diff_for_humans()
if seconds < 15:
return "now"
if seconds < 60:
return f"{int(seconds)} seconds ago"
if seconds < 3600:
return f"{int(seconds // 60)} minutes ago"
if seconds < 86400:
return f"{int(seconds // 3600)} hours ago"
days = diff.days
if days == 1:
return "yesterday"
if days < 7:
return f"{days} days ago"
if days < 14:
return "1 week ago"
if days < 30:
return f"{int(days // 7)} weeks ago"
if days < 60:
return "1 month ago"
if days < 365:
return f"{int(days // 30)} months ago"
if days < 730:
return "1 year ago"
return f"{int(days // 365)} years ago"
app/utils/decorators.py
+11
View File
@@ -0,0 +1,11 @@
def coroutine(func):
"""
Decorator: primes `func` by advancing to first `yield`
"""
def start(*args, **kwargs):
cr = func(*args, **kwargs)
next(cr)
return cr
return start
app/utils/hashing.py
+12 -16
View File
@@ -3,29 +3,25 @@ import hashlib
from unidecode import unidecode
def create_hash(*args: str, decode=False, limit=7) -> str:
def create_hash(*args: str, decode=False, limit=10) -> str:
"""
Creates a simple hash for an album
"""
str_ = "".join(args)
def remove_non_alnum(token: str) -> str:
token = token.lower().strip().replace(" ", "")
t = "".join(t for t in token if t.isalnum())
if t == "":
return token
return t
str_ = "".join(remove_non_alnum(t) for t in args)
if decode:
str_ = unidecode(str_)
str_ = str_.lower().strip().replace(" ", "")
str_ = "".join(t for t in str_ if t.isalnum())
str_ = str_.encode("utf-8")
str_ = hashlib.sha256(str_).hexdigest()
return str_[-limit:]
def create_folder_hash(*args: str, limit=7) -> str:
"""
Creates a simple hash for an album
"""
strings = [s.lower().strip().replace(" ", "") for s in args]
strings = ["".join([t for t in s if t.isalnum()]) for s in strings]
strings = [s.encode("utf-8") for s in strings]
strings = [hashlib.sha256(s).hexdigest()[-limit:] for s in strings]
return "".join(strings)
app/utils/network.py
+10 -12
View File
@@ -1,19 +1,18 @@
import requests
import socket as Socket
class Ping:
def has_connection(host="8.8.8.8", port=53, timeout=3):
"""
Checks if there is a connection to the internet by pinging google.com
Host: 8.8.8.8 (google-public-dns-a.google.com)
OpenPort: 53/tcp
Service: domain (DNS/TCP)
"""
@staticmethod
def __call__() -> bool:
try:
requests.get("https://google.com", timeout=10)
return True
except (requests.exceptions.ConnectionError, requests.Timeout):
return False
try:
Socket.setdefaulttimeout(timeout)
Socket.socket(Socket.AF_INET, Socket.SOCK_STREAM).connect((host, port))
return True
except Socket.error as ex:
return False
def get_ip():
@@ -26,4 +25,3 @@ def get_ip():
soc.close()
return ip_address
app/utils/parsers.py
+125 -5
View File
@@ -1,11 +1,21 @@
import re
from app.enums.album_versions import AlbumVersionEnum, get_all_keywords
from app.settings import SessionVarKeys, get_flag
def split_artists(src: str, with_and: bool = False):
exp = r"\s*(?: and |&|,|;)\s*" if with_and else r"\s*[,;]\s*"
artists = re.split(exp, src)
return [a.strip() for a in artists]
def split_artists(src: str):
"""
Splits a string of artists into a list of artists.
"""
separators: set = get_flag(SessionVarKeys.ARTIST_SEPARATORS)
for sep in separators:
src = src.replace(sep, ",")
artists = src.split(",")
artists = [a.strip() for a in artists]
return [a for a in artists if a]
def parse_artist_from_filename(title: str):
@@ -79,8 +89,118 @@ def parse_feat_from_title(title: str) -> tuple[list[str], str]:
return [], title
artists = match.group(1)
artists = split_artists(artists, with_and=True)
artists = split_artists(artists)
# remove "feat" group from title
new_title = re.sub(regex, "", title, flags=re.IGNORECASE)
return artists, new_title
def get_base_album_title(string: str) -> tuple[str, str | None]:
"""
Extracts the base album title from a string.
"""
pattern = re.compile(
rf"\s*(\(|\[)[^\)\]]*?({get_all_keywords()})[^\)\]]*?(\)|\])$",
re.IGNORECASE,
)
# TODO: Fix "Redundant character escape '\]' in RegExp "
match = pattern.search(string)
if match:
removed_block = match.group(0)
title = string.replace(removed_block, "")
return title.strip(), removed_block.strip()
return string, None
def get_anniversary(text: str) -> str | None:
"""
Extracts anniversary from text using regex.
"""
_end = "anniversary"
match = re.search(r"\b\d+\w*(?= anniversary)", text, re.IGNORECASE)
if match:
return match.group(0).strip().lower() + f" {_end}"
else:
return _end
def get_album_info(bracket_text: str | None) -> list[str]:
"""
Extracts album version info from the bracketed text on an album title string using regex.
"""
if not bracket_text:
return []
# replace all non-alphanumeric characters with an empty string
bracket_text = re.sub(r"[^a-zA-Z0-9\s]", "", bracket_text)
versions = []
for version_keywords in AlbumVersionEnum:
for keyword in version_keywords.value:
if re.search(keyword, bracket_text, re.IGNORECASE):
versions.append(version_keywords.name.lower())
break
if "anniversary" in versions:
anniversary = get_anniversary(bracket_text)
versions.insert(0, anniversary)
versions.remove("anniversary")
return versions
def get_base_title_and_versions(
original_album_title: str, get_versions=True
) -> tuple[str, list[str]]:
"""
Extracts the base album title and version info from an album title string using regex.
"""
album_title, version_block = get_base_album_title(original_album_title)
if version_block is None:
return original_album_title, []
if not get_versions:
return album_title, []
versions = get_album_info(version_block)
# if no version info could be extracted, accept defeat!
if len(versions) == 0:
album_title = original_album_title
return album_title, versions
def remove_bracketed_remaster(text: str):
"""
Removes remaster info from a track title that contains brackets using regex.
"""
return re.sub(
r"\s*[\\[(][^)\]]*remaster[^)\]]*[)\]]\s*", "", text, flags=re.IGNORECASE
).strip()
def remove_hyphen_remasters(text: str):
"""
Removes remaster info from a track title that contains a hypen (-) using regex.
"""
return re.sub(
r"\s-\s*[^-]*\bremaster[^-]*\s*", "", text, flags=re.IGNORECASE
).strip()
def clean_title(title: str) -> str:
"""
Removes remaster info from a track title using regex.
"""
if "remaster" not in title.lower():
return title
rem_1 = remove_bracketed_remaster(title)
rem_2 = remove_hyphen_remasters(title)
return rem_1 if len(rem_2) > len(rem_1) else rem_2
app/utils/progressbar.py
+16
View File
@@ -0,0 +1,16 @@
from tqdm import tqdm as _tqdm
def tqdm(*args, **kwargs):
"""
Wrapper for tqdm that sets globals.
"""
bar_format = "{percentage:3.0f}%|{bar:45}|{n_fmt}/{total_fmt}{desc}"
kwargs["bar_format"] = bar_format
if "desc" in kwargs:
print(f'INFO|{kwargs["desc"].capitalize()} ...')
kwargs["desc"] = ""
return _tqdm(*args, **kwargs)
app/utils/remove_duplicates.py
+31 -4
View File
@@ -2,21 +2,48 @@ from collections import defaultdict
from operator import attrgetter
from app.models import Track
from app.utils.hashing import create_hash
def remove_duplicates(tracks: list[Track]) -> list[Track]:
def remove_duplicates(tracks: list[Track], is_album_tracks=False) -> list[Track]:
"""
Remove duplicates from a list of Track objects based on the trackhash attribute.
Retain objects with the highest bitrate.
"""
hash_to_tracks = defaultdict(list)
tracks_dict = defaultdict(list)
# if is_album_tracks, sort by disc and track number
if is_album_tracks:
for t in tracks:
# _pos is used for sorting tracks by disc and track number
t._pos = int(f"{t.disc}{str(t.track).zfill(3)}")
# _ati is used to remove duplicates when merging album versions
t._ati = f"{t._pos}{create_hash(t.title)}"
# create groups of tracks with the same _ati
for track in tracks:
tracks_dict[track._ati].append(track)
tracks = []
# pick the track with max bitrate for each group
for track_group in tracks_dict.values():
max_bitrate_track = max(track_group, key=attrgetter("bitrate"))
tracks.append(max_bitrate_track)
return sorted(tracks, key=lambda t: t._pos)
# else, sort by trackhash
for track in tracks:
hash_to_tracks[track.trackhash].append(track)
# create groups of tracks with the same trackhash
tracks_dict[track.trackhash].append(track)
tracks = []
for track_group in hash_to_tracks.values():
# pick the track with max bitrate for each trackhash group
for track_group in tracks_dict.values():
max_bitrate_track = max(track_group, key=attrgetter("bitrate"))
tracks.append(max_bitrate_track)
app/utils/threading.py
+1 -2
View File
@@ -3,8 +3,7 @@ import threading
def background(func):
"""
a threading decorator
use @background above the function you want to run in the background
Runs the decorated function in a background thread.
"""
def background_func(*a, **kw):
app/utils/unicode.py
+5
View File
@@ -0,0 +1,5 @@
def handle_unicode(string: str):
"""
Handles Unicode errors by ignoring unicode characters
"""
return string.encode("utf-16", "ignore").decode("utf-16")
app/utils/wintools.py
+3 -1
View File
@@ -1,12 +1,14 @@
import platform
IS_WIN = platform.system() == "Windows"
# TODO: Check is_windows on app start in settings.py
def is_windows():
"""
Returns True if the OS is Windows.
"""
return platform.system() == "Windows"
return IS_WIN
def win_replace_slash(path: str):
assets/logo-fill.ico
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 4.2 KiB

manage.py
+54 -10
View File
@@ -2,17 +2,36 @@
This file is used to run the application.
"""
import logging
import mimetypes
import os
import setproctitle
from flask import request
from app.telemetry import Telemetry
from app.api import create_api
from app.arg_handler import HandleArgs
from app.functions import run_periodic_checks
from app.lib.watchdogg import Watcher as WatchDog
from app.settings import FLASKVARS
from app.periodic_scan import run_periodic_scans
from app.settings import FLASKVARS, Keys
from app.setup import run_setup
from app.start_info_logger import log_startup_info
from app.utils.filesystem import get_home_res_path
from app.utils.threading import background
mimetypes.add_type("text/css", ".css")
mimetypes.add_type("text/javascript", ".js")
mimetypes.add_type("text/plain", ".txt")
mimetypes.add_type("text/html", ".html")
mimetypes.add_type("image/webp", ".webp")
mimetypes.add_type("image/svg+xml", ".svg")
mimetypes.add_type("image/png", ".png")
mimetypes.add_type("image/vnd.microsoft.icon", ".ico")
mimetypes.add_type("image/gif", ".gif")
mimetypes.add_type("font/woff", ".woff")
mimetypes.add_type("application/manifest+json", ".webmanifest")
werkzeug = logging.getLogger("werkzeug")
werkzeug.setLevel(logging.ERROR)
@@ -20,12 +39,25 @@ app = create_api()
app.static_folder = get_home_res_path("client")
# @app.route("/", defaults={"path": ""})
@app.route("/<path:path>")
def serve_client_files(path):
def serve_client_files(path: str):
"""
Serves the static files in the client folder.
"""
return app.send_static_file(path)
js_or_css = path.endswith(".js") or path.endswith(".css")
if not js_or_css:
return app.send_static_file(path)
gzipped_path = path + ".gz"
if request.headers.get("Accept-Encoding", "").find("gzip") >= 0:
if os.path.exists(os.path.join(app.static_folder, gzipped_path)):
response = app.make_response(app.send_static_file(gzipped_path))
response.headers["Content-Encoding"] = "gzip"
return response
else:
return app.send_static_file(path)
@app.route("/")
@@ -39,7 +71,7 @@ def serve_client():
@background
def bg_run_setup() -> None:
run_setup()
run_periodic_checks()
run_periodic_scans()
@background
@@ -47,18 +79,30 @@ def start_watchdog():
WatchDog().run()
if __name__ == "__main__":
@background
def init_telemetry():
Telemetry.init()
def run_swingmusic():
Keys.load()
HandleArgs()
log_startup_info()
bg_run_setup()
start_watchdog()
init_telemetry()
setproctitle.setproctitle(
f"swingmusic - {FLASKVARS.FLASK_HOST}:{FLASKVARS.FLASK_PORT}"
)
if __name__ == "__main__":
run_swingmusic()
app.run(
debug=False,
threaded=True,
host=FLASKVARS.FLASK_HOST,
port=FLASKVARS.FLASK_PORT,
host=FLASKVARS.get_flask_host(),
port=FLASKVARS.get_flask_port(),
use_reloader=False,
)
# TODO: Organize code in this file: move args to new file, etc.
poetry.lock
Generated
+980 -507
View File
File diff suppressed because it is too large Load Diff
pyinstaller.config.ini
-3
View File
@@ -1,3 +0,0 @@
[DEFAULT]
build = False
pyproject.toml
+10 -1
View File
@@ -18,18 +18,27 @@ rapidfuzz = "^2.13.7"
tinytag = "^1.8.1"
Unidecode = "^1.3.6"
psutil = "^5.9.4"
show-in-file-manager = "^1.1.4"
pendulum = "^2.1.2"
flask-compress = "^1.13"
tabulate = "^0.9.0"
setproctitle = "^1.3.2"
posthog = "^3.0.2"
[tool.poetry.dev-dependencies]
pylint = "^2.15.5"
pytest = "^7.1.3"
hypothesis = "^6.56.3"
pyinstaller = "^5.9.0"
[tool.poetry.dev-dependencies.black]
version = "^22.6.0"
allow-prereleases = true
[tool.poetry.group.dev.dependencies]
pyinstaller = "^5.9.0"
[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"
screenshots/README.md
-12
View File
@@ -1,12 +0,0 @@
### Open the following links in new tabs (`ctrl + click`) to view screenshots:
> These images were taken on v1.10, they might be a lil bit outdated.
1. [Album page](https://github.com/geoffrey45/swingmusic/blob/master/screenshots/album-page.png?raw=true)
2. [Artist page](https://github.com/geoffrey45/swingmusic/blob/master/screenshots/artist-page-1.png?raw=true)
3. [Artist page continuation](https://github.com/geoffrey45/swingmusic/blob/master/screenshots/artist-page-2.png?raw=true)
4. [Artist discography](https://github.com/geoffrey45/swingmusic/blob/master/screenshots/artist-discography-page.png?raw=true)
5. [Folder page](https://github.com/geoffrey45/swingmusic/blob/master/screenshots/folder-page.png?raw=true)
6. [Favorite page](https://github.com/geoffrey45/swingmusic/blob/master/screenshots/favorites-page.png?raw=true)
7. [Playlist page](https://github.com/geoffrey45/swingmusic/blob/master/screenshots/playlist-page.png?raw=true)
8. [Queue page](https://github.com/geoffrey45/swingmusic/blob/master/screenshots/queue-page.png?raw=true)
9. [Search page](https://github.com/geoffrey45/swingmusic/blob/master/screenshots/search-page.png?raw=true)
10. [Settings page](https://github.com/geoffrey45/swingmusic/blob/master/screenshots/settings-page.png?raw=true)
screenshots/album-page.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 658 KiB

screenshots/album.webp
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 56 KiB

screenshots/album2.webp
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 199 KiB

Some files were not shown because too many files have changed in this diff Show More