MyClub/DOCS/FILES_MANAGEMENT_SYSTEM.md

Files Management System

Overview

A comprehensive files management system has been implemented to track, manage, and analyze all uploaded files on the server. The system provides:

• Complete file inventory with metadata tracking
• Usage tracking - know where each file is used across the system
• Unused files detection - identify orphaned files
• Duplicate detection - find duplicate files based on MD5 hash
• Safe deletion - warnings when deleting files that are in use

Features

1. All Files Management

• View all uploaded files with metadata (filename, size, type, upload date, uploader)
• Search and filter by filename and MIME type
• Copy file URLs to clipboard
• Delete files with usage warnings

2. Unused Files Detection

• Automatically detects files that aren't referenced by any entity
• Helps clean up disk space by identifying orphaned files
• Safe to delete without breaking functionality

3. Duplicate Files Detection

• Identifies files with identical content using MD5 hashing
• Groups duplicates together for easy management
• Helps optimize storage by removing redundant files

4. File Usage Tracking

The system automatically tracks file usage across:

• Articles - image_url, attachments
• Players - image_url
• Sponsors - logo_url
• Events - image_url, file_url
• Contacts - image_url
• Settings - club_logo_url, default_og_image_url
• Teams - logo_url

5. Safe Deletion

• Before deleting, the system shows where the file is used
• Prevents accidental deletion of files in use
• Force delete option available with warnings

Database Schema

uploaded_files Table

- id: Primary key
- filename: Original filename
- file_path: Physical path on server
- file_url: Public URL
- file_size: Size in bytes
- mime_type: MIME type (image/jpeg, etc.)
- uploaded_by_id: User who uploaded (nullable)
- created_at, updated_at, deleted_at: Timestamps

file_usages Table

- id: Primary key
- file_id: Reference to uploaded_files
- entity_type: Type of entity (article, player, sponsor, etc.)
- entity_id: ID of the entity
- field_name: Field where file is used (image_url, logo_url, etc.)
- created_at, updated_at: Timestamps

API Endpoints

Admin Routes (require admin role)

GET /api/v1/admin/files

List all files with usage information

• Query params: search, mime_type, sort_by, sort_order

GET /api/v1/admin/files/unused

Get files with no usage records

GET /api/v1/admin/files/duplicates

Get duplicate files grouped by MD5 hash

GET /api/v1/admin/files/:id/usages

Get detailed usage information for a specific file

DELETE /api/v1/admin/files/:id

Delete a file

• Query param: force=true to force delete even if in use

POST /api/v1/admin/files/scan

Scan uploads directory and sync with database

• Finds new files not in database
• Identifies orphaned database records

Frontend

Admin Page: /admin/soubory

The files management interface has three tabs:

1. Všechny soubory (All Files)

   • Searchable and filterable file list
   • Shows usage count for each file
   • Quick actions: copy URL, delete

2. Nepoužívané (Unused)

   • Lists files with zero usage
   • Safe to delete without breaking functionality
   • Helps with storage cleanup

3. Duplicity (Duplicates)

   • Groups files by content hash
   • Shows all duplicates in a group
   • Helps identify redundant files

Menu Location

Admin sidebar → Settings section → Soubory

Automatic File Tracking

The system automatically tracks files when:

1. File Upload (POST /api/v1/upload)

   • Creates database record for uploaded file
   • Records uploader and metadata

2. Entity Creation/Update

   • Articles: Tracks when saved with image_url or attachments
   • Players: Tracks when saved with image_url
   • Sponsors: Tracks when saved with logo_url
   • Events: Tracks when saved with image_url or file_url
   • Contacts: Tracks when saved with image_url
   • Settings: Tracks when saved with logo or OG image

File Tracker Service

Located in internal/services/file_tracker.go

Key Functions

• TrackFileUpload() - Record new file upload
• TrackFileUsage() - Record file usage by entity
• RemoveFileUsage() - Remove usage record
• UpdateFileUsages() - Update all usages for an entity
• TrackArticleFiles() - Track files in articles
• TrackPlayerFiles() - Track files in players
• TrackSponsorFiles() - Track files in sponsors
• TrackEventFiles() - Track files in events
• TrackContactFiles() - Track files in contacts
• TrackSettingsFiles() - Track files in settings
• TrackTeamFiles() - Track files in teams

Migration

Run database migration to create the tables:

# The migration will run automatically if RUN_MIGRATIONS=true in .env
# Or manually run:
migrate -path database/migrations -database "your-db-url" up

Migration files:

• 000010_create_files_table.up.sql - Creates tables
• 000010_create_files_table.down.sql - Drops tables

Usage Examples

Scan and sync files

POST /api/v1/admin/files/scan

Find unused files

GET /api/v1/admin/files/unused

Find duplicates

GET /api/v1/admin/files/duplicates

Check where a file is used

GET /api/v1/admin/files/123/usages

Delete a file (with warning if in use)

DELETE /api/v1/admin/files/123

Force delete a file even if in use

DELETE /api/v1/admin/files/123?force=true

Storage Optimization

1. Scan for unused files

   • Navigate to Files admin page → Unused tab
   • Review list of unused files
   • Delete unused files safely

2. Remove duplicates

   • Navigate to Files admin page → Duplicates tab
   • Review duplicate groups
   • Keep one copy, delete others
   • Update references if needed

3. Regular maintenance

   • Run periodic scans to sync database with filesystem
   • Monitor file usage to identify patterns
   • Clean up old unused files

Security

• All file management endpoints require admin authentication
• File deletion shows usage warnings
• Force delete requires explicit confirmation
• Upload tracking includes user attribution
• Deleted files are soft-deleted in database (deleted_at timestamp)

Technical Implementation

Backend Components

1. Models (internal/models/uploaded_file.go)

   • UploadedFile model
   • FileUsage model

2. Controller (internal/controllers/files_controller.go)

   • File listing and filtering
   • Usage tracking
   • Duplicate detection
   • File deletion

3. Service (internal/services/file_tracker.go)

   • Automatic tracking utilities
   • Entity-specific tracking methods

4. Routes (internal/routes/routes.go)

   • Admin routes for file management

Frontend Components

1. Service (frontend/src/services/files.ts)

   • API client functions
   • Type definitions

2. Admin Page (frontend/src/pages/admin/FilesAdminPage.tsx)

   • Files listing with tabs
   • Search and filter UI
   • Delete modals with warnings
   • Usage display

3. Navigation (frontend/src/components/admin/AdminSidebar.tsx)

   • Menu item in Settings section

Future Enhancements

Potential improvements:

• Bulk operations (delete multiple files)
• File replacement (update all references)
• Storage statistics and analytics
• Automatic cleanup of old unused files
• Image optimization on upload
• CDN integration support
• File versioning