Rewrite TODO as project handover

TODO.md

@@ -1,13 +1,188 @@
-- [x] 1. 프로젝트 폴더 구조 생성
+# AI Media Hub Handover
-- [x] 2. Dockerfile 및 Unraid XML 템플릿 작성
+
-- [x] 3. Gitea Actions CI/CD 파일 작성 (.gitea/workflows/build-push.yaml)
+## Project Summary
-- [x] 4. SQLite DB 모델 및 초기화 로직 (backend/models)
+- Project: `ai-media-hub`
-- [x] 5. Python yt-dlp 워커 스크립트 작성 (worker/downloader.py)
+- Goal: AI-assisted media discovery + ingest dashboard for Unraid
-- [x] 6. Go 백엔드 라우팅 및 파일 업로드(Zone B) 구현
+- Backend: Go
-- [x] 7. Go - Python 연동 로직 (Zone C 다운로드 실행)
+- Worker: Python + `yt-dlp` + `ffmpeg`
-- [x] 8. Google CSE 및 Gemini 2.5 Flash 연동 로직 (Zone A)
+- Frontend: HTML + Vanilla JS + Tailwind CDN
-- [x] 9. WebSocket 서버 및 진행률 방송 로직
+- Database: SQLite
-- [x] 10. 프론트엔드 메인 UI 구성 (Tailwind 3-Zone Layout)
+- Current search backend: `SearXNG`
-- [x] 11. 프론트엔드 JS 통신 로직 및 상태 바 렌더링 연동
+- Current vision/ranking backend: `Gemini 2.5 Flash`
-- [x] 12. 전체 기능 통합 테스트
+- Deployment target: single Docker container on Unraid
-- [x] 13. Git Init 및 자동 Push (Gitea)
+- Git remote: `https://git.savethenurse.com/savethenurse/ai-media-hub.git`
+
+## Current Architecture
+- `backend/main.go`
+  App bootstrap, env loading, static frontend serving, route registration
+- `backend/handlers/api.go`
+  Upload/download/search APIs, WebSocket progress broadcast, debug event broadcast
+- `backend/services/cse.go`
+  Actual search backend service
+  Despite filename, this is no longer Google CSE logic
+  It now wraps SearXNG search, source filtering, result enrichment, preview asset parsing
+- `backend/services/gemini.go`
+  Query translation, deterministic query expansion helper, Gemini vision scoring
+  Also extracts first video frame with `ffmpeg` when no thumbnail exists
+- `backend/models/db.go`
+  SQLite init + download history
+- `worker/downloader.py`
+  `yt-dlp` probe/download + ffmpeg clip extraction
+- `frontend/index.html`
+  Main dashboard UI, preview modal, debug log panel
+- `frontend/app.js`
+  API calls, WebSocket status bar, hover preview playback, debug logger panel, platform toggles
+- `frontend/style.css`
+  Custom styles, clamp helpers, slider thumb styles, debug panel scrollbar styles
+- `unraid-template.xml`
+  Unraid template for current `git.savethenurse.com` image source
+
+## Search Flow: Current Implementation
+1. User enters a query in Zone A.
+2. Frontend sends `/api/search` with:
+   - `query`
+   - selected `platforms`
+3. Backend translates the query to English in `GeminiService.TranslateQuery`.
+   Fallback order:
+   - Gemini translation
+   - Google Translate HTTP fallback
+   - small Korean media-term dictionary replacement
+4. Backend builds deterministic English search variants in `GeminiService.ExpandQuery`.
+5. Backend calls `SearchService.SearchMedia(...)`.
+6. Search service queries SearXNG for:
+   - `Envato`
+   - `Artgrid`
+   - `Google Video`
+7. Search service filters source URLs aggressively:
+   - Google Video: YouTube-only
+   - Envato: `elements.envato.com` item URLs only
+   - Artgrid: `artgrid.io/clip/...` only
+8. Search service enriches results:
+   - Envato: parses item page HTML for `og:image` and preview video URL
+   - Artgrid: attempts clip API + HTML parsing for thumbnails and preview sources
+9. Backend ranks all results locally.
+10. Backend evaluates all ranked results with Gemini vision in batches.
+11. Backend merges Gemini recommendations + fallback ranked items and returns JSON to frontend.
+12. Frontend renders cards and hover previews.
+
+## Direct Downloader Flow: Current Implementation
+1. User enters URL in Zone C.
+2. Frontend checks duplicate history via `/api/history/check`.
+3. Frontend loads preview metadata via `/api/download/preview`.
+4. Preview modal opens with:
+   - media preview
+   - duration
+   - crop dual-thumb slider
+   - quality select
+5. User confirms download.
+6. Backend launches Python worker.
+7. Worker downloads source with `yt-dlp`, clips with `ffmpeg`, emits JSON progress lines.
+8. Backend rebroadcasts progress over WebSocket.
+
+## Current Features Implemented
+- [x] Project folder structure
+- [x] Dockerfile
+- [x] Gitea workflow
+- [x] Unraid template
+- [x] SQLite download history
+- [x] File upload
+- [x] yt-dlp direct downloader
+- [x] Preview modal for direct download
+- [x] Crop selection slider
+- [x] Quality selection
+- [x] WebSocket realtime progress
+- [x] Search source toggles
+- [x] Search card hover preview support
+- [x] Debug log panel in frontend
+- [x] `.log` download from debug panel
+
+## Important Current Constraints / Known Problems
+- Search backend has been rewritten multiple times and is still the main unstable area.
+- Envato previews are parsed mainly from page HTML metadata / structured data.
+- Artgrid previews are partially inferred from:
+  - clip page HTML
+  - clip API attempts
+  - HLS preview handling in frontend
+- Search relevance is still not considered stable enough.
+- Gemini batch evaluation exists, but search quality can still degrade if upstream SearXNG results are noisy.
+- Frontend JavaScript was not linted with Node tooling in this environment because `node` is not installed here.
+
+## Frontend Debug Logger
+- UI button: bottom-right `Logs`
+- Files:
+  - `frontend/index.html`
+  - `frontend/app.js`
+  - `frontend/style.css`
+- Logs currently capture:
+  - API request / response
+  - WebSocket progress messages
+  - ignored WS debug messages
+  - status updates
+  - platform toggle state
+  - preview source attach / detach
+  - hover start / hover end
+  - modal preview open / close
+  - browser errors
+  - promise rejections
+  - backend debug broadcasts
+
+## Current Environment Variables
+- `APP_ROOT`
+- `APP_ADDR`
+- `SQLITE_PATH`
+- `DOWNLOADS_DIR`
+- `FRONTEND_DIR`
+- `WORKER_SCRIPT`
+- `SEARXNG_BASE_URL`
+- `SEARXNG_GOOGLE_VIDEO_ENGINE`
+- `SEARXNG_WEB_ENGINE`
+- `GEMINI_API_KEY`
+
+## Unraid Template Notes
+- Current image repository in template:
+  `git.savethenurse.com/savethenurse/ai-media-hub:latest`
+- Current registry in template:
+  `https://git.savethenurse.com`
+
+## Docker / Build Notes
+- Dockerfile uses:
+  - Go build stage
+  - static ffmpeg image stage
+  - Python runtime stage
+- Heavy apt ffmpeg install path was removed earlier to reduce build time.
+
+## Git / Push Workflow Used So Far
+- Branch: `main`
+- Remote: `origin`
+- All requested changes were committed and pushed incrementally to:
+  `https://git.savethenurse.com/savethenurse/ai-media-hub.git`
+
+## Recent Relevant Commits
+- `8ed1e84` Add in-app debug log panel
+- `823bf12` Reflect selected platforms in search status
+- `cceb040` Update platform status and HLS previews
+- `ad8afd5` Tighten source filters and add platform toggles
+- `27000db` Hide overlays during hover preview
+- `b78865d` Rewrite search flow and enrich preview assets
+- `de24886` Filter non-English expansions and prefer stock sources
+- `0bd458d` Boost translated search fallback and source priority
+
+## Next Priority Areas
+- [ ] Search backend quality stabilization
+  The search service is the main unresolved area.
+- [ ] Envato / Artgrid preview extraction hardening
+- [ ] Search result relevance validation against real user queries
+- [ ] Better matching between rendered description and actual linked asset
+- [ ] Add proper frontend build/lint step if Node becomes available
+
+## Verified Locally In This Environment
+- [x] `go build ./backend`
+- [x] Python syntax check for worker at various stages
+- [x] basic server boot / `/healthz` at various stages
+- [ ] full browser-level validation was not fully reproducible in this environment
+
+## Short Handover Summary
+- The codebase exists and runs.
+- Upload/download features mostly exist.
+- Search is implemented but is still the most fragile subsystem.
+- A visible debug logging panel now exists in the web UI and should be used first when continuing work.