GHStaK
862 lines
54 KiB
Markdown
862 lines
54 KiB
Markdown
# AI Media Hub Handover
|
|
|
|
## Working Rule
|
|
- This file is both backlog and handover log.
|
|
- Future plans written in this file should be written in Korean by default.
|
|
- When a meaningful coding/documentation task is completed, the workflow should aim to finish with a push to the git remote when the remote is available.
|
|
- Every task handled in this repository should add or update a corresponding work note in `TODO.md` before the task is considered complete.
|
|
- Git push authentication for this repo currently relies on the local credential file at `.local/git-credentials.psd1`; if push auth fails, retry using that file before treating the change as local-only.
|
|
- Every meaningful change should record:
|
|
- what changed
|
|
- why it changed
|
|
- how it was verified
|
|
- what is still risky or incomplete
|
|
- If a push fails or a change remains local-only, that must be written here explicitly.
|
|
|
|
## Current State At A Glance
|
|
- Project: `ai-media-hub`
|
|
- Goal: AI-assisted media discovery + ingest dashboard for Unraid
|
|
- Backend: Go
|
|
- Worker: Python + `yt-dlp` + `ffmpeg`
|
|
- Frontend: HTML + Vanilla JS + Tailwind CDN
|
|
- Database: SQLite
|
|
- Search backend: `SearXNG`
|
|
- AI translation / visual ranking: `Gemini 2.5 Flash`
|
|
- Deployment target: single Docker container on Unraid
|
|
- Git remote: `https://git.savethenurse.com/savethenurse/ai-media-hub.git`
|
|
|
|
## Current Status Summary
|
|
- Upload / direct download flow is implemented and broadly usable.
|
|
- Search is implemented end-to-end and now refactored into source-specific collectors.
|
|
- Search remains the main unstable subsystem.
|
|
- Envato metadata and preview extraction are much stronger than before, including additional hydration-data preview fallback.
|
|
- Artgrid metadata fidelity is improved, but stable public hover-video preview extraction is still not solved.
|
|
- Frontend now logs more useful API and debug information than earlier versions.
|
|
- A local self-test workflow now exists and should be run before container builds or pushes.
|
|
- A fresh-machine bootstrap was revalidated in a user-local toolchain setup on `2026-03-17`; `go test ./...` and `bash scripts/selftest.sh` now pass in that setup.
|
|
- Result modal sizing is now being constrained to the viewport, and modal-only source-summary translation is now part of the active implementation path.
|
|
- Card summaries now also translate lazily to Korean, and Gemini negative-assessment handling now drives stronger follow-up search behavior than before.
|
|
- Search preview delivery is now moving away from persistent on-disk preview caching toward live proxy / live transcode behavior, with Google Video preview reuse added to result cards and modal playback.
|
|
- The latest search-breadth / modal-fitting experiment from `5ca7aef` has been rolled back after live regression was confirmed.
|
|
|
|
## Current Architecture
|
|
- `backend/main.go`
|
|
- app bootstrap
|
|
- env loading
|
|
- static frontend serving
|
|
- route registration
|
|
- `backend/handlers/api.go`
|
|
- upload / download / search APIs
|
|
- WebSocket progress broadcast
|
|
- debug event broadcast
|
|
- search request orchestration only, with ranking/Gemini logic mostly moved out
|
|
- `backend/services/cse.go`
|
|
- SearXNG querying
|
|
- shared search helpers
|
|
- source-specific enrich helpers
|
|
- URL filtering / parsing utilities
|
|
- `backend/services/search_collectors.go`
|
|
- source-specific collectors:
|
|
- `envatoCollector`
|
|
- `artgridCollector`
|
|
- `googleVideoCollector`
|
|
- `backend/services/ranker.go`
|
|
- ranking
|
|
- Gemini candidate cap logic
|
|
- Gemini batch evaluation wrapper
|
|
- recommendation merge logic
|
|
- `backend/services/gemini.go`
|
|
- query translation
|
|
- deterministic query expansion
|
|
- Gemini vision scoring
|
|
- video frame extraction via `ffmpeg` when needed
|
|
- `backend/models/db.go`
|
|
- SQLite init
|
|
- download history
|
|
- `worker/downloader.py`
|
|
- `yt-dlp` probe / download
|
|
- `ffmpeg` clip extraction
|
|
- `frontend/index.html`
|
|
- main dashboard UI
|
|
- result viewer modal
|
|
- preview modal
|
|
- debug log panel
|
|
- `frontend/app.js`
|
|
- API calls
|
|
- WebSocket status bar
|
|
- result viewer modal
|
|
- hover preview playback
|
|
- direct download handoff for Google Video results
|
|
- debug logger panel
|
|
- platform toggles
|
|
- `frontend/style.css`
|
|
- custom styles
|
|
- clamp helpers
|
|
- slider thumb styles
|
|
- debug panel scrollbar styles
|
|
- `scripts/selftest.sh`
|
|
- local smoke test flow
|
|
- `scripts/mock_searxng.py`
|
|
- local mock SearXNG used by self-test
|
|
- `unraid-template.xml`
|
|
- Unraid template for current image source
|
|
|
|
## Search Flow: Current Implementation
|
|
1. User enters a query in Zone A.
|
|
2. Frontend sends `/api/search` with:
|
|
- `query`
|
|
- selected `platforms`
|
|
3. Backend translates the query in `GeminiService.TranslateQuery`.
|
|
- Gemini translation if available
|
|
- Google Translate HTTP fallback
|
|
- Korean media-term dictionary fallback
|
|
- explicit normalization for known compound phrases such as `사이버 펑크` -> `cyberpunk`
|
|
4. Backend builds deterministic English search variants in `GeminiService.ExpandQuery`.
|
|
5. `SearchService.SearchMedia(...)` orchestrates source-specific collectors.
|
|
6. Collectors query SearXNG separately for:
|
|
- Envato
|
|
- Artgrid
|
|
- Google Video
|
|
7. Each collector applies source-specific acceptance logic.
|
|
- Google Video: YouTube-only plus noise filtering
|
|
- Envato: `elements.envato.com` item URLs only
|
|
- Artgrid: accepts both:
|
|
- `artgrid.io/clip/...`
|
|
- `artlist.io/stock-footage/clip/...`
|
|
8. Artgrid canonical links are normalized to:
|
|
- `https://artgrid.io/clip/<id>/<slug>`
|
|
9. Results are enriched source-by-source.
|
|
- Envato:
|
|
- `VideoObject` JSON-LD preferred
|
|
- page meta preferred over search-engine proxy thumbnail
|
|
- preview mp4 extraction via JSON-LD / HTML parsing
|
|
- Python HTML fetch fallback used when Go HTTP fetch gets Cloudflare challenge pages
|
|
- Artgrid:
|
|
- page title / description / thumbnail cleaning
|
|
- homepage / challenge HTML is now rejected so generic site metadata does not overwrite clip metadata
|
|
- preview video extraction still not stable
|
|
10. Ranked results are passed through the shared ranker.
|
|
11. All ranked candidates are evaluated with Gemini Vision in batches.
|
|
12. Merge order now prefers:
|
|
- Gemini recommended items
|
|
- Gemini-reviewed non-recommended items
|
|
- keyword fallback items only if Gemini output is incomplete
|
|
13. Frontend renders cards, result viewer modal, and hover previews.
|
|
|
|
## Direct Downloader Flow: Current Implementation
|
|
1. User enters URL in Zone C.
|
|
2. Frontend checks duplicate history via `/api/history/check`.
|
|
3. Frontend loads preview metadata via `/api/download/preview`.
|
|
4. Preview modal opens with:
|
|
- media preview
|
|
- duration
|
|
- crop dual-thumb slider
|
|
- quality select
|
|
5. User confirms download.
|
|
6. Backend launches Python worker.
|
|
7. Worker downloads source with `yt-dlp`, clips with `ffmpeg`, emits JSON progress lines.
|
|
8. Backend rebroadcasts progress over WebSocket.
|
|
|
|
## Major Work Completed So Far
|
|
- Added local self-test workflow:
|
|
- `scripts/selftest.sh`
|
|
- `scripts/mock_searxng.py`
|
|
- Fixed translation fallback when Gemini key is missing.
|
|
- Added tests for translation fallback logic.
|
|
- Added HLS frontend wiring:
|
|
- `hls.js` script
|
|
- native HLS fallback
|
|
- Reduced search timeout risk by:
|
|
- limiting collector result caps
|
|
- limiting enrichment scope
|
|
- limiting Gemini Vision evaluation scope
|
|
- replacing oversized raw debug result payloads with summaries
|
|
- Improved Google Video filtering:
|
|
- rejects more music / trailer / BGM style noise
|
|
- Improved Envato fidelity:
|
|
- real title / description / thumbnail / preview from source page
|
|
- Improved Artgrid fidelity:
|
|
- accepts canonical Artlist URLs
|
|
- normalizes Artgrid clip URLs
|
|
- cleans title / description better
|
|
- Refactored search into source-specific collectors.
|
|
- Moved ranking and Gemini batch handling into `backend/services/ranker.go`.
|
|
- Fixed server-side 500 caused by Gemini candidate cap exceeding available ranked candidates.
|
|
- Improved frontend logging:
|
|
- raw non-JSON error body logging
|
|
- more compact debug payload rendering
|
|
- Changed hover preview playback to lazy attach on hover:
|
|
- attach source on `mouseenter`
|
|
- wait for readiness before `play()`
|
|
- detach source on `mouseleave`
|
|
- Added in-app result viewer modal for search results:
|
|
- results now open in a modal instead of directly opening a new tab
|
|
- modal now prefers internal preview media over embedded third-party iframes to avoid embed blocking
|
|
- external open button remains available
|
|
- Google Video results can now jump directly into the existing direct-download preview / crop flow from the result viewer
|
|
- Gemini reason generation is now intended to be Korean-first for readability
|
|
- Gemini Vision evaluation now covers all ranked results instead of only a top subset
|
|
- Search results now prioritize AI note text visually ahead of source summary
|
|
- Search query order and final top results now include light randomness so repeated searches are less static
|
|
|
|
## Current Features Implemented
|
|
- [x] Project folder structure
|
|
- [x] Dockerfile
|
|
- [x] Gitea workflow
|
|
- [x] Unraid template
|
|
- [x] SQLite download history
|
|
- [x] File upload
|
|
- [x] yt-dlp direct downloader
|
|
- [x] Preview modal for direct download
|
|
- [x] Crop selection slider
|
|
- [x] Quality selection
|
|
- [x] WebSocket realtime progress
|
|
- [x] Search source toggles
|
|
- [x] Search card hover preview support
|
|
- [x] Result viewer modal for search results
|
|
- [x] Google Video direct-download handoff from search results
|
|
- [x] Debug log panel in frontend
|
|
- [x] `.log` download from debug panel
|
|
- [x] Local self-test workflow
|
|
- [x] Source-specific search collectors
|
|
- [x] Shared ranker service layer
|
|
|
|
## Important Current Constraints / Known Problems
|
|
- Search backend quality is still the most fragile subsystem.
|
|
- Search relevance is still heuristic-heavy and not yet benchmarked against a durable real-query set.
|
|
- Embedded third-party result viewing is no longer relied on because many providers block iframe embedding with `X-Frame-Options` / CSP.
|
|
- Artgrid hover-video preview is still partial / unresolved:
|
|
- provided Artgrid HTML snapshots and downloaded asset bundles did not expose a stable public preview mp4/m3u8 URL
|
|
- public HTML often only exposes title / description / thumbnail / canonical URL
|
|
- Artgrid can still be sensitive to how SearXNG indexes canonical domains.
|
|
- Full browser-level validation is still not covered by local self-test.
|
|
- Frontend JavaScript still has no Node-based lint/build step in this environment.
|
|
- Search cards now separate source snippet from AI reason, but metadata fidelity still depends on source enrichment quality.
|
|
- Gemini notes are now intended to be Korean, but final output quality still depends on Gemini response consistency.
|
|
- Source Summary translation now depends on Google Translate HTTP availability; frontend silently falls back to original summary text if translation fails.
|
|
- The result modal should now stay within viewport height, but this still needs real browser confirmation on multiple short-height displays because CSS-only constraints were the source of the latest user-visible regression.
|
|
- Artgrid preview playback now has a server-side ffmpeg transcode path for `.m3u8` style preview URLs, but this trades storage savings for runtime CPU cost.
|
|
- The reverted `5ca7aef` experiment showed that simply widening collector caps and Gemini candidate count can backfire when the added candidates are weak: final visible count fell sharply even though backend raw candidate count increased.
|
|
- The local self-test script is better than before, but it is still a smoke test, not full integration coverage.
|
|
|
|
## Current Risks Around Search Quality
|
|
- Upstream SearXNG quality still controls the candidate pool.
|
|
- Gemini Vision can only rerank the candidates it receives.
|
|
- If source enrichment fails, Gemini may still judge a weaker proxy thumbnail or fallback image.
|
|
- Compound Korean intents are better handled now, but the translation path is still heuristic and can drift on niche concepts.
|
|
- Running Gemini Vision across all ranked results increases latency and token usage compared with the earlier capped approach.
|
|
|
|
## Frontend Debug Logger
|
|
- UI button: bottom-right `Logs`
|
|
- Files:
|
|
- `frontend/index.html`
|
|
- `frontend/app.js`
|
|
- `frontend/style.css`
|
|
- Logs currently capture:
|
|
- API request / response
|
|
- WebSocket progress messages
|
|
- ignored WS debug messages
|
|
- status updates
|
|
- platform toggle state
|
|
- result viewer modal open / close
|
|
- preview source attach / detach
|
|
- hover start / hover end
|
|
- hover play errors
|
|
- modal preview open / close
|
|
- browser errors
|
|
- promise rejections
|
|
- backend debug broadcasts
|
|
|
|
## Recent Change Log
|
|
- Date: `2026-03-24`
|
|
- What changed:
|
|
- Added a working-rule note that git push authentication for this repository should be retried with the local credential file `.local/git-credentials.psd1` before leaving work in a local-only state.
|
|
- Why it changed:
|
|
- The repository already stores the active git credential source locally, so the handover rules should point to it explicitly when push authentication fails.
|
|
- How it was verified:
|
|
- local file presence check for `.local/git-credentials.psd1`
|
|
- What is still risky or incomplete:
|
|
- The credential file is a local-machine dependency, so future environment changes still need to keep the file available and valid.
|
|
|
|
- Date: `2026-03-24`
|
|
- What changed:
|
|
- Added an operating rule that every completed task in this repository should also be reflected in `TODO.md`.
|
|
- Why it changed:
|
|
- The repository workflow was tightened so the handover file stays current after every task instead of only after larger batches of work.
|
|
- How it was verified:
|
|
- `git diff -- TODO.md`
|
|
- What is still risky or incomplete:
|
|
- This rule still depends on disciplined execution in each turn, so future work should keep verifying that `TODO.md` is updated before commit/push.
|
|
- Follow-up push of commit `a471c21` failed on `2026-03-24` with `Authentication failed for 'https://git.savethenurse.com/savethenurse/ai-media-hub/'`, so the latest rule update is currently local-only until credentials are restored.
|
|
|
|
- Date: `2026-03-24`
|
|
- What changed:
|
|
- Added a frontend-only `Video / Image` media-type toggle to Zone A.
|
|
- Kept the existing backend-connected video search flow as the default mode.
|
|
- Added an image-search prototype panel with sample prompt chips and test images.
|
|
- Added mock image-result cards so the image-search layout can be reviewed before backend image search exists.
|
|
- Why it changed:
|
|
- Image search is planned next, and the user wanted the Zone A UI shape in place first so the workflow can be tested before backend integration.
|
|
- How it was verified:
|
|
- `node --check frontend/app.js`
|
|
- What is still risky or incomplete:
|
|
- This is UI-only; image mode does not call a backend API yet.
|
|
- The test images currently use placeholder assets and do not represent final data contracts or modal behavior.
|
|
|
|
- Date: `2026-03-24`
|
|
- What changed:
|
|
- Added an operating rule that future plans recorded in this repo should be written in Korean by default.
|
|
- Added an operating rule that completed meaningful work should aim to end with a push to the git remote when available.
|
|
- Why it changed:
|
|
- The active collaboration workflow for this repository was updated so planning language and completion expectations stay explicit in the handover file.
|
|
- How it was verified:
|
|
- `git diff -- TODO.md`
|
|
- What is still risky or incomplete:
|
|
- Automatic push can still fail if remote auth or network state changes, so each turn should continue recording push failures explicitly when they happen.
|
|
|
|
- Date: `2026-03-16`
|
|
- What changed:
|
|
- Stabilized the Gemini visual-review path after widened search budgets caused full-batch “no candidate thumbnails or preview frames” failures.
|
|
- Google Video enrichment now always prefers the canonical YouTube `ytimg` thumbnail instead of keeping a potentially broken search-engine thumbnail.
|
|
- Gemini visual fetch now preserves the last concrete fetch error, retries with derived YouTube thumbnails when possible, and only falls back to the generic no-visual message after all image/frame paths fail.
|
|
- User-facing fallback warning text for the all-batches-no-visual case is now softened so ranked results can still be shown without surfacing the raw internal Gemini error string in the UI.
|
|
- Why it changed:
|
|
- The latest deployed build widened search enough that top-ranked candidates sometimes carried metadata without any fetchable image bytes, causing Gemini review to fail for every batch and surfacing an alarming warning even though ranked fallback results still existed.
|
|
- How it was verified:
|
|
- `go test ./...`
|
|
- `bash scripts/selftest.sh`
|
|
- What is still risky or incomplete:
|
|
- This improves YouTube-backed candidates immediately, but Envato/Artgrid thumbnails can still be inaccessible in some provider-side cases, so Gemini can still fall back to ranked results when source media is locked down.
|
|
|
|
- Date: `2026-03-16`
|
|
- What changed:
|
|
- Hid the expanded query-variant chip list from the search UI while leaving backend/debug query visibility intact.
|
|
- Reworked the result modal action panel so its primary CTA now follows source capability: `Direct Download` for Google Video and `Open Source` for Envato/Artgrid.
|
|
- Added a secondary action button area plus internal scrolling for long Source Summary text so long descriptions no longer push action buttons off-screen.
|
|
- Switched Google Video modal rendering from iframe-style embed-first behavior to a webpage-like preview panel, while keeping Artgrid/Envato on the existing media-mode path.
|
|
- Bumped the frontend asset version again to force browsers onto the updated modal behavior.
|
|
- Why it changed:
|
|
- The remaining user issues were now primarily UX/layout issues: too little visible result choice, hidden buttons under long summaries, query-variant clutter, and modal actions that did not match what each source can actually do.
|
|
- How it was verified:
|
|
- `go test ./...`
|
|
- `bash scripts/selftest.sh`
|
|
- What is still risky or incomplete:
|
|
- Live browser validation is still needed to confirm the widened search pool feels better in real queries and that the new Google Video panel feels sufficiently “webpage-like” in practice.
|
|
- `node` is still unavailable in this environment, so no frontend static syntax/build step was added here.
|
|
|
|
- Date: `2026-03-16`
|
|
- What changed:
|
|
- Expanded search breadth moderately by increasing base query count, collector query budgets, per-source caps, enrichment scope, and final visible result target while keeping Gemini review cap at `16`.
|
|
- Reworked recommendation action metadata so Google Video now advertises `Direct Download` as the primary CTA, while Envato and Artgrid advertise `Open Source`.
|
|
- Changed default modal media priority so Artgrid now prefers preview video ahead of thumbnail when both are available, and Google Video now defaults to a webpage-like thumbnail mode instead of embed-first.
|
|
- Added visible-count style debug summary fields to support checking whether the widened search budget actually increases user-facing choice.
|
|
- Why it changed:
|
|
- The UI was much healthier, but the remaining request from the user was to widen the pool of selectable results without undoing the recent quality gains, and to align modal CTA semantics with what each source can actually do.
|
|
- How it was verified:
|
|
- `go test ./...`
|
|
- What is still risky or incomplete:
|
|
- The widened search budget may increase latency on worse SearXNG days, so the frontend/UI half of this batch still needs to land before the full user-facing behavior is validated.
|
|
|
|
- Date: `2026-03-16`
|
|
- What changed:
|
|
- Rewired the result modal to consume backend media metadata instead of hard-coded source branches.
|
|
- Google Video now uses embed-first modal rendering again, with iframe timeout fallback to thumbnail/panel mode.
|
|
- Search cards now suppress low-value favicon/logo thumbnails and show a neutral “preview unavailable” media state instead of tiny site icons or placeholder-like junk.
|
|
- Bumped frontend asset version so browsers pick up the new modal logic.
|
|
- Why it changed:
|
|
- The UI was still rendering Google Video as a static image panel, and Envato/Artgrid cards were surfacing unusable thumbnails that made the results look broken even when metadata existed.
|
|
- How it was verified:
|
|
- `go test ./...`
|
|
- `bash scripts/selftest.sh`
|
|
- What is still risky or incomplete:
|
|
- Browser-level validation was still not fully reproducible here, so iframe timeout behavior and provider-specific rendering quirks still need live confirmation in the deployed UI.
|
|
- `node` is not installed in this environment, so frontend syntax/build verification is still limited to static inspection plus app boot smoke testing.
|
|
|
|
- Date: `2026-03-16`
|
|
- What changed:
|
|
- Hardened search result enrichment and recommendation metadata for preview recovery work.
|
|
- Added provider-aware fetch strategy for source HTML/JSON requests, broader Envato preview parsing, looser Artgrid HTML acceptance, and stronger thumbnail preservation rules.
|
|
- Added low-value thumbnail detection, ranking penalties for weak visuals, capped filler backfill, and response metadata fields for modal rendering (`mediaMode`, `embedUrl`, `previewBlockedReason`).
|
|
- Expanded debug summaries with usable-thumbnail and embed counts, and added unit coverage for the new parsing/ranking helpers.
|
|
- Why it changed:
|
|
- The latest production log showed Envato enrichment frequently failing, Artgrid enrichment collapsing on `403` plus HTML mismatch, and Gemini seeing too few usable visuals to do meaningful review.
|
|
- How it was verified:
|
|
- `go test ./...`
|
|
- What is still risky or incomplete:
|
|
- Frontend modal fallback behavior is not updated yet in this batch, so the new response metadata is not fully consumed until the UI patch lands.
|
|
- Envato source fetches may still fail on some pages if the provider changes challenge behavior again.
|
|
|
|
- Date: `2026-03-16`
|
|
- What changed:
|
|
- Added in-process query translation / expansion cache inside `GeminiService` so repeated identical searches can reuse the same English query and variant list without re-calling Gemini or Google Translate.
|
|
- Added in-process response caching for repeated SearXNG requests and for source fetches used during Envato / Artgrid enrichment.
|
|
- Added in-process Gemini visual cache for fetched thumbnails and extracted preview frames so repeated candidate evaluation no longer re-downloads the same asset or reruns `ffmpeg` every time.
|
|
- Tightened backend tests to cover the new cache helpers.
|
|
- Why it changed:
|
|
- Even after reducing query fan-out, repeated search passes and Gemini reevaluation were still paying duplicate translation, network, and media-processing cost inside the same running backend.
|
|
- How it was verified:
|
|
- `go test ./...`
|
|
- `bash scripts/selftest.sh`
|
|
- What is still risky or incomplete:
|
|
- These caches are process-local and TTL-based, so they do not survive container restarts and can still grow until expiry under heavy query diversity.
|
|
- Cache invalidation is intentionally simple; if upstream preview assets change rapidly, stale data can be served for a few minutes.
|
|
|
|
- Date: `2026-03-16`
|
|
- What changed:
|
|
- Result modal layout was rebuilt to match a top `16:9` embedded viewer with bottom-left full AI note and bottom-right action panel.
|
|
- Google Video results now load YouTube embed URLs in the modal viewer and keep the white `Direct Download` action in the lower-right panel.
|
|
- When Gemini evaluation comes back mostly negative or too weak, the backend now runs one supplemental search pass with broader intent variants and reevaluates the merged pool.
|
|
- Failed Gemini batch evaluations now retry sequentially candidate-by-candidate with a short delay so more candidates can still be processed when batch/token evaluation is unstable.
|
|
- Why it changed:
|
|
- The requested modal information hierarchy was different from the previous implementation.
|
|
- The user wanted negative Gemini feedback to trigger more exploration instead of stopping at the first pool.
|
|
- Batch-level Gemini failures were causing too many results to skip evaluation entirely.
|
|
- How it was verified:
|
|
- code-path inspection against the updated modal wiring and search flow
|
|
- What is still risky or incomplete:
|
|
- Non-YouTube third-party pages can still refuse iframe embedding via CSP or `X-Frame-Options`.
|
|
- Sequential Gemini retries improve coverage but also increase latency when the model is degraded.
|
|
|
|
- Date: `2026-03-16`
|
|
- What changed:
|
|
- Bumped frontend asset version and added result-modal initialization guards to avoid click failures when browser cache serves mismatched HTML/JS.
|
|
- Why it changed:
|
|
- The result modal stopped opening after the modal markup refactor, which is consistent with stale cached frontend assets or partially initialized modal DOM.
|
|
- How it was verified:
|
|
- static code inspection of modal DOM/JS bindings
|
|
- What is still risky or incomplete:
|
|
- Browser cache behavior itself was not fully reproduced here, so a hard refresh may still be needed in an already-open client session.
|
|
|
|
- Date: `2026-03-16`
|
|
- What changed:
|
|
- Envato preview extraction now also inspects `INITIAL_HYDRATION_DATA` when direct page meta / JSON-LD preview URLs are missing.
|
|
- Search result cards and result modal now surface AI note before source summary text.
|
|
- Google Video direct download action moved into the AI note area and now seeds Zone C input before opening the shared preview/download modal.
|
|
- Result modal no longer depends on third-party iframe embedding and instead shows internal preview media plus external-open fallback.
|
|
- Search flow now shuffles collector query order and lightly randomizes the top merged results to reduce identical repeated outputs.
|
|
- Why it changed:
|
|
- Some Envato items still missed preview URLs.
|
|
- Third-party iframe embedding was failing for blocked sites and creating a poor modal experience.
|
|
- The user wanted AI note to be the primary explanatory text and Google Video download action to be more obvious.
|
|
- Repeated searches returning the same ordering made the discovery experience feel too static.
|
|
- How it was verified:
|
|
- `go test ./...`
|
|
- `python3 -m py_compile worker/downloader.py scripts/mock_searxng.py`
|
|
- `bash scripts/selftest.sh`
|
|
- added unit coverage for Envato hydration preview extraction
|
|
- What is still risky or incomplete:
|
|
- Envato hydration structure could change again, so this fallback is still heuristic.
|
|
- Full browser-level validation of the revised result modal and button placement was not fully reproducible in this environment.
|
|
- Search randomness currently changes ordering and query traversal, but does not guarantee materially different source pools if upstream SearXNG returns a narrow candidate set.
|
|
|
|
## Current Environment Variables
|
|
- `APP_ROOT`
|
|
- `APP_ADDR`
|
|
- `SQLITE_PATH`
|
|
- `DOWNLOADS_DIR`
|
|
- `FRONTEND_DIR`
|
|
- `WORKER_SCRIPT`
|
|
- `SEARXNG_BASE_URL`
|
|
- `SEARXNG_GOOGLE_VIDEO_ENGINE`
|
|
- `SEARXNG_WEB_ENGINE`
|
|
- `GEMINI_API_KEY`
|
|
|
|
## Local Development Environment Notes
|
|
- This machine started without `go`, `pip`, `ffmpeg`, `yt-dlp`, or `node`.
|
|
- To unblock local development without root:
|
|
- installed Go `1.24.0` under `~/.local/tooling/go`
|
|
- installed `pip` in user site with `get-pip.py --user --break-system-packages`
|
|
- installed `imageio-ffmpeg` in user site and linked its bundled `ffmpeg` into `~/.local/bin/ffmpeg`
|
|
- installed `yt-dlp` standalone binary into `~/.local/bin/yt-dlp`
|
|
- linked `go` and `gofmt` into `~/.local/bin`
|
|
- appended `~/.local/bin` and `~/.local/tooling/go/bin` to `PATH` in `~/.bashrc`
|
|
- `node` is still not installed on this machine.
|
|
- This is acceptable for the current repo because there is still no Node-based frontend build or lint workflow in-tree.
|
|
- If future frontend work adds a Node toolchain, document the exact version and setup steps here before pushing.
|
|
|
|
## Local Self-Test Workflow
|
|
- Primary command:
|
|
- `bash scripts/selftest.sh`
|
|
- What it currently verifies:
|
|
- Go formatting for touched backend files
|
|
- Python syntax for worker + mock SearXNG
|
|
- `go test ./...`
|
|
- backend binary build
|
|
- local app boot with temp SQLite/download dirs
|
|
- `/healthz`
|
|
- `/api/search` using local mock SearXNG
|
|
- `/api/upload`
|
|
- Notes:
|
|
- search step now retries to reduce startup timing flakiness
|
|
- this is a smoke test, not a browser-level verification suite
|
|
|
|
## Verified Locally In This Environment
|
|
- [x] user-local Go toolchain available on `PATH`
|
|
- [x] user-local `ffmpeg` available on `PATH`
|
|
- [x] user-local `yt-dlp` available on `PATH`
|
|
- [x] `go build -o /tmp/ai-media-hub ./backend`
|
|
- [x] `go test ./...`
|
|
- [x] Python syntax check for worker + self-test helper
|
|
- [x] local app boot / `/healthz` through `scripts/selftest.sh`
|
|
- [x] local `/api/search` against mock SearXNG through `scripts/selftest.sh`
|
|
- [x] local `/api/upload` through `scripts/selftest.sh`
|
|
- [ ] full browser-level validation was not fully reproducible in this environment
|
|
- [ ] Node-based frontend lint/build step is still unavailable in this environment
|
|
|
|
## Recent Change Log
|
|
- Date: `2026-03-16`
|
|
- What changed:
|
|
- Added `/api/preview/stream` so remote preview assets are fetched through the backend instead of relying on the browser to load Envato / Artgrid media directly.
|
|
- MP4 previews are cached on disk under the downloads area, and HLS playlists are rewritten so segment fetches also flow through the same backend proxy route.
|
|
- Why it changed:
|
|
- Direct browser loading of remote preview URLs was still unstable and often failed due to upstream restrictions or missing headers.
|
|
- How it was verified:
|
|
- code-path inspection of preview proxy and playlist rewrite flow
|
|
- What is still risky or incomplete:
|
|
- HLS caching is not yet persisted segment-by-segment; current implementation rewrites playlists and proxies segment requests live.
|
|
|
|
- Date: `2026-03-16`
|
|
- What changed:
|
|
- Relaxed final recommendation merge so Gemini-reviewed non-negative items can still appear, and only a small preview-capable ranked filler set is used when the result list is otherwise too thin.
|
|
- Why it changed:
|
|
- The stricter `recommended=true only` merge made the visible result set collapse too aggressively in real searches.
|
|
- How it was verified:
|
|
- `go test ./...`
|
|
- `bash scripts/selftest.sh`
|
|
- What is still risky or incomplete:
|
|
- Filler results can still appear with the fallback reason when Gemini-reviewed positives are scarce, though the amount is now intentionally capped.
|
|
|
|
- Date: `2026-03-16`
|
|
- What changed:
|
|
- Google Video embed URL now uses `youtube-nocookie` with explicit `origin` to reduce player load failures.
|
|
- Gemini Vision prompt now forces a `Yes` or `No` verdict per candidate and only `Yes` candidates are merged into the final result set.
|
|
- Candidate visual fetch now prefers Envato / Artgrid preview video frames before thumbnails and sends a `Referer` header for thumbnail fetches.
|
|
- Envato / Artgrid enrichment now retries preview extraction once after a short delay when the first fetch still lacks a usable preview URL.
|
|
- Why it changed:
|
|
- The user reported YouTube player error `153`, too many fallback-style results, and source pages that appear to need additional loading time before preview URLs become visible.
|
|
- How it was verified:
|
|
- log review from `ai-media-hub-2026-03-16T05-05-58-704Z.log`
|
|
- `go test ./...`
|
|
- What is still risky or incomplete:
|
|
- If a YouTube video has embedding disabled by the uploader, no embed URL variant will fully bypass that restriction.
|
|
- Delay-and-retry HTML fetching cannot execute JavaScript, so pages that only expose preview URLs after full client-side rendering may still need a real browser-based fetch path later.
|
|
|
|
- Date: `2026-03-16`
|
|
- What changed:
|
|
- Reduced the heaviest search-stage caps slightly: fewer query variants per request, smaller per-source result caps, lower enrichment scope, and a bounded Gemini candidate set.
|
|
- Why it changed:
|
|
- The widened search configuration was pushing the request past the reverse-proxy timeout and surfacing `504 Gateway Time-out`.
|
|
- How it was verified:
|
|
- `go test ./...`
|
|
- `bash scripts/selftest.sh`
|
|
- What is still risky or incomplete:
|
|
- Search coverage is still broader than the original baseline, but there is now an explicit tradeoff between result volume and request latency.
|
|
|
|
- Date: `2026-03-16`
|
|
- What changed:
|
|
- Increased collector result caps and widened source-specific search query templates for Envato, Artgrid, and Google Video.
|
|
- Strengthened Gemini query-expansion and vision prompts with a professional video-editor framing.
|
|
- Restored result modal media fallback so Google Video uses YouTube embed while Envato and Artgrid can show preview video or thumbnail instead of blocked iframe pages.
|
|
- Expanded generic preview URL parsing so HTML-embedded `.mp4` and `.m3u8` sources are accepted more broadly.
|
|
- Why it changed:
|
|
- Search result volume was too low.
|
|
- The user wanted Gemini to reason more like a professional editor.
|
|
- Envato iframe pages were being refused, Google Video modal opening was broken, and preview extraction still missed known media URLs.
|
|
- How it was verified:
|
|
- local code inspection against attached Envato / Artgrid HTML samples
|
|
- `go test ./...`
|
|
- What is still risky or incomplete:
|
|
- The attached Artgrid HTML sample is a generic homepage shell, so preview extraction still depends on what the live clip page or downstream assets expose at runtime.
|
|
- Some providers can still refuse iframe rendering even when Artgrid pages currently appear to work.
|
|
|
|
## Unraid / Docker / CI Notes
|
|
- Dockerfile uses:
|
|
- Go build stage
|
|
- static ffmpeg image stage
|
|
- Python runtime stage
|
|
- Heavy apt ffmpeg install path was removed earlier to reduce build time.
|
|
- Gitea workflow builds and pushes:
|
|
- `git.savethenurse.com/savethenurse/ai-media-hub:latest`
|
|
- `git.savethenurse.com/savethenurse/ai-media-hub:${{ github.sha }}`
|
|
|
|
## Recent Relevant Commits
|
|
- `9637b76` Improve query intent handling and preview playback
|
|
- `6d9391b` Expand Artgrid query coverage to artlist canonical URLs
|
|
- `d8cc32e` Fix Gemini candidate cap causing search 500s
|
|
- `e426261` Fix Artgrid collector matching and split ranker
|
|
- `5aebbef` Refactor search into source-specific collectors
|
|
- `ae091c5` Improve source parsing from Envato and Artgrid HTML
|
|
- `06ea4f3` Restore Envato and Artgrid fallback search breadth
|
|
- `7dfb1ad` Stabilize search pipeline and improve preview diagnostics
|
|
- `6f3149a` Add local self-test flow and fix fallback regressions
|
|
- `f968458` Rewrite TODO as project handover
|
|
|
|
## Git / Push Status
|
|
- Current branch in ongoing work: `main`
|
|
- Current state:
|
|
- latest work in this environment has been pushed successfully multiple times after the earlier remote unpacker issue
|
|
- the older push failure note is historical context only and should not be treated as the current repo state
|
|
- on `2026-03-17`, local commits `6e9d20b` and `0495f34` were created during fresh-machine environment validation and documentation cleanup
|
|
- push from this machine was initially blocked by missing HTTPS auth helper configuration, but succeeded after explicit credentials were provided by the user
|
|
- Operational note:
|
|
- because the frontend is static and aggressively cached, browser hard refreshes are often required after UI / modal / preview changes
|
|
|
|
## Highest-Value Next Steps
|
|
- [ ] Reduce `/api/search` latency further without collapsing result count
|
|
- [ ] Rebuild the reverted search-expansion work from the previous stable baseline, but only after measuring where candidate quality collapses between ranked pool and final merge
|
|
- [ ] Build a repeatable repo-local bootstrap script or documented setup command set for non-root machines so fresh PC setup does not depend on shell history
|
|
- [ ] Improve Envato / Artgrid preview acquisition reliability so Gemini Vision sees real frames more often
|
|
- [ ] Browser-verify the new result modal at multiple viewport heights and confirm translated Source Summary readability on real long descriptions
|
|
- [ ] Evaluate whether the new Gemini supplemental-query generation is reducing irrelevant results on a small fixed benchmark query set
|
|
- [ ] Measure runtime cost of live Artgrid preview transcoding and decide whether bounded in-memory throttling or concurrency caps are needed
|
|
- [ ] Revisit Google Video UX:
|
|
- current YouTube embed was abandoned due error `153`
|
|
- current in-app panel is more reliable but less rich than a true embedded watch page
|
|
- [ ] Build collector-specific integration tests with recorded SearXNG samples
|
|
- [ ] Separate source enrichment tests from live network behavior using local fixtures
|
|
- [ ] Add a browser-level preview validation path, especially for hover video and preview proxy routing
|
|
- [ ] If Artgrid hover preview is still required, obtain one real clip HAR / DevTools network export and derive a stable preview URL parser
|
|
- [ ] Build a small fixed real-query benchmark set to evaluate search quality before further tuning
|
|
- [ ] If frontend tooling becomes available, add lint/build checks
|
|
|
|
## Short Handover Summary
|
|
- The codebase runs.
|
|
- Upload / direct-download features mostly exist and are broadly usable.
|
|
- Search is functional but still the least stable subsystem by a wide margin.
|
|
- Envato source fidelity is better than before, but Cloudflare / fetch failures still affect enrichment and preview acquisition.
|
|
- Artgrid source fidelity is improved, but query coverage and preview extraction are still unreliable.
|
|
- There is now a local self-test workflow.
|
|
- Backend debug logging is now much more detailed and intended to support exported log-file analysis from the in-app `Logs` panel.
|
|
|
|
## Current Reality Check
|
|
- Search request flow is now heavily instrumented.
|
|
- The frontend `Logs` panel can capture:
|
|
- API request start / completion
|
|
- SearXNG request / response counts
|
|
- collector query expansion
|
|
- enrichment start / finish
|
|
- Gemini translation / vision preparation / batch failures
|
|
- preview proxy fetch / cache events
|
|
- The latest broad issue pattern observed in logs is:
|
|
- too many SearXNG calls for a single request can still push total latency too high
|
|
- Envato / Artgrid often fail to provide enough preview-capable candidates for Gemini Vision
|
|
- Google Video is frequently the easiest source to retrieve and therefore can dominate final results
|
|
- YouTube embed error `153` made the prior Google modal approach unreliable
|
|
|
|
## Active Problems
|
|
- `504 Gateway Time-out`
|
|
- Root cause: `/api/search` can still become too expensive when query expansion, source collectors, enrichment, and Gemini batch retries stack together.
|
|
- Current mitigation: request-level time budget and partial-result return path.
|
|
- Residual risk: fewer reviewed results can be returned when the budget is exhausted.
|
|
- Too many Google Video-only result sets
|
|
- Root cause: Envato / Artgrid queries can still produce repeated `rawCount: 0` responses from SearXNG.
|
|
- Current mitigation: looser unquoted query variants for both sources.
|
|
- Residual risk: upstream SearXNG quality still dominates discovery.
|
|
- Gemini Vision partial or weak evaluation
|
|
- Root cause: many candidates still lack usable thumbnails / preview frames, so Gemini sees fewer real visuals than the raw result count suggests.
|
|
- Current mitigation: more verbose visual-fetch logging, preview-video-first strategy for Envato / Artgrid, and partial backfill from ranked candidates.
|
|
- Residual risk: if source media cannot be fetched, Gemini quality still degrades sharply.
|
|
- Envato / Artgrid preview instability
|
|
- Root cause: source HTML can be incomplete, fetches can fail, and some previews may only appear after client-side rendering or protected media access paths.
|
|
- Current mitigation: JSON-LD/meta/hydration parsing, delayed retry, preview proxy route, MP4 cache, and HLS playlist rewriting.
|
|
- Residual risk: a real browser-rendered fetch path may still be needed later for some pages.
|
|
- Google Video popup UX
|
|
- Root cause: YouTube embed error `153`.
|
|
- Current mitigation: dedicated in-app Google panel instead of direct embed.
|
|
- Residual risk: this is reliable but not as rich as showing the live watch page.
|
|
|
|
## Current Technical Notes
|
|
- Preview proxy route:
|
|
- `/api/preview/stream`
|
|
- MP4 responses can be cached to disk
|
|
- HLS playlists are rewritten so segment fetches also flow through the backend
|
|
- Frontend cache busting is done via `/app.js?v=...`
|
|
- If behavior in the browser does not match the latest backend/frontend code, the first assumption should be stale frontend assets until proven otherwise
|
|
|
|
## Recent Change Log
|
|
- Date: `2026-03-18`
|
|
- What changed:
|
|
- Resumed and completed the interrupted search-timeout mitigation work that had been left locally after the rollback to `f131cee`.
|
|
- Split the search-service deadline into:
|
|
- collection deadline
|
|
- enrichment deadline with a reserved window
|
|
- Reduced collector fan-out on the reverted baseline:
|
|
- fewer base queries
|
|
- no per-request query shuffling
|
|
- earlier stop when a collector repeatedly returns `0` results before producing any accepted item
|
|
- Raised `Google Video` max results to `12` so visible count does not collapse as hard when Envato / Artgrid are cold.
|
|
- Added unit coverage for the search/enrichment deadline split helper.
|
|
- Why it changed:
|
|
- The user-provided log `ai-media-hub-2026-03-18T04-44-11-440Z.log` showed:
|
|
- repeated collector passes with many `rawCount: 0`
|
|
- `search_service:deadline_reached`
|
|
- `partialDueToDeadline: true`
|
|
- final warning `search returned partial results to avoid gateway timeout`
|
|
- only `Google Video` surviving into the final result set with `resultCount: 8`
|
|
- The real bottleneck in that log was collector-side time waste before enrichment/Gemini, not another Gemini output-format issue.
|
|
- How it was verified:
|
|
- PowerShell with repo-local tooling:
|
|
- `go test ./...`
|
|
- `node --check frontend/app.js`
|
|
- What is still risky or incomplete:
|
|
- This should reduce timeout pressure and improve visible count in the common “Envato/Artgrid zero streak” case, but upstream SearXNG quality can still dominate the final pool.
|
|
- A full app-boot smoke flow was not reintroduced into this reverted baseline in this turn.
|
|
|
|
- Date: `2026-03-17`
|
|
- What changed:
|
|
- Reverted commit `5ca7aef` (`Strengthen search breadth and modal fitting`) to restore the previous stable search/modal baseline.
|
|
- Revalidated the rollback state locally.
|
|
- Why it changed:
|
|
- The user reported that search results became too sparse again and that `search returned partial results to avoid gateway timeout` reappeared.
|
|
- The provided log `ai-media-hub-2026-03-17T04-19-08-889Z.log` showed:
|
|
- backend candidate pool still reaching `30`
|
|
- Gemini candidate cap widened to `24`
|
|
- `visualRejectCount: 12`
|
|
- final visible result count collapsing to `5`
|
|
- That indicates the widened experiment increased weak/low-visual candidates and pushed the request back toward deadline pressure without improving visible result count.
|
|
- How it was verified:
|
|
- `go test ./...`
|
|
- `bash scripts/selftest.sh`
|
|
- What is still risky or incomplete:
|
|
- This rollback restores the earlier baseline but does not solve the underlying request to improve visible result count.
|
|
- The next attempt should start from the reverted baseline and target the actual bottleneck between ranked pool and final merge, instead of only widening raw query breadth.
|
|
|
|
- Date: `2026-03-17`
|
|
- What changed:
|
|
- If the first search pass plus Gemini filtering still leaves too few visible results, the backend now performs an additional coverage-expansion search/evaluation pass before final fallback filling.
|
|
- Search result cards and the result modal now reuse the existing direct-download preview probe path for Google Video, so YouTube-backed results can surface actual playable preview streams instead of staying thumbnail-only.
|
|
- Artgrid-style `.m3u8` previews now have a server-side ffmpeg transcode route so the frontend can request a directly playable MP4 stream when plain HLS playback is unreliable.
|
|
- Persistent preview-file caching to disk was removed from the preview proxy path so one-off preview traffic does not keep accumulating files under the downloads cache area.
|
|
- Modal spacing, media height, and text sizing were tightened again so the popup is more likely to fit without scrolling on shorter displays.
|
|
- Why it changed:
|
|
- The latest user feedback said the search result count had become too low after stricter Gemini filtering, the popup still felt too large, Google Video already had a better preview path in direct-download mode, Artgrid previews still were not reliably playable, and the server should prefer not retaining disposable preview artifacts on disk.
|
|
- How it was verified:
|
|
- `go test ./...`
|
|
- `bash scripts/selftest.sh`
|
|
- `python3 -m py_compile worker/downloader.py scripts/mock_searxng.py`
|
|
- What is still risky or incomplete:
|
|
- The extra coverage-expansion pass can improve visible count but may increase latency when upstream SearXNG quality is poor.
|
|
- Live ffmpeg transcoding avoids preview-file accumulation but may become CPU-heavy under concurrent preview playback.
|
|
- Real browser validation is still needed for the exact viewport in the latest screenshot and for actual Artgrid preview playback behavior.
|
|
|
|
- Date: `2026-03-17`
|
|
- What changed:
|
|
- Removed the visible `AI Recommended` badge from search cards.
|
|
- Extended lazy summary translation so result cards, not just the modal, now request Korean `Source Summary` text as they come into view.
|
|
- Reworked Gemini vision parsing and recommendation metadata so candidate assessments now distinguish `positive`, `unclear`, `irrelevant`, and `inappropriate`, with short search hints for weak/negative results.
|
|
- Removed the direct `Preview evidence pending` filler path from normal merge behavior and moved fallback filling to a later, more neutral stage.
|
|
- Upgraded supplemental search behavior so follow-up queries can be generated from Gemini feedback and provider/source mix instead of relying only on the old fixed fallback list.
|
|
- Tightened modal height handling again so the overlay and internal panels can scroll without pushing the popup past the viewport.
|
|
- Why it changed:
|
|
- The user reported that the modal could still overflow the browser, that the `AI Recommended` chip was unnecessary noise, that card-level source summaries should also be translated, and that weak/negative Gemini evaluations should trigger smarter additional searching instead of surfacing low-confidence filler results.
|
|
- How it was verified:
|
|
- `go test ./...`
|
|
- `bash scripts/selftest.sh`
|
|
- `python3 -m py_compile worker/downloader.py scripts/mock_searxng.py`
|
|
- What is still risky or incomplete:
|
|
- The new supplemental-query generation depends on Gemini availability for the smartest path and falls back to deterministic query building when Gemini text generation is unavailable.
|
|
- Card-level lazy translation reduces request burst compared with translating everything during `/api/search`, but still adds client-side translation traffic during scrolling.
|
|
- Real browser validation is still needed to confirm the modal no longer exceeds the viewport on the exact user display conditions shown in the screenshot.
|
|
|
|
- Date: `2026-03-17`
|
|
- What changed:
|
|
- Added `POST /api/translate/summary` so the result modal can translate `Source Summary` text to Korean on demand with in-process caching.
|
|
- Reworked result modal sizing so the popup stays within the viewport and the summary area scrolls internally instead of stretching the whole dialog off-screen.
|
|
- Replaced the false-positive near-deadline warning heuristic with explicit deadline metadata from search collection / enrichment / Gemini stages.
|
|
- Added an Artgrid API `403` cooldown guard so repeated clip enrichments stop re-hitting the same blocked JSON endpoint for a while and fall back to HTML parsing directly.
|
|
- Added backend tests for timeout-warning gating, summary translation caching, and Artgrid `403` skip behavior.
|
|
- Why it changed:
|
|
- The provided browser log showed a successful `~55s` search still surfacing `search returned partial results to avoid gateway timeout`, and the user reported that the result popup could overflow the viewport and that untranslated / very long source summaries made the modal feel too large.
|
|
- How it was verified:
|
|
- `go test ./...`
|
|
- `bash scripts/selftest.sh`
|
|
- `python3 -m py_compile worker/downloader.py scripts/mock_searxng.py`
|
|
- What is still risky or incomplete:
|
|
- Full browser-level confirmation of the resized modal and lazy summary translation still needs live UI validation.
|
|
- The Artgrid `403` guard reduces repeated waste, but it does not solve the underlying upstream access restriction.
|
|
|
|
- Date: `2026-03-17`
|
|
- What changed:
|
|
- Re-audited the repository structure and `TODO.md` handover state on a fresh machine.
|
|
- Bootstrapped a user-local development toolchain because the machine initially lacked `go`, `pip`, `ffmpeg`, and `yt-dlp`.
|
|
- Revalidated the documented self-test flow and updated this handover document with environment notes, current verification status, and next-step priorities.
|
|
- Committed the documentation update locally as `6e9d20b` (`Document fresh-machine setup and status`) and `0495f34` (`Record local-only push failure`), then completed the delayed push after credentials were supplied.
|
|
- Why it changed:
|
|
- Ongoing work will happen from different PCs, so the project needed an explicit record of what this repo requires to become runnable when root-level package installation is unavailable.
|
|
- How it was verified:
|
|
- `go version`
|
|
- `ffmpeg -version`
|
|
- `yt-dlp --version`
|
|
- `go test ./...`
|
|
- `bash scripts/selftest.sh`
|
|
- What is still risky or incomplete:
|
|
- This machine still has no `node`, so any future frontend build/lint workflow will need an explicit setup step before it can become part of local verification.
|
|
- The environment bootstrap done here is user-local rather than repo-automated, so reproducing it on another PC is still partly manual until a bootstrap script or setup doc lands.
|
|
- HTTPS push on this machine currently depends on manually supplied credentials, so a safer long-term credential helper or SSH setup is still worth documenting later.
|
|
|
|
- Date: `2026-03-16`
|
|
- What changed:
|
|
- When Gemini batch evaluation is only partially successful, the API now explicitly backfills the final list with preview-capable ranked candidates instead of leaving the visible result set too thin.
|
|
- Result modal behavior was switched so Envato and Artgrid open their actual item webpages inside the modal iframe again, while Google Video still uses the YouTube embed path.
|
|
- Frontend asset version was bumped again to force clients off stale preview-handling code.
|
|
- Why it changed:
|
|
- The new debug log showed `recommendedCount: 2` plus one failed Gemini batch, but the final visible list still collapsed to `6` items even though the raw pool was `25`.
|
|
- The user wants in-modal login/download for Envato and Artgrid rather than preview-only media mode.
|
|
- How it was verified:
|
|
- log review from `ai-media-hub-2026-03-16T06-00-31-359Z.log`
|
|
- code inspection of post-Gemini backfill and modal rendering path
|
|
- What is still risky or incomplete:
|
|
- Artgrid candidate volume is still often low because SearXNG is returning many `0`-result queries for current Artgrid query templates.
|
|
- If a browser still holds older frontend assets, a hard refresh may be required before the latest modal/preview behavior appears.
|
|
|
|
- Date: `2026-03-16`
|
|
- What changed:
|
|
- Significantly increased backend debug logging volume and routed it into the existing WebSocket `debug` stream so the in-app `Logs` panel captures much deeper request, search, preview, Gemini, and ranking traces.
|
|
- Added request-start/request-done API middleware logs with request body and truncated response body snapshots.
|
|
- Added service-level debug hooks for SearXNG requests, collector activity, enrichment passes, Gemini translation/vision steps, ranking batch stats, and preview proxy/cache behavior.
|
|
- Why it changed:
|
|
- The user wants to export detailed browser-visible log files and use them later for root-cause analysis instead of relying on sparse summaries.
|
|
- How it was verified:
|
|
- code inspection of debug hook wiring from backend services to WebSocket `debug` broadcasts
|
|
- What is still risky or incomplete:
|
|
- Very verbose logging can make the `Logs` panel noisier and larger, especially during long searches or preview playback.
|
|
|
|
- Date: `2026-03-16`
|
|
- What changed:
|
|
- Removed per-batch Gemini fallback injection so empty Gemini sub-results no longer automatically turn into many `"Gemini Vision 응답이 부족해..."` items.
|
|
- Relaxed final merge to keep more non-negative Gemini-reviewed candidates and allow a larger capped preview-capable filler set when the visible result list is too small.
|
|
- Slightly raised Envato / Artgrid caps and enrichment scope again after the stricter merge caused visible result count to collapse too far.
|
|
- Bumped frontend asset version to `20260316e` so clients pick up the newer preview-proxy behavior.
|
|
- Why it changed:
|
|
- The user still saw too few results and too many fallback-labeled items despite the earlier filtering changes.
|
|
- How it was verified:
|
|
- `go test ./...`
|
|
- What is still risky or incomplete:
|
|
- If the browser is holding an older cached `app.js`, a hard refresh may still be needed before the proxied preview path is actually used on the client.
|
|
|
|
- Date: `2026-03-16`
|
|
- What changed:
|
|
- Added a request-level time budget so search collection, enrichment, supplemental exploration, and Gemini evaluation stop early enough to return partial results before the reverse proxy reaches `504 Gateway Time-out`.
|
|
- Reduced query variant count and per-source caps slightly again to cut the worst-case number of SearXNG calls in a single search request.
|
|
- Why it changed:
|
|
- The new debug log showed `/api/search` taking about `90s`, with the timeout happening during the Gemini stage after a very large number of search and enrichment steps.
|
|
- How it was verified:
|
|
- `go test ./...`
|
|
- `bash scripts/selftest.sh`
|
|
- What is still risky or incomplete:
|
|
- Partial-result responses are now preferred over hard 504s, so some searches may return fewer reviewed items when the time budget is exhausted.
|
|
|
|
- Date: `2026-03-16`
|
|
- What changed:
|
|
- Loosened Envato and Artgrid query templates by adding unquoted variants so source discovery is less brittle when SearXNG has weak exact-phrase coverage.
|
|
- Replaced the Google Video modal embed with a dedicated in-app panel using thumbnail + metadata + action buttons, instead of relying on unstable YouTube embed playback.
|
|
- Bumped frontend asset version to `20260316g`.
|
|
- Why it changed:
|
|
- The latest log showed Envato and Artgrid repeatedly returning `rawCount: 0`, which is consistent with over-constrained exact-phrase queries.
|
|
- Google Video still hit YouTube player error `153`, so the modal needed a non-embed fallback that remains usable.
|
|
- How it was verified:
|
|
- log review from `ai-media-hub-2026-03-16T06-23-45-335Z.log`
|
|
- `go test ./...`
|
|
- What is still risky or incomplete:
|
|
- Even with looser queries, SearXNG quality still controls whether Envato and Artgrid candidates are discoverable for a given term.
|