Compare commits

..

15 Commits

Author SHA1 Message Date
GHStaK 914f10f502 Remove GIPHY prompt bar
build-push / docker (push) Successful in 4m37s
2026-03-24 17:27:30 +09:00
GHStaK e3dbedc59f Restore video search base URL handling
build-push / docker (push) Successful in 4m17s
2026-03-24 17:12:41 +09:00
GHStaK 73d820ddaa Fix GIPHY Unraid path target
build-push / docker (push) Successful in 4m22s
2026-03-24 17:05:19 +09:00
GHStaK f5ceb872e0 Fix GIPHY UI cache mismatch
build-push / docker (push) Successful in 4m18s
2026-03-24 16:50:20 +09:00
GHStaK e79d15de2e Simplify GIPHY result presentation
build-push / docker (push) Successful in 4m32s
2026-03-24 16:36:00 +09:00
GHStaK 3c6df2e777 Tolerate Gemini image expansion drift
build-push / docker (push) Successful in 4m23s
2026-03-24 16:29:36 +09:00
GHStaK 1fb9919ec3 Simplify GIPHY Unraid download config
build-push / docker (push) Successful in 4m55s
2026-03-24 16:18:03 +09:00
GHStaK 932f08642c Record GIPHY feature push failure
build-push / docker (push) Successful in 4m44s
2026-03-24 16:03:14 +09:00
GHStaK d63c467ef9 Add GIPHY image search feature 2026-03-24 16:02:49 +09:00
GHStaK 494a54fa46 Document local git credential source
build-push / docker (push) Successful in 4m33s
2026-03-24 15:15:31 +09:00
GHStaK 7772cd8064 Record push failure for TODO workflow update 2026-03-24 15:13:10 +09:00
GHStaK a471c21681 Require TODO updates for every task 2026-03-24 15:12:40 +09:00
GHStaK 89e25c560b Add image search prototype UI
build-push / docker (push) Successful in 4m38s
2026-03-24 11:40:54 +09:00
GHStaK 279a042561 Document Korean planning and push workflow
build-push / docker (push) Successful in 4m54s
2026-03-24 10:46:36 +09:00
GHStaK d3fb5e15e9 Reduce search timeout pressure on reverted baseline
build-push / docker (push) Successful in 4m22s
2026-03-18 14:01:39 +09:00
24 changed files with 1755 additions and 878 deletions
-4
View File
@@ -6,7 +6,3 @@ worker/__pycache__/
*.pyc *.pyc
node_modules/ node_modules/
dist/ dist/
.venv/
.tools/
.local/
*.log
+187 -144
View File
@@ -2,17 +2,16 @@
## Working Rule ## Working Rule
- This file is both backlog and handover log. - This file is both backlog and handover log.
- Future plans written in this file should be written in Korean by default.
- When a meaningful coding/documentation task is completed, the workflow should aim to finish with a push to the git remote when the remote is available.
- Every task handled in this repository should add or update a corresponding work note in `TODO.md` before the task is considered complete.
- Git push authentication for this repo currently relies on the local credential file at `.local/git-credentials.psd1`; if push auth fails, retry using that file before treating the change as local-only.
- Every meaningful change should record: - Every meaningful change should record:
- what changed - what changed
- why it changed - why it changed
- how it was verified - how it was verified
- what is still risky or incomplete - what is still risky or incomplete
- If a push fails or a change remains local-only, that must be written here explicitly. - If a push fails or a change remains local-only, that must be written here explicitly.
- Active planning and handoff notes should be written in Korean.
- After each meaningful completed task:
- update this file in context
- push the latest git state when operationally possible
- Local git credentials used for push automation must stay in an ignored local-only file and must never be committed.
## Current State At A Glance ## Current State At A Glance
- Project: `ai-media-hub` - Project: `ai-media-hub`
@@ -38,9 +37,7 @@
- Result modal sizing is now being constrained to the viewport, and modal-only source-summary translation is now part of the active implementation path. - Result modal sizing is now being constrained to the viewport, and modal-only source-summary translation is now part of the active implementation path.
- Card summaries now also translate lazily to Korean, and Gemini negative-assessment handling now drives stronger follow-up search behavior than before. - Card summaries now also translate lazily to Korean, and Gemini negative-assessment handling now drives stronger follow-up search behavior than before.
- Search preview delivery is now moving away from persistent on-disk preview caching toward live proxy / live transcode behavior, with Google Video preview reuse added to result cards and modal playback. - Search preview delivery is now moving away from persistent on-disk preview caching toward live proxy / live transcode behavior, with Google Video preview reuse added to result cards and modal playback.
- Search breadth and Gemini review budget were widened again because the latest user feedback still reported a thin visible result count even after smarter filtering. - The latest search-breadth / modal-fitting experiment from `5ca7aef` has been rolled back after live regression was confirmed.
- The codebase is now back on the broader-search / modal-fitting direction associated with `5ca7aef`, with an added Gemini deadline guard to reduce reverse-proxy `504` risk.
- Windows 11 local development now has a repo-local PowerShell bootstrap / run / self-test / push workflow built around `.venv`, `.tools`, and `.local` so the machine does not need global Go / ffmpeg / yt-dlp changes for this repo.
## Current Architecture ## Current Architecture
- `backend/main.go` - `backend/main.go`
@@ -223,7 +220,6 @@
- [x] Local self-test workflow - [x] Local self-test workflow
- [x] Source-specific search collectors - [x] Source-specific search collectors
- [x] Shared ranker service layer - [x] Shared ranker service layer
- [x] Windows 11 PowerShell local bootstrap / self-test / push workflow
## Important Current Constraints / Known Problems ## Important Current Constraints / Known Problems
- Search backend quality is still the most fragile subsystem. - Search backend quality is still the most fragile subsystem.
@@ -240,9 +236,7 @@
- Source Summary translation now depends on Google Translate HTTP availability; frontend silently falls back to original summary text if translation fails. - Source Summary translation now depends on Google Translate HTTP availability; frontend silently falls back to original summary text if translation fails.
- The result modal should now stay within viewport height, but this still needs real browser confirmation on multiple short-height displays because CSS-only constraints were the source of the latest user-visible regression. - The result modal should now stay within viewport height, but this still needs real browser confirmation on multiple short-height displays because CSS-only constraints were the source of the latest user-visible regression.
- Artgrid preview playback now has a server-side ffmpeg transcode path for `.m3u8` style preview URLs, but this trades storage savings for runtime CPU cost. - Artgrid preview playback now has a server-side ffmpeg transcode path for `.m3u8` style preview URLs, but this trades storage savings for runtime CPU cost.
- The provided Artgrid HTML sample still does not expose a direct preview `m3u8` or `mp4` URL by itself, and `yt-dlp` probe on the sample clip URL returned `Unsupported URL`, so fully reliable Artgrid playback still depends on live-page/runtime preview discovery succeeding elsewhere in the pipeline. - The reverted `5ca7aef` experiment showed that simply widening collector caps and Gemini candidate count can backfire when the added candidates are weak: final visible count fell sharply even though backend raw candidate count increased.
- Docker CLI is not installed in this environment, so container-image build verification still cannot be performed locally from this machine.
- Gemini batch warnings can still mix true model/output failures with harmless candidate-skip cases unless the recovery path filters those error classes before surfacing them.
- The local self-test script is better than before, but it is still a smoke test, not full integration coverage. - The local self-test script is better than before, but it is still a smoke test, not full integration coverage.
## Current Risks Around Search Quality ## Current Risks Around Search Quality
@@ -274,6 +268,149 @@
- backend debug broadcasts - backend debug broadcasts
## Recent Change Log ## Recent Change Log
- Date: `2026-03-24`
- What changed:
- Removed the remaining inline `Powered by GIPHY` / prompt-chip bar from Zone A image mode so the image search view now shows only the shared search controls and the results area.
- Why it changed:
- The user wanted that image-mode top strip removed entirely instead of reduced or restyled.
- How it was verified:
- static review of `frontend/index.html` and `frontend/app.js`
- What is still risky or incomplete:
- None beyond the usual need for a browser hard refresh if an older cached frontend bundle is still open in a tab.
- Date: `2026-03-24`
- What changed:
- Restored the video-search request path to tolerate a scheme-less `SEARXNG_BASE_URL` such as `192.168.1.66:8087` by normalizing it to `http://...` during search-service initialization.
- Added regression coverage so the video search service keeps accepting the older style base URL configuration used in live deployment.
- Why it changed:
- Real user logs showed video search failing immediately with `first path segment in URL cannot contain colon`, which traced back to a scheme-less SearXNG base URL in the deployed environment.
- How it was verified:
- log review of `ai-media-hub-2026-03-24T08-09-23-204Z.log`
- added unit coverage for scheme-less base URL normalization
- What is still risky or incomplete:
- Go tests could not be rerun in this environment because `go` is currently unavailable here, so this fix is verified by code-path review plus the added test only.
- Date: `2026-03-24`
- What changed:
- Corrected the Unraid template GIPHY download path mapping from `/downloads/giphy` to `/app/downloads/giphy` so it matches the backend default download directory layout.
- Why it changed:
- The previous template target path dropped the `/app` prefix and did not match the applications runtime default for `GIPHY_DOWNLOAD_DIR`.
- How it was verified:
- static review of `unraid-template.xml`
- What is still risky or incomplete:
- Existing Unraid installs that already created the older path mapping may need their template field refreshed or reapplied to align with the corrected container path.
- Date: `2026-03-24`
- What changed:
- Removed the large GIPHY image-mode info box entirely and replaced it with a minimal inline prompt bar plus `Powered by GIPHY` label.
- Hardened frontend visibility toggling so stale cached HTML/JS combinations do not crash on missing elements.
- Bumped the frontend asset version again so browsers are forced onto the latest image-search UI bundle after the GIPHY panel changes.
- Why it changed:
- Real user logs showed a client-side `Cannot read properties of null (reading 'classList')` error caused by stale frontend asset mismatch, which prevented image results from rendering, and the remaining large image-mode box was still not desired in the UI.
- How it was verified:
- log review of `ai-media-hub-2026-03-24T07-48-19-085Z.log`
- `node --check frontend/app.js`
- What is still risky or incomplete:
- Users with aggressively cached browser sessions may still need one hard refresh to fully drop older HTML/JS combinations already loaded in an open tab.
- Date: `2026-03-24`
- What changed:
- Simplified the GIPHY image-search UX so it presents raw search results instead of looking like an AI-evaluated result flow.
- Updated the image-mode copy to describe direct GIPHY search results, and changed the shared preview modal labels/content for GIPHY items from AI-note style metadata to plain result/source info.
- Why it changed:
- The image-search experience should behave like a straightforward provider search result browser, not like the video-side Gemini review flow.
- How it was verified:
- `node --check frontend/app.js`
- What is still risky or incomplete:
- This is a UX clarification pass; the backend still uses Gemini only for multilingual query expansion and does not do visual evaluation on GIPHY items.
- Date: `2026-03-24`
- What changed:
- Relaxed Gemini image-query expansion parsing so loose plain-text numbered lists can still be accepted when the model prepends explanatory text instead of returning a clean JSON object.
- Removed the GIPHY image-mode search meta box from the frontend so the image UI stays visually simpler.
- Stopped surfacing the Gemini image-expansion fallback warning directly in the image-search UI when the backend can still continue with usable fallback queries.
- Why it changed:
- Real log review showed Gemini image expansion sometimes returned text like `Here is the JSON requested`, which triggered fallback even though the model output still contained useful query candidates, and the extra meta box was not adding enough value to justify the space it consumed.
- How it was verified:
- log review of `ai-media-hub-2026-03-24T07-25-42-827Z.log`
- `node --check frontend/app.js`
- What is still risky or incomplete:
- This improves tolerance for one common Gemini formatting deviation, but fully free-form model output can still fall back if it does not contain recoverable query lines.
- Go tests still could not be rerun in this environment because `go` is currently unavailable here.
- Date: `2026-03-24`
- What changed:
- Removed the redundant `GIPHY Download Dir` variable field from the Unraid template and kept the dedicated `GIPHY Downloads` path mapping as the single user-facing download-path control.
- Why it changed:
- The earlier template exposed both a path mapping and a matching container-path variable for the same GIPHY download location, which was unnecessarily confusing in Unraid.
- How it was verified:
- static review of `unraid-template.xml`
- What is still risky or incomplete:
- The application still supports `GIPHY_DOWNLOAD_DIR` as an environment variable, but the Unraid template now intentionally relies on the path mapping plus the backend default container path to reduce duplicated inputs.
- Date: `2026-03-24`
- What changed:
- Replaced the earlier frontend-only image prototype with an integrated GIPHY image/GIF search flow.
- Added backend GIPHY search aggregation with dedupe, up to `100` results, secure download handling, and Gemini-driven multilingual image query expansion into exactly `5` English queries.
- Reused the existing result modal for enlarged GIPHY preview and download actions, and added an internal-scroll image results panel with visible `Powered by GIPHY` attribution.
- Updated startup config and Unraid template fields for `GIPHY_ENABLED`, `GIPHY_API_KEY`, `GIPHY_MAX_RESULTS`, `GIPHY_RATING`, `GIPHY_LANG`, `GIPHY_DOWNLOAD_DIR`, and `GEMINI_MODEL`.
- Added backend tests covering Gemini image-expansion parsing/fallback, GIPHY aggregation/cap behavior, download validation, and handler-level API paths.
- Why it changed:
- The app needed a production-usable image/GIF provider added incrementally without breaking the existing video search experience.
- How it was verified:
- `node --check frontend/app.js`
- static code review of backend/frontend wiring and new test coverage
- What is still risky or incomplete:
- This environment currently does not expose `go` or `gofmt`, so Go formatting and `go test ./...` could not be rerun locally in this turn even though new tests were added.
- Live browser QA and real GIPHY credential validation still need to be performed in a runtime environment with working API keys.
- Push of commit `d63c467` failed on `2026-03-24` with remote error `unable to create temporary object directory` / `unpacker error`, so the latest GIPHY feature batch is currently local-only until the remote accepts a retry.
- Date: `2026-03-24`
- What changed:
- Added a working-rule note that git push authentication for this repository should be retried with the local credential file `.local/git-credentials.psd1` before leaving work in a local-only state.
- Why it changed:
- The repository already stores the active git credential source locally, so the handover rules should point to it explicitly when push authentication fails.
- How it was verified:
- local file presence check for `.local/git-credentials.psd1`
- What is still risky or incomplete:
- The credential file is a local-machine dependency, so future environment changes still need to keep the file available and valid.
- Date: `2026-03-24`
- What changed:
- Added an operating rule that every completed task in this repository should also be reflected in `TODO.md`.
- Why it changed:
- The repository workflow was tightened so the handover file stays current after every task instead of only after larger batches of work.
- How it was verified:
- `git diff -- TODO.md`
- What is still risky or incomplete:
- This rule still depends on disciplined execution in each turn, so future work should keep verifying that `TODO.md` is updated before commit/push.
- Follow-up push of commit `a471c21` failed on `2026-03-24` with `Authentication failed for 'https://git.savethenurse.com/savethenurse/ai-media-hub/'`, so the latest rule update is currently local-only until credentials are restored.
- Date: `2026-03-24`
- What changed:
- Added a frontend-only `Video / Image` media-type toggle to Zone A.
- Kept the existing backend-connected video search flow as the default mode.
- Added an image-search prototype panel with sample prompt chips and test images.
- Added mock image-result cards so the image-search layout can be reviewed before backend image search exists.
- Why it changed:
- Image search is planned next, and the user wanted the Zone A UI shape in place first so the workflow can be tested before backend integration.
- How it was verified:
- `node --check frontend/app.js`
- What is still risky or incomplete:
- This is UI-only; image mode does not call a backend API yet.
- The test images currently use placeholder assets and do not represent final data contracts or modal behavior.
- Date: `2026-03-24`
- What changed:
- Added an operating rule that future plans recorded in this repo should be written in Korean by default.
- Added an operating rule that completed meaningful work should aim to end with a push to the git remote when available.
- Why it changed:
- The active collaboration workflow for this repository was updated so planning language and completion expectations stay explicit in the handover file.
- How it was verified:
- `git diff -- TODO.md`
- What is still risky or incomplete:
- Automatic push can still fail if remote auth or network state changes, so each turn should continue recording push failures explicitly when they happen.
- Date: `2026-03-16` - Date: `2026-03-16`
- What changed: - What changed:
- Stabilized the Gemini visual-review path after widened search budgets caused full-batch “no candidate thumbnails or preview frames” failures. - Stabilized the Gemini visual-review path after widened search budgets caused full-batch “no candidate thumbnails or preview frames” failures.
@@ -420,6 +557,13 @@
- `SEARXNG_GOOGLE_VIDEO_ENGINE` - `SEARXNG_GOOGLE_VIDEO_ENGINE`
- `SEARXNG_WEB_ENGINE` - `SEARXNG_WEB_ENGINE`
- `GEMINI_API_KEY` - `GEMINI_API_KEY`
- `GEMINI_MODEL`
- `GIPHY_ENABLED`
- `GIPHY_API_KEY`
- `GIPHY_MAX_RESULTS`
- `GIPHY_RATING`
- `GIPHY_LANG`
- `GIPHY_DOWNLOAD_DIR`
## Local Development Environment Notes ## Local Development Environment Notes
- This machine started without `go`, `pip`, `ffmpeg`, `yt-dlp`, or `node`. - This machine started without `go`, `pip`, `ffmpeg`, `yt-dlp`, or `node`.
@@ -433,24 +577,6 @@
- `node` is still not installed on this machine. - `node` is still not installed on this machine.
- This is acceptable for the current repo because there is still no Node-based frontend build or lint workflow in-tree. - This is acceptable for the current repo because there is still no Node-based frontend build or lint workflow in-tree.
- If future frontend work adds a Node toolchain, document the exact version and setup steps here before pushing. - If future frontend work adds a Node toolchain, document the exact version and setup steps here before pushing.
- Windows 11 repo-local workflow:
- `scripts/setup-dev.ps1`
- creates `.venv`
- installs `worker/requirements.txt` into the repo-local venv
- downloads repo-local Go into `.tools/go`
- downloads repo-local ffmpeg into `.tools/ffmpeg`
- creates local command shims under `.tools/bin` such as `python3.cmd`
- `scripts/enter-dev-shell.ps1`
- prepends `.venv`, `.tools/bin`, local Go, and local ffmpeg to `PATH`
- pins `GOPATH`, `GOMODCACHE`, and `GOCACHE` into `.local` so Go does not need user-profile write access
- `scripts/run-dev.ps1`
- launches the backend with Windows-friendly local environment wiring
- `scripts/selftest.ps1`
- PowerShell-native replacement for the Linux shell smoke test
- verifies `gofmt`, Python syntax, `go test ./...`, frontend syntax via `node --check`, backend build, mock SearXNG boot, `/healthz`, `/api/search`, and `/api/upload`
- `scripts/push.ps1`
- reads ignored local credentials from `.local/git-credentials.psd1`
- pushes without storing credentials in tracked files
## Local Self-Test Workflow ## Local Self-Test Workflow
- Primary command: - Primary command:
@@ -582,14 +708,11 @@
## Highest-Value Next Steps ## Highest-Value Next Steps
- [ ] Reduce `/api/search` latency further without collapsing result count - [ ] Reduce `/api/search` latency further without collapsing result count
- [ ] Rebuild the reverted search-expansion work from the previous stable baseline, but only after measuring where candidate quality collapses between ranked pool and final merge - [ ] Rebuild the reverted search-expansion work from the previous stable baseline, but only after measuring where candidate quality collapses between ranked pool and final merge
- [ ] Validate the reopened `5ca7aef` search-breadth direction against real proxy timeouts and visible result count before widening it any further
- [ ] Continue hardening Gemini batch parsing so truncated JSON and ignorable low-visual candidate failures do not inflate user-facing warning counts
- [ ] Build a repeatable repo-local bootstrap script or documented setup command set for non-root machines so fresh PC setup does not depend on shell history - [ ] Build a repeatable repo-local bootstrap script or documented setup command set for non-root machines so fresh PC setup does not depend on shell history
- [ ] Improve Envato / Artgrid preview acquisition reliability so Gemini Vision sees real frames more often - [ ] Improve Envato / Artgrid preview acquisition reliability so Gemini Vision sees real frames more often
- [ ] Browser-verify the new result modal at multiple viewport heights and confirm translated Source Summary readability on real long descriptions - [ ] Browser-verify the new result modal at multiple viewport heights and confirm translated Source Summary readability on real long descriptions
- [ ] Evaluate whether the new Gemini supplemental-query generation is reducing irrelevant results on a small fixed benchmark query set - [ ] Evaluate whether the new Gemini supplemental-query generation is reducing irrelevant results on a small fixed benchmark query set
- [ ] Measure runtime cost of live Artgrid preview transcoding and decide whether bounded in-memory throttling or concurrency caps are needed - [ ] Measure runtime cost of live Artgrid preview transcoding and decide whether bounded in-memory throttling or concurrency caps are needed
- [ ] If Artgrid playback is still required for previewless clips, investigate a browser-rendered fetch path or clip-page asset bundle analysis because static HTML + `yt-dlp` probe were both insufficient on the supplied sample
- [ ] Revisit Google Video UX: - [ ] Revisit Google Video UX:
- current YouTube embed was abandoned due error `153` - current YouTube embed was abandoned due error `153`
- current in-app panel is more reliable but less rich than a true embedded watch page - current in-app panel is more reliable but less rich than a true embedded watch page
@@ -657,130 +780,50 @@
## Recent Change Log ## Recent Change Log
- Date: `2026-03-18` - Date: `2026-03-18`
- What changed: - What changed:
- Reverted the recent Gemini output-format experimentation commits so the repo returns to the last simpler known-good search/Gemini state anchored by `91ee375`. - Resumed and completed the interrupted search-timeout mitigation work that had been left locally after the rollback to `f131cee`.
- Reverted commits:
- `f5d76fc` `Replace gemini batch JSON protocol`
- `b6a217c` `Harden single-candidate gemini recovery`
- `3be7971` `Reduce gemini partial batch noise`
- `513199f` `Harden gemini vision JSON recovery`
- The effective runtime baseline after the rollback is now the `91ee375` state plus the newer Windows 11 PowerShell workflow work.
- Why it changed:
- Repeated Gemini output-format changes were still producing real user-facing failures across multiple logs, including:
- `gemini vision failed for all batches`
- `gemini vision partially failed on 4 of 6 batches`
- extremely short truncated payloads such as `"{\"recommend"` and `"{\"recommendations\":[{\"index"`
- The user explicitly requested a rollback to a commit state that worked normally instead of continuing to stack more Gemini parsing experiments.
- How it was verified:
- `pwsh -NoProfile -File scripts/selftest.ps1`
- local git history inspection confirmed the rollback point as the pre-experiment Gemini baseline `91ee375`
- What is still risky or incomplete:
- This rollback intentionally gives up the later Gemini parsing experiments, so the codebase no longer contains those attempted mitigations.
- If the original upstream Gemini truncation issue already existed before those follow-up commits, it can still reappear and would need a cleaner redesign from the reverted baseline rather than more incremental patching on top of the discarded branch.
- Date: `2026-03-17`
- What changed:
- Fixed a search-budget regression where source collection could consume the full `SearchService` deadline and leave no time for Envato / Artgrid enrichment, causing Gemini to see only missing or low-value visuals.
- Split the search-service deadline into: - Split the search-service deadline into:
- collector deadline - collection deadline
- enrichment deadline with an explicit reserved window - enrichment deadline with a reserved window
- Added unit coverage for the new deadline split behavior. - Reduced collector fan-out on the reverted baseline:
- Stopped frontend preview-probe fallback from calling `/api/download/preview` for Artgrid items that do not already have a provider preview URL, so unsupported `yt-dlp` Artgrid probe errors no longer fire just from opening or hovering those results. - fewer base queries
- no per-request query shuffling
- earlier stop when a collector repeatedly returns `0` results before producing any accepted item
- Raised `Google Video` max results to `12` so visible count does not collapse as hard when Envato / Artgrid are cold.
- Added unit coverage for the search/enrichment deadline split helper.
- Why it changed: - Why it changed:
- The user-provided log `ai-media-hub-2026-03-17T07-01-21-282Z.log` showed: - The user-provided log `ai-media-hub-2026-03-18T04-44-11-440Z.log` showed:
- repeated collector passes with many `rawCount: 0`
- `search_service:deadline_reached` - `search_service:deadline_reached`
- immediate `search_service:enrich_start` -> `search_service:enrich_complete` - `partialDueToDeadline: true`
- `withPreview: 0` - final warning `search returned partial results to avoid gateway timeout`
- `withLowValueThumbnail: 12` - only `Google Video` surviving into the final result set with `resultCount: 8`
- repeated `candidate has no thumbnail or preview video` - The real bottleneck in that log was collector-side time waste before enrichment/Gemini, not another Gemini output-format issue.
- final warning `gemini vision returned no candidate evaluations`
- The same log also showed Artgrid preview probe failures from `yt-dlp` returning `Unsupported URL`, which were not helping user-facing preview behavior.
- How it was verified: - How it was verified:
- `pwsh -NoProfile -File scripts/selftest.ps1` - PowerShell with repo-local tooling:
- added Go tests for the search/enrichment deadline split helper - `go test ./...`
- `node --check frontend/app.js`
- What is still risky or incomplete: - What is still risky or incomplete:
- This preserves time for enrichment, but it does not guarantee that every live Envato / Artgrid page yields a usable preview URL. - This should reduce timeout pressure and improve visible count in the common “Envato/Artgrid zero streak” case, but upstream SearXNG quality can still dominate the final pool.
- Artgrid still depends on backend-enriched provider preview URLs for true video preview; if no provider preview is discovered, the UI will still fall back to thumbnail-only rendering. - A full app-boot smoke flow was not reintroduced into this reverted baseline in this turn.
- Date: `2026-03-17` - Date: `2026-03-17`
- What changed: - What changed:
- Added repo-local Windows 11 PowerShell workflows: - Reverted commit `5ca7aef` (`Strengthen search breadth and modal fitting`) to restore the previous stable search/modal baseline.
- `scripts/dev-tools.ps1` - Revalidated the rollback state locally.
- `scripts/setup-dev.ps1`
- `scripts/enter-dev-shell.ps1`
- `scripts/run-dev.ps1`
- `scripts/selftest.ps1`
- `scripts/push.ps1`
- Added ignored local directories for `.venv`, `.tools`, and `.local`.
- Created a local-only git credential file for automated push flow and kept it excluded from git.
- Pinned PowerShell tooling to repo-local Go caches under `.local` so `go test` no longer depends on writable user-profile Go paths.
- Why it changed: - Why it changed:
- The active development machine is now Windows 11, and the user requested that setup, verification, and push flows work through PowerShell while minimizing machine-wide side effects. - The user reported that search results became too sparse again and that `search returned partial results to avoid gateway timeout` reappeared.
- The previous local workflow was Linux-shell-oriented and did not directly cover Windows PowerShell usage. - The provided log `ai-media-hub-2026-03-17T04-19-08-889Z.log` showed:
- Initial Windows self-test attempts failed because Go wanted to write into the default user Go cache path, which was not reliably writable in this environment. - backend candidate pool still reaching `30`
- How it was verified: - Gemini candidate cap widened to `24`
- `pwsh -NoProfile -File scripts/setup-dev.ps1 -SkipPythonDeps -SkipGoDownload -SkipFFmpegDownload` - `visualRejectCount: 12`
- elevated `pwsh -NoProfile -File scripts/setup-dev.ps1` - final visible result count collapsing to `5`
- `pwsh -NoProfile -Command ". .\scripts\enter-dev-shell.ps1; python3 --version; yt-dlp --version; go version; ffmpeg -version | Select-Object -First 1"` - That indicates the widened experiment increased weak/low-visual candidates and pushed the request back toward deadline pressure without improving visible result count.
- `pwsh -NoProfile -File scripts/selftest.ps1`
- `git check-ignore -v .local\git-credentials.psd1 .venv .tools`
- What is still risky or incomplete:
- `scripts/setup-dev.ps1` still requires network access for Python package install and local Go / ffmpeg downloads on a truly fresh machine.
- The backend runtime still invokes `python3` by name, so Windows usage depends on entering the repo-local dev shell or otherwise exposing the generated shim on `PATH`.
- Push automation now uses an ignored local credential file, but credential rotation remains a manual step.
- Date: `2026-03-17`
- What changed:
- Restored hard/ignorable Gemini batch error filtering so low-value thumbnail skips and no-visual candidate skips do not count as user-facing partial batch failures when useful recommendations were still recovered.
- Added unit coverage for the Gemini batch error filtering behavior.
- Why it changed:
- The user reported the warning `gemini vision partially failed on 3 of 4 batches`.
- The provided log `ai-media-hub-2026-03-17T06-06-04-447Z.log` showed three different batch-failure classes mixed together:
- real Gemini JSON extraction failures
- low-value thumbnail skips
- no-visual candidate skips
- Only the first class should strongly influence the user-facing partial-failure warning.
- How it was verified: - How it was verified:
- `go test ./...` - `go test ./...`
- `bash scripts/selftest.sh` - `bash scripts/selftest.sh`
- What is still risky or incomplete: - What is still risky or incomplete:
- This reduces noisy partial-failure warnings, but it does not eliminate genuine Gemini JSON truncation or provider-side preview problems. - This rollback restores the earlier baseline but does not solve the underlying request to improve visible result count.
- If the model starts returning different malformed JSON patterns, the parser and warning logic may still need further hardening. - The next attempt should start from the reverted baseline and target the actual bottleneck between ranked pool and final merge, instead of only widening raw query breadth.
- Date: `2026-03-17`
- What changed:
- Reapplied the broader-search / modal-fitting codepath from `5ca7aef` as requested by the user.
- Added a targeted 504 mitigation by making Gemini batch recovery stop at the request deadline instead of continuing sequential single-candidate retries indefinitely.
- Kept an explicit Gemini vision JSON output token cap to reduce the chance of truncated model responses during larger structured batches.
- Why it changed:
- The user explicitly asked to return to the `5ca7aef` direction and then fix the live `504 Gateway Time-out`.
- The provided log `ai-media-hub-2026-03-17T04-59-24-566Z.log` showed backend search collection finishing in about `22s`, then Gemini batch failure plus sequential recovery continuing until the reverse proxy returned `504`.
- The direct cause was deadline-unaware Gemini recovery work, not initial search collection itself.
- How it was verified:
- `go test ./...`
- `bash scripts/selftest.sh`
- attempted `docker build -t ai-media-hub:test .` but `docker` is not installed in this environment
- What is still risky or incomplete:
- This reduces one concrete `504` path, but it does not guarantee that all reverse-proxy timeout cases are eliminated under worse upstream conditions.
- Container-image build verification still needs to happen on a machine that actually has Docker available.
- Date: `2026-03-17`
- What changed:
- Raised search breadth again by widening per-source collector caps, increasing the number of base queries considered, increasing per-collector query budgets, and expanding the Gemini candidate review budget.
- Added another browser-resolution-aware modal fitting pass: the result popup now recalculates shell width, shell height, media max height, and compact-mode behavior from `window.innerWidth` / `window.innerHeight` when opened or when the viewport changes.
- Removed the fixed minimum heights on the lower modal panels so they can shrink with the viewport instead of forcing the popup past the visible browser area.
- Checked the supplied Artgrid clip HTML and a direct `yt-dlp` probe against `https://artgrid.io/clip/355470/...`; the HTML still exposes metadata and thumbnail signals only, and the direct probe returned `Unsupported URL`.
- Why it changed:
- The user reported that the popup still overflowed the browser viewport and that the visible result set still felt too thin despite the earlier search-expansion work.
- The attached Artgrid HTML sample was intended to verify whether a stable preview URL could be extracted directly from static clip HTML.
- How it was verified:
- `go test ./...`
- `bash scripts/selftest.sh`
- `python3 -m py_compile worker/downloader.py scripts/mock_searxng.py`
- `python3 worker/downloader.py --mode probe --url 'https://artgrid.io/clip/355470/couple-woman-man-holding-hand'`
- What is still risky or incomplete:
- The new modal layout now adapts to actual browser viewport dimensions in code, but a live browser confirmation on the exact user display is still needed.
- Widening search breadth and Gemini review budget should improve visible count, but it can also increase latency and token cost under poor upstream conditions.
- The supplied Artgrid HTML sample alone was not enough to derive a directly playable preview URL, so previewless Artgrid clips may still fall back to thumbnail-only behavior.
- Date: `2026-03-17` - Date: `2026-03-17`
- What changed: - What changed:
+27 -27
View File
@@ -33,6 +33,7 @@ type App struct {
WorkerScript string WorkerScript string
SearchService *services.SearchService SearchService *services.SearchService
GeminiService *services.GeminiService GeminiService *services.GeminiService
GiphyService *services.GiphyService
Hub *Hub Hub *Hub
} }
@@ -149,6 +150,8 @@ func RegisterRoutes(router *gin.Engine, app *App) {
router.POST("/api/download", app.startDownload) router.POST("/api/download", app.startDownload)
router.POST("/api/translate/summary", app.translateSummary) router.POST("/api/translate/summary", app.translateSummary)
router.POST("/api/search", app.searchMedia) router.POST("/api/search", app.searchMedia)
router.POST("/api/giphy/search", app.searchGiphy)
router.POST("/api/giphy/download", app.downloadGiphy)
} }
func (a *App) debug(message string, data any) { func (a *App) debug(message string, data any) {
@@ -554,7 +557,7 @@ func (a *App) searchMedia(c *gin.Context) {
return return
} }
targetCount := 18 targetCount := 16
merged := services.MergeRecommendations(recommended, scored, targetCount) merged := services.MergeRecommendations(recommended, scored, targetCount)
if geminiErr != nil { if geminiErr != nil {
merged = services.BackfillRecommendations( merged = services.BackfillRecommendations(
@@ -564,34 +567,31 @@ func (a *App) searchMedia(c *gin.Context) {
"Gemini 배치 일부가 실패해 미리보기 가능한 상위 후보를 보강했습니다.", "Gemini 배치 일부가 실패해 미리보기 가능한 상위 후보를 보강했습니다.",
) )
} }
for pass := 0; pass < 2 && len(merged) < targetCount && time.Now().Before(deadline.Add(-4*time.Second)); pass++ { if len(merged) < targetCount && time.Now().Before(deadline.Add(-5*time.Second)) {
coverageQueries := buildCoverageQueries(req.Query, queryVariants, recommended, merged) coverageQueries := buildCoverageQueries(req.Query, queryVariants, recommended, merged)
if len(coverageQueries) == 0 { if len(coverageQueries) > 0 {
break a.debug("search coverage query variants", gin.H{"variants": coverageQueries, "variantCount": len(coverageQueries), "existingCount": len(merged)})
extraResults, extraMeta, extraErr := a.SearchService.SearchMediaWithDeadline(coverageQueries, enabledPlatforms, deadline.Add(-5*time.Second))
supplementalDeadlineLimited = supplementalDeadlineLimited || extraMeta.PartialDueToDeadline
if extraErr == nil && len(extraResults) > 0 {
results = mergeSearchResults(results, extraResults)
scored = services.RankSearchResults(strings.Join(coverageQueries[:min(len(coverageQueries), 3)], " "), results)
reviewedLinks := services.ReviewedRecommendationLinks(recommended)
supplementalCandidates := services.SelectUnevaluatedCandidates(scored, reviewedLinks, services.RemainingGeminiCapacity(recommended))
if len(supplementalCandidates) > 0 {
extraRecommended, extraStats, extraGeminiErr := services.EvaluateAllCandidatesWithGeminiWithDeadline(
a.GeminiService,
req.Query,
supplementalCandidates,
deadline.Add(-2*time.Second),
)
recommended = services.MergeUniqueRecommendations(recommended, extraRecommended)
geminiStats = services.MergeGeminiBatchStats(geminiStats, extraStats)
geminiErr = combineSearchWarnings(geminiErr, extraGeminiErr)
}
merged = services.MergeRecommendations(recommended, scored, targetCount)
}
} }
a.debug("search coverage query variants", gin.H{"variants": coverageQueries, "variantCount": len(coverageQueries), "existingCount": len(merged), "pass": pass + 1})
extraResults, extraMeta, extraErr := a.SearchService.SearchMediaWithDeadline(coverageQueries, enabledPlatforms, deadline.Add(-4*time.Second))
supplementalDeadlineLimited = supplementalDeadlineLimited || extraMeta.PartialDueToDeadline
if extraErr != nil || len(extraResults) == 0 {
break
}
results = mergeSearchResults(results, extraResults)
scored = services.RankSearchResults(strings.Join(coverageQueries[:min(len(coverageQueries), 3)], " "), results)
reviewedLinks := services.ReviewedRecommendationLinks(recommended)
supplementalCandidates := services.SelectUnevaluatedCandidates(scored, reviewedLinks, services.RemainingGeminiCapacity(recommended))
if len(supplementalCandidates) == 0 {
break
}
extraRecommended, extraStats, extraGeminiErr := services.EvaluateAllCandidatesWithGeminiWithDeadline(
a.GeminiService,
req.Query,
supplementalCandidates,
deadline.Add(-2*time.Second),
)
recommended = services.MergeUniqueRecommendations(recommended, extraRecommended)
geminiStats = services.MergeGeminiBatchStats(geminiStats, extraStats)
geminiErr = combineSearchWarnings(geminiErr, extraGeminiErr)
merged = services.MergeRecommendations(recommended, scored, targetCount)
} }
if len(merged) < targetCount { if len(merged) < targetCount {
merged = services.BackfillRecommendations(merged, scored, targetCount, "추가 검색 후에도 충분한 결과가 부족해 시각 자산이 있는 후보를 제한적으로 보강했습니다.") merged = services.BackfillRecommendations(merged, scored, targetCount, "추가 검색 후에도 충분한 결과가 부족해 시각 자산이 있는 후보를 제한적으로 보강했습니다.")
+98
View File
@@ -0,0 +1,98 @@
package handlers
import (
"net/http"
"strings"
"ai-media-hub/backend/models"
"ai-media-hub/backend/services"
"github.com/gin-gonic/gin"
)
func (a *App) searchGiphy(c *gin.Context) {
if a.GiphyService == nil || !a.GiphyService.Config.Enabled {
c.JSON(http.StatusServiceUnavailable, gin.H{"error": "giphy search is disabled"})
return
}
var req struct {
Query string `json:"query"`
MaxResults int `json:"maxResults"`
}
if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
req.Query = strings.TrimSpace(req.Query)
if req.Query == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "query is required"})
return
}
a.Hub.Broadcast("progress", gin.H{"type": "giphy-search", "status": "expanding query for GIPHY", "progress": 20})
resp, err := a.GiphyService.SearchImages(req.Query, req.MaxResults)
if err != nil {
a.debug("giphy:search_error", gin.H{"query": req.Query, "error": err.Error()})
a.Hub.Broadcast("progress", gin.H{"type": "giphy-search", "status": "giphy search failed", "progress": 100, "message": err.Error()})
c.JSON(http.StatusBadGateway, gin.H{"error": err.Error()})
return
}
a.debug("giphy:search_complete", gin.H{
"query": req.Query,
"expandedQueries": resp.ExpandedQueries,
"total": resp.Total,
"warning": resp.Warning,
})
a.Hub.Broadcast("progress", gin.H{"type": "giphy-search", "status": "giphy search complete", "progress": 100, "count": resp.Total})
c.JSON(http.StatusOK, resp)
}
func (a *App) downloadGiphy(c *gin.Context) {
if a.GiphyService == nil || !a.GiphyService.Config.Enabled {
c.JSON(http.StatusServiceUnavailable, gin.H{"ok": false, "error": "GIPHY_DISABLED", "message": "GIPHY download is disabled"})
return
}
var req struct {
ProviderID string `json:"providerId"`
Title string `json:"title"`
DownloadURL string `json:"downloadUrl"`
OriginalQuery string `json:"originalQuery"`
SelectedExpansionQuery string `json:"selectedExpansionQuery"`
}
if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"ok": false, "error": "INVALID_REQUEST", "message": err.Error()})
return
}
resp, err := a.GiphyService.DownloadMedia(services.GiphyDownloadRequest{
ProviderID: req.ProviderID,
Title: req.Title,
DownloadURL: req.DownloadURL,
OriginalQuery: req.OriginalQuery,
SelectedExpansionQuery: req.SelectedExpansionQuery,
})
if err != nil {
a.debug("giphy:download_error", gin.H{
"providerId": req.ProviderID,
"title": req.Title,
"error": err.Error(),
})
status := http.StatusBadGateway
if resp.Error == "INVALID_REQUEST" || resp.Error == "INVALID_DOWNLOAD_URL" {
status = http.StatusBadRequest
}
c.JSON(status, resp)
return
}
if a.DB != nil {
if recordID, dbErr := models.InsertDownload(a.DB, req.DownloadURL, "GIPHY", resp.SavedPath, "completed"); dbErr == nil {
_ = models.MarkDownloadCompleted(a.DB, recordID, "completed")
}
}
a.Hub.Broadcast("progress", gin.H{"type": "giphy-download", "status": "giphy download complete", "progress": 100, "fileName": resp.FileName})
c.JSON(http.StatusOK, resp)
}
+94
View File
@@ -0,0 +1,94 @@
package handlers
import (
"bytes"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"ai-media-hub/backend/services"
"github.com/gin-gonic/gin"
)
func TestSearchGiphyEndpointReturnsExpandedQueriesAndItems(t *testing.T) {
gin.SetMode(gin.TestMode)
mux := http.NewServeMux()
mux.HandleFunc("/generate", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"candidates":[{"content":{"parts":[{"text":"{\"queries\":[\"happy dog\",\"happy dog gif\",\"dog reaction\",\"dog meme gif\",\"animated dog sticker\"]}"}]}}]}`))
})
mux.HandleFunc("/v1/gifs/search", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"data":[{"id":"dog-1","title":"Happy Dog","slug":"happy-dog","rating":"g","url":"https://giphy.com/gifs/dog-1","images":{"original":{"url":"https://media.giphy.com/media/dog-1/giphy.gif","width":"480","height":"270"},"fixed_width":{"url":"https://media.giphy.com/media/dog-1/200w.gif","width":"200","height":"113"},"fixed_width_still":{"url":"https://media.giphy.com/media/dog-1/200w_s.gif"}}}]}`))
})
server := httptest.NewServer(mux)
defer server.Close()
gemini := services.NewGeminiService("dummy-key", "gemini-2.5-flash")
gemini.Client = &http.Client{Timeout: 2 * time.Second}
gemini.GenerateEndpoint = server.URL + "/generate"
giphy := services.NewGiphyService(services.GiphyConfig{
Enabled: true,
APIKey: "test-key",
MaxResults: 100,
BaseURL: server.URL,
}, gemini)
giphy.Client = &http.Client{Timeout: 2 * time.Second}
app := &App{GiphyService: giphy, Hub: NewHub()}
router := gin.New()
RegisterRoutes(router, app)
body := bytes.NewBufferString(`{"query":"행복한 강아지","maxResults":100}`)
req := httptest.NewRequest(http.MethodPost, "/api/giphy/search", body)
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
router.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("expected 200, got %d: %s", rec.Code, rec.Body.String())
}
var payload struct {
ExpandedQueries []string `json:"expandedQueries"`
Items []services.GiphyResult `json:"items"`
}
if err := json.Unmarshal(rec.Body.Bytes(), &payload); err != nil {
t.Fatalf("failed to decode response: %v", err)
}
if len(payload.ExpandedQueries) != 5 || len(payload.Items) == 0 {
t.Fatalf("unexpected payload: %#v", payload)
}
}
func TestDownloadGiphyEndpointRejectsNonGiphyHost(t *testing.T) {
gin.SetMode(gin.TestMode)
app := &App{
GiphyService: services.NewGiphyService(services.GiphyConfig{
Enabled: true,
APIKey: "test-key",
DownloadDir: t.TempDir(),
}, nil),
Hub: NewHub(),
}
router := gin.New()
RegisterRoutes(router, app)
body := bytes.NewBufferString(`{"providerId":"x","title":"bad","downloadUrl":"https://example.com/file.gif"}`)
req := httptest.NewRequest(http.MethodPost, "/api/giphy/download", body)
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
router.ServeHTTP(rec, req)
if rec.Code != http.StatusBadRequest {
t.Fatalf("expected 400, got %d: %s", rec.Code, rec.Body.String())
}
if !strings.Contains(rec.Body.String(), "INVALID_DOWNLOAD_URL") {
t.Fatalf("expected invalid host error, got %s", rec.Body.String())
}
}
+60 -1
View File
@@ -5,6 +5,8 @@ import (
"net/http" "net/http"
"os" "os"
"path/filepath" "path/filepath"
"strconv"
"strings"
"ai-media-hub/backend/handlers" "ai-media-hub/backend/handlers"
"ai-media-hub/backend/models" "ai-media-hub/backend/models"
@@ -17,8 +19,16 @@ func main() {
root := envOrDefault("APP_ROOT", "/app") root := envOrDefault("APP_ROOT", "/app")
dbPath := envOrDefault("SQLITE_PATH", filepath.Join(root, "db", "media.db")) dbPath := envOrDefault("SQLITE_PATH", filepath.Join(root, "db", "media.db"))
downloadsDir := envOrDefault("DOWNLOADS_DIR", filepath.Join(root, "downloads")) downloadsDir := envOrDefault("DOWNLOADS_DIR", filepath.Join(root, "downloads"))
giphyDownloadDir := envOrDefault("GIPHY_DOWNLOAD_DIR", filepath.Join(downloadsDir, "giphy"))
frontendDir := envOrDefault("FRONTEND_DIR", filepath.Join(root, "frontend")) frontendDir := envOrDefault("FRONTEND_DIR", filepath.Join(root, "frontend"))
workerScript := envOrDefault("WORKER_SCRIPT", filepath.Join(root, "worker", "downloader.py")) workerScript := envOrDefault("WORKER_SCRIPT", filepath.Join(root, "worker", "downloader.py"))
geminiAPIKey := os.Getenv("GEMINI_API_KEY")
geminiModel := envOrDefault("GEMINI_MODEL", "gemini-2.5-flash")
giphyEnabled := envBoolOrDefault("GIPHY_ENABLED", true)
giphyAPIKey := os.Getenv("GIPHY_API_KEY")
giphyMaxResults := envIntOrDefault("GIPHY_MAX_RESULTS", 100)
giphyRating := envOrDefault("GIPHY_RATING", "g")
giphyLang := envOrDefault("GIPHY_LANG", "en")
db, err := models.InitDB(dbPath) db, err := models.InitDB(dbPath)
if err != nil { if err != nil {
@@ -29,6 +39,25 @@ func main() {
if err := handlers.EnsurePaths(downloadsDir, workerScript); err != nil { if err := handlers.EnsurePaths(downloadsDir, workerScript); err != nil {
log.Fatal(err) log.Fatal(err)
} }
if geminiAPIKey == "" {
log.Printf("warning: GEMINI_API_KEY is not configured; query expansion will use fallback behavior")
}
if giphyEnabled && strings.TrimSpace(giphyAPIKey) == "" {
log.Fatal("GIPHY_ENABLED is true but GIPHY_API_KEY is not configured")
}
if err := os.MkdirAll(giphyDownloadDir, 0o755); err != nil {
log.Fatal(err)
}
geminiService := services.NewGeminiService(geminiAPIKey, geminiModel)
giphyService := services.NewGiphyService(services.GiphyConfig{
Enabled: giphyEnabled,
APIKey: giphyAPIKey,
MaxResults: giphyMaxResults,
Rating: giphyRating,
Lang: giphyLang,
DownloadDir: giphyDownloadDir,
}, geminiService)
app := &handlers.App{ app := &handlers.App{
DB: db, DB: db,
@@ -40,7 +69,8 @@ func main() {
os.Getenv("SEARXNG_GOOGLE_VIDEO_ENGINE"), os.Getenv("SEARXNG_GOOGLE_VIDEO_ENGINE"),
os.Getenv("SEARXNG_WEB_ENGINE"), os.Getenv("SEARXNG_WEB_ENGINE"),
), ),
GeminiService: services.NewGeminiService(os.Getenv("GEMINI_API_KEY")), GeminiService: geminiService,
GiphyService: giphyService,
Hub: handlers.NewHub(), Hub: handlers.NewHub(),
} }
app.SearchService.Debug = func(message string, data any) { app.SearchService.Debug = func(message string, data any) {
@@ -49,6 +79,9 @@ func main() {
app.GeminiService.Debug = func(message string, data any) { app.GeminiService.Debug = func(message string, data any) {
app.Hub.Broadcast("debug", gin.H{"message": message, "data": data}) app.Hub.Broadcast("debug", gin.H{"message": message, "data": data})
} }
app.GiphyService.Debug = func(message string, data any) {
app.Hub.Broadcast("debug", gin.H{"message": message, "data": data})
}
router := gin.Default() router := gin.Default()
handlers.RegisterRoutes(router, app) handlers.RegisterRoutes(router, app)
@@ -75,3 +108,29 @@ func envOrDefault(key, fallback string) string {
} }
return fallback return fallback
} }
func envBoolOrDefault(key string, fallback bool) bool {
value := strings.TrimSpace(strings.ToLower(os.Getenv(key)))
switch value {
case "1", "true", "yes", "on":
return true
case "0", "false", "no", "off":
return false
case "":
return fallback
default:
return fallback
}
}
func envIntOrDefault(key string, fallback int) int {
value := strings.TrimSpace(os.Getenv(key))
if value == "" {
return fallback
}
parsed, err := strconv.Atoi(value)
if err != nil {
return fallback
}
return parsed
}
+30 -8
View File
@@ -63,7 +63,7 @@ func NewSearchService(baseURL, googleVideoEngine, webEngine string) *SearchServi
webEngine = "google" webEngine = "google"
} }
return &SearchService{ return &SearchService{
BaseURL: strings.TrimRight(baseURL, "/"), BaseURL: normalizeBaseURL(baseURL),
GoogleVideoEngine: googleVideoEngine, GoogleVideoEngine: googleVideoEngine,
WebEngine: webEngine, WebEngine: webEngine,
Client: &http.Client{Timeout: 20 * time.Second}, Client: &http.Client{Timeout: 20 * time.Second},
@@ -77,6 +77,17 @@ func NewSearchService(baseURL, googleVideoEngine, webEngine string) *SearchServi
} }
} }
func normalizeBaseURL(raw string) string {
trimmed := strings.TrimSpace(raw)
if trimmed == "" {
return ""
}
if !strings.Contains(trimmed, "://") {
trimmed = "http://" + trimmed
}
return strings.TrimRight(trimmed, "/")
}
func (s *SearchService) SearchMedia(queries []string, enabledPlatforms map[string]bool) ([]SearchResult, SearchExecutionMeta, error) { func (s *SearchService) SearchMedia(queries []string, enabledPlatforms map[string]bool) ([]SearchResult, SearchExecutionMeta, error) {
return s.SearchMediaWithDeadline(queries, enabledPlatforms, time.Time{}) return s.SearchMediaWithDeadline(queries, enabledPlatforms, time.Time{})
} }
@@ -90,17 +101,16 @@ func (s *SearchService) SearchMediaWithDeadline(queries []string, enabledPlatfor
s.debug("search_service:start", map[string]any{ s.debug("search_service:start", map[string]any{
"queries": queries, "queries": queries,
"enabledPlatforms": enabledPlatforms, "enabledPlatforms": enabledPlatforms,
"deadlineSet": !deadline.IsZero(),
}) })
seen := map[string]bool{} seen := map[string]bool{}
sourceCounts := map[string]int{} sourceCounts := map[string]int{}
results := make([]SearchResult, 0, 90) results := make([]SearchResult, 0, 90)
var lastErr error var lastErr error
collectorZeroStreak := map[string]int{}
baseQueries := limitQueries(queries, 10) baseQueries := limitQueries(queries, 8)
shuffleStrings(baseQueries) primaryQueries := baseQueries[:minInt(len(baseQueries), 3)]
primaryQueries := baseQueries[:minInt(len(baseQueries), 4)]
runSearchPass := func(bases []string, onlyMissing bool) { runSearchPass := func(bases []string, onlyMissing bool) {
for _, base := range bases { for _, base := range bases {
if !collectionDeadline.IsZero() && time.Now().After(collectionDeadline) { if !collectionDeadline.IsZero() && time.Now().After(collectionDeadline) {
@@ -128,7 +138,6 @@ func (s *SearchService) SearchMediaWithDeadline(queries []string, enabledPlatfor
continue continue
} }
searchQueries := collector.BuildQueries(base) searchQueries := collector.BuildQueries(base)
shuffleStrings(searchQueries)
searchQueries = limitCollectorQueries(collector.Name(), searchQueries, onlyMissing) searchQueries = limitCollectorQueries(collector.Name(), searchQueries, onlyMissing)
s.debug("search_service:collector_queries", map[string]any{ s.debug("search_service:collector_queries", map[string]any{
"collector": collector.Name(), "collector": collector.Name(),
@@ -161,6 +170,11 @@ func (s *SearchService) SearchMediaWithDeadline(queries []string, enabledPlatfor
"rawCount": len(items), "rawCount": len(items),
"sourceCount": sourceCounts[collector.Name()], "sourceCount": sourceCounts[collector.Name()],
}) })
if len(items) == 0 && sourceCounts[collector.Name()] == 0 {
collectorZeroStreak[collector.Name()]++
} else {
collectorZeroStreak[collector.Name()] = 0
}
for _, item := range items { for _, item := range items {
item = normalizeResultForCollector(collector.Name(), item) item = normalizeResultForCollector(collector.Name(), item)
if item.Link == "" || seen[item.Link] || !collector.Accept(item) { if item.Link == "" || seen[item.Link] || !collector.Accept(item) {
@@ -173,6 +187,14 @@ func (s *SearchService) SearchMediaWithDeadline(queries []string, enabledPlatfor
break break
} }
} }
if collectorZeroStreak[collector.Name()] >= 2 && sourceCounts[collector.Name()] == 0 {
s.debug("search_service:collector_skip_after_zero_streak", map[string]any{
"collector": collector.Name(),
"base": base,
"streak": collectorZeroStreak[collector.Name()],
})
break
}
} }
} }
} }
@@ -1447,9 +1469,9 @@ func limitCollectorQueries(collector string, queries []string, onlyMissing bool)
limit := 2 limit := 2
switch collector { switch collector {
case "Envato", "Artgrid": case "Envato", "Artgrid":
limit = 5
case "Google Video":
limit = 4 limit = 4
case "Google Video":
limit = 3
} }
if onlyMissing { if onlyMissing {
limit-- limit--
+13 -9
View File
@@ -159,13 +159,13 @@ func TestLimitCollectorQueriesUsesSmallerBudgetForMissingPass(t *testing.T) {
queries := []string{"a", "b", "c", "d"} queries := []string{"a", "b", "c", "d"}
got := limitCollectorQueries("Artgrid", queries, true) got := limitCollectorQueries("Artgrid", queries, true)
if len(got) != 4 { if len(got) != 3 {
t.Fatalf("expected 4 queries for missing-pass Artgrid collector, got %d", len(got)) t.Fatalf("expected 3 queries for missing-pass Artgrid collector, got %d", len(got))
} }
got = limitCollectorQueries("Google Video", queries, false) got = limitCollectorQueries("Google Video", queries, false)
if len(got) != 4 { if len(got) != 3 {
t.Fatalf("expected 4 queries for Google Video collector, got %d", len(got)) t.Fatalf("expected 3 queries for Google Video collector, got %d", len(got))
} }
} }
@@ -182,6 +182,13 @@ func TestSearchServiceFetchCacheRoundTrip(t *testing.T) {
} }
} }
func TestNewSearchServiceNormalizesSchemeLessBaseURL(t *testing.T) {
service := NewSearchService("192.168.1.66:8087", "", "")
if service.BaseURL != "http://192.168.1.66:8087" {
t.Fatalf("expected normalized base url, got %q", service.BaseURL)
}
}
func TestSplitSearchDeadlinesReservesEnrichmentWindow(t *testing.T) { func TestSplitSearchDeadlinesReservesEnrichmentWindow(t *testing.T) {
deadline := time.Now().Add(20 * time.Second) deadline := time.Now().Add(20 * time.Second)
collectionDeadline, enrichmentDeadline := splitSearchDeadlines(deadline) collectionDeadline, enrichmentDeadline := splitSearchDeadlines(deadline)
@@ -192,17 +199,14 @@ func TestSplitSearchDeadlinesReservesEnrichmentWindow(t *testing.T) {
if !collectionDeadline.Before(enrichmentDeadline) { if !collectionDeadline.Before(enrichmentDeadline) {
t.Fatalf("expected collection deadline before enrichment deadline, got %v >= %v", collectionDeadline, enrichmentDeadline) t.Fatalf("expected collection deadline before enrichment deadline, got %v >= %v", collectionDeadline, enrichmentDeadline)
} }
if gap := enrichmentDeadline.Sub(collectionDeadline); gap < searchEnrichmentReserve-500*time.Millisecond {
t.Fatalf("expected reserve close to %v, got %v", searchEnrichmentReserve, gap)
}
} }
func TestSplitSearchDeadlinesDoesNotReserveWhenDeadlineIsTooClose(t *testing.T) { func TestSplitSearchDeadlinesDoesNotReserveWhenBudgetTooSmall(t *testing.T) {
deadline := time.Now().Add(2 * time.Second) deadline := time.Now().Add(2 * time.Second)
collectionDeadline, enrichmentDeadline := splitSearchDeadlines(deadline) collectionDeadline, enrichmentDeadline := splitSearchDeadlines(deadline)
if !collectionDeadline.Equal(enrichmentDeadline) { if !collectionDeadline.Equal(enrichmentDeadline) {
t.Fatalf("expected identical deadlines when budget is too tight, got %v and %v", collectionDeadline, enrichmentDeadline) t.Fatalf("expected identical deadlines, got %v and %v", collectionDeadline, enrichmentDeadline)
} }
} }
+109 -3
View File
@@ -20,6 +20,7 @@ import (
type GeminiService struct { type GeminiService struct {
APIKey string APIKey string
Model string
Client *http.Client Client *http.Client
GenerateEndpoint string GenerateEndpoint string
TranslateEndpoint string TranslateEndpoint string
@@ -69,11 +70,15 @@ type QueryExpansion struct {
Querywords []string `json:"querywords"` Querywords []string `json:"querywords"`
} }
func NewGeminiService(apiKey string) *GeminiService { func NewGeminiService(apiKey, model string) *GeminiService {
if strings.TrimSpace(model) == "" {
model = "gemini-2.5-flash"
}
return &GeminiService{ return &GeminiService{
APIKey: apiKey, APIKey: apiKey,
Model: model,
Client: &http.Client{Timeout: 40 * time.Second}, Client: &http.Client{Timeout: 40 * time.Second},
GenerateEndpoint: "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent", GenerateEndpoint: "https://generativelanguage.googleapis.com/v1beta/models/" + model + ":generateContent",
TranslateEndpoint: "https://translate.googleapis.com/translate_a/single", TranslateEndpoint: "https://translate.googleapis.com/translate_a/single",
visualCache: map[string]cachedVisualData{}, visualCache: map[string]cachedVisualData{},
translationCache: map[string]cachedStringValue{}, translationCache: map[string]cachedStringValue{},
@@ -99,6 +104,80 @@ func (g *GeminiService) ExpandQuery(query string) ([]string, error) {
return expanded, nil return expanded, nil
} }
func (g *GeminiService) ExpandImageQueries(query string) ([]string, error) {
trimmed := strings.TrimSpace(query)
if trimmed == "" {
return nil, fmt.Errorf("query is empty")
}
cacheKey := "image-expansion\n" + trimmed
if cached, ok := g.getCachedExpansion(cacheKey); ok {
g.debug("gemini:image_expand_cache_hit", map[string]any{"query": trimmed, "expanded": cached})
return cached, nil
}
fallback := buildFallbackImageQueries(trimmed, g.TranslateQuery(trimmed))
if g.APIKey == "" {
g.setCachedExpansion(cacheKey, fallback, 15*time.Minute)
return fallback, fmt.Errorf("gemini api key is not configured")
}
body := map[string]any{
"systemInstruction": map[string]any{
"parts": []map[string]string{{
"text": "Return exactly 5 concise English search queries for GIPHY image or GIF search. Respond with JSON only in this shape: {\"queries\":[\"...\",\"...\",\"...\",\"...\",\"...\"]}. Keep the queries meaning-preserving, practical, deduplicated, and concise.",
}},
},
"contents": []map[string]any{{
"parts": []map[string]string{{
"text": "User query: " + trimmed + "\nGenerate exactly 5 English search queries for GIPHY image or GIF search. Include a direct translation, a common phrasing, and only relevant related variants.",
}},
}},
"generationConfig": map[string]any{
"responseMimeType": "application/json",
"temperature": 0.2,
"maxOutputTokens": 160,
},
}
rawText, err := g.generateText(body)
if err != nil {
g.debug("gemini:image_expand_error", map[string]any{"query": trimmed, "error": err.Error()})
g.setCachedExpansion(cacheKey, fallback, 15*time.Minute)
return fallback, err
}
jsonText, err := extractJSONObject(rawText)
if err != nil {
if looseQueries := parseLooseImageExpansionLines(rawText); len(looseQueries) == 5 {
g.setCachedExpansion(cacheKey, looseQueries, 15*time.Minute)
g.debug("gemini:image_expand_success", map[string]any{"query": trimmed, "queries": looseQueries, "mode": "loose_text"})
return looseQueries, nil
}
g.debug("gemini:image_expand_parse_error", map[string]any{"query": trimmed, "error": err.Error()})
g.setCachedExpansion(cacheKey, fallback, 15*time.Minute)
return fallback, err
}
var payload struct {
Queries []string `json:"queries"`
}
if err := json.Unmarshal([]byte(jsonText), &payload); err != nil {
g.debug("gemini:image_expand_json_error", map[string]any{"query": trimmed, "error": err.Error(), "raw": truncateForError(rawText, 200)})
g.setCachedExpansion(cacheKey, fallback, 15*time.Minute)
return fallback, err
}
queries := normalizeImageExpansionQueries(payload.Queries)
if len(queries) != 5 {
err := fmt.Errorf("gemini image expansion returned %d queries", len(queries))
g.debug("gemini:image_expand_invalid_count", map[string]any{"query": trimmed, "queries": queries, "error": err.Error()})
g.setCachedExpansion(cacheKey, fallback, 15*time.Minute)
return fallback, err
}
g.setCachedExpansion(cacheKey, queries, 15*time.Minute)
g.debug("gemini:image_expand_success", map[string]any{"query": trimmed, "queries": queries})
return queries, nil
}
func (g *GeminiService) TranslateSummaryToKorean(text string) (string, error) { func (g *GeminiService) TranslateSummaryToKorean(text string) (string, error) {
trimmed := strings.TrimSpace(text) trimmed := strings.TrimSpace(text)
if trimmed == "" { if trimmed == "" {
@@ -306,7 +385,6 @@ User query: ` + query,
}, },
"generationConfig": map[string]any{ "generationConfig": map[string]any{
"responseMimeType": "application/json", "responseMimeType": "application/json",
"maxOutputTokens": 1400,
}, },
} }
@@ -728,6 +806,34 @@ func truncateForError(text string, limit int) string {
return trimmed[:limit] + "..." return trimmed[:limit] + "..."
} }
func parseLooseImageExpansionLines(text string) []string {
candidates := make([]string, 0, 8)
for _, line := range strings.Split(text, "\n") {
trimmed := strings.TrimSpace(line)
if trimmed == "" {
continue
}
trimmed = strings.TrimPrefix(trimmed, "- ")
trimmed = strings.TrimPrefix(trimmed, "* ")
trimmed = strings.TrimPrefix(trimmed, "1. ")
trimmed = strings.TrimPrefix(trimmed, "2. ")
trimmed = strings.TrimPrefix(trimmed, "3. ")
trimmed = strings.TrimPrefix(trimmed, "4. ")
trimmed = strings.TrimPrefix(trimmed, "5. ")
trimmed = strings.TrimSpace(strings.Trim(trimmed, "\"'`"))
lower := strings.ToLower(trimmed)
if strings.HasPrefix(lower, "here is") || strings.HasPrefix(lower, "json") || strings.HasPrefix(lower, "output") {
continue
}
candidates = append(candidates, trimmed)
}
queries := normalizeImageExpansionQueries(candidates)
if len(queries) < 5 {
return nil
}
return queries[:5]
}
func normalizeKoreanReason(reason string) string { func normalizeKoreanReason(reason string) string {
trimmed := strings.TrimSpace(reason) trimmed := strings.TrimSpace(reason)
if trimmed == "" { if trimmed == "" {
+75 -24
View File
@@ -15,7 +15,7 @@ func TestTranslateQueryFallsBackToGoogleWithoutGeminiKey(t *testing.T) {
})) }))
defer server.Close() defer server.Close()
service := NewGeminiService("") service := NewGeminiService("", "")
service.Client = &http.Client{Timeout: 2 * time.Second} service.Client = &http.Client{Timeout: 2 * time.Second}
service.TranslateEndpoint = server.URL service.TranslateEndpoint = server.URL
@@ -31,7 +31,7 @@ func TestTranslateQueryFallsBackToDictionaryWhenTranslateFails(t *testing.T) {
})) }))
defer server.Close() defer server.Close()
service := NewGeminiService("") service := NewGeminiService("", "")
service.Client = &http.Client{Timeout: 2 * time.Second} service.Client = &http.Client{Timeout: 2 * time.Second}
service.TranslateEndpoint = server.URL service.TranslateEndpoint = server.URL
@@ -50,7 +50,7 @@ func TestTranslateSummaryToKoreanUsesGoogleAndCaches(t *testing.T) {
})) }))
defer server.Close() defer server.Close()
service := NewGeminiService("") service := NewGeminiService("", "")
service.Client = &http.Client{Timeout: 2 * time.Second} service.Client = &http.Client{Timeout: 2 * time.Second}
service.TranslateEndpoint = server.URL service.TranslateEndpoint = server.URL
@@ -84,7 +84,7 @@ func TestBuildSupplementalQueriesReturnsGeneratedLines(t *testing.T) {
})) }))
defer server.Close() defer server.Close()
service := NewGeminiService("dummy-key") service := NewGeminiService("dummy-key", "")
service.Client = &http.Client{Timeout: 2 * time.Second} service.Client = &http.Client{Timeout: 2 * time.Second}
service.GenerateEndpoint = server.URL service.GenerateEndpoint = server.URL
@@ -99,6 +99,72 @@ func TestBuildSupplementalQueriesReturnsGeneratedLines(t *testing.T) {
} }
} }
func TestExpandImageQueriesReturnsExactlyFiveEnglishQueries(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"candidates":[{"content":{"parts":[{"text":"{\"queries\":[\"funny cat\",\"funny cat gif\",\"cute cat reaction\",\"cat meme gif\",\"animated cat sticker\"]}"}]}}]}`))
}))
defer server.Close()
service := NewGeminiService("dummy-key", "gemini-2.5-flash")
service.Client = &http.Client{Timeout: 2 * time.Second}
service.GenerateEndpoint = server.URL
queries, err := service.ExpandImageQueries("웃긴 고양이")
if err != nil {
t.Fatalf("expected image query expansion to succeed, got %v", err)
}
if len(queries) != 5 {
t.Fatalf("expected exactly 5 queries, got %#v", queries)
}
if queries[0] != "funny cat" || queries[4] != "animated cat sticker" {
t.Fatalf("unexpected image queries: %#v", queries)
}
}
func TestExpandImageQueriesFallsBackWhenGeminiFails(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.Error(w, "boom", http.StatusBadGateway)
}))
defer server.Close()
service := NewGeminiService("dummy-key", "gemini-2.5-flash")
service.Client = &http.Client{Timeout: 2 * time.Second}
service.GenerateEndpoint = server.URL
service.TranslateEndpoint = server.URL
queries, err := service.ExpandImageQueries("happy dog")
if err == nil {
t.Fatal("expected fallback warning error when gemini expansion fails")
}
if len(queries) != 5 {
t.Fatalf("expected fallback to still provide 5 queries, got %#v", queries)
}
if queries[0] != "happy dog" {
t.Fatalf("expected original query to be preserved in fallback, got %#v", queries)
}
}
func TestExpandImageQueriesAcceptsLoosePlainTextList(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"candidates":[{"content":{"parts":[{"text":"Here is the JSON requested\n1. cute cat\n2. cute cat gif\n3. cat reaction gif\n4. cat meme gif\n5. animated cat sticker"}]}}]}`))
}))
defer server.Close()
service := NewGeminiService("dummy-key", "gemini-2.5-flash")
service.Client = &http.Client{Timeout: 2 * time.Second}
service.GenerateEndpoint = server.URL
queries, err := service.ExpandImageQueries("고양이")
if err != nil {
t.Fatalf("expected loose plain-text list to be accepted, got %v", err)
}
if len(queries) != 5 || queries[0] != "cute cat" || queries[4] != "animated cat sticker" {
t.Fatalf("unexpected loose parsed queries: %#v", queries)
}
}
func TestSelectUnevaluatedCandidatesSkipsReviewedLinks(t *testing.T) { func TestSelectUnevaluatedCandidatesSkipsReviewedLinks(t *testing.T) {
ranked := []SearchResult{ ranked := []SearchResult{
{Link: "https://a.example"}, {Link: "https://a.example"},
@@ -123,13 +189,13 @@ func TestRemainingGeminiCapacityShrinksWithReviewedItems(t *testing.T) {
{Link: "https://a.example"}, {Link: "https://a.example"},
{Link: "https://b.example"}, {Link: "https://b.example"},
} }
if got := RemainingGeminiCapacity(reviewed); got != 22 { if got := RemainingGeminiCapacity(reviewed); got != 14 {
t.Fatalf("expected 22 remaining slots, got %d", got) t.Fatalf("expected 14 remaining slots, got %d", got)
} }
} }
func TestGeminiVisualCacheRoundTrip(t *testing.T) { func TestGeminiVisualCacheRoundTrip(t *testing.T) {
service := NewGeminiService("") service := NewGeminiService("", "")
service.setCachedVisual("image\nhttps://example.com/thumb.jpg", "abc", "image/jpeg", time.Minute) service.setCachedVisual("image\nhttps://example.com/thumb.jpg", "abc", "image/jpeg", time.Minute)
data, mimeType, ok := service.getCachedVisual("image\nhttps://example.com/thumb.jpg") data, mimeType, ok := service.getCachedVisual("image\nhttps://example.com/thumb.jpg")
@@ -142,7 +208,7 @@ func TestGeminiVisualCacheRoundTrip(t *testing.T) {
} }
func TestGeminiTranslationCacheRoundTrip(t *testing.T) { func TestGeminiTranslationCacheRoundTrip(t *testing.T) {
service := NewGeminiService("") service := NewGeminiService("", "")
service.setCachedTranslation("비 오는 도시", "rainy city", time.Minute) service.setCachedTranslation("비 오는 도시", "rainy city", time.Minute)
value, ok := service.getCachedTranslation("비 오는 도시") value, ok := service.getCachedTranslation("비 오는 도시")
@@ -155,7 +221,7 @@ func TestGeminiTranslationCacheRoundTrip(t *testing.T) {
} }
func TestGeminiExpansionCacheRoundTrip(t *testing.T) { func TestGeminiExpansionCacheRoundTrip(t *testing.T) {
service := NewGeminiService("") service := NewGeminiService("", "")
service.setCachedExpansion("city rain", []string{"city rain", "city rain stock footage"}, time.Minute) service.setCachedExpansion("city rain", []string{"city rain", "city rain stock footage"}, time.Minute)
value, ok := service.getCachedExpansion("city rain") value, ok := service.getCachedExpansion("city rain")
@@ -213,18 +279,3 @@ func TestMergeRecommendationsExcludesIrrelevantAndPendingFiller(t *testing.T) {
t.Fatalf("unexpected merged result: %#v", merged) t.Fatalf("unexpected merged result: %#v", merged)
} }
} }
func TestFilterHardGeminiErrorsIgnoresLowValueVisualFailures(t *testing.T) {
errs := []string{
"candidate thumbnail is low value",
"no candidate thumbnails or preview frames could be fetched for gemini vision",
"gemini vision JSON extraction failed: no complete JSON object found",
}
filtered := filterHardGeminiErrors(errs)
if len(filtered) != 1 {
t.Fatalf("expected only hard errors to remain, got %#v", filtered)
}
if !strings.Contains(filtered[0], "JSON extraction failed") {
t.Fatalf("unexpected filtered errors: %#v", filtered)
}
}
+617
View File
@@ -0,0 +1,617 @@
package services
import (
"encoding/json"
"fmt"
"io"
"mime"
"net/http"
neturl "net/url"
"os"
"path"
"path/filepath"
"regexp"
"strconv"
"strings"
"time"
)
const (
defaultGiphyAPIBaseURL = "https://api.giphy.com"
defaultGiphyMaxResults = 100
giphyBatchSize = 20
giphyDownloadSizeLimit = 50 * 1024 * 1024
)
type GiphyConfig struct {
Enabled bool
APIKey string
MaxResults int
Rating string
Lang string
DownloadDir string
BaseURL string
}
type GiphyService struct {
Config GiphyConfig
Client *http.Client
Gemini *GeminiService
Debug func(message string, data any)
BaseURL string
}
type GiphyResult struct {
Provider string `json:"provider"`
ProviderID string `json:"providerId"`
Link string `json:"link,omitempty"`
Title string `json:"title"`
SearchQuery string `json:"searchQuery"`
OriginalQuery string `json:"originalQuery,omitempty"`
PreviewURL string `json:"previewUrl"`
PreviewStillURL string `json:"previewStillUrl"`
FullURL string `json:"fullUrl"`
DownloadURL string `json:"downloadUrl"`
Width int `json:"width"`
Height int `json:"height"`
Rating string `json:"rating"`
SourcePageURL string `json:"sourcePageUrl"`
OpenURL string `json:"openUrl,omitempty"`
Source string `json:"source,omitempty"`
ActionLabel string `json:"actionLabel,omitempty"`
ActionType string `json:"actionType,omitempty"`
SecondaryActionLabel string `json:"secondaryActionLabel,omitempty"`
PreviewBlockedReason string `json:"previewBlockedReason,omitempty"`
Raw map[string]any `json:"raw,omitempty"`
}
type GiphySearchResponse struct {
Provider string `json:"provider"`
OriginalQuery string `json:"originalQuery"`
ExpandedQueries []string `json:"expandedQueries"`
Total int `json:"total"`
Items []GiphyResult `json:"items"`
Warning string `json:"warning,omitempty"`
}
type GiphyDownloadRequest struct {
ProviderID string `json:"providerId"`
Title string `json:"title"`
DownloadURL string `json:"downloadUrl"`
OriginalQuery string `json:"originalQuery"`
SelectedExpansionQuery string `json:"selectedExpansionQuery"`
}
type GiphyDownloadResponse struct {
OK bool `json:"ok"`
FileName string `json:"fileName,omitempty"`
SavedPath string `json:"savedPath,omitempty"`
Error string `json:"error,omitempty"`
Message string `json:"message,omitempty"`
}
type giphySearchAPIResponse struct {
Data []giphyAPIItem `json:"data"`
}
type giphyAPIItem struct {
ID string `json:"id"`
Title string `json:"title"`
Slug string `json:"slug"`
Rating string `json:"rating"`
URL string `json:"url"`
Images struct {
Original giphyRendition `json:"original"`
OriginalStill giphyRendition `json:"original_still"`
FixedWidth giphyRendition `json:"fixed_width"`
FixedWidthStill giphyRendition `json:"fixed_width_still"`
FixedWidthDownsample giphyRendition `json:"fixed_width_downsampled"`
FixedHeight giphyRendition `json:"fixed_height"`
PreviewGIF giphyRendition `json:"preview_gif"`
Downsized giphyRendition `json:"downsized"`
DownsizedLarge giphyRendition `json:"downsized_large"`
} `json:"images"`
}
type giphyRendition struct {
URL string `json:"url"`
MP4 string `json:"mp4"`
WebP string `json:"webp"`
Width string `json:"width"`
Height string `json:"height"`
}
func NewGiphyService(config GiphyConfig, gemini *GeminiService) *GiphyService {
if config.MaxResults <= 0 {
config.MaxResults = defaultGiphyMaxResults
}
if strings.TrimSpace(config.Rating) == "" {
config.Rating = "g"
}
if strings.TrimSpace(config.Lang) == "" {
config.Lang = "en"
}
if strings.TrimSpace(config.BaseURL) == "" {
config.BaseURL = defaultGiphyAPIBaseURL
}
return &GiphyService{
Config: config,
BaseURL: strings.TrimRight(config.BaseURL, "/"),
Client: &http.Client{Timeout: 20 * time.Second},
Gemini: gemini,
}
}
func (s *GiphyService) SearchImages(query string, requestedMax int) (GiphySearchResponse, error) {
response := GiphySearchResponse{
Provider: "giphy",
OriginalQuery: strings.TrimSpace(query),
}
if !s.Config.Enabled {
return response, fmt.Errorf("giphy is disabled")
}
if response.OriginalQuery == "" {
return response, fmt.Errorf("query is required")
}
if strings.TrimSpace(s.Config.APIKey) == "" {
return response, fmt.Errorf("giphy api key is not configured")
}
target := s.Config.MaxResults
if requestedMax > 0 {
target = minInt(requestedMax, s.Config.MaxResults)
}
if target <= 0 {
target = minInt(defaultGiphyMaxResults, s.Config.MaxResults)
}
expandedQueries, expansionErr := s.expandQueries(response.OriginalQuery)
response.ExpandedQueries = expandedQueries
if expansionErr != nil {
s.debug("giphy:query_expansion_fallback", map[string]any{
"query": response.OriginalQuery,
"queries": expandedQueries,
"error": expansionErr.Error(),
})
} else {
s.debug("giphy:query_expansion", map[string]any{
"query": response.OriginalQuery,
"queries": expandedQueries,
})
}
type queryState struct {
query string
offset int
enabled bool
}
states := make([]queryState, 0, len(expandedQueries))
for _, item := range expandedQueries {
states = append(states, queryState{query: item, enabled: true})
}
items := make([]GiphyResult, 0, target)
seen := map[string]bool{}
var allErrs []string
var successfulCalls int
fetchRound := func(limit int) bool {
progress := false
for idx := range states {
if !states[idx].enabled || len(items) >= target {
continue
}
batchLimit := minInt(limit, target-len(items))
found, err := s.searchQuery(states[idx].query, batchLimit, states[idx].offset, response.OriginalQuery)
if err != nil {
allErrs = append(allErrs, err.Error())
s.debug("giphy:query_error", map[string]any{
"query": states[idx].query,
"offset": states[idx].offset,
"error": err.Error(),
})
continue
}
successfulCalls++
s.debug("giphy:query_results", map[string]any{
"query": states[idx].query,
"offset": states[idx].offset,
"count": len(found),
})
if len(found) == 0 {
states[idx].enabled = false
continue
}
states[idx].offset += len(found)
before := len(items)
for _, item := range found {
if len(items) >= target {
break
}
if mergeUniqueGiphyResult(&items, item, seen) {
progress = true
}
}
if len(found) < batchLimit && len(items) == before {
states[idx].enabled = false
}
}
return progress
}
progress := fetchRound(minInt(giphyBatchSize, target))
for round := 0; len(items) < target && progress && round < 3; round++ {
progress = fetchRound(minInt(giphyBatchSize, target-len(items)))
}
response.Items = items
response.Total = len(items)
s.debug("giphy:search_complete", map[string]any{
"query": response.OriginalQuery,
"expanded": response.ExpandedQueries,
"total": response.Total,
"target": target,
"warning": response.Warning,
"successCalls": successfulCalls,
})
if response.Total == 0 && len(allErrs) > 0 {
return response, fmt.Errorf("giphy search failed: %s", strings.Join(uniqueStrings(allErrs, 3), "; "))
}
if len(allErrs) > 0 && response.Warning == "" {
response.Warning = "Some GIPHY requests failed, showing partial results."
}
return response, nil
}
func (s *GiphyService) DownloadMedia(req GiphyDownloadRequest) (GiphyDownloadResponse, error) {
if !s.Config.Enabled {
return GiphyDownloadResponse{OK: false, Error: "GIPHY_DISABLED", Message: "GIPHY is disabled"}, fmt.Errorf("giphy is disabled")
}
if strings.TrimSpace(req.ProviderID) == "" || strings.TrimSpace(req.DownloadURL) == "" {
return GiphyDownloadResponse{OK: false, Error: "INVALID_REQUEST", Message: "providerId and downloadUrl are required"}, fmt.Errorf("providerId and downloadUrl are required")
}
if !isAllowedGiphyDownloadURL(req.DownloadURL) {
return GiphyDownloadResponse{OK: false, Error: "INVALID_DOWNLOAD_URL", Message: "Only approved GIPHY media URLs are allowed"}, fmt.Errorf("download url is not on an approved giphy host")
}
if err := os.MkdirAll(s.Config.DownloadDir, 0o755); err != nil {
return GiphyDownloadResponse{OK: false, Error: "DOWNLOAD_DIR_FAILED", Message: "Failed to prepare GIPHY download directory"}, err
}
httpReq, err := http.NewRequest(http.MethodGet, req.DownloadURL, nil)
if err != nil {
return GiphyDownloadResponse{OK: false, Error: "DOWNLOAD_FAILED", Message: "Failed to build GIPHY download request"}, err
}
httpReq.Header.Set("User-Agent", "AI-Media-Hub/1.0")
resp, err := s.Client.Do(httpReq)
if err != nil {
return GiphyDownloadResponse{OK: false, Error: "DOWNLOAD_FAILED", Message: "Failed to download media from GIPHY"}, err
}
defer resp.Body.Close()
if resp.StatusCode >= 300 {
return GiphyDownloadResponse{OK: false, Error: "DOWNLOAD_FAILED", Message: "Failed to download media from GIPHY"}, fmt.Errorf("giphy download returned status %d", resp.StatusCode)
}
if resp.ContentLength > giphyDownloadSizeLimit {
return GiphyDownloadResponse{OK: false, Error: "DOWNLOAD_TOO_LARGE", Message: "GIPHY media exceeds the configured size limit"}, fmt.Errorf("giphy download too large: %d", resp.ContentLength)
}
contentType, _, _ := mime.ParseMediaType(resp.Header.Get("Content-Type"))
extension := determineGiphyExtension(req.DownloadURL, contentType)
fileName := buildGiphyFilename(req.ProviderID, req.Title, extension, time.Now())
targetPath := filepath.Join(s.Config.DownloadDir, fileName)
cleanTargetPath := filepath.Clean(targetPath)
cleanBaseDir := filepath.Clean(s.Config.DownloadDir)
if !strings.HasPrefix(cleanTargetPath, cleanBaseDir) {
return GiphyDownloadResponse{OK: false, Error: "INVALID_PATH", Message: "Resolved download path is invalid"}, fmt.Errorf("resolved giphy download path escaped base directory")
}
data, err := io.ReadAll(io.LimitReader(resp.Body, giphyDownloadSizeLimit+1))
if err != nil {
return GiphyDownloadResponse{OK: false, Error: "DOWNLOAD_FAILED", Message: "Failed to read media from GIPHY"}, err
}
if int64(len(data)) > giphyDownloadSizeLimit {
return GiphyDownloadResponse{OK: false, Error: "DOWNLOAD_TOO_LARGE", Message: "GIPHY media exceeds the configured size limit"}, fmt.Errorf("giphy download exceeded size limit during read")
}
if err := os.WriteFile(cleanTargetPath, data, 0o644); err != nil {
return GiphyDownloadResponse{OK: false, Error: "DOWNLOAD_FAILED", Message: "Failed to save media from GIPHY"}, err
}
s.debug("giphy:download_success", map[string]any{
"providerId": req.ProviderID,
"fileName": fileName,
"savedPath": cleanTargetPath,
})
return GiphyDownloadResponse{
OK: true,
FileName: fileName,
SavedPath: cleanTargetPath,
}, nil
}
func (s *GiphyService) expandQueries(query string) ([]string, error) {
if s.Gemini == nil {
return buildFallbackImageQueries(query, strings.TrimSpace(query)), fmt.Errorf("gemini service is not configured")
}
return s.Gemini.ExpandImageQueries(query)
}
func (s *GiphyService) searchQuery(query string, limit, offset int, originalQuery string) ([]GiphyResult, error) {
endpoint, err := neturl.Parse(s.BaseURL + "/v1/gifs/search")
if err != nil {
return nil, err
}
params := endpoint.Query()
params.Set("api_key", s.Config.APIKey)
params.Set("q", query)
params.Set("limit", strconv.Itoa(limit))
params.Set("offset", strconv.Itoa(max(offset, 0)))
params.Set("rating", s.Config.Rating)
params.Set("lang", s.Config.Lang)
endpoint.RawQuery = params.Encode()
req, err := http.NewRequest(http.MethodGet, endpoint.String(), nil)
if err != nil {
return nil, err
}
req.Header.Set("Accept", "application/json")
resp, err := s.Client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode >= 300 {
data, _ := io.ReadAll(io.LimitReader(resp.Body, 2048))
return nil, fmt.Errorf("giphy returned status %d: %s", resp.StatusCode, strings.TrimSpace(string(data)))
}
var payload giphySearchAPIResponse
if err := json.NewDecoder(resp.Body).Decode(&payload); err != nil {
return nil, err
}
items := make([]GiphyResult, 0, len(payload.Data))
for _, item := range payload.Data {
mapped := mapGiphyItem(item, query, originalQuery)
if mapped.ProviderID == "" || mapped.FullURL == "" {
continue
}
items = append(items, mapped)
}
return items, nil
}
func mapGiphyItem(item giphyAPIItem, searchQuery, originalQuery string) GiphyResult {
previewURL := firstNonEmpty(
item.Images.FixedWidthDownsample.URL,
item.Images.FixedWidth.URL,
item.Images.FixedHeight.URL,
item.Images.PreviewGIF.URL,
item.Images.Downsized.URL,
item.Images.Original.URL,
)
stillURL := firstNonEmpty(
item.Images.FixedWidthStill.URL,
item.Images.OriginalStill.URL,
item.Images.FixedWidth.URL,
item.Images.Downsized.URL,
item.Images.Original.URL,
)
fullURL := firstNonEmpty(
item.Images.Original.URL,
item.Images.DownsizedLarge.URL,
item.Images.Downsized.URL,
item.Images.FixedWidth.URL,
)
downloadURL := firstNonEmpty(
item.Images.Original.URL,
item.Images.DownsizedLarge.URL,
item.Images.FixedWidth.URL,
item.Images.Original.MP4,
)
width := atoiOrZero(firstNonEmpty(item.Images.Original.Width, item.Images.FixedWidth.Width, item.Images.DownsizedLarge.Width))
height := atoiOrZero(firstNonEmpty(item.Images.Original.Height, item.Images.FixedWidth.Height, item.Images.DownsizedLarge.Height))
title := strings.TrimSpace(item.Title)
if title == "" {
title = strings.ReplaceAll(strings.TrimSpace(item.Slug), "-", " ")
}
if title == "" {
title = "Untitled GIPHY"
}
return GiphyResult{
Provider: "giphy",
ProviderID: strings.TrimSpace(item.ID),
Link: strings.TrimSpace(item.URL),
Title: title,
SearchQuery: searchQuery,
OriginalQuery: originalQuery,
PreviewURL: previewURL,
PreviewStillURL: stillURL,
FullURL: fullURL,
DownloadURL: downloadURL,
Width: width,
Height: height,
Rating: strings.TrimSpace(item.Rating),
SourcePageURL: strings.TrimSpace(item.URL),
OpenURL: strings.TrimSpace(item.URL),
Source: "GIPHY",
ActionLabel: "Download",
ActionType: "giphy_download",
SecondaryActionLabel: "Open Original",
}
}
func mergeUniqueGiphyResult(items *[]GiphyResult, candidate GiphyResult, seen map[string]bool) bool {
for _, key := range giphyDedupKeys(candidate) {
if key == "" {
continue
}
if seen[key] {
return false
}
}
for _, key := range giphyDedupKeys(candidate) {
if key != "" {
seen[key] = true
}
}
*items = append(*items, candidate)
return true
}
func giphyDedupKeys(item GiphyResult) []string {
keys := []string{}
if item.ProviderID != "" {
keys = append(keys, "id:"+strings.ToLower(item.ProviderID))
}
if item.FullURL != "" {
keys = append(keys, "full:"+strings.ToLower(strings.TrimSpace(item.FullURL)))
}
if item.SourcePageURL != "" {
keys = append(keys, "source:"+strings.ToLower(strings.TrimSpace(item.SourcePageURL)))
}
return keys
}
func buildFallbackImageQueries(originalQuery, englishBase string) []string {
base := strings.TrimSpace(englishBase)
if base == "" {
base = strings.TrimSpace(originalQuery)
}
candidates := []string{
base,
base + " gif",
base + " reaction gif",
base + " meme gif",
base + " animated gif",
base + " reaction image",
base + " sticker",
}
queries := normalizeImageExpansionQueries(candidates)
if len(queries) > 5 {
return queries[:5]
}
for len(queries) < 5 {
queries = append(queries, fmt.Sprintf("%s gif %d", base, len(queries)+1))
}
return queries[:5]
}
func normalizeImageExpansionQueries(items []string) []string {
seen := map[string]bool{}
queries := make([]string, 0, 5)
for _, item := range items {
normalized := sanitizePlainEnglishLine(item)
normalized = strings.Join(strings.Fields(normalized), " ")
if normalized == "" {
continue
}
if !looksMostlyASCII(normalized) {
continue
}
key := strings.ToLower(normalized)
if seen[key] {
continue
}
seen[key] = true
queries = append(queries, normalized)
}
return queries
}
func isAllowedGiphyDownloadURL(rawURL string) bool {
parsed, err := neturl.Parse(strings.TrimSpace(rawURL))
if err != nil {
return false
}
host := strings.ToLower(parsed.Hostname())
if host == "" {
return false
}
if parsed.Scheme != "https" && parsed.Scheme != "http" {
return false
}
return host == "giphy.com" || strings.HasSuffix(host, ".giphy.com")
}
func buildGiphyFilename(providerID, title, extension string, now time.Time) string {
slug := sanitizeGiphyFilenameComponent(title)
if slug == "" {
slug = "untitled"
}
providerID = sanitizeGiphyFilenameComponent(providerID)
if providerID == "" {
providerID = "unknown"
}
ext := extension
if !strings.HasPrefix(ext, ".") {
ext = "." + ext
}
return fmt.Sprintf("giphy_%s_%s_%s%s", providerID, slug, now.Format("20060102_150405"), ext)
}
func sanitizeGiphyFilenameComponent(value string) string {
re := regexp.MustCompile(`[^a-z0-9]+`)
normalized := strings.ToLower(strings.TrimSpace(value))
normalized = re.ReplaceAllString(normalized, "-")
return strings.Trim(normalized, "-")
}
func determineGiphyExtension(rawURL, contentType string) string {
switch strings.ToLower(strings.TrimSpace(contentType)) {
case "image/gif":
return ".gif"
case "video/mp4":
return ".mp4"
case "image/webp":
return ".webp"
case "image/png":
return ".png"
case "image/jpeg":
return ".jpg"
}
parsed, err := neturl.Parse(rawURL)
if err == nil {
ext := strings.ToLower(path.Ext(parsed.Path))
switch ext {
case ".gif", ".mp4", ".webp", ".png", ".jpg", ".jpeg":
if ext == ".jpeg" {
return ".jpg"
}
return ext
}
}
return ".gif"
}
func atoiOrZero(value string) int {
parsed, err := strconv.Atoi(strings.TrimSpace(value))
if err != nil {
return 0
}
return parsed
}
func uniqueStrings(items []string, limit int) []string {
seen := map[string]bool{}
unique := make([]string, 0, minInt(len(items), limit))
for _, item := range items {
trimmed := strings.TrimSpace(item)
if trimmed == "" || seen[trimmed] {
continue
}
seen[trimmed] = true
unique = append(unique, trimmed)
if len(unique) >= limit {
break
}
}
return unique
}
func (s *GiphyService) debug(message string, data any) {
if s != nil && s.Debug != nil {
s.Debug(message, data)
}
}
+170
View File
@@ -0,0 +1,170 @@
package services
import (
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
func TestNewGiphyServiceAppliesDefaults(t *testing.T) {
service := NewGiphyService(GiphyConfig{Enabled: true}, nil)
if service.Config.MaxResults != 100 {
t.Fatalf("expected default max results 100, got %d", service.Config.MaxResults)
}
if service.Config.Rating != "g" || service.Config.Lang != "en" {
t.Fatalf("unexpected defaults: %#v", service.Config)
}
if service.BaseURL != defaultGiphyAPIBaseURL {
t.Fatalf("expected default base url %q, got %q", defaultGiphyAPIBaseURL, service.BaseURL)
}
}
func TestIsAllowedGiphyDownloadURL(t *testing.T) {
if !isAllowedGiphyDownloadURL("https://media2.giphy.com/media/test/giphy.gif") {
t.Fatal("expected media.giphy.com host to be allowed")
}
if isAllowedGiphyDownloadURL("https://example.com/file.gif") {
t.Fatal("expected non-giphy host to be rejected")
}
}
func TestBuildGiphyFilenameSanitizesInput(t *testing.T) {
got := buildGiphyFilename("ABC123", "Funny Cat!!!", ".gif", time.Date(2026, 3, 24, 15, 32, 12, 0, time.UTC))
want := "giphy_abc123_funny-cat_20260324_153212.gif"
if got != want {
t.Fatalf("expected %q, got %q", want, got)
}
}
func TestGiphySearchAggregatesDedupesAndCapsAt100(t *testing.T) {
mux := http.NewServeMux()
mux.HandleFunc("/generate", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"candidates":[{"content":{"parts":[{"text":"{\"queries\":[\"funny cat\",\"happy cat gif\",\"cat reaction\",\"cat meme\",\"animated cat sticker\"]}"}]}}]}`))
})
mux.HandleFunc("/v1/gifs/search", func(w http.ResponseWriter, r *http.Request) {
query := r.URL.Query().Get("q")
limit := atoiOrZero(r.URL.Query().Get("limit"))
offset := atoiOrZero(r.URL.Query().Get("offset"))
data := make([]map[string]any, 0, limit)
for idx := 0; idx < limit; idx++ {
id := fmt.Sprintf("%s-%d", strings.ReplaceAll(query, " ", "-"), offset+idx)
if idx == 0 {
id = fmt.Sprintf("shared-%d", offset)
}
data = append(data, map[string]any{
"id": id,
"title": fmt.Sprintf("%s %d", query, offset+idx),
"slug": fmt.Sprintf("%s-%d", strings.ReplaceAll(query, " ", "-"), offset+idx),
"rating": "g",
"url": "https://giphy.com/gifs/" + id,
"images": map[string]any{
"original": map[string]any{
"url": fmt.Sprintf("https://media.giphy.com/media/%s/giphy.gif", id),
"width": "480",
"height": "270",
},
"fixed_width": map[string]any{
"url": fmt.Sprintf("https://media.giphy.com/media/%s/200w.gif", id),
"width": "200",
"height": "113",
},
"fixed_width_still": map[string]any{
"url": fmt.Sprintf("https://media.giphy.com/media/%s/200w_s.gif", id),
},
},
})
}
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(map[string]any{"data": data})
})
server := httptest.NewServer(mux)
defer server.Close()
gemini := NewGeminiService("dummy-key", "gemini-2.5-flash")
gemini.Client = &http.Client{Timeout: 2 * time.Second}
gemini.GenerateEndpoint = server.URL + "/generate"
service := NewGiphyService(GiphyConfig{
Enabled: true,
APIKey: "test-key",
MaxResults: 100,
BaseURL: server.URL,
}, gemini)
service.Client = &http.Client{Timeout: 2 * time.Second}
resp, err := service.SearchImages("웃긴 고양이", 100)
if err != nil {
t.Fatalf("expected giphy search to succeed, got %v", err)
}
if len(resp.ExpandedQueries) != 5 {
t.Fatalf("expected 5 expanded queries, got %#v", resp.ExpandedQueries)
}
if resp.Total != 100 || len(resp.Items) != 100 {
t.Fatalf("expected capped 100 unique items, got total=%d len=%d", resp.Total, len(resp.Items))
}
seen := map[string]bool{}
for _, item := range resp.Items {
if seen[item.ProviderID] {
t.Fatalf("found duplicate providerId %q in aggregated results", item.ProviderID)
}
seen[item.ProviderID] = true
}
}
func TestDownloadMediaHappyPath(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "image/gif")
_, _ = w.Write([]byte("GIF89a"))
}))
defer server.Close()
serverURL, err := url.Parse(server.URL)
if err != nil {
t.Fatalf("failed to parse server url: %v", err)
}
tempDir := t.TempDir()
service := NewGiphyService(GiphyConfig{
Enabled: true,
APIKey: "test-key",
DownloadDir: tempDir,
}, nil)
service.Client = &http.Client{
Timeout: 2 * time.Second,
Transport: roundTripperFunc(func(req *http.Request) (*http.Response, error) {
clone := req.Clone(req.Context())
if strings.HasSuffix(clone.URL.Host, "giphy.com") {
clone.URL.Scheme = serverURL.Scheme
clone.URL.Host = serverURL.Host
clone.Host = serverURL.Host
}
return http.DefaultTransport.RoundTrip(clone)
}),
}
resp, err := service.DownloadMedia(GiphyDownloadRequest{
ProviderID: "abc123",
Title: "Funny Cat",
DownloadURL: "https://media.giphy.com/media/abc123/giphy.gif",
})
if err != nil {
t.Fatalf("expected download to succeed, got %v", err)
}
if !resp.OK {
t.Fatalf("expected ok response, got %#v", resp)
}
if filepath.Ext(resp.FileName) != ".gif" {
t.Fatalf("expected gif extension, got %q", resp.FileName)
}
if _, err := os.Stat(resp.SavedPath); err != nil {
t.Fatalf("expected saved file at %q: %v", resp.SavedPath, err)
}
}
+9 -48
View File
@@ -97,7 +97,7 @@ func RankSearchResults(query string, results []SearchResult) []SearchResult {
} }
func GeminiCandidateLimit(total int) int { func GeminiCandidateLimit(total int) int {
return min(total, 24) return min(total, 16)
} }
func EvaluateAllCandidatesWithGemini(service *GeminiService, query string, ranked []SearchResult) ([]AIRecommendation, GeminiBatchStats, error) { func EvaluateAllCandidatesWithGemini(service *GeminiService, query string, ranked []SearchResult) ([]AIRecommendation, GeminiBatchStats, error) {
@@ -105,7 +105,7 @@ func EvaluateAllCandidatesWithGemini(service *GeminiService, query string, ranke
} }
func EvaluateAllCandidatesWithGeminiWithDeadline(service *GeminiService, query string, ranked []SearchResult, deadline time.Time) ([]AIRecommendation, GeminiBatchStats, error) { func EvaluateAllCandidatesWithGeminiWithDeadline(service *GeminiService, query string, ranked []SearchResult, deadline time.Time) ([]AIRecommendation, GeminiBatchStats, error) {
const chunkSize = 6 const chunkSize = 8
const maxConcurrentBatches = 2 const maxConcurrentBatches = 2
if service == nil { if service == nil {
return nil, GeminiBatchStats{}, fmt.Errorf("gemini service is not configured") return nil, GeminiBatchStats{}, fmt.Errorf("gemini service is not configured")
@@ -186,8 +186,7 @@ func EvaluateAllCandidatesWithGeminiWithDeadline(service *GeminiService, query s
"error": batch.err.Error(), "error": batch.err.Error(),
}) })
} }
recovered, recoveredErrs := recoverGeminiBatchSequentially(service, query, ranked, batch.index*chunkSize, chunkSize, deadline) recovered, recoveredErrs := recoverGeminiBatchSequentially(service, query, ranked, batch.index*chunkSize)
hardErrs := filterHardGeminiErrors(recoveredErrs)
if len(recovered) > 0 { if len(recovered) > 0 {
stats.SequentialRetried++ stats.SequentialRetried++
stats.Succeeded++ stats.Succeeded++
@@ -198,9 +197,9 @@ func EvaluateAllCandidatesWithGeminiWithDeadline(service *GeminiService, query s
seen[item.Link] = true seen[item.Link] = true
merged = append(merged, item) merged = append(merged, item)
} }
if len(hardErrs) > 0 { if len(recoveredErrs) > 0 {
stats.Failed++ stats.Failed++
for _, recoveredErr := range hardErrs { for _, recoveredErr := range recoveredErrs {
if len(stats.Errors) < 5 { if len(stats.Errors) < 5 {
stats.Errors = append(stats.Errors, recoveredErr) stats.Errors = append(stats.Errors, recoveredErr)
} }
@@ -208,9 +207,6 @@ func EvaluateAllCandidatesWithGeminiWithDeadline(service *GeminiService, query s
} }
continue continue
} }
if len(hardErrs) == 0 {
continue
}
stats.Failed++ stats.Failed++
if len(stats.Errors) < 5 { if len(stats.Errors) < 5 {
stats.Errors = append(stats.Errors, batch.err.Error()) stats.Errors = append(stats.Errors, batch.err.Error())
@@ -295,7 +291,7 @@ func ReviewedRecommendationLinks(items []AIRecommendation) map[string]bool {
} }
func RemainingGeminiCapacity(reviewed []AIRecommendation) int { func RemainingGeminiCapacity(reviewed []AIRecommendation) int {
remaining := GeminiCandidateLimit(24) - len(ReviewedRecommendationLinks(reviewed)) remaining := GeminiCandidateLimit(16) - len(ReviewedRecommendationLinks(reviewed))
if remaining < 0 { if remaining < 0 {
return 0 return 0
} }
@@ -349,46 +345,11 @@ func MergeGeminiBatchStats(base, extra GeminiBatchStats) GeminiBatchStats {
return merged return merged
} }
func filterHardGeminiErrors(errs []string) []string { func recoverGeminiBatchSequentially(service *GeminiService, query string, ranked []SearchResult, startIndex int) ([]AIRecommendation, []string) {
filtered := make([]string, 0, len(errs)) recovered := make([]AIRecommendation, 0, 8)
for _, item := range errs {
if isIgnorableGeminiError(item) {
continue
}
filtered = append(filtered, item)
}
return filtered
}
func isIgnorableGeminiError(message string) bool {
lower := strings.ToLower(strings.TrimSpace(message))
if lower == "" {
return false
}
for _, token := range []string{
"no candidate thumbnails or preview frames could be fetched for gemini vision",
"candidate thumbnail is low value",
"candidate has no thumbnail or preview video",
"image url is empty",
} {
if strings.Contains(lower, token) {
return true
}
}
return false
}
func recoverGeminiBatchSequentially(service *GeminiService, query string, ranked []SearchResult, startIndex, chunkSize int, deadline time.Time) ([]AIRecommendation, []string) {
recovered := make([]AIRecommendation, 0, chunkSize)
errs := make([]string, 0, 4) errs := make([]string, 0, 4)
endIndex := min(startIndex+chunkSize, len(ranked)) endIndex := min(startIndex+8, len(ranked))
for idx := startIndex; idx < endIndex; idx++ { for idx := startIndex; idx < endIndex; idx++ {
if !deadline.IsZero() && time.Now().After(deadline) {
if len(errs) < 4 {
errs = append(errs, "sequential gemini recovery stopped at deadline")
}
break
}
recs, err := service.Recommend(query, []SearchResult{ranked[idx]}) recs, err := service.Recommend(query, []SearchResult{ranked[idx]})
if err != nil { if err != nil {
if len(errs) < 4 { if len(errs) < 4 {
+2 -2
View File
@@ -15,7 +15,7 @@ type searchCollector interface {
type envatoCollector struct{} type envatoCollector struct{}
func (envatoCollector) Name() string { return "Envato" } func (envatoCollector) Name() string { return "Envato" }
func (envatoCollector) MaxResults() int { return 16 } func (envatoCollector) MaxResults() int { return 12 }
func (envatoCollector) Enabled(enabledPlatforms map[string]bool) bool { func (envatoCollector) Enabled(enabledPlatforms map[string]bool) bool {
return len(enabledPlatforms) == 0 || enabledPlatforms["envato"] return len(enabledPlatforms) == 0 || enabledPlatforms["envato"]
} }
@@ -31,7 +31,7 @@ func (envatoCollector) Enrich(searcher *SearchService, result SearchResult) Sear
type artgridCollector struct{} type artgridCollector struct{}
func (artgridCollector) Name() string { return "Artgrid" } func (artgridCollector) Name() string { return "Artgrid" }
func (artgridCollector) MaxResults() int { return 16 } func (artgridCollector) MaxResults() int { return 12 }
func (artgridCollector) Enabled(enabledPlatforms map[string]bool) bool { func (artgridCollector) Enabled(enabledPlatforms map[string]bool) bool {
return len(enabledPlatforms) == 0 || enabledPlatforms["artgrid"] return len(enabledPlatforms) == 0 || enabledPlatforms["artgrid"]
} }
+181 -35
View File
@@ -5,6 +5,11 @@ const searchQuery = document.getElementById("searchQuery");
const searchResults = document.getElementById("searchResults"); const searchResults = document.getElementById("searchResults");
const searchWarning = document.getElementById("searchWarning"); const searchWarning = document.getElementById("searchWarning");
const queryVariants = document.getElementById("queryVariants"); const queryVariants = document.getElementById("queryVariants");
const searchModeTitle = document.getElementById("searchModeTitle");
const searchModeHint = document.getElementById("searchModeHint");
const searchSubmitButton = document.getElementById("searchSubmitButton");
const searchResultsViewport = document.getElementById("searchResultsViewport");
const mediaTypeToggles = Array.from(document.querySelectorAll("[data-media-type-toggle]"));
const platformToggles = Array.from(document.querySelectorAll("[data-platform-toggle]")); const platformToggles = Array.from(document.querySelectorAll("[data-platform-toggle]"));
const dropzone = document.getElementById("dropzone"); const dropzone = document.getElementById("dropzone");
const fileInput = document.getElementById("fileInput"); const fileInput = document.getElementById("fileInput");
@@ -13,6 +18,7 @@ const downloadForm = document.getElementById("downloadForm");
const downloadUrl = document.getElementById("downloadUrl"); const downloadUrl = document.getElementById("downloadUrl");
const downloadResult = document.getElementById("downloadResult"); const downloadResult = document.getElementById("downloadResult");
const cardTemplate = document.getElementById("searchCardTemplate"); const cardTemplate = document.getElementById("searchCardTemplate");
const imageCardTemplate = document.getElementById("imageCardTemplate");
const previewModal = document.getElementById("previewModal"); const previewModal = document.getElementById("previewModal");
const previewMediaFrame = document.getElementById("previewMediaFrame"); const previewMediaFrame = document.getElementById("previewMediaFrame");
const previewTitle = document.getElementById("previewTitle"); const previewTitle = document.getElementById("previewTitle");
@@ -38,9 +44,10 @@ const downloadLogs = document.getElementById("downloadLogs");
const debugLogList = document.getElementById("debugLogList"); const debugLogList = document.getElementById("debugLogList");
const debugSummary = document.getElementById("debugSummary"); const debugSummary = document.getElementById("debugSummary");
const resultModal = document.getElementById("resultModal"); const resultModal = document.getElementById("resultModal");
const resultModalShell = document.getElementById("resultModalShell");
const resultModalTitle = document.getElementById("resultModalTitle"); const resultModalTitle = document.getElementById("resultModalTitle");
const resultModalSource = document.getElementById("resultModalSource"); const resultModalSource = document.getElementById("resultModalSource");
const resultModalReasonLabel = document.getElementById("resultModalReasonLabel");
const resultModalSnippetLabel = document.getElementById("resultModalSnippetLabel");
const resultModalSnippet = document.getElementById("resultModalSnippet"); const resultModalSnippet = document.getElementById("resultModalSnippet");
const resultModalReason = document.getElementById("resultModalReason"); const resultModalReason = document.getElementById("resultModalReason");
const resultModalFrame = document.getElementById("resultModalFrame"); const resultModalFrame = document.getElementById("resultModalFrame");
@@ -57,9 +64,10 @@ const resultModalSecondaryAction = document.getElementById("resultModalSecondary
const closeResultModal = document.getElementById("closeResultModal"); const closeResultModal = document.getElementById("closeResultModal");
const resultModalReady = Boolean( const resultModalReady = Boolean(
resultModal && resultModal &&
resultModalShell &&
resultModalTitle && resultModalTitle &&
resultModalSource && resultModalSource &&
resultModalReasonLabel &&
resultModalSnippetLabel &&
resultModalSnippet && resultModalSnippet &&
resultModalReason && resultModalReason &&
resultModalFrame && resultModalFrame &&
@@ -91,6 +99,7 @@ const summaryTranslationInflight = new Map();
const resultPreviewCache = new Map(); const resultPreviewCache = new Map();
const resultPreviewInflight = new Map(); const resultPreviewInflight = new Map();
let cardSummaryObserver = null; let cardSummaryObserver = null;
let activeMediaType = "video";
const PREVIEW_PLACEHOLDER = "https://placehold.co/1280x720/0a0a0a/ffffff?text=Preview"; const PREVIEW_PLACEHOLDER = "https://placehold.co/1280x720/0a0a0a/ffffff?text=Preview";
function proxiedPreviewURL(src) { function proxiedPreviewURL(src) {
@@ -158,6 +167,9 @@ function setStatus(label, progress) {
} }
function setHidden(element, hidden, visibleDisplayClass = "flex") { function setHidden(element, hidden, visibleDisplayClass = "flex") {
if (!element) {
return;
}
element.classList.toggle("hidden", hidden); element.classList.toggle("hidden", hidden);
if (visibleDisplayClass) { if (visibleDisplayClass) {
element.classList.toggle(visibleDisplayClass, !hidden); element.classList.toggle(visibleDisplayClass, !hidden);
@@ -283,6 +295,70 @@ function renderQueryVariants(queries = []) {
queryVariants.classList.add("hidden"); queryVariants.classList.add("hidden");
} }
function syncMediaTypeButtons() {
for (const button of mediaTypeToggles) {
const type = button.dataset.mediaTypeToggle;
const active = type === activeMediaType;
button.classList.toggle("bg-white", active);
button.classList.toggle("text-black", active);
button.classList.toggle("text-zinc-300", !active);
}
}
function renderImageEmptyState(message) {
searchResults.innerHTML = "";
searchResults.innerHTML = `<div class="rounded-3xl border border-white/10 bg-black/30 p-5 text-sm text-zinc-400">${message}</div>`;
}
function renderImageResults(items = []) {
searchResults.innerHTML = "";
searchResults.classList.remove("xl:grid-cols-3");
searchResults.classList.add("xl:grid-cols-4");
if (!items.length) {
renderImageEmptyState("GIPHY에서 표시할 이미지/GIF를 찾지 못했습니다.");
return;
}
for (const item of items) {
const node = imageCardTemplate.content.firstElementChild.cloneNode(true);
const image = node.querySelector("img");
image.loading = "lazy";
image.src = item.previewStillUrl || item.previewUrl || item.fullUrl || PREVIEW_PLACEHOLDER;
image.alt = item.title;
node.querySelector(".image-card-tag").textContent = `GIPHY / ${item.searchQuery || "query"}`;
node.querySelector(".image-card-title").textContent = item.title;
node.querySelector(".image-card-caption").textContent = item.title || "Untitled GIPHY result";
node.querySelector(".image-card-meta").textContent = `${item.rating || "unrated"} / ${item.width || "?"}x${item.height || "?"}`;
node.addEventListener("click", () => openResultModal(item));
searchResults.appendChild(node);
}
}
function applyMediaTypeUI() {
const isImageMode = activeMediaType === "image";
syncMediaTypeButtons();
setHidden(queryVariants, true, "");
showWarning("");
searchResultsViewport.classList.toggle("image-results-scroll", isImageMode);
searchModeTitle.textContent = isImageMode ? "AI Image Discovery" : "AI Smart Discovery";
searchModeHint.textContent = isImageMode
? "GIPHY 이미지/GIF 검색 결과를 그대로 보여주는 모드입니다. 최대 100개 결과를 내부 스크롤로 탐색할 수 있습니다."
: "비디오 검색 모드입니다. 실제 검색 API와 연결되어 있습니다.";
searchQuery.placeholder = isImageMode ? "검색할 이미지를 설명하세요" : "한글 검색어를 입력하세요";
searchSubmitButton.textContent = isImageMode ? "Search GIPHY" : "AI Search";
for (const button of platformToggles) {
button.classList.toggle("hidden", isImageMode);
}
if (isImageMode) {
setStatus("giphy image mode", 0);
renderImageEmptyState("GIPHY 검색어를 입력하면 여기에 최대 100개의 이미지/GIF 결과가 표시됩니다.");
} else {
searchResults.classList.add("xl:grid-cols-3");
searchResults.classList.remove("xl:grid-cols-4");
searchResults.innerHTML = "";
setStatus(`active platforms: ${Array.from(activePlatforms).join(", ")}`, 0);
}
}
function syncPlatformButtons() { function syncPlatformButtons() {
for (const button of platformToggles) { for (const button of platformToggles) {
const platform = button.dataset.platformToggle; const platform = button.dataset.platformToggle;
@@ -438,30 +514,12 @@ function resetPreviewPlayer() {
function showModal(element) { function showModal(element) {
setHidden(element, false); setHidden(element, false);
if (element === resultModal) {
syncResultModalLayout();
}
} }
function hideModal(element) { function hideModal(element) {
setHidden(element, true); setHidden(element, true);
} }
function syncResultModalLayout() {
if (!resultModalReady) {
return;
}
const viewportWidth = Math.max(320, window.innerWidth || document.documentElement.clientWidth || 0);
const viewportHeight = Math.max(320, window.innerHeight || document.documentElement.clientHeight || 0);
const shellWidth = Math.min(1150, viewportWidth - 16);
const shellHeight = Math.min(840, viewportHeight - 12);
const mediaHeight = Math.max(150, Math.min(Math.floor(viewportHeight * 0.32), 320));
resultModalShell.style.setProperty("--result-modal-shell-width", `${shellWidth}px`);
resultModalShell.style.setProperty("--result-modal-shell-height", `${shellHeight}px`);
resultModalShell.style.setProperty("--result-modal-media-max-height", `${mediaHeight}px`);
resultModalShell.classList.toggle("result-modal-compact", viewportHeight < 900 || viewportWidth < 1280);
}
function buildResultModalEmbedURL(item) { function buildResultModalEmbedURL(item) {
if (item?.embedUrl) { if (item?.embedUrl) {
if (item.source === "Google Video" && item.embedUrl.includes("youtube-nocookie.com/embed/")) { if (item.source === "Google Video" && item.embedUrl.includes("youtube-nocookie.com/embed/")) {
@@ -500,6 +558,10 @@ function resetResultModalMedia() {
setHidden(resultModalGooglePanel, true, "flex"); setHidden(resultModalGooglePanel, true, "flex");
} }
function isGiphyResult(item) {
return String(item?.provider || "").toLowerCase() === "giphy" || String(item?.source || "").toLowerCase() === "giphy";
}
function showResultModalFrame(src) { function showResultModalFrame(src) {
if (!src) { if (!src) {
return; return;
@@ -609,7 +671,7 @@ async function fetchResultPreview(item) {
} }
const request = (async () => { const request = (async () => {
try { try {
const preview = await api("/api/download/preview", { const preview = await api("/api/download/preview", {
method: "POST", method: "POST",
headers: { "Content-Type": "application/json" }, headers: { "Content-Type": "application/json" },
body: JSON.stringify({ url: key }), body: JSON.stringify({ url: key }),
@@ -751,21 +813,42 @@ async function openResultModal(item) {
logEvent("result:modal:error", { message: "result modal is not fully initialized" }); logEvent("result:modal:error", { message: "result modal is not fully initialized" });
return; return;
} }
const giphyItem = isGiphyResult(item);
activeResultItem = item; activeResultItem = item;
activeResultModalSummaryRequest += 1; activeResultModalSummaryRequest += 1;
const summaryRequestId = activeResultModalSummaryRequest; const summaryRequestId = activeResultModalSummaryRequest;
resultModalTitle.textContent = item.title || "Untitled"; resultModalTitle.textContent = item.title || "Untitled";
resultModalSource.textContent = item.source || ""; resultModalSource.textContent = item.source || "";
resultModalReason.textContent = summarizeReason(item.reason) || "AI 노트가 없습니다."; resultModalReasonLabel.textContent = giphyItem ? "Result Info" : "AI Note";
const originalSummary = item.snippet || "원본 페이지에서 사용할 수 있는 설명이 없습니다."; resultModalSnippetLabel.textContent = giphyItem ? "Source" : "Source Summary";
resultModalReason.textContent = giphyItem
? [
`Rating: ${item.rating || "unrated"}`,
`Size: ${item.width || "?"} x ${item.height || "?"}`,
`Provider ID: ${item.providerId || "-"}`,
].join("\n")
: (summarizeReason(item.reason) || "AI 노트가 없습니다.");
const originalSummary = giphyItem
? [
"Powered by GIPHY",
item.sourcePageUrl || item.openUrl || item.link || "",
`Rating: ${item.rating || "unrated"}`,
].filter(Boolean).join("\n")
: (item.snippet || "원본 페이지에서 사용할 수 있는 설명이 없습니다.");
resultModalSnippet.textContent = originalSummary; resultModalSnippet.textContent = originalSummary;
resultModalOpenExternal.href = item.link || "#"; resultModalOpenExternal.href = item.openUrl || item.sourcePageUrl || item.link || "#";
resultModalDownload.classList.toggle("hidden", !item.actionType); resultModalDownload.classList.toggle("hidden", !item.actionType);
resultModalDownload.textContent = item.actionLabel || "Open Source"; resultModalDownload.textContent = item.actionLabel || "Open Source";
const showSecondary = Boolean(item.secondaryActionLabel && item.link); const showSecondary = Boolean(item.secondaryActionLabel && (item.openUrl || item.link));
resultModalSecondaryAction.classList.toggle("hidden", !showSecondary); resultModalSecondaryAction.classList.toggle("hidden", !showSecondary);
resultModalSecondaryAction.textContent = item.secondaryActionLabel || "Open Source"; resultModalSecondaryAction.textContent = item.secondaryActionLabel || "Open Source";
resetResultModalMedia(); resetResultModalMedia();
if (giphyItem) {
showResultModalThumbnail(item.fullUrl || item.previewUrl || item.previewStillUrl, item.title || "");
showModal(resultModal);
logEvent("result:modal:open", { title: item.title, source: item.source, link: item.link, provider: item.provider });
return;
}
const embedURL = buildResultModalEmbedURL(item); const embedURL = buildResultModalEmbedURL(item);
const fallbackReason = item.previewBlockedReason || "Embedded view was unavailable, switched to fallback preview."; const fallbackReason = item.previewBlockedReason || "Embedded view was unavailable, switched to fallback preview.";
let resolvedPreviewURL = item.previewVideoUrl || ""; let resolvedPreviewURL = item.previewVideoUrl || "";
@@ -816,6 +899,26 @@ function closeResultViewer() {
searchForm.addEventListener("submit", async (event) => { searchForm.addEventListener("submit", async (event) => {
event.preventDefault(); event.preventDefault();
if (activeMediaType === "image") {
setStatus("searching GIPHY", 10);
showWarning("");
try {
const data = await api("/api/giphy/search", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ query: searchQuery.value, maxResults: 100 }),
});
renderImageResults(data.items || []);
showWarning("");
logEvent("image-search:completed", { query: data.originalQuery || searchQuery.value, total: data.total || 0, expandedQueries: data.expandedQueries || [] });
setStatus("giphy search complete", 100);
} catch (error) {
renderImageEmptyState("GIPHY 검색 결과를 불러오지 못했습니다.");
showWarning(error.message);
setStatus("giphy search failed", 100);
}
return;
}
setStatus("preparing search", 5); setStatus("preparing search", 5);
showWarning(""); showWarning("");
try { try {
@@ -836,6 +939,14 @@ searchForm.addEventListener("submit", async (event) => {
} }
}); });
for (const button of mediaTypeToggles) {
button.addEventListener("click", () => {
activeMediaType = button.dataset.mediaTypeToggle || "video";
applyMediaTypeUI();
logEvent("media-type:update", { active: activeMediaType });
});
}
async function uploadFile(file) { async function uploadFile(file) {
const formData = new FormData(); const formData = new FormData();
formData.append("file", file); formData.append("file", file);
@@ -888,6 +999,35 @@ function closeModal() {
pendingDownload = null; pendingDownload = null;
} }
async function downloadGiphyItem(item) {
resultModalDownload.disabled = true;
const originalLabel = resultModalDownload.textContent;
resultModalDownload.textContent = "Downloading...";
try {
const data = await api("/api/giphy/download", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
providerId: item.providerId,
title: item.title,
downloadUrl: item.downloadUrl,
originalQuery: item.originalQuery,
selectedExpansionQuery: item.searchQuery,
}),
});
downloadResult.textContent = `${data.fileName} saved to ${data.savedPath}`;
setStatus("giphy download complete", 100);
logEvent("giphy:download:completed", { providerId: item.providerId, fileName: data.fileName, savedPath: data.savedPath });
} catch (error) {
downloadResult.textContent = error.message;
setStatus("giphy download failed", 100);
logEvent("giphy:download:error", { providerId: item.providerId, message: error.message, data: error.data || null });
} finally {
resultModalDownload.disabled = false;
resultModalDownload.textContent = originalLabel;
}
}
dropzone.addEventListener("dragover", (event) => { dropzone.addEventListener("dragover", (event) => {
event.preventDefault(); event.preventDefault();
dropzone.classList.add("border-white/60", "bg-white/[0.08]"); dropzone.classList.add("border-white/60", "bg-white/[0.08]");
@@ -956,10 +1096,14 @@ if (resultModalReady) {
} }
}); });
resultModalDownload.addEventListener("click", async () => { resultModalDownload.addEventListener("click", async () => {
if (!activeResultItem?.link) { if (!activeResultItem) {
return; return;
} }
const currentItem = activeResultItem; const currentItem = activeResultItem;
if (currentItem.actionType === "giphy_download") {
await downloadGiphyItem(currentItem);
return;
}
if (currentItem.actionType === "download") { if (currentItem.actionType === "download") {
try { try {
closeResultViewer(); closeResultViewer();
@@ -970,13 +1114,13 @@ if (resultModalReady) {
} }
return; return;
} }
window.open(currentItem.link, "_blank", "noopener,noreferrer"); window.open(currentItem.openUrl || currentItem.sourcePageUrl || currentItem.link, "_blank", "noopener,noreferrer");
}); });
resultModalSecondaryAction.addEventListener("click", () => { resultModalSecondaryAction.addEventListener("click", () => {
if (!activeResultItem?.link) { if (!activeResultItem) {
return; return;
} }
window.open(activeResultItem.link, "_blank", "noopener,noreferrer"); window.open(activeResultItem.openUrl || activeResultItem.sourcePageUrl || activeResultItem.link, "_blank", "noopener,noreferrer");
}); });
} }
previewModal.addEventListener("click", (event) => { previewModal.addEventListener("click", (event) => {
@@ -1050,11 +1194,6 @@ if (resultModalReady) {
} }
}); });
} }
window.addEventListener("resize", () => {
if (resultModalReady && !resultModal.classList.contains("hidden")) {
syncResultModalLayout();
}
});
for (const button of platformToggles) { for (const button of platformToggles) {
button.addEventListener("click", () => { button.addEventListener("click", () => {
const platform = button.dataset.platformToggle; const platform = button.dataset.platformToggle;
@@ -1097,8 +1236,15 @@ window.addEventListener("error", (event) => {
window.addEventListener("unhandledrejection", (event) => { window.addEventListener("unhandledrejection", (event) => {
logEvent("window:unhandledrejection", { reason: String(event.reason) }); logEvent("window:unhandledrejection", { reason: String(event.reason) });
}); });
window.addEventListener("keydown", (event) => {
if (event.key !== "Escape") {
return;
}
closeModal();
closeResultViewer();
});
connectWS(); connectWS();
syncPlatformButtons(); syncPlatformButtons();
setStatus(`active platforms: ${Array.from(activePlatforms).join(", ")}`, 0); applyMediaTypeUI();
logEvent("app:ready", { activePlatforms: Array.from(activePlatforms) }); logEvent("app:ready", { activePlatforms: Array.from(activePlatforms) });
+34 -9
View File
@@ -33,9 +33,16 @@
<div class="mb-4 flex items-center justify-between"> <div class="mb-4 flex items-center justify-between">
<div> <div>
<p class="text-xs uppercase tracking-[0.3em] text-zinc-500">Zone A</p> <p class="text-xs uppercase tracking-[0.3em] text-zinc-500">Zone A</p>
<h2 class="text-2xl font-semibold text-white">AI Smart Discovery</h2> <h2 id="searchModeTitle" class="text-2xl font-semibold text-white">AI Smart Discovery</h2>
</div> </div>
</div> </div>
<div class="mb-4 flex flex-wrap items-center gap-3">
<div class="inline-flex rounded-full border border-white/10 bg-black/30 p-1">
<button data-media-type-toggle="video" class="media-type-toggle rounded-full bg-white px-4 py-2 text-sm font-medium text-black transition">Video</button>
<button data-media-type-toggle="image" class="media-type-toggle rounded-full px-4 py-2 text-sm font-medium text-zinc-300 transition">Image</button>
</div>
<p id="searchModeHint" class="text-sm text-zinc-400">비디오 검색 모드입니다. 실제 검색 API와 연결되어 있습니다.</p>
</div>
<div class="mb-4 flex flex-wrap gap-3"> <div class="mb-4 flex flex-wrap gap-3">
<button data-platform-toggle="envato" class="platform-toggle rounded-full border border-white bg-white px-4 py-2 text-sm font-medium text-black">Envato</button> <button data-platform-toggle="envato" class="platform-toggle rounded-full border border-white bg-white px-4 py-2 text-sm font-medium text-black">Envato</button>
<button data-platform-toggle="artgrid" class="platform-toggle rounded-full border border-white bg-white px-4 py-2 text-sm font-medium text-black">Artgrid</button> <button data-platform-toggle="artgrid" class="platform-toggle rounded-full border border-white bg-white px-4 py-2 text-sm font-medium text-black">Artgrid</button>
@@ -43,11 +50,13 @@
</div> </div>
<form id="searchForm" class="flex flex-col gap-3 md:flex-row"> <form id="searchForm" class="flex flex-col gap-3 md:flex-row">
<input id="searchQuery" type="text" placeholder="한글 검색어를 입력하세요" class="flex-1 rounded-2xl border border-white/10 bg-black/40 px-5 py-4 text-base text-white outline-none ring-0 placeholder:text-zinc-500" /> <input id="searchQuery" type="text" placeholder="한글 검색어를 입력하세요" class="flex-1 rounded-2xl border border-white/10 bg-black/40 px-5 py-4 text-base text-white outline-none ring-0 placeholder:text-zinc-500" />
<button class="rounded-2xl border border-white bg-white px-7 py-4 text-base font-medium text-black transition hover:bg-zinc-200">AI Search</button> <button id="searchSubmitButton" class="rounded-2xl border border-white bg-white px-7 py-4 text-base font-medium text-black transition hover:bg-zinc-200">AI Search</button>
</form> </form>
<div id="searchWarning" class="mt-3 hidden rounded-2xl border border-amber-500/30 bg-amber-500/10 px-4 py-3 text-sm text-amber-200"></div> <div id="searchWarning" class="mt-3 hidden rounded-2xl border border-amber-500/30 bg-amber-500/10 px-4 py-3 text-sm text-amber-200"></div>
<div id="queryVariants" class="hidden"></div> <div id="queryVariants" class="hidden"></div>
<div id="searchResults" class="mt-6 grid gap-5 sm:grid-cols-2 xl:grid-cols-3"></div> <div id="searchResultsViewport" class="mt-6">
<div id="searchResults" class="grid gap-5 sm:grid-cols-2 xl:grid-cols-3"></div>
</div>
</article> </article>
<div class="grid gap-8"> <div class="grid gap-8">
@@ -150,7 +159,7 @@
</div> </div>
<div id="resultModal" class="fixed inset-0 z-50 hidden items-start justify-center overflow-y-auto bg-black/80 px-2 py-2 sm:px-4 sm:py-4"> <div id="resultModal" class="fixed inset-0 z-50 hidden items-start justify-center overflow-y-auto bg-black/80 px-2 py-2 sm:px-4 sm:py-4">
<div id="resultModalShell" class="result-modal-shell flex w-full max-w-6xl min-h-0 flex-col overflow-hidden rounded-3xl border border-white/10 bg-zinc-950 shadow-2xl"> <div class="result-modal-shell flex w-full max-w-6xl min-h-0 flex-col overflow-hidden rounded-3xl border border-white/10 bg-zinc-950 shadow-2xl">
<div class="flex items-center justify-between border-b border-white/10 px-4 py-3 sm:px-5 sm:py-4"> <div class="flex items-center justify-between border-b border-white/10 px-4 py-3 sm:px-5 sm:py-4">
<div class="min-w-0"> <div class="min-w-0">
<p id="resultModalSource" class="text-xs uppercase tracking-[0.25em] text-zinc-500"></p> <p id="resultModalSource" class="text-xs uppercase tracking-[0.25em] text-zinc-500"></p>
@@ -180,13 +189,13 @@
</div> </div>
</div> </div>
<div class="result-modal-details grid min-h-0 gap-3 px-3 py-3 sm:gap-4 sm:px-4 sm:py-4 lg:grid-cols-[1.5fr_0.8fr]"> <div class="result-modal-details grid min-h-0 gap-3 px-3 py-3 sm:gap-4 sm:px-4 sm:py-4 lg:grid-cols-[1.5fr_0.8fr]">
<div class="flex min-h-0 min-w-0 flex-col rounded-2xl border border-white/10 bg-white/[0.03] p-4"> <div class="flex min-h-[180px] min-w-0 flex-col rounded-2xl border border-white/10 bg-white/[0.03] p-4">
<p class="text-xs uppercase tracking-[0.25em] text-zinc-500">AI Note</p> <p id="resultModalReasonLabel" class="text-xs uppercase tracking-[0.25em] text-zinc-500">AI Note</p>
<div class="result-panel-scroll mt-3 min-h-0 flex-1 overflow-y-auto pr-2"> <div class="result-panel-scroll mt-3 min-h-0 flex-1 overflow-y-auto pr-2">
<p id="resultModalReason" class="whitespace-pre-wrap text-xs leading-6 text-zinc-200 sm:text-sm"></p> <p id="resultModalReason" class="whitespace-pre-wrap text-xs leading-6 text-zinc-200 sm:text-sm"></p>
</div> </div>
</div> </div>
<div class="flex min-h-0 min-w-0 flex-col rounded-2xl border border-white/10 bg-white/[0.03] p-4"> <div class="flex min-h-[200px] min-w-0 flex-col rounded-2xl border border-white/10 bg-white/[0.03] p-4">
<div class="mb-3 flex flex-col gap-2"> <div class="mb-3 flex flex-col gap-2">
<button id="resultModalDownload" type="button" class="hidden w-full rounded-2xl border border-white bg-white px-4 py-3 text-sm font-medium text-black transition hover:bg-zinc-200"> <button id="resultModalDownload" type="button" class="hidden w-full rounded-2xl border border-white bg-white px-4 py-3 text-sm font-medium text-black transition hover:bg-zinc-200">
Primary Action Primary Action
@@ -196,7 +205,7 @@
</button> </button>
</div> </div>
<div class="flex min-h-0 flex-1 flex-col"> <div class="flex min-h-0 flex-1 flex-col">
<p class="text-xs uppercase tracking-[0.25em] text-zinc-500">Source Summary</p> <p id="resultModalSnippetLabel" class="text-xs uppercase tracking-[0.25em] text-zinc-500">Source Summary</p>
<div class="result-summary-scroll mt-3 min-h-0 flex-1 overflow-y-auto pr-2"> <div class="result-summary-scroll mt-3 min-h-0 flex-1 overflow-y-auto pr-2">
<p id="resultModalSnippet" class="text-xs leading-6 text-zinc-300 sm:text-sm"></p> <p id="resultModalSnippet" class="text-xs leading-6 text-zinc-300 sm:text-sm"></p>
</div> </div>
@@ -224,6 +233,22 @@
</button> </button>
</template> </template>
<script src="/app.js?v=20260317d" defer></script> <template id="imageCardTemplate">
<button type="button" class="group overflow-hidden rounded-3xl border border-white/10 bg-black/30 text-left transition hover:border-white/30">
<div class="relative aspect-[4/3] overflow-hidden bg-zinc-900">
<img class="h-full w-full object-cover transition duration-500 group-hover:scale-105" alt="" />
<div class="absolute inset-x-0 bottom-0 bg-gradient-to-t from-black/80 via-black/10 to-transparent p-4">
<p class="image-card-tag text-[11px] uppercase tracking-[0.25em] text-zinc-300"></p>
</div>
</div>
<div class="space-y-2 p-5">
<h3 class="image-card-title line-clamp-2 text-base font-medium text-white"></h3>
<p class="image-card-caption line-clamp-3 text-sm text-zinc-300"></p>
<p class="image-card-meta text-[11px] uppercase tracking-[0.22em] text-zinc-500"></p>
</div>
</button>
</template>
<script src="/app.js?v=20260324b" defer></script>
</body> </body>
</html> </html>
+41 -13
View File
@@ -26,6 +26,45 @@ body {
-webkit-line-clamp: 3; -webkit-line-clamp: 3;
} }
.line-clamp-4 {
display: -webkit-box;
-webkit-box-orient: vertical;
overflow: hidden;
-webkit-line-clamp: 4;
}
.media-type-toggle {
min-width: 6rem;
}
.image-prompt-chip {
background: rgba(255, 255, 255, 0.03);
}
.image-prompt-chip:hover {
border-color: rgba(255, 255, 255, 0.28);
background: rgba(255, 255, 255, 0.08);
}
#searchResultsViewport {
min-height: 12rem;
}
#searchResultsViewport.image-results-scroll {
height: min(62dvh, 58rem);
overflow-y: auto;
padding-right: 0.35rem;
}
#searchResultsViewport.image-results-scroll::-webkit-scrollbar {
width: 10px;
}
#searchResultsViewport.image-results-scroll::-webkit-scrollbar-thumb {
background: rgba(255, 255, 255, 0.16);
border-radius: 9999px;
}
.dual-slider__thumb { .dual-slider__thumb {
touch-action: none; touch-action: none;
cursor: ew-resize; cursor: ew-resize;
@@ -60,8 +99,7 @@ body {
} }
.result-modal-shell { .result-modal-shell {
height: var(--result-modal-shell-height, min(calc(100dvh - 0.5rem), 860px)); height: min(calc(100dvh - 0.5rem), 860px);
max-width: var(--result-modal-shell-width, 72rem);
margin: auto; margin: auto;
} }
@@ -70,7 +108,7 @@ body {
} }
.result-modal-media-frame { .result-modal-media-frame {
max-height: var(--result-modal-media-max-height, min(34dvh, 22rem)); max-height: min(34dvh, 22rem);
} }
.result-modal-details { .result-modal-details {
@@ -80,16 +118,6 @@ body {
align-items: stretch; align-items: stretch;
} }
.result-modal-shell.result-modal-compact .result-modal-details {
gap: 0.5rem;
}
.result-modal-shell.result-modal-compact #resultModalReason,
.result-modal-shell.result-modal-compact #resultModalSnippet {
font-size: 0.75rem;
line-height: 1.35rem;
}
#resultModalSnippet, #resultModalSnippet,
#resultModalReason { #resultModalReason {
white-space: pre-wrap; white-space: pre-wrap;
-280
View File
@@ -1,280 +0,0 @@
Set-StrictMode -Version Latest
$ErrorActionPreference = "Stop"
$script:RepoRoot = Split-Path -Parent $PSScriptRoot
$script:ToolsRoot = Join-Path $script:RepoRoot ".tools"
$script:BinRoot = Join-Path $script:ToolsRoot "bin"
$script:GoRoot = Join-Path $script:ToolsRoot "go"
$script:GoBin = Join-Path $script:GoRoot "bin"
$script:FFmpegRoot = Join-Path $script:ToolsRoot "ffmpeg"
$script:FFmpegBin = Join-Path $script:FFmpegRoot "bin"
$script:VenvRoot = Join-Path $script:RepoRoot ".venv"
$script:VenvScripts = Join-Path $script:VenvRoot "Scripts"
$script:LocalRoot = Join-Path $script:RepoRoot ".local"
$script:GoPathRoot = Join-Path $script:LocalRoot "go"
$script:GoModCacheRoot = Join-Path $script:GoPathRoot "pkg\mod"
$script:GoBuildCacheRoot = Join-Path $script:LocalRoot "go-build-cache"
$script:CredentialFile = Join-Path $script:LocalRoot "git-credentials.psd1"
function Write-Step {
param([string]$Message)
Write-Host "[ai-media-hub] $Message"
}
function Invoke-CheckedCommand {
param(
[Parameter(Mandatory = $true)][string]$FilePath,
[string[]]$Arguments = @()
)
& $FilePath @Arguments
if ($LASTEXITCODE -ne 0) {
$joined = if ($Arguments.Count -gt 0) { " " + ($Arguments -join " ") } else { "" }
throw "명령 실행 실패: $FilePath$joined"
}
}
function Ensure-Directory {
param([Parameter(Mandatory = $true)][string]$Path)
if (-not (Test-Path -LiteralPath $Path)) {
New-Item -ItemType Directory -Path $Path | Out-Null
}
}
function Get-RepoRoot { return $script:RepoRoot }
function Get-ToolsRoot { return $script:ToolsRoot }
function Get-BinRoot { return $script:BinRoot }
function Get-GoRoot { return $script:GoRoot }
function Get-GoBin { return $script:GoBin }
function Get-FFmpegBin { return $script:FFmpegBin }
function Get-VenvRoot { return $script:VenvRoot }
function Get-VenvScripts { return $script:VenvScripts }
function Get-LocalRoot { return $script:LocalRoot }
function Get-GoPathRoot { return $script:GoPathRoot }
function Get-GoModCacheRoot { return $script:GoModCacheRoot }
function Get-GoBuildCacheRoot { return $script:GoBuildCacheRoot }
function Get-CredentialFile { return $script:CredentialFile }
function Resolve-SystemCommand {
param([Parameter(Mandatory = $true)][string]$Name)
try {
return (Get-Command $Name -ErrorAction Stop).Source
} catch {
return $null
}
}
function Resolve-PythonExe {
$candidates = @(
(Join-Path $script:VenvScripts "python.exe"),
(Resolve-SystemCommand -Name "python")
)
foreach ($candidate in $candidates) {
if ($candidate -and (Test-Path -LiteralPath $candidate)) {
return $candidate
}
}
throw "python.exe를 찾을 수 없습니다. Python 3.12+가 필요합니다."
}
function Resolve-BasePythonExe {
$all = @(Get-Command python -All -ErrorAction SilentlyContinue)
foreach ($item in $all) {
if ($item.Source -and $item.Source -notlike "*\.venv\*") {
return $item.Source
}
}
$fallback = Resolve-SystemCommand -Name "python"
if ($fallback -and $fallback -notlike "*\.venv\*") {
return $fallback
}
throw "시스템 python.exe를 찾지 못했습니다."
}
function Resolve-VenvPythonExe {
$pythonExe = Join-Path $script:VenvScripts "python.exe"
if (-not (Test-Path -LiteralPath $pythonExe)) {
throw ".venv가 아직 준비되지 않았습니다. scripts/setup-dev.ps1를 먼저 실행하세요."
}
return $pythonExe
}
function Use-LocalTooling {
Ensure-Directory -Path $script:ToolsRoot
Ensure-Directory -Path $script:BinRoot
Ensure-Directory -Path $script:LocalRoot
Ensure-Directory -Path $script:GoPathRoot
Ensure-Directory -Path $script:GoModCacheRoot
Ensure-Directory -Path $script:GoBuildCacheRoot
$segments = @(
$script:BinRoot,
$script:GoBin,
$script:FFmpegBin,
$script:VenvScripts,
$env:PATH
) | Where-Object { $_ -and $_.Trim() -ne "" }
$env:PATH = ($segments | Select-Object -Unique) -join ";"
$env:GOPATH = $script:GoPathRoot
$env:GOMODCACHE = $script:GoModCacheRoot
$env:GOCACHE = $script:GoBuildCacheRoot
}
function Ensure-Venv {
$venvPython = Join-Path $script:VenvScripts "python.exe"
if (Test-Path -LiteralPath $venvPython) {
return $venvPython
}
Ensure-Directory -Path $script:LocalRoot
$pythonExe = Resolve-SystemCommand -Name "python"
if (-not $pythonExe) {
throw "시스템 python이 없어 .venv를 만들 수 없습니다."
}
Write-Step ".venv 생성"
& $pythonExe -m venv $script:VenvRoot
return $venvPython
}
function Install-PythonRequirements {
param([switch]$UpgradePip)
$venvPython = Ensure-Venv
$systemPython = Resolve-BasePythonExe
if ($UpgradePip) {
Write-Step "pip 업그레이드"
Invoke-CheckedCommand -FilePath $systemPython -Arguments @("-m", "pip", "--python", $venvPython, "install", "--upgrade", "pip")
}
Write-Step "worker requirements 설치"
Invoke-CheckedCommand -FilePath $systemPython -Arguments @("-m", "pip", "--python", $venvPython, "install", "-r", (Join-Path $script:RepoRoot "worker\requirements.txt"))
}
function New-CmdShim {
param(
[Parameter(Mandatory = $true)][string]$Path,
[Parameter(Mandatory = $true)][string]$Target
)
$content = @(
"@echo off",
"set SCRIPT_DIR=%~dp0",
"`"%SCRIPT_DIR%$Target`" %*"
) -join "`r`n"
Set-Content -LiteralPath $Path -Value $content -Encoding ASCII
}
function Ensure-CommandShims {
Ensure-Directory -Path $script:BinRoot
$pythonShimTarget = "..\..\ .venv\Scripts\python.exe".Replace(" ", "")
New-CmdShim -Path (Join-Path $script:BinRoot "python3.cmd") -Target $pythonShimTarget
$ytDlpExe = Join-Path $script:VenvScripts "yt-dlp.exe"
if (Test-Path -LiteralPath $ytDlpExe) {
New-CmdShim -Path (Join-Path $script:BinRoot "yt-dlp.cmd") -Target "..\..\ .venv\Scripts\yt-dlp.exe".Replace(" ", "")
}
}
function Test-GoReady {
return [bool](Resolve-SystemCommand -Name "go")
}
function Test-FFmpegReady {
return [bool](Resolve-SystemCommand -Name "ffmpeg")
}
function Test-YtDlpReady {
return [bool](Resolve-SystemCommand -Name "yt-dlp")
}
function Install-LocalGo {
param(
[string]$Version = "1.24.0",
[string]$ArchiveUrl = ""
)
if (Test-Path -LiteralPath (Join-Path $script:GoBin "go.exe")) {
Write-Step "로컬 Go 이미 준비됨"
return
}
if (-not $ArchiveUrl) {
$ArchiveUrl = "https://go.dev/dl/go$Version.windows-amd64.zip"
}
Ensure-Directory -Path $script:ToolsRoot
$archivePath = Join-Path $script:LocalRoot "go-$Version.windows-amd64.zip"
Write-Step "로컬 Go 다운로드: $ArchiveUrl"
Invoke-WebRequest -Uri $ArchiveUrl -OutFile $archivePath
if (Test-Path -LiteralPath $script:GoRoot) {
Remove-Item -Recurse -Force $script:GoRoot
}
Expand-Archive -LiteralPath $archivePath -DestinationPath $script:ToolsRoot -Force
}
function Install-LocalFFmpeg {
param([string]$ArchiveUrl = "https://www.gyan.dev/ffmpeg/builds/ffmpeg-release-essentials.zip")
if (Test-Path -LiteralPath (Join-Path $script:FFmpegBin "ffmpeg.exe")) {
Write-Step "로컬 ffmpeg 이미 준비됨"
return
}
Ensure-Directory -Path $script:ToolsRoot
$archivePath = Join-Path $script:LocalRoot "ffmpeg-release-essentials.zip"
Write-Step "로컬 ffmpeg 다운로드: $ArchiveUrl"
Invoke-WebRequest -Uri $ArchiveUrl -OutFile $archivePath
$extractRoot = Join-Path $script:LocalRoot "ffmpeg-extract"
if (Test-Path -LiteralPath $extractRoot) {
Remove-Item -Recurse -Force $extractRoot
}
Expand-Archive -LiteralPath $archivePath -DestinationPath $extractRoot -Force
$binCandidate = Get-ChildItem -Path $extractRoot -Recurse -Filter "ffmpeg.exe" | Select-Object -First 1
if (-not $binCandidate) {
throw "ffmpeg.exe를 압축 파일에서 찾지 못했습니다."
}
$sourceBin = Split-Path -Parent $binCandidate.FullName
if (Test-Path -LiteralPath $script:FFmpegRoot) {
Remove-Item -Recurse -Force $script:FFmpegRoot
}
Ensure-Directory -Path $script:FFmpegBin
Copy-Item -Path (Join-Path $sourceBin "*") -Destination $script:FFmpegBin -Recurse -Force
}
function Get-AppEnvironment {
param(
[Parameter(Mandatory = $true)][string]$WorkspaceRoot,
[Parameter(Mandatory = $true)][string]$DownloadsDir,
[Parameter(Mandatory = $true)][string]$SqlitePath,
[Parameter(Mandatory = $true)][string]$AppAddr,
[string]$SearxUrl = "http://127.0.0.1:18080"
)
return @{
APP_ROOT = $WorkspaceRoot
APP_ADDR = $AppAddr
SQLITE_PATH = $SqlitePath
DOWNLOADS_DIR = $DownloadsDir
FRONTEND_DIR = (Join-Path $WorkspaceRoot "frontend")
WORKER_SCRIPT = (Join-Path $WorkspaceRoot "worker\downloader.py")
SEARXNG_BASE_URL = $SearxUrl
SEARXNG_GOOGLE_VIDEO_ENGINE = "google videos"
SEARXNG_WEB_ENGINE = "google"
}
}
function Import-GitCredential {
$credentialFile = Get-CredentialFile
if (-not (Test-Path -LiteralPath $credentialFile)) {
throw "git 자격정보 파일이 없습니다: $credentialFile"
}
return Import-PowerShellDataFile -Path $credentialFile
}
-10
View File
@@ -1,10 +0,0 @@
Set-StrictMode -Version Latest
$ErrorActionPreference = "Stop"
. (Join-Path $PSScriptRoot "dev-tools.ps1")
Use-LocalTooling
Write-Step "로컬 개발 PATH 적용 완료"
Write-Host (" Repo : " + (Get-RepoRoot))
Write-Host (" .venv : " + (Get-VenvRoot))
Write-Host (" .tools : " + (Get-ToolsRoot))
-31
View File
@@ -1,31 +0,0 @@
param(
[string]$Remote = "origin",
[string]$Branch = "",
[switch]$SetUpstream
)
Set-StrictMode -Version Latest
$ErrorActionPreference = "Stop"
. (Join-Path $PSScriptRoot "dev-tools.ps1")
Use-LocalTooling
$credential = Import-GitCredential
if (-not $Branch) {
$Branch = (& git branch --show-current).Trim()
}
if (-not $Branch) {
throw "현재 브랜치를 찾지 못했습니다."
}
$pair = "{0}:{1}" -f $credential.Username, $credential.Password
$token = [Convert]::ToBase64String([Text.Encoding]::UTF8.GetBytes($pair))
$gitArgs = @("-c", "http.extraheader=AUTHORIZATION: Basic $token", "push")
if ($SetUpstream) {
$gitArgs += @("--set-upstream", $Remote, $Branch)
} else {
$gitArgs += @($Remote, $Branch)
}
Write-Step "git push $Remote $Branch"
& git @gitArgs
-28
View File
@@ -1,28 +0,0 @@
param(
[string]$AppAddr = "127.0.0.1:8080",
[string]$SqlitePath = "",
[string]$DownloadsDir = "",
[string]$SearxUrl = "http://127.0.0.1:18080"
)
Set-StrictMode -Version Latest
$ErrorActionPreference = "Stop"
. (Join-Path $PSScriptRoot "dev-tools.ps1")
Use-LocalTooling
$repoRoot = Get-RepoRoot
if (-not $SqlitePath) {
$SqlitePath = Join-Path $repoRoot "db\media.dev.db"
}
if (-not $DownloadsDir) {
$DownloadsDir = Join-Path $repoRoot "downloads"
}
$envMap = Get-AppEnvironment -WorkspaceRoot $repoRoot -DownloadsDir $DownloadsDir -SqlitePath $SqlitePath -AppAddr $AppAddr -SearxUrl $SearxUrl
foreach ($entry in $envMap.GetEnumerator()) {
Set-Item -Path ("Env:" + $entry.Key) -Value $entry.Value
}
Write-Step "백엔드 실행"
Invoke-CheckedCommand -FilePath "go" -Arguments @("run", "./backend")
-158
View File
@@ -1,158 +0,0 @@
param(
[int]$MockPort = 18080,
[int]$AppPort = 18081,
[switch]$SkipGoFmt
)
Set-StrictMode -Version Latest
$ErrorActionPreference = "Stop"
. (Join-Path $PSScriptRoot "dev-tools.ps1")
Use-LocalTooling
$repoRoot = Get-RepoRoot
$tmpRoot = Join-Path (Get-LocalRoot) ("selftest-" + [guid]::NewGuid().ToString("N"))
Ensure-Directory -Path $tmpRoot
$mockProcess = $null
$appProcess = $null
$appStdoutTask = $null
$appStderrTask = $null
function Stop-TrackedProcess {
param($Process)
if ($null -ne $Process -and -not $Process.HasExited) {
Stop-Process -Id $Process.Id -Force
$Process.WaitForExit()
}
}
try {
if (-not $SkipGoFmt) {
Write-Step "gofmt"
Invoke-CheckedCommand -FilePath "gofmt" -Arguments @(
"-w",
(Join-Path $repoRoot "backend\main.go"),
(Join-Path $repoRoot "backend\handlers\api.go"),
(Join-Path $repoRoot "backend\models\db.go"),
(Join-Path $repoRoot "backend\services\cse.go"),
(Join-Path $repoRoot "backend\services\cse_test.go"),
(Join-Path $repoRoot "backend\services\search_collectors.go"),
(Join-Path $repoRoot "backend\services\ranker.go"),
(Join-Path $repoRoot "backend\services\gemini.go"),
(Join-Path $repoRoot "backend\services\gemini_test.go")
)
}
Write-Step "python syntax"
Invoke-CheckedCommand -FilePath (Resolve-VenvPythonExe) -Arguments @(
"-m",
"py_compile",
(Join-Path $repoRoot "worker\downloader.py"),
(Join-Path $repoRoot "scripts\mock_searxng.py")
)
Write-Step "go test"
Invoke-CheckedCommand -FilePath "go" -Arguments @("test", "./...")
Write-Step "frontend syntax"
Invoke-CheckedCommand -FilePath "node" -Arguments @("--check", (Join-Path $repoRoot "frontend\app.js"))
Write-Step "go build"
$binaryPath = Join-Path $tmpRoot "ai-media-hub.exe"
Invoke-CheckedCommand -FilePath "go" -Arguments @("build", "-o", $binaryPath, "./backend")
Write-Step "start mock searxng"
$mockLog = Join-Path $tmpRoot "mock-searxng.stdout.log"
$mockErrLog = Join-Path $tmpRoot "mock-searxng.stderr.log"
$mockProcess = Start-Process -FilePath (Resolve-VenvPythonExe) `
-ArgumentList @((Join-Path $repoRoot "scripts\mock_searxng.py"), "--port", $MockPort) `
-RedirectStandardOutput $mockLog `
-RedirectStandardError $mockErrLog `
-PassThru `
-WindowStyle Hidden
Write-Step "start app"
$downloadsDir = Join-Path $tmpRoot "downloads"
Ensure-Directory -Path $downloadsDir
$appLog = Join-Path $tmpRoot "app.log"
$envMap = Get-AppEnvironment `
-WorkspaceRoot $repoRoot `
-DownloadsDir $downloadsDir `
-SqlitePath (Join-Path $tmpRoot "media.db") `
-AppAddr ("127.0.0.1:{0}" -f $AppPort) `
-SearxUrl ("http://127.0.0.1:{0}" -f $MockPort)
$appStartInfo = New-Object System.Diagnostics.ProcessStartInfo
$appStartInfo.FileName = $binaryPath
$appStartInfo.WorkingDirectory = $repoRoot
$appStartInfo.UseShellExecute = $false
$appStartInfo.RedirectStandardOutput = $true
$appStartInfo.RedirectStandardError = $true
foreach ($entry in $envMap.GetEnumerator()) {
$appStartInfo.Environment[$entry.Key] = [string]$entry.Value
}
$appProcess = New-Object System.Diagnostics.Process
$appProcess.StartInfo = $appStartInfo
$null = $appProcess.Start()
$appStdoutTask = $appProcess.StandardOutput.ReadToEndAsync()
$appStderrTask = $appProcess.StandardError.ReadToEndAsync()
$healthUrl = "http://127.0.0.1:$AppPort/healthz"
$healthy = $false
for ($i = 0; $i -lt 30; $i++) {
try {
$health = Invoke-RestMethod -Uri $healthUrl -TimeoutSec 2
if ($health.status -eq "ok") {
$healthy = $true
break
}
} catch {
Start-Sleep -Seconds 1
}
}
if (-not $healthy) {
throw "/healthz 확인 실패"
}
Write-Step "verify search"
$searchPayload = @{
query = "city rain"
platforms = @("envato", "artgrid", "google video")
} | ConvertTo-Json
$search = Invoke-RestMethod -Method Post -Uri "http://127.0.0.1:$AppPort/api/search" -ContentType "application/json" -Body $searchPayload
$searchResults = @($search.results)
if ($null -eq $search.results -or $searchResults.Count -lt 2) {
throw "검색 결과가 너무 적습니다."
}
if (@($searchResults | Where-Object { -not $_.link }).Count -gt 0) {
throw "검색 결과에 link가 비어 있습니다."
}
Write-Step "verify upload"
$sampleFile = Join-Path $tmpRoot "sample.txt"
Set-Content -LiteralPath $sampleFile -Value "selftest upload" -Encoding UTF8
$form = @{
file = Get-Item -LiteralPath $sampleFile
}
$upload = Invoke-RestMethod -Method Post -Uri "http://127.0.0.1:$AppPort/api/upload" -Form $form
if (-not $upload.filename) {
throw "업로드 응답에 filename이 없습니다."
}
$uploadedPath = Join-Path $downloadsDir $upload.filename
if (-not (Test-Path -LiteralPath $uploadedPath)) {
throw "업로드된 파일이 downloads에 존재하지 않습니다."
}
Write-Step "selftest ok"
} finally {
Stop-TrackedProcess -Process $appProcess
Stop-TrackedProcess -Process $mockProcess
if ($appStdoutTask) {
$appStdoutTask.Wait()
$appStdoutTask.Result | Set-Content -LiteralPath (Join-Path $tmpRoot "app.log") -Encoding UTF8
}
if ($appStderrTask) {
$appStderrTask.Wait()
$appStderrTask.Result | Add-Content -LiteralPath (Join-Path $tmpRoot "app.log") -Encoding UTF8
}
}
-43
View File
@@ -1,43 +0,0 @@
param(
[switch]$SkipPythonDeps,
[switch]$SkipGoDownload,
[switch]$SkipFFmpegDownload,
[switch]$UpgradePip
)
Set-StrictMode -Version Latest
$ErrorActionPreference = "Stop"
. (Join-Path $PSScriptRoot "dev-tools.ps1")
Use-LocalTooling
Ensure-Directory -Path (Get-ToolsRoot)
Ensure-Directory -Path (Get-LocalRoot)
$pythonExe = Ensure-Venv
Write-Step "Python 사용 경로: $pythonExe"
if (-not $SkipPythonDeps) {
Install-PythonRequirements -UpgradePip:$UpgradePip
}
Ensure-CommandShims
if (-not $SkipGoDownload -and -not (Test-GoReady)) {
Install-LocalGo
}
if (-not $SkipFFmpegDownload -and -not (Test-FFmpegReady)) {
Install-LocalFFmpeg
}
Use-LocalTooling
Write-Step "준비 상태"
Write-Host (" python3: " + ((Resolve-SystemCommand -Name "python3") ?? "미설치"))
Write-Host (" python : " + ((Resolve-SystemCommand -Name "python") ?? "미설치"))
Write-Host (" yt-dlp : " + ((Resolve-SystemCommand -Name "yt-dlp") ?? "미설치"))
Write-Host (" go : " + ((Resolve-SystemCommand -Name "go") ?? "미설치"))
Write-Host (" ffmpeg : " + ((Resolve-SystemCommand -Name "ffmpeg") ?? "미설치"))
Write-Step "현재 셸에 PATH를 적용하려면 다음처럼 dot-source 하세요."
Write-Host " . .\scripts\enter-dev-shell.ps1"
+8 -1
View File
@@ -15,9 +15,16 @@
<Icon>https://raw.githubusercontent.com/selfhst/icons/main/png/google-gemini-light.png</Icon> <Icon>https://raw.githubusercontent.com/selfhst/icons/main/png/google-gemini-light.png</Icon>
<Config Name="WebUI Port" Target="8080" Default="8080" Mode="tcp" Description="Dashboard port" Type="Port" Display="always" Required="true" Mask="false">8080</Config> <Config Name="WebUI Port" Target="8080" Default="8080" Mode="tcp" Description="Dashboard port" Type="Port" Display="always" Required="true" Mask="false">8080</Config>
<Config Name="Downloads" Target="/app/downloads" Default="/mnt/user/appdata/ai-media-hub/downloads" Mode="rw" Description="Media output directory" Type="Path" Display="always" Required="true" Mask="false">/mnt/user/appdata/ai-media-hub/downloads</Config> <Config Name="Downloads" Target="/app/downloads" Default="/mnt/user/appdata/ai-media-hub/downloads" Mode="rw" Description="Media output directory" Type="Path" Display="always" Required="true" Mask="false">/mnt/user/appdata/ai-media-hub/downloads</Config>
<Config Name="GIPHY Downloads" Target="/app/downloads/giphy" Default="/mnt/user/appdata/ai-media-hub/giphy" Mode="rw" Description="Directory for downloaded GIPHY images and GIFs" Type="Path" Display="always" Required="true" Mask="false">/mnt/user/appdata/ai-media-hub/giphy</Config>
<Config Name="Database" Target="/app/db" Default="/mnt/user/appdata/ai-media-hub/db" Mode="rw" Description="SQLite database directory" Type="Path" Display="always" Required="true" Mask="false">/mnt/user/appdata/ai-media-hub/db</Config> <Config Name="Database" Target="/app/db" Default="/mnt/user/appdata/ai-media-hub/db" Mode="rw" Description="SQLite database directory" Type="Path" Display="always" Required="true" Mask="false">/mnt/user/appdata/ai-media-hub/db</Config>
<Config Name="SearXNG Base URL" Target="SEARXNG_BASE_URL" Default="http://searxng:8080" Mode="" Description="Base URL for the SearXNG instance" Type="Variable" Display="always" Required="true" Mask="false">http://searxng:8080</Config> <Config Name="SearXNG Base URL" Target="SEARXNG_BASE_URL" Default="http://searxng:8080" Mode="" Description="Base URL for the SearXNG instance" Type="Variable" Display="always" Required="true" Mask="false">http://searxng:8080</Config>
<Config Name="SearXNG Google Video Engine" Target="SEARXNG_GOOGLE_VIDEO_ENGINE" Default="google videos" Mode="" Description="Engine name used for Google video searches" Type="Variable" Display="always" Required="true" Mask="false">google videos</Config> <Config Name="SearXNG Google Video Engine" Target="SEARXNG_GOOGLE_VIDEO_ENGINE" Default="google videos" Mode="" Description="Engine name used for Google video searches" Type="Variable" Display="always" Required="true" Mask="false">google videos</Config>
<Config Name="SearXNG Web Engine" Target="SEARXNG_WEB_ENGINE" Default="google" Mode="" Description="Engine name used for Envato and Artgrid searches" Type="Variable" Display="always" Required="true" Mask="false">google</Config> <Config Name="SearXNG Web Engine" Target="SEARXNG_WEB_ENGINE" Default="google" Mode="" Description="Engine name used for Envato and Artgrid searches" Type="Variable" Display="always" Required="true" Mask="false">google</Config>
<Config Name="Gemini API Key" Target="GEMINI_API_KEY" Default="" Mode="" Description="Gemini API key" Type="Variable" Display="always" Required="true" Mask="true"/> <Config Name="GIPHY Enabled" Target="GIPHY_ENABLED" Default="true" Mode="" Description="Enable GIPHY image and GIF search" Type="Variable" Display="always" Required="true" Mask="false">true</Config>
<Config Name="GIPHY API Key" Target="GIPHY_API_KEY" Default="" Mode="" Description="GIPHY API key" Type="Variable" Display="always" Required="false" Mask="true"/>
<Config Name="GIPHY Max Results" Target="GIPHY_MAX_RESULTS" Default="100" Mode="" Description="Maximum number of aggregated GIPHY results to return" Type="Variable" Display="always" Required="true" Mask="false">100</Config>
<Config Name="GIPHY Rating" Target="GIPHY_RATING" Default="g" Mode="" Description="GIPHY content rating filter" Type="Variable" Display="always" Required="true" Mask="false">g</Config>
<Config Name="GIPHY Lang" Target="GIPHY_LANG" Default="en" Mode="" Description="Language hint sent to GIPHY search" Type="Variable" Display="always" Required="true" Mask="false">en</Config>
<Config Name="Gemini API Key" Target="GEMINI_API_KEY" Default="" Mode="" Description="Gemini API key" Type="Variable" Display="always" Required="false" Mask="true"/>
<Config Name="Gemini Model" Target="GEMINI_MODEL" Default="gemini-2.5-flash" Mode="" Description="Gemini model used for multilingual query expansion" Type="Variable" Display="always" Required="true" Mask="false">gemini-2.5-flash</Config>
</Container> </Container>