Expand preview handling and search coverage

TODO.md

@@ -32,6 +32,7 @@
 - A fresh-machine bootstrap was revalidated in a user-local toolchain setup on `2026-03-17`; `go test ./...` and `bash scripts/selftest.sh` now pass in that setup.
 - Result modal sizing is now being constrained to the viewport, and modal-only source-summary translation is now part of the active implementation path.
 - Card summaries now also translate lazily to Korean, and Gemini negative-assessment handling now drives stronger follow-up search behavior than before.
+- Search preview delivery is now moving away from persistent on-disk preview caching toward live proxy / live transcode behavior, with Google Video preview reuse added to result cards and modal playback.
 
 ## Current Architecture
 - `backend/main.go`
@@ -229,6 +230,7 @@
 - Gemini notes are now intended to be Korean, but final output quality still depends on Gemini response consistency.
 - Source Summary translation now depends on Google Translate HTTP availability; frontend silently falls back to original summary text if translation fails.
 - The result modal should now stay within viewport height, but this still needs real browser confirmation on multiple short-height displays because CSS-only constraints were the source of the latest user-visible regression.
+- Artgrid preview playback now has a server-side ffmpeg transcode path for `.m3u8` style preview URLs, but this trades storage savings for runtime CPU cost.
 - The local self-test script is better than before, but it is still a smoke test, not full integration coverage.
 
 ## Current Risks Around Search Quality
@@ -553,6 +555,7 @@
 - [ ] Improve Envato / Artgrid preview acquisition reliability so Gemini Vision sees real frames more often
 - [ ] Browser-verify the new result modal at multiple viewport heights and confirm translated Source Summary readability on real long descriptions
 - [ ] Evaluate whether the new Gemini supplemental-query generation is reducing irrelevant results on a small fixed benchmark query set
+- [ ] Measure runtime cost of live Artgrid preview transcoding and decide whether bounded in-memory throttling or concurrency caps are needed
 - [ ] Revisit Google Video UX:
   - current YouTube embed was abandoned due error `153`
   - current in-app panel is more reliable but less rich than a true embedded watch page
@@ -618,6 +621,24 @@
 - If behavior in the browser does not match the latest backend/frontend code, the first assumption should be stale frontend assets until proven otherwise
 
 ## Recent Change Log
+- Date: `2026-03-17`
+- What changed:
+  - If the first search pass plus Gemini filtering still leaves too few visible results, the backend now performs an additional coverage-expansion search/evaluation pass before final fallback filling.
+  - Search result cards and the result modal now reuse the existing direct-download preview probe path for Google Video, so YouTube-backed results can surface actual playable preview streams instead of staying thumbnail-only.
+  - Artgrid-style `.m3u8` previews now have a server-side ffmpeg transcode route so the frontend can request a directly playable MP4 stream when plain HLS playback is unreliable.
+  - Persistent preview-file caching to disk was removed from the preview proxy path so one-off preview traffic does not keep accumulating files under the downloads cache area.
+  - Modal spacing, media height, and text sizing were tightened again so the popup is more likely to fit without scrolling on shorter displays.
+- Why it changed:
+  - The latest user feedback said the search result count had become too low after stricter Gemini filtering, the popup still felt too large, Google Video already had a better preview path in direct-download mode, Artgrid previews still were not reliably playable, and the server should prefer not retaining disposable preview artifacts on disk.
+- How it was verified:
+  - `go test ./...`
+  - `bash scripts/selftest.sh`
+  - `python3 -m py_compile worker/downloader.py scripts/mock_searxng.py`
+- What is still risky or incomplete:
+  - The extra coverage-expansion pass can improve visible count but may increase latency when upstream SearXNG quality is poor.
+  - Live ffmpeg transcoding avoids preview-file accumulation but may become CPU-heavy under concurrent preview playback.
+  - Real browser validation is still needed for the exact viewport in the latest screenshot and for actual Artgrid preview playback behavior.
+
 - Date: `2026-03-17`
 - What changed:
   - Removed the visible `AI Recommended` badge from search cards.

backend/handlers/api.go

@@ -3,7 +3,6 @@ package handlers
 import (
 	"bufio"
 	"bytes"
-	"crypto/sha1"
 	"database/sql"
 	"encoding/json"
 	"errors"
@@ -144,6 +143,7 @@ func RegisterRoutes(router *gin.Engine, app *App) {
 	router.GET("/ws", app.handleWS)
 	router.GET("/api/history/check", app.checkDuplicate)
 	router.GET("/api/preview/stream", app.streamPreview)
+	router.GET("/api/preview/transcode", app.transcodePreview)
 	router.POST("/api/download/preview", app.previewDownload)
 	router.POST("/api/upload", app.uploadFile)
 	router.POST("/api/download", app.startDownload)
@@ -236,15 +236,6 @@ func (a *App) streamPreview(c *gin.Context) {
 		return
 	}
 
-	if isCacheablePreview(target, contentType) {
-		if cachedPath, err := a.cachePreviewResponse(target, contentType, resp.Body); err == nil {
-			a.debug("preview:proxy:cache_hit_write", gin.H{"target": target, "cachedPath": cachedPath})
-			c.File(cachedPath)
-			return
-		}
-		a.debug("preview:proxy:cache_write_error", gin.H{"target": target, "error": err.Error()})
-	}
-
 	if contentType != "" {
 		c.Header("Content-Type", contentType)
 	}
@@ -252,6 +243,61 @@ func (a *App) streamPreview(c *gin.Context) {
 	_, _ = io.Copy(c.Writer, resp.Body)
 }
 
+func (a *App) transcodePreview(c *gin.Context) {
+	target := strings.TrimSpace(c.Query("url"))
+	if target == "" {
+		c.JSON(http.StatusBadRequest, gin.H{"error": "url is required"})
+		return
+	}
+
+	headers := fmt.Sprintf("User-Agent: Mozilla/5.0\r\nReferer: %s\r\n", inferPreviewReferer(target))
+	cmd := exec.CommandContext(
+		c.Request.Context(),
+		"ffmpeg",
+		"-hide_banner",
+		"-loglevel", "error",
+		"-headers", headers,
+		"-i", target,
+		"-an",
+		"-vf", "scale='min(1280,iw)':-2",
+		"-c:v", "libx264",
+		"-preset", "veryfast",
+		"-crf", "30",
+		"-movflags", "frag_keyframe+empty_moov+faststart",
+		"-f", "mp4",
+		"pipe:1",
+	)
+	stdout, err := cmd.StdoutPipe()
+	if err != nil {
+		c.JSON(http.StatusBadGateway, gin.H{"error": err.Error()})
+		return
+	}
+	stderr, err := cmd.StderrPipe()
+	if err != nil {
+		c.JSON(http.StatusBadGateway, gin.H{"error": err.Error()})
+		return
+	}
+	if err := cmd.Start(); err != nil {
+		c.JSON(http.StatusBadGateway, gin.H{"error": err.Error()})
+		return
+	}
+
+	c.Header("Content-Type", "video/mp4")
+	c.Status(http.StatusOK)
+	_, copyErr := io.Copy(c.Writer, stdout)
+	errOutput, _ := io.ReadAll(io.LimitReader(stderr, 2048))
+	waitErr := cmd.Wait()
+	if copyErr != nil {
+		a.debug("preview:transcode:copy_error", gin.H{"target": target, "error": copyErr.Error()})
+		return
+	}
+	if waitErr != nil {
+		a.debug("preview:transcode:error", gin.H{"target": target, "error": waitErr.Error(), "stderr": strings.TrimSpace(string(errOutput))})
+		return
+	}
+	a.debug("preview:transcode:complete", gin.H{"target": target})
+}
+
 func (a *App) uploadFile(c *gin.Context) {
 	file, err := c.FormFile("file")
 	if err != nil {
@@ -508,15 +554,45 @@ func (a *App) searchMedia(c *gin.Context) {
 		return
 	}
 
-	merged := services.MergeRecommendations(recommended, scored, 16)
+	targetCount := 16
+	merged := services.MergeRecommendations(recommended, scored, targetCount)
 	if geminiErr != nil {
 		merged = services.BackfillRecommendations(
 			merged,
 			scored,
-			16,
+			targetCount,
 			"Gemini 배치 일부가 실패해 미리보기 가능한 상위 후보를 보강했습니다.",
 		)
 	}
+	if len(merged) < targetCount && time.Now().Before(deadline.Add(-5*time.Second)) {
+		coverageQueries := buildCoverageQueries(req.Query, queryVariants, recommended, merged)
+		if len(coverageQueries) > 0 {
+			a.debug("search coverage query variants", gin.H{"variants": coverageQueries, "variantCount": len(coverageQueries), "existingCount": len(merged)})
+			extraResults, extraMeta, extraErr := a.SearchService.SearchMediaWithDeadline(coverageQueries, enabledPlatforms, deadline.Add(-5*time.Second))
+			supplementalDeadlineLimited = supplementalDeadlineLimited || extraMeta.PartialDueToDeadline
+			if extraErr == nil && len(extraResults) > 0 {
+				results = mergeSearchResults(results, extraResults)
+				scored = services.RankSearchResults(strings.Join(coverageQueries[:min(len(coverageQueries), 3)], " "), results)
+				reviewedLinks := services.ReviewedRecommendationLinks(recommended)
+				supplementalCandidates := services.SelectUnevaluatedCandidates(scored, reviewedLinks, services.RemainingGeminiCapacity(recommended))
+				if len(supplementalCandidates) > 0 {
+					extraRecommended, extraStats, extraGeminiErr := services.EvaluateAllCandidatesWithGeminiWithDeadline(
+						a.GeminiService,
+						req.Query,
+						supplementalCandidates,
+						deadline.Add(-2*time.Second),
+					)
+					recommended = services.MergeUniqueRecommendations(recommended, extraRecommended)
+					geminiStats = services.MergeGeminiBatchStats(geminiStats, extraStats)
+					geminiErr = combineSearchWarnings(geminiErr, extraGeminiErr)
+				}
+				merged = services.MergeRecommendations(recommended, scored, targetCount)
+			}
+		}
+	}
+	if len(merged) < targetCount {
+		merged = services.BackfillRecommendations(merged, scored, targetCount, "추가 검색 후에도 충분한 결과가 부족해 시각 자산이 있는 후보를 제한적으로 보강했습니다.")
+	}
 	merged = services.RandomizeTopRecommendations(merged, 6)
 	for idx := range merged {
 		merged[idx] = services.DecorateRecommendationMedia(merged[idx])
@@ -676,6 +752,31 @@ func mergeSearchResults(base, extra []services.SearchResult) []services.SearchRe
 	return merged
 }
 
+func buildCoverageQueries(query string, existing []string, reviewed []services.AIRecommendation, merged []services.AIRecommendation) []string {
+	candidates := append([]string{}, existing...)
+	positiveHints := 0
+	for _, item := range reviewed {
+		if item.Assessment == "positive" && item.SearchHint != "" && positiveHints < 3 {
+			candidates = append(candidates, item.SearchHint)
+			positiveHints++
+		}
+	}
+	if len(merged) < 8 {
+		candidates = append(candidates,
+			query+" stock footage",
+			query+" lifestyle footage",
+			query+" candid couple footage",
+			query+" editorial scene",
+		)
+	} else {
+		candidates = append(candidates,
+			query+" establishing shot",
+			query+" cinematic b-roll",
+		)
+	}
+	return mergeSupplementalQuerySets(nil, candidates)
+}
+
 func combineSearchWarnings(base, extra error) error {
 	switch {
 	case base == nil:
@@ -843,39 +944,6 @@ func rewriteM3U8Playlist(body, target string) string {
 	return strings.Join(lines, "\n")
 }
 
-func isCacheablePreview(target, contentType string) bool {
-	lower := strings.ToLower(target + " " + contentType)
-	return strings.Contains(lower, ".mp4") || strings.Contains(lower, "video/mp4")
-}
-
-func (a *App) cachePreviewResponse(target, contentType string, body io.Reader) (string, error) {
-	if a.PreviewCacheDir == "" {
-		return "", fmt.Errorf("preview cache dir is not configured")
-	}
-	if err := os.MkdirAll(a.PreviewCacheDir, 0o755); err != nil {
-		return "", err
-	}
-
-	sum := sha1.Sum([]byte(target))
-	ext := ".bin"
-	if strings.Contains(strings.ToLower(target), ".mp4") || strings.Contains(strings.ToLower(contentType), "video/mp4") {
-		ext = ".mp4"
-	}
-	path := filepath.Join(a.PreviewCacheDir, fmt.Sprintf("%x%s", sum, ext))
-	if _, err := os.Stat(path); err == nil {
-		return path, nil
-	}
-	file, err := os.Create(path)
-	if err != nil {
-		return "", err
-	}
-	defer file.Close()
-	if _, err := io.Copy(file, body); err != nil {
-		return "", err
-	}
-	return path, nil
-}
-
 func summarizeOutput(prefix string, output []byte, err error) string {
 	trimmed := strings.TrimSpace(string(output))
 	if trimmed == "" && err != nil {

frontend/app.js

@@ -86,6 +86,8 @@ const hlsInstances = new WeakMap();
 const debugEntries = [];
 const summaryTranslationCache = new Map();
 const summaryTranslationInflight = new Map();
+const resultPreviewCache = new Map();
+const resultPreviewInflight = new Map();
 let cardSummaryObserver = null;
 const PREVIEW_PLACEHOLDER = "https://placehold.co/1280x720/0a0a0a/ffffff?text=Preview";
 
@@ -96,6 +98,25 @@ function proxiedPreviewURL(src) {
   return `/api/preview/stream?url=${encodeURIComponent(src)}`;
 }
 
+function transcodedPreviewURL(src) {
+  if (!src) {
+    return "";
+  }
+  return `/api/preview/transcode?url=${encodeURIComponent(src)}`;
+}
+
+function buildPlayablePreviewURL(src, source = "") {
+  const trimmed = String(src || "").trim();
+  if (!trimmed) {
+    return "";
+  }
+  const lower = trimmed.toLowerCase();
+  if (lower.includes(".m3u8") && (String(source || "").toLowerCase() === "artgrid" || lower.includes("artgrid") || lower.includes("artlist"))) {
+    return transcodedPreviewURL(trimmed);
+  }
+  return proxiedPreviewURL(trimmed);
+}
+
 function isLowValueThumbnailURL(src) {
   const lower = String(src || "").toLowerCase();
   if (!lower) {
@@ -471,7 +492,7 @@ function showResultModalVideo(src) {
   if (!src) {
     return;
   }
-  attachVideoSource(resultModalVideo, proxiedPreviewURL(src));
+  attachVideoSource(resultModalVideo, src);
   setHidden(resultModalVideo, false, "");
 }
 
@@ -555,6 +576,37 @@ async function translateCardSummary(node) {
   }
 }
 
+async function fetchResultPreview(item) {
+  const key = String(item?.link || "").trim();
+  if (!key) {
+    return null;
+  }
+  if (resultPreviewCache.has(key)) {
+    return resultPreviewCache.get(key);
+  }
+  if (resultPreviewInflight.has(key)) {
+    return resultPreviewInflight.get(key);
+  }
+  const request = (async () => {
+    try {
+      const preview = await api("/api/download/preview", {
+        method: "POST",
+        headers: { "Content-Type": "application/json" },
+        body: JSON.stringify({ url: key }),
+      });
+      resultPreviewCache.set(key, preview);
+      return preview;
+    } catch (error) {
+      logEvent("result:preview:fetch_failed", { link: key, source: item?.source || "", message: error.message });
+      return null;
+    } finally {
+      resultPreviewInflight.delete(key);
+    }
+  })();
+  resultPreviewInflight.set(key, request);
+  return request;
+}
+
 function ensureCardSummaryObserver() {
   if (cardSummaryObserver || typeof IntersectionObserver === "undefined") {
     return;
@@ -618,12 +670,20 @@ function renderResults(results) {
     node.dataset.summaryTranslated = "false";
     node.addEventListener("click", () => openResultModal(item));
     previewVideo.poster = usableThumbnail ? item.thumbnailUrl : "";
-    if (item.previewVideoUrl) {
-      const mediaArea = node.querySelector(".relative");
-      mediaArea.addEventListener("mouseenter", () => {
-        logEvent("preview:hover:start", { title: item.title, source: item.source, previewVideoUrl: item.previewVideoUrl });
+    const mediaArea = node.querySelector(".relative");
+    if (item.previewVideoUrl || item.source === "Google Video") {
+      mediaArea.addEventListener("mouseenter", async () => {
+        let previewURL = item.previewVideoUrl || "";
+        if (!previewURL && item.source === "Google Video") {
+          const preview = await fetchResultPreview(item);
+          previewURL = preview?.previewStreamUrl || "";
+        }
+        logEvent("preview:hover:start", { title: item.title, source: item.source, previewVideoUrl: previewURL });
+        if (!previewURL) {
+          return;
+        }
         overlays.forEach((overlay) => overlay.classList.add("hidden"));
-        startHoverPreview(previewVideo, proxiedPreviewURL(item.previewVideoUrl));
+        startHoverPreview(previewVideo, buildPlayablePreviewURL(previewURL, item.source));
       });
       mediaArea.addEventListener("mouseleave", () => {
         logEvent("preview:hover:end", { title: item.title, source: item.source });
@@ -666,7 +726,7 @@ async function prepareDirectDownload(targetUrl) {
   downloadResult.textContent = "preview loaded";
 }
 
-function openResultModal(item) {
+async function openResultModal(item) {
   if (!resultModalReady) {
     logEvent("result:modal:error", { message: "result modal is not fully initialized" });
     return;
@@ -688,7 +748,14 @@ function openResultModal(item) {
   resetResultModalMedia();
   const embedURL = buildResultModalEmbedURL(item);
   const fallbackReason = item.previewBlockedReason || "Embedded view was unavailable, switched to fallback preview.";
-  if (item.source === "Google Video" && item.mediaMode === "thumbnail") {
+  let resolvedPreviewURL = item.previewVideoUrl || "";
+  if (!resolvedPreviewURL && item.source === "Google Video") {
+    const preview = await fetchResultPreview(item);
+    resolvedPreviewURL = preview?.previewStreamUrl || "";
+  }
+  if (resolvedPreviewURL) {
+    showResultModalVideo(buildPlayablePreviewURL(resolvedPreviewURL, item.source));
+  } else if (item.source === "Google Video" && item.mediaMode === "thumbnail") {
     showResultModalGooglePanel(item, item.snippet || "Open source page or download directly.");
   } else if (item.mediaMode === "embed" && embedURL && embedURL !== "about:blank") {
     showResultModalFrame(embedURL);
@@ -703,7 +770,7 @@ function openResultModal(item) {
       window.clearTimeout(timeout);
     };
   } else if (item.mediaMode === "preview_video" && item.previewVideoUrl) {
-    showResultModalVideo(item.previewVideoUrl);
+    showResultModalVideo(buildPlayablePreviewURL(item.previewVideoUrl, item.source));
   } else if (item.mediaMode === "thumbnail" && hasUsableThumbnail(item.thumbnailUrl)) {
     showResultModalThumbnail(item.thumbnailUrl, item.title || "");
   } else {

frontend/index.html

@@ -151,7 +151,7 @@
 
     <div id="resultModal" class="fixed inset-0 z-50 hidden items-start justify-center overflow-y-auto bg-black/80 px-2 py-2 sm:px-4 sm:py-4">
       <div class="result-modal-shell flex w-full max-w-6xl min-h-0 flex-col overflow-hidden rounded-3xl border border-white/10 bg-zinc-950 shadow-2xl">
-        <div class="flex items-center justify-between border-b border-white/10 px-5 py-4">
+        <div class="flex items-center justify-between border-b border-white/10 px-4 py-3 sm:px-5 sm:py-4">
           <div class="min-w-0">
             <p id="resultModalSource" class="text-xs uppercase tracking-[0.25em] text-zinc-500"></p>
             <h3 id="resultModalTitle" class="mt-1 truncate text-xl font-semibold text-white"></h3>
@@ -161,7 +161,7 @@
             <button id="closeResultModal" type="button" class="rounded-full border border-white/10 px-3 py-2 text-xs uppercase tracking-[0.2em] text-zinc-300">Close</button>
           </div>
         </div>
-        <div class="border-b border-white/10 bg-black/40 p-3 sm:p-4">
+        <div class="border-b border-white/10 bg-black/40 p-2 sm:p-3">
           <div id="resultModalMediaFrame" class="result-modal-media-frame aspect-video overflow-hidden rounded-2xl border border-white/10 bg-black">
             <iframe id="resultModalFrame" class="hidden h-full w-full bg-white" referrerpolicy="no-referrer" allow="autoplay; fullscreen; encrypted-media; picture-in-picture" allowfullscreen></iframe>
             <video id="resultModalVideo" class="hidden h-full w-full bg-black object-contain" controls playsinline></video>
@@ -179,15 +179,15 @@
             </div>
           </div>
         </div>
-        <div class="result-modal-details grid min-h-0 gap-4 px-4 py-4 sm:gap-5 sm:px-5 sm:py-5 lg:grid-cols-[1.6fr_0.8fr]">
-          <div class="flex min-h-[220px] min-w-0 flex-col rounded-2xl border border-white/10 bg-white/[0.03] p-5">
+        <div class="result-modal-details grid min-h-0 gap-3 px-3 py-3 sm:gap-4 sm:px-4 sm:py-4 lg:grid-cols-[1.5fr_0.8fr]">
+          <div class="flex min-h-[180px] min-w-0 flex-col rounded-2xl border border-white/10 bg-white/[0.03] p-4">
             <p class="text-xs uppercase tracking-[0.25em] text-zinc-500">AI Note</p>
             <div class="result-panel-scroll mt-3 min-h-0 flex-1 overflow-y-auto pr-2">
-              <p id="resultModalReason" class="whitespace-pre-wrap text-sm leading-7 text-zinc-200"></p>
+              <p id="resultModalReason" class="whitespace-pre-wrap text-xs leading-6 text-zinc-200 sm:text-sm"></p>
             </div>
           </div>
-          <div class="flex min-h-[240px] min-w-0 flex-col rounded-2xl border border-white/10 bg-white/[0.03] p-5">
-            <div class="mb-4 flex flex-col gap-3">
+          <div class="flex min-h-[200px] min-w-0 flex-col rounded-2xl border border-white/10 bg-white/[0.03] p-4">
+            <div class="mb-3 flex flex-col gap-2">
               <button id="resultModalDownload" type="button" class="hidden w-full rounded-2xl border border-white bg-white px-4 py-3 text-sm font-medium text-black transition hover:bg-zinc-200">
                 Primary Action
               </button>
@@ -198,7 +198,7 @@
             <div class="flex min-h-0 flex-1 flex-col">
               <p class="text-xs uppercase tracking-[0.25em] text-zinc-500">Source Summary</p>
               <div class="result-summary-scroll mt-3 min-h-0 flex-1 overflow-y-auto pr-2">
-                <p id="resultModalSnippet" class="text-sm leading-7 text-zinc-300"></p>
+                <p id="resultModalSnippet" class="text-xs leading-6 text-zinc-300 sm:text-sm"></p>
               </div>
             </div>
           </div>
@@ -224,6 +224,6 @@
       </button>
     </template>
 
-    <script src="/app.js?v=20260317b" defer></script>
+    <script src="/app.js?v=20260317c" defer></script>
   </body>
 </html>

frontend/style.css

@@ -60,7 +60,7 @@ body {
 }
 
 .result-modal-shell {
-  height: min(calc(100dvh - 1rem), 920px);
+  height: min(calc(100dvh - 0.5rem), 860px);
   margin: auto;
 }
 
@@ -69,7 +69,7 @@ body {
 }
 
 .result-modal-media-frame {
-  max-height: min(42dvh, 28rem);
+  max-height: min(34dvh, 22rem);
 }
 
 .result-modal-details {
@@ -110,16 +110,16 @@ body {
 
 @media (max-height: 900px) {
   .result-modal-media-frame {
-    max-height: min(34dvh, 22rem);
+    max-height: min(28dvh, 18rem);
   }
 }
 
 @media (max-height: 720px) {
   .result-modal-shell {
-    height: min(calc(100dvh - 0.5rem), 780px);
+    height: min(calc(100dvh - 0.25rem), 700px);
   }
 
   .result-modal-media-frame {
-    max-height: min(28dvh, 15rem);
+    max-height: min(22dvh, 11rem);
   }
 }