Files

T

vincent 4c877fd7f9 Add FlightTube implementation plan

2026-08-29 02:19:23 +02:00

21 KiB

Raw Permalink Blame History

FlightTube Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: A Tauri desktop app that merges the user's YouTube subscriptions into one newest-first feed, downloads videos locally, and shows only downloaded videos when offline.

Architecture: Rust owns all I/O — Atom feed fetching, SQLite, the yt-dlp subprocess — and streams progress to the frontend over Tauri events. React renders and dispatches commands, holding no business logic. Pure parsing functions (CSV, XML, progress lines) are separated from I/O so they are unit-testable without a network or a subprocess.

Tech Stack: Tauri 2.11 · Rust 1.98 · React 19.2 · TypeScript · Vite 8.2 · Tailwind CSS 4.3 (@tailwindcss/vite, no config file) · rusqlite 0.40 (bundled) · quick-xml 0.42 · reqwest 0.13 · tokio 1.53 · csv 1.4 · chrono 0.4

Spec: docs/superpowers/specs/2026-08-29-flighttube-design.md

Global Constraints

Download format is fixed: -f "bv*[vcodec^=avc1]+ba[acodec^=mp4a]/bv*+ba/b" --merge-output-format mp4. Validated live; resolves to 1920x1080 avc1.640028 + mp4a.40.2. Never widen this — non-H.264 will not play in WKWebView, and in-app playback is a hard requirement.
No OAuth, no API keys, no secrets on disk. Metadata comes only from the public Atom endpoint https://www.youtube.com/feeds/videos.xml?channel_id=<id>.
Atom feeds return ~15 entries per channel. No backfill exists. Never write code that assumes deeper history.
Refresh must upsert, never delete-and-reinsert: download state lives alongside video rows and must survive a refresh.
Thumbnails must be cached to disk on refresh, or the offline feed renders broken images.
Feed concurrency 8; download concurrency 2.
Library path: ~/Movies/FlightTube, user-configurable.
Timestamps are stored as Unix seconds (INTEGER) so sorting is an indexed ORDER BY.
Verified toolchain: yt-dlp 2026.08.19, ffmpeg 9.0.1, rustc 1.98.0, Node 22.22.2.

File Structure

src-tauri/src/
  main.rs          entry, builder, state registration
  db.rs            schema, migrations, all SQL
  models.rs        Channel, Video, DownloadState, FeedFilter — shared types
  takeout.rs       subscriptions.csv -> Vec<Channel>          (pure + tested)
  feed.rs          Atom XML -> Vec<Video>, plus fetch          (parse is pure + tested)
  thumbs.rs        thumbnail download + disk cache
  downloader.rs    yt-dlp spawn, progress parsing              (parse is pure + tested)
  net.rs           connectivity probe
  commands.rs      Tauri command surface — delegates only
  fixtures/ltt_feed.xml   captured real Atom response for tests

src/
  main.tsx         React root
  App.tsx          layout, routing between feed and player
  api.ts           typed wrappers over invoke() + event listeners
  types.ts         TS mirrors of models.rs
  hooks/useFeed.ts        feed data + filters
  hooks/useDownloads.ts   live download progress map
  hooks/useConnectivity.ts
  components/Sidebar.tsx
  components/TopBar.tsx
  components/VideoRow.tsx
  components/DownloadButton.tsx
  components/Player.tsx
  components/Settings.tsx
  index.css        @import "tailwindcss" + theme tokens

Task 1: Scaffold and boot

Files:

Create: package.json, vite.config.ts, tsconfig.json, index.html, src/main.tsx, src/App.tsx, src/index.css
Create: src-tauri/Cargo.toml, src-tauri/tauri.conf.json, src-tauri/src/main.rs, src-tauri/capabilities/default.json

Interfaces:

Consumes: nothing
Produces: a running Tauri window rendering Tailwind-styled React
Step 1: Scaffold

npm create tauri-app@latest . -- --template react-ts --manager npm --yes
npm install
npm install -D @tailwindcss/vite tailwindcss

Step 2: Wire Tailwind v4 into Vite

vite.config.ts — add the plugin (v4 needs no tailwind.config.js):

import tailwindcss from "@tailwindcss/vite";
export default defineConfig({ plugins: [react(), tailwindcss()], /* keep tauri server block */ });

src/index.css — replace entire contents:

@import "tailwindcss";
@theme {
  --color-ink: #0f0f0f;
  --color-surface: #181818;
  --color-edge: #303030;
}

Step 3: Prove Tailwind renders

src/App.tsx:

export default function App() {
  return <div className="min-h-screen bg-ink text-white grid place-items-center">
    <h1 className="text-3xl font-semibold">FlightTube</h1>
  </div>;
}

Step 4: Run and verify

Run: npm run tauri dev Expected: a native window, dark #0f0f0f background, centered white "FlightTube". If the background is white, Tailwind is not wired — fix before continuing.

Step 5: Commit

git add -A && git commit -m "feat: scaffold Tauri + React + Tailwind v4"

Task 2: Takeout CSV parsing

Files:

Create: src-tauri/src/models.rs, src-tauri/src/takeout.rs
Modify: src-tauri/Cargo.toml (add csv, serde)

Interfaces:

Consumes: nothing
Produces: Channel { id: String, title: String, url: String } and takeout::parse_csv(input: &str) -> Result<Vec<Channel>, String>

Real Takeout header is Channel Id,Channel Url,Channel Title. Parse by header name, not position — column order has changed across Takeout versions and locales.

Step 1: Write failing tests

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn parses_standard_export() {
        let csv = "Channel Id,Channel Url,Channel Title\n\
                   UCabc,http://www.youtube.com/channel/UCabc,Linus Tech Tips\n";
        let out = parse_csv(csv).unwrap();
        assert_eq!(out.len(), 1);
        assert_eq!(out[0].id, "UCabc");
        assert_eq!(out[0].title, "Linus Tech Tips");
    }

    #[test]
    fn handles_commas_and_unicode_in_titles() {
        let csv = "Channel Id,Channel Url,Channel Title\n\
                   UCx,http://y.com/UCx,\"Kurzgesagt – In a Nutshell, Ltd\"\n";
        let out = parse_csv(csv).unwrap();
        assert_eq!(out[0].title, "Kurzgesagt – In a Nutshell, Ltd");
    }

    #[test]
    fn tolerates_reordered_columns() {
        let csv = "Channel Title,Channel Id,Channel Url\nVeritasium,UCz,http://y.com/UCz\n";
        let out = parse_csv(csv).unwrap();
        assert_eq!(out[0].id, "UCz");
        assert_eq!(out[0].title, "Veritasium");
    }

    #[test]
    fn skips_blank_lines_and_empty_file() {
        assert!(parse_csv("Channel Id,Channel Url,Channel Title\n\n").unwrap().is_empty());
        assert!(parse_csv("").is_err());
    }

    #[test]
    fn errors_on_missing_required_column() {
        assert!(parse_csv("Foo,Bar\n1,2\n").is_err());
    }
}

Step 2: Verify they fail

Run: cd src-tauri && cargo test takeout Expected: FAIL — parse_csv not found.

Step 3: Implement

Use csv::ReaderBuilder::new().flexible(true).from_reader(input.as_bytes()), read headers(), build a name→index map, error if Channel Id or Channel Title is absent, skip records whose id is empty, and derive url from the id when the URL column is missing.

Step 4: Verify they pass

Run: cd src-tauri && cargo test takeout Expected: 5 passed.

Step 5: Commit

git add -A && git commit -m "feat: parse Takeout subscriptions.csv"

Task 3: Atom feed parsing

Files:

Create: src-tauri/src/feed.rs, src-tauri/src/fixtures/ltt_feed.xml
Modify: src-tauri/Cargo.toml (add quick-xml with serde feature, reqwest with rustls-tls, tokio, chrono)

Interfaces:

Consumes: nothing
Produces:
- Video { id, channel_id, title, description, published: i64, thumb_url, views: i64, is_short: bool }
- feed::parse_atom(xml: &str) -> Result<Vec<Video>, String>
- async feed::fetch_channel(client: &reqwest::Client, channel_id: &str) -> Result<Vec<Video>, String>

A real response is already captured at /private/tmp/.../scratchpad/feed.xml — copy it to src-tauri/src/fixtures/ltt_feed.xml. Testing against a real payload rather than a hand-written one is the point.

Shorts are detected by the alternate link containing /shorts/. published is RFC 3339 → Unix seconds via chrono::DateTime::parse_from_rfc3339.

Step 1: Write failing tests

#[cfg(test)]
mod tests {
    use super::*;
    const FIXTURE: &str = include_str!("fixtures/ltt_feed.xml");

    #[test]
    fn parses_all_entries_from_real_feed() {
        let v = parse_atom(FIXTURE).unwrap();
        assert_eq!(v.len(), 15);
    }

    #[test]
    fn extracts_core_fields() {
        let v = parse_atom(FIXTURE).unwrap();
        let first = &v[0];
        assert_eq!(first.id, "tklAv8hcG9s");
        assert_eq!(first.channel_id, "UCXuqSBlHAE6Xw-yeJA0Tunw");
        assert!(first.title.contains("Linus"));
        assert!(first.published > 1_700_000_000);
        assert!(first.thumb_url.contains("tklAv8hcG9s"));
        assert!(first.views > 0);
    }

    #[test]
    fn detects_shorts() {
        let v = parse_atom(FIXTURE).unwrap();
        assert!(v[0].is_short, "first fixture entry is a /shorts/ link");
    }

    #[test]
    fn tolerates_missing_optional_fields() {
        let xml = r#"<feed xmlns="http://www.w3.org/2005/Atom"
          xmlns:yt="http://www.youtube.com/xml/schemas/2015"
          xmlns:media="http://search.yahoo.com/mrss/">
          <entry><yt:videoId>abc</yt:videoId><yt:channelId>UCq</yt:channelId>
            <title>Bare</title>
            <link rel="alternate" href="https://www.youtube.com/watch?v=abc"/>
            <published>2026-01-01T00:00:00+00:00</published>
          </entry></feed>"#;
        let v = parse_atom(xml).unwrap();
        assert_eq!(v[0].views, 0);
        assert_eq!(v[0].description, "");
        assert!(!v[0].is_short);
    }

    #[test]
    fn errors_on_malformed_xml() {
        assert!(parse_atom("<feed><entry>").is_err());
    }

    #[test]
    fn returns_empty_for_feed_with_no_entries() {
        let xml = r#"<feed xmlns="http://www.w3.org/2005/Atom"><title>Empty</title></feed>"#;
        assert!(parse_atom(xml).unwrap().is_empty());
    }
}

Step 2: Verify they fail

Run: cd src-tauri && cargo test feed

Step 3: Implement

Event-based quick_xml::Reader walking <entry> elements. Track the current element name; capture text for yt:videoId, yt:channelId, title, published, media:description; read the href attribute of link[rel=alternate] and the url attribute of media:thumbnail; read views from media:statistics. Ignore unknown elements. Return Err on quick_xml errors so malformed XML fails loudly.

Step 4: Verify they pass

Run: cd src-tauri && cargo test feed Expected: 6 passed.

Step 5: Commit

git add -A && git commit -m "feat: parse YouTube channel Atom feeds"

Task 4: yt-dlp progress parsing

Files:

Create: src-tauri/src/downloader.rs

Interfaces:

Consumes: nothing
Produces: Progress { downloaded: u64, total: Option<u64>, speed: Option<f64>, eta: Option<u64> } and downloader::parse_progress_line(line: &str) -> Option<Progress>

yt-dlp emits NA for unknown numeric fields, and interleaves ordinary log output with progress lines. The FTPROG sentinel disambiguates. A line that fails to parse must return None, never panic — a malformed progress line must not kill a download.

Step 1: Write failing tests

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn parses_complete_line() {
        let p = parse_progress_line("FTPROG 1048576 10485760 524288.0 18").unwrap();
        assert_eq!(p.downloaded, 1_048_576);
        assert_eq!(p.total, Some(10_485_760));
        assert_eq!(p.speed, Some(524_288.0));
        assert_eq!(p.eta, Some(18));
    }

    #[test]
    fn handles_na_fields_before_stream_resolves() {
        let p = parse_progress_line("FTPROG 4096 NA NA NA").unwrap();
        assert_eq!(p.downloaded, 4096);
        assert_eq!(p.total, None);
        assert_eq!(p.speed, None);
        assert_eq!(p.eta, None);
    }

    #[test]
    fn ignores_ordinary_yt_dlp_output() {
        assert!(parse_progress_line("[youtube] Extracting URL: https://...").is_none());
        assert!(parse_progress_line("[Merger] Merging formats into \"x.mp4\"").is_none());
        assert!(parse_progress_line("").is_none());
    }

    #[test]
    fn ignores_garbage_without_panicking() {
        assert!(parse_progress_line("FTPROG").is_none());
        assert!(parse_progress_line("FTPROG a b c d").is_none());
        assert!(parse_progress_line("FTPROG 1 2").is_none());
    }

    #[test]
    fn computes_percent() {
        let p = parse_progress_line("FTPROG 5000 10000 1.0 1").unwrap();
        assert_eq!(p.pct(), Some(50.0));
        let q = parse_progress_line("FTPROG 5000 NA NA NA").unwrap();
        assert_eq!(q.pct(), None);
    }
}

Step 2: Verify they fail

Run: cd src-tauri && cargo test downloader

Step 3: Implement

parse_progress_line splits on whitespace, requires the FTPROG prefix and exactly 5 tokens, and maps "NA" → None. downloaded must parse or the line is rejected. Add Progress::pct() returning Option<f64>.

Step 4: Verify they pass

Run: cd src-tauri && cargo test downloader Expected: 5 passed.

Step 5: Commit

git add -A && git commit -m "feat: parse yt-dlp progress output"

Task 5: Database layer

Files:

Create: src-tauri/src/db.rs
Modify: src-tauri/Cargo.toml (add rusqlite with bundled feature)

Interfaces:

Consumes: Channel (Task 2), Video (Task 3)
Produces: Db with open(path) -> Result<Db>, upsert_channels(&[Channel]) -> Result<usize>, list_channels() -> Result<Vec<ChannelWithCount>>, upsert_videos(&[Video]) -> Result<usize>, list_feed(&FeedFilter) -> Result<Vec<FeedItem>>, set_download_state(...), get_download(video_id)

Schema exactly as the spec defines. Wrap the connection in Mutex<Connection> for Tauri state.

Step 1: Write failing tests (all against Db::open_in_memory())

#[test] fn upserting_same_channel_twice_yields_one_row() { /* insert twice, assert count 1 */ }
#[test] fn refresh_preserves_download_state() {
    // upsert video, mark downloaded, upsert same video again with new title,
    // assert title updated AND download state still "done"
}
#[test] fn feed_is_sorted_newest_first() { /* three videos, assert descending published */ }
#[test] fn downloaded_only_filter_excludes_undownloaded() {}
#[test] fn hide_shorts_filter_excludes_shorts() {}
#[test] fn channel_filter_scopes_to_one_channel() {}
#[test] fn search_matches_title_case_insensitively() {}

Step 2: Verify they fail

Run: cd src-tauri && cargo test db

Step 3: Implement

upsert_videos uses INSERT … ON CONFLICT(id) DO UPDATE SET title=excluded.title, … touching only metadata columns — never the downloads table. That is what makes the preservation test pass. list_feed is a LEFT JOIN downloads with ORDER BY published DESC and WHERE clauses built from the filter.

Step 4: Verify they pass

Run: cd src-tauri && cargo test db Expected: 7 passed.

Step 5: Commit

git add -A && git commit -m "feat: SQLite schema and queries"

Task 6: Command surface and background work

Files:

Create: src-tauri/src/thumbs.rs, src-tauri/src/net.rs, src-tauri/src/commands.rs
Modify: src-tauri/src/main.rs, src-tauri/capabilities/default.json, src-tauri/tauri.conf.json

Interfaces:

Consumes: everything from Tasks 2–5
Produces: the nine commands and four events named in the spec
Step 1: Implement thumbs and net

thumbs::cache(client, url, dir) -> Result<PathBuf> — skip if the file already exists, name by video id, write bytes. net::is_online() — HEAD https://www.youtube.com with a 5s timeout.

Step 2: Implement refresh_feeds with bounded concurrency

futures::stream::iter(channels).map(fetch).buffer_unordered(8), upserting per channel as results land and emitting refresh:progress {done,total,channel}. A single channel's failure must not abort the run — collect errors and report a count.

Step 3: Implement download_video

Spawn via tokio::process::Command with the fixed format selector and --progress-template "FTPROG %(progress.downloaded_bytes)s %(progress.total_bytes)s %(progress.speed)s %(progress.eta)s". Read stdout lines, feed each to parse_progress_line, emit download:progress on Some. On exit code 0 mark done and record the real path; otherwise mark failed with captured stderr. Hold child handles in a Mutex<HashMap<String, Child>> so cancel_download can kill them. Gate concurrent downloads at 2 with a tokio::sync::Semaphore.

Step 4: Configure the asset protocol

tauri.conf.json → app.security.assetProtocol: { "enable": true, "scope": ["$HOME/Movies/FlightTube/**"] }, and add core:asset:default plus the opener and dialog permissions to capabilities/default.json. Without this the player shows a black rectangle and a CSP error.

Step 5: Verify the whole backend compiles and tests still pass

Run: cd src-tauri && cargo test && cargo build Expected: all tests pass, zero errors.

Step 6: Commit

git add -A && git commit -m "feat: Tauri commands, concurrent refresh, download manager"

Task 7: Feed UI

Files:

Create: src/types.ts, src/api.ts, src/hooks/useFeed.ts, src/hooks/useDownloads.ts, src/hooks/useConnectivity.ts, src/components/{Sidebar,TopBar,VideoRow,DownloadButton}.tsx
Modify: src/App.tsx

Interfaces:

Consumes: commands and events from Task 6
Produces: App rendering the working feed; Player mount point for Task 8
Step 1: Mirror Rust types in types.ts — Channel, FeedItem, DownloadState, FeedFilter. Keep field names identical to the serde output (snake_case) to avoid a mapping layer.
Step 2: Write api.ts — thin typed wrappers: importTakeout(), listChannels(), refreshFeeds(), listFeed(filter), downloadVideo(id), cancelDownload(id), deleteDownload(id), checkPrereqs(), plus onDownloadProgress(cb) / onRefreshProgress(cb) returning unlisten functions.
Step 3: Build the components — Sidebar (channel list + counts + active filter), TopBar (refresh with progress, search, downloaded-only, hide-Shorts, connectivity pill, settings), VideoRow (cached thumbnail, title, channel, relative time, views, DownloadButton), DownloadButton (idle → ring showing live pct → done badge; cancel while running).
Step 4: Verify against real data

Run: npm run tauri dev, import a real subscriptions.csv, refresh. Expected: feed populates newest-first across channels, thumbnails render, download shows live progress and completes.

Step 5: Commit

git add -A && git commit -m "feat: subscription feed UI"

Task 8: Player, offline mode, settings

Files:

Create: src/components/Player.tsx, src/components/Settings.tsx
Modify: src/App.tsx, src/hooks/useConnectivity.ts

Interfaces:

Consumes: Task 7 components, convertFileSrc from @tauri-apps/api/core
Produces: complete app
Step 1: Player — <video controls autoPlay src={convertFileSrc(path)} /> plus title, channel, description, and delete-download. Clicking an undownloaded row opens YouTube via the opener plugin instead.
Step 2: Offline mode — poll get_connectivity every 30s, listen to the webview's online/offline events, and add a manual override toggle. When offline (real or forced), force downloadedOnly on and show a banner.
Step 3: Settings — Takeout import via the dialog plugin, library path display, prereq status with yt-dlp and ffmpeg versions, and the brew install yt-dlp ffmpeg hint when missing.
Step 4: Verify offline behavior

Toggle Offline mode with downloads present. Expected: feed collapses to downloaded videos only, banner appears, playback still works, thumbnails still render from the disk cache.

Step 5: Commit

git add -A && git commit -m "feat: in-app player, offline mode, settings"

Self-Review

Spec coverage: Takeout import → T2/T8. Atom feeds → T3/T6. Thumbnail caching → T6. SQLite schema → T5. All nine commands and four events → T6. Downloads with fixed format selector, cancel, concurrency cap → T4/T6. Offline detection and filtering → T6/T8. In-app playback and asset protocol scope → T6/T8. UI incl. hide-Shorts → T7/T8. Prereq check → T6/T8. Testing strategy → T2/T3/T4/T5. No uncovered requirement.

Placeholders: none — every code step carries real code or a precise description of the transformation.

Type consistency: Channel, Video, Progress, FeedFilter, FeedItem, DownloadState are defined once and referenced identically throughout. parse_csv, parse_atom, parse_progress_line keep the same names in their defining and consuming tasks.

21 KiB Raw Permalink Blame History Unescape Escape

FlightTube Implementation Plan

Global Constraints

File Structure

Task 1: Scaffold and boot

Task 2: Takeout CSV parsing

Task 3: Atom feed parsing

Task 4: yt-dlp progress parsing

Task 5: Database layer

Task 6: Command surface and background work

Task 7: Feed UI

Task 8: Player, offline mode, settings

Self-Review

21 KiB

Raw Permalink Blame History