Discover. Understand. Research.
Created by Sanjay | Infinity Web Technology
Copyright © 2026 Sanjay. All rights reserved.
A standalone, local-first AI-powered research workspace desktop application engineered for academic researchers. Discover papers across permitted scholarly websites and repositories, collect normalized metadata, download authorized open-access documents, read PDFs offline in the integrated reader, perform literature review matrices, and organize a persistent local library.
Built with Tauri 2, Rust, React 19, TypeScript, Vite 8, and Tailwind CSS v4.
- Website-Driven Discovery (No-API Engine): Discovers literature by requesting permitted public HTML pages and extracting standard embedded metadata (Highwire Press, Dublin Core, Schema.org JSON-LD) and normal hyperlinks. Never calls third-party academic provider APIs, private/undocumented network endpoints, or cloud scraping proxies.
- Local-First & Sovereign: All bibliographies, collections, crawl state, and PDF files reside on the researcher's local machine in a local SQLite database (WAL mode) and managed document directory. Zero mandatory remote servers or cloud accounts.
- Integrated Offline PDF Reader: Embedded reader powered by sandboxed PDF.js with page navigation, zoom/fit modes, 90° rotation, full-text search, thumbnail previews, and persistent reading state.
- Strict Ethical & Safety Boundaries: Automated URL validation via Anti-SSRF guards (blocking private/loopback/metadata addresses), full robots.txt compliance, polite crawl delay enforcement, and magic-byte
%PDF-validation before file storage. Paywalls and bot challenges are never bypassed.
research-paper-scraper/
├── src/ # React 19 frontend
│ ├── app/ # App root shell (App.tsx)
│ ├── components/ # Reusable design system primitives & layout
│ │ ├── layout/ # AppShell, Sidebar, Toolbar, PageHeader
│ │ └── ui/ # Button, Badge, Input, SearchInput, Modal, ConfirmDialog, Drawer, EmptyState, CommandPalette, ToastContainer
│ ├── context/ # Centralized AppContext for view state, theme, and fixtures
│ ├── data/ # Centralized typed demo fixtures (demo-data.ts)
│ ├── features/ # Screen views
│ │ ├── dashboard/ # DashboardView
│ │ ├── discover/ # DiscoverView
│ │ ├── library/ # LibraryView
│ │ ├── downloads/ # DownloadsView
│ │ ├── collections/ # CollectionsView
│ │ ├── sources/ # SourcesView
│ │ ├── activity/ # ActivityView
│ │ ├── settings/ # SettingsView
│ │ └── paper-details/ # PaperDetailDrawer
│ ├── hooks/ # useApp hook
│ ├── lib/ # Typed Tauri IPC bridge (tauri-bridge.ts)
│ ├── styles/ # Tailwind CSS v4 entry point & theme tokens
│ ├── test/ # Frontend test setup, Vitest unit & integration suites
│ └── types/ # TypeScript domain models and IPC interfaces
├── src-tauri/ # Native Rust backend
│ ├── capabilities/ # Scoped Tauri 2 window permissions
│ ├── src/
│ │ ├── commands/ # IPC command entry points (app_health, app_ping)
│ │ ├── config/ # Application configuration & bounds validation
│ │ ├── errors/ # Internal AppError & serializable CommandError
│ │ ├── logging/ # Tracing initialization & credential redaction
│ │ ├── services/ # Business & diagnostic services (HealthService)
│ │ ├── state/ # Managed application state (AppState)
│ │ ├── lib.rs # Builder & command handler registration
│ │ └── main.rs # Native executable entry point
│ ├── Cargo.toml # Native dependencies & profiles
│ └── tauri.conf.json # Desktop window geometry & strict CSP
├── docs/
│ ├── phase-1/ # Approved requirements, PRD, source research, domain model, ADRs
│ ├── phase-2/ # Foundation architecture, security baseline, quality reports
│ └── phase-3/ # Screen inventory, design system, demo data spec, completion report
├── .github/workflows/ # CI quality workflows (ci.yml)
├── package.json # Frontend dependencies & npm scripts
└── vite.config.ts # Vite + Tailwind v4 + Vitest configuration
- Node.js:
v18+(Tested onv24.13.1) - npm:
v9+(Tested on11.8.0) - Rust:
v1.77+stable toolchain (Tested on1.99.1) - Platform Build Tools (Windows): Microsoft Visual Studio C++ Build Tools & Microsoft Edge WebView2 runtime (pre-installed on modern Windows 10/11).
npm installLaunches Vite dev server and native Tauri desktop window simultaneously:
npm run tauri devLaunches Vite dev server on http://localhost:1420:
npm run dev(Note: In browser mode, an amber status banner indicates desktop IPC is unavailable.)
All quality checks must pass cleanly before any code is committed:
# Type check with TypeScript compiler
npm run typecheck
# Lint with ESLint
npm run lint
# Check code formatting with Prettier
npm run format:check
# Format files with Prettier
npm run format:write
# Run frontend unit tests with Vitest
npm run test
# Build production frontend bundle into dist/
npm run build
# Run all frontend quality checks in one command
npm run check:all# Check Rust code formatting
cargo fmt --manifest-path src-tauri/Cargo.toml --check
# Format Rust code
cargo fmt --manifest-path src-tauri/Cargo.toml
# Run Clippy linter with warnings treated as errors
cargo clippy --manifest-path src-tauri/Cargo.toml -- -D warnings
# Run native Rust unit tests
cargo test --manifest-path src-tauri/Cargo.tomlTo build the native production desktop executable and installer:
npm run tauri buildBuild outputs are generated under src-tauri/target/release/bundle/.
- Phase 1 Specifications:
- Product Requirements Document (PRD)
- Academic Source Support Matrix
- Collection & Compliance Policy
- Domain Model & SQLite Schema
- Architecture Decision Records (ADRs)
- Phase 2 Foundation:
- Foundation Architecture
- Security Baseline
- Testing & Quality Gates
- Open Decisions & Technical Debt
- Phase 3 Modern UI & Design System:
- UI Screen Inventory
- Design System & Tokens
- Demo Data Specification
- Phase 3 Completion Report
- Phase 4 Local Backend & SQLite Database:
- ADR-005: SQLite Persistence & Database Access
- Local Database Architecture & Migrations
- Relational Data Model & FTS5
- Phase 4 Quality & Test Report
- Phase 5 Academic Connectors & Unified Discovery:
- Source Connectors & Specification Matrix
- Academic Search Architecture & Connector Framework
- Provider Setup, Rate Limits & Credentials
- Phase 5 Implementation & Audit Report
- Phase 6 Website Scraping Engine:
- Website Scraper Architecture
- Crawl Profiles Guide & Custom Rules
- Phase 6 Implementation & Audit Report
- Phase 7 Paper Processing, Deduplication & PDF Integrity:
- Processing Pipeline Architecture
- Metadata Normalization & Identifier Namespaces
- Intelligent Deduplication & Merge Policy
- PDF Processing & Document Integrity
- Phase 7 Testing & Verification Report