How Scraper Modules Work

SciPaperLoader uses a modular scraper architecture where each scraper module handles specific paper processing stages. Papers flow through different statuses as they are processed by various scrapers.

Key Concepts:

Input Statuses: Paper statuses this scraper can process
Output Statuses: Statuses papers get after processing
Processing Status: Temporary status while scraper works
Pipeline: Scrapers can be chained together

Status Flow Example:

New Processing Done

Papers transition through these statuses

System Configuration

Active Scraper Module: Loading...

Daily Volume Limit: Loading... papers

Total Available Modules: Loading...

Processing Pipeline: Multi-stage

Current Paper Counts:

Available Scraper Modules

Module Name	Description	Input Statuses	Success Output	Failure Output	Processing Status

Publisher Parser Overview

Publishers are detected from paper URLs and mapped to specific parser modules for content extraction.

Publisher	Papers	Parser Status	Parser Available