adds flask-migrate instructions to readme and deletes old developer instructions

DEVELOPMENT

@@ -1,3 +1,35 @@
+## Directory Structure
+
+Below is the directory and file layout for the `scipaperloader` project:
+
+```plaintext
+scipaperloader/
+├── app/
+│   ├── __init__.py        # Initialize Flask app and database
+│   ├── models.py          # SQLAlchemy database models
+│   ├── main.py            # Flask routes (main blueprint)
+│   ├── templates/         # Jinja2 templates for HTML pages
+│   │   ├── base.html      # Base layout template with Alpine.js and HTMX
+│   │   ├── index.html     # Home page template
+│   │   ├── upload.html    # CSV upload page template
+│   │   ├── schedule.html  # Schedule configuration page template
+│   │   └── logs.html      # Logs display page template
+│   └── static/            # Static files (CSS, JS, images)
+├── scraper.py             # Background scraper daemon script
+├── tests/
+│   └── test_scipaperloader.py  # Tests with a Flask test fixture
+├── config.py              # Configuration settings for different environments
+├── pyproject.toml         # Project metadata and build configuration
+├── setup.cfg              # Development tool configurations (linting, testing)
+├── Makefile               # Convenient commands for development tasks
+└── .venv/                 # Python virtual environment (not in version control)
+```
+
+- The **`app/`** package contains the Flask application code. It includes an `__init__.py` to create the app and set up extensions, a `models.py` defining database models with SQLAlchemy, and a `main.py` defining routes in a Flask Blueprint. The `templates/` directory holds HTML templates (with Jinja2 syntax) and `static/` will contain static assets (e.g., custom CSS or JS files, if any).  
+- The **`scraper.py`** is a **standalone** Python script acting as a background daemon. It can be run separately to perform background scraping tasks (e.g., periodically fetching new data). This script will use the same database (via SQLAlchemy models or direct database access) to read or write data as needed.  
+- The **`tests/`** directory includes a test file that uses pytest to ensure the Flask app and its components work as expected. A Flask fixture creates an application instance for testing (with an in-memory database) and verifies routes and database operations (e.g., uploading CSV adds records).  
+- The **configuration and setup files** at the project root help in development and deployment. `config.py` defines configuration classes (for development, testing, production) so the app can be easily configured. `pyproject.toml` and `setup.cfg` provide project metadata and tool configurations (for packaging, linting, etc.), and a `Makefile` is included to simplify common tasks (running the app, tests, etc.).  
+
 ## How to use the logger
 
 ### GUI Interactions:

README.md

@@ -82,6 +82,28 @@ The following environment variables can be set to configure Celery:
 Consider using
 [dotenv](https://flask.palletsprojects.com/en/3.0.x/cli/#environment-variables-from-dotenv).
 
+## Database Migrations with Flask-Migrate
+
+SciPaperLoader uses Flask-Migrate (based on Alembic) to handle database schema changes. This allows for version-controlled database updates that can be applied or rolled back as needed.
+
+### Database Migration Commands
+
+- `make db-migrate message="Description of changes"`: Create a new migration script based on detected model changes
+- `make db-upgrade`: Apply all pending migration scripts to the database
+- `make db-downgrade`: Revert the most recent migration
+- `make reset-db`: Reset the database completely (delete, initialize, and migrate)
+
+### Working with Migrations
+
+When you make changes to the database models (in `models.py`):
+
+1. Create a migration: `make db-migrate message="Add user roles table"`
+2. Review the generated migration script in the `migrations/versions/` directory
+3. Apply the migration: `make db-upgrade`
+4. To roll back a problematic migration: `make db-downgrade`
+
+Always create database backups before applying migrations in production using `make backup-db`.
+
 ## Deployment
 
 See [Deploying to Production](https://flask.palletsprojects.com/en/3.0.x/deploying/).