Add flexible import folder functionality
- Add IMPORT_FOLDER_NAME configuration variable - Support organized import (subfolders) or consolidated import (all to INBOX) - Implement get_destination_folder() method for folder mapping - Add email.utils import for proper date handling - Update logging to show source → destination folder mappings - Auto-create import folders as needed Configuration examples: - IMPORT_FOLDER_NAME=Imported → organized subfolders - IMPORT_FOLDER_NAME= → all emails to INBOX - Update .env.template with comprehensive explanations - Add Host Europe and German hosting provider examples - Include detailed configuration guides and troubleshooting - Add realistic migration scenarios - Update README.md with complete documentation - Add feature overview and configuration guide - Include usage examples and troubleshooting section - Document both import modes with clear examples
This commit is contained in:
Elmar Sönser
parent
bd54ae469b
commit
481e32bb73
3 changed files with 469 additions and 92 deletions
239
README.md
239
README.md
|
|
@ -1,20 +1,239 @@
|
|||
# Email Migration Script
|
||||
|
||||
A Python script to migrate emails from one IMAP account to another, preserving folder structure and metadata.
|
||||
A Python script to migrate emails from one IMAP account to another with flexible folder organization options.
|
||||
|
||||
## Features
|
||||
|
||||
- **Flexible Import Options**: Choose between organized folder structure or consolidated inbox
|
||||
- **Folder Filtering**: Include/exclude specific folders from migration
|
||||
- **Metadata Preservation**: Maintain email flags, dates, and other metadata
|
||||
- **Batch Processing**: Configurable batch sizes for optimal performance
|
||||
- **Comprehensive Logging**: Detailed logs for monitoring and troubleshooting
|
||||
- **SSL Security**: Secure connections with SSL/TLS support
|
||||
|
||||
## Quick Start
|
||||
|
||||
1. Edit .env file with your email credentials
|
||||
2. Run: python3 email_migration.py
|
||||
1. **Install**: No dependencies required - uses Python standard library
|
||||
2. **Configure**: Copy `.env.template` to `.env` and add your email credentials
|
||||
3. **Run**: `python3 email_migration.py`
|
||||
|
||||
## Requirements
|
||||
|
||||
- Python 3.6+ (pre-installed on Linux & macOS)
|
||||
- No additional dependencies required
|
||||
- IMAP access enabled on both email accounts
|
||||
- App passwords (recommended for Gmail, Yahoo, Outlook)
|
||||
|
||||
## Configuration
|
||||
Edit the .env file with your email account settings.
|
||||
## Import Folder Options
|
||||
|
||||
## Important Notes
|
||||
- Use app passwords, not regular passwords
|
||||
- Test with a small folder first
|
||||
- Check email_migration.log for detailed logs
|
||||
### Option 1: Organized Import (Recommended)
|
||||
|
||||
```env
|
||||
IMPORT_FOLDER_NAME=Imported
|
||||
```
|
||||
|
||||
**Result**: All source folders become subfolders within "Imported"
|
||||
- Source `INBOX` → Destination `Imported/INBOX`
|
||||
- Source `Sent` → Destination `Imported/Sent`
|
||||
- Source `Drafts` → Destination `Imported/Drafts`
|
||||
|
||||
**Benefits**:
|
||||
- Preserves original folder structure
|
||||
- Keeps imported emails separate from existing emails
|
||||
- Easy to locate and organize imported content
|
||||
|
||||
### Option 2: Consolidated Import
|
||||
|
||||
```env
|
||||
IMPORT_FOLDER_NAME=
|
||||
```
|
||||
|
||||
**Result**: All emails go directly to destination INBOX
|
||||
- All source folders → Destination `INBOX`
|
||||
|
||||
**Benefits**:
|
||||
- Simple single-folder result
|
||||
- Useful for merging multiple accounts
|
||||
- Good for basic email consolidation
|
||||
|
||||
## Configuration Guide
|
||||
|
||||
### Email Account Setup
|
||||
|
||||
```env
|
||||
# Source account (migrating FROM)
|
||||
SOURCE_IMAP_SERVER=imap.gmail.com
|
||||
SOURCE_EMAIL=old@gmail.com
|
||||
SOURCE_PASSWORD=your_app_password
|
||||
|
||||
# Destination account (migrating TO)
|
||||
DEST_IMAP_SERVER=imap.gmail.com
|
||||
DEST_EMAIL=new@gmail.com
|
||||
DEST_PASSWORD=your_app_password
|
||||
```
|
||||
|
||||
### Common IMAP Servers
|
||||
|
||||
| Provider | IMAP Server | Port | SSL |
|
||||
|----------|-------------|------|-----|
|
||||
| Gmail | `imap.gmail.com` | 993 | Yes |
|
||||
| Outlook/Hotmail | `outlook.office365.com` | 993 | Yes |
|
||||
| Yahoo | `imap.mail.yahoo.com` | 993 | Yes |
|
||||
| Apple iCloud | `imap.mail.me.com` | 993 | Yes |
|
||||
|
||||
### App Password Setup
|
||||
|
||||
**Gmail**: Google Account → Security → App Passwords
|
||||
**Yahoo**: Account Security → Generate app password
|
||||
**Outlook**: Account Security → App passwords
|
||||
|
||||
⚠️ **Important**: Use app passwords, not regular login passwords!
|
||||
|
||||
## Advanced Configuration
|
||||
|
||||
### Folder Filtering
|
||||
|
||||
```env
|
||||
# Only migrate specific folders
|
||||
INCLUDE_FOLDERS=INBOX,Sent,Important
|
||||
|
||||
# Skip unwanted folders
|
||||
EXCLUDE_FOLDERS=Trash,Spam,Junk
|
||||
```
|
||||
|
||||
### Performance Tuning
|
||||
|
||||
```env
|
||||
# Batch size (emails per batch)
|
||||
BATCH_SIZE=50 # Default balance
|
||||
BATCH_SIZE=10 # Conservative (slow connections)
|
||||
BATCH_SIZE=100 # Aggressive (fast connections)
|
||||
|
||||
# Connection timeout
|
||||
IMAP_TIMEOUT=60 # Default
|
||||
IMAP_TIMEOUT=120 # Slow connections
|
||||
```
|
||||
|
||||
### Logging and Debugging
|
||||
|
||||
```env
|
||||
LOG_LEVEL=INFO # Normal operation
|
||||
LOG_LEVEL=DEBUG # Detailed troubleshooting
|
||||
LOG_LEVEL=ERROR # Errors only
|
||||
```
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Example 1: Full Gmail Migration with Organization
|
||||
```bash
|
||||
# .env configuration
|
||||
SOURCE_EMAIL=old@gmail.com
|
||||
DEST_EMAIL=new@gmail.com
|
||||
IMPORT_FOLDER_NAME=OldAccount
|
||||
EXCLUDE_FOLDERS=Trash,Spam
|
||||
|
||||
# Run migration
|
||||
python3 email_migration.py
|
||||
```
|
||||
|
||||
### Example 2: Test Migration (INBOX only)
|
||||
```bash
|
||||
# .env configuration
|
||||
INCLUDE_FOLDERS=INBOX
|
||||
BATCH_SIZE=10
|
||||
LOG_LEVEL=DEBUG
|
||||
|
||||
# Run migration
|
||||
python3 email_migration.py
|
||||
```
|
||||
|
||||
### Example 3: Consolidate Multiple Accounts
|
||||
```bash
|
||||
# .env configuration
|
||||
IMPORT_FOLDER_NAME=
|
||||
# This puts all emails in main INBOX
|
||||
|
||||
# Run migration
|
||||
python3 email_migration.py
|
||||
```
|
||||
|
||||
## Monitoring Progress
|
||||
|
||||
The script provides detailed logging in multiple places:
|
||||
|
||||
1. **Console Output**: Real-time progress updates
|
||||
2. **Log File**: `email_migration.log` with detailed information
|
||||
3. **Final Summary**: Complete migration statistics
|
||||
|
||||
Example output:
|
||||
```
|
||||
Email Migration Script
|
||||
==================================================
|
||||
[INFO] Import folder configuration: All emails will be imported to subfolders within "Imported"
|
||||
[INFO] Found 5 folders to process
|
||||
[INFO] Processing folder: INBOX
|
||||
[INFO] Migrating 'INBOX' -> 'Imported/INBOX'
|
||||
[INFO] Downloaded 150/150 messages from INBOX
|
||||
[INFO] Uploaded 150/150 messages to Imported/INBOX
|
||||
[INFO] Folder 'INBOX' -> 'Imported/INBOX' completed: 150 downloaded, 150 uploaded
|
||||
|
||||
Migration completed!
|
||||
Folders processed: 5
|
||||
Total emails downloaded: 1250
|
||||
Total emails uploaded: 1250
|
||||
Errors encountered: 0
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
**Authentication Failed**
|
||||
- Use app passwords instead of regular passwords
|
||||
- Enable IMAP access in email account settings
|
||||
- Check server settings and ports
|
||||
|
||||
**Connection Timeout**
|
||||
- Increase `IMAP_TIMEOUT` value
|
||||
- Reduce `BATCH_SIZE` for stability
|
||||
- Check network connection
|
||||
|
||||
**Folder Creation Failed**
|
||||
- Verify destination account has folder creation permissions
|
||||
- Check for special characters in folder names
|
||||
- Some providers have folder name restrictions
|
||||
|
||||
**Partial Migration**
|
||||
- Check logs in `email_migration.log`
|
||||
- Re-run script - it will skip already migrated emails
|
||||
- Use `INCLUDE_FOLDERS` to retry specific folders
|
||||
|
||||
### Getting Help
|
||||
|
||||
1. **Enable Debug Logging**: Set `LOG_LEVEL=DEBUG`
|
||||
2. **Check Log File**: Review `email_migration.log`
|
||||
3. **Test Small First**: Use `INCLUDE_FOLDERS=INBOX` and `BATCH_SIZE=10`
|
||||
4. **Verify Credentials**: Test account access with email client first
|
||||
|
||||
## Security Notes
|
||||
|
||||
- Never commit `.env` file to version control
|
||||
- Use app passwords instead of account passwords
|
||||
- Ensure SSL is enabled (`*_IMAP_USE_SSL=True`)
|
||||
- Store credentials securely
|
||||
- Test with non-critical accounts first
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
email_migration/
|
||||
├── email_migration.py # Main script
|
||||
├── .env.template # Configuration template
|
||||
├── .env # Your configuration (create from template)
|
||||
├── email_migration.log # Generated log file
|
||||
├── temp_emails/ # Temporary files (auto-created)
|
||||
└── README.md # This file
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
This script is provided as-is for educational and personal use. Test thoroughly before production use.
|
||||
Loading…
Add table
Add a link
Reference in a new issue