🏦 VaultParser — Pure Rust Bank Statement Extractor

VaultParser is a high-performance, secure, offline-only library and interactive web service built in pure Rust for extracting tabular transaction ledgers from PDF bank statements.

It is designed to be 100% Python-free, utilizing native Rust parsers and coordinate-clustering engines to process PDF structures with zero external runtimes (including native support for decrypting AES-256 encrypted PDFs with empty passwords using lopdf 0.39.0).

---

🚀 Key Features

• 📦 Native Rust Library: Can be integrated directly into other Rust applications.
• 🛠️ Builder-Pattern Configurations: Programmatic setup of column boundaries, row tolerance margins, page trimming, and transaction row filters.
• 🏦 Indian Bank Presets: Out-of-the-box templates for:
  • HDFC Bank
  • State Bank of India (SBI)
  • Canara Bank
  • Union Bank of India
  • UCO Bank
  • Indian Bank
  • H P State Co-operative Bank (HPSCB)
  • ICICI Bank
  • Punjab National Bank (PNB)
  • Kotak Mahindra Bank
  • Axis Bank
  • Bank of Baroda (BOB)
  • YES Bank
  • IDFC FIRST Bank
  • IndusInd Bank
  • Himachal Pradesh Gramin Bank (HPGB)
  • Auto-detect (auto) plus custom JSON preset files* ✨ Interactive Web UI: Includes a web-based visual dashboard where you can:
  • Drag, double-click to add, or right-click to delete vertical column guides over the PDF canvas.
  • Crop header/footer regions in real-time.
  • Directly override cell values or delete rows with coordinates that persist across exports.
  • Instant Copy TSV to Clipboard for pasting directly into Excel / Google Sheets.
• 💾 Multi-format Exporters: Bulk compile transaction history to .xlsx (Excel), .csv, .tsv, or .json.
• 📊 Automated Totals Row: Automatically calculates total debit, credit, and row count summaries in exported ledgers.
• 🏷️ Transaction Categorization Engine: Enable with --categorize / -c to auto-tag transactions into categories (UPI & Transfers, Salary & Income, ATM & Cash, etc.). Unrecognized transactions are assigned to Suspense.
• 📅 Date Range Filtering: Filter transaction statements by date range using --from <YYYY-MM-DD> and --to <YYYY-MM-DD>.
• ⚡ Batch Processing & Format Override: Process whole directories of PDF statements in batch mode or force export formats using --format / -f.

---

📂 Project Structure

├── src/
│   ├── lib.rs          # Core library entry point
│   ├── config.rs       # Extraction parameter builder (ExtractionConfig)
│   ├── presets.rs      # Native bank column coordinate templates
│   ├── parser.rs       # Core Y-coordinate clustering engine & categorization
│   ├── models.rs       # Word, Row, and Table structures
│   ├── exporter.rs     # CSV, TSV, XLSX & JSON exporting utilities
│   ├── error.rs        # Custom library error handling
│   ├── main.rs         # Local web UI server binary
│   └── bin/
│       └── vpcli.rs    # Command line tool binary (supports batch mode, categorization, date filtering)
├── static/             # Frontend assets (HTML, style.css, app.js, pdf.js)
├── temp/               # Temporary parsing folder (automatically cleaned up)
└── Cargo.toml          # Cargo package file

---

📖 Library Usage Example

Add the library to your own Rust application to parse statements programmatically:

use vaultparser::{ExtractionConfig, BankPreset, extract_from_file};
use vaultparser::exporter::export_to_csv;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // 1. Start from a bank preset, then override options
    let mut config = vaultparser::BankPreset::Hdfc.config();
    config.categorize = true;
    config.from_date = Some("2023-01-01".to_string());
    config.to_date = Some("2023-03-31".to_string());

    // 2. Natively extract tabular data from statement
    let table = extract_from_file("statement.pdf", &config)?;
    println!("Parsed {} transaction rows!", table.rows.len());

    // 3. Export ledger to CSV bytes
    let csv_bytes = export_to_csv(&table)?;
    std::fs::write("statement_ledger.csv", csv_bytes)?;

    Ok(())
}

---

💻 Running the CLI Tool

You can run the extraction directly from the command line without opening a browser. The CLI utilizes clap for command-line parsing and option handling:

# Print results to stdout (default CSV, or use -f tsv / -f json):
cargo run --release --bin vpcli -- <input-pdf> <bank-preset> [options]

# Save CSV, TSV, Excel (.xlsx), or JSON results directly to a file:
cargo run --release --bin vpcli -- <input-pdf> <bank-preset> [output-file] [options]

# Extract with transaction categorization enabled:
cargo run --release --bin vpcli -- statement.pdf hdfc output.xlsx --categorize

# Extract with date range filtering:
cargo run --release --bin vpcli -- statement.pdf auto output.tsv --from 2023-01-01 --to 2023-03-31

# Batch process an entire folder of PDF statements:
cargo run --release --bin vpcli -- <input-dir> <bank-preset> --format tsv --output <output-dir>

Examples:

# Output HDFC statement directly to stdout as TSV with categorization
cargo run --release --bin vpcli -- "hdfc bank.pdf" hdfc -f tsv --categorize

# Save Union Bank statement output to statement.tsv
cargo run --release --bin vpcli -- "statement.pdf" union output.tsv

# Auto-detect bank preset, decrypt with password, and save output
cargo run --release --bin vpcli -- secure_statement.pdf auto output.csv --password "secret123"

# Batch process a folder of PDFs into TSV format with categorization
cargo run --release --bin vpcli -- ./statements/ sbi -f tsv --categorize --output ./converted/

---

🏃 Running the Web UI

1. Make sure you have the Rust toolchain installed.
2. Start the local server:

   cargo run --release

3. Open your browser and navigate to: http://localhost:8000/

---

📋 Step-by-Step Usage Guide

A. Using the Web Dashboard (Interactive Visual Extraction)

1. Upload Statement: Drag and drop your PDF bank statement into the upload zone.
2. Apply Preset: Select your bank (e.g. HDFC Bank) from the Preset Selector dropdown. The vertical column guides will automatically overlay on the PDF preview.
3. Refine Columns:
   • Move: Drag existing vertical red lines to match the statement's columns.
   • Add: Double-click anywhere on the PDF canvas to add a new vertical guide.
   • Remove: Right-click on any guide to delete it.
4. Trim Headers & Footers: Adjust the Top Trim and Bottom Trim slider percentages to exclude running page headers, summaries, or footers.
5. Review & Override Data:
   • Edit cells: Double-click any cell in the live preview table to correct typos.
   • Delete rows: Click the red ✕ on any row to delete headers or background noise.
   • Note: Manual edits are locked to PDF coordinates and persist when you switch pages or export!
6. Download Ledger: Click Export to Excel (.xlsx) or Export to CSV to save the formatted ledger.

B. Integrating the Rust Library in Your Code

1. Add Cargo Dependency: Add vaultparser to your project's Cargo.toml. To include the visual Web UI server and its async framework dependencies, enable the optional web feature:

    [dependencies]
    vaultparser = "0.4.0"
    # Or with the optional web UI:
    # vaultparser = { version = "0.4.0", features = ["web"] }

2. Define Configuration: Use pre-configured bank presets or customize via the builder pattern (which returns a Result for validation since v0.1.6):

   // Option A: Using a pre-configured bank preset
   let config = vaultparser::BankPreset::Hdfc.config();
   
   // Option B: Custom configuration with validation
   let config = vaultparser::ExtractionConfig::builder()
       .col_guides(vec![0.11, 0.42, 0.52, 0.62, 0.75, 0.88])
       .col_mappings(vec![
           "date".to_string(),
           "description".to_string(),
           "reference".to_string(),
           "skip".to_string(),
           "debit".to_string(),
           "credit".to_string(),
           "balance".to_string(),
       ])
       .y_top_trim(0.0)
       .y_bottom_trim(1.0)
       .build()?;

3. Run Extraction: Pass the PDF file path or byte buffer to the parser:

   let table = vaultparser::extract_from_file("my_statement.pdf", &config)?;

4. Save Export: Output the transaction data directly:

   let csv_data = vaultparser::exporter::export_to_csv(&table)?;
   std::fs::write("statement.csv", csv_data)?;

---

🔧 Accuracy Tuning

VaultParser is coordinate-based, so a few settings usually determine extraction quality:

• y_tolerance: increase for statements with uneven baselines or mixed font sizes; decrease if nearby rows are being merged.
• col_guides: align guides to the visual column boundaries. Overlap-based word assignment helps wide words, but bad guides still reduce accuracy.
• y_top_trim / y_bottom_trim: crop running headers, footers, page numbers, and summary blocks.
• skip_header_rows / skip_footer_rows: use when headers/footers repeat on every page.
• merge_multi_line: keep enabled for transaction descriptions that span multiple PDF lines.
• filter_only_date / filter_only_amount: remove non-transaction rows. Date/amount diagnostics in recent versions help catch clustering or date-format issues.

Recent versions also carry cross-page narrations, normalize Sept/ordinal dates, handle Indian/European decimals, and warn on balance continuity, out-of-sequence dates, and possible duplicate rows.

🧪 Testing

To run the built-in unit and integration test suite:

cargo test -- --nocapture

This runs coordinate/amount parsing tests, an extraction check against the sample HDFC statement, and a full integration extraction check on the user's statement hdfc bank.pdf to verify native decryption.

---

🪵 Logging & Diagnostics

The library utilizes the standard tracing crate for diagnostic logging. Both the Web UI and CLI binaries are pre-configured to output formatted logs to stderr, keeping stdout reserved for exported payloads.

You can control the log verbosity using the RUST_LOG environment variable:

• Show only start-up and high-level extraction counts (default):

  # Windows PowerShell
  $env:RUST_LOG="info"
  cargo run --bin vpcli -- "hdfc bank.pdf" hdfc

  # Linux/macOS
  export RUST_LOG=info
  cargo run --bin vpcli -- "hdfc bank.pdf" hdfc

• Show page-by-page word counts and layout details (debug):

  $env:RUST_LOG="debug"
  cargo run --bin vpcli -- "hdfc bank.pdf" hdfc

---

🛡️ License

Distributed under the GNU Affero General Public License v3.0 (AGPL-3.0). Offline-only, secure, and private by design.