FileFlash Backend

A high-performance backend for FileFlash personal cloud storage with advanced network optimizations.

Features

P2P File Transfer

• WebRTC-based P2P Transfers: Direct peer-to-peer file transfers
• WebSocket Signaling Server: Coordinates peer connections and file requests
• Peer Discovery Service: Automatically finds peers with shared files
• Hybrid Transfer Mode: Intelligent fallback between P2P and server transfers
• Multi-source Downloads: Download from multiple peers simultaneously

Network Optimizations

• Stream Transfer Service: Direct streaming with parallel chunk uploads
• Compression Service: Adaptive compression with multiple algorithms
• Delta Sync Service: Efficient synchronization using content-defined chunking
• HTTP/2 Connection Pool: Multiplexed streams with connection reuse
• Network-Aware Chunk Sizing: Dynamic chunk optimization based on network conditions

Core Features

• GraphQL API for efficient data fetching
• JWT-based authentication
• File upload/download with progress tracking
• Folder management with hierarchical structure
• File deduplication using checksums
• Compression support for various file types
• Range requests for partial downloads
• Parallel downloads for large files

Installation

1. Install dependencies:

   npm install
   

2. Set up environment variables:

   cp .env.example .env
   # Edit .env with your configuration
   

3. Initialize the database:

   npm run init-db
   

4. Start the server:

   npm start
   # For development
   npm run dev
   

Environment Variables

API Endpoints

GraphQL

• Endpoint: /graphql
• Playground: Available in development mode

REST Downloads

• Download File: GET /download/file/:id
• Parallel Download: GET /download/file/:id/parallel
• Session Status: GET /download/session/:sessionId
• Cancel Session: DELETE /download/session/:sessionId
• Download Stats: GET /download/stats

P2P Endpoints

• WebSocket Signaling: ws://localhost:4000/p2p
• P2P Info: GET /p2p/ws-info
• Connection Test: POST /p2p/test-connection
• P2P Stats: GET /p2p/stats

Authentication

All API requests require authentication using JWT tokens.

Header Format

Authorization: Bearer <token>

Network Optimization Services

Stream Transfer Service

• Direct client-to-storage streaming
• Parallel chunk uploads with configurable concurrency
• Range request support
• Transform streams for compression and encryption

Compression Service

• Multiple compression algorithms (gzip, brotli, lz4)
• Adaptive compression profiles
• Image optimization with Sharp
• Network-aware compression selection

Delta Sync Service

• Rabin fingerprinting for chunking
• Rolling checksums for large files
• Binary diff for small changes
• Efficient delta calculation

HTTP/2 Connection Pool

• Connection pooling with keep-alive
• Stream multiplexing
• Prioritization support
• Automatic cleanup

Network-Aware Chunk Sizing

• Dynamic chunk calculation
• Adaptive sizing based on bandwidth/latency
• Profile-based configurations
• Real-time adaptation

P2P Operations

GraphQL Queries

• p2pPeers: Get list of available peers
• p2pFileSources: Find peers that have a specific file
• p2pTransferStatus: Check status of a P2P transfer
• p2pNetworkStats: Get P2P network statistics

GraphQL Mutations

• initiateP2PTransfer: Start a P2P file transfer
• registerP2PFiles: Register files as available for P2P sharing
• updatePeerCapabilities: Update peer network capabilities
• cancelP2PTransfer: Cancel an ongoing P2P transfer
• requestP2PFile: Request a file from a specific peer

Database Schema

The system uses PostgreSQL with the following main tables:

• users: User accounts and storage quotas
• files: File and folder metadata
• physical_storage: Physical file locations and sizes

Security Features

• JWT-based authentication
• File ownership verification
• Input validation and sanitization
• CORS protection
• Helmet security headers
• File deduplication prevents unauthorized access

Performance Features

• Connection pooling
• Stream processing
• Parallel operations
• Caching layers
• Adaptive algorithms
• Compression optimization

Development

Running Tests

npm test

Database Migrations

The database schema is managed through SQL files in src/models/.

Adding New Services

1. Create service class in src/services/
2. Import and initialize in src/server.js
3. Add to resolvers or routes as needed

License

MIT