ebaysnipeextension/.kiro/specs/appwrite-cloud-storage/design.md

Design Document

Overview

This design implements a comprehensive migration from localStorage to AppWrite cloud storage for the Amazon Product Bar Extension. The solution provides user-based authentication, real-time synchronization, offline capabilities, and seamless data migration while maintaining the existing extension functionality.

Architecture

High-Level Architecture

graph TB
    subgraph "Chrome Extension"
        UI[Extension UI]
        AM[AppWriteManager]
        AS[AuthService]
        MS[MigrationService]
        OS[OfflineService]
        Cache[Local Cache]
    end
    
    subgraph "AppWrite Cloud"
        Auth[Authentication]
        DB[(Database)]
        Collections[Collections]
    end
    
    UI --> AM
    AM --> AS
    AM --> MS
    AM --> OS
    AM --> Cache
    AS --> Auth
    AM --> DB
    DB --> Collections

AppWrite Configuration

Connection Details:

• Project ID: 6963df38003b96dab5aa
• Database ID: amazon-extension-db
• API Endpoint: https://appwrite.webklar.com/v1
• Authentication: Required (Login only, no registration)

Collections:

• amazon-ext-enhanced-items - Enhanced product items
• amazon-ext-saved-products - Legacy basic products
• amazon_ext_blacklist - Blacklisted brands
• amazon-ext-enhanced-settings - User settings
• amazon-ext-migration-status - Migration tracking

Components and Interfaces

1. AppWriteManager

Central manager for all AppWrite operations, replacing localStorage managers.

class AppWriteManager {
  constructor(config) {
    this.client = new Client()
      .setEndpoint(config.endpoint)
      .setProject(config.projectId);
    
    this.databases = new Databases(this.client);
    this.account = new Account(this.client);
    
    this.databaseId = config.databaseId;
    this.collections = config.collections;
    this.authService = new AuthService(this.account);
    this.offlineService = new OfflineService();
  }

  // Core CRUD operations
  async createDocument(collectionId, data, documentId = null)
  async getDocument(collectionId, documentId)
  async updateDocument(collectionId, documentId, data)
  async deleteDocument(collectionId, documentId)
  async listDocuments(collectionId, queries = [])
  
  // User-specific operations
  async getUserDocuments(collectionId, queries = [])
  async createUserDocument(collectionId, data, documentId = null)
}

2. AuthService

Handles user authentication and session management.

class AuthService {
  constructor(account) {
    this.account = account;
    this.currentUser = null;
    this.sessionToken = null;
  }

  async login(email, password)
  async logout()
  async getCurrentUser()
  async isAuthenticated()
  async refreshSession()
  
  // Event handlers
  onAuthStateChanged(callback)
  onSessionExpired(callback)
}

3. MigrationService

Handles migration from localStorage to AppWrite.

class MigrationService {
  constructor(appWriteManager, legacyManagers) {
    this.appWriteManager = appWriteManager;
    this.legacyManagers = legacyManagers;
  }

  async migrateAllData()
  async migrateEnhancedItems()
  async migrateBasicProducts()
  async migrateBlacklistedBrands()
  async migrateSettings()
  async migrateMigrationStatus()
  
  async getMigrationStatus()
  async markMigrationComplete()
}

4. OfflineService

Manages offline capabilities and synchronization.

class OfflineService {
  constructor() {
    this.offlineQueue = [];
    this.isOnline = navigator.onLine;
    this.syncInProgress = false;
  }

  async queueOperation(operation)
  async syncOfflineOperations()
  async handleConflictResolution(localData, remoteData)
  
  isOnline()
  onOnlineStatusChanged(callback)
}

5. Enhanced Storage Managers

Updated versions of existing managers to use AppWrite instead of localStorage.

class AppWriteEnhancedStorageManager extends EnhancedStorageManager {
  constructor(appWriteManager) {
    super();
    this.appWriteManager = appWriteManager;
    this.collectionId = 'amazon-ext-enhanced-items';
  }

  async saveEnhancedItem(item, allowEmptyOptional = false)
  async getEnhancedItems()
  async getEnhancedItem(id)
  async updateEnhancedItem(id, updates)
  async deleteEnhancedItem(id)
}

Data Models

Enhanced Item Document

{
  $id: "unique_document_id",
  $createdAt: "2024-01-11T10:00:00.000Z",
  $updatedAt: "2024-01-11T10:00:00.000Z",
  userId: "user_id_from_auth",
  
  // Original EnhancedItem fields
  itemId: "B08N5WRWNW",
  amazonUrl: "https://amazon.de/dp/B08N5WRWNW",
  originalTitle: "Original Amazon Title",
  customTitle: "AI Enhanced Title",
  price: "29.99",
  currency: "EUR",
  titleSuggestions: ["Suggestion 1", "Suggestion 2", "Suggestion 3"],
  hashValue: "sha256_hash_value",
  createdAt: "2024-01-11T09:00:00.000Z",
  updatedAt: "2024-01-11T10:00:00.000Z"
}

Blacklisted Brand Document

{
  $id: "unique_document_id",
  $createdAt: "2024-01-11T10:00:00.000Z",
  $updatedAt: "2024-01-11T10:00:00.000Z",
  userId: "user_id_from_auth",
  
  brandId: "bl_1641891234567_abc123def",
  name: "Brand Name",
  addedAt: "2024-01-11T10:00:00.000Z"
}

User Settings Document

{
  $id: "user_settings_document_id",
  $createdAt: "2024-01-11T10:00:00.000Z",
  $updatedAt: "2024-01-11T10:00:00.000Z",
  userId: "user_id_from_auth",
  
  mistralApiKey: "encrypted_api_key",
  autoExtractEnabled: true,
  defaultTitleSelection: "first",
  maxRetries: 3,
  timeoutSeconds: 10,
  updatedAt: "2024-01-11T10:00:00.000Z"
}

Authentication Flow

Login Process

sequenceDiagram
    participant U as User
    participant E as Extension
    participant A as AuthService
    participant AW as AppWrite
    
    U->>E: Opens Extension
    E->>A: Check Authentication
    A->>AW: Get Current Session
    AW-->>A: No Session
    A-->>E: Not Authenticated
    E->>U: Show Login Form
    U->>E: Enter Credentials
    E->>A: Login(email, password)
    A->>AW: Create Session
    AW-->>A: Session Token
    A-->>E: Authentication Success
    E->>E: Initialize AppWrite Managers
    E->>U: Show Extension UI

Session Management

• Sessions are managed by AppWrite's built-in session handling
• Session tokens are stored securely (not in localStorage)
• Automatic session refresh before expiration
• Graceful handling of expired sessions with re-authentication prompt

Migration Strategy

Migration Process Flow

flowchart TD
    Start([User Logs In]) --> Check{Check Migration Status}
    Check -->|Not Migrated| Detect[Detect localStorage Data]
    Check -->|Already Migrated| Skip[Skip Migration]
    
    Detect --> HasData{Has Local Data?}
    HasData -->|Yes| Migrate[Start Migration]
    HasData -->|No| Complete[Mark Complete]
    
    Migrate --> Items[Migrate Enhanced Items]
    Items --> Products[Migrate Basic Products]
    Products --> Brands[Migrate Blacklisted Brands]
    Brands --> Settings[Migrate Settings]
    Settings --> Status[Update Migration Status]
    Status --> Cleanup[Cleanup localStorage]
    Cleanup --> Complete
    
    Complete --> End([Migration Complete])
    Skip --> End

Migration Implementation

class MigrationService {
  async migrateAllData() {
    try {
      // Check if migration already completed
      const status = await this.getMigrationStatus();
      if (status.completed) {
        return { success: true, message: 'Migration already completed' };
      }

      const results = {
        enhancedItems: await this.migrateEnhancedItems(),
        basicProducts: await this.migrateBasicProducts(),
        blacklistedBrands: await this.migrateBlacklistedBrands(),
        settings: await this.migrateSettings()
      };

      // Mark migration as complete
      await this.markMigrationComplete(results);
      
      return { success: true, results };
    } catch (error) {
      console.error('Migration failed:', error);
      return { success: false, error: error.message };
    }
  }
}

Offline Capabilities

Offline Strategy

1. Local Caching: Critical data cached locally for offline access
2. Operation Queuing: Offline operations queued for later sync
3. Conflict Resolution: Timestamp-based conflict resolution
4. Progressive Sync: Gradual synchronization when connectivity returns

Offline Implementation

class OfflineService {
  async queueOperation(operation) {
    const queuedOp = {
      id: generateId(),
      type: operation.type,
      collectionId: operation.collectionId,
      documentId: operation.documentId,
      data: operation.data,
      timestamp: new Date().toISOString(),
      retries: 0
    };
    
    this.offlineQueue.push(queuedOp);
    await this.saveQueueToStorage();
  }

  async syncOfflineOperations() {
    if (!this.isOnline() || this.syncInProgress) return;
    
    this.syncInProgress = true;
    
    for (const operation of this.offlineQueue) {
      try {
        await this.executeOperation(operation);
        this.removeFromQueue(operation.id);
      } catch (error) {
        operation.retries++;
        if (operation.retries >= 3) {
          this.moveToFailedQueue(operation);
        }
      }
    }
    
    this.syncInProgress = false;
    await this.saveQueueToStorage();
  }
}

Error Handling

Error Categories

1. Authentication Errors: Session expired, invalid credentials
2. Network Errors: Connection timeout, offline status
3. API Errors: Rate limiting, server errors
4. Data Errors: Validation failures, conflicts

Error Handling Strategy

class AppWriteErrorHandler {
  static handleError(error, context) {
    switch (error.type) {
      case 'user_unauthorized':
        return this.handleAuthError(error, context);
      case 'document_not_found':
        return this.handleNotFoundError(error, context);
      case 'network_failure':
        return this.handleNetworkError(error, context);
      default:
        return this.handleGenericError(error, context);
    }
  }

  static getUserFriendlyMessage(error) {
    const messages = {
      'user_unauthorized': 'Bitte melden Sie sich erneut an.',
      'network_failure': 'Netzwerkfehler. Versuchen Sie es später erneut.',
      'rate_limit_exceeded': 'Zu viele Anfragen. Bitte warten Sie einen Moment.',
      'document_not_found': 'Die angeforderten Daten wurden nicht gefunden.'
    };
    
    return messages[error.type] || 'Ein unerwarteter Fehler ist aufgetreten.';
  }
}

Correctness Properties

A property is a characteristic or behavior that should hold true across all valid executions of a system-essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.

Authentication Properties

Property 1: Valid Authentication Success For any valid user credentials, authentication should succeed and result in a valid session being stored securely Validates: Requirements 1.2, 1.3

Property 2: Invalid Authentication Failure For any invalid user credentials, authentication should fail and display appropriate error messages Validates: Requirements 1.4

Property 3: Session Reuse For any existing valid session, the extension should automatically use it without requiring re-authentication Validates: Requirements 1.5

Data Storage Properties

Property 4: User Data Isolation For any data operation, all stored data should be associated with the authenticated user ID and only accessible by that user Validates: Requirements 2.5, 7.1

Property 5: Collection Routing For any data type (enhanced items, blacklisted brands, settings, migration status), data should be stored in the correct AppWrite collection Validates: Requirements 2.1, 2.2, 2.3, 2.4

Migration Properties

Property 6: Complete Data Migration For any existing localStorage data, all data types (enhanced items, blacklisted brands, settings) should be successfully migrated to AppWrite Validates: Requirements 3.2, 3.3, 3.4

Property 7: Migration State Tracking For any migration operation, successful completion should result in proper migration status marking, and failures should provide detailed error information Validates: Requirements 3.5, 3.6

Synchronization Properties

Property 8: Real-time Data Sync For any data modification, changes should be immediately updated in AppWrite and reflected in the UI Validates: Requirements 4.1, 4.2

Property 9: Offline Change Queuing For any change made while offline, the change should be queued locally for later synchronization Validates: Requirements 4.3, 5.2

Property 10: Connectivity Restoration Sync For any network connectivity restoration, all queued offline changes should be automatically synchronized to AppWrite Validates: Requirements 4.4, 5.3

Property 11: Timestamp-based Conflict Resolution For any sync conflict, the system should resolve conflicts using the most recent timestamp Validates: Requirements 4.5, 5.4

Offline Capability Properties

Property 12: Offline Functionality For any offline state, the extension should continue to function using cached data Validates: Requirements 5.1

Error Handling Properties

Property 13: AppWrite Unavailability Fallback For any AppWrite service unavailability, the extension should fall back to localStorage temporarily Validates: Requirements 6.1

Property 14: Authentication Expiry Handling For any expired authentication session, the extension should prompt for re-authentication Validates: Requirements 6.2

Property 15: Rate Limiting Backoff For any API rate limit exceeded response, the extension should implement exponential backoff Validates: Requirements 6.3

Property 16: Data Corruption Recovery For any detected data corruption, the extension should attempt automatic recovery Validates: Requirements 6.4

Property 17: German Error Messages For any critical error, the extension should provide user-friendly error messages in German Validates: Requirements 6.5

Security Properties

Property 18: Sensitive Data Encryption For any sensitive data like API keys, the data should be encrypted before storing in AppWrite Validates: Requirements 7.2

Property 19: HTTPS Communication For any AppWrite communication, the extension should use secure HTTPS connections Validates: Requirements 7.3

Property 20: Automatic Inactivity Logout For any extended period of user inactivity, the extension should automatically log out the user Validates: Requirements 7.4

Property 21: No Local Credential Storage For any authentication operation, credentials should never be stored in localStorage Validates: Requirements 7.5

Performance Properties

Property 22: Intelligent Caching For any data loading operation, the extension should implement appropriate caching strategies Validates: Requirements 8.1

Property 23: Batch Operations for Large Datasets For any large dataset synchronization, the extension should use batch operations to minimize API calls Validates: Requirements 8.2

Property 24: Pagination for Large Collections For any large collection display, the extension should implement pagination Validates: Requirements 8.3

Property 25: Critical Operation Prioritization For any slow network condition, the extension should prioritize critical operations Validates: Requirements 8.4

Property 26: Frequent Data Preloading For any frequently accessed data, the extension should preload it to improve response times Validates: Requirements 8.5

Testing Strategy

Unit Testing

• AppWriteManager: Mock AppWrite SDK for isolated testing
• AuthService: Test authentication flows and session management
• MigrationService: Test data migration scenarios
• OfflineService: Test offline queuing and synchronization

Integration Testing

• End-to-End Migration: Test complete localStorage to AppWrite migration
• Authentication Flow: Test login, logout, and session management
• Data Synchronization: Test real-time sync across multiple instances
• Offline Scenarios: Test offline functionality and sync recovery

Property-Based Testing

Each correctness property will be implemented as a property-based test using fast-check library:

• Minimum 100 iterations per property test for comprehensive coverage
• Test tagging: Each test tagged with format Feature: appwrite-cloud-storage, Property {number}: {property_text}
• Mock AppWrite SDK for controlled testing environments
• Randomized test data generation for robust validation
• Edge case coverage through property-based input generation

Dual Testing Approach

• Unit tests: Verify specific examples, edge cases, and error conditions
• Property tests: Verify universal properties across all inputs
• Both approaches are complementary and necessary for comprehensive coverage