Runbook

Troubleshooting

Validation Cache Outdated

Symptoms: Requests fail with "Field mismatch" or "Validation error"

Diagnosis:

1. Check what schema the server has:

   GET /api/v1/metadata/fields?entity=Invoice

2. Compare with database Field records

Resolution:

• Restart server to clear in-memory cache
• Or trigger cache invalidation for tenant/entity

Slow Lookups

Symptoms: findMany on lookup fields taking > 500ms

Diagnosis:

1. Check for missing indexes on tenant table
2. Check validation cache hit rate

Resolution:

-- Check indexes on tenant table
SELECT indexname, indexdef 
FROM pg_indexes 
WHERE tablename = 'invoice';

-- Add index if missing
CREATE INDEX idx_invoice_customer ON invoice(customer_id);

Migration Failures

Symptoms: New field appears in UI but requests fail with column does not exist

Diagnosis:

# Check server logs for MigrationError
grep "MigrationError" /var/log/crm/server.log

Resolution:

1. Check tenant database schema:

   SELECT column_name, data_type 
   FROM information_schema.columns 
   WHERE table_name = 'invoice';

2. Re-trigger migration:

   // Re-save the field to trigger hook
   await FieldService.updateField(fieldId, { ... });

Maintenance

Adding a New Field Type

1. Update shared types: Add to FieldType enum in @repo/shared-types
2. Update EntityMetadataService: Handle new type in getFields
3. Update ValidationService: Map type to JSON Schema property
4. Update Migration Engine: Generate correct DDL for type

Manual Schema Sync

If tenant schema drifts from metadata:

import { TenantMigrationService } from "./services/migration-service";

await TenantMigrationService.syncSchema({
  org: "acme-corp",
  entity: "Invoice",
});

Database Commands

# Access tenant database
pnpm db:org:push    # Push schema to org DB
pnpm db:org:studio # Open Prisma Studio for org DB