LiveSync Setup and Configuration

Prerequisites

Before setting up LiveSync, ensure you have:

• An active Ably account with an API key
• A supported database (Postgres 10+ or MongoDB 4.0+)
• Database credentials with appropriate permissions
• Basic understanding of your database architecture

Setup Overview

Setting up LiveSync involves three main steps:

1. Configure your database - Set up required tables/collections
2. Create an integration rule - Connect your database to Ably
3. Integrate client SDKs - Subscribe to changes in your applications

Postgres Setup

Step 1: Create Required Tables

The Postgres database connector requires two tables in your database: an outbox table and a nodes table.

-- Create nodes table for cluster coordination
CREATE TABLE IF NOT EXISTS nodes (
  id TEXT PRIMARY KEY,
  expiry TIMESTAMP WITHOUT TIME ZONE NOT NULL
);

-- Create outbox table for change events
CREATE TABLE IF NOT EXISTS outbox (
  sequence_id  SERIAL PRIMARY KEY,
  mutation_id  TEXT NOT NULL,
  channel      TEXT NOT NULL,
  name         TEXT NOT NULL,
  rejected     BOOLEAN NOT NULL DEFAULT false,
  data         JSONB,
  headers      JSONB,
  locked_by    TEXT,
  lock_expiry  TIMESTAMP WITHOUT TIME ZONE,
  processed    BOOLEAN NOT NULL DEFAULT false
);

Step 2: Create Trigger Function

Create a trigger to notify the connector when new records are added to the outbox:

CREATE OR REPLACE FUNCTION public.outbox_notify()
RETURNS trigger AS $$
BEGIN
  PERFORM pg_notify('ably_adbc'::text, ''::text);
  RETURN NULL;
EXCEPTION
  WHEN others THEN
    RAISE WARNING 'unexpected error in %s: %%', SQLERRM;
    RETURN NULL;
END;
$$ LANGUAGE plpgsql;

-- Attach trigger to outbox table
CREATE TRIGGER public_outbox_trigger
AFTER INSERT ON public.outbox
FOR EACH STATEMENT EXECUTE PROCEDURE public.outbox_notify();

Step 3: Configure Database Permissions

Create a dedicated user with minimal required permissions:

CREATE ROLE livesync_user LOGIN PASSWORD 'secure_password' VALID UNTIL 'infinity';

-- Grant connection permission
GRANT CONNECT ON DATABASE your_database TO livesync_user;

-- Grant permissions on outbox table
GRANT SELECT, UPDATE, INSERT, DELETE ON public.outbox TO livesync_user;
GRANT USAGE, SELECT ON outbox_sequence_id_seq TO livesync_user;

-- Grant permissions on nodes table
GRANT SELECT, UPDATE, INSERT, DELETE ON public.nodes TO livesync_user;

Step 4: Create Integration Rule

Create a Postgres integration rule in the Ably dashboard:

1. Navigate to your Ably Dashboard
2. Select your application
3. Click the Integrations tab
4. Click New Integration Rule
5. Choose Postgres from the list

Configure the rule with the following settings:

Field	Description	Example
URL	Postgres connection string	postgres://user:pass@host:5432/dbname
Outbox table schema	Schema name for outbox table	public
Outbox table name	Name of the outbox table	outbox
Nodes table schema	Schema name for nodes table	public
Nodes table name	Name of the nodes table	nodes
SSL mode	SSL connection level	prefer / require / verify-ca / verify-full
SSL root certificate	CA certificate (if required)	Certificate content
Primary site	Data center location	us-east-1-A

Step 5: Test the Connection

Verify your setup by inserting a test record:

INSERT INTO outbox (
  mutation_id,
  channel,
  name,
  data
) VALUES (
  gen_random_uuid()::text,
  'test-channel',
  'test-event',
  '{"message": "Hello LiveSync!"}'
);

Monitor your Ably dashboard to confirm the message was published.

MongoDB Setup

Step 1: Enable Change Streams

Ensure your MongoDB deployment supports Change Streams:

• MongoDB Atlas: Change Streams are enabled by default
• Self-hosted: Requires a replica set or sharded cluster (MongoDB 4.0+)

Step 2: Configure Collection

Optionally enable pre and post images for full document access:

db.runCommand({
  collMod: 'your_collection',
  changeStreamPreAndPostImages: { enabled: true }
});

Step 3: Create Integration Rule

Create a MongoDB integration rule in the Ably dashboard:

1. Navigate to your Ably Dashboard
2. Select your application
3. Click the Integrations tab
4. Click New Integration Rule
5. Choose MongoDB from the list

Configure the rule with the following settings:

Field	Description	Example
URL	MongoDB connection string	mongodb://user:pass@host/db
Watch	What to watch	collection
Database name	Target database	myapp
Collection name	Target collection	users
Full Document	Include full document	updateLookup / whenAvailable / off
Full Document before change	Include previous state	whenAvailable / off
Pipeline	Change stream pipeline	See below
Primary site	Data center location	us-east-1-A

Step 4: Configure Pipeline

Define a MongoDB aggregation pipeline to route events to channels:

[
  {
    "$match": {
      "operationType": { "$in": ["insert", "update", "delete"] }
    }
  },
  {
    "$set": {
      "_ablyChannel": "users:updates"
    }
  }
]

Dynamic channel routing based on document fields:

[
  {
    "$set": {
      "_ablyChannel": {
        "$concat": [
          "users:",
          "$fullDocument.userId"
        ]
      }
    }
  }
]

Step 5: Configure Database Permissions

Create a user with Change Stream access:

db.createUser({
  user: 'livesync_user',
  pwd: 'secure_password',
  roles: [
    {
      role: 'read',
      db: 'your_database'
    }
  ]
});

// Grant Change Stream permissions
db.grantRolesToUser('livesync_user', [
  { role: 'changeStream', db: 'your_database' }
]);

Client SDK Integration

Install Ably SDK

Install the Ably SDK for your platform:

# JavaScript/Node.js
npm install ably

# Python
pip install ably

# Java
# Add to pom.xml or build.gradle

# Swift
# Add to Package.swift or use CocoaPods

Subscribe to Changes

Implement realtime subscriptions in your client application:

import { Realtime } from 'ably';

// Initialize with token auth (recommended for production)
const ably = new Realtime({
  authUrl: '/api/ably/auth' // Your token endpoint
});

// Get the channel
const channel = ably.channels.get('users:updates');

// Subscribe to all messages
await channel.subscribe((message) => {
  console.log('Change event:', message.data);
});

// Subscribe to specific event names
await channel.subscribe('user.created', (message) => {
  console.log('New user:', message.data);
});

await channel.subscribe('user.updated', (message) => {
  console.log('User updated:', message.data);
});

Authentication & Security

API Keys and Capabilities

Configure API keys with appropriate capabilities:

{
  "users:*": ["subscribe"],
  "public:*": ["subscribe", "history"]
}

Token Authentication

Implement token authentication for client applications:

// Backend token endpoint
app.get('/api/ably/auth', async (req, res) => {
  const tokenRequest = await ably.auth.createTokenRequest({
    clientId: req.user.id,
    capability: {
      [`users:${req.user.id}`]: ['subscribe']
    }
  });
  
  res.json(tokenRequest);
});

// Frontend client
const ably = new Realtime({
  authUrl: '/api/ably/auth',
  authMethod: 'GET'
});

Configuration Best Practices

Channel Naming

Use consistent channel naming conventions:

// Resource-based naming
'users:123'
'posts:456'
'comments:789'

// Hierarchical naming
'workspace:abc:tasks'
'workspace:abc:users'

// Event-type based
'orders:created'
'orders:updated'
'orders:cancelled'

Message Structure

Design consistent message payloads:

{
  "eventType": "user.updated",
  "timestamp": 1234567890,
  "data": {
    "id": "user123",
    "changes": {
      "email": "[e-mail]"
    }
  },
  "metadata": {
    "source": "api",
    "userId": "admin123"
  }
}

Error Handling

Implement robust error handling:

const channel = ably.channels.get('users:updates');

// Handle connection errors
ably.connection.on('failed', (stateChange) => {
  console.error('Connection failed:', stateChange.reason);
});

// Handle channel errors
channel.on('failed', (stateChange) => {
  console.error('Channel error:', stateChange.reason);
});

// Handle subscription errors
try {
  await channel.subscribe((message) => {
    console.log('Message:', message.data);
  });
} catch (error) {
  console.error('Subscription error:', error);
}

Monitoring and Debugging

Dashboard Metrics

Monitor your LiveSync integration in the Ably dashboard:

• Message throughput
• Connection count
• Error rates
• Integration rule status

Logging

Enable debug logging for troubleshooting:

const ably = new Realtime({
  key: 'YOUR_API_KEY',
  logLevel: 4 // 0=none, 1=errors, 2=warnings, 3=info, 4=debug
});

Testing

Test your integration with curl:

# Test outbox insertion (Postgres)
curl -X POST http://your-api/test-livesync \
  -H "Content-Type: application/json" \
  -d '{
    "channel": "test-channel",
    "name": "test-event",
    "data": {"message": "test"}
  }'

Next Steps

Database Synchronization

Learn database sync patterns and best practices

Conflict Resolution

Handle conflicts in distributed systems

Postgres Connector

Deep dive into Postgres-specific features

MongoDB Connector

Explore MongoDB Change Streams integration