Replicating AWS RDS MySQL to SQL Server on Windows

Building a Debezium CDC Proof of Concept

Introduction

Many organisations need a reliable way to move operational data from one database platform to another without relying on heavy batch jobs, manual exports, or nightly ETL windows. One practical approach is Change Data Capture, usually called CDC.

This blog explains how to build a proof of concept for near real-time replication from an existing AWS RDS MySQL database to a Microsoft SQL Server database running on a Windows machine using Debezium, Kafka Connect, and the Debezium JDBC sink connector.

The example source database endpoint used in this POC is:

onsysdb-01.ctrgy22e11p4.ap-southeast-2.rds.amazonaws.com

The target database is SQL Server running on a Windows machine.

The final architecture looks like this:

AWS RDS MySQL
        |
        | MySQL binary log / CDC
        v
Debezium MySQL Source Connector
        |
        v
Kafka Topic
        |
        v
Debezium JDBC Sink Connector
        |
        v
Microsoft SQL Server

Debezium is an open-source distributed platform for change data capture. For MySQL, Debezium reads the MySQL binary log and emits row-level INSERT, UPDATE, and DELETE changes into Kafka topics.

1. What Debezium Does

Debezium does not directly copy rows from MySQL to SQL Server by itself. Instead, it captures database changes and publishes them into Kafka.

For this POC, Debezium performs two main jobs:

1. Initial snapshot
   Captures the current rows from selected MySQL tables.

2. Continuous change capture
   Reads future INSERT, UPDATE, and DELETE events from the MySQL binary log.

After Debezium writes those changes into Kafka topics, a sink connector reads the topic and writes the data into SQL Server.

For the target database, this POC uses the Debezium JDBC sink connector. The JDBC sink connector can consume Debezium change events and write them into a relational database such as SQL Server. It also supports upsert behaviour and delete handling when configured correctly.

2. POC Scope

This POC demonstrates replication of one MySQL table into SQL Server.

The test will verify:

1. MySQL RDS connectivity
2. SQL Server connectivity
3. MySQL binary log configuration
4. Debezium source connector startup
5. Kafka topic creation
6. SQL Server sink connector startup
7. Initial data snapshot
8. INSERT replication
9. UPDATE replication
10. DELETE replication

The POC table used in this guide is:

debezium_poc_customer

3. Software Components

The POC uses the following components:

Windows machine
Docker Desktop
Debezium Kafka container
Debezium Connect container
Debezium MySQL source connector
Debezium JDBC sink connector
Microsoft JDBC Driver for SQL Server
AWS RDS MySQL
Microsoft SQL Server

The Microsoft JDBC Driver for SQL Server is the official Java database driver used by Java applications and tools to connect to SQL Server and Azure SQL.

4. Network Connectivity Checks

Before configuring Debezium, confirm that the Windows machine can reach both MySQL and SQL Server.

Run these commands from PowerShell:

Test-NetConnection onsysdb-01.ctrgy22e11p4.ap-southeast-2.rds.amazonaws.com -Port 3306
Test-NetConnection 192.168.1.51 -Port 1433

Expected result:

TcpTestSucceeded : True

If MySQL connectivity fails, check the AWS RDS security group. It must allow inbound traffic on port 3306 from your Windows machine, VPN range, or network.

If SQL Server connectivity fails, check:

SQL Server TCP/IP protocol is enabled
SQL Server is listening on port 1433
Windows Firewall allows inbound TCP 1433
SQL Server authentication mode allows SQL logins

To allow SQL Server traffic through Windows Firewall:

New-NetFirewallRule `
  -DisplayName "SQL Server 1433 Debezium POC" `
  -Direction Inbound `
  -Protocol TCP `
  -LocalPort 1433 `
  -Action Allow

5. Prepare AWS RDS MySQL for Debezium

Debezium requires MySQL binary logging to be enabled. More importantly, MySQL must use:

binlog_format = ROW
binlog_row_image = FULL

Debezium’s MySQL connector requires row-level binary log events. If binlog_format is not ROW, the connector fails with an error similar to:

The database server is not configured to use a ROW binlog_format,
which is required for this connector to work properly.

This check happens before Debezium starts reading any table data. It is not a table-level issue. It is a MySQL server or RDS parameter group issue.

Connect to RDS MySQL:

mysql -h onsysdb-01.ctrgy22e11p4.ap-southeast-2.rds.amazonaws.com -P 3306 -u admin -p

Check current settings:

SHOW VARIABLES LIKE 'log_bin';
SHOW VARIABLES LIKE 'binlog_format';
SHOW VARIABLES LIKE 'binlog_row_image';
SHOW VARIABLES LIKE 'server_id';

SELECT 
  @@hostname,
  @@server_id,
  @@global.binlog_format AS global_binlog_format,
  @@session.binlog_format AS session_binlog_format,
  @@global.binlog_row_image AS global_binlog_row_image,
  @@global.log_bin AS global_log_bin;

Expected result:

log_bin                  ON
global_binlog_format     ROW
session_binlog_format    ROW
global_binlog_row_image  FULL
global_log_bin           1

6. Configure the RDS Parameter Group

For Amazon RDS MySQL, binlog_format is controlled through the DB parameter group. If the database uses the default parameter group, create a custom parameter group and attach it to the DB instance. After changing the binary log format, a reboot is normally required before the change takes effect.

Check the current parameter group:

aws rds describe-db-instances `
  --db-instance-identifier onsysdb-01 `
  --query "DBInstances[0].DBParameterGroups"

Create a custom parameter group if required:

aws rds create-db-parameter-group `
  --db-parameter-group-name onsysdb01-debezium-mysql `
  --db-parameter-group-family mysql8.0 `
  --description "Parameter group for Debezium CDC"

Modify the parameter group:

aws rds modify-db-parameter-group `
  --db-parameter-group-name onsysdb01-debezium-mysql `
  --parameters `
  "ParameterName=binlog_format,ParameterValue=ROW,ApplyMethod=pending-reboot" `
  "ParameterName=binlog_row_image,ParameterValue=FULL,ApplyMethod=pending-reboot"

Attach it to the RDS instance:

aws rds modify-db-instance `
  --db-instance-identifier onsysdb-01 `
  --db-parameter-group-name onsysdb01-debezium-mysql `
  --apply-immediately

Reboot the RDS instance:

aws rds reboot-db-instance `
  --db-instance-identifier onsysdb-01

After reboot, reconnect to MySQL and verify again:

SHOW VARIABLES LIKE 'binlog_format';
SHOW VARIABLES LIKE 'binlog_row_image';
SHOW VARIABLES LIKE 'log_bin';

Expected:

binlog_format     ROW
binlog_row_image  FULL
log_bin           ON

7. Configure RDS Binary Log Retention

Debezium needs the MySQL binary logs to remain available long enough for the connector to read them. If the connector is stopped and RDS purges old binary logs, Debezium may not be able to resume from its previous position.

AWS provides the stored procedure mysql.rds_set_configuration to configure binary log retention hours for RDS MySQL. The default value for binlog retention hours is NULL, which means binary logs are not retained for external replication use.

Run:

SET SQL_SAFE_UPDATES = 0;

CALL mysql.rds_set_configuration('binlog retention hours', 48);

CALL mysql.rds_show_configuration;

Expected output:

name                     value
binlog retention hours   48

If MySQL Workbench returns this error:

Error Code: 1175. You are using safe update mode

then disable safe update mode for the session:

SET SQL_SAFE_UPDATES = 0;
CALL mysql.rds_set_configuration('binlog retention hours', 48);
CALL mysql.rds_show_configuration;

8. Create a MySQL User for Debezium

Create a dedicated MySQL user for Debezium.

CREATE USER 'debezium_user'@'%' IDENTIFIED BY 'YourStrongPasswordHere';

GRANT SELECT, RELOAD, SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT, LOCK TABLES
ON *.* TO 'debezium_user'@'%';

FLUSH PRIVILEGES;

For newer MySQL versions, if REPLICATION SLAVE is rejected, use:

GRANT SELECT, RELOAD, SHOW DATABASES, REPLICATION REPLICA, REPLICATION CLIENT, LOCK TABLES
ON *.* TO 'debezium_user'@'%';

FLUSH PRIVILEGES;

Validate using the Debezium user:

mysql -h onsysdb-01.ctrgy22e11p4.ap-southeast-2.rds.amazonaws.com -P 3306 -u debezium_user -p

Then run:

SELECT 
  @@hostname,
  @@server_id,
  @@global.binlog_format AS global_binlog_format,
  @@session.binlog_format AS session_binlog_format,
  @@global.binlog_row_image AS global_binlog_row_image,
  @@global.log_bin AS global_log_bin;

Both global and session binlog format should show:

ROW

9. Prepare SQL Server Target Database

Create a target database and login in SQL Server.

CREATE DATABASE DebeziumPOC;
GO

CREATE LOGIN debezium_sink
WITH PASSWORD = 'YourStrongSqlPasswordHere',
CHECK_POLICY = OFF;
GO

USE DebeziumPOC;
GO

CREATE USER debezium_sink FOR LOGIN debezium_sink
WITH DEFAULT_SCHEMA = dbo;
GO

ALTER ROLE db_owner ADD MEMBER debezium_sink;
GO

For a POC, db_owner is acceptable. For production, use least-privilege access.

10. Install and Start Debezium on Windows Using Docker

Create a working directory:

mkdir C:\debezium-rds-mssql-poc
cd C:\debezium-rds-mssql-poc

Create docker-compose.yml:

@'
services:
  kafka:
    image: quay.io/debezium/kafka:3.4
    container_name: dbz-kafka
    hostname: kafka
    ports:
      - "9092:9092"
    environment:
      CLUSTER_ID: "debezium-cluster-1"
      NODE_ID: "1"
      NODE_ROLE: "combined"
      KAFKA_CONTROLLER_QUORUM_VOTERS: "1@kafka:9093"
      KAFKA_LISTENERS: "PLAINTEXT://kafka:9092,CONTROLLER://kafka:9093"
      KAFKA_ADVERTISED_LISTENERS: "PLAINTEXT://kafka:9092"
      KAFKA_CONTROLLER_LISTENER_NAMES: "CONTROLLER"
      KAFKA_INTER_BROKER_LISTENER_NAME: "PLAINTEXT"

  connect:
    build:
      context: .
      dockerfile: Dockerfile.connect
    container_name: dbz-connect
    ports:
      - "8083:8083"
    environment:
      BOOTSTRAP_SERVERS: "kafka:9092"
      GROUP_ID: "1"
      CONFIG_STORAGE_TOPIC: "connect_configs"
      OFFSET_STORAGE_TOPIC: "connect_offsets"
      STATUS_STORAGE_TOPIC: "connect_statuses"
      CONNECT_PLUGIN_PATH: "/kafka/connect,/kafka/connect/debezium-connector-jdbc"
    depends_on:
      - kafka
'@ | Set-Content .\docker-compose.yml

Create Dockerfile.connect to include the SQL Server JDBC driver:

@'
FROM quay.io/debezium/connect:3.4

USER root

RUN mkdir -p /kafka/connect/debezium-connector-jdbc && \
    curl -fSL \
    https://repo1.maven.org/maven2/com/microsoft/sqlserver/mssql-jdbc/13.4.0.jre11/mssql-jdbc-13.4.0.jre11.jar \
    -o /kafka/connect/debezium-connector-jdbc/mssql-jdbc-13.4.0.jre11.jar && \
    chown -R kafka:kafka /kafka/connect/debezium-connector-jdbc

USER kafka
'@ | Set-Content .\Dockerfile.connect

Start the containers:

docker compose up -d --build

Check status:

docker ps

Validate Kafka Connect:

curl.exe http://localhost:8083/
curl.exe http://localhost:8083/connector-plugins

Expected connector plugins should include:

io.debezium.connector.mysql.MySqlConnector
io.debezium.connector.jdbc.JdbcSinkConnector

11. Create MySQL Test Table

Use a simple test table with a primary key. A primary key is important because the sink connector needs a reliable key for upserts and deletes.

Connect to MySQL and run:

USE YOUR_MYSQL_DATABASE;

DROP TABLE IF EXISTS debezium_poc_customer;

CREATE TABLE debezium_poc_customer (
    id INT NOT NULL,
    customer_code VARCHAR(30) NOT NULL,
    full_name VARCHAR(100) NOT NULL,
    email VARCHAR(150) NULL,
    status VARCHAR(20) NOT NULL DEFAULT 'ACTIVE',
    credit_limit DECIMAL(10,2) NOT NULL DEFAULT 0.00,
    notes VARCHAR(255) NULL,
    PRIMARY KEY (id),
    UNIQUE KEY uk_debezium_poc_customer_code (customer_code)
) ENGINE=InnoDB;

Insert test data:

INSERT INTO debezium_poc_customer
(id, customer_code, full_name, email, status, credit_limit, notes)
VALUES
(1, 'CUST-001', 'R Test Customer', 'r.test@example.com', 'ACTIVE', 1000.00, 'Initial Debezium test row'),
(2, 'CUST-002', 'Onsys Test Customer', 'onsys.test@example.com', 'ACTIVE', 2500.00, 'Second test row'),
(3, 'CUST-003', 'SQL Server Target Test', 'sqlserver.test@example.com', 'ACTIVE', 5000.00, 'Third test row');

COMMIT;

SELECT * FROM debezium_poc_customer;

12. Create SQL Server Target Table

You can allow Debezium to create the table automatically using schema evolution, but for a controlled POC, create the SQL Server table manually.

USE DebeziumPOC;
GO

IF OBJECT_ID('dbo.debezium_poc_customer', 'U') IS NOT NULL
BEGIN
    DROP TABLE dbo.debezium_poc_customer;
END
GO

CREATE TABLE dbo.debezium_poc_customer (
    id INT NOT NULL PRIMARY KEY,
    customer_code VARCHAR(30) NOT NULL,
    full_name VARCHAR(100) NOT NULL,
    email VARCHAR(150) NULL,
    status VARCHAR(20) NOT NULL,
    credit_limit DECIMAL(10,2) NOT NULL,
    notes VARCHAR(255) NULL
);
GO

SELECT * FROM dbo.debezium_poc_customer;

Expected result:

0 rows

13. Register the Debezium MySQL Source Connector

Create mysql-source.json:

cd C:\debezium-rds-mssql-poc

@'
{
  "name": "onsys-rds-mysql-source",
  "config": {
    "connector.class": "io.debezium.connector.mysql.MySqlConnector",
    "tasks.max": "1",

    "database.hostname": "onsysdb-01.ctrgy22e11p4.ap-southeast-2.rds.amazonaws.com",
    "database.port": "3306",
    "database.user": "debezium_user",
    "database.password": "YourStrongMysqlPasswordHere",

    "database.server.id": "5401",
    "topic.prefix": "onsysdb01",

    "database.include.list": "YOUR_MYSQL_DATABASE",
    "table.include.list": "YOUR_MYSQL_DATABASE.debezium_poc_customer",

    "snapshot.mode": "initial",
    "include.schema.changes": "false",

    "schema.history.internal.kafka.bootstrap.servers": "kafka:9092",
    "schema.history.internal.kafka.topic": "schemahistory.onsysdb01",

    "decimal.handling.mode": "string",
    "tombstones.on.delete": "false"
  }
}
'@ | Set-Content .\mysql-source.json

curl.exe -i -X POST `
  -H "Accept:application/json" `
  -H "Content-Type:application/json" `
  --data "@mysql-source.json" `
  http://localhost:8083/connectors

Check status:

curl.exe http://localhost:8083/connectors/onsys-rds-mysql-source/status

Expected:

"state":"RUNNING"

The task should also show:

"state":"RUNNING"

If the connector exists already, update it using the config endpoint:

(Get-Content .\mysql-source.json -Raw | ConvertFrom-Json).config |
  ConvertTo-Json -Depth 20 |
  Set-Content .\mysql-source-config.json

curl.exe -i -X PUT `
  -H "Accept:application/json" `
  -H "Content-Type:application/json" `
  --data "@mysql-source-config.json" `
  http://localhost:8083/connectors/onsys-rds-mysql-source/config

curl.exe -X POST "http://localhost:8083/connectors/onsys-rds-mysql-source/restart?includeTasks=true"

14. Register the SQL Server JDBC Sink Connector

The Debezium JDBC sink connector will read the MySQL table topic and write the result into SQL Server.

Create mssql-sink.json:

cd C:\debezium-rds-mssql-poc

@'
{
  "name": "mssql-jdbc-sink",
  "config": {
    "connector.class": "io.debezium.connector.jdbc.JdbcSinkConnector",
    "tasks.max": "1",

    "topics": "onsysdb01.YOUR_MYSQL_DATABASE.debezium_poc_customer",

    "connection.url": "jdbc:sqlserver://192.168.1.51:1433;databaseName=DebeziumPOC;encrypt=true;trustServerCertificate=true",
    "connection.username": "debezium_sink",
    "connection.password": "YourStrongSqlPasswordHere",

    "insert.mode": "upsert",
    "delete.enabled": "true",
    "primary.key.mode": "record_key",

    "schema.evolution": "basic",
    "collection.name.format": "${source.table}",
    "use.time.zone": "UTC"
  }
}
'@ | Set-Content .\mssql-sink.json

The connection string uses encrypt=true and trustServerCertificate=true. Microsoft documents these JDBC connection string properties for encrypted SQL Server connections.

curl.exe -i -X POST `
  -H "Accept:application/json" `
  -H "Content-Type:application/json" `
  --data "@mssql-sink.json" `
  http://localhost:8083/connectors

If the connector already exists, you will see:

HTTP/1.1 409 Conflict
Connector mssql-jdbc-sink already exists

That is normal. Update the existing connector instead:

(Get-Content .\mssql-sink.json -Raw | ConvertFrom-Json).config |
  ConvertTo-Json -Depth 20 |
  Set-Content .\mssql-sink-config.json

curl.exe -i -X PUT `
  -H "Accept:application/json" `
  -H "Content-Type:application/json" `
  --data "@mssql-sink-config.json" `
  http://localhost:8083/connectors/mssql-jdbc-sink/config

curl.exe -X POST http://localhost:8083/connectors/mssql-jdbc-sink/restart

Check status:

curl.exe http://localhost:8083/connectors/mssql-jdbc-sink/status

Expected:

"state":"RUNNING"

The JDBC sink connector requires delete mode to be explicitly enabled using delete.enabled=true. Deletes also require primary-key mapping so that the sink knows which target row to remove.

15. Monitor Kafka Topics

List Kafka topics:

docker compose exec kafka /kafka/bin/kafka-topics.sh `
  --bootstrap-server kafka:9092 `
  --list

Expected topic:

onsysdb01.YOUR_MYSQL_DATABASE.debezium_poc_customer

Read messages from the topic:

docker compose exec kafka /kafka/bin/kafka-console-consumer.sh `
  --bootstrap-server kafka:9092 `
  --topic onsysdb01.YOUR_MYSQL_DATABASE.debezium_poc_customer `
  --from-beginning `
  --property print.key=true `
  --max-messages 10

You should see JSON messages for the initial snapshot rows.

16. Validate Initial Snapshot in SQL Server

Run this in SQL Server:

USE DebeziumPOC;
GO

SELECT 
    id,
    customer_code,
    full_name,
    email,
    status,
    credit_limit,
    notes
FROM dbo.debezium_poc_customer
ORDER BY id;
GO

Expected rows:

1   CUST-001   Ranil Test Customer
2   CUST-002   Onsys Test Customer
3   CUST-003   SQL Server Target Test

Check row count:

SELECT COUNT(*) AS sqlserver_row_count
FROM dbo.debezium_poc_customer;
GO

Expected:

17. Test INSERT Replication

Run this in MySQL:

USE YOUR_MYSQL_DATABASE;

INSERT INTO debezium_poc_customer
(id, customer_code, full_name, email, status, credit_limit, notes)
VALUES
(4, 'CUST-004', 'Insert Replication Test', 'insert.test@example.com', 'ACTIVE', 7500.00, 'Inserted after Debezium started');

COMMIT;

SELECT * 
FROM debezium_poc_customer 
WHERE id = 4;

Validate in SQL Server:

USE DebeziumPOC;
GO

SELECT *
FROM dbo.debezium_poc_customer
WHERE id = 4;
GO

Expected result:

CUST-004 appears in SQL Server

18. Test UPDATE Replication

Run this in MySQL:

USE YOUR_MYSQL_DATABASE;

UPDATE debezium_poc_customer
SET 
    full_name = 'Updated Replication Test',
    status = 'SUSPENDED',
    credit_limit = 9999.99,
    notes = 'Updated from MySQL and replicated by Debezium'
WHERE id = 4;

COMMIT;

SELECT * 
FROM debezium_poc_customer 
WHERE id = 4;

Validate in SQL Server:

USE DebeziumPOC;
GO

SELECT *
FROM dbo.debezium_poc_customer
WHERE id = 4;
GO

Expected values:

full_name     Updated Replication Test
status        SUSPENDED
credit_limit  9999.99

19. Test DELETE Replication

Run this in MySQL:

USE YOUR_MYSQL_DATABASE;

DELETE FROM debezium_poc_customer
WHERE id = 4;

COMMIT;

SELECT * 
FROM debezium_poc_customer 
WHERE id = 4;

Validate in SQL Server:

USE DebeziumPOC;
GO

SELECT *
FROM dbo.debezium_poc_customer
WHERE id = 4;
GO

Expected result:

No rows returned

If the row still exists in SQL Server, check the sink connector configuration:

"delete.enabled": "true",
"primary.key.mode": "record_key"

20. End-to-End Validation Queries

Run this on MySQL:

USE YOUR_MYSQL_DATABASE;

SELECT 
    COUNT(*) AS mysql_count,
    SUM(credit_limit) AS mysql_total_credit
FROM debezium_poc_customer;

Run this on SQL Server:

USE DebeziumPOC;
GO

SELECT 
    COUNT(*) AS sqlserver_count,
    SUM(credit_limit) AS sqlserver_total_credit
FROM dbo.debezium_poc_customer;
GO

The counts and totals should match.

You can also compare individual rows:

-- MySQL
SELECT *
FROM debezium_poc_customer
ORDER BY id;

-- SQL Server
SELECT *
FROM dbo.debezium_poc_customer
ORDER BY id;

21. Connector Monitoring Commands

List all connectors:

curl.exe http://localhost:8083/connectors

Check MySQL source connector status:

curl.exe http://localhost:8083/connectors/onsys-rds-mysql-source/status

Check SQL Server sink connector status:

curl.exe http://localhost:8083/connectors/mssql-jdbc-sink/status

View MySQL source connector configuration:

curl.exe http://localhost:8083/connectors/onsys-rds-mysql-source/config

View SQL Server sink connector configuration:

curl.exe http://localhost:8083/connectors/mssql-jdbc-sink/config

Restart the MySQL source connector including failed tasks:

curl.exe -X POST "http://localhost:8083/connectors/onsys-rds-mysql-source/restart?includeTasks=true"

Restart the SQL Server sink connector:

curl.exe -X POST http://localhost:8083/connectors/mssql-jdbc-sink/restart

View container logs:

docker compose logs --tail=300 connect

Follow logs live:

docker compose logs -f connect

22. Common Issue: Connector Already Exists

When registering a connector, this error can appear:

HTTP/1.1 409 Conflict
Connector mssql-jdbc-sink already exists

This means the connector is already registered. Do not use POST again. Use PUT to update the connector configuration.

For example:

(Get-Content .\mssql-sink.json -Raw | ConvertFrom-Json).config |
  ConvertTo-Json -Depth 20 |
  Set-Content .\mssql-sink-config.json

curl.exe -i -X PUT `
  -H "Accept:application/json" `
  -H "Content-Type:application/json" `
  --data "@mssql-sink-config.json" `
  http://localhost:8083/connectors/mssql-jdbc-sink/config

Then restart:

curl.exe -X POST http://localhost:8083/connectors/mssql-jdbc-sink/restart

23. Common Issue: `binlog_format` Still Fails After Changing RDS

A very common error is:

The database server is not configured to use a ROW binlog_format

This is not fixed by changing the table. It is fixed at the RDS parameter group level.

Check using the same user Debezium uses:

mysql -h onsysdb-01.ctrgy22e11p4.ap-southeast-2.rds.amazonaws.com -P 3306 -u debezium_user -p

Then:

SELECT 
  @@global.binlog_format AS global_binlog_format,
  @@session.binlog_format AS session_binlog_format,
  @@global.binlog_row_image AS global_binlog_row_image,
  @@global.log_bin AS global_log_bin;

Expected:

global_binlog_format     ROW
session_binlog_format    ROW
global_binlog_row_image  FULL
global_log_bin           1

If it still does not show ROW, check:

The correct RDS parameter group was changed
The parameter group is attached to the correct DB instance
The DB instance was rebooted
The connector is pointing to the correct RDS endpoint
The Debezium user is connecting to the same server you tested manually

24. Common Issue: Docker Image Tag Not Found

If this error appears:

failed to resolve reference "quay.io/debezium/zookeeper:3.6": not found

use a known working Debezium image version for the POC, for example:

quay.io/debezium/kafka:3.4
quay.io/debezium/connect:3.4

Then rebuild:

docker compose down -v
docker compose up -d --build

25. Common Issue: Sink Is Running but SQL Server Has No Rows

If the sink connector status shows RUNNING but SQL Server is empty, check the following.

First, list Kafka topics:

docker compose exec kafka /kafka/bin/kafka-topics.sh `
  --bootstrap-server kafka:9092 `
  --list

Confirm the topic name exactly matches the sink connector:

"topics": "onsysdb01.YOUR_MYSQL_DATABASE.debezium_poc_customer"

Then check whether the topic has messages:

docker compose exec kafka /kafka/bin/kafka-console-consumer.sh `
  --bootstrap-server kafka:9092 `
  --topic onsysdb01.YOUR_MYSQL_DATABASE.debezium_poc_customer `
  --from-beginning `
  --property print.key=true `
  --max-messages 10

Then check SQL Server connectivity from inside the Docker container:

docker compose exec connect bash -c "echo > /dev/tcp/192.168.1.51/1433 && echo SQL_PORT_OPEN"

Expected:

SQL_PORT_OPEN

If this fails, the problem is network access from the Docker container to SQL Server.

26. Resetting the POC

For a POC only, you can reset everything:

curl.exe -X DELETE http://localhost:8083/connectors/onsys-rds-mysql-source
curl.exe -X DELETE http://localhost:8083/connectors/mssql-jdbc-sink

docker compose down -v
docker compose up -d --build

This deletes connector state, Kafka data, topics, and offsets. Do not use this reset approach in production unless you fully understand the impact.

27. Production Considerations

This Windows-based setup is suitable for a proof of concept, but not ideal for production.

For production, consider:

Run Debezium close to the RDS network
Use EC2, ECS, EKS, MSK, or a managed Kafka platform
Secure database credentials using secrets management
Monitor connector lag and task failures
Increase binlog retention based on recovery requirements
Avoid running production CDC on a desktop or laptop
Design table mappings carefully
Handle schema changes through a controlled release process
Test failover and restart behaviour
Back up connector configs and Kafka Connect offsets

Also review tables before including them in production replication:

Does the table have a primary key?
Are there large BLOB/TEXT columns?
Are there unsupported or awkward data types?
Does the application use frequent DDL changes?
Are deletes expected to be physically replicated?
Is SQL Server allowed to auto-create or alter target tables?

Conclusion

Debezium provides a powerful and flexible way to implement CDC-based replication from AWS RDS MySQL into SQL Server. The key is understanding the pipeline:

MySQL binlog
→ Debezium source connector
→ Kafka topic
→ JDBC sink connector
→ SQL Server table

The most important configuration requirement is that MySQL must use:

binlog_format = ROW
binlog_row_image = FULL

For AWS RDS, this normally means changing the DB parameter group, attaching it to the correct instance, and rebooting the database. Once MySQL is correctly configured, the Debezium MySQL connector can capture the initial snapshot and continue streaming changes into Kafka. The JDBC sink connector can then apply those changes to SQL Server using upsert and delete handling.

For a controlled POC, start with one table, verify insert/update/delete replication, then expand to additional tables after confirming stability and data type compatibility.

Building a Debezium CDC Proof of Concept

Introduction

1. What Debezium Does

2. POC Scope

3. Software Components

4. Network Connectivity Checks

5. Prepare AWS RDS MySQL for Debezium

6. Configure the RDS Parameter Group

7. Configure RDS Binary Log Retention

8. Create a MySQL User for Debezium

9. Prepare SQL Server Target Database

10. Install and Start Debezium on Windows Using Docker

11. Create MySQL Test Table

12. Create SQL Server Target Table

13. Register the Debezium MySQL Source Connector

14. Register the SQL Server JDBC Sink Connector

15. Monitor Kafka Topics

16. Validate Initial Snapshot in SQL Server

17. Test INSERT Replication

18. Test UPDATE Replication

19. Test DELETE Replication

20. End-to-End Validation Queries

21. Connector Monitoring Commands

22. Common Issue: Connector Already Exists

23. Common Issue: `binlog_format` Still Fails After Changing RDS

24. Common Issue: Docker Image Tag Not Found

25. Common Issue: Sink Is Running but SQL Server Has No Rows

26. Resetting the POC

27. Production Considerations

Conclusion

More from the team

How to Update SQL Server After Cloning and Renaming a Windows Server

How to Find and List Oracle Archive Logs for Specific Dates and RMAN Backups

Secure Your Reports: How to Enable SSL/TLS Certificate on SQL Server Reporting Services 2019

Need a hand with your environment?

Replicating AWS RDS MySQL to SQL Server on Windows

Building a Debezium CDC Proof of Concept

Introduction

1. What Debezium Does

2. POC Scope

3. Software Components

4. Network Connectivity Checks

5. Prepare AWS RDS MySQL for Debezium

6. Configure the RDS Parameter Group

7. Configure RDS Binary Log Retention

8. Create a MySQL User for Debezium

9. Prepare SQL Server Target Database

10. Install and Start Debezium on Windows Using Docker

11. Create MySQL Test Table

12. Create SQL Server Target Table

13. Register the Debezium MySQL Source Connector

14. Register the SQL Server JDBC Sink Connector

15. Monitor Kafka Topics

16. Validate Initial Snapshot in SQL Server

17. Test INSERT Replication

18. Test UPDATE Replication

19. Test DELETE Replication

20. End-to-End Validation Queries

21. Connector Monitoring Commands

22. Common Issue: Connector Already Exists

23. Common Issue: binlog_format Still Fails After Changing RDS

24. Common Issue: Docker Image Tag Not Found

25. Common Issue: Sink Is Running but SQL Server Has No Rows

26. Resetting the POC

27. Production Considerations

Conclusion

More from the team

How to Update SQL Server After Cloning and Renaming a Windows Server

How to Find and List Oracle Archive Logs for Specific Dates and RMAN Backups

Secure Your Reports: How to Enable SSL/TLS Certificate on SQL Server Reporting Services 2019

Need a hand with your environment?

23. Common Issue: `binlog_format` Still Fails After Changing RDS