Azure MySQL (Edge)

Prerequisites

• The Azure MySQL database instance is running. To check this:

  1. Log in to the Microsoft Azure Portal and select the Azure MySQL database you want to connect to Hevo.

  2. In the Overview tab, confirm that the Status field has the value Available.

     

• The MySQL version is 5.7 or 8.0. You can choose the MySQL version while creating the instance.

• Binary Log (BinLog) replication is enabled.

  Note: Hevo currently does not support ingesting data from read replicas.

• Hevo’s IP address(es) for your region is added to the Azure MySQL database IP Allowlist.

• The SELECT, RELOAD, SHOW DATABASES, and REPLICATION privileges are granted to the database user.

  Note: We recommend that you create a database user for configuring your Azure MySQL Source in Hevo. However, if you already have one, refer to section Grant privileges to the user.

• The database hostname and port number of the Source instance are available.

Perform the following steps to configure your Azure MySQL Source:

Set up MySQL Binary Logs for Replication

Hevo supports data ingestion from the MySQL database instance via binary logs (BinLog). A binary log is a collection of log files that records information about data modifications and data object modifications made on a MySQL database instance. Typically, binary logs are used for data replication and data recovery.

Perform the following steps to enable BinLog replication on your Azure MySQL database instance:

1. Log in to the Azure Portal.

2. Under Azure services, select More services.

   

3. On the All services page, search for and select All resources.

   

4. On the All resources page, click the MySQL database you want to connect to Hevo.

   

5. In the left navigation pane, under Settings, click Server Parameters.

6. On the Server Parameters page, do the following:

   

   1. Under the Top tab, search and update the values of the following parameters:

      Parameter Name	Value
      binlog_row_image	full
      binlog_expire_logs_seconds	A value greater than or equal to 259200 (three days).
      gtid_mode	ON
      enforce_gtid_consistency	ON

      Note: Enabling Global Transaction Identifiers (GTIDs) is recommended because it simplifies replication by uniquely identifying transactions. This makes it easier to track them and ensures that the primary and replica servers are in sync.

   2. Click Save.

7. Confirm under Notifications that your changes have been applied and the instance has restarted successfully before running the Pipeline, to avoid errors.

   

Allowlist Hevo IP addresses for your region

You need to allowlist the Hevo IP addresses for your region to enable Hevo to connect to your Microsoft Azure MySQL database. You can do this by creating firewall rules in your Microsoft Azure database settings as follows:

1. Log in to the Azure Portal.

2. Under Azure services, select More services.

   

3. On the All services page, search for and select All resources.

   

4. On the All resources page, click the MySQL database you want to connect to Hevo.

   

5. In the left navigation pane, under Settings, click Networking.

6. Create a Firewall Rule:

   1. Specify a Firewall Rule Name.

   2. Specify Hevo’s IP addresses in the Start IP address and End IP address fields as per your region, and then click Save to save the rule.

      Note: As Hevo has specific IP addresses and not a range, the value in Start IP address and End IP address fields is the same.

      

   3. Repeat this step to add the IP address for each applicable Hevo region.

Create a Database User and Grant Privileges

1. Create a database user (Optional)

2. Grant privileges to the user

The database user for Hevo requires the following privileges:

Connect to your Azure MySQL database as an admin user with an SQL client tool, such as MySQL Workbench, and run the following script. These commands grant only the privileges required by Hevo to ingest data from your Azure MySQL database.

# Grant Privileges to the Database User
GRANT RELOAD, SHOW DATABASES, REPLICATION CLIENT, REPLICATION SLAVE ON *.* TO <database_username>@'%';

# (Optional) Grant Hevo access to all Tables in the Database
GRANT SELECT ON <database_name>.* TO <database_username>;

# Grant Hevo access to the Tables in the Database that you want to replicate
GRANT SELECT, SHOW VIEW ON <database_name>.<table_name> TO <database_username>;

# (Optional) View the grants for the user:
SHOW GRANTS FOR <database_username>@'%';

# Finalize the User’s Permissions
FLUSH PRIVILEGES;

Note:

• Replace the placeholder values in the commands above with your own. For example, <database_username> with hevo.

• The SELECT, RELOAD, and SHOW DATABASES privileges are required only for the historical load.

• The REPLICATION SLAVE privilege is required only if you connect to a read replica. When it is granted to the authenticating user, the replica logs any updates received from the main database, maintaining a record of those changes in its log.

Retrieve the Database Hostname and Port Number (Optional)