Transferring data from an Oracle source endpoint
Yandex Data Transfer enables you to migrate data from an Oracle database and implement various scenarios of data transfer, processing and transformation. To implement a transfer:
- Explore possible data transfer scenarios.
- Prepare the Oracle database for the transfer.
- Set up an endpoint source in Yandex Data Transfer.
- Set up one of the supported data targets.
- Create a transfer and start it.
- Perform required operations with the database and control the transfer.
- In case of any issues, use ready-made solutions to resolve them.
Scenarios for transferring data from Oracle
-
Migration: Moving data from one storage to another. Migration often means migrating a database from obsolete local databases to managed cloud ones.
-
Uploading data to data marts is a process of transferring prepared data to storage for subsequent visualization.
For a detailed description of possible Yandex Data Transfer data transfer scenarios, see Tutorials.
Preparing the source database
Note
Some versions of Oracle use V_$
instead of V$
as the prefix for system objects. For example, V_$DATABASE
instead of V$DATABASE
.
If you get an error like "can only select from fixed tables/views
" when granting permissions to system objects, try changing the prefixes.
-
To prepare the source for the Snapshot transfer:
-
Create a user account the transfer will use to connect to the source:
CREATE USER <username> IDENTIFIED BY <password>; GRANT CREATE SESSION TO <username>;
-
Grant privileges to the created user:
GRANT SELECT ON V$DATABASE TO <username>; GRANT SELECT ON DBA_EXTENTS TO <username>; GRANT SELECT ON DBA_OBJECTS TO <username>; GRANT FLASHBACK ANY TABLE TO <username>;
If required, you can only grant the
FLASHBACK
privileges to the tables you need to copy rather than toANY TABLE
. -
Grant the user the privilege to read the tables
to be copied.
-
-
To prepare the source for the Replication transfer:
-
Create a user account the transfer will use to connect to the source:
CREATE USER <username> IDENTIFIED BY <password>; ALTER USER <username> DEFAULT tablespace USERS TEMPORARY tablespace TEMP; ALTER USER <username> quote unlimited on USERS; GRANT CREATE SESSION, execute_catalog_role, SELECT ANY TRANSACTION, SELECT ANY DISCTIONARY, CREATE PROCEDURE, LOGMINING TO <username>;
-
Grant privileges to the created user:
GRANT SELECT ON V$DATABASE TO <username>; GRANT SELECT ON V$LOG TO <username>; GRANT SELECT ON V$LOGFILE TO <username>; GRANT SELECT ON V$ARCHIVED_LOG TO <username>; GRANT SELECT ON dba_objects TO <username>; GRANT SELECT ON dba_extents TO <username>; GRANT EXECUTE ON SYS.DBMS_LOGMNR TO <username>; GRANT SELECT ON SYSTEM.LOGMNR_COL$ TO <username>; GRANT SELECT ON SYSTEM.LOGMNR_OBJ$ TO <username>; GRANT SELECT ON SYSTEM.LOGMNR_USER$ TO <username>; GRANT SELECT ON SYSTEM.LOGMNR_UID$ TO <username>;
-
Grant the user the privilege to read the tables
to be replicated. -
Enable Minimal Supplemental Logging
with primary keys as follows:ALTER DATABASE ADD SUPPLEMENTAL LOG DATA (PRIMARY KEY) COLUMNS;
-
-
If you are using the CDB environment
, configure the following settings:-
Create a user
Common User
:CREATE USER C##<username> IDENTIFIED BY <password> CONTAINER=all; ALTER USER C##<username> DEFAULT TABLESPACE USERS temporary tablespace TEMP CONTAINER=all; ALTER USER C##<username> quota unlimited on USERS CONTAINER=all; ALTER USER C##<username> SET container_data = (cdb$root, <your_PCB_name>) CONTAINER=current; GRANT CREATE SESSION, execute_catalog_role, SELECT ANY TRANSACTION, SELECT ANY DICTIONALY, CREATE PROCEDURE, LOGMINING, SET CONTAINER TO C##<username> CONTAINER=ALL;
If required, you can only specify the
cdb$root
container and the container with the tables you need to transfer. -
To allow the user to switch to the
cdb$root
container, grant them theALTER SESSION
privileges:GRANT ALTER SESSION TO C##<username>;
-
Grant privileges to the created user:
GRANT SELECT ON V$DATABASE TO C##<username> CONTAINER=ALL; GRANT SELECT ON V$LOG TO C##<username> CONTAINER=ALL; GRANT SELECT ON V$LOGFILE TO C##<username> CONTAINER=ALL; GRANT SELECT ON V$ARCHIVED_LOG TO C##<username> CONTAINER=ALL; GRANT SELECT ON dba_objects TO C##<username> CONTAINER=ALL; GRANT SELECT ON dba_extents TO C##<username> CONTAINER=ALL; GRANT EXECUTE ON SYS.DBMS_LOGMNR TO C##<username> CONTAINER=ALL; GRANT SELECT ON SYSTEM.LOGMNR_COL$ TO C##<username> CONTAINER=ALL; GRANT SELECT ON SYSTEM.LOGMNR_OBJ$ TO C##<username> CONTAINER=ALL; GRANT SELECT ON SYSTEM.LOGMNR_USER$ TO C##<username> CONTAINER=ALL; GRANT SELECT ON SYSTEM.LOGMNR_UID$ TO C##<username> CONTAINER=ALL;
-
Configuring the Oracle source endpoint
When creating or updating an endpoint, configure connection settings to Oracle.
The settings are given for a custom installation, where all fields are filled in manually.
-
Connection type: Select the Oracle cluster connection type
. -
For the Instance connection type:
-
Host: Enter the IP address or FQDN of the host you want to connect to.
-
Port: Set the number of the port that Data Transfer will use for the connection.
-
Type: Specify Oracle SID
or Service Name in the respective field below.
-
-
For the TNS connect string connection type, specify the Oracle RDBMS connection string in the respective field below.
-
User: Specify the username that Data Transfer will use to connect to the database.
-
Password: Enter the user's password to the database.
-
PDB container: Specify the name of the PDB container
to connect to. Leave the field empty if you are not using the CDB environment . -
Included tables: Data is only transferred from listed tables. This option is specified using regular expressions.
-
Excluded tables: Data from the listed tables is not transferred. This option is specified using regular expressions.
The lists include the name of the schema
(description of DB contents, structure, and integrity constraints) and the table name. Both lists support expressions in the following format:<schema_name>.<table_name>
: Fully qualified table name.<schema_name>.*
: All tables in the specified schema.<table_name>
: Table in the default schema.
Included and excluded table names must meet the ID naming rules in Oracle. For more information, see the Oracle documentation
. Double quotes within a table name are not supported. Escaping outer double quotes is not required. -
Convert the "NUMBER" type to "Int64": Enable this option to convert the type from
NUMBER
(without parameters) toINT64
. ParameterizedNUMBER (precision, scale)
types are automatically converted to integer types whenever possible.
Configuring the data target
Configure one of the supported data targets:
For a complete list of supported sources and targets in Yandex Data Transfer, see Available Transfers.
After configuring the data source and target, create and start the transfer.