Tales From A Lazy Fat DBA

Its all about Databases, their performance, troubleshooting & much more …. ¯\_(ツ)_/¯

Posts Tagged ‘DataPump’

Adapting Data Pump for the Cloud: Trailer Blocks in 23ai

Posted by FatDBA on November 1, 2024

Oracle Database 23c AI introduces a new trailer block format for Data Pump dump files, moving away from the traditional header block approach used in earlier versions like 21c. This new structure is designed specifically to support cloud storage environments, particularly within Oracle Cloud Infrastructure (OCI), and makes it easier to manage and transfer Data Pump files to object stores.

What’s Different with Trailer Blocks?

Trailer blocks are written only once the export process completes, contrasting with header blocks, which required upfront metadata and continuous updates throughout the job. This deferred approach means trailer blocks accumulate necessary metadata during the export process, allowing it to be stored in a more cloud-compatible format at the end. The switch to trailer blocks by default enables seamless storage of dump files in cloud object stores and reduces the limitations associated with local-only file systems.

Note : Trailer block files are compatible only with Oracle Database 23c AI and later. To ensure backward compatibility for previous Oracle versions, you can use the VERSION parameter to revert to the legacy header block format.

How to Use Trailer Blocks with Cloud Object Storage

When running Data Pump in Oracle 23c AI, trailer blocks allow straightforward use with object storage by configuring these essential parameters:

  1. Authenticate with Cloud Credentials: Ensure you have a credential object in place to establish the required connection with your object store.
  2. Configure DUMPFILE and CREDENTIAL Parameters: List export files using DUMPFILE and specify the credential to direct these files to the cloud.
  3. Set Log File Storage: Use the DEFAULT_DIRECTORY parameter to control log file storage. Failing to set up a valid cloud credential for object store URIs will lead to errors in dump file writing.

Steps for Cloud Storage Access

To export files directly to a cloud object store, ensure you have cloud credentials stored in an Oracle Wallet, with the location specified in your sqlnet.ora configuration. You will also need to add a certificate for authentication using orapki:

orapki wallet add -wallet <path_to_wallet> -trusted_cert -cert <certificate_file> -pwd <wallet_password>

This setup allows the Data Pump to use the CREDENTIAL parameter with the export command, establishing secure access to cloud storage.

Header Blocks vs. Trailer Blocks

In Oracle Database versions prior to 23c, Data Pump files used header blocks, where metadata was embedded at the start of each dump file. This format was limited to local storage and required metadata updates during export, which could be resource-intensive.

With Oracle Database 23c AI, trailer blocks are the default. This approach shifts metadata to the end of the file, streamlining the process for cloud storage. By using the VERSION parameter, you can control whether to retain the newer trailer block format or revert to header blocks based on compatibility needs.

Types of Trailer Blocks in Oracle 23c AI

Oracle 23c AI introduces two types of trailer blocks for handling Data Pump files:

  • Disk-Based Trailer Blocks: Directly written at the end of each primary table’s data in the dump file, providing a compact storage format.
  • Table-Based Trailer Blocks: Stored externally within the export’s primary table, conserving space by omitting individual trailer blocks for each data object. This approach efficiently manages storage while maintaining essential metadata for cloud-based and local export jobs.

Trailer blocks make Data Pump in Oracle Database 23c AI a flexible and cloud-ready solution, enabling efficient data movement across hybrid and cloud infrastructures. This updated approach supports larger cloud-based workflows by improving performance and storage efficiency for Data Pump exports.

Hope It Helped!
Prashant Dixit

Posted in Uncategorized | Tagged: , , , , , , , | Leave a Comment »

Export failed when trying using SYS user. * (ORA-28009: connection as SYS should be as SYSDBA or SYSOPER)

Posted by FatDBA on December 23, 2012

[oracle@localhost ~]$ expdp sys/oracle90 directory=dpump dumpfile=emp.dmp tables=larry.emp reuse_dumpfiles=true

Errors:
UDE-28009: operation generated ORACLE error 28009
ORA-28009: connection as SYS should be as SYSDBA or SYSOPER

SQL> show parameter dictionary

NAME                                 TYPE        VALUE
———————————— ———– ——————————
O7_DICTIONARY_ACCESSIBILITY          boolean     FALSE

 

With this parameter in False, it will not allow accessing anyone dictionary views, tables objects in SYS schema without SYSDBA/SYSOPER roles. O7_DICTIONARY_ACCESSIBILITY controls restrictions on SYSTEM privileges. If the parameter is set to true, access to objects in the SYS schema is allowed (Oracle7 behavior). The default setting of false ensures that system privileges that allow access to objects in “any schema” do not allow access to objects in the SYS schema.

 

SQL> alter system set O7_DICTIONARY_ACCESSIBILITY=TRUE scope=spfile;
System altered.

SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.

SQL> startup
ORACLE instance started.

Total System Global Area  523108352 bytes
Fixed Size                  1337632 bytes
Variable Size             406849248 bytes
Database Buffers          109051904 bytes
Redo Buffers                5869568 bytes
Database mounted.
Database opened.

SQL> show parameter dictionary

NAME                                 TYPE        VALUE
———————————— ———– ——————————
O7_DICTIONARY_ACCESSIBILITY          boolean     TRUE

[oracle@localhost ~]$ expdp sys/oracle90 directory=dpump dumpfile=emp.dmp tables=larry.emp reuse_dumpfiles=true

Export: Release 11.2.0.1.0 – Production on Sun Dec 23 12:12:29 2012
Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.

ORA-28002: the password will expire within 7 days            //   **** This error points to change password for user the table belongs to ‘Larry’****

Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 – Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
Starting “SYS”.”SYS_EXPORT_TABLE_01″:  sys/******** directory=dpump dumpfile=emp.dmp tables=larry.emp reuse_dumpfiles=true
Estimate in progress using BLOCKS method…
Processing object type TABLE_EXPORT/TABLE/TABLE_DATA
Total estimation using BLOCKS method: 64 KB
Processing object type TABLE_EXPORT/TABLE/TABLE
Processing object type TABLE_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS
. . exported “LARRY”.”EMP”                               8.046 KB       2 rows
Master table “SYS”.”SYS_EXPORT_TABLE_01″ successfully loaded/unloaded
******************************************************************************
Dump file set for SYS.SYS_EXPORT_TABLE_01 is:
/tmp/emp.dmp
Job “SYS”.”SYS_EXPORT_TABLE_01″ successfully completed at 12:12:47

 

Or —->>>>
Export it with SYSDBA role.

[oracle@localhost ~]$ expdp \’sys/oracle90 as sysdba\’ directory=dpump dumpfile=emp.dmp tables=larry.emp reuse_dumpfiles=true

Export: Release 11.2.0.1.0 – Production on Sun Dec 23 16:42:15 2012

Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.

Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 – Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
Starting “SYS”.”SYS_EXPORT_TABLE_01″:  “sys/******** AS SYSDBA” directory=dpump dumpfile=emp.dmp tables=larry.emp reuse_dumpfiles=true
Estimate in progress using BLOCKS method…

Posted in Advanced | Tagged: | 2 Comments »

Master Table – Data Pump Jobs

Posted by FatDBA on July 14, 2012

What is a Master Table & what’s the significance or importance of  it during a Data Pump Job ?

At the heart of the Data Pump operation is the Master Table.  This table is created at the beginning of a Data Pump operation and is dropped at the end of the successful completion of a Data Pump operation. The Master Table can also be dropped if the job is killed using the kill_job interactive command. If a job is stopped using the stop_job interactive command or if the job is terminated unexpectedly, the Master Table will be retained.
The Master Table is used to track the detailed progress information of a Data Pump job:

* The current set of dump files.
* The current state of every object exported or imported and their locations in the dump file set.
* The job’s user-supplied parameters.
* The status of every worker process.
* The state of current job status and restart information.

The Master Table is created in the schema of the current user running the Pump Dump export or import, and it keeps tracks of lots of detailed information.  As a result, this table can take up a significant amount of storage space.

[oracle@localhost /]$ expdp larry/champion tables=etr,orders directory=dpump

Export: Release 10.2.0.1.0 – Production on Tuesday, 01 May, 2012 23:27:46

Copyright (c) 2003, 2005, Oracle.  All rights reserved.

Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 – Production
With the Partitioning, OLAP and Data Mining options
Starting “LARRY”.”SYS_EXPORT_TABLE_01″:  larry/******** tables=etr,orders directory=dpump dumpfile=test.dmp logfile=testlog.log content=all
Estimate in progress using BLOCKS method…
Processing object type TABLE_EXPORT/TABLE/TABLE_DATA
Total estimation using BLOCKS method: 128 KB
Processing object type TABLE_EXPORT/TABLE/TABLE
Processing object type TABLE_EXPORT/TABLE/INDEX/INDEX
Processing object type TABLE_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS
Processing object type TABLE_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS
. . exported “LARRY”.”ETR”                            5.648 KB       7 rows
. . exported “LARRY”.”ORDERS”                            5.734 KB       5 rows
Master table “LARRY”.”SYS_EXPORT_TABLE_01″ successfully loaded/unloaded
******************************************************************************
Dump file set for LARRY.SYS_EXPORT_TABLE_01 is:
/u01/app/oracle/oradata/test.dmp
Job “LARRY”.”SYS_EXPORT_TABLE_01″ successfully completed at 23:27:51

Example: As highlighted in Bold the Master Table is successfully loaded and unloaded at the end of the Data Pump Job. Created in the same schema (Larry) which initiated the DP job as underlined created by the name of: SYS_EXPORT_TABLE_01

Let the Red Prevail …
Prashant D

Posted in Advanced | Tagged: , , | 2 Comments »