Tales From A Lazy Fat DBA

Its all about Databases, their performance, troubleshooting & much more …. ¯\_(ツ)_/¯

Posts Tagged ‘archiving’

DBMS_ILM HTML Dashboard Script .. The Missing Monitor for Oracle Archiving

Posted by FatDBA on September 15, 2025

Somneone on LinkedIn recently asked me if its possible tocollect all environment info, licensing signals (Heat Map / ADO / Advanced Compression), ILM policies, jobs, results, evaluation details, Heat Map status, partition placement and sizes, archive tablespace usage, and to have a “since last run” growth delta by snapshotting partition sizes via any reports. Here is something that I use… This script creates lightweight repo tables on first run and reuses them on subsequent runs to compute deltas and presents all results in a neat HTML report.

Paste this into a .sql file and run it with SQL*Plus or SQLcl. No lists, just code and the HTML it produces.

-- ilm_monitor_report.sql
-- HTML dashboard for DBMS_ILM monitoring with partition growth deltas.
-- Run as a DBA account with access to DBA_* views.

-- ===== User-configurable parameters =====
define REPORT_DIR    = '/u01/app/oracle/admin/reports'
define REPORT_NAME   = 'ilm_report_&&_DATESTAMP..html'
define OWNER_LIKE    = '%PSFT%'                -- filter objects by owner (e.g. 'PSFT' or '%')
define TABLE_LIKE    = '%'                    -- filter table name pattern (e.g. 'PAY_CHECKS%' or '%')
define ARCHIVE_TS    = 'ARCHIVE_TS'           -- archive tablespace name to highlight
define DAYS_BACK     = 30                     -- window for AWR/feature usage context
-- =======================================

column _DATESTAMP new_value _DATESTAMP
select to_char(sysdate,'YYYYMMDD_HH24MISS') as _DATESTAMP from dual;

set termout on pages 0 echo off feedback off verify off trimspool on lines 500 long 100000 longchunksize 100000
set markup html on spool on entmap off preformat off
spool &&REPORT_DIR/&&REPORT_NAME

prompt <style>
prompt body { font-family: system-ui, -apple-system, Segoe UI, Roboto, Arial, sans-serif; margin: 18px; }
prompt h1, h2 { margin-top: 24px; }
prompt table { border-collapse: collapse; width: 100%; margin: 12px 0 24px 0; }
prompt th, td { border: 1px solid #ddd; padding: 6px 8px; vertical-align: top; }
prompt th { background: #f6f8fa; text-align: left; }
prompt .ok { color: #137333; font-weight: 600; }
prompt .warn { color: #b26a00; font-weight: 600; }
prompt .bad { color: #c5221f; font-weight: 600; }
prompt .muted { color: #666; }
prompt .pill { padding: 2px 8px; border-radius: 12px; font-size: 12px; background:#eef2ff; }
prompt .archive { background:#fff7e6; }
prompt </style>

prompt <h1>DBMS_ILM Monitoring & Capacity Report</h1>
prompt <div class="muted">Generated: &&_DATESTAMP</div>

prompt <h2>Environment</h2>
WITH env AS (
  SELECT (SELECT name FROM v$database) db_name,
         (SELECT dbid FROM v$database) dbid,
         (SELECT instance_name FROM v$instance) inst,
         (SELECT host_name FROM v$instance) host,
         (SELECT version FROM v$instance) version,
         SYSDATE now_ts
  FROM dual
)
SELECT * FROM env;

prompt <h2>Heat Map & ADO Status</h2>
SELECT parameter_name, parameter_value
FROM   dba_heat_map_parameters
ORDER  BY parameter_name;

prompt <h2>Feature Usage Signals (Licensing awareness)</h2>
SELECT name,
       detected_usages,
       currently_used,
       TO_CHAR(last_usage_date,'YYYY-MM-DD HH24:MI') last_used
FROM   dba_feature_usage_statistics
WHERE  name IN (
  'Heat Map',
  'Automatic Data Optimization',
  'Advanced Compression',
  'Hybrid Columnar Compression'
)
ORDER BY name;

prompt <h2>Objects Under ILM Management (filter: owner like ''&&OWNER_LIKE'' and table like ''&&TABLE_LIKE'')</h2>
SELECT owner, object_name, object_type, ilm_level, enabled
FROM   dba_ilmobjects
WHERE  owner LIKE '&&OWNER_LIKE'
AND    object_name LIKE '&&TABLE_LIKE'
ORDER BY owner, object_name;

prompt <h2>ILM Policies</h2>
SELECT owner,
       object_name,
       policy_name,
       action,
       scope,
       condition,
       enabled,
       TO_CHAR(created,'YYYY-MM-DD HH24:MI') created
FROM   dba_ilm_policies
WHERE  owner LIKE '&&OWNER_LIKE'
AND    object_name LIKE '&&TABLE_LIKE'
ORDER  BY owner, object_name, policy_name;

prompt <h2>ILM Evaluation Details (why policies fired or not)</h2>
SELECT owner, object_name, policy_name,
       evaluation_result,
       evaluation_reason,
       TO_CHAR(evaluation_time,'YYYY-MM-DD HH24:MI') evaluation_time
FROM   dba_ilmevaluationdetails
WHERE  owner LIKE '&&OWNER_LIKE'
AND    object_name LIKE '&&TABLE_LIKE'
ORDER  BY evaluation_time DESC;

prompt <h2>ILM Jobs (background executions)</h2>
SELECT job_id,
       owner,
       object_name,
       policy_name,
       status,
       TO_CHAR(start_time,'YYYY-MM-DD HH24:MI') start_time,
       TO_CHAR(end_time,'YYYY-MM-DD HH24:MI')   end_time,
       message
FROM   dba_ilmjobs
WHERE  owner LIKE '&&OWNER_LIKE'
AND    object_name LIKE '&&TABLE_LIKE'
ORDER  BY start_time DESC;

prompt <h2>ILM Results</h2>
SELECT job_id,
       owner,
       object_name,
       policy_name,
       action,
       result,
       TO_CHAR(action_time,'YYYY-MM-DD HH24:MI') action_time,
       result_details
FROM   dba_ilmresults
WHERE  owner LIKE '&&OWNER_LIKE'
AND    object_name LIKE '&&TABLE_LIKE'
ORDER  BY action_time DESC;

prompt <h2>Heat Map Activity Snapshot (segment-level)</h2>
SELECT owner,
       segment_name,
       segment_type,
       access_count,
       TO_CHAR(last_access,'YYYY-MM-DD HH24:MI') last_access,
       TO_CHAR(last_modification,'YYYY-MM-DD HH24:MI') last_modification
FROM   dba_heat_map_segment
WHERE  owner LIKE '&&OWNER_LIKE'
AND    segment_name LIKE '&&TABLE_LIKE'
ORDER  BY NVL(last_access, last_modification) DESC NULLS LAST;

prompt <h2>Partition Placement & Size (highlighting ARCHIVE tablespace)</h2>
WITH p AS (
  SELECT p.table_owner owner,
         p.table_name,
         p.partition_name,
         p.tablespace_name,
         NVL(s.bytes,0) bytes
  FROM   dba_tab_partitions p
  LEFT   JOIN dba_segments s
         ON s.owner = p.table_owner
        AND s.segment_name = p.table_name
        AND s.partition_name = p.partition_name
        AND s.segment_type LIKE 'TABLE PARTITION%'
  WHERE  p.table_owner LIKE '&&OWNER_LIKE'
  AND    p.table_name  LIKE '&&TABLE_LIKE'
)
SELECT owner,
       table_name,
       partition_name,
       CASE WHEN tablespace_name = '&&ARCHIVE_TS' THEN
         '<span class="pill archive">'||tablespace_name||'</span>'
       ELSE tablespace_name END as tablespace_name,
       ROUND(bytes/1024/1024) AS mb
FROM   p
ORDER  BY owner, table_name, partition_name;

prompt <h2>Archive Tablespace Usage</h2>
WITH t AS (
  SELECT tablespace_name,
         SUM(bytes)/1024/1024 mb_alloc
  FROM   dba_data_files
  GROUP  BY tablespace_name
), f AS (
  SELECT tablespace_name,
         SUM(bytes)/1024/1024 mb_free
  FROM   dba_free_space
  GROUP  BY tablespace_name
)
SELECT t.tablespace_name,
       ROUND(t.mb_alloc,0) AS mb_alloc,
       ROUND(NVL(f.mb_free,0),0) AS mb_free,
       ROUND(t.mb_alloc - NVL(f.mb_free,0),0) AS mb_used,
       CASE WHEN t.tablespace_name = '&&ARCHIVE_TS'
            THEN '<span class="pill archive">archive target</span>'
            ELSE '<span class="muted">—</span>' END as note
FROM   t LEFT JOIN f USING (tablespace_name)
ORDER  BY t.tablespace_name;

prompt <h2>Partition Size Snapshot & Growth Since Last Run</h2>
BEGIN
  EXECUTE IMMEDIATE q'[
    CREATE TABLE ILM_MON_SNAP_PARTS (
      snap_ts         DATE DEFAULT SYSDATE NOT NULL,
      owner           VARCHAR2(128),
      table_name      VARCHAR2(128),
      partition_name  VARCHAR2(128),
      tablespace_name VARCHAR2(128),
      bytes           NUMBER
    )
  ]';
EXCEPTION WHEN OTHERS THEN
  IF SQLCODE != -955 THEN RAISE; END IF;
END;
/

INSERT /*+ append */ INTO ILM_MON_SNAP_PARTS (snap_ts, owner, table_name, partition_name, tablespace_name, bytes)
SELECT SYSDATE,
       p.table_owner,
       p.table_name,
       p.partition_name,
       p.tablespace_name,
       NVL(s.bytes,0)
FROM   dba_tab_partitions p
LEFT   JOIN dba_segments s
       ON s.owner = p.table_owner
      AND s.segment_name = p.table_name
      AND s.partition_name = p.partition_name
      AND s.segment_type LIKE 'TABLE PARTITION%'
WHERE  p.table_owner LIKE '&&OWNER_LIKE'
AND    p.table_name  LIKE '&&TABLE_LIKE';

COMMIT;

WITH latest AS (
  SELECT owner, table_name, partition_name, tablespace_name, bytes,
         snap_ts,
         ROW_NUMBER() OVER (PARTITION BY owner, table_name, partition_name ORDER BY snap_ts DESC) rn
  FROM   ILM_MON_SNAP_PARTS
),
curr AS (
  SELECT * FROM latest WHERE rn = 1
),
prev AS (
  SELECT l1.owner, l1.table_name, l1.partition_name, l1.tablespace_name, l1.bytes bytes_prev, l1.snap_ts snap_prev
  FROM   latest l1
  WHERE  rn = 2
)
SELECT c.owner,
       c.table_name,
       c.partition_name,
       c.tablespace_name,
       ROUND(c.bytes/1024/1024) AS mb_now,
       ROUND(NVL(p.bytes_prev,0)/1024/1024) AS mb_prev,
       ROUND((c.bytes - NVL(p.bytes_prev,0))/1024/1024) AS mb_delta,
       TO_CHAR(NVL(p.snap_prev, c.snap_ts),'YYYY-MM-DD HH24:MI') AS compared_to
FROM   curr c
LEFT   JOIN prev p
       ON p.owner = c.owner
      AND p.table_name = c.table_name
      AND p.partition_name = c.partition_name
ORDER  BY c.owner, c.table_name, c.partition_name;

prompt <h2>Quick Health Flags</h2>
WITH jobs AS (
  SELECT status FROM dba_ilmjobs WHERE start_time > SYSDATE - 7
),
flags AS (
  SELECT
    (SELECT CASE WHEN COUNT(*)=0 THEN 'No recent ILM jobs in last 7 days'
                 WHEN SUM(CASE WHEN status='SUCCEEDED' THEN 1 ELSE 0 END)=0 THEN 'ILM jobs exist but none succeeded'
                 ELSE 'Recent ILM jobs succeeded' END
       FROM jobs) AS job_health,
    (SELECT CASE WHEN (SELECT COUNT(*) FROM dba_ilm_policies WHERE owner LIKE '&&OWNER_LIKE' AND object_name LIKE '&&TABLE_LIKE')=0
                 THEN 'No ILM policies found for current filter'
                 ELSE 'ILM policies present' END
       FROM dual) AS policy_presence,
    (SELECT CASE WHEN (SELECT parameter_value FROM dba_heat_map_parameters WHERE parameter_name='HEAT_MAP') IN ('ON','AUTO')
                 THEN 'Heat Map enabled'
                 ELSE 'Heat Map disabled' END
       FROM dual) AS heat_map_state
  FROM dual
)
SELECT CASE WHEN job_health LIKE 'Recent%' THEN '<span class="ok">'||job_health||'</span>'
            WHEN job_health LIKE 'No recent%' THEN '<span class="warn">'||job_health||'</span>'
            ELSE '<span class="bad">'||job_health||'</span>' END AS job_health,
       CASE WHEN policy_presence LIKE 'ILM policies present' THEN '<span class="ok">'||policy_presence||'</span>'
            ELSE '<span class="warn">'||policy_presence||'</span>' END AS policy_presence,
       CASE WHEN heat_map_state = 'Heat Map enabled' THEN '<span class="ok">'||heat_map_state||'</span>'
            ELSE '<span class="bad">'||heat_map_state||'</span>' END AS heat_map_state
FROM   flags;

prompt <h2>Notes</h2>
prompt <p class="muted">
This report is filtered by owner like '<b>&&OWNER_LIKE</b>' and table like '<b>&&TABLE_LIKE</b>'.
Relocation of partitions using DBMS_ILM MOVE is included with Enterprise Edition. Any compression shown by feature-usage signals
may require Advanced Compression Option; Hybrid Columnar Compression is limited to engineered systems. Verify with your Oracle rep.
</p>

spool off
set markup html off
prompt Report written to &&REPORT_DIR/&&REPORT_NAME

Below is the sample report that was executed on one of my local test database where I had ILM configured.

This approach keeps everything lightweight and DBA-friendly. The first run creates a small snapshot table ILM_MON_SNAP_PARTS. Each subsequent run captures current partition sizes and shows the delta since the prior run, so you can literally see ILM moves and compression effects reflected in size changes and archive tablespace growth.

Hope It Helped!
Prashant Dixit
Database Architect @RENAPS
Reach us at : https://renaps.com/

Posted in Uncategorized | Tagged: , , , , , | Leave a Comment »

Stop Writing Purge Scripts. DBMS_ILM Is the Hidden Hero You Ignored

Posted by FatDBA on September 14, 2025

Enterprise databases accumulate vast amounts of historical data over time journal lines in Oracle E-Business Suite (EBS), payroll checks in PeopleSoft HCM, closed service requests in CRM, or audit logs in custom ERP systems. Left unchecked, this data creates storage bloat, longer backup windows, degraded query performance, and costly migrations. Traditionally, DBAs wrote custom purge scripts to handle “cold” data. But Oracle introduced a smarter, automated way: DBMS_ILM. With DBMS_ILM and the broader Information Lifecycle Management (ILM) framework, you can define policies that automatically compress, move, or archive data as it ages turning Oracle into a self-managing database.

Historical Timeline

  • Oracle 11g (2007–2012)
    • ILM was more of a concept than a feature.
    • DBMS_ILM package existed but was very limited, focusing mainly on manual compression/move operations.
  • Oracle 12c Release 1 (2013)
    • Heat Map introduced (tracks row/segment usage).
    • Automatic Data Optimization (ADO) became available.
    • DBMS_ILM matured to support policy-driven movement and compression.
    • Licensing Note: Compression features required Advanced Compression Option (ACO).
  • Oracle 12c Release 2 (2016)
    • Enhanced policy management, subpartition-level actions.
    • Support for Hybrid Columnar Compression (HCC) in engineered systems.
  • Oracle 18c / 19c (2018–2019)
    • DBMS_ILM + ADO stabilized as enterprise-ready.
    • Background ILM jobs integrated with DBMS_SCHEDULER.
    • Adopted widely in ERP/CRM archiving projects.
  • Oracle 21c / 23c (2021–2023)
    • Minor refinements but ILM model remained stable.
    • Greater focus on integration with Exadata and Cloud tiering.
    • In 23ai (23c), DBMS_ILM is still the core API for lifecycle policies, with Oracle pushing more cloud-managed automation in Autonomous DB.

Licensing is often misunderstood, so let’s break it down clearly:

FeatureOracle VersionLicense Requirement
Heat Map (track row/segment usage)12c+Enterprise Edition (included)
Basic ILM actions (MOVE partitions, mark READ ONLY)12c+Enterprise Edition (included)
Compression policies (OLTP, Advanced Row Compression)12c+Enterprise Edition + Advanced Compression Option (ACO)
Hybrid Columnar Compression (HCC)11g+Exadata, ZFS, or ACFS + ACO
Automatic Tiering to external storage12c+ILM Pack / Engineered Systems
Autonomous DB ILM policies19c+Included in Autonomous service pricing

Golden Rule:

  • Moving cold partitions to an archive tablespace = FREE (EE)
  • Compressing cold partitions = PAID (ACO)
  • Using Exadata HCC or tiering = Requires Engineered System licensing

Always validate with:

SELECT name, detected_usages, currently_used, last_usage_date
FROM   dba_feature_usage_statistics
WHERE  name LIKE '%Compression%' OR name LIKE '%Heat Map%' OR name LIKE '%ILM%';

This query helps confirm whether your test triggered paid options.

DBMS_ILM Components

  • Heat Map
    • Tracks data access (reads/writes).
    • Provides metrics for ILM decisions.
  • Automatic Data Optimization (ADO)
    • Executes policies when conditions are met.
    • Works on segments, partitions, or rows.
  • DBMS_ILM Package
    • API to create, enable, execute, and monitor ILM policies.
    • Provides manual overrides for testing.

Use Cases (ERP/CRM/PeopleSoft)

  1. PeopleSoft Payroll (PAY_CHECKS)
    • Compress after 2 years.
    • Move to ARCHIVE tablespace after 7 years.
  2. Oracle EBS GL (GL_JE_LINES)
    • Move journals > 10 years old.
    • Keep recent 3 years online.
  3. CRM Tickets
    • Close/compress tickets > 5 years.
    • Keep only 2 years active for customer service SLAs.
  4. Custom Banking ERP (TXN_AUDIT_LOG)
    • Auto-compress audit logs after 36 months.
    • Mark READ ONLY after 7 years.

Detailed Demo

6.1. Enable Heat Map

ALTER SYSTEM SET HEAT_MAP=ON SCOPE=BOTH;

6.2. Create Archive Tablespace

CREATE TABLESPACE ARCHIVE_TS 
DATAFILE '/u01/app/oracle/oradata/ARCHIVE01.dbf' SIZE 20G AUTOEXTEND ON;

6.3. Create Partitioned Table

CREATE TABLE psft.pay_checks
(
  check_id    NUMBER,
  emp_id      NUMBER,
  amount      NUMBER,
  check_date  DATE
)
PARTITION BY RANGE (check_date)
(
  PARTITION p2015 VALUES LESS THAN (TO_DATE('01-JAN-2016','DD-MON-YYYY')),
  PARTITION p2016 VALUES LESS THAN (TO_DATE('01-JAN-2017','DD-MON-YYYY')),
  PARTITION p2017 VALUES LESS THAN (TO_DATE('01-JAN-2018','DD-MON-YYYY')),
  PARTITION pmax  VALUES LESS THAN (MAXVALUE)
);

6.4. Insert Data

INSERT INTO psft.pay_checks
SELECT level, mod(level,1000), dbms_random.value(1000,5000),
       ADD_MONTHS(SYSDATE, -level)
FROM dual CONNECT BY level <= 200000;
COMMIT;

6.5. Add Policy

BEGIN
  DBMS_ILM.ADD_POLICY(
    object_owner     => 'PSFT',
    object_name      => 'PAY_CHECKS',
    policy_name      => 'PAYROLL_ARCHIVE_POLICY',
    action           => DBMS_ILM.MOVE,
    scope            => DBMS_ILM.SCOPE_PARTITION,
    tier_tablespace  => 'ARCHIVE_TS',
    condition        => 'CREATION_TIME < ADD_MONTHS(SYSDATE, -84)', -- 7 years
    enabled          => DBMS_ILM.ENABLED
  );
END;
/

6.6. Execute Policy

BEGIN
  DBMS_ILM.EXECUTE_ILM(
    object_owner => 'PSFT',
    object_name  => 'PAY_CHECKS',
    policy_name  => 'PAYROLL_ARCHIVE_POLICY'
  );
END;
/

Monitoring ILM

-- check policiess.
SELECT policy_name, action, condition, enabled
FROM   dba_ilm_policies
WHERE  object_name='PAY_CHECKS';

--Monitor Jobs
SELECT job_id, object_name, status, start_time, end_time
FROM   dba_ilmjobs;

-- check results
SELECT * FROM dba_ilmresults WHERE object_name='PAY_CHECKS';

Verify Partition Location

SELECT table_name, partition_name, tablespace_name
FROM   dba_tab_partitions
WHERE  table_name='PAY_CHECKS';

Heat Map Access Tracking

SELECT segment_name, access_count, last_access
FROM   dba_heat_map_segment
WHERE  segment_name='PAY_CHECKS';

Troubleshooting

IssueLikely CauseFix
Policy doesn’t fireHeat Map offALTER SYSTEM SET HEAT_MAP=ON
Partitions not movedNot partitionedPartition by DATE
ORA-4031 errorsMemory pressureTune ILM job resource usage
Compression errorsMissing ACO licenseRemove compression actions

Best Practices

  1. Always partition large tables by date before using ILM.
  2. Run in DEV and validate with ERP/CRM reports before PROD rollout.
  3. Align with compliance/legal retention rules (SOX, GDPR, HIPAA).
  4. Schedule ILM jobs in maintenance windows.
  5. Monitor dba_feature_usage_statistics to avoid license violations.
  6. Combine ILM with purging strategies for truly obsolete data.

Conclusion

Introduced in Oracle 11g as a concept and matured in 12c, DBMS_ILM is now a stable enterprise feature that brings policy-driven automation to Oracle databases.

  • For PeopleSoft Payroll, it can transparently archive checks older than 7 years.
  • For EBS Financials, it can compress historical journal entries.
  • For CRM logs, it can move cold data to cheaper storage.

When deployed with proper licensing (EE + ACO where needed), DBMS_ILM transforms data management from ad-hoc DBA scripts into a compliant, auditable, and automated lifecycle system.

It is not just a DBA feature… it’s a data governance framework, critical for ERP/CRM modernization and cloud migrations.

If you want to test it, start by enabling Heat Map in your DEV/TEST system, track your largest ERP/CRM tables for 30 days, then design your first MOVE-only ILM policy (no compression). Once comfortable, expand to compression (if licensed) and tiering strategies.

Hope It Helped!
Prashant Dixit
Database Architect @RENAPS
Reach us at : https://renaps.com/

Posted in Uncategorized | Tagged: , , , , , | Leave a Comment »