Hi All,
Entire last week I was looking in to one system wide performance issue where customer reported slowness in their database since they’d moved into a new infrastructure, after data migration. Though this wasn’t the production system, but kind of equally important, and the issue was impacting their acceptance tests. I was updated that few of the Materialized Views are there and their refresh times are now extended by a great margin, since they moved to this new system.
As soon I got the access on their system, I did a quick check on the database and I found a huge list of MV refresh jobs are scheduled and almost all of them are running with a huge delay, the performance was not at all consistent, sometimes they gets refreshed within few minutes and another run goes for few hours. This initially sounded little bizarre to me, until I did a comparive stuy between the previous (fast) and the new (slow) databases. What I observed is that there is more than 500% of extra data there in the business tables in this new (slow) database and with such a huge data, all those complex MV refresh queries ultimately got slowed down here in the new system.
For an example, one of the critical MV taking ~ 200 mins to refresh an is created with COMPLETE REFRESH mode.
create MATERIALIZED VIEW MV_MYTEST_BIGREFRESH
BUILD DEFERRED
REFRESH ON DEMAND
COMPLETE
AS
select abc,xyz,cast(value as ajaaj) as test ....
............
.................
SELECT *
FROM ( SELECT OWNER,
MVIEW_NAME,
CONTAINER_NAME,
REFRESH_MODE,
LAST_REFRESH_TYPE,
STALENESS,
round((LAST_REFRESH_END_TIME-LAST_REFRESH_DATE)*24*60,2) as REFRESH_TIME_MINS
FROM ALL_MVIEWS
WHERE LAST_REFRESH_TYPE IN ('FAST','COMPLETE')
)
ORDER BY REFRESH_TIME_MINS DESC;
OWNER MVIEW_NAME CONTAINER_NAME REFRESH_MODE LAST_REFRESH_TYPE STALENESS REFRESH_TIME_MINS
------- -------------------------------- -------------------------------- ------------ ----------------- -------------------------------------
PRASH MV_MYTEST_BIGREFRESH MV_MYTEST_BIGREFRESH DEMAND COMPLETE COMPLETE 196.75
Though I did multiple optimizations in order to stabilize performance, but would like to talk about few of the refresh tuning techniques that I have applied and finally worked for me.
Atomic_Refresh with value FALSE.
What caught my eye are all those long running DELETE statements running underneath those costly MV refresh jobs. This was the very first thing I tried with those complex and long running COMPLETE MV refresh, this is the optional parameter atomic_refresh of the procedure dbms_mview.refresh and is very useful. If the parameter is set to FALSE, the materialized view is deleted with a much faster TRUNCATE command. I immediately got the gain, as the runtime was dropped ~ 60% .. At the same time I have observed a great relax in redo generation as DELETE is replaced with TRUNCATE, and hence a great drop in ‘checkpoint incomplete‘ and ‘Log File Sync‘ waits.
The parameter atomic_refresh=FALSE works only with “complete” refresh, so “truncate” is only possible with “complete“. But there is one demerit too for this method, and is that no data is visible to the users during the refresh, so, choose wisely. If this is feasible in your environment, you can use the following command for a Complete Refresh:
BEGIN
dbms_mview.refresh(‘MV_MYTEST_BIGREFRESH_JOB’, method => ‘C’, atomic_refresh => FALSE);
END;
-- See below the REFRESH_TIME_MINS got dropped by ~ 60% after using setting atomic_refres = false
SELECT *
FROM ( SELECT OWNER,
MVIEW_NAME,
CONTAINER_NAME,
REFRESH_MODE,
LAST_REFRESH_TYPE,
STALENESS,
round((LAST_REFRESH_END_TIME-LAST_REFRESH_DATE)*24*60,2) as REFRESH_TIME_MINS
FROM ALL_MVIEWS
WHERE LAST_REFRESH_TYPE IN ('FAST','COMPLETE')
)
ORDER BY REFRESH_TIME_MINS DESC;
OWNER MVIEW_NAME CONTAINER_NAME REFRESH_MODE LAST_REFRESH_TYPE STALENESS REFRESH_TIME_MINS
------- -------------------------------- -------------------------------- ------------ ----------------- -------------------------------------
PRASH MV_MYTEST_BIGREFRESH MV_MYTEST_BIGREFRESH DEMAND COMPLETE COMPLETE 79.12
Adding PX (Parallelism) to the base tables or to the refresh command itself.
Though with atomic_refrsh FALSE, I was able to cut down a great percentage of the overall elapsed time, but still customer asked for me, maybe some 30% more …
I decided to try PX to the base tables or to the refresh statement, among other parameters, this procedure takes one parameter PARALLELISM. At the outset, it appears that the PARALLELISM parameter should invoke a parallel refresh of the materialized view. However, it does not; it simply changes the requested degree of parallelism if it already executes in parallel. You can invoke this procedure as:
BEGIN
dbms_mview.refresh(‘MV_MYTEST_BIGREFRESH_JOB’, method => ‘C’, atomic_refresh => FALSE, PARALLELISM=>4);
END;
-- See below the REFRESH_TIME_MINS got dropped by further ~30% after using PARALLELISM to the master/base MV tables.
SELECT *
FROM ( SELECT OWNER,
MVIEW_NAME,
CONTAINER_NAME,
REFRESH_MODE,
LAST_REFRESH_TYPE,
STALENESS,
round((LAST_REFRESH_END_TIME-LAST_REFRESH_DATE)*24*60,2) as REFRESH_TIME_MINS
FROM ALL_MVIEWS
WHERE LAST_REFRESH_TYPE IN ('FAST','COMPLETE')
)
ORDER BY REFRESH_TIME_MINS DESC;
OWNER MVIEW_NAME CONTAINER_NAME REFRESH_MODE LAST_REFRESH_TYPE STALENESS REFRESH_TIME_MINS
------- -------------------------------- -------------------------------- ------------ ----------------- -------------------------------------
PRASH MV_MYTEST_BIGREFRESH MV_MYTEST_BIGREFRESH DEMAND COMPLETE COMPLETE 55.08
You can also add PX to the base tables. When the master table of the materialized view has the PARALLEL attribute set to > 1, then the creation as well as the refresh processes will be parallelized. Whether or not you specify the PARALLELISM parameter in the REFRESH clause, doesn’t matter.
ALTER TABLE DIXIT_TEST_TABLE PARALLEL (DEGREE 4);You can also add PX hint to the MV definition too! When the materialized view is created with a PARALLEL hint, then the creation as well as the refresh processes, will be parallelized. Whether or not you specify the PARALLELISM parameter in the REFRESH clause, doesn’t matter.
This is how you can ad a parallel hint in the materialized view definition!
CREATE MATERIALIZED VIEW TEST_MV
AS
SELECT /*+ PARALLEL(DIXIT_TEST_TABLE, 4) */ MONTHLY_ID, SALARY_DATE, SUM(ANNUAL_COUNTS)
FROM DIXIT_TEST_TABLE GROUP BY MONTHLY_ID, SALARY_DATE;
Stats Collection:
This is very vital that all of the tables, Indexes which your MV refresh references via the code, should be fresh and well collected. Look of for possibilities to add column level stats i.e Histograms wherever there is a scope, specially for low cardinality columns. You can also leave this to the optimizer to decide, by using METHOD_OPT=>’FOR ALL COLUMNS SIZE AUTO’
Other techniques:
- Try for FAST/Incremental Refresh mode. With FAST refresh, only the changes since the last refresh are applied to the materialized view. Personally I have found the FAST refresh slower than the COMPLETE refresh mode, maybe because I used atomic_refresh parameter that replaced those costly DELETEs with fast TRUNCATE … Not sure why they called it FAST 🙂 To be sure that my materialized view can be fast refresh, we can also use explain_mview procedure and check the capability_name called “REFRESH_FAST”. The MV_CAPABILITIES TABLE is created in the owner’s schema by running the following script ..
SQL> @$ORACLE_HOME/rdbms/admin/utlxmv.sql
SQL> exec dbms_mview.explain_mview('MV_MYTEST_BIGREFRESH_JOB');
PL/SQL procedure successfully completed.
SQL> select capability_name,possible,related_text,msgtxt from mv_capabilities_table;
CAPABILITY_NAME POSSIBLE RELATED_TEXT MSGTXT
------------------------------ -------------------- ----------------------- ----------------------------------------------------------------------------
PCT N
REFRESH_COMPLETE Y
REFRESH_FAST Y <<<<<<<<---------------
REWRITE Y
PCT_TABLE N ID_ID_CANADA_HISTORY_T1 relation is not a partitioned table
PCT_TABLE N ID_ID_CANADA_HISTORY_TV relation is not a partitioned table
PCT_TABLE N DWH_SITE relation is not a partitioned table
REFRESH_FAST_AFTER_INSERT Y
REFRESH_FAST_AFTER_ONETAB_DML Y
REFRESH_FAST_AFTER_ANY_DML Y
REFRESH_FAST_PCT N PCT is not possible on any of the detail tables in the materialized view
REWRITE_FULL_TEXT_MATCH Y
REWRITE_PARTIAL_TEXT_MATCH Y
REWRITE_GENERAL Y
REWRITE_PCT N general rewrite is not possible or PCT is not possible on any of the detail tables
PCT_TABLE_REWRITE N ID_ID_CANADA_HISTORY_T1 relation is not a partitioned table
PCT_TABLE_REWRITE N ID_ID_CANADA_HISTORY_TV relation is not a partitioned table
PCT_TABLE_REWRITE N DWH_SITE relation is not a partitioned table
18 rows selected.
- Out-of-place refresh is a new option on DBMS_MVIEW. REFRESH in Oracle 12c. The complete refresh process builds a new table which seamlessly becomes the materialized view, the old table is dropped. It only works in conjunction with non-atomic refresh.
- DBMS_ADVISOR contains a procedure called TUNE_MVIEW that is used when working with the query rewrite mechanism. DBMS_MVIEW.EXPLAIN_MVIEW and DBMS_MVIEW.EXPLAIN_REWRITE that could be used to diagnose why a materialized view wasn’t being used for query rewrite. The DBMS_ADVISOR.TUNE_MVIEW procedure has the capability to add additional aggregate columns and materialized view logs to the view definition so that it becomes fast refreshable.
Well, all after I’ve performed above three methods, I was able to reduce the MV refresh downtimes by more than 92% … 🙂
Hope It Helped!
Prashant Dixit
