Hyperion Essbase Optimization: What is the Calc Cache?

The Essbase block storage option (BSO) is today almost exclusively defined by Planning applications. So this blog entry really is of interest to customers who are either using Planning, or continue to have planning-type Essbase applications - which is to say Essbase cubes that use functionality currently not provided by the aggregate storage option (ASO).

The graphic is a hypothetical view of the Calc Cache in relation to the kernel and the other Essbase BSO caches. The intention is to convey that the CALC CACHE is used by the calculator engine not to hold data, but to assist in aggregating data, which is another way to say create data blocks.

Essbase uses the calc cache to keep track of children and parents. This is what the CALC CACHE is.

The essence of the CALC CACHE is that Essbase is able to use it to track when to create parent blocks and what children are needed to do so without having to make a more expensive external call like, for example, to read the outline. So the calculator has a way to keep close to the Kernel when to create parent blocks and what children are involved.

That is really all you need to know about what the calc cache is. How to configure the calc cache, however, really requires a white paper discussion all to itself.

The effectiveness of the CALC CACHE is strictly cube dependent. And tuning it will require “different” just about everything from testing different amounts of memory allocated to the CALC CACHE, to altering outline order, to altering sparse/dense settings, even to possibly altering calc logic.

Essbase divides sparse dimensions into 2 groupings: some become anchor dimensions, and the others become bitmap dimensions. The RAM required to create the calc cache depends upon the size and number of the bitmap dimensions.

There are several ways that Essbase can establish a CALC CACHE and the reader is referred to the Essbase documentation for the technical details. What you are trying to accomplish in the CALC CACH is to provide Essbase with the best scratch pad you can given the amount of RAM and number of CPUs you have access to, as well as the nature of the sparse dimensions of your cube.

Establishing the most efficient CALC CACHE is one of the main technical reasons that Essbase best practices suggests to declare sparse dimensions from smallest to largest in the outline. The reason is that the most efficient CALC CACHE is created when there is a single anchor dimension and sufficient memory to instantiate multiple bitmaps for the bitmap dimensions. Placing the largest dimension at the bottom of the outline eliminates it from the bitmap dimensions, and thereby reduces the amount of RAM required for bitmap dimensions. The hourglass shaped outline is not somehow magical. Currently most cubes have to break that hourglass paradigm in order to achieve the most efficient outline order for the purposes of script calculations.

So, not all cubes are able to take advantage of the CALC CACHE because of the limitations imposed by the server environment (RAM, CPU and Outline). Each cube needs to be tuned individually to determine the optimal CALC CACHE setting, or even whether to turn the CALC CACHE OFF in order to favour completely parallel calculation (CALC PARALLEL and the CALC CACHE need to be tuned together).

It is important to know that searching through the bitmap is not indexed and results in the strong suggestion NOT to allocate more than 50 MB of RAM to the CALC CACHE, because the effectiveness of the CALC CACHE search algorithm tails off beyond 50MB. When Essbase is unable to achieve multiple bitmaps with a single anchor using 50 MB or less, break the so-called “hourglass” motif and move non-aggregating sparse dimensions below the last and largest sparse (anchor) dimension. The objective is to reserve memory for Essbase to be able to place as many aggregating dimensions as possible into the bitmap.

Anyone who has tested using the CALC CACHE and CALCPARALLEL will probably have noticed that there is some sort of relation that exists between these two commands.

This is because for every thread that an aggregation calculation spawns requires its own CALC CACHE. It is non-trivial for Essbase to manage multiple threads, multiple TASKDIMS and multiple CALC CACHEs. Moreover, assigning large amounts or RAM to the CALC CACHE when using CALC PARALLEL is one way to almost ensure destabilizing the Essbase server.

For example, consider a 32-bit Planning application where 5 business rules run concurrently. If each rule sets CACHE HIGH (and CALC CACHE HIGH is set in the Essbase.cfg file at ~200 million bytes) and CALCPARALLEL is set to 4, then one business rule of this single cube requires (5 x 4 x 200 million) ~ 4GB of RAM! The OS is only able to allocate ~2GB of memory to a single application. Rember, in a Planning-Essbase environment, you can set up to five cubes in a single application...

The example is not an exaggeration. I have seen precisely this configuration at numerous customer sites. And I am never invited to customer sites because Essbase is performing well.

Questions about BSO tuning, configuration and optimization? Let me know what they are and I will consider adding then as discussions here. Send suggestions to rick.sawa@oracle.com and use set the subject to Infogram.