Infrastructure Sizing for Essbase (part 3)

This entry concludes our discussion of estimating Essbase infrastructure specifications.


Understanding Essbase Processes

Batch Processing
There are two components to Essbase batch refresh processes. Meta data and data are updated and then data is pre-processed before being made available to users. Pre-processing (i.e. aggregation creation) is where the Administrator designs how Essbase computes and stores cube values in the effort to enhance query responsiveness. Computing and persisting data is demanding of processor, network and disk resources: During aggregation creation, Essbase systematically reads data in, calculates/aggregates, and writes data out, sometimes for extended (hours) periods of time.

Resource demands and best practices require that batch processing be carried out in isolation from end-user query activity. Batch routines are usually run this way so that the Admin can be sure that required server resources are able to be allocated to specific Essbase processes.

Running Batch in Isolation
When the batch processes are being run isolated, end user activities are temporarily suspended. Isolating events in time is the easiest way to ensure that the resources that are required to perform the batch processes are available without restriction. From an infrastructure perspective what is required is that the server be configured with sufficient resources to process the batch routines efficiently. From the user perspective, however, it means that there is potentially a significant period of time when the application is not available. Whereas this involves a minimal requirement for hardware, end user downtime is becoming less and less tolerable for enterprise analytic applications.

Running Batch in Parallel
Running in parallel means batch processes are run coincident with end-user activities. The architecture is configured with sufficient resources to enable efficient batch processing without any significant degradation in performance of end user activities. This minimizes the impact on the end user, but it maximizes infrastructure requirements. In this scenario there are possibly great periods of time when the infrastructure is under‑utilized.

Underestimating resources here has two unfortunate characteristics; most of the time the system appears under‑utilized. During times of peak use, however, the whole system becomes a bottleneck for batch as well as user processes.

Peak usage always coincides with the delivery of critical functionality. Failure during peak periods will undermine user confidence in the entire application.

Query Processing
The quality of the end-user experience of Essbase is defined by query performance characteristics. Query performance depends upon three factors:

· CPU availability - a query is a single threaded event and thus requires one CPU
· Query Size - the amount of data that Essbase is required to retrieve from disk
· Dynamic Processing - the number and degree of dynamic computations

The objective of developing an efficient Essbase computing environment is to create one where the resource requirements to support batch processes are not simultaneously competing for resources required for end-use querying. Large queries that take minutes to complete have a much greater chance of being concurrent with other processing requests. Tuning the cube to have more efficient response times mitigates this. However, at some point, the query response time bottom line is reached, and hardware appears as the only way to increase performance. This is achieved different ways. Bigger and faster hardware is augmented by scaling the environment horizontally across multiple different servers, or by scaling the environment vertically on servers, or both.

Storage Requirements
Oracle will recommend customers use dedicated storage whenever possible for Essbase. Many times performance limitations on Essbase will be Disk as well as CPU related. Applications calculating and reporting data which will drive high I/O utilization on a storage system. For SAN devices, dedicated fiber connections will be very beneficial. This configuration will be helpful specifically when calculating multiple applications simultaneously. While not necessary, additional throughput performance gains are achieved by splitting Block Storage Option applications to calculate and store the index and data files on different physical drives and controllers.

For an Aggregate Storage Option cube, data is stored in both the Temp and Default directories during data load and aggregation processing. Using separate physical drives and I/O channels for the location of these files, you can reduce I/O contention and there by improve overall performance. Breaking up the Aggregate .DAT files by storing large databases on different physical drives with different I/O channels can also improve query performance.

Network Speed and Bandwidth
Using a full duplex 1GB Network connection between any servers in the same zone that have EPM components that communicate is optimal. If all the EPM components are on one physical server then use a corresponding server network connection up to the maximum speed of the network throughput of routers to the network.

Other Performance Related Considerations

Multi-Threading Technologies
Multi-Threading processor technologies spawn two threads per physical CPU to emulate multiple processing cores. This way, the OS perceives twice as many logical CPUs as there are physical ones. The result can be, for some applications, to improve throughput by as much as ~30%.

Not all software applications are enhanced by the use of these technologies. In general it has been found that Essbase performance is not enhanced, and often demonstrates degraded performance.

Essbase performance is, however, application and cube dependent. A small number of customers anecdotally report increased responsiveness using multi-processor simulations. Considering that there are platform implementation differences, and that it is not possible to predict accurately how any given cube will respond to this technology, it is best to test to determine what is appropriate for your Essbase environment.

Multiple Cores and Floating Point Units (FPU)
This is perhaps an outdated topic because of recent trends in processor technologies. Essbase performs floating point arithmetic, so single FPU processors represent a bottleneck for Essbase. To take advantage of the performance gains from multiple-core technologies, ensure that each core is configured with its own FPU.

LPAR/VM Definitions
Virtual machine and some LPAR definitions of Essbase servers are being used to share resources. Whereas these configurations are supported by Oracle for Essbase, the nature of a shared environment is such that it is technically challenging to control server resources to ensure especially what processor resources are being allocated to Essbase. The result is that Essbase performance characteristics cannot be controlled to deliver a consistent level of performance. Where it is not possible to be sure that the server will be able to meet the end-user service level agreements, we recommend that you decide against using Essbase in a shared environment.

Estimating Core and RAM Requirements
If the requirement persists to estimate infrastructure specifications without processing requirement details being known, here are two methods that can be used. Both make assumptions about how to gauge concurrency. The second includes adding cores to support batch processing requirements and factors in a desired response time for end user querying.

Processor Cores per Application based on Active Users
This method is easy, but potentially expensive. When use-case scenarios are not well known, estimate processor and memory requirements the following way: Allocate six cores per Essbase application. Memory is allocated by adding 2GB of RAM to the server for each core.

The use-case scenario that this rule was devised for is an Essbase and Relational reporting environment with 2000 named users where 500 users are active. In large computing environments, this can easily lead to very pessimistic (i.e. expensive) estimates for core and RAM requirements.

Users per Core
Another option is to base the number of cores on the number of users, using batch concurrency and query response time assumptions. RAM estimation allocates between 2 and 4 GB of RAM per application rather than per core.

The examples assume that batch routines are run in parallel with end-user processes. For an application that batch routines are run in isolation, core estimates should be reduced accordingly.
The examples that follow are experienced estimates that can only be validated with realistic load testing. Adding to the overall complexity of using this method is to keep in mind how important report design is for performance, RAM and core requirements.

50 Users Per Core.
The method assumes a desired 15 second response time for queries/reports with the longest query taking no longer than 30 seconds. To this base number of cores, you add cores for required for parallel batch processing.

25 Users Per Core
This method assumes a desired 15 second response time for queries/reports where minimal increases in response times are required. Add an additional core for each report that runs for 60 seconds or more. To this base number, add the number of cores required for parallel batch processing to compute the total number of estimated cores.
______________________________________________________________________________________

John French

Principal Service Delivery Engineer
Oracle Advanced Customer Services - GLOBAL
Jensen Beach, FL

Richard (Rick) Sawa
Principal Service Delivery
Oracle Advanced Customer Services - GLOBAL
Columbus, OH