Basic Architectural Elements Skip Navigation
Essbase® Analytic Services Database Administrator's Guide | Update Contents | Previous | Next | Print | ? |
Information Map

Basic Architectural Elements


In this chapter, you will learn how Essbase Analytic Services improves performance by reducing storage space and speeding up data retrieval for multidimensional databases.

Note: The information in this chapter is designed for block storage databases. Some of the information is not relevant to aggregate storage databases. For detailed information on the differences between aggregate and block storage, see Comparison of Aggregate and Block Storage.

This chapter contains the following sections:

Attribute Dimensions and Standard Dimensions

Analytic Services has two types of dimensions: attribute dimensions and standard dimensions (non-attribute dimensions). This chapter primarily considers standard dimensions because Analytic Services does not allocate storage for attribute dimension members. Instead it dynamically calculates the members when the user requests data associated with them.

An attribute dimension is a special type of dimension that is associated with a standard dimension. For comprehensive discussion of attribute dimensions, see Working with Attributes.

Sparse and Dense Dimensions

Most data sets of multidimensional applications have two characteristics:

Analytic Services maximizes performance by dividing the standard dimensions of an application into two types: dense dimensions and sparse dimensions. This division allows Analytic Services to cope with data that is not smoothly distributed, without losing the advantages of matrix-style access to the data. Analytic Services speeds up data retrieval while minimizing the memory and disk requirements.

Most multidimensional databases are inherently sparse: they lack data values for the majority of member combinations. A sparse dimension is a dimension with a low percentage of available data positions filled.

For example, the Sample Basic database shown in Figure 23 includes the Year, Product, Market, Measures, and Scenario dimensions. Product represents the product units, Market represents the geographical regions in which the products are sold, and Measures represents the accounts data. Because not every product is sold in every market, Market and Product are chosen as sparse dimensions.

Most multidimensional databases also contain dense dimensions. A dense dimension is a dimension with a high probability that one or more data points is occupied in every combination of dimensions. For example, in the Sample Basic database, accounts data exists for almost all products in all markets, so Measures is chosen as a dense dimension. Year and Scenario are also chosen as dense dimensions. Year represents time in months, and Scenario represents whether the accounts values are budget or actual values.

Note: Caffeinated, Intro Date, Ounces, and Pkg Type are attribute dimensions that are associated with the Product dimension. Population is an attribute dimension that is associated with the Market dimension. Members of attribute dimensions describe characteristics of the members of the dimensions with which they are associated. For example, each product has a size in ounces. Attribute dimensions are always sparse dimensions and must be associated with a sparse standard dimension. Analytic Services does not store the data for attribute dimensions, Analytic Services dynamically calculates the data when a user retrieves it. For a comprehensive discussion about attribute dimensions, see Working with Attributes.

Figure 23: Sample Basic Database Outline

Data Blocks and the Index System

Analytic Services uses two types of internal structures to store and access data: data blocks and the index system.

Analytic Services creates a data block for each unique combination of sparse standard dimension members (providing that at least one data value exists for the sparse dimension member combination). The data block represents all the dense dimension members for its combination of sparse dimension members.

Analytic Services creates an index entry for each data block. The index represents the combinations of sparse standard dimension members. It contains an entry for each unique combination of sparse standard dimension members for which at least one data value exists.

For example, in the Sample Basic database outline shown in Figure 24, Product and Market are sparse dimensions.

Figure 24: Product and Market Dimensions from the Sample Basic Database

If data exists for Caffeine Free Cola in New York, then Analytic Services creates a data block and an index entry for the sparse member combination of Caffeine Free Cola (100-30) -> New York. If Caffeine Free Cola is not sold in Florida, then Analytic Services does not create a data block or an index entry for the sparse member combination of Caffeine Free Cola (100-30) -> Florida.

The data block Caffeine Free Cola (100-30) -> New York represents all the Year, Measures, and Scenario dimensions for Caffeine Free Cola (100-30) -> New York.

Each unique data value can be considered to exist in a cell in a data block. When Analytic Services searches for a data value, it uses the index to locate the appropriate data block as shown in Figure 25. Then, within the data block, it locates the cell containing the data value. The index entry provides a pointer to the data block. The index handles sparse data efficiently because it includes only pointers to existing data blocks.

Figure 25: Simplified Index and Data Blocks

Figure 26 shows part of a data block for the Sample Basic database. Each dimension of the block represents a dense dimension in the Sample Basic database: Time, Measures, and Scenario. A data block exists for each unique combination of members of the Product and Market sparse dimensions (providing that at least one data value exists for the combination).

Figure 26: Part of a Data Block for the Sample Basic Database

Each data block is a multidimensional array that contains a fixed, ordered location for each possible combination of dense dimension members. Accessing a cell in the block does not involve sequential or index searches. The search is almost instantaneous, resulting in optimal retrieval and calculation speed.

Analytic Services orders the cells in a data block according to the order of the members in the dense dimensions of the database outline.

A (Dense)
   a1
   a2
B (Dense)
   b1
       b11
       b12
   b2
       b21
       b22
C (Dense)
   c1
   c2
   c3
D (Sparse)
   d1
   d2
       d21
       d22
E (Sparse)
   e1
   e2
   e3 
 

The block in Figure 27 represents the three dense dimensions from within the combination of the sparse members d22 and e3 in the preceding database outline. In Analytic Services, member combinations are denoted by the cross-dimensional operator. The symbol for the cross-dimensional operator is ->. So d22, e3 is written d22 -> e3. A, b21, c3 is written A -> b21 -> c3.

Figure 27: Data Block Representing Dense Dimensions for d22 -> e3

Analytic Services creates a data block for every unique combination of the members of the sparse dimensions D and E (providing that at least one data value exists for the combination).

Data blocks, such as the one shown in Figure 27, may include cells that do not contain data values. A data block is created if at least one data value exists in the block. Analytic Services compresses data blocks with missing values on disk, expanding each block fully as it brings the block into memory. Data compression is optional, but is enabled by default. For more information, see Data Compression.

By carefully selecting dense and sparse standard dimensions, you can ensure that data blocks do not contain many empty cells. In Analytic Services, empty cells are known as missing or #MISSING data. You can also minimize disk storage requirements and maximize performance.

Selection of Sparse and Dense Dimensions

In most data sets, existing data tends to follow predictable patterns of density and sparsity. If you match patterns correctly, you can store the existing data in a reasonable number of fairly dense data blocks, rather than in many highly sparse data blocks.

When you add a dimension to an outline in Outline Editor, Analytic Services automatically sets the dimension as sparse. To help you determine whether dimensions should be dense or sparse, Analytic Services provides an automatic configuration feature.

To select automatic configuration of dense and sparse dimensions use the following method:


Tool
Topic
Location

Administration Services

Outline Editor - Properties Tab, Auto configure option

Setting Dimensions as Dense or Sparse

Essbase Administration Services Online Help



If you select automatic configuration, you cannot manually set the sparse or dense property for each dimension. Turn off automatic configuration to set the sparse and dense property manually. Attribute dimensions are always sparse dimensions. Keep in mind that you can associate attribute dimensions only with sparse standard dimensions.

Note: The automatic configuration of dense and sparse dimensions provides only an estimate. It cannot take into account the nature of the data you will load into your database or multiple user considerations.

Determining the Sparse-Dense Configuration for Sample Basic

Consider the Sample Basic database that is shipped with Analytic Services. The Sample Basic database represents data for The Beverage Company (TBC).

TBC does not sell every product in every market; therefore, the data set is reasonably sparse. Data values do not exist for many combinations of members in the Product and Market dimensions. For example, if Caffeine Free Cola is not sold in Florida, then data values do not exist for the combination Caffeine Free Cola (100-30)->Florida. So, Product and Market are sparse dimensions. Therefore, if no data values exist for a specific combination of members in these dimensions, Analytic Services does not create a data block for the combination.

However, consider combinations of members in the Year, Measures, and Scenario dimensions. Data values almost always exist for some member combinations on these dimensions. For example, data values exist for the member combination Sales->January->Actual because at least some products are sold in January. Thus, Year and, similarly, Measures and Scenario are dense dimensions.

The sparse-dense configuration of the standard dimensions in the Sample Basic database may be summarized as follows:

Analytic Services creates a data block for each unique combination of members in the Product and Market dimensions. Each data block represents data from the dense dimensions. The data blocks are likely to have few empty cells.

For example, consider the sparse member combination Caffeine Free Cola (100-30), New York, illustrated by Figure 28:

Dense and Sparse Selection Scenarios

The following scenarios show how a database is affected when you select different dense and sparse standard dimensions. Assume that these scenarios are based on typical databases with at least seven dimensions and several hundred members:

Scenario 1: All Sparse Standard Dimensions

If you make all dimensions sparse, Analytic Services creates data blocks that consist of single data cells that contain single data values. An index entry is created for each data block and, therefore, in this scenario, for each existing data value.

This configuration produces a huge index that requires a large amount of memory. The more index entries, the longer Analytic Services searches to find a specific block.

Figure 29: Database with All Sparse Standard Dimensions

Scenario 2: All Dense Standard Dimensions

If you make all dimensions dense, as shown in Figure 30, Analytic Services creates one index entry and one very large, very sparse block. In most applications, this configuration requires thousands of times more storage than other configurations. Analytic Services needs to load the entire block into memory when it searches for a data value, which requires enormous amounts of memory.

Figure 30: Database with All Dense Standard Dimensions

Scenario 3: Dense and Sparse Standard Dimensions

Based upon your knowledge of your company's data, you have identified all your sparse and dense standard dimensions. Ideally, you have approximately equal numbers of sparse and dense standard dimensions. If not, you are probably working with a non-typical data set and you need to do more tuning to define the dimensions.

Analytic Services creates dense blocks that can fit into memory easily and creates a relatively small index as shown in Figure 31. Your database runs efficiently using minimal resources.

Figure 31: An Ideal Configuration with Combination of Dense and Sparse Dimensions

Scenario 4: A Typical Multidimensional Problem

Consider a database with four standard dimensions: Time, Accounts, Region, and Product. In the following example, Time and Accounts are dense dimensions, and Region and Product are sparse dimensions.

The two-dimensional data blocks shown in Figure 32 represent data values from the dense dimensions: Time and Accounts. The members in the Time dimension are J, F, M, and Q1. The members in the Accounts dimension are Rev, Exp, and Net.

Figure 32: Two-dimensional Data Block for Time and Accounts

Analytic Services creates data blocks for combinations of members in the sparse standard dimensions (providing at least one data value exists for the member combination). The sparse dimensions are Region and Product. The members of the Region dimension are East, West, South, and Total US. The members in the Product dimension are Product A, Product B, Product C, and Total Product.

Figure 33 shows 11 data blocks. No data values exist for Product A in the West and South, for Product B in the East and West, and for Product C in the East. Therefore, Analytic Services has not created data blocks for these member combinations. The data blocks that Analytic Services has created have very few empty cells.

Figure 33: Data Blocks Created for Sparse Members on Region and Product

This example effectively concentrates all the sparseness into the index and concentrates all the data into fully utilized blocks. This configuration provides efficient data storage and retrieval.

Now consider a reversal of the dense and sparse dimension selections. In the following example, Region and Product are dense dimensions, and Time and Accounts are sparse dimensions.

As shown in Figure 34, the two-dimensional data blocks represent data values from the dense dimensions: Region and Product.

Figure 34: Two-Dimensional Data Block for Region and Product

Analytic Services creates data blocks for combinations of members in the sparse standard dimensions (providing at least one data value exists for the member combination). The sparse standard dimensions are Time and Accounts.

Figure 35 shows 12 data blocks. Data values exist for all combinations of members in the Time and Accounts dimensions; therefore, Analytic Services creates data blocks for all the member combinations. Because data values do not exist for all products in all regions, the data blocks have many empty cells. Data blocks with many empty cells store data inefficiently.

Figure 35: Data Blocks Created for Sparse Members on Time and Accounts

The Analytic Services Solution

When you create an optimized Analytic Services database, you need to consider carefully the following questions:

For more information on:



Hyperion Solutions Corporation link