The Memory Contract Manager (MCM) is a consolidated memory management component within MicroStrategy Intelligence Server that grants or denies memory requests for different tasks based on the amount of memory available for the Intelligence Server process.
MCM does not guarantee that a memory depletion scenario would not happen, but reduces the possibility of its occurrence.
When a request is granted or denied by the MCM,a message is logged before actual memory is allocated by the operating system to the DSSError.
Examples of task governed by the MCM:
- Database requests to the metadata or the warehouse
- SQL generation
- Analytical Engine processing
- Cache Creation/Update
- Report parsing and serialization for network transfer
- XML generation
Examples of task not governed by the MCM:
- Requests and caches metadata objects
- Requests and caches attribute elements
- Determines if report results are in cache
- Preparation of MicroStrategy Narrowcast Server request
- Preparation of datamart request
- Preparation of document
- Graphing
- Report Cross Tabulation
- Prompting
- Error Messages
The MCM is designed to protect MicroStrategy Intelligence Server in cases where a request will cause the system to approach a memory depletion, and not to manage the actual memory used by MicroStrategy Intelligence Server itself. It is possible that if MicroStrategy Intelligence Server is operating on a machine with insufficient resources, the MCM could quickly begin to deny requests. This situation would give the impression that large requests were submitted; when in reality the MCM is preventing memory depletion issues due to insufficient memory resources. Before going into some examples of how the MCM determines when requests will be denied, we would need to understand the different section of memory depletion such as virtual memory, virtual address space etc.
Virtual memory:
Virtual memory consists of the physical memory (RAM) on the machine and the system page file. When a machine runs out of virtual memory, processes on the machine are no longer able to process instructions. As a process on the server needs memory, it commits pages of the virtual memory for its own use.
Virtual Address Space:
Windows 32-bit Operating systems (OS) define a flat 32-bit virtual address map giving each application 4GB of virtual address space. However, all the applications share this space with the kernel using a 2GB-2GB divide. Depletions can occur when more user address space is requested but not available due to OS limits.
The virtual address space limitation is enforced by the operating system and may be either 2GB or 3GB depending on the specific Microsoft optimizations.
The virtual address space limitation is enforced by the operating system and may be either 2GB or 3GB depending on the specific Microsoft optimizations.
This is no longer a limitation with 64-bit operating systems as they can address close to 16 exabytes.
The pages within a process' address space can be found in one of three states within the Windows operating system:
| State | Description |
| Committed | Committed represents virtual address space pages which physical storage (in memory or on disk) has been allocated. A process' committed memory is also called private bytes. |
| Reserved | Reserved pages have been set aside for future use. The process cannot access a reserved page until it is committed, and there is no physical storage associated with it. The combination of reserved and committed address space is a process' virtual bytes. |
| Free | Free pages of address space are available to be committed or reserved. |
The private bytes and the virtual bytes for a process can be monitored using Microsoft Performance Monitor. The following depicts the virtual memory address states:
How does the MCM grant or deny a task?
The MCM uses a concept similar to contract-based leasing in its interaction with the tasks asking for memory. The MCM acts as a middle layer between the tasks and the OS kernel by keeping record of how much memory is available and how much memory has already been contracted out to the tasks on behalf of jobs.
After a task requests for a contract, it will provide the MCM with an estimate of how much memory is required for this new task. When memory is granted, the MCM updates its available memory (Contract List) and available memory decreases. After the task is complete or cancelled, the available memory is released. At this stage, the operations of the MCM are straightforward; however, once the available memory reaches a critical level and memory depletion may occur, additional processes kick in.
Note: MCM does not do memory allocations on behalf of the tasks from OS but only grants the component the privilege to allocate the amount of memory the component is asking for. The tasks allocate memory from the OS after receiving the grant.
Note: The MCM is active whenever it is enabled, even if the reserved memory is set to 0 MB. In this configuration, the MCM still monitors the amount of memory used and contracted to ensure memory depletion does not occur. To improve efficiency, the MCM will check for potential memory depletion every 10th request or any request over 10 MB.
During the Request Idle Mode, if the Memory Request Idle Time is reached and all requests have not yet met requirements and the next request, if it also exceeds requirements, will trigger MicroStrategy Intelligence Server to shut down and restart.
By default, the Memory Request Idle Time is five minutes. An idle time value of -1 will result in MicroStrategy Intelligence Server waiting indefinitely for requests.
Note: The request idle time is not enforced via an internal clock or scheduler. MCM checks whether the condition to fall out of the request idle mode is met every 10th request or with every request greater than 10MB. If such a condition is not met at this check, MCM checks for how much time has passed since the request idle mode was triggered. If this time is more than the request idle time setting, then a restart is issued. This helps eliminate unnecessary restarts in situations as the following.
Supposing that the request idle time is set to five minute, one request causes the request idle mode to be triggered but no more requests are submitted for quite a while, if there was a scheduled check and the request idle mode was not recovered at the fifth minute, the MicroStrategy Intelligence Server would have been started unnecessarily.
Comments