Behind the scenes at Labs
cancel
Showing results for 
Search instead for 
Did you mean: 

Getting started with Memory-Driven Computing: Tools you can use today

DanFeldman

 

DSC_2352.jpg

 

 

 

This post discusses some of the Memory-Driven Computing programming tools developed by system software researchers at Hewlett Packard Labs.  The full list of applications and tools is available in The Memory-Driven Computing Developer Toolkit.  Our focus in this post is on software that programmers can use today in conventional environments in order to start to learn about these exciting technologies.

Memory-Driven Computing shifts the emphasis in data processing systems to data in memory: instead of bringing data to processors, Memory-Driven Computing brings processing to data. Memory-Driven Computing allows us to ingest, store, and manipulate massive data sets while simultaneously reducing energy per bit by orders of magnitude.  The Machine research project at Hewlett Packard Labs has demonstrated a Memory-Driven Computing architecture that includes emerging technologies such as fast, persistent memory, photonics, and system-on-chip (SoC) processors. Recently, HPE unveiled the world’s largest single-memory computer and first Memory-Driven Computing architecture. In addition, HPE has products available today that lead the way to more cost-effective and energy-efficient computing:

  • Current HPE Superdome X systems feature massive, shared, multi-terabyte memories and hundreds of processor cores
  • Current HPE servers feature persistent memoryimplemented with NVDIMMs.

Future persistent memory products may employ novel non-volatile memory (NVM) device technologies such as Memristors.  Furthermore, fabric-attached memory (FAM) enables scaling far beyond today's cache-coherent, shared memory while preserving the convenience of byte-addressable memory.

The figure below presents one way to think about the various possible attributes of the memory in a Memory-Driven Computing environment: persistent, fabric attached and large.  In fact, a computer can implement some or all of these technologies and software tools can be designed to improve application operation in various combinations of them.  The Venn diagram depicts those three Memory-Driven Computing technologies and their intersections and maps a number of open source software tools released by Hewlett Packard Labs onto the Memory-Driven Computing implementation space.

Venn Diagram 7-17-2017.JPG 

Here are entry points to documentation on individual tools in the current release of our Memory Driven Computing Toolkit:

In order to understand the full power of Memory-Driven Computing and the tools listed above, it is helpful to understand the unique opportunities offered by each of the key Memory-Driven Computing technologies: Fabric-attached memory, persistent memory and (very) large memories.

Fabric-Attached Memory

Fabric-Attached Memory (FAM) promises to scale shared, byte-addressable memory far beyond today's cache-coherent shared memories. FAM offers shared access to non-cache-coherent memory, for massive numbers of system-on-chip (SoC) processors accessing the memory over a memory fabric. FAM also provides the necessary atomic primitives and cache flushing operations to enable software to share access to FAM safely.

Persistent Memory

Persistent memory refers to memory whose contents outlive the processes that allocate, populate, and manipulate the memory. Persistence must be defined with respect to events that terminate processes. For example, memory that is persistent with respect to mere process termination, normal or otherwise, is readily available on today's conventional computers running conventional operating systems (via file-backed memory mapping). By contrast, persistence with respect to operating system crashes or power outages requires additional hardware and/or software support.

Persistent memory is not the same as non-volatile memory (NVM); the latter term denotes memory device technologies (e.g., Memristor) that retain data in the absence of continuously supplied power. NVM can facilitate the implementation of memory that is persistent with respect to certain failures (e.g., power outages), but persistent memory can be implemented without NVM. For example, current HPE ProLiant Persistent Memory Servers employ NVDIMMs based on conventional volatile DRAM with flash storage and sufficient standby power to copy the contents of the DRAM to flash in the event that utility power is lost. The Memory-Driven Computing Developer Toolkit  addresses persistence rather than the specific memory technologies that implement persistence.

Today's applications manipulate persistent data indirectly, via file system interfaces (open/close/read/write) or database interfaces (SQL or put/get) that invoke complex layers of intermediate software between the application and the persistent data on block-addressable storage (hard disks or solid-state drives).

Legacy systems can still use filesystem and database system APIs to access data as storage in persistent memory, but they also still pay the overheads of invoking those complex layers of intermediate software. The alternative is to use persistent memory directly as byte-addressable memory.

In persistent memory programming, applications manipulate persistent data directly, in-place, using CPU instructions (LOAD and STORE). Persistent memory programming offers two main advantages over the conventional alternative: greatly simplified application software due to the elimination of separate data formats for memory and storage; and greatly improved performance due to the elimination of large, complex intermediate software layers such as file systems and relational database management systems.  This latter benefit also yields significant reductions in the energy required by no-longer necessary data movement and manipulation.

Hewlett Packard Labs has developed a range of technologies to help our customers realize the full value of Memory-Driven Computing and persistent memory. Our technologies address major challenges: organizing data, messaging, protecting the integrity of data from failures, allocating and managing memory, and predicting application performance.

One major concern surrounding persistent memory programming is the vulnerability of application data in persistent memory to corruption or loss due to failure. Application data in memory is deemed consistent if it satisfies whatever invariants or integrity constraints required by the applications that access it. Consistency is orthogonal to persistence - even if memory is persistent its contents need not be consistent. The main worry about persistent memory programming is that a failure (e.g., process crash, OS kernel panic, or power outage) that occurs while an application is updating its persistent data may corrupt or destroy the data. To realize the full benefits of persistent memory programming, application developers require mechanisms to update data in persistent memory from one consistent state to the next even in the presence of failures.

Another requirement for persistent memory programming is mechanisms that allow applications, administrators, and end users to allocate and manage persistent memory.

(Very) Large Memory

Very large main memory seems on the face of it to be an unproblematic continuation of ongoing trends in memory density and capacity.  Memory sizes in the 100s of TB and multiple PB range provide fantastic opportunities for rethinking application design – imagine keeping data structures like large graphs and maps that previously had to be serialized onto some kind of storage device and operated on in parts in main memory.  Applications would run much faster and scale much further.  However, top of the line desktop computers have main memory sizes well under a terabyte while servers are in the low terabyte range.  Exceptional high performance and mission critical devices, such as HPE’s Superdome X and Integrity MC990X systems support memories approaching 100TB.  Algorithms that perform well enough at today’s typical memory sizes will breakdown at scales that are multiple orders of magnitude larger.  Tools like MPGC, FOEDUS and Shoveller are designed to work at these extraordinary scales, thereby enabling scaling for the applications that rely on them.

Memory-Driven Computing tools you can use today

This section discusses a number of tools that are currently available for readers who are interested in exploring these novel technologies.  The table below complements the Venn diagram introduced in the first section of this post by showing salient characteristics of each of them and the sections that follow provide details on when and why you might want to use each of them.

 

ArtifactMatrix 7-17-2017.JPG

 

 

Persistent Memory Programming

Atlas enables conventional multi-threaded C/Pthreads software to employ persistent memory with crash resilience: Atlas guarantees that failures due to causes such as power outages, OS kernel panics, and application process crashes do not corrupt or destroy application data in persistent memory.

Data Organization

The Radix Tree is a user-space library that implements a radix tree based on FAM atomic instructions. The Radix Tree is a space-optimized trie suitable for efficiently storing and searching text and intrinsically hierarchical data; it supports transactional updates by multiple non-coherent SoCs in a shared FAM environment.

Log, Checkpoint, and Concurrent Transactions

Technologies that enable applications to log, checkpoint, and manipulate data with persistent memory include libnvwal, CRIU-PMEM, Managed Data Structures (MDS), and FOEDUS.

libnvwal is a user-space library for applications to manage write-ahead log on persistent memory (e.g., NVDIMM). CRIU-PMEM is an application-transparent, system level checkpointing tool using persistent memory.

The Managed Data Structures (MDS) library delivers a simple, high-level programming model that supports multi-threaded, multi-process creation, use and sharing of data structures in persistent memory, via APIs in Java and C++.  FOEDUS is a database engine roughly similar in operation to Berkeley DB but optimized for large-memory, manycore machines.

Tool

Targeted hardware

Intended use

Next release features

libnvwal

ProLiant Gen10 persistent memory

efficient write-ahead-logging in persistent memory

None

CRIU-PMEM

persistent memory, DRAM, HDD or SSD

application-transparent system level checkpointing

PMEM-awareness and performance optimization

MDS

ProLiant Gen10 persistent memory, mmapped file on HDD/SSD

convenient concurrent transactions on data structures in persistent memory

more data structures, larger scale

FOEDUS

large persistent memory and manycore machines

embeded database engine

None

 

Memory Allocation and Management

Technologies for memory management/allocation include ALPS, MPGC, Shoveller, and NVMM. ALPS provides a low-level abstraction layer that relieves the user from the details of mapping, addressing, and allocating shared, persistent memory. MPGC is a multi-process garbage collector for use with MDS or standalone; it addresses the special challenges of garbage collection of memory shared among independently developed processes. Shoveller is a scalable, parallel-log-structured memory allocator for key-value stores. NVMM is a library built atop the Librarian File System (LFS) that provides simple abstractions for accessing and allocating persistent memory.

Features

ALPS

MPGC

Shoveller

NVMM

Single/multi-shelf

Single

Single

N/A

Multi

Single/multi-process

Multi

Multi

Multi

Multi

Single/multi-node

see Note 1

Single

Single

Multi

Crash resilient?

Yes

Yes

No

WIP

Allocation size

Variable

Variable

Variable

Variable

Data structures

Volatile + persistent

Volatile + lock-free

Volatile

Persistent

APIs

Alloc/Free

Alloc

Get/Put/Del

Alloc/Free

Garbage collector

No

Yes, online

No

No

De-fragmentation

No

No

Yes

No

Grow capacity

No

Yes

No

Yes

Targeted hardware

CC-NUMA, FAM

persistent memory

large scale machines (e.g, SuperDomeX)

FAM, persistent memory

Intended use

memory allocation

memory management for persistent memory

key-value store

memory allocation for FAM

Next release features

none

performance, scale

none

crash recovery

Note 1: Multi for allocation, no remote free

Learn more

The Memory-Driven Computing Developer Toolkit is part of a Hewlett Packard Labs initiative to place important, enabling software technologies into open source so that developers can start exploring what it means to program for Memory-Driven Computing architectures – systems with massive pools of non-volatile memory, a fast memory fabric and task-specific processing.

Related links

Dan Feldman
Director, System Software Research
Hewlett Packard Labs
About the Author

DanFeldman

I manage the System Software Research group in Hewlett Packard Labs.