govt.nzAs datasets develop from megabytes to terabytes to petabytes, the cost of shifting knowledge from the block storage gadgets throughout interconnects into system memory, performing computation after which storing the massive dataset again to persistent storage is rising in terms of time and energy (watts). Moreover, heterogeneous computing hardware more and more needs entry to the identical datasets. For example, a common-objective CPU could also be used for assembling and preprocessing a dataset and scheduling duties, but a specialised compute engine (like a GPU) is way quicker at coaching an AI mannequin. A more efficient resolution is required that reduces the switch of massive datasets from storage on to processor-accessible memory. A number of organizations have pushed the business towards solutions to these problems by preserving the datasets in giant, byte-addressable, sharable memory. In the 1990s, the scalable coherent interface (SCI) allowed multiple CPUs to entry memory in a coherent means inside a system. The heterogeneous system structure (HSA)1 specification allowed memory sharing between gadgets of differing kinds on the identical bus.

Within the decade starting in 2010, the Gen-Z normal delivered a memory-semantic bus protocol with high bandwidth and low latency with coherency. These efforts culminated in the extensively adopted Compute Express Hyperlink (CXLTM) customary being used today. Because the formation of the Compute Express Hyperlink (CXL) consortium, Micron has been and stays an active contributor. Compute Express Hyperlink opens the door for saving time and energy. The new CXL 3.1 normal permits for byte-addressable, load-store-accessible memory like DRAM to be shared between completely different hosts over a low-latency, excessive-bandwidth interface using trade-commonplace elements. This sharing opens new doorways previously only potential by means of expensive, proprietary equipment. With shared Memory Wave techniques, the data can be loaded into shared memory once after which processed multiple times by a number of hosts and accelerators in a pipeline, with out incurring the cost of copying data to local memory, block storage protocols and latency. Moreover, some network data transfers can be eradicated.

For example, data may be ingested and stored in shared memory over time by a bunch linked to a sensor array. As soon as resident in memory, a second host optimized for this goal can clear and Memory Wave Audio preprocess the information, adopted by a third host processing the data. Meanwhile, the primary host has been ingesting a second dataset. The only info that must be passed between the hosts is a message pointing to the data to point it's ready for processing. The massive dataset never has to move or be copied, saving bandwidth, energy and memory house. One other example of zero-copy data sharing is a producer-client knowledge mannequin the place a single host is accountable for amassing information in memory, and then a number of different hosts eat the information after it’s written. As earlier than, the producer just needs to send a message pointing to the address of the information, signaling the other hosts that it’s prepared for consumption.

Zero-copy knowledge sharing could be additional enhanced by CXL memory modules having built-in processing capabilities. For example, if a CXL memory module can perform a repetitive mathematical operation or data transformation on an information object totally within the module, system bandwidth and power may be saved. These financial savings are achieved by commanding the memory module to execute the operation with out the info ever leaving the module using a functionality called close to memory compute (NMC). Additionally, the low-latency CXL fabric could be leveraged to send messages with low overhead very quickly from one host to another, between hosts and memory modules, or between memory modules. These connections can be used to synchronize steps and share pointers between producers and customers. Beyond NMC and communication advantages, advanced memory telemetry will be added to CXL modules to offer a new window into actual-world software site visitors in the shared devices2 with out burdening the host processors.

With the insights gained, working programs and administration software program can optimize information placement (memory tiering) and tune other system parameters to meet operating objectives, from efficiency to power consumption. Additional memory-intensive, worth-add functions such as transactions are also ideally suited to NMC. Micron is excited to combine large, scale-out CXL global shared memory and enhanced Memory Wave Audio features into our memory lake idea. As datasets develop from megabytes to terabytes to petabytes, the price of moving data from the block storage devices throughout interconnects into system memory, performing computation and then storing the big dataset again to persistent storage is rising when it comes to time and power (watts). Additionally, heterogeneous computing hardware increasingly wants entry to the same datasets. For example, a basic-goal CPU could also be used for assembling and preprocessing a dataset and scheduling tasks, but a specialized compute engine (like a GPU) is far faster at training an AI mannequin.

Edit

Pub: 14 Aug 2025 10:01 UTC

Views: 6