What Is an In-Memory Computing Chip? Architecture, Silicon Technologies, and Real-World Hardware

Quick answer

An in-memory computing chip reduces data movement by performing selected operations inside a memory array or very close to it. The term covers several architectures rather than one universal chip type. Analog compute-in-memory (CIM) can perform matrix operations through physical electrical behavior, digital CIM adds logic to or around memory arrays, and processing-in-memory (PIM) places programmable processing resources near high-capacity memory. These approaches can improve latency or energy efficiency for suitable data-intensive workloads, but the result depends on precision, memory capacity, peripheral circuits, software support, and the scope of the benchmark.

Why data movement matters

Conventional processors repeatedly move operands and intermediate results between compute units, caches, and external memory. For workloads with low arithmetic intensity, the time and energy spent transferring data can become more important than the arithmetic itself. Large embedding lookups, recommendation systems, graph analytics, database scans, and some neural-network layers are common examples. However, a workload should not be described as universally "memory-bound" without measuring the actual model, batch size, precision, memory hierarchy, and processor utilization.

In-memory computing changes where selected operations happen. Instead of fetching every weight or database item into a distant processor, the system performs part of the work where the data is stored. The potential benefit is therefore not simply "faster memory." It is a reduction in transfers across expensive interfaces. The tradeoff is that memory arrays, peripheral circuits, compilers, and algorithms must be designed together.

CIM, PIM, and near-memory terminology

Terminology varies across academic papers and vendors. The following definitions are useful for engineering comparison, but product documentation may use the labels differently.

The boundaries are not absolute. A DRAM module containing simple processors may be called PIM, while an SRAM macro that performs digital dot products may be called CIM. When comparing two platforms, inspect the data path and supported operations instead of relying on the marketing label.

How analog and digital in-memory computing work

Analog and mixed-signal CIM

In an idealized crossbar, each programmable cell represents a conductance related to a weight. Inputs are encoded as voltages, pulses, or time-domain signals. Currents accumulate along columns, allowing many multiply-accumulate contributions to be evaluated in parallel. In simplified form, an output can be written as y_j = sum(w_ijx_i). Real designs also need signed-weight encoding, multibit inputs and weights, reference paths, calibration, and conversion between digital and analog domains.

Analog CIM is attractive because the array uses device physics to perform many operations concurrently. The same physics creates engineering constraints: device-to-device variation, conductance drift, line resistance, noise, limited effective precision, programming overhead, and the energy and area of data converters. ADCs, DACs, drivers, and accumulation logic can dominate a design in some operating points, so array-only efficiency does not describe system efficiency.

Digital CIM

Digital CIM performs logic or arithmetic using digital states in or next to a memory macro. SRAM-based designs may activate modified bit cells and local logic to execute Boolean operations, population counts, bit-serial multiplication, or multiply-accumulate operations. Digital computation generally offers more predictable behavior and easier integration with standard digital design flows than analog CIM, but it may give up some density or energy advantages.

DRAM-PIM and near-memory processors

High-capacity PIM systems place digital processing resources close to DRAM banks, inside a memory stack, or on a memory module. They can operate on data at higher local bandwidth before sending reduced results to a host CPU or accelerator. This approach can be valuable for scans, filtering, embeddings, graph processing, and other bandwidth-heavy operations. It also requires a programming model that decides what runs near memory and what remains on the host.

Memory technologies used for in-memory computing

No memory technology is best for every in-memory computing design. Published cell size, endurance, retention, precision, and efficiency figures depend on process node, cell structure, operating conditions, and measurement method. For that reason, broad fixed ranges should not be treated as universal specifications.

Memory choice cannot be separated from the workload. Frequently updated weights favor different media from read-mostly inference. A small SRAM-CIM accelerator may offer excellent local performance but need external memory for a large model. A nonvolatile array may store weights efficiently but require calibration and endurance management.

Current research and commercial implementations

The market includes research chips, prototypes, development platforms, products in early commercialization, and products in volume production. These statuses are not interchangeable. Performance claims below are tied to the cited organization, workload, and date; vendor-reported results should not be generalized to unrelated systems.

These examples also show why a list of "commercial chips" needs status labels. A peer-reviewed prototype proves technical feasibility. A development card proves access for selected users. A production announcement indicates a different level of maturity, but system qualification, software support, customer volume, and independent benchmarks still matter.

Engineering limits and benchmark traps

Peripheral circuits can erase array-level gains

Analog arrays require drivers, converters, accumulators, calibration, control, and buffers. Digital CIM needs local arithmetic, data routing, and often bit-serial cycles. Report both the memory macro and the surrounding system. A TOPS/W number measured at an inner array cannot be compared directly with a board-level figure that includes external memory and host processing.

Precision is an end-to-end property

Do not classify all analog CIM as one fixed precision. Effective precision depends on cell states, input encoding, bit slicing, converter resolution, accumulation, calibration, and the neural network's tolerance to error. Some systems combine low-precision in-memory operations with higher-precision digital correction. Accuracy must be measured on the target model after mapping, not inferred from the nominal cell resolution.

Capacity and model mapping matter

If weights do not fit in the local arrays, the system may need tiling, repeated loading, or external memory. Those transfers can reduce the expected benefit. Sparse models, irregular access, attention layers, and dynamic shapes can also use the hardware differently from dense matrix-multiplication demonstrations.

Device behavior and reliability matter

Nonvolatile analog memories may exhibit write variability, retention drift, read noise, limited endurance, or temperature sensitivity. SRAM avoids many analog-storage effects but remains volatile and consumes more area. Evaluation should include calibration frequency, write traffic, error management, thermal conditions, and expected service life.

Software portability is still uneven

In-memory computing software stacks are often vendor-specific. A useful platform needs a compiler or graph mapper, supported operators, profiling tools, quantization or calibration workflows, host APIs, and a path for unsupported operations. Porting effort can outweigh hardware gains when the accelerated part of an application is small.

Training and inference have different requirements

Most current commercial positioning emphasizes inference or data-parallel acceleration. On-chip analog training remains an active research area because weight updates require high fidelity, repeated programming, and endurance management. It is therefore inaccurate to say analog training is impossible, but it should not be assumed to be a standard production capability.

Benchmark the whole workload

Compare systems only when precision, model, batch size, sparsity, accuracy target, power boundary, cooling, host contribution, and memory traffic are disclosed. Macro-level TOPS/W, chip-level throughput, system-level latency, and application-level cost answer different questions. There is no universal winner between analog CIM, digital CIM, and PIM.

How to evaluate an in-memory computing platform

Start with the bottleneck and deployment constraint, not the technology label.

1. Measure the bottleneck. Determine whether the application is limited by memory bandwidth, latency, arithmetic throughput, capacity, or communication between devices.

2. Define the accuracy and precision contract. Record the required model quality, numeric formats, accumulation precision, and allowed calibration.

3. Map the full workload. Identify which operators run in memory, which run on the host, and how much data crosses the boundary.

4. Check capacity and update behavior. Confirm whether weights or data fit locally and how often they must be rewritten.

5. Inspect the software path. Test compiler coverage, unsupported-operator fallback, debugging, profiling, model conversion, and deployment APIs.

6. Normalize benchmarks. Compare at the same model, batch size, precision, accuracy, and power boundary.

7. Verify maturity. Distinguish simulation, silicon prototype, sampling, development system, full production, and independently deployed product.

8. Run a representative proof of concept. Measure end-to-end latency, throughput, energy, host utilization, accuracy, and engineering effort on real data.

Procurement verification checklist

• Supported models, operators, data types, batch sizes, and sequence lengths

• Usable memory capacity after redundancy, metadata, and model partitioning

• Peak versus sustained throughput under the target accuracy requirement

• Power measurement boundary and required host, memory, networking, and cooling

• Toolchain version, framework support, licensing, and long-term maintenance

• Calibration, retention, endurance, error handling, and environmental limits

• Availability status, lead time, qualification evidence, and reference customers

Frequently asked questions

What is an in-memory computing chip?

It is a chip or memory-based platform that performs selected operations inside a memory array or close to stored data. The category includes analog CIM, digital CIM, DRAM-PIM, and near-memory architectures.

What is the difference between CIM and PIM?

CIM usually refers to operations performed within a memory array or its tightly coupled circuitry. PIM often refers to processing resources integrated in or near a memory device, bank, stack, or module. Vendor terminology overlaps, so the physical data path is more important than the label.

Are all in-memory computing chips analog?

No. Some designs use analog or mixed-signal crossbars, while others perform fully digital logic or arithmetic near SRAM or DRAM. Each approach has different precision, capacity, efficiency, and software tradeoffs.

Can in-memory computing replace a GPU?

Usually it acts as an accelerator rather than a complete replacement. A host CPU, GPU, or controller still handles unsupported operators, control flow, communication, preprocessing, and system management. The answer depends on workload coverage and software integration.

Can in-memory computing train AI models?

Research has demonstrated in-memory and analog training methods, but commercial activity is more concentrated on inference and data-intensive acceleration. Training requires accurate repeated weight updates, endurance management, and software support.

Which memory technology is best for CIM?

There is no universal best choice. SRAM offers mature, fast digital integration; DRAM offers capacity for PIM; ReRAM, PCM, and Flash can provide nonvolatile conductance states but introduce device and programming constraints. The workload and product maturity decide the practical fit.

How should TOPS/W claims be compared?

Use the same model, precision, batch size, sparsity, accuracy target, and power boundary. An array-only figure is not directly comparable with a complete chip or server result. Sustained application performance is more useful than an isolated peak number.

Is processing-in-memory commercially available?

Some PIM and digital in-memory platforms are commercially available, while others remain prototypes, demonstrations, samples, or early-production products. Confirm current availability, toolchain access, supported workloads, and customer qualification directly with the supplier.

Conclusion

In-memory computing is a family of architectures designed to reduce costly data movement. Analog CIM can exploit device physics for parallel matrix operations, digital CIM can bring predictable arithmetic into memory macros, and PIM can place programmable processing near high-capacity memory. None is automatically superior. The strongest evaluation combines workload profiling, accuracy requirements, capacity, peripheral overhead, software maturity, reliability, and current product status.

References

1. Nature Electronics: Computing-in-memory technologies and architectures for full-stack integration.

2. Nature Reviews Electrical Engineering: Memristor-based hardware accelerators for artificial intelligence.

3. IBM Research: An analog-AI chip for energy-efficient speech recognition and transcription.

4. IBM Research: New algorithms may enable training AI models on analog chips.

5. Analog or Digital In-memory Computing? Benchmarking through Quantitative Modeling.

6. Samsung Semiconductor: HBM-PIM and Aquabolt-XL demonstration announcement.

7. Samsung Semiconductor: LPDDR5X-PIM showcased at FMS 2026.

8. SK hynix: AiMX prototype announcement.

9. SK hynix: upgraded AiM-based AiMX showcase in 2025.

10. UPMEM: processing-in-memory architecture and reference platform.

11. d-Matrix: Corsair full-production announcement, June 2026.

12. d-Matrix: Corsair technical white paper.

13. GSI Technology: 2026 Form 10-K.

14. GSI Technology: Gemini-II APU tape-out announcement.