[![CRAN_Status_Badge](http://www.r-pkg.org/badges/version/memshare)](https://cran.r-project.org/package=memshare)
[![CRAN RStudio mirror downloads](https://cranlogs.r-pkg.org/badges/grand-total/memshare?color=blue)](https://r-pkg.org/pkg/memshare)
[![CRAN RStudio mirror downloads](https://cranlogs.r-pkg.org/badges/last-month/memshare?color=green)](https://r-pkg.org/pkg/memshare)
[![R-CMD-check](https://github.com/Mthrun/memshare/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/Mthrun/memshare/actions/workflows/R-CMD-check.yaml)

# memshare: Shared memory multithreading in R via C++17, with zero-copy access through ALTREP data structures.


---

# Overview

`memshare` enables multicore computation in R without redundant memory copies. Large vectors, matrices, or lists are stored once in shared memory and exposed to R processes as **ALTREP views**. This allows workers in a PSOCK cluster to operate on the same physical data while avoiding serialization overhead.

Key features:

- Shared memory allocation via C++17 back-end (`shm_open` on Unix, `MapViewOfFile` on Windows).
- ALTREP wrappers so R sees shared objects as native vectors or matrices.
- High-level parallels to `parallel::parApply` and `parallel::parLapply`:
  - `memApply()` — apply a function row/column-wise over a matrix in shared memory.
  - `memLapply()` — apply a function element-wise over a list in shared memory.
- Automatic cleanup of shared objects when all views are released or when the package is unloaded.
- Tested on Linux, macOS, and Windows.

---

# Installation

From CRAN:

```r
install.packages("memshare")
```

From GitHub (development version):

```r
remotes::install_github("mthrun/memshare")
```

System requirements: R ≥ 4.0 and a C++17 compiler.

# Quick start

## Example 1: Parallel correlations with a matrix

```r
library(memshare)
library(parallel)

set.seed(1)
n = 10000
p = 2000

X = matrix(rnorm(n * p), n, p)
y = rnorm(n)

res = memApply(
  X = X, MARGIN = 2,
  FUN = function(v, y) cor(v, y),
  VARS = list(y = y)
)
str(res)
```

## Example 2: List operations

```r
library(memshare)
library(parallel)

list_length = 1000
matrix_dim = 100

ListV = lapply(
     1:list_length,
     function(i) matrix(rnorm(matrix_dim * matrix_dim),
     nrow = matrix_dim, ncol = matrix_dim))

y = rnorm(matrix_dim)

namespace = "ID123"
res = memshare::memLapply(ListV, function(el, y) {
   el %*% y
}, NAMESPACE=namespace, VARS=list(y=y), MAX.CORES = 1)

```

Each element el of ListV is multiplied by y in parallel. The list resides once in shared memory.

## Concepts

- Pages: memory regions owned by the current R session that loaded the package.

- Views: ALTREP wrappers exposing shared memory variables (read/write capable).

- Namespaces: string identifiers defining a shared memory context across sessions.

When the package is detached, all handles and associated shared memory pages are released, unless another R process still holds references.


## Shared-memory lifecycle

`memshare` exposes explicit lifecycle functions so you can control when data is
placed in shared memory and when it is freed.

### `registerVariables(namespace, variableList)`
Allocate shared memory and copy R objects (matrices or vectors, or lists for `memLapply`) into it.
- `namespace`: character(1). Identifier of the shared memory context shared across processes.
- `variableList`: a **named** list of objects to register. Names become the keys under which
  you can later retrieve views.

**Example**

```r
library(memshare)

ns <- "my_namespace"
X  <- matrix(rnorm(1e4), 100, 100)
y  <- rnorm(100)

registerVariables(ns, list(X = X, y = y))
# Now X and y live once in shared memory and can be accessed from other R processes
```

### `releaseVariables(namespace, variableNames)`
Delete variables from the shared memory space. Shared regions are only removed when **no active views remain**.
- `namespace`: character(1) used above.
- `variableNames`: character vector of variable names to free.

**Example**

```r
# After all workers have released their views:
releaseVariables(ns, c("X", "y"))
```

### Working with views: `retrieveViews()` and `releaseViews()`
To avoid duplication, workers attach to shared memory by **views**:
- `retrieveViews(namespace, c("X","y"))` returns ALTREP-backed objects that behave like ordinary R matrices/vectors.
- Always call `releaseViews(namespace, ...)` when finished so that the backing memory can be reclaimed.

**Example (worker-side)**

```r
vlist <- retrieveViews(ns, c("X","y"))
# use vlist$X and vlist$y
releaseViews(ns, c("X","y"))
```

> Tip: `memApply()` and `memLapply()` manage views for you automatically, but the low-level API above is useful for custom workflows.

### Manual

The full manual for users or developers is available here:
[Package documentation](https://CRAN.R-project.org/package=memshare/memshare.pdf)

## References <a name="references"/>

[Thrun/Märte, 2026]  Thrun, M. C., & Märte, J.: Memshare: Memory Sharing for Multicore Computation in R with an Application to Feature Selection by Mutual Information using PDE, R Journal, Vol. 17(4), pp. 306 - 322, doi 10.32614/RJ-2025-043, 2026.

[Thrun et al., 2020] Thrun, M.C., Gehlert, T., & Ultsch, A.: Analyzing the Fine Structure of Distributions, PLOS ONE, 15(10), e0238835, 2020.

[Ultsch, 2005] Ultsch, A.: Pareto Density Estimation: A Density Estimation for Knowledge Discovery, Proceedings of the 28th Annual Conference of the German Classification Society, Springer, 2005

