R: Deterministic, Zero-Copy Parallel Execution for R

# Deterministic, Zero-Copy Parallel Execution for R

## Documentation for package ‘shard’ version 0.1.1

- DESCRIPTION file
- User guides, package vignettes and other documentation
- Package README

## Help Pages

A B C D E F I L M N P Q R S T U V W misc

## -- A --

adapter | Adapter Registry for Class-Specific Deep Sharing
affinity | CPU Affinity + mmap Advice (Advanced)
affinity_supported | Check whether CPU affinity is supported
altrep | ALTREP Shared Vectors
arena | Arena Semantic Scope
arena_depth | Get Current Arena Depth
as.array.shard_buffer | Coerce a Shared Memory Buffer to Array
as.double.shard_buffer | Coerce a Shared Memory Buffer to Double
as.integer.shard_buffer | Coerce a Shared Memory Buffer to Integer
as.logical.shard_buffer | Coerce a Shared Memory Buffer to Logical
as.matrix.shard_buffer | Coerce a Shared Memory Buffer to Matrix
as.raw.shard_buffer | Coerce a Shared Memory Buffer to Raw
as.vector.shard_buffer | Coerce a Shared Memory Buffer to a Vector
as_shared | Create a shared vector from an existing R vector
as_tibble | Materialize a shard table handle as a data.frame/tibble
as_tibble.shard_dataset | Materialize a dataset handle into a data.frame/tibble
as_tibble.shard_row_groups | Materialize a row-groups handle into a data.frame/tibble
as_tibble.shard_table_buffer | Materialize a fixed table handle or buffer
as_tibble.shard_table_handle | Materialize a table handle into a data.frame/tibble
attr<-.shard_shared_vector | Set an Attribute on a Shared Vector
attributes<-.shard_shared_vector | Set Attributes on a Shared Vector
available_backings | Get available shared memory backing types

## -- B --

bool | Column Types
buffer | Shared Memory Buffers
buffer_advise | Advise access pattern for a buffer
buffer_close | Close a Buffer
buffer_diagnostics | Buffer Diagnostics
buffer_info | Get Buffer Info
buffer_open | Open an Existing Buffer
buffer_path | Get Buffer Path

## -- C --

close.shard_deep_shared | Close a Shared Object
close.shard_shared | Close a Shared Object
close.shard_shared_vector | Close a Shared Object
collect | Collect a shard table into memory
collect.shard_dataset | Collect a dataset handle into memory
collect.shard_row_groups | Collect a row-groups handle into memory
collect.shard_table_handle | Collect a table handle into memory
coltypes | Column Types
copy_report | Data Copy Report
cow_report | Copy-on-Write Policy Report

## -- D --

diagnostics | Diagnostics API
dim.shard_buffer | Dimensions of a Shared Memory Buffer
dim<-.shard_shared_vector | Set dim on a Shared Vector
dimnames<-.shard_shared_vector | Set dimnames on a Shared Vector
dispatch | Task Dispatch Engine
dispatch_chunks | Dispatch Chunks to Worker Pool

## -- E --

ergonomics | Ergonomic Apply/Lapply Wrappers

## -- F --

factor_col | Categorical column type
fetch | Fetch Data from a Shared Object
fetch.default | Fetch Data from a Shared Object
fetch.shard_deep_shared | Fetch Data from a Shared Object
fetch.shard_shared | Fetch Data from a Shared Object
float64 | Column Types

## -- I --

idx_range | Contiguous index range
int32 | Column Types
in_arena | Check if Currently Inside an Arena
is_block_view | View Predicates
is_shared | Check if Object is Shared
is_shared_vector | Check if an object is a shared vector
is_view | View Predicates
is_windows | Check if running on Windows
iterate_row_groups | Iterate row groups

## -- L --

length.shard_buffer | Length of a Shared Memory Buffer
length.shard_descriptor | Length of a shard_descriptor Object
length.shard_descriptor_lazy | Length of a shard_descriptor_lazy Object
list_kernels | List registered kernels

## -- M --

materialize | Materialize Shared Object
materialize.default | Materialize Shared Object
materialize.shard_shared | Materialize Shared Object
materialize.shard_view_block | Materialize a block view into an R matrix
materialize.shard_view_gather | Materialize a gather view into an R matrix
mem_report | Memory Usage Report

## -- N --

names<-.shard_shared_vector | Set Names on a Shared Vector

## -- P --

pin_workers | Pin shard workers to CPU cores
pool | Worker Pool Management
pool_create | Create a Worker Pool
pool_dispatch | Dispatch Task to Worker
pool_get | Get the Current Worker Pool
pool_health_check | Check Pool Health
pool_lapply | Parallel Dispatch with Async Workers
pool_sapply | Parallel sapply with Supervision
pool_status | Get Pool Status
pool_stop | Stop the Worker Pool
print.arena_result | Print an arena_result object
print.shard_apply_policy | Print a shard_apply_policy Object
print.shard_buffer | Print a Shared Memory Buffer
print.shard_deep_shared | Print a Deep-Shared Object
print.shard_descriptor | Print a shard_descriptor Object
print.shard_descriptor_lazy | Print a shard_descriptor_lazy Object
print.shard_dispatch_result | Print a shard_dispatch_result Object
print.shard_health_report | Print a shard_health_report Object
print.shard_idx_range | Print a shard_idx_range object
print.shard_pool | Print a shard_pool Object
print.shard_reduce_result | Print a shard_reduce_result Object
print.shard_report | Print a shard_report Object
print.shard_result | Print a shard_result Object
print.shard_segment | Print a Shared Memory Segment
print.shard_shared | Print a Shared Object
print.shard_shared_vector | Print a Shared Vector
print.shard_tiles | Print a shard_tiles object
print.shard_view_block | Print a shard_view_block object
print.shard_view_gather | Print a shard_view_gather object
print.shard_worker | Print a shard_worker Object

## -- Q --

queue | Chunk Queue Management

## -- R --

raw_col | Column Types
recommendations | Performance Recommendations
register_kernel | Register a shard kernel
report | Generate Shard Runtime Report
results | Extract Results from shard_map
row_layout | Row layout for fixed-row table outputs
rss | RSS Monitoring Utilities

## -- S --

schema | Define a table schema
scratch_diagnostics | Scratch pool diagnostics
scratch_matrix | Get a scratch matrix
scratch_pool_config | Configure scratch pool limits
segment | Shared Memory Segment
segment_advise | Advise OS about expected access pattern for a segment
segment_close | Close a shared memory segment
segment_create | Create a new shared memory segment
segment_info | Get segment information
segment_open | Open an existing shared memory segment
segment_path | Get the path or name of a segment
segment_protect | Make a segment read-only
segment_read | Read raw data from a segment
segment_report | Shared Memory Segment Report
segment_size | Get the size of a segment
segment_write | Write data to a segment
set_affinity | Set CPU affinity for the current process
shards | Shard Descriptor Creation
shards_list | Create Shards from an Explicit Index List
shard_apply_matrix | Apply a Function Over Matrix Columns with Shared Inputs
shard_apply_policy | Apply Wrapper Policy
shard_crossprod | Parallel crossprod() using shard views + output buffers
shard_get_adapter | Get Adapter for an Object
shard_lapply_shared | Apply a Function Over a List with Optional Auto-Sharing
shard_list_adapters | List Registered Adapters
shard_map | Parallel Execution with shard_map
shard_reduce | Streaming Reductions over Shards
shard_register_adapter | Register an Adapter for Class-Specific Traversal
shard_share_hook | Deep Sharing Hook for Custom Classes
shard_share_hook.default | Deep Sharing Hook for Custom Classes
shard_unregister_adapter | Unregister an Adapter
share | Zero-Copy Shared Objects
shared_advise | Advise access pattern for a shared input vector/matrix
shared_diagnostics | Get diagnostics for a shared vector
shared_info | Get Information About a Shared Object
shared_reset_diagnostics | Reset diagnostic counters for a shared vector
shared_segment | Get the underlying segment from a shared vector
shared_vector | Create a shared vector from a segment
shared_view | Create a view (subset) of a shared vector
share_open | Open an Existing Shared Object by Path
stream_count | Stream row count
stream_filter | Stream-filter a dataset/row-groups into a new partitioned dataset
stream_group_count | Stream group-wise count
stream_group_sum | Stream group-wise sum
stream_map | Stream over row-groups/datasets and map
stream_map.shard_dataset | Stream over row-groups/datasets and map
stream_map.shard_row_groups | Stream over row-groups/datasets and map
stream_reduce | Stream over row-groups/datasets and reduce
stream_reduce.shard_dataset | Stream over row-groups/datasets and reduce
stream_reduce.shard_row_groups | Stream over row-groups/datasets and reduce
stream_sum | Stream sum of a numeric column
stream_top_k | Stream top-k rows by a numeric column
string_col | Column Types
succeeded | Check if shard_map Succeeded

## -- T --

table_buffer | Allocate a fixed-row table buffer
table_diagnostics | Table Diagnostics
table_finalize | Finalize a table buffer or sink
table_finalize.shard_table_buffer | Finalize a table buffer
table_finalize.shard_table_sink | Finalize a sink
table_sink | Create a table sink for row-group or partitioned outputs
table_write | Write tabular results into a table buffer or sink
table_write.shard_table_buffer | Write into a table buffer
table_write.shard_table_sink | Write a shard's row-group output
task_report | Task Execution Report

## -- U --

utils | Utility Functions

## -- V --

view | Create a view over a shared matrix
views | Zero-copy Views
view_block | Create a contiguous block view
view_diagnostics | View diagnostics
view_gather | Create a gather (indexed) view over a shared matrix
view_info | Introspection for a view

## -- W --

worker | Individual Worker Control

## -- misc --

[.shard_buffer | Extract Buffer Elements
[.shard_descriptor | Subset Shard Descriptor
[.shard_descriptor_lazy | Subset a shard_descriptor_lazy Object
[<-.shard_buffer | Assign to Buffer Elements
[<-.shard_shared_vector | Subset-assign a Shared Vector
[[.shard_descriptor | Get Single Shard
[[.shard_descriptor_lazy | Extract a Single Shard from a shard_descriptor_lazy Object
[[<-.shard_shared_vector | Double-bracket Subset-assign a Shared Vector