# Getting started with rcloner

    library(rcloner)
    has_rclone <- rclone_available()
    if (!has_rclone) {
      message("rclone is not installed on this system. ",
              "Code chunks that require rclone are skipped. ",
              "Install with install_rclone().")
    }

## Overview

`rcloner` provides an R interface to rclone, a command-line program that supports over 40 cloud storage backends, including:

- S3-compatible stores: Amazon S3, MinIO, Ceph, Cloudflare R2, Backblaze B2, …
- Google Cloud Storage and Google Drive
- Azure Blob Storage
- Dropbox, OneDrive, Box, and many others

All file operations (copy, sync, list, move, delete, …) use the same consistent interface regardless of the storage backend.

## Installation

Install from CRAN:

    install.packages("rcloner")

Or the development version from GitHub:

    # install.packages("pak")
    pak::pak("boettiger-lab/rcloner")

### Installing the rclone binary

`rcloner` automatically locates a system-installed rclone binary. If rclone is not already on your `PATH`, install it with:

    install_rclone()

This downloads the appropriate pre-built binary from https://downloads.rclone.org for your operating system and architecture and stores it in a user-writable directory — no system privileges required.

Check the installed version:

    rclone_version()
    #> rclone v1.60.1-DEV
    #> - os/version: ubuntu 24.04 (64 bit)
    #> - os/kernel: 6.14.0-32-generic (x86_64)
    #> - os/type: linux
    #> - os/arch: amd64
    #> - go/version: go1.22.2
    #> - go/linking: dynamic
    #> - go/tags: none

## Configuring a remote

`rcloner` manages cloud storage credentials through remotes, which are named configurations stored in rclone’s config file.

### Amazon S3

    rclone_config_create(
      "aws",
      type     = "s3",
      provider = "AWS",
      access_key_id     = Sys.getenv("AWS_ACCESS_KEY_ID"),
      secret_access_key = Sys.getenv("AWS_SECRET_ACCESS_KEY"),
      region            = "us-east-1"
    )

### MinIO / S3-compatible

    rclone_config_create(
      "minio",
      type     = "s3",
      provider = "Minio",
      access_key_id     = Sys.getenv("MINIO_ACCESS_KEY"),
      secret_access_key = Sys.getenv("MINIO_SECRET_KEY"),
      endpoint          = "https://minio.example.com"
    )

### Listing configured remotes

    rclone_listremotes()

## Listing objects

`rclone_ls()` returns a data frame of objects at a given path.

### Local paths (no credentials needed)

    # List a local directory
    rclone_ls(tempdir(), files_only = TRUE)
    #> [1] Path     Name     Size     MimeType ModTime  IsDir
    #> <0 rows> (or 0-length row.names)

### Remote paths

    # List a bucket on a configured remote
    rclone_ls("aws:my-bucket")

    # Recursive listing
    rclone_ls("aws:my-bucket/data/", recursive = TRUE)

    # Directories only
    rclone_lsd("aws:my-bucket")

`rclone_ls()` parses `rclone lsjson` output and returns a data frame with columns `Path`, `Name`, `Size`, `MimeType`, `ModTime`, and `IsDir`.

## Copying and syncing files

### Copy

`rclone_copy()` copies files from source to destination, skipping identical files. It never deletes destination files.

    src  <- tempfile()
    dest <- tempfile()
    dir.create(src)
    dir.create(dest)
    writeLines("hello from rcloner", file.path(src, "readme.txt"))

    rclone_copy(src, dest)
    list.files(dest)
    #> [1] "readme.txt"

### Copy to/from the cloud

    # Upload a local directory to S3
    rclone_copy("/local/data", "aws:my-bucket/data")

    # Download a file from S3
    rclone_copy("aws:my-bucket/report.csv", "/local/downloads/")

    # Copy a URL directly to cloud storage (no local intermediate)
    rclone_copyurl(
      "https://raw.githubusercontent.com/tidyverse/readr/main/inst/extdata/mtcars.csv",
      "aws:my-bucket/mtcars.csv"
    )

### Sync

`rclone_sync()` makes the destination identical to the source, deleting destination files that are not in the source. Use with care.

    rclone_sync("aws:my-bucket/data", "/local/backup")

### Move

`rclone_move()` copies files and then deletes the source.

    rclone_move("aws:staging/file.csv", "aws:archive/2024/file.csv")

## Other file operations

    # Read a remote file into R
    contents <- rclone_cat("aws:my-bucket/config.yaml")

    # Get metadata for an object
    rclone_stat("aws:my-bucket/data.csv")

    # Total size of a path
    rclone_size("aws:my-bucket")

    # Create a bucket/directory
    rclone_mkdir("aws:new-bucket")

    # Delete files (keeps directories)
    rclone_delete("aws:my-bucket/old-data/")

    # Remove a path and all its contents
    rclone_purge("aws:my-bucket/scratch")

    # Generate a public link (where supported)
    rclone_link("aws:my-bucket/report.html")

    # Get storage quota info
    rclone_about("aws:")

## Using the low-level rclone() wrapper

Every rclone subcommand is accessible via the `rclone()` function, which accepts a character vector of arguments:

    # Equivalent to: rclone version
    rclone("version")

    # Run any rclone command
    rclone(c("check", "aws:bucket", "/local/backup", "--one-way"))

## Migrating from minioclient

If you are migrating from the `minioclient` package, the function mapping is:

`minioclient` | `rcloner`
--- | ---
`mc_alias_set()` | `rclone_config_create()`
`mc_cp()` | `rclone_copy()`
`mc_mv()` | `rclone_move()`
`mc_mirror()` | `rclone_sync()`
`mc_ls()` | `rclone_ls()`
`mc_cat()` | `rclone_cat()`
`mc_mb()` | `rclone_mkdir()`
`mc_rb()` | `rclone_purge()`
`mc_rm()` | `rclone_delete()`
`mc_du()` | `rclone_size()`
`mc_stat()` | `rclone_stat()`
`mc()` | `rclone()`

The main difference is that `rcloner` uses remotes (e.g. `"aws:bucket"`) rather than aliases (e.g. `"alias/bucket"`). Remote configuration is done with `rclone_config_create()` instead of `mc_alias_set()`.