# Interact with local models

## Intro

LlamaGPT-Chat is a command line chat application. It integrates with the LLM via C++. This means that it will only work with LLM’s that have been in the same language. There are few available today that are free to download and use. Because they are in C++, the models are able to run locally on your computer, even if there is no GPU available. Most of the models are quite fast in their responses. By integrating with LlamaGPT-Chat, able to access multiple types of LLM’s with only one additional back-end within `chattr`.

## Installation

To get LlamaGPT-Chat working you machine you will need two things:

1. A version of LlamaGPT-Chat that works in your computer

2. An LLM file that works with with the chat program

### Install LLamaGPT-Chat

LlamaGPT-Chat will need a “compiled binary” that is specific to your Operating System. For example, for Windows, a compiled binary should be an .exe file.

The GitHub repository offers pre-compiled binaries that you can download and use: Releases. Depending on the system’s security, the pre-compiled program may blocked from running. If that is the case, you will need to “build” the program in your computer. The instructions to do this are here.

### LLM file

The GitHub repository contains a list of compatible models. Download at least one of the models, and make note of where it was saved to in your computer.

## Setup

To start, instruct `chattr` to switch the LLamaGPT back-end using `chattr_use()`

    library(chattr)

    chattr_use("llamagpt")
    #>
    #> ── chattr
    #> • Provider: LlamaGPT
    #> • Path/URL: ~/LlamaGPTJ-chat/build/bin/chat
    #> • Model: ~/ggml-gpt4all-j-v1.3-groovy.bin
    #> • Label: GPT4ALL 1.3 (LlamaGPT)

If this is the first time you use this interface, confirm that the path of to the compiled program, and the model matches to what you have in your machine. To check that use `chattr_defaults()`

    chattr_defaults()
    #>
    #> ── chattr ──────────────────────────────────────────────────────────────────────
    #>
    #> ── Defaults for: Default ──
    #>
    #> ── Prompt:
    #> • Use the R language, the tidyverse, and tidymodels
    #>
    #> ── Model
    #> • Provider: LlamaGPT
    #> • Path/URL: ~/LlamaGPTJ-chat/build/bin/chat
    #> • Model: ~/ggml-gpt4all-j-v1.3-groovy.bin
    #> • Label: GPT4ALL 1.3 (LlamaGPT)
    #>
    #> ── Model Arguments:
    #> • threads: 4
    #> • temp: 0.01
    #> • n_predict: 1000
    #>
    #> ── Context:
    #> Max Data Files: 0
    #> Max Data Frames: 0
    #> ✖ Chat History
    #> ✖ Document contents

If either, or both, paths are incorrect, correct them by updating the `path`, and `model` arguments in `chattr_defaults()`

    chattr_defaults(path = "[path to compiled program]", model = "[path to model]")
    #>
    #> ── chattr ──────────────────────────────────────────────────────────────────────
    #>
    #> ── Defaults for: Default ──
    #>
    #> ── Prompt:
    #> • Use the R language, the tidyverse, and tidymodels
    #>
    #> ── Model
    #> • Provider: LlamaGPT
    #> • Path/URL: [path to compiled program]
    #> • Model: [path to model]
    #> • Label: GPT4ALL 1.3 (LlamaGPT)
    #>
    #> ── Model Arguments:
    #> • threads: 4
    #> • temp: 0.01
    #> • n_predict: 1000
    #>
    #> ── Context:
    #> Max Data Files: 0
    #> Max Data Frames: 0
    #> ✖ Chat History
    #> ✖ Document contents

So that you do not need to change those defaults every time you start a new R session, use `chattr_defaults_save()`. That will create a YAML file in your working directory. `chattr` will use that file to override the defaults.

    chattr_defaults_save()

Use `chattr_test()` to confirm that the `chattr` is able to communicate with LLamaGPT-Chat, and that the model it will use with your R session is accessible

    chattr_test()
    #> ✔ Model started sucessfully
    #> ✔ Model session closed sucessfully

## Model Arguments

The arguments sent to the model can be modified in `chattr_defaults()` by modifying the `model_arguments` argument. It expects a list object. Here are the arguments it sets by default:

    chattr_defaults()$model_arguments
    #> $threads
    #> [1] 4
    #>
    #> $temp
    #> [1] 0.01
    #>
    #> $n_predict
    #> [1] 1000

Here is an example of adding `batch_size` to the defaults:

    chattr_defaults(model_arguments = list(batch_size = 40))
    #>
    #> ── chattr ──────────────────────────────────────────────────────────────────────
    #>
    #> ── Defaults for: Default ──
    #>
    #> ── Prompt:
    #> • Use the R language, the tidyverse, and tidymodels
    #>
    #> ── Model
    #> • Provider: LlamaGPT
    #> • Path/URL: [path to compiled program]
    #> • Model: [path to model]
    #> • Label: GPT4ALL 1.3 (LlamaGPT)
    #>
    #> ── Model Arguments:
    #> • batch_size: 40
    #> • threads: 4
    #> • temp: 0.01
    #> • n_predict: 1000
    #>
    #> ── Context:
    #> Max Data Files: 0
    #> Max Data Frames: 0
    #> ✖ Chat History
    #> ✖ Document contents

To see the most current list of available model arguments go to: Detailed command list.

IMPORTANT - `chattr` passes the arguments directly to LLamaGPT-chat as a command line flag, except `model`. `chattr` will use the value in `chattr_defaults(model = "[path to model]")` instead.