Author: Luka Krajnc
Date: 2022-03-22

# Loading and automatic trimming of density profiles

#### Luka Krajnc

#### 2022-03-22

# Introduction

The use of resistance drilling for density assessment is becoming more and more used across different disciplines. This R package provides a way to do the density analysis in R by loading the measurements, trimming them automatically or by hand. It is also possible to remove trends from profiles and measure ring widths. Please note that this package was written with a focus on forestry, where density profiles are usually obtained on living standing trees. Other uses are also possible, wherever you are using density profiles.

Let’s load the library to see what it can do:

    library(densitr)

# Loading resistance drilling density profiles

In the current version of densitr, only files (*.dpa) created by Rinntech Resistograph® devices are supported. It was tested to work on files produced by R650-RC Resistograph®. Other file types are currently not supported, they might be added later on. Contributions and pull requests are always welcome.

Never try to load *.dpa files directly from the Resistograph device into R, always copy them first onto your computer and load them from there. Try at your own risk. This vignette uses some example density profiles recorded using an Rinntech Resistograph® R650-SC, which are included with this package. There is a total of 15 profiles, representing various tree species. Some are drilled bark-to-bark, while others were drilled bark-to-pith because of their dimensions.

Loading a single file is simple:

    dp <- dpload(dp.file = system.file("extdata", "00010001.dpa", package = "densitr"))
    dp
    #>
    #>
    #> Density profile ID:  00010001
    #>
    #> Total length:    11521 x 1/100 mm
    #> Trimmed: start not trimmed, end not trimmed
    #>
    #> Data preview:
    #>  amplitude position       ID
    #>          0        1 00010001
    #>          0        2 00010001
    #>          0        3 00010001
    #>          0        4 00010001
    #>          0        5 00010001
    #>          0        6 00010001
    #> ...

The function returns an S3 object with the class of dp, which is essentially a list with two items. In the first, `$data`, you will find a data frame with all of the actual profile data. Column position of that data frame holds the horizontal position as saved by the device when drilling. Column amplitude holds actual density values as recorded by the device. In the second list of an dpa object, `$footer`, you will find additional information about the individual density profile, as recorded by the device.

View dp measurement data:

    head(dp$data)
    #>   amplitude position       ID
    #> 1         0        1 00010001
    #> 2         0        2 00010001
    #> 3         0        3 00010001
    #> 4         0        4 00010001
    #> 5         0        5 00010001
    #> 6         0        6 00010001

View dp additional data:

    head(dp$footer)
    #>         ID Length yMin yMax    xUnit yUnit      Keycode     Date Time Version
    #> 1 00010001 115 mm  100 1500 1/100 mm   rel 00010001.dpa 20191126 1025     200
    #>   Project ResistographProfileNumber Revision End_Of_Drilling
    #> 1    0001                         1 1355BExt     SWITCH_BACK

We can also plot an individual density profile:

    plot(dp)

When plotting density profiles, units for both axis are extracted from `$footer`, as is the measurement ID.

Loading several files is also easy, just specify the argument `dpa.directory` with a folder path. This returns a list of density profiles. By default this function works recursively and also looks into all subfolder, set the `recursive = FALSE` avoid that.

    dp.list <- dpload(dp.directory = system.file("extdata", package = "densitr"))
    #> found 15 density profiles, loading...
    #> loaded 15 density profiles

Inspect the list by displaying the first two items:

    head(dp.list, 2)
    #> $`00010001`
    #>
    #>
    #> Density profile ID:  00010001
    #>
    #> Total length:    11521 x 1/100 mm
    #> Trimmed: start not trimmed, end not trimmed
    #>
    #> Data preview:
    #>  amplitude position       ID
    #>          0        1 00010001
    #>          0        2 00010001
    #>          0        3 00010001
    #>          0        4 00010001
    #>          0        5 00010001
    #>          0        6 00010001
    #> ...
    #>
    #> $`00010002`
    #>
    #>
    #> Density profile ID:  00010002
    #>
    #> Total length:    11430 x 1/100 mm
    #> Trimmed: start not trimmed, end not trimmed
    #>
    #> Data preview:
    #>  amplitude position       ID
    #>          0        1 00010002
    #>          0        2 00010002
    #>          4        3 00010002
    #>          7        4 00010002
    #>          7        5 00010002
    #>          7        6 00010002
    #> ...

# Trimming

A typical resistance drilling measurement starts with an increase in resistance values in between the measurement start and the immersion of the needle in the wood. Similarly, if the needle exits of the opposite of the wood, there will be a decrease in values due to lack of resistance. Bark-to-bark measurements should be trimmed on both sides, while bark-to-pith only at the beginning.

## Looking under the hood

densitr provides two functions that try to automate start and end detection: `dpdetect_s` and `dpdetect_e` by using checkpoint binary segmentation. They are usually not used manually, see section on automatic trimming. Let’s find where the first profile in our list actually starts:

    dpdetect_s(dp.list[[1]])
    #> [1] 1949

The function return the horizontal position (equivalent to row number), where the start was detected. See function documentation for more information on how it works. The same function can also return a diagnostic plot.

    dpdetect_s(dp.list[[1]], return.plot = TRUE)

End detection is also possible on bark-to-bark profiles:

    dpdetect_e(dp.list[[1]])
    #> [1] 10655

Diagnostic plot can also be displayed when detecting profile end. If either start or end is not detected, it will also print a warning message.

## Automatic trimming

In order to trim an individual profile after obtaining start and end profiles, you would have to subset the profile using start and end positions. Calling `dptrim` on a density profile will automate the whole process. It will try do detect start and end positions and trim those portions away, returning a profile without the starting or ending portion of the profile.

    dptrim(dp.list[[1]])
    #>
    #>
    #> Density profile ID:  00010001
    #>
    #> Total length:    8707 x 1/100 mm
    #> Trimmed: start trimmed, end trimmed
    #>
    #> Data preview:
    #>  amplitude position       ID
    #>        429     1949 00010001
    #>        429     1950 00010001
    #>        430     1951 00010001
    #>        432     1952 00010001
    #>        433     1953 00010001
    #>        434     1954 00010001
    #> ...

`dptrim` is also capable of displaying a diagnostic plot:

    dptrim(dp.list[[1]], return.plot = TRUE)

`dptrim` can only be called on individual profiles. In order to run automatic trimming on a list of density profiles use `dptriml`. While this is just a convenience wrapper for `pbapply::pblapply` or `lapply`, it also provides a trimming report at the end:

    dp.trimmed  <- dptriml(dp.list)
    #> started trimming 15 files
    #> ########################################
    #> trimming report:
    #> analysed 15 file(s)
    #> start detection failed in: 0 file(s)
    #> end detection failed in: 6 file(s).
    #> ########################################
    #> end fail(s):
    #> 00050045, 00050046, 00050012, 00050013, 00050036, 00050038

If you have `pbapply` installed, you will get a nice progress bar displaying current progress. This is useful when trimming a large number of profiles at once. Note, you can also supply `dptriml` with an additional argument `cl`, which specifies number of cores to run trimming in parallel. This will significantly speed up the whole process. `pbapply` package is also required for this.

The equivalent functions to `dptrim` and `dptriml` to trim the profiles only at the beginning are `dptrim_s` and `dptriml_s`.

## What to do when automatic trimming fails?

Automatic trimming sometimes fails to detect the starting or ending point within the density profile. Failing to detect ending point is usually a consequence of the profile being bark-to-pith instead of bark-to-bark. In those cases (an others) you can trim the measurement by hand. See vignette on Manual profile trimming for more information.