---
title: "cols4all: introduction"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{cols4all: introduction}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  fig.width=6, 
  fig.height=3,
  comment = "#>"
)
hook_output <- knitr::knit_hooks$get("output")
knitr::knit_hooks$set(output = function(x, options) {
   lines <- options$output.lines
   if (is.null(lines)) {
     return(hook_output(x, options))  # pass to default hook
   }
   x <- unlist(strsplit(x, "\n"))
   more <- "..."
   if (length(lines)==1) {        # first n lines
     if (length(x) > lines) {
       # truncate the output, but add ....
       x <- c(head(x, lines), more)
     }
   } else {
     x <- c(more, x[lines], more)
   }
   # paste these lines together
   x <- paste(c(x, ""), collapse = "\n")
   hook_output(x, options)
 })

```

```{r setup, include=FALSE}
library(cols4all)
```



## Introduction

Color is essential in many charts and maps. There are a lot of color palettes to choose from: most visualization software tools have their own palettes and there are many other series of color palettes, such as ColorBrewer (Harrower and Brewer, 2003). To make life easier for R users, there are a couple of packages that contain a large collection of palettes, most notably `pals` (Wright, 2021) with 139 and `paletteer` (Hvitfeldt, 2021) with 2569 (!) palettes. However, people often cannot see the trees through the forest, so therefore they tend to stick with the color palettes they know, or with the most popular ones.

The `cols4all` package also contains a large collection of palettes (to be precise `r length(c4a_palettes())` at the time of writing), but with the central question: **which palettes are good and why?** There is no simple answer, since there are many aspects to take into account, which may have opposite effects. In `cols4all` we examine the following aspects:

* **Color Blind Friendliness** First and most importantly, is a palette suitable for colorblind people? About 1 out of 12 humans have a color vision deficiency, and we want to include them; hence the package name "colors for all".

* **HCL Analysis** Does a palette work well for statistical purposes? We analyse this with the HCL (hue-chroma-luminance) color space. Concrete questions for these analysis are the following. How **vivid** are colors? Do the colors stand out about equally (which we call **fairness**)? What hue ranges are used for quantitative color palettes?

<!--First of all, we check how vivid (chroma) the colors are: vivid colors are preferable for coloring small objects (dots) while pastel colors are preferable for large areas such as maps. Secondly, we also check whether the palette is fair, by which we mean that no colors stand out more than others. Finally, we check the hues of the palettes:-->

* **Nameability** (in development) Easy to name colors are also easy to remember. Therefore, it is beneficial to use a color palette with easy nameable colors.

* **Contrast** What is the contrast between the colors in the palette? We focus on two specific questions: 1) Is there sufficient contrast with the background color (usually white or black)? This is especially needed for text but also point- and line charts. 2) Are border lines needed to separate colors?

* **3D Blues** Does the palette contain blue colors that could result in a floating effect?

Currently, **cols4all** contains palettes from several popular and lesser known color palette series: `r paste0("\"", paste(setdiff(head(c4a_series(as.data.frame = FALSE), -1), "misc"), collapse = "\", \""), "\"")` , and `r paste0("\"", tail(c4a_series(as.data.frame = FALSE), 1), "\"")`. Stand-alone palettes that are included have been put in the series "misc" (miscellaneous). Own palettes series can be added as well.

Color palettes are organized and made consistent with each other. For instance, all sequential palettes go from light to dark. Furthermore, for each color palette a color for missing values is assigned, which is especially important for spatial data visualization. Currently we support several types: *categorical* (qualitative) palettes, *sequential* palettes, *diverging* palettes, and *bivariate* palettes (divided into four subtypes).

```{r}
#| fig.alt = "Dashboard",
#| out.width = 650,
#| echo = FALSE

knitr::include_graphics("dash-3.png")
```

The figure above shows the main tool of this package, a dashboard. The first tab page contains an overview table of all palettes (given the input selection criteria). The other tab pages are for further palette-wise analyses.

Color palettes also are important in areas other than data visualization, such as arts, web design, and fashion. The same aspects apply as well here, but obviously different criteria apply: the colors of a dress don't have to be distinguished by colorblind people, websites often contain colors that purposely stand out more than others, and there are paintings that purposely use visual illusions.


## Related R packages

The foundation of this package is another R package: [**colorspace**](https://colorspace.r-forge.r-project.org/). We use this package to analyse colors. For this purpose and specifically for color blind friendliness checks, we additionally use [**colorblindcheck**](https://github.com/Nowosad/colorblindcheck).

There are a few other pacakges with a large collection of color palettes, in particular [**pals**](https://kwstat.github.io/pals/) and [**paletteer**](https://github.com/EmilHvitfeldt/paletteer). There are a few features that distinguishes **cols4all** from those packages:

*	Color palettes are characterized and analysed. Properties such as color blindness, fairness (whether colors stand out about equally), and contrast are determined for each palette.

*	Bivariate color palettes are available (besides the three main palette types: categorical, sequential, and diverging).

*	Own color palettes can be loaded and analysed.

*	Color for missing values are made explicit.

*   Palettes are made consistent with each other to enable comparison. For instance, black and white are (by default) removed from categorical palettes. Another standard that we adapt to is that all sequential palettes go from light to dark and not the other way round.

*   There is native support for **ggplot2** and **tmap** (as of the
    upcoming version 4).
    
*	There are a couple of exporting options, including (bibtex) citation.



## Using cols4all
 
### Installation

The `cols4all` is available on CRAN and can be installed as follows

```{r, eval=FALSE}
install.packages("cols4all", dependencies = TRUE)
```

The latest development version can be installed as follows:

```{r, eval=FALSE}
remotes::install_github("mtennekes/cols4all", dependencies = TRUE)
```


### Getting started

Load the package:

```{r, eval=TRUE}
library(cols4all)
```

The main tool is a dashboard (shown above), which is started with:

```{r, eval=FALSE}
c4a_gui()
```


What types and series are available?

```{r}
c4a_types()

c4a_series()
```

How many palettes per type x series?

```{r}
c4a_overview()
```

What palettes are available, e.g diverging from the hcl series?

```{r}
# Diverging palettes from the 'hcl' series
c4a_palettes(type = "div", series = "hcl")
```

Give me the colors!

```{r}
# select purple green palette from the hcl series:
c4a("hcl.purple_green", 11)

# get the associated color for missing values
c4a_na("hcl.purple_green")
```

Plot these colors:

```{r, fig.height = 2}
c4a_plot("hcl.purple_green", 11, include.na = TRUE)
```



### Using cols4all palettes in ggplot2

```{r}
library(ggplot2)
data("diamonds")
diam_exp = diamonds[diamonds$price >= 15000, ]

# discrete categorical scale
ggplot(diam_exp, aes(x = carat, y = price, color = color)) +
	geom_point(size = 2) +
	scale_color_discrete_c4a_cat("carto.safe") +
	theme_light()

# continuous diverging scale
ggplot(diam_exp, aes(x = carat, y = depth, color = price)) +
	geom_point(size = 2) +
	scale_color_continuous_c4a_div("wes.zissou1", mid = mean(diam_exp$price)) +
	theme_light()
```		

### Overview of functions

Main functions:

*   `c4a_gui` Dashboard for analyzing the palettes
*   `c4a` Get the colors from a palette (`c4a_na` for the associated color for missing values)
*	`c4a_plot` Plot a color palette

Palette names and properties:

*   `c4a_palettes` Get available palette names
*   `c4a_series` Get available series names
*   `c4a_types` Get implemented types
* 	`c4a_overview` Get an overview of palettes per series x type.
*   `c4a_citation` Show how to cite palettes (with bibtex code).
*   `c4a_info` Get information from a palette, such as type and maximum number
    of colors
*   `.P` Environment via which palette names can be browsed with
    auto-completion (using `$`)


Importing and exporting palettes:

*   `c4a_data` Build color palette data
*   `c4a_load` Load color palette data
*   `c4a_sysdata_import` Import system data
*   `c4a_sysdata_export` Export system data

Edit color palette data

*   `c4a_duplicate` Duplicates a color palette
*   `c4a_modify` Modifies palette colors 

ggplot2

* 	`scale_<aesthetic>_<mapping>_c4a_<type>` e.g. `scale_color_continuous_c4a_div` Add scale to ggplot2. 



## Color spaces

Visual perception, in particular colors, is complex. For the purpose of information visualization it is explained in an accessible way by Ware (2019). In this vignette, we briefly describe the color spaces that are needed to understand the functionality of **cols4all**.

The human eye has three types of cones (specific photoreceptor cells in the retina), which are sensitive to light of different wavelengths: long, medium, and short. Their peak sensitivities (of 565, 540, and 440nm respectively) correspond to the primary colors red, green, and blue respectively. In computer graphics, a color is usually defined as RGB, which is a mixture of those three primary colors. RGB colors are often represented in hex-format (e.g. #FF00FF), where the first two characters specify red, the third and fourth green and the last two blue. Therefore #FF00FF is purple. A standard RGB space in computer graphics is _sRGB_ (which exactly defines the primary colors, the white point, and a so-called gamma correction which is needed to perceive a grayscale linearly).

To analyse color palettes, we also use the polarLUV color space, which is a transformation of the CIELUV space. In this space, known as _HCL_, a color is defined by three variables: 

* **Hue** The hue of a color is how we call casually name colors: e.g. yellow, purple, blue, orange, etc. It is defined as a polar coordinate, so takes values from 0 to 360 degrees.
* **Chroma** The vividness or intensity of the color. The higher, the more colorful. The maximum chroma depends on the hue. Grayscale colors have 0 chroma. The maximum value depends the hue and luminance. 
* **Luminance** The amount of light emitted from an object (e.g. a computer screen). It is similar to _brightness_, although the latter is a relative measure. It takes values from 0 to 100.







## Indicators

### Colorblind friendliness

First of all, we use the distance measure between two colors A and B from CIE as defined in 2000 (see Lindbloom, Bruce Justin. Delta E (CIE 2000).).

For simplicity, we apply two labels: colorblind friendly <font size="4">&#9786;</font> and colorblind unfriendly <font size="4">&#128064;</font>. A palette without such label is classified in between. 
Our definition of these labels depend on the palette type:


* (**categorical**) The minimum distance metric (`min_dist`) is the minimum distance between any two colors for any color vision deficiency type. 
  - <font size="4">&#9786;</font> (colorblind friendly): `min_dist >= 10`
  - <font size="4">&#128064;</font> (colorblind unfriendly): `min_dist <= 2`
  
    These numbers are parameters that can be changed via `c4a_options`.
* (**sequential**) The minimum step metric (`min_step`) is used, which is the smallest distance between any two adjacent colors (for any color vision deficiency type).
  - <font size="4">&#9786;</font> (colorblind friendly): `min_step >= 5`
  - <font size="4">&#128064;</font> (colorblind unfriendly): `min_dist <= 1`
  
    Again, these numbers are parameters that can be changed via `c4a_options`.
    Note that this metric is not very useful when a continuous scale is used. In that case, it is recommended to analyse the palette with say 7 or 9 colors.
* (**diverging**) The `min_step` metric used in a similar way as for sequential palettes. In addition, the inter wing distance metric (`inter_wing_dist`) is used, which is the minimum color distance between any color in the left wing to any color in the right wing of the palette (for any color vision deficiency type). 
  - <font size="4">&#9786;</font> (colorblind friendly): `min_step >= 5` and `inter_wing_dist >= 10`
  - <font size="4">&#128064;</font> (colorblind unfriendly): `min_dist <= 1` or `inter_wing_dist <= 4`
  
    These parameters can be changed via `c4a_options`.
* (**bivariate (seq x seq)**) Three sequential palettes are extracted: a) the first column, b) the bottom row, and c) the diagonal. These are combined to form three diverging palettes: ab, ac, and bc. For each of these diverging palette, the same checks are applied as described above for regular diverging palettes.
* (**bivariate (seq x cat)**) For each row, the same checks apply as for a categorical palette
* (**bivariate (seq x div)**) Three sequential palettes are extracted: a) the first column, b) the middle column column, and c) the last column. These are combined to form three diverging palettes: ab, ac, and bc. For each of these diverging palette, the same checks are applied as described above for regular diverging palettes.
* (**bivariate (seq x desaturated)**) A diverging palette is formed from the first and the last column. The same checks are applied as described above for regular diverging palettes.

### HCL analysis

Regarding the used **hues**, there is no simple good and bad. However, for quantitative palettes we distinguish 

* (**sequential**) 
  - Single hue palettes (&#128396;) which are recommended for quantitative analysis, but harder to distinguish colors
  - Spectral (&#127752;) which are easy to distinguish colors, but less suitable for quantitative analysis
* (**diverging**)
  - Two-hue palettes (&#x262F;) each wing has its own distinct hue, which is highly recommended
  - Spectral (&#127752;) which are easy to distinguish colors, but less suitable for quantitative analysis
 

Regarding the used **chroma** / **vividness**, we label a color palette

- **vivid** (<font size="4">&#x1f576;</font>) if it contains a color with a chroma value of 100 or higher
- **pastel** (<font size="4">&#10057;</font>) if all colors have chroma values of at most 70

These threshold numbers can be configured via options (to do).

Pastel (low chroma) colors are recommended for *space-filling visualizations*, like maps and bar charts. Vivid (high chroma) colors for *small objects*, such as dots, lines, and text labels.

We also use the **chroma** and **luminance** values of palette colors to find out whether some colors stand out more than other colors. Ideally, a categorical palette should contain colors that stand out about equally (for otherwise, one color will draw more attention than another, which may bias our perception and interpretation of the shown data). Colors with a high chroma value stand out more than less chromatic colors. Furthermore, against a bright background, dark colors (low luminance) stand out more, while bright colors (high luminance) stand out more against a dark background. Luminance is only considered for categorical palettes, because quantitative palettes (sequential and diverging) often map a numeric variable to luminance.

Let us define the luminance range (`Lrange`) as the maximum minus the minimum luminance value of the palette and the chroma range (`Crange`) as the maximum chroma value minus the minimum chroma value.

Regarding **Fairness** we consider two metrics: the luminance range (`Lrange`) and the chroma range (`Crange`). We call a palette:

- **fair** (<font size="4">&#9825;</font>) if `Crange <= 50` and (for categorical palettes only) `Lrange <= 30`
- **unfair** (<font size="4">&#10799;</font>) if `Crange <= 80` or (for categorical palettes only) `Lrange >= 50`

Note that harmonic color palettes are usually not color blind friendly. Furthermore, when the luminance values of the colors are about equal, the contrast of those colors is low, which requires the use of border lines (see below).

### Naming

This property is still in development. Details will follow.

### Contrast

The border between two colored shapes appears **wobbly** when the colors are equally luminant (bright), no matter what hue (red, blue, etc.) they have. This visual illusion is called **equiluminance**.

The **contrast ratio** is a measure for equiluminance, calculated as `CR = (L1 + 0.05) / (L2 + 0.05)`, where `L1` and `L2` are the luminances (normalized between 0 and 1) of the lighter and darker colors, respectively. Note that the minimum contrast ratio is 1 and the maximum 21.

In the overview table, there are three flags that indicate low contrast (`CR < 1.2`):

- <font size="1">&#127937;</font>: Between two colors of the palettes
- <font size="1">&#127987;</font>: Between one color of the palette and white
- <font size="1">&#127988;</font>: Between one color of the palette and black

The go-to solution to prevent wobbly borders is by using black or white (depending of the lightness of the colors) **border lines**.




### 3D Blues

Pure blue colors, by which we mean a 0 or very low number for the R (red) and G (green) channels and a high number for the B (blue) channel may cause a visual illusion called **chromostereopsis**. This effect is especially prominent when blue objects are shown next to red objects against a black background.

The table column "3D Blues" indicates when a palette contains a pure blue color. The used symbol is B7#9 (which refers to a music chord used in blues music). 


## References

Harrower M, Brewer CA (2003). “ColorBrewer.org: An Online Tool for Selecting Colour Schemes for Maps.” _The Cartographic Journal_,
*40*(1), 27-37.

Hvitfeldt E. (2021). paletteer: Comprehensive Collection of Color Palettes. version 1.3.0. https://github.com/EmilHvitfeldt/paletteer

Ware, C (2019). Information Visualization - Perception for Design - 4th Edition. Elsevier

Wright K (2021). _pals: Color Palettes, Colormaps, and Tools to Evaluate Them_. R package version 1.7,
  <https://CRAN.R-project.org/package=pals>.
  
  
