Metadata-Version: 2.1
Name: airflow-metrics
Version: 0.1.1
Summary: Airflow plugin for automatically sending metrics from Airflow to Datadog
Home-page: https://github.com/getsentry/airflow-metrics
Author: zylphrex
Author-email: zylphrex@gmail.com
Maintainer: zylphrex
Maintainer-email: zylphrex@gmail.com
License: UNKNOWN
Keywords: airflow,datadogmetrics,plugin
Platform: MacOS
Platform: Unix
Platform: Windows
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Plugins
Classifier: License :: OSI Approved :: Apache Software License
Description-Content-Type: text/markdown
Requires-Dist: datadog (==0.29.3)
Requires-Dist: google-api-python-client (==1.7.9)
Requires-Dist: pandas-gbq (==0.10.0)

[![Build Status](https://travis-ci.com/getsentry/airflow-metrics.svg?token=TJpWxpbKGxDuV8CPPRzL&branch=master)](https://travis-ci.com/getsentry/airflow-metrics)

# airflow-metrics

`airflow-metrics` is an Airflow plugin for automatically sending metrics from Airflow to Datadog.

**Tested For**: `apache-airflow==1.10.2`

## Installation

```shell
pip install airflow-metrics
```

## Setup

`airflow-metrics` will report all metrics to Datadog, so create an `airflow` connection with your Datadog api key.

```shell
airflow connections --add --conn_id datadog_default --conn_type HTTP --conn_extr '{"api_key": "<your api key>"}'
```

**Note**: If you skip this step, your `airflow` installation should still work but no metrics will be reported.

## Usage

That's it! `airflow-metrics` will now begin sending metrics from Airflow to Datadog automatically.

## Configuration

By default, `airflow-metrics` will begin extracting metrics from Airflow as you run your DAGs and send them to overflow. You can opt out of it entirely or opt out of a subset of the metrics by setting these configurations in your `airflow.cfg`

```
[airflow_metrics]

airflow_metrics_enabled = True
airflow_metrics_tasks_enabled = True
airflow_metrics_bq_enabled = True
airflow_metrics_gcs_to_bq_enabled = True
airflow_metrics_requests_enabled = True
airflow_metrics_thread_enabled = True`
```

## Limitations

`airflow-metrics` starts a thread to report some metrics, and is not supported when using sqlite as your database.

## Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

### Getting Started

Set up your virtual environment for python3 however you like.

```shell
pip install -e .
airflow initdb
airflow connections --add --conn_id datadog_default --conn_type HTTP --conn_extr '{"api_key": ""}'
```

**Note**: The last step is necessary, otherwise the plugin will not initialize correctly and will not collect metrics. But you are free to add a dummy key for development purposes.

### Running Tests

```shell
pip install -r requirements-dev.txt
pytest
```


