Excerpt Lines from Files
========================
.. image:: https://travis-ci.org/fvafrCU/excerpts.svg?branch=master
    :target: https://travis-ci.org/fvafrCU/excerpts
.. image:: https://codecov.io/github/fvafrCU/excerpts/coverage.svg?branch=master
    :target: https://codecov.io/github/fvafrCU/excerpts
.. image:: https://img.shields.io/pypi/v/excerpts.svg
    :target: https://pypi.python.org/pypi/excerpts

Introduction
------------

Ever read or wrote source files containing sectioning comments?
Especially in scientific and/or data analysis scripts I see a lot like
<<>>=
### Collect Data
@
or 
<<>>=
###
### Remove Outliers 
###
@
or even
<<>>=
### 2.1 Descriptive Statistics
@

(`RStudio <https://rstudio.com>`_, an IDE for the
`programming language R <https://www.r-project.org/>`_ has
even come up with their own
`code sectioning and folding feature
<https://support.rstudio.com/hc/en-us/articles/200484568-Code-Folding-and-Sections>`_
that requires comments like
<<>>=
# Hypothesis Testing ----
@
)

If these comments are markdown style section comments, we can excerpt them and
set a table of contents.

A First Example
---------------
Suppose you have the following code:
<<echo = False>>=
file_name = 'tests/files/hello.py'
with open(file_name, 'r') as fin:
    print(fin.read())
@
We can excerpt the comments marked by '%':
<<echo = False>>=
import excerpts
with open(file_name) as infile:
    lines = infile.readlines()

excerpted = excerpts.excerpt(lines = lines, comment_character="#", 
    magic_character="%")
print (''.join(str(p) for p in excerpted))
@
which is valid 
`markdown <https://daringfireball.net/projects/markdown/>`_
that we can convert using 
`pandoc <https://www.pandoc.org>`_
.

A Bit More Elaborated
---------------------
Suppose you have a file 'tests/files/some_file.txt' reading:
<<echo = False>>=
file_name = 'tests/files/some_file.txt'
with open(file_name, 'r') as fin:
    print(fin.read())
@

Then excerpting the marked comments via
<<>>=
import excerpts
file_name = 'tests/files/some_file.txt'
with open(file_name) as infile:
    lines = infile.readlines()

excerpted = excerpts.excerpt(lines = lines, comment_character="#", 
    magic_character="%")
@

<<evaluate = False>>=
print (''.join(str(p) for p in excerpted))
@
gives
<<echo = False>>=
print (''.join(str(p) for p in excerpted))
@
which again is valid 
`markdown <https://daringfireball.net/projects/markdown/>`_
for 
`pandoc <https://www.pandoc.org>`_
.

Working with Files
~~~~~~~~~~~~~~~~~~
If you want to excerpt from a file and run pandoc on the result, you can use

<<>>=
excerpts.excerpts(file_name = file_name, comment_character="#", 
    magic_character="%", output_path="output", run_pandoc=True,
    pandoc_formats="html")
@
to generate 
`this file. <output/some_file.html>`_

Command Line Interface
......................
Excerpts has a command line interface that you may call from your
operating systems' command line instead of from python3:
<<echo = False >>=
import subprocess
print(subprocess.check_output(['excerpts', '-h']).decode('UTF-8').split(sep = 'convert')[0])
@

PEP8 
----
PEP8 requires each "line of a block comment starts with a # and a single space".
excerpts takes care of this requirement by removing a single comment character
that is followed by a space and a sequence of comments characters.
Should you need to disable this behaviour, you can set allow_pep8 to False.


Requirements
------------

Excerpts needs python3.

Installation
------------
Try 
  pip3 install git+git://github.com/fvafrcu/excerpts --upgrade --user

  
