% Generated by roxygen2: do not edit by hand

% Please edit documentation in R/RcppCWB_package.R

\docType{package}

\name{RcppCWB-package}

\alias{RcppCWB-package}

\alias{RcppCWB}

\title{Rcpp Bindings for the Corpus Workbench (CWB).}

\description{

The \code{RcppCWB} package is a wrapper library to expose core functions of

the \code{Open Corpus Workbench} (CWB). This includes the low-level

functionality of the \code{Corpus Library} (CL) as well as capacities to use

the query syntax of the \code{Corpus Query Processor} (CQP).

}

\section{The Idea Behind RcppCWB}{

The \code{Open Corpus Workbench} (CWB) is an indexing and querying engine

popular in corpus-assisted research. Its core aim is to support working

efficiently with large, structurally and linguistically annotated corpora.

First of all, the CWB includes tools to index and compress corpora. Second,

the \code{Corpus Library} (CL) offers low-level functionality to retrieve

information from CWB indexed corpora. Third, the \code{Corpus Query

Processor} (CQP) offers a syntax that allows to perform anything from

simple to complex queries, using different annotation layers of corpora.

The CWB is a classical tool which has inspired a set of developments. A

persisting advantage of the CWB is its mature, open source code base that

is actively maintained by a community of developers. It is used as a robust

and efficient backend for widely used tools such as

TXM(\url{https://txm.gitpages.huma-num.fr/textometrie/}) or CQPweb

(\url{https://cwb.sourceforge.io/cqpweb.php}). Its uncompromising C

implementation guarantees speed and makes it well suited to be integrated

with R at the same time.

The package \code{RcppCWB} is a follow-up on the \code{rcqp} package that

has pioneered to expose CWB functionality from within R. Indeed, the

\code{rcqp} package, published at CRAN in 2015, offers robust access to CWB

functionality. However, the "pure C" implementation of the \code{rcqp}

package creates difficulties to make the package portable to Windows. The

primary purpose of the \code{RcppCWB} package is to reimplement a wrapper

library for the CWB using a design that makes it easier to achieve

cross-platform portability.

Even though \code{RcppCWB} functions may be used directly, the package is

designed to serve as an interface to CWB indexed corpora in packages with

higher-level functionality. In this regard, \code{RcppCWB} is the backend

of the \code{polmineR} package. It is deliberately open to be used in other

contexts. The package may stimulate using linguistically annotated, indexed

and compressed corpora on all platforms. The paradigm of working with text

as linguistic data may benefit from \code{RcppCWB}.

}

\section{Implementation}{

When building the package, the first step is to compile the relevant parts

of the CWB on Linux and macOS machines. On Windows, cross-compiled binaries

are downloaded from a GitHub repository of the PolMine Project

(\url{https://github.com/PolMine/libcl}). Second, \code{Rcpp} wrappers are

compiled and make the relevant functions of the Corpus Library and CQP

accessible. In addition to genuine CWB functions, \code{RcppCWB} offers a

set of higher level functions implemented using \code{Rcpp} for common

performance critical tasks.

}

\section{Getting Started with RcppCWB}{

To understand the data storage model of the CWB, in particular the notions

of positional and structural attributes (s- and p-attributes), the vignette

of the \code{rcqp} package is a very good starting point (see references).

The CWB 'Corpus Encoding Tutorial' explains how to create your own corpus,

the 'CQP Query Language Tutorial' introduces the syntax of CQP (see

references).

The \code{RcppCWB} package includes a sample corpus (REUTERS, the data also

included in the \code{tm} package). The examples in the documentation

of the functions may be a good starting point to understand how to use

\code{RcppCWB}.

}

\section{Digging Deeper}{

The original paper of Christ (1994) explains the design choices of the CWB.

The indexing and compression techniques of the CWB (Huffman coding) are

explained in Witten et al. (1999).

}

\section{Acknowledgements}{

The work of the all developers of the CWB is gratefully acknowledged. There

is a particular intellectual debt to Bernard Desgraupes and Sylvain

Loiseau, and the \code{rcqp} package they developed as the original R

wrapper to expose the functionality of the CWB.

}

\examples{

ids <- cl_cpos2id("REUTERS", cpos = 1:20, p_attribute = "word", registry = get_tmp_registry())

tokens <- cl_id2str("REUTERS", id = ids, p_attribute = "word", registry = get_tmp_registry())

print(paste(tokens, collapse = " "))

cqp_query("REUTERS", query = '[]{5} "oil" []{5}')

cpos_matrix <- cqp_dump_subcorpus("REUTERS")

concordances_oil <- apply(

cpos_matrix, 1,

function(row){

ids <- cl_cpos2id("REUTERS", p_attribute = "word", cpos = row[1]:row[2], get_tmp_registry())

tokens <- cl_id2str("REUTERS", p_attribute = "word", id = ids, get_tmp_registry())

paste(tokens, collapse = " ")

}

)

}

\references{

Christ, O. 1994. "A modular and flexible architecture for an integrated

corpus query system", in: Proceedings of COMPLEX '94, pp. 23-32. Budapest.