RcppCWB/docs/articles/vignette.html at RcppDeepState · RcppDeepState/RcppCWB

History

323 lines (295 loc) · 28.2 KB

Raw

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

<!DOCTYPE html>

<html lang="en">

<head>

<title>Writing performance code with RcppCWB • RcppCWB</title>

<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.4.1/jquery.min.js" integrity="sha256-CSXorXvZcTkaix6Yvo6HppcZGetbYMGWSFlBw8HfCJo=" crossorigin="anonymous"></script><link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.4.1/css/bootstrap.min.css" integrity="sha256-bZLfwXAP04zRMK2BjiO8iu9pf4FbLqX6zitd+tIvLhE=" crossorigin="anonymous">

<script src="https://cdnjs.cloudflare.com/ajax/libs/clipboard.js/2.0.6/clipboard.min.js" integrity="sha256-inc5kl9MA1hkeYUt+EC3BhlIgyp/2jDIyBLS6k3UxPI=" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/headroom/0.11.0/headroom.min.js" integrity="sha256-AsUX4SJE1+yuDu5+mAVzJbuYNPHj/WroHuZ8Ir/CkE0=" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/headroom/0.11.0/jQuery.headroom.min.js" integrity="sha256-ZX/yNShbjqsohH1k95liqY9Gd8uOiE1S4vZc+9KQ1K4=" crossorigin="anonymous"></script><link href="../pkgdown.css" rel="stylesheet">

<script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js" integrity="sha256-nvJJv9wWKEm88qvoQl9ekL2J+k/RWIsaSScxxlsrv8k=" crossorigin="anonymous"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/config/TeX-AMS-MML_HTMLorMML.js" integrity="sha256-84DKXVJXs0/F8OTMzX4UR909+jtl4G7SPypPavF+GfA=" crossorigin="anonymous"></script><!--[if lt IE 9]>

<![endif]-->

</head>

Toggle navigation

</button>

<a class="navbar-link" href="../index.html">RcppCWB</a>

0.6.0

</div>

<li>

<a href="../reference/index.html">Reference</a>

</li>

Articles

</a>

<li>

<a href="../articles/vignette.html">Writing performance code with RcppCWB</a>

</li>

</ul>

</li>

<li>

<a href="../news/index.html">Changelog</a>

</li>

</ul>

<li>

</a>

</li>

</ul>

</div>

</div>

</div>

</header><div class="row">

<h1 data-toc-skip>Writing performance code with RcppCWB</h1>

<h4 data-toc-skip class="author">Andreas Blaette

(<a href="mailto:[email protected]" class="email">[email protected]</a>)</h4>

Source: <a href="https://github.com/PolMine/RcppCWB/blob/HEAD/vignettes/vignette.Rmd" class="external-link"><code>vignettes/vignette.Rmd</code></a>

<div class="hidden name"><code>vignette.Rmd</code></div>

</div>

<h2 id="rationale">Rationale<a class="anchor" aria-label="anchor" href="#rationale"></a>

</h2>

The RcppCWB package exposes the functionality of the Corpus Workbench

(CWB) to R, so that R users can benefit from the performance of the C

code of the CWB. Ease of use and performance should be great most of the

time. But there are scenarios when the interface between R and C/C++ is

a bottleneck for achieving sufficient performance. In this case, using

CWB functionality in C++ functions exposed to R using

<code><a href="https://rdrr.io/pkg/Rcpp/man/cppFunction.html" class="external-link">Rcpp::cppFunction()</a></code> or <code><a href="https://rdrr.io/pkg/Rcpp/man/sourceCpp.html" class="external-link">Rcpp::sourceCpp()</a></code> may

solve issues with performance and memory limitations.

</div>

<h2 id="basics">Basics<a class="anchor" aria-label="anchor" href="#basics"></a>

</h2>

Writing C++ functions that use CWB functionality requires loading the

Rcpp and RcppCWB package.

<code class="sourceCode R"><a href="https://rdrr.io/r/base/library.html" class="external-link">library</a>(<a href="https://www.rcpp.org" class="external-link">Rcpp</a>)

<a href="https://rdrr.io/r/base/library.html" class="external-link">library</a>(<a href="https://github.com/PolMine/RcppCWB" class="external-link">RcppCWB</a>)</code></pre></div>

We need to be aware that the default functions for accessing the CWB

functionality involve passing length-one character vectors used for

looking up the C representation of structural or positional attributes

for corpora that have been loaded. It is more efficient to perform this

lookup only once. Following this rationale, a set of functions exposes

CWB functionality closer to the C logic, passing pointers to attributes

that have been looked up.

This functionality can also be used from R. For instance, we look up

the p-attribute “word” of the “REUTERS” corpus as follows.

<code class="sourceCode R">p_attr_word <- <a href="../reference/cl_rework.html">p_attr</a>(

corpus = "REUTERS",

p_attribute = "word",

registry = <a href="../reference/tmp_registry.html">get_tmp_registry</a>()

)</code></pre></div>

And we use <code><a href="../reference/cl_rework.html">cpos_to_str()</a></code> to decode the first words of

the corpus.

<pre><code>## [1] "Diamond" "Shamrock" "Corp" "said" "that" "effective"

## [7] "today" "it" "had" "cut" "its"</code></pre>

While this may also be useful when writing R code, this lower-level

functionality is particularly well-suited for writing high-performance

C++ code exposed to R.

</div>

<h2 id="inline-c-functions">Inline C++ functions<a class="anchor" aria-label="anchor" href="#inline-c-functions"></a>

</h2>

We start with a first simple scenario, which uses

<code><a href="https://rdrr.io/pkg/Rcpp/man/cppFunction.html" class="external-link">cppFunction()</a></code> to source an inline C++ function in an R

session.

<code class="sourceCode R"><a href="https://rdrr.io/pkg/Rcpp/man/cppFunction.html" class="external-link">cppFunction</a>(

'Rcpp::StringVector get_str(SEXP corpus, SEXP p_attribute, SEXP registry, Rcpp::IntegerVector cpos){

SEXP attr;

Rcpp::StringVector result;

attr = RcppCWB::p_attr(corpus, p_attribute, registry);

result = RcppCWB::cpos_to_str(attr, cpos);

return(result);

}',

depends = "RcppCWB"

)</code></pre></div>

This is not a very interesting example, but using the function

works:

<code class="sourceCode R">get_str("REUTERS", "word", RcppCWB::<a href="../reference/tmp_registry.html">get_tmp_registry</a>(), 0:50)</code></pre></div>

<pre><code>## [1] "Diamond" "Shamrock" "Corp" "said" "that"

## [6] "effective" "today" "it" "had" "cut"

## [11] "its" "contract" "prices" "for" "crude"

## [16] "oil" "by" "1.50" "dlrs" "a"

## [21] "barrel" "The" "reduction" "brings" "its"

## [26] "posted" "price" "for" "West" "Texas"

## [31] "Intermediate" "to" "16.00" "dlrs" "a"

## [36] "barrel" "the" "copany" "said" "The"

## [41] "price" "reduction" "today" "was" "made"

## [46] "in" "the" "light" "of" "falling"

## [51] "oil"</code></pre>

</div>

<h2 id="source-c-function">Source C++ function<a class="anchor" aria-label="anchor" href="#source-c-function"></a>

</h2>

To provide a more interesting real-life example, we demonstrate a

solution to the following scenario: It may be necessary to decode an

entire corpus, and to write the tokens of corpus regions to a file in a

line-by-line manner. Computing word embeddings may require this input

format, for instance.

But if the corpus is really large, decoding the corpus entirely and

then writing everything to disk may hit memory limitations. Decoding the

tokens of the corpus successively and writing content to the output file

on the spot is an obvious solution, but moving data between the R/C++/C

interface for every single token is excessively slow. A pure C++

implementation will be much more effective.

The following C++ file that relies on CWB functions as exposed by

RcppCWB addresses the scenario.

<div class="sourceCode" id="cb8"><pre class="sourceCode cpp"><code class="sourceCode cpp"><a href="#cb8-1" aria-hidden="true" tabindex="-1"></a>// [[Rcpp::depends(RcppCWB)]]

<a href="#cb8-2" aria-hidden="true" tabindex="-1"></a>#include <Rcpp.h>

<a href="#cb8-3" aria-hidden="true" tabindex="-1"></a>#include <RcppCWB.h>

<a href="#cb8-5" aria-hidden="true" tabindex="-1"></a>#include <stdio.h>

<a href="#cb8-6" aria-hidden="true" tabindex="-1"></a>#include <iostream>

<a href="#cb8-7" aria-hidden="true" tabindex="-1"></a>#include <fstream>

<a href="#cb8-8" aria-hidden="true" tabindex="-1"></a>#include <cstdlib>

<a href="#cb8-11" aria-hidden="true" tabindex="-1"></a>// [[Rcpp::export]]

<a href="#cb8-12" aria-hidden="true" tabindex="-1"></a>int write_token_stream(SEXP corpus, SEXP p_attribute, SEXP s_attribute, SEXP registry, SEXP attribute_type, Rcpp::StringVector filename) {

<a href="#cb8-14" aria-hidden="true" tabindex="-1"></a> int i, n, region_size;

<a href="#cb8-15" aria-hidden="true" tabindex="-1"></a> Rcpp::IntegerVector region(2);

<a href="#cb8-16" aria-hidden="true" tabindex="-1"></a> std::ofstream outdata;

<a href="#cb8-18" aria-hidden="true" tabindex="-1"></a> n = RcppCWB::attribute_size(corpus, s_attribute, attribute_type, registry);

<a href="#cb8-20" aria-hidden="true" tabindex="-1"></a> outdata.open(filename[0]);

<a href="#cb8-21" aria-hidden="true" tabindex="-1"></a> if( !outdata ) {

<a href="#cb8-22" aria-hidden="true" tabindex="-1"></a> std::cerr << "Error: file could not be opened" << std::endl;

<a href="#cb8-23" aria-hidden="true" tabindex="-1"></a> exit(1);

<a href="#cb8-24" aria-hidden="true" tabindex="-1"></a> }

<a href="#cb8-26" aria-hidden="true" tabindex="-1"></a> for (i = 0; i < n; i++){

<a href="#cb8-27" aria-hidden="true" tabindex="-1"></a> region = RcppCWB::struc2cpos(corpus, s_attribute, registry, i);

<a href="#cb8-28" aria-hidden="true" tabindex="-1"></a> region_size = region[1] - region[0] + 1;

<a href="#cb8-30" aria-hidden="true" tabindex="-1"></a> Rcpp::IntegerVector cpos(region_size);

<a href="#cb8-31" aria-hidden="true" tabindex="-1"></a> cpos = Rcpp::seq(region[0], region[1]);

<a href="#cb8-33" aria-hidden="true" tabindex="-1"></a> Rcpp::StringVector values(region_size);

<a href="#cb8-34" aria-hidden="true" tabindex="-1"></a> values = RcppCWB::cpos2str(corpus, p_attribute, registry, cpos);

<a href="#cb8-36" aria-hidden="true" tabindex="-1"></a> int j;

<a href="#cb8-37" aria-hidden="true" tabindex="-1"></a> for (j = 0; j < values.length(); j++){

<a href="#cb8-38" aria-hidden="true" tabindex="-1"></a> outdata << values(j);

<a href="#cb8-39" aria-hidden="true" tabindex="-1"></a> if (j < values.length() - 1){

<a href="#cb8-40" aria-hidden="true" tabindex="-1"></a> outdata << " ";

<a href="#cb8-41" aria-hidden="true" tabindex="-1"></a> }

<a href="#cb8-42" aria-hidden="true" tabindex="-1"></a> }

<a href="#cb8-43" aria-hidden="true" tabindex="-1"></a> outdata << std::endl;

<a href="#cb8-44" aria-hidden="true" tabindex="-1"></a> }

<a href="#cb8-45" aria-hidden="true" tabindex="-1"></a> outdata.close();

<a href="#cb8-47" aria-hidden="true" tabindex="-1"></a> return 0;

This code can be sourced, compiled and exposed to R using

<code><a href="https://rdrr.io/pkg/Rcpp/man/sourceCpp.html" class="external-link">sourceCpp()</a></code>.

<code class="sourceCode R"><a href="https://rdrr.io/pkg/Rcpp/man/sourceCpp.html" class="external-link">sourceCpp</a>(file = <a href="https://rdrr.io/r/base/system.file.html" class="external-link">system.file</a>(package = "RcppCWB", "cpp", "fastdecode.cpp"))</code></pre></div>

We exemplify that everything works as intended using the (smallish)

REUTERS corpus. So we create the output …

<code class="sourceCode R">outfile <- <a href="https://rdrr.io/r/base/tempfile.html" class="external-link">tempfile</a>(fileext = ".txt")

write_token_stream(

corpus = "REUTERS",

p_attribute = "word",

s_attribute = "id",

attribute_type = "s",

registry = RcppCWB::<a href="../reference/tmp_registry.html">get_tmp_registry</a>(),

filename = outfile

)</code></pre></div>

… and read it (showing the content selectively) to convey that the

corpus data has been exported as intended.

<code class="sourceCode R"><a href="https://rdrr.io/r/base/readLines.html" class="external-link">readLines</a>(outfile) |>

<a href="https://rdrr.io/r/base/lapply.html" class="external-link">lapply</a>(substr, 1, 75) |>

<a href="https://rdrr.io/r/base/unlist.html" class="external-link">unlist</a>()</code></pre></div>

<pre><code>## [1] "Diamond Shamrock Corp said that effective today it had cut its contract pri"

## [2] "OPEC may be forced to meet before a scheduled June session to readdress its"

## [3] "Texaco Canada said it lowered the contract price it will pay for crude oil "

## [4] "Marathon Petroleum Co said it reduced the contract price it will pay for al"

## [5] "Houston Oil Trust said that independent petroleum engineers completed an an"

## [6] "Kuwait s Oil Minister in remarks published today said there were no plans f"

## [7] "Indonesia appears to be nearing a political crossroads over measures to der"

## [8] "Saudi riyal interbank deposits were steady at yesterday's higher levels in "

## [9] "The Gulf oil state of Qatar recovering slightly from last year's decline in"

## [10] "Saudi Arabian Oil Minister Hisham Nazer reiterated the kingdom's commitment"

## [11] "Saudi crude oil output last month fell to an average of 3.5 mln barrels per"

## [12] "Deputy oil ministers from six Gulf Arab states will meet in Bahrain today t"

## [13] "Saudi Arabian Oil Minister Hisham Nazer reiterated the kingdom's commitment"

## [14] "Kuwait's oil minister said in a newspaper interview that there were no plan"

## [15] "The port of Philadelphia was closed when a Cypriot oil tanker Seapride II r"

## [16] "A study group said the United States should increase its strategic petroleu"

## [17] "A study group said the United States should increase its strategic petroleu"

## [18] "Unocal Corp's Union Oil Co said it lowered its posted prices for crude oil "

## [19] "The New York Mercantile Exchange set April one for the debut of a new proce"

## [20] "Argentine crude oil production was down 10.8 pct in January 1987 to 12.32 m"</code></pre>

</div>

<h2 id="moving-ahead">Moving ahead<a class="anchor" aria-label="anchor" href="#moving-ahead"></a>

</h2>

Writing C++ functions is obviously more demanding than writing R

code. But using CWB functionality as exposed by RcppCWB in C++ functions

that can be used from R may be a great solution to performance and

memory issues. Rcpp brings writing C++ code much closer to what R users

are acquainted with, making writing high-performance C++ close much

easier. So we encourage considering this option when pure R solutions

are not fast enough.

</div>

<nav id="toc" data-toggle="toc"><h2 data-toc-skip>Contents</h2>

</nav>

</div>

Developed by Andreas Blaette, Bernard Desgraupes, Sylvain Loiseau.

</div>

Site built with <a href="https://pkgdown.r-lib.org/" class="external-link">pkgdown</a> 2.0.6.

</div>

</footer>

</div>

</body>

</html>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FilesExpand file tree

vignette.html

Latest commit

History

vignette.html

File metadata and controls