Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions biojava-tornadovm/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
data/
.tornado/
96 changes: 96 additions & 0 deletions biojava-tornadovm/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
# biojava-tornadovm

GPU paths for two BioJava hot spots, with [TornadoVM](https://github.com/beehive-lab/TornadoVM). The
classes are drop-in replacements and return **exactly** the same results as the BioJava code they
replace:

| BioJava | Drop-in replacement | Result |
|---|---|---|
| `org.biojava.nbio.structure.asa.AsaCalculator` | `org.biojava.nbio.tornadovm.asa.TornadoAsaCalculator` | bit-identical per-atom and per-residue ASA |
| `Alignments.getAllPairsScores` / `getAllPairsScorers` (`GLOBAL`, `LOCAL`) | `org.biojava.nbio.tornadovm.align.TornadoAlignments` | identical scores and scorers |

Without a TornadoVM runtime, below a size threshold, or after a GPU failure, the classes run an exact
multi-threaded CPU path instead.

This module needs **JDK 21** and is not part of the default build: it builds only with the `tornadovm`
profile, so the rest of BioJava keeps its JDK 11 baseline.

## Run it

Linux with an NVIDIA GPU, using [SDKMAN](https://sdkman.io). For AMD or Intel GPUs, install
`tornadovm 8.0.0-jdk21-opencl` and set `TORNADO_BACKEND=opencl`.

```bash
# 1. JDK 21 and TornadoVM 8.0.0
curl -s "https://get.sdkman.io" | bash && source ~/.sdkman/bin/sdkman-init.sh # if you have no SDKMAN yet
sdk install java 21.0.2-open
sdk install tornadovm 8.0.0-jdk21-cuda # answering "n" to "set as default" is fine
export JAVA_HOME=$HOME/.sdkman/candidates/java/21.0.2-open

# 2. Build BioJava with this module
git clone https://github.com/biojava/biojava.git && cd biojava
mvn -Ptornadovm -pl biojava-tornadovm -am install -DskipTests

# 3. The tests on the GPU, then the benchmarks
cd biojava-tornadovm
scripts/test-gpu.sh # JUnit under TornadoVM: GPU results == BioJava results (23 tests)
./bench.sh # parity tests + ASA and all-pairs benchmarks (downloads 5 PDB entries)
./bench.sh asa # or one benchmark: asa | sw
```

The scripts find `tornadovm 8.0.0-jdk21-<backend>` in SDKMAN by its version, whatever your default
is. If it is missing, they install it with `sdk install`, without making it the default. Without
SDKMAN, they download the SDK from the
[TornadoVM release](https://github.com/beehive-lab/TornadoVM/releases/tag/v8.0.0) into `.tornado/`.
`TORNADO_SDK=/path/to/sdk` uses a given SDK.

`mvn -Ptornadovm -pl biojava-tornadovm test` runs the same tests on a plain JVM, which exercises the
CPU paths.

## Use it in an application

```java
// before: new AsaCalculator(structure, AsaCalculator.DEFAULT_PROBE_SIZE, 1000, nThreads, false)
TornadoAsaCalculator asa = new TornadoAsaCalculator(structure, AsaCalculator.DEFAULT_PROBE_SIZE,
AsaCalculator.DEFAULT_N_SPHERE_POINTS, nThreads, false);
double[] perAtom = asa.calculateAsas();

// before: Alignments.getAllPairsScores(seqs, PairwiseSequenceScorerType.GLOBAL, gaps, matrix)
double[] scores = TornadoAlignments.getAllPairsScores(seqs, PairwiseSequenceScorerType.GLOBAL, gaps, matrix);
```

Start the application on JDK 21 with the TornadoVM SDK's JVM options. Without them, it runs on the
CPU paths:

```bash
export TORNADOVM_HOME=$HOME/.sdkman/candidates/tornadovm/8.0.0-jdk21-cuda
sed "s|\${TORNADOVM_HOME}|$TORNADOVM_HOME|g" $TORNADOVM_HOME/tornado-argfile.template > tornado.args
$JAVA_HOME/bin/java @tornado.args -cp my-app.jar:<dependencies> com.example.Main
```

`tornado-api` is a `provided` dependency: at run time it comes from the SDK, so do not bundle it.

- `-Dbiojava.tornado=off` forces the CPU paths; `-Dbiojava.tornado=force` ignores the size thresholds
(GPU from 1,000 atoms for ASA, and from 1e7 DP cells in total for alignment).
- `-Dasa.fused.device=<driver>:<device>` and `-Dsw.scores.device=<driver>:<device>` choose the device;
`tornado --devices` lists them.
- GPUs without FP64 (e.g. Intel iGPUs) are detected. ASA then uses a single-precision kernel and
recomputes near-boundary cases exactly on the CPU, so results stay identical.
- `*_IDENTITIES`, `*_SIMILARITIES`, `KMERS` and `WU_MANBER` scorers are delegated to BioJava.

## Results

TornadoVM 8.0.0, CUDA, RTX 4090; i9-13900K with 32 threads. End to end, including transfers. All
results are identical to BioJava's. "Lean Java" is the GPU kernel's algorithm in plain Java on all
cores, which is also this module's CPU path.

| Workload | BioJava | lean Java (32 thr) | GPU | vs BioJava | vs lean Java |
|---|---:|---:|---:|---:|---:|
| ASA 4V6X, 238k atoms | 558 ms (32 thr) | 109 ms | 35.7 ms | 15.6x | 3.0x |
| ASA 3J3Q, 2.4M atoms | 6.92 s (32 thr) | 1.16 s | 373 ms | 18.5x | 3.1x |
| All-pairs global, 200 proteins, 20k pairs | 9.97 s | 417 ms | 17.0 ms | 586x | 24.5x |
| All-pairs local, 200 proteins, 20k pairs | 10.5 s | 293 ms | 17.0 ms | 615x | 17.2x |
| All-pairs global, 1,000 proteins, 500k pairs | (skipped) | 15.0 s | 265 ms | | 56.6x |

Tested on an RTX 4090 with the CUDA and OpenCL backends, and on an Intel UHD 770 with OpenCL. The 23
GPU parity tests pass on all three.
29 changes: 29 additions & 0 deletions biojava-tornadovm/bench.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
#!/usr/bin/env bash
# One command: fetch TornadoVM (if needed) and the test data, build, run the parity tests on the GPU and the benchmarks.
# ./bench.sh everything (~10 min, most of it BioJava's own CPU baselines)
# ./bench.sh asa|sw one benchmark
set -euo pipefail
cd "$(dirname "$0")"
source scripts/env.sh
tornado --devices | grep -E "device=|--" | head -4

mkdir -p data/pdb
for id in 1smt 4hhb 1aon 4v6x 3j3q; do
[[ -s data/pdb/$id.cif.gz ]] || curl -sfL -o data/pdb/$id.cif.gz "https://files.rcsb.org/download/$id.cif.gz"
done
cp -n src/test/resources/PF00104_small.fasta data/ 2>/dev/null || true
(cd .. && mvn -q -B -Ptornadovm -pl biojava-tornadovm -am -DskipTests install)

what="${1:-all}"
if [[ "$what" == all ]]; then
echo "== parity tests on the GPU"; scripts/test-gpu.sh 2>/dev/null | grep -E "tests (successful|failed)"
fi
if [[ "$what" == all || "$what" == asa ]]; then
echo "== ASA (Shrake-Rupley, 1000 sphere points)"
scripts/run.sh org.biojava.nbio.tornadovm.bench.AsaBench 1SMT 4HHB 1AON 4V6X 3J3Q 2>/dev/null | grep -vE "cold|DEBUG|INFO|WARN"
fi
if [[ "$what" == all || "$what" == sw ]]; then
echo "== All-pairs alignment scores (BLOSUM62, gaps 10/1)"
scripts/run.sh org.biojava.nbio.tornadovm.bench.SwBench random:40:100:300 random:200:100:500 2>/dev/null | grep -vE "DEBUG|INFO|WARN"
scripts/run.sh -Dbench.skipcpu=true org.biojava.nbio.tornadovm.bench.SwBench fasta:data/PF00104_small.fasta random:1000:200:500 2>/dev/null | grep -vE "DEBUG|INFO|WARN|threads in"
fi
103 changes: 103 additions & 0 deletions biojava-tornadovm/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<artifactId>biojava</artifactId>
<groupId>org.biojava</groupId>
<version>7.3.1-SNAPSHOT</version>
</parent>
<artifactId>biojava-tornadovm</artifactId>
<name>biojava-tornadovm</name>
<description>GPU paths for BioJava hot spots (ASA, all-pairs alignment scores) with TornadoVM. Needs JDK 21; built with -Ptornadovm.</description>

<licenses>
<license>
<name>GNU LGPL v2</name>
<url>http://www.gnu.org/licenses/old-licenses/lgpl-2.1.txt</url>
<distribution>repo</distribution>
</license>
</licenses>

<url>http://www.biojava.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<tornadovm.version>8.0.0-jdk21</tornadovm.version>
</properties>
<build>
<plugins>
<!-- TornadoVM 8.0.0 for JDK 21 uses the foreign-memory API, a preview feature in JDK 21 -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<release>21</release>
<compilerArgs>
<arg>--enable-preview</arg>
</compilerArgs>
</configuration>
</plugin>
<!-- Plain JVM: the tests exercise the exact CPU paths; scripts/test-gpu.sh runs them on a GPU -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>-Xmx1500M --enable-preview</argLine>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<excludes>
<exclude>demo/**</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.biojava</groupId>
<artifactId>biojava-structure</artifactId>
<version>7.3.1-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.biojava</groupId>
<artifactId>biojava-alignment</artifactId>
<version>7.3.1-SNAPSHOT</version>
</dependency>
<!-- Compile-time API only: at run time the classes come from the TornadoVM SDK (its argument file). -->
<dependency>
<groupId>io.github.beehive-lab</groupId>
<artifactId>tornado-api</artifactId>
<version>${tornadovm.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-params</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j2-impl</artifactId>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
</dependency>
</dependencies>
</project>
50 changes: 50 additions & 0 deletions biojava-tornadovm/scripts/env.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Source me. Selects JDK 21 and a TornadoVM 8.0.0 SDK for JDK 21.
# JAVA_HOME a JDK 21 (default: $JAVA_HOME if it is 21, else SDKMAN's 21.0.2-open)
# TORNADO_SDK an existing TornadoVM 8.0.0 JDK 21 SDK (optional)
# TORNADO_BACKEND cuda (default) or opencl
# Without TORNADO_SDK the SDK comes from SDKMAN (tornadovm 8.0.0-jdk21-<backend>, installed if missing),
# or, without SDKMAN, from the GitHub release, downloaded into .tornado/.
TORNADO_VERSION=8.0.0
_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"

if [[ -z "${JAVA_HOME:-}" ]] || ! "$JAVA_HOME/bin/java" -version 2>&1 | grep -q '"21\.'; then
JAVA_HOME="$HOME/.sdkman/candidates/java/21.0.2-open"
fi
if ! "$JAVA_HOME/bin/java" -version 2>&1 | grep -q '"21\.'; then
echo "error: TornadoVM $TORNADO_VERSION (jdk21 build) needs JDK 21: sdk install java 21.0.2-open, or set JAVA_HOME" >&2
return 1 2>/dev/null || exit 1
fi
export JAVA_HOME

_backend="${TORNADO_BACKEND:-cuda}"
_sdkman="${SDKMAN_DIR:-$HOME/.sdkman}"
if [[ -z "${TORNADO_SDK:-}" && -s "$_sdkman/bin/sdkman-init.sh" ]]; then
_candidate="$_sdkman/candidates/tornadovm/$TORNADO_VERSION-jdk21-$_backend"
if [[ ! -x "$_candidate/bin/tornado" ]]; then
echo "Installing TornadoVM $TORNADO_VERSION-jdk21-$_backend with SDKMAN ..." >&2
# shellcheck disable=SC1091
source "$_sdkman/bin/sdkman-init.sh"
# "n": do not make it the default TornadoVM (that would change TORNADOVM_HOME in every shell)
printf "n\n" | sdk install tornadovm "$TORNADO_VERSION-jdk21-$_backend" >&2
fi
[[ -x "$_candidate/bin/tornado" ]] && TORNADO_SDK="$_candidate"
fi
if [[ -z "${TORNADO_SDK:-}" ]]; then
_name="tornadovm-$TORNADO_VERSION-jdk21-$_backend-linux-amd64"
TORNADO_SDK="$_root/.tornado/tornadovm-$TORNADO_VERSION-$_backend"
if [[ ! -x "$TORNADO_SDK/bin/tornado" ]]; then
echo "Downloading $_name ..." >&2
mkdir -p "$_root/.tornado"
curl -fL --progress-bar -o "$_root/.tornado/$_name.tar.gz" \
"https://github.com/beehive-lab/TornadoVM/releases/download/v$TORNADO_VERSION/$_name.tar.gz"
tar -xzf "$_root/.tornado/$_name.tar.gz" -C "$_root/.tornado"
rm "$_root/.tornado/$_name.tar.gz"
if [[ ! -x "$TORNADO_SDK/bin/tornado" ]]; then
TORNADO_SDK="$(dirname "$(find "$_root/.tornado" -path '*/bin/tornado' | head -1)")/.."
TORNADO_SDK="$(cd "$TORNADO_SDK" && pwd)"
fi
fi
fi
# the launcher prefers TORNADOVM_HOME over TORNADO_SDK: set both, or a stale TORNADOVM_HOME wins
export TORNADO_SDK TORNADOVM_HOME="$TORNADO_SDK"
export PATH="$TORNADO_SDK/bin:$JAVA_HOME/bin:$PATH"
22 changes: 22 additions & 0 deletions biojava-tornadovm/scripts/run.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#!/usr/bin/env bash
# Runs a main class of this project under the TornadoVM runtime.
# scripts/run.sh [-Dprop=value ...] <main-class> [args...]
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
source "$ROOT/scripts/env.sh"
if [[ ! -f "$ROOT/target/cp.txt" || "$ROOT/pom.xml" -nt "$ROOT/target/cp.txt" ]]; then
(cd "$ROOT" && mvn -q -B dependency:build-classpath -Dmdep.outputFile=target/cp.txt -Dmdep.includeScope=test \
-DexcludeArtifactIds=tornado-api \
-DexcludeGroupIds=org.junit.jupiter,org.junit.platform,org.junit.vintage,org.opentest4j,org.apiguardian >/dev/null)
fi
# JUnit is left out: the SDK puts its own JUnit on the module path, and two versions do not mix.
# The SDK's JVM options (module path, exports, native library path), with $TORNADOVM_HOME filled in.
ARGS="$ROOT/target/tornado.args"
if [[ ! -f "$ARGS" || "$TORNADOVM_HOME/tornado-argfile.template" -nt "$ARGS" ]] || ! grep -qF "$TORNADOVM_HOME/lib" "$ARGS"; then
sed "s|\${TORNADOVM_HOME}|$TORNADOVM_HOME|g" "$TORNADOVM_HOME/tornado-argfile.template" > "$ARGS"
fi
JVM=()
while [[ $# -gt 0 && "$1" == -* ]]; do JVM+=("$1"); shift; done
CP="$ROOT/target/classes:$ROOT/target/test-classes:$(cat "$ROOT/target/cp.txt")"
MAIN="$1"; shift
exec "$JAVA_HOME/bin/java" "@$ARGS" -Xmx24g -Dtornado.device.memory=20GB ${JVM[@]+"${JVM[@]}"} -cp "$CP" "$MAIN" "$@"
10 changes: 10 additions & 0 deletions biojava-tornadovm/scripts/test-gpu.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/usr/bin/env bash
# Runs the JUnit tests under the TornadoVM runtime, so that the GPU paths are exercised
# (a plain `mvn test` has no TornadoVM runtime and only checks the CPU fallbacks).
# JUnit's console launcher comes from the TornadoVM SDK (JUnit 6, `execute` subcommand).
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
source "$ROOT/scripts/env.sh"
(cd "$ROOT/.." && mvn -q -B -Ptornadovm -pl biojava-tornadovm -am -DskipTests install && cd "$ROOT" && mvn -q -B test-compile)
exec "$ROOT/scripts/run.sh" --add-modules=org.junit.jupiter -Dbiojava.tornado.expectGpu=true org.junit.platform.console.ConsoleLauncher execute \
--disable-banner --details=tree --scan-classpath "$ROOT/target/test-classes" "$@"
Loading
Loading