ISLR_LIBRAS is a toolkit and dataset repository from Malta Lab for benchmarking sign-language recognition models. It provides:

• ✔️ A curated Brazilian Sign Language dataset (MALTA-LIBRAS) and its intersections with other open datasets
• ✔️ Scripts to download, preprocess, and convert videos into tensors
• ✔️ End-to-end training and evaluation pipelines using PyTorch Lightning
• ✔️ Utilities to aggregate results and reproduce experiments from our paper

• 🗂️ Data Downloading
• ⚙️ Dataset Preparation
• 🛠️ Installation
• 🚀 Usage
• 🧰 Scripts Overview
• 📊 Evaluation & Results
• 🤝 Contributing
• 📄 License
• 📫 Contact

Due to data policies, we can’t host the videos directly, but all sources are public! 🙌

Sources for each dictionary in MALTA-LIBRAS (web-scraping via 'videos_download/download_videos.py'):

• INES V2 📘: http://www.acessibilidadebrasil.org.br/libras/
• INES V3 📗: http://www.acessibilidadebrasil.org.br/libras_3/
• Corpus Libras (UFSC) 📙: https://corpuslibras.ufsc.br/
• SignBank (UFSC_V2) 📓: https://signbank.libras.ufsc.br/pt
• Spread the Sign 🌐: https://www.spreadthesign.com/pt.br/search/
• V-LIBRASIL (UFPE) 🎥: https://libras.cin.ufpe.br/
• USP 🏛️: https://edisciplinas.usp.br/mod/glossary/view.php?id=197645
• UFV 🎓: https://sistemas.cead.ufv.br/capes/dicionario/
• YouTube ▶️: a curated SharePoint link is provided in the repo for direct download.

Other datasets:

• WLASL (American SL) 🇺🇸: download via Kaggle
• SLOVO (Czech SL) 🇨🇿: clone from https://github.com/hukenovs/slovo.git

Annotations:

• 💡 Full MALTA-LIBRAS glossary: 'dataset_intersections/glossary.csv'
• 💡 Experiment subset: 'dataset_intersections/matched_labels_with_tensors.csv'

All data is also available in tensor format at: https://huggingface.co/datasets/MALTA-Lab/MALTA_LIBRAS/tree/main

1. Build tensors for public benchmarks
   bash python build_tensor_dataset.py
   --wl dataset_path/WLASL
   --slovo dataset_path/SLOVO
   --minds dataset_path/MINDS
   --output_dir tensors/

2. Build tensors for MALTA_LIBRAS
   bash python vuild_test_dataset_tensors.py
   --annotations dataset_intersections/glossary.csv
   --videos_dir path/to/downloaded_videos
   --output_dir tensors/malta_libras

git clone https://github.com/Malta-Lab/ISLR_LIBRAS.git cd ISLR_LIBRAS

python3 -m venv .venv source .venv/bin/activate

pip install --upgrade pip pip install -r requirements.txt

pip install jupyterlab

Launch training with:

python train.py
--data_dir tensors/
--dataset MALTA_LIBRAS
--model resnet3d
--epochs 50
--batch_size 16
--gpus 1
--seed 42
--output_dir lightning_logs/

After training, generate results:

python create_results.py
--logs_dir lightning_logs/
--output_csv results/summary.csv

build_tensor_dataset.py — prepare tensors for WLASL, SLOVO, MINDS

build_test_dataset_tensors.py — prepare tensors for MALTA-LIBRAS

train.py — model training pipeline (PyTorch Lightning)

create_results.py — aggregate .csv outputs into summary tables

delete_ckpt.py — clean up old checkpoints

metrics_evaluation.ipynb — analyze experiment results

dataset_intersections/ — CSVs of overlapping labels

videos_download/ — scripts to fetch videos

Use metrics_evaluation.ipynb to visualize per-class accuracy, confusion matrices, and learning curves.

Checkpoints, logs, and CSVs are stored under lightning_logs/<experiment_name>/.

This repo is under MIT license.

Malta Lab – https://github.com/Malta-Lab