Utilities

Various utilities and scripts

Data Setup Utility

This utility helps in setting up data for Parquet, Iceberg and Delta tables.

Give input in `trino_input.csv`

Then Execute `python TrinoQueryGen.py`. This will generate `output_data.csv` which will queries to run.

Then Execute `python ExecuteQueryOnTrino.py`. This will execute those queries on the trino setup.

Now data should be generated.

Trino Docker Package

This package contains docker compose file for setting local data lake using Trino, Minio and Hive Metastore. It also uses postgres as database for metastore. Two handy scripts are provided to start and stop dockerized trino setup.

Name		Name	Last commit message	Last commit date
datasetup		datasetup
trino-docker		trino-docker
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md

Navigation Menu

Repository navigation

About

Repository files navigation

Utilities

Data Setup Utility

Give input in `trino_input.csv`

Then Execute `python TrinoQueryGen.py`. This will generate `output_data.csv` which will queries to run.

Then Execute `python ExecuteQueryOnTrino.py`. This will execute those queries on the trino setup.

Now data should be generated.

Trino Docker Package

`start_trino.sh` - This setups trino + Minio + Hive Metastore cluster.

`stop_trino.sh` - This destroys above created setup.

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

Utilities

Data Setup Utility

Give input in trino_input.csv

Then Execute python TrinoQueryGen.py. This will generate output_data.csv which will queries to run.

Then Execute python ExecuteQueryOnTrino.py. This will execute those queries on the trino setup.

Now data should be generated.

Trino Docker Package

start_trino.sh - This setups trino + Minio + Hive Metastore cluster.

stop_trino.sh - This destroys above created setup.

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

Give input in `trino_input.csv`

Then Execute `python TrinoQueryGen.py`. This will generate `output_data.csv` which will queries to run.

Then Execute `python ExecuteQueryOnTrino.py`. This will execute those queries on the trino setup.

`start_trino.sh` - This setups trino + Minio + Hive Metastore cluster.

`stop_trino.sh` - This destroys above created setup.