Skip to content

Commit bc150e6

Browse files
AlchemistZoroAlchemistZoro
committed
readme change
1 parent 72ee20b commit bc150e6

11 files changed

Lines changed: 44 additions & 100 deletions

File tree

‎1.sh‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# CUDA_VISIBLE_DEVICES=5 ./run_conceptlm.sh obqa --data_dir data/ --emp False --use_wandb True -k 1
2+
# CUDA_VISIBLE_DEVICES=5 ./run_conceptlm.sh obqa --data_dir data/ --emp False --use_wandb True -k 3
3+
# CUDA_VISIBLE_DEVICES=5 ./run_conceptlm.sh obqa --data_dir data/ --emp False --use_wandb True -k 7
4+
# CUDA_VISIBLE_DEVICES=5 ./run_conceptlm.sh obqa --data_dir data/ --emp False --use_wandb True --mix_number 2
5+
# CUDA_VISIBLE_DEVICES=5 ./run_conceptlm.sh obqa --data_dir data/ --emp False --use_wandb True --mix_number 3
6+
CUDA_VISIBLE_DEVICES=5 ./run_conceptlm.sh obqa --data_dir data/ --emp False --use_wandb True --mix_number 5
7+
# CUDA_VISIBLE_DEVICES=6 ./run_conceptlm.sh obqa --data_dir data/ --emp False --use_wandb True --mix_number 10
8+
# CUDA_VISIBLE_DEVICES=6 ./run_conceptlm.sh obqa --data_dir data/ --emp False --use_wandb True --mix_number 20

‎2.sh‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
CUDA_VISIBLE_DEVICES=5 ./run_conceptlm.sh obqa --data_dir data/ --emp False --use_wandb True
2+
CUDA_VISIBLE_DEVICES=5 ./run_conceptlm.sh csqa --data_dir data/ --emp False --use_wandb True
3+
CUDA_VISIBLE_DEVICES=6 ./run_conceptlm.sh obqa --data_dir data/ --emp False --use_wandb True --gnn_dim 100
4+
CUDA_VISIBLE_DEVICES=6 ./run_conceptlm.sh obqa --data_dir data/ --emp False --use_wandb True --gnn_dim 300

‎3.sh‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
CUDA_VISIBLE_DEVICES=7 ./run_conceptlm.sh obqa --data_dir data/ --emp True --use_wandb True -k 1
2+
CUDA_VISIBLE_DEVICES=7 ./run_conceptlm.sh obqa --data_dir data/ --emp True --use_wandb True -k 3
3+
CUDA_VISIBLE_DEVICES=7 ./run_conceptlm.sh obqa --data_dir data/ --emp True --use_wandb True -k 5
4+
CUDA_VISIBLE_DEVICES=7 ./run_conceptlm.sh obqa --data_dir data/ --emp True --use_wandb True -k 7
5+
CUDA_VISIBLE_DEVICES=7 ./run_conceptlm.sh csqa --data_dir data/ --emp True --use_wandb True -k 5

‎README.md‎

Lines changed: 26 additions & 80 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,4 @@
1-
# ConceptLM: Graph REASoning Enhanced Language Models for Question Answering
2-
3-
This repo provides the source code & data of our paper [ConceptLM: Graph REASoning Enhanced Language Models for Question Answering](https://arxiv.org/abs/2201.08860) (ICLR 2022 spotlight). If you use any of our code, processed data or pretrained models, please cite:
4-
```bib
5-
@inproceedings{zhang2021conceptlm,
6-
title={ConceptLM: Graph REASoning Enhanced Language Models},
7-
author={Zhang, Xikun and Bosselut, Antoine and Yasunaga, Michihiro and Ren, Hongyu and Liang, Percy and Manning, Christopher D and Leskovec, Jure},
8-
booktitle={International Conference on Learning Representations},
9-
year={2021}
10-
}
11-
```
12-
13-
<p align="center">
14-
<img src="./figs/conceptlm.png" width="600" title="ConceptLM model architecture" alt="">
15-
</p>
1+
# ConceptLM
162

173
## 1. Dependencies
184

@@ -66,92 +52,52 @@ The script to download and preprocess the [MedQA-USMLE](https://github.com/jind1
6652
### Directly download preprocessed data
6753
For your convenience, if you don't want to preprocess the data yourself, you can download all the preprocessed data [here](https://drive.google.com/drive/folders/1T6B4nou5P3u-6jr0z6e3IkitO8fNVM6f?usp=sharing). Download them into the top-level directory of this repo and unzip them. Move the `medqa_usmle` and `ddb` folders into the `data/` directory.
6854

69-
### Resulting file structure
7055

71-
The resulting file structure should look like this:
72-
73-
```plain
74-
.
75-
├── README.md
76-
├── data/
77-
├── cpnet/ (prerocessed ConceptNet)
78-
├── csqa/
79-
├── train_rand_split.jsonl
80-
├── dev_rand_split.jsonl
81-
├── test_rand_split_no_answers.jsonl
82-
├── statement/ (converted statements)
83-
├── grounded/ (grounded entities)
84-
├── graphs/ (extracted subgraphs)
85-
├── ...
86-
├── obqa/
87-
├── medqa_usmle/
88-
└── ddb/
89-
```
9056

9157
## 3. Training ConceptLM
9258
To train ConceptLM on CommonsenseQA, run
9359
```
9460
CUDA_VISIBLE_DEVICES=0 ./run_conceptlm.sh csqa --data_dir data/
9561
```
96-
CSQA with pool
97-
```
98-
CUDA_VISIBLE_DEVICES=3 ./run_conceptlm.sh csqa --data_dir data/ --use_wandb True --emp False -mbs 4
99-
```
100-
You can specify up to 2 GPUs you want to use in the beginning of the command `CUDA_VISIBLE_DEVICES=...`.
10162

102-
Similarly, to train ConceptLM on OpenbookQA, run
63+
Debug on OBQA
10364
```
104-
CUDA_VISIBLE_DEVICES=0 ./run_conceptlm.sh obqa --data_dir data/
65+
CUDA_VISIBLE_DEVICES=0 ./run_conceptlm.sh obqa --data_dir data/ --emp True --debug True
10566
```
10667

107-
Embedding pool experiment test
108-
```
109-
CUDA_VISIBLE_DEVICES=1 ./run_conceptlm.sh obqa --data_dir data/ --use_wandb True --emp False
110-
```
11168

112-
Debug
69+
## 4. Experimental expansion
70+
### BASE MODEL
11371
```
114-
CUDA_VISIBLE_DEVICES=0 ./run_conceptlm.sh obqa --data_dir data/ --emp True --debug True
72+
./run_conceptlm.sh obqa --data_dir data/ --emp False --use_wandb True
73+
./run_conceptlm.sh csqa --data_dir data/ --emp False --use_wandb True
11574
```
11675

117-
To train ConceptLM on MedQA-USMLE, run
76+
### Different number of mixed coding layers
11877
```
119-
CUDA_VISIBLE_DEVICES=0 ./run_conceptlm__medqa_usmle.sh
78+
./run_conceptlm.sh obqa --data_dir data/ --emp False --use_wandb True -k 1
79+
./run_conceptlm.sh obqa --data_dir data/ --emp False --use_wandb True -k 3
80+
./run_conceptlm.sh obqa --data_dir data/ --emp False --use_wandb True -k 7
12081
```
121-
122-
## 4. Pretrained model checkpoints
123-
You can download a pretrained ConceptLM model on CommonsenseQA [here](https://drive.google.com/file/d/1QPwLZFA6AQ-pFfDR6TWLdBAvm3c_HOUr/view?usp=sharing), which achieves an IH-dev acc. of `79.0` and an IH-test acc. of `74.0`.
124-
125-
You can also download a pretrained ConceptLM model on OpenbookQA [here](https://drive.google.com/file/d/1-QqyiQuU9xlN20vwfIaqYQ_uJMP8d7Pv/view?usp=sharing), which achieves an test acc. of `84.8`.
126-
127-
You can also download a pretrained ConceptLM model on MedQA-USMLE [here](https://drive.google.com/file/d/1j0QxiBiGbv0s9PhseSly6V6uiHWU5IEt/view?usp=sharing), which achieves an test acc. of `38.5`.
128-
129-
## 5. Evaluating a pretrained model checkpoint
130-
To evaluate a pretrained ConceptLM model checkpoint on CommonsenseQA, run
82+
### Entity encoding node
13183
```
132-
CUDA_VISIBLE_DEVICES=0 ./eval_conceptlm.sh csqa --data_dir data/ --load_model_path /path/to/checkpoint
84+
./run_conceptlm.sh obqa --data_dir data/ --emp True --use_wandb True -k 1
85+
./run_conceptlm.sh obqa --data_dir data/ --emp True --use_wandb True -k 3
86+
./run_conceptlm.sh obqa --data_dir data/ --emp True --use_wandb True -k 5
87+
./run_conceptlm.sh obqa --data_dir data/ --emp True --use_wandb True -k 7
88+
./run_conceptlm.sh csqa --data_dir data/ --emp True --use_wandb True -k 5
13389
```
134-
Again you can specify up to 2 GPUs you want to use in the beginning of the command `CUDA_VISIBLE_DEVICES=...`.
135-
136-
Similarly, to evaluate a pretrained ConceptLM model checkpoint on OpenbookQA, run
90+
### Different number of interaction nodes
13791
```
138-
CUDA_VISIBLE_DEVICES=0 ./eval_conceptlm.sh obqa --data_dir data/ --load_model_path /path/to/checkpoint
92+
./run_conceptlm.sh obqa --data_dir data/ --emp False --use_wandb True --mix_number 2
93+
./run_conceptlm.sh obqa --data_dir data/ --emp False --use_wandb True --mix_number 3
94+
./run_conceptlm.sh obqa --data_dir data/ --emp False --use_wandb True --mix_number 5
95+
./run_conceptlm.sh obqa --data_dir data/ --emp False --use_wandb True --mix_number 10
96+
./run_conceptlm.sh obqa --data_dir data/ --emp False --use_wandb True --mix_number 20
13997
```
140-
To evaluate a pretrained ConceptLM model checkpoint on MedQA-USMLE, run
98+
### Subgraphs of different number of nodes
14199
```
142-
INHERIT_BERT=1 CUDA_VISIBLE_DEVICES=0 ./eval_conceptlm.sh medqa_usmle --data_dir data/ --load_model_path /path/to/checkpoint
100+
./run_conceptlm.sh obqa --data_dir data/ --emp False --use_wandb True --gnn_dim 100
101+
./run_conceptlm.sh obqa --data_dir data/ --emp False --use_wandb True --gnn_dim 300
143102
```
144103

145-
## 6. Use your own dataset
146-
- Convert your dataset to `{train,dev,test}.statement.jsonl` in .jsonl format (see `data/csqa/statement/train.statement.jsonl`)
147-
- Create a directory in `data/{yourdataset}/` to store the .jsonl files
148-
- Modify `preprocess.py` and perform subgraph extraction for your data
149-
- Modify `utils/parser_utils.py` to support your own dataset
150-
151-
## 7. Acknowledgment
152-
This repo is built upon the following work:
153-
```
154-
QA-GNN: Question Answering using Language Models and Knowledge Graphs
155-
https://github.com/michiyasunaga/qagnn
156-
```
157-
Many thanks to the authors and developers!

greaselm.py renamed to conceptlm.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -577,7 +577,7 @@ def main(args):
577577

578578
args.hf_version = transformers.__version__
579579

580-
with wandb.init(project="KG-LM", config=args, name=args.run_name, resume="allow", id=wandb_id, settings=wandb.Settings(start_method="fork"), mode=wandb_mode):
580+
with wandb.init(project="CLM-TEST", config=args, name=args.run_name, resume="allow", id=wandb_id, settings=wandb.Settings(start_method="fork"), mode=wandb_mode):
581581
print(socket.gethostname())
582582
print ("pid:", os.getpid())
583583
print ("screen: %s" % subprocess.check_output('echo $STY', shell=True).decode('utf'))

eval_greaselm.sh renamed to eval_concept.sh

File renamed without changes.

‎exp.csv‎

Lines changed: 0 additions & 19 deletions
This file was deleted.

‎figs/greaselm.png‎

-134 KB
Binary file not shown.

modeling/modeling_greaselm.py renamed to modeling/modeling_conceptlm.py

File renamed without changes.

run_greaselm.sh renamed to run_conceptlm.sh

File renamed without changes.

0 commit comments

Comments
 (0)