Skip to content

Add encoding cache and lazy-train mechanism#50

Merged
ljshou merged 69 commits intomasterfrom
dev/fecheng
Aug 2, 2019
Merged

Add encoding cache and lazy-train mechanism#50
ljshou merged 69 commits intomasterfrom
dev/fecheng

Conversation

@chengfx
Copy link
Copy Markdown
Contributor

@chengfx chengfx commented May 24, 2019

There are two major updates in this PR.

  1. Encoding Cache
    Now encoding cache mechanism is supported. It's a lazy-build progress. So we will build it in the first epoch and then reuse it in the rest epoch.
  2. Lazy Train
    We will load chunk_size cases to train in every part of an epoch avoiding out of memory in large training data

Feixiang Cheng and others added 30 commits April 25, 2019 22:09
@chengfx
Copy link
Copy Markdown
Contributor Author

chengfx commented May 28, 2019

tasks                       GPU/CPU           old accuracy/AUC        accuracy/new AUC
english_text_matching           GPU             0.96655                 0.967
english_text_matching           CPU             0.96655                 0.967
chinese_text_matching           GPU             0.70001                 0.7
chinese_text_matching           CPU             0.70001                 0.7
quora_question_pairs            GPU             0.72596                 0.72086
quora_question_pairs            CPU             0.72596                 0.72086
knowledge_distillation          CPU             0.66329                 0.6628427083333334

@chengfx
Copy link
Copy Markdown
Contributor Author

chengfx commented Aug 1, 2019

New auto-test result @adolphk-yk

tasks			     GPU/CPU	old accuracy/AUC	new accuracy/AUC 
english_text_matching		GPU		0.96655			0.97075
english_text_matching		CPU		0.96655			0.97075
chinese_text_matching		GPU		0.70001			0.7
chinese_text_matching		CPU		0.70001			0.7
quora_question_pairs		GPU		0.72596			0.721861
quora_question_pairs		CPU		0.72596			0.721861
knowledge_distillation		CPU		0.66329			0.6580784722222223

@ljshou ljshou merged commit 58ad563 into master Aug 2, 2019
@ljshou ljshou deleted the dev/fecheng branch August 2, 2019 12:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants