GitHub - timespacce/math_exp_bert: Google's Bidirectional Encoder Representations From Transformers - the predecessor of GPT-3 - implemented for Multi-GPU and TPU and trained on 105M sequences (27B tokens).

timespacce / math_exp_bert Public

Notifications You must be signed in to change notification settings
Fork 1
Star 0

Google's Bidirectional Encoder Representations From Transformers - the predecessor of GPT-3 - implemented for Multi-GPU and TPU and trained on 105M sequences (27B tokens).

0 stars 1 fork Branches Tags Activity

Star

Notifications

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 161 Commits
configs		configs
vocabs		vocabs
.gitignore		.gitignore
bert_embedding.py		bert_embedding.py
configuration.py		configuration.py
data_loader.py		data_loader.py
encoder.py		encoder.py
environment.yml		environment.yml
gelu.py		gelu.py
main.py		main.py
mha.py		mha.py
optimization.py		optimization.py
transformer.py		transformer.py
validation.py		validation.py

About

Google's Bidirectional Encoder Representations From Transformers - the predecessor of GPT-3 - implemented for Multi-GPU and TPU and trained on 105M sequences (27B tokens).