gpu branch #266
Merged
gpu branch #266
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
shumingma
reviewed
May 15, 2025
gpu/README.md
Outdated
|
|
||
| Within each block, values are stored contiguously in memory and permuted to facilitate efficient access and processing. | ||
|
|
||
| See `convert_convert_checkpoint.py` for details. |
shumingma
reviewed
May 15, 2025
gpu/convert_checkpoint.py
Outdated
| @@ -0,0 +1,106 @@ | |||
| # Copyright (c) Meta Platforms, Inc. and affiliates. | |||
Contributor
There was a problem hiding this comment.
may remove this as the conversion script is written by us.
shumingma
reviewed
May 15, 2025
gpu/convert_checkpoint.py
Outdated
| from einops import rearrange | ||
| from safetensors.torch import save_file | ||
| import model | ||
| from pack_weight import convert_weight_int8_to_int2_adsbrain, weight_repack, convert_weight_int8_to_int2 |
Contributor
There was a problem hiding this comment.
may remove 'convert_weight_int8_to_int2_adsbrain' if it's not used.
shumingma
reviewed
May 15, 2025
gpu/pack_weight.py
Outdated
|
|
||
| return ret | ||
|
|
||
| def convert_weight_int8_to_int2_adsbrain(weight): |
shumingma
reviewed
May 15, 2025
gpu/test.py
Outdated
| from torch.utils import benchmark | ||
| from torch import nn | ||
|
|
||
| from pack_weight import convert_weight_int8_to_int2, weight_repack, convert_weight_int8_to_int2_adsbrain |
Contributor
There was a problem hiding this comment.
weight_repack and convert_weight_int8_to_int2_adsbrain can be removed?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.