[WIP][Feature commit] Initial commit for supporting IPv6 stack in Ray Clus…#40332
[WIP][Feature commit] Initial commit for supporting IPv6 stack in Ray Clus…#40332jjacobj84 wants to merge 3 commits intoray-project:masterfrom
Conversation
…ter.
Changes in the PR include the following
1 : Refactoring python code base to support IPv6 addressing.
Replacing `split(":")` with `rsplit(":", 1)` to be compatible with IPv4 and IPv6 address
2 : Implemented a utility function `build_grpc_address` to support IPv6 addressing format for GRPC C-core implementation as per https://grpc.github.io/grpc/cpp/md_doc_naming.html
IPv4 format : <address>:<port>
IPv6 format : "ipv6":[<address>]:<port>
3 : Integrated `build_grpc_address` in `init_grpc_channel` and `_connect_channel` to support IPv6 based client connectivity
4 : Refactoring `GcsClientOptions` and `PythonGcsPublisher` address split to support IPv6 address format
5 : Changes in `BuildChannel` for building GRPC tunnels to support IPv6 addressing format
**Scope of this commit is limited to the following based on validation**
1 : Manually starting head and worker node using `--node-ip-address` parameter with IPv6 address.
Test also covered connectivity between head and worker nodes using IPv6 protocol
**Head Node Start Command**
`ray start --head --node-ip-address 2600:1f18:8ce:5f00:1198:a288:1745:1979`
**Pending**
- Adding Unit testcase.
- Validating CI/CD flow is good.
**Scope for next PR would include the following**
1 : Validating autodiscovery of head and worker node using IPv6 stack
2 : Autodiscovery of IPV6 node ip address based on google's DNS query and preferring IPv6 over IPv4 in the case of dual stack.
Signed-off-by: Japson Jeyasekaran <jjacobj@amazon.com>
Signed-off-by: Japson Jeyasekaran <jjacobj@amazon.com>
Signed-off-by: Japson Jeyasekaran <jjacobj@amazon.com>
|
Thanks for the contribution! We will take a look at this PR shotly :) |
|
is it still in WIP state now? |
|
ping on ^ |
|
|
gotcha. please ping me when it is ready! |
|
@jjacobj84 do you mind if I cherry-pick your My python code changes are complete but the C++ and headers changes are missing. |
This is the literal changes from @jjacobj84 from while I will build upon in later commits. Specifically files: - src/ray/gcs/gcs_client/gcs_client.h - src/ray/gcs/pubsub/gcs_pub_sub.cc - src/ray/rpc/grpc_client.h Sam Gleske (samrocketman) has updated these files resolving conflicts but left them in a similar form Japson originally contributed. See also -------- ray-project#40332 Signed-off-by: Japson Jeyasekaran <jjacobj@amazon.com> Signed-off-by: Sam Gleske <sam.mxracer@gmail.com> Signed-off-by: Sam Gleske <sgleske@integralads.com> Co-authored-by: Sam Gleske <sam.mxracer@gmail.com> Co-authored-by: Sam Gleske <sgleske@integralads.com>
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.
|
|
This pull request has been automatically closed because there has been no more activity in the 14 days Please feel free to reopen or open a new pull request if you'd still like this to be addressed. Again, you can always ask for help on our discussion forum or Ray's public slack channel. Thanks again for your contribution! |
This is the literal changes from @jjacobj84 from while I will build upon in later commits. Specifically files: - src/ray/gcs/gcs_client/gcs_client.h - src/ray/gcs/pubsub/gcs_pub_sub.cc - src/ray/rpc/grpc_client.h Sam Gleske (samrocketman) has updated these files resolving conflicts but left them in a similar form Japson originally contributed. See also -------- ray-project#40332 Signed-off-by: Japson Jeyasekaran <jjacobj@amazon.com> Signed-off-by: Sam Gleske <sam.mxracer@gmail.com> Signed-off-by: Sam Gleske <sgleske@integralads.com> Co-authored-by: Sam Gleske <sam.mxracer@gmail.com> Co-authored-by: Sam Gleske <sgleske@integralads.com>
Why are these changes needed?
Initial commit for supporting IPv6 stack in Ray Cluser
Changes in the PR include the following
1 : Refactoring python code base to support IPv6 addressing.
- Replacing
split(":")withrsplit(":", 1)to be compatible with IPv4 and IPv6 address2 : Implemented a utility function
build_grpc_addressto support IPv6 addressing format for GRPC C-core implementation as per https://grpc.github.io/grpc/cpp/md_doc_naming.html3 : Integrated
build_grpc_addressininit_grpc_channeland_connect_channelto support IPv6 based client connectivity4 : Refactoring
GcsClientOptionsandPythonGcsPublisheraddress split to support IPv6 address format5 : Changes in
BuildChannelfor building GRPC tunnels to support IPv6 addressing formatScope of this commit is limited to the following based on validation
1 : Manually starting head and worker node using
--node-ip-addressparameter with IPv6 address.Test also covered connectivity between head and worker nodes using IPv6 protocol
e.g
ray start --head --node-ip-address 2600:1f18:8ce:5f00:1198:a288:1745:1979Pending items in this PR
Scope for next PR would include the following
1 : Validating autodiscovery of head and worker node using IPv6 stack
2 : Autodiscovery of IPV6 node ip address based on google's DNS query and preferring IPv6 over IPv4 in the case of dual stack.
Related issue number
Checks
git commit -s) in this PR.scripts/format.shto lint the changes in this PR.method in Tune, I've added it in
doc/source/tune/api/under thecorresponding
.rstfile.