Skip to content

[New Scheduler] Implement KeepAliveService #5067

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
style95 merged 4 commits into from
Mar 11, 2021
Merged

[New Scheduler] Implement KeepAliveService #5067

style95 merged 4 commits into from
Mar 11, 2021

Conversation

@KeonHee
Copy link
Member

@KeonHee KeonHee commented Feb 15, 2021 •
edited
Loading

Description

  • In invoker and scheduler components, each host has a lease. This service manages lease to not disappear due to timeout.
  • Each host writes health data to etcd with this lease. When lease disappears by timeout, each host's health data disappear together, so it can be determined as an unhealthy host by the scheduler.
  • Because it is the service that has the greatest influence on the health state of the component, a dedicated dispatcher is allocated.
  • https://cwiki.apache.org/confluence/display/OPENWHISK/LeaseKeepAliveService

Related issue and scope

  • I opened an issue to propose and discuss this change (#????)

My changes affect the following components

  • API
  • Controller
  • Message Bus (e.g., Kafka)
  • Loadbalancer
  • Invoker
  • Scheduler
  • Intrinsic actions (e.g., sequences, conductors)
  • Data stores (e.g., CouchDB)
  • Tests
  • Deployment
  • CLI
  • General tooling
  • Documentation

Types of changes

  • Bug fix (generally a non-breaking change which closes an issue).
  • Enhancement or new feature (adds new functionality).
  • Breaking change (a bug fix or enhancement which changes existing behavior).

Checklist:

  • I signed an Apache CLA.
  • I reviewed the style guides and followed the recommendations (Travis CI will check :).
  • I added tests to cover my changes.
  • My changes require further changes to the documentation.
  • I updated the documentation where necessary.

@codecov-io
Copy link

codecov-io commented Feb 15, 2021 •
edited
Loading

Codecov Report

Merging #5067 (dd4e051) into master (ed58b23) will decrease coverage by 46.07%.
The diff coverage is 78.78%.

Impacted file tree graph

@@             Coverage Diff             @@
##           master    #5067       +/-   ##
===========================================
- Coverage   81.61%   35.54%   -46.08%     
===========================================
  Files         204      205        +1     
  Lines        9950    10011       +61     
  Branches      447      457       +10     
===========================================
- Hits         8121     3558     -4563     
- Misses       1829     6453     +4624
Impacted Files Coverage Δ
...rg/apache/openwhisk/core/scheduler/Scheduler.scala 0.00% <0.00%> (ø)
...openwhisk/core/service/LeaseKeepAliveService.scala 84.21% <84.21%> (ø)
...in/scala/org/apache/openwhisk/common/Logging.scala 71.85% <100.00%> (-13.93%) ⬇️
.../scala/org/apache/openwhisk/core/WhiskConfig.scala 95.45% <100.00%> (+0.05%) ⬆️
...a/org/apache/openwhisk/common/ConfigMapValue.scala 0.00% <0.00%> (-100.00%) ⬇️
.../apache/openwhisk/core/controller/Namespaces.scala 0.00% <0.00%> (-100.00%) ⬇️
...pache/openwhisk/core/controller/CorsSettings.scala 0.00% <0.00%> (-100.00%) ⬇️
...che/openwhisk/core/entitlement/RateThrottler.scala 0.00% <0.00%> (-100.00%) ⬇️
...he/openwhisk/core/entitlement/KindRestrictor.scala 0.00% <0.00%> (-100.00%) ⬇️
...penwhisk/core/database/cosmosdb/CosmosDBUtil.scala 0.00% <0.00%> (-100.00%) ⬇️
... and 136 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ed58b23...dd4e051. Read the comment docs.

@KeonHee KeonHee closed this Feb 16, 2021
@KeonHee KeonHee reopened this Feb 16, 2021
@KeonHee KeonHee closed this Feb 16, 2021
@KeonHee KeonHee reopened this Feb 16, 2021
bdoyle0182
bdoyle0182 reviewed Feb 22, 2021
View reviewed changes
common/scala/src/main/scala/org/apache/openwhisk/core/service/LeaseKeepAliveService.scala Outdated
case class Lease(id: Long, ttl: Long) extends KeepAliveServiceData

// Events received by the actor
case object ReGrantLease
Copy link
Contributor

@bdoyle0182 bdoyle0182 Feb 22, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
case object ReGrantLease
case object RegrantLease

Copy link
Member Author

@KeonHee KeonHee Feb 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated it to RegrantLease

common/scala/src/main/scala/org/apache/openwhisk/core/service/LeaseKeepAliveService.scala Outdated
implicit val ec: ExecutionContextExecutor = context.dispatcher

private val leaseTimeout = loadConfigOrThrow[Int](ConfigKeys.etcdLeaseTimeout).seconds
private var worker: Option[Cancellable] = None
Copy link
Contributor

@bdoyle0182 bdoyle0182 Feb 22, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if this var is thread safe since it gets set in a Future

Copy link
Member Author

@KeonHee KeonHee Feb 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By including the worker in the state data, I removed the var value.

@KeonHee KeonHee changed the title [New Scheduler] Implement KeepAliveService [WIP] [New Scheduler] Implement KeepAliveService Feb 24, 2021
@KeonHee KeonHee changed the title [WIP] [New Scheduler] Implement KeepAliveService [New Scheduler] Implement KeepAliveService Feb 28, 2021
@KeonHee KeonHee changed the title [New Scheduler] Implement KeepAliveService [WIP] [New Scheduler] Implement KeepAliveService Feb 28, 2021
@KeonHee KeonHee changed the title [WIP] [New Scheduler] Implement KeepAliveService [New Scheduler] Implement KeepAliveService Feb 28, 2021
@ningyougang
Copy link
Contributor

ningyougang commented Mar 2, 2021

LGTM

@KeonHee
Copy link
Member Author

KeonHee commented Mar 4, 2021

@bdoyle0182

I've reflected all of your reviews. please review again :)

@style95 style95 mentioned this pull request Mar 8, 2021
Merged
21 tasks
@style95
Copy link
Member

style95 commented Mar 9, 2021

I would merge this in 24 hours for subsequent PRs.

@bdoyle0182
Copy link
Contributor

bdoyle0182 commented Mar 9, 2021

I would merge this in 24 hours for subsequent PRs.

LGTM

@style95 style95 merged commit e05aa44 into apache:master Mar 11, 2021
@codecov-commenter
Copy link

codecov-commenter commented Jul 25, 2025

Codecov Report

Attention: Patch coverage is 78.78788% with 14 lines in your changes missing coverage. Please review.

Project coverage is 75.26%. Comparing base (ed58b23) to head (dd4e051).
Report is 232 commits behind head on master.

Files with missing lines Patch % Lines
...openwhisk/core/service/LeaseKeepAliveService.scala 84.21% 9 Missing ⚠️
...rg/apache/openwhisk/core/scheduler/Scheduler.scala 0.00% 5 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (ed58b23) and HEAD (dd4e051). Click for more details.

HEAD has 3 uploads less than BASE
Flag BASE (ed58b23) HEAD (dd4e051)
8 5
Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #5067      +/-   ##
==========================================
- Coverage   81.61%   75.26%   -6.36%     
==========================================
  Files         204      212       +8     
  Lines        9950    10276     +326     
  Branches      447      460      +13     
==========================================
- Hits         8121     7734     -387     
- Misses       1829     2542     +713

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bdoyle0182 bdoyle0182 bdoyle0182 left review comments

@ningyougang ningyougang ningyougang approved these changes

Assignees

No one assigned

Labels

scheduler

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@KeonHee @codecov-io @ningyougang @style95 @bdoyle0182 @codecov-commenter