It's a project named "DataCleaner" which is responsible for the device message tackling. It's using SparkStreaming&KAKFA to tackle the stream data of devies. To simplify the coding process, Scala which supports functional programming is used as the main programing language. This layer tackle the device message from "DeviceGate" layer according to the configuration which can be configure dynamically. For now, only simple configurations, for example value transformation, average value calculation, are supported, and more abundant features are under development. After tackling, tackled messages will be stored into mongodb or hdfs and some information will be pushed to "APIBackend" layer through KAFKA for further use.
-
Notifications
You must be signed in to change notification settings - Fork 0
heheave/datacleaner
About
DataClean layer
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published