Skip to content

heheave/datacleaner

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

13 Commits
 
 
 
 
 
 
 
 

Repository files navigation

It's a project named "DataCleaner" which is responsible for the device message tackling. It's using SparkStreaming&KAKFA to tackle the stream data of devies. To simplify the coding process, Scala which supports functional programming is used as the main programing language. This layer tackle the device message from "DeviceGate" layer according to the configuration which can be configure dynamically. For now, only simple configurations, for example value transformation, average value calculation, are supported, and more abundant features are under development. After tackling, tackled messages will be stored into mongodb or hdfs and some information will be pushed to "APIBackend" layer through KAFKA for further use.

About

DataClean layer

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published