Skip to content

Overview

Info

This document is a work in progress.

Data Collector Service, collects and stores many kinds of event data into Novo’s Smart Data Lake so that Novo can use such data for various analysis to enable personalization related features and functionality in addition to product developments & improvements. Such as:

  • Home area detection
  • Predictive Cards
  • Personalization
  • RoadSense

The Data Collector Service consists of two components - Data Collector and Data Source

Data Collector

On the edge, DataCollector works like a typical messaging middleware; when an event is reported by a producer, it will notify all consumers that subscribed to this event. In general, DataCollector concentrates on two kinds of events - user operation events from the application and sensor events from the vehicle. With a supported event data model provided as an API, user will construct a specific event and then send a request to DataCollector which contains the event data.

Important capabilities of Data Collector service include:

  • Support various event data models for application & sensor events
  • Packing & Compression to save bandwidth when uploading data to the cloud
  • Support realtime customized consumer when events arrive
  • Accomodate data caps for network usage
  • Local database to store events when network is not available
  • Upload data to Cloud Data Lake
  • Enables data analysis for powerful insights

The data collector package provides classes to send application and vehicle sensor events to cloud, and support realtime event consumption on the edge.

Please see the API reference for the main classes related to Data Collector.

The DataCollectorClient currently supports 4 requests (Send, Subscribe, Unsubscribe and Query Bucket Usage). Note that although these requests do currently support "asyncCall" API, using the synchronous "execute" API is recommended since it is a non-blocking call.

Note that although these 4 requests do currently support "asyncCall" API, using the synchronous "execute" API is recommended since it is a non-blocking call.

Refer to Send Event Request Subscribe Event Request Unsubscribe Event Request Query Bucket Usage Request sections for details.

Data Source

Data Source is provided as an optional component to facilitate generation of data from various data sources such as “android”, “vehicle”, “OBD”, etc.

Data Source can help produce event data regularly and automatically at pre-configured intervals and send them to the Data Collector component.

Currently, only GPS probe data collection is supported.

Relationship between Data Source & Data Collector:
image info