From the course: Predictive Customer Analytics
Join today to access over 23,400 courses taught by industry experts.
Design data processing pipelines - Python Tutorial
From the course: Predictive Customer Analytics
Design data processing pipelines
- A very important aspect of successful customer analytics is the data pipeline and processing infrastructure within your business. This infrastructure ensures that data is available for analytics in a timely manner and guarantees its accuracy. The following are some of the recommended best practices for building data processing pipelines. Provide mechanisms for republishing and reprocesing data to catch up for gaps and interruptions. Work with data provider organizations to make sure that any gaps in data availability, security, and format are taken care of. Separate realtime and historical data feeds if required. Realtime data is expected to be fast, but can suffer from data loss. Historical data, however, is batch-oriented, and accuracy of data is far more important. While it is a good design goal to build the same pipeline for both realtime and historical, do not try to overdesign to make this happen. It only makes…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.