A service-oriented framework for ETL implementation

Abstract

The development of analytical systems imposes several challenges related not only to the amount and heterogeneity of the involved data but also to the constant need to readapt and evolve to overcome new business challenges. The data modelling layer represents the mapping between the domain and technical knowledge, however, to organize raw data into a form that can be used for analytics, specific Extract, Transform and Load (ETL) processes should be applied. ETL systems are recognized as a critical and tightly coupled system component that encapsulates data-level requirements that are hard to implement and maintain. In a Big Data era, adaptability and extensibility are important characteristics to hold when developing analytical systems. Thus, to provide more consistent, reliable, flexible, and reusable ETL processes, a service-oriented implementation for ETL development is proposed.