Miran Levar (2015) Web-based platform for dataflow processing. MSc thesis.
Abstract
This Thesis presents the design and implementation of a web-based general-purpose data processing platform. The platform consists of a storage system, processing system, and an application server with an exposed RESTful interface. Through this interface clients, such as web browser applications, can access and control data processing as well as view the results of it. The processing system is inspired by dataflow processing, defining each process as a black box with inputs and outputs. Connecting the outputs to inputs of other processes enables data to flow between processes and allows for creation of processing pipelines. Triggers – conditions under which a process can begin work on available input data – enable automation of the processing pipelines. Such a system allows users who lack programming expertise to easily create and run complex dataflow operations.
Actions (login required)