A system for parallel execution of data-flow graphs

Andrej Bukošek (2013) A system for parallel execution of data-flow graphs. EngD thesis.

Preview

PDF Download (5Mb)

Abstract

The goal of this thesis was to create a system for performing arbitrary operations on data in parallel using a data-flow graph. In this thesis we developed a system for parallel execution of data-flow graphs. Every node in the graph performs an operation, which operates on data passed to it via incoming graph edges and sends results over outgoing edges. We developed an execution engine for executing operations in parallel. Each operation is implemented in a dynamically loadable module or using a domain-specific language, which was developed specifically for this thesis. We also developed and implemented a compiler for this language. The domain-specific language is functional and strongly typed. We designed its type system to be modular. Every data type is implemented in an external dynamically loadable module, which the compiler loads during its initialization. Each module contains functions for generating an intermediate representation for the LLVM system, which optimizes it and translates it into machine code. As an example usage of our system we developed operations for image manipulation, compositing, and rendering of 3D scenes. Such a set of operations is commonly used in the film industry for the creation of special effects. We developed a renderer based on the path tracing algorithm, which creates an image from the description of a 3D scene. This method is based on a physically-correct simulation of light bouncing around the scene. The system, which consists of a compiler, execution engine, and a library of operations, is functioning satisfactory and fulfills the set goals.

Actions (login required)