AnyDSL-Ray - An Open, Scalable, and Portable Ray Tracing Infrastructure

Ray tracing as a technology lies at the intersection of many different visual computing technologies. Many visual computing related projects – both inside and outside of the Intel-VCI – use or target ray tracing technology.

However, today’s ray tracing implementations suffer from a range of limitations: They are often monolithic applications rather than exible libraries on which other projects can build up, they typically scale only along some of the axes of image quality, model size, hardware resources, and application type. Also, many of them are not easily extensible, or tied to a specic hardware platform. This is true for systems like Nvidia Optix, which is tied to a specic hardware architecture: The Open Source system Manta, which lacks high-quality renderers and exposes the programmer to low-level SSE coding; Embree focuses on individual kernels and does not provide a full rendering solution (yet), or PBRT, which is a complete rendering but is missing the necessary performance optimizations.

The project aims at closing this gap by designing and building a ray tracing infrastructure that will be fully open source, that can be used as a library/tool, that uses the proper compiler technology to be portable across a wide range of hardware platforms while offering close to optimal performance on each of them, and that explicitly aims at providing scalability across the different dimensions of ray tracing technology needed by users. We approach the goal of the project in two stages: The basic AnyDSL compiler as well as the AnyDSL-Ray ray tracing system.

We are building on top of the AnyDSL compiler project, a project that aims at offering high-level domain-specific libraries (DSL) while allowing them to be compiled down to highly efficient and hardware specific code using user-space libraries of refinement transformations. By now, AnyDSL is at a stage at which larger applications can use the infrastructure even though further refinements and extensions are still needed for AnyDSL.

Within AnyDSL-Ray, we have created ray traversal abstractions. From these, we can generate high-performance variants of ray traversal routines. The resulting code is on a par with Embree’s hand-coded kernels on the CPU and even outperforms state-of-the-art GPU implementations.

Current status of the activities:

  • AnyDSL offers some unique compiler tools that will be instrumental for a generic high-performance ray tracing library. This includes the code that maps the abstractions used in the generic code to both CPU and GPU target HW. The development of the ray tracing code has identified some limitations of AnyDSL specifically with regards to the type system that will be fixed with the upcoming new type system for AnyDSL and other work.
  • The ray tracing code mentioned above includes a full ray tracer with global illumination (using path tracing), transparency, textures, and advanced materials (like glass). By directly using the new DSL capabilities, we are defining and formulating the architecture to be scalable across all of the above domains (quality, complexity, hardware, applications, etc.), while offering the necessary hooks to perform the generic and hardware-specific optimizations through appropriate refinements of the DSL constructs. The goal is to get the performance close to that of the highly optimized code in Embree, which required a large number of person years to develop. However, we will be able to benefit from many of the architectural and algorithmic designs decisions in Embree [Siggraph’14].

Creating a collaborative research platform: the AnyDSL-Ray ray tracing system aims at providing a very flexible yet high-performance basis for other projects to build upon. To that end, we need to bring a renderer into form that can easily be used by external developers through an Open-Source approach. This includes a suitably flexible API for different types and arrangements of geometry, advanced shading support, advanced acceleration structures and their algorithms. We will mainly use the XML3D scene description approach as the basis for this work. We also plan to include advanced lighting simulation algorithms along the lines of our four Siggraph papers [SigAsia’12, SigAsia’13, 2x Siggraph’14], mainly focusing on the Vertex Connection and Merging approach [Georgiev et al., SigAsia’12] and its mapping to highly parallel architectures [Davidovic et al., Siggraph’14].

Several internal Intel-VCI and German Research Center for Artificial Intelligence (DFKI) projects will guide the design of AnyDSL-Ray, including DreamSpace, an EU project aiming at real-time photorealistic imaging; the Future Internet Projects for extending XML3D towards high-quality rendering solution (also in the cloud) with portable material models, Display as a Service (DaaS) for providing rendering services within DaaS, and more.

Project Team

Principal Investigator
Prof. Dr.-Ing. Philipp Slusallek