Procedural Generation with Applied Intuition and Toyota

Originally published on Medium

At Applied Intuition we are building software infrastructure to safely develop, test, and deploy autonomous vehicles (AV) at scale. Applied Intuition’s extensive integrated development environment enables customers to simulate every component of their tech stack.

While much of what we are working on is more traditional simulation for the motion planning, controls, and perception modules of an AV’s software stack, we have also been building large scale simulated environments.

Our team has created a pipeline for generating environments procedurally (automatically created) that is extensible to many different use cases. For use cases that require environments to be both generated and modified quickly, there is no substitute for procedural generation.

Using this approach we recently worked with the UX team of Toyota Research Institute — Advanced Development (TRI-AD) to create realistic environments for the Lexus LF30 virtual reality (VR) experience at the Tokyo Motor Show.

The Purpose

The goal for Applied Intuition and TRI-AD’s UX team was to develop a rapid iteration user experience framework in which a vehicle’s user interfaces (UI) could be tested in VR. The framework was also designed to operate in cooperation with an articulated six degrees of freedom test setup. This setup was then demonstrated at the Tokyo Motor Show, with the Lexus LF30.

Lexus LF30 Vehicle

Procedural environment generation was critical for this application as environments could be quickly adjusted and tuned for optimal user experience.

The Drive

The environment was modeled based on the look of Barcelona and was iterated to give passengers a feeling of general comfort within the vehicle. The full simulation was required to run at 90 frames per second, including the vehicle rig, the world rendering, the custom UI in the vehicle and several other features. The environment was designed to mimic the Barcelona backdrop of the original vehicle reveal video.

Applied Intuition helped in making the world configurable, allowing the track to be continuously modified and updated. This made it possible to tune the route intensity for passenger comfort, as well as optimize sight-lines for both visual purposes and performance management. The toolset also allowed for on the small changes to smaller environment features such as the density of vegetation and rocks. Creating a new version of the track took approximately 3 minutes to calculate and import, allowing for extremely fast iteration cycles.

The Tech

The project was built in Unity, with assets place procedurally to ensure the world could be altered up until the final days before the demonstration. The procedural placement was done with Houdini and the Houdini engine.

Houdini Engine Controls Within Unity

The Houdini Engine allows us to place instances in an efficient manner. Applied Intuition made controls that allowed for tuning the number of instances live in engine, as well as providing tools to change things like orientation and random seeds scale distributions. Having this level of adjustability meant that different parts of the track could be adjusted for performance live, without having to go back and forth between Houdini and Unity. Performance was maintained while also being able to improve the visual quality in sections of the drive where there was performance headroom.

The procedural setup allowed for various controls, such as the terrain height, the path of the road (which will affect the terrain height locally) the camber of the road, the banking of the road etc. Below is an example of two different road paths that were tested (notice the differences in the red mesh shading around the edges of the road). The driving scenario can be made more or less intense with a quick modification to the input curve.

Iterating on the Design of the Drive Route (Top), Leading to the Final Product (Bottom)

The experience ended with a vista point, which was continuously moved and adjusted to achieve the best views and user enjoyment.

Vista Point at the End of the Drive

The Final Product

We would like to congratulate TRI-AD’s UX team and Lexus on a very well received demonstration, the interior of the Lexus LF30 Electrified concept was one of the most discussed topics of the Tokyo Motor show, and we’re proud to have a part in letting people experience that sensation. Thank you to TRI-AD’s UX team for an incredible experience.

The Applied Intuition Team