Multi-Fidelity Optimization for Inverse Material Rendering

Image quality metrics are typically used as loss functions for inverse material rendering. However, all these metrics have drawbacks. When optimizing with low-fidelity (LF) metrics, shorter runtimes can be achieved, but this also produces lower quality results. On the other hand, optimizing using high-fidelity (HF) metrics produces higher quality results, but causes significantly longer runtimes. We present an alternative method of applying image quality metrics to inverse material rendering by using multi-fidelity optimization (MFO). This optimization method is a two-stage process, where an LF optimization is utilized to quickly generate the starting point for an HF optimization. By evaluating the MFO approach for multiple material optimization scenarios, we show that this method produces high quality results, while having shorter runtimes than optimizations applying HF metrics. Given the efficacy of applying multi-fidelity optimization to inverse material rendering, we expect this technique to be useful for any optimization applying image quality metrics, encouraging a broader evaluation of MFO in inverse rendering.