PENERAPAN STYLE TRANSFER PADA OBJEK VIDEO DENGAN MENGGUNAKAN KOMBINASI STABLE DIFFUSION DAN CONTROLNET SERTA LOW-RANK ADAPTATION

Abstract

With the advancement of technology, there has been a significant transformation in human creative processes, particularly in the fields of art and design. Various software and hardware tools are now available to support digital exploration and expression of ideas; however, their use often requires specialized skills and substantial costs. One of the main challenges in video production is maintaining visual style consistency, especially during post-production stages such as visual effects and colour grading. This study aims to implement a style transfer technique based on the Stable Diffusion model combined with ControlNet to achieve precise artistic style transformation on each video frame. Additionally, the Low-Rank Adaptation approach is employed to fine-tune the Stable Diffusion model using activation tokens to generate images in the target style. This approach successfully performed style transfer and produced images with SSIM scores of 0.7058, 0.7772, and 0.7703 at ControlNet Strength values of 1.0, 0.8, and 0.6, respectively.