Generative Image Manifold: Drag Your GAN, Interactive Point-based Manipulation 2023

We will go into more detail about GANs in this article because people want to learn more about it. Deep generative models, such as generative adversarial networks (GANs), have shown remarkable effectiveness in generating random optical images. In real-world applications, the ability to control combined image inputs is essential for learning-based image synthesis methods. For example, social media users may want to change the position, shape, expression and body posture of a person or animal in a photo-realistic photo, professional media editors may need quickly sketch out specific scene layouts for movies, and car designers may want to interactively change the shape of their designs.

Diverse creative images

To satisfy diverse user goals, an ideal controlled image synthesis technique should have the following characteristics. 1) Flexibility: It can control many different spatial characteristics, such as the position, attitude, expression, and arrangement of produced objects or creatures. 2) Accuracy: It must be able to manage spatial characteristics accurately. 3. Generality: Must be applicable to many different types of objects, not specific to any one object. This work attempts to fully satisfy all of these characteristics, whereas previous works have only fully met one or two of them. Most legacy techniques rely on supervised learning, using manually annotated data or prior 3D models to train the GAN in a controlled manner.

Recently, the synthesis of visual instructions with text has received attention. As a result, these techniques sometimes manage a limited number of spatial characteristics or provide users with limited editing capabilities. They must also apply to new object types. However, text instructions must increase adaptability and accuracy when changing spatial characteristics. For example, it cannot be used to move an object to a specific number of pixels. The authors of this work consider a powerful but underutilized interaction point-based operation for flexible, precise, and comprehensive GAN control. The goal is to shift the handle points in the direction of the appropriate target point by clicking as many handle points and target points as desired on the image.

The method most similar to our scenario studies traction-based manipulation. The user has control over multiple partial properties using this point-based operation, independent of object categories. Compared to that problem, the problem presented in this paper has two additional challenges: They consider managing some points that their approach has difficulty in doing, and they also require points handle reaching target points precisely, something their approach cannot do. DO.

Categories: Trending
Source: Tekmonk Bio