JPH0560148B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION Field of Industrial Application The present invention relates to an apparatus for creating a new image by generating and composing a plurality of images, and more specifically, an apparatus for creating a new image by generating and composing a plurality of images. The present invention relates to a device that generates an image based on a stored image of the object, and generates a composite image observed by changing the viewpoint while preserving the relative relationship between the images.

Configuration of conventional example and its problems This device is used for selecting products such as costumes and furniture, for example. In the case of furniture, for example, images of a television receiver, sofa, sideboard, etc. are combined and replaced with an image of the room to create an image that looks as if the product the customer is currently selecting is installed. It is also applied to making selection easier.

In order to generate images that change the size and relative position of multiple target objects observed from many directions, as well as change the viewpoint of the observer, it is necessary to memorize images actually observed from many directions. Alternatively, there is a method of storing all three-dimensional shapes of the target object and generating and combining them based on the data, but both have the disadvantage that the amount of image data to be stored is enormous. In particular, in the latter case, for the reasons mentioned above, only simple structures can be used in practice.

In addition, when combining multiple target objects on the same screen, for example, the target object and the corresponding point in the memory image,
Information indicating individual characteristics such as the viewpoint and shooting conditions when the target object is imaged with an image input device, the size and position of the target object, and their interrelationships is required. There are cases where it is necessary to perform a large number of operations to match an object with a stored image or to determine the positional relationship between a plurality of target objects.

Purpose of the Invention The present invention classifies viewing viewpoints, reduces the image storage capacity for each, allows the operator of the device to input necessary information, and recognizes the structure of the target object based on that information. Synthetic image generation that can generate and synthesize images of multiple target objects observed from different directions, and generate complex composite images in response to changes in the relative relationships of multiple target objects and changes in the observer's viewpoint. The purpose is to provide a device.

Configuration of the Invention In order to achieve the above object, the present invention provides a reference image memory that stores a reference image, an addition image memory that stores an addition image, and a three-dimensional shape of a target object included in the images stored in these image memories. An image/shape correspondence device that sets and defines a shape model corresponding to the target object, a distance and a direction when the target object is stored, and stores the stored image and the shape model in association with each other; The distance and direction of the target object from the viewpoint are set, and the target object in the stored image is determined from the difference between the distance and direction from the viewpoint at the time of memorization and the distance and direction from a different viewpoint from the time of memorization. converting the image into an image from a different viewpoint than when it was stored;
The amount of deformation is calculated from a magnification ratio that represents how many times the size is to be made, a moving distance that represents how much the position is changed, and a rotation angle that represents how many times the position is rotated, and based on the amount of deformation, each of the above a geometric transformation device that performs geometric transformation of a stored image according to the shape model;
an image display device that synthesizes and displays the output of the image/shape correspondence device and the output of the geometric transformation device, and generates a composite image observed from a different viewpoint from that at the time of capturing each of the stored images. This is what I did.

DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments of the present invention will be described based on the drawings. The drawing shows essential blocks of an embodiment of the present invention. Reference image memory 1 and addition image memory 2
(collectively referred to as image memories) each store a stored image of a reference image or an added image of the target object. The stored images in the image memories 1 and 2 are combined and displayed on the image display device 13 via the image/shape correspondence device 14 or the geometric transformation device 15.

Next, in order to generate an image of the target object, the shape model generator 3 generates a shape model signal closest to the shape of the target object in response to the output of the image memory 1 and/or 2. For this purpose, for example, basic geometrical shape models such as a rectangular parallelepiped, cylinder, cone, triangular pyramid, etc. are stored in the shape model generator 3 in advance, and the operator observes the target object and determines the basic geometrical shape closest to it. This is achieved by selecting a model. For example, if a television receiver is considered as the target object, a rectangular parallelepiped shape model is selected. Next, a shape model signal for the selected shape model is generated by the shape model generator 3, and this shape model signal passes through the shape model calculation device 4 and is combined with the image signal of the stored image in the image memory 1 and/or 2. The signals are superimposed by the adding circuit 7 and displayed on the image display device 13.

That is, the shape model calculation device 4 calculates the amount of three-dimensional deformation of the image displayed on the image display device 13 according to the control signal from the feature input device 6, and calculates the amount of three-dimensional deformation of the image displayed on the image display device 13, and size,
Then, the three-dimensional position is considered based on the geometric shape value of the target object, and the surface of the shape model and the image position of the stored image are matched. Here, the amount of deformation refers to the need to change the size, move, or rotate the memorized image in order to make the memorized image that memorizes the target object at the time of imaging into a visible target object when observed from a different position or angle. This corresponds to the magnification ratio, moving distance, and rotation angle at that time. The amount of deformation is calculated by comparing the distance and direction from the viewpoint of the target object when the image is captured and the distance from the viewpoint and the direction of the target object when observed from a different position or angle. This shape model signal is added with deformation information by the shape model calculation device 4 and is transferred to the addition circuit 7. The operator of this device manipulates the projected image of the basic geometric shape model to create a shape model that is most in contact with the target object. The creation of a shape model with the largest number of contact points is performed by the operator inputting a plurality of positions of corresponding points between the stored image and the shape model using the feature input device 6, which is comprised of a joystick, a mouse, a tablet, or the like. The feature input device 6 inputs the size of the target object, for example, in the case of a conical model, the radius and height, the attribute of transparency,
And the image input device is configured to input conditions such as zoom, distance, angle, etc. when capturing an image of the target object. A shape model is created in this manner, and the shape model is stored in the shape model storage device 5 in correspondence with the stored image of the target object. Of course, in the case of a complex-shaped target object, a plurality of shape models can be associated with it. This device regards the shape of the target object as the shape of the shape model stored in the shape model storage device 5, and regards the patterns (textures) on each surface of the object as the stored images stored in the image memories 1 and 2. perform a function. The addition circuit 7 superimposes the image signal of the stored image and the shape model signal.

Next, the shape model stored in the shape model storage device 5 is converted into the visual coordinate transformation devices 8 and 9 of the geometric transformation device 15.
Coordinate transformation is applied. the visual coordinate conversion device 8;
Reference numeral 9 indicates that the operator of this device can change the target visual coordinate conversion devices 8 and 9 independently of the viewpoint selected when capturing the reference image in the reference image memory 1 and/or the added image in the added image memory 2. By inputting a desired viewpoint using the visual coordinate input device 12 and transmitting a control signal, the target object is converted into an image observed from that viewpoint based on the shape model signal. The stored image that has passed through the visual coordinate conversion devices 8 and 9 is sent to an addition circuit 1.
Synthesized with 0. For example, one input of the addition circuit 10 composed of a multiplication circuit and a summation circuit is used as a reference image, and the other input is used as an addition image. The input signals of the reference image and the addition image include the density of each point and, if defined, the shape model and the distance and direction from the viewpoint. To synthesize these, the density of the image seen in front when changing the viewpoint is used as the density of each point. The addition circuit 10 compares the distances between the reference image and the addition image at each point, and adds the one closer to the viewpoint to the output image. The viewpoint from which the entire image is observed can be changed by further performing coordinate transformation on the synthesized image output using the visual coordinate transformation device 11 to determine the magnification, movement distance, and rotation angle. for example,
It has the excellent advantage that a room in which furniture is installed can be observed from the front of the room or from other rooms, and the customer's ideas can be concretely expressed. The image display device 13 converts the input image signal into an analog quantity and displays it on, for example, a cathode ray tube of a television receiver. The output of the visual coordinate conversion device 11 and the output from the adder circuit 7 of the image/shape correspondence device 14 can be connected to the image display device 13 and combined and displayed on the image display device 13.

When this device is applied to interior design as an application example, a composite image of a certain piece of furniture and a room selected as an added image can be stored in the reference image memory 1 as a reference image, and then another added image can be stored as a reference image. By combining selected pieces of furniture with other furniture, the decoration of the room can be completed, and by repeating this process, a large number of images can be combined with a simple circuit configuration.

Effects of the Invention As described above, the present invention can generate memorized images of multiple target objects with simple operations based on shape information, and can interactively create a composite image that does not feel strange. In addition to having the excellent effect of
The target object can be observed from various viewpoints,
It has the advantage that concrete expression can be obtained in a short time.

[Brief explanation of the drawing]

1 is a main part block diagram showing an embodiment of the present invention in the drawings. FIG. 1...Reference image memory, 2...Additional image memory, 3...Shape model generator, 4...Shape model calculation device, 5...Shape model storage device, 6...
Feature input device, 7...Addition circuit, 8...Visual coordinate conversion device, 9...Visual coordinate conversion device, 10...Addition circuit, 11...Visual coordinate conversion device, 12...Visual coordinate input device, 13... ...Image display device, 14...Image/
Shape correspondence device, 15... Geometric conversion device.

Claims (1)

[Scope of Claims] 1. A reference image memory that stores a reference image, an addition image memory that stores an addition image, a shape model corresponding to a three-dimensional shape of a target object included in images stored in these image memories, An image/shape correspondence device that sets and defines the distance and direction when the target object is stored, and stores the stored image in association with the shape model, and the distance of the target object from a different viewpoint from when it was stored. and a direction, and from the difference between the distance and direction from the viewpoint at the time of memorization and the distance and direction from a different viewpoint from the time of memorization, the target object in the memory image is different from the time of memorization. In order to convert the image from the viewpoint, we calculate the amount of deformation, which consists of a scaling factor that shows how many times the size should be made, a moving distance that shows how much the position should be changed, and a rotation angle that shows how many times the image should be rotated. , a geometric transformation device that performs geometric transformation of each of the stored images according to the shape model based on the amount of deformation; and a composite display of the output of the image/shape correspondence device and the output of the geometric transformation device. and an image display device that generates a composite image observed from a different viewpoint from that at which each of the stored images is captured. 2. The image/shape correspondence device is configured to include a feature input device, a shape model generator, a shape model calculation device, and a shape model storage device, and the shape model generator has a plurality of basic geometric shape models stored in advance. The basic geometric shape model that is stored, selected by the operator from among them, and designated from the feature input device is used by the shape model calculation device to calculate the geometric shape of the target object input by the operator from the feature input device. A shape model is set by changing the size based on the value, a surface of the shape model is associated with an image position of the stored image, and the result is stored in a shape model storage device. A composite image generation device according to claim 1. 3. The composite image generation device according to claim 2, wherein the shape model calculation device performs perspective transformation of the shape model, and the output thereof and the stored image can be displayed simultaneously. 4. A geometric transformation device that generates an image obtained by observing a stored image from a viewpoint different from that at which the target object was imaged is configured to include a viewing coordinate input device and a viewing coordinate conversion device, and the viewing coordinate input device inputs the coordinates and direction of the viewpoint. and the visual coordinate conversion device is a device that converts the stored image into a coordinate system of the viewpoint selected by the visual coordinate input device. Composite image generator. 5 a visual coordinate conversion device that converts the stored image into the coordinate system of the viewpoint selected by the visual coordinate input device;
Claim 1 is characterized in that the reference image, the added image, and the composite image are provided separately, or the input of the visual coordinate conversion device is switched between the reference image, the added image, and the composite image. Composite image generator.