JPH09159413A - Image processor and modeling unit - Google Patents

Abstract

PROBLEM TO BE SOLVED: To synthesize a plurality of images while matching to the actual dimensions of object in an image without inputting the dimensional information of object by calculating the dimensions of an object, present in an image being picked up, on the image. SOLUTION: An image pickup section 10 picks up the image of an object and a dimensional information measuring section 12 extracts the dimensional information from image pickup conditions and the information of image being picked up. A data preserving section 14 produces and preserves a data representative of correspondence between the information of image being picked up with the dimensional information. A data read-out section 16 reads out the preserved data while keeping the correspondence between image information and dimensional information. An image pickup section 10, i.e., an ordinary electronic camera, and means for operating the dimensional information are employed as means for extracting the dimensional information. A medium for preserving or reading data includes a floppy disc, a magnetooptical disc, etc.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and a model creating apparatus which store image information and size information of an object in association with each other and retrieve and use them.

[0002]

2. Description of the Related Art In recent years, an apparatus has been proposed which provides a simulated experience to a user by using computer graphics. As an example of this, Japanese Patent Laid-Open No. 4-192066 discloses a "commodity simulated experience showroom system" as follows. This system is supposed to display a stereoscopic image of computer graphics on a goggle type display and provide the user with a feeling as if the product is in front of the user. In this system,
The shape of each product is given as three-dimensional shape data.

Further, as an apparatus for presenting a product to a user in a state close to the usage environment thereof, for example, Japanese Patent Laid-Open No. 5-619.
No. 62, "Environmental model creation device for product evaluation"
Has the following disclosure. This environment model creating device measures the three-dimensional position information of an object (product) based on an indoor environment photograph by adding human judgment to it, and calculates the mapping position and size according to the actual shape. , Which is said to be a relocation device. This device is supposed to support a customer's decision making by observing the product or simulating matching with the surrounding interior while the product is installed in an actual use environment.

[0004]

DISCLOSURE OF THE INVENTION Problems to be Solved by the Invention
In Japanese Patent Laid-Open No. 62-62, the measurement of the three-dimensional shape and size of an object is performed based on an environmental photograph in which an environmental constituent object is present, and human judgment is added to this. Therefore, the environmental photograph taken by the conventional technique does not include dimension information, and there is no description from the viewpoint of measuring and calculating actual dimensions from the environmental photograph. Therefore, it is necessary to separately measure and input the dimensions of the object. Further, it is not described from the viewpoint of synthesizing the imaged image of the object according to the actual size.

Further, when the imaged object is captured in the model created by inputting the use environment data, the image of the object in the environment photograph has dimensional information although it is based on the environment photograph. Therefore, it is necessary to separately measure and input the dimensions of the object in order to handle the created model as it is.

The present invention has been made in view of the above problems, and by calculating the size of an object existing in the image from a captured image, the image of the object can be obtained without inputting the size information of the object. Multiple images can be combined according to the actual dimensions of the target object inside. Furthermore, without inputting the actual dimensions of the imaged object, you can enter parameters such as actual dimensions for the model created. An object of the present invention is to provide an image processing device and a model creating device capable of incorporating the imaged object.

[0007]

In order to achieve the above object, an image processing apparatus of the present invention comprises an image pickup means for picking up an image of an object, a measuring means for measuring dimensional information of the object, and the image pickup means. It is characterized by further comprising a storage means for storing the image information captured and captured by the above and the dimension information in association with each other.

Further, the image processing apparatus of the present invention comprises an image pickup means for picking up an object, a measuring means for measuring the dimension information of the object, image information picked up by the image pickup means and the dimension information. And a storing means for storing the image information and the size information stored in the storing means while keeping the corresponding relationship, and at least two or more dimensions taken out by the extracting means. A conversion unit that converts the magnification of the image information so that the unit lengths of the images in which the respective objects are captured using information are matched, and the image information converted by the conversion unit are combined. And a synthesizing means.

Further, the model creating apparatus of the present invention comprises an image pickup means for picking up an image of an object to create an image of the object, a measuring means for measuring dimensional information of the object, the image pickup means and the measuring means. From the calculation means for calculating the shape and size of the target object, the model image creation means for creating a model image equivalent to the actual environment by inputting parameters, the target object image taken by the imaging means, and the Magnification converting means for converting the magnifications of the object image and the model image so that the unit lengths of the model images created by the model image creating means are the same, and the object image is the model image. Locating means for locating the object image in the model image and arranging it at an arbitrary position, a position changing means for changing the position of the object image arranged in the model image, and a module for arranging the object image. Characterized by comprising a display means for displaying an Le images.

That is, in the image processing apparatus of the present invention, the object is imaged by the image pickup means, and the dimension information of the object is measured by the measuring means. Then, the image information captured and captured by the image capturing unit and the dimension information are stored in association with each other by the storage unit.

[0011]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the arrangement of an image processing apparatus according to the first embodiment of the present invention.

This image processing apparatus comprises an object image pickup section 10 for picking up an image of an object, a dimension information measurement section 12 for measuring the dimension of the object, and image information and dimension information measurement from the object image pickup section 10. The data storage unit 14 stores the size information from the unit 12 in association with each other, and the data read unit 16 reads out these data from the data storage unit 14.

In the image processing apparatus thus constructed, first, the object image pickup section 10 picks up an image of the object, and the dimension information measuring section 12 extracts the dimension information from the image pickup condition and the picked-up image information. . Next, data in which the captured image information and the dimension information are associated with each other is stored in the data storage unit 1.
The data created in step 4 and associated with this data saving unit 14 is saved. Then, the data stored in the data storage unit 14 is read by the data reading unit 16 while maintaining the correspondence relationship between the image information and the dimension information.

Here, a method of extracting the dimension information corresponding to the image information by the dimension information measuring unit 12 from the image pickup condition and the imaged image will be described. As a device for performing this extraction, an ordinary electronic camera as the object imaging unit 10 and a calculation means for calculating dimensional information are used.

Assuming that the focal length f of the lens system used in the electronic camera and the focal position s at the time of photographing (distance to the image pickup device behind the lens), the magnification α of the lens system is α = (s / f-1) can be represented. If the pitch of one pixel of the image sensor is d, the actual size of one pixel in the captured image is αd.

Note that the dimension information may be actual dimensions or numerical information for which the actual dimensions can be easily calculated, or values that can calculate object dimensions such as the focal length and the focal position of the optical system of the object imaging section 10. Good. By using data such as the focal length and the focal position, the function of calculating the dimensional information can be externally independent. In the data storage unit 14 or the data reading unit 16, a medium used for storing or reading data in which image information and size information are associated with each other is,
A floppy disk (FD) used as a recording medium for digital data, a magneto-optical disk, a hard disk, or the like, or another recording medium may be used.

FIG. 2 is an example of an object image picked up by the image processing apparatus, and FIG. 3 is a diagram showing a data format in which image information and size information are recorded in association with each other. FIG.
(A) is the actual size of 1 pixel pitch at the beginning of the data 1
8 shows the data format in which the actual size of the image corresponding to one pixel is recorded and the luminance information 20 of the first, second, third, ... Pixels is recorded after the head portion. In FIG. 3B, the actual size 22 of the horizontal or vertical axis of the image is recorded at the beginning of the data, and the luminance information 20 of the first, second, third ... Pixels is recorded after this beginning. FIG. 3 shows the format.
(C) shows a data format in which the number of pixels per unit length 24 is recorded in the head portion, and similarly, the luminance information 20 of the first, second, third, ... Pixels is recorded after the head portion. is there.

In FIG. 3D, arbitrary pixels A and B are set on the image, and the coordinates 26 and 28 of the pixels A and B and the actual size 30 between the pixels A and B are set from the beginning of the data. As described above, FIG. 3 (e) shows the focal length 32 of the lens system and the focal position 34 of the captured image recorded at the head. Note that the object shown in FIG. 2 is actually a three-dimensional object, but here it is treated as two-dimensional without considering depth information.

FIG. 4 is an example of an object image stored as a three-dimensional image, and the coordinate system of the image is shown by x, y, z. FIG. 5 is a diagram showing a data format in which a three-dimensional image as shown in FIG. 4 is recorded in association with image information and size information. In this data format, the actual size 40 corresponding to the unit length of the world coordinate system, the viewpoint position 42, and the line-of-sight direction 44 are recorded in order from the beginning of the data, and further the image data represented by the world coordinate system 46 on the rear side thereof. Are recorded as 1st, 2nd, 3rd ... In addition,
Instead of the actual size 40 corresponding to the unit length of the world coordinate system, the number of pixels corresponding to the unit length of the world coordinate system may be used. The world coordinate system specifies the position of an object point in three-dimensional space coordinates.

As described above, according to the first embodiment, the image information and the dimension information of the object are stored in association with each other, and the image information and the dimension information are read out and the object existing in the image is read. The size of the object can be known, and the actual size of the object existing between the images can be compared.

Further, by using the world coordinate system, the image information of the object in the three-dimensional space can be stored or read out in association with the size information thereof. Next, an image processing apparatus according to the second embodiment of the present invention will be described.

FIG. 6 is a block diagram showing the arrangement of the image processing apparatus according to the second embodiment. This image processing device
Corresponds the object image pickup unit 10 that picks up an image of the object, the dimension information measurement unit 12 that measures the dimensions of the object, the image information from the object image pickup unit 10 and the dimension information from the dimension information measurement unit 12. A data storage unit 14 that additionally stores the data, a data read unit 16 that reads a plurality of image information and dimension information corresponding thereto from the data storage unit 14, and a scaling factor that scales the image information so as to match the dimension information. The image conversion unit 50 includes an image conversion unit 50, an image storage unit 52 that stores image information whose magnification has been converted, and an image composition unit 54 that combines the image information stored in the image storage unit 52.

FIG. 7 is a diagram showing an example of an image processed by the image processing apparatus. As in the first embodiment, the image processing apparatus captures an image of an object by the object imaging unit 10 and extracts dimension information from the imaging condition and image information captured by the dimension information measuring unit 12. Then, the data storage unit 14 creates data in which the image information and the size information are associated with each other, and the data storage unit 14 stores the associated data.

Next, the data reading section 16 reads a plurality of images having a correspondence relationship between the image information and the dimension information from the data storage section 14. At this time, the plurality of data images read out correspond to those of the automobile 5 shown in FIG.
6 and a garage 58 shown in FIG. 7B. The screen shown in FIG. 7 is composed of 512 × 256 pixels in the horizontal and vertical directions, and the actual dimensions corresponding to one pixel are 1.0 and 1.4, respectively. The unit used is cm.

Further, the magnification information conversion unit 50 extracts the dimension information, and performs magnification conversion of the image information so that the length per unit of the dimension information matches between the image information. That is, for example, the automobile 56 of the image shown in FIG. 7A is reduced to 1 / 1.4, and the garage 58 shown in FIG. 7B is held without being scaled. Thereafter, these images are stored in the image storage unit 52, and the image combining unit 9 aligns the position of the reduced automobile 56 of FIG. 7A and the position of the garage 58 of FIG. As shown in c), an image of the automobile 56 in the garage 58 is created. The garage steel frame portion of the automobile 56 is subjected to a rear surface treatment so as to fit inside the garage 58. The images combined as described above need only correspond to the actual dimensions of the objects existing in the images, and the data format stored in the image storage unit 52 may be any of the formats shown in FIG. Good.

Incidentally, in the image magnification conversion in the magnification information conversion section 50, the automobile 56 shown in FIG. 7A is reduced in order to make the dimensions of the two images match.
The garage 58 shown in (b) may be enlarged.

As described above, according to the second embodiment, the image information and the dimension information of the object are stored in association with each other, and the image information and the dimension information are read out and the object existing in the image is read. The size of the object can be known, and the actual size of the object existing between the images can be compared.

Further, it is possible to synthesize images having different photographing magnifications in sizes according to actual dimensions, that is, to synthesize images photographed at different magnifications on a scale based on actual dimensions. Further, since each image has corresponding size information, the size information can be used to convert the magnification to match the actual size and to combine the images.

Next, a model creating device according to a third embodiment of the present invention will be described. FIG. 8 is a block diagram showing the configuration of the model creating device according to the third embodiment.
This model creating apparatus is a three-dimensional model that takes in an object measurement function that takes in an image and size information of an object, and creates a model using actual dimensions and takes in the object measured by the object measurement function in this model. It includes a creation function and a model display function for displaying the model created by the three-dimensional model creation function.

The object measuring function includes an object image pickup section 10 for picking up an image of an object, a dimension information measuring section 12 for measuring the size of the object, and an object shape for measuring a two-dimensional image or a three-dimensional shape of the object. The recognition unit 60 is included. In particular, as a method for measuring a three-dimensional image of an object, a stereo method using a parallax using a twin-lens camera, a moving image in which a moving image is taken and at the same time, shooting conditions, that is, a position, a direction, a focus condition, etc. are monitored. Method, and a method of measuring while changing the focal length of the lens of the imaging unit. Further, there are a method of irradiating a slit light in a grid pattern and measuring the pattern thereof, a method of using an image moire, and the like.

Next, the model creating function will be described. The model creation function stores a data storage unit 14 that stores image information from the object imaging unit 10 and dimension information from the dimension information measurement unit 12 in association with each other, and data associated from the data storage unit 14. A data reading unit 16 for reading,
A magnification information conversion unit 50 that performs magnification conversion of image information so as to match the size information, an image storage unit 52 that stores the image information output from the magnification information conversion unit 50, and actual size or assumption of the object. Dimension parameter input unit 62 for inputting parameters for reproducing the model, a model creation unit 64 for creating a model based on the parameters received from the dimensional parameter input unit 62, and the image storage unit 52. The image synthesizing unit 54 synthesizes the created image and the model created by the model creating unit 64.

The model display function is performed by the image synthesizing unit 54.
The model display unit 66 that displays the image that is combined in. The model creating apparatus configured as described above is the second
In the same manner as in the above embodiment, an image of the target object is captured by the target object imaging unit 10, dimension information is extracted from the imaging condition and the captured image information by the dimension information measuring unit 12, and the captured image information and the dimension information are acquired. Data corresponding to and is created and the data is stored in the data storage unit 14.

Next, the data reading section 16 reads a plurality of images having a correspondence relationship between the image information and the dimension information from the data storage section 14. Further, the size information is extracted by the magnification information conversion unit 50 so that the size information matches the size information and the size parameter input unit 6
Based on parameters such as actual dimensions input from 2, image information magnification conversion is performed so that the magnification is the same as that of the model created by the model creation unit 64, that is, the dimension information matches. The magnification information conversion unit 50 may perform the magnification conversion on the image information captured by the object imaging unit 10 based on parameters such as actual dimensions input from the dimension parameter input unit 62. . The image whose magnification is converted by the magnification information conversion unit 50 is stored in the image storage unit 52.

The model creating section 64 creates a model based on the actual dimensions input from the dimension parameter input section 62. Further, the image information saved in the image saving unit 52 and the model created by the model creating unit 64 are arbitrarily changed in position and orientation by the image combining unit 54 and are combined. The combined image information and model are
It is displayed on the model display unit 66.

The parameters input to the dimension parameter input unit 62 include the position of the object and its size, that is, the height, width, height, and color of the object, but are not limited thereto. Any parameter may be used as long as it is necessary to reproduce the assumed model.

As described above, according to the third embodiment, by storing the image information and the size information of the object in association with each other, the image information and the size information are read out and the object existing in the image is read. The size of the object can be known, and the actual size of the object existing between the images can be compared.

Further, since the image information and the size information correspond to each other, it is possible to grasp the size relationship between the object and the model image at an arbitrary position. In addition, the set position of the imaged object can be freely changed within the layout.

Further, according to the model creating apparatus, it is possible to create a model room such as a house and perform a simulation of arranging furniture that actually exists. In the present embodiment, since the simulation is performed based on the actual size, it is possible to study the layout of the furniture, for example, to diagnose whether or not the furniture fits in a certain gap. Also, by imaging the furniture in your own house with this model creation device,
Floor plans for room remodeling can be modeled based on actual dimensions, which can be useful in planning room remodeling.

FIG. 9 is a diagram showing an example of a specific configuration of the image processing apparatus and the model creating apparatus of the above embodiment. The image processing apparatus and the model creating apparatus include a camera 70 for picking up an image of an object, a data processing unit 72 for calculating dimension information of the measured image, a computer 74 for controlling the entire image processing apparatus and model creating apparatus, It comprises a data storage device 76, an image display device 78, and a parameter input device 80.

A binocular camera is useful as the camera 70 in order to image and process a three-dimensional object as an object. Further, the camera 70 is equipped with a position sensor 70a for measuring its own position and a wireless transmitter 70b. Then, the image information and the position information of the object are transmitted by the wireless transmitter 70.
b, and is received by the data processing unit 72 having a data receiver 72a. Although it is not necessary for the data communication method to be wireless in particular, if wireless is used, an object existing in a wide range can be imaged with other devices fixed.

The computer 74 creates a three-dimensional space model, takes in an object, and changes the installation position of the model on the image display device 78. In the parameter input device 80, actual dimensions necessary for model creation are input as parameters. In this parameter input device 80,
A keyboard, mouse, or joystick is used. The image display device 78 includes a CRT, a liquid crystal display, a head mounted display (HMD).
Etc. can be used. In addition, the data storage device 76,
A hard disk, an optical disk, a RAM, etc. are used.

FIG. 10 is a diagram showing an example of an image processed by using the model creating apparatus of the above embodiment. In this figure, FIG. 10B shows a table 82 imaged by the camera 70, and FIG. 10C shows a lighting fixture 84 imaged by the camera 70. These images are taken from a plurality of directions, and complete three-dimensional shape data is created by the data processing unit 72 for each of the table 82 and the luminaire 84.

Next, the parameter is input to the parameter input device 80 by the operation of the operator, and the parameter is input as shown in FIG.
A room model as shown in is created. In order to create such a room model, actual dimensions such as room size, ceiling height, door size and installation position, window size and installation position, etc. are used as the parameter input device. It is input to the computer 74 via 80.

FIG. 10 (d) was synthesized by incorporating the table 82 shown in FIG. 10 (b) and the lighting fixture 84 shown in FIG. 10 (c) into the room model shown in FIG. 10 (a). It is an image. Since the three-dimensional shape data of the table 82 and the lighting equipment 84 is grasped, it is possible to deal with the movement of the position and the change of the installation direction in the screen shown in FIG.

According to the above embodiment of the present invention, the following configuration can be obtained. (1) Image capturing means for capturing an image of an object, measuring means for measuring the dimension information of the object, and storage means for storing the image information captured by the image capturing means and the dimension information in association with each other. An image processing apparatus comprising: (2) Image pickup means for picking up an image of the object, measuring means for measuring the dimension information of the object, and storage means for storing the image information captured by the image pickup means and the dimension information in association with each other. , The image information and the dimension information stored in the storage means are taken out while keeping the corresponding relationship, and at least two or more pieces of the dimension information taken out by the extracting means are used to identify each of the objects. It is provided with a conversion unit that converts the magnification of the image information so that the unit lengths of the captured images match, and a combining unit that combines the image information converted by the conversion unit. Image processing device. (3) An image pickup means for picking up an image of an object to create an object image, a measuring means for measuring dimensional information of the object, and a shape and size of the object from the image pickup means and the measuring means. A calculation means for calculating, a model image creation means for inputting parameters and creating a model image equivalent to an actual environment, an object image taken by the imaging means, and a model image created by the model image creation means. Magnification converting means for converting the magnification of the object image and the model image so that the unit lengths match between the images, and the object image is taken into the model image,
Arrangement means for arranging at an arbitrary position, position changing means for changing the position of the object image arranged in the model image, and display means for displaying the model image on which the object image is arranged. A model creating device characterized in that (4) The image processing device according to (1), wherein the image information is composed of a set of pixels, and the size information is an actual size of an image corresponding to one pixel. (5) The image processing device according to (1), wherein the image information is described in a world coordinate system, and the size information is an actual size for a unit length of the world coordinate system.

[0046]

As described above, according to the present invention, the size of an object existing in the image is calculated from the picked-up image, so that the size information of the object can be calculated in the image without inputting the size information of the object. It is possible to combine multiple images according to the actual size of the target object, and further, without inputting the actual size of the imaged target, for the model created by inputting parameters such as actual size, It is possible to provide an image processing device and a model creation device capable of incorporating the imaged object.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an image processing apparatus according to a first embodiment.

FIG. 2 is a diagram showing an example of an object image captured by the image processing apparatus.

FIG. 3 is a diagram showing a data format in which image information and dimension information are associated and recorded.

FIG. 4 is a diagram showing an example of an object image stored as a three-dimensional image.

5 is a diagram showing a data format in which a three-dimensional image as shown in FIG. 4 is recorded in association with image information and dimension information.

FIG. 6 is a block diagram showing a configuration of an image processing apparatus according to a second embodiment.

FIG. 7 is a diagram showing an example of an image processed by the image processing apparatus.

FIG. 8 is a block diagram showing a configuration of a model creating device according to a third embodiment.

FIG. 9 is a diagram showing an example of a specific configuration of the image processing apparatus and the model creating apparatus of the above-described embodiment.

FIG. 10 is a diagram showing an example of an image processed using the model creating device according to the embodiment.

[Explanation of symbols]

10 ... Object imaging unit, 12 ... Dimension information measuring unit, 14 ... Data storage unit, 16 ... Data reading unit, 50 ... Magnification information conversion unit, 52 ... Image storage unit, 54 ... Image combining unit, 56 ...
Car, 58 ... Garage, 60 ... Object shape recognition unit, 62
... Dimension parameter input section, 64 ... Model creation section, 66 ...
Model display unit, 70 ... Camera, 70a ... Position sensor, 7
0b ... Wireless transmitter, 72 ... Data processing unit, 72a
... data receiver, 74 ... computer, 76 ... data storage device, 78 ... image display device, 80 ... parameter input device.

Claims (3)

[Claims] 1. An image pickup unit for picking up an image of an object, a measuring unit for measuring the dimension information of the object, and a storage for storing the image information captured by the image pickup unit and the dimension information in association with each other. An image processing device comprising: 2. An image pickup means for picking up an image of a target object, a measuring means for measuring the dimension information of the target object, and an image information captured by the image pickup means and the size information are stored in association with each other. Means, an extracting means for extracting the image information and the dimension information stored in the storing means while keeping the correspondence relationship, and at least two or more of the dimension information extracted by the extracting means for each target. A conversion unit for converting the magnification of the image information so that the unit lengths of the images of the objects are matched, and a combination unit for combining the image information converted by the conversion unit. An image processing device characterized by: 3. An image pickup means for picking up an image of an object to create an object image, a measuring means for measuring dimensional information of the object, a shape and a size of the object from the image pickup means and the measuring means. Calculating means for calculating the height, a model image creating means for creating a model image equivalent to an actual environment by inputting parameters, an object image taken by the imaging means, and a model created by the model image creating means Magnification conversion means for converting the magnification of the object image and the model image so that the unit lengths of the images match, and the object image is taken into the model image, and an arbitrary position Arranging means for arranging the object image, position changing means for changing the position of the object image arranged in the model image, and display means for displaying the model image in which the object image is arranged Model generating apparatus characterized by comprising a.