JPH0831126B2 - Method of creating composite image in spectacle wearing simulation apparatus - Google Patents

Abstract

PURPOSE:To reduce the difference of impressions between a person who actually puts on his/her spectacles and a synthetic image of the person and the spectacle frame, by displaying said synthetic image on a display device based on a synthetic image of an unspectacled person displayed on a display device. CONSTITUTION:The frame synthetic magnifying power and the synthetic position are calculated for production of a synthetic image of a person 2 and a spectacle frame via a CPU 7, a spectacle frame data memory 8 and a program memory 9. Thus synthetic image data on the person 2 and the frame is obtained and this synthetic image is displayed on a display device 12 via a D/A converter 6. Thus an optional picture size is correctly synthesized and the person 2 is photographed in a picture size in response to the purpose of application. Then the spectacle frame is synthesized with a correct position, size and tilt so that the difference of impressions can be reduced between a spectacled synthetic image of the person 2 and the person 2 who actually puts on his/her spectacles.

Description

Description: TECHNICAL FIELD The present invention relates to a synthetic image creating method in a spectacle wearing simulation device. That is, the present invention creates an image of a spectacle image for a person image not wearing spectacles,
The present invention relates to a synthetic image creating method in a spectacle wearing simulation device for displaying a synthetic image for wearing spectacles.

(Prior Art) Conventionally, when purchasing eyeglasses at an eyeglass store or the like,
When selecting a desired spectacle frame separately from the prescription, in order to know the condition of one's face on which the frame is applied, it is performed by directly applying the frame and looking at one's face reflected in a mirror.

However, for people with strong myopia, astigmatism, etc., if the glasses are removed, the image will be blurred even at a fairly close position. Therefore, it is impossible to correctly know the condition of one's face when wearing only the spectacle frame. For this reason, a device described in Japanese Patent Application Laid-Open No. 61-80222 has been proposed as a spectacle wearing simulation device capable of knowing the wearing state of the spectacles while wearing them. This method uses an image synthesizing processing technique, and a person image without glasses is taken by a video camera, and an image pickup signal of the person image is AD-converted and taken into an image storage device as a digital signal. The display signal is created by combining the data of the person image and the data of the eyeglass frame stored in the device, and the state in which the person wears glasses is displayed on the display device to perform eyeglass matching.

In the device, when a person image is stored, the person image captured by the video camera is displayed at a predetermined display position on the display device, and the face of the person is also displayed at a predetermined display position on the eyeglass frame. The video camera is operated so that it looks like an image.

This is because the ability of the computer may be smaller if the size, position, and angle of the image of the spectacle frame are fixed. Therefore, a square frame indicating the position of the eyeglass frame is displayed on the display device, and while the display signal of this square frame and the image pickup signal from the video camera are combined and displayed, the video camera is moved to move to this frame. The image is taken so that the part of the face on which you wear glasses is inserted. This eliminates the need to correct the data of the spectacle frame based on the size, position, and inclination of the face, and also reduces the ability of the computer side.

Furthermore, the method transfers the data of the eyeglass frames to the image recording device from another storage device and combines them.

(Problems to be solved by the invention) However, in this method, the image of the person captured by the video camera is the image of the person without glasses, and the person wearing the glasses for vision correction removes the glasses. Then you will see the display device. In this case, the person has a blurred image,
Since it is not possible to gaze clearly, the face cannot be aligned on the display device by itself.

Even if a person with relatively good eyesight is facing the video camera and looking at the display device, the video camera and the display device are not at the same position. Will be done.

Further, in general, when a person faces a camera or the like and the image is taken, the human face does not stand still horizontally and has a habit of slightly tilting the face to the left or right. When displaying a composite image The size of the eyeglass frame,
Even if the position (tilt) is slightly shifted, it may cause discomfort, and it may be quite different from the case of wearing an actual eyeglass frame, and it is difficult to display an accurate composite image of a person image and an eyeglass frame image. It was.

Further, in the method of the above-mentioned JP-A-61-28022, it is necessary to accurately display a person image in a designated screen size, and therefore the video camera is moved to adjust the size of the person in order to photograph the person. Since it must be done, a photographer or a remote control device capable of adjusting a photographing position, an angle and the like is required.

That is, in order to capture the imaged face in the frame of the eyeglass frame position of the display device screen, the size of the face must match the position of the fixed display screen.

For that purpose, it is necessary to make precise and fine adjustments, such as moving the photographing means such as a video camera back and forth, and moving it left and right, and the adjustment work is complicated and It takes time.

The present invention eliminates such drawbacks. First, when capturing an image with a camera, the precise position of the subject, the inclination of both eyes with respect to the horizontal line of the display screen, the size of the fixed screen and the size of the imaged person. This makes it possible to capture a human image with a natural expression without the need for matching, and secondly, to accurately combine the position, inclination, and size of the image of the spectacle frame with the human image.

(Means for Solving Problems) The present invention includes a CPU, an AD converter, an image storage memory, a DA converter, a spectacle frame data memory loaded at a first combining magnification, a program memory, and an operating means. A computer, an image pickup device, a display device, and a distance measuring means,
The computer creates a composite image of the image data of the spectacle frame captured at the first composite magnification and the image data of the human image from the image capturing device, and the computer displays the image on the display device. A method for creating a composite image in a spectacle wearing simulation device that displays the state of wearing spectacles, which is detected from a still image on a display screen of a person who does not wear spectacles and is taken into the computer by the imaging device. Based on the corneal vertex position coordinates of the left and right eyes and the distance information between the person and the imaging device measured by the distance measuring means, a second composite magnification of the display image on the display screen for the actual person is calculated. ,
Calculating a composite position of the eyeglass frame on the display screen and a third composite magnification of the human image and the eyeglass frame image based on the second composite magnification, and displaying the composite image of the person and the eyeglass frame on the display device. The present invention provides a method for creating a composite image in a characteristic eyewear wearing simulation device.

(Operation) A spectacle wearing simulation device of the present invention is a computer having a CPU, an AD converter, an image storage memory, a DA converter, a spectacle frame data memory loaded at a first combining magnification, a program memory, and an operating means. A video camera and a display device are provided with distance measuring means, and a person takes an image with the video camera without glasses. Therefore, the person to be imaged without glasses need only look at the gaze target and need not pay attention to the position and inclination of the face.

Next, the image pickup signal of the person image is taken into the image memory through the A / D converter and is displayed as a still image on the display device through the D / A converter. From this display image, the corneal vertex coordinates (X _L , Y _L ) of the left and right eyes are calculated according to the dot numbers in the X and Y directions.
(X _R , Y _R ) is detected. From these data, the inclination θ of the straight line connecting the corneal vertex positions of the left eye and the right eye with respect to the X direction (horizontal direction) of the display device is And the tilt angles of both eyes are calculated. The midpoint (center coordinate) of the corneal vertex position coordinates of the left and right eyes (X ₀ ,
From Y ₀ ), the rotation center reference of the frame for rotation of the frame image combination can be obtained. Its center coordinates are Is calculated as

Next, the distance (l) between the person and the video camera measured by the distance measuring means and the distance (L) between the frame and the camera calculated as a fixed value in advance when the frame data is captured (L). The composite magnification (m) for creating a composite image of a person and a frame is It is calculated by the formula. Therefore, since the formulas (1), (2), and (3) are programmed and read into the program memory, the CPU, the spectacle frame data memory, and the program memory are used for frame composition for creating a composite image of a person and a frame. The magnification and the composite position are calculated, composite image data of the frame and the person is created, and the composite image of the frame and the person is displayed on the display device through the D / A converter.

Therefore, since it is possible to correctly combine images with an arbitrary screen size, it is possible to capture an image with a screen size according to the purpose of use. Also,
Since the spectacle frames can be combined at the correct position, size, and inclination, it is possible to reduce the difference in impression from the case of wearing an actual frame.

(Embodiment) An embodiment of the present invention will be described in detail with reference to FIGS. 1 to 5B. FIG. 1 is a diagram for explaining the configuration of a spectacle wearing simulation device, FIG. 2 is a diagram showing a display screen on which a person image is displayed, and FIG. 3 is a basic spectacle wearing simulation method. It is a flow chart shown.

In FIG. 1, a spectacle wearing simulation device (hereinafter referred to as the present device) 1 is a video camera 3 for capturing an image of a person 2.
And A / D converter 4, image memory 5, D / A converter 6, CPU
7, frame data memory 8, program memory 9,
A computer device 11 and a display device 12 which are composed of an operating means 10.
And a distance measuring device 13.

The video camera 3 is used as an image input device for capturing an image of a person with glasses removed, and when capturing the person 2,
Faced in a certain direction, preset gaze target
Focus on 14.

In this case, the condition that the eyeglasses are not worn is a prerequisite, and the setting of the gaze target and the like described above is arbitrary, and imaging in a normal natural relaxed state is preferable. In addition to the video camera, a scanner or the like can be used as the other imaging device.

Next, the image pickup signal from the video camera 3 which is this image pickup device is taken into the image storage memory 5, and the image pickup signal of this person image is further taken into the image recording memory 5 via the A / D converter 4. The data thus captured in the image memory 5 is displayed as a still image on the display device 12 via the D / A converter 6. (Shown in FIG. 2) From this image, the corneal vertex position coordinates (X _R , Y _R ) and (X _L , Y _L ) of the left and right eyes are detected from the operating means 10, and the person in these images is detected. Image memory 5, image memory 5,
The CPU 7, the frame data memory 8 and the program memory 9 create composite image data of the person image and the eyeglass frame, and display the composite image on the display device 12 via the D / A converter 6.

The video camera 3, the computer device 11, and the display device 12 of this embodiment use an image synthesizing system of a computer which is well known in the configuration and mechanical device mechanism, and the computer of the image pickup signal is transmitted to the display device. The device and principle of D / A conversion, A / D conversion, or image display of an image pickup signal until it is displayed are well known, and the well-known image processing technique is used, so detailed description will be omitted.

Therefore, the data loaded into the image recording memory 5 is RG
Data for each of the three colors of B or a video signal such as NTSC may be used. The CPU 7 performs arithmetic processing, and the frame data memory 8 has previously captured the frame data by the video camera at a fixed acquisition magnification. The image signal of the video camera is converted into a digital signal through A / D conversion to obtain an image. The data is taken into the memory, the data is read into the floppy disk, etc. as numerical data, and the frame data memory is read out whenever necessary.

Further, processing techniques such as reading and reading of the imaged data of this frame into and from the storage device are well known, and are not a feature of the present application.

For the method of capturing eyeglass frame images, refer to Japanese Patent Application No. 62-43.
As shown in No. 98, the color of each pixel of the display device is read in the eyeglass frame data, and the background color of the eyeglass frame depends on how much the color exists in the scanning line direction of the display screen. Read the color of. Therefore, the color of the spectacle frame and the background color of the spectacle frame must be different.

Further, it is preferable that the color of the background is uniform. According to this method, a clear spectacle frame image can be captured. Next, a program for creating a composite image of a person and a frame (which will be described later) is loaded into the program memory 9 and is read out whenever necessary. The capacity of the image recording memory 5 is preferably such that a human image and a spectacle frame image can be displayed on the display device 12 in natural colors and shapes (contours). Then use 8 bits for each color).

Further, since the display screen of the display device 12 is composed of a group of dots (in this embodiment, vertical 480 dots, 0.3
7mm / dot, 640 dots wide, 0.37mm / dot), and the memory address and color contents (RGB) of the computer are assigned to each dot.

That is, RGB components represent individual luminance levels and represent one color as one vector. The brightness level step depends on the number of bits of the computer. For example, in the case of 8 bits for each RGB, 2 ⁸ = 256 steps for each color, and the number of color combinations of the ³ colors is 256 ³ = 16777216, that is, a system capable of expressing 16777216 colors.

The operation means 10 is composed of a keyboard for inputting the IPD or frame data type name, and a touch panel for inputting coordinate positions on the display screen.

In this embodiment, the touch panel is a resistance film type transparent touch panel. In order to detect touch in this structure, two polyester sheets are attached to the screen of the display device with conductive wires made of a transparent resistance film in a grid pattern. Usually, the polyester sheets are separated from each other, and when pushed with a finger, any one of a large number of intersecting two lines conducts to detect this contact point.

The touch panel position detection method includes a capacitive method, an acoustic method, an optical method, and the like, and in addition to the touch panel method, there is a light pen method as an input means from the display screen. In the above, means other than this embodiment can be used, and there is no particular limitation.

A known distance measuring means 13 can be used, and the distance measuring means is not particularly limited. For example, the distance measuring method shown in FIG. 4 or FIGS. 5A and 5B is preferable. In FIG. 4, the distance measuring device 108 includes a light receiving lens 103 and a light receiving sensor 104 for the human face 101.
The light receiving device 102 and the light projecting lens 106 are configured by a light projecting device 105 including a light source 107, a waveform shaping circuit 109, and a calculation unit of an arithmetic circuit 110.

By adjusting the distance between the light projecting lens 106 and the light source 107, the irradiation light 111 emitted from the light projecting device 105 is irradiated on the human face 101 in a point or line shape, and the irradiated position is determined by the light receiving device. Ten
It can be measured at 2. That is, this is performed by moving the light receiving lens 106 so that an image of the illuminated bright spot on the human face 101 is formed on the light receiving sensor 104 of the light receiving device 102. Here, in the light receiving device 102, the light receiving sensor 1
The signal from 04 enters the waveform shaping circuit 109, undergoes waveform shaping, and enters the arithmetic circuit 110. As a result, the light receiving sensor 104
Depending on the image position x from the upper optical axis 112, the distance measuring device 108 and the human face
The distance l with 101 is obtained from the following equation.

However, X ₀ is the distance from the light wheel 112 of the light projecting lens 106, Z ₀ is the distance between the light receiving lens 103 and the light receiving sensor 104, and L ₀
Is the light receiving lens 10 from the point P where the irradiation light 111 intersects the optical axis 112.
The distance is up to 3. As a light receiving sensor, CCD, P
SD etc. can be used.

FIG. 5A shows the structure of a distance measuring device 210 that houses a sounding body 203.
203 A gate circuit 204, a gate pulse generation circuit 205, a sound receiving sensor 206, a waveform shaping circuit 207, a phase comparison circuit 208, and an arithmetic circuit 209.

Here, in FIG. 5A, the continuous signal from the original oscillation circuit 202 enters the gate circuit 204, and the gate pulse generation circuit 205
This signal becomes intermittent due to the gate pulse from and enters the sounding body 203 to generate intermittent sound.

The sound wave generated from the sounding body 203 hits the human face 201, is reflected, and enters the sound receiving sensor 206.

The signal from the sound receiving sensor 206 is waveform shaped by the waveform shaping circuit 207 and enters the phase comparison circuit 208.

As shown in FIG. 5B, the phase comparison circuit 208 compares the gate pulse (a) from the gate pulse generation circuit 205 and the signal (b) from the waveform shaping circuit 207 and outputs the phase difference t.

The phase difference t enters the arithmetic circuit 209, and the distance measuring device 2 is calculated from the following equation.
The distance between 10 and the human face 201 is obtained.

(However, v is the speed of sound in the air) Therefore, according to the present invention, with the configuration shown in FIG. 1, the still image on the display screen of the person who does not wear the glasses taken into the computer by the video camera is used. The composite magnification of the display image on the display screen with respect to the actual person is calculated from the corneal vertex position coordinates of the left and right eyes detected from the image and the distance information between the person and the video camera measured by the distance measuring means. Then, the spectacle frame composition position on the display screen and the composition magnification of the person image and the eyeglass frame image are calculated based on the composition magnification, and a composite image of the person and the eyeglass frame is displayed on the display device. 6 illustrates a method for creating a composite image in a simulation unit.

The specific method will be described in detail below with reference to the flow chart of FIG.

(Steps 1 to 3) The person 2 is captured by the video camera 3 and displayed as a still image on the display image. The image data of this person is not captured only at a specific position on the display screen and at a specific magnification of a specific size, but is captured in the state of a constant, natural, relaxed face. The image pickup signal from is displayed as a still image on the display device via the computer device.

Here, the distance (l) between the person 2 and the video camera 3 is measured by the distance measuring means.

(Steps 4 to 5) Next, the left eye cornea apex position coordinates (X _L , Y _L ) and the right eye corneal apex position coordinates (X _R , Y _R ) of the person image on this display screen are input by the touch panel. Type in the computer.

(Steps 6 to 8) Here, the detection of the image position information of both eyes and the determination of the composite magnification of the frame and the person are performed. That is, the tilt angle of both eyes with respect to the horizontal direction of the screen and the center position of both eyes are detected. Therefore, the position coordinates of the left and right eyes on the display screen can be easily obtained from the input means described above by the dot numbers in the X direction and the Y direction on the screen. Therefore, from these data, the cornea of the left eye and the right eye. The inclination θ of the straight line connecting the vertices (positions) with respect to the X direction (horizontal direction) of the display device is Is required.

Next, in order to synthesize the spectacle frame with the display image of the person, the spectacle frame image must also have the same inclination as the display image of the person. This inclination has already been obtained by the equation (1), and it is obtained by calculating the midpoint (center coordinates) (X ₀ , Y ₀ ) of the corneal vertex position coordinates of the left eye and the right eye. A rotation center reference for rotation can be determined.
Its center coordinates are Becomes

Here, for the composite magnification (m) for creating a composite image of the person and the frame by the distance measuring means, the distance (L) between the frame and the camera at the time of capturing the frame data is a fixed value. Since it has been calculated, It is calculated by the formula.

Further, the synthetic position information of the frame data is obtained from the above-mentioned equations (1) and (2). Therefore, since these image synthesizing methods are programmed and are in the program memory, the synthesizing magnification of the frame for producing a synthetic image of the person and the frame based on the corneal vertex position coordinates and the distance measuring means described above at any time, The combined position is determined using the equations (1) to (3).

(Steps 9 to 13) The frame can be selected by drawing a frame such as a catalog or by the actual object. Since these are registered as frame data in the computer in advance, the numerical value of the model etc. can be selected. Data can be read out by an input operation with a keyboard or the like, and further, a process of creating a composite image of a person and a frame is performed, and the composite image is sent out to the display image and displayed.

Therefore, until the favorite frame is determined, the favorite frame is selected such that the input operation of the frame data is continued.

The above is the basic flow, for example, changing the complexion of the person image to simulate the sunshine state,
It is also possible to perform simulation by changing the color of the lens by using a well-known image synthesizing technique.

(Effects of the Invention) According to the present invention, a person to be imaged without glasses need only look at the gazing target, and need not pay attention to the position and inclination of the face. Therefore, it is possible to perform the operation by itself without the need for a dedicated operator as in the conventional case. Further, since the images can be properly combined in any screen size, it is possible to take an image with a screen size according to the purpose of use. In addition, since the spectacle frames can be combined at the correct position, size, and inclination, the difference in impression from when wearing an actual frame can be reduced, and further, for processing eyeglasses such as eye points on the spectacle frame. The necessary data can be easily measured with high accuracy.

[Brief description of drawings]

FIG. 1 is a diagram for explaining the configuration of the eyeglasses simulation device of this embodiment, FIG. 2 is a partially enlarged view of the display device screen of this embodiment, and FIG. 3 is a diagram of the operation of the eyeglasses simulation device of this embodiment. It is a figure which shows a flow chart. Further, FIGS. 4 and 5A are schematic diagrams showing a distance measuring method, and FIG. 5B is a diagram showing a phase difference between pulses. (Explanation of reference numerals) 1 ... Simulation device for wearing spectacles, 2 ... Person, 3
…… Video camera, 11 …… Computer device, 12 ……
Display device.

Claims (1)

[Claims] 1. A computer having a CPU, an AD converter, an image storage memory, a DA converter, a frame data memory of a spectacle frame captured at a first composite magnification, a program memory, and an operating means, an imaging device, and a display. And a distance measuring means. The computer creates a composite image of the image data of the spectacle frame captured at the first composite magnification and the image data of the human image from the image capture device. A method for creating a composite image in a spectacle wearing simulation device for displaying a state in which a person wears spectacles on the display device, wherein a display screen of a person who does not wear spectacles and is taken into the computer by the imaging device Of the corneal apex position coordinates of the left and right eyes detected from the still image in the image of the person and the imaging device measured by the distance measuring unit. A second combination magnification of the display image on the display screen with respect to the actual person is calculated based on the distance information, and the eyeglass frame combination position on the display screen, the person image and the eyeglass frame are calculated based on the second combination magnification. A third method for producing a composite image in a spectacle wearing simulation device, characterized in that a third composite magnification of an image is calculated and a composite image of a person and a spectacle frame is displayed on the display device.