JP7343760B2 - Image processing program, image processing method, and image processing device - Google Patents

Description

The present invention relates to an image processing program, an image processing method, and an image processing apparatus.

Before buying clothing, people usually try it on. At this time, the person trying on the clothes changes into the clothing they plan to purchase in a space such as a fitting room, and confirms the clothing they plan to purchase by reflecting the clothes they are trying on in a mirror.

On the other hand, in recent years, a system has been developed that photographs a person trying on clothes with a camera and displays the photographed appearance of the person trying on clothes on a display. For example, a display device installed on the back of a half mirror that reflects the user's whole body images the user from a different direction than the half mirror, and displays the captured image so that it does not overlap with the mirror image of the user. A technology has been proposed that makes it possible to visually recognize the figure in multiple directions. Furthermore, a technique has been proposed that allows a user to photograph a person trying on clothes, store the image of the person trying on clothes, and selectively display the stored images of the person trying on clothes.

Furthermore, as a related technique, a technique has been proposed in which the display size of graphic images and character images is changed depending on the distance between the user and the display screen.

Japanese Patent Application Publication No. 2010-87569 Japanese Patent Application Publication No. 2001-157201 Japanese Patent Application Publication No. 2008-96776

However, in the above-described system in which the person trying on the clothes is photographed with a camera and displayed on a display, the ratio of the image of the person trying on the clothes on in the vertical direction may become uneven. For example, when an image of the same height as the person trying on the garment is displayed, the lower body of the image appears smaller than the upper body of the person trying on the garment. Such non-uniformity in appearance becomes more noticeable as the distance between the person trying on the item and the display becomes closer, and the difference between the visible image and the actual item becomes larger. If the image they see differs from the real thing, it becomes difficult for the person trying on the clothes to correctly judge the length of the clothes they are trying on, for example.

In one aspect, it is an object of the present invention to provide an image processing program, an image processing method, and an image processing device that make the appearance of oneself closer to the real thing when an image of oneself is viewed up close. do.

In order to solve the above problems, an image processing program is provided. The image processing program causes a computer to display an image of a person taken with a camera on a display device that the person can view, based on the height of the person's eyes and the distance between the person and the display surface of the display device. The enlargement/reduction ratio calculated for each pixel of the photographed image is obtained, each pixel of the photographed image is enlarged/reduced based on the enlargement/reduction ratio to generate an enlarged/reduced image, the enlarged/reduced image is displayed on a display device , and the height of the display surface is The first size of the pixels of the photographed image when the distance is equally divided in the direction, and the second size of the pixels of the photographed image when the angle is equally divided in the height direction of the display surface with the eye as the center. The process of calculating the scaling ratio is executed based on the size of the image .

Furthermore, in order to solve the above problems, an image processing method is provided in which a computer executes processing similar to processing using the above image processing program.
Furthermore, in order to solve the above problem, an image processing apparatus is provided that performs processing similar to processing using the above image processing program.

According to one aspect, it is possible to make the appearance of oneself closer to the real thing when one looks at an image taken of oneself up close.

FIG. 1 is a diagram for explaining an example of an image processing apparatus according to a first embodiment. FIG. 3 is a diagram illustrating an example of the configuration of an image processing system according to a second embodiment. It is a figure which shows an example of the difference in the appearance of a try-on figure. FIG. 3 is a diagram showing an example of conversion from a distance coordinate system to an angular coordinate system. FIG. 3 is a diagram showing the correspondence between display images and angles. 1 is a diagram illustrating an example of a hardware configuration of an image processing device. FIG. 2 is a diagram illustrating an example of functional blocks of an image processing device. FIG. 6 is a diagram illustrating an example of dividing the display screen into equal distances in the y-axis direction. FIG. 3 is a diagram illustrating an example of equal angular division of the display screen in the y-axis direction. FIG. 6 is a diagram for explaining calculation of the distance from the bottom edge of the display to the top edge of the pixel (θ _a ≦90°). FIG. 6 is a diagram for explaining calculation of the distance from the bottom edge of the display to the top edge of the pixel (θ _a ≦90°). FIG. 6 is a diagram for explaining calculation of the distance from the bottom edge of the display to the top edge of a pixel (90°<θ _a ). FIG. 6 is a diagram for explaining calculation of the distance from the bottom edge of the display to the top edge of a pixel (90°<θ _a ). FIG. 6 is a diagram showing an example of generation of an enlarged/reduced image. It is a figure which shows the example of an image display and the image seen from the person trying it on. 7 is a flowchart illustrating an example of an operation for calculating a first scaling ratio. FIG. 3 is a diagram illustrating an example of an image processing operation based on first scaling ratio calculation. FIG. 7 is a diagram showing an example of a table showing the correspondence between toe pixel positions, top of head pixel positions, and parameter information. 7 is a flowchart illustrating an example of an image processing operation based on second scaling factor calculation. 12 is a flowchart illustrating an example of toe pixel position calculation processing. It is a flowchart which shows an example of the calculation process of the top pixel position. FIG. 7 is a diagram illustrating an example of an image processing operation based on a second scaling factor calculation. 12 is a flowchart illustrating an example of an image processing operation based on a third scaling factor calculation. FIG. 7 is a diagram illustrating an example of an image processing operation based on a third scaling factor calculation.

The present embodiment will be described below with reference to the drawings.
[First embodiment]
FIG. 1 is a diagram for explaining an example of an image processing apparatus according to the first embodiment. The image processing device 1 includes a receiving section 1a and a control section 1b. Further, a camera 2 and a display device 3 are connected to the image processing device 1.

This image processing device 1 displays a photographed image of a person 4 taken by a camera 2 on a display device 3, and in a situation where the person 4 himself/herself visually recognizes the displayed photographed image, the image of the person 4 displayed on the display device 3 is displayed. This is to make the appearance of the image when viewed by the person 4 closer to the real thing. In particular, when the distance between the person 4 and the display surface of the display device 3 is short, the ratio of the image of the person 4 itself seen from the person 4 in the vertical direction becomes uneven. Therefore, the image processing device 1 changes the display state of the photographed image so that the ratio of the image of the person 4 itself in the vertical direction as seen from the person 4 becomes close to that of the actual image.

The receiving unit 1a is realized, for example, as a communication interface. The receiving unit 1a receives an image of a person 4 taken by the camera 2.
The control unit 1b is realized, for example, as a processor. In order to change the display state of the photographed image as described above, the control unit 1b acquires the scaling ratio for each pixel of the photographed image. More specifically, the enlargement/reduction ratio is obtained for each pixel arranged in the vertical direction in the photographed image. Such a scaling factor for each pixel is calculated based on the height H1 of the eye 4a of the person 4 and the distance D1 between the person 4 and the display surface of the display device 3.

Note that the scaling ratio may be stored in a storage device (not shown) for each combination of the height H1 and the distance D1, and read out from the control unit 1b, for example. Alternatively, the scaling ratio may be calculated by the control unit 1b.

The control unit 1b generates an enlarged/reduced image by enlarging/reducing each pixel of the captured image based on the enlargement/reduction ratio, and causes the display device 3 to display the generated enlarged/reduced image.
The ratio of the image of the person 4 itself seen from the person 4 in the vertical direction changes depending on the relative positional relationship between the eyes 4a of the person 4 and the display surface of the display device 3. For example, the closer the distance D1 between the person 4 and the display surface of the display device 3 is, the greater the ratio of the image of the person 4 itself seen from the person 4 in the vertical direction becomes unequal. Further, when the distance D1 is the same, the image of the person 4 on the display screen appears to be enlarged in the vertical direction from the person 4 as it gets closer to the eye 4a, and appears to be reduced in the vertical direction as it gets farther from the eye 4a.

As described above, the control unit 1b scales each pixel of the photographed image using the scaling ratio calculated for each pixel of the photographed image based on the height H1 and the distance D1, and creates a scaled image obtained by scaling. is displayed on the display device 3. Thereby, regardless of the height H1 and the distance D1, the ratio of the image of the person 4 itself seen from the person 4 in the vertical direction can be made closer to the real thing. Therefore, when the image of the person 4 is viewed up close, the appearance of the person 4 can be made closer to the real person.

The lower side of FIG. 1 shows an example of an image when the image of the person 4 is displayed in life-size on the display device 3, and an example of the image when the image is viewed from the person 4. At this time, the height of the eye 4a of the person 4 on the display device 3 becomes H1, similar to the eye 4a of the real person. When the photographed image 5a of the person 4 is displayed as is on the display device 3 in this state, in the image 6a visually recognized by the person 4, the ratio of the upper body of the person 4 appears larger than the actual ratio. The ratio of the upper body becomes relatively larger as the distance D1 becomes shorter.

On the other hand, in the enlarged/reduced image 5b generated from the captured image 5a based on the enlargement/reduction ratio, the ratio of the lower body is larger than that of the upper body in the area below the height of the eye 4a. When such an enlarged/reduced image 5b is displayed on the display device 3, the ratio of the upper body to the lower body of the image 6b visually recognized by the person 4 appears to be the same as that of the real person.

[Second embodiment]
Next, a second embodiment will be described in which the functions of the image processing device 1 of FIG. 1 are applied to a system that photographs a person trying on clothes with a camera and displays the person trying on the clothes on a display.

FIG. 2 is a diagram showing an example of the configuration of an image processing system according to the second embodiment. The image processing system 1-1 includes a camera 20, a display 30, and an image processing device 10. Camera 20 and display 30 are installed inside fitting room 40. It is assumed that the display 30 is installed perpendicularly to the bottom surface of the fitting room 40.

The camera 20 photographs the person trying on the clothes in the fitting room 40. The image processing device 10 performs image processing on the photographed image transmitted from the camera 20 so that the ratio of the upper body to the lower body of the person trying on the suit is close to the real one. The display 30 displays an image of the person trying on the clothes, which has been subjected to image processing by the image processing device 10 . Note that as the display 30, for example, a liquid crystal display or an organic EL (Electroluminescence) display can be used.

Next, before explaining the details of the image processing system 1-1 according to the second embodiment, FIG. Explain using. In addition, in FIG. 3, it is assumed that the image of the person trying on the clothes taken by the camera 20 is displayed as is on the display 30 without changing the size ratio.

FIG. 3 is a diagram illustrating an example of a difference in appearance when trying on clothes. In the upper side of FIG. 3, the x-axis is taken in a direction perpendicular to the display surface of the display 30, and the y-axis is taken in the height direction. Further, the intersection between the display surface of the display 30 and the bottom surface (ground) of the fitting room 40 is defined as the origin of the coordinate system. Further, the try-on person image R0 is a schematic representation of the try-on person R when the photographed image is displayed as is on the display 30.

As shown in the upper part of FIG. 3, two points are taken on the x-axis. Let xb be the point where is farther away.

When try-on person R exists at position xa, the image displayed on display 30 based on the photographed image of try-on person R looks like image g1 from try-on person R. At this time, the try-on person image R0 appears in the image g1 in approximately life-size. However, in the image g1, the proportion of the upper body of the try-on person image R0 is larger than the proportion of the lower body, and the proportions of both are unequal with respect to the real thing. In this case, for the try-on person R, the upper body of the person trying on the clothes appears larger than the real thing, and the lower body of the person trying on the clothes looks smaller than the real thing.

On the other hand, when the try-on person is located at a farther position xb, the image displayed on the display 30 based on the photographed image of the try-on person R looks like image g2 from the try-on person R. In image g2, the proportions of the upper body and lower body are approximately the same as in the real thing, but the whole body image is smaller. In this case, for the try-on person R, the overall image appears small even though the balance between the upper and lower body clothes looks similar to the real thing.

In this way, if the photographed image is displayed as it is on the display 30, the closer the try-on person R gets to the display 30, the more uneven the ratio of the upper body to the lower body becomes, and the more the try-on person R approaches the display 30, the more uneven the ratio of the upper body to the lower body becomes. It looks bigger than the proportion of the lower body. Therefore, it becomes difficult for the try-on person R to correctly judge the state of the try-on, such as the balance between the clothes on the upper body and the lower body.

On the other hand, as the try-on person R moves away from the display 30, the ratio of the upper body to the lower body approaches the actual ratio, but the display 30 ends up displaying the whole body trying on the clothes in a smaller size. Therefore, if the try-on person R wants to see the ratio of the upper and lower body in the same way as the real thing, he or she will have to stand at a distance from the display 30, but since the full-body image will be displayed small, in this case as well, the try-on condition will be different. It is difficult to judge correctly.

Furthermore, by cutting out an area including the try-on person R from the photographed image and displaying the cut-out area on the display 30, the try-on person image R0 on the display 30 can be kept at a constant size even when the try-on person R moves away from the display 30. can be maintained. However, even if such processing is performed, the further the try-on person R is from the display 30, the smaller the size of the try-on person image R0 visually recognized by the try-on person R remains.

Next, the reason why the appearance of the try-on clothes as seen from the try-on person differs depending on the distance between the display 30 and the try-on person will be explained using FIGS. 4 and 5. The try-on person image R0 on the display 30 is displayed in a distance coordinate system expressed by distance. On the other hand, the try-on person R visually recognizes the try-on person image R0 in an angular coordinate system expressed by an angle centered on the eyes of the try-on person R. The reason why the ratio of the upper body and lower body changes according to the distance between the display 30 and the person trying on the garment is that such a coordinate system conversion is performed in the process from when the image of the person trying on the garment R0 is displayed to being visually recognized by the person trying on the garment. This is because you will be exposed.

FIG. 4 is a diagram showing an example of conversion from a distance coordinate system to an angular coordinate system. FIG. 4 shows, as an example, a case where the try-on person image R0 is displayed in life-size on the display 30. At this time, the height of the try-on person R and the height of the try-on person image R0 are equal.

The y-coordinate of the ground is y ₀ , the y-coordinate of the head of try-on R is y _h , the y-coordinate of the eye of try-on R is y _e , and the position of half the height of try-on R is Let the y coordinate be _yL . Further, let the distance from y ₀ to y _L be y _b , and let the distance from y _L to y _h be _yu (y _u =y _b ).

Here, on the acute angle side formed by the light ray B1 that enters the eyes of the try-on person R from the top of the head of the try-on person image R0, and the light ray B2 that enters the eyes of the try-on person R from a position half the height of the try-on person image R0, Let the angle be θ _u . Further, the acute angle between the light ray B2 and the light ray B3 which enters the eyes of the try-on person R from the feet of the try-on person image R0 is defined as θ _b . At this time, when the try-on person image R0 is displayed on the display 30 and the try-on person R looks at the displayed try-on person image R0, the distance y _u is converted to an angle θ _u , and the distance yb is converted to an angle θ _b . Ru.

FIG. 5 is a diagram showing the correspondence between display images and angles. An image g3 shown in FIG. 5 shows an image that appears in the eyes of the try-on person R when the try-on person R views the photographed image displayed on the display 30 as shown in FIG. In image g3, the upper half of the body corresponding to the distance y _u is converted to an angle θ _u , and the lower half of the body corresponding to the distance y _b is converted to an angle θ _b . The closer the distance between the display 30 and the try-on person R is, the larger the angle θ _u becomes relative to the angle θ _b.As shown in FIG. I can see it. Therefore, it becomes difficult for the try-on person R to correctly judge the length of the clothes, etc.

The present embodiment has been made in view of these points, and when displaying the trying-on appearance of the try-on person R photographed by the camera 20 on the display 30, it does not depend on the distance between the display 30 and the try-on person R. In this case, the upper and lower body of the person trying on the suit R is converted and displayed so that they can be seen at the same proportions as the real thing. 4 and 5, when the size of the pixels on the display surface of the display 30 in the y-axis direction is converted to the size in the angular coordinate system centered on the eyes of the try-on person R, The closer the image is to the eye level of R, the more the image is enlarged, and the further it is from the eye level, the more the image is reduced. Therefore, the image processing device 10 enlarges or reduces the vertical size of each pixel of the captured image displayed on the display 30 at a conversion rate that is opposite to the above conversion rate, and displays the enlarged/reduced captured image. 30. Further, the image processing device 10 determines the enlargement/contraction ratio according to the distance between the display surface of the display 30 and the person trying on the garment R, and the height of the eye of the person trying on the garment R.

FIG. 6 is a diagram illustrating an example of the hardware configuration of the image processing device. The entire image processing apparatus 10 is controlled by a processor (computer) 100. Processor 100 may be a multiprocessor. The processor 100 is, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), or a PLD (Programmable Logic Device). Further, the processor 100 may be a combination of two or more elements among a CPU, an MPU, a DSP, an ASIC, and a PLD.

A memory 101 and a plurality of peripheral devices are connected to the processor 100 via a bus 103.
Memory 101 is used as a main storage device of image processing device 10 . The memory 101 temporarily stores at least a portion of OS (Operating System) programs and application programs to be executed by the processor 100. The memory 101 also stores various data required for processing by the processor 100. The memory 101 is also used as an auxiliary storage device of the image processing device 10, and stores OS programs, application programs, and various data. Note that the image processing device 10 may include, as an auxiliary storage device, a semiconductor storage device such as a flash memory or an SSD (Solid State Drive), or a magnetic recording medium such as an HDD (Hard Disk Drive).

Peripheral devices connected to the bus 103 include an input/output interface 102 and a network interface 104.
A camera 20 and a display 30 are connected to the input/output interface 102. The input/output interface 102 receives data of an image taken by the camera 20 and transmits it to the processor 100. Further, the input/output interface 102 causes the display 30 to display an image according to instructions from the processor 100. Further, the input/output interface 102 can be connected to an information input device such as a keyboard or a mouse, and transmits signals sent from the information input device to the processor 100.

The input/output interface 102 also functions as a communication interface for connecting peripheral devices. For example, the input/output interface 102 can be connected to an optical drive device that reads data recorded on an optical disc using laser light or the like. Optical discs include CDs (Compact Discs), DVDs (Digital Versatile Discs), and Blu-ray Discs (BD, registered trademark).

Further, the input/output interface 102 can connect a memory device or a memory reader/writer. The memory device is a recording medium equipped with a communication function with the input/output interface 102. A memory reader/writer is a device that writes data to or reads data from a memory card. A memory card is a card-type recording medium.

The network interface 104 connects to a network and performs network interface control. Network interface 104 transmits and receives data between a network and processor 100. As the network interface 104, for example, a NIC (Network Interface Card), a wireless LAN (Local Area Network) card, etc. can be used.

With the hardware configuration as described above, the processing functions of the image processing device 10 can be realized.
FIG. 7 is a diagram illustrating an example of functional blocks of an image processing device. The image processing device 10 includes a control section 11, a storage section 12, and an interface section 13. The control section 11 includes an enlargement/reduction ratio calculation section 11a and an image enlargement/reduction section 11b.

The enlargement/reduction ratio calculation unit 11a calculates the size in 1 of the pixel in the y-direction when the display surface of the display 30 is equally divided in the y-axis direction, and the size _in 1 of the pixel in the y-direction with the eye of the try-on person as the center. The scaling ratio is calculated from the size i _n 2 of the pixel in the y direction when the pixel is divided into equal angles. The enlargement/contraction ratio is determined by adjusting each pixel of the photographed image so that a point equidistant on the y-axis (that is, with respect to the vertical direction of the display surface of the display 30) appears to be equidistant in the field of view of the person trying on the garment. This is a coefficient for enlarging or reducing the size in the y-axis direction.

"Equal distance division" indicates that the display surface is divided at equal distances in the y-axis direction. In this distance-equal division, the display surface is divided by the number of pixels in the y-axis direction of the photographed image. Therefore, each divided area corresponds to each pixel that is parallel in the y-axis direction of the captured image when the captured image is drawn on the display screen, and the size i _n 1 corresponds to the nth pixel on the captured image. Indicates the size of the area to be drawn in the y-axis direction.

Note that the number of divisions in the equal distance division is the number of pixels in the y-axis direction of the captured image (source image) before being input to the display 30. Therefore, the number of divisions in equal distance division is not necessarily the same as the number of pixels in the y-axis direction of the display 30.

"Equal angle division" indicates that when the viewing angle in the y-axis direction from the eyes of the wearer to the display surface is divided by the same angle, the display surface is divided by the dividing line. "Pixels when divided into equal angles" (or "pixels when divided into equal angles") refers to each divided area projected onto the display screen when the above-mentioned viewing angle is divided at the same angle.

The number of divisions in equal angle division is the same as the number of divisions in equal distance division. Then, the nth pixel when the display surface is divided into equal distances is associated with the nth pixel when the display surface is divided into equal angles, and the sizes of both nth pixels are i _n 1, i _n 2 is used to find the enlargement/contraction ratio for the n-th pixel in the photographed image. Therefore, the enlargement/contraction ratio is calculated for each pixel arranged in parallel in the y direction in the photographed image.

In addition, when calculating the pixel size _in 2 by equal angle division, the enlargement/reduction ratio calculation unit 11a calculates the distance between the display 30 and the person trying on the suit (first parameter) and the height of the eye of the person trying on the suit (second parameter). The pixel size i _n 2 is calculated based on the following parameters. At this time, the scaling ratio calculation unit 11a obtains the scaling ratio by at least one of the first, second, and third scaling ratio calculations according to the method of obtaining parameter information including the first and second parameters. I can do it.

In the first scaling ratio calculation, the scaling ratio is calculated using fixed value parameter information. In the second scaling ratio calculation, the scaling ratio is calculated using parameter information retrieved using the toe pixel position and the top pixel position of the wearer as keys. In the third expansion/contraction ratio calculation, parameter information is obtained from the actually measured three-dimensional (3D) position of the eye of the person trying on the suit, and the expansion/contraction ratio is calculated. Note that details of the first, second, and third scaling ratio calculations will be described later with reference to FIGS. 16 to 24.

The image enlarging/reducing unit 11b multiplies the captured image of the person trying on the clothes taken by the camera 20 by an enlargement/reduction ratio to generate an enlarged/reduced image, and causes the display 30 to display the enlarged/reduced image. The storage unit 12 stores an image taken by the camera 20 and information (parameter information) required for calculating the enlargement/reduction ratio.

The interface unit 13 performs interface control with devices connected to the image processing device 10. For example, it performs a process of receiving an image taken by the camera 20 and a process of transferring an enlarged/reduced image to the display 30.

Note that the control unit 11 is realized by the processor 100 in FIG. 6, for example. In this case, the processing of the control unit 11 is realized by the processor 100 executing a program. Further, the storage unit 12 is realized, for example, by the memory 101 in FIG. 6. The interface unit 13 is realized, for example, by the input/output interface 102 shown in FIG.

Next, the scaling ratio calculation process in the image processing device 10 will be described in detail using FIGS. 8 to 13. Note that illustration of the camera 20 installed on the y-axis will be omitted hereinafter.
FIG. 8 is a diagram showing an example of dividing the display surface into equal distances in the y-axis direction. Here, the number of pixels in the vertical direction (y-axis direction) of the photographed image is I [pixel], and each divided area (each Find the size of the pixel) in the y-axis direction. This is the same as finding the size of each pixel in the y-axis direction of the captured image when the captured image is virtually drawn on the display surface of the display 30 while maintaining the number of pixels. It is assumed that the lower end of the display surface of the display 30 is at the ground level.

By the equal distance division, the display surface is equally divided into I pixels p _n (1≦n≦I). At this time, the size i _n 1 [mm/pixel] of the pixel p _n in the y-axis direction is calculated by the following formula (1), assuming that the height of the display surface of the display 30 is y _v [mm]. . Since the pixels are equally divided by distance, the sizes of all pixels p ₁ , . . . , p _n , . . . , p _I have the same value.

Next, consider the area that the display surface of the display 30 occupies in the viewing angle of the person trying on the suit R in the y-axis direction. First, as an angle conversion, a unit conversion formula from radian to degree is shown in the following equation (2a), and a unit conversion equation from degree to radian is shown in the following equation (2b).

The smaller angle (viewing angle) between the light ray b1 entering the eye of the try-on person R from the upper end of the display surface of the display 30 and the light ray b2 entering the eye of the try-on person R from the lower end of the display surface is Θ. do. Also, let Θ _u be the acute angle between the light ray b1 and the vertical line ha drawn from the eyes of the try-on person R to the ground, and let Θ _d be the acute angle between the light ray b2 and the vertical line ha. . The distance between the display surface of the display 30 and the person trying on the garment R is x _r , and using this distance x _r and the eye height y _e of the person trying on the garment R, the angles Θ, Θ _u , and Θ _d are calculated using the following formulas. Calculated by (3), (4), and (5).

FIG. 9 is a diagram showing an example of equal angle division of the display surface in the y-axis direction. The display surface of the display 30 shall be equally divided at angles such that the angle formed by the upper and lower ends of the pixels (divided areas) p _n (1≦n≦I) and the eyes of the try-on person R is uniform. . This means that the viewing angle of the try-on person R in the y-axis direction with respect to the display screen is equally divided into I parts. Also, the distance from the bottom edge of the display screen to the top edge of pixel p _n is y _n (first top edge distance), and the distance from the bottom edge of the display screen to the top edge of pixel p _n-1 (that is, the bottom edge of pixel p _n ) is Let y _n-1 (second upper end distance).

Let _{θ n be} the acute angle between the light ray b11 entering the eye of the try-on person R from the upper end of the pixel p n and the light ray b 12 entering the eye of the try-on person R from the upper end of the pixel p _n-1 . Note that the acute angle between the light ray b11 and the vertical line ha is defined as θ _a . At this time, since the angle θ _n is one angle obtained by equally dividing the viewing angle Θ shown in FIG. 8 into I pieces, θ _n for the pixel p _n is calculated by the following equation (6). Since the angles are equally divided, the magnitudes of all angles (θ ₁ , . . . , θ _n , . . . , θ _I ) have the same value (referred to as θ _e ).

As shown in FIG. 9, the size of each pixel on the y-axis (size in the distance coordinate system) due to equal angle division differs from pixel to pixel, but it appears to be the same size to the person trying it on. Therefore, by scaling up or down the size i _n 1 of pixel p _n resulting from equal distance division to the size i _n 2 of pixel p _n resulting from equal angle division, each pixel resulting from equal distance division (i.e., each pixel of the photographed image ) can be shown to be the same size in the y-axis direction to the try-on person R.

First, the size i _n 2 of the pixel p _n resulting from equal angle division is calculated using the following equation (7).

Further, the height y _n of the upper end of pixel p _n in equation (7) is calculated by the following equations (8a) and (8b). The height y _n-1 of the upper end of the pixel p _n-1 is similarly determined from equations (8a) and (8b).

Therefore, the scaling ratio r _n for pixel p _n is calculated by dividing the size i _n 2 of pixel p _n by equal angle division into the size i _n 1 of pixel p _n by equal distance division, as shown in the following equation (9). Calculated by dividing by .

Here, the above equation (8a) will be explained using FIGS. 10 to 13. 10 and 11 are diagrams for explaining calculation of the distance from the bottom edge of the display to the top edge of the pixel. 10 and 11 show the case where the angle θ _a is 90° or less.

In FIG. 10, first, the control unit 11 calculates an equal division angle θ _e by equally dividing the viewing angle Θ of the try-on person R with respect to the display surface of the display 30 (formula (6)). The control unit 11 equalizes the acute angle θ _a 1 (first angle) between the light ray b11 that enters the eye of the try-on person R from the upper end of the pixel p _n (first pixel) and the vertical line ha. It is calculated based on the division angle θ _e (=θ _n ). In this calculation process, _θ a obtained by substituting n of pixel p _n using equation (8b) is set as θ _a 1. The angle θ _a 1 is 90° or less. In this case, the control unit 11 subtracts the angle θ _a 1 from 90° to obtain the angle (90°−θ _a 1) (second angle).

On the other hand, if we connect the eye A of the try-on person R, the point B at the _height of the eye A on the display 30, and the point C at the upper end of the pixel _pn on the display 30, we can form a triangle ABC (the interior angle of point B). is a right angle) is formed. The control unit 11 multiplies the tangent of the angle (90°-θ _a 1) in the triangle ABC by the distance x _r (first parameter) between the display 30 and the try-on person R to obtain the distance d1 ( the first distance). Then, the control unit 11 subtracts the distance d1 from the eye height y _e (second parameter) of the try-on person R to calculate the height y _n of the upper end of the pixel p _n (equation (8a)). when θ _a ≦90°).

In FIG. 11, the control unit ₁₁ similarly controls the acute angle θ _a 2 (the third angle) is calculated based on the equal division angle θ _e (=θ _n−1 ). In this calculation process, _θ a obtained by substituting (n-1) of pixel p _n-1 using equation (8b) is set as θ _a 2. The angle θ _a 2 is 90° or less. In this case, the control unit 11 subtracts the angle θ _a 2 from 90° to obtain the angle (90°−θ _a 2) (fourth angle).

On the other hand, if we connect the eye A of the try-on person R, the point B at the height _ye of the eye A on the display 30, and the point D at the upper end of the pixel _pn-1 on the display 30, a triangle ABD (point B The interior angles of are right angles) are formed. The control unit 11 multiplies the tangent of the angle (90°-θ _a 2) in the triangle ABD by the distance x _r (first parameter), which is the distance between the display 30 and the person trying on the garment R, to determine the distance. d2 (second distance) is calculated. Then, the control unit 11 subtracts the distance d2 from the eye height y _e (second parameter) of the try-on person R to calculate the height _{y n-1} of the upper end of the pixel p _n-1 (formula (8a) when θ _a ≦90°).

In this way, when θ _a ≦90°, the size i _n 2 of pixel p _n by equal angle division can be calculated using equation (7).
12 and 13 are diagrams for explaining calculation of the distance from the bottom edge of the display to the top edge of the pixel. 12 and 13 show the case where the angle θ _a exceeds 90°.

In FIG. 12, first, the control unit 11 calculates an equal division angle θ _e by equally dividing the viewing angle Θ of the try-on person R with respect to the display surface of the display 30 (formula (6)). The control unit 11 equalizes the obtuse angle θ _a 3 (fifth angle) between the light ray b13 that enters the eye of the try-on person R from the upper end of the pixel p _n (first pixel) and the vertical line ha. It is calculated based on the division angle θ _e (=θ _n ). In this calculation process, _θ a obtained by substituting n of pixel p _n using equation (8b) is set as θ _a 3. The angle θ _a 3 exceeds 90°. In this case, the control unit 11 subtracts 90° from the angle θ _a 3 to obtain the angle (θ _a 3-90°) (sixth angle).

On the other hand, if we connect eye A of try-on person R, point B at the height of eye _A on the display 30, and point E at the upper end of pixel _pn on the display 30, we will find a triangle ABE (internal angle of point B). is a right angle) is formed. The control unit 11 multiplies the tangent of the angle (θ _a 3-90°) in the triangle ABE by the distance x _r (first parameter) between the display 30 and the try-on person R to obtain a distance d3 ( 3rd distance) is calculated. Then, the control unit 11 calculates the height y _n of the upper end of the pixel p _n by adding the distance d3 to the eye height _{y e} (second parameter) of the try-on person R (equation (8a)). when 90°< _θa ).

In FIG. 13, the control unit ₁₁ similarly controls the obtuse angle θ _a 4 (the seventh angle) is calculated based on the equal division angle θ _e (=θ _n−1 ). In this calculation process, _θ a obtained by substituting (n-1) of pixel p _n-1 using equation (8b) is set as θ _a 4. The angle θ _a 4 exceeds 90°. In this case, the control unit 11 subtracts 90° from the angle θ _a 4 to obtain the angle (θ _a 4-90°) (eighth angle).

On the other hand, if we connect the eye A of the try-on person R, the point B at the height _ye of the eye A on the display 30, and the point F at the upper end of the pixel p _n-1 on the display 30, a triangle ABF (point B The interior angles of are right angles) are formed. The control unit 11 multiplies the tangent of the angle (θ _a 4-90°) in the triangle ABF by the distance x _r (first parameter) between the display 30 and the try-on person R to obtain a distance d4 ( 3rd distance) is calculated. Then, the control unit 11 calculates the height y _n-1 of the upper end of the pixel p _n-1 by adding the distance d4 to the eye height y _e (second parameter) of the try-on person R (formula (8a) when 90°<θ _a ).

In this way, in the case of 90°<θ _a , the size i _n 2 of the pixel p _n by equal angle division can be calculated using equation (7).
FIG. 14 is a diagram illustrating an example of generating an enlarged/reduced image. An enlarged/reduced image g20 is generated by applying a corresponding enlargement/reduction ratio to each pixel of the photographed image g10. In this process, first, the vertical size of each pixel of the photographed image is scaled up or down by multiplying the vertical size _in of the pixel p _n of the photographed image by the scaling ratio r _n . The size i _n 3 of the pixel p _n after scaling is calculated by the following equation (10). However, the pixel sizes i _n and i _n 3 are the sizes in the image coordinate system, not in the real world coordinate system where the display 30 and the try-on person R exist.

図１４に示すように、撮影画像ｇ１０の各画素が拡縮されることで、拡縮後の画素による変換画像ｇ１０ａが生成される。そして、変換画像ｇ１０ａの各画素の画素値を補間演算して、変換画像ｇ１０ａを撮影画像ｇ１０と同じ画素数の画像に変換することで、拡縮画像ｇ２０が生成される。拡縮画像ｇ２０はディスプレイ３０に入力され、ディスプレイ３０に表示される。

As shown in FIG. 14, by scaling each pixel of the captured image g10, a converted image g10a is generated using the pixels after scaling. Then, by interpolating the pixel value of each pixel of the converted image g10a and converting the converted image g10a into an image with the same number of pixels as the photographed image g10, an enlarged/reduced image g20 is generated. The enlarged/reduced image g20 is input to the display 30 and displayed on the display 30.

FIG. 15 is a diagram showing an example of an image display and an image seen from the person trying on the item. If the photographed image g10 illustrated in FIG. 15 is displayed on the display 30, the displayed photographed image g10 shows the person trying on the clothes at almost life size. In this case, the eye height of the try-on person in the displayed photographed image g10 is approximately equal to the eye height of the actual try-on person. In addition, in this photographed image g10, the upper and lower body of the person trying on the clothes is shown in approximately the same proportion as the actual person. However, when the try-on person looks at the display 30 on which this photographed image g10 is displayed, the try-on person sees the upper body enlarged in the vertical direction and the lower body reduced in the vertical direction, as shown in the example of FIG. .

On the other hand, in the enlarged/reduced image g20 obtained by enlarging/reducing this photographed image g10, the image is reduced vertically in the area close to the eye position of the try-on person, and in the area far from the eye position of the try-on person (for example, at the bottom of the image). In the side area), the image is enlarged vertically. As a result, in the image of the person trying on the suit in the enlarged/reduced image g20, the upper body appears smaller and the lower body appears larger than in the original photographed image g10. When the try-on person looks at the display 30 on which such an enlarged/reduced image g20 is displayed, the upper and lower body of the try-on person appears in almost the same proportion as the real thing, as shown in the image g20a, as in the original photographed image g10. become.

In addition, in the enlarged/reduced image g20, the closer the distance between the display 30 and the person trying on the clothes is, the greater the reduction rate of the upper body and the greater the enlargement rate of the lower body. On the other hand, in the enlarged/reduced image g20, the farther the distance between the display 30 and the person trying on the clothes, the smaller the reduction ratio of the upper body, the smaller the enlargement ratio of the lower body, and the closer the ratio of the upper body to the lower body becomes. However, when the person trying on the garment looks at the display 30 on which the enlarged/reduced image g20 is displayed, the upper and lower body of the wearer can be seen at a ratio close to the real thing, regardless of the distance between the display 30 and the person trying on the garment.

Next, the first expansion/contraction rate calculation by the expansion/contraction rate calculation section 11a of the control section 11 will be explained using FIGS. 16 and 17. In the first expansion/contraction ratio calculation, the distance _xr between the display 30 and the person trying on the suit and the _height of the eye of the person trying it on, ye, are prepared in advance as fixed values, and the expansion/contraction rate is calculated based on the parameter information of the fixed value. It is something to do.

For example, the distance x _r between the display 30 and the person trying on the clothes is set based on the average size of the floor area of the fitting room (about 90 cm x about 90 cm). In the storage unit 12, two-thirds of the size (approximately 60 cm) in the depth direction (direction perpendicular to the display surface of the display 30) in the fitting room is stored in advance as a fixed value of the distance x _r . . Furthermore, the storage unit 12 stores in advance the intermediate value (164 cm) between the average height of Japanese men (171 cm) and the average height of Japanese women (158 cm) as a fixed eye height _ye .

Further, the height _yv of the display surface of the display 30 is initially set in the storage unit 12 when the image processing device 10 is started to be used. In this way, the storage unit 12 stores in advance the distance x _r between the display 30 and the person trying on the item, the eye height y _e of the person trying on the item, and the height y _v of the display surface of the display 30 as parameter information. Ru.

FIG. 16 is a flowchart illustrating an example of the operation of calculating the first scaling ratio.
[Step S11] The enlargement/reduction ratio calculation unit 11a acquires a captured image captured by the camera 20. At this time, the enlargement/reduction ratio calculation unit 11a performs processing on the photographed image to remove distortion of the lens of the camera 20 in the vertical direction, so that the vertical ratio of the image in the photographed image approaches the actual ratio. It is desirable to correct it as follows.

[Step S12] The expansion/contraction rate calculation unit 11a acquires parameter information for calculating the expansion/contraction rate from the storage unit 12. As parameter information, the distance x _r between the display 30 and the person trying on the item, the height _ye of the eyes of the person trying on the item, and the height y _v of the display surface of the display 30 are acquired. Further, the enlargement/reduction ratio calculation unit 11a recognizes the number I of pixels in the vertical direction of the acquired captured image.

[Step S13] The scaling ratio calculation unit 11a calculates the scaling ratio for each pixel based on the acquired parameter information and the recognized number of pixels I.
[Step S14] The image enlarging/reducing unit 11b multiplies the captured image by the calculated enlargement/reducing ratio to generate an enlarged/reduced image.

[Step S15] The control unit 11 transfers the enlarged/reduced image to the display 30.
FIG. 17 is a diagram illustrating an example of an image processing operation based on the first scaling factor calculation.
[Step S21] The camera 20 photographs the person trying on the clothes, and the photographed image g10 is transferred to the control unit 11.

[Step S22] The enlargement/reduction ratio calculation unit 11a extracts parameter information from the storage unit 12. Further, the enlargement/reduction ratio calculation unit 11a recognizes the number I of pixels in the vertical direction of the acquired captured image.
[Step S23] The scaling ratio calculation unit 11a calculates the scaling ratio based on the parameter information and the number of pixels I.

Note that when the number of pixels I and the height _yv of the display surface of the display 30 are also fixed values, the storage unit 12 stores the fixed values of distance _xr , height _ye , number of pixels I, and height _yv. The enlargement/contraction ratio calculated based on this may be stored in advance. In this case, the scaling ratio is acquired from the storage unit 12 in steps S22 and S23.

[Step S24] The image enlarging/reducing unit 11b multiplies the photographed image g10 by an enlarging/reducing ratio to generate an enlarged/reduced image g20.
[Step S25] The control unit 11 transfers the enlarged/reduced image g20 to the display 30.

[Step S26] The display 30 displays the enlarged/reduced image g20.
Next, the second expansion/contraction rate calculation by the expansion/contraction rate calculation unit 11a will be explained using FIGS. 19 to 22. In the second scaling factor calculation, among the parameter information, the distance _xr between the display 30 and the person trying on the suit and the _height of the eyes of the person trying on the suit, ye, are used for the toe pixel position and the top of the head in the image of the person trying on the suit in the photographed image. The enlargement/contraction ratio is calculated based on the pixel position.

Here, as an example, it is assumed that the values of distance x _r and _height ye corresponding to a plurality of combinations of toe pixel positions and top of head pixel positions are stored in advance in the storage unit 12 . Furthermore, for the sake of simplicity, it is assumed that the number I of pixels in the vertical direction of the photographed image is a fixed value and is stored in advance in the storage unit 12 as parameter information.

FIG. 18 is a diagram showing an example of a table showing the correspondence between toe pixel positions, top of head pixel positions, and parameter information. Tables 12a and 12b shown in FIG. 18 are stored in the storage unit 12 and used when the second scaling ratio calculation is performed.

In the table 12a, the distance _xr between the display 30 and the person trying on the item is registered in association with a plurality of values for the toe pixel position P1. The toe pixel position P1 is detected as a pixel number in the vertical direction of the toe of the person trying on the wearer in the photographed image. If the position and direction of the camera 20 are determined in advance, the distance _xr can be estimated from the toe pixel position P1. The distance _xr between the display 30 and the person trying on the item is read out from the table 12a using the detected toe pixel position P1 as a key.

In the table 12b, the eye height _ye of the try-on wearer is registered in association with each combination of the toe pixel position P1 and the top pixel position P2. The top of the head pixel position P2 is detected as a pixel number in the vertical direction of the top of the head of the person trying on the wearer in the photographed image. If the position and direction of the camera 20 are determined in advance, the position and size of the average try-on person's face can be estimated from the combination of the toe pixel position P1 and the top pixel position P2, and the height can be determined based on these. y _e is estimated. The eye height _ye of the try-on person is read out from the table 12b using the detected toe pixel position P1 and top of head pixel position P2 as keys.

FIG. 19 is a flowchart illustrating an example of the image processing operation based on the second scaling factor calculation.
[Step S31] The control unit 11 acquires a photographed image taken by the camera 20. Note that, similarly to step S11 above, it is desirable that the control unit 11 at this time perform processing on the captured image to remove distortion of the lens of the camera 20 in the vertical direction.

[Step S32] The enlargement/reduction ratio calculation unit 11a detects the toe of the person trying on the suit from the photographed image, and calculates the toe pixel position P1.
[Step S33] The enlargement/reduction ratio calculation unit 11a detects the top of the head of the person trying on the suit from the photographed image, and calculates the top of the head pixel position P2.

[Step S34] The enlargement/reduction ratio calculation unit 11a converts the parameter information stored in the storage unit 12 (the distance x _r between the display and the person trying on the garment and the height of the eyes of the person trying it on _ye ) into the calculated toe pixel position. P1 and the parietal pixel position P2 are used as keys to read and acquire the data. That is, the distance x _r is obtained from the table 12a using the toe pixel position P1 as a key, and the height _ye is obtained from the table 12b using the toe pixel position P1 and the top pixel position P2 as keys. The enlargement/reduction ratio calculation unit 11a also obtains the number of pixels I in the vertical direction of the captured image and the height _yv of the display surface of the display 30 from the storage unit 12 as other parameter information.

[Step S35] The expansion/contraction rate calculation unit 11a calculates the expansion/contraction rate based on the acquired parameter information.
[Step S36] The image enlarging/reducing unit 11b multiplies the captured image by the calculated enlargement/reducing ratio to generate an enlarged/reduced image.

[Step S37] The control unit 11 transfers the enlarged/reduced image to the display 30.
FIG. 20 is a flowchart illustrating an example of toe pixel position calculation processing. The process in FIG. 20 corresponds to the process in step S32 in FIG.

[Step S41] The enlargement/reduction ratio calculation unit 11a performs edge enhancement processing in the y-axis direction (vertical direction) on the luminance value of the photographed image using an edge enhancement filter.
[Step S42] The enlargement/reduction ratio calculation unit 11a selects the bottom line (horizontal pixel row) in the photographed image as a toe position search target.

[Step S43] The enlargement/reduction ratio calculation unit 11a searches for a pixel whose luminance value is equal to or greater than a threshold value from the left end to the right end of the search target line in the edge-enhanced photographed image.
[Step S44] The enlargement/reduction ratio calculation unit 11a determines whether a pixel whose luminance value is equal to or greater than a threshold value is detected. If the corresponding pixel is detected, the scaling factor calculation unit 11a determines the pixel number in the vertical direction of the current search target line as the toe pixel position P1, and ends the process. On the other hand, if the corresponding pixel is not detected, the process advances to step S45.

[Step S45] The enlargement/reduction ratio calculation unit 11a moves the line to be searched upward by one column (one pixel).
[Step S46] The enlargement/reduction ratio calculation unit 11a determines whether the moved line to be searched is the topmost line of the photographed image. If the moved search target line is not the topmost line, the process returns to step S43. On the other hand, if the search target line that has been moved is the topmost line, the toe pixel cannot be detected, it is regarded as an error, and the process is terminated.

Note that, if an error occurs in calculating the toe pixel position, the scaling ratio calculation unit 11a may, for example, shift to the first scaling ratio calculation based on parameter information of fixed values stored in advance in the storage unit 12. can.

FIG. 21 is a flowchart illustrating an example of a process for calculating the pixel position of the top of the head. The process in FIG. 21 corresponds to the process in step S33 in FIG. 19.
[Step S51] The enlargement/reduction ratio calculation unit 11a acquires the photographed image subjected to the edge enhancement process in step S42 of FIG. 20.

[Step S52] The enlargement/reduction ratio calculation unit 11a selects the upper end line (horizontal pixel row) in the photographed image as a search target for the top of the head position.
[Step S53] The enlargement/reduction ratio calculation unit 11a searches for a pixel whose luminance value is equal to or greater than a threshold value from the left end to the right end of the search target line in the edge-enhanced photographed image.

[Step S54] The enlargement/reduction ratio calculation unit 11a determines whether a pixel whose luminance value is equal to or greater than a threshold value is detected. If the corresponding pixel is detected, the scaling factor calculation unit 11a determines the pixel number in the vertical direction of the current search target line as the top pixel position P2, and ends the process. On the other hand, if the corresponding pixel is not detected, the process advances to step S55.

[Step S55] The enlargement/reduction ratio calculation unit 11a moves the search target line downward by one column (one pixel).
[Step S56] The enlargement/reduction ratio calculation unit 11a determines whether the moved line to be searched is the lowest line of the photographed image. If the search target line that has been moved is not the lowest line, the process returns to step S53. On the other hand, if the search target line that has been moved is the lowest line, the top of the head pixel cannot be detected, and this is regarded as an error, and the process is terminated.

Note that, if an error occurs in calculating the top pixel position, the scaling ratio calculation unit 11a may, for example, shift to a first scaling ratio calculation based on parameter information of fixed values stored in advance in the storage unit 12. I can do it.

In the process shown in FIG. 20 described above, the lower end position of the person trying on the suit in the photographed image after edge enhancement is detected as the toe position. In addition, in the process of FIG. 21, the upper end position of the person trying on the garment in the photographed image after edge enhancement is detected as the position of the top of the head. Through such processing, the toe pixel position and the top pixel position can be estimated by simple processing without using a plurality of cameras 20 or a dedicated distance sensor.

FIG. 22 is a diagram illustrating an example of an image processing operation based on the second scaling factor calculation.
[Step S61] The camera 20 photographs the person trying on the clothes and transfers the photographed image to the control unit 11.

[Step S62] The enlargement/reduction ratio calculation unit 11a calculates the toe pixel position from the photographed image g10.
[Step S63] The enlargement/reduction ratio calculation unit 11a calculates the top of the head pixel position from the photographed image g10.

[Step S64] The enlargement/reduction ratio calculation unit 11a extracts the distance _xr between the display and the person trying on the clothes from the table 12a, which is included in the parameter information, using the toe pixel position as a key. Furthermore, the enlargement/reduction ratio calculation unit 11a extracts the eye height _ye of the person trying on the wearer from the table 12b, which is included in the parameter information, using the toe pixel position and the top pixel position as keys. Further, the enlargement/reduction ratio calculation unit 11a obtains the number of pixels I in the vertical direction of the photographed image and the height _yv of the display surface of the display 30 from the storage unit 12 as other parameter information.

[Step S65] The expansion/contraction rate calculation unit 11a calculates the expansion/contraction rate based on the parameter information.
[Step S66] The image enlarging/reducing unit 11b multiplies the photographed image g10 by the enlarging/reducing ratio to generate an enlarged/reduced image g20.

[Step S67] The control unit 11 transfers the enlarged/reduced image g20 to the display 30.
[Step S68] The display 30 displays the enlarged/reduced image g20.
Next, the third expansion/contraction ratio calculation will be explained using FIGS. 23 and 24. In the third scaling factor calculation, the distance _xr between the display and the person trying on the item and the _height of the eye of the person trying it on, among the parameter information, are actually measured, and the scaling factor is calculated based on these measured values. be. In the following explanation, as an example, two cameras (stereo cameras) are used to measure the 3D position of the try-on's eyes, but as another example, a depth sensor may be used to measure the 3D position of the try-on's eyes. good.

FIG. 23 is a flowchart illustrating an example of the image processing operation based on the third scaling factor calculation.
[Step S71] The cameras 20a and 20b (see FIG. 24) photograph the person trying on the clothes and transfer the photographed images to the control unit 11. Note that, similarly to step S11 above, at this time, it is desirable that the control unit 11 performs processing on each captured image to remove distortion of the lens of the camera 20 in the vertical direction.

[Step S72] The enlargement/reduction ratio calculation unit 11a calculates the difference between the person trying on the garment and the background from each photographed image, and extracts the area of the person trying on the garment (person area).
[Step S73] The enlargement/reduction ratio calculation unit 11a detects a circular shape from the human region to detect the head region of the person trying on the suit.

[Step S74] The enlargement/contraction rate calculation unit 11a detects the eye area of the wearer by detecting a circular shape from the head area.
[Step S75] The enlargement/reduction ratio calculation unit 11a detects the center coordinates of the two eye regions from each of the photographed images. The enlargement/reduction ratio calculation unit 11a uses triangulation to measure the 3D position (3D coordinate position) of the eye based on the center coordinates detected from each captured image. As a result, the distance x _r between the display 30 and the person trying on the item and the height _ye of the eye of the person trying on the item are determined.

[Step S76] The enlargement/reduction ratio calculation unit 11a acquires the remaining parameter information (the number of vertical pixels I of the photographed image and the height _yv of the display surface of the display 30) from the storage unit 12.
[Step S77] The enlargement/reduction ratio calculation unit 11a calculates the size based on the distance _xr between the display 30 and the person trying on the suit, the _height of the eye of the person trying on the suit, determined at step S75, and the parameter information obtained at step S76. , calculate the scaling ratio.

[Step S78] The image enlarging/reducing unit 11b multiplies the captured image by the calculated enlargement/reducing ratio to generate an enlarged/reduced image.
[Step S79] The control unit 11 transfers the enlarged/reduced image to the display 30.

FIG. 24 is a diagram illustrating an example of an image processing operation based on the third scaling factor calculation.
[Step S81] The cameras 20a and 20b photograph the person trying on the clothes and transfer the photographed images g11 and g12 to the control unit 11.

[Step S82] The enlargement/reduction ratio calculation unit 11a extracts human regions from the photographed images g11 and g12, and generates human region images g11-1 and g12-1.
[Step S83] The enlargement/reduction ratio calculation unit 11a extracts the head region from the human region images g11-1 and g12-1 to generate head regions g11-2 and g12-2.

[Step S84] The enlargement/reduction ratio calculation unit 11a extracts the eye region from the head regions g11-2 and g12-2, and generates eye region images g11-3 and g12-3.
[Step S85] The enlargement/reduction ratio calculation unit 11a measures the 3D position of the try-on's eye by triangulation from the eye center coordinates of the two eye area images g11-3 and g12-3.

[Step S86] The enlargement/contraction rate calculation unit 11a obtains the distance _xr between the display and the person trying on the garment and the height _ye of the eye of the person trying on the garment, among the parameter information, from the 3D position of the eye. Furthermore, the enlargement/reduction ratio calculation unit 11a acquires the remaining parameter information (the number of vertical pixels I of the photographed image and the height _yv of the display surface of the display 30) from the storage unit 12.

[Step S87] The expansion/contraction rate calculation unit 11a calculates the expansion/contraction rate based on the parameter information.
[Step S88] The image enlarging/reducing unit 11b multiplies the photographed image g11 (or photographed image g12) by an enlargement/reduction ratio to generate an enlarged/reduced image g20.

[Step S89] The control unit 11 transfers the enlarged/reduced image g20 to the display 30.
[Step S90] The display 30 displays the enlarged/reduced image g20.
As explained above, in the second embodiment, when the display surface of the display 30 is equally divided in the y-axis direction, the pixel size _{i in} 1 in the y-direction and the display surface are The enlargement/contraction ratio is calculated from the size _in 2 of the pixel in the y direction when the pixel is equally divided at angles in the y axis direction. Then, by applying an enlargement/reduction ratio to the photographed image, a corrected enlarged/reduced image is generated so that when the try-on person views it through the display 30, the upper body and lower body of the try-on person's image appear at a ratio close to the real thing. , is displayed on the display 30. As a result, even when the person trying on the item approaches the display 30, the person trying on the item can see the person trying on the item in a state similar to the actual item, and serviceability can be improved.

Note that the processing functions of the image processing apparatuses 1 and 10 of the present invention described above can be realized by a computer. In this case, a program is provided that describes the processing contents of the functions that the image processing apparatuses 1 and 10 should have. By executing the program on the computer, the above processing functions are realized on the computer.

A program that describes processing contents can be recorded on a computer-readable recording medium. Computer-readable recording media include magnetic storage devices, optical disks, semiconductor memories, and the like. Magnetic storage devices include hard disk drives (HDD), magnetic tapes, and the like. Optical discs include CDs, DVDs, and Blu-ray discs.

When a program is distributed, for example, a portable recording medium such as a CD on which the program is recorded is sold. It is also possible to store the program in the storage device of the server computer and transfer the program from the server computer to another computer via a network.

A computer that executes a program stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. Then, the computer reads the program from its own storage device and executes processing according to the program. Note that a computer can also directly read a program from a portable recording medium and execute processing according to the program. Furthermore, each time a program is transferred from a server computer connected via a network, the computer can sequentially execute processing according to the received program.

1 Image processing device 1a Receiving unit 1b Control unit 2 Camera 3 Display device 4 Person 4a Eye 5a Photographed image 5b Enlarged/reduced image 6a, 6b Image

Claims (7)

to the computer,
When displaying a photographed image of a person with a camera on a display device that the person views,
obtaining an enlargement/contraction ratio calculated for each pixel of the photographed image based on the eye height of the person and the distance between the person and the display surface of the display device;
generating a scaled image by scaling each pixel of the photographed image based on the scale ratio, and displaying the scaled image on the display device;
a first size of pixels of the photographed image when the distance is equally divided in the height direction of the display surface, and an angle is equally divided in the height direction of the display surface with the eye as the center with respect to the display surface; Calculating the enlargement/contraction ratio based on the second size of the pixels of the photographed image when
An image processing program that executes processing. The enlargement/contraction ratio for each pixel in the photographed image is
The shorter the distance, the vertical width of a pixel whose vertical position is closer to the eye height when the photographed image is drawn on the display surface, and the vertical width of a pixel whose position is farther from the eye height. is calculated so that it becomes larger,
The image processing program according to claim 1. The expansion/contraction rate is
the first size in the vertical direction of each pixel of the photographed image when the photographed image is drawn on the display surface;
the second size in the vertical direction of each divided area projected onto the display surface when the viewing angle from the eye to the display surface from the eye located at a position based on the height and the distance is divided by the same angle; Sato,
Calculated based on
The image processing program according to claim 1 or 2. A pixel selected from the photographed image when each pixel arranged in the vertical direction in the photographed image and each divided region arranged in the vertical direction on the display surface are selected one by one in the same direction. The scaling factor for the selected pixel is calculated based on the ratio of the first size for the divided area and the second size for the divided area corresponding to the selected pixel.
The image processing program according to claim 3. In acquiring the enlargement/reduction ratio, the height and the distance are calculated based on the positions of the upper end and the lower end of the person in the photographed image, and the enlargement/reduction ratio is calculated based on the calculated height and the distance.
The image processing program according to any one of claims 1 to 4. The computer is
When displaying a photographed image of a person with a camera on a display device that the person views,
obtaining an enlargement/contraction ratio calculated for each pixel of the photographed image based on the eye height of the person and the distance between the person and the display surface of the display device;
generating a scaled image by scaling each pixel of the photographed image based on the scale ratio, and displaying the scaled image on the display device;
a first size of pixels of the photographed image when the distance is equally divided in the height direction of the display surface, and an angle is equally divided in the height direction of the display surface with the eye as the center with respect to the display surface; Calculating the enlargement/contraction ratio based on the second size of the pixels of the photographed image when
Image processing method. a receiving unit that receives an image of a person taken by a camera;
When displaying the photographed image on a display device that the person views, calculate each pixel of the photographed image based on the height of the person's eyes and the distance between the person and the display surface of the display device. a control unit that obtains an enlargement/reduction ratio, generates an enlarged/reduced image by enlarging/reducing each pixel of the photographed image based on the enlargement/reduction ratio, and displays the enlarged/reduced image on the display device ;
The control unit is configured to determine a first size of pixels of the photographed image when the distance is equally divided in the height direction of the display surface, and a first size of the pixels of the photographed image in the height direction of the display surface with the eye as the center relative to the display surface. Calculating the enlargement/contraction ratio based on a second size of pixels of the photographed image when the angle is equally divided into
Image processing device.

Images