JP2014204289A - Image display device, image display system, and image display method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image display device capable of watching an image obtained by imaging a subject in a desired direction in simple configuration, an image display system and an image display method.SOLUTION: A storage section stores a plurality of first images of a subject captured from a predetermined position correspondingly to a first imaging order while the subject is rotated in one direction at least once. A first calculation section calculates a first imaging direction of an imaging apparatus relative to the subject by extracting a position of a second image of a specified object on the subject in at least two first images. The first calculation section calculates first imaging directions of the plurality of first images on the basis of the first imaging order and at least two first imaging directions. A setting section sets a second imaging direction of an image to be displayed. An image selection section selects an image of the first imaging direction corresponding to the second imaging direction from among the plurality of first images. A display section displays the selected first image.

Description

  The present invention relates to an image display device, an image display system, and an image display method.

  For example, you may want to check the rear view with little effort using figure-viewing. At home, at the time of changing clothes, makeup, and in-store examples, such as when trying on a boutique or a clothing store, checking a hairstyle at a beauty salon, or trying on a kimono store or a rental kimono store. However, it cannot be easily confirmed because there is a part that cannot be seen by conventional postures or an invisible part.

  For this reason, for example, there is known an example in which a rear-view image captured by an imaging device is wirelessly transmitted and displayed as an electronic image in front of the appearance user. There is also an example in which a reflecting dihedral mirror composed of left and right reflecting plane mirrors having an opening angle of approximately 90 degrees and a direct viewing plane mirror placed in front of the reflecting diplexer are opposed to each other in an upright posture. Are known. In this example, the object image is positioned within the mirror surface on the reflection plane mirror side of either the left or right side of the reflecting dihedral mirror from the center line of the reflecting dihedral mirror, and the object image has a rear view, a side view, and a front view. Each virtual image is simultaneously projected on a direct viewing plane mirror (see, for example, Patent Documents 1 and 2).

  There is also known an image display device that is arranged in front of a subject whose subject is the subject and displays a rear image. In this image display device, a video camera that captures a subject is disposed on the rear surface of the subject, and the video camera transmits an image signal obtained by capturing an image of the rear surface of the subject to the image display device, thereby displaying an image. An image of the rear surface of the subject is displayed on the device. As another example, an image photographing system for photographing a subject from a plurality of directions is also known. In this image capturing system, the actuator changes the position and orientation of the camera in response to the movement signal, and the camera captures the subject in response to the capturing timing signal (see, for example, Patent Documents 3 and 4).

  An example in which a video camera and a touch panel display with a camera are combined is also known. In such an example, for example, the captured still images can be displayed side by side to compare the coordinates. If you try on clothes and take a still picture, you can check the impression of a natural back without having to turn around and check. You can shoot and save your try-on as a still image with full high-definition, and view and compare multiple saved images simultaneously. Furthermore, a delay mirror is also known in which an image obtained by photographing a scene in front of a mirror is always reflected on the mirror with a delay of 3 seconds (see, for example, Non-Patent Documents 1 to 3).

Japanese Utility Model Publication No. 5-55955 JP 2009-134295 A JP 2000-138926 A JP 2004-248231 A

Yui Ogasawara “Interactive Mirror” Reflecting Back View, [online], April 15, 2010, ITmedia News, Internet <URL: http://www.itmedia.co.jp/news/articles/1004/15/ news078.html> PUMA STORE Harajuku INTERACTIVE MIRROR-YouTube, March 3, 2011, Internet <URL: http://www.youtube.com/watch?v=_X2-_t5f_IA> “Delay Mirror” that allows you to check your back thoroughly, [online], January 16, 2012, ITmedia, Internet <URL: http://nlab.itmedia.co.jp/nl/articles/1201/16 /news097.html>

  However, for example, as described above, in the case of an image display apparatus that takes a still image and compares it, the angle cannot be changed freely. An image capturing system that captures a subject from a plurality of directions requires an actuator in addition to the camera. In addition, in the case of a delay video, it is not possible to view it carefully at the desired angle.

  According to one aspect, an object is to provide an image display device, an image display system, and an image display method capable of viewing an image obtained by photographing a subject from a desired direction with a simple configuration.

  An image display device according to one aspect includes a storage unit, a first calculation unit, a setting unit, an image selection unit, and a display unit. The storage unit captures the plurality of first images of the plurality of first images of the object captured by the imaging device disposed at a predetermined position while the object rotates at least once in one direction. Stored in association with the first shooting order. The first calculation unit extracts the position of the second image of the specific object on the object in at least two of the plurality of first images, thereby extracting the position of the object with respect to the object. A first photographing direction indicating a relative posture of the photographing apparatus is calculated. The first calculation unit calculates a first shooting direction of each of the plurality of first images based on the first shooting order and the calculated at least two first shooting directions. The setting unit sets a second shooting direction of an image to be displayed. The image selection unit selects an image in the first shooting direction corresponding to the second shooting direction from the plurality of first images. The display unit displays the selected first image.

  An image display system according to another aspect includes an imaging device, a storage unit, a first calculation unit, a setting unit, an image selection unit, and a display unit. The imaging device is arranged at a predetermined position and captures a plurality of first images of the object while the object rotates at least once in one direction. The storage unit stores the plurality of photographed first images in association with a first photographing order in which the plurality of first images are photographed. The first calculation unit extracts the position of the second image of the specific object on the object in at least two of the plurality of first images, thereby extracting the position of the object with respect to the object. A first photographing direction indicating a relative posture of the photographing device is calculated in at least two of the first images. The first calculation unit calculates a first shooting direction of each of the plurality of first images based on the first shooting order and the calculated at least two first shooting directions. The setting unit sets a second shooting direction of an image to be displayed. The image selection unit selects an image in the first shooting direction corresponding to the second shooting direction from the plurality of first images. The display unit displays the selected first image.

  According to the image display device, the image display system, and the image display method of the above-described aspect, it is possible to view an image obtained by photographing a subject from a desired direction with a simple configuration.

It is a figure explaining the concept of the image display method by 1st Embodiment. It is a figure which shows the example of a display by the display apparatus by 1st Embodiment. It is a block diagram which shows an example of the hardware constitutions of the image display system by 1st Embodiment. It is a block diagram which shows an example of a structure of the image display system by 1st Embodiment. It is a block diagram which shows an example of a detailed structure of the stored image angle calculation part by 1st Embodiment. It is a figure which shows notionally the method of calculating a rotation angle from the image sequence by 1st Embodiment. It is a figure which shows the image table by 1st Embodiment. It is a flowchart which shows operation | movement of the image display system by 1st Embodiment by 1st Embodiment. It is a figure which shows notionally the image display method by 2nd Embodiment. It is a block diagram which shows an example of a structure of the stored image angle calculation part by 2nd Embodiment. It is a figure which shows notionally the image display method by 3rd Embodiment. It is a block diagram which shows an example of a structure of the image display system by 3rd Embodiment. It is a figure which shows notionally the image display method by 4th Embodiment. It is a figure which shows notionally the image display method by 5th Embodiment. It is a figure which shows the hardware constitutions of a standard computer.

(First embodiment)
The image display system 1 according to the first embodiment will be described below with reference to the drawings. FIG. 1 is a diagram for explaining the concept of the image display method according to the first embodiment. As shown in FIG. 1, in the image display method according to the first embodiment, an image display system 1 is used. The image display system 1 includes a camera 3, a display device 5, and a half mirror 7. The camera 3 is a photographing device capable of continuously photographing moving images or still images. The display device 5 is a device that displays an image taken by the camera 3. The half mirror 7 is a half mirror provided on the display surface of the display device 5. The image displayed on the display device 5 can be viewed through the half mirror 7. When no image is displayed on the display device 5, the half mirror 7 functions as a mirror.

  In the present embodiment, the object to be photographed is a person 11. In the field of view of the camera 3, the person 11 passes through the center of the body and rotates in the rotation direction 13 with the vertical axis as the rotation axis. The rotation direction 13 is indicated by a bidirectional arrow. The rotation may be in any direction, but it is preferable to rotate in any one direction during a series of photographing. The rotation is preferably performed by the person 11 himself.

  The image sequence 100 is a fixed camera 3 while the person 11 rotates one axis or more about a vertical axis as a rotation axis, for example, a plurality of still images or moving images of the whole body, half body, or face. These images are obtained by shooting and stored in a storage unit to be described later. In the example of FIG. 1, images 100-1 to 100-11 are shown together with an angle indicating the calculated shooting direction. An arbitrary image included in the image sequence 100 is hereinafter referred to as an image 100-n.

  Even during reproduction, the camera 3 captures the person 11 standing in the field of view of the camera 3. For example, the camera 3 stands in the field of view of the camera 3 of the image display system 1 and looks at the image displayed on the display device 5 while passing through the center of the body and centering on the vertical axis as in the rotation direction 15. The rotating person 11 is photographed. At this time, the image display system 1 detects the rotation of the person 11 in the rotation direction 15 and selects an image to be displayed on the display device 5 according to the detected rotation.

  FIG. 2 is a diagram illustrating a display example 25 by the display device 5. As shown in FIG. 2, in the display example 25, the display surface of the main body 9 includes a mirror unit 17 and a screen 19. In the display example 25, the mirror unit 17 shows a mirror image of the person 11, and the screen 19 shows the rear view of the person 11. When the person 11 rotates in the rotation direction 23, the image display system 1 detects the rotation, and displays an image captured at a rotation angle corresponding to the detected rotation angle on the screen 19.

  FIG. 3 is a block diagram illustrating an example of a hardware configuration of the image display system 1. As illustrated in FIG. 3, the image display system 1 includes a photographing device 31, an image display device 33, an arithmetic processing device 35, a storage device 37, and an input device 39. The photographing device 31 is a device that photographs a moving image or a still image such as the camera 3. The image display device 33 is a device that displays an image, such as the display device 5.

  The arithmetic processing device 35 is a device that performs arithmetic processing for controlling the operation of the image display system 1. The storage device 37 is a Read Only Memory (ROM), a Random Access Memory (RAM), an external storage device, a portable storage medium, and a driving device thereof. In the example of FIG. 1, the arithmetic processing device 35 and the storage device 37 may be incorporated in the display device 5, for example. The input device 39 is an input device capable of inputting information by a user operation, such as a keyboard and a touch panel. In the example of FIG. 1, the input device 39 can be a touch panel provided on the surface of the half mirror 7, for example.

  FIG. 4 is a block diagram illustrating an example of functions of the image display system 1. As shown in FIG. 4, the image display system 1 has functions of an imaging unit 51, an image sequence storage unit 52, an accumulated image angle calculation unit 53, a captured image angle calculation unit 55, a display setting unit 57, and an image display unit 61. doing. The photographing unit 51 controls the camera 3 to photograph the person 11, for example. The image sequence storage unit 52 stores, in the storage device 37, a plurality of images captured by the camera 3, for example, while the person 11 rotates once in association with the imaging order. The accumulated image angle calculation unit 53 calculates a rotation angle of the person 11 at the time of photographing each image 100-n by performing predetermined processing described later on the image sequence 100.

  The captured image angle calculation unit 55 calculates the rotation angle of the person 11 photographed at the time of reproduction. The display setting unit 57 sets the display center angle as the center of the rotation angle of the image displayed on the display device 5. In this setting, for example, the angle input by the user from the input device 39 can be set as the display center angle. The image selection unit 59 selects an image to be displayed based on information from the image sequence storage unit 52, the captured image angle calculation unit 55, and the display setting unit 57. A method for selecting an image will be described later. The image display unit 61 causes the display device 5 to display the image selected by the image selection unit 59.

  FIG. 5 is a block diagram illustrating an example of a detailed configuration of the accumulated image angle calculation unit 53. As shown in FIG. 5, the accumulated image angle calculation unit 53 includes an approximate front face extraction unit 71, an approximate front face angle calculation unit 73, an approximate front face angle setting unit 75, an approximate front face adjacent image angle difference calculation unit 77, and an adjacent An image pixel movement amount calculation unit 79, a unit angle difference calculation unit 81, and an approximate front face image angle difference summation unit 83 are included. The accumulated image angle calculation unit 53 includes a back image extraction unit 85, a back surface adjacent image pixel movement amount calculation unit 87, a back surface adjacent image angle difference calculation unit 89, a back surface adjacent image angle difference summation unit 91, an angle difference summation unit 93, and an angle adjustment. A coefficient calculation unit 95 is included. Furthermore, the accumulated image angle calculation unit 53 includes a back surface adjacent image adjustment angle difference calculation unit, a back surface image angle calculation unit 99, and a back surface image angle setting unit 101.

  In the present embodiment, the approximate front face extraction unit 71 extracts an image from which face parts such as eyes, nose and mouth can be detected cleanly from the image sequence 100 stored in the image sequence storage unit 52. “Precise detection” means that, for example, an angle calculation unit 73 (to be described later) can calculate the face angle by detecting a face part in the image 100-n. The face image at this time is referred to as a “substantially front face”, and an image including the “substantially front face” is referred to as a “substantially front face image”.

  The “front face” refers to a face image that is determined to be facing directly in front of the camera 3. The “front face” may be a face image that is closest to the state of facing the “right front” among the photographed face images. For example, the rotation angle of the person 11 in the image including the front face can be 0 degree.

  The approximate front face angle calculation unit 73 calculates the left and right rotation angles (also referred to as face orientations) of the extracted face image based on the arrangement of the face parts. This rotation angle is obtained from, for example, a face image obtained by capturing a face by detecting a nasal cavity and a face end point of one face existing in the face image, and detecting the detected nasal cavity and the detected end point. It is calculated based on information about the distance in the image. At this time, for example, “frontal” may be a state in which the nasal cavity is at the center of the face end point, or a state in which the center of the face end point is the center of both eyes.

  The approximate front face angle setting unit 75 sets the calculated rotation angle as an attribute to the approximate front face image extracted by the approximate front face extraction unit 71 in the image sequence storage unit 52. That is, the approximate front face angle setting unit 75 stores the image and the rotation angle in the image sequence storage unit 52 in association with each other. The approximate front face adjacent image angle difference calculation unit 77 calculates an angle difference between adjacent images of the approximate front face image. An adjacent image refers to an image 100-n in which the shooting order is continuous. For example, when an image is used by thinning a predetermined number in the shooting order, the adjacent image is an image in which the shooting order is continuous among the used images.

  The adjacent image pixel movement amount calculation unit 79 calculates the left and right movement amount of the face part between adjacent images of the approximate front face in units of pixels. In this case, a facial part that faces the camera 3 as much as possible is used. The unit angle difference calculation unit 81 calculates “the angle difference per unit pixel of the left / right movement amount” based on the movement amount calculated by the adjacent image pixel movement amount calculation unit 79.

  The rear image extraction unit 85 extracts an image other than the approximate front face image (hereinafter referred to as “rear image”) and the adjacent approximate front face from the image sequence storage unit 52. Instead of extracting the adjacent frontal face images, the range of the rotation angle including the frontal face can be set in advance, for example, approximately ± 30 degrees of the front face. The back side adjacent image pixel movement amount calculation unit 87 calculates, for example, the left and right movement amount of the head portion image between adjacent images of the back image in units of pixels using the optical flow technique. For example, based on the spatial and temporal changes in luminance between two adjacent images and the unit angle calculated by the unit angle difference calculation unit 81, the rotation angle difference of the object between the two adjacent images is calculated. In this case, the image for calculating the amount of left-right movement uses an image of a portion facing the camera as much as possible (that is, a portion close to the position of the rotation axis in the left-right direction of the image in this example).

  The back side adjacent image angle difference calculation unit 89 multiplies the movement amount between adjacent images of the back image by the “angle difference per unit pixel of the left and right movement amount” calculated by the unit angle difference calculation unit 81. The angle difference Δθ is calculated.

  The approximate front face image angle difference summation unit 83 obtains the sum of the angle differences between adjacent images in the image designated as the “substantially front face image”. The back side adjacent image angle difference summation unit 91 obtains the sum of the angle differences between the adjacent images of the “back side image”. The angle difference summation unit 93 obtains the sum of the angle difference sum of the approximate front face image and the angle difference sum of the back image. Ideally this value should be 360 degrees. However, in actuality, there are cases where the angle does not reach 360 degrees due to an error or the like. Therefore, the error is absorbed in the “back image” portion.

  The angle adjustment coefficient calculation unit 95 calculates a coefficient for dispersing the error with respect to the total 360 degrees of the angle differences to the angle of the back image. For example, when the total angle of the rear image is 330 degrees and the total approximate front face image angle is 40 degrees, the error E of error E = (330 + 40) −360 (degrees) is dispersed and absorbed by 330 degrees. In this example, the adjustment coefficient is coefficient C = (E + 330) / 330.

  The back side adjacent image adjustment angle difference calculation unit 97 performs calculation by multiplying the angle difference between adjacent images of the back image by a coefficient C for error absorption. That is, the back side adjacent image adjustment angle difference calculation unit 97 multiplies the angle difference calculated by the back side adjacent image angle difference calculation unit 89 by the coefficient C. The back image angle calculation unit 99 calculates the rotation angle of the person 11 in each image using the calculated angle difference between adjacent images. The rear image angle setting unit 101 stores the calculated rotation angle in association with each image 100-n in the image sequence storage unit 52.

  Returning to FIG. 4, the image selection method based on the reproduced image by the image selection unit 59 will be described. For example, when the person 11 is photographed by the camera 3 during reproduction, the rotation angle calculated by the captured image angle calculation unit 55 is φ (degrees). At this time, the captured image angle calculation unit 55 calculates the rotation angle φ based on the face orientation detected from the angle of the frontal face or the rotation angle φ in the rear image in the image captured during reproduction. In addition, the center angle of the display angle set by the display setting unit 57 is set as a display center angle θc (degrees). For example, when the center of the image to be displayed is the front and back, θc = 180 degrees can be set. For example, the image selection unit 59 can set an angle that is a rotation angle of an image to be displayed on the image display device 33 to display angle = θc−φ (degrees). Note that the image selection unit 59 preferably uses, as the display angle, an angle obtained by adding or subtracting an angle proportional to the rotation angle φ detected in the image taken at the time of reproduction to the display center angle θc. For example, the reproduction angle θd = θc−Kφ can be set. Here, K is a real number.

  FIG. 6 is a diagram conceptually illustrating a method for calculating the rotation angle from the image sequence 100. As illustrated in FIG. 6, the image sequence 100 includes a plurality of images 100-n captured while the person 11 rotates at least once, arranged in the order of capturing. In the image sequence 100, ranges 121 and 125 indicate ranges in which a face orientation can be calculated by detecting a face part and a rotation angle can be calculated. The range 123 is a range in which a rotation angle corresponding to a certain point on the image or the number of moving pixels of the image is calculated based on the relationship between the number of moving pixels of the face part calculated from the ranges 121 and 125 and the rotation angle. . In the range 123, the rotation angle is calculated by adjusting the calculated rotation angle by equally distributing the error.

  FIG. 7 is a diagram illustrating the image table 110 according to the first embodiment. The image table 110 is data in which a captured image is associated with a shooting order and a rotation angle. As shown in FIG. 7, in the image table 110, the shooting order 111, the image data 113, and the rotation angle 115 are associated with each other. The shooting order 111 is the shooting order of the images 100-n in the image sequence 100. The image data 113 is information indicating the image data of the image 100-n. The rotation angle 115 is a rotation angle calculated for the corresponding image. The image table 110 may include other information such as information indicating whether the image is a front face image or a back image.

  The above processing will be further described with reference to FIG. FIG. 8 is a flowchart showing the operation of the image display system 1 according to the first embodiment. In the image display system 1, for example, each processing is performed by the arithmetic processing device 35 reading and executing a program stored in the storage device 37. Here, for example, each function shown in FIGS. 4 and 5 is performed. It demonstrates as what performs a process.

  As illustrated in FIG. 8, the photographing unit 51 photographs a person 11 (user) rotating, for example, around the vertical axis by the camera 3 and accumulates the image sequence 100 in the image sequence accumulation unit 52 (S131). The accumulated image angle calculation unit 53 calculates the rotation angle θi of the photographed person 11 in each image 100-n accumulated in the image sequence accumulation unit 52, and associates them with each other as an attribute of each image 100-n. The data is stored in the image sequence storage unit 52 as in the table 110.

  The image sequence storage unit 52 repeats the above processing until the photographing is finished (S133: NO). When shooting is completed (S133: YES), the display setting unit 57 sets the angle θc of the reproduction center image and the reproduction magnification K (S134).

  The photographing unit 51 photographs the person 11 (S135). The captured image angle calculation unit 55 calculates the rotation angle φ of the photographed person 11 (S136). Based on the rotation angle φ of the person 11, a reproduction angle θd that is a rotation angle corresponding to the image to be reproduced is calculated. Here, θd = θc−Kφ. The image selection unit 59 selects the image sequence 100-n corresponding to the reproduction angle θd as a reproduction image by referring to the image table 110 (S137). The image display unit 61 displays the selected image (S138). If it is not determined that the reproduction has ended (S139: NO), the arithmetic processing unit 35 repeats the processing from S135, and if it is determined that the reproduction has ended, the processing ends (S139).

  As described above in detail, according to the image display system 1 according to the present embodiment, the camera 3 shoots, for example, the person 11 rotating around the vertical axis as at least one rotation. The image sequence storage unit 52 stores the shooting order and the shot image 100-n in association with each other. The accumulated image angle calculation unit 53 calculates the face orientation using a face component in an image having a generally front face. The image sequence storage unit 52 stores the calculated face orientation in association with the image 100-n. The accumulated image angle calculation unit 53 calculates the relationship between the pixel movement amount and the rotation angle based on the image 100-n and the detected face orientation. Further, the accumulated image angle calculation unit 53 determines the rotation angle in the rear image based on the relationship between the pixel movement amount calculated by, for example, the optical flow and the pixel movement amount calculated by the accumulated image angle calculation unit 53 and the rotation angle. calculate. The accumulated image angle calculation unit 53 further corrects the rotation angle in the back image so that the rotation of one rotation is 360 degrees.

  The captured image angle calculation unit 55 calculates the rotation angle φ of the person 11 captured by the camera 3 at the time of reproduction, and based on the display center angle θc set by the display setting unit 57 and the reproduction magnification K, the image to be reproduced The reproduction angle θd is calculated. The image selection unit 59 selects an image from the image sequence storage unit 52 based on the reproduction angle θd calculated by the captured image angle calculation unit 55. The image display unit 61 displays the image selected by the image selection unit 59.

  As described above, according to the image display system 1 according to the first embodiment, the rotation angle indicating the photographing direction can be calculated from the face orientation in the image having the almost front face that can be detected by the facial part. Further, by calculating the relationship between the pixel movement amount and the rotation angle in an image in which a facial part can be detected, the rotation angle can also be calculated in the rear image. Thus, in this embodiment, it is not necessary to use a turntable or a moving device in order to obtain the shooting direction. Therefore, it is possible to calculate the rotation angle indicating the photographing direction of the photographed person with only a simple preparation of installing the display device 5 integrated with the camera 3 at the site. In addition, since there is no need for the user to get on the turntable, there is no concern that a safety problem will occur such as the user breaking the balance at the start of rotation of the turntable.

  The person 11 is requested to rotate on the spot, and it is photographed. However, even if the angular velocity varies due to the rotation of the supporting leg during rotation, even if the angular velocity varies, the specific object of the photographed image (or The rotation angle can be calculated based on the movement of a specific point. For this reason, it is possible to accurately calculate the actual rotation angle of the subject. Therefore, the image at the time of reproduction can be smoothed, and an image taken from a desired angle can be surely confirmed without any sense of incongruity. In addition, it is not necessary to measure the head size of the person 11 or the distance at the time of photographing, and the rotation angle can be calculated only from the photographed image. Therefore, it is not necessary to go through complicated procedures, and simple use is possible. Since the angle indicating the shooting direction is adjusted so that the total angle difference between adjacent images is 360 degrees per rotation, the angle can be calculated more accurately.

  At the time of reproduction, the display center angle θc can be set in advance. Further, by capturing the person 11 with the camera 3 and obtaining the rotation angle at the time of reproduction, it is possible to display the image 100-n in the direction rotated by an angle proportional to the obtained rotation angle from the display center angle θc. become. Therefore, the appearance from a favorite angle can be confirmed using natural gestures such as rotation of the user's face as an operation input. Furthermore, since the half mirror 7 is provided, various modifications such as enabling both a mirror image and an image to be observed are possible.

(Second Embodiment)
Hereinafter, an image display system according to the second embodiment will be described with reference to FIGS. 9 and 10. FIG. 9 is a diagram conceptually illustrating an image display method according to the second embodiment, and FIG. 10 is a block diagram illustrating an example of a configuration of an accumulated image angle calculation unit according to the second embodiment. In addition, about the structure and operation | movement similar to the image display system 1 by 1st Embodiment, the same code | symbol is attached | subjected and duplication description is abbreviate | omitted.

  Similar to the image display system 1, the image display system according to the second embodiment has the hardware configuration shown in FIG. 3, and replaces the stored image angle calculation unit 53 in the image display system 1 shown in FIG. The accumulated image angle calculation unit 240 is provided.

  As shown in FIG. 9, in the present embodiment, the accumulated image angle calculation unit 240 detects two front faces. In other words, the accumulated image angle calculation unit 240 uses two images (for example, an image 100-1 and an image 100-11) that are different in one rotation that are determined to include the front face based on the face orientation using the face recognition technology. )To detect. The accumulated image angle calculation unit 240 sets a rotation angle for each image 100-n on the assumption that images taken between two front face images are taken at equal intervals.

  As shown in FIG. 10, the accumulated image angle calculation unit 240 includes an approximate front face extraction unit 71, an approximate front face angle calculation unit 73, a front face identification unit 241, and an angle setting unit 243. When the rotation angle of the image including the front face is calculated by the front face angle calculation unit 73, the front face specification unit 241 specifies the front face image. The angle setting unit 243 calculates a rotation angle corresponding to each image 100-n based on the image including the identified front face and the shooting order of the other images 100-n.

  As described above, according to the image display system according to the second embodiment, two front face images are specified, and an image in the shooting order between the two images is shot at an equal rotation angle. Set the rotation angle.

  As described above, according to the image display system according to the second embodiment, it is possible to calculate the rotation angle even in the rear image, as in the first embodiment. At this time, it is not necessary to use a turntable or a moving device in order to obtain the shooting direction. Therefore, it is possible to calculate the rotation angle indicating the photographing direction of the photographed person with only a simple preparation of installing the display device 5 integrated with the camera 3 at the site. In addition, since there is no need for the user to get on the turntable, there is no concern that a safety problem will occur such as the user breaking the balance at the start of rotation of the turntable.

  Furthermore, according to the second embodiment, it is possible to reduce the processing amount by the arithmetic processing unit 35 as compared with the calculation method of the rotation angle in the image display system 1 according to the first embodiment, which is simplified. The rotation angle can be calculated.

(Third embodiment)
Hereinafter, an image display system according to the third embodiment will be described with reference to FIGS. 11 and 12. FIG. 11 conceptually illustrates an image display method according to the third embodiment, and FIG. 12 is a block diagram illustrating an example of a configuration of an accumulated image angle calculation unit according to the third embodiment. In addition, about the structure and operation | movement similar to the image display system 1 by 1st or 2nd embodiment, the same code | symbol is attached | subjected and duplication description is abbreviate | omitted. Similar to the image display system 1, the image display system according to the third embodiment has the hardware configuration shown in FIG.

  As shown in FIG. 11, in the present embodiment, the rotation angle is not calculated for a captured image that is captured during reproduction. Here, the image 100-1 is detected as a front image determined to include a front face. In addition, a reproduction center image is detected as an image 100-8 as an image displayed when a front image is detected in an image captured during reproduction.

  In the present embodiment, the image display system determines whether or not the image captured at the time of reproduction matches each of the image sequence 100, and detects a matching image that matches. In the example of FIG. 11, the matching image is the image 100-3. At this time, it is assumed that the match image is x (x is an integer) from the front image. In the example of FIG. 11, x = 2. The display image is the −xth image from the reproduction image. In the example of FIG. 11, the image 100-6.

  FIG. 12 is a block diagram showing an example of the configuration of the image display system 250 according to the third embodiment. As shown in FIG. 12, the image display system 250 includes an imaging unit 51, an image sequence storage unit 52, an accumulated image angle calculation unit 53, a captured image comparison unit 251, an image selection unit 253, a display setting unit 57, and an image display unit 61. have.

  The image display system 250 includes a captured image comparison unit 251 instead of the captured image angle calculation unit 55 in the image display system 1 and an image display unit 61 instead of the image selection unit 59.

  The captured image comparison unit 251 captures the person 11 at the time of reproduction, compares the captured image with the image 100-n stored in the image sequence storage unit 52, and detects the best matching image. The comparison for detecting the coincidence image can be performed by a known method, for example, calculating a correlation degree related to luminance. The image selection unit 253 calculates the number x of the matching images matched by the captured image comparison unit 251 in the shooting order from the front image, and for example, the −xth image from the display image set by the display setting unit 57. Is detected as a display image.

  As described above, according to the image display system 250 according to the third embodiment, it is not necessary to calculate the rotation angle in the image taken at the time of reproduction. In the present embodiment, the shooting direction is expressed as, for example, the number x of images in the shooting order from the front image.

  As described above, in the present embodiment, as in the first and second embodiments, the shooting direction can be calculated as the number of images corresponding to the shooting order even in the back image. At this time, it is not necessary to use a turntable or a moving device in order to obtain the shooting direction. Therefore, it is possible to calculate the photographing direction indicating the photographing direction of the photographed person with only a simple preparation of installing the display device 5 integrated with the camera 3 at the site. In addition, since there is no need for the user to get on the turntable, there is no concern that a safety problem will occur such as the user breaking the balance at the start of rotation of the turntable.

  Furthermore, according to the third embodiment, there is no need to calculate the rotation angle of an image taken during reproduction. Further, for example, even if only an image whose face is not visible at the time of reproduction is captured, a matching image can be detected by comparison with the image sequence 100, and a corresponding display image can be displayed. The present embodiment is an image display method that is advantageous in an environment where the degree of coincidence of images is easily calculated.

(Fourth embodiment)
The image display system according to the fourth embodiment will be described below with reference to FIG. FIG. 13 is a diagram conceptually illustrating an image display method according to the third embodiment. In addition, about the structure and operation | movement similar to the image display system by 1st-3rd embodiment, the same code | symbol is attached | subjected and duplication description is abbreviate | omitted. The image display method in the image display system according to the fourth embodiment can be used as a modification in the first to third embodiments. Therefore, the hardware configuration and the functional block configuration are the same as any of the configurations described in the first to third embodiments.

  As shown in FIG. 13, in the fourth embodiment, the camera 3 captures the person 11 while the person 11 rotates more than a predetermined angle over one rotation. For example, the image sequence storage unit 52 captures, for example, the image 200-14 until the person 11 rotates about 45 degrees after the image 200-11 in which the person 11 rotates once, as in the image sequence 200, for example. So that the image is taken. Then, when playing back an image whose shooting order is close to the image 200-11, for example, included in the range 202, an image close to the image 200-11 is displayed.

  As described above, according to the display method according to the fourth embodiment, a smoother display can be obtained when the vicinity of an image in which the person 11 is rotated once is displayed.

(Fifth embodiment)
The image display system according to the fifth embodiment will be described below with reference to FIG. FIG. 14 is a diagram conceptually illustrating an image display method according to the fifth embodiment. In addition, about the structure and operation | movement similar to the image display system by 1st-4th embodiment, the same code | symbol is attached | subjected and duplication description is abbreviate | omitted. The image display method in the image display system according to the fifth embodiment can be used as a modification in the first to fourth embodiments. Therefore, the hardware configuration and the functional block configuration are the same as any of the configurations described in the first to fourth embodiments.

  As shown in FIG. 14, in the display example 230 in the fifth embodiment, the front image 232 and the back image 234 are simultaneously displayed on one screen.

  According to the image display system according to the fifth embodiment, for example, if it is only the rear view, it is difficult to understand that if the orientation of the body changes, the image will also change, but the presence of the front view changes the orientation of the body. Can be understood intuitively.

  The present invention is not limited to the embodiments described above, and various configurations or embodiments can be adopted without departing from the gist of the present invention. For example, in the first to fifth embodiments, the face rotation angle is calculated at the time of image reproduction, but the calculated angle is not limited to this. For example, a head, a shoulder, or other images may be used. The calculation method of the rotation angle is not limited to the above method, and other conventionally known methods may be used.

  The display example in the display device 5 is not limited to the above, and the layout, display timing, and the like may be changed as appropriate. For example, the display device 5 may display an image on the entire surface of the main body 9. Further, the display device 5 may simultaneously display images taken from various directions. Display timing can be displayed by delaying the change in the rotation angle measured by the change in the face angle measured during playback, or by arranging the images taken from the front facing to the maximum rotation angle. It can be transformed, such as displaying.

  Here, an example of a computer that is commonly applied to cause a computer to perform the operations of the image display methods according to the first to fifth embodiments will be described. FIG. 15 is a block diagram illustrating an example of a hardware configuration of a standard computer. As shown in FIG. 15, a computer 300 includes a central processing unit (CPU) 302, a memory 304, an input device 306, an output device 308, an external storage device 312, a medium driving device 314, a network connection device, and the like via a bus 310. It is connected.

  The CPU 302 is an arithmetic processing unit that controls the operation of the entire computer 300. The memory 304 is a storage unit for storing in advance a program for controlling the operation of the computer 300 or using it as a work area when necessary when executing the program. The memory 304 is, for example, a random access memory (RAM), a read only memory (ROM), or the like. The input device 306 is a device that, when operated by a computer user, acquires various information input from the user associated with the operation content and sends the acquired input information to the CPU 302. Keyboard device, mouse device, etc. The output device 308 is a device that outputs a processing result by the computer 300, and includes a display device and the like. For example, the display device displays text and images according to display data sent by the CPU 302. The display device can be, for example, the display device 5.

  The external storage device 312 is a storage device such as a hard disk, and stores various control programs executed by the CPU 302, acquired data, and the like. The medium driving device 314 is a device for writing to and reading from the portable recording medium 316. The CPU 302 can perform various control processes by reading and executing a predetermined control program recorded on the portable recording medium 316 via the recording medium driving device 314. The portable recording medium 316 is, for example, a Compact Disc (CD) -ROM, a Digital Versatile Disc (DVD), a Universal Serial Bus (USB) memory, or the like. The network connection device 318 is an interface device that manages transmission / reception of various data performed between the outside by wired or wireless. A bus 310 is a communication path for connecting the above devices and the like to exchange data.

  A program that causes a computer to execute the image display methods according to the first to fourth embodiments is stored in, for example, the external storage device 312. The CPU 302 reads a program from the external storage device 312 and causes the computer 300 to perform an image display operation. At this time, first, a control program for causing the CPU 302 to perform image display processing is created and stored in the external storage device 312. Then, a predetermined instruction is given from the input device 306 to the CPU 302 so that the control program is read from the external storage device 312 and executed. The program may be stored in the portable recording medium 316.

  The storage device 37 and the image sequence storage unit 52 are examples of a storage unit, the stored image angle calculation unit 53 is an example of a first calculation unit, and the display setting unit 57 is an example of a setting unit. The display device 5, the image display device 33, and the output device 308 are examples of a display unit. The camera 3 and the imaging unit 51 are an example of an imaging device. The back image angle calculation unit 99 is an example of a fourth calculation unit, and the captured image angle calculation unit 55 is an example of a fifth calculation unit. The CPU 302 is an example of first to fifth calculation units and a setting unit.

Regarding the above embodiment, the following additional notes are disclosed.
(Appendix 1)
First imaging in which the plurality of first images are captured from a plurality of first images of the object captured by an imaging device disposed at a predetermined position while the object rotates at least once in one direction. A storage unit for storing in association with the order;
By extracting the position of the second image of the specific object on the object in at least two of the plurality of first images, the relative posture of the imaging device with respect to the object And calculating a first shooting direction of each of the plurality of first images based on the first shooting order and the calculated at least two first shooting directions. A first calculation unit that
A setting unit for setting the second shooting direction of the image to be displayed;
An image selection unit for selecting an image in the first shooting direction corresponding to the second shooting direction from the plurality of first images;
A display for displaying the selected first image;
An image display device comprising:
(Appendix 2)
further,
The first calculation unit includes:
A second calculating unit that calculates an angle indicating the first photographing direction based on a position where the second image is extracted with respect to the first image from which the second image is extracted;
A third calculator that calculates a relationship between the position difference and the angle difference in at least two of the first images;
With respect to the first image from which the second image is not extracted, the relation and the first imaging order are based on two spatial and temporal changes in luminance in the two adjacent first images. A fourth calculator that calculates an angle indicating the first imaging direction by calculating a rotation angle difference of the object between the first images;
The image display device according to appendix 1, further comprising:
(Appendix 3)
The object is a person;
The second image is an image of a facial part,
The first angle calculation unit calculates the angle indicating the first shooting direction by calculating a face orientation based on the face part when the face part is extracted from the first image. The image display device according to appendix 2, wherein:
(Appendix 4)
In the fourth calculation unit, the total sum of the rotation angle differences between the plurality of first images adjacent to each other in the first imaging order taken between two images taken from the first direction is 360. The image display device according to appendix 2 or appendix 3, wherein the rotation angle difference calculated when the second image is not extracted is uniformly corrected so that the second image is extracted.
(Appendix 5)
Furthermore, the first calculation unit extracts at least two images obtained by capturing the object from a first direction by extracting a second image from the plurality of first images, and the first calculation unit extracts the first image. For a predetermined number of the first images in which the imaging order is between the two extracted images from the first direction, the angle indicating the first imaging direction is set to 360 degrees and the predetermined number The image display device according to claim 1, wherein the angle is calculated as an angle obtained by adding the values divided by the first shooting order.
(Appendix 6)
further,
The storage unit stores a plurality of first images of the object photographed from the same position while the rotating object rotates at least one rotation exceeding a predetermined angle in one direction.
The image selection unit, when continuously displaying the first images of the object, preferentially selects the first images whose shooting order is closer to each other. The image display device according to any one of 1 to appendix 5.
(Appendix 7)
The display unit simultaneously displays the first image in a second shooting direction and the first image in a shooting direction corresponding to a direction different from the second shooting direction by 180 degrees. The image display device according to any one of appendix 1 to appendix 6.
(Appendix 8)
An imaging device that is arranged at a predetermined position and that captures a plurality of first images of the object while the object rotates at least once in one direction;
A storage unit that stores the plurality of photographed first images in association with a first photographing order in which the plurality of first images are photographed;
By extracting the position of the second image of the specific object on the object in at least two of the plurality of first images, the relative posture of the imaging device with respect to the object Is calculated in at least two of the first images, and based on the first imaging order and the calculated at least two first imaging directions, the plurality of first images are calculated. A first calculation unit for calculating each first imaging direction;
A setting unit for setting the second shooting direction of the image to be displayed;
An image selection unit for selecting an image in the first shooting direction corresponding to the second shooting direction from the plurality of first images;
An image display system comprising: a display unit configured to display the selected first image.
(Appendix 9)
The imaging device further captures a third image different from the first image of the object,
Furthermore, in the third image, a fifth calculation unit that calculates a third shooting direction in which the target object is shot by extracting the position of the second image of the specific object on the target object. ,
Have
The image selection unit selects one image from the plurality of the first images according to a first shooting direction corresponding to the third shooting direction and the second shooting direction. The image display system according to appendix 8.
(Appendix 10)
The imaging device further captures a third image different from the first image of the object,
Further, by extracting a first image closest to the third image captured by the imaging device from a plurality of the first images, a third imaging direction in which the object is captured is determined. A fifth calculation unit for calculating,
Have
The image selection unit selects one image from the plurality of the first images according to a first shooting direction corresponding to the third shooting direction and the second shooting direction. The image display system according to appendix 8.
(Appendix 11)
further,
The first calculation unit includes:
A second calculating unit that calculates an angle indicating the first photographing direction based on a position where the second image is extracted with respect to the first image from which the second image is extracted;
A third calculator that calculates a relationship between the position difference and the angle difference in at least two of the first images;
With respect to the first image from which the second image is not extracted, the relationship and the first imaging order are based on the spatial and temporal changes in luminance in the two adjacent first images. A fourth calculator that calculates an angle indicating the first imaging direction by calculating a rotation angle difference of the object between the first images;
The image display system according to any one of appendix 8 to appendix 10, further comprising:
(Appendix 12)
The image display device
First imaging in which the plurality of first images are captured from a plurality of first images of the object captured by an imaging device disposed at a predetermined position while the object rotates at least once in one direction. By extracting the position of the second image of the specific object on the object in at least two of the first images stored in association with the order, the object with respect to the object is extracted. A first photographing direction indicating a relative posture of the photographing device is calculated, and each of the plurality of first images is calculated based on the first photographing order and the calculated at least two first photographing directions. Calculate the first shooting direction,
Set the second shooting direction of the image to be displayed,
Selecting an image of the first shooting direction corresponding to the second shooting direction from the plurality of first images;
Displaying the selected first image;
An image display method characterized by the above.
(Appendix 13)
The image display system
While the object rotates at least once in one direction, a plurality of first images of the object are taken from a predetermined position,
Storing the plurality of photographed first images in association with a first photographing order in which the plurality of first images are photographed;
By extracting the position of the second image of the specific object on the object in at least two of the plurality of first images, the relative posture of the imaging device with respect to the object And calculating a first shooting direction of each of the plurality of first images based on the first shooting order and the calculated at least two first shooting directions. And
Set the second shooting direction of the image to be displayed,
Selecting an image of the first shooting direction corresponding to the second shooting direction from the plurality of first images;
An image display method comprising displaying the selected first image.
(Appendix 14)
further,
Taking a third image different from the first image of the object;
In the third image, by extracting the position of the second image of the specific object on the object, a third imaging direction in which the object is imaged is calculated,
14. The image according to appendix 13, wherein an image corresponding to the first shooting direction corresponding to the third shooting direction and the second shooting direction is selected from a plurality of the first images. Display method.
(Appendix 15)
further,
Taking a third image different from the first image of the object;
By extracting a first image closest to the third image captured by the imaging device from a plurality of the first images, a third imaging direction in which the object is captured is calculated. ,
14. The image according to appendix 13, wherein an image corresponding to the first shooting direction corresponding to the third shooting direction and the second shooting direction is selected from a plurality of the first images. Display method.

DESCRIPTION OF SYMBOLS 1 Image display system 3 Camera 5 Display apparatus 7 Half mirror 9 Main body 11 Person 13 Rotation direction 15 Rotation direction 17 Mirror part 19 Screen 21 Rotation direction 23 Rotation direction 25 Display example 31 Shooting device 33 Image display device 35 Arithmetic processing device 37 Storage device 39 Input Device 51 Shooting Unit 52 Image Sequence Storage Unit 53 Stored Image Angle Calculation Unit 55 Captured Image Angle Calculation Unit 57 Display Setting Unit 59 Image Selection Unit 61 Image Display Unit 71 General Front Face Extraction Unit 73 General Front Face Angle Calculation Unit 75 Front face angle setting section 77 General front face adjacent image angle difference calculation section 79 Adjacent image pixel movement amount calculation section 81 Unit angle difference calculation section 83 General front face image angle difference summation section 85 Back image extraction section 87 Back surface adjacent image pixel movement amount Calculation unit 89 Back side adjacent image angle difference calculation unit 91 Back side adjacent image angle difference summation unit 93 Angle difference summation unit 95 Angle adjustment coefficient calculation unit 97 Surface adjacent image adjustment angle difference calculation unit 99 back image angle calculator 101 back image angle setting unit 100 image sequence 100-n image 110 image table 111 imaging sequence 113 image data 115 rotation angle

Claims (12)

First imaging in which the plurality of first images are captured from a plurality of first images of the object captured by an imaging device disposed at a predetermined position while the object rotates at least once in one direction. A storage unit for storing in association with the order;
By extracting the position of the second image of the specific object on the object in at least two of the plurality of first images, the relative posture of the imaging device with respect to the object And calculating a first shooting direction of each of the plurality of first images based on the first shooting order and the calculated at least two first shooting directions. A first calculation unit that
A setting unit for setting the second shooting direction of the image to be displayed;
An image selection unit for selecting an image in the first shooting direction corresponding to the second shooting direction from the plurality of first images;
A display for displaying the selected first image;
An image display device comprising: further,
The first calculation unit includes:
A second calculating unit that calculates an angle indicating the first photographing direction based on a position where the second image is extracted with respect to the first image from which the second image is extracted;
A third calculator that calculates a relationship between the position difference and the angle difference in at least two of the first images;
With respect to the first image from which the second image is not extracted, the relation and the first imaging order are based on two spatial and temporal changes in luminance in the two adjacent first images. A fourth calculator that calculates an angle indicating the first imaging direction by calculating a rotation angle difference of the object between the first images;
The image display device according to claim 1, further comprising: The object is a person;
The second image is an image of a facial part,
The first angle calculation unit calculates the angle indicating the first shooting direction by calculating a face orientation based on the face part when the face part is extracted from the first image. The image display device according to claim 2.   In the fourth calculation unit, the total sum of the rotation angle differences between the plurality of first images adjacent to each other in the first imaging order taken between two images taken from the first direction is 360. 4. The image display device according to claim 2, wherein the rotation angle difference calculated when the second image is not extracted is uniformly corrected so that the second image is not extracted. 5.   Furthermore, the first calculation unit extracts at least two images obtained by capturing the object from a first direction by extracting a second image from the plurality of first images, and the first calculation unit extracts the first image. For a predetermined number of the first images in which the imaging order is between the two extracted images from the first direction, the angle indicating the first imaging direction is set to 360 degrees and the predetermined number The image display apparatus according to claim 1, wherein the angle is calculated as an angle obtained by adding the values divided by the first imaging order. further,
The storage unit stores a plurality of first images of the object photographed from the same position while the rotating object rotates at least one rotation exceeding a predetermined angle in one direction.
The said image selection part preferentially selects the said 1st image with which the said imaging | photography order is mutually closer, when displaying the said 1st image of the said object continuously. The image display device according to any one of claims 1 to 5.   The display unit simultaneously displays the first image in a second shooting direction and the first image in a shooting direction corresponding to a direction different from the second shooting direction by 180 degrees. The image display device according to claim 1. An imaging device that is arranged at a predetermined position and that captures a plurality of first images of the object while the object rotates at least once in one direction;
A storage unit that stores the plurality of photographed first images in association with a first photographing order in which the plurality of first images are photographed;
By extracting the position of the second image of the specific object on the object in at least two of the plurality of first images, the relative posture of the imaging device with respect to the object Is calculated in at least two of the first images, and based on the first imaging order and the calculated at least two first imaging directions, the plurality of first images are calculated. A first calculation unit for calculating each first imaging direction;
A setting unit for setting the second shooting direction of the image to be displayed;
An image selection unit for selecting an image in the first shooting direction corresponding to the second shooting direction from the plurality of first images;
An image display system comprising: a display unit configured to display the selected first image. The imaging device further captures a third image different from the first image of the object,
Furthermore, in the third image, a fifth calculation unit that calculates a third shooting direction in which the target object is shot by extracting the position of the second image of the specific object on the target object. ,
Have
The image selection unit selects one image from the plurality of the first images according to a first shooting direction corresponding to the third shooting direction and the second shooting direction. The image display system according to claim 8. The imaging device further captures a third image different from the first image of the object,
Further, by extracting a first image closest to the third image captured by the imaging device from a plurality of the first images, a third imaging direction in which the object is captured is determined. A fifth calculation unit for calculating,
Have
The image selection unit selects one image from the plurality of the first images according to a first shooting direction corresponding to the third shooting direction and the second shooting direction. The image display system according to claim 8. The image display device
First imaging in which the plurality of first images are captured from a plurality of first images of the object captured by an imaging device disposed at a predetermined position while the object rotates at least once in one direction. By extracting the position of the second image of the specific object on the object in at least two of the first images stored in association with the order, the object with respect to the object is extracted. A first photographing direction indicating a relative posture of the photographing device is calculated, and each of the plurality of first images is calculated based on the first photographing order and the calculated at least two first photographing directions. Calculate the first shooting direction,
Set the second shooting direction of the image to be displayed,
Selecting an image of the first shooting direction corresponding to the second shooting direction from the plurality of first images;
Displaying the selected first image;
An image display method characterized by the above. The image display system
While the object rotates at least once in one direction, a plurality of first images of the object are taken from a predetermined position,
Storing the plurality of photographed first images in association with a first photographing order in which the plurality of first images are photographed;
By extracting the position of the second image of the specific object on the object in at least two of the plurality of first images, the relative posture of the imaging device with respect to the object And calculating a first shooting direction of each of the plurality of first images based on the first shooting order and the calculated at least two first shooting directions. And
Set the second shooting direction of the image to be displayed,
Selecting an image of the first shooting direction corresponding to the second shooting direction from the plurality of first images;
An image display method comprising displaying the selected first image.