JP2015023512A - Imaging apparatus, imaging method and imaging program for imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To acquire a panoramic image capable of enhancing a subject etc. viewed by a photographer.SOLUTION: An imaging apparatus includes: imaging means for imaging a range including an object and at least an eye part of a photographer; image processing means for acquiring the image data of the imaging means; eye line direction determination means for determining the eye line direction of the photographer in the image data acquired by the image processing means; main object determination means for determining a main object in the object the photographer intends to image, on the basis of the eye line direction determined by the eye line direction determination means; and enhancement display means for performing enhanced display of the main object determined by the main object determination means.

Description

  The present invention relates to a photographing apparatus, a photographing method, and a photographing program for a photographing apparatus that photograph a subject in a wide photographing range such as an omnidirectional, full circumference, and panorama.

  For example, Patent Document 1 discloses a technique of an image capturing apparatus that captures an object in an omnidirectional image capturing range. This patent document 1 detects a direction in which an imaging unit that captures a panoramic image in a 360 ° direction along a predetermined imaging plane and a component surface of the imaging reproduction apparatus housing in which the imaging unit is disposed are opposed to each other. Detection means, recording means for recording image data of a panoramic image obtained by photographing with the imaging means in association with information indicating the direction detected by the detection means, display means for displaying an image, and image recorded by the recording means Display control means for displaying a panoramic image on the display means so that a predetermined reference direction is the center of the image based on information indicating the direction recorded in association with the data. It is disclosed that a person who views a panoramic image can easily understand the direction in which the subject in the panoramic image was located at the time of shooting.

JP 2011-40898 A

  In a wide range of shooting such as omnidirectional shooting, panoramic shooting, panoramic shooting, it is not possible to know which part of the image of the wide angle of view the photographer was watching and which subject was shot for the purpose. It was only possible to convey only an image with a weak impression that the intention was not transmitted by simple recording.

  An object of the present invention is to provide a photographing apparatus and a photographing device capable of transmitting an image having a wide angle of view such as a panoramic image and an omnidirectional image, and yet capable of acquiring an image capable of conveying a main object viewed by a photographer. A method and an imaging program for an imaging apparatus are provided.

  An imaging apparatus according to a main aspect of the present invention includes an imaging unit that captures an image in a range including a subject and at least an eye part of a photographer, an image processing unit that acquires image data of the imaging unit, and the image processing unit. Eye direction determination means for determining the eye direction of the photographer in the acquired image data, and the main subject in the subject to be photographed by the photographer based on the eye direction determined by the eye direction determination means Main subject determining means for determining a subject; and highlighting means for highlighting the main subject determined by the main subject determining means.

  An imaging method according to a main aspect of the present invention is to shoot a subject by an imaging unit having an imaging range including the subject and at least an eye part of the photographer, acquire image data of the imaging range by an image processing unit, The eye direction of the photographer is determined by the eye direction determination means, and the main subject in the subject to be photographed by the photographer is highlighted based on the determined eye direction.

  An imaging program of an imaging apparatus according to a main aspect of the present invention includes an imaging function that causes a computer to capture an image within a range including a subject and at least an eye part of a photographer, and an image processing function that acquires image data of the imaging function. The photographer intends to photograph based on the eye direction determined by the eye direction determination function and the eye direction determination function for determining the eye direction of the photographer in the image data acquired by the image processing function. A main subject determination function for determining a main subject in the subject to be determined, and a highlight display function for highlighting the main subject determined by the main subject determination function.

  ADVANTAGE OF THE INVENTION According to this invention, the imaging device which can acquire the wide-angle image which can emphasize the to-be-photographed object etc. which the photographer was looking at, the imaging | photography method, and the imaging | photography program of an imaging device can be provided.

1 is a block configuration diagram showing an embodiment of a photographing apparatus according to the present invention. The block diagram which shows the imaging optical system in the same apparatus. The figure which shows the mode of the omnidirectional imaging | photography when acquiring a panoramic image by imaging | photography of the same apparatus. The figure which shows the imaging | photography result of the omnidirectional imaging | photography by the same apparatus. The figure which shows the panoramic image acquired by the same apparatus. The figure for demonstrating the setting of a reference | standard gaze direction in the same apparatus. The figure which shows the state when setting a reference | standard gaze direction by the presence or absence of a photographer's nostril part with the same apparatus. The figure which shows the state in which the eyes | visual_axis of a photographer are orient | assigned to the to-be-photographed object when image | photographing with the same apparatus. The figure which shows the state when determining a photographer's pupil position by the same apparatus. The figure which shows the state when determining a photographer's pupil position by the same apparatus. The figure which shows the state when setting a reference | standard gaze direction by the photographer's pupil position with the same apparatus. The figure which shows that a photographer's eyes | visual_axis direction exists ahead by the same apparatus. The figure which shows that a photographer's eyes | visual_axis direction exists in the left side toward self from the same apparatus. The figure which shows that a photographer's eyes | visual_axis direction exists on the right side toward self by the same apparatus. The figure which shows that a photographer's eyes | visual_axis direction exists in the left side toward self from the same apparatus. The figure which shows that a photographer's eyes | visual_axis direction exists on the right side toward self by the same apparatus. The figure which shows the setting of angle (phi) of the vertical width in the panoramic image acquired by the apparatus. The figure which shows the setting of the emphasis range with respect to the group of persons which consist of several persons with the same apparatus. The figure for demonstrating the basis of the setting of the emphasis range by the parameter | index in the apparatus. The camera control flowchart in the same apparatus. FIG. 3 is a flowchart for determining the direction of the eye of the apparatus in the apparatus. The direction determination flowchart of a gaze direction in the same apparatus.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram of the photographing apparatus 10. The apparatus 10 captures a subject in a photographing range of approximately 360 ° in almost all directions (hereinafter, abbreviated as “omnidirectional”) and acquires, for example, a panoramic image thereof. An optical system 11 is provided.
FIG. 2 shows a configuration diagram of the photographing optical system 11. The imaging optical system 11 has a plurality of lenses having a function as a fisheye lens, and captures light in a visual field region around the entire circumference around the optical axis P. Of course, it is not necessary to be strictly around the entire circumference, and an optical system that does not shoot in the optical axis direction and shoots only a panoramic belt-like area may be used. If the optical axis direction is included, it can be said that it is omnidirectional. However, these are collectively referred to as omnidirectional below. Of course, depending on how the fisheye lens and wide-angle lens are used, omnidirectional photography is possible. An imaging unit 12 is provided on the optical axis P of the imaging optical system 11. The image capturing unit 12 converts light captured by the image capturing optical system 11 into an electrical digital signal to generate image data. The imaging unit 12 is provided on the substrate 13.

  The imaging optical system 11 may be fixed to the apparatus main body 10a or may be rotatably provided. In the case of the fixed type, the user adjusts the angle of the camera. However, when the camera can be rotated, it is possible to shoot in the zenith direction by gravity or the force of the actuator without being strictly held. The photographing optical system 11 includes, for example, a spherical rotating image pickup body 13 and is provided so as to rotate with respect to the apparatus main body 10a. An imaging optical system 11, an imaging unit 12 provided on a substrate 14, a weight 14, and the like are provided inside the rotary imaging body 13. The imaging optical system 11, the imaging unit 12, the substrate 14, the weight 15, and the like rotate integrally with the rotation of the rotating imaging body 13. Since the weight 15 is provided on the extended line of the optical axis P of the imaging optical system 11 below the rotary image pickup body 13, even if it rotates within the range of the rotatable solid angle θ, gravity by the weight 15 is applied. Accordingly, the rotary imaging body 13 including the photographing optical system 11 is rotated and moved toward the zenith regardless of the inclination of the apparatus main body 10a. Further, a rotation restricting pin 16 is provided on the extended line of the optical axis P of the photographing optical system 11 below the rotating imaging body 13. The rotation restricting pin 16 restricts the rotating image pickup body 13 within the range of the rotatable solid angle θ. When omnidirectional shooting is performed while looking at the zenith direction, there is a high possibility that the eyes and face of the photographer are also shot.

  In addition, the apparatus 10 includes a control unit 20 composed of a computer. The control unit 20 controls the entire apparatus 10, and includes an imaging unit 12, a microphone 21, an operation determination unit 22, a rear camera (eye sensor) 23, a recording unit 24, and a communication unit 25. The attitude determination unit 26, the front display unit 27, the front side touch panel 28, the back display unit 29, and the back side touch panel 30 are connected. The control unit 20 includes an internal memory.

The microphone 21 collects sounds of a photographer who performs a photographing operation of the apparatus 10 and a main subject such as a person, and outputs these sound detection signals. The microphones 21 may be selectable depending on the shooting style (how to hold the camera). One or more microphones 21 are selected to collect sound, for example, in stereo.
The operation determination unit 22 includes various operation members that are manually operated by the photographer, for example, an operation member for performing focus adjustment with respect to the main subject, operation of a diaphragm for adjusting the brightness of the subject, a release button, a movie button, and the like. , A mode dial, a selection key, a power button, and the like, and determines an operation on these operation members and sends an operation determination signal to the control unit 20.

Although not used much in the present apparatus 10, it is possible to use an eyepiece type electronic viewfinder (EVF) in combination so that the subject can be observed without being influenced by outside light. For example, when a rear camera (eye sensor) 23 is provided on the rear surface of the apparatus 10 and an eyepiece type electronic viewfinder (EVF) is provided, the photographer's eyes are within a predetermined distance with respect to the EVF. It detects whether or not it is approaching and enables EVF.
In the recording unit 24, for example, a panoramic image acquired by photographing a subject in a photographing range over 360 °, an image obtained by cutting out a specific area, the photographer A detected by the face detecting unit 41, and the like. Image data including a face is recorded. The recording unit 24 is, for example, a memory card.
The communication unit 25 performs data communication including, for example, a photographed image and an operation signal between the apparatus 10 and the external apparatus.
The posture determination unit 26 detects the posture of the apparatus 10 or a change in posture. This posture detection performs camera shooting mode switching and camera shake correction.

The front display unit 27 is provided in front of the apparatus 10 and displays a live view image, a panoramic image acquired by photographing, and the like. The front display unit 27 is, for example, a liquid crystal display or an organic EL display.
The front-side touch panel 28 is provided on the front surface of the front display unit 27, detects a press by a photographer's touch operation, and outputs a touch coordinate signal indicating the touch operation position. For example, when the photographer performs a touch operation while the live view image is displayed on the front display unit 27, the front-side touch panel 28 may function as if the release switch was performed. .
When the photographic optical system looks at the zenith direction and performs omnidirectional shooting while looking at such a front-side display unit 27, it is highly likely that the photographer's eyes and face are also shot. In the following embodiment, the feature is obtained by photographing when the front display unit 27 is used. Focus and exposure may be adjusted to the touch position. The front side is an expression when the optical axis direction is considered to be the front side, and in the mode where the optical axis is actually directed toward the zenith direction, the display unit is on the top surface, so the user looks at this and composes etc. Can be confirmed. When looking at this, it becomes a posture to look into from a certain direction of the optical system, and there is a high probability that the photographer's face (eye) is also captured by the photographing optical system, and it is determined from the viewing direction which is the photographer It is also possible to do.
That is, here, it is preferable that the display panel is disposed on the same plane as the photographing lens is disposed, so that a photographer who views an image obtained by the imaging unit on the display unit can be viewed on the same imaging unit. Can be judged. A subject in the direction that the photographer is facing can be photographed at the same time, and can be applied without difficulty to the purpose of taking a picture of what is being seen, as with a general camera. Of course, it goes without saying that it is convenient and fun to shoot with the photographer at once.
In this case, it is possible to reduce the size and thickness by arranging lenses and displays on a wide surface. It is also advantageous for taking pictures with the optical axis directed at you, and taking selfies. Of course, even if the imaging result is not displayed in the same direction as when the photographing optical system is provided, the same effect can be obtained when the display unit is provided on the surface facing the photographer. A movable display unit may be used. In these cases, however, the equipment becomes larger and thicker.
Although not used in the present apparatus 10, the rear display unit 29 is provided on the rear surface of the present apparatus 10 and displays a live view image, a panoramic image acquired by photographing, and the like. The back display unit 29 is, for example, a liquid crystal display or an organic EL display.
The rear-side touch panel 30 is provided on the front surface of the rear display unit 29, detects a press by a photographer's touch operation, and outputs a touch coordinate signal indicating the touch operation position. Similarly to the front side touch panel 28, when the photographer touches the rear side touch panel 30 while displaying a live view image on the rear display unit 29, for example, a release switch is performed for the touch operation. You may make it function as a thing. These display units are selectively enabled depending on the shooting mode, posture, and the like.

  The control unit 20 includes an internal memory and the like, and the shooting program of the apparatus 10 is stored in advance in the internal memory. The photographing result may be recorded in this internal memory or the like. The control unit 20 realizes the following functions by executing the photographing program. In other words, the photographing program causes the control unit 20 including a computer to photograph within a range including the subject and at least the eye of the photographer, an image processing function for acquiring image data of the photographing function, and an image processing function. The eye direction determination function for determining the eye direction of the photographer in the image data acquired by the method, and the main function for determining the main subject in the subject to be photographed by the photographer based on the eye direction determined by the eye direction determination function. A subject determination function and a highlight display function for highlighting the main subject determined by the main subject determination function are realized.

Here, the highlight display of the main subject will be described. 3A to 3C are diagrams for explaining acquisition of a panoramic image by photographing of the apparatus 10, FIG. 3A shows a state of omnidirectional photographing, FIG. 3B shows a photographing result, and FIG. This shows the panoramic image processed from the shooting result. When a necessary part is cut out from all directions and displayed wide, it becomes a panoramic image. As the definition of wide or panorama, there are those whose horizontal length is longer than the aspect ratio of the display unit of the camera, and whose horizontal length is longer than the vertical length by, for example, twice or more.
As shown in FIG. 3A, a photographer A is a photographer A who is the photographer himself, a person such as a father (hereinafter referred to as a father) B as a main subject, and a person such as a child (hereinafter referred to as a child). A state where three persons with C are photographed by omnidirectional photographing is shown. It is assumed that the sight line direction (sight line direction) S of the photographer A is directed to the father B.

When omnidirectional shooting is performed in the state shown in FIG. 3A, a shooting result as shown in FIG. 3B is obtained, and a circular shooting result corresponding to the image circle of the imaging optical system 11 is obtained. In FIG. 3B, based on the eye line direction S of the photographer A, the main subject to be photographed by the photographer A, here, the father B including the child C, is determined, and the father B including the child C is highlighted. Is attached with a circular index F.
As shown in FIG. 3C, an annular index F is added to highlight father B on the panoramic image.

The control unit 20 includes an image processing unit 40, a face detection unit 41, a line-of-sight direction determination unit 42, a main subject determination unit 43, an emphasis display unit 44, and display control in order to realize the highlight display of the main subject. Unit 45 and a voice detection unit 46. The sound detection unit 46 receives a sound detection signal output from the microphone 21 and detects the sound of the photographer and the main subject such as a person.
The face detection unit 41 can compare the shadow pattern of the image with the image dictionary of the shadow pattern recorded in the recording unit 24 to determine the face part in the image, or to determine which person's face part. . Further, the face detection unit 41 can determine the part of the eye based on the shadow pattern of which part of the face image.
Note that the eye direction determination unit 42 determines the position, size, arrangement, and the like of white eyes and black eyes, and determines the eye direction.

The image processing unit 40 receives omnidirectional image data output from the imaging unit 12 and performs image processing such as white balance correction and gamma correction on the image data.
The face detection unit 41 detects the face of a person from omnidirectional image data.
When the photographer is included in the omnidirectional image data acquired by the image processing unit 40, the line-of-sight direction determination unit 42 determines the line-of-sight direction S of the photographer A.
Specifically, the eye direction determination unit 42 receives a person in the omnidirectional image data detected by the face detection unit 41, here, the face of the photographer A as a person, The line-of-sight direction S is determined based on at least one of or both of the presence or absence of the nostril part and the pupil position of the eye part. As for the determination of the photographer, it is possible to determine from the direction and area of the face image to be photographed as described above, and the face detection result and the camera owner may be determined. If the face can be detected, the position of the eye can be determined from the shadow pattern of the face.
In the determination of the line-of-sight direction S, the line-of-sight direction determination unit 42 determines whether or not the line-of-sight direction S of the photographer A is on the apparatus 10 side, and if the line-of-sight direction S is on the apparatus 10 side, the book The line-of-sight direction S on the apparatus 10 side is set as the origin direction (reference line-of-sight direction) G as shown in FIG. After setting the reference eye direction G, the eye direction determination unit 42 compares the eye direction G of the photographer A with the eye direction S of the photographer A to compare the eye direction S of the photographer A. judge.

  That is, when the photographer A intends to perform panoramic photography centering on the photographer A himself and the father B including the child C, the photographer A first looks at the father B including the child C as shown in FIG. The father B including the child C and the photographer A themselves are placed within the photographing range of the apparatus 10. Thereafter, the photographer A looks at the live view image displayed on the front display unit 27 as shown in FIG. 4 and the father B including the child C within the photographing range of the apparatus 10 and the photographer A himself / herself. Confirm that is included. Thereafter, the photographer A pushes the release button of the apparatus 10 or touches the touch panel 28 on the front display unit 27, so that the apparatus 10 performs a photographing operation, which is shown in FIG. 3B. Such a photographing result is obtained, and a panoramic image as shown in FIG. 3C is acquired.

Accordingly, the eye direction determination unit 42 sets the eye direction S as the reference eye direction G when the photographer A directs the eye direction S from the father B including the child C toward the apparatus 10 as shown in FIG. Set. This line-of-sight direction is repeated, but the line of sight itself cannot be determined, and the line of sight is estimated by the arrangement and relative change of white and black eyes.
After setting the reference eye direction G, the eye direction determination unit 42 compares the reference eye direction G with the eye direction S of the photographer A to determine the eye direction S of the photographer A.
In determining the line-of-sight direction S of the photographer A, the line-of-sight direction determination unit 42 is based on the comparison result between the reference line-of-sight direction G and the line-of-sight direction S of the photographer A (in the eye portion). It is determined whether the change in the position of the black eye or the difference between the distribution of the white eye and the black eye) is on the left side, the right side, or the front side with respect to the reference eye direction G. Of course, the determination may be made by looking at the change in the position of the black eye (or the position of the white eye) on the basis of the pattern of the nose muscles, the hole of the nose and the contour pattern of the face. Alternatively, a reference position for white eyes and black eyes may be determined, and it may be determined which one is being viewed from the difference between the reference positions.

  In addition, the eye direction determination unit 42 determines the main subject such as a person in the eye direction S of the photographer A determined as described above, and the sound of the photographer A and the main subject such as a person detected by the sound detection unit 46. If it is determined that there is a conversation between the photographer A and the main subject such as a person based on the photographer A and each image of the main subject such as a person, the pupil position of the photographer A and the determined left side The right eye direction or the forward eye direction S is associated, and the data of this association is stored in an internal memory or the like. The association data may be recorded in the recording unit 24 such as a memory card.

Here, an example of setting of the reference eye direction G of the photographer A's eye direction S and determination of the eye direction S of the photographer A will be described.
As described above, the eye direction determination unit 42 is based on one or both of the presence or absence of a nostril portion in the face of the photographer A and the pupil position of the eye in the image data acquired by the image processing unit 40. A specific example of determining the eye direction S will be described.

  FIG. 5A shows that the line-of-sight direction S of the photographer A is directed to the front display unit 27 of the apparatus 10. This line-of-sight direction S is substantially equal to that directed to the photographing optical system 11 of the apparatus 10. At this time, when the photographer A's line-of-sight direction S is directed to the front display unit 27 of the present apparatus 10, the reference line-of-sight direction G is used. is there. For this reason, in the image photographed by the imaging unit 12 through the photographing optical system 11 of the apparatus 10, the nostril portion of the photographer A's face is hardly photographed.

  On the other hand, FIG. 5B shows that the line-of-sight direction S of the photographer A is directed to the father B including the child C as shown in FIG. 3A. The face of the photographer A at this time is entirely upward. As a result, the nostril portion of the photographer A is photographed in the image photographed by the imaging unit 12 through the photographing optical system 11 of the apparatus 10.

Therefore, if the nose portion of the photographer A is captured in the image data acquired by the image processing unit 40 as shown in FIG. It is determined that it is directed to the front display unit 27 of the apparatus 10, and the line-of-sight direction S at this time is set as the reference line-of-sight direction G.
As shown in FIG. 5B, the eye direction determination unit 42 determines that the eye direction S of the photographer A is a child if the nostril portion of the photographer A is not photographed in the image data acquired by the image processing unit 40. It is determined that it is directed toward the father B including C, and the reference eye direction G is not set.

Next, a specific example in which the eye direction S is determined by the eye direction determination unit 42 based on the pupil position of the eye in the face of the photographer A in the image data will be described.
FIG. 6A shows that the line-of-sight direction S of the photographer A is directed to the father B including the child C. At this time, in the image photographed by the imaging unit 12 through the photographing optical system 11 of the present apparatus 10, the eye of the photographer A is photographed with the pupil and white eyes around the pupil.

FIG. 6B shows that the line-of-sight direction S of the photographer A is directed higher than the father B including the child C. At this time, the image photographed by the imaging unit 12 is that the eye of the photographer A is photographed in two regions of the pupil and white eye, for example, two regions of the pupil position above and the white eye position below. Become.
FIG. 6C determines that the line-of-sight direction S of the photographer A is directed to the front display unit 27 of the apparatus 10, and sets the line-of-sight direction S to the reference line-of-sight direction G. At this time, the image photographed by the imaging unit 12 is that the eye of the photographer A is photographed in two regions of the pupil and the white eye, for example, two regions of the pupil position below and the white eye position above. Become.

  Therefore, as shown in FIG. 6C, the eye direction determination unit 42 shoots the photographer if the image data acquired by the image processing unit 40 is captured in, for example, two regions where the pupil position is lower and the white eye position is upper. It is determined that the line-of-sight direction S of A is directed to the front display unit 27 of the apparatus 10, and the line-of-sight direction S at this time is set as the reference line-of-sight direction G. In the state shown in FIGS. 6A and 6B, the reference eye direction G is not set.

Next, a specific example when the eye direction determination unit 42 determines whether the eye direction of the photographer A is on the left side, the right side, or the front side with respect to the reference eye direction G will be described.
FIGS. 7A to 7C show the case where the face of the photographer A faces forward and only the pupil of the eye is directed forward, leftward (left side), and rightward (right side). FIG. It shows that the direction S is ahead. Note that the photographer A is photographed in a state of facing the front with respect to the photographing optical system 11 of the apparatus 10, and the nose muscle H of the nose portion of the photographer A exists between both eyes. The eye direction determination unit 42 has a pupil located at the center of the eye part on the image data acquired by the image processing unit 40, and the white eye part is photographed substantially equally on both the left and right sides of the pupil. It is determined that the line-of-sight direction S of the photographer A is ahead.

FIG. 7B shows that the line-of-sight direction S of the photographer A is on the left side from the self. Since the pupil is positioned on the right side of the eye part and the white eye part is photographed on the left side of the pupil on the image data acquired by the image processing unit 40, the eye direction determination unit 42 has the eye direction of the photographer A. It is determined that S is on the left side from itself.
FIG. 7C shows that the line-of-sight direction S of the photographer A is on the right side from the self. Since the pupil is positioned on the left side of the eye part on the image data acquired by the image processing unit 40 and the white eye part is captured on the right side of the pupil, the eye direction determination unit 42 has the eye direction of the photographer A. It is determined that S is on the right side from itself.

8A and 8B show a case where the face of the photographer A faces in the lateral direction and the pupil of the eye part faces in the left direction (left side) and right direction (right side). Among these, FIG. 8A shows that the line-of-sight direction S of the photographer A is on the left side from the self. Since the pupil is positioned on the right side of the eye part and the white eye part is photographed on the left side of the pupil on the image data acquired by the image processing unit 40, the eye direction determination unit 42 has the eye direction of the photographer A. It is determined that S is on the left side from itself.
FIG. 8B shows that the line-of-sight direction S of the photographer A is on the right side of the photographer A. Since the pupil is positioned on the left side of the eye part on the image data acquired by the image processing unit 40 and the white eye part is captured on the right side of the pupil, the eye direction determination unit 42 has the eye direction of the photographer A. It is determined that S is on the right side from itself.

The main subject determination unit 43 exists on the main subject in the subject to be photographed by the photographer A based on the eye direction S determined by the eye direction determination unit 42, that is, on the extension line of the eye direction S, for example, FIG. A father B including a child C as shown in FIG. 3C is determined.
This main subject determination means 43 has the nostril portion of the photographer A, the pupil position is downward, and there is no deviation in the nose and pupil positions of the photographer A, or there is no nostril portion of the photographer. If there is no vertical displacement in the position and there is no deviation in the nose and pupil positions of the photographer A, it is determined that the line-of-sight direction S of the photographer A is on the apparatus 10 side.

The highlight display unit 44 highlights the main subject determined by the main subject determination unit 43. Specifically, the highlighting unit 44 is for emphasizing an image region corresponding to a main subject in the image data acquired by the image processing unit 40, for example, a father B including a child C as shown in FIGS. 3B and 3C. Image processing, for example, an index for emphasis is added to a main subject such as a father B including a child C, an arrangement position of the main subject is changed, or a gray level of a region corresponding to the main subject is relatively changed At least one of the following.
The highlighting index F is, for example, an index F formed in an annular shape for highlighting the father B including the child C on the panoramic image as shown in FIGS. 3B and 3C. The index F is not limited to an annular shape, and may be another shape such as an elliptical shape or a rectangular shape. Further, as the index F, the arrangement position on the image data of the main subject such as the father B including the child C may be changed, and further, the gray level of the area of the main subject such as the father B including the child C may be changed. For example, the brightness of the same area is increased, or image processing is performed so that a main subject such as father B including child C is irradiated with a spotlight, and further, contrast is lowered, exposure is changed, and the like. May be.

  The highlighting section 44 has a highlight range for highlighting the main subject in the image data. FIG. 9 sets the vertical width of the panoramic image acquired by the apparatus 10 to the angle φ. For example, when the apparatus 10 is photographed by placing the apparatus 10 at the height position of the chest of the father B as the main subject, the horizontal angle linking the apparatus 10 and the height position of the chest of the father B is set. An angle range (−10 ° to 50 ° = 60 °) with a lower angle of −10 ° and an upper angle of 50 ° with respect to the line. It should be noted that if the range emphasized by the index F is performed within the angle range (−10 ° to 50 ° = 60 °), the emphasized region becomes difficult to understand. For example, the range emphasized to 30 °, which is half of the angle range 60 °. Set.

For example, when the index F is an ellipse, when a group of persons D composed of a plurality of persons are photographed as shown in FIG. 10, the upper side is 25 ° and the lower side is 10 ° with respect to the center of the group of persons D. Set as the emphasis range.
Here, the basis for setting the emphasis range by the index F will be described with reference to FIG.
Assume that the photographer A photographs the main subject such as the person E with the apparatus 10 at a height of 1.2 m from the ground. The extension of the person E is, for example, 1.7 m, and shooting is performed with the distance between the apparatus 10 and the person E being, for example, 1 m. Further, tracing in the horizontal direction J from the height position of the photographing optical system 11 of the apparatus 10 corresponds to the height position of the chest of the person E.
Under such conditions, in order to photograph the person E by the apparatus 10 and the face part from the chest of the person E enters the photographing range, an upper angle θ (= 25 °) with respect to the horizontal direction J is set. I need it. Note that θ = arctan (0.5) ≈25 °. However, if only the chest from the face of the person E is photographed, there is no background of the person E and there is a place that is not suitable for viewing the image, so the angle θ is doubled to make the upper angle 50 °.

Further, when the highlight display unit 44 determines that there is no sound detected by the sound detection unit 46 and the main subject is a group of persons consisting of a plurality of persons by the eye direction determination unit 42, the highlight display unit 44 displays a highlight display on the group of people. Do.
The display control unit 45 displays the image data acquired by the image processing unit 40, such as a live view image or a panoramic image, on the front display unit 27 or the rear display unit 29.

Next, the operation of the apparatus configured as described above will be described according to the camera control flowchart shown in FIG.
In step S101, the control unit 20 determines whether or not the shooting mode is set. If the result of this determination is that shooting mode has been set, the control unit 20 determines in step S102 whether or not the rotary imaging body 13 including the shooting optical system 11 is facing the zenith. If the result of this determination is that it is facing the zenith, the control unit 20 displays a live view image of the subject on the front display unit 27 and the rear display unit 29 in step S103. In other words, the imaging optical system 11 captures light in the visual field region over the entire circumference around the optical axis P by a plurality of lenses having a function as a fisheye lens. The imaging unit 12 converts light captured by the imaging optical system 11 into an electrical analog signal, converts the analog signal into a digital signal, and generates image data. The image processing unit 40 receives omnidirectional image data output from the imaging unit 12 and performs image processing such as white balance correction and gamma correction on the image data. The display control unit 45 displays the image data acquired by the image processing unit 40, here the live view image, on the front display unit 27 or the rear display unit 29.

  Note that, as a result of the determination in step S101, if the mode is not the shooting mode but another mode, for example, the playback mode, the control unit 20 moves to the playback mode. If the result of determination in step S <b> 102 is that the rotating imaging body 13 including the imaging optical system 11 is not facing the zenith, the control unit 20 proceeds to the normal imaging mode in step S <b> 104.

  In step S105, the control unit 20 determines whether or not it is the front display unit 27 that displays the live view image, that is, the front side front display unit 27 provided with the photographing optical system 11 in the present apparatus 10. Confirm that the live view image is being displayed. If the display of the live view image is the rear display unit 29, the control unit 20 switches the display of the live view image to the front display unit 27.

  Thereafter, when the photographer is included in the omnidirectional image data acquired by the image processing unit 40 in step S111, the eye direction determination unit 42 of the control unit 20 determines the eye direction S of the photographer A. It is determined, and it is determined whether or not the eye direction S is the apparatus 10 side (camera side). That is, for example, when the photographer A performs panoramic photographing centering on the photographer A himself / herself and the father B including the child C as shown in FIG. 3A, the live view image in the apparatus 10 is shown in FIG. Is directed to the front display 27 side on which is displayed or the father B side including the child C.

In such a situation, in determining whether or not the eye direction S of the photographer A is on the apparatus 10 side, the eye direction determination unit 42 obtains the image data acquired by the image processing unit 40 as shown in FIG. 5A. If the nostril portion of the photographer A is photographed, it is determined that the line-of-sight direction S of the photographer A is directed to the front display unit 27 (the device 10 side) of the device 10.
As a result, the line-of-sight direction determination unit 42 determines the line-of-sight direction S of the photographer A that is directed to the front display unit 27 (the apparatus 10 side) of the apparatus 10 as shown in FIG. Set as G.
The line-of-sight direction determination unit 42 records image data including the face of the photographer A when the reference line-of-sight direction G is set as an initial pattern of the face of the photographer A, for example, in the recording unit 24 or the like. The image data of the initial pattern of the photographer A's face is captured in the photographer A when the face detection unit 41 detects the photographer A's face from the image data acquired by the image processing unit 40. It is acquired that the face part is included.
The face detection unit 41 can compare the shadow pattern of the image with the image dictionary of the shadow pattern recorded in the recording unit 24 to determine the face part in the image, or to determine which person's face part. .

In step S113, the line-of-sight direction determination unit 42 determines whether the line-of-sight direction S of the photographer A is the left side, the right side, or the front with respect to the reference line-of-sight direction G, that is, the line-of-sight direction S of the photographer A. It is determined whether or not the orientation has already been determined.
Here, since it is not yet determined whether the sight line direction S of the photographer A is the left side, the right side, or the front side with respect to the reference sight line direction G, the control unit 20 performs step S115 through step S115. Returning to S102, in Step S111 again, it is determined whether or not the eye direction S of the photographer A is the apparatus 10 side (camera side).

At this time, since it has been determined that the eye direction S of the photographer A is on the apparatus 10 side, the eye direction determination unit 42 records the initial pattern of the face of the photographer A in step S121. It is determined whether or not.
As a result of this determination, since the initial pattern of the photographer A's face has already been recorded in step S112, the eye direction determination unit 42 determines the initial pattern of the photographer A's face and the current pattern in step S122. The face part of the photographer A in the image data acquired by the image processing unit 40 (hereinafter referred to as the current photographer A's face part) is compared.

  As a result of this comparison, in step S123, the line-of-sight direction determination unit 42 causes the current photographer A's face to face upward from the initial pattern of the photographer A's face, that is, as illustrated in FIG. It is determined whether or not the direction S is directed higher than the father B including the child C. If the result of this determination is that the current photographer's A face is not upward, the eye direction determination unit 42 returns to step S102 through step S115, assuming that there is no upward determination in step S124.

  If the current photographer A's face is upward, the line-of-sight direction determination unit 42 determines that the current photographer A's face is rightward with respect to the initial pattern of the photographer A's face in step S131. As shown in 7C, it is determined whether or not the sight line direction S of the photographer A is on the right side of the photographer A. As a result of the determination, if the current photographer A's face is facing right, the eye direction determination unit 42 determines in step S132 that the current photographer A's face is facing right.

  On the other hand, if the current photographer A's face is not facing right, the eye direction determination unit 42 determines in step S133 that the current photographer A's face is facing left relative to the initial pattern of the photographer A's face. That is, as shown in FIG. 7B, it is determined whether or not the sight line direction S of the photographer A is on the left side of the photographer A. As a result of this determination, if the current photographer A's face is facing left, the eye direction determination unit 42 determines that the current photographer A's face is facing left in step S134.

If the current photographer A's face is not facing left, since it is already determined that the current photographer A's face is not facing right (step S131), the line-of-sight direction determining unit 42 determines in step S135. It is determined that the current photographer A's face is facing forward.
With the above processing, the eye direction determination unit 42 determines the direction of the eye direction S of the photographer A as to whether the eye direction S of the photographer A is on the left side, the right side, or the front with respect to the reference eye direction G. Exit.

The control unit 20 returns to step S102 through step S115, and in step S113 again, the direction determination of whether the eye direction S of the photographer A is on the left side, the right side, or the front side with respect to the reference eye direction G is performed. After the determination as to whether or not the process has been completed, since the determination has already been completed, the process proceeds to step S114.
In this step S114, the main subject determining unit 43, as shown in FIGS. 3B and 3C, for example, main subjects in the subject to be photographed by the photographer A based on the eye direction S determined by the eye direction determining unit 42. A father B including a child C is determined.
The emphasis display unit 44 performs image processing for emphasis on an image region corresponding to a main subject in the image data acquired by the image processing unit 40, for example, a father B including a child C as shown in FIGS. 3B and 3C. For example, at least of adding an index for emphasis to a main subject such as father B including child C, changing the arrangement position of the main subject, or relatively changing the gray level of an area corresponding to the main subject Do one. For example, as shown in FIGS. 3B and 3C, the highlighting index F is formed in a ring shape for highlighting the father B including the child C on the panoramic image. The image data to which the emphasis index F is added is displayed on the front display unit 27 as a live view image.

  In step S115, the control unit 20 determines whether to perform shooting. That is, when the release button of the operation determination unit 22 is pressed or the front touch panel 28 is touched, the control unit 20 records the panoramic image data at the time of the operation in the recording unit 24. That is, when omnidirectional shooting is performed as shown in FIG. 3A, a circular shooting result corresponding to the image circle of the imaging optical system 11 as shown in FIG. 3B is obtained. Based on S, a main subject to be photographed by the photographer A, here, a father B including the child C, is determined, and a circular index F is added to highlight the father B including the child C. The control unit 20 develops a circular imaging result corresponding to the image circle of the imaging optical system 11 to form a panoramic image as shown in FIG. 3C, and an annular index F for highlighting the father B on the panoramic image. Is added.

Next, specific determination of whether or not the eye direction S of the photographer A is on the apparatus 10 side by the eye direction determination unit 42 will be described with reference to the apparatus side eye direction determination flowchart shown in FIG. . The line-of-sight direction (especially the user) is originally represented by an arrow such as S in FIGS. 5A and 5B, but this arrow itself cannot be detected. Therefore, in this embodiment, the direction of the face is detected from the nose and the shadow of the eye, and the direction of the arrow is determined in consideration of the arrangement of white eyes and black eyes. For example, as shown in FIG. 7A, if the nose is at the center and the eye part is the target, it is determined that the line of sight corresponds to FIGS. 5A and 5B. At this time, if the pattern of the nostril cannot be determined, it can be determined that the line of sight is in FIG. 5A. For example, this may be recorded as a reference line of sight, and a line of sight in another direction may be estimated by comparing with this. If the nostril can be detected, it can be estimated that the line of sight is shown in FIG. 5B. When the line of sight moves left and right, it can be determined by whether the position of the black eye is biased with respect to the nose. All such analogies will be described below as the (user's) eye direction determination.
In step S201, the line-of-sight direction determination unit 42 is a face portion existing in an image area corresponding to the side where the operation determination unit 22 is provided in the image data acquired by the image processing unit 40, that is, the photographer A's. The face part is detected by the face detection part 41. If the face is not detected, the eye direction determination unit 42 ends the determination of whether or not the eye direction S of the photographer A is on the apparatus 10 side.

  In step S202, the line-of-sight direction determination unit 42 has the face portion of the photographer A facing upward as shown in FIG. 5B, and the line-of-sight direction S of the photographer A is directed to the father B including the child C. The nostril portion of the photographer A is photographed, and the pupil position of the photographer A is positioned downward, the line-of-sight direction S is on the lower side, and the line-of-sight direction S of the photographer A is directed to the front display unit 27 of the apparatus 10. It is determined whether or not.

As a result of this determination, if the face of the photographer A is upward and the eye direction S of the photographer A is downward, the eye direction determination unit 42 is acquired by the image processing unit 40 in step S203. From the image data, for example, as shown in FIG. 7A, it is determined whether or not the nose muscle H of the photographer A's nose exists between both eyes, and the pupil is located at the center of the eyes and there is no left-right displacement. .
As a result of this determination, if the nasal nose muscle H of the photographer A exists between both eyes, and if the pupil is located at the center of the eye and there is no left-right displacement, the eye direction determination unit 42 performs step S204. , It is determined that the line-of-sight direction S of the photographer A is on the apparatus 10 side.

If the result of the determination in step S202 is that the face of the photographer A is upward and the line-of-sight direction S of the photographer A is not on the lower side, the line-of-sight direction determination unit 42 in FIG. As shown, it is determined from the image data acquired by the image processing unit 40 whether or not the nostril portion of the photographer A's face is photographed and there is no upward / downward shift of the photographer A's pupil position. To do.
As a result of the determination, if the nostril portion of the photographer A's face is hardly photographed and there is no upward or downward displacement of the photographer A's pupil position, the eye direction determination unit 42 proceeds to step S203. .
As a result of the determination, if the nostril portion of the face of the photographer A is photographed and the pupil position of the photographer A is shifted upward or downward, the eye-gaze direction determination unit 42 captures in step S206. It is determined that the eye direction S of the person A is other than the apparatus 10 side.

Next, the direction determination (steps S132, 134, 135) of the eye direction S by the eye direction determination unit 42 will be described with reference to the direction determination flowchart of the eye direction S shown in FIG. Note that the direction of the eye line direction S of the photographer A is one of the left side, the right side, and the front side with respect to the reference eye direction G, and the direction of the eye direction S is represented as X.
In step S300, the line-of-sight direction determination unit 42 determines whether or not the direction of the line-of-sight direction S of the photographer A can be determined according to each pupil position shown in FIGS. 7A to 7C, 8A, and 8B, for example.

  As a result of the determination, if the determination is impossible, the eye direction determination unit 42 collects the image of the face of the photographer A from the image data acquired by the image processing unit 40 and the microphone 21 in step S301. The relationship between the photographer A and the sound of a main subject such as a father B including a child C is determined. That is, in step S302, the eye direction determination unit 42 has the face portion of the photographer A on the direction X of the eye direction S, that is, on the left side, the right side, or an extension line of the eye direction S ahead of the reference eye direction G. It is determined whether or not.

If the result of this determination is that the photographer A's face is on the left side, right side, or an extension of the front eye direction S with respect to the reference eye direction G, the eye direction determining unit 42 determines whether the left side in step S303 Then, it is determined whether or not there is a change in the mouth of the face of the photographer A on the right side or the extension line in the eye direction S on the front, for example, whether the mouth has a movement such as opening and closing. That is, it is determined whether or not the photographer A has a conversation with a main subject such as a father B including a child C.
If the result of this determination is that there is a change in the mouth of the photographer A, the eye direction determination unit 42 determines whether or not the sound emitted from the mouth of the photographer A has been collected in step S304. That is, since the microphone 21 collects sound in stereo, the control unit 20 can determine the direction of the collected sound from the sound signal output from the microphone 21. Accordingly, the line-of-sight direction determination unit 42 associates the direction of the collected sound with the position of the photographer A in the image data acquired by the image processing unit 40, so that the sound is received from the face of the photographer A. It is determined whether or not it is emitted.

As a result of this determination, if sound is emitted from the face of the photographer A, the eye direction determination unit 42 determines, for example, on the left side, right side, or front side based on the audio signal output from the microphone 21 in step S305. It is determined whether or not sound is emitted from the face of the photographer A.
As a result of this determination, if the sound is emitted from the face of the photographer A whose eye direction S is on the left side, the right side, or an extension of the front eye direction S, the eye direction determining unit 42 determines in step S306. The eye direction S obtained based on the audio signal output from the microphone 21 corresponds to the left, right, or front of the eye position of the eye of the photographer A, for example, FIGS. 7A to 7C, FIG. Each pupil position shown in FIG. 8B is associated and recorded in, for example, the recording unit 24 and is made into a database.

In step S310, the highlighting unit 44 displays a main subject such as a father B including a child C existing on an extension line of the eye direction S when the eye direction S of the photographer A is the left side, the right side, or the front. Emphasize processing.
Note that, as a result of the determination in step S305, if there is no sound from a main subject such as father B including a child C existing on the left side, the right side, or an extension line in the forward line-of-sight direction S when viewed from the photographer A, In step S307, the direction determination unit 42 determines whether or not there is one main subject such as a person on the left side, the right side, or an extension of the forward eye direction S as viewed from the photographer A.

If the result of this determination is that there is only one main subject such as a person, the highlighting section 44 determines that the person or the like present on the left side, the right side of the photographer A, or an extension line in the forward eye direction S in step S310. The face part of the main subject is emphasized.
If the main subject such as a person is not one person, the highlighting unit 44 determines that the main part such as a plurality of persons existing on the left line, the right side of the photographer A, or an extension line in the forward eye direction S in step S312. Emphasize each face of the subject group.

As a result of the determination in step S302, if there is no main subject such as a person existing on the direction X in the eye direction S, that is, on the left side, the right side, or an extension line in the front eye direction S with respect to the reference eye direction G. In step S311, the line-of-sight direction determination unit 42 emphasizes the main subject such as a person with the index F in the angle range of 30 °, for example, on the left, right, or forward line-of-sight direction S.
As a result of the determination in step S300, for example, if the direction of the eye direction S of the photographer A can be determined according to the respective pupil positions shown in FIGS. 7A to 7C, 8A, and 8B, the eye direction determination unit 42 In S321, the direction of the eye line direction S of the photographer A is determined according to the respective pupil positions shown in FIGS. 7A to 7C, 8A, and 8B.

As described above, according to the above-described embodiment, when the subject is photographed in the omnidirectional photographing range and the photographer A is included in the image data obtained from the omnidirectional photographing, the line-of-sight direction of the photographer A S is determined, and based on the determined eye direction S, a main subject such as a father B including a child C among subjects to be photographed by the photographer A is determined, and the determined main subject is highlighted. Therefore, it is possible to acquire a panoramic image that can emphasize the main subject such as the father B including the child C in the subject that the photographer A was looking at.
In the above embodiment, we also touched on image determination such as the outline of the face and nostrils, but of course, the method of presetting the reference position and the dictionary image, the eye of each eye Needless to say, the same effect can be achieved with only the eye image, such as recording an image pattern. Moreover, as long as it has an imaging optical system and an imaging optical system capable of simultaneously capturing an image of a photographer's eye and a direction viewed by the photographer, the image is not limited to the omnidirectional or the entire periphery.

  In the above-described emphasis, the index F formed in a ring shape is used, the shape is not limited to the ring shape, and other shapes such as an ellipse shape and a square shape are used. To change the arrangement position, and further change the intensity level of the main subject area such as father B including the child C to increase the brightness of the same area, for example, or to the main subject such as father B including the child C. The image may be emphasized by performing image processing as if it were irradiated with a spotlight, further reducing the contrast, changing the exposure, or the like.

  A region to be emphasized, for example, a range to be emphasized by an index F formed in a ring shape, is set to, for example, an angle range of 30 °, or 25 ° on the upper side and 10 ° on the lower side with respect to the center of a group of persons D composed of a plurality of persons. By setting “°” as the emphasis range, it is possible to make it suitable for the size of the main subject such as father B including child C for emphasis.

Therefore, it is clear which part of the panoramic image the photographer was watching, and which subject was photographed for the purpose. It is possible to obtain a panoramic image in which the subject, the subject that the photographer was looking at, and the subject that the photographer was impressed with can be understood. Therefore, when viewing a panoramic image at a later date, the subject viewed at the time of shooting can be easily understood, which is a great help in viewing the panoramic image.
By limiting the left, right, or front eye direction S of the photographer A, the face portion of the main subject such as a person existing on the extension line of the left direction, right side, or front eye direction S of the photographer A can be obtained. Emphasis processing is possible.

  In addition, the present invention is characterized by a user interface using a user's face, and is not necessarily a photographing apparatus. Needless to say, the present invention can be applied to a control box for operating an object such as an operation member of an industrial or medical device such as a remote controller of an apparatus or an operation part of a stationary apparatus. In other words, the camera can be applied to a device that wirelessly operates a highlighted part or the like or a device that performs intensive observation. In an operating device that can image an object and an operator, a technique such as the present invention is required to determine and operate the orientation of the face, and any device with such conditions can be applied. . It is also effective for a device such as an in-vehicle device that the user wants to quickly communicate with the device. Therefore, the present invention belongs to an information processing apparatus.

  Note that each processing method performed by the imaging apparatus according to the above-described embodiment, that is, the processing illustrated in each flowchart is stored as a program that can be executed by the control unit 2. This program is stored in a storage medium of an external storage device such as a memory card (ROM card, RAM card, etc.), magnetic disk (floppy (registered trademark) disk, hard disk, etc.), optical disk (CD-ROM, DVD, etc.), semiconductor memory, etc. The above-described processing can be executed by reading the stored program and controlling the operation by the read program.

  10: photographing apparatus, 11: photographing optical system, 12: imaging unit, 10a: main body of the apparatus, 13: rotating imaging body, 14: substrate, 15: weight, 16: rotation regulating pin, 20: control unit, 21: microphone 22: Operation determination unit, 23: Rear camera (eye sensor), 24: Recording unit, 25: Communication unit, 26: Posture determination unit, 27: Front display unit, 28: Front side touch panel, 29: Rear display unit, 30 : Rear side touch panel, 40: image processing unit, 41: face detection unit, 42: eye direction determination unit, 43: main subject determination unit, 44: highlight display unit, 45: display control unit, 46: audio detection unit.

Claims (11)

Photographing means for photographing within a range including the subject and at least the eye part of the photographer;
Image processing means for obtaining image data of the photographing means;
Eye direction determination means for determining the eye direction of the photographer in the image data acquired by the image processing means;
Main subject determination means for determining a main subject in the subject to be photographed by the photographer based on the eye direction determined by the eye direction determination means;
Highlighting means for highlighting the main subject determined by the main subject determining means;
An imaging apparatus comprising:   The eye direction determination unit determines the eye direction based on at least one of or both of a nostril portion and a pupil position of an eye of the photographer in the image data. The imaging apparatus according to 1.   The eye direction determination means determines whether the eye direction of the photographer is on the apparatus side, and if the eye direction is on the apparatus side, the eye direction on the apparatus side is used as a reference. The imaging apparatus according to claim 2, wherein the eye direction of the photographer is determined by comparing the reference eye direction with the eye direction of the photographer.   The main subject determination means includes the photographer's nostril portion, the pupil position is downward, and the photographer's nose muscle and the pupil position are not displaced, or the photographer's nostril portion. However, if there is no vertical displacement in the pupil position and there is no displacement in the photographer's nose and pupil position, it is determined that the eye direction of the photographer is the apparatus side. The imaging device according to claim 3.   The line-of-sight direction determination means is based on a comparison result between the reference line-of-sight direction and the photographer's line-of-sight direction, and the line-of-sight direction of the photographer is any of the left side, the right side, or the front with respect to the reference line-of-sight direction. The imaging device according to claim 3, wherein it is determined whether or not Voice detection means for detecting voice of the photographer and the main subject;
The line-of-sight direction determining means includes the photographer based on the main subject in the determined line-of-sight direction, the sound detected by the sound detection means, and each image of the photographer and the main subject. When determining that there is a conversation with the main subject, associating the pupil position of the photographer with the eye direction on either the left side, the right side, or the front;
The imaging device according to claim 5, wherein: When the eye direction determination means determines that the sound is not detected by the sound detection means and the main subject is a group of persons consisting of a plurality of persons,
The highlighting means highlights the group of people.
The imaging device according to claim 6.   The highlight display unit performs an image processing for emphasis on an image area corresponding to the main subject in the image data, and adds an emphasis index to the main subject. The photographing apparatus according to any one of claims 1 to 7, wherein at least one of the following is changed: or a relative gray level of an area corresponding to the main subject is changed.   The photographing apparatus according to claim 1, wherein the highlighting unit has a highlight range in which the main subject is highlighted in the image data. The subject is photographed by photographing means having a photographing range including the subject and at least the eye part of the photographer,
The image data of the shooting range is acquired by an image processing means,
The eye direction of the photographer is determined by eye direction determination means,
Highlighting a main subject in the subject that the photographer intends to shoot based on the determined line-of-sight direction;
An imaging method characterized by the above. On the computer,
A shooting function for shooting in a range including the subject and at least the eye of the photographer;
An image processing function for acquiring image data of the photographing function;
A line-of-sight direction determination function for determining a line-of-sight direction of the photographer in the image data acquired by the image processing function;
A main subject determination function for determining a main subject in the subject to be photographed by the photographer based on the eye direction determined by the eye direction determination function;
A highlighting function for highlighting the main subject determined by the main subject determining function;
Shooting program of the imaging device for realizing.

Images