JP6168630B2 - Image selection display method and apparatus - Google Patents

Description

  The present invention relates to an image selection display method and apparatus, and more particularly to an image that displays the same object and that selects an image from a plurality of images with different focus positions (focus positions).

  There exists patent document 1 as a prior art example of this kind. In Patent Document 1, the focal position is changed in multiple steps or steplessly according to one release operation, and image data is generated for the changed subject images at a plurality of different focal positions. The focusing range can be changed arbitrarily.

JP 2003-143461 A

  It is an object of the present invention to select and display an image according to a person's biological signal from a plurality of images that display the same target and are in different positions.

  In order to solve this problem, the present invention adopts the following configuration.

In the present invention, a value transmitted to the optic nerve in the act of human viewing is acquired as a biological signal, and based on this biological signal, a position coordinate on a camera photograph or CG image or an image focused on an object is retrieved and displayed. To do.
In the present invention, an object imaged in the brain is specified by a biological signal such as an electroencephalogram, and an image focused on the object is searched and displayed based on the biological signal.

  Here, the present invention targets not only camera photographs but also CG images. In the case of a camera photograph, the focused position coordinates or object on the image differ depending on the focus position of the optical system when the photograph is taken. The focus position is also called a focus position, a focus position, and a focus distance. On the other hand, in the case of a CG image, the focused position coordinate or object on the image is the position coordinate or object of a portion displayed sharply in an image including a portion displayed blurred and a portion displayed sharply. It shall be said. The focused position coordinate on the image is a coordinate representing a pixel position in the image plane, and includes a rectangular range specified by a plurality of coordinates.

  ADVANTAGE OF THE INVENTION According to this invention, it is an image which displays the same object, Comprising: An image can be selected and displayed according to a human biosignal from the several image from which the position which focused is different.

The block diagram by 1st Embodiment of this invention. The block diagram by 2nd Embodiment of this invention. FIG. 3 is a structural diagram of data stored in the image storage means of FIG. 2. The block diagram by 3rd Embodiment of this invention. The block diagram by 4th Embodiment of this invention. The block diagram by 5th Embodiment of this invention. The block diagram by 6th Embodiment of this invention.

[First Embodiment]
Hereinafter, a first embodiment of the present invention will be described. FIG. 1 is a configuration diagram of an image selection display system according to the first embodiment. In FIG. 1, the image storage unit 1 stores a plurality of images i. Each image i is an image displaying the same object, but the focused position is different for each image. For example, there are multiple images that show the same room, but one image is in focus on the table in front, and the other image is in focus on the sofa in the back. That's it. The “in-focus position” here means a position in focus in the image plane, and is a position specified by the coordinates of the pixel. This in-focus position is referred to as a “focused position”.

  However, the image data i is not limited to a photographic image taken through an optical system, but may be a CG (Computer Graphics) image produced by a computer. In this case, the focused position refers to the position of the portion where the target is displayed sharply with respect to the portion where the target is displayed blurred.

  Further, the image i is not limited to an independent still image, and may be a moving image frame. A plurality of images that display the same target and have different focused positions are considered to have a set of a plurality of images i for each frame.

  The image i described above is associated with the data of the focused position p1 for each image i. For example, the position coordinate of the pixel corresponding to the table is associated with the image i focused on the table as the focused position p1. Similarly, the position coordinate of the pixel corresponding to the sofa is associated with the image i focused on the sofa as the focused position p1. Here, the focused position may be represented by the coordinates of one point of the pixel, but is a coordinate range specified by the coordinates of a plurality of pixels. Here, the focused position p1 is a rectangular range specified by the pixel coordinates of two diagonal points. A range of a circle specified by the center point and the radius may be a focused position.

  The image storage means 1 stores the image data of the image i described above and the data of the focused position p1 in association with each image. Hereinafter, a set of a plurality of images i displaying the same object is referred to as an “image set”.

  Further, the image selection display system shown in FIG. 1 includes an image display means 3 for displaying an image i read from the image storage means 1, and an image plane of a person who views the image i displayed on the image display means 3. And a line-of-sight coordinate calculation means 2 for inputting the coordinates of the line of sight at. Further, the control means 4 for repeating the process of reading out from the image storage means 1 and displaying on the image display means 3 an image i in which the coordinates of the line of sight input from the line-of-sight coordinate calculation means 2 match the focused position p1. I have.

  The line-of-sight coordinate calculation means 2 inputs, for example, the coordinates of the line of sight (gaze point coordinates) on the image plane obtained based on the amount of human eyeball movement, brain waves, etc. The image plane here is an image plane of an image displayed on the image display means 3. Known techniques of this type include Japanese Patent No. 3201333, Japanese Patent Application Laid-Open No. 5-333395, Japanese Patent Application Laid-Open No. 9-34631, Japanese Patent Application Laid-Open No. 9-21557, and the like. The effective value of the line-of-sight position coordinate input from the line-of-sight position input means 2 is within the range of the ordinate and abscissa of the pixel on the image plane.

  The image display means 3 may be any device that can display an image based on image data. For example, a liquid crystal display monitor.

  The control means 4 is a processor that executes a control circuit or a control program. The control unit 4 can acquire the position coordinates of the line of sight input from the line-of-sight coordinate calculation unit 2, and can search the image storage unit 1 for data of the image i in which this position coordinate and the focused position p1 are matched. Data of the obtained image i is read from the image storage means 1 (line-of-sight coordinate matching S1). Next, the control unit 4 displays an image based on the read data of the image i on the image display unit 3 (compatible image display S2). At this time, a person who sees the image sees a sharp target image at a position where the user's line of sight is poured, and appears as a blurred target image around the target image. The control means 4 repeats the above processes S1 and S2. Then, when the viewer moves his / her line of sight, the sharply visible portion of the target image moves in accordance with the movement of the line of sight.

  As described above, according to the present embodiment, when a person viewing the image looks at the image while moving his / her line of sight, the focused image is displayed at a position where the line of sight is always poured.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. FIG. 2 is a configuration diagram of the second embodiment. In the image selection display system shown in FIG. 2, the image storage means 1 stores a plurality of the above-described image sets for displaying the same object for each aperture value (image brightness) of the optical system. Further, it has condition selection means 5 for selecting an aperture value by selection of a person who views the image (hereinafter referred to as “user” as appropriate). Then, the control unit 4 reads an image from the image set corresponding to the aperture value selected according to the input signal from the condition selection unit 5. In the present embodiment, it is assumed that the plurality of images i included in the image set are photographic images or moving image frame images taken by the camera through the optical system.

  Further, the image storage unit 1 stores a plurality of image sets obtained by photographing the same target (subject) for each focal length (target size) of the optical system. The condition selection means 5 also has a function of selecting the focal length according to the intention of the person viewing the image. Then, the control unit 4 reads an image from the image set corresponding to the focal length selected according to the input signal from the condition selection unit 5. Other configurations are the same as those of the first embodiment.

  More specifically, in this embodiment, the structure of data stored in the image storage means 1 is as shown in FIG. 3, for example. The image storage means 1 stores a focal position, an image i, a focused position p1, an aperture value, and a focal length in association with each other. In this example, one image set is configured for every three rows surrounded by a thick frame. The image set is a bundle of a set of images obtained by photographing the same subject while changing the focal position. A plurality of image sets are stored, and each image set is photographed with different settings of the aperture value and the focal length. “Focus position” indicates the focal position of the optical system at the time of photographing with an index of 1 to 3. For convenience of illustration, only three focal positions are shown. However, in practice, it is preferable to capture a large number of images by changing the focal position steplessly. Since “image i” is different image data for each photographing, i (1) to i (18) are indicated by different symbols. As described above, the data of “focused position p1” represents the position in focus in the image plane, and when the focal length of the optical system is different, the angle of view of the subject is different. Assuming that the focused position in the plane is also different, different symbols p1 (1) to p1 (9) are attached. On the other hand, if the focal lengths are the same even if the aperture values of the optical system are different, it is assumed that the focused positions will be substantially the same, and the focused positions of the images i (10) to i (18) are set to p1. (1) to p1 (9). “Aperture value” indicated a different value with an index of 1 or 2. It is desirable to take a set of images at more aperture values. “Focal distance” showed different values with an index of 1 to 3. It is desirable to take image sets at more focal lengths. Here, “focal length” may be replaced with “zoom magnification”. The one shown in FIG. 3 is merely an example. The number of focal positions, the number of aperture values, and the number of focal lengths are not limited to this example. For example, a large number of images may be taken while zooming steplessly.

  There are several possible configurations for the condition selection means 5, but one is that the user manually operates a physical input device and inputs selection signals for the aperture value and focal length. Further, a configuration in which an aperture value and a focal length selection signal are input by a user's biological signal is conceivable. For example, there is known an example in which driving of a vehicle can be controlled by an electroencephalogram. Similarly, the condition selection means 5 can be configured to input a selection signal of an aperture value and a focal length by the user's electroencephalogram.

  The control unit 4 holds the aperture value and the focal length value selected according to the input signal from the condition selection unit 5 in the storage unit as condition parameters. In the line-of-sight coordinate matching processing S1, an image in which the position coordinates of the line of sight match the focused position p1 is read from the image set satisfying the condition parameters and displayed on the image display unit 3. For example, if the aperture value 1 and the focal length 2 are designated as the condition parameters, the image i is read out from the second image set from the top in FIG.

  Even if it does in this way, besides having the same operation effect as a 1st embodiment, an image can be seen, selecting an aperture value and a focal length.

  Here, in the present embodiment, both the aperture value and the focal length can be selected, but only one of them may be selected.

[Third Embodiment]
Next, a third embodiment of the present invention will be described with reference to FIG. FIG. 4 is a configuration diagram of the third embodiment. The image selection display system shown in FIG. 4 includes a first image storage unit 6 that stores an image i separately from the image storage unit 1 in the above-described embodiment. In the present embodiment, the above-described image storage unit 1 is the second image storage unit 1. A focused position determination unit 7 is provided between the first image storage unit 6 and the second image storage unit 1. The focused position determination means 7 reads the image i from the first image storage means 6, determines the focused position p1 in the image plane of the read image i, and associates the determined position p1 with the image i. The image is stored in the second image storage unit 1. Other configurations are the same as those of the first embodiment.

  The image i stored in the first image storage unit 6 is the same as the image i stored in the image storage unit 1 in the first embodiment. The focused position determination unit 7 determines the focused position p1 in the image plane of each image i, and associates each image i with the focused position p1 and stores it in the second image storage unit 1.

There are several possible methods for the focused position determination means 7 to determine the position p1.
One is a method in which the image i is displayed on a monitor, and a person designates the focused position p1 of the image i by visual observation using a physical input device such as a pointing device.
Also, it is generally known that whether or not a part of an image is blurred can be determined by a color difference between the part and the surrounding area (whether or not the image is sharply different from the surrounding color). . At that time, Fourier transform is used, but the spectrum is analyzed by an FFT (Fast Fourier Transform) algorithm by computer processing. The focused position determination means 7 may be configured to automatically determine the focused position p1 in the image i using this FFT algorithm. The determination of blur by FFT may be performed for each area obtained by dividing an image into matrix areas. Alternatively, the outline extraction process of the object included in the image may be performed, and the blur may be determined for the object for which the outline has been extracted with respect to the coordinates in the image plane of the object.

Even if it does in this way, besides having the same effect as a 1st embodiment, the information which linked image i and focused position p1 can be generated.
Here, as in the second embodiment, when associating the aperture value and the focal length when captured with the camera with the image i, the aperture value is stored in advance when the data of the image i is stored in the first image storage unit 6. And the value of the focal length may be stored in association with each other.

  Further, in the present embodiment, the focused position determination means 7 is provided in the preceding stage of the second image storage means 1, and the association between the image i stored in the second image storage means 1 and the focused position p1 is made in advance. Although the configuration is such that the function of the determination unit 7 is provided in the control unit 4 and the control unit 4 performs the line-of-sight coordinate matching processing S1, the configuration in which the focused position p1 of each image i is determined is performed. Good.

[Fourth Embodiment]
Next, a fourth embodiment of the present invention will be described. FIG. 5 is a configuration diagram of the fourth embodiment. The image selection display system shown in FIG. 5 includes an imaging device C for capturing an image i stored in the first image storage means 5. Other configurations are the same as those of the third embodiment.

  The imaging apparatus C includes an imaging optical system, an imaging element that receives light that has passed through the optical system, a shutter mechanism that is positioned between the optical system and the imaging element, and a drive unit that drives the optical system and the shutter mechanism. And. In addition, a control unit that controls the operation of the drive unit, an operation unit that inputs an instruction to the control unit, and a display unit that displays various types of information are provided.

  When the shutter button is pressed by the operation of the operation unit, the control unit drives the drive unit to drive the shutter mechanism at the same time while continuously changing the focus position of the photographing optical system from one end to the other end. The subject is continuously shot (hereinafter referred to as “stepless focus shooting”). Then, a large number of images i obtained by continuous shooting are stored in the first image storage means 6. During this single shooting, the aperture value and focal length of the optical system are fixed, and the aperture value and focal length value at the time of shooting are stored in association with each image i. When one shooting is finished, the control unit repeatedly executes stepless focus shooting while changing the combination of the aperture value and the focal length of the optical system. It is preferable that the aperture value is sequentially selected for all aperture values that the photographing apparatus C allows selection. Further, it is preferable that the focal length is changed steplessly within a range allowed by the optical system.

  Even if it does in this way, besides having the same effect as a 3rd embodiment, from photography of an image to generation of an image set can be performed automatically. Here, the first image storage unit 6 may be housed in the housing of the imaging apparatus C. The image storage means 1 and 6 are not limited as long as they are memory media capable of storing data. In addition, the above description has been described on the assumption that the still image is continuously shot. However, instead of continuously shooting still images, a moving image may be shot, and each frame of the shot moving image may be set as each image i. .

[Fifth Embodiment]
Next, a fifth embodiment of the present invention will be described with reference to FIG. In the present embodiment, the image storage unit 1 stores the data of the image i and the data of the focused position p2 in association with each image. In the present embodiment, the image i is an image displaying a plurality of objects, and is focused on a certain object among them. That is, the image i is an image in which the focused object is displayed sharply and its surroundings and other objects are blurred. An object is an article, a creature, a character, or the like that appears in an image. The focused object is different for each image i. Therefore, the focused position p2 is different for each image i. However, each image i is an image that displays the same target as in the first embodiment.

  The focused position p2 represents the position of the focused object in the image plane. In the present embodiment, the focused position p2 is data representing the position of the focused object in the image plane in the pixel coordinate range.

  The control means 4 reads out the data of the image i from which the coordinates of the gazing point input from the line-of-sight coordinate calculation means 2 are included in the coordinate range of the focused position p2 from the image storage means 1. According to this, the image i focused on the object at the gazing point is read (line-of-sight object matching S11).

  Other configurations are the same as those of the first embodiment. According to the present embodiment, when an image is viewed while moving the line of sight, the image focused on the object that is always pouring the line of sight is displayed.

[Sixth Embodiment]
Next, a sixth embodiment of the present invention will be described with reference to FIG. In the present embodiment, the image storage means 1 stores image i data and electroencephalogram pattern p3 data in association with each other. Each image i is the same as that in the fifth embodiment described above, and is an image displaying a plurality of objects, and each image is focused on a different object. The electroencephalogram pattern p3 is data obtained by patterning an electroencephalogram generated when an image viewer images an object on the brain. The electroencephalogram pattern generated when an object is imaged differs depending on the object to be imaged. Therefore, the electroencephalogram pattern p3 generated when an image of the same object as the focused object in an image i is imaged and stored in association with the image i. Similarly, for other images i, the electroencephalogram pattern p3 corresponding to the focused object is stored in association with each other.

  An electroencephalogram pattern generation means 12 is connected to the control means 4. The electroencephalogram pattern generation means 12 acquires an electroencephalogram signal as a biological signal from a person who images the object on the brain, patterns the acquired electroencephalogram signal, and inputs the pattern to the control means 4. For patterning the electroencephalogram signal, the same algorithm as that for generating the electroencephalogram pattern p3 stored in the image storage means 1 is used. For example, it is conceivable to perform a fast Fourier transform on an electroencephalogram signal and convert a feature amount pattern into data in the frequency domain.

  When a viewer images an object included in the image displayed on the image display unit 3 on the brain, an electroencephalogram pattern corresponding to the image of the object is input from the electroencephalogram pattern generation unit 12 to the control unit 4. The control unit 4 searches the image storage unit 1 for an image i associated with an electroencephalogram pattern p3 that matches or is similar to the electroencephalogram pattern input from the electroencephalogram pattern generation unit 12 (electroencephalogram object matching S21). As a result, if the corresponding image i can be searched, the image i is read from the image storage means 1 and displayed on the image display means 3 (applicable image display S2). According to this, an image focused on the object imaged on the brain by the viewer is displayed on the image table means 3.

  When the control means 4 repeats the processes of S21 and S2, an image focused on the object imaged by the viewer on the brain is always displayed on the image display means 3.

  The present invention is not limited to the configuration of each embodiment described above. It is also possible to combine the configurations of the above-described embodiments.

DESCRIPTION OF SYMBOLS 1 Image memory | storage means, 2nd image memory | storage means 2 Eye-gaze coordinate calculating means 3 Image display means 4 Control means 5 Condition selection means 6 1st image storage means 7 Focused position determination means C Imaging device i Image p1, p2 Focused P3 focus object brain wave pattern

Claims (6)

A plurality of images that display the same target, differ in the focused position for each image, display the focused position relatively sharply, and display relatively blurred surroundings as an image set in advance. An image storage unit that stores and stores in advance the coordinates representing the focused position for each image;
The gaze point coordinates on the image plane of the person viewing the image are acquired, an image in which the gaze point coordinate and the coordinate of the focused position are matched is selected from the image set, read out from the image storage unit, and displayed. A control unit that repeatedly executes an operation to be displayed on the unit,
An image selection display device in which a focused position for each image is independent of a position of an object displayed on the image. The image selection display device according to claim 1, wherein the image set includes a plurality of images obtained by stepless focus photographing. The image storage unit stores a plurality of the image sets for each aperture value, and the control unit obtains an aperture value selected according to an input signal, and the image from the image set corresponding to the aperture value. the selected read out from the image storage unit, the image selection display device according to claim 1 or 2. The image storage unit stores a plurality of the image sets for each focal length, and the control unit obtains a focal length selected according to an input signal, and the image from the image set corresponding to the focal length. the selected read out from the image storage unit, the image selection display device according to any one of claims 1 to 3. A plurality of images that display the same target, differ in the focused position for each image, display the focused position relatively sharply, and display relatively blurred surroundings as an image set in advance. An image selection and display method using an image storage unit that stores and stores in advance the coordinates representing the focused position for each image.
The step of acquiring the gaze point coordinates on the image plane of the person viewing the image, and selecting an image in which the gaze point coordinate and the coordinate of the focused position are matched from the image set, and reading the image from the image storage unit Repeatedly executing the step of displaying on the display unit,
An image selection display method in which the focused position for each image is independent of the position of an object displayed on the image. A plurality of images that display the same target, differ in the focused position for each image, display the focused position relatively sharply, and display relatively blurred surroundings as an image set in advance. An image selection display program using an image storage unit that stores and stores in advance the coordinates representing the focused position for each image,
An image in which the process of acquiring the gazing point coordinates on the image plane of the person viewing the image and the gazing point coordinates and the coordinates of the focused position are selected is selected from the image set, and is read from the image storage unit. The processing to be displayed on the display unit is repeatedly executed by the processor,
An image selection display program in which the focused position for each image is independent of the position of an object displayed on the image.

Images