JP2015148880A - Electronic device and password management program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an electronic device and password management program that make it difficult to estimate a password, and enable high security to be ensured.SOLUTION: An electronic device comprises: a display unit that displays an image; an operation unit that can be operated by a user in responce to the image; a processing unit that causes the display unit to perform a prescribed operation in responce to at least an operation of the user; a display control unit that causes the display unit to display a password input-purpose image in which an in-focus position is changed to a plurality of positions and an object is provided over a plurality of in-focus positions; an extraction unit that extracts a plurality of selectable objects via the operation unit from the plurality of objects displayed in the display unit; and an input unit that when the object is selected via the operation unit, as to each of the objects, inputs information associated with an object to be selected and selection order of the object thereto as a password of a lock of a prescribed operation or a lock release thereof.

Description

  The present invention relates to an electronic device and a password management program.

  As an example of an electronic device such as a mobile phone or a portable information terminal, a display unit that displays an image such as a still image or a moving image, an operation unit that can be operated by a user, and an operation of the display unit according to a user operation. A configuration having a processing unit to be controlled is known. In such a configuration, a password is set as necessary to limit various operations of the electronic device.

  In this case, the user can input a password by setting a character string such as numerals or English symbols as a password and inputting the set character string via the operation unit. However, such passwords tend to select user-specific character strings such as names and dates of birth, and it has been pointed out that passwords may be stolen.

  On the other hand, as described in Patent Document 1, for example, password input areas divided into a plurality of unit areas are provided on the touch panel screen, and predetermined unit areas are touched one by one in a predetermined order. It is disclosed that the order of touch operations is input as a password.

JP 2011-164737 A

  However, the configuration of Patent Document 1 has a problem that a password is easily estimated when a fingerprint of a user remains on the touch panel screen.

  In view of the circumstances as described above, it is an object of the present invention to provide an electronic device and a password management program that are difficult to estimate a password and can ensure high security.

  According to the first aspect of the present invention, a display unit that displays an image, an operation unit that can be operated by a user corresponding to the image, and a process that causes the display unit to perform a predetermined operation in accordance with at least a user operation. A display control unit for displaying a password input image on which the focus position can be changed to a plurality of positions and an object is provided over the plurality of focus positions, and a plurality of objects displayed on the display unit An extraction unit that extracts a plurality of objects that can be selected through the operation unit, and when an object is selected through the operation unit, for each object, the object to be selected and the selection order of the object An electronic device is provided that includes an input unit that inputs the associated information as a password for locking or unlocking a predetermined operation.

  According to the second aspect of the present invention, a display unit that displays an image and an operation unit that a user can operate according to the image are provided, and at least the display unit performs a predetermined operation according to the user's operation. A process for causing the control device of the electronic device to change the in-focus position to a plurality of positions and displaying an image for password input in which an object is provided over the plurality of in-focus positions on the display unit, and a plurality of displayed on the display unit A plurality of objects that can be selected from the object via the operation unit, and when an object is selected via the operation unit, for each object, the selected object and the selection order of the object, A password management program is provided for executing a process of inputting information associated with the password as a password for locking or unlocking a predetermined operation.

  According to the aspects of the present invention, it is difficult to estimate a password, and an electronic device and a password management program that can ensure high security can be provided.

(A) And (b) is a figure which shows the structure of the information terminal which is an example of the electronic device which concerns on 1st Embodiment. It is a figure which shows an example of the image for password input. (A)-(e) is a figure which shows the example of the image for password input displayed on a display part. (A) is a table | surface which shows the relationship between an in-focus position and the object extracted by the extraction part, (b) is a table | surface which shows the correspondence of the attribute of an object, and the object which belongs to the attribute. It is a flowchart which shows the operation | movement which inputs a password. (A)-(m) is a figure which shows the display state of a display part in the operation | movement which inputs a password. It is sectional drawing which shows schematic structure of the optical system part in the light field camera which is an example of an imaging device. It is a figure explaining the pixel arrangement | sequence and incident area of an image pick-up element. It is a block diagram which shows the structure of a light field camera. It is a figure explaining the reconstruction of an image.

Embodiments of the present invention will be described below with reference to the drawings.
1A and 1B are diagrams illustrating a configuration of an information terminal 100 that is an example of an electronic device.
As shown in FIGS. 1A and 1B, the information terminal 100 includes a housing 101, a display unit 102, an operation unit 103, and a control device CONT. Examples of the information terminal 100 include a mobile phone and a mobile information terminal.

  The housing 101 is formed in, for example, a rectangular plate shape, and houses the display unit 102, the operation unit 103, and the control device CONT. The casing 101 is formed in a size that can be held by a user with one hand, for example.

  The display unit 102 is disposed on the plate surface side of the housing 101 and includes a display panel such as a liquid crystal panel or an organic EL panel. The display unit 102 can display images such as still images and moving images under the control of the control device CONT.

  The operation unit 103 includes a touch panel 103 a disposed on the display surface of the display unit 102. The touch panel 103a inputs an operation signal to the control device CONT when the user touches it. As the touch panel 103a, for example, a capacitance type contact sensor is used. Instead of the capacitance method, an electromagnetic induction method, an infrared method, a surface acoustic wave method, a resistance film method, a matrix switch method, etc. Also good. Further, as the touch panel 103a, for example, a so-called multi-touch panel system that can detect multiple contacts at the same time and can input multiple points may be used.

  By using the touch panel 103 a as the operation unit 103, the user can operate corresponding to the image displayed on the display unit 102. The operation unit 103 is not limited to the touch panel 103a, and for example, an eye sensor may be used. In this case, an operation signal is input to the control device CONT according to the position of the user's eyes.

  The control device CONT includes a processing unit 104, a display control unit 105, an extraction unit 106, an input unit 107, and a storage unit 108.

  The processing unit 104 causes the display unit 102 to perform a predetermined operation in accordance with at least a user operation. Examples of the predetermined operation include an operation of executing a program stored in the storage unit 108, an operation of displaying an image stored in the storage unit 108 on the display unit 102, and an operation of performing various calculations.

  For example, the processing unit 104 causes the user to input a password when the user locks or unlocks various operations of the information terminal 100. The processing unit 104 causes this operation to be performed based on the password management program stored in the storage unit 108. The password management program will be described later.

  The display control unit 105 controls display on the display unit 102. For example, according to the operation of the processing unit 104, the display control unit 105 selects an image stored in the storage unit 108 and causes the display unit 102 to display the image. For example, the display unit 105 causes the display unit 102 to display a password input image when the user inputs a password.

Here, the password input image will be described. FIG. 2 is a diagram showing the password input image P. As shown in FIG.
As shown in FIG. 2, the password input image P is an image used when the user inputs a password. The password input image P is an image generated based on information on a light field (light space) acquired by, for example, a light field camera (see FIG. 9 or the like) described later. As shown in FIG. 2, the password input image P can change the focus position to a plurality of positions (focus positions P1 to P5). The in-focus position P1 to the in-focus position P5 are arranged in order of increasing distance from the shooting position. Therefore, the focus position P1 is closest to the shooting position, and the focus position P5 is farthest from the shooting position.

  The password input image P includes a plurality of objects O as subjects. For these objects O, for example, subjects of images taken by a light field camera or the like may be employed. Further, the object O may be added to the captured image, or the subject may be deleted from the captured image.

  For example, the password input image P includes an object O for each of the five in-focus positions P1 to P5. Four objects O of a star shape O11, a star shape O12, a heart shape O13, and a wire mesh O14 are arranged at the focus position P1. Two objects O, a person O21 and a person O22, are arranged at the in-focus position P2. The object O of the tree O31 is arranged at the focus position P3. Three objects O, which are a tree (two) O41, a house O42, and a tree (one) O43, are arranged at the in-focus position P4. Two objects O, a mountain O51 and a forest O52, are arranged at the in-focus position P5.

  3A to 3E are diagrams showing examples of password input images P displayed on the display unit 102. FIG. As shown in FIGS. 3A to 3E, the password input image P is displayed in a state where the objects O arranged at the in-focus positions P1 to P5 overlap each other.

  FIG. 3A shows a display example of the password input image P when the focus position P1 is focused. As shown in FIG. 3A, the display unit 102 clearly displays the objects O, which are the star shape O11, the star shape O12, the heart shape O13, and the wire mesh O14, which are arranged at the in-focus position P1. On the display unit 102, the other objects O arranged at the in-focus positions P2 to P5 are displayed with a coarse image quality. The object O arranged at the focus positions P2 to P5 is displayed so as to become coarser as the distance from the focus position P1 increases.

  FIG. 3B shows a display example of the image P when the focal point is focused on the in-focus position P2. As shown in FIG. 3B, the display unit 102 clearly displays the objects O21 and O22 arranged at the in-focus position P2. On the display unit 102, the other objects O arranged at the in-focus positions P1, P3 to P5 are displayed in a state where the image quality is rough. The object O arranged at the focus positions P1, P3 to P5 is displayed so as to become coarser as the distance from the focus position P2 increases.

  FIG. 3C shows a display example of the image P when the focal point is focused on the in-focus position P3. As shown in FIG. 3C, the object of the tree O31 arranged at the in-focus position P3 is clearly displayed on the display unit 102. On the display unit 102, other objects arranged at the focus positions P1, P2, P4, and P5 are displayed in a state where the image quality is coarse. The objects arranged at the focus positions P1, P2, P4, and P5 are displayed so as to become coarser as the distance from the focus position P3 increases.

  FIG. 3D shows a display example of the image P when the focal point is focused on the in-focus position P4. As shown in FIG. 3D, the display unit 102 clearly displays the objects of the tree (two) O41, the house O42, and the tree (one) O43 arranged at the in-focus position P4. . On the display unit 102, other objects arranged at the focus positions P1 to P3 and P5 are displayed in a state where the image quality is coarse. The objects arranged at the focus positions P1 to P3 and P5 are displayed so as to become coarser as the distance from the focus position P4 increases.

  FIG. 3E shows a display example of the image P when the focus is set at the in-focus position P5. As shown in FIG. 3E, the display unit 102 clearly displays each object of the mountain O51 and the forest O52 arranged at the in-focus position P5. On the display unit 102, other objects arranged at the in-focus positions P1 to P4 are displayed in a state where the image quality is rough. The objects arranged at the focus positions P1 to P4 are displayed so as to become coarser as the distance from the focus position P5 increases.

  Subsequently, the extraction unit 106 illustrated in FIG. 1B extracts a plurality of objects O that can be selected via the operation unit 103 from the plurality of objects O of the password input image P displayed on the display unit 102. . The extraction unit 106 changes the object O to be extracted every time the object O is selected.

  Further, the extraction unit 106 extracts the object O included in the focus position within a predetermined range with reference to the focus position of the password input image P displayed on the display unit 102. In the present embodiment, as the in-focus position within the predetermined range, for example, the display unit 102 includes the in-focus position displayed, and includes up to two adjacent in-focus positions with respect to the in-focus position. The in-focus position is not included.

  FIG. 4A is a table showing the relationship between the in-focus position and the object O extracted by the extraction unit 106. The vertical column of the table shown in FIG. 4A is a position where the password input image P displayed on the display unit 102 is focused. The horizontal column of the table shown in FIG. 4 is the in-focus position to be extracted by the extraction unit 106.

  As shown in FIG. 4A, when the password input image P is focused at the focus position P1, the extraction unit 106 extracts the object O arranged at the focus positions P1 to P3. Further, the object O arranged at the focus positions P4 and P5 is not extracted. In this case, the object O arranged at the focus positions P1 to P3 can be selected by the operation unit 103, and the object O arranged at the focus positions P4 and P5 cannot be selected by the operation unit 103.

  Further, when the password input image P is focused at the focus position P2, the extraction unit 106 extracts the object O arranged at the focus positions P1 to P4. Further, the object O arranged at the in-focus position P5 is not extracted. In this case, the object O arranged at the focus positions P1 to P4 can be selected by the operation unit 103, and the object O arranged at the focus position P5 cannot be selected by the operation unit 103.

  In addition, when the password input image P is focused at the focus position P3, the extraction unit 106 extracts the objects O arranged at the focus positions P1 to P5. In this case, all the objects O arranged at the focus positions P1 to P5 can be selected by the operation unit 103.

  When the password input image P is focused at the focus position P4, the extraction unit 106 extracts the object O arranged at the focus positions P2 to P5. Further, the object O arranged at the in-focus position P1 is not extracted. In this case, the object O arranged at the focus positions P2 to P5 can be selected by the operation unit 103, and the object O arranged at the focus position P1 cannot be selected by the operation unit 103.

  When the password input image P is focused at the focus position P5, the extraction unit 106 extracts the object O arranged at the focus positions P3 to P5. Further, the object O arranged at the focus positions P1 and P2 is not extracted. In this case, the object O arranged at the focus positions P3 to P5 can be selected by the operation unit 103, and the object O arranged at the focus positions P1 and P2 cannot be selected by the operation unit 103.

  Further, the extraction unit 106 can extract a plurality of objects O as one object O. For example, the extraction unit 106 extracts a plurality of objects O having common attributes as one object O. FIG. 4B is a table showing the correspondence between the attributes of the object O and the objects O belonging to the attribute.

  As shown in FIG. 4 (b), the extraction unit 106 uses, for example, a symbol mark (hereinafter referred to as attribute (A)), a person (hereinafter referred to as attribute (I)), an artifact ( Hereinafter, it can be classified into attributes such as an attribute (c)), a natural object (hereinafter abbreviated as attribute (d)), and a landscape (hereinafter abbreviated as attribute (e)).

  The attribute (a) includes, for example, the above-mentioned star shapes O11, O12, heart shape O13, and the like. The attribute (a) includes, for example, the above-described persons O21 and O22. The attribute (c) includes, for example, the above-described wire mesh O14 and house O42. The attribute (d) includes, for example, the above trees O31, O41, O43, forest O52, and the like. The attribute (e) includes, for example, the above mountain O51. Note that the extraction unit 106 may extract a plurality of objects O included in the same in-focus position as one object O instead of each attribute as described above. In this case, each object O is an object corresponding to the in-focus positions P1 to P5.

  When the object O is selected via the operation unit 103, the input unit 107 locks information regarding the selected object O and the selection order of the object O with respect to each of the objects O in a predetermined operation lock. Or enter it as the unlock password.

  The storage unit 108 stores data of the password input image P. The storage unit 108 stores programs, data, and the like related to the processes performed by the processing unit 104, the display control unit 105, the extraction unit 106, and the input unit 107. Examples of such a program include the above-described password management program.

  The password management program can change the focus position to a plurality of positions (focus positions P1 to P5), and display the password input image P on which the object O is provided over the plurality of focus positions on the display unit 102. Processing, extraction processing for extracting a plurality of objects O that can be selected from the plurality of objects O displayed on the display unit 102 via the operation unit 103, and when an object O is selected via the operation unit 103 For each of the objects O, the control device CONT performs an input process of inputting information associating the selected object O with the selection order of the object O as a password for locking or unlocking a predetermined operation. The password management program causes the display control unit 105 to perform the above display process. Further, the password management program causes the extraction unit 106 to perform the above extraction process. Further, the password management program causes the input unit 107 to perform the above input processing.

  Next, the operation of the information terminal 100 configured as described above will be described. In the present embodiment, an operation for allowing a user to input a password when setting a password for locking various operations of the information terminal 100 or when releasing the lock by a password will be described as an example.

  FIG. 5 is a flowchart showing an operation for inputting a password. 6A to 6M are diagrams showing the display state of the display unit 102 in the operation of inputting a password. Hereinafter, a description will be given with reference to FIGS. 5 and 6.

  First, as shown in FIG. 5, when the user inputs a password, the control device CONT causes the display control unit 105 to display the password input image P on the display unit 102 (step S01). By this operation, the initial screen of the password input image P is displayed on the display unit 102 as shown in FIG. On the initial screen, the password input image P is focused at the focus position P2.

  Next, the control unit CONT causes the extraction unit 106 to extract a plurality of objects O that can be selected via the operation unit 103 from the plurality of objects O displayed on the display unit 102 (step S02). In this case, since the password input image P is focused on the focus position P2 and displayed on the display unit 102, the extraction unit 106 focuses as described with reference to the table of FIG. The object O arranged at the positions P1 to P4 is extracted. Further, the extraction unit 106 does not extract the object O placed at the in-focus position P5. Thereby, the object O arranged at the in-focus positions P1 to P4 can be selected by the operation unit 103, and the object O arranged in the arrangement at the in-focus position P5 cannot be selected by the operation unit 103. Note that the password input image P is displayed on the display unit 102 in a state of being focused on the focus position P2. Therefore, the object O arranged at the focus position P1 or P3 is displayed in a rough state. In the present embodiment, it is possible to select the object O displayed roughly as described above.

  Next, the control device CONT instructs the user to select the first object O, for example, by the processing unit 104 (step S03). In this case, the processing unit 104 may cause the display unit 102 to display content indicating that the object O is selected by the touch panel 103a, for example.

  The control device CONT determines whether the user has selected the object O (step S04). If the control device CONT determines that the object O is not selected (NO in step S04), the control device CONT returns to step S3 again and repeatedly determines whether or not the object O has been input.

  If it is determined that the object O has been selected (YES in step S04), the control device CONT stores the contents of the selected object O and the order of selection in the storage unit 108 for the next operation. Move. In the present embodiment, as shown in FIG. 6B, for example, the star O11 is selected by the user. When the star O11 is selected by the user, the control device CONT focuses the password input image P at the focus position P1 of the selected star O11 and displays it on the display unit 102 (step S05). Accordingly, as shown in FIG. 6C, the focus position of the password input image P is switched from the focus position P2 to the focus position P1.

  Next, the control unit CONT causes the extraction unit 106 to extract a plurality of objects O that can be selected via the operation unit 103 from the plurality of objects O displayed on the display unit 102 (step S06). Thus, the extraction unit 106 changes the object O to be extracted every time the object O is selected.

  In this case, since the password input image P is focused on the focus position P1 and displayed on the display unit 102, the extraction unit 106 focuses as described with reference to the table of FIG. The object O arranged at the positions P1 to P3 is extracted. Further, the extraction unit 106 does not extract the object O arranged at the focus positions P4 and P5. Thereby, the object O arranged at the focusing positions P1 to P3 can be selected by the operation unit 103, and the object O arranged at the focusing positions P4 and P5 cannot be selected by the operation unit 103.

  Next, the control device CONT instructs the user to select whether to select the next object O or the selection of the object O, for example, by the processing unit 104 (step S07). In this case, for example, the processing unit 104 causes the display unit 102 to display the content indicating that the object O is selected by the touch panel 103a, and presses the button indicating that the selection of the object O is completed using the touch panel 103a. Display.

  The control device CONT determines whether the user has selected the object O (step S08). When determining that the object O has not been selected (NO in step S08), the control device CONT determines whether or not the selection of the object O has been completed (step S09). If it is determined that the selection of the object O has not been completed (NO in step S09), steps S08 and S09 are repeated again.

  If the control device CONT determines that the object O has been selected (YES in step S04), the control device CONT repeatedly performs the processing from step S05 to step 07. In the present embodiment, as shown in FIGS. 6D to 6M, a case where a password is selected six times will be described as an example. In this case, first, as shown in FIG. 6D, for example, a heart shape O13 is selected as the second object O by the user. When the heart shape O13 is selected by the user, the control device CONT focuses the password input image P at the focus position P1 of the selected heart shape O13 and displays it on the display unit 102. Accordingly, as shown in FIG. 6E, the focus position of the password input image P remains the focus position P1.

  Thereafter, the control device CONT performs the operations of Steps S06 to 07 as described above. Then, as shown in FIG. 6F, for example, a tree O31 is selected as the third object O by the user. When the tree O31 is selected by the user, the control device CONT focuses the password input image P at the focus position P3 of the selected tree O31 and displays it on the display unit 102. Accordingly, as shown in FIG. 6G, the focus position of the password input image P is changed from the focus position P1 to the focus position P3.

  Thereafter, as shown in FIG. 6 (h), FIG. 6 (j), and FIG. 6 (l), the user uses the house O42 as the fourth object O, the forest O52 as the fifth object O, and the sixth object O. Tree O41 is selected as the object O. In this case, as shown in FIG. 6 (i), FIG. 6 (k), and FIG. 6 (m), the control device CONT uses the display control unit 105 to focus the focus position P4 of the house O42 and the focus of the forest O52, respectively. The focus position is changed to the position P5 and the focus position P4 of the tree O41 and displayed on the display unit 102.

  Then, after the user selects the sixth object O, when the selection completion button is pressed in step S09, the control unit CONT is selected for each of the six objects O by the input unit 107. Information associating the object O with the selection order of the object O is input as a password (step S10). Thereby, the input of the password is completed.

  As a result of the above operation, as shown in FIGS. 6C to 6M, the portion of the surface of the display unit 102 (touch panel 103a) corresponding to the selected object O is printed with the user's fingerprint. Traces F1 to F6 are left. However, it is difficult to accurately estimate the password because it is not possible to determine which position of the focus positions P1 to P5 has selected the object O simply by looking at the fingerprint marks F1 to F6. Further, since the rule for selecting the object O is not stored, but the object O is selected from the password input image P that matches the set rule, the user can easily remember and forget it.

  As described above, in the present embodiment, the focus input position P can be changed to a plurality of positions (focus positions P1 to P5), and the object O is provided over the plurality of focus positions P1 to P5. Is displayed on the display unit 102, a plurality of selectable objects O are extracted from the plurality of objects O displayed on the display unit 102 through the operation unit 103, and the object O is selected through the operation unit 103. In this case, for each object O, information that associates the selected object O with the selection order of the object O is input as a password for locking or unlocking a predetermined operation, making it difficult to estimate the password. . Thereby, the information terminal 100 which can ensure high security can be provided.

  In the present embodiment, since the extraction unit 106 classifies the object O into attributes (A) to (E), for example, when canceling a password, the same attribute as the attribute of the object O set as the password is used. Can be set so that the password can be released even if a different object O is selected. Therefore, the password input image P does not necessarily have a single configuration as in the prior art, and an object O having the same attribute exists in the password input image P, and the object O can be input as a password for cancellation. Any state is acceptable.

  In this case, the display control unit 105 may display the object O that satisfies such a condition at a random position from the plurality of objects O held by the information terminal 100. Thereby, the selection position and selection order of the user on the display unit 102 (touch panel 103a) are the change in the position of the object O, the distance of the object O arranged at each of the focus positions P1 to P5, and the object O. Many patterns can be taken in combination with attribute information. For this reason, even when the same password is canceled, the possibility of having the same selection position and selection order is extremely low. Therefore, the password cannot be released only by others seeing and repeating the user's selection order when releasing the password. For this reason, the information terminal 100 can be firmly locked.

Second Embodiment
Next, a second embodiment of the present invention will be described.
In the present embodiment, a light field camera 1 that is a means for acquiring the password input image P will be described as an example of an electronic device.

  FIG. 7 is a cross-sectional view illustrating a schematic configuration of an optical system portion in a light field camera 1 which is an example of an imaging apparatus. In FIG. 7, the right direction (optical axis direction) of the paper surface is the X-axis, the upward direction of the paper surface orthogonal to the X-axis is the Z-axis, and the direction from the back of the paper surface orthogonal to the X-axis and Z-axis is the front direction. Is the Y axis. FIG. 7 is a cross-sectional view when the optical system portion in the light field camera 1 is cut along a plane including the optical axis.

  The light field camera 1 of the present embodiment acquires information on a light field (light space) that is a light ray set in a three-dimensional space, and generates an image based on the acquired information on the light field. In the following description, the light field information is referred to as light ray information. A typical function of the light field camera 1 is a refocus function for generating an image whose focal length is changed by post-processing after shooting. Therefore, the light field camera 1 is also called a refocus camera.

  The light field camera 1 of the present embodiment includes a photographing optical system (imaging optical system) 11 and a microlens array 12 as its optical system part. FIG. 7 also shows the image sensor 20 that is a configuration other than the optical system portion in the light field camera 1.

  The photographing optical system 11 is a single focus lens composed of a plurality of lens groups. The photographing optical system 11 forms an image of a light beam from the subject in the vicinity of its focal plane (imaging plane). Further, the photographing optical system 11 includes an aperture stop (not shown) that controls an aperture value (F value) during photographing. Since the aperture value can be changed by reconstructing the image after shooting, the aperture diameter of the aperture stop may be fixed. As shown in FIG. 7, a microlens array 12 and an image sensor 20 are sequentially arranged near the focal plane of the photographing optical system 11.

  The microlens array 12 has a configuration in which a plurality of microlenses (positive lenses) having positive power are two-dimensionally arranged. In the microlens array 12 shown in FIG. 7, only three microlenses are arranged in the Z-axis direction, but actually, a large number of microlenses are arranged in the Z-axis direction and the Y-axis direction.

The image pickup device 20 is disposed on the rear side of the microlens array 12 (opposite side of the photographing optical system 11) with a slight gap. The image sensor 20 receives light beams that have passed through the microlenses of the microlens array 12 by a plurality of photoelectric conversion elements, and photoelectrically converts the light beams received by the photoelectric conversion elements to generate a plurality of pixel signals. The plurality of photoelectric conversion elements are arranged in a matrix, and each photoelectric conversion element forms a pixel. In the present embodiment, the image sensor 20 is a CCD (Charge Coupled Device) or CMOS (Complementary).
It consists of a two-dimensional solid-state image sensor such as a metal-oxide semiconductor).

  In the example shown in the partially enlarged view of FIG. 7, a plurality of pixels are arranged on the rear side of each of the plurality of microlenses 12a, 12b, and 12c. Specifically, three pixels 21a, 21b, 21c are arranged on the rear side of the micro lens 12a, three pixels 22a, 22b, 22c are arranged on the rear side of the micro lens 12b, and on the rear side of the micro lens 12c. Three pixels 23a, 23b, and 23c are arranged.

  Three pixels 21a, 21b, and 21c are pixels 21 corresponding to the microlens 12a, three pixels 22a, 22b, and 22c are pixels 22 corresponding to the microlens 12b, and three pixels 23a, 23b, and 23c are microscopic. This is a pixel 23 corresponding to the lens 12c. In the partially enlarged view of FIG. 7, three pixels are shown as pixels corresponding to each of the microlenses 12a, 12b, and 12c, but actually, a large number of pixels are arranged in the Z-axis direction and the Y-axis direction ( (See FIG. 8 described later).

  Next, acquisition of a light field by the light field camera 1 will be briefly described with reference to FIG. In FIG. 7, it is assumed that the subject at the position A is focused on the position of the microlens array 12 by the photographing optical system 11. The light beam from the point A1 on the subject passes through the partial region 11a on the pupil of the photographing optical system 11, passes through the microlens 12a, and is received by the pixel 21c of the image sensor 20. Further, another light ray from the point A1 on the subject passes through the partial region 11b on the pupil of the photographing optical system 11, passes through the microlens 12a, and is received by the pixel 21b of the imaging device 20. Further, another light ray from the point A1 on the subject passes through the partial region 11c on the pupil of the photographing optical system 11, passes through the microlens 12a, and is received by the pixel 21a of the imaging device 20.

  Similarly, the light beam from the point A2 on the subject passes through the partial area 11a on the pupil of the photographing optical system 11, passes through the microlens 12b, and is received by the pixel 22c of the image sensor 20. Further, another ray from the point A2 on the subject passes through the partial region 11b on the pupil of the photographing optical system 11, passes through the microlens 12b, and is received by the pixel 22b of the image sensor 20. Further, another light ray from the point A2 on the subject passes through the partial region 11c on the pupil of the photographing optical system 11, passes through the microlens 12b, and is received by the pixel 22a of the imaging device 20.

  Further, the light beam from the point A3 on the subject passes through the partial area 11a on the pupil of the photographing optical system 11, passes through the microlens 12c, and is received by the pixel 23c of the image sensor 20. Further, another light ray from the point A3 on the subject passes through the partial region 11b on the pupil of the photographing optical system 11, passes through the microlens 12c, and is received by the pixel 23b of the image sensor 20. Further, another light ray from the point A3 on the subject passes through the partial region 11c on the pupil of the photographing optical system 11, passes through the microlens 12c, and is received by the pixel 23a of the image sensor 20. As described above, the light beam from the subject at the position A is received by the pixels of the image sensor 20 located on the rear side of the microlenses 12a, 12b, and 12c, whereby the brightness and the traveling direction are acquired as light beam information. The

  Of the three pixels corresponding to the microlenses 12a, 12b, and 12c, the image generated by the pixel signals of the lower pixels 21c, 22c, and 23c is an image of light rays that have passed through the partial region 11a. In addition, the image generated by the pixel signals of the middle pixels 21b, 22b, and 23b is an image of light rays that have passed through the partial region 11b. Further, the image generated by the pixel signals of the upper pixels 21a, 22a, and 23a is an image of light rays that have passed through the partial region 11c. For example, when the number of pixels corresponding to each microlens is N, N reconstructed images configured by arranging pixel values of pixels located at the same position with respect to the microlens are configured to have the photographic optical system 11 N stereo image groups acquired by dividing into partial regions are formed.

  FIG. 8 is a diagram for explaining the pixel array and the incident area of the image sensor 20. 8 shows a state where the image sensor 20 is observed from the X-axis direction of FIG. In the image sensor 20 shown in FIG. 8, a region where pixels are arranged is referred to as a pixel region 200. In the pixel region 200, for example, 10 million or more pixels are arranged in a matrix. The circular area in the pixel area 200 is an area where light rays that have passed through one microlens are incident. Accordingly, a plurality of circular regions are formed for each of the plurality of microlenses. These regions are referred to as incident regions 201, 202. For example, 586 × 440 microlenses are arranged. However, such an arrangement number is an example, and a larger or smaller number may be used. In the example shown in FIG. 8, the diameter of one microlens is 13 pixels. However, the number of pixels is not limited to this.

  As described above, the light field camera 1 extracts and reconstructs pixel values of pixels at the same position in the incident area (the same position in the Y-axis direction and the Z-axis direction in the incident area) from all the microlenses. . The reconstructed image group obtained in this way corresponds to an image group in which a subject is observed from different partial areas of the photographing optical system 11. Therefore, these reconstructed image groups have parallax according to the position of the partial region and the distance to the subject. For example, as shown in FIG. 8, a reconstructed image composed of pixel values of pixels of each microlens located at the same position as the pixel 201a in the incident area 201 and each micro located at the same position as the pixel 201b in the incident area 201 It has parallax with the reconstructed image composed of pixel values of the lens pixels.

  FIG. 9 is a block diagram showing a configuration of the light field camera 1. The light field camera 1 shown in FIG. 9 includes an imaging unit 10, an image processing unit 30, a work memory 40, a display unit 50, an operation unit 55, a recording unit 60, an audio output unit 65, and a system control unit 70. The imaging unit 10 includes a photographing optical system 11, a microlens array 12, a drive control unit 13, an imaging element 20, and a drive unit 21. Since the photographing optical system 11 and the microlens array 12 have been described with reference to FIG. 7, the description thereof is omitted in FIG.

  For example, a Bayer array color filter array is disposed on the imaging surface of the imaging element 20. The image sensor 20 makes the light beam that has passed through the photographing optical system 11 and the microlens array 12 enter. The image sensor 20 photoelectrically converts an incident light beam to generate a pixel signal of each pixel of an analog signal. In the imaging unit 10, an amplifier (not shown) amplifies the pixel signal of each pixel of the analog signal with a predetermined gain factor (amplification factor). Further, an A / D converter (not shown) converts the pixel signal of each pixel of the analog signal amplified by the amplifier into the pixel signal of each pixel of the digital signal. Then, the imaging unit 10 outputs RAW data including pixel signals of each pixel of the digital signal to the image processing unit 30. The light ray information acquired by the light field camera 1 is specified by RAW data output from the imaging unit 10.

  The drive control unit 13 performs aperture stop drive control based on control information transmitted from the system control unit 70 in order to adjust the aperture diameter of the aperture stop included in the photographing optical system 11. The drive unit 21 is a control circuit that controls driving of the image sensor 20 under imaging conditions set based on an instruction signal from the system control unit 70. Examples of imaging conditions include exposure time (shutter speed), frame rate, gain, and the like.

  Here, the exposure time refers to the time from when the photoelectric conversion element of the image sensor 20 starts to accumulate after the start of charge accumulation according to incident light. The frame rate is a value representing the number of frames processed (displayed or recorded) per unit time in a moving image. The unit of the frame rate is represented by fps (Frames Per Second). The gain refers to a gain factor (amplification factor) of an amplifier that amplifies the pixel signal. By changing this gain, the ISO sensitivity can be changed. This ISO sensitivity is a photographic film standard established by ISO and represents how much light the photographic film can record. However, generally, ISO sensitivity is also used when expressing the sensitivity of the image sensor 20. In this case, the ISO sensitivity is a value representing the ability of the image sensor 20 to capture light.

  The image processing unit 30 performs various kinds of image processing on the RAW data including the pixel signal of each pixel, and generates image data in a predetermined file format (for example, JPEG format). The image data generated by the image processing unit 30 includes image data of still images, moving images, and live view images. The live view image is an image displayed on the display unit 50 by sequentially outputting the image data generated by the image processing unit 30 to the display unit 50. The live view image is used for the user to confirm the image of the subject imaged by the imaging unit 10. The live view image is also called a through image or a preview image.

  The image processing unit 30 uses the work memory 40 as a work space to generate image data set at an arbitrary focal length by performing predetermined image processing on RAW data composed of pixel signals of each pixel. Specifically, the image processing unit 30 generates image data set at an arbitrary focal length by performing the following image processing.

  FIG. 10 is a diagram for explaining image reconstruction. As shown in FIG. 10, the image processing unit 30 has pixel values (values indicated by pixel signals) of the pixels 201a, 202a... At the same position with respect to the respective incident areas 201, 202. To extract. The image processing unit 30 rearranges the extracted pixel values to generate reconstructed image data 400a. When the number of pixels corresponding to each of the incident areas 201, 202... Is N, N reconstructed image data are generated.

  The image processing unit 30 translates the N reconstructed image data groups by a predetermined amount in accordance with a predetermined focal length. Then, the image processing unit 30 adds and averages the N reconstructed image data groups after the parallel movement. Thereby, image data set to a predetermined focal length is generated. Since the image of the subject existing at a depth corresponding to the amount of movement is aligned, a sharp contour image is formed. An image of a subject that does not exist at a depth corresponding to the amount of movement is blurred. The image processing described above is an example, and the image processing unit 30 may generate image data set at an arbitrary focal length by executing image processing different from the image processing described above. For example, the image processing unit 30 calculates a light beam acquired by each pixel of the image sensor 20 by performing a predetermined calculation. Then, the image processing unit 30 generates image data set to a predetermined focal length by projecting the calculated light beam onto a virtual image plane set to a predetermined focal length.

  In the present embodiment, the image processing unit 30 has a focal position (focal point) such as image data focused on the foremost subject, image data focused on the farthest subject, and image data focused on a subject at an intermediate position. A plurality of image data having different distances are generated.

  The image processing unit 30 performs various types of image processing on image data set at an arbitrary focus using the work memory 40 as a work space. For example, the image processing unit 30 generates RGB image data by performing color signal processing (tone correction) on a signal obtained by the Bayer array. The image processing unit 30 performs image processing such as white balance adjustment, sharpness adjustment, gamma correction, and gradation adjustment on the RGB image data. Further, the image processing unit 30 performs a process of compressing in a predetermined compression format (JPEG format, MPEG format, etc.) as necessary.

  The image processing unit 30 detects one or more subjects from the image data. In the present embodiment, the image processing unit 30 detects the subject by specifying the boundary of the subject region based on the contrast and color change between the bright and dark portions in the image data. Note that the image processing unit 30 may detect a subject of a human subject using a known face detection function. Further, the image processing unit 30 may detect a moving subject by comparing a plurality of pieces of image data obtained in time series for a moving subject (moving subject). In addition to the face detection, the image processing unit 30 may detect a human body included in the image data as a subject as described in, for example, Japanese Patent Application Laid-Open No. 2010-16621 (US2010 / 0002940). .

  Further, the image processing unit 30 detects (calculates) a distance (focal length) to each detected subject. For example, the image processing unit 30 detects the parallax of each subject in a plurality of reconstructed image data (for example, reconstructed image data composed of pixel values of pixels at two specific positions in the incident region), and uses the detected parallax. Based on this, the distance to each subject is calculated.

  The image processing unit 30 outputs the generated image data to the recording unit 60. In addition, the image processing unit 30 outputs the generated image data to the display unit 50. In addition, the image processing unit 30 outputs position data indicating the detected size of the subject and the position in the image data in units of pixels to the recording unit 60. In addition, the image processing unit 30 outputs distance data indicating the detected distance to each subject to the recording unit 60.

  The work memory 40 temporarily stores RAW data, image data, and the like when image processing by the image processing unit 30 is performed. The display unit 50 displays an image captured by the imaging unit 10 and various information. The display unit 50 includes a display screen 51 configured by, for example, a liquid crystal display panel. The image displayed on the display unit 50 includes a still image, a moving image, and a live view image.

  The operation unit 55 includes a release switch (a switch that is pressed when shooting a still image), a moving image switch (a switch that is pressed when shooting a moving image), various operation switches, and the like that are operated by the user. The operation unit 55 outputs a signal corresponding to the operation by the user to the system control unit 70. The recording unit 60 has a card slot into which a storage medium such as a memory card can be mounted. The recording unit 60 stores the image data and various data generated by the image processing unit 30 in a recording medium mounted in the card slot. The recording unit 60 has an internal memory. The recording unit 60 can also record the image data and various data generated by the image processing unit 30 in the internal memory. The audio output unit 65 performs audio output from the speaker based on the instruction signal from the system control unit 70.

  The system control unit 70 controls the overall processing and operation of the light field camera 1. The system control unit 70 has a CPU (Central Processing Unit). The system control unit 70 performs control to output the image data generated by the image processing unit 30 to the display unit 50 and display an image (live view image, still image, moving image) on the display screen 51 of the display unit 50. . Further, the system control unit 70 performs control to read out the image data recorded in the recording unit 60 and output the image data to the display unit 50 and display an image on the display screen 51 of the display unit 50.

  In addition, the system control unit 70 instructs the predetermined imaging conditions in order to perform imaging with predetermined imaging conditions (exposure time, frame rate, gain, etc.) on the imaging surface (pixel area 200) of the imaging device 20. A signal is output to the drive unit 21. The system control unit 70 also instructs the image processing unit 30 to execute image processing with predetermined control parameters (control parameters such as color signal processing, white balance adjustment, gradation adjustment, and compression rate). The instruction signal is output to the image processing unit 30. Further, the system control unit 70 controls the recording unit 60 to record the image data by causing the recording unit 60 to output the image data generated by the image processing unit 30. Further, the system control unit 70 causes the recording unit 60 to output the position data and distance data of each subject detected by the image processing unit 30, and records the image data, the position data, and the distance data in association with each subject. Control for recording in the unit 60 is performed. In addition to the above control, the system control unit 70 also performs control such as adjustment of the aperture diameter of the aperture stop provided in the photographing optical system 11.

  The light field camera 1 configured as described above may further have a configuration corresponding to the display control unit 105, the extraction unit 106, and the input unit 107 described in the first embodiment. In this case, when a password is input to the light field camera 1, the password can be input by the method described in the above embodiment. Thereby, it is difficult to estimate the password, and the information terminal 100 that can ensure high security can be provided. A configuration corresponding to each part of the light field camera 1 may be mounted on the information terminal 100.

The technical scope of the present invention is not limited to the above-described embodiment, and appropriate modifications can be made without departing from the spirit of the present invention.
For example, in the first embodiment, the object O arranged in the password input image P has been described as an example of the configuration in which the object O is arranged at a fixed position on the display unit 102. However, the present invention is not limited to this. Absent. For example, when inputting a preset password, the display control unit 105 may change and display the display position of the object O on the display unit 102.

  At this time, the display control unit 105 changes the display position so that the plurality of objects O can be selected in the set order (predetermined order). That is, the display position of the object O is changed so that the object O to be selected next is not arranged at a position that is not extracted by the extraction unit 106. As a result, since the fingerprint trace remaining on the display unit 102 is different every time it is input, it is difficult to estimate the password only by looking at the fingerprint trace.

  Moreover, although the said embodiment demonstrated the example using the image produced | generated based on the information of the light field as the password input image P, it is not limited to this. For example, as the password input image P, an image acquired by continuous shooting (focus bracketing) while changing the focus position may be used. In this case, since a plurality of images are used, it is only necessary to associate the images and embed a marker in the object O. Further, instead of a series of images acquired by the focus bracket, a new password input image P may be generated by combining predetermined objects O.

  DESCRIPTION OF SYMBOLS 1 ... Light field camera (electronic device), 100 ... Information terminal (electronic device), 102 ... Display part, 103 ... Operation part, 104 ... Processing part, 105 ... Display control part, 106 ... Extraction part, 107 ... Input part, O ... object, P ... password input image, P1 to P5 ... focus position

Claims (11)

A display for displaying an image;
An operation unit that a user can operate in accordance with the image;
A processing unit that causes the display unit to perform a predetermined operation in accordance with at least a user operation;
A display control unit that can change the in-focus position to a plurality of positions and display a password input image in which an object is provided across the plurality of in-focus positions on the display unit;
An extraction unit that extracts a plurality of objects that can be selected from the plurality of objects displayed on the display unit via the operation unit;
When the object is selected via the operation unit, for each of the objects, information associating the selected object with the selection order of the object is used as a password for locking or unlocking the predetermined operation. An electronic device comprising: The electronic device according to claim 1, wherein the extraction unit changes the object to be extracted every time the object is selected. The electronic device according to claim 2, wherein the extraction unit extracts the object included in the focus position within a predetermined range with reference to the focus position of the password input image displayed on the display unit. . The electronic device according to any one of claims 1 to 3, wherein the extraction unit extracts a plurality of the objects as one object. The electronic device according to claim 4, wherein the extraction unit extracts a plurality of the objects having common attributes as one object. The electronic device according to claim 4, wherein the extraction unit extracts a plurality of the objects included in the same in-focus position as one object. The display control unit causes the password input image to be focused at the focus position of the selected object and displayed on the display unit every time selection by the operation unit is performed. 6. The electronic device according to claim 1. The display control unit changes and displays the display position of the object in the display unit so that a plurality of the objects can be selected in a predetermined order. 8. The electronic device described. The electronic device according to claim 1, wherein the operation unit includes a touch panel provided on the display unit. The password input image is an image generated based on information of a light field that is a set of rays in a three-dimensional space, or an image obtained by continuous shooting while changing the in-focus position. The electronic device according to claim 9. In a control device for an electronic device that includes a display unit that displays an image and an operation unit that a user can operate according to the image, and that causes the display unit to perform a predetermined operation according to at least a user operation,
A process of displaying on the display unit a password input image in which an in-focus position can be changed to a plurality of positions and an object is provided over a plurality of the in-focus positions;
A process of extracting a plurality of objects selectable via the operation unit from the plurality of objects displayed on the display unit;
When the object is selected via the operation unit, for each of the objects, information associating the selected object with the selection order of the object is used as a password for locking or unlocking the predetermined operation. Password management program that executes the process of entering as.