JP5976174B2 - Image processing apparatus, control method therefor, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, a control method therefor, and a program, and more particularly, to an image processing apparatus that displays a display item for operation, a control method therefor, and a program.

  A digital camera equipped with a touch panel is known. In the touch panel, display items for operation are displayed on the screen, thereby reducing operation parts and improving operability. For example, in a digital camera, when a user performs a flick operation in a region where an image is displayed on a touch panel in a state where an image is displayed on a display unit, a function of feeding an image is realized. In addition, a method for improving the operability of the touch panel by stereoscopically displaying display items for operation is also disclosed (see, for example, Patent Document 1 and Patent Document 2).

  In patent document 1, the display item for operation is displayed in three dimensions, and when a user operation is performed on the display item, the state is changed in three dimensions. As a result, the user can recognize that the operation has been performed reliably.

  In Patent Document 2, a sensor for detecting the approach to the touch panel is provided before the user touches the touch panel, and the display item for operation displayed on the display unit is switched from the stereoscopic display to the two-dimensional display according to the detection of the sensor. . This improves operability.

Japanese Patent Laid-Open No. 10-105735 Japanese Patent Application Laid-Open No. 2004-280496

  However, in the methods according to Patent Document 1 and Patent Document 2, since display items displayed in a three-dimensional manner on a screen are operated via a flat touch panel, display items that can be operated without a sense of incongruity for the user are limited to a part such as buttons. The

  In addition, when a display item displayed in 3D is operated on the touch panel, if the image feed is further operated on the touch panel as in the past, both the display item displayed in 3D and the display item displayed in 2D such as a display image are displayed on the touch panel. Will be operated. Such an operation method is difficult for the user to understand.

  An object of the present invention is to provide an image processing apparatus, a control method thereof, and a program for improving the operability of an operation for a display item for operation displayed three-dimensionally, and an operation display item for two-dimensional display. is there.

In order to achieve the above object, an image processing apparatus comprising an image processing apparatus, image plan view of the display unit of the touch panel, a three-dimensional display item for inputting an operation on the image A display control unit that controls to display the image on the display unit of the touch panel so as to be superimposed on the image; a calculation unit that calculates a distance between an operator for operating the image and the touch panel; and When the calculated distance is greater than a second threshold value, control is performed to invalidate the input of the operator's instruction to the image , and when the distance calculated by the calculation unit is smaller than the second threshold value, an input control means for controlling so as to enable input of an instruction of the operator for the image, wherein the display control unit, involved in the distance calculated by the calculating means Not the displayed image on the display unit of the touch panel, when the distance calculated by the calculating means is smaller than a first threshold value, and controls to display the 3-dimensional display items, calculated by said calculation means When the calculated distance is smaller than the second threshold, the control is performed so that the three-dimensional display item is not displayed, and the input control unit is configured so that the distance calculated by the calculation unit is smaller than the first threshold. Control is performed so as to validate the input of an instruction from the operator for the three-dimensional display item, and the first threshold value is larger than the second threshold value.

  According to the present invention, it is possible to provide an image processing apparatus, a control method thereof, and a program for improving the operability of an operation on a stereoscopic display operation item and an operation on a two-dimensional operation display item. it can.

1 is an external view of a digital camera as an example of an image processing apparatus according to an embodiment of the present invention. It is a block diagram which shows the structural example of the digital camera in FIG. 6 is a diagram for explaining the principle of detecting the distance from the display unit to the finger and the position of the finger with respect to the display screen area of the display unit 102. FIG. 4A shows a configuration in the parallax barrier method, and FIG. 4B shows a configuration in the lenticular method. It is the figure which showed an example of the structure of the display image generation part of FIG. 2 at the time of implement | achieving a display part using a parallax division | segmentation system. It is a figure for demonstrating the method in which the non-contact operation detection part in FIG. 2 detects a motion of the operator's finger | toe as an operation element from the several acquired operation element image data. FIG. 3 is a diagram showing screen contents displayed on a display unit in a playback mode in which the system control unit in FIG. 2 displays image data stored in a recording medium on a display unit for viewing purposes in accordance with an operator's instruction. . It is the figure which looked down at the digital camera from the diagonal direction from the display part side. It is a flowchart which shows the procedure of the switching process performed by the system control part in FIG. 2 in 1st Embodiment. It is a figure which shows an example of a structure of the display image generation part in FIG. 2 at the time of implement | achieving a display part with the liquid crystal display in which only two-dimensional display is possible. It is a figure which shows the expression method of a three-dimensional display item, and (A)-(C) are all figures which show the expression method of a rectangular parallelepiped. It is a flowchart which shows the procedure of the switching process performed by the system control part in FIG. 2 in 2nd Embodiment. (A) shows the figure which displayed the outline of the 3D display item overlapped with the display of image display data, and (B) shows the figure which displayed the shadow of the 3D display item overwriting the display of image display data. FIG.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[First Embodiment]
FIG. 1 is an external view of a digital camera 100 as an example of an image processing apparatus according to an embodiment of the present invention.

  In FIG. 1, a display unit 102 is a display unit (display unit) that displays images and various types of information, and displays display items that are operated by an operator that is a finger. Details of this display item will be described later. The shutter button 103 is an operation unit for issuing a shooting instruction. The power switch 104 switches between power-on in the shooting mode, power-on in the playback mode, and power-off. The operation input imaging unit L and the operation input imaging unit R are imaging units for imaging the position and operation of a finger operating the digital camera 100. The operation input imaging unit L and the operation input imaging unit R are arranged at a certain interval above the display unit 102. The recording medium 107 is a recording medium such as a memory card or a hard disk and is configured to be detachable from the main body of the digital camera 100.

  FIG. 2 is a block diagram illustrating a configuration example of the digital camera 100 in FIG.

  In FIG. 2, the photographing lens 209 includes a focus lens. The shutter 210 has an aperture function. The imaging unit 211 is configured by a CCD, a CMOS element, or the like that converts an optical image into an electrical signal. The A / D converter 212 converts an analog signal into a digital signal. The A / D converter 212 is used to convert an analog signal output from the imaging unit 211 into a digital signal. The barrier 208 covers the imaging unit including the imaging lens 209 of the digital camera 100, thereby preventing the imaging system including the imaging lens 209, the shutter 210, and the imaging unit 211 from being dirty or damaged.

  The image processing unit 213 performs resize processing such as predetermined pixel interpolation and reduction and color conversion processing on the data from the A / D converter 212 or the data from the memory control unit 215. The image processing unit 213 performs predetermined calculation processing using the captured image data, and the system control unit 221 performs exposure control and distance measurement control based on the obtained calculation result.

  Thereby, AF (autofocus) processing, AE (automatic exposure) processing, and EF (flash pre-emission) processing of the TTL (through-the-lens) method are performed. The image processing unit 213 further performs predetermined arithmetic processing using the captured image data, and also performs TTL AWB (auto white balance) processing based on the obtained arithmetic result.

  Output data from the A / D converter 212 is directly written into the memory 219 via the image processing unit 213 and the memory control unit 215 or via the memory control unit 215. The memory 219 stores image data obtained by the imaging unit 211 and converted into digital data by the A / D converter 212 and image data to be displayed on the display unit 102.

  The memory 219 has a storage capacity sufficient to store a predetermined number of still images, a moving image and sound for a predetermined time.

  The memory 219 also serves as an image display memory (video memory). The D / A converter 218 converts the image display data stored in the memory 219 into an analog signal and supplies it to the display unit 102.

  Thus, the display image data written in the memory 219 is displayed on the display unit 102 via the D / A converter 218. The display unit 102 performs display according to the analog signal from the D / A converter 218 on a display such as an LCD.

  The nonvolatile memory 226 is an electrically erasable / recordable memory, and for example, an EEPROM or the like is used. The nonvolatile memory 226 stores constants, programs, and the like for operating the system control unit 221. Here, the program is a program for executing various flowcharts described later in the present embodiment.

  The system control unit 221 controls the entire digital camera 100. By executing the program recorded in the non-volatile memory 226 described above, each process of the present embodiment to be described later is realized.

  As the system memory 227, a RAM is used in this embodiment. In the system memory 227, constants and variables for operation of the system control unit 221 and programs read from the nonvolatile memory 226 are expanded. The system control unit also performs display control by controlling the memory 219, the D / A converter 218, the display unit 102, and the like.

  The shutter button 103 includes a first shutter switch and a second shutter switch, and is an operation means for inputting various operation instructions to the system control unit 221.

  The first shutter switch is turned on when the shutter button 103 provided in the digital camera 100 is being operated, so-called half-press (shooting preparation instruction), and the first shutter switch signal SW1 is generated. In response to the first shutter switch signal SW1, operations such as AF (autofocus) processing, AE (automatic exposure) processing, AWB (auto white balance) processing, and EF (flash pre-emission) processing are started.

  The second shutter switch is turned on when the operation of the shutter button 103 is completed, that is, when it is fully pressed (shooting instruction), and a second shutter switch signal SW2 is generated. In response to the second shutter switch signal SW2, the system control unit 221 starts a series of shooting processing operations from reading a signal from the imaging unit 211 to writing image data on the recording medium 107.

  The power control unit 224 includes a battery detection circuit, a DC-DC converter, a switch circuit that switches a block to be energized, and the like, and detects whether or not a battery is attached, the type of battery, and the remaining battery level. Further, the power control unit 224 controls the DC-DC converter based on the detection result and an instruction from the system control unit 221 and supplies a necessary voltage to each unit including the recording medium 107 for a necessary period.

  The power supply unit 225 includes a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, an AC adapter, or the like. An I / F 220 (hereinafter referred to as an interface) is an interface with a recording medium 107 such as a memory card or a hard disk. The recording medium 107 is a recording medium such as a memory card, and includes a semiconductor memory, a magnetic disk, or the like.

  As another operation unit, a touch panel 223 capable of detecting contact with the display unit 102 is provided. The touch panel 223 is formed integrally with the display unit 102. The touch panel control unit 222 corresponds to second recognition means for recognizing the operation content operated by the finger touching the display unit 102 (touch panel 223). Specifically, the touch panel control unit 222 can detect the following operation on the touch panel 223. Touching the touch panel with a finger or pen (hereinafter referred to as touchdown). The touch panel is touched with a finger or pen (hereinafter referred to as touch-on). Moving while touching the touch panel with a finger or pen (hereinafter referred to as “move”). The finger or pen that was touching the touch panel is released (hereinafter referred to as touch-up).

  A state where nothing touches the touch panel (hereinafter referred to as touch-off). The touch panel control unit 222 notifies the system control unit 221 of these operations and the position coordinates where the finger or pen touches the touch panel, and the system control unit 221 performs any operation on the touch panel based on the notified information. It is determined whether or not.

  Regarding the move, the moving direction of the finger or pen moving on the touch panel can also be determined for each vertical component / horizontal component on the touch panel based on the change of the position coordinates. It is also assumed that a stroke is drawn when touch-up is performed on the touch panel through a certain move from touch-down. The operation of drawing a stroke quickly is called a flick.

  A flick is an operation of quickly moving a certain distance while touching a finger on the touch panel and then releasing it, in other words, an operation of quickly tracing a finger on the touch panel. If it is detected that the moving is performed at a predetermined speed or more over a predetermined distance, and a touch-up is detected as it is, it can be determined that a flick has been performed.

  In addition, when it is detected that the movement is performed at a predetermined distance or more and less than a predetermined speed, it is determined that the drag has been performed.

  The operation input imaging unit L and the operation input imaging unit are for imaging an operator such as an operator's finger. The operator image data captured by the operation input imaging unit L and the operation input imaging unit R is directly written into the memory 219 via the image processing unit 213 and the memory control unit 215 or via the memory control unit 215. It is.

  The distance / position detection unit 216 is a derivation unit that derives the distance between the finger and the display unit 102. The distance / position detection unit 216 detects the operator's finger as the operation unit from the operation unit image data, and detects the distance from the display unit 102 to the finger and the position of the finger with respect to the display screen area of the display unit 102. Then, the system control unit 221 is notified. The system control unit 221 controls to change the display form on the display unit 102 according to the distance from the display unit 102 to the finger and the position of the finger with respect to the area of the display unit 102.

  The non-contact operation detection unit 217 detects the movement of the finger by calculating the amount of change of the position of the operator's finger, which is the operator, at regular intervals from a plurality of continuously obtained operation element image data. Notify the system controller 221. Accordingly, the non-contact operation detection unit 217 corresponds to a first recognition unit that recognizes the operation content operated without the finger touching the display unit 102.

  The system control unit 221 recognizes an operation instruction with respect to the digital camera 100 based on the movement trajectory of the finger when the operator's finger is not touching the display unit 102, and performs digital processing based on the recognized operation instruction. The entire camera 100 is controlled.

  The display image generation unit 214 generates display data to be displayed on the display unit 102. The system controller 221 controls the distance / position detector 216 to detect the operator's finger as an operator from the operator image data captured by the operation input imaging unit L and the operation input imaging unit R. To do.

  FIG. 3 is a diagram for explaining the principle of detecting the distance from the display unit 102 to the finger and the position of the finger with respect to the area of the display screen of the display unit 102.

In FIG. 3, when the xyz space is assumed, the center positions P _L and PR that are virtual viewpoint positions of the operation input imaging unit L and the operation input imaging unit _R are both on the xz plane. To do. In addition, it is assumed that the imaging surfaces 301 and 302 such as the CCDs of the operation input imaging unit L and the operation input imaging unit R are on the xy plane.

Operator's finger when in the position of P, the virtual viewpoint position P _L of the operation input imaging section L an operation input for the imaging unit _R, the straight line connecting the P _R and P, the intersection of the imaging surface _{Let the} three-dimensional coordinates be Q _L and QR, respectively. The two-dimensional coordinates on the imaging surface of this intersection point are respectively (x _L , y _L ), (x _R , y _R ), the distance between the viewpoint positions of the operation input imaging units L and 6 and the operation input imaging unit R. the 2w, viewpoint position _P L, the distance from _{P R} to the imaging surface to as d.

The unit vector in the x-axis direction is i, the unit vector in the y-axis direction is j, and the unit vector in the z-axis direction is k. In this case, the following formula is established.
P _L = −wi−dk
P _R = wi-dk
Q _L = (x _L −w) i + y _L j
Q _R = (x _R + w) i + y _R j
A straight line connecting the P _L and _{Q L,} the straight line connecting the _{P R} and _{Q R,} in order to intersect at P, the following equation holds.
P = P _L + m (Q _L −P _L ) = P _R + n (Q _R −P _R )
Solving this equation for m and n yields:
m = n = 2w / (x _L -x _R )
As a result, the finger position Pxy on the xy plane can be obtained by the following equation.
Pxy = (mx _L −w) i + my _L j
Further, the distance Pz from the display unit 102 to the finger can be obtained by the following equation.
Pz = (m−1) d
In the present embodiment, the distance from the display unit 102 to the finger is obtained from the above formula.

  Next, the display unit 102 will be described. The display unit 102 is configured by a liquid crystal display capable of autostereoscopic viewing. The display unit 102 capable of autostereoscopic viewing may be realized using a parallax dividing method such as a parallax barrier (parallax barrier) method or a lenticular method, or may use a DFD (Depth Focus 3D) method. May be realized.

  4A shows a configuration in the parallax barrier method, and FIG. 4B shows a configuration in the lenticular method.

  4A, in the case of the parallax barrier method, in the display unit 102, a left-eye pixel region 401L and a right-eye pixel region 401R are alternately set in the horizontal direction of a pixel row formed of TFT liquid crystal. . In this figure, an observer's right eye 404 and an observer's left eye 403 are assumed. The light from the left-eye pixel region 401L reaches only the left eye 403 by the parallax barrier 402 that is installed in front of the pixel row and includes vertical slits. And it is comprised so that the light from the pixel region 401R for right eyes may reach only the right eye 404. FIG.

  The parallax barrier 402 may be configured by a switch liquid crystal that can be electronically controlled. Further, the parallax barrier 402 may be set behind the pixel row and in front of the light source.

  In FIG. 4B, in the case of the lenticular method, the left-eye pixel area 405L and the right-eye pixel area 405R are alternately set in the display unit 102 in the horizontal direction of the screen. Then, it is configured such that the light from the left eye pixel region 405L reaches only the left eye and the light from the right eye pixel region 405R reaches only the right eye by the semi-cylindrical lenticular lens 406. Thus, in the parallax division method, it is possible to give a sense of depth by projecting different images to the left and right eyes of a human.

  Further, in the case of the DFD method, the display unit 102 arranges two TFT liquid crystals in front and back with an appropriate interval, and displays an image of a brightness ratio corresponding to the distance on the two TFT liquid crystals. Specifically, the closer the object is, the higher the luminance of the corresponding image portion displayed on the TFT liquid crystal placed in the front portion. Thereby, a continuous depth feeling can be produced between the front and rear TFT liquid crystals.

  FIG. 5 is a diagram illustrating an example of the configuration of the display image generation unit 214 in FIG. 2 when the display unit 102 is realized using a parallax division method.

  The display image generation unit 214 includes a background image generation unit 501, a right-eye display item generation unit 502, a left-eye display item generation unit 503, a two-dimensional display item generation unit 504, and a stereoscopic image generation unit 505.

  The background image generation unit 501 reads out image data from the memory 219 via the memory control unit 215, generates image display data, and outputs the image display data to the stereoscopic image generation unit 505.

  The right-eye display item generation unit 502 reads the right-eye bitmap data of the stereoscopic-compatible display item from the memory 219 via the memory control unit 215, generates right-eye display data, and outputs the right-eye display data to the stereoscopic image generation unit 505. .

  The left-eye display item generation unit 503 reads the left-eye bitmap data of the stereoscopic corresponding display item from the memory 219 via the memory control unit 215, generates left-eye display data, and outputs the left-eye display data to the stereoscopic image generation unit 505. .

  Here, the stereoscopic display compatible display item is a display image configured to give a sense of depth when displayed on a liquid crystal display capable of autostereoscopic viewing. For example, operation switches, buttons, dials, and icons are virtually expressed in a space that is a predetermined distance away from the display area of the display unit 102.

  In the digital camera 100 according to the present embodiment, the operator's finger recognizes the movement of the finger in a non-contact state with the touch panel 223, and operates the digital camera 100 by replacing the operation instruction with respect to the stereoscopic display display item. can do.

  The two-dimensional display item generation unit 504 reads the bitmap data for the two-dimensional display item from the memory 219 via the memory control unit 215, generates the two-dimensional display item display data, and outputs it to the stereoscopic image generation unit 505. To do.

  The two-dimensional display item is a display image configured to prevent a sense of depth when displayed on a liquid crystal display capable of autostereoscopic viewing. Or the display image comprised so that a feeling of depth may be clearly reduced compared with the display item corresponding to a stereoscopic vision may be sufficient.

  In the digital camera 100 according to the present embodiment, the operator performs an operation instruction by performing a flick operation or a touch-down operation on the touch panel 223 with respect to the two-dimensional display item displayed on the display unit 102 by the operator. Can be entered.

  The stereoscopic image generation unit 505 generates a stereoscopic image from the image display data, right-eye display data, left-eye display data, and two-dimensional display item display data, and writes the stereoscopic image to the memory 219 via the memory control unit 215.

  Specifically, right-eye display data and 2-dimensional display item display data are superimposed on the image display data to generate a right-eye stereoscopic image. In addition, the left-eye display data and the two-dimensional display item display data are superimposed on the image display data to generate a left-eye stereoscopic image.

  Then, the stereoscopic image generation unit 505 divides the right-eye stereoscopic image and the left-eye stereoscopic image into strip regions in the vertical direction of the screen, and the divided right-eye stereoscopic image and left-eye stereoscopic image are It is generated by alternately arranging in the direction.

  At this time, the same right-eye stereoscopic image and left-eye stereoscopic image for the image data are generated, and in this state, the operator sees the planar image as a planar display.

  The right-eye stereoscopic image and the left-eye stereoscopic image for the two-dimensional display item are generated in the same way, and in this state, it appears as a planar display item to the operator.

  In addition, the right-eye stereoscopic image and the left-eye stereoscopic image with respect to the stereoscopic-compatible display item have parallax, and thus appear to the operator as a stereoscopic display item.

  In the digital camera 100 of the present embodiment, it is possible to independently control whether to display image data, a two-dimensional display item, and a stereoscopic vision display item or not.

  FIG. 6 is a diagram for explaining a method in which the non-contact operation detection unit 217 in FIG. 2 detects the movement of the operator's finger as the operator from a plurality of continuously acquired operator image data. .

  6A illustrates a method of detecting the operator's finger as the operator from a plurality of continuously acquired operator image data and calculating the locus thereof. 6B to 6E show trajectory patterns recorded in the nonvolatile memory 226. FIG.

  In FIG. 6A, the non-contact operation detection unit calculates a correlation value between small pixel blocks on the screen in two consecutive frames of operator image data, and calculates a motion vector of the operator's finger.

  In FIG. 6A, motion vectors calculated at regular time intervals are indicated by V (t), V (t + 1), V (t + 2),..., V (t + 7). The system control unit 221 recognizes a set of continuous vectors as a trajectory of the operator's finger.

  Further, locus patterns 601 to 604 in FIGS. 6B to 6E indicate locus patterns recorded in the nonvolatile memory 226. Each of the trajectory patterns 601 to 604 is associated with an operation instruction for the stereoscopic display item.

  For example, the system control unit 221 determines that the set of vectors V (t) to V (t + 3) matches the locus pattern 601 as a result of matching with the locus patterns 601 to 604. As a result, the system control unit 221 executes an operation instruction associated with the trajectory pattern 601.

  Similarly, the system control unit 221 determines that the set of vectors V (t + 4) to V (t + 6) matches the trajectory pattern 604, and executes an operation instruction associated with the trajectory pattern 604.

  7 is displayed on the display unit 102 in a reproduction mode in which the system control unit 221 in FIG. 2 displays the image data stored in the recording medium 107 on the display unit 102 for the purpose of browsing in accordance with an instruction from the operator. It is the figure which showed the screen content.

  FIG. 7A shows a state where a display 702 indicating the content of displaying image display data generated from image data is displayed on the display unit 102. FIG. 7B shows a state in which the outline 704 of the stereoscopic view display item is displayed in an overlapping manner on the display 702 of the image display data. FIG. 7C shows a state in which the stereoscopic display display item 705 is displayed by being overwritten on the display 702 of the image display data. FIG. 7D shows a state in which a two-dimensional display item for inputting an operation instruction is displayed.

  In FIG. 7A, the operator can input an operation instruction by flicking the finger 703 on the touch panel 223.

  When the operator flicks the display 702, the system control unit 221 determines that the image feed operation instruction is instructed by the operator, and the shooting time is one before the image data currently displayed on the display 702. Or, update to the display of the next image data.

  In FIG. 7B, since only the outline 704 of the display item corresponding to stereoscopic vision is displayed, the area that blocks the display 702 of the image display data can be reduced, and the hindrance to the browsing action of the image data by the operator is reduced. be able to. The contour 704 appears to the operator as a three-dimensional display item.

  In FIG. 7C, the stereoscopic display corresponding item 705 appears to the operator as a stereoscopic display item. The shooting date 708 is a part of the display of the stereoscopic view display item 705 and is a display showing the shooting date of the image data to be displayed. Arrows 706 and 707 are a part of the display of the stereoscopic corresponding display item 705, and are arrows indicating guidance for the operator to operate the stereoscopic corresponding display item 705.

  When the system control unit 221 recognizes that the operator's finger 703 has moved in the direction of the arrow 706 or 707 and determines that the operator's finger 703 matches the trajectory patterns 601 to 604 of FIG. It is determined that the operation instruction is instructed by the operator. Then, the content of the shooting date 708 indicating the shooting date of the image data is changed to the shooting date before or after the currently displayed shooting date, and the display 702 is changed to the following. Update to display the image data corresponding to the shooting date.

  In FIG. 7D, when the operator touches down the two-dimensional display item 709, the system control unit 221 determines that an enlargement instruction is input, and the display 702 is obtained by enlarging a partial area of the image data. . When the operator touches down the two-dimensional display item 710, the system control unit 221 determines that an instruction to delete image data indicated by the display 702 has been input.

  As described above, the system control unit 221 of the digital camera 100 according to the present embodiment controls to change the display form on the display unit 102 according to the distance from the display unit 102 to the finger.

  FIG. 8 is a view of the digital camera 100 viewed from an oblique direction from the display unit 102 side.

  In FIG. 8, how the display is controlled according to the distance from the display unit 102 to the operator's finger 703 will be described.

  The distance between the display unit 102 and the operator's finger 703 is represented by dx. First, when the system control unit 221 determines that dx is less than or equal to a predetermined threshold value d1 (first threshold value), the system control unit 221 displays a stereoscopic view display item.

  At this time, if it is determined that the operator's finger 703 is in close proximity to the stereoscopic display item, non-contact operation instruction input is validated. In other words, the system control unit 221 recognizes an operation instruction to the digital camera 100 based on the locus of movement data of the operator's finger 703 detected by the non-contact operation detection unit 217. At this time, in order to prevent malfunction, control is performed so as to invalidate the input of operation instructions by operating the touch panel.

  Further, when it is determined that dx is equal to or less than d2 (second threshold), which is a threshold smaller than d1, the system control unit 221 deletes the stereoscopic display corresponding item and displays a two-dimensional display item. That is, the display mode of the display item is switched. At this time, the system control unit 221 disables non-contact operation instruction input. In addition, control is performed so as to enable input of operation instructions by operating the touch panel.

  FIG. 9 is a flowchart illustrating a procedure of switching processing executed by the system control unit 221 in FIG. 2 in the first embodiment.

  The processing shown in the flowchart of FIG. 9 is executed by the system control unit 221 based on the program read into the system memory 227. When the operator switches the power switch 104 to power-on in the reproduction mode, the flowchart of FIG. 9 is started.

  In FIG. 9, the image data stored in the recording medium 107 is read out and expanded in the memory 219, and the image display data generated from the image data is displayed on the display unit 102 as shown in FIG. Step S901).

  Further, the operation input imaging unit L and the operation input imaging unit R are activated and controlled, and the operator's finger detected by the distance / position detection unit 216 and the display unit 102 on the basis of the operation element image data captured by these. And the detection operation of the finger position with respect to the area of the display unit 102 are started.

  Next, it is determined whether or not the distance between the operator's finger 703 and the display unit 102 is equal to or less than d1 (step S902). As a result of the determination in step S902, when the finger 703 cannot be detected, or even if it can be detected, the distance is determined to be d1 or less (NO in step S902), step S903 is repeated.

  On the other hand, when the distance is equal to or less than d1 (YES in step S902), as shown in FIG. 7B, the contour 704 of the stereoscopic display display item is displayed in an overlapping manner on the display 702 of the image display data (step S903).

  In the state in which the stereoscopic view display item is displayed with an outline, the operator cannot input an operation instruction to the stereoscopic view display item. However, the operator can understand that the stereoscopic view display item can be operated when the finger is brought closer. In addition, since it is possible to recognize at which position the display item corresponding to the stereoscopic view is arranged, when inputting an operation instruction using the display item corresponding to the stereoscopic view, it is possible to grasp which position the finger 703 should be moved to. .

  Next, it is determined whether the distance between the operator's finger 703 and the display unit 102 is d2 or less, or whether the operator's finger 703 is in contact with the display unit 102 (touch panel 223) (step S904).

  If the result of determination in step S904 is negative (NO in step S904), it is determined whether or not the distance between the finger 703 and the display unit 102 is equal to or greater than d1 (step S906). When it is determined that it is greater than or equal to d1 (YES in step S906), the display of the outline 704 of the stereoscopic view display item is deleted (step S907), and the process returns to step S902.

  On the other hand, as a result of the determination in step S906, when the distance is determined to be less than d1 (NO in step S906), it is determined whether or not the finger 703 is close to the contour 704 of the stereoscopic display-compatible display item (step S908). This can be determined from the relationship between the detection result of the finger position with respect to the area of the display unit 102 in the distance / position detection unit 216 and the display position of the stereoscopic display display item.

  As a result of the determination in step S908, when it is determined that the finger 703 has approached the contour 704 of the stereoscopic view display item (NO in step S908), the process returns to step S904.

  On the other hand, as a result of the determination in step S908, when close to the contour 704 (YES in step S908), as shown in FIG. 7C, the stereoscopic display display item 705 is displayed by overwriting the display 702 of the image display data. (Step S909).

  The stereoscopic view corresponding display item 705 appears to the operator as a stereoscopic display item. At the same time, the shooting date 708 indicating the shooting date of the image data, which is a part of the display of the stereoscopic display item 705, and arrows 706 and 707 for operating the stereoscopic display item 705 are displayed. Is displayed.

  Thereafter, the system control unit 221 validates the input of an operation instruction in a non-contact manner, and starts a finger movement detection operation in the non-contact operation detection unit 217. At this time, in order to prevent malfunction, control is performed so as to invalidate the input of operation instructions by operating the touch panel. By displaying the stereoscopic display item 705 in front of the display 702 of the image display data, a display capable of recognizing that the input of the non-contact operation instruction by operating the stereoscopic display item is valid. Realized.

  In step S909, the stereoscopic corresponding display item 705 is displayed at the same position as the position where the outline 704 of the stereoscopic corresponding display item in FIG. An operation instruction can be input without shifting.

  Next, based on the movement trajectory of the finger 703, an operation instruction input recognition operation is performed on the stereoscopic display item 705, and it is determined whether or not there is an operation on the stereoscopic display item 705 (step S913).

  As a result of the determination in step S913, when an operation is performed on the stereoscopic view display item (YES in step S913), processing according to the operation is executed (step S914), and the process returns to step S913.

  The operation is an operation such as the operator's finger 703 moving along a predetermined locus in the direction of the arrow 706 or 907 as described above. When the system control unit 221 recognizes the operation, it is determined that there has been an image feed operation by designating a date.

  More specifically, in the display state of FIG. 7C, the shooting date 708 is changed to the shooting date before or after the currently displayed shooting date, and the display 702 is changed. The display is updated to display image data corresponding to the next shooting date.

  On the other hand, if there is no operation as a result of the determination in step S913 (NO in step S913), it is determined whether or not the proximity between the finger 703 and the stereoscopic display item 705 has been released (step S915). This can be determined from the relationship between the detection result of the finger position with respect to the area of the display unit 102 in the distance / position detection unit 216 and the display position of the stereoscopic display display item.

  As a result of the determination in step S915, when the proximity is not released (NO in step S915), the process returns to step S913. On the other hand, when the proximity is canceled (YES in step S915), the stereoscopic corresponding display item 705 is deleted. Then, as shown in FIG. 7B, the contour 704 of the stereoscopic view display item is displayed in an overlapping manner on the display 702 of the image display data (step S916), and the process returns to step S904. Further, at this time, the non-contact input is invalidated, and the finger detection operation in the non-contact operation detection unit 217 is stopped. In addition, when the input of the operation instruction by the operation of the touch panel is invalidated in step S909, the control may be performed so as to re-enable it.

  If the result of the determination in step S904 is affirmative (YES in step S904), the display of the outline 704 or the stereoscopic display corresponding item 705 is deleted and the two-dimensional display items 709 and 710 are displayed (step S905). The display is switched to the display of FIG.

  Next, the touch panel operation by the operator is recognized, and it is determined whether or not there is an operation (step S910). As a result of the determination in step S910, when there is a touch panel operation (YES in step S910), an operation for image data or a two-dimensional display item is executed according to the operation of the touch panel. More specifically, when the operator flicks the display 702, the system control unit 221 determines that an operation instruction for image forwarding is instructed by the operator. Then, the display 702 is updated to display the image data whose shooting time is one before or one after the currently displayed image data. Alternatively, when the operator touches down the two-dimensional display item 709 or the two-dimensional display item 710, the system control unit 221 executes enlargement display and deletion operations, respectively.

  On the other hand, if the result of determination in step S910 is that there is no touch panel operation (NO in step S910), it is determined whether the distance between the finger 703 and the display unit 102 is greater than or equal to d1, and if it is less than d1 (NO in step S912), step Return to S910. When it is equal to or greater than d1 (YES in step S912), the process returns to step S902.

  In the processing described above, steps S903 and 909 correspond to first display control means for controlling the display unit 102 so as to three-dimensionally display display items that can be operated without the finger 703 touching the display unit 102. Step S905 corresponds to a second display control unit that controls the display unit 102 so as to two-dimensionally display a display item that can be operated when the finger 703 contacts the display unit 102. In steps S903 and 909, the display unit 102 is controlled to display the display item three-dimensionally by generating a parallax image of the display item, as described with reference to FIG.

  As described above, according to the present embodiment, the operator can operate the stereoscopic display display item by recognizing the movement of the finger without bringing the finger into contact with the touch panel 223. In addition, since the 2D display item can be operated by touching a touch panel integrated with the display unit 102, an intuitive operation method can be realized.

  In addition, since the operation using the stereoscopic view display item and the operation using the two-dimensional display item are switched according to the position of the operator's finger, an operation method easy to understand for the operator can be realized.

  Further, when the operator's finger as an operator approaches the distance d1 or less from the display unit 102, the outline of the stereoscopic display display item is displayed overlapping the display of the image display data. Accordingly, the operator can understand that the stereoscopic display display item can be operated when the finger is brought closer, and can recognize where the stereoscopic display display item is arranged. As a result, when an operation instruction is input using the stereoscopic view compatible display item, it is possible to grasp to which position the finger should be moved.

  In addition, when the finger is close to the stereoscopic display display item, the stereoscopic display display item is displayed in front of the display of the image display data, so that the operation instruction input to the stereoscopic display item is valid. Can be displayed in a recognizable manner.

  Further, when the operator's finger as an operator is further approached and approaches the distance d2 or less from the display unit 102, the stereoscopic display display item is deleted, so that the operation on the 2D display item is displayed. It becomes easy to operate on image data.

  According to the processing of FIG. 9 described above, the display unit 102 is controlled so as to display the display items that can be operated without the finger 703 touching the display unit 102 (steps S903 and 909). Then, the display unit 102 is controlled so that a display item that can be operated by touching the display unit 102 with the finger 703 is two-dimensionally displayed. Next, the operation content operated without the finger 703 touching the display unit 102 is recognized (step S910). Further, the operation content operated by the finger 703 touching the display unit 102 is recognized. As a result, the difference between the operation on the display item for operation displayed in three dimensions and the operation on the display item for operation displayed in two dimensions becomes clear, so that the operability can be improved.

  Furthermore, according to the processing of FIG. 9, when the distance between the finger 703 and the display unit 102 is derived to be d1 or less (first threshold value or less) determined in advance, the display unit is displayed in a three-dimensional manner. 102 is controlled. Further, when the distance is derived to be d2 or less (second threshold or less) smaller than d1, the display unit 102 is controlled so as to delete the display item displayed in three dimensions. Thereby, the operativity of operation with respect to the display item for operation displayed in three dimensions and operation with respect to the display item for operation displayed in two dimensions can be improved.

  In the description of the present embodiment, the operator's finger, which is an operator, is detected from the operator image data captured by the two imaging units of the operation input imaging unit L and the operation input imaging unit R. The distance from the display unit 102 to the finger and the position of the finger with respect to the display screen area of the display unit 102 are detected. However, the operator's finger is detected from the operator image data captured by one of the operation input imaging units, and the distance from the display unit 102 to the finger is calculated based on the information on the size of the finger. You may make it do. In this case, a single operation input imaging unit can be provided.

  In the description of the present embodiment, the distance / position detection unit 216 detects the operator's finger that is an operation unit from the operation unit image data captured by the operation input imaging unit L and the operation input imaging unit R. doing. Then, the distance from the display unit 102 to the finger and the position of the finger with respect to the display screen area of the display unit 102 are detected.

  However, a touch panel device as described in Japanese Patent Application Laid-Open No. 2009-258903 may be used to perform signal detection by performing multi-electrode coupling. Thus, it is possible to detect that the operator's finger is close to the display unit 102 and to detect the position of the finger with respect to the area of the display screen of the display unit 102 in the close state.

  It is also possible to detect the movement of the finger by continuously detecting the position of the finger, and to recognize an operation instruction for the stereoscopic display corresponding item based on the locus. In this case, the operation input imaging unit L and the operation input imaging unit R can be omitted.

  In the flowchart of FIG. 9, the two-dimensional display items 709 and 710 are displayed at a timing when the operator's finger 703 approaches the distance d2 or less from the display unit 102. However, the image display data may be displayed at the same time as it is displayed on the display unit 102 in step S901. In this case, even if the operator does not move the finger, the operator can recognize an operation that can be performed by the touch panel operation.

  In the flowchart of FIG. 9, in step S904, the system control unit 221 determines whether the distance between the operator's finger 703 and the display unit 102 is close to d2 or less, or whether the operator's finger 703 has touched the touch panel 223. Is determined. When the determination is affirmative, the process proceeds to step S905, where the system control unit 221 erases the display of the outline 704 of the stereoscopic corresponding display item or the stereoscopic corresponding display item 705. However, the stereoscopic display display item may be displayed in a reduced size so that it can be identified that the operation on the two-dimensional display item or the image data is preferential without being deleted. In this case, the operator can recognize the contents that can be operated in a non-contact state even when the operation is possible by touching the touch panel.

[Second Embodiment]
In the second embodiment, a case will be described in which the display unit 102 is configured by a liquid crystal display that cannot perform autostereoscopic viewing and can only perform two-dimensional display. In the following description, only the configuration different from the first embodiment will be described.

  FIG. 10 is a diagram illustrating an example of the configuration of the display image generation unit 214 in FIG. 2 when the display unit 102 is realized by a liquid crystal display capable of only two-dimensional display.

  In FIG. 10, the display image generation unit 214 includes a background image generation unit 1001, a stereoscopic display item generation unit 1002, a 2D display item generation unit 1003, and a display image synthesis unit 1004.

  The background image generation unit 1001 reads image data from the memory 219 via the memory control unit 215, generates image display data, and outputs the image display data to the display image synthesis unit 1004.

  The stereoscopic display item generation unit 1002 reads out stereoscopic display item bitmap data from the memory 219 via the memory control unit 215, generates stereoscopic display item display data, and outputs the generated display data to the display image synthesis unit 1004.

  Here, the stereoscopic display item is a display image expressed on a two-dimensional liquid crystal display so that the operator recognizes that it is a three-dimensional object. For example, a switch, button, dial, or icon for operation is displayed in three dimensions. Express with display items.

  FIG. 11 is a diagram illustrating a method of expressing a stereoscopic display item, and any of (A) to (C) is a diagram illustrating a method of expressing a rectangular parallelepiped.

  FIG. 11A shows a rectangular parallelepiped 1101. By displaying only lines observable from one direction among the ridge lines of the rectangular parallelepiped 1101, the operator can recognize the solid object.

  FIG. 11B represents a method of adding a shadow to the surface in accordance with the direction of the surface of the cuboid 1102 and the light irradiation direction, assuming that the cuboid 1102 is observed under a certain light source. . Since the realism is higher than the expression of only the ridge line, the operator can recognize the object as a three-dimensional object more than in the case of FIG.

  FIG. 11C shows a method of adding a shadow 1104 generated in a portion where the direct light from the light source does not hit the rectangular parallelepiped 1103 in addition to the method of applying a shadow to the three-dimensional object of FIG. Since the realism further increases, the operator can recognize the object as a three-dimensional object more than in the case of FIG.

  In the digital camera 100 of the present embodiment, an operation instruction can be input by recognizing the movement of the operator's finger and operating this stereoscopic display item.

  Returning to the description of FIG. 10, the two-dimensional display item generation unit 1003 reads the bitmap data for the two-dimensional display item from the memory 219 via the memory control unit 215, generates the two-dimensional display item display data, and displays the display data. The image is output to the image composition unit 1004.

  The two-dimensional display item is a display image configured to prevent a sense of depth or a three-dimensional effect when displayed on a two-dimensional display liquid crystal display. Or the display image comprised so that a feeling of depth and a three-dimensional effect may become apparently few compared with a three-dimensional display item may be sufficient.

  In the digital camera 100 of this embodiment, the operator performs an operation instruction by performing a flick operation or a touchdown operation on the touch panel 223 with respect to the two-dimensional display item displayed on the display unit 102 with the finger 703. Can be entered.

  The display image composition unit 1004 superimposes the display data for 3D display items and the display data for 2D display items on the image display data, and writes them to the memory 219 via the memory control unit 215. At this time, whether or not to display the image data, the two-dimensional display item, and the stereoscopic display item can be controlled independently.

  FIG. 12 is a flowchart illustrating a procedure of switching processing executed by the system control unit 221 in FIG. 2 in the second embodiment.

  Note that the flowchart of FIG. 12 is the same processing except that the stereoscopic display item is displayed instead of displaying the stereoscopic display item in the flowchart shown in FIG. .

  In FIG. 12, steps S1201 and S1202 are the same as steps S901 and S902 of FIG.

  In FIG. 12, it is determined whether or not the distance between the operator's finger 703 and the display unit 102 is equal to or less than d1 (step S1202). As a result of the determination in step S1202, if the finger 703 cannot be detected or if the distance is determined to be d1 or less even if it can be detected (NO in step S1202), step S1203 is repeated.

  On the other hand, when the distance is equal to or less than d1 (YES in step S1202), as shown in FIG. 13A, the outline 1301 of the stereoscopic display item is displayed in an overlapping manner on the display 702 of the image display data (step S1203).

  Steps S1204 to S1208 in FIG. 12 are the same as steps S904 to S908 in FIG.

  As a result of the determination in step S1208, when the finger 703 approaches the contour 1301 (YES in step S1208), the shadow 1303 of the stereoscopic display item is overwritten on the display 702 of the image display data as shown in FIG. It is displayed (step S1209). In step S1208, the distance / position detection unit 216 can determine the relationship between the detection result of the finger position with respect to the area of the display unit 102 and the display position of the contour.

  Steps S1210 to S1216 in FIG. 12 are the same as steps S910 to S916 in FIG.

  As described above, even when the display unit 102 is configured with a liquid crystal display that cannot perform autostereoscopic viewing and can only perform two-dimensional display, the operator can operate the stereoscopic display item on the display unit 102 without contact. Further, since the two-dimensional display item is operated by touching a touch panel integrated with the display unit 102, an intuitive operation method can be realized. An image processing apparatus that can be operated using both a stereoscopic display item and a two-dimensional display item can be realized.

  In addition, since the operation using the stereoscopic display item and the operation using the two-dimensional display item are switched according to the position of the operator's finger, an operation method that is easy to understand for the operator can be realized.

  As described above, according to the present embodiment, the display item for operation displayed in 3D is operated without contact with the display unit, and the display item for operation displayed in 2D is operated with contact with the display unit. Therefore, the operation can be performed intuitively and the operability is improved.

  In addition, an operation method easy to understand for the operator can be realized because the operation for the display item displayed in three dimensions and the operation for the display item displayed in two dimensions are switched according to the position of the operator.

  Further, when the operator's finger as an operator approaches the screen, the display of the display item displayed in a three-dimensional manner is suppressed, and the display is overlapped with the display of the image display data. Thereby, the operator can understand that the display item displayed in three dimensions becomes operable when the finger is brought closer. In addition, since it is possible to recognize at which position the display item of stereoscopic display is arranged, when inputting an operation instruction using the display item displayed stereoscopically, it is possible to grasp to which position the finger should be moved. .

  In addition, when the finger is in close proximity to the display item displayed in 3D, the display item displayed in 3D is displayed in front of the image display data. Can be displayed in a recognizable manner.

  Further, when the operator's finger as an operator is further approached, the display item displayed in 3D is erased, so that the operation on the two-dimensional display item and the operation on the displayed image data are facilitated.

  Note that the processing of the above-described embodiment may provide a system or apparatus with a storage medium that records software program codes that embody each function. The functions of the above-described embodiments can be realized by the computer (or CPU or MPU) of the system or apparatus reading out and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. As a storage medium for supplying such a program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, or the like can be used. Alternatively, a CD-ROM, CD-R, magnetic tape, nonvolatile memory card, ROM, or the like can be used.

  The functions of the above-described embodiments are not only realized by executing the program code read by the computer. Including the case where the OS (operating system) running on the computer performs part or all of the actual processing based on the instruction of the program code, and the functions of the above-described embodiments are realized by the processing. It is.

  Further, the program code read from the storage medium may be written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. After that, the CPU of the function expansion board or function expansion unit performs part or all of the actual processing based on the instruction of the program code, and the functions of the above-described embodiments are realized by the processing. Is also included.

100 Digital camera 102 Display unit 213 Image processing unit 214 Display image generation unit 215 Memory control unit 216 Distance / position detection unit 217 Non-contact operation detection unit 221 System control unit 222 Touch panel control unit 223 Touch panel 501, 1001 Background image generation unit 502 Right eye Display item generation unit 503 Right-eye display item generation unit 504 Two-dimensional display item generation unit 505 Stereoscopic image generation unit 703 Finger 704 Outline 705 Stereoscopic display item 706, 707 Arrows 709, 710 Two-dimensional display item 1002 Stereoscopic display item Generation unit 1003 Two-dimensional display item generation unit 1004 Display image composition unit L, R Operation input imaging unit

Claims (5)

An image processing apparatus,
Display control means for controlling an image to be displayed in a plane on the display unit of the touch panel, and to display a three-dimensional display item for inputting an operation on the image in a three-dimensional manner on the display unit of the touch panel .
A calculating means for calculating a distance between an operator for operating the image and the touch panel;
When the distance calculated by the calculating means is larger than a second threshold value, control is performed so as to invalidate the input of the operator's instruction to the image , and the distance calculated by the calculating means is greater than the second threshold value. If small, comprising input control means for controlling to enable the input of instructions of the operator for the image ,
The display control means displays the image on the display unit of the touch panel regardless of the distance calculated by the calculation means, and when the distance calculated by the calculation means is smaller than a first threshold, the 3 Control to display a three-dimensional display item, and control to not display the three-dimensional display item when the distance calculated by the calculation means is smaller than the second threshold,
The input control means, when the distance calculated by the calculation means is smaller than the first threshold, to control to enable the input of the instruction of the operator for the three-dimensional display item,
The image processing apparatus according to claim 1, wherein the first threshold value is larger than the second threshold value. The three-dimensional display items or the image processing different from each associated, if it is determined that an operation for the three-dimensional display item or the image, corresponding to the three-dimensional display items or the image the engineered The image processing apparatus according to claim 1, further comprising processing means for executing the processing. Wherein the display control unit by generating a disparity image of the three-dimensional display items, according to claim 1, wherein the controller controls the three-dimensional display items to stereoscopic display on a display unit of the touch panel Image processing apparatus. A control method for an image processing apparatus, comprising:
Image plan view of the display unit of the touch panel, and a display control step of controlling to so that showing the stereoscopic Table 3D shows the item for inputting an operation on the image on the display unit of the touch panel superimposed on the image,
A calculation step of calculating a distance between an operator for operating the image and the touch panel;
When the distance calculated by the calculation step is larger than a second threshold value, control is performed so as to invalidate the input of the operator's instruction to the image , and the distance calculated by the calculation step is greater than the second threshold value. If it is smaller, the input control step for controlling to enable the input of instructions of the operator for the image ,
In the display control step, regardless of the distance calculated in the calculation step, the image is displayed on the display unit of the touch panel, and when the distance calculated in the calculation step is smaller than a first threshold, the 3 Control to display a three-dimensional display item, and control to not display the three-dimensional display item when the distance calculated in the calculation step is smaller than the second threshold,
In the input control step, when the distance calculated in the calculation step is smaller than the first threshold, control is performed so as to enable the input of the instruction of the operator for the three-dimensional display item,
The method of controlling an image processing apparatus, wherein the first threshold is larger than the second threshold. The program which makes a computer function as each means of the image processing apparatus of any one of Claims 1 thru | or 3.