JP2016071558A - Display control device, control method, control program, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To achieve a touch interface easy to use for a user.SOLUTION: A display control device includes: an operation prediction section (41) for detecting movement of a body of a user who moves the body in order to contact to a touch panel (12); and a display control part (42) for displaying an operation image on a display part (13) when the movement of the body is detected by the operation prediction section (41).SELECTED DRAWING: Figure 1

Description

  The present invention relates to a display control device that displays a touch-operable image on a display device including a display unit and a touch panel formed integrally with the display unit.

  In recent years, a display device such as a television (having a touch interface) that can be operated by a touch operation has become widespread. The touch interface is an intuitive user interface, and when using a display device such as a television in a place other than home, one of the reasons for the widespread use is that it is easy to operate without knowledge of products. Examples of using the display device in places other than home include use in a hotel room, use in a hospital (such as a hospital room or waiting room), and use of a television set in a running machine or the like in a sports gym. .

  In a display device equipped with a touch interface, an image for selecting a channel or selecting a menu is displayed by a touch operation, and further operations are performed on the displayed image, thereby changing the channel or setting for a television. The process to change can be performed. However, two touch operations are required until such processing is executed, and it is difficult to say that a highly convenient touch interface is realized.

  As a solution to this problem, techniques described in Patent Documents 1 to 3 have been proposed. Patent Documents 1 and 2 describe techniques for controlling image display without a touch operation by a gesture. Patent Document 3 describes a technique for displaying an image by detecting that a finger is approaching.

  Various techniques have been proposed to realize a touch interface that is easy for the user to use. For example, Patent Document 4 describes a technique for moving an image displayed at the position of a finger approaching. Japanese Patent Application Laid-Open No. 2004-228688 describes a technique for registering user hand information in advance and displaying an appropriate image for each of the right hand and the left hand.

JP 2011-1000039 A (published May 19, 2011) JP2013-016060A (published January 24, 2013) JP 2002-358162 A (released on December 13, 2002) JP 2014-102851 A (released June 5, 2014) JP 2011-150672 A (released on August 4, 2011)

  However, the techniques disclosed in Patent Documents 1 and 2 have a problem in that gestures to be performed must be remembered in advance and are not suitable for temporarily using a television in a place other than home. In addition, there are users who feel embarrassed to perform gestures in the presence of people around them.

  Further, in the techniques of Patent Documents 3 and 4, since the image is displayed near the finger by detecting that the finger is close to the screen, the image is not displayed until just before the user touches the screen. Therefore, there is a problem that it is inconvenient for the user because it is necessary to confirm the position touched immediately before the touch operation (for example, the position of the button that the user wants to touch in the image).

  As another solution, for example, an image can be displayed by voice input, but there are also users who feel embarrassed to perform voice input in the state where there are people around like a gesture.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a display control device or the like that realizes a touch interface that is easy for the user to use.

  In order to solve the above problems, a display control device according to one embodiment of the present invention can be operated by a touch operation on the touch panel on a display device including a display unit and a touch panel formed integrally with the display unit. A display control device that displays an operation image on the display unit, the detection unit detecting a body movement of a user who moves the body to touch the touch panel, and the body movement detected by the detection unit A display control unit that displays the operation image on the display unit.

  In order to solve the above problems, a method for controlling a display control device according to one embodiment of the present invention includes a display device including a display unit and a touch panel formed integrally with the display unit. A control method of a display control device for displaying an operation image operable by an operation on the display unit, wherein a detection step of detecting a movement of a body of a user who moves the body to touch the touch panel, and a detection step And displaying the operation image on the display unit when the movement of the body is detected.

  According to one aspect of the present invention, there is an effect of realizing a touch interface that is easy for a user to use.

It is a block diagram which shows an example of the principal part structure of the television which concerns on Embodiment 1 of this invention. It is a figure which shows an example of the hardware constitutions of the television shown in FIG. It is a figure which shows an example of the picked-up image image | photographed with the television shown in FIG. It is a figure which shows an example of the picked-up image image | photographed with the television shown in FIG. FIG. 2 is a screen diagram showing an example of an operation panel and menu images in the television shown in FIG. 1. FIG. 3 is a screen diagram illustrating a display example of an operation panel in the television illustrated in FIG. 1. 3 is a flowchart illustrating an example of a flow of image display processing in the television illustrated in FIG. 1. It is a block diagram which shows an example of the principal part structure of the television which concerns on Embodiment 2 of this invention. 9 is a flowchart illustrating an example of a flow of image display processing in the television illustrated in FIG. 8. It is a block diagram which shows an example of the principal part structure of the television which concerns on Embodiment 3 of this invention. (A) is a schematic diagram which shows the calculation method of the shortest distance of the television shown in FIG. 10, and a user, (b) is a schematic diagram which shows the method of specifying the area | region which a pointer (finger) touches. . 11 is a flowchart illustrating an example of a flow of image display processing in the television illustrated in FIG. 10.

  Hereinafter, embodiments of the present invention will be described in detail. In the embodiments described below, a television (display device) integrated with the display control device of the present invention will be described as an example. The display control device and the display device may be separate. The display control device may be any device that is connected to the display device in a wired or wireless manner so as to be able to display an image on the display device. For example, a tuner separate from the TV, a recording device such as an HDD recorder, The present invention can also be applied to game machines, personal computers connected to displays, and the like.

  Note that dimensional relationships such as length, size, and width and shapes in the drawings are appropriately changed for clarity and simplification of the drawings, and may differ from actual dimensions and shapes.

Embodiment 1
An embodiment of the present invention will be described below with reference to FIGS.

(Hardware configuration of TV 10)
First, the hardware configuration of the television 10 (display device) will be described with reference to FIG. FIG. 2 is a diagram illustrating an example of a hardware configuration of the television 10. As shown in FIG. 2, the television 10 includes an operation acquisition unit 11, a touch panel 12, a display unit 13, a control unit 14, a tuner 21, a stream separation unit 22, a graphics processing unit 23, a video decoding unit 24, an audio decoding unit 25, A scaling unit 26, a scaling unit 27, a mixing unit 28, a blend unit 29, a DAC 31, and a speaker 32 are provided.

  The operation acquisition unit 11 is a device that acquires operation information related to a user's operation. Details of the operation acquisition unit 11 will be described later.

  The touch panel 12 is for a user to input an instruction to the television 10. The touch panel 12 displays a touch surface that receives contact (approach) of an indicator (such as a finger or a pen), contact / non-contact (approach / non-approach) between the indicator and the touch surface, and a contact (approach) position thereof. It is comprised with the touch sensor for detecting. The touch sensor may be realized by any sensor as long as it can detect contact / non-contact (approach / non-approach) between the indicator and the touch surface. The touch sensor is realized by, for example, a pressure sensor, a capacitance sensor, an optical sensor, or the like. The touch sensor may be a camera that captures an indicator approaching the touch surface. The touch panel 12 outputs information (hereinafter referred to as touch information) on the position (coordinates) at which the detected indicator has contacted (approached) to the processing instruction unit 43 described later.

  The display unit 13 is a display device that displays information processed by the television 10 as an image or video in a display area. For example, the display unit 13 displays a video corresponding to a television broadcast signal, an operation panel (operation image) generated by the control unit 14, and the like. For example, the display unit 13 is realized by a display device such as an LCD (liquid crystal display).

  In the present embodiment, the touch panel 12 and the display unit 13 are integrally formed.

  The control unit 14 performs overall control of each unit included in the television 10. Details of the control unit 14 will be described later.

  The tuner 21 selects a channel and extracts a stream of a channel designated in advance from a digital broadcast wave. The stream extracted by the tuner 21 may include BML (Broadcast Markup Language) information for displaying data broadcasting.

  The stream separation unit 22 separates (DEMUX) the transport stream (TS) extracted from the tuner 21 into a video signal, an audio signal, and a data signal (caption information, BML information, etc.).

  The graphics processing unit 23 converts the separated data signal and an image signal such as an operation panel output from the control unit 14 into graphics. The graphic generated by the graphics processing unit 23 is output to the scaling unit 26.

  The video decoding unit 24 decodes the separated video signal. The video signal decoded by the video decoding unit 24 is output to the scaling unit 27.

  The audio decoding unit 25 decodes the separated audio signal. The audio signal decoded by the audio decoding unit 25 is output to the mixing unit 28.

  The scaling unit 26 adjusts the size of the graphic generated by the graphics processing unit 23. At this time, the scaling unit 26 adjusts the size of the graphic (operation panel or the like) according to the scaling information supplied from the graphics processing unit 23. The graphic image signal whose size has been adjusted is output to the blend unit 29.

  The scaling unit 27 adjusts the size of the video signal output from the video decoding unit 24. The scaling unit 27 refers to information indicating the size of the display area of the display unit 13 acquired from the blend unit 29 and information indicating the size of the graphic (such as an operation panel) whose size is adjusted by the scaling unit 26. Adjust the size of the video signal. The video signal whose size has been adjusted is output to the blend unit 29.

  The mixing unit 28 outputs the audio signal decoded by the audio decoding unit 25.

  In accordance with an instruction from the control unit 14, the blend unit 29 arranges the video signal and the graphic and performs synthesis. Then, the synthesized video signal and graphic are displayed on the display unit 13.

  A DAC (Digital-Analog Converter) 31 changes a digital signal to an analog signal for an audio signal.

  The speaker 32 outputs the analog signal converted by the DAC 31 as sound.

(Main components of the television 10)
Next, with reference to FIG. 1, the main part structure of the television 10 is demonstrated. FIG. 1 is a block diagram illustrating an example of a main configuration of the television 10.

  As shown in FIG. 1, the television 10 includes an operation acquisition unit 11, a touch panel 12, a display unit 13, a control unit 14, and a storage unit 15. Note that the details of the touch panel 12 and the display unit 13 have already been described, and a description thereof will be omitted here.

  In addition, as described above, the television 10 is configured to be integrated with the display control device 1. A portion surrounded by a broken line shown in FIG.

  The operation acquisition unit 11 includes a camera 111 (imaging unit). The camera 111 is a photographing device that continuously photographs a user who is using the television 10 (viewing a program displayed on the television 10). The camera 111 should just be provided in the position which can image | photograph the user using the television 10. FIG. The motion acquisition unit 11 outputs video captured by the camera 111 to the operation prediction unit 41 (detection unit) described later as motion information. Specifically, the camera 111 continuously captures video as long as the television 10 is activated, and the operation acquisition unit 11 continues to output the captured video to the operation prediction unit 41. In the present embodiment, the camera 111 is described as capturing video (moving images). However, the camera 111 continuously captures images (still images) every predetermined time (for example, 1 second). There may be. Hereinafter, an image captured by the camera 111 is referred to as a captured image.

  The control unit 14 includes an operation prediction unit 41, a display control unit 42, and a processing instruction unit 43.

  The processing instruction unit 43 instructs each unit of the control unit 14 to perform processing according to touch information. Specifically, when the process instruction unit 43 acquires touch information from the touch panel 12, the process instruction unit 43 specifies a process corresponding to the touch operation. Then, the process instruction unit 43 instructs each unit of the control unit 14 to execute the specified process. For example, the processing instruction unit 43 acquires touch information indicating a touch operation on a button on the operation panel, and instructs the display control unit 42 to display an image indicated by the touched button.

  The operation prediction unit 41 determines whether or not the user is about to perform a touch operation on the touch panel 12 according to operation information (specifically, a captured video). The operation prediction unit 41 includes a face area determination unit 411 and a left and right area determination unit 412. The operation prediction unit 41 supplies the captured image supplied from the operation acquisition unit 11 to the face area determination unit 411.

  The face area determination unit 411 identifies a face area that is an area where the user's face is displayed from the captured video, and performs a determination regarding the identified face area. Here, the detail of the process which the face area | region determination part 411 performs is demonstrated with reference to FIG. FIG. 3 is a diagram illustrating an example of a captured image captured by the camera 111. In the following description, captured images may be collectively referred to as “captured images 141”. The captured images 141a to 141e show a part of the captured images as images.

  When the face area determination unit 411 acquires the captured image 141, the face area determination unit 411 first specifies the face area. Since the existing technology can be used to specify the face area, the description thereof is omitted here. Next, the face area determination unit 411 determines whether or not the specified face area 143 occupies a range equal to or greater than a face area threshold 511 described later in the center area 142 set in advance in the acquired captured video 141. Here, in FIG. 3A, it is assumed that the face area 143 is equal to or greater than the face area threshold 511 in the captured video 141a. As determined in the example of FIG. 3A, when it is determined that the range of the face area threshold 511 or more is occupied, the face area determination unit 411 supplies the determination result to the display control unit 42. Note that the center area 142 in this embodiment is an area specified by drawing two boundary lines on the captured image 141, as shown in FIG. Specifically, the center area 142 is an area where the user's face is assumed to be reflected, and the two boundary lines are drawn so as to specify the area. However, the center region 142 is not limited to this example. Further, the ratio of the face area 143 in the captured video 141 may be determined.

  On the other hand, when it is determined that the range does not occupy the face area threshold value 511 or more, the face area determination unit 411 specifies a change in the captured video 141 and performs a process of determining the specified change. Specifically, it is determined whether or not the amount of change in the face area 143 (specifically, the enlargement ratio of the face area) is greater than or equal to a face change threshold 512, which will be described later, in the center area 142 set in advance in the captured image 141. To do. Here, in (b) of FIG. 3, when a user who uses the television 10 (views a program) in a position not close to the television 10 approaches the television 10, the face in the captured image 141 c is displayed. The area 143 is larger than the face area 143 of the captured video 141b. If the face area determination unit 411 determines that this change is greater than or equal to the face change threshold 512, the face region determination unit 411 supplies the determination result to the display control unit 42.

  On the other hand, when it is determined that the face change threshold 512 is not exceeded, the position information of the face area 143 is supplied to the left and right area determination unit 412.

  Note that the face area determination unit 411 determines whether or not the face area 143 occupies a range of the face area threshold value 511 or more in the center area 142 of the photographed video 141, and the face area 143 in the center area 142 of the photographed video 141. The configuration may be such that only one of whether or not the change amount (magnification rate of the face region 143) is equal to or greater than the face change threshold 512 may be determined.

  Further, the face area determination unit 411 may determine the distance between the television 10 and the user using the captured video acquired from the motion acquisition unit 11 before the above-described determination. Specifically, when the face area determination unit 411 determines that the size of the specified face area 143 is less than a predetermined threshold, the face area determination unit 411 determines that the television 10 and the user are sufficiently separated. When it is determined that the face is sufficiently separated, the face area determination unit 411 ends the process. In other words, the face area determination unit 411 determines whether to display the operation panel 132 only when the distance between the television 10 and the user is close to some extent.

  The left and right region determination unit 412 performs determination regarding the right region and the left region. Here, the detail of the process which the left-right area determination part 412 performs is demonstrated with reference to FIG. FIG. 4 is a diagram illustrating an example of a captured image captured by the camera 111.

  As illustrated in FIG. 4, the left and right area determination unit 412 divides the captured image 141 into a left area 144 and a right area 145 according to the position information of the face area 143 supplied from the face area determination unit 411. Specifically, the captured image 141 is divided into a left region 144 and a right region 145 by drawing a boundary line at the center of the face region 143. And the process which identifies the change of the picked-up image 141 and determines the specified change is performed. Specifically, it is determined whether or not a predetermined change has occurred in the left region 144 or the right region 145 of the captured video 141. Note that the division of the captured video 141 is not limited to the example in which the boundary line is drawn at the center of the face region 143 as described above. For example, the user's right arm and the upper arm of the left arm are identified, and the region including the upper arm of the right (left) arm is first identified as the left region 144 (right region 145), and the left region 144 (right side) in the captured image 141 is identified. The region other than the region 145) may be the right region 145 (left region 144).

  Here, the example of the predetermined change described above will be described in detail. The predetermined change is not limited to the one described below, and may be a change that occurs when the user tries to perform a touch operation on the touch panel 12 in the left region 144 or the right region 145.

  For example, the left / right region determination unit 412 may determine whether the rate of change in the left region 144 or the right region 145 is equal to or greater than a left / right change threshold value 513 described below.

  Further, when the left or right region determination unit 412 recognizes the left hand or the right hand in the left region 144 or the right region 145, the left / right region determination unit 412 determines whether the recognized rate of change of the left hand or the right hand is equal to or greater than a threshold value (not shown). May be.

  Further, the left / right area determination unit 412 may determine whether or not a rod-like object (for example, an index finger or a pen) has appeared in the left area 144 or the right area 145.

  In addition, the left and right area determination unit 412 may determine whether or not the forearm (the part of the human arm from the base of the hand to the elbow) has appeared in the left area 144 or the right area 145.

  Further, the left / right region determination unit 412 may determine whether a moving object has appeared from the left end of the left region 144 or the right end of the right region 145.

  Note that the left and right region determination unit 412 may execute all of the five determinations described above, or may execute at least one of them. Hereinafter, the left and right area determination unit 412 according to the present embodiment will be described as a configuration that determines only whether the rate of change in the left area 144 or the right area 145 is equal to or greater than a left and right change threshold value 513 described later. Here, the “change rate” is obtained by quantifying the difference between an image obtained by cutting a certain moment in the captured image 141 and an image obtained by cutting a moment after a predetermined time (for example, one second). In addition, since the existing technique can be used for the above-described detection of the rod-like object, the forearm, and the moving object appearing from the left end of the left region 144 or the right end of the right region 145, detailed description thereof is omitted here. .

  Here, in FIG. 4A, in the captured image 141 d, a user using the television 10 is performing an operation of extending the right hand forward to touch the touch panel 12 at a position not close to the television 10. . As a result, the left region 144 in the photographed video 141d is greatly changed from the left region 144 in the photographed video 141b. When it is determined that the change rate in this change is equal to or greater than the left / right change threshold 513, the left / right area determination unit 412 supplies the determination result to the display control unit 42.

  4B, in the captured image 141e, the user who uses the television 10 at a position not close to the television 10 performs an operation of extending the left hand to touch the touch panel 12. . As a result, the right area 145 in the photographed video 141e is greatly changed from the right area 145 in the photographed video 141b. When it is determined that the change rate in this change is equal to or greater than the left / right change threshold 513, the left / right area determination unit 412 supplies the determination result to the display control unit 42.

  On the other hand, when it is determined that the rate of change does not exceed the left / right change threshold value 513, the left / right area determination unit 412 supplies information indicating that there is no change in the captured image 141 to the face area determination unit 411. When the face area determination unit 411 is supplied with the information, the face area determination unit 411 performs the above-described process again on the captured video 141.

  The display control unit 42 displays an image (video) on the display unit 13. The display control unit 42 includes an image generation unit 421 and an image position determination unit 422. When the display control unit 42 is supplied with a determination result from the operation prediction unit 41 (specifically, the face region determination unit 411 or the left and right region determination unit 412), the display control unit 42 outputs the determination result to the image generation unit 421 and the image position determination unit 422. To supply.

  The image generation unit 421 generates an image to be displayed on the display unit 13. Specifically, when the determination result is supplied, the image generation unit 421 controls the graphics processing unit 23 to generate an operation panel (operation image) that can be operated by a user's touch operation. Specifically, when the image generation unit 421 is supplied with the determination result, the image generation unit 421 reads out image data 514 (to be described later) according to the determination result, and displays the operation panel indicated by the read image data 514 in the graphics processing unit 23. Generate.

  More specifically, when the image generation unit 421 is supplied with a determination result that the rate of change in the left region 144 or the right region 145 of the captured video 141 is equal to or greater than the left / right change threshold 513, the user is using his / her hand (fingertip). Image data 514 for displaying an operation panel for performing a touch operation is read, and the operation panel indicated by the image data 514 is generated by the graphics processing unit 23.

  Details of the operation panel generated by the image generation unit 421 will be described with reference to FIG. FIG. 5 is a screen diagram showing an example of the operation panel and menu image on the television 10.

  When the determination result that the rate of change in the left region 144 or the right region 145 of the captured image 141 is greater than or equal to the left / right change threshold 513 is supplied, the image generation unit 421 sends the graphics processing unit 23 the graphics processing unit 23 according to the read image data 514. An operation panel 132a shown in FIG. 5A is generated.

  In addition, the image generation unit 421 determines that the face area 143 occupies a range equal to or greater than the face area threshold 511 in the center area 142 or the amount of change in the face area 143 in the center area 142 of the captured image 141 is greater than or equal to the face change threshold 512. When the determination result is supplied, image data 514 for displaying an operation panel for performing a touch operation with a pen or the like held by the user is read, and the operation panel indicated by the image data 514 is displayed as graphics. The data is generated by the processing unit 23.

  Specifically, the determination result that the face area 143 occupies the range of the face area threshold 511 or more in the center area 142, or the change amount of the face area 143 in the center area 142 of the photographed video 141 is the face change threshold 512 or more. When the determination result is supplied, the image generation unit 421 causes the graphics processing unit 23 to generate the operation panel 132b shown in FIG. 5C according to the read image data 514.

  In the operation panel 132b, each button (a touch interface simulating buttons included in the operation panel 132b) is larger than the operation panel 132a shown in FIG. That is, the operation panel 132b is designed to make it easy to touch each button when the user performs a touch operation with a pen or the like held in the mouth.

  The situation “a user performs a touch operation with a pen or the like held in his / her mouth” is a situation that may occur when a user who cannot use both hands in a hospital or the like operates the television 10. When operating with a pen or the like in the mouth, there is a problem that the operation is more difficult than when operating with a hand. In such a case, the operation panel 132 is designed like the operation panel 132b shown in FIG. 5C, thereby realizing a touch interface that is easy for the user to operate.

  On the operation panel 132b shown in FIG. 5C, a cross key (“OK”) included in the operation panel 132a shown in FIG. 5A is touched by touching a button labeled “cursor”. And four arrow buttons around the button) are displayed. Further, on the operation panel 132b, a numeric keypad included in the operation panel 132a is displayed by touching a button described as “number”.

  Further, the image generation unit 421 controls the graphics processing unit 23 to cause the scaling unit 26 to output an image signal of the operation panel and scaling information corresponding to the supplied determination result. Here, the scaling information is information indicating the size when the operation panel is displayed on the display unit 13. Accordingly, the image generation unit 421 can cause the scaling unit 26 to adjust the size of the operation panel.

  Adjustment of the size of the operation panel will be described with reference to FIG. FIG. 6 is a screen diagram showing a display example of the operation panel on the television shown in FIG. 6 (b) to 6 (d) show some cases when the user is viewing the television with the television screen 131 displayed on the entire display area of the display unit 13 as shown in FIG. 6 (a). It is a figure which shows the image displayed as a result of trying to perform touch operation with respect to the touch panel 12 by operation | movement.

  The image generation unit 421 determines that the face area 143 occupies a range of the face area threshold 511 or more in the center area 142, or the amount of change of the face area 143 in the center area 142 of the captured image 141 is the face change threshold 512 or more. When the determination result is supplied, scaling information indicating the size as shown in FIG. 6B is controlled by the graphics processing unit 23 and output to the scaling unit 26. As a result, the scaling unit 26 adjusts the size of the operation panel 132 so that the size is as shown in FIG.

  When the image generation unit 421 is supplied with a determination result that the rate of change in the left region 144 or the right region 145 of the captured video 141 is equal to or greater than the left / right change threshold 513, the image generation unit 421 is illustrated in (c) or (d) of FIG. The scaling information indicating the size and the graphics processing unit 23 are controlled and output to the scaling unit 26. As a result, the scaling unit 26 adjusts the size of the operation panel 132 so as to have a size as shown in (c) or (d) of FIG.

  Then, the scaling unit 26 outputs the scaled image signal of the operation panel 132 to the blend unit 29. On the other hand, the image generation unit 421 instructs the image position determination unit 422 to determine the image position.

  Regarding the size adjustment of the television screen 131, as described above, the scaling unit 27 has been adjusted by the scaling unit 26 and the information indicating the size of the display area of the display unit 13 acquired from the blend unit 29. This is performed with reference to information indicating the size of the operation panel 132. The scaling unit 27 outputs the size-adjusted video signal of the television screen 131 to the blending unit 29.

  Further, the image generation unit 421 generates an image different from that of the operation panel 132 by controlling the graphics processing unit 23 in accordance with an instruction from the processing instruction unit 43. Details of this processing will be described with reference to FIG. 5 again.

  FIG. 5A shows an example of the operation panel 132 in FIG. 6C or 6D. In other words, it is an example of the operation panel 132 that is displayed when a determination result that the rate of change in the left region 144 or the right region 145 of the captured image 141 is equal to or greater than the left-right change threshold 513 is supplied. When the processing instruction unit 43 acquires touch information indicating that the position of the tool button 151 on the operation panel 132a illustrated in FIG. 5A is touched, the processing instruction unit 43 displays a tool image 133 on the image generation unit 421. Instruct. Upon receiving the instruction, the image generation unit 421 controls the graphics processing unit 23 to generate a tool image 133 as shown in FIG. Specifically, the image generation unit 421 reads the image data 514 of the tool image 133 in response to an instruction from the processing instruction unit 43 and causes the graphics processing unit 23 to generate the tool image 133.

  Then, the image generation unit 421 controls the graphics processing unit 23 to output the image signal of the tool image 133 generated by the scaling unit 26. At this time, the image generation unit 421 controls the graphics processing unit 23 to output the size adjustment result of the operation panel 132a, that is, the scaling information of the operation panel 132a shown in FIG. As a result, the tool image 133 is adjusted to the same size as the operation panel 132a shown in FIG.

  On the other hand, FIG. 5C shows an example of the operation panel 132 in FIG. 6B. In other words, the determination result that the face area 143 occupies the range of the face area threshold 511 or more in the center area 142, or the determination that the amount of change of the face area 143 in the center area 142 of the captured video 141 is the face change threshold 512 or more. It is an example of the operation panel 132b displayed when a result is supplied. When the processing instruction unit 43 acquires touch information indicating that the position of the tool button 151 on the operation panel 132b illustrated in FIG. 5C is touched, the processing instruction unit 43 displays a tool image 134 on the image generation unit 421. Instruct. Upon receiving the instruction, the image generation unit 421 controls the graphics processing unit 23 to generate a tool image 134 as shown in FIG. Specifically, the image generation unit 421 reads the image data 514 of the tool image 134 in response to an instruction from the processing instruction unit 43 and causes the graphics processing unit 23 to generate the tool image 134.

  Then, the image generation unit 421 controls the graphics processing unit 23 to output the image signal of the tool image 134 generated by the scaling unit 26. At this time, the image generation unit 421 controls the graphics processing unit 23 to output the size adjustment result of the operation panel 132b, that is, the scaling information of the operation panel 132b shown in FIG. As a result, the tool image 134 is adjusted to the same size as the operation panel 132b shown in FIG.

  Thereafter, the display control unit 42 displays the tool image 133 (or tool image 134) at the position where the operation panel 132a (or operation panel 132b) was displayed.

  The image position determination unit 422 determines the position of the image (video) displayed on the display unit 13. Specifically, upon receiving an instruction from the image generation unit 421, the image position determination unit 422 determines the position of the operation panel 132 according to the determination result supplied from the display control unit 42.

  When the image position determination unit 422 is supplied with the determination result that the rate of change in the right region 145 of the captured video 141 is equal to or greater than the left-right change threshold 513, the position of the operation panel 132 is displayed as illustrated in FIG. Is a region at a predetermined distance from the left-hand side end of the user in the display unit 13. The “user's left (right) hand side end” means the end of the user's left (right) hand side facing the television 10 in the display unit 13 or the like. That is, in the case of the television 10 shown in FIG. 5, the left (right) hand side end is the left (right) end. The same applies to “the left (right) hand side end of the user” used thereafter.

  When the image position determination unit 422 is supplied with the determination result that the rate of change in the left region 144 of the captured video 141 is equal to or greater than the left-right change threshold 513, as illustrated in FIG. The position is set to a region at a predetermined distance from the user's right-hand side end on the display unit 13.

  Further, the image position determination unit 422 determines that the face area 143 occupies a range of the face area threshold 511 or more in the center area 142 or the amount of change of the face area 143 in the center area 142 of the photographed video 141 is the face change threshold 512 or more. When the determination result is supplied, the position of the operation panel 132 is set as an area including the center of the display unit 13 as shown in FIG.

  Further, the image position determining unit 422 determines the position of the television screen 131 according to the determined position of the operation panel 132. Specifically, when the image position determination unit 422 is supplied with the determination result that the rate of change in the left region 144 or the right region 145 of the captured video 141 is equal to or greater than the left / right change threshold 513, (c) in FIG. As shown in (d), an area where the operation panel 132 is not displayed in the display area of the display unit 13 is an area where the television screen 131 is displayed.

  On the other hand, the image position determination unit 422 determines that the face area 143 occupies a range of the face area threshold 511 or more in the center area 142 or the amount of change in the face area 143 in the center area 142 of the captured image 141 is a face. When the determination result that the change threshold value is 512 or more is supplied, the area for displaying the television screen 131 remains the entire display area of the display unit 13 as shown in FIG. At this time, the image position determination unit 422 determines to display the operation panel 132 superimposed on the television screen 131.

  Then, the image position determination unit 422 outputs the determined position information of the operation panel 132 and the television screen 131 to the blend unit 29 by controlling the graphics processing unit 23. The blend unit 29 combines the size-adjusted television screen 131 and the operation panel 132 according to the acquired position information. The blend unit 29 displays the combined operation panel 132 and television screen 131 on the display unit 13.

  Note that the display control unit 42 determines that the face area 143 occupies the face area threshold 511 or more in the center area 142, or the amount of change in the face area 143 in the center area 142 of the captured image 141 is greater than or equal to the face change threshold 512. When the determination result is supplied, the operation panel 132 may be displayed in the entire display area (all areas) of the display unit 13. As a result, the operation panel 132 is maximized, so that it is possible to realize a touch interface that is easier for a user who performs a touch operation with a pen or the like held in the mouth.

  The storage unit 15 is a storage device that stores various data used in the television 10 (display control device 1). The storage unit 15 stores at least a face area threshold 511, a face change threshold 512, and a left / right change threshold 513.

  The face area threshold value 511 is a numerical value (threshold value) for determining that the user is approaching the television 10 in an attempt to perform a touch operation on the touch panel 12. Specifically, the face area occupies the central area 142. It is a threshold value of the ratio. This numerical value may be a numerical value indicating the ratio of the face area 143 to the central area 142 when the user is about to perform a touch operation with his / her fingertip, and the numerical value is not particularly limited. That is, when the number is greater than or equal to the above, the user is closer to the television 10 than when the user performs a touch operation with the fingertip. Therefore, it is highly likely that the user is about to perform a touch operation on the touch panel 12 with a pen or the like held in the mouth. Note that when the face area determination unit 411 determines the ratio of the face area 143 to the entire captured image 141, the face area threshold 511 is the shooting when the user is about to perform a touch operation with his / her fingertip. This is a numerical value indicating the ratio of the face area 143 to the image 141.

  The face change threshold 512 is a numerical value (threshold) for determining that the user is approaching the television 10 in an attempt to perform a touch operation on the touch panel 12, and more specifically, enlargement of the face region 143 in the captured video The rate threshold. This numerical value indicates the enlargement ratio of the face area 143 when the user is going to perform a touch operation with his / her fingertip from the state where the user is watching a program or the like using the television 10 as usual. Any numerical value may be used, and the numerical value is not particularly limited. In other words, when the enlargement ratio of the face area 143 is equal to or larger than that when the user who wants to perform the touch operation with the fingertip brings the face closer, the user brings the face closer (in detail, in addition to the mouth). It can be determined that a touch operation is being performed (by using a pen or the like).

  The left-right change threshold value 513 is a numerical value (threshold value) for determining that the user has moved the right arm or the left arm in an attempt to perform a touch operation on the touch panel 12, and more specifically, the left region 144 or the right region 145 in the captured video. Is the threshold of the rate of change at. This numerical value indicates that the right arm or the left arm is applied to the television 10 (specifically, the display unit 13) in order to perform a touch operation on the touch panel 12 from a state in which the user is viewing a program or the like using the television 10 as usual. Any numerical value that is sufficiently lower than the rate of change in the left region 144 or the right region 145 when it is moved is not particularly limited. That is, when the user moves the right arm or the left arm in order to perform the touch operation, the rate of change is equal to or greater than the left / right change threshold 513, and therefore the user is about to perform the touch operation with his / her fingertip. Can be judged.

  Note that the left / right change threshold 513 is changed according to the content of a predetermined change that the left / right region determination unit 412 determines, or another left / right change threshold 513 is added. For example, when the left / right region determination unit 412 recognizes the left hand or the right hand in the left region 144 or the right region 145 and determines whether the recognized change rate of the left hand or right hand is equal to or greater than a threshold value, the left / right change The threshold value 513 may be a value indicating the rate of change of the left hand or the right hand when only the index finger is extended from the open hand state.

  The image data 514 is information for generating the operation panel 132 and the tool image 133. For example, the image data 514 may be a script for generating the operation panel 132 or the tool image 133.

(Flow of image display processing)
Next, a flow of image display processing in the present embodiment will be described with reference to FIG. FIG. 7 is a flowchart illustrating an example of the flow of image display processing in the television 10. In the description here, the operation panel for manual operation is described as the operation panel 132a (the operation panel shown in FIG. 5A), and the operation panel for operation with a pen or the like in the mouth is By describing the operation panel 132b shown in FIG. 5C (the operation panel shown in FIG. 5C), the two will be described separately.

  First, when the operation prediction unit 41 is supplied with the captured video 141 from the motion acquisition unit 11, the operation prediction unit 41 supplies the captured video 141 to the face area determination unit 411.

  When acquiring the captured image 141 (S1), the face region determining unit 411 specifies a face region 143 that is a region where the user's face is displayed from the captured image 141 (S2). Then, the face area determination unit 411 determines whether the specified face area 143 occupies the face area threshold 511 or more in the center area 142 (S3, detection step). Specifically, the face area determination unit 411 reads the face area threshold value 511 from the storage unit 15, and the numerical value indicating the ratio of the area occupied by the identified face area 143 in the central area 142 is equal to or greater than the face area threshold value 511. It is determined whether or not.

  When it is determined that the face area threshold value 511 is reached (YES in S3), the face area determination unit 411 supplies the determination result to the display control unit 42. The display control unit 42 displays the operation panel 132b for face operation according to the supplied determination result (S9, display step). Specifically, the display control unit 42 causes the operation panel 132b for face operation to be superimposed on the television screen 131 and displayed. More specifically, first, the display control unit 42 supplies the supplied determination result to the image generation unit 421. When the image generation unit 421 is supplied with a determination result that the face region 143 occupies the range of the face region threshold 511 or more in the central region 142, the image generation unit 421 controls the graphics processing unit 23 to perform an operation according to the read image data 514. The panel 132b (specifically, the operation panel 132b shown in FIG. 5C) is generated. Then, the image generation unit 421 controls the graphics processing unit 23 to cause the scaling unit 26 to output the image signal of the operation panel 132b and scaling information corresponding to the supplied determination result. Thereby, the scaling part 26 adjusts the size of the operation panel 132b so that it may become a magnitude | size as shown to (b) of FIG.

  Then, the scaling unit 26 outputs the scaled image signal of the operation panel 132 b to the blend unit 29. On the other hand, the image generation unit 421 instructs the image position determination unit 422 to determine the image position.

  Upon receiving the above instruction, the image position determination unit 422 determines the position of the operation panel 132b at the center of the display unit 13 as shown in FIG. 6B according to the determination result supplied from the display control unit 42. Include area. Further, the image position determination unit 422 leaves the area for displaying the television screen 131 as the entire display area of the display unit 13. At this time, the image position determination unit 422 determines to display the operation panel 132 b superimposed on the television screen 131. Then, the image position determination unit 422 outputs the determined position information of the operation panel 132 b and the television screen 131 to the blend unit 29 by controlling the graphics processing unit 23.

  The blending unit 29 combines the size-adjusted television screen 131 and the operation panel 132b as shown in FIG. 6B according to the acquired position information. Then, the blend unit 29 causes the display unit 13 to display the synthesized operation panel 132b and the television screen 131.

  On the other hand, when it is determined that the identified face area 143 is less than the face area threshold 511 (NO in S3), the face area determination unit 411 determines that the amount of change in the identified face area 143 is the center area 142. It is determined whether or not the face change threshold is 512 or more (S4, detection step). Specifically, the face area determination unit 411 reads the face change threshold value 512 from the storage unit 15, calculates the change amount of the face area 143 (specifically, the enlargement ratio of the face area 143) in the center area 142, It is determined whether or not the enlargement ratio is greater than or equal to the face change threshold 512. If it is determined that the face change threshold value 512 is greater than or equal to 512 (YES in S4), the face area determination unit 411 supplies the determination result of the determination to the display control unit 42. The display control unit 42 executes the process of step S9 described above.

  On the other hand, when it is determined that the enlargement ratio of the face area 143 is less than the face change threshold 512 (NO in S4), the face area determination unit 411 sends the position information of the face area 143 to the left and right area determination unit 412. Supply.

  The left and right area determination unit 412 divides the captured image 141 into a left area 144 and a right area 145 around the face area 143 according to the supplied position information of the face area 143 (S5). Then, the left / right area determination unit 412 determines whether or not there is a change indicating a change rate of the left / right change threshold 513 or more in the captured video 141 (S6, detection step). Specifically, the left / right region determination unit 412 reads the left / right change threshold value 513 from the storage unit 15, calculates the change rate of the change occurring in the left region 144 and the right region 145, and the change rate is equal to or greater than the left / right change threshold value 513. It is determined whether or not.

  First, when it is determined that a change greater than or equal to the left / right change threshold 513 occurs in the left region 144 (“is left region” in S6), the left / right region determination unit 412 sends the determination result to the display control unit 42. Supply.

  Then, the display control unit 42 displays the operation panel 132a for the right hand operation according to the supplied determination result (S7, display step). Specifically, the display control unit 42 displays the operation panel 132a in an area from the end on the right hand side of the user on the display unit 13 to a predetermined distance, and from the left end of the operation panel 132 on the display unit 13 on the left hand side of the user. The TV screen 131 is displayed in the area up to the end of the screen. More specifically, first, the display control unit 42 supplies the supplied determination result to the image generation unit 421. When the image generation unit 421 is supplied with the determination result that the rate of change in the left region 144 of the captured video 141 is equal to or greater than the left-right change threshold 513, the image generation unit 421 controls the graphics processing unit 23 to respond to the read image data 514. The operation panel 132a (specifically, the operation panel 132a shown in FIG. 5A) is generated. Then, the image generation unit 421 controls the graphics processing unit 23 to cause the scaling unit 26 to output an image signal of the operation panel 132a and scaling information corresponding to the supplied determination result. Thereby, the scaling unit 26 adjusts the size of the operation panel 132a so as to have a size as shown in FIG.

  Then, the scaling unit 26 outputs the scaled image signal of the operation panel 132a to the blend unit 29. On the other hand, the image generation unit 421 instructs the image position determination unit 422 to determine the image position.

  Upon receiving the instruction, the image position determination unit 422 determines the position of the operation panel 132a on the display unit 13 according to the determination result supplied from the display control unit 42, as shown in FIG. The region from the right-hand end of the to the predetermined distance. The image position determination unit 422 sets the position of the television screen 131 as an area from the left end of the operation panel 132a to the end of the display unit 13 on the left hand side of the user. Then, the image position determination unit 422 outputs the determined position information of the operation panel 132 a and the television screen 131 to the blend unit 29 by controlling the graphics processing unit 23.

  The blending unit 29 combines the size-adjusted television screen 131 and the operation panel 132a as shown in FIG. 6D in accordance with the acquired position information. Then, the blend unit 29 causes the display unit 13 to display the synthesized operation panel 132a and the television screen 131.

  On the other hand, when it is determined that a change greater than or equal to the left / right change threshold 513 has occurred in the right region 145 (“is right region” in S6), the left / right region determination unit 412 performs display control of the determination result. To the unit 42.

  Then, the display control unit 42 displays the left-hand operation panel 132a according to the supplied determination result (S8, display step). Specifically, the display control unit 42 displays the operation panel 132a in a region from the end on the left hand side of the user on the display unit 13 to a predetermined distance, and the right side of the user on the display unit 13 from the right end of the operation panel 132a. The TV screen 131 is displayed in the area up to the end of the screen. More specifically, first, the display control unit 42 supplies the supplied determination result to the image generation unit 421. When the image generation unit 421 is supplied with the determination result that the rate of change in the right region 145 of the captured video 141 is greater than or equal to the left / right change threshold 513, the image generation unit 421 controls the graphics processing unit 23 to respond to the read image data 514. The operation panel 132a is generated. Then, the image generation unit 421 controls the graphics processing unit 23 to cause the scaling unit 26 to output an image signal of the operation panel 132a and scaling information corresponding to the supplied determination result. As a result, the scaling unit 26 adjusts the size of the operation panel 132a so as to have a size as shown in FIG.

  Then, the scaling unit 26 outputs the scaled image signal of the operation panel 132a to the blend unit 29. On the other hand, the image generation unit 421 instructs the image position determination unit 422 to determine the image position.

  Upon receiving the instruction, the image position determination unit 422 determines the position of the operation panel 132a on the display unit 13 according to the determination result supplied from the display control unit 42, as shown in FIG. The region from the left-hand side end to a predetermined distance. In addition, the image position determination unit 422 sets the position of the television screen 131 as an area from the right end of the operation panel 132a to the end of the display unit 13 on the right hand side of the user. Then, the image position determination unit 422 outputs the determined position information of the operation panel 132 a and the television screen 131 to the blend unit 29 by controlling the graphics processing unit 23.

  The blending unit 29 combines the size-adjusted television screen 131 and the operation panel 132a as shown in FIG. 6C according to the acquired position information. Then, the blend unit 29 causes the display unit 13 to display the synthesized operation panel 132a and the television screen 131.

  On the other hand, when it is determined that no change of the left / right change threshold 513 or more has occurred in any region (“None” in S6), that is, the change rate in both the left region 144 and the right region 145 is left and right. If it is determined that the change threshold value is less than 513, the left and right area determination unit 412 supplies information indicating that there is no change in the captured image 141 to the face area determination unit 411. When the information is supplied, the face area determination unit 411 executes the image display process again from the process of step S1.

(Operational effect of the television 10 according to the first embodiment)
As described above, the television 10 (specifically, the display control device 1) according to the present embodiment acquires a video image of the user using the camera 111, and the user performs a touch operation on the touch panel 12 in the video image. When the action to be executed is specified, the operation panel 132 is displayed at a position where the user can easily perform the touch operation.

  For example, when the user specifies an action to perform a touch operation with the right hand, the operation panel 132a is displayed in an area of a predetermined distance from the user's right hand side end on the display unit 13. Conversely, when the user specifies an action to perform a touch operation with the left hand, the operation panel 132a is displayed in an area at a predetermined distance from the end portion of the display unit 13 on the left hand side of the user. In addition, when the user specifies an action of bringing a face closer to the display unit 13 (to try to perform a touch operation with an indicator in the mouth), the operation panel 132 b is displayed on the display unit 13.

  As a result, the operation panel 132 can be displayed by predicting the movement of the user, so that the user does not need to perform a touch operation for displaying the operation panel 132. That is, conventionally, a process execution instruction for the television 10 (for example, changing a channel, which has been realized by two touch operations, that is, a touch operation for displaying the operation panel 132 and a touch operation on the operation panel 132) , An execution instruction for changing the setting) can be realized by a single touch operation. Therefore, when a user with a physical disability uses the television 10 in a hospital or the like, it is possible to instruct the television 10 to execute a process with a minimum necessary touch operation.

  In addition, since the operation to perform the touch operation is specified using the video captured by the camera 111, the operation panel 132 is displayed at a relatively early timing in the operation for the user to perform the touch operation. be able to. As a result, the user can recognize the position to be touched at an early stage, and thus can perform a touch operation on the operation panel 132 without being confused.

  In addition, since the display position, size, button layout, and the like of the operation panel 132 are changed according to the user's operation, the touch interface that is easy for the user to use can be realized.

[Embodiment 2]
The following will describe another embodiment of the present invention with reference to FIGS. For convenience of explanation, members having the same functions as those described in the embodiment are given the same reference numerals, and descriptions thereof are omitted.

(Main components of the television 20)
First, with reference to FIG. 8, the structure of the main part of the television 20 (display device) will be described. FIG. 8 is a block diagram illustrating an example of a main configuration of the television 20. As shown in FIG. 8, the television 20 is configured to be integrated with the display control device 2. A portion surrounded by a broken line shown in FIG. Note that the hardware configuration of the television 20 is the same as the hardware configuration of the television 10 described in the first embodiment, and a description thereof will be omitted.

  As illustrated in FIG. 8, the television 20 includes an operation acquisition unit 17, a control unit 18, and a storage unit 19 instead of the operation acquisition unit 11, the control unit 14, and the storage unit 15 described in the first embodiment.

  Unlike the motion acquisition unit 11 described in the first embodiment, the motion acquisition unit 17 includes a distance sensor 112 (distance measurement unit) in addition to the camera 111.

  The distance sensor 112 is a distance measuring sensor that measures the distance between the television 20 and an object. Specifically, the distance sensor 112 includes an object at a predetermined position (for example, the center position of the captured image) in a captured image captured by the camera 111, and a surface (front surface) provided with the display unit 13 in the television 20. Measure the distance. The action acquisition unit 17 outputs a distance value (distance information) that is a measurement result by the distance sensor 112 together with the captured image to the operation prediction unit 44 described later. Specifically, the camera 111 continuously captures images as long as the television 20 is activated. The distance sensor 112 continuously measures the distance as long as the television 20 is activated. Then, the operation acquisition unit 11 continues to output the captured video and the measured distance value to the operation prediction unit 41. It should be noted that the position where the distance sensor 112 is provided is the surface (front surface) provided with the object at a predetermined position (for example, the center position of the captured image) in the captured image captured by the camera 111 and the display unit 13 in the television 20. The position is not particularly limited as long as the distance can be measured.

  Unlike the control unit 14 described in the first embodiment, the control unit 18 includes an operation prediction unit 44 instead of the operation prediction unit 41.

  Unlike the operation prediction unit 41 described in the first embodiment, the operation prediction unit 44 includes a distance determination unit 413 in addition to the face region determination unit 411 and the left and right region determination unit 412. The operation prediction unit 44 supplies the distance value acquired from the operation acquisition unit 17 to the distance determination unit 413.

  The distance determination unit 413 performs determination regarding the distance value measured by the distance sensor 112. Specifically, the distance determination unit 413 reads a distance threshold 515 described later from the storage unit 19 and determines whether the distance value is less than the distance threshold 515. If it is determined that the distance is less than the threshold value 515, the operation prediction unit 44 is instructed to cause the captured image 141 to be supplied to the face area determination unit 411.

  On the other hand, when it is determined that the acquired distance value is equal to or greater than the distance threshold 515, the comparison between the acquired distance value and the distance threshold 515 is continued until the acquired distance value is less than the distance threshold 515.

  Unlike the storage unit 15 described in the first embodiment, the storage unit 19 further stores a distance threshold 515.

  The distance threshold value 515 is a numerical value (threshold value) for determining whether or not the user is about to perform a touch operation on the touch panel 12, and in detail, is a threshold value of a distance value measured by the distance sensor 112. This numerical value may be a numerical value indicating the distance between the front surface of the television 20 and the user's trunk when the user is at the farthest position where the user can extend the arm with respect to the touch panel 12 and perform the touch operation with the fingertip. That's fine. That is, the acquired distance value being less than the distance threshold 515 means that the user is at a position where the touch operation on the touch panel 12 can be performed, and the operation panel 132 may be displayed by the touch operation. That is.

(Flow of image display processing)
Next, the flow of image display processing in the present embodiment will be described with reference to FIG. FIG. 9 is a flowchart illustrating an example of the flow of image display processing in the television 20.

  First, when the operation prediction unit 44 is supplied with the captured image 141 and the distance value from the operation acquisition unit 11, the operation prediction unit 44 supplies the distance determination unit 413 with the distance value.

  When the distance determination unit 413 acquires the distance value (S11), the distance determination unit 413 determines whether the acquired distance value is less than the distance threshold 515 (S12). Specifically, the distance threshold value 515 is read from the storage unit 19, and whether or not the acquired distance value is less than the distance threshold value 515 is compared.

  If it is determined that the distance threshold is 515 or more (NO in S12), the process returns to step S11. On the other hand, when it is determined that the distance is less than the threshold 515 (YES in S12), the distance determination unit 413 instructs the operation prediction unit 44 to supply the captured image 141 to the face area determination unit 411.

  The subsequent processing from step S13 to step S21 is the same as the processing from step S1 to S9 in FIG. 7 described in the first embodiment, and thus description thereof is omitted here.

(Operational effect of the television 20 according to the second embodiment)
As described above, the television 20 (specifically, the display control device 2) according to the present embodiment captures images of the user using the camera 111 when the user and the television 20 are closer than a predetermined distance. , And an action for the user to perform a touch operation on the touch panel 12 is specified.

  As a result, even if the user performs any operation at a position away from the predetermined distance (a position where the user does not reach the touch panel 12 even if the user reaches out), the operation panel 132 is not displayed. Therefore, erroneous display of the operation panel 132 can be prevented.

[Embodiment 3]
The following will describe still another embodiment of the present invention with reference to FIGS. For convenience of explanation, members having the same functions as those described in the embodiment are given the same reference numerals, and descriptions thereof are omitted.

(Main components of the television 30)
First, with reference to FIG. 10, a configuration of main parts of the television 30 (display device) will be described. FIG. 10 is a block diagram illustrating an example of a main configuration of the television 30. As shown in FIG. 10, the television 30 has a configuration integrated with the display control device 3. A portion surrounded by a broken line shown in FIG. Note that the hardware configuration of the television 30 is the same as the hardware configuration of the television 10 described in the first embodiment, and thus the description thereof is omitted.

  As illustrated in FIG. 10, the television 30 includes an operation acquisition unit 51, a control unit 52, and a storage unit 53 instead of the operation acquisition unit 11, the control unit 14, and the storage unit 15 described in the first embodiment.

  Unlike the motion acquisition unit 11 described in the first embodiment, the motion acquisition unit 51 includes a first distance sensor 113 (distance measurement unit) and a second distance sensor 114 (distance measurement unit) instead of the camera 111.

  The first distance sensor 113 is a distance measuring sensor that measures the distance between the television 30 and an object. Specifically, the first distance sensor 113 is provided at the left hand side end of the user in the television 30. Then, the first distance sensor 113 measures the distance from the provided position to the object closest to the television 30. This will be described with reference to FIG. FIG. 11A is a schematic diagram illustrating a method of calculating the shortest distance between the television 30 and the user. As illustrated in FIG. 11A, the user U brings the fingertip F close to the touch panel 12 in order to perform a touch operation on the touch panel 12. At this time, the first distance sensor 113 measures the distance from the provided position to the object closest to the television 30. That is, in the example of FIG. 11A, the distance to the fingertip F is measured. On the other hand, for example, when the user is watching the program with both arms lowered, the first distance sensor 113 measures the distance from the provided position to the user's nose.

  The second distance sensor 114 is the same as the first distance sensor 113 except for the position where it is provided. The second distance sensor 114 is provided at the right hand side end of the user in the television 30. For example, as shown in FIG. 11A, the second distance sensor 114 also measures the distance from the provided position to the fingertip F closest to the television 30 in the same manner as the first distance sensor 113.

  The action acquisition unit 51 uses two distance values (a first distance value la and a second distance value lb), which are measurement results obtained by the first distance sensor 113 and the second distance sensor 114, as an operation prediction unit described later. Output to 45. Specifically, the first distance sensor 113 and the second distance sensor 114 continuously measure the distance as long as the television 30 is activated. Then, the operation acquisition unit 51 operates the first distance value la (distance information) measured by the first distance sensor 113 and the second distance value lb (distance information) measured by the second distance sensor 114 as an operation prediction unit. Continue to output to 45.

  Unlike the control unit 14 described in the first embodiment, the control unit 52 includes an operation prediction unit 45 instead of the operation prediction unit 41.

  Unlike the operation prediction unit 41 described in the first embodiment, the operation prediction unit 44 includes a distance determination unit 414, a calculation unit 415, and a region specification unit 416 instead of the face region determination unit 411 and the left and right region determination unit 412. . When the operation prediction unit 44 acquires the first distance value and the second distance value from the operation acquisition unit 51, the operation prediction unit 44 supplies the first distance value and the second distance value to the distance determination unit 414.

  The distance determination unit 414 calculates the shortest distance between the object and the television 30 using the distance values measured by the first distance sensor 113 and the second distance sensor 114, and the shortest distance is less than a distance threshold 516 described later. It is determined whether or not. Here, calculation of the shortest distance by the distance determination unit 414 and determination regarding the shortest distance will be described with reference to FIG. The distance determination unit 414 calculates the shortest distance ly between the television 30 and the user using the first distance value la and the second distance value lb.

  Specifically, the first distance sensor 113 and the second distance sensor 114 measure the distance to the same object (fingertip F in the example of FIG. 11A). When the distance value (first distance value) from a certain object to the first distance sensor 113 is la, and the distance value (second distance value) from the object to the second distance sensor 114 is lb If the measurable range of the distance in the first distance sensor 113 and the second distance sensor 114 is only on the front side of the television 30, the position of the object (fingertip F) is determined at one location. More specifically, the position at which the circle with the radius la centering on the position of the first distance sensor 113 and the circle with the radius lb centering on the position of the second distance sensor 114 intersect is the object (fingertip). F).

  Since the position of the object (fingertip F) can be specified in this way, the distance determination unit 414 can calculate the shortest distance ly from the television 30 to the object. Specifically, when the distance determination unit 414 obtains the first distance value la and the second distance value lb, the distance from the position of the second distance sensor 114 to the fingertip F shown in FIG. The length lx in the X-axis direction is specified. Then, the shortest distance ly is calculated from the second distance value lb and the specified length lx using the three-square theorem.

  In the example of FIG. 11A, the length in the X-axis direction from the position of the second distance sensor 114 to the fingertip F is set to lx, but from the position of the first distance sensor 113 to the fingertip F. The length in the X-axis direction may be lx. In this case, the distance determination unit 414 calculates the shortest distance ly from the first distance value la and the specified length lx.

  In this calculation, if accurate calculation is performed, a component perpendicular to the paper surface in FIG. 11 (distance perpendicular to the paper surface between the fingertip F and the first distance sensor 113 and the second distance sensor 114). Need to be considered. However, since the difference between the distance to be measured in consideration of the component perpendicular to the paper surface and the shortest distance ly described above is small and within an allowable range, it is described as not considered in the present embodiment.

  Specifically, when considering a component perpendicular to the paper surface, the distance to be measured is “the distance between the straight line connecting the first distance sensor 113 and the second distance sensor 114 and the object (fingertip F)”. It becomes. This distance is hereinafter referred to as distance lc. Here, the measurement range in the direction perpendicular to the paper surface of the first distance sensor 113 and the second distance sensor 114 (from the object at a position detectable by the first distance sensor 113 and the second distance sensor 114 from the first It is assumed that a range of angles formed between a straight line perpendicular to a straight line connecting the distance sensor 113 and the second distance sensor 114 and a normal direction of the touch panel 12 is −30 ° to + 30 °. Then, it is assumed that there is an object (fingertip F) at a position that is the upper limit (or lower limit) of the measurement range, and the distance lc is 30 cm. At this time, the shortest distance ly is ly = lc × cos (30 °) = 30 cm × cos (30 °) ≈26 cm. That is, the error between the shortest distance ly and the distance lc is about 4 cm at the maximum, and this difference is within an allowable range (the distance lc can be approximated to the shortest distance ly), and thus is not considered in this embodiment. As described.

  Then, the distance determination unit 414 compares the calculated shortest distance ly with the distance threshold 516 read from the storage unit 53 and determines whether or not the shortest distance ly is less than the distance threshold 516. When it is determined that the distance is less than the threshold 516, the first distance value la and the second distance value lb are supplied to the calculation unit 415, and the calculation unit 415 uses the region determination to determine the region. Instructs the value to be calculated.

  On the other hand, when it is determined that the shortest distance ly is greater than or equal to the distance threshold 516, the calculation and determination of the shortest distance ly is continued until the shortest distance ly becomes less than the distance threshold 516.

  The distance determination by the distance determination unit 414 is not limited to this example. For example, the distance determination unit 414 determines whether or not the first distance value 1a and the second distance value lb are equal to or smaller than a threshold value (or less than the threshold value), and the first distance value 1a and the second distance value lb. Is determined to be less than or equal to the threshold value (or less than the threshold value), the first distance value la and the second distance value lb are supplied to the calculation unit 415 and are used by the calculation unit 415 to determine the region. It may be instructed to calculate the region determination value. The threshold value is, for example, a numerical value indicating the length of a general human arm. Thereby, when it is determined that the first distance value 1a and the second distance value lb are equal to or smaller than the threshold value (or less than the threshold value), that is, the user extends the arm with respect to the touch panel 12 and performs the touch operation with the fingertip. It is feasible.

  The calculation unit 415 calculates an area determination value. Specifically, when the calculation unit 415 is instructed to calculate the region determination value from the distance determination unit 414, the calculation unit 415 uses the supplied first distance value la and second distance value lb to determine the region determination value. Calculate the value. Then, the calculated area determination value is supplied to the area specifying unit 416.

  Here, the calculation of the region determination value will be described in detail with reference to FIG. FIG. 11B is a schematic diagram illustrating a method for specifying an area touched by an indicator (finger). As shown in FIG. 11B, the calculation unit 415 equally divides the width of the television 30 (the length in the X-axis direction) in advance. In the present embodiment, as shown in FIG. 11B, an example in which the width is divided into six parts will be described. However, there is no particular limitation on how to divide the width. Hereinafter, each of the six divided widths is referred to as a “unit”. Then, the calculation unit 415 specifies the length rx of one unit, and divides the first distance value la and the second distance value lb by the specified length rx, thereby making the first distance value made dimensionless. Ra and the second distance value rb are calculated.

  In the above example, the calculation unit 415 divides the width of the television 30 into equal parts, that is, the lengths of the respective units are equal. However, the present invention is not limited to this example. For example, the calculation unit 415 may divide each unit so that the lengths of the units are not equal.

Then, the calculation unit 415 calculates the region determination value by subtracting the square of the dimensionless second distance value rb from the square of the dimensionless first distance value ra. That is, the region determination value is a numerical value calculated by calculating ra ² −rb ² . The calculation unit 415 supplies the calculated region determination value to the region specifying unit 416. In addition, the calculation unit 415 supplies the area specifying unit 416 in advance with equal information indicating how much the width of the television 30 is divided.

The area specifying unit 416 specifies an area on the touch panel 12 that includes a position where the user is about to perform a touch operation. Specifically, the region specifying unit 416 includes a position where the user is about to perform a touch operation by comparing the supplied region determination value (ra ² −rb ² ) with a region determination threshold 517 described later. Identify the areas that will be

  Here, a method for specifying a region will be described with reference to FIG.

  First, as illustrated in FIG. 11A, the area specifying unit 416 divides the touch panel 12 into a plurality of areas in advance. In the present embodiment, the touch panel 12 is equally divided into the area A, the area B, and the area C, but the boundary between two adjacent units and the two adjacent boundaries in the width of the television 30 described above. It is not limited to this example as long as it is equally divided so as to match.

  Further, the area specifying unit 416 sets coordinates at the boundaries of each unit using the equalization information supplied from the calculation unit 415. In the present embodiment, with the emphasis on ease of explanation, as described above, it is assumed that there is no component in the vertical direction on the paper surface in FIG. 11 (handled as two-dimensional coordinates). The three-dimensional coordinates include a component in the direction perpendicular to the paper surface.

  In the present embodiment, the center of the television 30 in front of the television 30 is defined as an origin O (0, 0). Then, coordinates are set at the boundary of each unit as shown in FIG. In the example of FIG. 11B, since it is divided into six equal parts, the coordinates set at the right-hand side end of the television 30 are (3, 0), and the coordinates set at the left-hand side end of the television 30 Becomes (−3,0). At this time, since the boundary between the two areas coincides with a part of the boundary between the two units, coordinates are set at the boundary between the two areas. Specifically, the coordinate at the boundary between the region A and the region B is (−1, 0), and the coordinate at the boundary between the region B and the region C is (1, 0).

Here, it is assumed that the fingertip F is at the boundary between the region B and the region C. When the coordinate of the fingertip F is (xt, yt), xt = 1. Here, using the three-square theorem, ra ² = {1-(− 3)} ² + yt ² and rb ² = (3-1) ² + yt ² . From these two formulas, the region determination value is ra ² −rb ² = 4 ² −2 ² = 16−4 = 12.

Similarly, when the fingertip F is at the boundary between the region A and the region B, the region determination value is ra ² −rb ² = −12.

  That is, in the example of FIG. 11B, when the region determination value is −12 or less, it can be specified that the fingertip F is about to touch a position included in the region A. Further, when the region determination value is larger than −12 and smaller than 12, it can be specified that the fingertip F is about to touch a position included in the region B. Furthermore, when the region determination value is larger than 12, it can be specified that the fingertip F is about to touch a position included in the region C.

  As described above, the region specifying unit 416 compares the supplied region determination value with the region determination threshold value 517 that is −12 and 12 in this embodiment, so that the user performs a touch operation on the touch panel 12. Identify the area that contains the position you are trying to execute. The area specifying unit 416 supplies information indicating the specified area to the display control unit 42.

  Unlike the storage unit 15 described in the first embodiment, the storage unit 53 does not store the face area threshold value 511, the face change threshold value 512, and the left / right change threshold value 513. The storage unit 53 stores a distance threshold value 516 and a region determination threshold value 517 instead of these.

  The distance threshold value 516 is a numerical value (threshold value) for determining whether or not the user is about to perform a touch operation on the touch panel 12, and more specifically, the first distance value la measured by the first distance sensor 113. And the threshold of the shortest distance ly obtained from the second distance value lb measured by the second distance sensor 114. This numerical value is a numerical value indicating the distance between the user's nose tip and the front surface of the television 30 when the user is at the farthest position where the user can extend the arm with respect to the touch panel 12 and perform the touch operation with the fingertip. Good.

  The area determination threshold value 517 is a numerical value (threshold value) for specifying an area including a position where the user is about to perform a touch operation. As described above, in the present embodiment, the numerical values are −12 and 12, but are not limited to these numerical values.

Here, the numerical value stored as the region determination threshold 517 is generalized. Assume that the fingertip F is at the boundary between the region B and the region C, the coordinates set at the left-hand side end of the television 30 are (x0, y0), and the coordinates set at the right-hand side end of the television 30 are ( x1, y1). At this time, ra ² = (xt−x0) ² + (yt−y0) ² and rb ² = (xt−x1) ² + (yt−y0) ² . Here, since y0 = y1 = 0, ra ² = (xt−x0) ² + yt ² and rb ² = (xt−x1) ² + yt ² .

When the value of the region determination threshold 517 is ^d, a ^{d = ra 2 -rb 2 = -2} × xt × x0 + x0 2 + 2 × xt × x1-x1 2. Here, as shown in FIG. 11B, when the origin O is determined so that x0 = −x1, the value of the region determination threshold when the fingertip F is at the boundary between the region B and the region C is d = 2 × xt × x1 + x1 ² + 2 × xt × x1−x1 ² = 4 × xt × x1. On the other hand, the value of the region determination threshold 517 when the fingertip F is at the boundary between the region A and the region B is −d. As described above, the value of the region determination threshold 517 can be generalized as d = 4 × xt × x1 and −d.

  In the present embodiment, the position where the plurality of distance sensors are provided is the number and the positions provided if the area specifying unit 416 is provided at different positions so that the area that the user is trying to touch can be specified. There are no particular restrictions on the positions. That is, the number and positions of the plurality of distance sensors in the present embodiment are determined depending on how the touch panel 12 is divided into a plurality of regions. For example, when the touch panel 12 is divided vertically (in the direction perpendicular to the paper surface in FIG. 11), the first distance sensor 113 and the second distance sensor 114 are respectively provided at the upper end portion and the lower end portion of the television 30, for example. What is necessary is just to arrange.

(Flow of image display processing)
Next, a flow of image display processing in the present embodiment will be described with reference to FIG. FIG. 12 is a flowchart illustrating an example of the flow of image display processing in the television 30.

  First, when the operation prediction unit 45 is supplied with the first distance values la and lb from the motion acquisition unit 11, the operation prediction unit 45 supplies the first distance value la and the second distance value lb to the distance determination unit 414.

  When the distance determination unit 414 obtains the distance values (first distance value la and second distance value lb) detected by each distance sensor (S31), the distance determination unit 414 obtains the distance values from the first distance value la and second distance value lb. It is determined whether or not the obtained shortest distance ly is less than a distance threshold (S32). Specifically, the position of the object (for example, the fingertip F) is specified from the first distance value la and the second distance value lb, and the length in the X-axis direction from the position of the second distance sensor 114 to the fingertip F is determined. Specify lx. Then, the shortest distance ly is calculated from the second distance value lb and the specified length lx using the three-square theorem. Then, the calculated shortest distance ly is compared with the distance threshold 516 read from the storage unit 53.

  If the distance determination unit 414 determines that the distance threshold value 516 or more is reached (NO in S32), the process returns to step S31. On the other hand, when it is determined that the distance is less than the threshold 516 (YES in S32), the distance determination unit 414 supplies the first distance value la and the second distance value lb to the calculation unit 415 and calculates the calculation unit. Instruct 415 to calculate the region determination value.

In response to the instruction from the distance determination unit 414, the calculation unit 415 calculates the region determination value (ra ² −rb ² ) (S32). Specifically, the calculation unit 415 divides the supplied first distance value la and second distance value lb by the length rx of each unit of the width of the television 30 that is divided in advance. Then, the first distance value ra and the second distance value rb which are made dimensionless are calculated. Then, the region determination value is calculated by calculating ra ² −rb ² . The calculation unit 415 supplies the calculated region determination value to the region specifying unit 416.

  The region specifying unit 416 determines the magnitude relationship between the calculated region determination value and the values d and -d of the region determination threshold value 517 (S34). Specifically, the region specifying unit 416 reads the region determination threshold 517 from the storage unit 53, compares it with the supplied region determination value, and determines the magnitude relationship between the two values.

When the region specifying unit 416 determines that the region determination value is equal to or greater than the value d of the region determination threshold 517 (“d ≦ (ra ² −rb ² )” in S34), the user touches the region C illustrated in FIG. Identify the region that contains the location where the operation is to be performed. Then, information indicating the region C is supplied to the display control unit 42. When the display control unit 42 is supplied with the information indicating the region C, the display control unit 42 displays the operation panel 132 for the right hand operation (S35). In addition, since the specific process which the display control part 42 performs in order to display the operation panel 132 for right-hand operation is the same as the process demonstrated in Embodiment 1, description here is abbreviate | omitted.

On the other hand, when the region specifying unit 416 determines that the region determination value is equal to or smaller than the value −d of the region determination threshold value 517 (“(ra ² −rb ² ) ≦ −d” in S34), FIG. A region A to be shown is specified as a region including a position where the user is about to perform a touch operation. Then, information indicating the region A is supplied to the display control unit 42. When the information indicating the area A is supplied, the display control unit 42 displays the left-hand operation panel 132 (S36). In addition, since the specific process which the display control part 42 performs in order to display the operation panel 132 for left-hand operation is the same as the process demonstrated in Embodiment 1, description here is abbreviate | omitted.

Further, when the region specifying unit 416 determines that the region determination value is larger than the value −d of the region determination threshold value 517 and smaller than d (“−d <(ra ² −rb ² ) <d” in S34), FIG. A region B shown in FIG. 11 is specified as a region including a position where the user is about to perform a touch operation. Then, information indicating the region B is supplied to the display control unit 42. When the information indicating the region B is supplied, the display control unit 42 displays the operation panel 132 for face operation (S37). The specific process executed by the display control unit 42 to display the operation panel 132 for face operation is the same as the process described in the first embodiment, and thus the description thereof is omitted here.

(Operational effect of the television 30 according to the third embodiment)
As described above, the television 30 (specifically, the display control device 3) according to the present embodiment performs an operation in which a user tries to perform a touch operation on the touch panel 12 without using the camera 111, and the touch operation. An area including the position of the touch panel 12 that is expected to be executed can be specified.

  Accordingly, it is possible to realize a touch interface that is easy for the user to use without using large-capacity data such as video and images. Therefore, the processing load of the television 30 (specifically, the display control device 3) can be suppressed.

[Example of software implementation]
The control blocks (in particular, the operation prediction unit 41, the display control unit 42, the processing instruction unit 43, the operation prediction unit 44, and the operation prediction unit 45) of the televisions 10 to 30 (specifically, the display control devices 1 to 3) are integrated. It may be realized by a logic circuit (hardware) formed on a circuit (IC chip) or the like, or may be realized by software using a CPU (Central Processing Unit).

  In the latter case, the display control devices 1 to 3 include a CPU that executes instructions of a program that is software that realizes each function, and a ROM (Read Only) in which the program and various data are recorded so as to be readable by a computer (or CPU). Memory) or a storage device (these are referred to as “recording media”), a RAM (Random Access Memory) for expanding the program, and the like. And the objective of this invention is achieved when a computer (or CPU) reads the said program from the said recording medium and runs it. As the recording medium, a “non-temporary tangible medium” such as a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. The program may be supplied to the computer via an arbitrary transmission medium (such as a communication network or a broadcast wave) that can transmit the program. The present invention can also be realized in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.

[Summary]
A display control device (1) according to an aspect 1 of the present invention is configured to perform a touch operation on the touch panel on a display device (television 10) including a display unit (13) and a touch panel (12) formed integrally with the display unit. A display control device that displays an operation image that can be operated by the display unit, the detection unit (operation prediction unit 41) detecting the movement of the user's body moving the body to touch the touch panel, and the detection A display control unit (42) for displaying the operation image on the display unit when the movement of the body is detected by the unit.

  According to said structure, when the motion of the user's body who moved the body trying to touch a touch panel is detected, an operation image is displayed on a display part. As a result, the operation image is displayed on the display unit of the display device before the user touches the touch panel. Therefore, the user does not need to perform a touch operation for displaying the operation image. Therefore, touch operations performed by the user can be reduced, and a touch interface that is easy for the user to use can be realized.

  In addition, since the movement of the body of the user who moves his body to touch the touch panel is detected, the user does not need to perform a special operation (for example, gesture or voice input) to display the operation image. That is, even if there are people around the user, the user can perform an operation for displaying the operation image (an operation for performing a touch operation) without any embarrassment.

  In addition, since the movement of the user's body that has moved to touch the touch panel is detected, the operation image is displayed before the user touches the touch panel. Therefore, since the user can recognize the position to be touched in the operation image at an early stage, the user can execute the touch operation without being confused.

  In the display control device according to aspect 2 of the present invention, in the aspect 1, the detection unit detects the movement of the user's right hand and the movement of the left hand as the movement of the user's body, and the display control unit When the movement of the right hand is detected by the detection unit, the operation image is displayed in an area on the right hand side of the user in the display unit, while when the movement of the left hand is detected, the operation image is displayed. You may display on the area | region of the said user's left hand side in the said display part.

  According to the above configuration, when the user is about to touch the touch panel with the right hand, the operation image is displayed in the area on the right hand side of the user in the display unit, and when the user is about to touch the touch panel with the left hand, The image is displayed in an area on the left hand side of the user in the display unit.

  As a result, the distance by which the user moves his / her hand is minimized, and a touch interface that is easy to use for the user can be realized.

  Also, if the operation panel is displayed on the opposite side of the hand moved by the user, the user's own arm will interfere with the viewing of the image or video displayed on the display device. For example, when the display device is a television, the user's own arm is in the way of watching the program.

  On the other hand, according to the above configuration, since the operation panel is displayed in the region of the hand moved by the user, the user's arm does not interfere with the visual recognition of the image or video.

  In the aspect 1 or 2, the display control device according to aspect 3 of the present invention detects the movement of the user's face approaching the touch panel as the movement of the user's body, and the display control unit When the movement of the user's face approaching the touch panel is detected by the detection unit, the operation image may be displayed in a region including the center of the display unit or in the entire region of the display unit.

  According to the above configuration, when the movement of the user's face approaching the touch panel is detected, the operation image is displayed in an area including the center of the display unit or in the entire area.

  Thereby, even when the user tries to execute a touch operation with an indicator such as a pen in his mouth, an operation image can be displayed.

  Here, a situation in which the user performs a touch operation with an indicator such as a pen in his mouth is, for example, an attempt to display an operation image when a user with both hands is viewing a TV in a hospital or the like. It is a situation to do. At this time, the user's face is considered to be in front of the center of the display unit in most cases. Therefore, by displaying the operation image in an area including the center of the display unit, when the user tries to perform a touch operation with an indicator such as a pen in his mouth, the distance to move the face is minimized. be able to. As a result, a situation where the user performs a touch operation with an indicator such as a pen in his mouth, in other words, a situation where it is more difficult to perform a touch operation at a position desired by the user than when performing a touch operation with a fingertip. In addition, it is possible to make it easy to perform a touch operation at a position desired by the user.

  In addition, if the operation image is displayed in the entire area of the display unit, the operation image is maximized, so that each item included in the operation image is enlarged. The operation can be made easier.

  The display control apparatus according to aspect 4 of the present invention further includes a photographing unit (camera 111) that continuously photographs the user in any one of the above aspects 1 to 3, and the detection unit photographs by the photographing unit. The motion of the user's body may be detected from the recorded video or image.

  According to said structure, the motion of the user's body who is going to touch a touch panel from the image | video or image which image | photographed the user continuously is detected. Thereby, it is possible to specify whether or not the user is about to touch the touch panel at an early stage in the user's operation. Therefore, since the operation image can be displayed at an early stage in the operation of the user trying to perform the touch operation, the user can recognize the position to be touched in the operation image at an early stage, and the touch operation can be performed without confusion. Can be executed.

  The display control apparatus according to aspect 5 of the present invention is the display control apparatus according to aspect 4, wherein the detection unit has a rate of change of the user's body motion in the video or image captured by the imaging unit equal to or greater than a predetermined rate of change. The display control unit may display the operation image on the display unit when the detection unit determines that the change rate is equal to or higher than the predetermined change rate.

  According to the above configuration, the operation image is displayed on the display unit when the change rate of the movement of the user's body in the captured video or image is equal to or higher than the predetermined change rate. Thereby, since it can be specified that the user has moved the body to touch the touch panel, the operation image can be displayed on the display unit. Further, when the rate of change is less than a predetermined rate of change, that is, when the user's body movement is a small movement that is not a movement to touch the touch panel, the operation image is not displayed. Therefore, it is possible to prevent erroneous display such that an operation image is displayed on the display unit even though the user does not intend to touch the touch panel.

  Specifically, when the user moves the body to touch the touch panel, the user in the captured video or image changes greatly from the state before the body is moved. As a result, the rate of change of the user's body movement in the video or image increases. That is, when the rate of change is large, the user is trying to touch the touch panel.

  Then, if the predetermined change rate is set to a value sufficiently lower than the change rate when the user moves to touch the touch panel from the state before moving the body, the predetermined change By comparing the rate and the rate of change of the user's body movement in the video or image, it can be determined whether or not the user is touching the touch panel. Thereby, when the user moves his body to touch the touch panel, the operation image can be displayed on the display unit.

  In the display control device according to aspect 6 of the present invention, in the aspect 5, the detection unit divides the video or image captured by the imaging unit into left and right with the user's face as the center, and the display control unit When the change rate in the left area of the video or image divided by the detection unit is determined to be greater than or equal to the predetermined change rate, the operation image is displayed on the display unit on the right hand side of the user. On the other hand, when it is determined that the change rate in the right region is equal to or higher than the predetermined change rate, the operation image may be displayed in the left hand side region of the user in the display unit.

  According to the above configuration, the captured video or image is divided into a left region and a right region with the user's face as the center. When the rate of change in the left region is a predetermined rate of change, the operation image is displayed in the region on the right hand side of the display unit. Further, when the rate of change in the right region is a predetermined rate of change, the operation image is displayed in the region on the left hand side of the display unit.

  As a result, in the video or image in which the user's face is captured, it is accurately determined whether the user's right hand or left hand has moved even when the user's face is not at the center of the video or image. be able to.

  The display control device (2) according to aspect 7 of the present invention further includes a distance measuring unit (distance sensor 112) that measures the distance between the display device and the user according to any one of the aspects 4 to 6. When the distance measured by the distance measurement unit is less than a predetermined threshold, the detection unit may detect the movement of the user's body from the video or image captured by the imaging unit.

  According to said structure, when a distance of a display apparatus and a user is less than a predetermined threshold value, a user's motion is detected from the image | photographed image | video or image. In other words, when the distance between the display device and the user is equal to or greater than the predetermined threshold, even if the user moves, the user's movement is not detected, and thus the operation image is not displayed. That is, it is possible to detect the user's movement only when there is a possibility that the user is going to touch the touch panel.

  Specifically, a numerical value indicating a distance between the display device and the user's trunk when the user is at the farthest position where the user can touch the touch panel with a fingertip by extending the arm with respect to the touch panel is a threshold value If the distance measured by the distance measuring unit is equal to or greater than the threshold, the user is in a position where the user cannot touch the touch panel. That is, even if the user moves his / her body at such a position, the movement is not a movement to touch the touch panel. In such a case, the display control apparatus according to this aspect does not detect the user's movement. That is, the display control apparatus according to this aspect detects the user's movement only when the user is at a position where the user can touch the touch panel. Therefore, erroneous display of the operation image can be prevented.

  In the display control device (3) according to aspect 8 of the present invention, in the aspect 1, the detection unit is configured to select an area to be touched by movement of the user's body among a plurality of areas obtained by dividing the touch panel in advance. In particular, the display control unit may display the operation image in a display region of the display unit that overlaps the region specified by the detection unit.

  According to said structure, the area | region which the user is going to touch is specified among the several area | regions which divided the touch panel previously, and an operation image is displayed on the area | region which the user is going to touch. In other words, the operation image is not displayed at a position greatly deviated from the position that the user is trying to touch. As a result, the distance by which the user moves his / her hand is minimized, and a touch interface that is easy to use for the user can be realized.

  The display control device according to aspect 9 of the present invention is the display control device according to aspect 8, in which a plurality of distances between the display device and the user, which are provided at different positions so as to identify the region to be touched, are measured. The distance measurement unit may be further included, and the detection unit may specify the region to be touched based on the distances respectively measured by the plurality of distance measurement units.

  According to said structure, the area | region which a user is going to touch is pinpointed based on the distance which the some distance measurement part each measured.

  Thereby, the area | region where a user is going to perform touch operation can be pinpointed only by the measurement of the distance by a some distance sensor. That is, it is possible to determine whether or not the user is about to touch the touch panel and the position when the user is about to touch without using a video or an image of the user. Therefore, it is possible to display the operation image at a position that is easy for the user to operate without causing the display control apparatus to execute a process with a large load such as a process for video or an image.

  Note that the positions of the plurality of distance measuring units are determined depending on how the touch panel is divided. That is, using the respective distances measured by the plurality of distance measuring units, the positions of the plurality of distance measuring units are determined so that the region to be touched can be identified. For example, when the touch panel is divided into left and right, two distance measurement units may be provided at the right end and the left end of the display device, respectively, and when the touch panel is divided up and down, two distance measurement units are displayed. What is necessary is just to provide in the upper end part and lower end part of an apparatus, respectively.

  The display control device control method according to the tenth aspect of the present invention provides a display device including a display unit and a touch panel integrally formed with the display unit, and displays an operation image operable by a touch operation on the touch panel. The display control device is displayed on the touch panel with a detection step (S3, S4, S6) for detecting the movement of the body of the user who moved the body to touch the touch panel, and the detection step A display step (S7, S8, S9) for displaying the operation image on the display unit when motion is detected.

  According to this control method, the same function and effect as those of the display control apparatus according to aspect 1 are obtained.

  The display control apparatus according to each aspect of the present invention may be realized by a computer. In this case, the display control apparatus is operated by causing the computer to operate as each unit (limited to software elements) included in the display control apparatus. A control program for a display control apparatus realized by a computer and a computer-readable recording medium on which the control program is recorded also fall within the scope of the present invention.

  The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention. Furthermore, a new technical feature can be formed by combining the technical means disclosed in each embodiment.

  INDUSTRIAL APPLICABILITY The present invention can be used for a display control device that displays a touch-operable image on a display device such as a television provided with a display unit and a touch panel formed integrally with the display unit.

DESCRIPTION OF SYMBOLS 1 Display control apparatus 2 Display control apparatus 3 Display control apparatus 10 Television (display apparatus)
12 Touch Panel 13 Display Unit 20 Television (Display Device)
30 Television (display device)
41 Operation prediction unit (detection unit)
42 display control unit 44 operation prediction unit (detection unit)
45 Operation prediction unit (detection unit)
111 Camera (shooting unit)
112 Distance sensor (distance measuring unit)
113 1st distance sensor (distance measurement part)
114 Second distance sensor (distance measuring unit)
S3 detection step S4 detection step S6 detection step S7 display step S8 display step S9 display step

Claims (12)

A display control device that causes a display device including a display unit and a touch panel formed integrally with the display unit to display an operation image operable by a touch operation on the touch panel on the display unit,
A detection unit that detects the movement of the body of the user who moved the body to touch the touch panel;
A display control device, comprising: a display control unit that displays the operation image on the display unit when movement of the body is detected by the detection unit. The detection unit distinguishes and detects the movement of the user's right hand and the left hand as the movement of the user's body,
The display control unit displays the operation image in a region on the right hand side of the user in the display unit when the movement of the right hand is detected by the detection unit, while when the movement of the left hand is detected, The display control apparatus according to claim 1, wherein the operation image is displayed in an area on the left hand side of the user in the display unit. The detection unit detects a movement of the user's face approaching the touch panel as a movement of the user's body,
The display control unit displays the operation image in a region including the center of the display unit or the entire region of the display unit when the detection unit detects a movement of the user's face approaching the touch panel. The display control device according to claim 1, wherein the display control device is a display control device. It further includes a photographing unit that continuously photographs the user,
The display control apparatus according to claim 1, wherein the detection unit detects a movement of the body of the user from a video or an image captured by the imaging unit. The detection unit determines whether the change rate of the movement of the user's body in the video or image shot by the shooting unit is equal to or higher than a predetermined change rate,
The display control apparatus according to claim 4, wherein the display control unit displays the operation image on the display unit when the detection unit determines that the rate of change is equal to or greater than the predetermined change rate. The detection unit divides the video or image captured by the imaging unit into left and right with the user's face as the center,
The display control unit displays the operation image on the display unit when the change rate in the left region of the video or image divided by the detection unit is equal to or higher than the predetermined change rate. When the change rate in the right region is determined to be greater than or equal to the predetermined change rate, the operation image is displayed in the left hand region of the user in the display unit. The display control apparatus according to claim 5. A distance measuring unit for measuring a distance between the display device and the user;
The detection unit detects movement of the body from an image or an image captured by the imaging unit when the distance measured by the distance measurement unit is less than a predetermined threshold. 7. The display control device according to any one of 6. The detection unit identifies an area to be touched by a movement of the user's body among a plurality of areas obtained by dividing the touch panel in advance.
The display control apparatus according to claim 1, wherein the display control unit displays the operation image in a display region of the display unit that overlaps the region specified by the detection unit. A plurality of distance measuring units that measure the distance between the display device and the user, which are provided at different positions so that the region to be touched can be specified;
The display control apparatus according to claim 8, wherein the detection unit specifies the region to be touched based on the distances measured by the plurality of distance measurement units. A control method of a display control device for causing a display device including a display unit and a touch panel integrally formed with the display unit to display an operation image operable by a touch operation on the touch panel on the display unit,
A detection step of detecting the movement of the body of the user who moved the body to touch the touch panel;
And a display step of displaying the operation image on the display unit when the body movement is detected in the detection step.   A control program for causing a computer to function as the display control device according to claim 1, wherein the control program causes the computer to function as the detection unit and the display control unit.   The computer-readable recording medium which recorded the control program of Claim 11.

Images