JP2009169794A - Display device, control method thereof, input device, control method thereof, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate a clear transferred image including an object approaching an input/output display. <P>SOLUTION: A trigger detection part 51 detects as a trigger an input to a capture button 12 or an input to a region of the capture button of an input/output display 11. An image generation part 52 generates a transferred image including an object approaching the input/output display 11 from a light reception image based on light reception at the input/output display 11. A display control unit 53 displays a display image based on the generated transferred image to the input/output display 11. The device is applicable to a display device having an input/output display. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a display device, a control method thereof, an input device, a control method thereof, and a program, and more particularly, for example, in a display device having an input / output display, a transferred image showing an object approaching the input / output display is more The present invention relates to a display device capable of generating a clear image, a control method thereof, an input device, a control method thereof, and a program.

  A display constituting a display device includes what is called an input / output display having a function of receiving light from the outside with an optical sensor (see, for example, Patent Document 1 and Patent Document 2). For example, the input / output display irradiates light with a backlight and receives light reflected and returned by an object approaching the input / output display. Alternatively, the input / output display receives light emitted from an object having a light source.

  In the display device having the input / output display, a transfer image in which an object approaching the input / output display is reflected from the received light image based on the light received by the input / output display, that is, the transfer image is generated. Has been proposed as a display image on an input / output display.

For example, in the example of FIG. 1, the heart-shaped stamp surface of the stamp to be transferred is brought closer to the input / output display shown as a transparent rectangle from the other side of the input / output display. At this time, in an ideal example as shown in FIG. 1, the portion of the stamp surface (black heart-shaped portion in the figure) in the display image (for example, the transferred image itself) on the input / output display is the original stamp surface. It becomes a clear one that closely matches the shape of.
Japanese Patent Laid-Open No. 2000-019478 JP 2004-127272 A

  However, in the display device having the input / output display described above, if the transfer image is generated by, for example, averaging the received light image, as shown in FIGS. 2A and 2B, the display image ( For example, the portion of the stamp surface to be transferred (the black heart-shaped portion in the figure) in the transferred image itself becomes a locus of the stamp surface in the approaching process, and is unclear and does not match the original stamp surface. For example, even if the received light image is used as it is as the transfer image, if the stamp surface to be transferred moves relative to the input / output display, the portion of the stamp surface in the transfer image is still blurred and unclear. End up.

  The present invention has been made in view of such a situation. For example, in a display device having an input / output display, a transfer image showing an object approaching the input / output display is generated as a clearer image. Is to be able to.

  A display device according to a first aspect of the present invention is a display device including a display light receiving unit that displays an image and receives light from the outside, and inputs an input to an input unit or a predetermined region of the display light receiving unit. Based on the trigger detection means to detect as a trigger and the trigger detected by the trigger detection means, an object close to the display light receiving means is shown from the received light image based on the light received by the display light receiving means. Image generation means for generating a transfer image, and display control means for causing the display light receiving means to display a display image based on the transfer image generated by the image generation means.

  A control method or program according to the first aspect of the present invention is a program for causing a computer to execute a control method or control process of a display device including a display light receiving unit that displays an image and receives light from the outside. A trigger detection step that detects an input to a unit or an input to a predetermined region of the display light receiving means as a trigger, and light reception based on light reception by the display light receiving means based on the trigger detected by the processing of the trigger detection step From the image, an image generation step for generating a transfer image in which an object close to the display light receiving unit is reflected, and a display image based on the transfer image generated by the processing of the image generation step are displayed on the display light receiving unit. A program for causing a computer to execute a control method or a control process including a display control step to be displayed. .

  An input device according to a second aspect of the present invention is an input device including a light receiving unit that receives light from the outside, and is a received light image based on light reception by the light receiving unit, and the frame at that time and the frame Trigger detection means for detecting, as a trigger, a difference value indicating a difference between the light reception images of frames close in time, and the light reception based on the trigger detected by the trigger detection means Image generating means for generating, from the image, a transfer image showing an object close to the light receiving means.

  The input device according to the second aspect of the present invention may further include storage means for storing the transfer image generated by the image generation means.

  A storage device is connected to the input device according to the second aspect of the present invention, and further provided with a transmission means for transmitting the transfer image generated by the image generation means to the storage device. The transfer image transmitted from the transmission unit of the input device can be received and stored.

  A display device is connected to the input device of the second aspect of the present invention, and further provided with a transmission means for transmitting the transfer image generated by the image generation means to the display device. The transfer image transmitted from the transmission unit of the input device can be received, and a display image based on the transfer image can be displayed on a display unit.

  The input device according to the second aspect of the present invention may further include display control means for displaying a display image based on the transfer image generated by the image generation means on a display unit.

  The control method or program according to the second aspect of the present invention is a program for causing a computer to execute a control method or control process for an input device including a light receiving means for receiving light from the outside. A trigger detection step for detecting, as a trigger, a difference value indicating a difference between the received light images of the frame at that time and a frame temporally close to the frame at the time, A control method or a control process including an image generation step of generating, from the received light image, a transfer image showing an object close to the light receiving means based on the trigger detected by the trigger detection step. A program to be executed by a computer.

  An input device according to a third aspect of the present invention is an input device including a light receiving unit that receives light from the outside, and triggers by detecting a frame-like pattern from a received light image based on light reception by the light receiving unit. Trigger detecting means for detecting, and image generating means for generating a transfer image showing an object in proximity to the light receiving means from the light receiving image based on the trigger detected by the trigger detecting means. .

  The input device according to the third aspect of the present invention may further include storage means for storing the transfer image generated by the image generation means.

  A storage device is connected to the input device according to the third aspect of the present invention, and further provided with a transmission means for transmitting the transfer image generated by the image generation means to the storage device. The transfer image transmitted from the transmission unit of the input device can be received and stored.

  A display device is connected to the input device according to the third aspect of the present invention, and further includes transmission means for transmitting the transfer image generated by the image generation means to the display device. The transfer image transmitted from the transmission unit of the input device can be received, and a display image based on the transfer image can be displayed on a display unit.

  The input device according to the third aspect of the present invention may further include display control means for causing a display unit to display a display image based on the transfer image generated by the image generation means.

  A control method or program according to a third aspect of the present invention is a program that causes a computer to execute a control method or control process of an input device including a light receiving unit that receives light from the outside. A trigger detecting step for detecting a trigger by detecting a frame-shaped pattern from the received light image, and a proximity of the light receiving means from the received light image based on the trigger detected by the processing of the trigger detecting step. A program for causing a computer to execute a control method or a control process including an image generation step of generating a transfer image in which a moving object is reflected.

  In the first aspect of the present invention, an input to an input unit or an input to a predetermined area of the display light receiving unit is detected as a trigger, and light reception based on light reception by the display light receiving unit is performed based on the detected trigger. From the image, a transfer image showing an object close to the display light receiving means is generated, and a display image based on the generated transfer image is displayed on the display light receiving means.

  In a second aspect of the present invention, a difference value indicating a difference between the received light image based on the light received by the light receiving means and the received light image of the frame close to the frame in time is a predetermined value. A range is detected as a trigger, and based on the detected trigger, a transfer image in which an object close to the light receiving means is reflected is generated from the received light image.

  In the third aspect of the present invention, a trigger is detected by detecting a frame-like pattern from a received light image based on light received by the light receiving means, and the received light from the received light image based on the detected trigger. A transfer image in which an object close to the means is shown is generated.

  According to the first to third aspects of the present invention, for example, a transfer image in which an object close to an input / output display is shown can be generated as a clearer image.

  Embodiments of the present invention will be described below. Correspondences between the constituent elements of the present invention and the embodiments described in the specification or the drawings are exemplified as follows. This description is intended to confirm that the embodiments supporting the present invention are described in the specification or the drawings. Therefore, even if there is an embodiment which is described in the specification or the drawings but is not described here as an embodiment corresponding to the constituent elements of the present invention, that is not the case. It does not mean that the form does not correspond to the constituent requirements. Conversely, even if an embodiment is described herein as corresponding to a configuration requirement, that means that the embodiment does not correspond to a configuration requirement other than the configuration requirement. It's not something to do.

  Embodiments to which the present invention is applied will be described below with reference to the drawings.

  FIG. 3 is a diagram showing an external configuration example of the display device 1 to which the present invention is applied.

  The display device 1 is provided with, for example, an input / output display 11 on the front surface (front surface in the figure), and a capture button 12 is provided in the lower left portion of the figure surrounding the input / output display 11 on the front surface.

  For example, the user brings the stamp surface of the stamp shown in FIG. 1 or the like to be transferred close to the input / output display 11, for example, close to the stamp surface parallel to the input / output display 11. When it is determined that it is touching or approaching by a short distance, an input is made to the capture button 12.

  The display device 1 detects an input to the capture button 12 as a trigger, and generates a transfer image showing an object (in this example, a stamp surface) close to the input / output display 11 from a received light image based on the trigger. Then, a first transfer image generation display process for displaying a display image on the input / output display 11 based on the transfer image is performed.

  Next, an example of the internal configuration of the display device 1 will be described with reference to FIG.

  The display device 1 includes a capture button 12, an input / output panel 31, a control unit 32, and an image memory 54.

  The capture button 12 supplies an input signal indicating that to the control unit 32 when there is an input to the capture button 12, for example, when the user presses the capture button 12 or releases it after releasing it.

  The input / output panel 31 includes the input / output display 11, and displays the display image on the input / output display 11 based on the display data of the display image supplied from the control unit 32. A light reception image based on the reception of light from the outside is generated and supplied to the control unit 32.

  The control unit 32 includes, for example, a CPU (Central Processor Unit), a RAM (Random Access Memory), and the like, and controls each unit of the display device 1. For example, the control unit 32 detects a trigger based on the input signal supplied from the capture button 12, and generates a transfer image from the received light image based on the detected trigger among the received light images supplied from the input / output panel 31. The display data of the display image is generated based on the transferred image and supplied to the input / output panel 31.

  The input / output panel 31 includes an input / output display 11, a light reception signal processing unit 41, and a display signal processing unit 42.

  The input / output display 11 displays a display image based on the display signal supplied from the display signal processing unit 42. The input / output display 11 includes a plurality of optical sensors 11A distributed throughout the input / output display 11, and generates a light reception signal corresponding to the amount of light received from the outside by the optical sensor 11A. This is supplied to the received light signal processing unit 41.

  Based on the light reception signal supplied from the input / output display 11, the light reception signal processing unit 41 has, for example, a portion where the object of the input / output display 11 is close (high) and has nothing close to it. Is generated for each frame and supplied to the control unit 32.

  The display signal processing unit 42 performs predetermined processing on the display data of the display image supplied from the control unit 32, and supplies the display signal obtained as a result to the input / output display 11.

  The control unit 32 includes a trigger detection unit 51, an image generation unit 52, a display control unit 53, and an image memory 54.

  The trigger detection unit 51 detects a trigger based on the input signal supplied from the capture button 12.

  When a trigger is detected by the trigger detection unit 51, the image generation unit 52 generates a transfer image from the received light image based on the trigger among the received light images supplied from the input / output panel 31, and the display control unit 53 and the image memory 54 and supply.

  The display control unit 53 generates display data of the display image based on the transfer image supplied from the image generation unit 52 and supplies the display data to the display signal processing unit 42 of the input / output panel 31.

  The image memory 54 stores a transfer image and the like supplied from the image generation unit 52.

  Next, the operation of the display device 1 in the first transfer image generation / display process will be described with reference to the flowchart of FIG.

  In step S <b> 1, the trigger detection unit 51 of the control unit 32 of the display device 1 determines whether a trigger has occurred based on the input signal supplied from the capture button 12. For example, the trigger detection unit 51 determines whether or not there is a trigger based on whether or not an input signal indicating that there is an input to the capture button 12 is supplied. If it is determined in step S1 that there is no trigger, the process returns to step S1 and the subsequent processing is performed in the same manner.

  On the other hand, if it is determined in step S1 that there is a trigger (that is, if a trigger is detected), in step S2, the image generation unit 52 uses the trigger in the received light image supplied from the input / output panel 31. A transfer image is generated from the received light image based on this, and supplied to the display control unit 53 and the image memory 54.

  Specifically, for example, the image generation unit 52 generates a processed image that has been subjected to an average process or the like from a received light image based on a trigger among the received light images supplied from the input / output panel 31, and from the processed image, An image obtained by extracting a predetermined portion is generated as a transfer image.

  That is, for example, the image generation unit 52 detects the received light image of the frame when the trigger is detected as the processed image, the received light image n (n> 0) frames before the frame when the trigger is detected, and the trigger. Received image after m (m> 0) frames from the frame at the time of detection, an image obtained by averaging the received images of a predetermined number of frames before the trigger was detected, and a predetermined number before the trigger was detected An image obtained by weighted averaging the received light images of the frames with a predetermined weight, an image obtained by averaging the received light images of a predetermined number of frames after the trigger is detected, and a predetermined number of frames after the trigger is detected An image obtained by weighted averaging of received light images with a predetermined weight, an image obtained by averaging received light images of a predetermined number of frames before and after the trigger is detected, and a predetermined number of frames before and after the trigger is detected An image or the like obtained by weighting and averaging the received light image of the system with a predetermined weight is generated.

  Then, the image generation unit 52 has only the largest object part of the object part in which the object is shown in the generated processed image, or the area of the object part in which the object is shown in the generated process image. Only the object part that is equal to or higher than a predetermined threshold, only the top N object parts of the object part in which the object in the generated processed image is reflected, or the entire processed image is extracted as a transfer image.

  In step S3, the display control unit 53 displays the transfer image supplied from the image generation unit 52 as a foreground, for example, a portion where the object of the transfer image is reflected, for example, an empty image as a background. An image is created and the display data is supplied to the display signal processing unit 42 of the input / output panel 31. In this way, the display control unit 53 causes the input / output display 11 to display a display image based on the transfer image supplied from the image generation unit 52.

  And it returns to step S1 and the subsequent processes are performed similarly.

  As described above, the display device 1 detects an input to the capture button 12 as a trigger, that is, when an object close to the input / output display 11 is assumed to be in a state suitable for transfer. Since the trigger is detected at the same time, for example, a clearer transfer image can be generated by generating the transfer image from the received light image based on the trigger.

  Next, an external configuration example of the display device 71 will be described with reference to FIG.

  6 corresponding to those in FIG. 3 are denoted by the same reference numerals, and the description thereof is omitted as appropriate.

  The display device 71 in FIG. 6 is obtained by deleting the capture button 12 of the display device 1 in FIG. 3. In the display device 71, instead of providing the capture button 12, a capture button as an image is displayed on the input / output display 11. It is displayed in the lower left of the figure.

  For example, the user brings the stamp surface of the stamp shown in FIG. 1 or the like to be transferred close to the input / output display 11, for example, close to the stamp surface parallel to the input / output display 11. When it is determined, input is performed to the capture button display area on the input / output display 11.

  The display device 71 detects an input to the display area of the capture button as a trigger, generates a transfer image showing an object close to the input / output display 11 from a light reception image based on the trigger, and displays the transfer image on the transfer image. Based on this, a second transfer image generation display process for displaying a display image on the input / output display 11 is performed.

  Next, an example of the internal configuration of the display device 71 will be described with reference to FIG.

  In FIG. 7, the same reference numerals are given to the corresponding parts in FIG. 4, and description thereof will be omitted as appropriate.

  The display device 71 includes an input / output panel 31 and a control unit 81.

  7 includes a trigger detection unit 91 in place of the trigger detection unit 51 of the control unit 32 in FIG.

  The trigger detection unit 91 detects a trigger based on the received light image supplied from the input / output panel 31.

  Next, the operation of the display device 71 in the second transfer image generation display process will be described with reference to the flowchart of FIG.

  In step S <b> 21, the trigger detection unit 91 of the control unit 81 of the display device 71 determines whether or not there is a trigger based on the received light image supplied from the input / output panel 31. For example, the trigger detection unit 91 determines whether or not there is an input to the capture button display area on the input / output display 11, that is, for example, the part corresponding to the capture button display area in the received light image from the input / output panel 31. It is determined whether or not there is a trigger depending on whether or not the luminance value is within a predetermined range. If it is determined in step S21 that there is no trigger, the process returns to step S1 and the subsequent processing is performed in the same manner.

  On the other hand, when it is determined in step S21 that a trigger has occurred (that is, when a trigger is detected), in step S22, the image generation unit 52 supplies from the input / output panel 31 as in step S2 of FIG. A transfer image is generated from the received light image based on the trigger among the received light images to be supplied to the display control unit 53 and the image memory 54.

  In step S23, as in step S3 in FIG. 5, the display control unit 53 uses the transfer image supplied from the image generation unit 52 as the foreground, for example, the object portion shown in the transfer image, for example, what A display image with a blank image as a background is created, and the display data is supplied to the display signal processing unit 42 of the input / output panel 31. In this way, the display control unit 53 causes the input / output display 11 to display a display image based on the transfer image supplied from the image generation unit 52.

  And it returns to step S21 and subsequent processing is performed similarly.

  As described above, since the display device 71 detects the input to the display area of the capture button as a trigger instead of using the input to the capture button as a trigger, there is no need to provide a capture button. A clear transfer image can be generated.

  Next, a configuration example of the display device 111 will be described with reference to FIG.

  In FIG. 9, the same reference numerals are given to corresponding portions in FIG. 7, and description thereof will be omitted as appropriate.

  For example, the user brings the stamp surface of the stamp shown in FIG.

  The display device 111 is a light reception image from the input / output panel 31 and triggered by a difference value indicating a difference between the light reception image of the frame at that time and a frame temporally close to the frame within a predetermined range. A transfer image is generated from the received light image that is detected and based on the trigger, and a third transfer image generation / display process for displaying the display image on the input / output display 11 is performed based on the transfer image.

  The display device 111 includes an input / output panel 31 and a control unit 121.

  9 includes a trigger detection unit 131 instead of the trigger detection unit 91 of the control unit 81 in FIG.

  The trigger detection unit 131 detects a trigger based on the received light image supplied from the input / output panel 31.

  The trigger detection unit 131 includes a difference calculation unit 131A and a trigger detection unit 131B.

  The difference calculation unit 131A calculates a difference between predetermined light reception images supplied from the input / output panel 31.

  The trigger detection unit 131B detects a trigger based on the difference between the received light images calculated by the difference calculation unit 131A.

  Next, the operation of the display device 111 in the third transfer image generation display process will be described with reference to the flowchart of FIG.

  In this example, a light-receiving image of a binary image in which the value of the portion where the object is close to the input / output display 11 is 1 and the value of the portion where nothing is close is 0 is supplied from the input / output panel 31. Shall be.

  In step S41, the difference calculation unit 131A of the trigger detection unit 131 of the control unit 121 of the display device 111 is a received light image supplied from the input / output panel 31, and the frame at that time and a frame close in time to the frame. The difference between the received light images is calculated.

  Specifically, for example, when using a light reception image of a binary image, the difference between the light reception images is equivalent to taking an exclusive OR (XOR (Exclusive OR)) between the light reception images. The unit 131A calculates the difference between the received light images by taking the XOR of the received light images.

  That is, for example, the difference calculation unit 131A receives the received light image of the (N−1) th frame at time N−1 shown in FIG. 11A and the received light of the Nth frame at time N immediately after the (N−1) th frame shown in FIG. 11B. By taking an XOR with the image, the difference between the received light images shown in FIG. 11C is calculated.

  In step S42, the trigger detection unit 131B obtains a difference value indicating the difference between the received light images based on the difference between the received light images calculated by the difference calculation unit 131A, and whether or not there is a trigger based on the difference value. Determine whether.

  Specifically, for example, the trigger detection unit 131B obtains the number of pixels in a portion where the difference between the received light images is, for example, 1 as the difference value indicating the difference between the received light images, and the difference value is a predetermined value. It is determined whether or not there is a trigger depending on whether or not it is within a range (for example, below a predetermined threshold).

  For example, the trigger detection unit 131B obtains the number of pixels of a portion having a value of, for example, 1 (black portion in the drawing) as a difference value indicating a difference between the received light images illustrated in FIG. 11C.

  If it is determined in step S42 that there is no trigger, the process waits for the received light image of the next frame to be supplied from the input / output panel 31, returns to step S41, and the subsequent processing is performed in the same manner.

  On the other hand, when it is determined in step S42 that there is a trigger (that is, when a trigger is detected), in step S43, the image generation unit 52 supplies from the input / output panel 31 as in step S2 of FIG. A transfer image is generated from the received light image based on the trigger among the received light images to be supplied to the display control unit 53 and the image memory 54.

  In step S43, the display control unit 53 uses the transfer image supplied from the image generation unit 52 as the foreground, for example, based on the transfer image supplied from the image generation unit 52, as in step S3 of FIG. A display image with a blank image as a background is created, and the display data is supplied to the display signal processing unit 42 of the input / output panel 31. In this way, the display control unit 53 causes the input / output display 11 to display a display image based on the transfer image supplied from the image generation unit 52.

  And it returns to step S41 and the subsequent processes are performed similarly.

  As described above, the display device 111 detects that the difference value indicating the difference between the received light image of the frame at that time and the frame close to the frame in time is within a predetermined range as a trigger. That is, since the trigger is detected when an object close to the input / output display 11 is assumed to be in a state suitable for transfer, a transfer image is generated from the received light image based on the trigger. For example, a clearer transfer image can be generated.

  Furthermore, since the display device 111 automatically detects the trigger from the received light image generated for each frame without being based on the input from the outside, for example, a clear transfer image can be generated more easily. it can.

  The trigger detection unit 131B detects the difference as a trigger even if the difference value indicating the difference between the received light images is within a predetermined range for only one frame. In addition, the trigger detection unit 131B detects the difference between the received light images. It may be detected as a trigger that the difference value shown is within a predetermined range for n (n> 1) frames continuously. In this way, for example, it is possible to detect a trigger when it is more reliably assumed that an object close to the input / output display 11 is in a state suitable for transfer.

  In this example, the received light image is a binary image, but may be a gray image or a color image, for example.

  A third transfer image generation / display process in the case of using this gray light reception image will be described.

  In this case, in step S41, the difference calculation unit 131A performs binarization processing with a predetermined threshold on the light reception image of the gray image supplied from the input / output panel 31, and the light reception image of the binary image obtained as a result thereof. , The difference between the received light images of the binary image of the frame at that time and the frame temporally close to that frame is calculated.

  Specifically, the difference calculation unit 131A, for example, a portion having a luminance value (pixel value) equal to or greater than a predetermined threshold with respect to the gray light reception image from the input / output panel 31 (that is, an object on the input / output display 11). A binarization process is performed in which the value of the portion where the two are close to each other is set to 1, and the value of the portion whose luminance value (pixel value) is smaller than the predetermined threshold (that is, the portion where nothing is close) is set to zero.

  In step S42, the trigger detection unit 131B obtains a difference value indicating a difference between the light reception images of the binary image based on the difference between the light reception images of the binary image calculated by the difference calculation unit 131A, and the difference value is obtained. Based on the above, it is determined whether or not there is a trigger.

  If it is determined in step S42 that there is no trigger, the process waits for the received light image of the next frame to be supplied from the input / output panel 31, returns to step S41, and the subsequent processing is performed in the same manner.

  On the other hand, when it is determined in step S42 that there is a trigger (that is, when a trigger is detected), in step S43, the image generation unit 52 supplies from the input / output panel 31 as in step S2 of FIG. From the received light image of the gray image, a transfer image of the gray image is generated from the received light image of the gray image based on the trigger, and is supplied to the display control unit 53 and the image memory 54.

  In step S43, the display control unit 53, based on the gray image transfer image supplied from the image generation unit 52, for example, the object portion shown in the gray image transfer image, as in step S3 of FIG. Foreground, for example, a display image with an empty image as a background is created, and the display data is supplied to the display signal processing unit 42 of the input / output panel 31. In this way, the display control unit 53 causes the input / output display 11 to display a gray image display image as shown in FIG. 12, for example, based on the gray image transfer image supplied from the image generation unit 52. .

  And it returns to step S41 and the subsequent processes are performed similarly.

  Next, a third transfer image generation / display process in the case of using a light-receiving image of a color image will be described.

  The color image referred to here is, for example, an RGB (red green blue) color system color image, and is composed of each color image (R image, G image, B image).

  In this case, in step S41, the difference calculation unit 131A determines the difference between the received light image of the color image supplied from the input / output panel 31 and the received light image of the color image of the frame close in time to the current frame. Is calculated.

  Specifically, the difference calculation unit 131A calculates, for each color image of the light reception image of the color image from the input / output panel 31, the difference between the color image of the frame at that time and the color image of the frame temporally close to that frame.

  In step S42, the trigger detection unit 131B obtains a difference value indicating a difference between the light reception images of the color image based on the difference between the light reception images of the color image calculated by the difference calculation unit 131A, and uses the difference value as the difference value. Based on this, it is determined whether or not there is a trigger.

  Specifically, for example, the trigger detection unit 131B determines the number of pixels in which the absolute value of the difference between the color images calculated by the difference calculation unit 131A is within a predetermined range (for example, a predetermined threshold or more), and receives the color image. It is obtained as a difference value indicating a difference between images.

  The processing after step S43 is processing in which the gray image is replaced with a color image in the third transfer image generation / display processing in the case where the light reception image of the gray image described above is used, and description thereof is omitted.

  Next, another example of the third transfer image generation / display process in the case of using a light-receiving image of a color image will be described.

  In this example, in step S41, the difference calculation unit 131A applies a luminance image such as HSV (hue saturation value) or HLS (hue saturation lightness) to the light-receiving image of the color image supplied from the input / output panel 31. The luminance image obtained as a result of the conversion is further binarized at a predetermined threshold value, and the received light image of the binary image obtained as a result is obtained at that time and its frame. The difference between the received light images of the binary image of the frame close in time to the frame is calculated.

  In step S42, the trigger detection unit 131B obtains a difference value indicating a difference between the light reception images of the binary image based on the difference between the light reception images of the binary image calculated by the difference calculation unit 131A, and the difference value is obtained. Based on the above, it is determined whether or not there is a trigger.

  The processing after step S43 is processing in which the gray image is replaced with a color image in the third transfer image generation / display processing in the case where the light reception image of the gray image described above is used, and description thereof is omitted. Note that a transfer image of a color image is generated by the third transfer image generation / display process, and based on the transfer image of the color image, a color image as shown in FIG. A display image is displayed.

  As described above, the display device 111 can generate a transfer image of a gray image or a color image from a light reception image of a gray image or a color image, and can display a display image of a gray image or a color image.

  Note that when the transfer device generates the transfer image, the display device 111 may not generate the transfer image until the received light image satisfies a predetermined condition.

  For example, the display device 111 has a predetermined size portion (in the drawing, including a portion of an object (a heart-shaped black portion in the drawing) shown in a transfer image (display image) as shown in FIG. The transfer image is not generated until the pixel value (luminance value) of the received light image of the portion corresponding to the broken-line square is within a predetermined range (for example, a predetermined threshold value or less).

  Next, a configuration example of the display device 151 will be described with reference to FIG.

  In FIG. 15, the same reference numerals are given to corresponding portions in FIG. 7, and description thereof will be omitted as appropriate.

  For example, the user brings the stamp surface of the stamp shown in FIG.

  The display device 151 detects a frame-like pattern such as a circular frame or a hexagon surrounding the stamp surface to be transferred, as shown in FIGS. 16A and 16B, from the light-receiving image from the input / output panel 31. A trigger image is detected, a transfer image is generated from the received light image based on the trigger, and a fourth transfer image generation display process for displaying a display image on the input / output display 11 is performed based on the transfer image.

  The display device 151 includes an input / output panel 31 and a control unit 161.

  The control unit 161 in FIG. 15 is provided with a trigger detection unit 171 instead of the trigger detection unit 91 of the control unit 81 in FIG.

  The trigger detection unit 171 detects a trigger by detecting a predetermined pattern from the received light image supplied from the input / output panel 31.

  Next, the operation of the display device 151 in the fourth transfer image generation / display process will be described with reference to the flowchart of FIG.

  In step S61, the trigger detection unit 171 of the control unit 161 of the display device 151 performs, for example, a circular or hexagonal shape as illustrated in FIGS. 16A and 16B on the received light image supplied from the input / output panel 31. Pattern matching is performed using the frame-shaped pattern as a detection target, and whether or not a trigger has occurred is determined based on whether or not the frame-shaped pattern is detected by the pattern matching. If it is determined in step S61 that there is no trigger, the process returns to step S61, and the subsequent processing is performed in the same manner.

  On the other hand, when it is determined in step S61 that there is a trigger (that is, when a trigger is detected), in step S62, the image generation unit 52 uses the trigger in the received light image supplied from the input / output panel 31. A transfer image is generated from the received light image based on this, and supplied to the display control unit 53 and the image memory 54.

  Specifically, for example, the image generation unit 52 has a circular or hexagonal frame shape detected by the trigger detection unit 171 in the processed image generated from the received light image supplied from the input / output panel 31. Only the part of the object shown inside the pattern is extracted as a transfer image.

  In step S63, the display control unit 53 uses the transfer image supplied from the image generation unit 52 as the foreground, for example, based on the transfer image supplied from the image generation unit 52, as in step S3 of FIG. A display image with a blank image as a background is created, and the display data is supplied to the display signal processing unit 42 of the input / output panel 31. In this way, the display control unit 53 causes the input / output display 11 to display a display image based on the transfer image supplied from the image generation unit 52.

  And it returns to step S61 and the subsequent processes are performed similarly.

  As described above, the display device 151 detects a trigger by detecting a predetermined pattern from the received light image supplied from the input / output panel 31, that is, an object close to the input / output display 11. Since the trigger is detected when it is assumed that the image is suitable for transfer, for example, a clearer transfer image can be generated by generating a transfer image from the received light image based on the trigger. it can.

  The display device 151 stores and displays the generated transfer image by itself. However, for example, the display device 151 stores and displays the transfer image on another device connected to the display device 151. It may be.

  For example, in the example of FIG. 18, a PC 181 is connected to the display device 151 via the network 182. The display device 151 transmits the generated transfer image to the PC 181, and the PC 181 receives the transfer image transmitted from the display device 151, stores the transfer image in a storage unit (not shown), or stores the transfer image in the transfer image. A display image based on this is displayed on the display 181A.

  In the above, as shown in FIG. 19A, the display control unit 53 uses the object part in which the object of the transfer image is reflected as it is as the display image, but otherwise performs predetermined image processing on the object part. You may make it a display image.

  For example, in the example of FIG. 19B, the display control unit 53 enlarges the object portion in which the object of the transfer image is reflected and performs image processing to move to a predetermined position to obtain a display image. In the example of FIG. 19C, The display control unit 53 reduces the object portion in which the object of the transfer image is reflected, and performs image processing for moving to a predetermined position to obtain a display image. In the example of FIG. 19D, the display control unit 53 rotates the object part in which the object of the transfer image is reflected 90 degrees counterclockwise, and performs image processing to move to a predetermined position to obtain a display image. In this example, the display control unit 53 enlarges the object portion in which the object of the transfer image is reflected, rotates 90 degrees counterclockwise, and performs image processing to move to a predetermined position to obtain a display image. In the example of FIG. 19F, the display control unit 53 rotates the object part in which the object of the transfer image is reflected 90 degrees counterclockwise, reduces it, and performs image processing to move it to a predetermined position to obtain a display image. In the example of FIG. 19G, the display control unit 53 performs an image process for inverting the object part in which the object of the transfer image is reflected in the vertical direction to obtain a display image.

  Note that the display control unit 53 enlarges the object part in which the object of the transfer image is reflected at an arbitrary enlargement ratio or reduction at an arbitrary reduction ratio, rotates at an arbitrary rotation angle, You may make it perform the image processing which combined multiple of the movement to a position, and the reversal of the left-right direction, or the reversal of an up-down direction.

  Further, as shown in FIG. 20, the display control unit 53 synthesizes a plurality of images obtained by performing different image processing on the object portion in which the object of the transfer image is reflected to form a display image. Also good.

  The input / output display 11 irradiates, for example, light on an object close to the input / output display 11 and receives light reflected by the object and returning. In such a case, an object that easily reflects light of a frequency that is easily received by the input / output display 11 is used as an object to be transferred. The input / output display 11 may irradiate invisible light such as infrared light as light to irradiate a nearby object. By doing so, it becomes possible to separately handle the visible light irradiated by the input / output display 11 for displaying an image and the light for irradiating the object.

  The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a program recording medium in a general-purpose personal computer or the like.

  FIG. 21 is a block diagram illustrating a configuration example of hardware of a computer that executes the above-described series of processing by a program.

  In the computer, a CPU 191, a ROM (Read Only Memory) 192, and a RAM 193 are connected to each other by a bus 194.

  An input / output interface 195 is further connected to the bus 194. The input / output interface 195 includes an input unit 196 composed of a keyboard, mouse, microphone, etc., an output unit 197 composed of a display, a speaker, etc., a storage unit 198 composed of a hard disk or a non-volatile memory, and a communication unit 199 composed of a network interface. A drive 200 for driving a removable medium 201 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is connected.

  In the computer configured as described above, the CPU 191 loads the program stored in the storage unit 198 into the RAM 193 via the input / output interface 195 and the bus 194 and executes the program, for example. Is performed.

  The program executed by the computer (CPU 191) is, for example, a magnetic disk (including a flexible disk), an optical disk (CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc), etc.), a magneto-optical disk, or a semiconductor. The program is recorded on a removable medium 201 which is a package medium including a memory or the like, or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

  The program can be installed in the storage unit 198 via the input / output interface 195 by attaching the removable medium 201 to the drive 200. In addition, the program can be received by the communication unit 199 via a wired or wireless transmission medium and installed in the storage unit 198. In addition, the program can be installed in the ROM 192 or the storage unit 198 in advance.

  The program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.

  The embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention.

It is a figure which shows a mode that the object which is approaching an input-output display is transcribe | transferred. It is a figure which shows the example of a transfer image (display image). It is a figure which shows the example of an external appearance structure of the display apparatus 1 to which this invention is applied. 3 is a diagram illustrating an example of an internal configuration of a display device 1. FIG. It is a flowchart for demonstrating operation | movement of the display apparatus 1 in a 1st transfer image production | generation display process. 3 is a diagram illustrating an example of an external configuration of a display device 71. FIG. 4 is a diagram illustrating an internal configuration example of a display device 71. FIG. It is a flowchart for demonstrating operation | movement of the display apparatus 71 in a 2nd transfer image production | generation display process. 3 is a diagram illustrating a configuration example of a display device 111. FIG. It is a flowchart for demonstrating operation | movement of the display apparatus 111 in a 3rd transfer image production | generation display process. It is a figure for demonstrating the difference calculation by the difference calculating part 131A. It is a figure which shows the example of the transfer image (display image) of a gray image. It is a figure which shows the example of the transfer image (display image) of a color image. It is a figure for demonstrating the conditions which restrict | limit the production | generation of a transfer image. 3 is a diagram illustrating a configuration example of a display device 151. FIG. It is a figure which shows the example of a frame-shaped pattern. It is a flowchart for demonstrating operation | movement of the display apparatus 151 in a 4th transfer image production | generation display process. FIG. 6 is a diagram illustrating a state in which a display image based on a transfer image is displayed on a PC 181 connected to a display device 151. 6 is a diagram illustrating an example of image processing by a display control unit 53. FIG. 6 is a diagram illustrating an example of image processing by a display control unit 53. FIG. It is a block diagram which shows the structural example of the hardware of the computer to which this invention is applied.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Display apparatus, 11 Input / output display, 11A Optical sensor, 12 Capture button, 31 Input / output panel, 32 Control part, 41 Light reception signal processing part, 42 Display signal processing part, 51 Trigger detection part, 52 Image generation part, 53 Display Control unit, 54 image memory, 71 display device, 81 control unit, 91 trigger detection unit, 111 display device, 121 control unit, 131 trigger detection unit, 131A difference calculation unit, 131B trigger detection unit, 151 display device, 161 control unit , 171 trigger detection unit, 181 PC, 181A display, 182 network, 191 CPU, 192 ROM, 193 RAM, 194 bus, 195 input / output interface, 196 input unit, 197 output unit, 198 storage unit, 199 communication unit, 200 drives, 201 removable media

Claims (44)

In a display device comprising display light receiving means for displaying an image and receiving light from the outside,
Trigger detection means for detecting an input to the input unit or an input to a predetermined area of the display light receiving means as a trigger;
An image generating means for generating a transfer image showing an object close to the display light receiving means from a light receiving image based on light received by the display light receiving means based on the trigger detected by the trigger detecting means; ,
A display device comprising: display control means for causing the display light receiving means to display a display image based on the transfer image generated by the image generating means. The input unit is a button,
The display device according to claim 1, wherein the input to the input unit is an input made by a user pressing the button or releasing it after pressing the button. The image generating means includes
The received image of the frame when the trigger is detected,
The received light image n (n> 0) frames before the frame when the trigger is detected;
The received light image m (m> 0) frames after the frame when the trigger is detected;
An image obtained by averaging the received light images of a predetermined number of frames before the frame when the trigger is detected;
An image obtained by weighted averaging the received light images of a predetermined number of frames before the frame when the trigger is detected;
An image obtained by averaging the received light images of a predetermined number of frames after the frame when the trigger is detected,
An image obtained by weighted averaging the received light images of a predetermined number of frames after the frame when the trigger is detected;
An image obtained by averaging the received light images of a predetermined number of frames including a frame when the trigger is detected,
The display device according to claim 1, wherein the transfer image is generated from a processed image that is an image obtained by weighted averaging the received light images of a predetermined number of frames including a frame when the trigger is detected. The image generating means includes
Only the object part having the largest area among the object parts in which the object is reflected in the processed image,
Only the object part whose area of the object part in which the object in the processed image is reflected is a predetermined threshold or more,
In the processed image, the area of the object part in which the object is reflected is only the top N object parts,
Alternatively, the display device according to claim 3, wherein the entire processed image is extracted as the transfer image. The display control means, for the object portion where the object of the transfer image is reflected,
Enlargement at any enlargement ratio or reduction at any reduction ratio,
Rotation at any rotation angle,
Move to any position,
2. The display device according to claim 1, wherein the display light receiving unit displays the display image as the display image by performing image processing combining one or more of left-right reversal or vertical reversal. The display device according to claim 5, wherein the display control unit synthesizes a plurality of images obtained by performing the different image processing on an object portion in which an object of the transfer image is reflected, and forms the display image. The display device according to claim 1, wherein the light reception image is a binary image, a gray image, or a color image. The display device according to claim 1, wherein the display light receiving unit irradiates the object with light and receives light reflected by the object and returning. The display device according to claim 8, wherein the object is one that easily reflects light having a frequency that is easily received by the display light receiving unit. The display device according to claim 8, wherein the display light receiving unit emits infrared light. In a control method of a display device including a display light receiving means for displaying an image and receiving light from the outside,
A trigger detection step for detecting, as a trigger, an input to an input unit or an input to a predetermined region of the display light receiving unit;
Image generation for generating a transfer image in which an object close to the display light receiving unit is shown from a light reception image based on light reception by the display light receiving unit based on the trigger detected by the processing of the trigger detection step Steps,
A display control step of causing the display light receiving means to display a display image based on the transfer image generated by the processing of the image generation step. A program that causes a computer to execute control processing of a display device that includes a display light receiving unit that displays an image and receives light from the outside,
A trigger detection step for detecting, as a trigger, an input to an input unit or an input to a predetermined region of the display light receiving unit;
Image generation for generating a transfer image in which an object close to the display light receiving unit is shown from a light reception image based on light reception by the display light receiving unit based on the trigger detected by the processing of the trigger detection step Steps,
A program for causing a computer to execute a control process including: a display control step for causing the display light receiving means to display a display image based on the transfer image generated by the process of the image generation step. In an input device having a light receiving means for receiving light from the outside,
Detected by using, as a trigger, a light reception image based on light reception by the light receiving means, and a difference value indicating a difference between the light reception images of a frame at that time and a frame temporally close to the frame within a predetermined range Trigger detecting means for
An input device comprising: an image generation unit configured to generate a transfer image in which an object close to the light receiving unit is shown based on the trigger detected by the trigger detection unit. The input device according to claim 13, wherein the trigger detection unit detects, as a trigger, that the difference value is within the predetermined range for n consecutive frames. The image generating means includes
The received image of the frame when the trigger is detected,
The received light image n (n> 0) frames before the frame when the trigger is detected;
The received light image m (m> 0) frames after the frame when the trigger is detected;
An image obtained by averaging the received light images of a predetermined number of frames before the frame when the trigger is detected;
An image obtained by weighted averaging the received light images of a predetermined number of frames before the frame when the trigger is detected;
An image obtained by averaging the received light images of a predetermined number of frames after the frame when the trigger is detected,
An image obtained by weighted averaging the received light images of a predetermined number of frames after the frame when the trigger is detected;
An image obtained by averaging the received light images of a predetermined number of frames including a frame when the trigger is detected,
The input device according to claim 13, wherein the transfer image is generated from a processed image that is an image obtained by weighted averaging the received light images of a predetermined number of frames including a frame when the trigger is detected. The image generating means includes
Only the object part having the largest area among the object parts in which the object is reflected in the processed image,
Only the object part whose area of the object part in which the object in the processed image is reflected is a predetermined threshold or more,
In the processed image, the area of the object part in which the object is reflected is only the top N object parts,
The input device according to claim 15, wherein the entire processed image is extracted as the transfer image. The input device according to claim 13, further comprising storage means for storing the transfer image generated by the image generation means. A storage device is connected to the input device,
A transmission means for transmitting the transfer image generated by the image generation means to the storage device;
The input device according to claim 13, wherein the storage device receives and stores the transfer image transmitted from the transmission unit of the input device. A display device is connected to the input device,
A transmission means for transmitting the transfer image generated by the image generation means to the display device;
The input device according to claim 13, wherein the display device receives the transfer image transmitted from the transmission unit of the input device, and displays a display image based on the transfer image on a display unit. The input device according to claim 13, further comprising a display control unit that causes a display unit to display a display image based on the transfer image generated by the image generation unit. The display control means, for the object portion where the object of the transfer image is reflected,
Enlargement at any enlargement ratio or reduction at any reduction ratio,
Rotation at any rotation angle,
Move to any position,
The input device according to claim 20, wherein image processing is performed by combining one or more of left-right direction inversion and up-down direction inversion, and the display image is displayed on the display unit. The input device according to claim 21, wherein the display control unit synthesizes a plurality of images obtained by performing different image processing on an object portion in which an object of the transfer image is reflected, and forms the display image. The input device according to claim 13, wherein the light reception image is a binary image, a gray image, or a color image. When the image generation means generates the transfer image, the pixel value of the received light image in a portion corresponding to a predetermined size portion including the object portion in which the object is reflected in the transfer image is within a predetermined range. The input device according to claim 13, wherein the transfer image is not generated until. The input device according to claim 13, wherein the display light receiving unit irradiates the object with light and receives light reflected by the object and returning. The input device according to claim 25, wherein the object is one that easily reflects light having a frequency that is easily received by the display light receiving means. The display device according to claim 25, wherein the display light receiving unit emits infrared light. In the control method of the input device including the light receiving means for receiving light from the outside,
Detected by using, as a trigger, a light reception image based on light reception by the light receiving means, and a difference value indicating a difference between the light reception images of a frame at that time and a frame temporally close to the frame within a predetermined range Trigger detection step to perform,
An image generation step of generating a transfer image showing an object close to the light receiving means from the light reception image based on the trigger detected by the processing of the trigger detection step. A program that causes a computer to execute control processing of an input device including a light receiving unit that receives light from the outside,
Detected by using, as a trigger, a light reception image based on light reception by the light receiving means, and a difference value indicating a difference between the light reception images of a frame at that time and a frame temporally close to the frame within a predetermined range Trigger detection step to perform,
Based on the trigger detected by the processing of the trigger detection step, an image generation step of generating, from the received light image, a transfer image showing an object in proximity to the light receiving means is performed on a computer. The program to be executed. In an input device having a light receiving means for receiving light from the outside,
Trigger detecting means for detecting a trigger by detecting a frame-like pattern from a light-receiving image based on light reception by the light-receiving means;
An input device comprising: an image generation unit configured to generate a transfer image in which an object close to the light receiving unit is shown based on the trigger detected by the trigger detection unit. The image generating means includes
The received image of the frame when the trigger is detected,
The received light image n (n> 0) frames before the frame when the trigger is detected;
The received light image m (m> 0) frames after the frame when the trigger is detected;
An image obtained by averaging the received light images of a predetermined number of frames before the frame when the trigger is detected;
An image obtained by weighted averaging the received light images of a predetermined number of frames before the frame when the trigger is detected;
An image obtained by averaging the received light images of a predetermined number of frames after the frame when the trigger is detected,
An image obtained by weighted averaging the received light images of a predetermined number of frames after the frame when the trigger is detected;
An image obtained by averaging the received light images of a predetermined number of frames including a frame when the trigger is detected,
31. The input device according to claim 30, wherein the transfer image is generated from a processed image that is an image obtained by weighted averaging the received light images of a predetermined number of frames including a frame when the trigger is detected. 32. The input device according to claim 31, wherein the image generation unit extracts, as the transfer image, only an object part in which the object is present inside the frame-shaped pattern in the processed image. The input device according to claim 30, further comprising storage means for storing the transfer image generated by the image generation means. A storage device is connected to the input device,
Further comprising transmission means for transmitting the transfer image generated by the image generation means to the storage device,
The input device according to claim 30, wherein the storage device receives and stores the transfer image transmitted from the transmission unit of the input device. A display device is connected to the input device,
A transmission means for transmitting the transfer image generated by the image generation means to the display device;
The input device according to claim 30, wherein the display device receives the transfer image transmitted from the transmission unit of the input device and displays a display image based on the transfer image on a display unit. The input device according to claim 30, further comprising display control means for causing a display unit to display a display image based on the transfer image generated by the image generation means. The display control means, for the object portion where the object of the transfer image is reflected,
Enlargement at any enlargement ratio or reduction at any reduction ratio,
Rotation at any rotation angle,
Move to any position,
37. The input device according to claim 36, wherein image processing is performed by combining one or more of left-right reversal or up-down reversal and displayed on the display unit as the display image. The input device according to claim 37, wherein the display control unit combines the plurality of images obtained by performing the different image processing on an object portion in which the object of the transfer image is reflected to form the display image. The input device according to claim 30, wherein the light reception image is a binary image, a gray image, or a color image. The input device according to claim 30, wherein the display light receiving unit irradiates the object with light and receives light reflected by the object and returning. 41. The input device according to claim 40, wherein the object is one that easily reflects light having a frequency that is easily received by the display light receiving means. The display device according to claim 40, wherein the display light receiving unit irradiates infrared light. In the control method of the input device including the light receiving means for receiving light from the outside,
A trigger detection step of detecting a trigger by detecting a frame-like pattern from a light-receiving image based on light reception by the light-receiving means;
An image generation step of generating a transfer image showing an object close to the light receiving means from the light reception image based on the trigger detected by the processing of the trigger detection step. A program that causes a computer to execute control processing of an input device including a light receiving unit that receives light from the outside,
A trigger detection step of detecting a trigger by detecting a frame-like pattern from a light-receiving image based on light reception by the light-receiving means;
Based on the trigger detected by the processing of the trigger detection step, an image generation step of generating, from the received light image, a transfer image showing an object in proximity to the light receiving means is performed on a computer. The program to be executed.

Images