JP4852634B2 - External device cooperation apparatus and method and external device cooperation control program - Google Patents

Description

  The present invention relates to an external device cooperation apparatus and method and an external device cooperation control program, and in particular, an external device cooperation device for recognizing a gesture based on an image photographed using a photographing unit and operating the external device, and The present invention relates to a method and an external device cooperation control program.

  In fields such as interactive digital signage, attention has been focused on systems that detect changes in objects and people and change information presented interactively based on the changes. Such a system generally takes a technique that makes the person interested in the experience on the spot by changing the information display according to the movement of the person's limbs, and attracts interest to the information presented as a result ( For example, see Patent Document 1). For this reason, new stimuli are always required, and it is necessary to constantly evolve actions for making the user interested and effects provided by the system.

Japanese Patent Laid-Open No. 20056-215863

  However, the experience provided by the interactive digital signage system is limited to that place, and the only way to maintain interest in information is to rely on human memory. It was difficult to do. The challenge was to make users who have finished using interactive digital signage such as shopping malls interested in the information presented continuously.

  The present invention has been made in view of the above points, and an object of the present invention is to provide an external device cooperation apparatus and method and an external device cooperation control program capable of continuously appealing information presented for a long time to a user.

  FIG. 1 is a principle configuration diagram of the present invention.

The present invention (claim 1) is an external device linkage apparatus for controlling an external device,
Image recognition means 110 including means for acquiring an image from a connected camera, extracting a specific object region from the image in real time, and instructing projection of the video to the specific object;
Storage means for storing video content in advance;
A video display unit that acquires video content from the storage unit and displays the video content on the specific object when the image recognition unit 110 instructs to project a video onto the specific object;
Detecting that a particular object in the recognized finger position by image recognition unit 110 has accumulated over a specified time, the gesture recognition unit 120 for generating a trigger at the time,
Based on the trigger by the gesture recognition unit 120 has, for executing the specified operation of the external equipment, the external interface unit 140 for transmitting and receiving data corresponding to a video content and.

Further, the present invention (Claim 2) is the external device cooperation apparatus according to Claim 1,
Means for displaying on the specific object an icon corresponding to the processing status of the external device.

  FIG. 2 is a diagram for explaining the principle of the present invention.

The present invention (claim 3) is an external apparatus cooperative method for controlling an external device,
Images recognition means obtains an image from a connected camera, the extracts from the image the specific object region in real time (step 1), an image including a means for indicating the projected image with respect to the specific object A recognition step;
When the video display means is instructed to project the video on the specific object, the video display step acquires the video content from the storage means storing the video content and displays the video content on the specific object; and
Gesture recognition means, and detects that the specific object in the recognized specified position by the image recognition step has accumulated over a specified time, the gesture recognition step of generating a trigger at that time (Step 2),
External interface means, based on the trigger generated by the gesture recognition step, for performing a specified operation of the external equipment, transmitting and receiving transmitted and received between the data corresponding to the video content to an external device (step 3) And do.

Further, the present invention (Claim 4 ) is the external device cooperation method according to Claim 3,
An icon corresponding to the processing status of the external device is displayed on the specific object.

The present invention (Claim 5 ) is an external device linkage control program for causing a computer to function as each means constituting the external device linkage apparatus according to Claim 1 or 2 .

  As described above, according to the present invention, a virtual object such as video content displayed on the palm of the hand is converted into a physical object such as a printed card and brought back home. The user who uses the signage can recall the memory of the experience, and can continue to appeal the information presented for a long time to the user.

It is a principle block diagram of this invention. It is a figure for demonstrating the principle of this invention. It is a block diagram of the external apparatus cooperation apparatus in one embodiment of this invention. It is a figure which shows the relationship between the video content and print content in one embodiment of this invention. It is a flowchart of the process in one embodiment of this invention. It is a figure which shows the mode of printing in one embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  The present invention makes it possible to output a physical object having an entity from an experience with a virtual object and take it home. Specifically, image content corresponding to video content displayed on the palm can be printed on a print medium (for example, paper).

  FIG. 3 shows a configuration of the external device cooperation apparatus according to the embodiment of the present invention.

  The external device cooperation apparatus 100 shown in the figure is a personal computer (PC), and an imaging unit such as a camera 10 and an output unit such as a printer 20 and a projector 30 are connected as external devices.

  The external device cooperation apparatus 100 includes an image recognition unit 110, a gesture recognition unit 120, a video display unit 130, an external interface unit 140, and a storage unit 150. The storage unit 150 is a storage medium such as a hard disk and stores video content.

  The image recognition unit 110 continues to extract a specific object region such as a palm in real time from an image captured using an imaging unit such as the camera 10.

  The video display unit 130 continuously projects the video content extracted from the storage unit 150 onto the palm.

  For example, when the image recognizing unit 110 detects that the palm has moved to a designated position described as “print”, the gesture recognizing unit 120 detects that the user has stayed for a designated time of 2 seconds. A trigger for interlocking with an external device (for example, the printer 30) is generated.

  The external interface unit 140 transmits and receives data for executing an arbitrary designated operation to an external device such as the printer 30. Thereby, a virtual name object projected and displayed on a specific object such as a palm can be taken out as a physical object having an entity.

  This will be specifically described below.

  As a specific example, as shown in FIG. 4, it is assumed that a print content file name corresponding one-to-one with a palm and video is acquired and output from the set printer 30. Correspondence between the video content being displayed, the print content, and the output destination printer is performed in the print content definition file. Further, during the execution of the content printing process, an image indicating that printing is in progress may be projected onto the palm of the hand while reproducing the sound effects. It is assumed that a plurality of printers that print print contents can be set. The target printer is assumed to have a corresponding driver installed and recognized as a printer device by the OS.

  In the print content definition file, the paper size and orientation (vertical, horizontal) for each printer are described in advance. Also, the settings (paper type, paper feed method, full screen printing, etc.) depending on the printer model are not set, and the printer default settings are used as they are. It is assumed that printing is performed in a printable area of the set paper at the maximum size that maintains the aspect ratio.

  Assuming that the above settings have been made, the printer 30 executes print processing in response to a notification from the external interface unit 140. Alternatively, the number of executable print jobs may be set in the action definition file, and processing may not be performed when the number of jobs exceeds the set number.

  In addition, if the content for printing corresponding to the video content being displayed on the palm is not defined, no processing is performed.

  When printing by the printer 30 is started, an icon corresponding to the processing status is projected on the palm. The projected printing situation is, for example, the following three patterns.

• Print spooling-if the print job is spooled and waiting to be printed;
-Printing process in progress-When printing process is in progress;
-Error-When a printing error occurs (deletes after a specified time)
After the printing is finished, the sound effect for the end is reproduced, and the original video content that has been displayed is displayed again on the palm.

  FIG. 5 shows the flow of the above processing, and FIG. 6 shows the state of printing. In the following example, as shown in FIG. 6, it is assumed that the external device includes a camera 10, a projector 20, and a printer 30 in the same casing.

  Step 101) When the palm of the hand stays on the external device 10 for an arbitrary time, the camera 10 of the external device captures the palm, and the image recognition unit 110 tracks the captured image of the palm as a specific object. To do. Thereby, the gesture recognition unit 120 detects the retention of the palm and causes the external interface 140 to generate a trigger. The palm image is taken with a video camera. For example, an image of about 30 frames per second is obtained using an IEEE1394 connection color camera. In addition, the video display unit 130 acquires video content corresponding to the situation from the storage unit 150 and outputs the video to the palm of the user's hand using the projector 20. It should be noted that video content corresponding to the situation can be acquired from the storage unit 150 by describing a conversion table in an XML format setting file.

  Specifically, in the example of FIG. 6, the palm is placed over a mark described as “PRINT” for an arbitrary time (for example, 2 seconds) or longer, and the image recognition unit 110 uses the palm as a specific object. Extract. Then, when the gesture recognition unit 120 detects that the extracted specific object stays for a predetermined time (for example, 2 seconds or more), a trigger is generated.

  Step 102) Upon receiving a trigger from the gesture recognition unit 120, the external interface 140 determines whether the print content corresponding to the content projected and displayed by the projector 20 exists in the storage unit 150. Exit.

  Step 103) If there is a print content corresponding to the content displayed in Step 102, the print job is spooled.

  Step 104) The video display unit 130 displays the icon being spooled on the palm. More specifically, the external interface 140 inquires of the printer about the situation, and if there is a print job being executed, it is spooled, and the icon is also displayed as being spooled. When there is no print job being executed, the same processing as usual (displaying a printer icon and issuing a print job and returning to the original video content when the job transfer is completed) is performed.

  Step 105) The external interface 140 instructs the printer 30 of the external device to print the print content acquired from the storage unit 150. As a result, the printer 30 starts printing the designated content. When the image is printed, as shown in FIG. 5, an image related to the content is printed on the card from the bottom of the printer 30, and the user can take the card home.

  Step 106) A printing icon is displayed.

  Step 107) It is determined whether printing has succeeded. If successful, the process proceeds to Step 108. If unsuccessful, the process proceeds to Step 109.

  Step 108) A print error icon is displayed and the process is terminated.

  Step 109) The original content is displayed and the process is terminated.

  Also, the operation of each component of the external device cooperation apparatus 100 shown in FIG. 3 is constructed as a program and installed and executed on a computer used as the external device cooperation apparatus, or distributed through a network. Is possible.

  Further, the constructed program can be stored in a portable storage medium such as a hard disk, a flexible disk, or a CD-ROM, and can be installed or distributed in a computer.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

  The present invention can be applied to a technique for changing information to be provided interactively based on a change in the motion of an object or a person. Specifically, the present invention can be applied to a case where novelty goods are distributed at a product PR event or the like. By applying the present invention, one's original novelty goods can be distributed, and the appeal of messages that companies want to convey to users can be improved.

10 Shooting means, camera (external equipment)
20 Projector (external device)
30 Printer (external device)
DESCRIPTION OF SYMBOLS 100 External apparatus cooperation apparatus 110 Image recognition means, Image recognition part 120 Gesture recognition means, Gesture recognition part 130 Image | video display part 140 External interface means, External interface part 150 Storage part

Claims (5)

An external device linkage apparatus for controlling an external device,
Image recognition means including means for acquiring an image from a connected camera, extracting a specific object region from the image in real time, and instructing the specific object to project a video;
Storage means for storing video content in advance;
A video display unit that acquires video content from the storage unit and displays the video content on the specific object when the image recognition unit instructs the projection of the video onto the specific object;
Detecting that a particular object in the recognized specified position by the image recognition means is retained over a specified time, the gesture recognition unit for generating a trigger at the time,
Based on the trigger by the gesture recognition means, for performing a specified operation of the external equipment, and external interface means for transmitting and receiving data corresponding to the video content,
An external device cooperation apparatus characterized by comprising: In the external device cooperation apparatus according to claim 1,
The external device cooperation apparatus according to claim 1 , further comprising means for causing the specific object to display an icon corresponding to the processing status of the external device. An external apparatus cooperative method for controlling an external device,
Images recognition means obtains an image from a connected camera, extracts a specific object region in real time from within the image, and the image recognition step comprising means for indicating the projected image with respect to the specific object,
When the video display means is instructed to project a video on the specific object, the video display step acquires the video content from the storage means storing the video content and displays the video content on the specific object; and
Gesture recognition means detects that a particular object in the recognized specified position by the image recognition step has accumulated over a specified time, the gesture recognition step of generating a trigger at the time,
External interface means, on the basis of the trigger generated by the gesture recognition step, for performing a specified operation of the external equipment, and transmitting and receiving step for transmitting and receiving between the data corresponding to the video content to an external device,
The external device cooperation method characterized by performing. In the external device cooperation method according to claim 3,
An icon corresponding to the processing status of the external device is displayed on the specific object.
The external device cooperation method according to claim 3. External cooperative control program for causing a computer to function as each means constituting the external device linkage apparatus according to claim 1 or 2.