JP2015080265A - Apparatus selection system, apparatus selection device, and apparatus selection program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an apparatus selection system allowing plural users to efficiently use specific apparatuses under control.SOLUTION: The apparatus selection system includes: plural imaging units that take images of plural users 81-84 issuing instruction signals to plural apparatuses under control; a user identification section that identifies each of the users based on pieces of image information taken by the imaging units; an apparatus identification section that identifies apparatuses under control based on the orientation of the users taken by the imaging units; a signal determination section that receives instruction signals issued by the users to determine the instruction signals; an instruction signal setting section that makes different settings to plural functions which are executable by the apparatus under control; and a control section that gives the users identified by the user identification section an authority to carry out functions on the respective apparatuses under control based on instruction signals determined by the signal determination section and the signals for the functions set by the instruction signal setting section.

Description

  In the embodiment described in this specification, a plurality of information processing devices such as a multifunction peripheral (MFP) having a plurality of functions are set as control target devices, and a plurality of users are identified using a technique such as a spatial hand gesture. The present invention relates to a technique for acquiring the right to execute different functions in a control target device.

  Conventionally, in order for one operator to select one device from a plurality of devices and perform control such as ON-OFF, a selection method for selecting the device to be controlled, and the selected control target device As an operation method, a method using a spatial hand gesture has been proposed (Patent Document 1).

  In these methods, a spatial hand gesture performed by the operator is captured by the camera, and the device selected by the operator is recognized by image processing. These devices to be controlled are a lighting fixture, a curtain opening / closing drive unit, a CD player, a television, and a video, and operate different types of devices with one operator, and the operator is not selected.

JP 2009-037344 A

  An object of the embodiment described in the present specification is to provide a device selection system, a device selection device, and a device selection program that can efficiently use a specific device to be controlled by a plurality of users.

  The device selection system according to the present embodiment connects at least a plurality of devices to be controlled and a server via a network, and is based on an instruction signal issued from a user to a device to be controlled having a plurality of executable functions. A device selection system that allows a specific user to acquire a right to execute different functions, a photographing unit that photographs a plurality of users that issue instruction signals to the plurality of control target devices, and the photographing unit A user identifying unit that identifies each user that emits an instruction signal based on the shooting information captured by the camera, a device identifying unit that identifies a control target device based on the orientation of the user captured by the imaging unit, and an instruction signal issued by the user A signal determination unit that determines an instruction signal, and a plurality of executable devices Specified by the user specifying unit based on the instruction signal setting unit for setting different signal contents for the function, the instruction signal determined by the signal determination unit, and the signal content for the function set by the instruction signal setting unit And a control unit that obtains each user with the authority to execute a different function on the control target device specified by the user.

  The device selection apparatus according to the present embodiment is connected to at least a plurality of control target devices via a network, and is specific to a control target device having a plurality of executable functions based on an instruction signal issued from a user. A device selection device that allows a user to acquire an authority to execute different functions, and each user that issues an instruction signal based on information captured by a plurality of users that issue an instruction signal to the plurality of devices to be controlled A user specifying unit for specifying the image, a device specifying unit for specifying the device to be controlled based on the orientation of the imaged user, a signal determining unit for receiving a captured user instruction signal and determining the instruction signal An instruction signal setting for setting different signal contents for a plurality of functions executable by the control target device Control unit specified by the user for each user specified by the user specifying unit based on the signal, the instruction signal determined by the signal determining unit, and the signal content for the function set by the instruction signal setting unit And a control unit that obtains authority to cause each device to execute different functions.

  The device selection program according to the present embodiment performs processing for acquiring the authority to allow a specific user to execute a different function for a specific control target device based on an instruction signal issued from the user, with at least a plurality of control target devices. A program for causing a device selection device connected via a network to execute an instruction signal based on information captured by a plurality of users who issue instruction signals to the plurality of control target devices. A plurality of devices that can be executed by the control target device by specifying each user, specifying the control target device based on the orientation of the imaged user, receiving the captured user instruction signal, determining the instruction signal Different signal contents are set for each function, and the determined instruction signal and the set function The device selection apparatus is configured to execute a process for acquiring, for each identified user, authority to execute different functions for each identified device to be controlled based on the signal content to be performed. .

The figure which shows the whole structure of the selection system of the apparatus by this embodiment. The front view of the apparatus of FIG. FIG. 3 is a detailed view of the photographing unit in FIG. 2. The control block diagram of the server shown in FIG. The flowchart which shows the flow of the initial process by the server shown in FIG. The flowchart which shows the flow of the calling process of FIG. The flowchart which shows the flow of the selection process of FIG. 6 is a flowchart showing a flow of operation processing in FIG. 5. The figure which shows an example of the definition of gesture. The figure which shows an example of the definition of the gesture with respect to each control object apparatus for every user.

  Hereinafter, a device selection system according to an embodiment will be described with reference to the drawings.

  FIG. 1 is a diagram showing the overall configuration of a device selection system, and FIG. 2 is a front view of the device.

  In FIG. 1, a plurality of information processing devices 11, 12, and 13 that constitute a control target device group 10, a plurality of personal computers (PCs) 31, 32, and 33, a server 40, and a plurality that constitutes an external camera group 60. The cameras 61 and 62 are connected to the network 70.

  The plurality of information processing apparatuses 11, 12, and 13 will be described below as a multifunction peripheral (MFP) having a plurality of functions such as a scanner function, a printer function, a Web browser function, and a FAX function. The plurality of MFPs 11, 12, and 13 may be arranged in one place in the office, for example, or may be arranged in appropriate places. Further, not all MFPs may have the same function. For example, only one MFP includes a developing device that separately accommodates both a decolorable toner and a non-decolorable toner. It is good.

  As shown in FIG. 2, the MFPs 11, 12, and 13 are configured such that, in this embodiment, the image forming unit 14, the paper storage unit 15 that stores printing paper is stored in the apparatus body, and the paper in the paper storage unit 15 is stored in the image forming unit 14. A paper transport unit 16 for transporting and discharging out of the apparatus is disposed. Further, the main body of the apparatus includes a scanner unit 18 for reading a document from the automatic document feeder 17 and a document placed on the platen glass, an operation panel 19 having an operation unit and a display unit, and operations such as a spatial hand gesture. A gesture recognition unit 20 for recognizing a person's gesture, a network interface 21, a speaker 22, and a card authentication unit 23.

  The operation panel 19 is used to operate the MFP 11 (12, 13). For example, in the case of network printing, a job is temporarily stored in a queue. The user releases the job stored in the queue by operating the operation panel 19. This function of temporarily storing print jobs in a queue is effective as a means for preventing information leakage. In other words, when printing is started as soon as a job is received without being stored, if there is a state in which the printed matter is left on the paper discharge unit of the MFP for a long time, it is touched by a third party in the meantime. There is a fear. For this reason, when the operator places his authentication card over the card authentication unit 23 of the MFP selected by the operator, the print job of the operator once stored in the server 40 is printed from the selected MFP. FIG. 1 shows a state in which the D operator 84 holds his / her authentication card over the card authentication unit 23 and waits for printing out.

  On the other hand, the operators 81, 82, 83 seated at the desks on which the PCs 31, 32, 33 are placed and the MFPs 11, 12, 13 are AC operators by the gesture recognition unit 20 of the MFPs 11, 12, 13. As a range in which the hand gestures 81, 82, and 83 can be recognized, the range is, for example, within a range of several meters to several tens of meters. The A user 81 and the PC 31, the B user 82 and the PC 32, and the C user 83 and the PC 33 are associated with each other, and data for printing from the PCs 31 to 33 is stored in the server 40.

  The gesture recognizing unit 20 is provided with a color camera 203 and an infrared camera 204 for recognizing the depth of a captured image on a horizontal bar 202 provided at the upper end of the column 201. In addition, a notification lamp 205 is provided at the upper end of the column 201. When the gesture recognition unit 20 of the MFP detects a gesture, the notification lamp 205 of the MFP is turned on to notify the operator of the control state for the MFP that is the control target device. For example, when indicating that it is a call target, the lamp is lit to indicate that it has been selected as an operation target. In the present embodiment, the notification lamp 205 blinks when it is a call target, is turned on when it is selected, and is turned off when it is in a standby state. At that time, the control state may be notified by voice through the speaker 22.

  The color camera 203 and the infrared camera 204 capture the inside of the office in a wide angle range, and can perform face authentication of a plurality of operators 81 to 83 at a time. As shown in FIG. 3, the gesture recognizing unit 20 may be configured separately from the MFPs 11 to 13. For example, the gesture recognizing unit 20 is provided with a column 201 on the base 207 and a microphone on a horizontal bar 202 attached to the top of the column 201. 206 is further provided. The microphone 206 is used for voice recognition of the operator.

  In the present embodiment, by arranging the gesture recognition unit 20 in each MFP, when the operators 81 to 83 perform a gesture toward the MFP for which the operation authority is desired to be acquired, the intention of acquisition is obtained by the cameras 203 and 204 of the MFP. You can recognize that there is. The color camera 203 acquires moving image data for face recognition of the operators 81-83. The operator is photographed as a moving image, and the infrared camera 204 acquires data for specifying the depth of the subject and the gesture.

  In addition, a plurality of cameras 51 and 52 constituting the external camera group 50 are appropriately arranged on the ceiling or wall of the office, the personal computers 31, 32, and 33, and the operator is photographed, so that the camera of the gesture recognition unit 20 is taken. Can be used as an aid to the gesture recognition unit 20.

  As shown in FIG. 4, the server 40 includes a processor 41, a storage unit 42, an output unit 43 having a display 431 and a speaker 22, an input unit 44, a network interface unit 45, a plurality of individual identification units 46, and a plurality of simultaneous gestures. A detection unit 47, a gesture determination unit 48, a device selection availability determination unit 49, a device selection completion notification unit 50, and a device operation unit 51 corresponding to the gesture are connected to the line 52.

  The processor 41 makes various determinations in the server 40 and issues instructions to each unit. The storage unit 42 stores information used by the processor 41 for control. The input unit 44 is an input unit of a user interface and receives operations from the keyboard 441 and the mouse 442 operated by the A user 81 to the C user 83. A display 431 of the output unit 43 is an output unit of a user interface, and displays information to the user. The network interface unit 45 inputs / outputs information between the MFPs 11 to 13 as control target devices and the network 70, and the MFPs 11 to 13 of the control target device group 10 and the cameras 61 and 62 of the external camera group 60 are connected. Connected.

  The multi-operator personal identification unit 46 performs face recognition processing and voice recognition based on shooting information from the camera, voice information from the microphone, and face image information and voice information of each of the users 81 to 83 registered in advance. Identify users by processing.

  The multiple simultaneous gesture detection unit 47 simultaneously detects a plurality of user gestures in real time.

  The gesture determination unit 48 determines whether the content of the gesture is “calling a device”, “selecting a device”, or “operating a device” based on, for example, a hand gesture performed by an image-processed user. For the determination, as shown in FIG. 9, the operation of the hand gesture and the definition of the control target device are controlled, and as shown in FIG. 10, the control target device (MFP) 11 for each of a plurality of AC users. A correspondence table in which operation contents are defined corresponding to different gestures every 13 is used. In FIG. 10, even if the contents of the gesture are different in the call, selection, and operation control items, the same control is executed if the user is different.

  The device selection availability determination unit 49 determines whether the control target device is in a state where it can be called or selected.

  The device selection completion notification unit 50 notifies the user that there is a device to be controlled that has been selected by a gesture. For example, the notification lamp 205 and the speaker 22 shown in FIG. Or, an instruction is given to play a voice message such as “Mr. XX, can be operated” from the speaker 22.

  The device operation unit 51 corresponding to the gesture instructs the MFPs 11 to 13 of the control contents defined according to the gesture to the MFPs 11 to 13 that are the devices to be controlled.

  Next, initial processing of the device selection system by the server 40 will be described based on the flowchart shown in FIG.

  The processor 41 of the server 40 receives the moving image from each gesture recognition unit 20 and the external camera group 60 of the control target device group 10 via the network interface unit 45 (ACT1), and proceeds to ACT2.

  In ACT2, the processor 41 determines whether or not a gesture is detected, and proceeds to ACT3 when a gesture is detected.

  In ACT3, the processor 41 specifies the number of users who are making a gesture using the individual specifying unit 46 of the plurality of operators, and proceeds to ACT4.

  In ACT4, the processor 41 determines whether or not the user is zero (0), and if not, proceeds to ACT5, otherwise returns to ACT1.

  In ACT5, when there are a plurality of users, it is determined whether or not the gesture of these users is a call to the control target device. If it is a call gesture, the process proceeds to ACT 6 to perform a call process for the control target device, and if it is not a call process, the process proceeds to ACT 7.

  In ACT7, it is determined whether or not the gesture is a selection gesture. If the gesture is a selection gesture, the process proceeds to ACT8 and the control target device is selected. If not, the process proceeds to ACT9 and the control target device is operated. When the processes of ACT6, ACT8, and ACT9 are completed, the process returns to ACT4.

  The calling process shown in ACT6 of FIG. 5 will be described based on the flowchart of FIG.

  When the calling process is started, in ACT 21, the processor 41 specifies who is calling which control target device from the photographing information of the camera, and proceeds to ACT 22. Here, the specification of the control target device is determined based on, for example, whether the orientation of the specified user's face is facing the control target device, and whether the specified user's gesture indicates the control target device. The number of control target devices to be called may be limited to one, and a plurality of devices in the vicinity thereof may be targeted. This can change the behavior according to the settings of the entire system. Further, the determination for specifying the user specifies the user who is performing the gesture from the face recognition or the position and angle of the camera and the seating chart.

  In ACT 22, the processor 41 inquires of the device to be controlled about the device state of the specified device to be controlled, and proceeds to ACT 23.

  In ACT23, when the device state satisfies the call execution condition, the process proceeds to ACT24, instructing the call target instruction to the control target apparatus, and then proceeds to ACT25. As a call response instruction, a patrol lamp blinks to indicate a call, or a sound indicating a call is output from the speaker 22.

  If the call execution condition is not satisfied in ACT 23, this processing operation is terminated. As a case where the call execution condition is not satisfied, for example, a case where the control target device has already been used by another user (during operation) can be exemplified.

  In ACT 25, the processor 41 records temporary information in the storage. For example, the control device number that issued the call instruction, the user number, and the device state = “calling” are recorded in the storage, and the call processing is terminated.

  The selection process shown in ACT8 of FIG. 5 will be described based on the flowchart of FIG.

  When the selection process is started, in ACT 31, the processor 41 specifies which control target device is selected from the imaging information of the camera, and proceeds to ACT 32. The identification method is the same as in the case of call processing.

  In ACT 32, the processor 41 inquires the record information of the control target device group temporarily stored from the storage, and proceeds to ACT 33.

  In ACT 33, if there is a record of the specified device to be controlled as a result of the inquiry and the device state is “calling”, the process proceeds to ACT 34. If the device state has been selected, this selection process is performed. finish.

  In ACT 34, the processor 41 instructs the selected control target device to perform a selection response instruction, and proceeds to ACT 35.

  In ACT 35, the processor 41 records temporary information in the storage. For example, the control target device number for which the call instruction is issued, the user number, and the device status “selected” are recorded in the storage. In addition, the control target device that has received the selection response instruction turns on the patrol lamp so as to indicate that it has been selected, or outputs a sound that indicates that it has been selected from the speaker 22, and ends this selection process.

  The operation process shown in ACT9 of FIG. 5 will be described based on the flowchart shown in FIG.

  When the operation process is started, the processor 41 specifies which control target device is to be operated from the photographing information of the camera, and proceeds to ACT42. The identification method is the same as in the case of call processing.

  In ACT 42, the processor 41 inquires the record information of the control target device group stored from the storage, and proceeds to ACT 43.

  In ACT 43, if there is a record of the specified control target device and the device state is “selected”, the process proceeds to ACT 44, and if not, this operation processing is terminated.

  In ACT 44, the control target device whose operation instruction is specified is instructed, and the process proceeds to ACT 45.

  In ACT 45, when the operation of the specified control target device ends, the processor 41 receives an operation execution end notification and proceeds to ACT 46.

  In ACT 46, the processor 41 deletes the information temporarily recorded in the storage and ends the operation process.

  The process shown in FIG. 5 will be described with reference to FIG. FIG. 9 shows information defining the relationship between the spatial hand gesture and the operation to be actually performed, and the operation is uniquely assigned to the spatial hand gesture.

  For example, when the A user 81 uses a plurality of MFPs 11, 12, and 13 to print, for example, the printing information of the A user 81 stored in the server 40, it is unknown which MFP can be used. Therefore, for example, in order to call the MFP 11 and the MFP 12 as candidates for use, for example, the A user 81 calls the MFP 11 MFP 12, which is a plurality of control target devices arranged slightly away from each other by shaking his / her hand. . The plurality of called MFPs 11 and 12 return a response (blinking the notification lamp 205). Next, when the user points to and designates the MFP 11, a selection process is executed, and the designated MFP 11 turns on the notification lamp 205. Then, the MFP 12 that has not been selected turns off the notification lamp 205.

  Since the A user 81 has selected the MFP 11, the A user 81 performs an operation process on the designated MFP 11 by performing a spatial hand gesture (putting his thumb up), and the server 40 receives the A user via the MFP. When the authentication information is received from the authentication card, the print job stored in the queue is printed from the MFP 11.

  In the present embodiment, spatial hand gestures are allocated as shown in FIG. 10 so that a plurality of users 81, 82, 83 can select a plurality of MFPs 11, 12, 13.

  Here, even if the A user 81 and the B user 82 call and select the same MFP 11, if the A user 81 wants to use the print function and the B user 82 uses the scanner function, a new gesture is made for a plurality of functions. Can be used at the same time.

  Further, when only the MFP 13 includes a developing unit containing the erasing toner, when the A user 81 and the B user 82 compete for the use of the erasing printing function, if the priority order is determined, for example, the C user 83 Can be prevented from interrupting.

  The gesture recognizing unit 20 has a role of photographing a user's face by a camera and calls, selections made while the user interacts with the server 40 in order to identify a user who intends to use a plurality of control target devices, It has a role of reading a spatial hand gesture as a signal of an operation instruction by a camera.

  In the present embodiment, a spatial hand gesture using a user's own hand is used in three stages as shown in FIG. Although the user is specified by using, a call, selection, and operation instruction signal is not limited to the spatial hand gesture. For example, the gesture recognition unit 20 uses a remote control device using a near infrared light emitting element. The remote control device may be provided with a light receiving unit to output various instruction signals such as a call instruction, a selection instruction, and an operation instruction together with information specifying the user from the remote control apparatus.

  According to the present embodiment, it is possible to acquire one or more control target devices simultaneously and independently using a pointing medium such as a spatial hand gesture defined for each of a plurality of users. became. Then, a desired operation can be performed by operating the control target device or the function for which the use authority has been acquired. For example, execution of a print job can release and print only the jobs stored by each user.

  In addition, when the D user 84 in FIG. 1 has already performed an operation by operating a button on the operation panel, or when one user is performing an operation by executing a spatial hand gesture, Functions cannot be used by spatial hand gestures.

  In this embodiment, the function for implementing the invention is recorded in advance in the apparatus. However, the present invention is not limited to this, and the same function may be downloaded from the network to the apparatus, and the same function is recorded. What is stored in the medium may be installed in the apparatus. The recording medium may be any form as long as the recording medium can store the program and can be read by the apparatus, such as a CD-ROM. In addition, the function obtained by installing or downloading in advance may be realized in cooperation with an OS (operating system) inside the apparatus.

  The above-described embodiments are merely examples, and can be implemented in various other forms without departing from the spirit or main features thereof. Therefore, the above-mentioned embodiment is only a mere illustration in all points, and should not be interpreted limitedly. The interpretation of the technical matters indicated by the claims is not limited to the text of the specification. Further, all modifications, various improvements, alternatives and modifications belonging to the equivalent scope of the claims are all within the scope of the present invention.

11-13 Information processing equipment (MFP)
20 Gesture recognition units 31-33 Personal computer (PC)
40 servers 70 networks 81-84 users

Claims (5)

Connect at least multiple devices to be controlled and a server via a network, and execute different functions for specific users on the devices to be controlled that have multiple functions that can be executed based on instruction signals issued by the user. A device selection system for obtaining authority to
An imaging unit that images a plurality of users that issue instruction signals to the plurality of devices to be controlled;
A user identifying unit that identifies each user that emits an instruction signal based on photographing information photographed by the photographing unit;
A device identifying unit that identifies a device to be controlled based on the orientation of the user photographed by the photographing unit;
A signal determination unit that receives an instruction signal issued by the user and determines the instruction signal;
An instruction signal setting unit that sets different signal contents for a plurality of functions that can be executed by the device to be controlled;
Based on the instruction signal determined by the signal determination unit and the signal content for the function set by the instruction signal setting unit, for each user specified by the user specifying unit, to each control target device specified by the user A control unit for acquiring authority to execute different functions;
A selection system for equipment.   The device selection system according to claim 1, wherein the instruction signal is a spatial hand gesture. Authority to allow a specific user to execute different functions on a control target device that has multiple functions that can be executed based on an instruction signal that is connected to at least a plurality of control target devices via a network. A device selection device for acquiring
A user identifying unit that identifies each user that emits an instruction signal based on information captured by a plurality of users that emit instruction signals to the plurality of control target devices;
A device identification unit that identifies a device to be controlled based on the orientation of the captured user;
A signal determination unit that receives an instruction signal of a photographed user and determines the instruction signal;
An instruction signal setting unit that sets different signal contents for a plurality of functions that can be executed by the device to be controlled;
Based on the instruction signal determined by the signal determination unit and the signal content for the function set by the instruction signal setting unit, for each user specified by the user specifying unit, to each control target device specified by the user A control unit for acquiring authority to execute different functions;
A device selection device.   The device selection apparatus according to claim 3, wherein the device to be controlled is a multifunction device having at least a scanner function and a print function. A device connected to at least a plurality of control target devices via a network for processing to acquire a right to execute a different function for a specific user for a specific control target device based on an instruction signal issued by the user A program for causing a selection device to execute,
Identify each user that emits an instruction signal based on information taken by a plurality of users that emit an instruction signal to the plurality of control target devices,
Based on the orientation of the imaged user, identify the controlled device,
Receiving a photographed user instruction signal, determining the instruction signal,
Set different signal contents for a plurality of functions that can be executed by the device to be controlled,
Based on the determined instruction signal and the signal content for the set function, a process for causing each identified user to acquire the authority to cause the control target device identified by the user to execute a different function, A device selection program for causing the device selection apparatus to execute.