JP5979976B2 - Information processing apparatus and control method thereof - Google Patents

Description

  The present invention relates to a technique for estimating a direction in which an operator is present.

2. Description of the Related Art Conventionally, apparatuses that read an original placed on a platen and form an image mainly use the original to be read under the platen to read the content of the original. Such an apparatus is often provided with an operation panel, and the position where the operator operates is limited. Therefore, it is not necessary for the apparatus to recognize the position of the operator. However, in recent years, in order to improve operability related to a document reading operation, there is a document camera such as a document camera that photographs a document placed on the document table from the top of the document table. In addition, there is a projector that is provided on the top of the document table and can project an image that can be used as a user interface for operation to an arbitrary place or orientation on the document table. Since an apparatus having these functions can be operated from various directions, the apparatus needs to recognize the position of the operator.
In order to solve such a problem, conventionally, there has been a technique for estimating the position of the operator from an image or a video obtained by photographing the hand operated by the operator or the movement of the hand. For example, there is an apparatus that detects a moving direction of a hand that is pushed out to a projection table for operation, determines a display position or a display direction of a user interface according to the detected direction, and projects it (Patent Document 1).

JP 2009-64109 A

  However, in Patent Document 1, when a hand is moved in a horizontal direction with respect to an axis connecting the operator and the apparatus, a deviation of at least 90 ° occurs between the direction in which the operator is present and the direction in which the user interface is displayed. For this reason, the direction in which the operator is present may be erroneously estimated. Accordingly, an object of the present invention is to estimate the direction in which the operator is present with high accuracy.

In order to solve the above problems, an information processing apparatus according to the present invention is an information processing apparatus that acquires image data of the document by imaging a document placed on a document table, a document from the top of the platen analyzes the image data obtained by the imaging means for capturing an upper base, and an image analysis means that acquires position information of the operator's arm to set the document and the document, the document obtained by the image analysis unit And direction estimation means for estimating the direction in which the operator is present based on the position information of the operator's arm, and display on the document table in accordance with the direction in which the operator is estimated as estimated by the direction estimation means. And display control means for controlling the direction of the image .

According to the present invention, an image can be displayed for an operator who sets a document on a document table .

The hardware block diagram of the information processing apparatus 100, and its external view. 2 is a functional configuration diagram of the information processing apparatus 100. FIG. The figure which showed an example of the image data which imaged on the original stand at the time of original setting The figure which showed an example of the processing flow of the information processing apparatus.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following embodiments do not limit the present invention related to the scope of claims, and all combinations of features described in the present embodiments are essential to the solution means of the present invention. Not necessarily. In addition, about the same structure, the same code | symbol is attached | subjected and demonstrated.

<First Embodiment>
FIG. 1A is a hardware configuration diagram of the information processing apparatus 100 according to the present embodiment.

  In FIG. 1A, a CPU 101 is a system control unit and controls the information processing apparatus 100 as a whole. The ROM 102 is a read-only memory that stores programs and parameters that do not need to be changed. The RAM 103 is a rewritable memory that temporarily stores programs and data supplied from an external device. The sensor 104 is preferably one that can detect that an object has entered the document table of the information processing apparatus 100, and may be an infrared sensor, for example. The imaging unit 105 is installed on the upper part of the document table, and images the document table. In the present embodiment, the detection range of the sensor 104 and the imaging range of the imaging unit 105 are on the minimum rectangular area including the document table. The image analysis unit 106 analyzes and edits the image obtained by the imaging unit 105. The image data picked up when the sensor 104 detects the entry of an object is an analysis target in the image analysis unit 106. The direction estimation unit 107 identifies the document and the hand holding the document from the captured image data, and estimates the direction in which the operator is present. A display unit 108 is an interface with a monitor or a projector for displaying captured image data and a user interface. A system bus 108 connects the units 101 to 108 so that they can communicate with each other. In addition to the ROM as the storage medium for supplying the program code to the configuration described above, there are the following. For example, a flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape, memory card, DVD, and the like may be added to the configuration of the information processing apparatus of this embodiment. Further, there may be an input unit that is an interface with an input device such as a pointing device or a keyboard that receives data from a user and inputs data.

  FIG. 2 is a functional configuration diagram of the information processing apparatus 100 according to the present embodiment.

  When a document to be processed is set on the document table, the detection unit 201 detects that the document has entered the document table. Upon receiving the detection by the detection unit 201, the imaging unit 202 starts imaging on the document table. The image analysis unit 203 analyzes the image data obtained by the imaging unit 202. If the analysis identifies the original and the hand holding the original or if there is no difference from the previous time image data, the imaging is stopped, and the direction estimation unit 204 determines the direction of the operator based on the analysis result. presume. The display unit 205 displays the user interface and image data in the estimated direction.

  FIG. 1B is an external view of the information processing apparatus 100 according to this embodiment.

  The document table 301 has a circular shape, and the operator sets the document from all directions. A camera 302 serving as an imaging unit 202 is installed on the upper part of the document table, and images a document 303 and the like placed on the document table. It should be noted that the imaging range of the camera 302 can capture an image of a minimum rectangular area including the document table 301 and cannot capture images of an operator outside the document table 301. The data format that can be captured by the camera 302 may be a still image or a moving image as long as the image data that can be analyzed by the image analysis unit 203 can be generated. A projector 304 corresponding to the display unit 205 projects data captured by the user interface and the camera 302 to an arbitrary position and direction on the document table. In the example of FIG. 1B, a state in which the operation button 305 is projected on the document table 301 is shown.

  In the present embodiment, the operation button 305 is projected by the projector 304, but may be displayed on a display. In the present embodiment, an example in which the document table is circular has been described, but the document table may not be circular. However, when the document table is circular, the operator can set the document from various directions as compared with the case where the document table is rectangular.

  FIG. 4 is a processing flow of the information processing apparatus 100 in the present embodiment. First, when the operator attempts to place a document on the document table, the detection unit 201 detects that the document has entered the space on the document table (step 401). When the entry of the document is detected, the imaging unit 202 starts imaging on the document table (step 402). Note that the imaging may be a moving image, or still images may be taken at regular time intervals. The image analysis unit 203 analyzes the image data obtained by imaging, and recognizes the document and the hand or arm that sets the document (step 403). Also, document position information, hand shape, and arm position information are acquired. Note that the imaging by the camera 302 may be terminated at the timing when the recognition is successful, or the imaging may be terminated when there is no difference from the image data obtained immediately before imaging.

  In step 404, the image analysis unit 203 determines whether the hand for setting the document shown in the image data is the right hand or the left hand. The discrimination method may be pattern matching with feature extraction. Whether the hand is a hand for setting a document is determined to be a hand for setting a document if it is in contact with the document or overlaps the document. If it is determined in step 404 whether the hand setting the document is the right hand or the left hand, the process proceeds to step 405.

  In step 405, the direction estimation unit 204 estimates the direction in which the operator is present from the information on the right hand and the left hand. A specific estimation method in step 405 will be described with reference to FIGS. 3 (a) and 3 (b). FIG. 3A shows an example of image data captured by the camera 302 in FIG. 1B on the document table 501 when the operator is setting the document 504 with the right hand. FIG. 3B shows an example of image data captured by the camera 302 in FIG. 1B on the document table 501 when the operator is setting the document 504 with the left hand. Reference numeral 502 in FIG. 3A denotes an imaging area of the camera 302. A point A that is the tip 505 of the hand (503 in FIG. 3A and 509 in FIG. 3B) that sets the document in the image data, and a point B that is the coordinates 506 of the hand region farthest from the tip 2 points are detected. If the arm is in the image data, the arm is also taken into account for the detection of the two points. A position where the operator is present (here, the viewpoint position of the operator) 508 is a point P, and the thumb side when the palm is placed on the surface of the document table with respect to the line segment AB is referred to as the inside. A straight line 507 is a straight line passing through the point A and perpendicular to the line segment AB. Unless the operator is unnatural in the act of placing the document 504 on the document table 501, the angle θ formed in the order of the points B, A, and P is 0 ° <θ <90 ° and is inward of the line segment AB. A point P exists in the region. Therefore, in consideration of the expected value that the deviation between the direction in which the operator is actually present and the estimated direction is reduced, the direction of the angle θ = 45 ° as viewed from the inside and the point A with respect to the line segment AB is operated. It is estimated that the person is in the direction. Note that θ = 45 ° is an example, and it may be 0 ° <θ <90 °.

  If it is determined in step 404 that the right hand or left hand has failed, the process proceeds to step 406. The reason why the determination of whether the right hand or the left hand is unsuccessful may be that a part of the hand is hidden by the original when the original is held by the hand, so that the determination cannot be made.

  In step 406, it is determined whether the document is in the left or right direction with respect to the arm. FIGS. 3C and 3D show an example of a case where Yes is determined in step 406. FIG. 3C shows image data obtained by capturing the document 604 on the right side of the arm with the camera 302 of FIG. 1B on the document table 601. FIG. 3 (d) shows image data obtained by capturing the document 604 on the left side with the camera 302 of FIG. 1 (b) on the document table 601. FIG. Reference numeral 602 in FIG. 3C denotes an imaging area of the camera 302. Two points, point A at the tip 605 and point B which is the coordinate 606 of the hand region farthest from point A of the part recognized as a hand or arm (603 in FIG. 3C and 609 in FIG. 3D) To detect. If the arm is in the image data, the arm is also taken into account for the detection of the two points. A straight line 607 is a straight line passing through the point A and perpendicular to the line segment AB. If the area recognized as the document 604 is on the right side as shown in FIG. 3C or the left side as shown in FIG. 3D with respect to the line segment AB, it is determined as Yes in step 406. If it is determined as Yes in step 406, the process proceeds to step 407.

  In step 407, the direction estimation unit 204 estimates the direction in which the operator is present from the relative position information between the document and the arm. A specific estimation method in step 407 will be described with reference to FIG. 3C and FIG. A position where the operator is present (here, the viewpoint position of the operator) 606 is a point P, and the side where the area recognized as the document 604 exists with respect to the line segment AB is called the inside. Unless the operator is unnatural in the act of placing the document on the document table 601, the angle θ formed in the order of the points B, A, and P is 0 ° <θ <90 °, and the area inside the line segment AB There is a point P. Therefore, in consideration of the expected value that the deviation between the direction in which the operator is actually present and the estimated direction is reduced, the direction of the angle θ = 45 ° as viewed from the inside and the point A with respect to the line segment AB is operated. It is estimated that the person is in the direction.

  FIG. 3E shows image data obtained by capturing the document on the document table 701 with the camera 302 in FIG. For example, when an original place is provided next to the original stage and images are taken by sliding the originals horizontally one by one from the original place, the original can be held as shown in FIG. Reference numeral 702 in FIG. 3E denotes an imaging area of the camera 302. In such a case, it is determined No in step 406.

  If it is determined No in step 406, the image analysis unit 203 analyzes the contents of the document 704 (step 408). As a result of analyzing the document content, the image analysis unit 203 determines whether or not the orientation of the document has been identified (step 409). In this embodiment, it is assumed that the document content is analyzed to recognize characters and images, and the orientation of the document is determined from the orientation of the characters and images. However, the present invention is not limited to characters and images. Further, it is possible to analyze at least one of the characters and images in the document, determine the orientation of at least one of the analyzed characters and images, and use the determined orientation as the orientation of the document. If the orientation of the document can be determined in step 409, the process proceeds to step 410.

  In step 410, the direction estimation unit 204 estimates the direction in which the operator is present from the relative position information of the document and the arm and the orientation of the document. A specific estimation method in step 410 will be described with reference to FIG. Two points are detected: a point A which is the tip 705 of the hand 703 holding the document 704 shown in the image data, and a point B which is the coordinates 706 of the hand region farthest from the tip 705. If the arm is in the image data, the arm is also taken into account for the detection of the two points. A straight line 707 is a straight line passing through the point A and perpendicular to the line segment AB. Unless the operator is unnatural in the act of placing the document on the document table 701, it is unlikely that the operator is in the direction opposite to the hand 703 with respect to the straight line 707 when viewed from the front end 705. In consideration of the high possibility that the operator can set the document on the document table, it is estimated that the operator's viewpoint is in a direction satisfying ± 45 ° with respect to the document orientation 708. A direction that is an intermediate value of angles satisfying these conditions viewed from the tip 705 is estimated as the direction in which the operator is present. If it is determined No in step 409, that is, if the analysis of the document content fails or the orientation of the document cannot be identified, the process proceeds to step 411.

  In step 411, the direction estimation unit 204 estimates that the direction in which the arm is present is the direction in which the operator is present. Specifically, it is estimated that there is an operator in the extending direction of the point B with respect to the line segment AB in FIG.

  In step 412, a user interface for operation is displayed on the display unit 205 in the direction in which the operator is estimated from any one of steps 405, 407, 410, and 411. As a display method, in this embodiment, the image is projected from the projector 304 in FIG.

  If the right hand and left hand are successfully identified in step 404, the identified hand may be presumed to be the dominant hand of the operator, and the user interface may be projected on the document table on the dominant hand side with respect to the document. In addition, when a book or the like is imaged on the document table, it is conceivable that the image is captured with the hand held so as not to turn the page. The user's hand is likely to be the other hand, so if you hold the book with your right hand, the left side of the book, and if you hold the book with your left hand, the user interface on the right side of the book. You may project. In this embodiment, the user interface for operation is given as an example of information to be projected. However, the present invention is not limited to this, and when something is displayed to the operator, it is displayed in the estimated direction of the operator. .

  Although the embodiment according to the present invention has been described above, as described above, the information processing apparatus is a general-purpose information processing apparatus such as a normal personal computer and can be realized by a computer program that operates on the general-purpose information processing apparatus. is there. Therefore, it is obvious that the present invention falls within the category of computer programs. In that case, the information processing apparatus 101 functions as each unit. In general, the computer program is stored in a computer-readable storage medium such as a CDROM, and can be executed by setting it in a corresponding drive of the computer and copying or installing it in the system. Therefore, it is obvious that the present invention includes such a computer-readable storage medium as a category.

Claims (15)

An information processing apparatus for acquiring image data of a document by imaging a document placed on a document table,
Image analysis means for analyzing the image data obtained by the image pickup means for picking up an image on the document table from the upper part of the document table, and obtaining position information of the arm of the operator who sets the document and the document;
Direction estimation means for estimating a direction in which the operator is present based on position information of the manuscript and the operator's arm acquired by the image analysis means;
Display control means for controlling the direction of an image to be displayed on the document table in accordance with the direction of the operator estimated by the direction estimation means;
An information processing apparatus comprising: The image analysis means further analyzes the orientation of the document,
The information processing apparatus according to claim 1, wherein the direction estimation unit estimates a direction in which the operator is present based on the position information and a direction of the document.   The information processing apparatus according to claim 1, wherein the display control unit displays a user interface on the document table in a direction estimated by the direction estimation unit. Detecting means for detecting that a document has entered the document table;
4. The information processing apparatus according to claim 1, wherein the image pickup unit starts image pickup when an entry of a document is detected by the detection unit. 5.   5. The direction estimation unit estimates a direction in which the operator is located with respect to the document table in accordance with a relative position between the document and the operator's arm. The information processing apparatus according to item 1.   The direction estimating means determines whether the operator's arm is a right hand or a left hand as a result of the analysis by the image analysis means, depending on whether the operator's arm is a right hand or a left hand. If the direction of the operator is estimated and the result of the analysis by the image analysis means is that it is not possible to identify whether the operator's arm is a right hand or a left hand, the relative position of the manuscript and the operator's arm 5. The information processing apparatus according to claim 1, wherein a direction in which the operator is located with respect to the document table is estimated in accordance with the information processing apparatus. If the position of the document relative to the operator's arm can be specified as either the right or left as a result of the analysis by the image analysis unit, the direction estimating unit is responsive to the relative position of the document and the operator's arm. If the direction of the operator with respect to the document table is estimated and the position of the document relative to the operator's arm is not identifiable as either left or right as a result of the analysis by the image analysis means, the document 7. The information according to claim 1, wherein a direction in which the operator is located with respect to the document table is estimated using an orientation of the document represented by an analysis result of the content of the document. Processing equipment.   The direction estimating means further estimates the direction of the operator with respect to the document table using the direction of the operator's arm obtained from the result of the analysis by the image analyzing means. The information processing apparatus according to any one of claims 1 to 7. The information processing apparatus includes an imaging unit that captures an image on the document table from the top of the document table, and a projector that projects an image on the document table from the top of the document table.
The information processing apparatus according to claim 1, wherein the display control unit displays the image on the document table by controlling the projector. A control method of an information processing apparatus for acquiring image data of a document by imaging a document placed on a document table,
An image analysis step in which image analysis means analyzes image data obtained by an image pickup means for picking up an image on the original platen from above the original platen, and acquires position information of the manuscript and the arm of the operator who sets the original document; ,
A direction estimating unit that estimates a direction in which the operator is present based on position information of the manuscript and the operator's arm; and
A display control step for controlling a direction of an image to be displayed on the document table in accordance with the estimated direction of the operator;
A method for controlling an information processing apparatus, comprising: The image analysis step analyzes image data obtained by the imaging means, analyzes position information of the manuscript and an operator's arm for setting the manuscript, and an orientation of the manuscript,
The method according to claim 10, wherein the direction estimating step estimates a direction in which the operator is present based on the position information and the orientation of the document.   The method for controlling an information processing apparatus according to claim 10, wherein the display control step includes a display step of displaying a user interface in a direction estimated in the direction estimation step.   The detection means has a detection step of detecting that a document has entered the document table, and starts imaging on the document table when entry of the document is detected in the detection step. The control method of the information processing apparatus according to claim 10.   A program for causing a computer to function as each unit included in the information processing apparatus according to claim 1 by being read and executed by a computer.   A computer-readable storage medium storing the program according to claim 14.

Images