JP2015015634A - Information processing apparatus, direction identification method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing apparatus which can easily identify a direction in which a communication device to be a communication partner exists.SOLUTION: An information processing apparatus 100 has a camera 107, a storage unit 113, a face identification unit 110, and a direction determination unit 112. The storage unit 113 associates registered face images of a plurality of persons with a communication device which is owned by each person to store them therein. The face identification unit 110, on the basis of the face image of each person detected from the image including face images of the persons which are captured by the camera 107 and the registered face images, identifies the person associated with a detected face image. The direction identification unit 112 identifies the direction in which the communication apparatus which is owned by the identified person exists, in accordance with an area in which the face image in the image which is captured by the camera 107 is displayed.

Description

  The present invention relates to an information processing apparatus that transmits electronic data to a communication device.

  A portable information processing device such as a smart phone is generally equipped with an imaging device such as a camera. In such an information processing apparatus, a person can be identified from an image captured by the imaging apparatus using face recognition technology. Further, Patent Literature 1 discloses a technique for specifying a communication device owned by a specified person as a communication partner. In Patent Document 1, a correspondence relationship between a face image and a communication device owned by a person of the face image is stored in advance in the information processing apparatus. The information processing apparatus specifies a communication device owned by the person of the face image from the face image recognized from the image captured by the image capturing apparatus and the stored content. The identified communication device becomes a communication partner of the information processing apparatus.

JP 2011-172089 A

  An owner of an information processing apparatus may browse an image such as a photograph displayed on the display screen of the information processing apparatus together with another person. At this time, the image data of the image being viewed may be transmitted from this information processing apparatus to a communication device owned by another person.

  For example, the user A may provide a photograph being browsed from a smartphone owned by the user A (smartphone A) to a smartphone (smartphone B) owned by the user B who browses the photograph together with the user A. is there. In such a case, in Patent Document 1, the user A images the user B with the smart phone A and specifies the smart phone B as a communication partner.

  In an information processing apparatus including two cameras, such as a smart phone, it is possible to take an image of user B while browsing a photograph by using a camera (in-camera) provided on the display screen side. By using the in-camera, it is possible to easily identify the smart phone B as a communication partner. If it is possible to transmit image data by performing a touch operation (drag) in the direction of the smart phone B after specifying the communication partner, for example, easy and intuitive communication is possible. For this purpose, it is necessary to specify the direction (position) of the smart phone B specified as the communication partner with respect to the smart phone A.

  In order to solve such a conventional problem, it is a main object of the present invention to provide an information processing apparatus capable of easily specifying the direction in which a communication device as a communication partner exists.

  An information processing apparatus of the present invention that solves the above problems includes an imaging unit, a storage unit that stores registered face images of a plurality of persons in association with communication devices owned by each person, and a person imaged by the imaging unit Face identifying means for identifying a person associated with the detected face image based on the face image detected from the image including the face image and the registered face image stored in the storage means; Direction specifying means for specifying a direction in which a communication device owned by the person specified by the face specifying means exists in accordance with an area in which the face image in the image picked up by the image pickup means is displayed. It is characterized by that.

  According to the present invention, it is possible to easily identify the direction in which a communication device serving as a communication partner exists by the area where the face image is displayed in the captured image.

(A) is a hardware block diagram of an information processing apparatus, (b) is a functional block diagram of an information processing apparatus. FIG. 4 is an exemplary diagram of a registration table. (A), (b) is explanatory drawing of the utilization scene of information processing apparatus. (A)-(f) is explanatory drawing of the transmission / reception operation | movement of image data. (A), (b) is a flowchart showing the process of information processing apparatus. (A)-(d) is an illustration figure of the image containing a recipient's face image. (A)-(c) is a flowchart showing the process of step S203. (A), (b) is a flowchart showing the process of information processing apparatus. FIG. 4 is an exemplary diagram in a case where it is difficult to specify a direction in which a communication device exists. (A) is a hardware block diagram of an information processing apparatus, (b) is a functional block diagram of an information processing apparatus. The flowchart showing the process of step S203. The figure showing a mode that a recipient's face image is reflected. The figure showing a mode that the inclination of the main body of information processing apparatus changes.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. This embodiment is an example of the present invention, and is not intended to limit the scope of the present invention only to them.

<First Embodiment>
FIG. 1A is a hardware configuration diagram of the information processing apparatus 100 according to the first embodiment. The information processing apparatus 100 is realized by a device having a communication function and an imaging function, such as a smart phone or a tablet PC (Personal Computer).

  Each component of the information processing apparatus 100 is controlled by a CPU (Central Processing Unit) 101. The CPU 101 executes a computer program read from a ROM (Read Only Memory) 102 using a RAM (Random Access Memory) 103 as a work area in order to control each component. The ROM 102 is a memory that stores various computer programs for controlling the respective components. Note that the CPU 101 may load a computer program from an external storage device connected to the information processing apparatus 100 to the RAM 103 and execute the computer program.

  An HDD (Hard Disk Drive) 104 is an example of an external storage device. The HDD 104 stores the above computer program and various electronic data. In addition to the HDD 104, the external storage device may be realized by, for example, a medium (recording medium) and an external storage drive for realizing access to the medium. As such a medium, for example, a flexible disk (FD), a CD-ROM, a DVD, a USB memory, an MO, a flash memory, or the like can be used. Further, the external storage device may be provided in a server device or the like connected via a network.

  The touch panel 105 is an input / output device that is provided on one surface of the information processing apparatus 100 and is a combination of a display and a position input device such as a touch pad. The touch panel 105 displays various images and acquires an instruction by a user operation. The communication interface (I / F) 106 performs two-way communication by wire or wireless with other information processing apparatuses, communication devices, external storage devices, and the like by a known communication technique. In addition to direct communication, indirect communication via a relay device is also possible with other communication devices. The camera 107 is an imaging device that captures an image or video of a person. The camera 107 of this embodiment is an in-camera provided on the same surface as the main camera provided on the back side of the information processing apparatus 100 and the display of the touch panel 105. That is, the in-camera is installed at a position and a direction in which a face of a person who is browsing the display can be photographed.

  The CPU 101, ROM 102, RAM 103, HDD 104, touch panel 105, communication I / F 106, and camera 107 are connected to each other via a system bus 108 so that data can be transmitted and received.

  FIG. 1B is a functional block diagram showing functions formed in the information processing apparatus 100. Each function is realized by the CPU 101 reading a computer program from the ROM 102 and executing it. The information processing apparatus 100 includes an imaging unit 109, a face specifying unit 110, an image processing unit 111, a direction specifying unit 112, a storage unit 113, a UI screen generating unit 114, a transmitting unit 115, a determining unit 116, a direction acquiring unit 117, and an input. A unit 118 and a receiving unit 119 are formed. In the present embodiment, each functional block is realized by software, but may be realized by hardware. The information processing apparatus 100 identifies a person from the image captured by the camera 107 and determines a recipient from the identified person. A communication device owned by the receiver is a communication partner. Further, the information processing apparatus 100 identifies the direction of the communication device that is the communication partner with respect to the information processing apparatus 100 according to the display area of the person in the image captured by the camera 107. In order to implement such processing, each functional block is formed. Note that the communication device to be a communication partner is a device having a display function and a communication function. For example, another information processing apparatus 100 may be the communication partner.

  The imaging unit 109 transmits image data of an image captured by the camera 107 to the face specifying unit 110. The face identifying unit 110 performs a process for identifying a person in the image based on the image data received from the imaging unit 109. The face identifying unit 110 identifies a person by detecting a face image using a known face recognition technique.

  The image processing unit 111 specifies the display area of the face image of the person specified by the face specifying unit 110, detects the center coordinates of the display area of the face image, and detects the position of the organ of the face image such as the eyes and nose (organ Image processing such as detection). These image processes are also realized using a known technique. The display area of the face image is hereinafter referred to as “face area”. The display area of the face image is a face outline and an area inside the face image. Further, a rectangular or circular area including the face image may be used as the face image display area.

  The direction specifying unit 112 uses the result of image processing by the image processing unit 111 to determine the positional relationship (relative to the communication device owned by the information processing apparatus 100 and the person (receiver) specified by the face specifying unit 110. Direction).

  A UI (User Interface) screen generation unit 114 generates a UI screen for transmitting electronic data based on a touch operation (touch, drag, etc.) by the owner on the touch panel 105. The determination unit 116 compares the direction in which the communication device owned by the receiver with respect to the information processing apparatus 100 exists with the direction of the touch operation, and determines whether or not they match. If they match, the determination unit 116 causes the UI screen generation unit 114 to generate a UI screen indicating that the electronic data is moved to the communication device owned by the recipient. If they do not match, the determination unit 116 causes the UI screen generation unit 114 to generate a UI screen indicating that electronic data is not transmitted to the communication device owned by the recipient.

  The transmission unit 115 transmits data to other communication devices. In particular, in the present embodiment, when the determination result by the determination unit 116 matches, the transmission unit 115 matches the communication device owned by the receiver with the determination result generated by the electronic data and UI screen generation unit 114. UI screen data representing the UI screen is transmitted. The receiving unit 119 receives data from other communication devices. In particular, in the present embodiment, the receiving unit 119 receives a completion notification indicating that electronic data and UI screen data have been received from a communication device owned by the receiver. The transmission unit 115 and the reception unit 119 are realized by the communication I / F 106.

  The input unit 118 detects a user touch operation performed by the touch panel 105. The direction acquisition unit 117 acquires the operation direction (instruction direction) of the user's touch operation detected by the input unit 118.

  The storage unit 113 is realized by the RAM 103 and the HDD 104 and stores a registration table regarding a plurality of persons registered in the information processing apparatus 100. The owner of the information processing apparatus 100 is registered in the registration table together with other persons. The registration table is used to specify a person in the image based on the image data received from the imaging unit 109. FIG. 2 is an exemplary diagram of a registration table. In the registration table, registration data of a user who is a person registered in the information processing apparatus 100 is recorded. The registration data includes a user registration face image (registered face image) 309, a user ID 310 for identifying the user, a device ID 311 for identifying a communication device owned by the user, and whether the user is a sender of electronic data. Alternatively, attribute data 312 indicating whether the user is a recipient is included. In the example of FIG. 2, user A is registered as a sender and user B is registered as a receiver. User A is the owner of the information processing apparatus 100. The recipient is a person specified by the face specifying unit 110, and is changed to a person other than the user B (user C) according to the image captured by the camera 107. The registered face image 309 is associated with a user ID 310, a device ID 311, and attribute data 312. The registered face image 309 makes it possible to specify the user ID 310, the device ID 311 and the attribute data 312.

  The face identifying unit 110 identifies the person in the image by comparing the detected face image with the registered face image 309 registered in the registration table. The face identifying unit 110 compares the detected face image with the registered face image 309 using a known image matching technique. For example, the comparison is performed based on the correspondence between the feature points of the detected face image and the registered face image 309.

  3A and 3B are explanatory diagrams of specific examples of usage scenes of the information processing apparatus 100. FIG. FIG. 3A is a diagram illustrating a state where a user A 301 who is the owner of the information processing apparatus 100 and a user B 302 who is not the owner browse a list of images displayed on the touch panel 105 of the information processing apparatus 100. is there.

  FIG. 3B is a diagram of the information processing apparatus 100 viewed from the user A301 and the user B302 side. FIG. 3B shows a situation where a user A 301 and a user B 302 browsing an image list displayed on the touch panel 105 appear in an image 308 captured by the in-camera 304. The image 308 includes the face image 307 of the user A301 and the face image 306 of the user B302. Note that the image captured by the in-camera 304 does not need to be displayed on the touch panel 105, but may be displayed to present the user A301 and the user B302 appearing in the in-camera 304. In the present embodiment, the user A301 is the sender and the user B302 is the receiver, and the image is transmitted from the image list displayed on the touch panel 105 to the communication device owned by the user B302 by the sender's touch operation.

  FIG. 4 illustrates an example in which image data of a photograph (display image) is transmitted and received between the information processing apparatus 100 of the sender (user A301) and the communication device 402 owned by the receiver (user B302) through a UI screen. FIG. In the present embodiment, an example in which image data of a photograph is transmitted / received will be described. However, the transmitted / received data is not limited to this, and may be electronic data of another format.

  In FIG. 4A, a photograph 403 is displayed on the information processing apparatus 100 of the sender. The sender transmits this photograph 403 to the communication device 402. FIG. 4B illustrates a state in which the information processing apparatus 100 and the communication device 402 are brought close to each other and electronic data can be transmitted and received. By being close to each other, the information processing apparatus 100 and the communication device 402 can perform electronic data by proximity wireless communication.

  In FIG. 4C, the sender performs a touch operation (drag) 404 on the photograph 403 in the direction in which the communication device 402 exists on the touch panel 105 of the information processing apparatus 100. A part of the photograph 403 is displayed on the communication device 402. The proportion of photographs displayed on the communication device 402 is determined based on the operation amount (finger movement amount) of the touch operation 404 performed by the sender on the touch panel 105.

  In FIG. 4D, the receiver who is the owner of the communication device 402 receives a photo by touching (dragging) 405 a part of the photo 403 displayed on the communication device 402. The direction of the touch operation 405 performed by the receiver is the same as the direction of the touch operation 404 performed by the sender in the information processing apparatus 100. That is, the direction of the touch operation 405 is opposite to that of the information processing apparatus 100 when viewed from the communication device 402. FIG. 4E illustrates a state in which the transmission of electronic data of the photograph 403 to the communication device 402 has been completed. When the transmission of electronic data is completed normally, the state shown in FIG.

  FIG. 4F illustrates a case where the sender performs a touch operation 406 in a direction in which the communication device 402 does not exist. In this case, electronic data is not transmitted. A UI screen indicating that electronic data cannot be transmitted is displayed on the touch panel 105 of the information processing apparatus 100.

  FIG. 5 is a flowchart showing processing of the information processing apparatus 100 for realizing the operation described in FIG. This process is started by activation of predetermined application software, activation of the information processing apparatus 100, selection by an owner of an image to be transmitted, or the like, and is repeatedly executed every time an image captured by the camera 107 is updated.

  FIG. 5A is a flowchart showing processing for identifying a direction in which the communication device 402 owned by the receiver exists by the information processing apparatus 100.

  The imaging unit 109 transmits image data of an image captured by the camera 107 to the face specifying unit 110. The face identifying unit 110 receives the image data transmitted from the imaging unit 109, detects all face images included in the image based on the received image data, and identifies the recipient (S201). The face specifying unit 110 specifies a recipient from the face image with reference to the registration table. If the detected face image is not registered in the registration table, the face specifying unit 110 ignores this face image. In this example, the face identifying unit 110 detects the face image 307 of the user A301 and the face image 306 of the user B302 from the image 308 in FIG. The face identifying unit 110 identifies the user A301 who is a sender from the detected face image 307 with reference to the registration table. Further, the face identification unit 110 identifies the user B302 from the detected face image 306 with reference to the registration table, and sets the user B302 as a receiver. The face identifying unit 110 sets the recipient by setting the attribute data 312 of the registration data of the user B302 to “recipient”. As a result, “device B” is specified as the communication device 402 of the communication partner.

  When the recipient is identified by the face identifying unit 110, the image processing unit 111 identifies a face area that is a display area of the recipient's face image 306 (S202). After specifying the face area of the recipient, the direction specifying unit 112 specifies the direction of the communication device 402 (device B) with respect to the information processing apparatus 100 according to the position of the face area (S203). Details of the process of specifying the direction in which the communication device 402 exists will be described later. As described above, the process of specifying the direction of the communication device 402 owned by the receiver with respect to the information processing apparatus 100 ends.

  FIG. 5B is a flowchart showing processing for transmitting image data of a photograph (display image) to the communication device 402. This process is executed in parallel with the process of FIG.

  The input unit 118 determines whether or not the sender (user A301) has performed a touch operation (drag) on the photograph being displayed (the photograph 403 in FIG. 4A) (S204). The input unit 118 stands by until there is a touch operation. When there is a touch operation (S204: Y), the direction acquisition unit 117 acquires the direction in which the touch operation was performed (instruction direction by dragging) (S205). The direction acquisition unit 117 receives touch operation information representing the content of the touch operation from the input unit 118, and acquires an instruction direction by the touch operation based on the content of the received touch operation information.

  The determination unit 116 determines whether or not the instruction direction acquired by the direction acquisition unit 117 matches the direction of the communication device 402 identified in step S203 with respect to the information processing apparatus 100 (S206).

  If they match (S206: Y, FIG. 4C), the UI screen generation unit 114 generates a UI screen for transmitting the image data of the photograph (S207). The UI screen generation unit 114 receives touch operation information from the input unit 118, and generates a UI screen based on the received touch operation information. The transmission unit 115 transmits photo image data and UI screen data representing the generated UI screen to the communication device 402 (S208). When the reception unit 119 receives a completion notification indicating that the reception has been completed from the communication device 402, the transmission process of the image data of the photograph ends (S209: Y).

  If they do not match in step S206 (S206: N, FIG. 4 (f)), the UI screen generation unit 114 generates a UI screen for not transmitting the photo image data, and ends the processing (S210). The UI screen is displayed on the touch panel 105 of the information processing apparatus 100. Such a UI screen is, for example, a display indicating that text cannot be transmitted or a display in which a photograph cannot protrude from the display screen of the touch panel 105 of the information processing apparatus 100 even when dragging. In the latter case, the photo can be dragged to the edge of the display screen, but it is a UI screen in which the photo does not move when dragging further.

  As described above, by specifying the direction (position) of the communication device 402 with respect to the information processing apparatus 100, whether or not to transmit image data is determined according to the subsequent operation direction of the touch panel 105. Therefore, the sender can transmit electronic data by an easy and intuitive operation.

  In step S208, the communication device 402 receives photo image data and UI screen data. The UI screen data includes information such as a moving distance by a touch operation. The communication device 402 generates and displays a UI screen for the communication device 402 based on the UI screen data. The recipient performs a touch operation 405 on the display screen of the communication device 402. This operation content is transmitted from the communication device 402 to the information processing apparatus 100. The UI screen generation unit 114 of the information processing apparatus 100 generates the UI screen in consideration of the operation content transmitted from the communication device 402 when generating the UI screen in step S207. Thereby, the display as shown in FIG.

  In the present embodiment, an example in which whether or not to transmit image data is determined according to the direction in which the touch panel 105 is touched has been described. However, whether or not image data can be transmitted may be determined by other operations. For example, the movement (gesture) of the sender's hand may be detected by an image of the in-camera 304 or an infrared sensor, and whether or not image data can be transmitted is determined based on the direction of the movement.

  Further, the process of FIG. 5A and the process of FIG. 5B may be performed at the end of the process of step S205, for example. In this case, after the process of FIG. 5A is completed, the processes after step S206 are performed.

  Processing for identifying the direction in which the communication device 402 exists in step S203 will be described. FIG. 6 is an exemplary view of an image 308 including the recipient's face image 306 captured by the in-camera 304. The direction (position) of the communication device 402 with respect to the information processing apparatus 100 is specified based on the position of the face image 306 in the image 308, the orientation of the face image 306, or the positional relationship with the sender's face image 307. FIG. 7 is a flowchart showing the process of step S203.

  FIG. 6A is an exemplary diagram in a case where the direction in which the communication device 402 exists is determined by the position of the center coordinate 503 of the face area of the recipient of the image 308. The image 308 is divided into four regions 505 to 508 by dotted lines 504. FIG. 7A is a flowchart showing the process of step S203 using such an image 308.

  The image processing unit 111 detects the center coordinates 503 of the face area specified in step S202 of FIG. 5A (S601). The center coordinates of the face area are the coordinates of the center point obtained from the center-of-gravity coordinates of the face area and the coordinates of the top and bottom and the left and right ends of the face area.

  The image processing unit 111 identifies a region in the image 308 to which the detected center coordinate 503 belongs (S602). In the example of FIG. 6A, the center coordinate 503 belongs to the region 505. The direction specifying unit 112 specifies the direction in which the communication device 402 is present for the information processing apparatus 100 based on the area 505 to which the central coordinate 503 belongs, and ends the process (S603).

  In the example of FIG. 6A, when the center coordinate 503 belongs to the area 505, the direction specifying unit 112 specifies that the communication device 402 is in the left direction of the information processing apparatus 100. When the central coordinate 503 belongs to the area 506, the direction specifying unit 112 specifies that the communication device 402 is in the front direction of the information processing apparatus 100. When the central coordinates 503 belong to the region 507, the direction specifying unit 112 specifies that the communication device 402 is in the right direction of the information processing apparatus 100. When the central coordinate 503 belongs to the area 508, the direction specifying unit 112 specifies that the communication device 402 is in the backward direction of the information processing apparatus 100. In this example, the position of the center coordinate 503 is used as a reference, but the direction may be specified based on another position of the face image 306 (characteristic portion such as eyes).

  FIG. 6B is an exemplary diagram in a case where the direction in which the communication device 402 exists is determined by the orientation of the face image 306 of the recipient of the image 308. The information processing apparatus 100 identifies the orientation of the face image 306 based on the positions of organs such as eyes and nose in the face image 306. The information processing apparatus 100 uses a straight line 509 passing through the positions of the left and right eyes of the face image 306, a straight line 510 perpendicular to the straight line 509 and passing through the nose position, and an intersection 511 between the straight line 510 and the edge of the image 308. The direction in which 402 exists is specified. FIG. 7B is a flowchart showing the process of step S203 using the image 308 as described above.

  The image processing unit 111 detects the positions of facial organs (eyes and nose) from the face image 306 (S604). The image processing unit 111 detects the positions of eyes and nose by a known image recognition technique. The image processing unit 111 identifies the orientation of the face image 306 based on the detected eye and nose positions (S605). Here, the direction of the face is a direction from the eyes of the straight line 510 toward the nose. The image processing unit 111 identifies an intersection 511 between a line obtained by extending the straight line 510 from the eyes to the nose and the edge of the image 308 (S606). The direction specifying unit 112 specifies the direction in which the communication device 402 exists for the information processing apparatus 100 based on the intersection 511, and ends the process (S607).

  In the example of FIG. 6B, since the intersection 511 is at the left edge of the image 308, the direction specifying unit 112 specifies that the communication device 402 is in the left direction of the information processing apparatus 100. Similarly, the direction specifying unit 112 specifies that the communication device 402 is in the front direction of the information processing apparatus 100 when the intersection 511 is at the upper edge of the image 308. The direction specifying unit 112 specifies that the communication device 402 is in the right direction of the information processing apparatus 100 when the intersection 511 is on the right edge of the image 308. The direction specifying unit 112 specifies that the communication device 402 is in the rear direction of the information processing apparatus 100 when the intersection 511 is at the lower edge of the image 308.

  FIG. 6C is an exemplary diagram in a case where the direction in which the communication device 402 exists is determined by the distance from the center coordinates 503 of the face area of the image 308 to the edges 512 to 515 of the image 308. FIG. 7C is a flowchart showing the process of step S203 using such an image 308.

  The image processing unit 111 detects the center coordinates 503 of the face area in the same manner as in step S601 in FIG. 7A (S601). The image processing unit 111 identifies the edge closest to the center coordinate 503 among the edges 512 to 515 of the image 308 (S608). The image processing unit 111 calculates the distance from the center coordinate 503 to each of the edge parts 512 to 515, and specifies the edge part having the smallest distance as the nearest edge part. In FIG. 6C, the edge 512 is specified as the nearest edge.

  The direction specifying unit 112 specifies the direction in which the communication device 402 is present with respect to the information processing apparatus 100 based on the nearest edge 512, and ends the process (S609). In the example of FIG. 6C, since the left edge 512 of the image 308 is the nearest, the direction specifying unit 112 specifies that the communication device 402 is in the left direction of the information processing apparatus 100. When the upper edge 513 of the image 308 is the nearest, the direction specifying unit 112 specifies that the communication device 402 is in the front direction of the information processing apparatus 100. When the right edge 514 of the image 308 is the nearest neighbor, the direction specifying unit 112 specifies that the communication device 402 is in the right direction of the information processing apparatus 100. When the lower edge 515 of the image 308 is the nearest, the direction specifying unit 112 specifies that the communication device 402 is in the rear direction of the information processing apparatus 100.

  FIG. 6D shows the direction in which the communication device 402 exists by the intersection 519 between the straight line 518 passing through the center coordinate 516 of the sender's face area and the center coordinate 503 of the receiver's face area and the edge of the image 308. It is an illustration figure in the case of deciding. FIG. 8 is a flowchart showing processing for identifying the direction in which the communication device 402 exists from such an image 308.

  FIG. 8A is a flowchart showing processing for identifying the direction in which the communication device 402 exists by the information processing apparatus 100. The same processes as those in the flowchart of FIG. 5 are denoted by the same step numbers.

  When the face specifying unit 110 specifies the face area of the receiver in steps S201 and S202, the face specifying part 110 specifies the face area of the sender from the image 308 by the same processing (S701 and 702). The face identifying unit 110 detects the sender's face image 307 and identifies the sender based on the registration data of the user A in the registration table of FIG. In addition, the face specifying unit 110 specifies the sender's face area from the detected face image 307. When the identification of the sender's face area is completed, the direction in which the communication device 402 exists is identified (S203).

  FIG. 8B is a flowchart showing the process of step S203.

  The image processing unit 111 detects the center coordinates 503 of the face area of the receiver and the center coordinates 516 of the face area of the sender (S703). This process is the same as the process of step S601 in FIG. The image processing unit 111 identifies the positional relationship between the detected center coordinates 503 and 516 (S704). In FIG. 6D, the image processing unit 111 identifies an intersection 519 between the straight line 518 extended from the center coordinate 516 in the direction of the center coordinate 503 and the edge of the image 308. This intersection point 519 represents the positional relationship between the center coordinates 503 and 516. The direction specifying unit 112 specifies the direction in which the communication device 402 is present with respect to the information processing apparatus 100 based on the positional relationship of the center coordinates 503 and 516, and ends the process (S705).

  In FIG. 6D, since the intersection 519 is at the left edge of the image 308, the direction specifying unit 112 specifies that the communication device 402 is in the left direction of the information processing apparatus 100. When the intersection is on the upper edge of the image 308, the direction specifying unit 112 specifies that the communication device 402 is in the front direction of the information processing apparatus 100. When the intersection is at the right edge of the image 308, the direction specifying unit 112 specifies that the communication device 402 is in the right direction of the information processing apparatus 100. When the intersection is at the lower edge of the image 308, the direction specifying unit 112 specifies that the communication device 402 is in the backward direction of the information processing apparatus 100.

  As described above, the direction in which the communication device 402 is present with respect to the information processing apparatus 100 can be specified by a simple operation. Note that the direction can be specified with higher accuracy by adding processing or adding a configuration such as a sensor as described below.

Second Embodiment
FIG. 9 is an exemplary diagram in the case where it is difficult to specify the direction in which the communication device owned by the receiver exists from the display position and orientation of the face image.

  FIG. 9A is a diagram illustrating a state in which the communication device 402 owned by the receiver 803 is located in front of the information processing apparatus 801 owned by the sender. A face image 806 of the receiver 803 and a face image 805 of the sender are reflected in the image 804 captured by the camera of the information processing device 801. This image 804 is not displayed on the display of the information processing apparatus 801.

  In the information processing device 801 and the communication device 402 having such a positional relationship, the sender performs a touch operation (drag) with the touch panel of the information processing device 801 in the direction 807 where the communication device 402 exists. As a result, the image data is transmitted from the information processing apparatus 801 to the communication device 402. When the sender performs a touch operation in another direction (for example, the direction 808), the image data is not transmitted.

  FIG. 9B is a diagram illustrating a state where the communication device 402 is located in the left direction of the information processing apparatus 801. In the information processing device 801 and the communication device 402 having such a positional relationship, the sender performs a touch operation (drag) on the touch panel of the information processing device 801 in the direction 809 where the communication device 402 exists. As a result, the image data is transmitted from the information processing apparatus 801 to the communication device 402. When the sender performs a touch operation in another direction (for example, the direction 810), the image data is not transmitted.

  In FIG. 9A and FIG. 9B, the face image 805 of the sender and the face image 806 of the receiver 803 are in the same position and in the same direction as the image 804 captured by the in-camera of the information processing device 801. Reflect. Therefore, it is difficult to specify the direction (position) in which the communication device 402 is present with respect to the information processing apparatus 801 in the image 804 as illustrated in FIGS. 9A and 9B.

  FIG. 9C illustrates a region in which it is difficult to specify the direction in which the communication device 402 exists from the position and direction of the face image in the image 804 as described above. When the recipient's face image 806 is reflected in the designated areas 811, 812, the direction in which the communication device 402 exists cannot be specified. The designated areas 811 and 812 are stored in the information processing apparatus 801 in advance.

  In the second embodiment, even when it is difficult to specify the direction in which the communication device 402 exists, the information processing device 801 that can specify the direction (position) in which the communication device 402 exists with respect to the information processing device 801. I will provide a. FIG. 10 is a diagram illustrating the configuration of the information processing apparatus 801. The same components as those of the information processing apparatus 100 of the first embodiment in FIG. Description of the same configuration is omitted. FIG. 10A is a hardware configuration diagram of the information processing apparatus 801. The information processing apparatus 801 has a configuration in which an inclination detector 1101 is added to the information processing apparatus 100 of the first embodiment.

  The inclination detector 1101 detects the inclination (posture) of the main body of the information processing apparatus 801. As the inclination detector 1101, a gyro sensor, an acceleration sensor, or a geomagnetic sensor can be used. The inclination detector 1101 detects the three-axis inclinations of the pitch, roll, and yaw of the main body of the information processing apparatus 801.

  FIG. 10B is a functional block diagram of the information processing apparatus 801. The information processing apparatus 801 has a configuration in which an inclination determination unit 1102 is added to the functions configured in the information processing apparatus 100 of the first embodiment. The inclination determination unit 1102 analyzes the detection result of the inclination detector 1101 and determines the inclination (posture) of the main body of the information processing apparatus 801.

  FIG. 11 is a flowchart showing the process of step S203 by the information processing apparatus 801. The same step number is attached | subjected to the process same as the process of Fig.7 (a) of 1st Embodiment.

  The image processing unit 111 detects the center coordinates of the face area of the receiver 803 as in step S601 in FIG. 7A (S601). The image processing unit 111 determines whether or not the center coordinates of the detected face area are included in any of the designated areas 811 and 812 in FIG. 9C (S1201). When the center coordinates are included in any of the designated areas 811 and 812 (S1201: Y), the direction specifying unit 112 performs direction analysis in which the communication device 402 exists using supplementary information described later (S1202). The supplemental information is information other than the face area of the receiver 803 that can specify the direction in which the communication device 402 exists. A direction in which the communication device 402 exists is obtained by performing a direction analysis using the supplementary information. The direction specifying unit 112 specifies the direction in which the communication device 402 exists for the information processing device 801 based on the result of the direction analysis and the center coordinates of the face area of the receiver 803, and ends the processing (S1203).

  When the center coordinate of the face area of the receiver 803 is not included in any of the designated areas 811 and 812 (S1201: N), the processing after step S602 in FIG. 7A is performed. The direction specifying unit 112 specifies the direction in which the communication device 402 exists and ends the process (S603).

  Regarding the processing in step S1202, a case where a change in the face area of the recipient 803 is used as supplementary information will be described.

  FIG. 12 is a diagram illustrating a state in which the face image of the receiver is reflected in images continuously captured by the information processing apparatus 801 owned by the sender. FIG. 12A is a diagram illustrating a case where the communication device 402 is located in the front direction of the information processing apparatus 801 (the state illustrated in FIG. 9A). FIG. 12B is a diagram illustrating a case where the communication device 402 is located in the left direction of the information processing device 801 (state in FIG. 9B).

  In the case of FIG. 12A, the face images 905, 906, and 907 of the receiver 803 are reflected in the image 804 captured by the camera 107 of the information processing apparatus 801 together with the face image 805 of the sender. In the images 804 of (a-1) and (a-2), the face images 905 and 906 of the recipient 803 are protruding from the image 804. The protruding portion is a portion that is not imaged, but is displayed in FIGS. 12A and 12B in order to make it easier to understand the appearance of the image.

  The image 804 is generated in the order of (a-1), (a-2), and (a-3). (A-1) shows a part of the face image 905 of the receiver 803, (a-2) shows a half 906 of the face image of the receiver 803, and (a-3) shows the entire face image 907 of the receiver 803. It is crowded. In FIG. 12A, the face images 905 to 907 of the recipient 803 are moving from the top to the bottom in the image 804. By identifying the movement of the face images 905 to 907, the direction of the communication device 402 relative to the information processing device 801 is specified. When the face image 905 to 907 of the receiver 803 moves from the top to the bottom in the image 804, the information processing device 801 specifies that the communication device 402 is forward.

  In the case of FIG. 12B, as shown in (b-1) to (b-3), the face images 908 to 910 of the receiver 803 are moved from the left to the right in the image 804. In this case, the information processing device 801 specifies that the communication device 402 is in the left direction.

  In this way, the information processing device 801 specifies the direction in which the communication device 402 exists, using the movement direction of the face image of the receiver 803 as supplementary information. In order to acquire supplementary information, image data representing the image 804 captured by the camera 107 is stored in the storage unit 113 for a predetermined time (for example, 2 seconds). In the process of step S1202, the image processing unit 111 reads the image data from the storage unit 113 and analyzes the direction in which the position of the face area changes.

  Regarding the processing in step S1202, a case will be described in which a change in the tilt (posture) of the main body of the information processing apparatus 801 owned by the sender is used as supplementary information.

  FIG. 13 is a diagram illustrating how the inclination of the main body of the information processing apparatus 801 changes. FIG. 13A shows a change in the inclination of the main body of the information processing apparatus 801 when the communication device 402 is located in the forward direction of the information processing apparatus 801 (FIG. 9A). FIG. 13B shows a change in the inclination of the main body of the information processing apparatus 801 when the communication device 402 is located in the left direction of the information processing apparatus 801 (FIG. 9B).

  In FIG. 13A, the information processing apparatus 801 rotates with respect to the rotation axis 1005 (pitch). The information processing apparatus 801 rotates in the order of (a-1) to (a-3). FIG. 13A shows a state where the information processing apparatus 801 is tilted forward. This movement of the information processing apparatus 801 is, for example, a movement that causes the receiver 803 to browse a picture to be transmitted in the future, and is also a preparatory operation for transmitting picture image data to the communication device 402 owned by the receiver 803. is there.

  When the communication device 402 is located in the forward direction of the information processing apparatus 801, the information processing apparatus 801 is tilted in the forward direction in this way. The inclination determination unit 1102 can acquire and determine the change in inclination (change in pitch), thereby specifying the direction in which the communication device 402 exists.

  In FIG.13 (b), the information processing apparatus 801 rotates with respect to the rotating shaft 1006 (roll). As shown in (b-1) to (b-3), the main body of the information processing apparatus 801 is tilted leftward. The inclination determination unit 1102 acquires and determines the change in inclination (change in roll), thereby specifying that the communication device 402 is located in the left direction of the information processing apparatus 801 (FIG. 9B). )).

  The above is an explanation of the processing when the center coordinates of the face area of the receiver 803 exist in the designated area 811 in FIG. Even when the center coordinates of the face area of the receiver 803 exist in the designated area 812 in FIG. 9C, the direction in which the communication device 402 exists can be specified by the same processing.

  The case where the inclination of the main body of the information processing apparatus 801 owned by the sender is used as supplementary information will be described for the processing in step S1202.

  As described above, the operation in which the sender tilts the information processing apparatus 801 is a movement that causes the receiver 803 to browse a picture to be transmitted, and transmits image data of the photograph to the communication device 402 owned by the receiver 803. It is also a preparation operation. Therefore, the sender stops the operation by tilting the information processing apparatus 801 to the receiver side to some extent. For example, in the state of (a-3) in FIG. 13A or (b-3) in FIG. 13B, the sender stops the movement of tilting the information processing apparatus 801. The inclination of the information processing apparatus 801 in the stopped state is used as supplementary information.

  The inclination determination unit 1102 determines the inclination of the information processing apparatus 801 and specifies the direction in which the communication device 402 exists based on the determination result. When the information processing apparatus 801 is inclined forward, it is specified that the communication device 402 is forward. When the information processing apparatus 801 is tilted leftward, it is specified that the communication device 402 is leftward. If the information processing device 801 is tilted to the right, the communication device 402 is specified to be in the right direction. When the information processing apparatus 801 is tilted backward, it is specified that the communication device 402 is backward. The inclination of the information processing apparatus 801 is not always unidirectional. Therefore, the inclination determination unit 1102 determines that the information processing apparatus 801 is inclined in the maximum inclination direction among the four directions of front, right, left, and rear.

  You may use combining the above supplementary information. For example, the direction in which the communication device 402 exists is specified according to each of the change in the face area of the receiver 803, the change in the inclination of the main body of the information processing apparatus 801, and the inclination of the information processing apparatus 801, The direction may be determined. Moreover, a priority may be provided for each supplemental information, and the direction may be specified by supplemental information having a high priority. In this case, when the specified direction lacks accuracy, the direction is specified by supplementary information having the next highest priority.

  Thus, in 2nd Embodiment, the direction where the communication apparatus 402 exists is determined using supplementary information. Therefore, even when it is difficult to determine from the position of the face area of the receiver 803, the direction in which the communication device 402 exists can be accurately identified.

<Other variations>
Although the embodiment according to the present invention has been described above, as described above, the information processing apparatus is a general-purpose information processing apparatus such as a normal personal computer and can be realized by a computer program that operates on the general-purpose information processing apparatus. is there. Therefore, it is obvious that the present invention falls within the category of computer programs. In general, the computer program is stored in a computer-readable storage medium such as a CDROM, and can be executed by setting it in a corresponding drive of the computer and copying or installing it in the system. Therefore, it is obvious that the present invention includes such a computer-readable storage medium as a category.

  DESCRIPTION OF SYMBOLS 100,801 ... Information processing apparatus, 101 ... CPU, 102 ... ROM, 103 ... RAM, 104 ... HDD, 105 ... Touch panel, 106 ... Communication I / F, 107 ... Camera, 108 ... System bus, 1101 ... Tilt detector, DESCRIPTION OF SYMBOLS 109 ... Imaging part, 110 ... Face specific part, 111 ... Image processing part, 112 ... Direction specific part, 113 ... Memory | storage part, 114 ... UI screen production | generation part, 115 ... Transmission part, 116 ... Determination part, 117 ... Direction acquisition part , 118... Input unit, 119... Reception unit, 1102.

Claims (14)

Imaging means;
Storage means for storing registered face images of a plurality of persons in association with communication devices owned by each person;
The person associated with the detected face image is identified based on the face image detected from the image including the face image of the person imaged by the imaging unit and the registered face image stored in the storage unit Face identification means to
Direction specifying means for specifying a direction in which a communication device owned by the person specified by the face specifying means exists according to a region in which the face image in the image picked up by the image pickup means is displayed; Characterized by comprising,
Information processing device. Detecting means for detecting a center coordinate of a region where the detected face image is displayed;
The direction specifying unit specifies a direction in which a communication device owned by the person specified by the face specifying unit exists according to a position of the central coordinate in the image.
The information processing apparatus according to claim 1. Organ detecting means for detecting the position of a facial organ from the detected face image;
The direction specifying unit specifies a direction in which a communication device owned by the person specified by the face specifying unit exists according to the face orientation specified by the detected position of the facial organ. Features
The information processing apparatus according to claim 1. Detecting means for detecting a center coordinate of a region where the detected face image is displayed;
The direction specifying unit specifies a direction in which a communication device owned by the person specified by the face specifying unit exists according to an edge of the image closest to the center coordinate.
The information processing apparatus according to claim 1. Detecting means for detecting a center coordinate of a region where the detected face image is displayed;
The storage means stores a registered face image of the owner of the information processing apparatus together with registered face images of other persons,
The face identifying unit detects the face image of the owner and the face image of another person from the image captured by the imaging unit, and refers to the registered face image stored in the storage unit, Identify the owner and other person associated with each detected face image;
The direction specifying means is a communication owned by the specified other person according to the center coordinates of the area where the face image of the owner is displayed and the center coordinates of the area where the face image of the other person is displayed. It is characterized by identifying the direction in which the device exists,
The information processing apparatus according to claim 1. Detecting means for detecting a center coordinate of a region where the detected face image is displayed;
The storage means stores a designated area, which is an area in the image, that cannot identify a direction in which a communication device owned by the person specified by the face specifying means exists even if the face image is displayed. And
The direction specifying means can specify a direction in which the communication device exists if the center coordinate is included in the designated area, and can perform direction analysis using supplemental information that is information other than the area where the face image is displayed. Performing a direction analysis and specifying a direction in which a communication device owned by the person specified by the face specifying means exists according to the result of the direction analysis and the position of the center coordinate,
The information processing apparatus according to claim 1. It has a display on one side of the main body of the information processing device,
The imaging means is an in-camera that is provided on the same surface as the display and is installed at a position and a direction in which a face of a person who is browsing the display can be photographed. The image captured by the in-camera is the display Is not displayed in the
The information processing apparatus according to claim 1. Imaging means;
Input means for detecting the direction of the touch operation on the touch panel;
Storage means for storing registered face images of a plurality of persons in association with communication devices owned by each person;
The person associated with the detected face image is identified based on the face image detected from the image including the face image of the person imaged by the imaging unit and the registered face image stored in the storage unit Face identification means to
Direction specifying means for specifying a direction in which a communication device owned by the person specified by the face specifying means exists according to a region in which the face image in the image picked up by the image pickup means is displayed;
Determining means for determining whether or not the direction of the touch operation detected by the input means matches the direction specified by the direction specifying means;
A transmission means for transmitting electronic data to a communication device owned by the person specified by the face specifying means when they match as a result of the determination by the determination means;
Information processing device. Imaging means;
First storage means for storing a registered face image of a plurality of persons in association with a communication device owned by each person;
Based on the face image detected from the image including the face image of the person imaged by the imaging unit and the registered face image stored in the first storage unit, the image is associated with the detected face image. A face identification means for identifying a person,
Detecting means for detecting center coordinates of a region where the detected face image is displayed;
A second memory for storing a designated area, which is an area in the image, in which a direction in which a communication device owned by the person specified by the face specifying means is present cannot be specified even if the face image is displayed. Means,
If the center coordinate is included in the designated area, a communication device owned by the person specified by the face specifying means according to the area where the face image in the image picked up by the image pickup means is displayed. The supplementary information, which is information other than the area where the face image is displayed, can be used to identify the direction in which the communication device exists if the existing direction is specified and the central coordinate is not included in the designated area. Direction identification means for performing direction analysis and identifying a direction in which a communication device owned by the person identified by the face identification means exists according to the result of the direction analysis and the position of the central coordinate. And
Information processing device. Inclination detecting means for detecting the inclination of the main body of the information processing apparatus,
The direction specifying unit performs a direction analysis in which a communication device owned by the person specified by the face specifying unit exists with the tilt of the main body as supplementary information.
The information processing apparatus according to claim 9. Inclination detection means for detecting a change in the inclination of the main body of the information processing apparatus,
The direction specifying unit performs a direction analysis in which a communication device owned by the person specified by the face specifying unit exists using supplemental information on a change in tilt of the main body,
The information processing apparatus according to claim 9. The direction specifying means may be a communication device owned by the person specified by the face specifying means, using as a supplementary information a change in the area in which the face image is displayed in the images continuously captured by the imaging means. It is characterized by analyzing the existing direction,
The information processing apparatus according to claim 9. A method executed by an information processing apparatus comprising: an imaging unit; and a storage unit that stores registration face images for registration of a plurality of persons in association with communication devices owned by each person,
Based on the face image detected from the image including the face image of the person imaged by the imaging means and the registered face image stored in the storage means, the person associated with the detected face image is determined. Identifying steps;
Identifying a region in which the face image is displayed in the image captured by the imaging means;
Identifying a direction in which a communication device owned by the identified person is present in accordance with an area in which the identified face image is displayed,
Direction identification method. In a computer equipped with imaging means,
Storage means for storing a registered face image for registration of a plurality of persons in association with a communication device owned by each person;
Based on the face image detected from the image including the face image of the person imaged by the imaging means and the registered face image stored in the storage means, the person associated with the detected face image is determined. Face identification means to identify,
Direction specifying means for specifying a direction in which a communication device owned by the person specified by the face specifying means exists according to an area in which the face image in the image picked up by the image pickup means is displayed;
Computer program to form.