JP2016035625A - Information processing apparatus, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique that enables operations to be easily performed around an ear.SOLUTION: The presently disclosed invention provides an information processing apparatus characterized by comprising a detection unit that remotely detects movements of an object that is present around an ear of a user. According to the presently disclosed invention, it is possible to remotely detect the movements of an object via the detection unit, so the movements of the object, for example, movements of a hand of a user, can be remotely detected. Thus, operations around an ear can be performed easily, though of course, the effects obtainable from the presently disclosed invention are not limited to performing said operations.SELECTED DRAWING: Figure 1

Description

  The present disclosure relates to an information processing apparatus, an information processing method, and a program.

  Patent document 1 discloses the technique which provides a rod-shaped operation part in the temple part of a head mounted display (HMD). In this technique, the user manually operates the rod-shaped operation unit. That is, the user performs an operation around the ear.

JP 2014-49934 A

  However, in this technique, since the operation unit is in a position that is difficult for the user to see, the user needs to find the operation unit while moving his hand around the temple unit. For this reason, it takes time and effort to operate around the ear.

  For this reason, the technique which can perform operation around an ear easily was calculated | required.

  According to the present disclosure, there is provided an information processing apparatus including a detection unit that remotely detects a motion of an object existing around a user's ear.

  According to the present disclosure, an information processing method including remotely detecting a motion of an object existing around a user's ear by a detection unit is provided.

  According to the present disclosure, the computer recognizes the movement of the object based on the detection information output from the detection unit that remotely detects the movement of the object existing around the user's ear, and responds to the movement of the object. A program is provided that realizes a control function for performing the processing.

  According to the present disclosure, the motion of the object can be remotely detected by the detection unit.

  As described above, according to the present disclosure, since the movement of the object can be remotely detected by the detection unit, the movement of the object, for example, the movement of the user's hand can be remotely detected. Therefore, the operation around the ear can be easily performed. The technology according to the present disclosure may have any of the effects described in the present specification or other effects.

It is a perspective view showing an example of an information processor concerning an embodiment of this indication, and a head mounting member with which an information processor was equipped. It is a block diagram which shows the structure of an image sensor. It is a hardware block diagram of an image sensor. It is a block diagram which shows the structure of information processing apparatus. It is a hardware block diagram of an image analyzer. It is a top view which shows a mode that the user with which the head mounting member was mounted | closed brings a hand close to the circumference | surroundings of an ear | edge. It is a side view which shows an example of the direction where a user moves the hand around an ear | edge. It is a side view which shows the state which the user moved the hand forward. It is explanatory drawing which shows the state which the user approached the circumference | surroundings of the ear | edge. It is a flowchart which shows the outline | summary of the process by information processing apparatus. It is a flowchart which shows the procedure of the process by information processing apparatus. It is a flowchart which shows the procedure of the process by information processing apparatus. It is a state transition diagram of an information processor. It is explanatory drawing which shows an example of the captured image which an image sensor acquires. It is explanatory drawing which shows an example of the captured image which an image sensor acquires. It is explanatory drawing which shows an example of the captured image which an image sensor acquires. It is explanatory drawing which shows an example of the captured image which an image sensor acquires. It is explanatory drawing which shows an example of the captured image which an image sensor acquires. It is explanatory drawing which shows an example of the captured image which an image sensor acquires. It is explanatory drawing which shows an example of the captured image which an image sensor acquires. It is explanatory drawing which shows an example of the captured image which an image sensor acquires. It is explanatory drawing which shows an example of the captured image which an image sensor acquires. It is explanatory drawing which shows an example of the captured image which an image sensor acquires. It is explanatory drawing which shows an example of the captured image which an image sensor acquires. It is a state transition diagram which shows the modification of the process by information processing apparatus. It is a state transition diagram which shows the modification of the process by information processing apparatus. It is explanatory drawing which shows an example of the captured image which an image sensor acquires. It is explanatory drawing which shows the modification of the detection target by an image sensor. It is explanatory drawing which shows the example of the shape of a hand. It is explanatory drawing which shows an example of the captured image which an image sensor acquires. It is explanatory drawing which shows an example of the captured image which an image sensor acquires. It is explanatory drawing which shows an example of the captured image which an image sensor acquires. It is explanatory drawing which shows an example of the captured image which an image sensor acquires. It is explanatory drawing which shows an example of the captured image which an image sensor acquires. It is explanatory drawing which shows the area | region imaged with an image sensor, and the area | region which is not imaged. It is explanatory drawing which shows an example of the captured image which an image sensor acquires. It is a perspective view which shows the modification of the head mounting member with which the information processing apparatus was mounted | worn. It is a perspective view which shows the modification of a structure of information processing apparatus. It is explanatory drawing which shows the modification of a structure of information processing apparatus.

  Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

The description will be made in the following order.
1. 1. Configuration of information processing apparatus 1-1. Overall configuration 1-2. Configuration of image sensor 1-3. 1. Configuration of image analysis apparatus 2. Overview of hand gestures 3. Outline of processing by information processing apparatus 4. Procedure of processing by information processing apparatus Various modifications 5-1. First modification of information processing apparatus 5-2. Second modification of information processing apparatus 5-3. Third modification of information processing apparatus 5-4. Fourth modification example of information processing apparatus 5-5. Fifth Modification of Information Processing Device 5-6. Sixth Modification of Information Processing Device 5-7. Seventh modification of information processing apparatus 5-8. Eighth Modification of Information Processing Device 5-9. Ninth Modification of Information Processing Device 5-10. 10th modification of information processing apparatus 5-11. Eleventh modification of information processing apparatus 5-12. Twelfth modification of information processing apparatus 5-13. Thirteenth Modification of Information Processing Device 5-14. 14th modification of information processing apparatus

<1. Configuration of information processing apparatus>
(1-1. Overall configuration)
First, the overall configuration (outline) of the information processing apparatus 1 according to the present embodiment will be described with reference to FIG. As illustrated in FIG. 1, the information processing apparatus 1 includes an image sensor (detection unit) 10 and an image analysis apparatus 20. The image sensor 10 and the image analysis device 20 are connected by a signal line 30. Further, the image analysis device 20 is connected to the operation target device 200 by a signal line 40. The image sensor 10 and the image analysis device 20 are mounted on the head mounting member 100. The head mounting member 100 shown in FIG. 1 is glasses and is mounted on the head of the user U (see FIG. 6 and the like). Of course, the head mounting member 100 is not limited to glasses. Other examples include headphones, head mounted displays, goggles, earphones, helmets, hats and the like. The image sensor 10 and the image analysis device 20 are mounted on a portion of the head mounting member 100 that is mounted on the ear U10 of the user U, that is, the temple unit 110. The temple part 110 may be comprised with a raw material (for example, metal) with high heat conductivity. Thereby, the heat generated in the image sensor 10 and the image analysis device 20 can be efficiently transmitted to the temple unit 110, so that the image sensor 10 and the image analysis device 20 are efficiently cooled.

  The image sensor 10 remotely detects an object existing around the user's U ear (right ear) U10. Here, the remote detection means detecting an object that is not in contact with the image sensor 10. The image sensor 10 can also detect an object in contact with the image sensor 10. The object to be detected by the image sensor 10 is, for example, the hand (right hand) U1 of the user U. Specifically, the image sensor 10 acquires a captured image by imaging the periphery of the ear U10 and outputs captured image information (detection information) regarding the captured image to the image analysis device 20.

  The image analysis device 20 analyzes (recognizes) the motion of the object based on the captured image information, and performs processing according to the motion of the object. For example, the image analysis device 20 generates instruction information corresponding to the movement of the object and outputs the instruction information to the operation target device 200. The operation target device 200 performs various processes based on the instruction information. The operation target device 200 is a device that receives instruction information from the image analysis apparatus 20 and executes processing according to the instruction information. The type of the operation target device 200 is not particularly limited. The operation target device 200 may be, for example, an audio device, a display, a personal computer (including a desktop type and a notebook type), a mobile phone, a smartphone, a smart tablet, a camera attached to the body of the user U, an in-vehicle device, and the like. The specific content of the instruction information differs depending on the operation target device 200. When the operation target device 200 is an audio device, the instruction information is, for example, information for instructing fast forward, reproduction, rewind, and the like.

  In particular, when the operation target device 200 is a camera attached to the body of the user U (hereinafter also referred to as “motion imaging camera”), the operation target device 200 captures an image while the user U is performing any operation. Do. Therefore, the user U may have difficulty in accurately operating the operation target device 200. Therefore, when the operation target device 200 is a motion imaging camera and the user U skis, the information processing apparatus 1 is attached to goggles (head-mounted member 100). In this case, while enjoying skiing, the user U moves the hand U around the ear U10 and performs a hand gesture. In response to this, the image sensor 10 detects a hand gesture remotely. The image analysis apparatus recognizes the hand gesture and performs processing corresponding to the hand gesture (for example, recording start and recording end instruction to the operation target device 200). Therefore, the user U can easily operate the operation target device 200.

  Therefore, the user U moves the hand U1 to the periphery of the ear U10, and then performs some kind of hand gesture (gesture operation using the hand U1) to perform processing corresponding to the hand gesture on the operation target device 200. Can be made. Therefore, in this embodiment, unlike the technique disclosed in Patent Document 1, it does not take time to search for the operation unit. Therefore, in this embodiment, the user U can easily perform an operation around the ear U10.

(1-2. Configuration of Image Sensor 10)
Next, the configuration of the image sensor 10 will be described based on FIGS. 1 to 3 and FIG. 6. As shown in FIG. 1, the image sensor 10 is mounted outwardly on the temple portion 110 of the head mounting member 100 (that is, the lens surface of the image sensor 10 faces the outside of the temple portion 110). The image sensor 10 may be fixed to the temple portion 110 or may be detachable. Here, as a method for making the image sensor 10 detachable from the temple part 110, for example, the image sensor 10 may be fixed to the temple part 110 using screws or clips. In this case, the image sensor 10 and the image analysis device 20 can be removed from the head mounting member 100 by removing the screw or the clip. Thereby, the head mounting member 100 can be used also as normal spectacles.

  The image sensor 10 is preferably provided on the front side of the temple portion 110 (side closer to the front portion 120) so that the hair of the user U is not captured in the captured image as much as possible.

  The installation position of the image sensor 10 is not limited to the temple unit 110. In other words, the image sensor 10 may be provided at any position as long as it can capture the periphery of the user U's ear U10. Therefore, the image sensor 10 may be attached to the front part 120. In this case, the periphery of the ear U10 can be imaged by widening the angle of view of the image sensor 10 or by directing the optical axis toward the periphery of the ear U10. Further, in FIG. 1, the image sensor 10 is installed in the temple unit 110 for the right ear. This is because it is assumed that the user U performs a hand gesture with the right hand. Therefore, the image sensor 10 may be provided in the temple part for the left ear. In this case, the user U performs a hand gesture with the left hand.

  As shown in FIG. 2, the image sensor 10 includes an imaging unit (detection unit) 11 and a communication unit 12. The imaging unit 11 images the periphery of the user U's ear U10. The object to be detected by the imaging unit 11 is, for example, the hand U1 of the user U as described above. FIG. 6 shows an imageable range (detectable range) 10 </ b> A of the imaging unit 11. The imageable range 10A includes an area around the ear U10. In particular, in the present embodiment, the imageable range 10A includes a region facing the opening surface of the ear U10. Therefore, the imaging unit 11 can image an object present around the ear U10, particularly an object facing the opening surface of the ear U10. That is, the imaging unit 11 can remotely detect the movement of an object existing around the ear U10, particularly an object facing the opening surface of the ear U10.

  For example, as illustrated in FIG. 6, the imaging unit 11 can image the hand U1 when the user U moves the hand U1 from the front of the user U to the opening surface of the ear U10. That is, the imaging unit 11 can remotely detect the movement of the hand U1. Of course, the imageable range 10A is not limited to the example shown in FIG. That is, the imageable range 10A only needs to include at least the area around the ear U10 of the user U. The imaging unit 11 acquires a captured image by imaging, and outputs captured image information regarding the captured image to the communication unit 12. Here, the captured image is a color image and includes a plurality of pixels. Each pixel is associated with position information (coordinate information).

  The communication unit 12 is connected to the image analysis device 20 via the signal line 30 and outputs captured image information to the image analysis device 20. Note that the communication unit 12 may output the captured image information to the image analysis device 20 by wireless communication. In this case, the signal line 30 becomes unnecessary.

  FIG. 3 shows a hardware configuration of the image sensor 10. The image sensor 10 includes an imaging mechanism 11a and a communication device 12a as a hardware configuration. The imaging mechanism 11a and the communication device 12a are connected by a bus or the like. The image sensor 10 implements the above-described imaging unit (detection unit) 11 and communication unit 12 with these hardware configurations. These hardware configurations are realized by various hardware configurations necessary for imaging, electronic circuits, and the like. Various hardware configurations necessary for imaging include, for example, a lens, an imaging device, and the like. The imaging mechanism 11a acquires a captured image by imaging an object that exists in the imageable range 10A. And the imaging mechanism 11a produces | generates the captured image information regarding a captured image. Therefore, the imaging mechanism 11a is a substantial operation subject of the image sensor 10. The communication device 12 a is connected to the image analysis device 20 via the signal line 30 and outputs captured image information to the image analysis device 20. The communication device 12a may output the captured image information to the image analysis device 20 by wireless communication. In this case, the signal line 30 becomes unnecessary.

(1-3. Configuration of Image Analysis Device 20)
Next, the configuration of the image analysis device 20 will be described based on FIGS. 1, 4, and 5. As shown in FIG. 1, the image analysis device 20 is installed in the temple portion 110 of the head mounting member 100. The image analysis apparatus 20 may be fixed to the temple unit 110 or may be detachable. The method for making the image analysis device 20 detachable from the temple unit 110 is the same as that of the image sensor 10. Further, the image analysis device 20 may not be provided in the temple unit 110. That is, since the image analysis device 20 is a device that analyzes captured image information, the image analysis device 20 may be provided at any position as long as the captured image information can be analyzed. Modification examples regarding the installation position of the image analysis device 20 will be described later.

  As shown in FIG. 4, the image analysis device 20 includes a storage unit 21, a communication unit 22, and a control unit 23. The storage unit 21 stores a program necessary for the image analysis apparatus 20 to realize the storage unit 21, the communication unit 22, and the control unit 23. Further, the storage unit 21 stores captured image information, intermediate data when image analysis is performed, a final result of image analysis, and the like. The communication unit 22 is connected to the image sensor 10 via the signal line 30 and acquires captured image information from the image sensor 10. Then, the communication unit 22 outputs the captured image information to the control unit 23. The communication unit 22 is connected to the operation target device 200 via the signal line 40, and outputs the instruction information given from the control unit 23 to the operation target device 200. The communication unit 22 may perform wireless communication with the image sensor 10 and the operation target device 200. In this case, the signal lines 30 and 40 are not necessary.

  The control unit 23 controls each component in the image analysis device 20 and performs the following processing. That is, the control unit 23 analyzes (recognizes) the motion of the object based on the captured image information, and performs processing according to the motion of the object. For example, the control unit 23 generates instruction information corresponding to the movement of the object and outputs the instruction information to the operation target device 200. The operation target device 200 performs various processes based on the instruction information.

  The image analysis device 20 includes a memory 21a, a communication device 22a, and a CPU 23a as a hardware configuration. These hardware configurations are connected by, for example, a bus. The image analysis apparatus 20 implements the storage unit 21, the communication unit 22, and the control unit 23 described above with these hardware configurations. These hardware configurations are realized by an electronic circuit or the like.

  The memory 21a is composed of a ROM, a RAM, a nonvolatile memory, and the like. The memory 21 a stores programs necessary for the image analysis apparatus 20 to realize the storage unit 21, the communication unit 22, and the control unit 23. The memory 21a stores captured image information, intermediate data when image analysis is performed, a final result of image analysis, and the like.

  The communication device 22 a is connected to the image sensor 10 via the signal line 30 and acquires captured image information from the image sensor 10. Further, the communication device 22 a is connected to the operation target device 200 via the signal line 40 and outputs instruction information to the operation target device 200. Note that the communication device 22a may perform wireless communication with the image sensor 10 and the operation target device 200. In this case, the signal lines 30 and 40 are unnecessary. The CPU 23a reads and executes the program stored in the memory 21a. Therefore, the CPU 23a is a substantial operation subject of the image analysis apparatus 20.

  In addition, when the image sensor 10, the image analysis device 20, and the operation target device 200 are connected by the signal lines 30 and 40, the communication unit 22 and the communication device 22a may be omitted. In this case, the storage unit 21 and the control unit 23 (that is, the memory 21a and the CPU 23a), the image sensor 10, and the operation target device 200 are directly connected via the bus and the signal lines 30 and 40.

<2. Overview of hand gestures>
Next, based on FIG.6 and FIG.7, the hand gesture which the user U performs with respect to said information processing apparatus 1 is demonstrated. Of course, the hand gesture described below is an example, and the user U may perform other types of hand gestures.

  As shown in FIG. 6, the image sensor 10 captures an area around the ear U10. The region around the ear U10 includes a region facing the opening surface of the ear U10. Accordingly, as shown in FIGS. 6 and 7, the user U moves the hand U1 from the front toward the opening surface with the hand U1 open, and temporarily stops at a position facing the opening surface (operation (1)). That is, the user U moves the hand U1 to the side of the user U's head. Here, the direction (for example, “front”, “back”, “side”, etc.) of the present embodiment means a direction based on the gaze direction of the upright user. That is, the direction in which the user's line of sight faces is the front, the reverse direction of the front is the back, and the direction intersecting the user's line of sight is the side. Further, the user U may move the hand U1 in a direction other than the front, for example, from the side toward the opening surface of the ear U1.

  Then, the user U closes (holds) the hand U1. Thereafter, the user U performs any of the following operations (2) to (5). That is, the user U moves the hand horizontally and forward with the hand U1 closed (operation (2)). Or the user U moves a hand horizontally and back, with the hand U1 closed (operation (3)). Alternatively, the user U moves the hand vertically downward with the hand closed (operation (4)). Alternatively, the user U moves the hand vertically upward with the hand closed. As will be described later, the image analysis apparatus 20 performs processing according to operations (1) to (5).

  Thus, in this embodiment, the user U performs a hand gesture on the side of the user U's head. The reason for this will be described with reference to FIGS. The hand gesture is preferably performed with a posture as easy as possible for the user U. The hand gesture is preferably performed so as not to block the user U's line of sight as much as possible. Therefore, the present inventor conducted an experiment using a plurality of users U (subjects) in order to examine what kind of posture is easy for the user U. Specifically, each subject received the posture shown in FIG. 8 (posture A) and the posture shown in FIG. 9 (posture B). Posture A is a posture that projects the hand U1 forward, and posture B is a posture that holds the hand U1 on the side of the user U's head. As a result, most subjects answered that posture B is more comfortable than posture A. Furthermore, in the posture A, the arm and the hand U1 block the line of sight ahead, but in the posture B, the arm and the hand U1 hardly block the line of sight. That is, the user U can perform a hand gesture while gazing at a front object (not shown). Therefore, in this embodiment, the hand gesture is performed on the side of the user U's head.

<3. Overview of processing by information processing device>
Next, an outline of processing performed by the image analysis apparatus 20 will be described with reference to FIG. In step S <b> 1, the image sensor 10 acquires a captured image by imaging the periphery of the user U's ear U <b> 10. Then, the image sensor 10 transmits captured image information regarding the captured image to the image analysis device 20.

  The communication unit 22 of the image analysis device 20 acquires captured image information and outputs it to the control unit 23. The control unit 23 recognizes the movement of the user U's hand U1 by analyzing the captured image information. Then, based on the analysis result, the control unit 23 determines whether or not the hand U1 of the user U has moved from the front of the user U to the periphery of the ear U10 (that is, the imageable range 10A of the image sensor 10). That is, the control unit 23 determines whether or not the operation (1) described above has been performed. When it is determined that the operation (1) has been performed, the control unit 23 proceeds to step S2, and when it is determined that the operation (1) has not been performed, the control unit 23 returns to step S1.

  In step S <b> 2, the image sensor 10 acquires a captured image by imaging the periphery of the user U's ear U <b> 10. Then, the image sensor 10 transmits captured image information regarding the captured image to the image analysis device 20. The communication unit 22 of the image analysis device 20 acquires captured image information and outputs it to the control unit 23. The control unit 23 recognizes the movement of the user U's hand U1 by analyzing the captured image information. Then, the control unit 23 determines whether or not the user's hand has moved far from the periphery of the ear U10 (that is, out of the imageable range 10A of the image sensor 10). When it is determined that the user's hand has moved far from the periphery of the ear U10, the control unit 23 returns to step S1, and when it is determined that the user's hand has remained around the ear U10, the control unit 23 Proceed to S3.

  In step S <b> 3, the image sensor 10 acquires a captured image by imaging the periphery of the user U's ear U <b> 10. Then, the image sensor 10 transmits captured image information regarding the captured image to the image analysis device 20. The communication unit 22 of the image analysis device 20 acquires captured image information and outputs it to the control unit 23. The control unit 23 recognizes the movement of the user U's hand U1 by analyzing the captured image information. The control unit 23 performs processing according to the movement of the user U's hand U1. Then, the control part 23 complete | finishes this process.

<4. Processing procedure by information processing apparatus>
Next, a processing procedure performed by the image analysis apparatus 20 will be described with reference to flowcharts shown in FIGS. In step S10, the control unit 23 resets the time t indicated by the timer to zero. Here, the timer is set in the storage unit 21. The value of t increases by 1 every time the image sensor 10 performs imaging. Further, the state (Mode) of the hand U1 is turned off.

  In step S <b> 20, the image sensor 10 acquires a captured image by imaging the periphery of the user U's ear U <b> 10. Then, the image sensor 10 transmits captured image information regarding the captured image to the image analysis device 20. The communication unit 22 of the image analysis device 20 acquires captured image information and outputs it to the control unit 23. The control unit 23 analyzes the captured image information. Specifically, the control unit 23 detects a skin color area from the captured image. Specifically, the control unit 23 detects pixels whose pixel values are skin color from the captured image. Then, the control unit 23 sets the number (area) of the detected pixels as S (t) and sets the center of gravity position as M (t). S (t) is a scalar value, and M (t) is a two-dimensional vector. By detecting the skin color portion, it is possible to know the size of the hand (S (t)) and the position of the hand (M (t)) in the image. Note that a more robust method may be applied to the recognition of the hand and its movement.

  In step S30, the control unit 23 checks the value of Mode. When Mode is OFF, the control unit 23 proceeds to step S40. When the Mode is STANDBY, the control unit 23 proceeds to Step S7. When the Mode becomes READY, the control unit 23 proceeds to Step S100 illustrated in FIG.

  In step S40, the control unit 23 determines whether or not M (t) has entered the frame from the left end of the captured image with the passage of time (that is, increase in time t). Here, as shown in FIG. 6, when the user U moves the hand U1 from the front to the periphery of the ear U10, the hand U1 enters the imageable range 10A from the left side of the imageable range 10A in plan view. Therefore, in this case, M (t) enters the frame from the left end of the captured image. Therefore, in step S40, the control unit 23 determines whether or not the user U's hand U1 has entered the imageable range 10A from the front.

Specifically, for example, the control unit 23 acquires captured images for the past several frames (for example, 5 frames), and examines the value of M (t) in these captured images. That is, the control unit 23 checks whether M (t) has moved from the left end of the captured image to the center together with t. Here, examples in which M (t) moves from the left end of the captured image to the center are shown in FIGS. 14 to 19 show the captured image 10B at times t-5, t-4, t-3, t-2, t-1, and t (= current time). Hand image U1a shows user U's hand U1. In the example shown in FIGS. 14 to 19, M (t) of the hand image U <b> 1 a moves from the left end of the captured image to the center as the time t increases. In FIGS. 14 to 19, images other than the hand image U1a are omitted.

  If the control unit 23 determines that M (t) has entered the frame from the left end of the captured image, the control unit 23 proceeds to step S50. Otherwise, the control unit 23 proceeds to step S90. In step S50, the control unit 23 sets Mode to STANDBY. Thereafter, the control unit 23 proceeds to step S90.

  In step S60, the control unit 23 determines whether or not S (t) has decreased with the passage of time (that is, increase in time t). Here, when the user U closes the hand U1, the skin color area in the captured image is narrowed, so S (t) is small. Therefore, in step S60, the control unit 23 determines whether or not the user U has closed the hand U1.

  Specifically, for example, the control unit 23 acquires captured images for the past several frames, and examines the value of S (t) in these captured images. That is, the control unit 23 determines whether or not the scalar value of S (t) has decreased as time t increases. Note that it is possible that the value of S (t) gradually decreases as the hand U1 goes out of the frame. Therefore, the control unit 23 further determines whether or not M (t) is substantially constant (variation within a predetermined threshold value) regardless of an increase in time t when making a stricter determination. Also good.

  If it is determined that S (t) has decreased, the control unit 23 proceeds to step S70, and otherwise proceeds to step S90. In step S70, the control unit 23 sets Mode to READY. Thereafter, the control unit 23 proceeds to step S90.

  In step S80, the control unit 23 determines whether or not M (t) is out of frame with the passage of time (that is, increase in time t). That is, the control unit 23 determines whether or not the hand U1 of the user U has moved outside the imageable range 10A of the image sensor 10 (that is, around the ear U10).

  Specifically, for example, the control unit 23 acquires captured images for the past several frames, and examines the value of M (t) in these captured images. That is, the control unit 23 determines whether or not M (t) has moved from the center of the image to the periphery with the increase of t, and finally moved out of the captured image. If the control unit 23 determines that M (t) is out of frame as time passes, the control unit 23 proceeds to step S190 shown in FIG. Otherwise, the control unit 23 proceeds to step S90.

  In step S90, the control unit 23 waits for a unit time, and then increases the time t indicated by the timer by one. Thereafter, the control unit 23 returns to Step S20.

  In step S100 illustrated in FIG. 12, the control unit 23 determines whether or not M (t) is out of the frame from the left end of the captured image as time elapses (that is, time t increases). Here, as shown in FIGS. 6 and 7, when the user U performs the operation (2), the hand U1 moves to the left from the center of the imageable range 10A in plan view, and then the imageable range 10A. Deviate from. Therefore, in this case, M (t) frames out from the left end of the captured image. Therefore, in step S100, the control unit 23 determines whether or not the user U has performed the operation (2).

  Specifically, for example, the control unit 23 acquires captured images for the past several frames, and examines the value of M (t) in these captured images. That is, the control unit 23 determines whether or not M (t) has moved from the center of the image to the left end with the increase of t, and finally moved out of the captured image.

  If it is determined that the user U has performed the operation (2), the control unit 23 proceeds to step S110. Otherwise, the control unit 23 proceeds to step S120. In step S120, the control unit 23 generates instruction information for instructing the operation target device 200 to perform the process D1. The communication unit 22 transmits instruction information to the operation target device 200.

  Here, the content of the process D1 differs depending on the operation target device 200, but the process D1 is preferably associated with the content of the hand gesture. That is, since the operation (2) is a gesture operation for moving the hand forward, the user U imagines that some state quantity is displaced in the positive direction. Therefore, it is preferable that the process D1 is to shift the state quantity related to the operation target device 200 in the positive direction. For example, when the operation target device 200 is an audio device, the process D1 is preferably fast-forwarding or playback. The distinction between fast-forwarding and reproduction may be performed at a frame-out speed, for example. That is, fast-forwarding is performed when the frame-out speed is equal to or higher than a predetermined value, and reproduction is performed when the frame-out speed is lower than the predetermined value. Further, when the operation target device 200 is a display, the process D1 is preferably image enlargement or the like. The frame-out speed and the image enlargement speed may be associated with each other. In this way, the user U can more intuitively operate the operation target device 200 by associating the process D1 with the content of the hand gesture. Thereafter, the control unit 23 proceeds to step S190.

  In step S120, the control unit 23 determines whether or not M (t) is out of the frame from the right end of the captured image as time elapses (that is, time t increases). Here, as shown in FIGS. 6 and 7, when the user U performs the operation (3), the hand U1 moves to the right from the center of the imageable range 10A in plan view, and then the imageable range 10A. Deviate from. Therefore, in this case, M (t) is out of the frame from the right end of the captured image. Therefore, in step S100, the control unit 23 determines whether or not the user U has performed the operation (3).

  Specifically, for example, the control unit 23 acquires captured images for the past several frames, and examines the value of M (t) in these captured images. That is, the control unit 23 determines whether or not M (t) has moved from the center of the image to the right end with the increase of t, and finally moved out of the captured image.

  If it is determined that the user U has performed the operation (3), the control unit 23 proceeds to step S130, and otherwise proceeds to step S140. In step S130, the control unit 23 generates instruction information for instructing the operation target device 200 to perform the process D2. The communication unit 22 transmits instruction information to the operation target device 200.

  Here, the content of the process D2 differs depending on the operation target device 200, but the process D2 is preferably associated with the content of the hand gesture. That is, since the operation (2) is a gesture operation for moving the hand backward, the user U imagines that some state quantity is displaced in the negative direction. Therefore, it is preferable that the process D2 is to displace the state quantity related to the operation target device 200 in the negative direction. For example, when the operation target device 200 is an audio device, the process D1 is preferably rewinding or reverse playback. The distinction between rewinding and reverse reproduction may be performed at a frame-out speed, for example. That is, rewinding may be performed if the frame-out speed is equal to or higher than a predetermined value, and reverse playback is performed if the frame-out speed is lower than the predetermined value. When the operation target device 200 is a display, the process D2 is preferably image reduction or the like. The frame-out speed and the image reduction speed may be associated with each other. Thus, the user U can operate the operation target device 200 more intuitively by associating the process D2 with the content of the hand gesture. Thereafter, the control unit 23 proceeds to step S190.

  In step S140, the control unit 23 determines whether M (t) is out of the frame from the lower end of the captured image as time elapses (that is, time t increases). Here, as illustrated in FIG. 7, when the user U performs the operation (4), the hand U1 moves vertically downward from the center of the imageable range 10A in a side view, and then deviates from the imageable range 10A. . Therefore, in this case, M (t) frames out from the lower end of the captured image. Therefore, in step S100, the control unit 23 determines whether or not the user U has performed the operation (4).

  Specifically, for example, the control unit 23 acquires captured images for the past several frames, and examines the value of M (t) in these captured images. That is, the control unit 23 determines whether or not M (t) moves from the center of the image to the lower end as t increases, and finally moves outside the captured image. Here, examples in which M (t) moves from the center of the captured image to the lower end are shown in FIGS. 20 to 24 and 14 show the captured image 10B at times t-5, t-4, t-3, t-2, t-1, and t (= current time). Hand image U1a shows user U's hand U1. In the example shown in FIGS. 20 to 24 and 14, M (t) of the hand image U <b> 1 a moves from the center of the captured image to the lower end as the time t increases.

  When it is determined that the user U has performed the operation (4), the control unit 23 proceeds to step S150, and otherwise proceeds to step S160. In step S160, the control unit 23 generates instruction information for instructing the operation target device 200 to perform the process D3. The communication unit 22 transmits instruction information to the operation target device 200.

  Here, the content of the process D3 differs depending on the operation target device 200, but the process D3 is preferably associated with the content of the hand gesture. That is, since the operation (4) is a gesture operation for moving the hand downward, the user U imagines that some state quantity is displaced in the negative direction. Therefore, it is preferable that the process D3 is to displace the state quantity related to the operation target device 200 in the negative direction. For example, when the operation target device 200 is an audio device, the process D3 is preferably volume down or the like. When the operation target device 200 is a display, the process D2 is preferably image reduction or the like. In this way, the user U can more intuitively operate the operation target device 200 by associating the process D3 with the content of the hand gesture.

  In step S160, the control unit 23 determines whether or not M (t) is out of the frame from the upper end of the captured image as time elapses (that is, time t increases). Here, as shown in FIG. 7, when the user U performs the operation (5), the hand U1 moves vertically upward from the center of the imageable range 10A in a side view, and then deviates from the imageable range 10A. . Therefore, in this case, M (t) is out of the frame from the upper end of the captured image. Therefore, in step S100, the control unit 23 determines whether or not the user U has performed the operation (5).

  Specifically, for example, the control unit 23 acquires captured images for the past several frames, and examines the value of M (t) in these captured images. That is, the control unit 23 determines whether or not M (t) moves from the center of the image to the upper end as t increases, and finally moves outside the captured image.

  When it is determined that the user U has performed the operation (5), the control unit 23 proceeds to step S170, and otherwise proceeds to step S180. In step S170, the control unit 23 generates instruction information for instructing the operation target device 200 to perform the process D4. The communication unit 22 transmits instruction information to the operation target device 200.

  Here, the content of the process D4 differs depending on the operation target device 200, but the process D4 is preferably associated with the content of the hand gesture. That is, since the operation (5) is a gesture operation for moving the hand upward, the user U imagines that some state quantity is displaced in the positive direction. Therefore, it is preferable that the process D4 is to shift the state quantity related to the operation target device 200 in the positive direction. For example, when the operation target device 200 is an audio device, the process D4 is preferably volume up. When the operation target device 200 is a display, the process D2 is preferably image enlargement or the like. In this way, the user U can more intuitively operate the operation target device 200 by associating the process D4 with the contents of the hand gesture.

  In step S180, the control unit 23 determines whether or not M (t) has been framed out of the captured image in a mode other than the operations (2) to (5) as time passes (that is, the time t increases). To do. As such an aspect, for example, M (t) moves in an oblique direction in the captured image.

  Specifically, for example, the control unit 23 acquires captured images for the past several frames, and examines the value of M (t) in these captured images. That is, the control unit 23 determines whether or not M (t) moves in the oblique direction as t increases and finally moves out of the captured image.

  When the control unit 23 determines that M (t) has been out of the captured image in a mode other than the operations (2) to (5) with the passage of time (that is, the increase in time t), step S190 is performed. Otherwise, the process returns to step S90 shown in FIG.

  In step S190, the control unit 23 changes (shifts) the state of the hand U1 to OFF. Thereafter, the process returns to step S90.

  FIG. 13 is a state transition diagram showing state transitions by the above processing. When Mode is OFF, the control unit 23 stands by until M (t) enters the frame from the left end of the captured image.

  When M (t) enters the frame from the left end of the captured image, the control unit 23 changes the state of the hand U1 from OFF to STANDBY. The control unit 23 waits until S (t) decreases, that is, until the user U closes the hand U1. Note that the control unit 23 transitions the state of the hand U1 from STANDBY to OFF when M (t) is out of the captured image.

  When S (t) becomes small, the controller 23 changes the state of the hand U1 from STANDBY to READY. The control unit 23 stands by until any one of the operations (2) to (5) is performed. When any of the operations (2) to (5) is performed (that is, when M (t) is out of the frame from any of the left end, the right end, the upper end, and the lower end of the captured image), the control unit 23 Instruction information corresponding to the operation is generated. Thereafter, the control unit 23 shifts the state of the hand U1 to OFF. Control part 23 makes the state of hand U1 change to OFF, without generating directions information, when M (t) carries out a frame out from a picked-up picture in modes other than operation (2)-(5).

  Thus, according to the present embodiment, the user U moves the hand U1 to the periphery of the ear U10, for example, a position facing the opening surface, and performs a hand gesture. Then, the information processing apparatus 1 causes the operation target device 200 to perform processing according to the content of the hand gesture. Therefore, since the user can cause the operation target device 200 to perform a desired process simply by moving the hand U1 around the ear U10 and performing a hand gesture, the user can easily perform the operation around the ear U10. Can be done.

  That is, according to the present embodiment, the information processing apparatus 1 includes the image sensor 10 that remotely detects the movement of an object existing around the user's U ear U10. Therefore, the image sensor 10 can remotely detect such a hand gesture when the user U moves the hand U1 around the ear U10 and performs a hand gesture. Therefore, the user U can easily perform an operation around the ear.

  Here, the image sensor 10 remotely detects the movement of an object present at a position facing the opening surface of the user U's ear U10. Here, since the opening surface of the ear U10 is a position that is easy for the user U to understand, the user U can easily move the hand U1 to this position. Therefore, the user U can easily perform an operation around the ear.

  Further, the information processing apparatus 1 recognizes the movement of the object based on the captured image information output from the image sensor 10 and performs a process according to the movement of the object. Therefore, the user can cause the information processing apparatus 1 to perform processing according to the hand gesture.

  Further, the information processing apparatus 1 recognizes the movement of the object based on the captured image information output from the image sensor 10, and the object is within the imageable range 10A from a specific direction (preferably in front of the user U). It is determined whether or not the vehicle has entered. If the information processing apparatus 1 determines that the object has entered the imageable range 10A from the front of the user U, the information processing apparatus 1 shifts the state of the object to the standby state. Then, the information processing apparatus 1 recognizes the movement of the object based on the captured image information output from the image sensor 10 after the state of the object has shifted to the standby state, and performs processing according to the movement of the object.

  That is, in the present embodiment, the gesture recognition start trigger is that the object (hand) has moved from the specific direction (preferably in front of the user U) to the periphery of the ear U10. The operation of moving the hand U1 from the front of the user U around the ear U10 (operation (1)) is often a natural operation for the user U. That is, the user U can make the information processing apparatus 1 perform gesture recognition only by performing a natural operation.

  Furthermore, the information processing apparatus 1 determines whether the shape of the object is a specific shape (for example, a shape with a hand closed) based on the imageable information output from the image sensor 10 after the state of the object has shifted to the standby state. Determine whether. If the information processing apparatus 1 determines that the shape of the object is a specific shape, the information processing apparatus 1 shifts the state of the object to the ready state. Then, the information processing apparatus 1 recognizes the movement of the object based on the captured image information output from the image sensor 10 after the state of the object shifts to the ready state, and performs processing according to the movement of the object. . Thereby, the information processing apparatus 1 can perform gesture recognition more accurately.

<5. Various modifications>
(5-1. First Modification of Information Processing Device)
Next, a first modification of the information processing apparatus 1 will be described based on FIG. In the first modification, the control unit 23 performs the following process when the state of the hand U1 is STANDBY.

  That is, the control unit 23 determines whether the hand U1 has performed a tap operation based on the captured image information. When the tap operation is performed, the control unit 23 generates instruction information for instructing the operation target device 200 to perform the process D5. The content of the process D5 is preferably different depending on the type of tap operation (the number of tap operations). For example, when the operation target device 200 is an audio device, it is preferable that the process D5 is playback, reverse playback, fast forward, rewind, or the like according to the number of taps. When the operation target device 200 is a display, it is preferable that the process D5 is image enlargement, image reduction, or the like according to the number of taps. Note that the control unit 23 maintains the state of the hand U1 at STANDBY.

(5-2. Second Modification of Information Processing Device)
Next, a second modification of the information processing apparatus 1 will be described based on FIG. The second modification differs from the processes shown in FIGS. 11 to 13 in the processes after STANDBY.

  Specifically, when the state of the hand U1 shifts to the STANDBY state, the control unit 23 waits until S (t) becomes small. Then, when S (t) becomes small, the control unit 23 further determines whether or not S (t) has decreased to a predetermined threshold value or less. If the control unit 23 determines that S (t) has decreased to a predetermined threshold value or less, the control unit 23 changes the state of the hand U1 to the first READY state (READY state shown in FIG. 13). Then, the control part 23 performs the process shown in FIGS. On the other hand, when determining that S (t) is larger than the predetermined threshold, the control unit 23 changes the state of the hand U1 to the second READY state shown in FIG.

  The second READY state is a state where the hand U1 is holding a virtual dial. An example of a captured image corresponding to the second READY state is shown in FIG. In the captured image 10B shown in FIG. 27, a hand image U1a holding the virtual dial 10C is drawn. S (t) of the hand image U1a is larger than S (t) of the hand image U1a shown in FIG.

  And the control part 23 determines whether the hand U1 rotated based on the captured image information. When the hand U1 rotates, the control unit 23 generates instruction information for instructing the operation target device 200 to perform the process D6. The content of the process D6 is preferably different depending on the rotation content (rotation direction and speed). For example, when the operation target device 200 is an audio device, it is preferable that the process D6 is reproduction, reverse reproduction, fast forward, rewind, or the like according to the rotation content. For example, if the rotation direction is right rotation, the process D6 is reproduction or fast forward. These distinctions may be made based on the speed of the number of times. Further, when the operation target device 200 is a display, it is preferable that the process D6 is image enlargement, image reduction, or the like according to the rotation content. The speed of rotation may be associated with the speed of image enlargement or reduction. Note that the control unit 23 maintains the state of the hand U1 in the second READY state. When M (t) is out of frame, the control unit 23 shifts the state of the hand U1 to OFF.

  As described above, in the second modification, when the user U performs a hand gesture by grasping and turning the virtual dial, a process corresponding to the hand gesture is performed. That is, in the second modification, the control unit 23 changes the state of the hand U1 to a different READY state according to the shape of the hand U1. Thereafter, the control unit 23 performs processing according to the movement of the hand U1 and the type of the READY state. In this case, the user U can perform more various hand gestures.

(5-3. Third Modification of Information Processing Device)
Next, a third modification of the information processing apparatus 1 will be described based on FIG. In the third modified example, as shown in FIG. 28, the user U is caused to wear a specific color ring U2 on his / her finger and perform a hand gesture. The ring U2 moves in conjunction with the user U's hand. The image sensor 10 remotely senses the movement of the ring U2 by imaging the ring U2. When recognizing the hand gesture (for example, when performing steps S40, S60, S80, S100, S120, S140, S160, and S180), the control unit 23 uses the movement of the ring U2 instead of the movement of the hand U1. Recognize as a hand gesture. When the image sensor 10 images a hand not wearing the ring U <b> 2, the control unit 23 recognizes the hand as a hand of another person other than the user U. In this case, the control unit 23 does not recognize the hand gesture. Therefore, in the third modified example, it is difficult to mistake a hand gesture by another person's hand as a hand gesture by the user U. That is, more robust gesture recognition is possible.

  Of course, the object that can be remotely detected by the image sensor 10 is not limited to a hand or a ring. For example, the image sensor 10 may remotely detect the movement of a stick, a pen, or the like. In this case, the user U may move a stick, pen, or the like within the imageable range 10 </ b> A of the image sensor 10.

(5-4. Fourth Modification of Information Processing Device)
In the above-described embodiment, the control unit 23 recognizes two states of the hand, that is, an open state and a gripped state. In the fourth modification, the control unit 23 recognizes another shape. An example of the shape is shown in FIG. In the example shown in FIG. 29, the thumb and little finger of the hand U1 are extended, and the other three fingers are bent. For example, after performing the above-described operation (1), the user U changes the shape of the hand U1 to the shape shown in FIG. Thereafter, the user U performs the operations (2) to (5) described above. The control unit 23 recognizes the hand gesture and performs the above-described processing (such as an instruction to the operation target device 200). Others who are not conscious of hand gestures rarely have their hands in the shape shown in FIG. Therefore, in the fourth modified example, it is difficult to mistake a hand gesture by another person's hand as a hand gesture by the user U. That is, more robust gesture recognition is possible.

(5-5. Fifth Modification of Information Processing Device)
In the fifth modification, the fingerprint of the user U is registered in the storage unit 21 in advance. Then, the control unit 23 performs fingerprint recognition when recognizing a hand gesture (for example, when performing steps S40, S60, S80, S100, S120, S140, S160, and S180). That is, the control unit 23 compares a fingerprint drawn on the captured image with a pre-registered fingerprint, and recognizes a hand gesture when they match. Therefore, in the fifth modification, it is difficult to mistake a hand gesture by another person's hand as a hand gesture by the user U. That is, more robust gesture recognition is possible.

(5-6. Sixth Modification of Information Processing Device)
In the sixth modification, the control unit 23 recognizes the orientation of the hand image in the captured image when recognizing the hand gesture (for example, when performing steps S40, S60, S80, S100, S120, S140, S160, and S180). Is analyzed. That is, as shown in FIGS. 30 and 31, if the hand image U1a of the user U is drawn on the captured image 1B, the wrist of the hand image U1a is directed to the lower left or the lower direction and the fingertip is directed to the upper or the upper right direction Should be facing. This is apparent from the position of the hand U1 shown in FIG. Conversely, as shown in FIGS. 32 and 33, when the hand image U20a of another person is depicted in the captured image 1B, the wrist of the hand image 20a faces in another direction (for example, right direction, upper left direction, etc.). There are many.

  Therefore, the control unit 23 determines whether or not the hand image is the user U's hand image U1a based on the orientation of the hand image. When the control unit 23 determines that the hand image is the user U's hand image U1a, the control unit 23 recognizes the hand gesture. Therefore, in the sixth modified example, it is difficult to mistake a hand gesture by another person's hand as a hand gesture by the user U. That is, more robust gesture recognition is possible.

(5-7. Seventh Modification of Information Processing Device)
In the seventh modification, the image sensor 10 also performs distance measurement. Examples of such an image sensor 10 include a stereo camera (having two sets of the imaging mechanism 11a). Another specific example of the image sensor 10 is a camera that performs distance measurement using the phase difference of the image plane.

  The control unit 23 recognizes the hand gesture (for example, when performing steps S40, S60, S80, S100, S120, S140, S160, and S180), the hand is a predetermined distance (for example, 30 cm) from the image sensor 10. It is determined whether or not it exists inside. When the control unit 23 determines that the hand is within a predetermined distance from the image sensor 10, the control unit 23 recognizes the hand as the hand U1 of the user U and recognizes the hand gesture. On the other hand, when the control unit 23 determines that the distance from the hand to the image sensor 10 is greater than the predetermined distance, the control unit 23 recognizes the hand as a hand of another person. In this case, the control unit 23 does not recognize the hand gesture. Therefore, in the seventh modified example, it is difficult to mistake a hand gesture by another person's hand as a hand gesture by the user U. That is, more robust gesture recognition is possible.

(5-8. Eighth Modification of Information Processing Device)
As described above, the image sensor 10 images the side of the user U. Therefore, the image sensor 10 can image an object that exists in the blind spot of the user U (that is, it is difficult or invisible to the user U). Therefore, the control unit 23 recognizes an object that approaches the user U from the blind spot of the user U based on the captured image. For example, as shown in FIG. 34, when the automobile image 10D is drawn in the captured image 1B, the control unit 23 determines that there is an automobile in the blind spot. Furthermore, the control unit 23 determines whether or not the vehicle suddenly approaches the user U. For example, when the automobile image 10D is enlarged over time, the control unit 23 may determine that the automobile is rapidly approaching. This determination may be performed using the seventh modification. When it is determined that there is an object that approaches the user U from the blind spot of the user U, the control unit 23 performs an emergency process. Examples of the emergency process include a process of turning off the operation target device 200 and notifying the user U that the danger is imminent. As a method of notifying the user U, there is a method of notifying using the operation target device 200 (for example, using the speaker if the operation target device 200 is an audio device). As another method, there is a method of installing a speaker or the like in the image analysis apparatus 20 and informing using the speaker. According to the eighth modification, the user U can recognize that there is an object that is rapidly approaching the user U.

(5-9. Ninth Modification of Information Processing Device)
The control unit 23 turns on / off the power of the image sensor 10 and the image analysis device 20 in conjunction with turning on / off the power of the operation target device 200. That is, when the operation target device 200 is turned on, a command (signal) to turn on the power is sent from the operation target device 200 to the image analysis device 20 via the signal line 40. Upon receiving this command, the control unit 23 turns on the power of the image sensor 10 and the image analysis device 20. The same applies to OFF. Accordingly, when the operation target device 200 is not used, the image sensor 10 and the image analysis device 20 do not consume power. In this case, the signal line 40 is used for bidirectional communication.

(5-10. Tenth Modification of Information Processing Device)
When the mode of the hand U1 is turned off, the control unit 23 turns on a portion corresponding to the left region 10B-1 of the captured image 10B in the image sensor of the image sensor 10, as shown in FIG. The other area 10B-2 is turned off. That is, the image sensor 10 captures an image using only a portion corresponding to the left region 10B-1 in the image sensor. That is, when the operation (1) serving as the trigger for hand gesture recognition is performed, the hand image U1a is first drawn in the left region 10B-1 as shown in FIGS. Therefore, if only the left area 10B-1 can be imaged, it can be determined whether or not the user U has performed the operation (1). According to the tenth modification, the power consumption of the image sensor 10 is further reduced.

(5-11. Eleventh Modification of Information Processing Device)
As shown in FIG. 36, the control unit 23 determines that the hair of the user U is applied to the image sensor 10 when a plurality of line images 10E extending in the vertical direction (vertical direction) are drawn on the captured image 10B. To do. In this case, since the imaging is disturbed by the hair of the user U, the control unit 23 performs an imaging disturbance process. Examples of the processing at the time of imaging disturbance include processing of turning off the power of the operation target device 200 and notifying the user U that the hair is applied to the image sensor 10. The method for notifying the user U may be the same method as in the above-described eighth modification. According to the eleventh modification, the user can recognize that appropriate imaging cannot be performed due to the hair.

(5-12. Twelfth Modification of Information Processing Device)
FIG. 37 shows the information processing apparatus 1 according to the twelfth modification. In this modification, the information processing apparatus 1 is mounted on the head mounting member 300. The head mounting member 300 is a headphone. In this example, the information processing apparatus 1 is mounted on an ear-cup (ear cup) portion of the head mounting member 300. The manner of mounting is the same as that shown in FIG. Also according to this modification, the same effect as described above can be obtained.

(5-13. Thirteenth Modification of Information Processing Device)
FIG. 33 shows an information processing apparatus 1 according to a thirteenth modification. In this modification, the signal line 30 is extended and the image analysis device 20 is detached from the head-mounted member 100. In this example, the head mounting member 100 is reduced in weight. The user U1 may put the image analysis device 20 in a clothes pocket, for example.

(5-14. Fourteenth Modification of Information Processing Device)
In the information processing apparatus 1 described above, an object is remotely detected using the image sensor 10, but any sensor that can remotely detect an object is applicable to the information processing apparatus 1. For example, if only the movement of the object is detected remotely, the information processing apparatus 1 may use proximity sensors 51 to 54 as shown in FIG.

  The proximity sensors 51 to 54 are arranged in the case 50. Two proximity sensors 51 to 54 are arranged in each of the vertical direction (y direction) and the horizontal direction (x direction). The case 50 is disposed at the same position as the image sensor 10 of FIG. The proximity sensors 51 to 54 are sensors that can detect that an object is approaching. The control unit 23 identifies the sensor that reacts first and the sensor that reacts last among the proximity sensors 51 to 54, so that the object (hand) of the sensor that reacts last from the position of the sensor that reacts first. You can see that it has moved to the position. The control unit 23 may perform gesture recognition using this information.

  Needless to say, the above modifications may be arbitrarily combined. For example, the first modification and the second modification may be combined. In this case, the control unit 23 performs processing according to the tap operation and the operation of rotating the virtual dial. Further, the first or second modification and the third to seventh modifications may be combined. In this case, the user U performs a tap operation and an operation of rotating a virtual dial, and the control unit 23 performs processing corresponding to these. Moreover, the control part 23 performs the process (process which improves robustness) which concerns on a 3rd modification-a 7th modification with the above.

  Moreover, you may combine an 8th modification and another modification. In this case, the control unit 23 performs an emergency process when the object suddenly approaches from the blind spot of the user U while performing a process according to another modification. Moreover, you may combine a 9th modification and another modification. In this case, the control unit 23 performs processing according to another modification, and turns off the image sensor 10 and the image analysis device 20 in conjunction with the power source of the operation target device 200.

  Moreover, you may combine a 10th modification and another modification. In this case, while performing the process according to another modification, the control unit 23 turns on the region drinking corresponding to the left region 10B-1 in the image sensor until the operation (1) is performed.

  Moreover, you may combine an 11th modification and another modification. In this case, the control unit 23 performs the process according to another modification, while performing the imaging disturbance process when the image sensor 10 is covered with hair.

  Further, any one of the twelfth to fourteenth modifications may be combined with another modification. In this case, the information processing apparatus 1 has a configuration shown in any of the 12th to 14th modified examples. Furthermore, the control part 23 performs the process which concerns on another modification.

  In addition, the said effect is an illustration to the last, Comprising: This embodiment may have either the effect described in this specification, or another effect.

  The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the technical scope of the present disclosure is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field of the present disclosure can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that it belongs to the technical scope of the present disclosure.

The following configurations also belong to the technical scope of the present disclosure.
(1)
An information processing apparatus comprising a detection unit that remotely detects a movement of an object existing around a user's ear.
(2)
The information processing apparatus according to (1), wherein the detection unit remotely detects a movement of an object existing at a position facing an opening surface of the user's ear.
(3)
The information processing apparatus according to (1) or (2), wherein the detection unit remotely detects at least one of the user's hand and an interlocking member that moves in conjunction with the user's hand.
(4)
Any one of (1) to (3), further comprising a control unit that recognizes the movement of the object based on detection information output from the detection unit and performs processing according to the movement of the object. The information processing apparatus described.
(5)
The control unit recognizes the movement of the object based on the detection information output from the detection unit, determines whether the object has entered the detectable range of the detection unit from a specific direction,
When it is determined that the object has entered the detectable range of the detection unit from the specific direction, the state of the object is shifted to a standby state,
The method according to (4), wherein the movement of the object is recognized based on detection information output from the detection unit after the state of the object has shifted to the standby state, and processing according to the movement of the object is performed. Information processing device.
(6)
The control unit recognizes the movement of the object based on the detection information output from the detection unit, and determines whether the object has entered the detectable range of the detection unit from the front of the user. The information processing apparatus according to (5).
(7)
The control unit determines whether the shape of the object is a specific shape based on detection information output from the detection unit after the state of the object has shifted to the standby state,
When it is determined that the shape of the object is the specific shape, the state of the object is shifted to the ready state,
Recognizing the movement of the object based on detection information output from the detection unit after the state of the object has shifted to the ready state, and performing a process according to the movement of the object. 6) The information processing apparatus described.
(8)
There are a plurality of the specific shapes,
The control unit matches the specific shape of the plurality of specific shapes based on detection information output from the detection unit after the state of the object has shifted to the standby state. Determine whether or not to
When it is determined that the shape of the object matches any one of the plurality of specific shapes, the state of the object is shifted to a ready state according to the shape of the object,
Based on detection information output from the detection unit after the state of the object has shifted to the ready state, the movement of the object is recognized, and processing according to the type of the movement of the object and the ready state is performed. The information processing apparatus according to (7).
(9)
The information processing apparatus according to any one of (1) to (8), wherein the detection unit is mounted on a head mounting member that can be mounted on the head of the user.
(10)
The information processing apparatus according to (9), wherein the detection unit is mounted on a portion of the head mounting member that is mounted on an ear of the user.
(11)
The information processing apparatus according to any one of (1) to (10), wherein the detection unit is an image sensor capable of capturing an image around the user's ear.
(12)
An information processing method including remotely detecting a motion of an object existing around a user's ear by a detection unit.
(13)
The information processing method according to (12), further comprising: recognizing a motion of the object based on detection information output from the detection unit and performing a process according to the motion of the object.
(14)
Based on the detection information output from the detection unit, recognize the movement of the object, determine whether the object has entered the detectable range of the detection unit from the front of the user,
When it is determined that the object has entered the detectable range of the detection unit from the front of the user, transition to a standby state,
The information processing method according to (13), wherein the movement of the object is recognized based on the detection information output from the detection unit after transitioning to the standby state, and processing according to the movement of the object is performed.
(15)
On the computer,
Based on detection information output from a detection unit that remotely detects the movement of an object existing around the user's ear, a control function that recognizes the movement of the object and performs processing according to the movement of the object is realized. ,program.
(16)
The control function recognizes the movement of the object based on the detection information output from the detection unit and determines whether the object has entered the detectable range of the detection unit from the front of the user. And
When it is determined that the object has entered the detectable range of the detection unit from the front of the user, transition to a standby state,
The program according to (15), wherein the program recognizes the movement of the object based on detection information output from the detection unit after shifting to the standby state, and performs processing according to the movement of the object.

DESCRIPTION OF SYMBOLS 1 Information processing apparatus 10 Image sensor 20 Image analysis apparatus 21 Storage part 22 Communication part 23 Control part

Claims (16)

  An information processing apparatus comprising a detection unit that remotely detects a movement of an object existing around a user's ear.   The information processing apparatus according to claim 1, wherein the detection unit remotely detects a movement of an object existing at a position facing an opening surface of the user's ear.   The information processing apparatus according to claim 1, wherein the detection unit remotely detects at least one of the user's hand and an interlocking member that moves in conjunction with the user's hand.   The information processing apparatus according to claim 1, further comprising: a control unit that recognizes the movement of the object based on detection information output from the detection unit and performs processing according to the movement of the object. The control unit recognizes the movement of the object based on the detection information output from the detection unit, determines whether the object has entered the detectable range of the detection unit from a specific direction,
When it is determined that the object has entered the detectable range of the detection unit from the specific direction, the state of the object is shifted to a standby state,
The information according to claim 4, wherein the movement of the object is recognized based on detection information output from the detection unit after the state of the object has shifted to the standby state, and processing according to the movement of the object is performed. Processing equipment.   The control unit recognizes the movement of the object based on the detection information output from the detection unit, and determines whether the object has entered the detectable range of the detection unit from the front of the user. The information processing apparatus according to claim 5. The control unit determines whether the shape of the object is a specific shape based on detection information output from the detection unit after the state of the object has shifted to the standby state,
When it is determined that the shape of the object is the specific shape, the state of the object is shifted to the ready state,
The information according to claim 5, wherein the movement of the object is recognized based on detection information output from the detection unit after the state of the object has shifted to the ready state, and processing corresponding to the movement of the object is performed. Processing equipment. There are a plurality of the specific shapes,
The control unit matches the specific shape of the plurality of specific shapes based on detection information output from the detection unit after the state of the object has shifted to the standby state. Determine whether or not to
When it is determined that the shape of the object matches any one of the plurality of specific shapes, the state of the object is shifted to a ready state according to the shape of the object,
Based on detection information output from the detection unit after the state of the object has shifted to the ready state, the movement of the object is recognized, and processing according to the type of the movement of the object and the ready state is performed. The information processing apparatus according to claim 7.   The information processing apparatus according to claim 1, wherein the detection unit is mounted on a head mounting member that can be mounted on the head of the user.   The information processing apparatus according to claim 9, wherein the detection unit is mounted on a portion of the head mounting member that is mounted on the user's ear.   The information processing apparatus according to claim 1, wherein the detection unit is an image sensor capable of capturing an image around the user's ear.   An information processing method including remotely detecting a motion of an object existing around a user's ear by a detection unit.   The information processing method according to claim 12, comprising: recognizing the movement of the object based on detection information output from the detection unit and performing processing according to the movement of the object. Based on the detection information output from the detection unit, recognize the movement of the object, determine whether the object has entered the detectable range of the detection unit from the front of the user,
When it is determined that the object has entered the detectable range of the detection unit from the front of the user, the state of the object is shifted to a standby state,
The information according to claim 13, wherein the movement of the object is recognized based on the detection information output from the detection unit after the state of the object has shifted to the standby state, and processing according to the movement of the object is performed. Processing method. On the computer,
Based on detection information output from a detection unit that remotely detects the movement of an object existing around the user's ear, a control function that recognizes the movement of the object and performs processing according to the movement of the object is realized. ,program. The control function recognizes the movement of the object based on the detection information output from the detection unit and determines whether the object has entered the detectable range of the detection unit from the front of the user. And
When it is determined that the object has entered the detectable range of the detection unit from the front of the user, the state of the object is shifted to a standby state,
The program according to claim 15, wherein the movement of the object is recognized based on detection information output from the detection unit after the state of the object has shifted to the standby state, and processing corresponding to the movement of the object is performed. .

Images