JP2010117917A - Motion detection apparatus and operation system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a motion detection apparatus, which allows a user to perform remote operation, in an operation system configured so that operation is performed in reference to a screen projected by a laser project, without turning away from the screen or leaning forward. <P>SOLUTION: The motion detection apparatus includes a screen and a laser projector which projects image on the screen by use of laser light. The user moves the hand so as to interrupt the optical path of the laser light to make a shadow of the hand on the screen. A motion detection part takes an image of the screen by a camera and detects a motion of the user based on the shadow of the hand obtained from the image to confirm an instruction content. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a motion detection device including a projection unit that projects an image using laser light, and an operation system using the motion detection device.

  Conventionally, when operating an on-vehicle car navigation device, there are two types of operation methods, a contact type and a remote type. Examples of the contact type include a method of operating a button arranged on the dashboard or directly touching a touch panel screen. However, when the driver operates with these methods, it is inconvenient because it is necessary to get out of the driver's seat or extend his hand far away.

  As an example of the remote type, there is a method of giving an instruction using a remote controller. However, since the operation cannot be performed without looking at the remote control, the operation is performed while comparing the screen with the remote control at hand, which takes time. As a countermeasure against this problem, Japanese Patent Application Laid-Open No. 2007-083834 (Patent Document 1) makes it possible to operate the touch panel facing forward by projecting the image of the operation unit forward using the reflection of the mirror. However, in this method, since the direction in which the user puts his hand and the direction of the screen to be viewed by the user are different, it is difficult to say that an intuitive operation can be performed.

On the other hand, there is a method described in Japanese Patent Application Laid-Open No. H08-095707 (Patent Document 2) as a display system for a user to perform a remote operation while looking at a screen, not limited to a car navigation device. In this method, infrared light or the like is projected to detect the position of the stick held by the user, and the user's operation content is recognized based on the position information. However, in this method, it is difficult to accurately specify the position of the rod due to the light diffraction phenomenon.
Japanese Patent Laid-Open No. 2007-083834 JP-A-8-095707

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technique that enables a user to intuitively perform a remote operation while looking at a screen.

  In order to achieve the above object, the present invention employs an operation detection apparatus having the following configuration. That is, a screen, a projection unit that projects an image onto the screen using laser light, a photographing unit having a screen photographing unit that photographs the screen, and an operation detection unit that detects a user's operation, An operation detection device that detects a user's operation based on position information of a blocking area where an optical path of a laser beam is blocked.

  The present invention uses position information of a blocking area (shadow) displayed on a screen as a method of detecting an operation performed by a user in order to instruct an operation content. That is, when the projection unit projects an image on the screen with laser light, the user places a hand or the like at a position that blocks the optical path. Thereby, the shadow which the laser beam was interrupted on a screen arises. Therefore, the screen photographing means (camera or the like) photographs the screen and acquires a screen photographed image. The motion detection unit detects the user's motion based on the position information of the shadow in the screen shot image.

According to the present invention configured as described above, the user can intuitively remotely operate the apparatus while looking at the shadow reflected on the screen. In addition, since the user does not need to touch the screen directly, the user can perform a comfortable operation without leaning out or extending his hand. Furthermore, by using the laser beam, a clear shadow without diffraction can be acquired based on the straightness of the laser beam.

  Note that the motion detection unit may detect the motion based not only on the position information of the shadow at a certain temporary point but also with time. As a result, the user can instruct operation details in more detail.

  In the embodiment of the present invention, user photographing means can be provided in the photographing unit. In this method, the optical path of laser light is photographed from the projection unit side, and a user-captured image is acquired. Here, if a body such as the user's hand blocks the laser beam, a part of the image should be projected on the user's hand. Therefore, the motion detection unit recognizes an image reflected in the user's hand from the user-captured images, and detects the user's motion based on the recognized image.

  Furthermore, in the implementation of the present invention, a three-dimensional photographing unit can be provided in the photographing unit. This is to obtain a three-dimensional photographed image by photographing the vicinity of the screen. The motion detection unit three-dimensionally analyzes the movement of the user's hand in the three-dimensional captured image and detects the user's motion.

  As described above, according to the present invention configured by adding the user photographing means or the three-dimensional photographing means, it is possible to detect the user's action based on the image reflected in the user's hand or the movement of the user's hand. Thus, the detection accuracy can be improved by combining with the method based on the shadow reflected on the screen.

  In the implementation of the present invention, it is possible to provide at least one reference marker which is an area having a predetermined brightness in the projected image. Here, when disturbance light is reflected on the screen, the brightness contrast on the screen becomes low, and a situation in which the user's action cannot be detected based on the shadow position information may occur. At this time, the motion detection unit can estimate the extent of the influence of disturbance light by comparing the brightness of the reference marker in the screen image captured by the imaging unit with the brightness at the time of projection.

  According to the present invention configured as described above, when the motion detection unit cannot detect the user's motion, it is determined whether or not the cause is the influence of disturbance light. Increases brightness and makes shadows stand out.

  The present invention can be realized as an operation system including a motion detection device and an information processing unit that executes processing based on a user motion detected by the motion detection device. Furthermore, the operating device can be used in a vehicle.

  According to the motion detection device or the operation system according to the present invention, the user can intuitively perform remote operation while looking at the screen.

  Exemplary embodiments of the present invention will be described in detail below with reference to the drawings. In the following embodiments, an example will be described in which a driver who is a user performs an operation and operates the device in a vehicle equipped with a motion detection device and a car navigation device.

<Embodiment 1>
In the present embodiment, a method for detecting a user's action using position information of a blocking area (shadow) on the screen will be described. That is, when the laser projector projects an image with a laser beam on the screen, the user blocks the laser beam with a hand or the like. This is a method of detecting the operation of the user based on the shadow generated on the screen and recognizing the operation content intended by the user.

(Operation system configuration)
The configuration of the operation system according to the present embodiment will be described with reference to the block diagram shown in FIG. The laser projector 1 projects the image created by the image creation unit 6 with laser light, and corresponds to the projection unit in the present invention. The screen 2 is an area for displaying an image projected from the laser projector 1.

  In the present embodiment, it is assumed that the user sits and operates in the driver's seat, the laser projector 1 is installed on the left rear of the driver's seat, and the screen 2 is installed on the dashboard on the left front of the driver's seat. ing. The screen 2 is usually composed of a white planar area having an area that can be visually recognized by the user, but is not necessarily limited thereto. For example, an image may be projected on the dashboard itself, and the color may be other than white.

  As shown in the figure, when the user blocks the laser beam by hand, a shadow (blocking area in the present invention) is generated on the screen 2. The screen photographing camera 3 photographs the screen 2 and acquires a screen photographed image, and corresponds to the screen photographing means in the present invention. The screen camera 3 is preferably installed at a certain angle from the laser projector 1 so that the user's hand does not appear in the screen shot image.

  The motion detection unit 4 detects a user's motion based on the screen shot image, and includes a shadow detection unit 41, a pattern storage unit 42, a pattern comparison unit 43, and a temporal analysis unit 44. The shadow detection unit 41 recognizes the brightness contrast in the screen shot image, and detects a shadow area blocked by the laser beam. At that time, referring to the image projected by the laser projector 1, it is determined whether the contrast in the screen shot image is due to a shadow or the original projected image has a contrast, and the accuracy of shadow detection may be improved. it can. The pattern storage unit 42 is a storage device, and stores image pattern data for recognizing the hand shape and the user's action. There are two types of data: one for determining whether the detected shadow is a hand shape, and the other for recognizing what kind of instruction the hand shape is.

  The pattern comparison unit 43 recognizes the shape of the hand shadow included in the screen shot image by matching the area detected by the shadow detection unit 41 with the image pattern stored in the pattern storage unit 42. At this time, the pattern comparison unit 43 first determines whether or not the region that the shadow detection unit 41 determines to be a shadow is a hand shadow, and if it is a hand, recognizes what the user instruction is. . The temporal analysis unit 44 detects the user's action by analyzing the shape of the hand shadow over time. The motion detection unit 4 transmits the user motion to the information processing unit 5. Note that the operation detection unit 4 may be configured by a dedicated circuit, or may be realized as a program that functions using a CPU.

  The information processing unit 5 recognizes an operation requested by the user based on the user's motion detected by the motion detection unit 4 and controls the image creating unit 6 and other devices. The image creating unit 6 creates an image to be presented to the user and transmits it to the laser projector 1.

(Processing flow)
Hereinafter, a processing flow when the user selects a destination from a map on the screen and starts navigation will be described with reference to the flowchart of FIG. The screen shot image transition diagrams shown in FIGS. 4A to 4C are referred to as necessary. Specifically, first, a laser projector projects an image onto a screen and displays a map. Subsequently, the user (driver) shades the laser beam with his hand to make a shadow on the screen, and selects the destination building “ABC Kaikan” at the fingertip portion of the shadow. Subsequently, the camera shoots the screen. The motion detection unit detects the user's motion based on the shadow position information and recognizes the user's instruction content. The car navigation device sets the destination according to the instruction content and starts guidance.

  In this embodiment, when the user selects an object on the screen, a shadow fingertip is placed on the object and held for one second. As the object, for example, a building on a map or a button on a screen can be a target.

  In step S201, the laser projector 1 projects a map image toward the screen 2, and, as shown in FIG. 4A, issues a message requesting the user to select a destination and waits for input. When the user finds the destination “ABC Hall” by looking at the map, the user moves his hand to block the optical path of the laser beam so that the shadow of the fingertip indicates “ABC Hall”. This state is shown in FIG.

  In step S202, the screen photographing camera 3 photographs the screen 2 and obtains a screen photographed image. In step S203, the motion detection unit 4 attempts to detect the user's motion based on the screen shot image, and determines whether there is an instruction from the user. This step will be described in detail later.

  When there is no instruction from the user (step S204 = N), the input waiting from the user is continued and an attempt is made to detect the user's operation again at a predetermined interval. On the other hand, if there is an instruction from the user (step S204 = Y), the process proceeds to step S205. “ABC Kaikan” is set as the destination, and a message is output as shown in FIG. This completes the selection of the destination and starts navigation.

(Processing flow for user motion detection)
With reference to the flowchart of FIG. 3, processing in which the motion detection unit detects the user's motion based on the position information of the fingertip shadow will be described.

  In step S301, the shadow detection unit 41 searches for an area determined to be a shadow from the screen shot image. In this case, for example, it is possible to use a method of detecting a region having a lightness lower than that of the surroundings based on the brightness contrast. Subsequently, the pattern comparison unit 43 determines whether the shadow of the fingertip is included by matching the shape of the shadow with the image pattern stored in the pattern storage unit 42. When there is no shadow of the fingertip (step S301 = N), the process proceeds to step S304, “no instruction” is set in the memory, and the process is terminated. On the other hand, if there is a shadow of the fingertip (step S301 = Y), the process proceeds to step S302, and “ABC Hall”, which is the object selected by the fingertip, is specified.

  In step S303, the temporal analysis unit 44 determines whether the object selection by the fingertip has continued for 1 second. If it does not continue (step S303 = N), it is determined that it is not a selection operation by the user but is simply moving, and the process proceeds to step S304. On the other hand, when it continues for 1 second (step S303 = Y), it judges that the user selected the destination and progresses to step S305. In step S305, “instructed” is set in the memory, and the process ends.

  In this embodiment, as a method for selecting an object on the map, the shadow of the fingertip is placed on the object and held for one second. However, the method is not limited to this. For example, it can be considered that the selection is made in a shorter time.

  According to the motion detection device of the present embodiment, when the user operates the car navigation device, it can be remotely operated using the shadow of the hand, so that the user can intuitively perform the remote operation while viewing the map displayed on the screen. It becomes possible. Also, since the user does not have to touch the screen directly, there is no need to lean out or extend his hand. In addition, by using a laser projector, a diffraction phenomenon does not occur regardless of the distance between the projection unit and the screen, so that the degree of freedom in arranging the operation system in the vehicle is improved.

<Embodiment 2>
In the present embodiment, a method for detecting a user's action using an image projected on the user's hand in addition to the shadow of the fingertip in the screen shot image described in the first embodiment will be described.

(Operation system configuration)
Hereinafter, the configuration of the operation system of this embodiment will be described with reference to the block diagram shown in FIG. In addition, the same number is attached | subjected to the same block as Embodiment 1, and description is abbreviate | omitted.

  The user photographing camera 7 is a camera for acquiring an image projected on the hand when the user blocks the laser beam with the hand to cause a shadow on the screen, and corresponds to the user photographing means in the present invention. Therefore, the user photographing camera 7 is installed in the same direction as the laser projector 1 toward the screen 2. Hereinafter, an image photographed by the user photographing camera 7 is described as a user photographed image.

(Processing flow)
Hereinafter, the processing flow of this embodiment will be described with reference to the flowchart of FIG. 6, the projected image of FIG. 8A, and the screen shot image of FIG. 8B.

  Here, a process when the projection image is divided into four areas of 2 × 2 in the vertical and horizontal directions and the user selects one of them will be described. Specifically, first, a laser projector projects an image onto a screen. Subsequently, the user extends his / her hand and selects one of the four areas by the shadow of the fingertip. At this time, the laser beam is blocked by the hand, so that a shadow is generated on the screen. At the same time, the blocked laser beam is projected onto the user's hand. Therefore, an image projected on the user's hand is acquired from the image captured by the user imaging camera 7, and the position of the fingertip is recognized by comparing with an original projection image. As a result, the user's action is detected and it is determined which area has been selected.

  In step S <b> 601, the laser projection 1 projects the image created by the image creation unit 8 onto the screen 2. In this image, as shown in FIG. 8A, the screen is divided into four areas.

  In step S602, the screen shooting camera 3 captures the screen 2 and obtains a screen shot image. Further, the user photographing camera 7 photographs the optical path of the laser light from the same side as the laser projection 1 to obtain a user photographed image.

  In step S603, the motion detection unit 4 tries to detect the user's motion from the screen shot image. The detection is executed according to the flowchart of FIG. 3 as in the first embodiment. As a result, when there is a clear shadow as shown in FIG. 8B and the user's action can be detected (step S604 = Y), the process proceeds to step S607, and the area designated by the user is selected. On the other hand, when a clear shadow cannot be detected due to the influence of ambient light or the like, and the instruction from the user cannot be confirmed (step S604 = N), the process proceeds to step S605.

In step S605, the motion detection unit 4 tries to detect the user's motion from the user-captured image,
Returns “Instructed” or “Not Instructed”. This step will be described in detail later. If there is a user instruction (step S606 = Y), the process proceeds to step S607, and the selection process is terminated. On the other hand, when there is no user instruction (step S606 = N), the user enters a selection waiting state, and the user's operation is detected again at a predetermined interval.

(Processing flow for detecting user actions from user-captured images)
With reference to the flowchart in FIG. 7 and the user-captured image shown in FIG.

  First, the pattern comparison unit 43 analyzes the user-captured image and tries to find the shape of the hand by matching with a pattern stored in the pattern storage unit 42. If the hand shape cannot be found (step S701 = N), the process proceeds to step S705, “no instruction” is set in the memory, and the process ends. On the other hand, when the hand shape is found as shown in FIG. 8C (step S701 = Y), the process proceeds to step S702, and an image reflected in the hand is acquired. Subsequently, in step S703, the image shown in the hand is analyzed and compared with the original projection image received from the image creating unit 6, and the region selected by the fingertip (in this case, the upper left of the screen) is specified.

  In step S704, the user-captured image is continuously observed to determine whether selection by the fingertip has continued for 1 second. If it does not continue (step S704 = N), it is determined that it is not a selection by the user but is merely in the middle of an operation, and the process proceeds to step S705, where “no instruction” is set in the memory and the process is terminated. On the other hand, if the selection continues for 1 second (step S704 = Y), the process proceeds to step S706, where “instructed” is set and the process ends.

  According to the motion detection device of the present embodiment, even when the shadow is not clear in the screen shot image, it is possible to detect the user's motion based on the image reflected in the hand, and improve the detection accuracy. be able to.

  In the present embodiment, the user-captured image is analyzed only when the shadow is not detected in the screen-captured image, but the present invention is not limited to this. For example, it is possible to improve the detection accuracy by always analyzing the user-captured image and using it together with the screen-captured image.

<Embodiment 3>
In this embodiment, in order to detect the user's movement, in addition to the fingertip shadow described in the first embodiment, the user's hand movement is acquired by the camera. In that case, the feature is that the shape and movement of the user's hand are three-dimensionally analyzed. Hereinafter, differences from the second embodiment will be described.

(Operation system configuration)
The motion detection apparatus according to the present embodiment includes a three-dimensional imaging camera that images a region near the screen, and acquires a three-dimensional captured image by capturing the movement of the user's hand in the region. This camera corresponds to the three-dimensional imaging means in the present invention. The motion detection unit three-dimensionally analyzes the movement of the user's hand based on the three-dimensional captured image, and detects the user's motion.

  As such a three-dimensional imaging camera, for example, a camera that performs analysis using light reflection can be used. That is, this is a device that emits oscillating light from the camera to the area near the screen and measures the phase shift when the light hits the user's hand and is reflected. In this case, it is preferable to install a three-dimensional imaging camera on the screen side so that the user's body and laser light can capture the image. In addition, a stereoscopic video can be acquired using a stereo camera as a three-dimensional imaging camera.

(Processing flow)
The processing flow of the present embodiment is almost the same as that of the second embodiment, except that the user's action detected based on the user-captured image in the second embodiment is performed based on the three-dimensional captured image.

  According to the motion detection device of the present embodiment, even when the shadow in the screen shot image or the image in the hand shot in the user shot image is unclear, the user motion is detected based on the movement of the user's hand. Therefore, the detection accuracy can be improved. In addition, since the three-dimensional imaging camera acquires a three-dimensional hand movement, it is particularly effective when recognizing a movement operation. The use of the three-dimensional captured image is not limited to the case where the user's action cannot be detected from the screen captured image or the user captured image. By using both simultaneously, the detection accuracy can be improved.

<Embodiment 4>
This embodiment is an effective method for a scene in which ambient light enters the screen, the brightness of the entire screen increases, and the contrast decreases.

  In such a case, since it is difficult to recognize the shadow of the hand on the screen, it is necessary to improve the contrast. For this purpose, first, an area having a predetermined brightness called a reference marker is set in the projection image, and projection is performed. Then, the influence of ambient light is estimated by comparing the brightness of the original projected image and the screen shot image. If the influence of disturbance light is large, the brightness of the projected image is increased to improve the contrast, and the shadow detection is attempted.

(Operation system configuration)
The configuration of the operation system according to the present embodiment will be described with reference to the block diagram shown in FIG. The components themselves are the same as in the first embodiment. The image creation unit 6 provides reference markers 8 at two locations on the upper right and lower left of the image. Regardless of the image projected by the laser projector 1, the reference marker 8 is output so as to have a constant brightness.

(Processing flow)
The processing flow will be described with reference to the flowchart of FIG. As in the first embodiment, a case where the user selects a destination from a map and causes the car navigation device to start navigation will be described as an example.

  In step S1001, the laser projector 1 projects a map image toward the screen 2, and waits for a user to select a destination. In step S <b> 1002, the screen shooting camera 3 captures the screen 2 and transmits the screen captured image to the operation detection unit 4. In step S1003, the motion detection unit attempts to detect the user's motion from the shadow of the hand according to the flow of FIG. If an instruction from the user can be detected (step S1004 = Y), the process proceeds to step S1007, where the destination is set and navigation is started. Note that the processing up to this point is the same as that of the first embodiment, and the operation when the user's operation cannot be detected (step S1004 = N) is a feature of this embodiment.

  That is, when the brightness is not yet increased (step S1005 = N), disturbance light estimation and brightness determination at the next projection are performed. This step will be described in detail later. Subsequently, the process proceeds to step S1001 again, and the image is projected with the brightness determined earlier. As a result, the brightness increases, so that the contrast in the screen shot image is improved, and the user's action based on the shadow of the hand can be detected in step S1003.

On the other hand, when the brightness has been increased (step S1005 = Y), the reason why the shadow cannot be detected is that the laser light is not cut off and the shadow itself does not exist on the screen, regardless of the influence of disturbance light. Conceivable. Therefore, the operation detection unit continues to wait for input from the user, and tries to detect the user's operation again at a predetermined interval.

(Disturbance light estimation and projection brightness determination)
With reference to the flowchart of FIG. 11, the ambient light estimation and brightness determination method at the next projection will be described.

  The motion detection unit 4 detects the brightness of the reference marker in the screen shot image in step S1101 and compares it with the brightness of the reference marker at the time of projection in step S1102. Here, if the contrast is lowered due to the influence of ambient light and the shadow of the hand cannot be detected, the brightness of the reference marker in the screen shot image should be higher than that at the time of projection. Therefore, in step S1103, it is determined whether the reference marker brightness of the screen shot image is greater than or equal to a reference value, for example, more than twice that at the time of projection. If the reference value is exceeded (S1103 = Y), it is determined that the influence of disturbance light is large, and the process proceeds to step S1004 to increase the brightness at the next projection.

  On the other hand, in the case of S1103 = N, it is determined that the influence of disturbance light is not so great, so the brightness at the next projection is not changed.

  According to the motion detection device of this embodiment, when a hand shadow cannot be detected from a screen shot image, the influence of ambient light is estimated based on the brightness of the reference marker, the brightness at the next projection is determined, and the shadow is detected. Can be possible.

  In this embodiment, there are two reference markers. However, the number and size of the reference markers may be changed as appropriate according to the surrounding conditions such as the screen size and the projection distance. Further, although the brightness is increased only once, it may be increased a plurality of times. Furthermore, when determining the amount of increase in brightness, if the influence of ambient light is large, the amount of increase in brightness can be increased accordingly.

<Modifications with different user actions>
In each of the above-described embodiments, the method of selecting a target by placing a shadow of a fingertip on an object on a screen for one second has been taken up as a user action. However, the user's action is not limited to selecting an object, but can be defined in various situations. Examples are described below.

(Drag and drop operation)
Here, a case where a user's action of dragging and dropping an object on the screen is detected will be described. As an example to which this operation can be applied, there is a case where the user changes the scale of the map displayed on the screen. That is, the user selects a desired scale by dragging and dropping the pointer on the scroll bar representing the scale of the map.

  FIGS. 12A to 12C are diagrams showing transition of screen shot images in this modification. The screen configuration will be described with reference to FIG. The upper part of the screen is a map on which roads, buildings (XYZ buildings), and the like are displayed. The lower part of the screen shows the scale of the map, and a scroll bar 121 and a pointer 122 are displayed. If the pointer 122 is on the right side of the scroll bar 121, a wide range is displayed, and if the pointer 122 is on the left side, a narrow range is displayed in detail. When the user wants to change the scale, the user first touches the pointer 122 with the shadow of the fingertip, moves the scroll bar to a position indicating the desired scale, and then moves the shadow of the fingertip away from the vicinity of the scroll bar 121.

  FIG. 13 shows a flowchart in this example. In step S1301, first, the laser projection 1 displays a map on the screen 2. Subsequently, in step S1302, the detection of the shadow of the fingertip is periodically attempted by the method of any of the above-described embodiments, and an operation by the user is awaited.

  As shown in FIG. 12A, when the user touches the pointer 122 with the shadow of the fingertip (step S1302 = Y), it is considered that the scroll operation has started. At this time, as shown in FIG. 12B, the user slides the pointer 122 on the scroll bar 121 and designates a desired scale. Subsequently, in step S1303, it is periodically determined whether or not the shadow of the fingertip has moved away from the pointer 122. When the shadow of the fingertip has moved away from the pointer (step S1303 = Y), it is determined whether or not the position of the pointer has been changed. If the pointer position has been changed (step S1304 = Y), the scale of the map is changed as shown in FIG. On the other hand, if the pointer position is not changed (step S1304 = N), the process is terminated without changing the scale.

(Move the hand shadow on the screen)
Here, a case where a user's motion of moving a hand shadow on the screen is detected will be described. As an example to which this operation can be applied, there is a case where the user wants to scroll a map displayed on the screen. Here, the user scrolls the map to the left by moving the hand from the right to the left on the screen.

  When a map is displayed on the screen, as shown in FIG. 14A, the user blocks the laser beam with a hand with his / her finger extended so that a shadow is projected on the screen. Next, the user moves his hand in the direction of scrolling the screen. In this example, when the user moves the shadow of the hand in the upper left direction, the map is also scrolled to the upper left as shown in FIG.

(Operation to make the hand a specific shape)
Here, a case will be described in which when a specific hand shape is associated with an operation content, a user's action for making the hand a shape is detected.

  FIG. 15A shows the shape of a hand that can be used in cases where some numerical value is increased, such as increasing the volume of a car navigation device or car audio, or increasing the temperature of an air conditioner. As shown in the figure, the user conveys the operation content to the motion detection device by holding his hand, turning his thumb up, and shaking his hand.

  FIG. 15B shows the shape of the hand when the user gives consent. For example, as shown in the figure, when a dialog requesting approval is displayed as “Are you sure?”, The user creates a ring with the thumb and index finger to convey the intention of approval.

(Applicability of the present invention)
Although the car navigation device mounted on the vehicle has been described in the above embodiment, the present invention is not limited to this. For example, it can also be used for various devices such as an air conditioner and an audio device that perform functions according to user instructions.

  In addition, the motion detection device and the operation system are not necessarily mounted on a vehicle, and can be installed in a house and applied to home appliances, for example.

FIG. 2 is a block diagram illustrating a configuration of an operation system according to the first embodiment. 5 is a flowchart of processing according to the first embodiment. The flowchart of the process which detects a user's operation | movement based on a shadow. FIG. 4 is a diagram illustrating transition of a screen shot image according to the first embodiment. FIG. 4 is a block diagram illustrating a configuration of an operation system according to a second embodiment. 10 is a flowchart of processing according to the second embodiment. The flowchart of the process which detects a user's operation | movement based on the image shown in the hand. 9 is an image for explaining processing of the second embodiment. FIG. 9 is a block diagram illustrating a configuration of an operation system according to a fourth embodiment. 10 is a flowchart of processing according to the fourth embodiment. 10 is a flowchart of a next projection brightness determination process according to the fourth embodiment. The figure which shows the operation | movement of the user who changes the scale of a map. The flowchart which shows a user's operation | movement which changes the reduced scale of a map. The figure which shows the operation | movement of the user who scrolls a map. The figure which shows the operation content matched with the shape of a specific hand.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Laser projector 2 Screen 3 Camera for screen photography 4 Motion detection part 6 Image preparation part 7 Camera for user photography 8 Reference marker

Claims (7)

A motion detection device that captures a predetermined area and detects a user's motion from the captured moving image,
Screen,
A projection unit that projects an image on the screen using laser light;
A photographing unit having a screen photographing unit for photographing the screen and obtaining a screen photographed image;
An operation detection apparatus comprising: an operation detection unit configured to detect a user's operation based on position information of a blocking area that is an area where laser light is blocked in the screen shot image. The motion detection apparatus according to claim 1, wherein the motion detection unit detects a user's motion based on a change with time of position information of the blocking area. The photographing unit has a user photographing unit that obtains a user-captured image by photographing an optical path of laser light from the projection unit side,
The motion detection unit detects an image projected on the user's body if there is a portion of the user's captured image where the user's body blocks the optical path of the laser beam, and performs the user's motion based on the image. The motion detection device according to claim 1, wherein the motion detection device is detected. The photographing unit includes a three-dimensional photographing unit that obtains a three-dimensional photographed image by photographing the movement of the user's hand in the vicinity of the screen,
The motion detection apparatus according to claim 1, wherein the motion detection unit detects a user motion based also on the three-dimensional captured image. The image projected by the projection unit is provided with at least one reference marker that is a region having a predetermined brightness by the projection unit,
When the motion detection unit cannot recognize the presence of the blocking area, the motion detection unit compares the brightness of the reference marker in the screen shot image with the brightness of the reference marker at the time of projection, and the screen shot image 5. The motion detection device according to claim 1, wherein, when it is determined that the brightness of the entire image is lower than a predetermined value, the brightness of the entire image is increased to perform projection. The motion detection device according to any one of claims 1 to 5,
An operation system comprising: an information processing unit that executes a process according to a user action detected by the action detection device. The operation system is mounted on a vehicle,
The operation system according to claim 6, wherein the information processing unit controls a car navigation device.

Images