TWI428808B - Controlling system and method via motion detection - Google Patents

Description

Somatosensory touch operating system and method

The invention relates to a touch interaction operation of an image display screen, in particular to a somatosensory touch operating system and a somatosensory touch operation method.

When a computer device and a projection device are used for presentation, the problem often faced by the user is that there is no interaction between the light pen and the image display screen, so that the operation command of the computer device cannot be generated by the movement of the light pen in the projection screen.

One of the common methods for solving the above problems is that the image display screen is projected on a large touch panel, and the touch panel is directly used to determine whether the image display screen is touched. However, large touch panels are costly and their arrangement makes the projection device unfavorable for handling between different locations.

In other prior art, there is also a technical means for obtaining a sampling screen by the image capturing device, analyzing the sampling screen, and determining whether the user touches the image display screen with a hand or a light pen.

For example, the new patent M396446 of the Republic of China first defines a detection area. When the image of the detection area is blocked, it means that the user touches the area, but when the detection area is blocked, the user does not necessarily point. Touch the area. The Chinese patent CN 03102675.3 uses a plurality of image capturing devices to obtain the position of the hand or the light pen, and determines whether the hand or the light pen touches the image display screen; CN200510009897.5 in mainland China and US 6,760,009 in the US are also obtained by using multiple image capturing devices. The position of the hand or the light pen, but it is impossible to judge whether the hand or the light pen touches the image display screen. In the foregoing prior art, a plurality of image capturing devices must be used, and the image capturing devices must pass through a calibration program, so that the touch operating system must be subjected to a calibration process before use, thereby being disadvantageous for handling between different places.

In view of the above problems, the present invention provides a somatosensory touch operating system and a somatosensory touch operation method, which are relatively simple technical means for discriminating whether an image display screen is touched.

The present invention provides a somatosensory touch operating system for discriminating whether an image display screen is touched to generate a pair of execution actions that should be touched. The somatosensory touch operating system comprises a specified wavelength light projection device, an image capture device, a selective filter element, and a data processing device.

The designated wavelength light projection device is configured to project a specified wavelength of light toward the image display screen to reflect the specified wavelength of light by the image display screen into a specific wavelength image. The image capturing device is disposed at a distance from the designated wavelength light projection device. The image capturing device continuously captures the image through the selective filtering component toward the image display screen, and the selective filtering component causes the image capturing device to obtain only a specific wavelength image; wherein the specific wavelength image includes a plurality of pixels, and each pixel You can define a coordinate in a particular wavelength image. a data processing device for separating a foreground image from a specific wavelength image and a shadow image falling on the image display screen, defining a feature point in the foreground image, and defining a shadow corresponding to the feature point in the shadow image point.

The data processing device further sets a distance threshold. When the relative distance between the feature point coordinates and the shadow point coordinates is less than the distance threshold value, the data processing device determines that the image display screen is touched, and generates a corresponding touch execution action.

The present invention further provides a somatosensory touch operation method for discriminating whether an image display screen is touched to generate a pair of touch action, including the following steps: projecting a specified wavelength of light toward the image display screen to specify The wavelength light is reflected by the image display screen; the image display screen continuously captures the specific wavelength image formed by the reflection of the specified wavelength light; and the pixel matrix of the specific wavelength image is used to establish a label system on the specific wavelength image; Separating a foreground image and a shadow image falling on the image display screen; defining a feature point in the foreground image, and defining a shadow point corresponding to the feature point in the shadow image; setting a distance threshold value and comparing the feature points The relative distance between the coordinates and the coordinates of the shadow point, whether the relative distance is less than the distance threshold value; and when the relative distance is less than the distance threshold value, determining that the feature point of the foreground image has touched the image display screen in the three-dimensional space, and Corresponding touch action.

The effect of the invention is that the image display screen does not need to be provided with a sensor, but the image analysis method is used to detect the touch of the human body on the image display screen, and the technical means required for the touch operation are simplified. In addition, when determining the coordinates of the present invention, the pixel matrix of the specific wavelength image itself is used as the coordinate system, and the calibration procedure of the external coordinate system is not required, and the pre-program for operating the somatosensory touch operating system is greatly simplified.

Referring to FIG. 1 , a somatosensory touch operating system 100 according to a first embodiment of the present invention is configured to determine whether an image display screen F is touched to generate a pair of touches. action. The somatosensory touch operating system 100 includes a specified wavelength light projection device 110, an image capture device 120, a selective filter element 130, and a data processing device 140.

Please refer to FIG. 5 , the specific embodiment of the image display screen F includes the image projected by the liquid crystal projector 150 and the DLP projection device 150. Please refer to FIG. 6 , and the image display screen F can also be displayed. It is a display surface of a non-projection type display device 160 such as a liquid crystal display or a plasma display, and is preferably a large size.

Referring to FIG. 1 and FIG. 2, the designated wavelength light projection device 110 is configured to project a specified wavelength light I toward the image display screen F, so that the designated wavelength light I is displayed by the image display screen F. Reflected as a specific wavelength image S. The specified wavelength light I is the wavelength light that the image display screen F does not emit, so as to prevent the display operation of the image display screen F itself from interfering with the somatosensory touch operating system 100; preferably, the specified wavelength light I is invisible light, thereby avoiding the specified wavelength The light I interferes with the user to view the image display screen F; the specific embodiment of the specified wavelength light I is infrared light, wherein the infrared light is the wavelength light that the image display screen F does not emit under the normal display operation, and is invisible and harmless to the human eye. .

Referring to FIG. 1 and FIG. 2, the image capturing device 120 and the designated wavelength light projection device 110 are separated by a distance G, and the image is captured toward the image display screen F. The selective filter element 130 is located between the image capture device 120 and the designated wavelength light projection device 110, and the selective filter element 130 allows the image capture device 120 to capture only a specific wavelength image. In a specific implementation, the image capturing device 120 can be a COMPLEMENTARY METAL-OXIDE SEMICONDUCTOR (CMOS) or a CHARGE COUPLED DEVICE (CCD).

The image capturing device 120 is electrically connected to the data processing device 140, and the data processing device 140 receives the specific wavelength image S obtained by the image capturing device 120, as shown in FIG. 1 and FIG. The specific wavelength image S includes a plurality of pixels, and each pixel defines a coordinate in the specific wavelength image S.

Referring to FIG. 3 and FIG. 4, after the data processing device 140 acquires the specific wavelength image S, a foreground image O and a shadow falling on the image display screen F are separated from the specific wavelength image S. The image B defines a feature point C in the foreground image O, and defines a shadow point C" corresponding to the feature point C in the shadow image B.

The manner in which the foreground image O is separated is determined by the intensity of the reflection of the specified wavelength light I. In general, the reflection intensity of the specified wavelength light I reflected by the image display screen F falls within an average reflection intensity interval, and the foreground image O is used by an obstacle, such as a standing in front of the image display screen F. The reflection intensity of the specified wavelength light I reflected is falling outside the average reflection intensity interval and greater than a reflection intensity lower limit value; thus, the data processing device 140 can be driven by a specific wavelength. The image S separates the foreground image O. The shadow image B is because the specified wavelength light I projected onto the image display screen F is blocked by the obstacle, and is formed in the dark portion of the image display screen F, and the reflected intensity of the reflected wavelength light I falls. The average reflection intensity interval is smaller than the reflection intensity lower limit value; thus, the data processing device 140 can separate the shadow image B from the specific wavelength image S. The aforementioned lower limit of the reflection intensity is smaller than the minimum value of the average reflection intensity interval.

Referring to "Fig. 3" and "Fig. 4", the feature point C defined in the foreground image O can be the user's hand. The method for finding the feature point C is to store the feature model in the data processing device 140. By comparing the feature model with the foreground image O, the pixel group corresponding to the feature model in the foreground image O can be found as the feature point. C. Similarly, in the application example in which the hand is the feature point C, the shadow point C′ corresponding to the feature point C is the shadow of the hand, so the feature image and the shadow image B may be compared to the shadow image. Find the pixel group that matches the feature model in B as the shadow point C". The foregoing method of finding the feature point C and the shadow point C" is only an example. In the field of image processing, the person having the usual knowledge can complete the definition of the feature point C and the shadow point according to the interpretation of the feature point C and the shadow point C". The technical means of C". The aforementioned feature point C is not necessarily the hand of the user, but may be a pen, a rod or the like.

When the feature point C continuously moves in the specific wavelength image S, the shadow point C" also changes position. As described above, the specific wavelength image S includes a plurality of pixels, and each pixel can define the specific wavelength image. A coordinate in S. Therefore, the data processing device 140 can generate the feature point C coordinate and the shadow point C" coordinate by analyzing the feature point C and the shadow point C" at the position of the specific wavelength image S.

A distance threshold value is further set in the data processing device 140. When the relative distance between the coordinate of the feature point C and the coordinate of the shadow point C" is smaller than the threshold value of the distance, the coordinates of the feature point C and the shadow point C" are regarded as overlapping.

As shown in Fig. 4, when the relative distance between the coordinate of the feature point C and the shadow point C" is smaller than the threshold value of the distance, even if it is equal to zero, the user who generates the foreground image O touches the image display screen by hand. At this time, the data processing device 140 determines that the feature point C of the foreground image O has touched the image display screen F in the three-dimensional space, and generates an execution action corresponding to the touch.

For example, when using a slide to perform a briefing, the user can touch it as an operation of "changing a page", so that the data processing device 140 performs "changing a page" and changing the projection displayed by the image display screen F. The user can use a continuous touch as an operation of "back to the previous page", so that the data processing device 140 performs "back to the previous page".

As shown in FIG. 5, a specific embodiment of the data processing device 140 can be a computer host that loads and executes an operating system and loads and executes an application according to the user's operation. The data processing device 140 generates a display signal. As described above, the specific embodiment of the image display screen F is an image frame projected by the projection device 150. The projection device 150 receives the display signal of the data processing device 140 to perform projection to generate an image frame on the image display screen F.

As shown in FIG. 6, another specific embodiment of the image display screen F is a display surface of a display device 160. The display device 160 receives the display signal of the data processing device 140 to generate an image frame on the display surface to become the image display screen F.

Referring to the "figure 5" and "figure 6", as in the application example described above, the data processing device 140 can change the display output when the "replace page" or "return to the previous page" execution operation is performed. The signal causes the projection device 150 or the display device 160 to change the content of the image display screen F.

Referring to FIG. 7 , a somatosensory touch operating system 100 according to a second embodiment of the present invention is configured to determine whether an image display screen F is touched to generate a pair of touches. action. The somatosensory touch operating system 100 includes a specified wavelength light projection device 110, an image capture device 120, a selective filter element 130, and a data processing device 140.

As shown in FIG. 7, the image capturing device 120 maintains a separation distance G from the designated wavelength light projection device 110. Therefore, the somatosensory touch operating system 100 of the second embodiment further includes a frame 170. The image capturing device 120 and the designated wavelength light projection device 110 are disposed on the frame 170 to maintain the separation distance G.

Referring to FIG. 8 , a somatosensory touch operating system 100 according to a third embodiment of the present invention includes a specified wavelength light projection device 110 , a selective filter component 130 , and an image capture device . 120. A data processing device 140, a projection device 150, and a frame 170. The image capturing device 120, the designated wavelength light projection device 110, and the projection device 150 are disposed on the frame 170 to maintain the separation distance G.

As shown in FIG. 9, it is a somatosensory touch operation method proposed by the present invention for discriminating whether an image display screen F is touched to generate a pair of execution actions that should be touched.

An embodiment of the coordinate positioning method for indicating a light spot according to the present invention provides a code for the data processing device 140 to load and execute, and performs the following steps; the code is stored in a computer readable storage medium. Or transmitted through a computer program product.

A specified wavelength light projection device 110 is projected toward the image display screen F to cause the designated wavelength light I to be reflected by the image display screen F into a specific wavelength image S, as shown in step 110.

The designated wavelength light I may be continuously projected onto the image display screen F, or may be intermittently projected onto the image display screen F by a flash frequency method. As described above, the designated wavelength light I is preferably a specific wavelength light that the image display screen F does not emit, and is invisible light, such as infrared light.

Then, the data processing device 140 continues to capture the specific wavelength image S formed by the specified wavelength light I toward the image display screen F by the image capturing device 120, as shown in step 120.

The selective filtering component 130 is disposed between the image capturing device 120 and the image display screen F. The selective filtering component 130 can be disposed between the image capturing device 120 and the image display screen F. The image capturing device 120 is caused to acquire only the specific wavelength image S.

The data processing device 140 receives the specific wavelength image S, and establishes a calibration system on the specific wavelength image S by using the pixel matrix of the specific wavelength image S, as shown in step 130.

Next, the data processing device 140 separates a foreground image O and a shadow image B falling on the image display screen F from the specific wavelength image S, as shown in step 140.

As shown in FIG. 10, in step 140, the manner of separating the foreground image O and the shadow image B includes the following steps.

The data processing device 140 sets an average reflection intensity interval and a reflection intensity lower limit value, as shown in step 141; wherein the reflection intensity lower limit value is smaller than the minimum value of the average reflection intensity interval.

The data processing device 140 separates, from the specific wavelength image S, a pixel group whose reflection intensity of the specified wavelength light I falls outside the average reflection intensity interval and is greater than the lower limit of the reflection intensity, and is defined as the foreground image O, As shown in step 142.

At the same time, the data processing device 140 separates, from the specific wavelength image S, a pixel group whose reflection intensity of the specified wavelength light I falls outside the average reflection intensity interval and is smaller than the lower limit of the reflection intensity, and is defined as the shadow image. B, as shown in step 143.

Referring to FIG. 9 again, the data processing device 140 defines a feature point C in the foreground image O, and defines a shadow point C′ corresponding to the feature point C in the shadow image B, as in step Step. 150 is shown.

As shown in Fig. 11, in the step 150, the manner of defining the feature point C and the shadow point C" includes the following steps.

The data processing device 140 loads the feature model as shown in step 151. The feature model is based on the outline of a particular subject, such as the user's hand, pen, and body. And the aforementioned outline includes those obtained by observing a specific subject from different viewing angles.

The data processing device 140 compares the feature model with the foreground image O, finds a pixel group that matches the feature model in the foreground image O, and defines the pixel group as the feature point C, as shown in step 152.

The data processing device 140 compares the feature model and the shadow image B to find a pixel group that matches the feature model in the shadow image B, and defines the pixel group as a shadow point C", as shown in step 153.

Referring again to FIG. 9, the data processing device 140 sets a distance threshold as shown in step 160.

The data processing device 140 continuously compares the relative distance between the feature point C coordinate and the shadow point C" coordinate to determine whether the relative distance is smaller than the distance threshold value, as shown in step 171 and step 172.

When the relative distance is less than the distance threshold, the data processing device 140 determines that the feature point C of the foreground image O has touched the image display screen F in the three-dimensional space, and generates an action corresponding to the touch, such as a step. Step 180 is shown.

In the somatosensory touch operating system 100 and the somatosensory touch operation method proposed by the present invention, it is not necessary to provide a sensor on the image display screen F. The invention can detect the touch of the human body on the image display screen F by means of image analysis. In the above image analysis, other analysis steps can be combined to further obtain information such as touch coordinates and simplify the touch operation. The technical means needed. In addition, when determining the coordinates of the present invention, the pixel matrix of the specific wavelength image S itself is used as the coordinate system, and the calibration procedure of the external coordinate system is not required, and the pre-program of the operating touch operating system 100 is greatly simplified.

100. . . Somatosensory touch operating system

110. . . Specified wavelength light projection device

120. . . Image capture device

130. . . Selective filter element

140. . . Data processing device

150. . . Projection device

160. . . Display device

170. . . frame

F. . . Image display

I. . . Specified wavelength light

G. . . Spacing distance

S. . . Specific wavelength image

O. . . Foreground image

B. . . Shadow image

C. . . Feature points

C"...shadow point

1 and 2 are schematic views of a somatosensory touch operating system according to a first embodiment of the present invention.

3 and 4 are schematic views of a specific wavelength image in the first embodiment of the present invention.

FIGS. 5 and 6 show a variation of the somatosensory touch operating system in the first embodiment of the present invention.

FIG. 7 is a schematic diagram of a somatosensory touch operating system according to a second embodiment of the present invention.

FIG. 8 is a schematic diagram of a somatosensory touch operating system according to a third embodiment of the present invention.

FIG. 9 , FIG. 10 and FIG. 11 are flowcharts of a somatosensory touch operation method of the projection system of the present invention.

Claims (10)

A somatosensory touch operating system for discriminating whether an image display screen is touched to generate a pair of touch-to-touch execution operations, the somatosensory touch operating system comprising: a specified wavelength light projection device for facing the image The display screen projects a specified wavelength of light, so that the specified wavelength of light is reflected by the image display screen into a specific wavelength image; an image capturing device maintains a separation distance from the specified wavelength light projection device and displays the image toward the image The screen captures an image; a selective filtering component is configured to enable the image capturing device to acquire only the specific wavelength image; wherein the specific wavelength image includes a plurality of pixels, and each pixel defines one of the specific wavelength images And a data processing device for separating a foreground image and a shadow image falling on the image display screen from the specific wavelength image, and defining a feature point in the foreground image, and defining the feature image in the shadow image a shadow point corresponding to the feature point; wherein the data processing device further sets a distance threshold, when the feature point coordinates The relative distance between the coordinates of the shadow point is less than the distance threshold value, the data processing device determining the image display screen is touched, the touch is generated to perform an action should be. The somatosensory touch operating system of claim 1, wherein the specified wavelength of light is infrared. The somatosensory touch operating system of claim 1, wherein the data processing device sets an average reflection intensity interval and a reflection intensity lower limit value that is smaller than a minimum value of the average reflection intensity interval; the data processing device is configured by the specific wavelength image a pixel group in which the reflection intensity of the specified wavelength light falls outside the average reflection intensity interval and is greater than the reflection intensity lower limit value, defined as the foreground image, and the data processing device is in the specific wavelength image And separating a pixel group whose reflection intensity of the specified wavelength light falls outside the average reflection intensity interval and smaller than the reflection intensity lower limit value, and is defined as the shadow image. The somatosensory touch operating system of claim 1, wherein the data processing device stores a feature model for comparing the feature model with the foreground image to find a pixel group in the foreground image that matches the feature model, As the feature point; and the data processing device compares the feature model with the shadow image, and finds a pixel group conforming to the feature model in the shadow image as a shadow point. The somatosensory touch operating system of claim 1, further comprising a frame, wherein the image capturing device and the designated wavelength light projection device are disposed on the frame to maintain the separation distance. A somatosensory touch operation method for discriminating whether an image display screen is touched to generate a pair of execution actions that should be touched, comprising: projecting a specified wavelength of light toward the image display screen, so that the specified wavelength light is The image display screen reflects; the image display screen continuously captures a specific wavelength image formed by the specified wavelength light reflection; and the pixel matrix of the specific wavelength image establishes a label system on the specific wavelength image; from the specific wavelength image Separating a foreground image and a shadow image falling on the image display screen; defining a feature point in the foreground image, and defining a shadow point corresponding to the feature point in the shadow image; setting a distance threshold value, Determining whether the relative distance is smaller than the distance threshold value than the coordinate between the coordinate of the feature point and the coordinate of the shadow point; and determining the feature point of the foreground image in three dimensions when the relative distance is less than the distance threshold value; The image display screen has been touched in the space, and an execution action corresponding to the touch is generated. The somatosensory touch operation method of claim 6, wherein the specified wavelength light is continuously projected on the image display screen. The somatosensory touch operation method of claim 6, wherein the designated wavelength light is intermittently projected on the image display screen by using a flash frequency method. The method of claim 2, wherein the method for separating the foreground image and the shadow image comprises the steps of: setting an average reflection intensity interval-reflection intensity lower limit value; wherein the reflection intensity lower limit value is smaller than the average value a minimum value of the reflection intensity interval; in the specific wavelength image, a pixel group in which the reflection intensity of the specified wavelength light falls outside the average reflection intensity interval and is greater than the lower limit of the reflection intensity is defined as the foreground image And a pixel group in which the reflection intensity of the specified wavelength light falls outside the average reflection intensity interval and is smaller than the reflection intensity lower limit value in the specific wavelength image, and is defined as the shadow image. The method of claim 2, wherein the method of determining the feature point and the shadow point comprises the steps of: loading a feature model; and comparing the feature model with the foreground image to find a match in the foreground image. The pixel group of the feature model defines the pixel group as the feature point; and compares the feature model with the shadow image to find a pixel group corresponding to the feature model in the shadow image, and defines the pixel group as the shadow point.