TW201405376A - Remote device and power saving method of interactive system - Google Patents

Abstract

There is provided a remote device including an image sensor, a readout circuit and a processing unit. The image sensor sequentially captures a first image and a second image containing at least one reference light image. The readout circuit is configured to read a first image data of the first image and a second image data of the second image from the image sensor. The processing unit is configured to calculate an image feature of the at least one reference light image according to the first image data and control the readout circuit to only read the second image data of a range of interest in the second image according to the image feature. There is further provided a power saving method of an interactive system.

Description

Power saving method for remote control device and interactive system

The present invention relates to an interactive system, and more particularly to a power saving method for a remote control device and an interactive system for reading only a part of pixel data of a sensing array for object tracking.

A remote control device for a game, such as the "indicator positioning device for a camera" of the Republic of China invention patent No. I267754, which can be applied to a light gun game. The index positioning device is provided with a control circuit in the photographic indicator device, which are respectively connected. There are cameras, computing units, and communication interfaces. The communication interface is connected to the host, the camera is provided with a filter in front of the camera, and a plurality of light-emitting elements that can be photographed by the camera are disposed on the screen. When a user uses the photographic indicator device to execute a program executed by the host, when the camera captures the display screen, since the camera has a filter, the light source other than the spectrum emitted by the illuminating element is filtered out, so the camera shoots Only the light source of the light-emitting element appears in the picture. Then, the screen is transferred to the calculation unit to calculate the coordinate value of the camera aiming point and transmitted to the host, so that the host can use the coordinate value to perform related control.

However, in actual use, in order to improve the convenience of the remote control device, the remote control device uses the wireless communication to communicate data with the host, and uses a battery module to supply the overall power required by the remote control device. Since the remote control unit contains several energy-consuming components, it must be done The amount of energy consumed by each component is reduced to extend the life of the battery module. Usually, in order to improve the accuracy of calculating the coordinates of the aiming point, the camera system preferably uses a higher image capturing frequency to capture the image, but the high image capturing rate relatively increases the computing load of the computing unit, thereby increasing the remote control device. The overall consumption of electrical energy and reduce the life of the battery module.

The Republic of China Invention Patent Publication No. 200912700 discloses a "displacement detecting device and a power saving method thereof". The displacement detecting device should be a mouse, and includes a light source and an image capturing unit. The image capturing unit continuously captures an image formed by the light source projected onto a surface. The displacement detecting device calculates a position difference of the reference object in the continuous image to obtain a motion vector. When the motion vector value is less than a threshold value within a predetermined time, the displacement detecting device is confirmed to be in an idle state. Not an operational state. If the displacement detecting device is in an idle state, the light-emitting frequency of the light source is lowered, thus reducing power consumption. Similarly, the Republic of China Patent Publication No. 200943131 discloses a "displacement detecting device and a power saving method thereof", which adjusts the light-emitting period of the light source according to the displacement amount of the displacement detecting device during the two consecutive illumination periods of the light source. And hope to reduce power consumption.

Different from the aforementioned two patent specifications, the mouse is referred to as the "interactive imaging system, interactive device and its operation method", which can be adjusted in real time. The sampling frequency of the image module in the interactive device is changed to save power consumption of the interactive device as a whole. Regardless of reducing the sampling frequency of the image module or reducing the light-emitting frequency of the light source, these prior art techniques still capture the complete frame and process the data of the complete picture, so the power consumption cannot be effectively reduced.

The patents that may be related to this case can be found in U.S. Patent Nos. 7,839,388; US 7,738,013; US 7,738,013; US 7,619,669; US 20070024587 and the like.

In view of this, the present invention further provides a power saving method for a remote control device and an interactive system, which can select only a part of pixel data of a sensing array according to image features of the reference light source image, thereby saving the overall image sensing device. Energy consumption.

An object of the present invention is to provide a power saving method for a remote control device and an interactive system that can perform interactive operation without reading all pixel data of the sensing array of the image sensor.

Another object of the present invention is to provide a power saving method for a remote control device and an interactive system, which can estimate a range of interest of a next image according to an image feature of at least one reference light source image in a current image. And the energy consumption is reduced by reading only the pixel data of the range of interest in the sensing array.

The invention provides a remote control device comprising an image sensor, a reading circuit and a processing unit. The image sensor is sequentially A first image and a second image including at least one reference light source image are captured. The reading circuit reads one of the first image data of the first image and the second image data of the second image from the image sensor. The processing unit is configured to calculate an image feature of the at least one reference light source image according to the first image data, and control the read circuit to read only the second image data of a range of interest in the second image according to the image feature. Wherein the range of interest is smaller than the second image.

In one embodiment, the image feature includes one of image position of the reference light source image and at least one of an image size, an image brightness, and an action vector.

In one embodiment, the processing unit determines a first attention range in the first image according to an image position of the at least one reference light source image, and estimates the motion according to the motion vector of the reference light source image relative to the first attention range. A second range of interest in the second image or a second range of interest in the second image is estimated according to an extended parameter; wherein the motion vector and the extended parameter are a preset value or a plurality of pages according to the second image The image is calculated.

In one embodiment, the image sensor further extracts a pre-processed image including the at least one reference light source image, and the read circuit reads a portion of the column image data of the pre-processed image; The processing unit may determine a first range of interest according to the at least one reference light source image in the partial image data of the preprocessed image, and control the reading circuit to read only the first image in the first image. Focusing on the first image data of the range to calculate the image feature; wherein the partial column image data is image data of an odd column or an even column of the preprocessed image.

In one embodiment, the processing unit controls the reading circuit to further read the second image data of the second image outside the range of interest to perform debugging.

The present invention further provides a power saving method for an interactive system, comprising the steps of: sequentially capturing, by an image sensor, a preprocessed image and a first image including at least one reference light source image; and reading by using a read circuit Part of the image data of the pre-processed image; determining, by the processing unit, a first range of interest according to the at least one reference light source image in the portion of the image data of the pre-processed image; using the read circuit to read only the first image a first image data of the first attention range in the image; and calculating, by the processing unit, an image feature of the at least one reference light source image according to the first image data.

In one embodiment, the power saving method of the interactive system further includes the following steps: using the image sensor to capture another second image including the at least one reference light source image; and using the processing unit to control the image according to the image feature The reading circuit reads only the second image data of one of the second regions of interest in the second image, and the second region of interest is smaller than the second image.

The invention further provides a power saving method for an interactive system, including The following steps: sequentially capturing, by an image sensor, a first image and a second image including at least one reference light source image; reading a total image data of the first image by using a reading circuit; using a processing unit Calculating an image feature of the at least one reference light source image according to the entire image data of the first image; and controlling, by the processing unit, the read circuit to read only one image of a target range in the second image according to the image feature Data, and the range of interest is smaller than the second image.

In the power saving method of the remote control device and the interactive system according to the embodiment of the present invention, since the reading circuit only needs to read part of the image data of the sensing array, the programmable gain amplifier (PGA) and the analog digital converter can be effectively saved. Energy consumption of pixel data processing components such as (ADC) and processing units. In addition, since the time for reading the image data is relatively reduced, the frame rate can be relatively increased to increase the system reaction speed.

The above and other objects, features, and advantages of the present invention will become more apparent from the accompanying drawings. In the description of the present invention, the same components are denoted by the same reference numerals and will be described in the foregoing.

The present invention is applicable to the execution of an interactive game program of a game console, remote browsing of a webpage on a monitor, and operation of other remote video and audio systems (eg, smart TV; smart TV), but due to the game host It is the most common at present, so the following is an example of a game console.

Referring to Fig. 1, there is shown a schematic diagram of an interactive image system in accordance with an embodiment of the present invention. The interactive imaging system 10 includes an interactive control device 11, a host 12, a light source 13, and an image display 14. The embodiment of the host computer 12 can be a game console, a computer system host or an interactive image player, and can execute an interactive game program to generate an interactive image for display on the image display 14 for user manipulation. The host computer 12 includes a wireless module 121 and a processor 122. The wireless module 121 and the wireless module 111 of the interactive control device 11 transmit data (RF1 and RF2) bidirectionally. The processor 122 can use the data according to the data. Play interactive games. The host 12 can also be connected to the image display 14 in a wired or wireless manner, or the host 12 can be directly disposed in the image display 14.

A cursor is displayed on the screen of the image display 14. Examples of the image include, but are not limited to, a bullet point of a light gun and an arrow of an image. The interactive control device 11 includes the wireless module 111, a control circuit 112, and an image sensor 113. The image sensor 113 can capture the image of the plurality of reference points 131 in the light source 13, and the control circuit 112 according to the reference point 131. The image determines the position, direction of movement and amount of movement of the interactive control device 11, that is, the positioning and displacement of the aforementioned cursor.

The embodiment of the reference point 131 of the light source 13 can be, but is not limited to, a plurality of infrared light emitting diodes or light emitting units of a specific wavelength. The reference points of various shapes of the arranged components are electrically connected to the host 12 or the image display 14, and the power required for illumination can be supplied by an independent power source. The light source 13 can also be integrated into the display 14 or the host 12. In addition, the number of reference points 131 is not limited to six in the figure, and may be one or plural. The interactive control device 11 captures the images of the reference points 131 to determine the relative position and/or angular change of the interactive control device 11 and the reference point to relatively control the position and movement of the cursor on the image display 14. The interactive control device 11 is configured to control a game program or a computer software executed by the host computer 12. When the executor of the host 12 is a game software, the interactive device 11 can be, for example, but not limited to, a light gun, a cue stick, a golf club, a tennis racket, a bat, a racquet And table tennis bats to control the game. When the executor of the host computer 12 is a computer software, the interactive control device 11 can be used as an indicator (cursor) locating device to control the execution of the computer software.

2A-2D shows that the interactive control device of the present invention captures an image including a plurality of reference point images. The images 20a to 20d in FIGS. 2A to 2D are images in which a plurality of reference points are imaged in the image sensor 113. Figure 2A is a complete image captured in the idle state, and the reference point image is the imaged area of the reference point (bright spot). The reference point images 21 to 23 in the figure are merely examples, and are not limited to pixels of one unit, and one unit in the figure may also represent a plurality of pixels. In addition, the images 20a-20d are acquired by the interactive control device 11 in an idle state. The image, but the decision mechanism for switching from the operating state to the idle state can adopt any method disclosed in the prior art. For example, the foregoing method for judging the entry into the idle state by using the motion vector is within the protection scope of the present invention.

In Fig. 2A, the image 20a has three reference point images 21~23. The white area represents the three reference point images 21~23, and the other thin point array areas are scanned by the image sensor 113 for the undetected reference point image. The pixel unit, the image 20a is obtained by completely scanning each pixel unit by the image sensor 113.

In the image of FIG. 2B, the image sensor 113 scans the pixel cells corresponding to the three reference point images in the image 20a, and the other pixel cells are turned off to effectively reduce the power consumption. The pixel array area in the middle represents the pixel unit that is not turned on. That is, the control circuit 112 of the interactive control device 11 turns on the pixel cells of the three reference point images 21 to 23 in the corresponding image 20a of the image sensor 113, and scans only the pixel cells. In the figure, the reference point images 21 and 23 are still scanned in the original position, but the reference point image 22 cannot be scanned in the original scanned area, and the control circuit 112 switches the interactive control device 11 from the idle state to the operating state.

In order to confirm that at least one reference point image in the image 20b does not exist in the original position in the image 20a, and the interactive control device 11 is switched to the operating state, the control circuit 112 compares a threshold value with the image 20b. The brightness information of the pixels of the three reference point images 21 to 23 in the corresponding image 20a, for example, the brightness value or the brightness average of the pixel unit of the reference point image 22 in the corresponding image 20a in the image 20b is lower than a predetermined value. The threshold value can be judged that the reference point image 22 or the bright spot has left the original position in the image 20a. On the other hand, if the brightness of the pixel unit is higher than a predetermined threshold, it can be assumed that the reference point image 22 is still in the original position in the image 20a. If the reference point images 21 and 23 are also in their original positions, the interactive control can be assumed. The device 11 is still idle. When it is confirmed that the interactive control device 11 is still in an idle state, the control circuit 112 can further reduce the sampling frequency of the image sensor 113 or reduce the lighting frequency of the light source 13 to save power consumption. According to the system requirements, it can be decided to switch to the operating state when a bright spot disappears, or remain in an idle state as long as a bright spot persists, and further can be set to have at least N bright spots to remain in an idle state.

In an embodiment, the threshold may include one or more, for example, all pixels of the image 20b may be compared with the same threshold; or each pixel is compared with a different threshold. At this time, the thresholds may be, for example, a ratio or an offset of one of the grayscale values of the respective pixels in the previous image before entering the idle state.

In Fig. 2B, only the pixel cells of the three reference point images in the image 20a are scanned, but the interactive control device 11 may be mistakenly left out of the idle state due to noise or fine shaking of the desktop. So you can image three in three The pixel of the reference point image is used as a reference, and the plurality of pixel grids are expanded to the periphery and the scanning range is selected. For example, in FIG. 2A, three pixel areas 241-243 are used as the scanning range, or as the attention area, that is, other areas. The pixel grid is turned off when scanning the next screen. In the present invention, the pixel area may be the aforementioned extended scanning range or the pixel range in which the original reference point image is located. The 2C picture is the image 20c obtained by the next picture scanning, and the reference point images 22 to 23 in the pixel areas 242 to 243 are not changed in position, but the position of the reference point image 21 in the pixel area 241 is changed. Since the reference point image 21 is still located in the pixel area 241, the control circuit 112 can judge that the interactive control device 11 has not left the idle state. Or if it is detected that at least one bright spot exists in the pixel area 241, it can be judged that the interactive control device 11 has not left the idle state. The so-called highlight presence or judgment, that is, the brightness value of a pixel is greater than a predetermined threshold, it can be considered that the pixel is a bright spot.

Referring to FIG. 2D, the pixel regions 244 and 245 in the image 20d select the entire column of pixels as the scanning range according to the positions of the three reference point images 21 to 23 in the image 20a. In other embodiments, the entire column of pixels may be selected as the scanning range. The scanning range of the foregoing embodiments may be set to the same size, enlarged or reduced pixel corresponding to the reference images 21-23.

The foregoing embodiments are illustrated by way of a simplified diagram, which is further illustrated by a practical example. The pixel size of an image sensor It is 128×96. If there are two reference point images in the first image, and the image size of each reference point is about 30~50 pixels. The two pixel areas are set to 50 pixel cells, so the image sensor only needs to turn on the pixel of 0.81%, and the calculation is as follows: (50 × 2) / (128 × 96) = 0.00813. Similarly, the circuit module such as signal and image processing in the interactive control device 11 can reduce the number of processing pixels in the same proportion, thereby further reducing power consumption.

Figure 3 is a flow chart of the method of operation of the interactive imaging system of the present invention. Referring to step 31, in the idle state, the interactive control device turns on all the pixel cells of the image sensor to capture a first image. Selecting at least one pixel region according to the position of the at least one reference point image in the first image, and the pixel region may be the same range or an expanded range of the corresponding pixel grid including the reference point image, as shown in step 32. According to step 33, the image sensor scans the pixel cells in the pixel area to obtain a second image. As shown in step 34, comparing a threshold value Th with brightness information of at least one pixel in the pixel region in the second image, for example, a brightness value or a brightness average value, to confirm whether the interactive control device is still in the idle state. . As shown in step 35, when it is confirmed that the interactive control device is still in an idle state, the sampling frequency of the image sensor can be further reduced, or the light-emitting frequency of the light source can be reduced to save power consumption. If the sampling frequency or the illuminating frequency has dropped to the lowest frequency value, there is no need to downgrade, and jump back to step 33. On the other hand, if the brightness value of the pixel is lower than the threshold Th, it can be determined that the reference point image is not in the first image. The central location requires the interactive control device to enter an operational state, as shown in step 36.

Please refer to FIG. 4A and FIG. 4B. FIG. 4A is a schematic diagram showing an interactive system according to an embodiment of the present invention; FIG. 4B is a block diagram showing a remote control device of the interactive system according to an embodiment of the present invention. The interactive system of the embodiment of the present invention comprises a remote control device 4, an image device 5 and at least one reference light source (for example, two reference light sources 51 are displayed here); wherein the reference light source 51 can be coupled to the image device 5 or independently The power supply is separated from the image device 5, and there is no particular limitation. The reference light source 51 preferably emits invisible light or emits light in a predetermined illumination mode to distinguish it from ambient light; the reference light source 51 can be, for example, a light emitting diode, a laser diode or other active and passive light source, without particular limitation. . In other embodiments, the reference light source 51 can be displayed in the image of the image device 5. It should be noted that the number of reference light sources 51 is determined according to the algorithm used to track the reference light source 51, and is not limited to a specific number.

The image device 5 can be, for example, a smart TV, a game console screen, a projector screen, a computer screen, or the like for displaying images. In this embodiment, the remote control device 4 is configured to capture an image including the at least one reference light source 51 and change image (eg, position change and/or angle change) of the at least one reference light source image in the captured image. Interacting with the image device 5, for example, controlling an application executed by the image device 5, displayed An image or a cursor or the like; wherein a function of interacting with a display device by a remote controller is known, and thus will not be described herein. The present invention mainly aims to reduce the energy consumption of the remote control device 4 during operation.

The remote control device 4 includes an image sensor 41, a read circuit 42 and a processing unit 43. The image sensor 41 captures images at a fixed or adjustable sampling frequency. The read circuit 42 The image sensor 41 reads all or part of the image data of the captured image and transmits the image data to the processing unit 43. The processing unit 43 performs post processing on the image data and controls the reading circuit 42 (for example, transmitting a control signal). The area where the image data is read from the image sensor 41 is selected, and the processing unit 43 can interact with the image device 5 according to the image data.

Referring to Figures 4B and 5, the operation of the interactive system of the first embodiment of the present invention will be described.

The image sensor 41 is preferably an active array image sensor, such as a CMOS image sensor, including a sensing array (eg, shown as an 8×8 sensing array) Capture images at a sampling frequency. For example, in FIG. 5, the image sensor 41 sequentially captures a first image 411 and a second image 412 respectively including at least one reference light source image I ₅₁ and I ₅₁ '. In this embodiment, the first image 411 and the second image 412 represent image data actually sensed by the sensing array of the image sensor 41, that is, the first image 411 and the second image 412 are The image is formed by measuring each pixel data (ie, grayscale value) of the array. The reading circuit 42 reads the first image data of the first image 411 and the second image data of the second image 412 from the image sensor 41. In this embodiment, the first image data is the first image data. All of the pixel data of the first image 411, and the second image data is part of the pixel data of the second image 412. In other words, in the embodiment, the reading circuit 42 can select a part of the pixel data of the image captured by the image sensor 41. In an embodiment, since the pixel grayscale values of the reference light source images I ₅₁ and I ₅₁ ′ are greater than the background gray scale values (ie, the pixel regions other than the reference light source images I ₅₁ and I ₅₁ ′), in this embodiment, Determining the pixel regions of the first image 411 and the second image 412 with the pixel grayscale value greater than a threshold value as the reference light source images I ₅₁ and I ₅₁ ', and the threshold value may be determined according to different applications; The manner in which the reference light source image is recognized from the image is well known and will not be described here.

The processing unit 43 can be a digital signal processor (DSP) for calculating an image feature of the at least one reference light source image I ₅₁ according to the first image data, for example, including an image position of the reference light source image I ₅₁ and At least one of an image size, an image brightness, and a motion vector. The processing unit 43 determines a range of interest (ROI) of the sensing array according to the image feature, so as to control the reading circuit 42 to read only the second image of the ROI of the second image 412. In order to save energy consumption of the remote control device 4 during operation, for example, the read circuit 42, the programmable gain amplifier (PGA), the analog digital converter (ADC), and the energy consumption of the processing unit 43 are included. The ROI of interest is smaller than the second image 412. For example, assuming that the size of the sensing array is 128×96, the size of the reference light source image I ₅₁ is 50. In the embodiment considering two reference light sources 51, the ROI of the attention range is 0.81% of the sensing array, so The amount of image data read and post-processed can be greatly reduced.

In addition, since the plurality of pixels of the sensing array are arranged in an array, the ROI _{2 of} interest may be, for example, a rectangular pixel region (for example, FIG. 5) or a plurality of column pixel regions (for example, the pixel column R ₂ to R of FIG. 6 ). ₄ or pixel column R ₂ ~ R ₈ ). In other embodiments, the shape of the ROI of interest is preferably a predetermined shape pixel region determined according to the shape of the reference light source 51, and is not limited to those disclosed in FIGS. 5 and 6. It should be noted that the area filled with the sparse points in the second image 412 in FIG. 5 represents the pixel area that is not read.

In one embodiment, the processing unit 43 determines a first attention range ROI ₁ in the first image 411 according to an image size and an image position of the at least one reference light source image I ₅₁ , and is relative to the first attention range ROI ₁ Estimating a second range of interest ROI ₂ in the second image 412 according to an action vector MV of the reference source image I ₅₁ (for example, a direction vector of the reference source image I ₅₁ to I ₅₁ ' shown in the upper right diagram); The pixel position may be, for example, a center of gravity or a center position of the reference light source image I ₅₁ . The image size may be, for example, the number of pixels in the image whose gray scale value is greater than the threshold value. According to different embodiments, the range of interest ROI ₁ The ROI ₂ may select an image size greater than or equal to one of the reference light source images I ₅₁ , I ₅₁ '. In this embodiment, the motion vector MV may be a preset value or may be calculated by the processing unit 43 according to the plurality of images before the second image 412 (including or not including the first image 411); for example, Determining the preset value of the motion vector MV according to an operation parameter of the interactive system (for example, including a size of the reference light source 51 and an operating distance of the remote control device 4), or according to the second image 412 A relationship (e.g., correlation) is obtained; wherein the motion vector MV includes, for example, information on the direction of movement and the amount of movement.

In another embodiment, the processing unit 43 determines a first attention range ROI ₁ in the first image 411 according to an image size and an image position of the at least one reference light source image I ₅₁ , and according to an extended parameter dp (right The following figure is used to estimate a second range of interest ROI ₂ in the second image 412; wherein the extended parameter dp can be a preset value or the processing unit 43 calculates the image based on the plurality of images before the second image 412. For example, the action vector MV obtained by the front image of the second image 412 (including or not including the first image 411) is used as the extended parameter dp, or the extended parameter dp is preset according to an operation parameter of the interactive system. For example, the extended parameter dp is extended to two pixels). In addition, the image size in the above two embodiments may also be set in advance according to the operating parameters of the interactive system (for example, the size of one of the reference light sources 51 and the operating distance of one of the remote control devices 4), so the processing unit 43 is only based on the at least one. A first attention range ROI ₁ in the first image 411 can be determined by referring to one of the image positions of the light source image I ₅₁ .

Referring to Figures 4B and 6, the operation of the interactive system of the second embodiment of the present invention will be described. The embodiment saves the energy consumption of the remote control device 4 when the image sensor 41 does not capture the at least one reference light source image I ₅₁ .

In the embodiment, the image sensor 41 captures a pretreatment image 410 including at least one reference light source image I ₅₁ before the first image 411 is captured, that is, the preprocessed image 410 is not It is used to calculate the image change of the reference light source 51 and is only used to determine whether the reference light source image is captured and its image position. The reading circuit 42 reads a portion of the image data of the pre-processed image 410. The partial image data is, for example, the image data of the odd-numbered column or the even-numbered column of the pre-processed image 410, but is not limited thereto. In other embodiments, at least one column of pixels may be skipped after reading the plurality of columns of image data each time. In other words, when the processing unit 43 determines that the image sensor 41 has not captured the at least one reference light source image I ₅₁ , the reading circuit 42 is controlled to read only the sensing array of the image sensor 41 . The portion of the image data is used as the pre-processed image 410. When the processing unit 43 determines that the image sensor 41 has captured the at least one reference source image I ₅₁ , the processing unit 43 is based on the pre-processed image 410. The at least one reference light source image I ₅₁ determines a first range of interest ROI ₁ in the partial image data, and controls the reading circuit 42 to read only the first target range ROI ₁ in the first image 411. An image data; the processing unit 43 calculates an image feature of the at least one reference light source image I ₅₁ according to the first image data of the first attention range ROI ₁ , and controls the reading circuit 42 to read only according to the image feature. Taking the second image data of the second region of interest ROI ₂ in the second image 412, wherein the reading circuit 42 controls the reading of the second region of interest ROI ₂ in the second image 412 according to the image feature. The second image data Formula similar to the first embodiment and therefore omitted herein. The difference between the second embodiment and the first embodiment is that the reading circuit 42 first reads a part of the column image data of the pre-processed image 410 to determine a first range of interest ROI ₁ and reads only the first image 411. the first range of interest ROI in the image data of the first ₁ and not the entire range of interest ROI of _an image data; wherein, the first ROI size range of interest, for example, according to _an interactive system operating parameters (e.g., the reference light source 51 One size and one operating distance of the remote control device 4 are preset. It should be noted that, in the pre-processed image 410, the first image 411, and the second image 412 in FIG. 6 , the area filled with the sparse points indicates the pixel area that is not read. Further, in other embodiments, the same operation can be performed based on a pixel column.

Further, since the processing unit 43 may misjudge the reference source image I _51, the first and second embodiments of the present invention, the processing unit 43 may control the reading circuit 42 reads the second image further in the first 412 The second image data of the partial range other than the range ROI ₂ is used to confirm whether the other reference light source image in the second image 412 is not included in the second attention range ROI ₂ . In addition, since the reading circuit 42 only needs to read a part of the pixel area of the sensing array of the image sensor 41, the sampling frequency can be further increased to relatively increase the system reaction speed. In addition, in order to simplify the illustration, only one reference light source image I ₅₁ is shown in FIGS. 5 and 6. It can be understood that the operation of a reference light source image as shown in the figure can be extended to the operation of a plurality of reference light source images.

Referring to FIG. 7, a flowchart of a power saving method of the interactive system according to the first embodiment of the present invention includes the following steps: capturing a first image including at least one reference light source image (step _S61 ); Reading all image data of the first image (step _S62 ); calculating image characteristics of the at least one reference light source image according to the total image data of the first image (step _S63 ); extracting the at least one reference a second image of the light source image (step _S64 ); reading only one image of a range of interest in the second image according to the image feature (step _S65 ); and updating the second focus to the first focus And updating the second image data to the first image data (step _S66 ), and then returning to step _S63 .

Please refer to FIGS. 4A, 4B, 5 and 7 simultaneously, and then explain the implementation of the power saving method of the interactive system according to the first embodiment of the present invention. the way.

Step _S61 : The image sensor 41 first captures a first image 411 including at least one reference light source image I ₅₁ .

Step _S62 : The reading circuit 42 sequentially reads all the image data of the first image 411.

Step _S63 : The processing unit 43 calculates an image feature of the at least one reference light source image I ₅₁ according to the entire image data of the first image 411. As described above, the image feature includes one image position of the reference light source image. And at least one of an image size, an image brightness, and an action vector.

Step _S64 : Next, the image sensor 41 captures a second image 412 including one of the at least one reference light source image (eg, I ₅₁ ').

Step _S65 : The processing unit 43 determines a range of interest ROI of the sensing array according to the image feature, so as to control the reading circuit 42 to read only one image of the ROI in the second image 412 without reading. Take image data other than the ROI of the range of interest. In an embodiment, the processing unit 43 determines a first attention range ROI ₁ in the first image 411 according to the image position of the at least one reference light source image I ₅₁ , and according to the first reference range ROI ₁ according to the reference A motion vector MV of the source image I ₅₁ estimates a second range of interest ROI ₂ in the second image 412 (as shown in the upper right diagram of FIG. 5); in another embodiment, the processing unit 43 can use the reference source image I. one of the ₅₁ images of the first image position determining a first range of interest 411 ROI _1, and relative to the first range of interest ROI ₁ estimated in the second image a second range of interest ROI ₂ 412 according to an expansion parameter PD ( As shown in the bottom right of Figure 5). As described above, the motion vector MV and the extended parameter pd may be a preset value or the processing unit 43 calculates the image based on the plurality of images before the second image 412. Since the purpose of the embodiment is to save energy, the second range of interest ROI _{2 is} smaller than the second image 412. In other embodiments, if a preset extended parameter is used and the operating parameters of the interactive system are known, the first region of interest ROI ₁ may not be determined before the first image 411, and only the first image 411 needs to be known. The image position of the reference light source image I ₅₁ can directly set the second region of interest ROI ₂ according to the preset expansion parameter.

Step _S66 : Finally, the processing unit 43 updates the second range of interest ROI ₂ to the first range of interest ROI ₁ and updates the second image data to the first image data. Then, it returns to step _S63 .

After returning to step _S63 , the processing unit 43 calculates an updated image feature of the at least one reference light source image I ₅₁ according to the updated first image data (ie, step _S63 ); and updates the second image to a The newly captured image (ie, step _S64 ), that is, the image sensor 41 retrieves an image as the updated second image 412; and the processing unit 43 controls the reading according to the updated image feature. The circuit 42 reads only the second image data of one of the second regions of interest ROI ₂ in the updated second image 412 (ie, step _S65 ).

The processing unit 43 can track the reference light source image according to the correlation between the first image 411 and the second region of interest ROI ₂ and interact with the image device 5 according to the tracking result, and execute steps S ₆₃ -S ₆₆ cyclically to continue Track the action of the reference source image. For example, a sampling frequency loop step S ₆₃ ~ S _66.

In this embodiment, in order to avoid the occurrence of the erroneous control, the processing unit 43 controls the reading circuit 42 to read the portion of the second image 412 other than the second range of interest ROI ₂ every other preset time. Image data to confirm whether there is an unread reference light source image; wherein the preset time can be determined according to the required accuracy and different applications.

Please refer to FIG. 8 , which shows a flowchart of a power saving method of the interactive system according to the second embodiment of the present invention, including the following steps: capturing a pre-processed image including at least one reference light source image (step S ₇₁ ); And reading a portion of the image data of the pre-processed image (step _S72 ); determining a first range of interest according to the at least one reference light source image in the partial image data of the pre-processed image (step _S73 ); ???the first image of the at least one reference light source image (step _S74 ); reading only the first image data of the first attention range in the first image (step _S75 ); calculating the first image data according to the first image data At least one image feature of the reference light source image (step _S76 ); capturing a second image including the at least one reference light source image (step _S77 ); and reading only one second of the second image according to the image feature Focusing on one of the second image data of the range (step _S78 ); and updating the second range of interest to the first range of interest and updating the second image data to the first image data (step _S79 ), and then returning Step _S76 . Since the purpose of the embodiment is to save energy, the second range of interest is smaller than the second image.

Referring to Figures 4A, 4B, 6 and 8 simultaneously, an embodiment of the power saving method of the interactive system of the second embodiment of the present invention will be described.

Step _S71 : The image sensor 41 captures a pre-processed image 410. As previously described, the pre-processed image 410 is used to confirm whether the reference source image has entered the field of view of the image sensor 41.

Step _S72 : In order to save energy, the reading circuit 42 reads only a part of the column image data of the pre-processed image 410; wherein the partial column image data is an image of the odd-numbered column or the even-numbered column of the pre-processed image 410. Data, but not limited to this, it is also possible to skip at least one column of image data after each capture of at least two columns of image data, as long as the image data of the pre-processed image 410 is captured. Yes, there are no specific restrictions. For example, in the middle upper diagram of Fig. 6, the reading circuit 42 reads only the blank pixel columns without reading the pixel columns filled in the sparse dots. When the image sensor 41 does not capture the reference light source image I 51 of the reference light source ₅₁ , the process returns to step _S71 , and when the reference light source image I _{51 of} the reference light source 51 has been captured, the process proceeds to step S ₇₃ .

Step _S73 : The processing unit 43 determines a first attention range ROI _{1 of the} sensing array according to the at least one reference light source image I ₅₁ in the partial image data of the preprocessed image 410, for example, the processing unit 43 according to the The image position of the at least one reference light source image I ₅₁ determines the position of the first attention range ROI ₁ , and the size of the first attention range ROI ₁ may be according to an operation parameter of the interactive system, for example, one of the reference light sources 51 and / or one of the remote control devices 4 is determined by the operating distance.

Step _S74 : Next, the image sensor 41 captures a first image 411 including one of the at least one reference light source image I ₅₁ .

Step _S75 : When the first range of interest ROI ₁ has been determined, the processing unit 43 controls the reading circuit 42 to read only the first image data of the first region of interest ROI ₁ in the first image 411 without reading. The image data other than the first range of interest ROI _{1 is} taken to reduce energy consumption. As described above, in the embodiment, the sampling frequency can be simultaneously increased to increase the system reaction speed.

Step _S76 : The processing unit 43 may calculate an image feature of the at least one reference light source image I ₅₁ according to the first image data; as described above, the image feature may include an image position of the reference light source image I ₅₁ And at least one of an image size, an image brightness, and an action vector.

Step _S77 : Next, the image sensor 41 captures a second image 412 including one of the at least one reference light source image (eg, I ₅₁ ').

Step S _78: The decision processing unit 43 according to the image sensing array wherein one of said second range of interest ROI _2, and controls the reading circuit 42 reads only the second image 412 in the second of the range of interest ROI ₂ The image data other than the second range of interest ROI _{2 is} not read by a second image data; wherein the embodiment is the same as the step S _{65 of the} first embodiment, and thus no further details are provided herein.

Step _S79 : Finally, the processing unit 43 updates the second attention range ROI ₂ to the first attention range ROI ₁ and updates the second image data to the first image data. Then, it returns to step _S76 .

After returning to step _S76 , the processing unit 43 calculates an updated image feature of the at least one reference light source image I ₅₁ according to the updated first image data (ie, step _S76 ); and updates the second image to a The newly captured image (ie, step _S77 ), that is, the image sensor 41 retrieves an image as the updated second image 412; and the processing unit 43 controls the reading according to the updated image feature. The circuit 42 reads only the second image data of one of the second regions of interest ROI ₂ in the updated second image 412 (ie, step _S78 ).

The processing unit 43 can track the reference light source image according to the correlation between the first region of interest ROI ₁ and the second region of interest ROI ₂ and interact with the image device 5 according to the tracking result, and sequentially perform steps S ₇₆ to S ₇₉ to continuously track the motion of the reference light source image. For example, steps S ₇₆ to S ₇₉ can be performed in a sampling frequency cycle.

Similarly, in addition to loop step step S ₇₆ ~ S _79, the control in order to avoid misuse, the interactive system may be set to a predetermined time interval from the step S ₇₈ back to the second embodiment of the re-execution step S ₇₁ of the present invention The power saving method prevents other reference light source images from being outside the ROI of the attention area, that is, even if the reference light source image has been captured, the pre-processed image 410 can be captured once every preset time to increase the operation precision. degree. In addition, the processing unit 43 can control the read circuit 42 to read the second image data of the updated second portion of the second image range 412 other than the second range of interest ROI ₂ at a predetermined time. The purpose is also to increase operational accuracy. In this embodiment, the preset time may be set according to different applications such as system accuracy and system parameters.

In summary, the power-saving method of the conventional interactive system still has the problem that the system energy consumption cannot be fully saved. Therefore, the present invention further provides an interactive system (Fig. 4A), a remote control device (Fig. 4B), and a power saving method thereof (Figs. 7-8), which can read only part of the image data of a sensing array. Post-processing to save system energy. In addition, when the image sensor does not capture the reference light source image, the system energy consumption can be further reduced by reading only part of the column image data.

Although the present invention has been disclosed in the foregoing embodiments, it is not intended to limit the present invention, and any of the technical fields of the present invention are Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, the scope of the invention is defined by the scope of the appended claims.

10‧‧‧Interactive imaging system

13‧‧‧Light source

11‧‧‧Interactive control device

131‧‧‧ reference point

111‧‧‧Wireless Module

14‧‧‧Image display

112‧‧‧Control circuit

21~23‧‧‧ reference point image

113‧‧‧Image Sensor

20a~20d‧‧‧ images

12‧‧‧Host

241~245‧‧‧pixel area

121‧‧‧Wireless Module

31~36‧‧‧Steps

122‧‧‧Processor

4‧‧‧Remote control

41‧‧‧Image sensor

410‧‧‧Pre-processing images

411‧‧‧ first image

412‧‧‧Second image

42‧‧‧Read circuit

43‧‧‧Processing unit

5‧‧‧Video device

51‧‧‧Reference light source

I ₅₁ , I ₅₁ '‧‧‧ reference source image

ROI ₁ ‧‧‧First Focus

ROI ₂ ‧‧‧second focus

MV‧‧‧ action vector

Dp‧‧‧ expansion parameters

S ₆₁ ~S ₆₆ ‧‧‧Steps

S ₇₁ ~S ₇₉ ‧‧‧Steps

R ₁ ~R ₈ ‧‧‧pixel columns

1 is a schematic diagram showing an interactive image system according to an embodiment of the present invention.

2A-2D shows that the interactive control device of the present invention captures an image including a plurality of reference point images.

Figure 3 is a flow chart showing the operation method of the idle state confirmation of the present invention.

Figure 4A shows a schematic diagram of an interactive system in accordance with an embodiment of the present invention.

FIG. 4B is a block diagram showing the remote control device of the interactive system according to the embodiment of the present invention.

Fig. 5 is a view showing the operation of the interactive system of the first embodiment of the present invention.

Figure 6 is a diagram showing the operation of the interactive system of the second embodiment of the present invention.

Fig. 7 is a flow chart showing the power saving method of the interactive system of the first embodiment of the present invention.

Figure 8 is a flow chart showing the power saving method of the interactive system of the second embodiment of the present invention.

4‧‧‧Remote control

41‧‧‧Image sensor

5‧‧‧Video device

51‧‧‧Reference light source

Claims (22)

A remote control device includes: an image sensor for sequentially capturing a first image and a second image including at least one reference light source image; and a reading circuit for reading the first image from the image sensor a first image data of the image and a second image data of the second image; and a processing unit configured to calculate an image feature of the at least one reference light source image according to the first image data, and according to the image feature Controlling the reading circuit to read only the second image data of a range of interest in the second image, wherein the range of interest is smaller than the second image. The remote control device of claim 1, wherein the image feature comprises one of image position of the reference light source image and at least one of an image size, an image brightness, and an action vector. According to the remote control device of claim 1, wherein the range of interest is a preset shape pixel area or a plurality of column pixel areas. The remote control device of claim 1, wherein the processing unit determines a first attention range in the first image according to an image position of the at least one reference light source image, and is based on the reference light source image with respect to the first attention range One motion vector estimates a second range of interest in the second image or estimates a second range of interest in the second image based on an extended parameter. The remote control device of claim 4, wherein the motion vector and the extended parameter are a preset value or the processing unit calculates the image based on the plurality of images before the second image. The remote control device of claim 1, wherein the image sensor further extracts a preprocessed image including the at least one reference light source image before the first image is captured, and the read circuit reads the preprocess Part of the image is the image data. The remote control device of claim 6, wherein the processing unit determines a first range of interest according to the at least one reference light source image in the partial image data of the preprocessed image, and controls the read circuit to read only the The first image data of the first range of interest in the first image to calculate the image feature. According to the remote control device of claim 6, wherein the part of the image data is the image data of the odd or even columns of the preprocessed image. The remote control device of claim 1, wherein the processing unit controls the reading circuit to read the second image data of the second image in a portion other than the range of interest. According to the remote control device of claim 1, wherein the range of interest is greater than or equal to an image size of the reference light source image. The remote control device of claim 1, wherein the at least one reference light source is fixed to an image device. An energy saving method for an interactive system, comprising the following steps: Using a video sensor to sequentially capture a pre-processed image and a first image including at least one reference light source image; and reading a portion of the column image data of the pre-processed image by using a read circuit; The at least one reference light source image in the partial image data of the pre-processed image determines a first range of interest; the read circuit is used to read only the first image data of the first attention range in the first image; The processing unit calculates an image feature of the at least one reference light source image according to the first image data. According to the power saving method of claim 12, wherein the part of the image data is the image data of the odd or even columns of the preprocessed image. The power saving method of claim 12, wherein the processing unit determines the first attention range according to an image position of the at least one reference light source image. According to the power saving method of claim 12, the method further includes: using the image sensor to further capture a second image including the at least one reference light source image; and using the processing unit to control the image according to the image feature The reading circuit reads only the second image data of one of the second regions of interest in the second image, and the second region of interest is smaller than the second image. According to the power saving method of claim 15, the method further includes: updating the second attention range to the first attention range and updating the second image data to the first image data; and using the processing unit according to the updated The first image data is used to calculate an updated image feature of the at least one reference light source image; the second image is updated as a newly captured image; and the processing unit controls the read circuit to read only according to the updated image feature. And taking the second image data of one of the second attention ranges in the updated second image. According to the power saving method of claim 16, wherein the pre-processed image is captured every predetermined time. According to the power saving method of claim 16, the method further comprises: controlling, by the processing unit, the reading circuit to read the updated second portion of the second image in the second image. The second image data. An energy saving method for an interactive system, comprising the steps of: sequentially capturing, by an image sensor, a first image and a second image including at least one reference light source image; and reading the first image by using a reading circuit Calculating, by using a processing unit, image features of the at least one reference light source image according to the image data of the first image; and controlling, by the processing unit, the read circuit to read only the first image image One of the two images is of interest in one of the range of image data, and the range of interest is smaller than the second image. According to the power saving method of claim 19, the method further comprises: controlling, by the processing unit, the reading circuit to read the image data of the second image in a portion other than the range of interest. The power saving method of claim 19, wherein the processing unit determines a first attention range in the first image according to an image position of the at least one reference light source image, and is based on the reference light source relative to the first attention range One of the image motion vectors estimates a second range of interest in the second image or estimates a second range of interest in the second image based on an extended parameter. The power saving method according to claim 21, wherein the motion vector and the extended parameter are a preset value or the processing unit calculates the image according to the plurality of images before the second image.