WO2019120052A1

WO2019120052A1 - Method and apparatus for decoding information transmitted by optical source

Info

Publication number: WO2019120052A1
Application number: PCT/CN2018/117868
Authority: WO
Inventors: 方俊; 牛旭恒
Original assignee: 北京外号信息技术有限公司
Priority date: 2017-12-19
Filing date: 2018-11-28
Publication date: 2019-06-27
Also published as: CN109936713A; CN109936713B

Abstract

Disclosed is a method for decoding information transmitted by an optical source. The optical source is configured to be capable of working in at least two modes, wherein in different modes, when the optical source is photographed by means of a rolling shutter imaging device, obtained images of the optical source display different pre-determined appearances so as to represent different data. The method comprises: continuously photographing an optical source by means of a rolling shutter imaging device; acquiring several frames of continuous images of the optical source; with regard to each frame of image of the optical source, respectively extracting a starting portion and an ending portion of the image; analyzing the extracted starting portion and ending portion to determine data represented by each starting portion and each ending portion; and executing a data removal operation on data represented by the ending portion of each frame of image and on data represented by the starting portion of the next following frame of image to determine a data sequence transmitted by the optical source.

Description

Method and apparatus for decoding information transmitted by a light source

Technical field

The present invention belongs to the field of optical information technology, and more particularly to a method for decoding information transmitted by a light source.

Background technique

Barcodes and QR codes have been widely adopted to encode information. When these barcodes and QR codes are scanned with a specific device or software, the corresponding information is identified. However, the recognition distance between the barcode and the two-dimensional code is very limited. For example, for a two-dimensional code, when scanning with a mobile phone camera, the phone must typically be placed at a relatively short distance, typically about 15 times the width of the two-dimensional code. Therefore, for long-distance recognition (for example, 200 times the width of the two-dimensional code), barcodes and two-dimensional codes are usually not implemented, or very large barcodes and two-dimensional codes must be customized, but this will bring about an increase in cost. And in many cases it is impossible to achieve due to various other restrictions.

A CMOS imaging device is a widely used imaging device, as shown in FIG. 1, including an array of image sensitive cells (also referred to as image sensors) and some other components. The image sensor array can be a photodiode array, with each image sensor corresponding to one pixel. Each column of image sensors corresponds to a column amplifier, and the output signal of the column amplifier is sent to an A/D converter (ADC) for analog-to-digital conversion and then output through an interface circuit. For any image sensor in the image sensor array, it is now cleared at the beginning of the exposure, and then the signal value is read after waiting for the exposure time. CMOS imaging devices typically employ rolling shutter imaging. In CMOS imaging devices, data readout is serial, so clear/exposure/readout can only be done line-by-line in a pipeline-like manner and will be processed after all rows of the image sensor array have been processed. It is synthesized into one frame of image. Thus, the entire CMOS image sensor array is actually progressively exposed (in some cases CMOS image sensor arrays may also be exposed in multiple lines at a time), which results in small delays between rows. Due to this small delay, when the light source flashes at a certain frequency, some undesired streaks appear on the image taken by the CMOS imaging device, which affects the shooting effect.

It has been found that it is theoretically possible to use the fringes on the image taken by the CMOS imaging device to convey information (similar to a bar code) and to try to pass as much information as possible through the stripes, but this usually requires the CMOS imaging device and the light source to be as far as possible Close, and preferably always at a roughly fixed distance, and also requires fine time synchronization, precise identification of the boundaries of individual stripes, accurate detection of the width of each stripe, etc., thus, in practice its stability and Reliability is not satisfactory and is not widely used.

Summary of the invention

One aspect of the invention relates to a method for decoding information conveyed by a light source, the light source being configured to operate in at least two modes, wherein in a different mode, when the image is passed through a rolling shutter The image of the light source obtained when the light source is photographed presents a different predetermined appearance to represent different data, and the method includes:

Continuously photographing the light source by a rolling shutter imaging device;

Obtaining a plurality of consecutive frames of the light source;

Extracting a start portion and an end portion of the image for each frame image of the light source;

The extracted initial portion and the ending portion are analyzed to determine data represented by each of the initial portion and the ending portion;

A data removal operation is performed on the data represented by the end portion of each frame image and the data represented by the beginning portion of the next frame image immediately thereafter to determine the data sequence delivered by the light source.

Advantageously, wherein said starting portion is part of an earliest image of each frame image of said light source, said end portion being part of a latest image of each frame image of said light source.

Preferably, wherein the lengths of the start portion and the end portion are both less than or equal to 1/2 of the total length of each frame image of the light source.

Preferably, wherein the appearance relates to a pattern, a color, or a combination thereof.

Preferably, wherein the different predetermined appearance comprises at least one predetermined stripe.

Preferably, wherein the analyzing the extracted initial portion and the ending portion to determine data represented by each of the initial portion and the ending portion comprises:

Analyze the appearance of each start and end; and

Determining data represented by each of the start portion and the end portion based on the appearance, wherein if a certain start portion or end portion includes a combination of different predetermined appearances, the data represented by the start portion or the end portion is considered to be "uncertain".

Preferably, wherein performing the data removal operation on the data represented by the end portion of each frame image and the data indicated by the beginning portion of the next frame image immediately thereafter includes:

Retaining one of the data represented by the end portion of each frame of image is the same as the data represented by the beginning portion of the next frame image immediately following;

If one of the data represented by the end portion of each frame image and the data indicated by the beginning portion of the next frame image is "unsure", another data is retained.

Preferably, the method further includes deleting the data if the data represented by the beginning portion of the first frame image of the light source or the end portion of the last frame image of the light source is "unsure"; otherwise, retaining The data.

Preferably, wherein the frame rate of the rolling shutter imaging device is equal to the signal frequency of the light source.

Preferably, wherein the direction of the scanning line of the rolling shutter imaging device is substantially perpendicular to the length direction of the light source.

Another aspect of the invention relates to an apparatus for decoding information transmitted by a light source, comprising a rolling shutter imaging device, a processor and a memory, wherein the memory stores a computer program, the computer program being processed The device can be used to implement the above method when executed.

Another aspect of the invention relates to a storage medium in which is stored a computer program that, when executed, can be used to implement the method described above.

Another aspect of the invention is an apparatus for decoding information conveyed by a light source, the light source being configured to operate in at least two modes, wherein in a different mode, when the image is captured by a rolling shutter The image of the light source obtained when the light source is photographed presents a different predetermined appearance to represent different data, and the apparatus includes:

a module for continuously photographing the light source by a rolling shutter imaging device;

a module for acquiring a plurality of consecutive frames of the light source;

a module for extracting a start portion and an end portion of the image for each frame image of the light source;

a module for analyzing the extracted start and end portions to determine data represented by each of the start portion and the end portion;

A module for performing a data removal operation on the data represented by the end portion of each frame image and the data represented by the beginning portion of the next frame image to determine the data sequence delivered by the light source.

DRAWINGS

The embodiments of the present invention are further described below with reference to the accompanying drawings, in which:

1 is a schematic view of a CMOS imaging device;

2 is a pattern of an image acquired by a CMOS imaging device;

Figure 3 is a light source in accordance with one embodiment of the present invention;

Figure 4 is a light source in accordance with another embodiment of the present invention;

5 is an imaging timing chart of a CMOS imaging device;

6 is another imaging timing diagram of a CMOS imaging device;

Figure 7 shows an image of the CMOS imaging device at different stages when the light source is operating in the first mode;

8 illustrates an imaging timing diagram of a CMOS imaging device when the light source operates in the first mode, in accordance with an embodiment of the present invention;

9 illustrates an imaging timing diagram of a CMOS imaging device when the light source operates in the second mode, in accordance with an embodiment of the present invention;

10 illustrates an imaging timing diagram of a CMOS imaging device when a light source operates in a first mode in accordance with another embodiment of the present invention;

11 shows an imaging timing diagram of a CMOS imaging device for implementing a stripe different from that of FIG. 8 in accordance with another embodiment of the present invention;

Figures 12-13 show two striped images of a light source obtained at different settings;

Figure 14 shows a streak-free image of the obtained light source;

Figure 15 shows a five-frame continuous image of the light source obtained with the signal frequency of the light source being equal to the frame rate of the CMOS imaging device;

One embodiment of extracting the beginning portion and the ending portion of each frame image shown in FIG. 15 is shown in FIG. 16;

Another embodiment of extracting the beginning portion and the ending portion of each frame image shown in Fig. 15 is shown in Fig. 17.

Detailed ways

The present invention will be further described in detail below with reference to the accompanying drawings.

One embodiment of the present invention is directed to an optical communication device capable of transmitting different information by emitting different lights. The optical communication device is also referred to herein as a "light tag" and both are used interchangeably throughout this application. The optical communication device includes a light source and a controller capable of controlling the light source to operate in two or more modes by a light source control signal, the two or more modes including the first mode and the second mode, Wherein, in the first mode, the light source control signal has a first frequency such that an attribute of light emitted by the light source changes at a first frequency to deliver first information, and in the second mode, the light source The attribute of the emitted light changes or does not change at the second frequency to deliver second information different from the first information.

The attribute of light in this application refers to any property that the CMOS imaging device can recognize, for example, it may be an attribute that the human eye can perceive, such as the intensity, color, and wavelength of light, or other attributes that are not perceptible to the human eye. For example, the intensity, color or wavelength of the electromagnetic wavelength outside the visible range of the human eye changes, or any combination of the above properties. Thus, a change in the properties of light can be a single property change, or a combination of two or more properties can change. When the intensity of the light is selected as an attribute, it can be achieved simply by selecting to turn the light source on or off. In the following, for the sake of simplicity, the light source is turned on or off to change the properties of the light, but those skilled in the art will appreciate that other ways to change the properties of the light are also possible. It should be noted that the attribute of the light varying at the first frequency in the first mode may be the same as or different from the attribute of the light changing at the second frequency in the second mode. Preferably, the properties of the light that change in the first mode and the second mode are the same.

When the light source operates in the first mode or the second mode, the light source can be imaged using a CMOS imaging device or a device having a CMOS imaging device (eg, a cell phone, a tablet, smart glasses, etc.), that is, by rolling the shutter Imaging. Hereinafter, a mobile phone as a CMOS imaging device will be described as an example, as shown in FIG. 2 . The line scanning direction of the mobile phone is shown as a vertical direction in FIG. 2, but those skilled in the art can understand that the line scanning direction can also be a horizontal direction depending on the underlying hardware configuration. In addition, it will be appreciated that the solution of the present invention is applicable to any imaging device that employs a rolling shutter imaging method.

The light source can be a light source of various forms as long as one of its properties that can be perceived by the CMOS imaging device can be varied at different frequencies. For example, the light source may be an LED light, an array of a plurality of LED lights, a display screen or a part thereof, and even an illuminated area of light (for example, an illuminated area of light on a wall) may also serve as a light source. The shape of the light source may be various shapes such as a circle, a square, a rectangle, a strip, an L, or the like. Various common optical devices can be included in the light source, such as a light guide plate, a soft plate, a diffuser, and the like. In a preferred embodiment, the light source may be a two-dimensional array of a plurality of LED lamps, one dimension of which is longer than the other dimension, preferably a ratio of between about 6-12:1. For example, the LED light array can be composed of a plurality of LED lamps arranged in a row. The LED light array can be rendered as a substantially rectangular light source when illuminated, and the operation of the light source is controlled by a controller.

Figure 3 illustrates a light source in accordance with one embodiment of the present invention. When imaging the light source shown in FIG. 3 using a CMOS imaging device, it is preferable to make the long side of the light source shown in FIG. 3 and the scanning line direction of the CMOS imaging device (for example, the line scanning direction of the mobile phone shown in FIG. 2) ) Vertical or substantially vertical to image as many stripes as possible under otherwise identical conditions. However, sometimes the user does not understand the line scanning direction of the mobile phone. In order to ensure that the mobile phone can recognize in various postures, and the maximum recognition distance can be achieved under both the vertical screen and the horizontal screen, the light source can be a plurality of rectangular shapes. Combination, for example, an L-shaped light source as shown in FIG.

In another embodiment, the light source may not be limited to a planar light source, but may be implemented as a stereoscopic light source, for example, a strip-shaped cylindrical light source, a cubic light source, or the like. The light source can be placed, for example, on a square, suspended at a substantially central location of an indoor venue (eg, a restaurant, a conference room, etc.) so that a nearby user in each direction can capture the light source through the mobile phone, thereby obtaining the light source. Information.

FIG. 5 shows an imaging timing diagram of a CMOS imaging device, each of which corresponds to a row of sensors of the CMOS imaging device. When imaging is performed on each line of the CMOS imaging sensor array, two stages are mainly involved, namely, exposure time and readout time. The exposure time of each line may overlap, but the readout time does not overlap.

It should be noted that only a small number of rows are schematically illustrated in FIG. 5, and in actual CMOS imaging devices, depending on the resolution, there are typically thousands of rows of sensors. For example, for 1080p resolution, it has 1920 x 1080 pixels, the number 1080 indicates 1080 scan lines, and the number 1920 indicates 1920 pixels per line. For 1080p resolution, the read time per line is approximately 8.7 microseconds (ie, 8.7 x 10 ^-6 seconds).

If the exposure time is too long, resulting in a large overlap of exposure time between adjacent lines, it may appear as a significant transitional fringe when imaging, for example, multiple strips between pure black pixel rows and pure white pixel rows have different grays. The pixel row of degrees. The present invention is expected to be able to present pixel lines as sharp as possible. For this reason, the exposure time of a CMOS imaging device (such as a mobile phone) can be set or adjusted (for example, set or adjusted by an APP installed on a mobile phone) to select a relative Short exposure time. In a preferred embodiment, the exposure time can be made approximately equal to or less than the readout time of each row. Taking the 1080p resolution as an example, the readout time of each line is approximately 8.7 microseconds. In this case, it is conceivable to adjust the exposure time of the mobile phone to about 8.7 microseconds or less. Fig. 6 shows an imaging timing chart of the CMOS imaging device in this case. In this case, the exposure time of each line does not substantially overlap, or the number of overlapping portions is small, so that stripes having relatively clear boundaries can be obtained at the time of imaging, which is more easily recognized. It should be noted that FIG. 6 is only a preferred embodiment of the present invention, and a longer (for example, twice or three times, four times or four times the readout time of each row, etc.) or a shorter exposure time is also feasible. of. For example, in the imaging process of the striped image shown in Figs. 12 and 13 of the present application, the readout time per line is approximately 8.7 microseconds, and the exposure time per line set is 14 microseconds. Further, in order to exhibit streaks, the length of one cycle of the light source may be set to about twice or more of the exposure time, and preferably may be set to about four times or more of the exposure time.

Figure 7 is a view showing an image of a CMOS imaging device at different stages when the light source is operated in the first mode using a controller, in which the property of the light emitted by the light source is changed at a certain frequency, in this example Medium to turn the light source on and off.

The upper part of Fig. 7 shows a state change diagram of the light source at different stages, and the lower part shows an image of the light source on the CMOS imaging device at different stages, wherein the row direction of the CMOS imaging device is vertical and from the left Scan to the right. Since the image captured by the CMOS imaging device is progressively scanned, when the high-frequency flicker signal is captured, the portion of the obtained image on the image corresponding to the imaging position of the light source forms a stripe as shown in the lower part of FIG. 7, specifically, In time period 1, the light source is turned on, in which the scanning line of the leftmost portion of the exposure exhibits bright streaks; in time period 2, the light source is turned off, in which the scanned lines of the exposure exhibit dark stripes; in time period 3, the light source is turned on, The scanned lines exposed during this time period exhibit bright streaks; in time period 4, the light source is turned off, during which the scanned lines of exposure exhibit dark stripes.

The frequency of the flashing of the light source can be set by the light source control signal, or the length of the light strip can be adjusted each time the light source is turned on and off, and the longer opening or closing time generally corresponds to a wider stripe. For example, for the case shown in FIG. 6, if the light source is turned on and off each time, the exposure time is set to be substantially equal to the exposure time of each line of the CMOS imaging device (this exposure time can be set by the APP installed on the mobile phone or manually set. ), it is possible to present stripes with a width of only one pixel when imaging. In order to enable long-distance identification of optical tags, the narrower the stripes, the better. However, in practice, the stripe having a width of only one pixel may be less stable or less recognizable due to light interference, synchronization, etc., therefore, in order to improve the stability of recognition, it is preferable to realize a stripe having a width of two pixels. . For example, for the case shown in FIG. 6, the stripe having a width of about two pixels can be realized by setting the duration of each turn on or off of the light source to be approximately equal to about twice the exposure time of each line of the CMOS imaging device. Specifically, as shown in FIG. 8, wherein the signal of the upper portion of FIG. 8 is a light source control signal, the high level corresponds to the turn-on of the light source, and the low level corresponds to the turn-off of the light source. In the embodiment shown in FIG. 8, the duty ratio of the light source control signal is set to about 50%, and the exposure duration of each line is set to be substantially equal to the readout time of each line, but those skilled in the art can understand that other Settings are also possible, as long as they can show distinguishable stripes. For simplicity of description, the synchronization between the light source and the CMOS imaging device is used in FIG. 8 such that the time of turning on and off of the light source substantially corresponds to the start or end time of the exposure time of a certain line of the CMOS imaging device, but the field The skilled person will understand that even if the two are not synchronized as shown in Fig. 8, they can exhibit significant streaks on the CMOS imaging device. At this time, there may be some transition stripes, but there must be a line of exposure when the light source is always off ( That is, the darkest stripe) is the line that is exposed when the light source is always on (ie, the brightest stripe), which is separated by one pixel. The light and dark variations (i.e., fringes) of such pixel rows can be easily detected (e.g., by comparing the brightness or grayscale of some of the pixels in the imaged area of the source). Furthermore, even if there is no line (ie, the darkest stripe) that is exposed when the light source is always off and a line that is exposed when the light source is always on (ie, the brightest stripe), if there is a light-on portion t1 less than a certain time during the exposure time a line having a small proportion of the length of the entire exposure time (ie, a darker stripe), and a line in which the light source opening portion t2 is greater than a certain length of time or a proportion of the entire exposure time (ie, a brighter stripe) during the exposure time, and T2-t1> light and dark stripe difference threshold (for example, 10 microseconds), or t2/t1> light and dark stripe ratio threshold (for example, 2), and the brightness change between these pixel lines can also be detected. The light and dark stripe difference threshold and the ratio threshold are related to the optical label illumination intensity, the photosensitive device property, the shooting distance, and the like. Those skilled in the art will appreciate that other thresholds are also possible as long as computer-resolvable stripes are present. When the streaks are identified, the information conveyed by the light source at this time, such as binary data 0 or data 1, can be determined.

The stripe recognition method according to an embodiment of the present invention is as follows: obtaining an image of the optical label, and dividing the imaging area of the light source by means of projection; collecting stripe in different configurations (for example, different distances, different light source flicker frequencies, etc.) Images and unstripe pictures; normalize all collected pictures to a specific size, such as 64*16 pixels; extract each pixel feature as input feature, build a machine learning classifier; perform two-class discrimination to determine a striped picture Still a non-striped picture. For stripe recognition, one of ordinary skill in the art can also process by any other method known in the art, which will not be described in detail.

For a strip light source with a length of 5 cm, when using a mobile phone that is currently on the market, setting the resolution to 1080p, when shooting 10 meters away (that is, the distance is 200 times the length of the light source), The strip light source occupies about 6 pixels in its length direction, and if each stripe width is 2 pixels, it will appear in a range of widths of a plurality of apparent pixels within the width of the 6 pixels. At least one distinct stripe that can be easily identified. If a higher resolution is set, or optical zoom is used, the stripe can be recognized at a greater distance, for example, when the distance is 300 or 400 times the length of the light source.

The controller can also operate the light source in the second mode. In one embodiment, in the second mode, the light source control signal can have another frequency than the first mode to change the properties of the light emitted by the light source, such as turning the light source on and off. In one embodiment, the controller can increase the turn-on and turn-off frequencies of the light source in the second mode compared to the first mode. For example, the frequency of the first mode may be greater than or equal to 8000 times/second, and the frequency of the second mode may be greater than the frequency of the first mode. For the situation illustrated in Figure 6, the light source can be configured to turn the light source on and off at least once during the exposure time of each row of the CMOS imaging device. FIG. 9 shows a case where the light source is turned on and off only once during the exposure time of each line, wherein the signal of the upper portion of FIG. 9 is a light source control signal whose high level corresponds to the turn-on of the light source, and the low level corresponds to the light source. The light source is turned off. Since the light source is turned on and off in the same way during the exposure time of each line, the exposure intensity energy obtained at each exposure time is roughly equal, so there is no significant difference in brightness between the individual pixel rows of the final image of the light source. So there are no stripes. Those skilled in the art will appreciate that higher turn-on and turn-off frequencies are also possible. In addition, for simplicity of description, synchronization between the light source and the CMOS imaging device is used in FIG. 9 such that the turn-on time of the light source substantially corresponds to the start time of the exposure time of a certain line of the CMOS imaging device, but those skilled in the art can It is understood that even if the two are not synchronized as in Fig. 9, there is no significant difference in brightness between the respective pixel rows of the final image of the light source, so that no streaks exist. When the streaks are not recognized, the information conveyed by the light source at this time, such as binary data 1 or data 0, can be determined. For the human eye, the light source of the present invention does not perceive any flicker when operating in the first mode or the second mode described above. In addition, in order to avoid the flicker phenomenon that may be perceived by the human eye when switching between the first mode and the second mode, the duty ratios of the first mode and the second mode may be set to be substantially equal, thereby realizing in different modes. The roughly the same luminous flux.

In another embodiment, in the second mode, direct current may be supplied to the light source such that the light source emits light whose properties do not substantially change, thereby obtaining one of the light sources obtained when the light source is photographed by the CMOS image sensor. No streaks appear on the frame image. In addition, in this case, substantially the same luminous flux in different modes can also be achieved to avoid flicker that may be perceived by the human eye when switching between the first mode and the second mode.

Figure 8 above describes an embodiment in which stripes are rendered by varying the intensity of the light emitted by the source (e.g., by turning the light source on or off). In another embodiment, as shown in Figure 10, The wavelength or color of the light emitted by the light source changes to present stripes. In the embodiment shown in Fig. 10, the light source includes a red light that emits red light and a blue light that emits blue light. The two signals in the upper portion of FIG. 10 are a red light control signal and a blue light control signal, respectively, wherein a high level corresponds to the turn-on of the corresponding light source and a low level corresponds to the turn-off of the corresponding light source. The red light control signal and the blue light control signal are phase shifted by 180°, that is, the two levels are opposite. The red light control signal and the blue light control signal enable the light source to alternately emit red light and blue light outward, so that when the light source is imaged by the CMOS imaging device, red and blue stripes can be presented.

By determining whether or not there is a streak on a portion of the image of one frame taken by the CMOS imaging device corresponding to the light source, information transmitted by each frame of the image, such as binary data 1 or data 0, can be determined. Further, by taking a continuous multi-frame image of the light source by the CMOS imaging device, a sequence of information composed of binary data 1 and 0 can be determined, and information transmission of the light source to the CMOS imaging device (for example, a mobile phone) is realized. In one embodiment, when a continuous multi-frame image of the light source is captured by the CMOS imaging device, control may be performed by the controller such that the switching time interval between the operating modes of the light source is equal to the length of time that a complete frame of the CMOS imaging device is imaged Thereby, the frame synchronization of the light source and the imaging device is realized, that is, information of 1 bit is transmitted per frame. For a shooting speed of 30 frames per second, 30 bits of information can be transmitted per second, and the encoding space reaches 2 ^30. The information can include, for example, a start frame mark (frame header), an optical tag ID, a password, a verification code. , URL information, address information, timestamps or different combinations thereof, and so on. The order relationship of the above various kinds of information can be set in accordance with a structuring method to form a packet structure. Each time a complete packet structure is received, it is considered to obtain a complete set of data (a packet), which can be read and verified. The following table shows the packet structure in accordance with one embodiment of the present invention:

Frame header

Attribute (8bit)

Data bit (32bit)

Check digit (8bit)

End of frame

In the above description, the information conveyed by the frame image is determined by judging whether or not there is a streak at the imaging position of the light source in each frame of image. In other embodiments, different information conveyed by the frame image may be determined by identifying different fringes at the imaging location of the light source in each frame of image. For example, in the first mode, the property of the light emitted by the light source is controlled by the light source control signal having the first frequency to be changed at the first frequency so as to be present on the image of the light source obtained when the light source is photographed by the CMOS image sensor. a first stripe; in the second mode, the light source control signal having the second frequency is used to control the property of the light emitted by the light source to vary at a second frequency, thereby enabling the light source obtained when the light source is photographed by the CMOS image sensor A second stripe different from the first stripe is present on the image. The difference in stripes may be based, for example, on different widths, colors, brightnesses, etc., or any combination thereof, as long as the difference can be identified.

In one embodiment, stripes of different widths may be implemented based on different light source control signal frequencies. For example, in the first mode, the light source may operate as shown in FIG. 8 to achieve a width of approximately two pixels. a stripe; in the second mode, the durations of the high level and the low level in each period of the light source control signal in FIG. 8 can be respectively changed to twice the original, as shown in FIG. A second stripe with a width of approximately four pixels is implemented.

In another embodiment, stripes of different colors may be implemented. For example, the light source may be set to include a red light that emits red light and a blue light that emits blue light. In the first mode, the blue light may be turned off. And the red lamp works as shown in Fig. 8 to realize red and black stripes; in the second mode, the red lamp can be turned off, and the blue lamp works as shown in Fig. 8, thereby realizing blue and black stripes. . In the above embodiment, the red and black stripes and the blue and black stripes are realized using the light source control signals having the same frequency in the first mode and the second mode, but it is understood that the first mode and the second mode may also be used. Light source control signals at different frequencies.

In addition, those skilled in the art can understand that more than two kinds of information can be further represented by implementing more than two kinds of stripes. For example, in an embodiment including the red light and the blue light in the above light source, the third mode can be further set. In this third mode, the red and blue lights are controlled in the manner shown in Figure 10 to achieve a red-blue stripe, a third type of information. Obviously, alternatively, another type of information, that is, the fourth type of information, can be further transmitted through the fourth mode in which the stripes are not presented. A plurality of the above four modes may be arbitrarily selected for information transfer, and other modes may be further combined as long as different patterns generate different stripe patterns (including a stripe-free pattern).

Figure 12 shows the use of 1080p resolution imaging for LEDs that are flashing at a frequency of 16,000 times per second (each period has a duration of 62.5 microseconds with an on duration and a closure duration of approximately 31.25 microseconds each) The stripe on the image obtained by the experiment, with the exposure time of each line set to 14 microseconds. As can be seen from Figure 12, stripes of approximately 2-3 pixel width are presented. Figure 13 shows that the blinking frequency of the LED lamp in Figure 12 is adjusted to 8000 times per second (the duration of each cycle is 125 microseconds, wherein the opening duration and the closing duration are each about 62.5 microseconds), under other conditions. Streaks on the image obtained by experiment under constant conditions. As can be seen from Figure 13, a stripe of approximately 5-6 pixel width is presented. Figure 14 shows the adjustment of the blinking frequency of the LED lamp of Figure 12 to 64,000 times per second (the duration of each cycle is 15.6 microseconds, wherein the opening duration and the closing duration are each about 7.8 microseconds), under other conditions. The image obtained by experiment without change has no streaks on it, because the length of each line of exposure is 14 microseconds, which basically covers one opening time and one closing time of the LED lamp.

In the above, for convenience of description, the square wave is taken as an example to describe the light source control signal having the corresponding frequency, but those skilled in the art can understand that the light source control signal can also use other waveforms such as a sine wave, a triangular wave or the like.

The use of one light source is described above, and in some embodiments, two or more light sources may also be employed. The controller can independently control the operation of each light source.

In one embodiment, the optical tag may further include one or more positioning indicia located adjacent the information delivery source, the positioning indicia being, for example, a lamp of a particular shape or color, the lamp being, for example, capable of remaining constantly illuminated during operation. The location identification can help a user of a CMOS imaging device, such as a cell phone, to easily discover optical tags. In addition, when the CMOS imaging device is set to a mode in which the optical tag is photographed, the imaging of the positioning mark is conspicuous and easy to recognize. Thus, the one or more location markers disposed adjacent the information transfer light source can also assist the handset in quickly determining the location of the information transfer light source to facilitate identifying whether the imaged region corresponding to the information transfer light source has streaks. In one embodiment, in identifying whether there are streaks, the location identification may first be identified in the image such that the approximate location of the optical tag is found in the image. After the location identification is identified, one or more regions in the image may be determined based on the relative positional relationship between the location identification and the information delivery light source, the region encompassing an imaging location of the information delivery light source. These areas can then be identified to determine if there are streaks or what stripes are present.

Compared with the recognition distance of about 15 times of the two-dimensional code in the prior art, the identification distance of at least 200 times of the optical label of the present invention has obvious advantages. The long-distance recognition capability is especially suitable for outdoor recognition. Taking a recognition distance of 200 times as an example, for a light source with a length of 50 cm set on a street, a person within 100 meters from the light source can pass the mobile phone and the light source. Interact. In addition, the solution of the present invention does not require the CMOS imaging device to be located at a fixed distance from the optical tag, nor does it require time synchronization between the CMOS imaging device and the optical tag, and does not require accurate detection of the boundaries and widths of the individual stripes. Therefore, it has extremely strong stability and reliability in actual information transmission.

When the optical tag is used to transmit the data information, the light source in the optical tag can be controlled to switch between different working modes based on the information to be transmitted at a certain signal frequency, thereby continuously transmitting different information. The signal frequency refers to the number of times the light source of the optical tag transmits data per second. For example, a light source of an optical tag can be configured to pass data at a signal frequency of 30 times/second. In the case of a signal frequency of 30 times/second, for the case where binary data 1 is delivered in the first mode and binary data 0 is delivered in the second mode, the controller in the optical tag can determine based on the sequence of binary data to be transmitted. Which mode the light source should operate in each time slice of 1/30 second. For example, for the binary data sequence 110101, the light source in the optical tag can be configured to sequentially operate in the first mode, the first mode, the second mode, and the first mode in six consecutive time slices of length 1/30 second. , the second mode, the first mode.

In the case where the CMOS imaging device is not synchronized with the light source, when the CMOS imaging device is used for image acquisition of the light source, it may happen that the mode switching of the light source occurs when the light source is imaged, which causes the image at the light source The splicing of two stripe patterns (including a stripe-free pattern) is presented. This situation is very unlikely to occur when the CMOS imaging device is far from the source. For example, when a light source in an optical tag transmits data at a predetermined signal frequency of 30 times/second, each data transfer of the light source has a duration of 1/30 second, that is, approximately 33,000 microseconds. When the CMOS imaging device is far from the light source, the length direction of the image of the captured light source will occupy only about a few tens of pixel rows. As described earlier, the exposure time of each line of the CMOS imaging device is about several microseconds (for example, 14 microseconds), and there may be an overlap between exposure times of adjacent lines, so the total of these tens of pixel rows The exposure time is approximately a few hundred microseconds. Compared to the light source, which lasts about 33,000 microseconds of information transfer in each mode, the probability that the total exposure time falls just at the mode switching moment of the light source is very small. When the CMOS imaging device is close to the light source, for example, when the length direction of the image of the captured light source occupies about thousands of pixel rows, since the total exposure time of thousands of pixel rows is relatively long, the total The probability that the exposure time falls just at the mode switching moment of the light source will be significantly improved. In order to solve the above problem, the sampling frequency (that is, the frame rate) of the CMOS imaging device (for example, a mobile phone) can be made to be greater than or equal to twice the signal frequency, for example, if the light source transmits data at a signal frequency of 30 times/second, The CMOS imaging device can be configured to take a picture at a frame rate of at least 60 frames per second.

However, in some cases, in order to increase the signal frequency of the light source or to reduce the frame rate requirement for the CMOS imaging device, it is desirable to enable data transfer even when the signal frequency of the light source is equal to the frame rate of the CMOS imaging device. An implementation method is described below.

Figure 15 shows a five-frame continuous image of the light source obtained with the signal frequency of the light source equal to the frame rate of the CMOS imaging device. The image of the light source can be obtained by continuously photographing the optical tag by the CMOS imaging device and extracting an image of the light source from each frame of the captured image. In the embodiment shown in FIG. 15, the white portion of the image of the light source represents a first type of fringe pattern corresponding to the first mode of the light source for transmitting binary data 1, in the image of the light source The gray portion represents a second stripe pattern corresponding to the second mode of the light source for delivering binary data 0. As can be seen from Fig. 15, two stripe patterns appear in the second frame, the third frame, and the fifth frame image of the light source, which means that the light source operating mode happens when the light source is imaged in these frames. Switching. Switching of the operating mode of the light source does not occur during imaging of the first and fourth frame images of the light source, but it will be appreciated that the imaging period of the first and fourth frame images actually involves the light source in two consecutive time slices The two data passes are only the working mode of the light source or the data transmitted in the two consecutive time slices are the same (in this embodiment, both are the first mode). It will be appreciated that the five consecutive frames in Figure 15 actually relate to six consecutive time slices of the source during information transfer, or six consecutive modes of operation. And it can be understood that four complete time slices (ie, second, third, fourth, and fifth time slices) are covered from the imaging start time of the first frame image of the light source to the imaging end time of the fifth frame image, because The imaging start time of one frame of image is located during the first time slice, and the imaging end time of the fifth frame image is located during the sixth time slice.

In order to decode the working mode sequence or the data sequence of the multi-frame image representation of the light source, a decoding method is proposed, which includes:

The optical tag containing the light source is continuously photographed by a CMOS imaging device. By this continuous shooting, a series of continuous images containing the light source can be obtained.

Get several consecutive frames of the light source. Several frames of continuous images of the light source can be obtained using various methods known in the art, for example, the imaging position of the light source can be determined from a series of successive images of the captured light source, thereby extracting several consecutive frames of the light source .

For each frame of the light source, the beginning and end portions of the image are extracted separately. Because the CMOS imaging device is progressively scanned, each frame of the source is not imaged simultaneously, but in a prioritized order. The beginning of the image refers to the portion of the earliest image of the image, and the end portion of the image refers to the portion of the image that is the latest image. Each CMOS imaging device has an inherent scanning mode that can be known in advance. In the embodiment shown in FIG. 15 and the following description, it is assumed that the scanning line direction of the CMOS imaging device is a vertical direction, and scanning is started from the left side line. Therefore, in the embodiment shown in Fig. 15, the portion located at the left end of the image is the starting portion, and the portion located at the right end of the image is the ending portion. The lengths of the start portion and the end portion are selected to be less than or equal to 1/2 of the total length of each frame image of the light source to ensure that there is no combination of stripe patterns in at least one of the start portion and the end portion. The lengths of the starting portion and the ending portion are preferably equal, but not necessarily equal.

The extracted start and end sections are analyzed to determine the data represented by each of the start and end sections. For example, for the embodiment shown in FIG. 15, if there is only the first stripe pattern in the start portion or the end portion, it can be determined that it represents binary data 1; if there is only the second stripe in the start portion or the end portion The pattern can be determined to represent binary data 0. If there are two stripe patterns in a certain starting part or ending part, the data represented by it cannot be determined, which can be referred to as "unsure" here, and is indicated by "*" hereinafter.

A data removal operation is performed on the data represented by the end portion of each frame image and the data indicated by the beginning portion of the next frame image to determine the data sequence delivered by the light source. The data removal operation may, for example, include the rule that if the two data are the same, one of them is retained; if one of the data is "unsure", another data is retained.

One embodiment of extracting the start portion and the end portion of each frame image shown in Fig. 15 is shown in Fig. 16. In this embodiment, the image on the left side of the broken line A is extracted as the start portion, and the image on the right side of the broken line B is extracted as the end portion, wherein the lengths of the start portion and the end portion are equal, and each of the images of the light source is 1/4 of the total length. Thus, ten initial parts and ending parts are obtained. In this embodiment, there is no combination of stripe patterns in each of the starting portion or the ending portion. By analyzing the ten initial and ending parts, the corresponding data sequence represented by the ten initial parts and the ending part can be determined: 1, 1, 1, 0, 0, 1, 1, 1, 1, 0. Taking the data represented by each end portion and the data indicated by the immediately preceding part as a group, four sets of data are obtained, which are (1, 1), (0, 0), (1, 1), ( 1, 1). By performing a data removal operation on each of the sets of data, a data sequence can be obtained: 1, 0, 1, 1, which is the end of imaging from the imaging start time of the first frame image of the light source to the fifth frame image. The data transmitted by the four complete time slices (ie, the second, third, fourth, and fifth time slices) covered at the moment.

Another embodiment of extracting the start portion and the end portion of each frame image shown in Fig. 15 is shown in Fig. 17. In the embodiment of Fig. 17, the image on the left side of the broken line A is still extracted as the start portion, and the image on the right side of the broken line B is extracted as the end portion, but different from the manner shown in Fig. 16, the start portion and the end The length of the portion is longer and thus there is a combination of stripe patterns in some of the starting portions. By analyzing the ten initial and ending parts, the corresponding data sequence represented by the ten initial parts and the ending part can be determined: 1, 1, *, 0, *, 1, 1, 1, *, 0. Taking the data represented by each end portion and the data indicated by the immediately preceding part as a group, four sets of data are obtained, which are (1, *), (0, *), (1, 1), ( 1,*). By performing a data removal operation on each of the sets of data, the same data sequence can be obtained: 1, 0, 1, 1, which is the image from the imaging start time of the first frame image of the light source to the fifth frame image. The data transmitted by the four complete time slices (ie, the second, third, fourth, and fifth time slices) covered by the end of imaging.

It will be understood by those skilled in the art that for the embodiment shown in FIG. 16 and FIG. 17, the data transmitted by the first time slice and the sixth time slice can be further determined, because the initial portion of the first frame image represents Data 1, the end portion of the fifth frame image represents data 0, regardless of the end portion of the image of one frame before the image of the first frame and the beginning portion of the image of the sixth frame, the data is retained after the data removal operation is performed. And 0, that is, the data transmitted by the first time slice and the sixth time slice are data 1 and 0, respectively. If the data of the beginning portion of the first frame image or the end portion of the fifth frame image is judged as "unsure" in the previous step, it can be simply deleted. Therefore, the data removal rule for the start portion of the first frame image and the end portion of the last frame image may be: if the data it represents is "unsure", the data is deleted; otherwise, the data is retained.

The optical tag of the present invention is used as an example to describe a method for decoding information transmitted by a light source in an optical tag. However, those skilled in the art can understand that the above decoding method is not limited to the optical tag of the present invention, but may be Suitable for any light source configured to be capable of operating in at least two modes, wherein in different modes, different images are presented on the image of the light source obtained when the light source is photographed by the CMOS image sensor Exterior. This appearance may, for example, relate to a pattern, a color, or a combination thereof. For example, for a light source that simply transmits information by emitting different colors of light in different modes, when the light source is photographed by a CMOS image sensor, images of different colors of the light source are obtained in different modes of the light source. . It can be understood that when the light source is photographed by the CMOS image sensor, it is also possible that a mode switching occurs just when one frame of the light source is taken, so that a color jump occurs on one frame of the light source. Obviously, the above described decoding method of the present invention can be applied to the decoding of the light source.

References in the specification to "individual embodiments," "some embodiments," "one embodiment," or "an embodiment" or "an" In at least one embodiment. Thus, appearances of the phrases "in the various embodiments", "in some embodiments", "in one embodiment", or "in an embodiment" example. Furthermore, the particular features, structures, or properties may be combined in any suitable manner in one or more embodiments. Thus, the particular features, structures, or properties shown or described in connection with one embodiment may be combined, in whole or in part, with the features, structures, or properties of one or more other embodiments without limitation, as long as the combination is not Logical or not working. In addition, the various elements in the drawings of the present application are only for the purpose of illustration and illustration.

Having thus described several aspects of at least one embodiment of the present invention, it is understood that various changes, modifications and improvements can be readily made by those skilled in the art. Such changes, modifications, and improvements are intended to be within the spirit and scope of the invention.

Claims

A method for decoding information conveyed by a light source, the light source being configured to operate in at least two modes, wherein, in a different mode, when the light source is photographed by a rolling shutter imaging device The image of the light source exhibits a different predetermined appearance to represent different data, the method comprising:

Continuously photographing the light source by a rolling shutter imaging device;

Obtaining a plurality of consecutive frames of the light source;

Extracting a start portion and an end portion of the image for each frame image of the light source;

The extracted initial portion and the ending portion are analyzed to determine data represented by each of the initial portion and the ending portion;

A data removal operation is performed on the data represented by the end portion of each frame image and the data represented by the beginning portion of the next frame image immediately thereafter to determine the data sequence delivered by the light source.
The method of claim 1 wherein said starting portion is part of an earliest image of each frame image of said light source, said end portion being part of a latest image of each frame image of said light source .
The method of claim 1, wherein the length of the start portion and the end portion are both less than or equal to 1/2 of a total length of each frame image of the light source.
The method of claim 1 wherein the appearance relates to a pattern, a color, or a combination thereof.
The method of claim 1 wherein said different predetermined appearances comprise at least one predetermined stripe.
The method according to any one of claims 1 to 5, wherein the analyzing the extracted start portion and end portion to determine data represented by each of the start portion and the end portion comprises:

Analyze the appearance of each start and end; and

Determining data represented by each of the start portion and the end portion based on the appearance, wherein if a certain start portion or end portion includes a combination of different predetermined appearances, the data represented by the start portion or the end portion is considered to be "uncertain".
The method according to claim 6, wherein said performing data removal operation on said data indicated by an end portion of each frame image and data indicated by a start portion of a next frame image immediately thereafter comprises:

Retaining one of the data represented by the end portion of each frame of image is the same as the data represented by the beginning portion of the next frame image immediately following;

If one of the data represented by the end portion of each frame image and the data indicated by the beginning portion of the next frame image is "unsure", another data is retained.
The method according to claim 6, further comprising deleting the data if the data indicated by the start portion of the first frame image of the light source or the end portion of the last frame image of the light source is "undefined" ; otherwise, retain the data.
The method according to any one of claims 1 to 5, wherein the frame rate of the rolling shutter imaging device is equal to the signal frequency of the light source.
The method according to any one of claims 1 to 5, wherein the direction of the scanning line of the rolling shutter imaging device is substantially perpendicular to the length direction of the light source.
An apparatus for decoding information conveyed by a light source, comprising a rolling shutter imaging device, a processor, and a memory, wherein the memory stores a computer program that can be used to implement when executed by the processor The method of any of claims 1-10.
A storage medium having stored therein a computer program, which when executed, can be used to implement the method of any of claims 1-10.
An apparatus for decoding information conveyed by a light source, the light source being configured to operate in at least two modes, wherein, in a different mode, when the light source is photographed by a rolling shutter imaging device The image of the light source exhibits a different predetermined appearance to represent different data, the device comprising:

a module for continuously photographing the light source by a rolling shutter imaging device;

a module for acquiring a plurality of consecutive frames of the light source;

a module for extracting a start portion and an end portion of the image for each frame image of the light source;

a module for analyzing the extracted start and end portions to determine data represented by each of the start portion and the end portion;

A module for performing a data removal operation on the data represented by the end portion of each frame image and the data represented by the beginning portion of the next frame image to determine the data sequence delivered by the light source.