TWI564849B - Real-time pedestrian countdown displayer - Google Patents

Description

Pedestrian countdown display instant recognition method

The invention relates to a method for instantly recognizing a pedestrian countdown display, in particular to a method for promptly recognizing a pedestrian countdown display, which is embedded in a hardware and can be integrated into a pedestrian warning auxiliary recognition system.

According to the schematic diagram of the pedestrian countdown display 10 shown in FIG. 1, it is divided into two upper and lower lamp faces, the upper part is a countdown countdown display 12 for the pedestrian light to be changed, and the lower part is an animation consisting of seven actions. The green man walks the animated face 11, when the countdown countdown shows 12 is over, the little green man walks the animation face 11 to express fast passage, which can remind the pedestrian to pay attention to the green light of the traffic sign light in the front is about to change into a red light state, so that the pedestrian can Knowing in advance that the light changes after the remaining seconds can avoid danger during walking.

For the second time, the elderly, the blind and the color blind are visually impaired among the pedestrians. The visually impaired person needs tools such as voice, sensor, pedestrian navigation, guide cane or guide dog to assist the guidance, vision and Walking, however, the visually impaired person cannot be effectively applied to the pedestrian countdown display 10 by the above-mentioned auxiliary tools, because the color characteristics, position distribution and contour of the small green man walking animation surface 11 and the countdown countdown display 12 are different. Therefore, it is more appropriate for a person with visual impairment to judge the color characteristics, position distribution and contour of the small green person walking the animated face 11 and the countdown countdown display 12, and to assist the visual obstacle to the mechanical vision.

However, it is still impossible to instantly recognize the pedestrian countdown display 10 by mechanical visually judging the color characteristics, position distribution and contour of the small green man walking animation face 11 and the countdown countdown display 12, as follows:

(1). The pedestrian countdown display 10 is located in the environment, except that the small green man walking animation face 11 and the countdown countdown display 12 are respectively green and orange, and the environment also has green or orange color and other colors, so that In the process of scanning the environment, it takes a lot of time to filter out the small green man to walk the animated face 11 and the countdown countdown display 12, so that the mechanical vision is not easy to instantly recognize the pedestrian countdown display 10.

(2). Countdown countdown display 12 varies by time to produce different numbers, each of the digital contours is different and the change is fast, and the countdown countdown display 12 tilt produces the inner angle, the inner angle needs to calculate the corrected angle, but it is easy to cause the number The distortion of the contour makes it difficult for mechanical vision to instantly recognize the changing state of the digital contour.

(3). In summary, the mechanical vision cannot instantly recognize the pedestrian countdown display 10, and it is not easy to expand to the mobile device. Therefore, the aforementioned mechanical vision is difficult to implement on the pedestrian countdown display 10, so that there is still room for improvement, and there is still room for improvement.

The main purpose of the present invention is to provide an instant recognition method for a pedestrian countdown display, wherein the color characteristics of the environment are continuously converted in different color spaces, and the color of the small green person walking animation surface and the countdown countdown display can be filtered out. Features, and scanning the color features in a sliding window, thereby providing an immediate enhancement in the effectiveness of the pedestrian countdown display.

Another object of the present invention is to provide a pedestrian pedestrian countdown display instant recognition method, which uses an inner distance shape context to recognize a digital contour displayed by a countdown countdown, and performs an oblique correction number to eliminate the inner angle before the inner distance shape context recognition And the digital contour and the digital template are matched by dynamic programming, and thus the effect of instant recognition of the countdown countdown display is enhanced.

Another object of the present invention is to provide an instant recognition method for a pedestrian countdown display, the method of which can be embedded in a mobile device integrated into a pedestrian warning auxiliary recognition system, the pedestrian alarm The auxiliary identification system cooperates with the cloud system to instantly recognize the enhancements that can be expanded in the warning and the cloud.

To achieve the above object, the method of the present invention comprises: a) inputting at least one frame, the frame being presented with a plurality of pixels, each of the pixels being converted into an HSV color space in an RGB color space; b) producing a highly saturated pixel Through the mask, the frame is rendered as a highly saturated pixel to form an unmasked image that is converted into a CIELAB color space in the RGB color space; c) a mask is formed through which a high-brightness pixel passes. The unmasked image presents a high-brightness pixel, and the unmasked image is narrowed down and the energy distribution is estimated, and the color statistical model of the image of the small green person and the color count model of the countdown countdown display image is established. The statistical model is input to the unmasked image, so that the frame has a plurality of orange signs and a plurality of green signs, wherein an orange sign is adjacent to one of the green marks to present an upper and lower relationship; d) providing a sliding window, the sliding The upper part of the window is set to match the orange mark as the search object, the lower part is set to match the green mark as the search object, and the frame is scanned by the sliding window; and e). The degree value is a degree of matching between the color statistic and the unmasked image in the sliding window after estimating the color statistic in the sliding window, and the matching degree sets a threshold value when the density value is greater than or equal to the The threshold value identifies the orange and green signs in the upper and lower relationship, which can be used to indicate the pedestrian countdown display.

According to the foregoing feature, wherein: c) the color statistical model training comprises: inputting a training image; masking low-saturation pixels and low-brightness pixels in the training image; recording the first and the first of the training images in the CIELab color space The color distribution position of each pixel of the second channel; the input of the next training image; and the storage of the final color statistical model.

According to the foregoing feature, the color adjustment of the first channel is between red and green and the color of the second channel is adjusted between yellow and blue, and the color adjustment range values of the first and second channels are all -109 ~110.

According to the foregoing feature, wherein: e) the pedestrian countdown display has a small green person walking animation face and a countdown countdown display, the countdown timer countdown display recognition comprises: detecting the position of the small green person walking the animation face; Cutting the countdown countdown display located above the small green man's animation surface; using the inner distance shape context to identify the digital outline displayed by the countdown countdown, the digital outline has a plurality of contour points, each of the contour points a shape context, each of the shape contexts being normalized into a uniform vector; a matching of the digital contour of the countdown countdown display and a blurred digital template produces a large minimum feature difference; the blur detection of the countdown countdown display number is utilized The weighed zero crossing is a horizontal scan line to analyze the one-dimensional difference around the number. The one-dimensional difference is obtained from the histogram and the sum of the statistical weights. The number is known according to the zero-crossing threshold. The degree of blur presented; and changing the number of the fuzzy digital template to match the countdown countdown display To the survival of the fittest.

According to the foregoing feature, the countdown countdown display located above the animation face of the little green man is cut: comprising: inputting an image, the image has a number; super-resolution enhancing the image; transforming two values in the image The enhanced grayscale renders the number; removes the noise of the digit; obliquely corrects the digit; normalizes the digit; and identifies the digit outline.

A pedestrian warning auxiliary recognition system, comprising: the above-mentioned pedestrian countdown display instant recognition method; and a mobile device comprising: an image input unit for capturing video, the video generating a continuous frame; a memory storage The unit is electrically connected to the image input unit for temporarily storing the continuous frame and the pedestrian countdown display instant recognition method; an operation processing unit is electrically connected to the memory storage unit for calculating the pedestrian countdown display The instant recognition method processes the continuous frame; and an output result unit is electrically connected to the memory storage unit for outputting the change state of the small green person walking animation surface and the countdown countdown display.

According to the foregoing features, the present invention may further include a cloud system through the network The road is combined with the pedestrian warning auxiliary identification system to enable the little green person to go to the cloud system by taking the change state of the animation face and the countdown timer countdown display.

By means of the above-mentioned technical means, the pedestrian recognizable display instant recognition method of the present invention, the frame is continuously converted in two color spaces, and the non-orange and green objects or backgrounds can be excluded, and the statistics of the small green people walking the animation surface and the countdown The color feature of the chronograph countdown display is converted into two marks, so that the sliding window can scan the two marks in real time, and the number displayed by the countdown countdown is super-analyzed, two values are changed, noise is removed, and the oblique correction is performed. After normalizing a series of processing, and identifying the digital contour by the inner distance shape context, the difference between the digital contour and the digital template is matched by dynamic programming, and the instant identification method can be implemented in the pedestrian warning auxiliary recognition system and the cloud system.

S1~S12‧‧‧Steps

S81~S86‧‧‧ steps

S111~S119‧‧‧Steps

S1121~S1128‧‧‧Steps

20‧‧‧ mobile devices

21‧‧‧Image input unit

22‧‧‧Memory storage unit

23‧‧‧Operation Processing Unit

24‧‧‧Output result unit

30‧‧‧Car-assisted identification system

40‧‧‧Cloud System

Figure 1 is a schematic diagram of a conventional pedestrian countdown display.

2A is a flow chart of the instant recognition of the pedestrian countdown display of the present invention.

2B is a schematic illustration of a frame of the present invention presenting an RGB color space.

2C is a schematic diagram showing the passage of a highly saturated pixel in the frame of the present invention.

2D is a schematic diagram showing the RGB color space of the unmasked image of the present invention.

2E is a schematic diagram showing the passage of a high-brightness pixel in an unmasked image of the present invention.

2F is a schematic diagram showing the green mark and the orange mark in the present invention.

2G is a schematic diagram of the present invention using sliding window detection.

2H is a schematic illustration of a pedestrian countdown display of the present invention.

Figure 3 is a flow chart of the color statistical model training of the present invention.

4A is a flow chart of the present invention identifying a countdown countdown display.

Figure 4B is a schematic illustration of the figures of the present invention.

Figure 4C is a schematic illustration of the identification of a digital outline by the present invention.

4D is a partial enlarged view of FIG. 4C.

4E is a schematic diagram of the simulated proximity recognition number of the present invention.

4F is a schematic diagram of the analog remote recognition number of the present invention.

Figure 5A is a flow chart of digital pre-processing of the present invention

Figure 5B is a schematic illustration of the unclear figures of the present invention.

Figure 5C is a schematic illustration of the numerical representation of the invention.

Figure 5D is a schematic illustration of the oblique correction number of the present invention.

Figure 6A is a schematic illustration of the pedestrian alert assisted identification system of the present invention.

6B is a schematic diagram of the pedestrian warning assisted identification system of the present invention for identifying a pedestrian countdown display.

6C is a schematic diagram of the pedestrian warning auxiliary recognition system of the present invention for recognizing a countdown countdown display.

First, the method for the instant recognition of the pedestrian countdown display of the present invention is preferably the steps of the preferred embodiment. Please refer to the flowchart of FIG. 2A, and cooperate with FIG. 2B~2H, FIG. 3, FIG. 4A~4D and FIG. 5A~5D, in one The movie uses the brightness information and saturation information for each frame.

As shown in FIG. 2A, step S1: beginning with the present invention, as shown in FIG. 2B to FIG. 2C, step S2: input at least one frame, the frame is presented with a plurality of pixels, and step S3: each pixel is in RGB color. The space is converted into the HSV color space, step S4: making a mask through which the high saturated pixel passes, so that the frame presents a high saturated pixel to form an unmasked image. In this embodiment, the frame generates high saturation via the HSV color space. The pixel is configured to set a high saturation threshold according to the high saturation pixel to form a saturation filter (first high pass filter), which can filter out each of the high saturation pixels in the frame.

As shown in FIG. 2D~2E, step S5: the unmasked image is converted in RGB color space. CIELAB color space, step S6: making a high-brightness pixel through the mask, so that the unmasked image is presented with high-brightness pixels, step S7: reducing the unmasked image and estimating the energy distribution, the implementation In an example, the frame generates a high-brightness pixel through the CIELAB color space, so that the mask sets a high-brightness threshold according to the high-brightness pixel to form a luminance filter (second high-pass filter), which can filter out the frame. Each of the high-brightness pixels, through the above-described frame, cascades the luminance filter via the saturation filter to instantly filter out desired color features.

2F, step S8: after establishing a small green person moving animated image and a color statistical model of the countdown countdown display image, the color statistical model is input to the unmasked image, so that the frame has a plurality of orange signs and plural The green mark, one of the orange signs is adjacent to one of the green marks, and the relationship between the upper part and the lower part is displayed. In this embodiment, as shown in FIG. 3, step S81: the color statistical model training starts, step S82: inputting a training image, Step S83: Blocking the low-saturation pixel and the low-brightness pixel in the training image, step S84: recording the color distribution position of the training image in each pixel of the first and second channels of the CIELab color space, the first channel The color adjustment is between red and green and the color of the second channel is adjusted between yellow and blue, and the color adjustment range values of the first and second channels are all -109~110, step S85: inputting the next training image If there is no next training image, the process proceeds to step S86: the last color statistical model is stored. Otherwise, if there is a next training image in step S85, the process goes to step S82.

As shown in FIG. 2G~2H, step S9: providing a sliding window, the upper part of the sliding window is set to match the orange color mark as the search object, and the lower part is set to match the green mark as the search object, and the frame is scanned by the sliding window. S10: Find a maximum density value, which is a degree of matching between the color statistics and the unmasked image in the sliding window after estimating the color statistics in the sliding window, and the matching degree sets a threshold value, when the The density value (the color statistic in the sliding window / the unmasked image in the sliding window) is greater than or equal to the threshold value, and the recognition is presented, The lower relationship orange, green mark, step S11: the pedestrian countdown display can be marked, step S12: end, otherwise, if the density value is not greater than or equal to the threshold value, step S12 is reached to step S12.

Through the above step S11: the pedestrian countdown display can be marked. In conjunction with the flowchart shown in FIG. 4A, the pedestrian countdown display has a small green person walking animation face and a countdown countdown display, and the countdown timer countdown display recognition includes Step S111: detecting the position of the small green person walking the animation surface, step S112: cutting out the countdown countdown display located above the small green person walking animation surface, as shown in FIG. 5A, comprising: step S1121: input An image having a number, the step S1122: super resolution (SR) is used to enhance the image, as shown in FIG. 5B and FIG. 5C, wherein the image is parsed and the low number is less clear, and the image is parsed after being enhanced. The high presentation number is clear, step S1123: transforming the two values in the image to enhance the gray scale to display the significant number, step S1124: removing the noise of the number, step S1125: obliquely correcting the number, as shown in FIG. 5D. The (a) picture shows the RGB image (img) as a number, (b) the picture is a converted RGB image into a binarized image (bw), and (c) the picture uses a saturated mask (sat masked) to cover the binarized image. To make the numbers around The message disappears, (d) the picture is the open binary image, the digital image is unclear, and the (e) picture is the feedback. The RGB image is compared with the binarized image to make the damage. The digital replies to the partial image, (f) the figure shows that the corrected number is not skewed to the correct angle (cor ang), and the above is the process of oblique correction of the numbers 1, 3, 8, 13 and 17, step S1146: normalize the Number, step S1127: Identify the digital outline, step S1128: result output, if displayed as a double digit, then step S113: cut into tens and single digits.

Further, in step S114, the inner contour shape context is used to identify the digital contour displayed by the countdown countdown, and the inner distance shape context is used as a shape descriptor, as described in Haibin Ling et al ., the inner-distance replaces the Euclidean distance to extend the The shape context is not described, and the present invention applies the inner distance shape context to identify the countdown countdown display, as shown in FIGS. 4A and 4B, which identifies a number to form a digital outline, and the digital outline has a plural number Contour points, and draw five contour points for comparison, as shown in Figure 4C, which are five (a), (b), (c), (d), and (e) The contour points each have a shape context, and each of the shape contexts is normalized into a uniform vector, and the number is corrected in accordance with the aforementioned skew, and the inner angle is not calculated to form an inner-distance shape context without inner-angle. , IDSC-IA).

In addition, in step S115, a large minimum difference between the digital contour and the digital template displayed by the countdown countdown is generated by dynamic programming (DP). If there is no large minimum feature difference, the step is entered. S116: displaying the result in the user interface, step S117: waiting for the next timing number, and step S114, identifying the digital contour displayed by the countdown countdown using the inner distance shape context.

Then, if there is a large minimum feature difference, the process proceeds to step S118: the number of the countdown countdown display is blurred, and as shown in FIG. 4E, (a) and (b) are simulated near and far distance recognition centers respectively. Producing a weighed zero crossing of 0.24833, 0.81342, which is a horizontal scan line that analyzes the one-dimensional difference around the number, which is presented in a histogram and the sum of the statistical weights. In this way, the degree of blurring of the number can be known according to the threshold of the zero-crossing rate.

Still further, in step S119, the digital template is changed to match the digital contour displayed by the countdown countdown to the weakened strong, as shown in FIG. 4F, wherein (a) is a digital template in the first channel of the CIELAB color space. Generating brightness into a gray level image, and then converting the gray level image into a binarized template; (b) the picture is a fuzzy digital template obtained by Gaussian blurring using a digital template; (c) The degree of blurring for the digital value is obtained by the threshold of the zero-crossing rate. According to the above, the degree of blurring between the matching of the number and the fuzzy digital template is changed. If the matching is unsuccessful, the digital template is changed, step S114: using Distance shape context recognizes the countdown The numerical outline of the countdown display.

As shown in FIG. 6A, a pedestrian alert assisted identification system 30 integrates with a mobile device 20 through the pedestrian countdown display instant recognition method (steps 1 to 12). The method includes: an image input unit 21 for capturing video, the video generating a continuous frame; a memory storage unit 22 electrically connected to the image input unit 21 for temporarily storing the continuous frame, the pedestrian countdown display An instant identification method (steps 1 to 12); an operation processing unit 23 electrically connected to the memory storage unit 22 for calculating the pedestrian countdown display instant recognition method (step 1 to step 12) to process the continuous frame; And an output result unit 24 electrically connected to the memory storage unit 22 for outputting the change state of the small green person walking animation surface and the countdown countdown display, and cooperating with a cloud system 40, which integrates the pedestrian through the network The warning auxiliary recognition system 30 is configured to enable the small green person to move to the cloud system 40 by moving the animation surface and the change state of the countdown timer display.

As shown in FIG. 6B, (a) is an instant identification of the green (Greenwalking green) produced by the small green man walking in the mask and the orange (Gouter orange) generated by the countdown, and at ( b) The picture shows the 40th frame (frame), which only takes 0.97974 seconds. Then, as shown in FIG. 6C, the picture (a) shows the number 3 (Timer Recognition: 3) of the instant recognition countdown, and (b) The picture shows the 102nd frame (frame), which only costs 0.87623.

Therefore, since the pedestrian recognizable display instant recognition method is written into an application program, a mobile device with a shooting function is embedded through a software interface to form a pedestrian warning auxiliary recognition system, which can be worn on a visually impaired person to make the visual obstacle The person can instantly recognize the pedestrian countdown display, and the visual instant recognition is superior to tools such as sound, sensor, pedestrian navigation, guide cane or guide dog.

To sum up, the technical means disclosed in the present invention have the invention patents such as "novelty", "progressiveness" and "available for industrial use", and pray for the patent to benefit the government. Invention, no sense of morality.

The drawings and the descriptions of the present invention are merely preferred embodiments of the present invention, and those skilled in the art, which are subject to the spirit of the present invention, should be included in the scope of the patent application.

S1~S12‧‧‧Steps

Claims (5)

An instant recognition method for a pedestrian countdown display comprises: a) inputting at least one frame, the frame being presented with a plurality of pixels, each of the pixels being converted into an HSV color space in an RGB color space to form another frame, the other frame having Hue pixel, saturated pixel, brightness pixel; b). Making a high saturated pixel through the mask, so that the other frame presents a highly saturated pixel to form an unmasked image, the unmasked image is converted in RGB color space CIELAB color space; c). Making a high-brightness pixel through the mask, so that the unmasked image presents high-brightness pixels, and the unmasked image is narrowed down and the energy distribution is estimated, and a small green is established. After the person passes the animated image and the countdown countdown displays the color statistical model of the image, the color statistical model is input to the unmasked image, so that the other frame has a plurality of orange signs and a plurality of green signs, wherein one of the orange signs is close to One of the green signs shows the relationship between the upper part and the lower part; d) provides a sliding window, and the upper part of the sliding window is set to match the orange color mark as the search object, and the lower part is set Matching the green mark to search for the object, and scanning the other frame with the sliding window; and e). the color statistics in the sliding window and the unmasked image in the sliding window have a density value, and the sliding The degree of matching between the color statistics in the window and the unmasked image in the sliding window is set to a threshold value. When the density value is greater than or equal to the threshold value, the orange and green flags in the upper and lower relationship are identified. The method for real-time identification of a pedestrian countdown display according to the first aspect of the invention, wherein the color statistical model training step of step c further comprises: inputting a training image; masking low saturation pixels and low in the training image a brightness pixel; recording a color distribution position of the training image in each pixel of the first and second channels of the CIELab color space; Enter the next training image; and store the final color statistical model. The method for real-time identification of a pedestrian countdown display according to the second aspect of the invention, wherein the color adjustment of the first channel is between red and green and the color of the second channel is adjusted between yellow and blue, and the The color adjustment range values of the first and second channels are all -109~110. The method for real-time identification of a pedestrian countdown display according to any one of claims 1 to 3, further comprising the step f: identifying a position of the small green person walking the animated image; cutting out the small green person The countdown countdown display image above the animated image; the inner distance shape context is used to identify the digital outline of the countdown countdown image, the digital outline having a plurality of contour points, each of the contour points having a shape context, each The shape context is normalized into a unified vector; a matching minimum difference between the digital contour of the countdown countdown display image and the blurred digital template is generated; the fuzzy detection of the countdown countdown display image number, the use of the zero-crossing rate The weighed zero crossing analyzes the one-dimensional difference around the number with a horizontal scan line. The one-dimensional difference is obtained from the histogram and the sum of the statistical weights. According to the zero-crossing threshold, the number is represented by the number. Degree of blur; and changing the fuzzy digital template to match the digital outline of the countdown countdown display image Survival of the fittest. The method for instantly recognizing a pedestrian countdown display as described in claim 4, wherein the step of cutting the countdown countdown image displayed above the small green person's animated image further comprises: inputting an image, The image has a number; the super-resolution enhances the image; transforming the two values in the image enhances the grayscale to render the significant number; removing the noise of the digit; obliquely correcting the digit; normalizing the digit; and identifying the digital contour.