CN109523551B - A method and system for obtaining the walking posture of a robot - Google Patents

Abstract

The invention relates to a method and a system for obtaining the walking posture of a robot, and belongs to the technical field of image processing. The method includes the following steps: (1) collecting an image of a background scene, an image of a marker point and an image sequence of a robot walking in the background scene, and the marker point is fixed on the robot to mark the walking track at the fixed position; (2) ) Taking the image of the background scene as a reference frame, segmenting out a partial image from the image sequence to form a foreground image sequence, the partial image including the image of the robot; (3) Taking the image of the marked point as a template, matching out each image sequence from the foreground image sequence. Mark the points and obtain their coordinate data; (4) Calculate the walking posture of the robot during the walking process according to the coordinate data of each marked point. The method can effectively reduce the cost of equipment required for obtaining the walking posture and the calculation amount of subsequent image processing, and can be widely used in fields such as robots.

Description

A method and system for obtaining the walking posture of a robot

This application is a divisional application for an invention patent with the application number of CN201711394246.1 and the invention titled "A method and system for obtaining the walking posture of a target".

technical field

The present invention relates to the technical field of image processing, in particular, to a method and a system for acquiring a walking posture of a robot.

Background technique

Wearable exoskeleton robots are suitable for the rehabilitation treatment of stroke, hemiplegia, stroke and other patients. In order to obtain rehabilitation treatment data, it is usually by capturing the movement posture of the joints of the rehabilitation personnel to judge the limb posture of the rehabilitation personnel after the exoskeleton robot is loaded. , and reconstruct the walking posture of the rehabilitation personnel based on the limb posture change data in at least one walking cycle.

Optical tracking motion measurement method is usually used to obtain the motion posture of the rehabilitation personnel's joints. This measurement method is based on computer vision technology, and the motion measurement of the target object is completed by tracking and calculating the specific mark points on the target object through cameras and other devices. Commonly used There are two types of marking points: reflective marking points and luminous marking points. The size and shape of the marking points can be set as required.

A typical optical tracking motion measurement system usually uses multiple cameras arranged around the performance venue, with the overlapping field of view of the cameras being the workspace. In order to facilitate subsequent image processing, the subject is usually required to wear black tights and affix special optical markers on key parts such as major joints. Before measuring, the system must first complete the calibration, and the camera can start to capture the movements of the subject, save, analyze, and process the captured image sequence to identify the optical landmarks in the image and calculate it at every moment. The spatial position of the object can be reconstructed to reconstruct the trajectory of the target. In order to obtain the accurate motion trajectory of the target, the camera is required to shoot at a high shooting rate.

Existing commonly used measurement systems require the use of multiple cameras, which leads to high costs and a large amount of calculation for subsequent image processing. In addition, they have high requirements on the illumination and reflection of the test site, which limits their application scenarios and is not conducive to wearable wearables. The application of skeletal robot in the aforementioned patient rehabilitation treatment.

SUMMARY OF THE INVENTION

The main purpose of the present invention is to provide a method for obtaining the walking posture of a robot, so as to reduce the required equipment cost while reducing the calculation amount of subsequent image processing; another purpose of the present invention is to provide a system for obtaining the walking posture of a robot, In order to reduce the cost of the required equipment, the calculation amount of the subsequent image processing is reduced.

In order to achieve the above-mentioned main purpose, the present invention provides a method for obtaining a walking posture of a robot, which includes a collection step, a segmentation step, a matching step and a calculation step; wherein, the collection step includes collecting an image of a background scene, an image of marked points, and the robot walking in the background scene. The image sequence of the process, the marking point is fixed on the robot to mark the walking trajectory at the fixed position; the segmentation step includes taking the image of the background scene as a reference frame, and segmenting the partial image from the image sequence to form the foreground image sequence, and the partial image is divided into the foreground image sequence. The image includes the image of the robot; the matching step includes using the image of the marked point as a template, matching each marked point from the foreground image sequence and obtaining its coordinate data; walking posture.

Taking the background scene as the reference frame, segment the image part that contains at least the robot from each frame of the image sequence, and use the image local area as the object of subsequent image processing—the foreground image, so that most of the background image does not need to be processed in the subsequent processing. image processing to effectively reduce the amount of calculation of subsequent image processing; at the same time, the image of the marked point is used as a template, and the position of the marked point is partially matched from the segmented image, so that the icon-type mark point can be used for marking, and at the same time, it can be The cost of the required equipment is effectively reduced by adopting the monocular camera to collect the image system.

A specific solution is to perform a decolorization step after the acquisition step and before the segmentation step, and the decolorization step includes converting the image of the background scene, the image of the marker points and the sequence of images into grayscale images. In the subsequent segmentation step and the matching step, the grayscale image is used as the object for processing, which can further reduce the calculation amount of the subsequent processing.

A more specific solution is that the segmentation step includes a construction step, a binarization step and a cropping step; wherein, the construction step includes performing the grayscale value of each frame of image and the corresponding pixel on the reference frame in the decolorized image sequence. The absolute value of the difference is processed to construct a difference frame sequence; the binarization step includes binarizing the difference frame sequence based on a predetermined threshold, and the robot and the background scene are represented by black and white respectively; the cropping step includes Based on the coordinate data of the color region of the robot, the foreground region sequence is cut out from the decolorized image sequence by using the rectangular boundary.

A further specific solution is to perform expansion processing on the difference frame after the binarization process after the binarization step and before the selection step, so as to further clarify the boundary line between the robot and the background and facilitate subsequent selection processing; The color area of the robot is the color area after the expansion process; after the decolorization step and before the segmentation step, the image sequence after the decolorization process is smoothed to reduce the noise introduced in the process of shooting.

A preferred solution is that the marking points include joint marking points fixed at each joint position of the walking mechanism of the robot, and the matching step includes a pre-matching step and a re-matching step; wherein, the pre-matching step includes taking the template as a reference, traversing the foreground area, calculating In the foreground area, the negative correlation degree R(x, y) of the local area centered on the pixel point with the coordinates (x, y) and the template, and based on the negative correlation degree being less than the preset threshold, the pixel cluster composition is obtained. There is a pre-selected marker cluster for characterizing the local area; the re-matching step is included in a pre-selected marker cluster, and the coordinates of the marker in the local area are represented by the pixel with the minimum negative correlation; the negative correlation R ( The formula for calculating x, y) is:

Among them, T(x', y') is the gray value of the pixel point whose coordinates are (x', y') in the template, and the coordinates of the pixel point on the template are the coordinates in the coordinate system constructed with the center point as the origin, I( x+x', y+y') is the gray value of the pixel whose coordinates are (x+x', y+y') in the foreground area, and the coordinate of the pixel in the foreground area is the pixel's coordinate in the image sequence. coordinate.

A more preferred solution is that the image of the marked point includes the front view image, the left oblique view image and the right oblique view image of the marked point, so as to solve the processing of the image captured by the monocular camera when the viewing angle is not correct; after the rematching step and before the calculation step. , according to the matched coordinates of the marked point, obtain the color of the local area at the corresponding point on the color image in the image sequence, and filter out the real marked point whether the local area matches the color on the marked point. Using the color information in the color image, the obtained markers are screened, which effectively avoids the problem of false markers being matched in the matching step.

Another preferred solution is that the image sequence is acquired by a monocular camera.

Another preferred solution is that the marking point includes a center portion and an annular portion surrounding the center portion, one of the center portion and the annular portion has a white surface, and the other has a black surface. The marking point is set to be composed of two circular parts with obvious color difference and nested with each other and an annular step, which can effectively improve the recognition accuracy, and at the same time, it is not necessary to limit the appearance color of the robot.

In order to achieve the above-mentioned other object, the system for obtaining the walking posture of a robot provided by the present invention includes a processor and a memory, and the memory stores a computer program. When the computer program is executed by the processor, the following receiving steps, segmentation steps, matching steps and calculation steps can be implemented. step; wherein, the receiving step includes receiving the image of the background scene collected by the camera and the image of the marked point, and receiving the image sequence of the robot walking process in the background scene collected by the monocular camera, and the marked point is fixed on the robot for use. The walking track at the fixed position of the marker; the segmentation step includes taking the image of the background scene as a reference, and segmenting the partial image from the image sequence to form the image sequence to be processed, and the partial image includes the image of the robot; the matching step includes taking the image of the marker point as the reference. A template is used to match each marker point from the foreground image sequence and obtain its coordinate data; the calculation step includes calculating the walking posture of the robot during the walking process according to the coordinate data of each marker point.

The specific scheme is to convert the image of the background scene, the image of marked points and the image sequence into grayscale images after the acquisition step and before the segmentation step; the marked points include joint marked points fixed at each joint position of the walking mechanism of the robot ; The marking point includes a center part and an annular part surrounding the center part, one of the center part and the annular part has a white surface, and the other has a black surface.

Description of drawings

1 is a work flow diagram of an embodiment of a method for obtaining a walking posture of a target according to the present invention;

2 is a schematic diagram of a process of segmenting an image in an embodiment of a method for obtaining a walking posture of a target according to the present invention, wherein FIG. 2( a ) is a background scene image serving as a reference frame for segmentation, and FIG. 2( b ) is an image to be segmented. Figure 2(c) is a schematic diagram of the difference frame, Figure 2(d) is the image after binarization processing, Figure 2(e) is the image after dilation processing, Figure 2 (f) is a schematic diagram of segmenting the foreground image from the image with a rectangular boundary;

Fig. 3 is the template of the marked point in the recognition step in the embodiment of the method for obtaining the walking posture of the target according to the present invention, wherein Fig. 3 (a) is the template image under the right oblique view, and Fig. 3 (b) is facing The template image under the viewing angle, Figure 3(c) is the template image under the left oblique viewing angle;

4 is a schematic process diagram of a pre-matching step in an embodiment of a method for obtaining a walking posture of a target according to the present invention;

5 is a schematic diagram of a process of calculating a walking attitude in a calculation step in an embodiment of a method for obtaining a walking attitude of a target according to the present invention;

FIG. 6 is a schematic structural block diagram of an embodiment of a walking attitude detection system according to the present invention.

The present invention will be further described below with reference to the embodiments and the accompanying drawings.

Detailed ways

In the following embodiments, the person after the load of the wearable exoskeleton robot is the target person, and the walking posture of the lower limb is obtained as an example to illustrate the method and system of the present invention for obtaining the walking posture of the target object, but the present invention The application scenarios of the method and system are not limited to the application scenarios shown in the following embodiments, and can also be used to obtain the walking postures of other objects such as robots and robot dogs.

Method embodiment

Referring to FIG. 1 , the method for obtaining the walking posture of a target object of the present invention includes a collection step S1 , a decolorization step S2 , a denoising step S3 , a segmentation step S4 , a matching step S5 , a screening step S6 and a calculation step S7 .

1. Collecting step S1 , collecting an image of a background scene, an image of a marker point, and an image sequence of the target object walking in the background scene, and the marker point is fixed on the target object to mark the walking track at the fixed position.

As shown in Figure 2(b), in order to mark the posture of the lower limbs of a person during walking, at least one marking point must be set at the three joints of the ankle joint, knee joint and hip joint of the lower limbs of the person; in this implementation In an example, the marking point is an icon-type marking point, as shown in FIG. 4 , which specifically includes a white center portion and a black annular portion arranged around the center portion, and of course, it can also be set to include a black center portion and a black center portion surrounding the center portion. The arranged white annular part is formed by setting the marking point to be a part with a larger black and white color difference, so that after the color removal process in the color removal step S2, the primary color difference is still retained at the marked point image to facilitate subsequent identification processing. ; In addition, the image marking point is set as a center part and a ring part surrounding the center part, which is convenient for correction when the viewing angle is not correct to obtain the center point position of the center part. Of course, only single-color or more than three-color marking points can be set. For single-color image marking points, it is preferable to use a color that still has a large color difference from the surrounding color at the fixed position after decolorization; The structure of each part is not limited to the above-mentioned circular structure. For example, it can also be set into four squares spliced together, and among the two adjacent squares, one is black and the other is white. Obtain the center position of the marked point at the intersection of the chromatic aberration blocks. For the image that needs to be decolorized later, it is preferably black and white contrast color; of course, it can be set to other multi-color stitching structures that still have large color difference after decolorization; if the image does not need to be decolorized, you can also Use a color structure with a large color difference for flattening.

The images collected by the camera are captured in the form of data frames, and multiple frames of images are rapidly iterated in sequence to form a video stream. Each frame of the video stream contains the gait information of the target. In the storage data of the video stream, each frame of image captured by the camera exists in the form of a matrix array. In this embodiment, the camera used to obtain the background scene images and image sequences is a monocular camera installed on the side of the person's walking path. The person's walking path is preferably a straight path. All are within the viewing angle of the monocular camera, and the monocular camera can be installed at the central axis position of the overall background scene image.

Usually, each frame of image captured by the camera is represented by the BGR color space format by default. BGR has three channels, namely the three matrices in the array, which respectively represent the blue, green and red parts of the three primary colors. The color of each pixel can be divided into the mixture of these three colors according to different proportions. The scale data of each color is saved in the corresponding position of each channel.

2. Decoloring step S2, performing decoloring processing on the collected background scene image, marker point image and image sequence, so as to convert the color image into a grayscale image.

Although the image data in BGR format retains the most optical information, not all operations need to utilize all the optical information in the process of finding marker points. Compressing a three-channel matrix array into one channel, although at the cost of losing some data, can better present the really useful information and speed up the search for markers. In the form of an image, the color image is compressed into a grayscale image, but the description of the grayscale image, like the color image, still reflects the overall and local distribution and characteristics of the chromaticity and brightness levels of the entire image. The matrix transformation is performed according to the following equation 1:

Y=0.114·B+0.587·G+0.299·R

Among them, Y represents the matrix of the grayscale image, and the larger the value is, the whiter the pixel at this position is, and vice versa.

3. Noise reduction step S3, performing smoothing processing on the decolorized image sequence.

Due to factors such as natural vibration, illumination changes, or hardware problems, each frame of image acquired contains noise. By smoothing the data frame, the noise can be effectively removed and the interference of the noise to the subsequent detection process can be avoided.

In this embodiment, the smoothing process is to use Gaussian blur to process each pixel of the image, that is, for any pixel, the weighted average of its surrounding pixels is taken, and the weights are assigned according to a normal distribution. The larger the point weight, the less weight the farther point is. In the actual smoothing operation, it is found that taking the target pixel as the center, taking a 21 × 21 matrix, and weighting and averaging the surrounding pixels in the matrix has the best noise removal effect.

4. Segmentation step S4, taking the image of the background scene as a reference frame, segmenting a partial image from the image sequence to form a foreground image sequence, and the partial image includes the image of the target object.

In the actual shooting environment, the target person completely walks through the lens field of the monocular camera from left to right, and during this process, the marker points at the three joint points of the fixed lower limbs are exposed to the camera. In the process of capturing optical information, the monocular camera maps the three-dimensional environment into a two-dimensional plane. From the perspective of the obtained two-dimensional video, since the camera is still, each frame of the image in the entire video stream can be roughly divided into two parts, the foreground and the background; the foreground refers to the location where the moving target person is located. Local area, background refers to the area that fills the entire environment scene except the target person, that is, other areas except the foreground area; as the target person walks into the viewfinder from one end of the lens field of view, and walks out of the scene from the other end. In the viewfinder frame, the foreground area is constantly sliding relative to the background area in the video, while the background area remains stationary in the entire video, and different areas are occluded by the moving foreground area. From the perspective of the area ratio of the entire video frame, the foreground area occupies a smaller area, while the background area occupies a larger area. Since the area where the target marker to be calibrated is located is the foreground area, scanning the background area will not only consume computing power and waste time, but may even solve the wrong result when the matching threshold is set low, further interfering with the marking point data. extract.

If the foreground area can be segmented from the entire data frame, then the marker point search can be performed only for this area, ignoring the background area that accounts for the majority of the area, which can greatly improve the search efficiency.

Therefore, in this embodiment, by dividing the foreground region from the data frame as the subsequent image processing object, the calculation amount thereof can be effectively reduced, and the positioning accuracy of the target marker point can be improved at the same time. According to the actual situation, at the beginning of the video, if the tester has not yet entered the camera, it is agreed that the first frame of the video stream belongs to the background area, which constitutes the background scene image. In the image processing of , the foreground and background are divided based on the reference background frame.

In this embodiment, the segmentation basis of the foreground region is threshold binarization, which specifically includes a construction step S41 , a binarization step S42 , a dilation step S43 and a clipping step S44 .

(1) Constructing step S41 , performing absolute difference processing on the grayscale values of each frame of image in the decolorized image sequence and the corresponding pixel points on the reference frame to construct a difference value frame sequence.

For any data frame M, it is calculated according to the following formula 2:

absdiff(I)=|M(I)-M _o (I)|

Among them, M _o is the reference background frame, M(I) represents the frame to be processed, I represents a specific position in the data frame, and absdiff is a matrix array whose elements represent the difference between the gray values of the pixels at the I position the absolute value of .

In the grayscale image obtained by decolorization, the value range of each element point is 0-255. It is easy to know that the value of each element in absdiff is also in the range of 0-255, and absdiff can also be regarded as a gray degree image, and the result is shown in Figure 2(c).

(2) Binarization step S42 , based on a predetermined threshold, the difference frame sequence is binarized, and the target object and the background scene are represented by black and white respectively.

By setting a threshold, the absdiff is binarized into a pure black and white image, and the white and black areas roughly represent the distribution of the foreground and background. The result is shown in Figure 2(d).

(3) Dilation step S43 , performing dilation processing on the difference frame after the binarization process.

The binarized image will inevitably have a lot of noise, because each data point in the area at the junction of the foreground and the background is always large or small relative to the threshold, and it is difficult to divide the ideal black- White dividing line. Dilation can be used to trim burrs and remove noise.

First, define a structure matrix {E _ij , i, j=1, 2, 3, 4, 5} with a size of 5×5. The generation rule of the structure matrix is shown in the following formula 3:

也要改成合适的数字，其中

是个经验值。The size of the structure matrix can be adjusted according to the actual situation, and correspondingly, the regular

is also changed to a suitable number, where

is an experience value.

Second, after generating the structuring element, use it to traverse the entire image, and obtain the dilated binary image according to the following rules:

dilate(x, y)=max absdiff(x+x', y+y'), E(x', y')≠0

Among them, (x, y) is the coordinates of the pixel to be processed, and E(x', y') is the element of the matrix {E _ij }.

In the obtained dilate image, the white part is expanded into a more continuous area, and the boundary is more obvious. The result is shown in Figure 2(e).

(4) the selection step S44, based on the coordinate data of the target object color region, the foreground region sequence is cut out from the decolorized image sequence by using a rectangular boundary, the rectangular boundary completely contains the target object color region, and the target object color region is The color area that characterizes the target.

According to the distribution of the white part, a complete area is framed on the data frame of the image sequence with a rectangular boundary, which is used as the foreground area, and the remaining area is the background area. The result is shown in Figure 2(f).

5. Matching step S5 , using the image of the marked point as a template, each marked point is matched from the foreground image sequence and its coordinate data is obtained.

In this step, the three pre-collected marker images are used as matching templates, and then the foreground regions segmented in the preceding steps will be traversed based on the three matching templates. Specifically, the pre-matching step S51 and the re-matching step S52 are included.

(1) pre-matching step S51, taking the template as a benchmark, traverse the foreground area, and calculate the negative correlation degree R(x, y) between the local area and the template in the foreground area centered on the pixel point with coordinates (x, y), And according to the reference that the negative correlation degree is less than the preset threshold, the acquired pixel point cluster constitutes a preselected marker point cluster used to represent that the local area has one marker point.

When the template traverses to a certain local area centered on (x, y) in the foreground area, the following formula 4 is used to calculate their normalized sum of squared differences (SQDIFF_NORMED):

Among them, R is a variable related to the degree of template correlation, which is used to represent the negative correlation degree. The smaller the value of R, the better the matching degree of the corresponding pixel and its surrounding area with the template; T(x', y') is the gray value of the pixel point whose coordinates are (x', y') in the template, the coordinates of the pixel point on the template are the coordinates in the coordinate system constructed with the center point as the origin, I(x+x', y+y') is the gray value of the pixel whose coordinates are (x+x', y+y') in the foreground area, and the coordinates of the pixel in the foreground area are the coordinates of the pixel in the image sequence, usually the upper left in the image. The corner or lower left corner is the origin.

Because, in the actual situation detection, the marked points on the target person cannot always face the camera, so the marked image captured by the camera is not always the concentric circles with good geometry as shown in Figure 3(b). , the irregular ellipse shape as shown in Figure 3(a) and Figure 3(c) may also appear. Therefore, three templates are designed for each marker point with respect to its frontal, left oblique and right oblique viewing angles, as shown in Figures 3(b), 3(a) and 3(c).

When traversing, calculate the local matching degree between the three templates and the foreground area, and keep the one with the smallest result. The matching degree is calculated by the following formula 5:

After the traversal is completed, the R values corresponding to most of the pixels are too large, indicating that the region cannot match the template; the R value of a few pixels drops to a small range, indicating that the region is very close to the template. In this embodiment, 0.1 is taken as the threshold value, and the pixel area with the R value greater than this value is discarded. Otherwise, the area is considered to be the area where the marker point is located, and the center pixel coordinate data is recorded.

(2) Re-matching step S52, in one of the preselected marker point clusters, the coordinates of the marker points in the local area are represented by the pixel points with the smallest negative correlation R.

Checking the coordinate data obtained by the template matching layer, it can be found that there is a large overlap, that is, the points with better matching degree are often clustered together, and the sum of the squares of their differences is less than the threshold, so they are all recorded. However, these The matching windows corresponding to the points will overlap significantly, and the center of the overlap is generally a certain marker point. Since this cluster of coordinate points all represent the same marker point, the best representative one can be selected from a cluster to represent the marker point, and the remaining coordinates are discarded as noise. Use the following formula 6 to screen markers:

(x, y) = minR(x', y'), (x', y') ∈ Range

Among them, Range is the area where a cluster of coordinate points is located, and the coordinate point with the lowest R value is selected as the result of the best matching degree among the coordinate points in the cluster to represent the marked point here.

6. Screening step S6: According to the matched coordinates of the marked points, obtain the color of the local area at the corresponding point on the color image in the image sequence, and screen out the real marked point whether the local area matches the color on the marked point.

In the matching step S5, the obtained coordinate points are theoretically the positions of the respective marker points, but in rare cases, misjudgments in the matching process due to changes in other regions in the foreground region cannot be ruled out.

Since the foreground area is constantly changing, and the misjudgment area will not cause continuous interference, the misjudgment occurs randomly and continuously. To eliminate this interference, gamut filtering is adopted for each target region as a simple additional discrimination rule.

Color gamut screening utilizes color images that have not been involved in marker point recognition before. In this screening step, this part of the information is utilized by comparing the colors of coordinate points to further optimize the recognition results.

The specific process of color gamut screening is as follows:

First, convert the image in BGR format to an image in HSV format through the following equations 7 to 9:

V=max(R, G, B)

In the HSV color space, the three parameters H, S, and V represent hue, saturation, and lightness, respectively. Compared with the BGR space, it is more suitable for comparing the similarity of two colors. It is stipulated that for a coordinate point to be measured, if its core color is white, it will pass the test. If the core color is not white, it will not pass the test, and the coordinate point will be discarded. Determining whether a color belongs to the white range is based on the S and V parameters. It is stipulated that the following formulas 10 and 11 are satisfied, and they belong to white:

0≤S≤0.5

0.5≤V≤1

This method can be used as an auxiliary means to quickly screen whether the located markers meet the requirements. Because the data it faces has been relatively reliably screened in the previous layers, the accuracy requirements are not too high.

7. Calculation step S7, calculating the walking posture of the target object during the walking process according to the obtained coordinate data of each marked point.

As shown in FIG. 5 , according to the coordinates of the three marked points at a certain moment, that is, the coordinates of the target person’s hip joint 21 , knee joint 22 and ankle joint 23 , the coordinates of the three joints are (x1, y1 ) ), (x2, y2) and (x3, y3), so that the angle θ between the mid-thigh axis 24 and the vertical and the angle α between the mid-thigh axis 24 and the mid-axis 25 of the calf can be calculated for the The two angle values represent the walking posture of the lower limbs of the target person at the moment. The specific calculation formulas of the included angle and the included angle are shown in the following formulas 12 and 13:

Wherein, vector r1=(0, 1), vector r2=(x2-x1, y2-y1), and vector r3=(x3-x2, y3-y2).

Of course, other combinations of angles can be used to represent the walking posture of the walking person's lower limbs, and other physical quantities other than the angles can also be used to represent the walking posture of the target person.

System embodiment

Referring to FIG. 6, the system 1 for obtaining the walking posture of the target object of the present invention includes a monocular camera, a processor and a memory, and the memory stores a computer program. When the computer program is executed by the processor, it can realize the receiving step S1, the color removal step S2, Noise reduction step S3, segmentation step S4, matching step S5, screening step S6 and calculation step S8.

The receiving step S1 is to receive the data collected by the camera. The specific collection process in the receiving step S1 and the specific contents of the decolorizing step S2 to the calculating step S7 have been described in detail in the above method embodiments, and will not be repeated here.

In the above embodiment, the decolorization step S2, the noise reduction step S3 and the screening step S6 are not mandatory steps, but optional optimization steps, that is, the decolorization step S2 can further reduce the calculation amount of the subsequent image processing, and by reducing the The noise step S3 further improves the subsequent matching accuracy, and the screening step S7 can eliminate misjudged markers.

Claims (6)

1. a method for obtaining robot walking posture, is characterized in that, comprises: The collection step is to collect the image of the background scene, the image of the mark point and the image sequence of the walking process of the robot in the background scene, and the mark point is fixed on the robot to mark the walking trajectory at the fixed position ; the image sequence is collected by a monocular camera; Decoloring step, converting the image of the background scene, the image of the marker point and the image sequence into a grayscale image; The segmentation step, taking the image of the background scene as a reference frame, and segmenting a partial image from the image sequence to form a foreground image sequence, the partial image including the image of the robot; the segmentation step includes a construction step, a binary image The step of decolorization and the step of cropping; the step of constructing includes performing the difference and absolute value processing on the gray value of each frame of image in the decolorized image sequence and the gray value of the corresponding pixel on the reference frame, and constructing the difference value frame sequence; the binarization step includes performing a binarization process on the difference frame sequence based on a predetermined threshold, and using black and white to represent the robot and the background scene respectively; the cropping step includes based on The coordinate data of the robot color area, the foreground area sequence is cut out from the decolorized image sequence by using a rectangular boundary, the rectangular boundary completely contains the robot color area, and the robot color area is the color that characterizes the robot area; In the matching step, using the image of the marked point as a template, each marked point is matched from the foreground image sequence and its coordinate data is obtained; Joint marking points, the matching step includes a pre-matching step and a re-matching step; The pre-matching step includes taking the template as a reference, traversing the foreground areas of the foreground area sequence, and calculating the difference between the local area centered on the pixel point with coordinates (x, y) in the foreground area and the template. Negative correlation degree R(x, y), and based on the negative correlation degree being less than a preset threshold as a reference, obtain a pixel cluster to form a pre-selected marker cluster for representing that the local area has a marker; The re-matching step includes in one of the pre-selected marking point clusters, representing the coordinates of the marking points in the local area with the pixel point with the least negative correlation; The calculation formula of the negative correlation degree R(x, y) is:

Wherein, T(x', y') is the gray value of the pixel point whose coordinates are (x', y') in the template, and the coordinates of the pixel point on the template take the center point as the origin in the coordinate system constructed Coordinates, I(x+x', y+y') is the gray value of the pixel whose coordinates are (x+x', y+y') in the foreground area, and the coordinates of the pixel on the foreground area are the coordinates of the pixel in the image sequence; The calculation step is to calculate the walking posture of the robot during the walking process according to the coordinate data of the marked points. 2. method according to claim 1, is characterized in that: After the binarization step and before the cropping step, dilation is performed on the difference frame after the binarization process; The robot color area is the color area after expansion processing; smoothing the decolorized image sequence after the decolorization step and before the segmentation step; The smoothing process is to use Gaussian blur to process each pixel of the image. 3. method according to claim 1, is characterized in that: The image of the marked point includes a front view image, a left oblique perspective image and a right oblique perspective image of the marked point; After the re-matching step and before the calculation step, the color of the local area at the corresponding point on the color image in the image sequence is obtained according to the coordinates of the matched marker point, and whether the color of the local area and the marker point is Match and filter out the real markers. 4. The method according to any one of claims 1 to 3, wherein: The marking point includes a center portion and an annular portion surrounding the center portion. One of the center portion and the annular portion has a white surface, and the other has a black surface. 5. A system for obtaining the walking posture of a robot, comprising a processor and a memory, wherein the memory stores a computer program, wherein, when the computer program is executed by the processor, the following steps can be realized: The receiving step is to receive the image of the background scene collected by the camera and the image of the marker point, and receive the image sequence of the robot walking process in the background scene collected by the monocular camera, and the marker point is fixed at the The robot is used to mark the walking trajectory at the fixed position; Decoloring step, converting the image of the background scene, the image of the marker point and the image sequence into a grayscale image; The segmentation step, taking the image of the background scene as a reference frame, and segmenting a partial image from the image sequence to form a foreground image sequence, the partial image including the image of the robot; the segmentation step includes a construction step, a binary image The step of decolorization and the step of cropping; the step of constructing includes performing the difference and absolute value processing on the gray value of each frame of image in the decolorized image sequence and the gray value of the corresponding pixel on the reference frame, and constructing the difference value frame sequence; the binarization step includes performing a binarization process on the difference frame sequence based on a predetermined threshold, and using black and white to represent the robot and the background scene respectively; the cropping step includes based on The coordinate data of the robot color area, the foreground area sequence is cut out from the decolorized image sequence by using a rectangular boundary, the rectangular boundary completely contains the robot color area, and the robot color area is the color that characterizes the robot area; In the matching step, using the image of the marked point as a template, each marked point is matched from the foreground image sequence and its coordinate data is obtained; Joint marking points, the matching step includes a pre-matching step and a re-matching step; The pre-matching step includes taking the template as a reference, traversing the foreground areas of the foreground area sequence, and calculating the difference between the local area centered on the pixel point with coordinates (x, y) in the foreground area and the template. Negative correlation degree R(x, y), and based on the negative correlation degree being less than a preset threshold as a reference, obtain a pixel cluster to form a pre-selected marker cluster for representing that the local area has a marker; The re-matching step includes in one of the pre-selected marking point clusters, representing the coordinates of the marking points in the local area with the pixel point with the least negative correlation; The calculation formula of the negative correlation degree R(x, y) is:

Among them, T(x', y') is the gray value of the pixel point whose coordinates are (x', y') in the template, and the coordinates of the pixel point on the template take the center point as the origin in the constructed coordinate system. Coordinates, I(x+x', y+y') is the gray value of the pixel whose coordinates are (x+x', y+y') in the foreground area, and the coordinates of the pixel on the foreground area are the coordinates of the pixel in the image sequence; The calculation step is to calculate the walking posture of the robot during the walking process according to the coordinate data of the marked points. 6. The system of claim 5, wherein: The marking point includes a center portion and an annular portion surrounding the center portion. One of the center portion and the annular portion has a white surface, and the other has a black surface.

Images