KR102642691B1 - apparatus for recognizing measurement value and correcting distortion of instrument panel image and camera - Google Patents

Abstract

The instrument panel image-based measurement value recognition device according to the present invention relates to an instrument panel image-based measurement value recognition device that recognizes measured values from an instrument panel image captured by an instrument panel photographing device that photographs the instrument panel of a measuring device. an input unit that receives an image of a driving instrument panel from which the instrument panel of a driving measuring device is captured; and generating a driving instrument panel correction image by correcting distortion of the driving instrument panel image, performing preprocessing for straight line detection on the driving instrument panel correction image to generate a driving instrument panel preprocessing image, and generating a plurality of straight lines from the driving instrument panel preprocessing image. Detects, selects a guideline outline corresponding to the outline of the guideline of the instrument panel among the plurality of straight lines based on one or more of the straight line length, straight line position, and straight line angle of each of the plurality of straight lines, and determines the guideline outline angle of the guideline outline. It may include a processor that calculates the measured value indicated by the guideline based on the measured value indicated by the guideline.

Description

Measurement value recognition device that corrects instrument panel image distortion and imaging device distortion {apparatus for recognizing measurement value and correcting distortion of instrument panel image and camera}

The present invention relates to a measurement value recognition device based on an instrument panel image, and more specifically, to a measurement value recognition device based on an instrument panel image capable of recognizing measurement values measured from a measurement device based on an image of the instrument panel taken from an instrument panel photographing device. will be.

Factories that produce specific products have recently been operated by automated systems. Previously, the automation system of these factories was not connected to an external network and was operated by configuring an independent network for each factory automation system, but recently, these factory automation systems have been connected to an external network to control and produce There are active attempts to reuse data produced from devices for production and management.

Attempts to connect each device of this factory automation system to a network are being made in the form of industrial IoT (INDUSTRIAL IoT) due to the recent increase in interest in the Internet of Things (INTERNET OF THINGS) and the generalization of technology. This industrial IoT does not just network and automate only factory production facilities, but also involves even detailed parts of the factory system in IoT, enabling automatic control and various management, as well as using various internal and external data for factory operation. It is possible to greatly improve operational efficiency.

In particular, by applying IoT, various devices with various protocols can participate in the system, making it possible to configure an advanced system compared to the automation system used in the past.

The individual facilities that make up a large-scale smart factory interact organically with each other. For example, products pretreated in a pretreatment facility are transferred to the next processing facility during the manufacturing process using a transfer robot. In this connection structure of facilities, if a failure occurs in one facility, the entire manufacturing line including the failed facility must be stopped, so predicting facility failure is very important in the operation of a smart factory.

In addition, it is also very important to quickly and accurately collect and utilize measurement values from various measuring devices, control the operation of the smart factory based on the collected measurement values, and take action for each situation.

At this time, in order to obtain measurement values used to predict failure or determine the status of the smart factory, a large number of sensors must be installed in the smart factory and the signals must be received periodically, so the cost of building a smart factory is high. It is predicted.

Korean Patent Publication No. 10-2019-0062739

The purpose of the present invention is to correct the distortion of the instrument panel image captured by an instrument panel imaging device installed on the instrument panel at an angle close to the instrument panel, and to provide instrument panel image-based measurement that can recognize the measured value of the measurement device from the corrected instrument panel image. To provide a value recognition device.

In addition, the purpose of the present invention is to select a guideline outline corresponding to the guideline outline among the straight lines detected from the dashboard image, obtain the angle of the guideline center line from the guideline outline, and calculate the measurement value indicated by the guideline. To provide a value recognition device.

The objects of the present invention are not limited to the objects mentioned above, and other objects and advantages of the present invention that are not mentioned can be understood by the following description and will be more clearly understood by the examples of the present invention. Additionally, it will be readily apparent that the objects and advantages of the present invention can be realized by the means and combinations thereof indicated in the patent claims.

The instrument panel image-based measurement value recognition device according to the present invention relates to an instrument panel image-based measurement value recognition device that recognizes measured values from an instrument panel image captured by an instrument panel photographing device that photographs the instrument panel of a measuring device. an input unit that receives an image of a driving instrument panel from which the instrument panel of a driving measuring device is captured; and generating a driving instrument panel correction image by correcting distortion of the driving instrument panel image, performing preprocessing for straight line detection on the driving instrument panel correction image to generate a driving instrument panel preprocessing image, and generating a plurality of straight lines from the driving instrument panel preprocessing image. Detects, selects a guideline outline corresponding to the outline of the guideline of the instrument panel among the plurality of straight lines based on one or more of the straight line length, straight line position, and straight line angle of each of the plurality of straight lines, and determines the guideline outline angle of the guideline outline. It may include a processor that calculates the measured value indicated by the guideline based on the measured value indicated by the guideline.

Preferably, the processor may generate the driver instrument panel correction image by correcting one or more of perspective distortion and instrument panel imaging device distortion of the driver instrument panel image.

Preferably, the processor performs one or more of region-of-interest extraction to extract only the region of interest from the dashboard correction image and color binarization to binarize the color information of the dashboard correction image to pre-process the instrument panel correction image. It can be created as an image.

Preferably, the processor may delete a straight line whose straight line length is less than the reference straight line length among the plurality of straight lines.

Preferably, the processor sets an area less than a first radius from the rotation center of the pointer as a first area, and sets an area more than a first radius and less than a second radius from the rotation center of the pointer as a second area, Among the plurality of straight lines, straight lines whose starting and ending points are not included in the first and second areas may be deleted.

Preferably, the processor may delete a straight line whose start and end points are included in the first area among the plurality of straight lines.

Preferably, the processor groups straight lines overlapping the first area among a plurality of straight lines that have not been deleted into a first straight line group, and groups straight lines overlapping in the second area among the plurality of straight lines into a second straight line group. And, if the straight line angle difference between the straight line angles of each straight line included in the first straight line group and the straight line angle included in the second straight line group is within the reference angle difference range, the corresponding straight line can be selected as the guideline outline.

Preferably, the processor classifies the guideline outlines into a first guideline outline group and a second guideline outline group based on the guideline outline angle of each of the guideline outlines, and the first guideline outline included in the first guideline outline group is Calculate a guideline center line angle based on the guideline outline angle and the second guideline outline angle of the guideline outline included in the second guideline outline group, and the guideline center line angle, the minimum measured value guideline angle of the measuring device, and the maximum measured value guideline. The measured value indicated by the guideline can be calculated based on the angle, minimum measured value, and maximum measured value.

The instrument panel image-based measurement value recognition device according to the present invention sets two areas in the distortion-corrected instrument panel image, selects the guide outline from the straight lines of the two areas using the straight angle, and then obtains the angle of the guide center line, and performs measurement. The measured values of the device can be accurately recognized.

Figure 1 is a diagram illustrating a dashboard image-based measurement value recognition device and a dashboard of the measurement device, a dashboard imaging device, and a server according to an embodiment of the present invention.
Figure 2 is an exploded perspective view of a dashboard imaging device connected to a dashboard image-based measurement value recognition device according to an embodiment of the present invention.
Figure 3 is a block diagram of the instrument panel image-based measurement value recognition device according to an embodiment of the present invention.
Figure 4 is an image of an instrument panel for calibration used in an instrument panel image-based measurement value recognition device according to an embodiment of the present invention.
Figure 5 is a rotated instrument panel image for calibration used in an instrument panel image-based measurement value recognition device according to an embodiment of the present invention.
Figure 6 is a correction image of the instrument panel for calibration used in the instrument panel image-based measurement value recognition device according to an embodiment of the present invention.
Figure 7 is a pre-processing image of the instrument panel for calibration used in the instrument panel image-based measurement value recognition device according to an embodiment of the present invention.
Figure 8 is an image of a driving instrument panel used in an instrument panel image-based measurement value recognition device according to an embodiment of the present invention.
Figure 9 is a rotated driving instrument panel image used in an instrument panel image-based measurement value recognition device according to an embodiment of the present invention.
Figure 10 is a correction image of the driving instrument panel used in the instrument panel image-based measurement value recognition device according to an embodiment of the present invention.
Figure 11 is a pre-processing image of the driving instrument panel used in the instrument panel image-based measurement value recognition device according to an embodiment of the present invention.
FIG. 12 is a diagram illustrating a plurality of straight lines detected from a pre-processed image of a driving instrument panel by an instrument panel image-based measurement value recognition device according to an embodiment of the present invention.
Figures 13 and 14 are diagrams for explaining a process in which the instrument panel image-based measurement value recognition device selects a guideline outline from a plurality of straight lines according to an embodiment of the present invention.
FIG. 15 is a diagram illustrating a process in which the instrument panel image-based measurement value recognition device calculates the measurement value indicated by the pointer according to an embodiment of the present invention.

Hereinafter, various embodiments of the present invention are described with reference to the accompanying drawings. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include various modifications, equivalents, and/or alternatives to the embodiments of the present invention. In connection with the description of the drawings, similar reference signs may be used for similar components.

In this document, expressions such as “have,” “may have,” “includes,” or “may include” refer to the presence of the corresponding feature (e.g., numerical value, function, operation, or component such as a part). , and does not rule out the existence of additional features.

In this document, expressions such as “A or B,” “at least one of A or/and B,” or “one or more of A or/and B” may include all possible combinations of the items listed together. . For example, “A or B,” “at least one of A and B,” or “at least one of A or B” (1) includes at least one A, (2) includes at least one B, or (3) It can refer to all cases including both at least one A and at least one B.

As used herein, expressions such as "first", "second", "first", or "second" may modify various elements in any order and/or importance, and may refer to one element. It is only used to distinguish from other components and does not limit the components. For example, a first user device and a second user device may represent different user devices regardless of order or importance. For example, a first component may be renamed a second component without departing from the scope of rights described in this document, and similarly, the second component may also be renamed to the first component.

A component (e.g., a first component) is “(operatively or communicatively) coupled with/to” another component (e.g., a second component). When “connected to” is mentioned, it should be understood that any component may be directly connected to the other component or may be connected through another component (e.g., a third component). On the other hand, when a component (e.g., a first component) is referred to as being “directly connected” or “directly connected” to another component (e.g., a second component), it is said that the component and the other component are It may be understood that no other components (e.g., third components) exist between components.

The expression "configured to" used in this document may be used depending on the context, for example, "suitable for," "having the capacity to." )", "designed to", "adapted to", "made to", or "capable of". . The term “configured (or set) to” may not necessarily mean “specifically designed to” in hardware. Instead, in some contexts, the expression “a device configured to” may mean that the device is “capable of” working with other devices or components. For example, the phrase "control configured (or set) to perform operations A, B, and C" means that a dedicated processor (e.g., an embedded processor) to perform those operations, or by executing one or more software programs stored in memory, , may refer to a general-purpose processor (e.g., CPU or application processor) capable of performing the corresponding operations.

In particular, in this specification, “~device” may include one or more of a central processing unit (CPU), an application processor (AP), and a communication processor (CP). .

In this specification, “~device” refers to all types of hardware devices including at least one processor, and depending on the embodiment, it may be understood as encompassing software configurations operating on the hardware device. For example, “~device” can be understood as including, but is not limited to, a mechanical drive device, smartphone, tablet PC, desktop, laptop, and user clients and applications running on each device.

Terms used in this document are merely used to describe specific embodiments and may not be intended to limit the scope of other embodiments. Singular expressions may include plural expressions, unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by a person of ordinary skill in the technical field described in this document. Among the terms used in this document, terms defined in general dictionaries may be interpreted to have the same or similar meaning as the meaning they have in the context of related technology, and unless clearly defined in this document, they may be interpreted in an ideal or excessively formal sense. It is not interpreted. In some cases, even terms defined in this document cannot be interpreted to exclude embodiments of this document.

1 is a diagram illustrating a dashboard image-based measurement value recognition device and a dashboard of the measurement device, a dashboard imaging device, and a server according to an embodiment of the present invention, and FIG. 2 is a diagram showing a dashboard image-based measurement according to an embodiment of the present invention. This is an exploded perspective view of the instrument panel imaging device connected to the value recognition device.

Referring to FIGS. 1 and 2 , the instrument panel image-based measurement value recognition device 100 according to an embodiment of the present invention receives the instrument panel image captured from the instrument panel photographing device 200 and determines the instructions of the measurement device (M). The measured value indicated by (N) can be calculated.

Here, the measuring device M may be a device that displays measured values only in an analog manner and cannot output the measured values through communication or the like.

For example, the measuring device M may be a device that measures pressure, flow rate, power, voltage, current, etc., and displays the measured values obtained through these measurements on a dashboard P composed of a measuring needle or gauge.

Meanwhile, note that the measuring device M may be circular in shape with a predetermined thickness, but its shape is not limited as long as it represents measured values.

Here, the instrument panel photographing device 200 is disposed close to the measurement device M, photographs the instrument panel P of the measurement device M, and outputs the captured instrument panel image to the instrument panel image-based measurement value recognition device 100. can do.

The instrument panel photographing device 200 may include a housing portion 210, a bracket portion 220, and a camera module 230.

The housing portion 210 houses a camera module 230 that photographs the instrument panel (P), and one side of the bracket portion 220 is coupled to the housing portion 210 and the other side is fastened to one side of the measuring device (M). Thus, the housing portion 210 can be mounted on the measurement device (M) so that the shooting direction of the camera module 230 is toward the instrument panel (P).

At this time, the bracket portion 220 includes a first plate 221 and a second plate 222, and the first lower surface of the first plate 221 is in contact with one side of the measuring device M, and the second plate 221 The other end of 222 is connected to one end of the first plate 221, and the second lower surface is in contact with the other side of the housing portion 210.

Specifically, the second plate 222 has a second lower surface at a predetermined connection angle with the first lower surface of the first plate 221 so that the shooting direction of the camera module 230 is toward the central area of the instrument panel (P). It can be connected to form a .

That is, the second plate 222 and the first plate 221 may be connected at a connection angle. Here, the connection angle may be a predetermined angle corresponding to the shooting direction of the camera module 230.

In addition, the connection angle formed by the second plate 222 and the first plate 221 may mean an internal angle, and the second plate 222 and the second plate ( The angle may be such that the housing portion 210 coupled with 222) is not disposed.

That is, the second plate 222 may be connected to the first plate 221 at a connection angle such that one end located on the opposite side of the first plate 221 is away from the instrument panel P.

The instrument panel imaging device 200 described above may generate a distorted instrument panel image due to its structure, the connection angle formed by the second plate 222 and the first plate 221, and the location where it is installed in the measurement device M. .

Specifically, compared to the instrument panel image taken in the direction perpendicular to the front of the instrument panel (P), that is, from the front of the instrument panel (P), the instrument panel image taken by the instrument panel imaging device 200 is taken at a distance close to the instrument panel (P). By being photographed and created at an angle, perspective distortion and instrument panel imaging device distortion may occur.

Here, the instrument panel photographing device distortion is a distortion caused by the optical structure of the instrument panel photographing device 200, and the fisheye lens distortion and instrument panel photography caused by the fisheye lens provided in the camera module 230 of the instrument panel photographing device 200 This may include distortion caused by the lens and image sensor of the device 200 being twisted and arranged.

Additionally, the instrument panel imaging device 200 can capture the instrument panel P at a rotated angle, thereby generating an image of the instrument panel captured at a rotated angle relative to the instrument panel observation angle.

Accordingly, the instrument panel image-based measurement value recognition device 100 according to an embodiment of the present invention measures the structure of the instrument panel imaging device 200, the connection angle formed by the second plate 222 and the first plate 221, and the measurement value. Calibration is performed considering the location where it is installed in the device (M), rotation, distortion correction and preprocessing are performed on the instrument panel image, a guideline outline is selected after detecting a straight line from the instrument panel image, and the guideline outline angle of the guideline outline is adjusted. Based on this, the measured value indicated by the instrument panel's instructions can be calculated.

Hereinafter, the function of the instrument panel image-based measurement value recognition device 100 according to an embodiment of the present invention described above will be described in detail.

Figure 3 is a block diagram of the instrument panel image-based measurement value recognition device according to an embodiment of the present invention.

Referring to FIG. 3, the instrument panel image-based measurement value recognition device 100 according to an embodiment of the present invention may include an input unit 110, a processor 120, a communication unit 130, and a storage unit 140. .

The input unit 110 may receive an image captured from the instrument panel photographing device 200.

At this time, the input unit 110 may receive a calibration chessboard image from the instrument panel photographing device 200, which is used to set the focal length of the instrument panel photographing device 200 and obtain a distortion correction coefficient.

The processor 120 may obtain a distortion correction coefficient using the above-described chessboard image for calibration.

Here, the distortion correction coefficient may be a coefficient used to correct distortion caused by the optical structure of the dashboard imaging device 200, and the processor 120 is composed of functions and codes executed in the “Open CV” programming library. You can obtain the distortion correction coefficients (k1, k2, p1, p2) by inputting the chessboard image for calibration into the distortion correction coefficient acquisition program.

<Distortion correction coefficient acquisition program>

import numpy as np

import cv2 as cv

import glob

# termination criteria

criteria = (cv.TERM_CRITERIA_EPS + cv.TERM_CRITERIA_MAX_ITER, 30, 0.001)

# prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(6,5,0)

objp = np.zeros((6*7,3), np.float32)

objp[:,:2] = np.mgrid[0:7,0:6].T.reshape(-1,2)

# Arrays to store object points and image points from all the images.

objpoints = [] # 3d point in real world space

imgpoints = [] # 2d points in image plane.

images = glob.glob('*.jpg')

for fname in images:

img = cv.imread(fname)

gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)

# Find the chess board corners

ret, corners = cv.findChessboardCorners(gray, (7,6), None)

# If found, add object points, image points (after refining them)

if ret == True:

objpoints.append(objp)

corners2 = cv.cornerSubPix(gray,corners, (11,11), (-1,-1), criteria)

imgpoints.append(corners)

# Draw and display the corners

cv.drawChessboardCorners(img, (7,6), corners2, ret)

cv.imshow('img', img)

cv.waitKey(500)

cv.destroyAllWindows()

Meanwhile, note that the processor 120 can use various types of distortion correction coefficient acquisition programs as long as it can obtain the distortion correction coefficient used to correct the distortion of the instrument panel photographing device.

Meanwhile, the following will describe the process of performing calibration on the instrument panel (P) of the measuring device (M).

Figure 4 is a calibration instrument panel image used in the instrument panel image-based measurement value recognition device according to an embodiment of the present invention, and Figure 5 is a rotated instrument panel image used in the instrument panel image-based measurement value recognition device according to an embodiment of the present invention. It is an image of the instrument panel for calibration, and FIG. 6 is a correction image of the instrument panel for calibration used in a measurement value recognition device based on the instrument panel image according to an embodiment of the present invention, and FIG. 7 is a measurement value based on the instrument panel image according to an embodiment of the present invention. This is a pre-processed image of the instrument panel for calibration used in recognition devices.

Referring further to FIGS. 4 to 7, the input unit 110 may receive an instrument panel image for calibration, which is used to calibrate the instrument panel P of the measurement device M, from the instrument panel photographing device 200. .

At this time, as shown in FIG. 4, the instrument panel imaging device 200 may photograph the instrument panel P of the measurement device M that is not in operation and generate an instrument panel image for calibration.

As shown in FIG. 5 , the processor 120 may rotate the instrument panel image for calibration based on the installation angle of the instrument panel imaging device 200. Specifically, the processor 120 adjusts the instrument panel for calibration as much as the installation angle at which the instrument panel imaging device 200 is installed based on the straight line connecting the rotation center (R) of the pointer (N) of the instrument panel (P) and the bottom of the instrument panel (P). You can rotate the image.

Through this, the processor 120 can change the instrument panel image for calibration that was not captured at a correct angle into an image at a correct angle.

Thereafter, the processor 120 may correct the distortion of the instrument panel image for calibration and generate a correction image of the instrument panel for calibration.

Specifically, as shown in FIG. 6 , the processor 120 may generate an instrument panel correction image for calibration by correcting one or more of perspective distortion and instrument panel imaging device distortion of the instrument panel image for calibration.

At this time, the processor 120 corrects perspective distortion using the “cv2.warpAffine()” function executed in the “Open CV” programming library and the “cv2.unditor()” function executed in the “Open CV” programming library. The instrument panel imaging device distortion can be corrected using the conversion matrix and distortion correction coefficient.

Thereafter, as shown in FIG. 7 , the processor 120 may perform preprocessing to set information used for calibration on the calibration instrument panel correction image to generate a preprocessed image of the instrument panel for calibration.

Specifically, the processor 120 may perform preprocessing by extracting only a region of interest (ROI) from the driver's instrument panel correction image.

Finally, in the process of performing calibration on the instrument panel (P) of the measurement device (M), the processor 120 calculates the rotation center (R) of the pointer (N) from the calibration instrument panel pre-processing image, as shown in FIG. The minimum measured value guideline angle (Amin), the maximum measured value guideline angle (Amax), the minimum measured value (Vmin) and maximum measured value (Vmax) of the measuring device (M), and the measuring unit (U) of the measuring device (M). It can be recognized and set.

Specifically, the processor 120 recognizes numbers from the calibration instrument panel preprocessing image, sets the smallest number among the recognized numbers as the minimum measured value (Vmin), and sets the largest number among the recognized numbers as the maximum measured value (Vmin). Vmax) can be set.

In addition, the processor 120 sets the angle of a straight line connecting the center position of the number representing the minimum measured value (Vmin) to the rotation center (R) of the pointer (N) as the minimum measured value pointer angle (Amin), and The angle of a straight line connecting the center position of the number representing the value (Vmax) to the rotation center (R) of the pointer (N) can be set as the maximum measured value pointer angle (Amax).

Here, the minimum measured value guideline angle (Amin) and the maximum measured value guideline angle (Amax) are the angles based on the straight line connecting the rotation center (R) of the pointer (N) of the instrument panel (P) and the bottom of the instrument panel (P). It can be.

Additionally, the processor 120 may recognize characters from a pre-processed image of the instrument panel for calibration, and set a character that is the same as a pre-stored measurement unit character among the recognized characters as the measurement unit (U).

Here, the pre-stored measurement unit character may be a character representing a commonly used measurement unit.

Meanwhile, the processor 120 may set the center position of the pre-processed image of the instrument panel as the rotation center (R) of the pointer (N), or may receive coordinate values from the user and set it as the rotation center (R) of the pointer (N). .

Meanwhile, the processor 120 controls the minimum measured value guideline angle (Amin), the maximum measured value guideline angle (Amax), the minimum measured value (Vmin) and the maximum measured value (Vmax) of the measuring device (M), and the measuring device (M). Information corresponding to each measurement unit (U) may be input and set from the user.

Meanwhile, the following will explain the process of recognizing the measurement value measured from the measuring device (M) from the driving instrument panel image taken of the instrument panel (P) of the measuring device (M) in operation.

Figure 8 is a driving instrument panel image used in the instrument panel image-based measurement value recognition device according to an embodiment of the present invention, and Figure 9 is a rotated driving image used in the instrument panel image-based measurement value recognition device according to an embodiment of the present invention. It is an instrument panel image, and FIG. 10 is a correction image of the driving instrument panel used in the instrument panel image-based measurement value recognition device according to an embodiment of the present invention, and FIG. 11 is an instrument panel image-based measurement value recognition device according to an embodiment of the present invention. This is a pre-processing image of the driving instrument panel used, and FIG. 12 is a diagram showing a plurality of straight lines detected from the pre-processing image of the driving instrument panel by the instrument panel image-based measurement value recognition device according to an embodiment of the present invention, and FIGS. 13 and 14 are It is a diagram to explain the process of selecting a guideline outline from a plurality of straight lines by the instrument panel image-based measurement value recognition device according to an embodiment of the present invention, and Figure 15 is a diagram for measurement value recognition based on the instrument panel image according to an embodiment of the present invention. This is a drawing to explain the process by which the device calculates the measured value indicated by the guideline.

Referring further to FIGS. 8 to 15, the input unit 110 inputs the driving instrument panel image in which the instrument panel (P) of the driving measurement device (M) is photographed from the instrument panel imaging device 200, as shown in FIG. 8. You can receive it.

Here, driving means that the measuring device (M) is operating to measure measured values, and when the measuring device (M) is operating, the pointer (N) on the instrument panel (P) indicates the measured value measured in real time. You can.

The processor 120 may perform rotation, distortion correction, and preprocessing on the driving instrument panel image, detect a straight line, and select a straight line corresponding to the outline of the pointer N among the detected straight lines as the pointer outline.

At this time, the process of the processor 120 rotating, correcting distortion, and pre-processing the driving instrument panel image may be similar to the process of rotating, correcting distortion, and pre-processing the image for calibration.

As shown in FIG. 9 , the processor 120 may rotate the driving instrument panel image based on the installation angle of the instrument panel imaging device 200. Specifically, the processor 120 stores the driving instrument panel image as much as the installation angle at which the instrument panel imaging device 200 is installed based on the straight line connecting the rotation center (R) of the pointer (N) of the instrument panel (P) and the bottom of the instrument panel (P). can be rotated.

Through this, the processor 120 can change the driving instrument panel image that was not captured at the correct angle into an image at the correct angle.

Thereafter, the processor 120 may correct the distortion of the driving instrument panel image and generate a correction image of the driving instrument panel.

Specifically, as shown in FIG. 10 , the processor 120 may generate a driver instrument panel correction image by correcting one or more of perspective distortion and instrument panel imaging device distortion of the driver instrument panel image.

At this time, the processor 120 corrects perspective distortion using the “cv2.warpAffine()” function executed in the “Open CV” programming library and the “cv2.unditor()” function executed in the “Open CV” programming library. The instrument panel imaging device distortion can be corrected using the conversion matrix and distortion correction coefficient.

Thereafter, as shown in FIG. 11, the processor 120 may perform preprocessing for straight line detection on the driver's dashboard correction image to generate a driver's dashboard preprocessed image.

Specifically, the processor 120 performs one or more of region-of-interest extraction, which extracts only the region of interest from the dashboard correction image, and color binarization, which binarizes the color information of the driver instrument panel correction image, to convert the driver instrument panel correction image into a pre-processed image of the driver instrument panel. can be created.

Here, color binarization may be a preprocessing process that changes each pixel of the driver's dashboard correction image, which consists of pixels containing a plurality of color information, into one of two color information. For example, color binarization may be a process of changing the color information of each pixel of the driver's dashboard correction image to black or white.

At this time, the processor 120 can perform color binarization using the “cv2.cvtColor()” function executed in the “Open CV” programming library.

As shown in FIG. 12, the processor 120 may detect a plurality of straight lines from the pre-processed image of the driving instrument panel.

Specifically, the processor 120 can detect only straight lines among the outlines that make up the pre-processed image of the driving instrument panel. At this time, the processor 120 extracts the outline from the driving instrument panel preprocessing image by performing Canny transformation through the “cv2.Canny()” function executed in the “Open CV” programming library, and executes the “Open CV” programming library. Multiple straight lines can be detected by performing Hough line transformation through the “cv2.HoughLinesP()” function.

Thereafter, the processor 120 may select an indicator outline corresponding to the outline of the indicator of the instrument panel from among the plurality of straight lines based on one or more of the straight line length, straight line position, and straight line angle of each of the plurality of straight lines.

Specifically, the processor 120 may delete a straight line whose straight line length is less than the reference straight line length among the plurality of straight lines.

That is, the processor 120 can delete straight lines that are not long straight lines, such as the outside of the needle.

At this time, the processor 120 may set the reference straight line length to 5% of the horizontal or vertical length of the pre-processed image of the driving instrument panel.

Meanwhile, as shown in FIG. 13, the processor 120 sets the area less than the first radius from the rotation center of the pointer N as the first area O1, and sets the area less than the first radius from the rotation center of the pointer N to the first radius O1. The area greater than or equal to the second radius may be set as the second area O1.

At this time, the processor 120 may set the first radius to 10% of the horizontal or vertical length of the driving instrument panel preprocessing image.

At this time, the processor 120 may set the second radius to 40% of the horizontal or vertical length of the pre-processed image of the driving instrument panel.

As shown in FIG. 13 , the processor 120 may delete a straight line whose start and end points are not included in the first area O1 and the second area O2 among a plurality of straight lines.

That is, the processor 120 can delete straight lines located on the outside of a plurality of straight lines.

As shown in FIG. 14 , the processor 120 may delete a straight line whose start and end points are included in the first area O1 among a plurality of straight lines.

That is, the processor 120 may delete straight lines cut off within the first area O1, excluding straight lines extending outside the first area O1.

Thereafter, the processor 120 groups the straight lines overlapping the first area O1 among the plurality of straight lines that have not been deleted into a first straight line group, and the straight lines overlapping the second area O2 among the plurality of straight lines into the second straight line group. Can be grouped into straight line groups.

If the straight line angle difference between the straight line angles of each straight line included in the first straight line group and each straight line included in the second straight line group is within the reference angle difference range, the processor 120 may select the straight line as the guideline outline.

That is, if the straight line angle difference between any one of the straight lines included in the first straight line group and the straight line angle of any one of the straight lines included in the second straight line group is within the reference angle difference range, the processor 120 separates the two straight lines. You can select by the guideline outline.

Here, the standard angle difference range may be “-1 degree to 1 degree,” and the straight angle of each straight line is a straight line connecting the rotation center (R) of the pointer (N) of the instrument panel (P) and the bottom of the instrument panel (P). It may be an angle based on .

The processor 120 may classify the guideline outlines into a first guideline outline group and a second guideline outline group based on the guideline outline angle of each guideline outline.

Specifically, the processor 120 may classify the two guideline outlines into the same group if the guideline outline angle difference between the guideline outline angles of any two guideline outlines is within the reference angle difference range.

Thereafter, the processor 120 provides instructions based on the first guideline outline angle of the guideline outline N1 included in the first guideline outline group and the second guideline outline angle of the guideline outline N2 included in the second guideline outline group. The center line angle can be calculated.

At this time, as shown in FIG. 15, the processor 120 calculates the average of the first guideline outline angles of the guideline outlines N1 included in the first guideline outline group as the first guideline outline angle AN1, and the second guideline outline angle AN1. The average of the second guideline outline angles of the guideline outlines (N2) included in the guideline outline group may be calculated as the second guideline outline angle (AN2).

Thereafter, the processor 120 may calculate the average of the guideline outline angles of the first guideline outline angle (AN1) and the second guideline outline angle (AN2) as the guideline center line angle (AN') of the guideline center line (N').

Finally, the processor 120 based on the guideline centerline angle (AN'), the minimum measured value guideline angle (Amin), the maximum measured value guideline angle (Amax), the minimum measured value (Vmin), and the maximum measured value (Vmax). The measured value indicated by the pointer (N) can be calculated.

The processor 120 can calculate the measured value indicated by the pointer N using Equation 1 below.

Here, V is the measured value indicated by the guideline (N), VN' is the guideline center line angle, Amin is the minimum measured value guideline angle, Amax is the maximum measured value guideline angle, Vmin is the minimum measured value, and Vmax is the maximum measured value.

Through this, the processor 120 allows the instrument panel imaging device 200 to be tilted at a certain angle, rather than in an area perpendicular to the front of the instrument panel P, so as not to interfere with the process of the operator visually checking the instrument panel P. Even if a distorted instrument panel image is input when installed at (P), the instrument panel image is corrected to accurately calculate the center line angle formed by the center line of the pointer (N), and the measurement value indicated by the pointer (N) can be calculated with high accuracy. .

Meanwhile, the processor 120 may control the communication unit 130 to transmit the calculated measurement value to the server 300 in real time.

When a request to store measurement values is received from the outside, the server 300 may store the measurement values received from the communication unit 130 of the instrument panel image-based measurement value recognition device 100 in a database according to time series.

Additionally, when a measurement value inquiry request is received from the outside, the server 300 may read the measurement value corresponding to the inquiry condition from the database and transmit the measurement value to the outside.

Here, the inquiry condition may be a condition that matches the measurement time of the measured value and the measurement target device.

Meanwhile, the processor 120 may perform the operation of each of the above-described components, and may include one or more cores (not shown) and a graphics processing unit (not shown) and/or a connection passage for transmitting and receiving signals with other components ( For example, it may include a bus, etc.).

The processor 120 may be configured to perform the operation of each of the components described above by executing one or more instructions stored in the storage unit 140.

The storage unit 140 may store programs (one or more instructions) for processing and controlling the processor 120. Programs stored in the storage unit 140 may be divided into a plurality of modules according to their functions.

So far, the present invention has been examined with a focus on preferred embodiments. A person skilled in the art to which the present invention pertains will understand that the present invention can be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered from an illustrative rather than a restrictive perspective. The scope of the present invention is indicated in the claims rather than the foregoing description, and all differences within the equivalent scope should be construed as being included in the present invention.

As described above, although the present invention has been described with limited examples and drawings, the present invention is not limited thereto, and the technical idea of the present invention and the following will be understood by those skilled in the art to which the present invention pertains. Of course, various modifications and variations are possible within the scope of equivalence of the patent claims to be described.

100: Instrument panel image-based measurement value recognition device
110: input unit
120: processor
130: Department of Communications
140: storage unit
200: Instrument panel photographing device
300: Server

Claims (1)

In a measurement value recognition device that recognizes measured values from a dashboard image captured by a dashboard imaging device that photographs the instrument panel of a measuring device, and corrects instrument panel image distortion and photographing device distortion,
an input unit that receives a driving instrument panel image of the instrument panel of a driving measurement device from the instrument panel photographing device; and
A driving instrument panel correction image is generated by correcting the distortion of the driving instrument panel image, preprocessing for straight line detection is performed on the driving instrument panel correction image to generate a driving instrument panel preprocessing image, and a plurality of straight lines are drawn from the driving instrument panel preprocessing image. Detect, select a guideline outline corresponding to the outline of the guideline of the instrument panel from a plurality of straight lines based on one or more of the straight line length, straight line position, and straight line angle of each of the plurality of straight lines, and based on the guideline outline angle of the guideline outline It includes a processor that calculates the measured value indicated by the guideline,
The processor is
Based on the guideline outline angle of each of the guideline outlines, the guideline outlines are classified into a first guideline outline group and a second guideline outline group, and the first guideline outline angle of the guideline outline included in the first guideline outline group and the second guideline outline group are classified into a first guideline outline group and a second guideline outline group. 2 A guideline center line angle is calculated based on the second guideline outline angle of the guideline outline included in the guideline outline group, and the guideline center line angle, the minimum measured value guideline angle of the measuring device, the maximum measured value guideline angle, the minimum measured value, and Calculate the measured value indicated by the guideline based on the maximum measured value,
The processor is
Generating the driving instrument panel correction image by correcting one or more of perspective distortion and instrument panel imaging device distortion of the driving instrument panel image,
The processor is
Generating the driving instrument panel correction image as a pre-processed image of the driving instrument panel by performing one or more of region of interest extraction for extracting only the region of interest from the driving instrument panel correction image and color binarization for binarizing color information of the driving instrument panel correction image,
The color binarization is
Characterized in that the processor changes the color information of each pixel constituting the driver instrument panel correction image to one of two predetermined color information.
A measurement value recognition device that corrects instrument panel image distortion and imaging device distortion.