KR20240020149A - Artificial intelligence performance inspection system improved in function of acoustic analysis for used cars and processing method thereof and audio-video visualization device for the same - Google Patents

Abstract

The present invention includes the steps of (a) a performance checklist creation module inquiring detailed vehicle model information through the vehicle license plate number of a used car; (b) the performance checklist generating module collecting unstructured data including an acoustic signal detected from a mechanical or electronic device included in the used car using an acoustic sensor; (c) the performance checklist generation module performing an AI performance inspection by diagnosing the collected unstructured data through computer analysis and generating an AI performance checklist by reflecting the resulting AI performance inspection results; and (d) the unstructured data transmission unit provides the AI performance checklist generated by the performance checklist generation module to the customer. Step (c) includes: generating visualization graphics from a result of computer analysis of the unstructured data; synchronizing the visualization graphics with the sound signal to create a video file; A process of outputting a video file on the AI performance checklist is performed, and when the video file is played, a process of displaying an indicator moving along the time axis on the visualization processing graphic is performed, and a process is performed through a speaker in conjunction with the movement of the indicator. Disclosed is a processing method of a used car AI performance inspection system characterized by performing processing to reproduce the corresponding sound.

Description

Used car AI performance inspection system and processing method with improved acoustic analysis function and audio-video visualization device therefor

This application claims priority based on Korean Patent Application No. 10-2022-0098209, filed on August 5, 2022, and all contents disclosed in the specification and drawings of the application are used in this application.

The present invention relates to used car AI performance inspection technology with improved sound analysis function. More specifically, the emotional quality of noise generated when the vehicle's engine is running or while the vehicle is running can be checked online, and an emotional quality index is created and measured. It is about a used car AI performance inspection system and processing method with improved acoustic analysis function that allows you to check the analysis results and supporting data online through computers, AI, etc., and an audio image visualization device for the same.

Generally, when trading used cars, people who want to buy a car visit a large local used car market and choose a car by being introduced to cars for sale there or stopping by to see the cars for sale in person. This was the transaction method, but even in these cases, judgments were often made by looking at the car's car registration, insurance accident history, and performance inspection table. In most cases, if you knew a little about the car, you started the engine yourself, and even did an actual test drive. The reality is that there are almost no cases unless it is an expensive car.

Recently, at a time when online transactions are becoming more active and non-face-to-face transactions are also becoming more active, buying a used car based on the status of a used car that has only been confirmed in writing, at least online, has to be judged based on only a small amount of information, making it an unsatisfactory transaction. Accordingly, since customers who purchase these used cars do not have information about driving or engine condition, etc., unfair transactions take place in a situation of extreme information asymmetry, and when a problem occurs with a car traded in this way, the customer who purchased the car has no information. Based on the asymmetry, people think they have been scammed, and this leads to a climate of distrust in the used car sales industry as a whole. This distrust leads to the traditional desire to help small and medium-sized businesses and small merchants, but instead to trustworthy or branded companies. They are causing harm to both customers and used car industry workers by making them more welcoming of well-known large corporations entering the used car industry.

Regarding the performance condition inspection method and the certificate, which are essential for trading used cars, currently, a professional inspector visually inspects parts for repair, replacement, and leakage of various parts and issues a certificate, but the actual purchase From the customer's perspective, online transactions and non-face-to-face transactions are increasing, and even if they purchase a vehicle online or offline, it is common for them to be unaware of the emotional quality that occurs when the vehicle's engine is running or while the vehicle is driving.

As an alternative, Patent Document 1 can provide more reliable used car price information by calculating the performance inspection cost expected in the future and calculating the used car market price by reflecting the performance inspection warranty insurance for the expected performance inspection cost. We are launching a used car trading system that reflects performance inspection warranty insurance to reduce the burden of used car performance inspection costs on used car buyers through inspection warranty insurance.

However, the used car industry still needs auxiliary means to expand customers' options, and such means are not limited to information delivered on paper, but rather provide direct or indirect experience of the car's driving or the sounds generated by the engine. In this way, an intelligent performance inspection service suitable for the era of the 4th industrial revolution is needed.

As another alternative, Patent Document 2, for which the present applicant has previously applied and been granted a patent, collects and analyzes unstructured data such as sound information generated from used cars, analyzes the normal or abnormal state of the vehicle, and visualizes the resulting performance inspection results. A used car AI performance inspection system provided to customers is presented.

However, the existing intelligent performance inspection service system has the advantage of intuitively checking for abnormalities in automobile parts such as engines through the final result graph, but it provides detailed information on which characteristics are problematic and which are normal. There are limitations in delivering information to customers, so improvements are required.

Patent Document 1: Registered Patent Publication No. 10-1970641 (registered on April 15, 2019) Patent Document 2: Registered Patent Publication No. 10-2305809 (registered on September 17, 2021)

The present invention was created in consideration of the above points, and collects unstructured data such as acoustic signals to analyze the normal or abnormal state of the vehicle and supports the performance inspection results to be intuitively confirmed through vision and hearing. The purpose is to provide a used car AI performance inspection system and processing method with improved acoustic analysis functions.

Another purpose of the present invention is to provide an acoustic analysis function that can provide indicators of emotional quality by classifying acoustic data generated from the driving or engine of a car in detail according to criteria such as frequency bands and analyzing them more precisely and accurately through AI learning. The purpose is to provide an improved used car AI performance inspection system and processing method.

In order to achieve the above object, the present invention includes the steps of (a) a performance checklist creation module inquiring detailed vehicle model information through the vehicle license plate number of a used car; (b) the performance checklist generating module collecting unstructured data including an acoustic signal detected from a mechanical or electronic device included in the used car using an acoustic sensor; (c) the performance checklist generation module performing an AI performance inspection by diagnosing the collected unstructured data through computer analysis and generating an AI performance checklist by reflecting the resulting AI performance inspection results; and (d) the unstructured data transmission unit provides the AI performance checklist generated by the performance checklist generation module to the customer. Step (c) includes: generating visualization graphics from a result of computer analysis of the unstructured data; synchronizing the visualization graphics with the sound signal to create a video file; A process of outputting a video file on the AI performance checklist is performed, and when the video file is played, a process of displaying an indicator moving along the time axis on the visualization processing graphic is performed, and a process is performed through a speaker in conjunction with the movement of the indicator. Provides a processing method for a used car AI performance inspection system, characterized in that it performs processing to reproduce the corresponding sound.

In step (c), the visualization processing graphics processes the sound source file corresponding to the sound signal to generate a three-dimensional image format including a time axis, frequency axis, and intensity axis, divides the time axis into N sections, and divides the time axis into N sections. N image files can be generated by sequentially imaging the indicator for each section from the section to the Nth section, and can be combined in order from the 1st image file to the Nth image file and synchronized with the sound signal.

In step (c), the acoustic signal is divided into frequency bands according to predetermined standards, the state in each frequency band is compared with reference data, scored, and displayed on the AI performance checklist. Perform a segmentation analysis process, , the segmentation analysis process processes the acoustic signal according to criteria including low frequency, midrange, high frequency, regularity, and irregularity. It is subdivided, the status of each area is scored, and displayed on the AI performance checklist, and step (c) is for each of the low frequency, midrange, high frequency, regularity, and irregularity areas in the segmentation analysis process. The scores and total scores can be visualized in tables and polygonal graphs and displayed on the AI performance checklist, and in the segmentation analysis process, each area can be decomposed into a number of characteristic elements, analyzed, and scored.

According to another aspect of the present invention, detailed vehicle model information is searched through the vehicle license plate number of the used car, and unstructured data including sound generated from the mechanical or electronic device of the used car is collected using an acoustic sensor, and the collected unstructured data is collected. A performance checklist creation module that diagnoses data through computer analysis, performs an AI performance inspection, and generates an AI performance checklist by reflecting the resulting AI performance inspection results; And providing the AI performance checklist generated by the performance checklist generation module to the customer. An unstructured data transmission unit; and the performance checklist creation module includes: a vehicle model information inquiry unit that obtains the license plate number of the used car and searches vehicle model information; an acoustic signal acquisition unit that collects acoustic signals generated from mechanical or electronic devices of the used car; An acoustic information pre-processing unit that digitizes the collected acoustic signals to generate acoustic data; Vehicle model information DB that stores standard vehicle sound data; a vehicle type information mapping unit that maps the sound data with the vehicle type information DB to detect abnormal signs of the vehicle; an acoustic data analyzer that analyzes the acoustic data to score characteristics and conditions of the vehicle; A performance information report processor that generates an abnormality report when an abnormality is found in the sound data and generates a normal report when there is no abnormality and reflects it in the AI performance inspection results; And an unstructured data visualization processing unit that generates visualization graphics from the results of computer analysis of the unstructured data and displays the graphics on an AI performance checklist, wherein the unstructured data visualization processing unit generates visualization graphics for acoustic signals. An audio-visual visualization device comprising a processing unit and a synchronization processing unit that synchronizes the visualization graphics generated by the graphics processing unit with the sound signal to create a video file, wherein the audio-video visualization device is configured to: A used car AI performance inspection system is provided, characterized in that it performs a function of displaying an indicator moving along the time axis on the visualization processing graphic and a function of playing the corresponding sound through a speaker in conjunction with the movement of the indicator.

The audio-visual visualization device processes a sound source file corresponding to the sound signal to generate the visualization graphics having a three-dimensional image format including a time axis, frequency axis, and intensity axis, and divides the time axis into N sections. N image files can be generated by sequentially imaging the indicator for each section from the 1st section to the Nth section, and the 1st image file to the Nth image file can be combined in order and synchronized with the sound signal.

The acoustic data analyzer includes a region-specific characteristic value DB that stores characteristic information of acoustic data by characteristic; A reference data characteristic value DB that stores characteristic values of acoustic data of a specific vehicle or standard vehicle for each vehicle type, which serves as a standard for analyzing acoustic data; A frequency domain extraction unit that removes noise from the input acoustic data and extracts the time interval required for normalization processing and analysis; a characteristic information processing unit that separates the extracted time sections according to criteria including low frequency, midrange, high frequency, regularity, and irregularity; A characteristic value storage processor for each frequency domain that decomposes each separated region into a plurality of characteristic elements and stores characteristic information of the acoustic data for each characteristic in the region-specific characteristic value DB; A comparative deviation processing unit that analyzes the deviation between the characteristic value of the reference data and the characteristic value to be analyzed; and a deviation score visualization processing unit that scores the analyzed deviations, calculates a score appropriate for each characteristic deviation, and visualizes and displays the resulting calculation results in a score table and a polygon graph.

According to another aspect of the present invention, a graphics processing unit that generates visualization processing graphics for acoustic signals; And a synchronization processing unit that synchronizes the visualization graphics generated by the graphics processing unit with the sound signal to create a video file, wherein the synchronization processing unit moves along the time axis on the visualization graphics when playing the video file. It can perform the function of displaying an indicator and playing the corresponding sound through a speaker in conjunction with the movement of the indicator.

The synchronization processing unit processes the sound source file corresponding to the sound signal to generate the visualization processing graphics to have a three-dimensional image format including a time axis, frequency axis, and intensity axis, divides the time axis into N sections, and divides the time axis into N sections. N image files can be generated by sequentially imaging the indicator for each section from the Nth section to the Nth section, and can be combined in order from the 1st image file to the Nth image file and synchronized with the sound signal.

The used car AI performance inspection system and processing method with improved sound analysis function according to the present invention has the following effects.

First, by generating a visualization graphic (e.g., a spectrogram) from the results of computer analysis of the acoustic signal, synchronizing it with the acoustic signal, and compounding it, the analysis results can be intuitively understood by the user.

Second, automobile parts are organically connected to each other and move in different cycles. Existing used car performance inspection technologies based on acoustic data analysis record the sound of each part and check whether the sound is heard. However, due to the nature of mechanical parts, the same results cannot be obtained depending on errors and wear during production, so this approach is used. There is a problem with not being able to accurately diagnose the condition of the car. On the other hand, the present invention does not find the natural frequency of individual components, but separates them by specific frequency bands, analyzes periodic and aperiodic patterns within them, learns them, and uses them for diagnosis, thereby achieving detailed performance inspection and reliability of inspection data. can be improved.

Third, by accumulating error discovery data by vehicle model information, similar failure and maintenance factors caused by similar parts can be identified in advance, and detailed maintenance information can be provided to accumulate detailed failure and maintenance history for the vehicle.

Fourth, the AI performance inspection results are visualized and displayed on the customer terminal through a user interface (UI) that compares the normal performance state and the deteriorated performance state in a graph, allowing the customer to intuitively check the normal/abnormal state of the car. there is. A more remarkable effect can be achieved by synchronizing and linking the acoustic signal and the spectrogram.

Fifth, the AI performance inspection results provided to customers become information for additional judgment by customers to help resolve information asymmetry when trading used cars. Through this, customer trust is broken and the home of small and medium-sized merchants is industrialized by large corporations. The 4th industrialization becomes possible by restoring customers' trust in quality and further revitalizing it through progress in online transactions.

Sixth, as the intelligent performance inspection system and its services spread into the daily use area of cars through parking lot exit gates, etc., car owners will be able to estimate the remaining value of their car as a used car, so that the car can be operated, maintained, and maintained. Systematized information can be linked to decision-making, such as buying and selling, making it possible to utilize AI-recommended maintenance times and sales points in the era of self-driving cars.

The following drawings attached to this specification illustrate preferred embodiments of the present invention, and serve to further understand the technical idea of the present invention together with the detailed description of the invention described later, so the present invention includes the matters described in such drawings. It should not be interpreted as limited to only .
Figure 1 is a block diagram showing the configuration of a used car AI performance inspection system according to a preferred embodiment of the present invention.
FIG. 2 is a block diagram showing the configuration of the performance checklist creation module in FIG. 1.
FIG. 3 is a block diagram showing the configuration of the acoustic data analyzer in FIG. 2.
Figure 4 is a procedure diagram showing the process of performing a used car AI performance inspection processing method according to a preferred embodiment of the present invention.
5 to 7 are tables illustrating sound information and vehicle type information processed by the used car AI performance inspection system according to a preferred embodiment of the present invention.
Figure 8 is a spectrogram comparing the normal state and abnormal state of a gasoline engine.
Figures 9 to 12 are spectrograms showing examples of visual processing of AI performance inspection results according to the present invention.
Figure 13 is a computer capture screen illustrating the AI performance checklist created according to the present invention.
FIG. 14 is a block diagram showing the configuration of an audio-video visualization device included in the unstructured data visualization processing unit in FIG. 2.
FIG. 15 is a computer capture screen showing an example of generating N image files by imaging an indicator in the audio-video visualization device of FIG. 14.
FIGS. 16 to 18 are computer capture screens showing an example of playing a video generated by the audio-video visualization device of FIG. 14 using a video player.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

Figure 1 is a block diagram showing the configuration of a used car AI performance inspection system according to a preferred embodiment of the present invention.

Referring to Figure 1, the used car AI performance inspection system 204 according to a preferred embodiment of the present invention includes an online performance checklist request receiver 202, an online performance checklist output unit and access rights transmission unit 206, and an unstructured data request receiver. (207), an unstructured data transmission unit 208, and a performance checklist creation module 102. In addition, the used car AI performance inspection system 204 is for on-site visiting customers (car buyers) 101 and includes a performance checklist request receiver 202 and a performance checklist document output unit 203.

The used car AI performance inspection system 204 includes a structured data DB 209 that stores text data that fits into the area of a normal automobile performance checklist, and an access rights DB that stores data to provide performance inspection results only to approved users. It is configured to include or be linked with (210) and an unstructured data DB 211 that stores unstructured data generated from mechanical or electronic devices of the car through interior/exterior photography, sound, images, and sensors of the used car.

The performance checklist creation module 102 searches detailed car model information through the vehicle license plate number of the used car, and collects unstructured data generated from the car's mechanical or electronic devices through interior/exterior photography, sound, video, and sensors of the used car. AI performance inspection is performed to check vehicle performance by collecting and diagnosing the collected unstructured data through computer analysis.

As shown in FIG. 2, the performance checklist generation module 102 includes an acoustic signal acquisition unit 103, a vehicle model information inquiry unit 104, an acoustic information pre-processing unit 105, a vehicle model information mapping unit 106, and acoustic data. It includes an analyzer 107, an unstructured data visualization processing unit 108, a vehicle model information DB 111, and a performance information report processing unit 109.

The sound signal acquisition unit 103 collects sound signals generated from mechanical and electronic devices of the vehicle.

The collection of acoustic signals to identify abnormal signs in a car is performed by an acoustic sensor such as a microphone installed in the passenger seat, engine room, lower part of the car, or on the outside of the car. Since various mechanical mechanisms such as wheels and drivetrains are combined within a car, it is effective to place them in areas where various internal and external forces act. When installed outside a car, it can be installed at the bottom or side of the entrance to places where it is easy to provide services to cars (parking lots, gas stations, car washes, maintenance shops, drive-in cafes, etc.), including one side of a used car dealer's facility. . In such places, it is advantageous for acoustic sensors such as microphones to detect frequencies exceeding the audible frequency (20 to 20,000 Hz) to detect mechanical abnormalities. Acoustic sensors collect mechanical sounds when the vehicle moves at a constant speed or is stationary. The acoustic signals collected in this way are analyzed through an analysis computer connected directly or via communication to find unusual frequency patterns.

Recognition of vehicle license plate numbers is performed through cameras installed at the service provider's gate facilities. If the vehicle number is matched with the detailed vehicle type information of the vehicle within the system or through an external service company and created into a database, abnormal signs applicable only to specific vehicles that entered and exited the vehicle can be grouped and determined by vehicle type to recognize and identify abnormal symptoms according to the characteristics of the vehicle type. It can be used as targeted data by determining whether it is about a car.

The vehicle model information search unit 104 obtains the vehicle license plate number of the car and searches the vehicle model information. The vehicle model information is obtained from the additional information provider or public information API from the license plate number recognized through the camera installed at the gate facility.

In other words, the detailed model information of the car in question can be obtained, for example, by reading the license plate number '12-ga 1234' and then reading 'Vehicle identification number (VIN) = KN12345678A123456; Brand=Hyundai; Model = Grandeur IG; trim=3.0; Gasoline; Details Trim = Exclusive Special; mission=auto; Drive = front wheel; Model=2017 model; Production = February 2019; Format name=HG4EBK-G; Riding Capacity=5; Motor type=G6DG; displacement=2999; body length=4920; bodywidth=1860; Airbag = Advanced; option=aroundview; tires=19 inches; Obtain detailed classification information such as ‘Sunroof=Y’.

The acoustic information pre-processing unit 105 digitizes the collected acoustic signals and generates acoustic data. Specifically, the acoustic information pre-processing unit 105 processes the collected acoustic signals through Fourier transformation and noise removal, as shown in FIGS. 4A to 4C, to generate acoustic data consisting of frequency components generated from the vehicle's mechanical devices.

The vehicle type information mapping unit 106 maps sound data with the vehicle type information DB 111, which stores various error case information for each vehicle type, to detect abnormal signs of the vehicle.

The acoustic data analyzer 107 analyzes the pattern of acoustic information for the acoustic data generated by the acoustic information pre-processing unit 105 and detects abnormal symptom events 401 to 404 of the vehicle. In this regard, Figures 4 to 6 schematically show a pattern analysis process of acoustic information.

The acoustic data analyzer 107 is a module that analyzes and processes the characteristics of time series acoustic data. As shown in FIG. 3, the acoustic data analyzer 107 includes a normalization and analysis section extraction unit 107a, a frequency (high/middle/low/audible) region extraction unit 107b, a characteristic information processing unit 107c, and a frequency It includes a feature value storage processor for each region (107d), a feature value DB for each region (107e), a comparison deviation processor (107f), a deviation score visualization processor (107g), and a reference data feature value DB (107h).

The extraction unit 107b for each frequency (high/middle/low/audible) region removes noise from the input sound data and extracts the time interval (usually in seconds) required for normalization processing and analysis.

The characteristic information processing unit 107c separates the extracted time section (e.g., 4-second section data) by frequency domain into low frequency, midrange, high frequency, and entire audible frequency domain. Here, each frequency range can be varied in various ways depending on the characteristics of the acoustic data and the information to be found.

The characteristic value storage processing unit 107d for each frequency domain decomposes each of the separated regions into a plurality of characteristic elements (for example, about 40 to 70) and stores the characteristic information of the sound data for each characteristic in the characteristic value DB 107e for each region. Save and process. In addition, the characteristic value storage processing unit 107d for each frequency domain stores the total value analyzed and scored for each region in the characteristic value DB 107e for each region. In other words, the data separated by area contains information on the characteristics of the frequency of a specific area during a specific time period, where various characteristics of the sound data within the corresponding section (high, low, upward, downward, periodicity, and non-periodicity of the frequency) etc.) are extracted by the frequency domain characteristic value storage processing unit 107d and stored in the domain characteristic value DB 107e.

The region-specific characteristic value DB 107e is a database in which characteristic values of acoustic data to be analyzed are stored.

The comparison deviation processing unit 107f analyzes the deviation between the characteristic value of the reference data and the characteristic value to be analyzed.

The deviation score visualization processing unit 107g scores the analyzed deviations, calculates a score appropriate for each characteristic deviation, and displays the corresponding score table (see 1 in FIG. 13) as well as a polygon-shaped graph (see 2 in FIG. 13). ) is visualized and displayed.

The reference data characteristic value DB 107h is a database in which characteristic values of sound data of a specific standard vehicle (or vehicle type) are stored.

Each part of a car generates sound in a different frequency band. As described above, it is possible to determine whether the relevant part is normal or abnormal by subdividing the frequency band and measuring periodic activity. For example, Figure 8 shows a spectrogram for a normal state of a gasoline engine (a) and a state in which only three cylinders are operating due to a spark plug failure (b). Referring to (b) of FIG. 8, it can be seen that an abnormal pattern appears in the frequency band of approximately 10,000 to 13,000 Hz. Therefore, by analyzing the characteristic information of this section, the abnormal state of the relevant engine can be efficiently determined.

The unstructured data visualization processing unit 108 visualizes the AI performance inspection results and generates a vehicle performance inspection graph. The AI performance check results may include acoustic waveform graphs, images, tables, etc. used in the pattern analysis process of acoustic information as shown in FIGS. 4 to 6. More preferably, the unstructured data visualization processing unit 108 displays the AI performance inspection results at the customer terminal through a user interface (UI) that compares the normal performance state and the deteriorated performance state in a graph, as shown in 4 of FIG. 13. Perform visualization processing on the AI performance inspection result data so that it is displayed in . In this way, by displaying both the performance degradation state and the normal state graph, customers can intuitively understand whether performance is degraded. Specifically, the unstructured data visualization processing unit 108 generates a spectrogram that represents the change in amplitude over time (horizontal axis) and frequency (vertical axis) of the acoustic signal in terms of density or color difference. In addition, the unstructured data visualization processing unit 108 displays the AI performance inspection result spectrogram obtained through various analyzes on the AI performance checklist, as shown in FIGS. 9 to 12.

The performance information report processing unit 109 generates an abnormality report when an abnormality is found in the acoustic information, and when there is no abnormality, it generates a normal report and reflects it in the AI performance inspection results. The abnormal symptom report or normal report generated by the performance information report processing unit 109 is stored in the vehicle model information DB 111.

The performance checklist creation module 102 is equipped with an SNS error reporting processing unit 102i to increase database reliability of the vehicle model information DB 111 and enable rapid data updates. When an error for a specific vehicle model is reported through SNS such as Internet news or Twitter, the SNS error reporting processing unit 102i analyzes the error details and reports the vehicle model information DB 111 when it is determined that the vehicle model has a defect. Update the error data by reflecting it.

The unstructured data transmission unit 208 provides the AI performance check results generated by the performance checklist creation module 102 to the customer along with the basic performance check results. At this time, the AI performance check result includes visualized information generated by the unstructured data visualization processing unit 108.

Preferably, the used car AI performance inspection system 204 provides resume service for AI performance inspection even after the sale of the used car is completed.

Figure 4 is a flowchart showing a service method performed by a used car AI performance inspection system according to a preferred embodiment of the present invention.

As shown in Figure 4, the intelligent performance inspection service method according to a preferred embodiment of the present invention includes a car seller 100, an intelligent performance inspection service system 204, an on-site car buyer 101, and an online car. It is performed through interaction between buyers 101'.

First, when a used vehicle for sale is registered and received by the automobile seller 100 (step S104 and step S105), the intelligent performance inspection service system 204 is installed within the vehicle after the car wash of the sales vehicle is completed (step S106). / Complete the performance check by sequentially performing the processes of external photo taking (step S107), acoustic information collection (step S108), basic performance check (step S109), and AI performance check (step S110) (step S111).

Thereafter, when a performance checklist request is received from an on-site car buyer 101 (step S112) or a performance checklist request is received from an online car buyer 101' (step S113), the intelligent performance check service system 204 First, a basic performance checklist is issued and provided to the on-site car buyer (101) or online car buyer (101') (step S114). At this time, the performance checklist document output unit 203 provided in the intelligent performance inspection service system 204 prints the performance checklist 115, 116 on which documents and/or photos related to the basic performance information of the sold vehicle are posted offline or online. to provide.

Additionally, an AI performance checklist 118 is issued and transmitted to the online car buyer 101' (step S117). The AI performance checklist 118 is issued by reflecting structured data in the form of documents or photos as well as unstructured performance inspection data such as sound and video unique to the vehicle being sold.

Specifically, the intelligent performance inspection service system 204 performs preprocessing of Fourier transform and noise removal on the acoustic information signal obtained from the vehicle sold after requesting vehicle model information, and performs pattern analysis through computer learning. Analyze periodic, non-periodic, and unusual information. In addition, the intelligent performance inspection service system 204 maps the analyzed information with the searched vehicle model information and the error history for each vehicle type (user information such as user information from its own DB, public API or SNS, etc.) held by itself, to prevent the occurrence of the error. Evaluate and analyze whether the problem is limited to the current vehicle or a problem depending on the characteristics of each vehicle type. The analyzed reporting information is reflected when issuing the AI performance checklist.

The intelligent performance inspection service system 204 visualizes the AI performance inspection results through the performance checklist creation module 102 and generates a vehicle performance inspection graph. The AI performance inspection results reflect the acoustic waveform graphs, images, tables, etc. used in the pattern analysis process of acoustic information. The performance checklist creation module 102 compares and displays the normal performance state and the deteriorated performance state in a graph according to the AI performance inspection results. In addition, the performance checklist generation module 102 subdivides the acoustic signal collected by the acoustic signal acquisition unit 103 into at least a low-frequency band, a mid-range region, and a high-frequency band, and scores the status of each region to determine AI performance. Mark it on the checklist. More preferably, the intelligent performance inspection service system 204 subdivides the acoustic signal into low-frequency band, mid-range, high-frequency band, regular and irregular region, etc., as shown in FIG. 13, and reports the status for each region. can be scored and displayed on the AI performance checklist.

Referring again to FIG. 2, the unstructured data visualization processing unit 108 processes the results of computer analysis of unstructured data to generate visualization graphics (e.g., spectrograms) and synchronizes them with sound signals to create a video, thereby processing the visualization. It includes an audio-visual visualization device 108' that has the function of further enhancing understanding of graphics. The video file generated by the audio-visual visualization device 108' can be output on an AI performance checklist and provided to the customer.

As shown in FIG. 14, the audio-visual visualization device 108' includes a graphics processing unit 108a that generates visualization processing graphics for an acoustic signal, and the visualization processing graphics generated by the graphics processing unit 108a into an acoustic signal. It includes a synchronization processing unit 108b that synchronizes with and creates a video file.

The graphics processing unit 108a processes sound source files corresponding to sound signals and generates visualization graphics with a three-dimensional image format including a time axis, frequency axis, and intensity axis. Preferably, the visualization processing graphic may be a spectrogram that represents changes in amplitude with respect to time (horizontal axis) and frequency (vertical axis) of an acoustic signal in terms of density or color difference, as described above.

Additionally, the graphics processing unit 108a generates image files including an indicator of a predetermined shape that passes through the time axis or is placed on the time axis on the generated visualization processing graphics. Preferably, the indicator may be a linear leader in the form of a vertical line. According to a modified example of the present invention, the indicator may be configured in the form of an icon of a predetermined shape. In Figure 15, the graphics processing unit 108a divides the time axis of the visualization processing graphics into N sections and sequentially images indicators in the form of red leader lines for each section from the 1st section to the Nth section to generate N image files. An example is shown. That is, each of the N image files includes an indicator, and each indicator is imaged at a different position on the time axis.

The synchronization processing unit 108b combines the first image file to the Nth image file in order and synchronizes them with the sound signal, thereby substantially converting the visualization processing graphics into a video file. That is, the synchronization processing unit 108b performs the function of displaying an indicator moving along the time axis on the visualization processing graphics when playing a video file and playing the corresponding sound through a speaker in conjunction with the movement of the indicator.

Preferably, the audio-image visualization device 108' has the graphics processing unit 108a convert the sound source file (mp3, wav format, etc.) of the sound signal and visualize it in the form of a three-dimensional (e.g. time, frequency, intensity) image. The process of generating graphics and the process of dividing sections by calculating the position of the The axis is divided by [in the case of 60 frames, 60 frames x 5 seconds = 300 sections]) and the process of imaging the leader line according to the divided section of the . After the graphic processing unit 108a creates a group of image files containing 300 instruction lines, the synchronization processing unit 108b performs a process of combining each image file in order to form one frame and producing an image.

As shown in Figures 16 to 18, the audio-video visualization device synchronizes sound signals and visualization processing graphics and provides a video file in which an indicator in the form of a leader line moves along the time axis in conjunction with the reproduction of the sound signal. Figures 16 to 18 show playing a video file using GOM Player ^TM , a known video player, and capturing the computer screen at random time intervals.

According to another aspect of the present invention, there is a procedure for inquiring detailed vehicle model information through the license plate number of a used car, and atypical information generated from the mechanical or electronic devices of the car through interior/exterior photography of the used car and sound, video, and sensors. The procedure for collecting data, and the collected unstructured data are diagnosed through computer analysis to check performance. If abnormalities are found in the acoustic information, an abnormality report is generated, and if there are no abnormalities, a normal report is generated. The AI performance inspection procedure is reflected in the AI performance inspection results, and the AI performance inspection results are visualized to show changes in time, frequency, and amplitude of the acoustic signals in the normal performance state and deteriorated performance state of the used car, as well as differences in density or color. A recording medium (e.g., magnetic recording medium, CD-ROM, flash memory, etc.) is provided that records a program that can execute a visualization process on a computer by generating a spectrogram indicated by and displaying it on the AI performance checklist.

According to another aspect of the present invention, a procedure for processing a sound source file corresponding to an acoustic signal to generate visualization processing graphics having a three-dimensional image format including a time axis, frequency axis, and intensity axis, and dividing the time axis into N sections. A procedure for generating N image files by dividing and sequentially imaging indicators for each section from the 1st section to the Nth section, and a procedure for combining them in order from the 1st image file to the Nth image file and synchronizing them with the audio signal. A recording medium (eg, magnetic recording medium, CD-ROM, flash memory, etc.) recording a program that can be executed on a computer is provided.

According to the present invention, which has the above-described configuration, when buying or selling a car, not only the basic performance check of the car, but also information suitable for additional mechanical and electronic device characteristics are collected and the AI performance check result report is provided to the customer offline or online. can be provided. According to the present invention, the analysis results can be intuitively understood by the user by generating visualization graphics (e.g., a spectrogram) from the results of computer analysis of the acoustic signal, synchronizing them with the acoustic signal, and compounding them. In addition, there is a significant effect of obtaining detailed AI performance inspection results by dividing into specific frequency bands, analyzing periodic and aperiodic patterns occurring within them, learning them, and using them for diagnosis. In addition, by providing even the original collected data to authorized customers, it is possible to resolve the extreme information asymmetry that occurs when information about the vehicle for sale is not accurately conveyed to the buyer, and provides emotional quality for purchasing a vehicle online. It is possible to promote the online use of used car transactions by allowing indirect judgment.

Although the present invention has been described above with limited examples and drawings, the present invention is not limited thereto, and the technical idea of the present invention and the description below will be understood by those skilled in the art to which the present invention pertains. Of course, various modifications and variations are possible within the scope of equivalence of the patent claims.

102: Performance checklist creation module 107: Acoustic data analyzer
108: Unstructured data visualization processing unit 108': Audio-video visualization device
108a: graphics processing unit 108b: synchronization processing unit
202: Performance checklist request receiving unit 203: Performance checklist document output unit
204: Intelligent performance inspection service system 205: Online performance inspection list request reception unit
206: Online performance checklist output unit and access rights transmission unit
207: Unstructured data request receiver 208: Unstructured data transmission unit
209: Structured data DB 210: Access rights DB
211: Unstructured data DB

Claims (8)

(a) a step in which the performance checklist creation module searches detailed vehicle model information through the vehicle license plate number of a used car;
(b) the performance checklist generating module collecting unstructured data including an acoustic signal detected from a mechanical or electronic device included in the used car using an acoustic sensor;
(c) the performance checklist generation module performing an AI performance inspection by diagnosing the collected unstructured data through computer analysis and generating an AI performance checklist by reflecting the resulting AI performance inspection results; and
(d) The unstructured data transmission unit provides the AI performance checklist generated by the performance checklist generation module to the customer. Including steps;
In step (c),
A process of generating visualization graphics from the results of computer analysis of the unstructured data;
A process of synchronizing the visualization graphics with the sound signal to create a video file;
Perform the process of outputting the video file on the AI performance checklist,
When playing the above video file,
A processing method of a used car AI performance inspection system, characterized in that the process of displaying an indicator moving along the time axis on the visualization processing graphic and the process of playing the corresponding sound through a speaker in conjunction with the movement of the indicator. The method of claim 1, wherein step (c) is:
The visualization processing graphic processes the sound source file corresponding to the sound signal and generates a three-dimensional image format including a time axis, frequency axis, and intensity axis,
Dividing the time axis into N sections and sequentially imaging the indicator for each section from the 1st section to the Nth section to generate N image files,
A processing method of a used car AI performance inspection system, characterized in that the first image file to the Nth image file is combined in order and synchronized with the acoustic signal. The method of claim 2, wherein step (c) is:
Perform a segmentation analysis process to classify the acoustic signal into frequency bands according to predetermined standards, score the status in each frequency band by comparing it with reference data, and display it on the AI performance checklist,
The segmentation analysis process processes the acoustic signal according to criteria including low frequency, midrange, high frequency, regularity, and irregularity. Subdivide and score the status of each area and display it on the AI performance checklist,
In step (c),
In the segmentation analysis process, the scores and total scores for each of the low-frequency, mid-range, high-frequency, regularity, and irregularity areas are visualized in a table and polygon graph and displayed on the AI performance checklist,
A used car AI performance inspection processing method, characterized in that in the segmentation analysis process, each area is decomposed into a plurality of characteristic elements, analyzed, and scored. Detailed vehicle model information is searched through the used car's license plate number, and unstructured data including sounds generated from the mechanical or electronic devices of the used car are collected using an acoustic sensor, and the collected unstructured data is diagnosed through computer analysis. A performance checklist creation module that performs an AI performance inspection and generates an AI performance checklist by reflecting the resulting AI performance inspection results; and
Providing customers with the AI performance checklist generated by the performance checklist generation module Includes an unstructured data transmission unit,
The performance checklist creation module is,
A vehicle type information search unit that obtains the vehicle number of the used car and searches vehicle type information;
a sound signal acquisition unit that collects sound signals generated from mechanical or electronic devices of the used car;
An acoustic information pre-processing unit that digitizes the collected acoustic signals to generate acoustic data;
Vehicle model information DB that stores standard vehicle sound data;
a vehicle type information mapping unit that maps the sound data with the vehicle type information DB to detect abnormal signs of the vehicle;
an acoustic data analyzer that analyzes the acoustic data to score characteristics and conditions of the vehicle;
A performance information report processor that generates an abnormality report when an abnormality is found in the sound data and generates a normal report when there is no abnormality and reflects it in the AI performance inspection results; and
An unstructured data visualization processing unit that generates visualization graphics from the results of computer analysis of the unstructured data and displays them on an AI performance checklist,
The unstructured data visualization processing unit,
An audio-visual visualization device having a graphics processing unit that generates visualization graphics for sound signals and a synchronization processing unit that synchronizes the visualization graphics generated by the graphics processing unit with the sound signals to create a video file,
The audio-visual visualization device,
A used car AI performance inspection system that performs a function of displaying an indicator moving along the time axis on the visualization graphics when playing the video file and a function of playing the corresponding sound through a speaker in conjunction with the movement of the indicator. . The method of claim 4, wherein the audio-visual visualization device,
Processing a sound source file corresponding to the sound signal to generate the visualization processing graphic having a three-dimensional image format including a time axis, frequency axis, and intensity axis,
Dividing the time axis into N sections and sequentially imaging the indicator for each section from the 1st section to the Nth section to generate N image files,
A used car AI performance inspection system that combines the first image file to the Nth image file in order and synchronizes them with the acoustic signal. The method of claim 4, wherein the acoustic data analyzer,
A characteristic value DB for each area that stores characteristic information of sound data by characteristic;
A reference data characteristic value DB that stores characteristic values of acoustic data of a specific vehicle or standard vehicle for each vehicle type, which serves as a standard for analyzing acoustic data;
A frequency domain extraction unit that removes noise from the input acoustic data and extracts the time interval required for normalization processing and analysis;
a characteristic information processing unit that separates the extracted time sections according to criteria including low frequency, midrange, high frequency, regularity, and irregularity;
A characteristic value storage processor for each frequency domain that decomposes each separated region into a plurality of characteristic elements and stores characteristic information of the acoustic data for each characteristic in the region-specific characteristic value DB;
A comparative deviation processing unit that analyzes the deviation between the characteristic value of the reference data and the characteristic value to be analyzed; and
A used car AI performance inspection system comprising a deviation score visualization processing unit that scores the analyzed deviations, calculates a score appropriate for each characteristic deviation, and visualizes and displays the resulting calculation results in a score table and polygon graph. a graphics processing unit that generates visualization processing graphics for acoustic signals; and
A synchronization processing unit that synchronizes the visualization graphics generated by the graphics processing unit with the sound signal to create a video file,
The synchronization processing unit,
An audio-visual visualization device that performs a function of displaying an indicator moving along the time axis on the visualization graphics when playing the video file and a function of playing the corresponding sound through a speaker in conjunction with the movement of the indicator. . The method of claim 7, wherein the synchronization processing unit,
Processing a sound source file corresponding to the sound signal to generate the visualization processing graphics to have a three-dimensional image format including a time axis, frequency axis, and intensity axis,
Dividing the time axis into N sections and sequentially imaging the indicator for each section from the 1st section to the Nth section to generate N image files,
An audio-visual visualization device that combines the first image file to the Nth image file in order and synchronizes them with the sound signal.