CN116055349A - Method, device and equipment for detecting stream data - Google Patents

Method, device and equipment for detecting stream data Download PDF

Info

Publication number
CN116055349A
CN116055349A CN202211725891.8A CN202211725891A CN116055349A CN 116055349 A CN116055349 A CN 116055349A CN 202211725891 A CN202211725891 A CN 202211725891A CN 116055349 A CN116055349 A CN 116055349A
Authority
CN
China
Prior art keywords
tested
streaming data
assertion
data
point data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211725891.8A
Other languages
Chinese (zh)
Inventor
薛野
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yunkong Zhixing Technology Co Ltd
Original Assignee
Yunkong Zhixing Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yunkong Zhixing Technology Co Ltd filed Critical Yunkong Zhixing Technology Co Ltd
Priority to CN202211725891.8A priority Critical patent/CN116055349A/en
Publication of CN116055349A publication Critical patent/CN116055349A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/08Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/08Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0852Delays
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/10Active monitoring, e.g. heartbeat, ping or trace-route
    • H04L43/106Active monitoring, e.g. heartbeat, ping or trace-route using time related information in packets, e.g. by adding timestamps
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/50Testing arrangements

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Environmental & Geological Engineering (AREA)
  • Health & Medical Sciences (AREA)
  • Cardiology (AREA)
  • General Health & Medical Sciences (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The embodiment of the specification discloses a method, a device and equipment for detecting stream data. The method comprises the following steps: the tested program sends the to-be-tested streaming data generated in the first time period of the tested program to the kafka message middleware server; the streaming data to be tested comprises start tangential point data and end tangential point data; the kafka message middleware server distributes the streaming data to be tested to a test server; the test server detects whether the starting tangent point data meets the pre-assertion or not to obtain a first detection result; the test server detects whether the ending tangent point data meets post-assertion or not to obtain a second detection result; and if the first detection result indicates that the start tangent point data meets the pre-assertion, and if the second detection result indicates that the end tangent point data meets the post-assertion, determining that the streaming data to be tested passes detection.

Description

Method, device and equipment for detecting stream data
Technical Field
The present invention relates to the field of data detection, and in particular, to a method, an apparatus, and a device for detecting streaming data.
Background
Streaming data is a set of sequential, massive, fast, continuously arriving data sequences, which can generally be regarded as a dynamic data set that continues over time with an unlimited growth, spanning between hundreds of milliseconds and seconds.
In the technical field of automatic driving, automatic driving data are transmitted in a streaming data mode, real-time dynamic of an automatic driving vehicle in the driving process can be completely recorded only in a streaming data mode, a tested program can send a message to a middleware server every 100 milliseconds, a large amount of data can be generated in the running process of the automatic driving vehicle, and the large amount of data are required to be detected for better controlling the automatic driving vehicle.
At present, program detection mainly uses white box test, in the automatic test process, the output of function type calculation is checked by using a mode of assertion, the function type method is called to output only one result at a time, the assertion method does not need to perform continuous detection, but the data of stream data is continuous stream, the correctness of the data stream cannot be continuously detected in real time by common assertion of stream data, and the detection efficiency is low in the prior art only through manual detection for sequential detection and time delay detection of stream data.
In the technical field of automatic driving, data exist in a streaming data type, streaming data are output in a streaming form after a series of calculation of a program, an intermediate state in a program processing process can be printed through log4j (an open source project of Apache) and other log technologies, then manual comparison is carried out on data which are expected to be checked, or logs are checked on a kibana (a data analysis and visualization platform of open source) page after being collected through an ELK (a combination of three open source software) log collecting tool, or finally the logs are collected through a big data standard part, and the detection efficiency of the streaming data generated in the operation process of an automatic driving vehicle in the prior art is low.
Therefore, how to efficiently detect streaming data is a technical problem to be solved.
Disclosure of Invention
The embodiment of the specification provides a method, a device and equipment for detecting stream data, so as to solve the problem of low detection efficiency in the existing program detection method.
In order to solve the above technical problems, the embodiments of the present specification are implemented as follows:
the method for detecting stream data provided in the embodiment of the present specification includes:
the tested program sends the to-be-tested streaming data generated in the first time period of the tested program to the kafka message middleware server; the streaming data to be tested comprises start tangential point data and end tangential point data;
the kafka message middleware server distributes the streaming data to be tested to a test server;
the test server detects whether the starting tangent point data meets the pre-assertion or not to obtain a first detection result;
the test server detects whether the ending tangent point data meets post-assertion or not to obtain a second detection result;
and if the first detection result indicates that the start tangent point data meets the pre-assertion, and if the second detection result indicates that the end tangent point data meets the post-assertion, determining that the streaming data to be tested passes detection.
The device for detecting stream data provided in the embodiment of the present specification includes:
and a sending module: the tested program sends the to-be-tested streaming data generated in the first time period of the tested program to the kafka message middleware server; the streaming data to be tested comprises start tangential point data and end tangential point data;
the kafka message middleware server distributes the streaming data to be tested to a test server;
and a judging module: the test server detects whether the starting tangent point data meets the pre-assertion or not to obtain a first detection result;
the test server detects whether the ending tangent point data meets post-assertion or not to obtain a second detection result;
and a determination module: and if the first detection result indicates that the start tangent point data meets the pre-assertion, and if the second detection result indicates that the end tangent point data meets the post-assertion, determining that the streaming data to be tested passes detection.
The embodiment of the present disclosure provides a detection device for streaming data, including:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores instructions executable by the at least one processor to enable the at least one processor to:
the tested program sends the to-be-tested streaming data generated in the first time period of the tested program to the kafka message middleware server; the streaming data to be tested comprises start tangential point data and end tangential point data;
the kafka message middleware server distributes the streaming data to be tested to a test server;
the test server detects whether the starting tangent point data meets the pre-assertion or not to obtain a first detection result;
the test server detects whether the ending tangent point data meets post-assertion or not to obtain a second detection result;
and if the first detection result indicates that the start tangent point data meets the pre-assertion, and if the second detection result indicates that the end tangent point data meets the post-assertion, determining that the streaming data to be tested passes detection.
One embodiment of the present specification achieves the following advantageous effects: the tested program sends the to-be-tested streaming data generated in the first time period of the tested program to the kafka message middleware server; the streaming data to be tested comprises start tangential point data and end tangential point data; the kafka message middleware server distributes the streaming data to be tested to a test server; the test server detects whether the starting tangent point data meets the pre-assertion or not to obtain a first detection result; the test server detects whether the ending tangent point data meets post-assertion or not to obtain a second detection result; and if the first detection result indicates that the start tangent point data meets the pre-assertion, and if the second detection result indicates that the end tangent point data meets the post-assertion, determining that the streaming data to be tested passes detection. And respectively sending the start tangent point data and the end tangent point data to the kafka message middleware server, detecting whether the start tangent point data meets the pre-assertion or not by using a test server, and detecting whether the end tangent point data meets the post-assertion or not, thereby continuously detecting the correctness of the streaming data in real time and improving the detection efficiency of the streaming data.
Drawings
In order to more clearly illustrate the embodiments of the present description or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the present application, and that other drawings may be obtained according to these drawings without inventive effort to a person skilled in the art.
Fig. 1 is an application scenario schematic diagram of a method for detecting streaming data in an embodiment of the present disclosure;
fig. 2 is a flow chart of a method for detecting streaming data according to an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of a device for detecting streaming data according to an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of a streaming data detection device according to an embodiment of the present disclosure.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of one or more embodiments of the present specification more clear, the technical solutions of one or more embodiments of the present specification will be clearly and completely described below in connection with specific embodiments of the present specification and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present specification. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without undue burden, are intended to be within the scope of one or more embodiments herein.
For terms used in the embodiments of the present specification, explanation is first given of:
kafka: is a high throughput distributed publish-subscribe messaging system.
Software development kit (softwaredevelopment kit, english abbreviation SDK): typically, a collection of development tools are created by some software engineers for a particular software package, software framework, hardware platform, operating system, etc. when the application software is built.
JUnit: a Java language unit test framework is used for writing and running repeatable tests, namely, a Junit test, namely, a so-called white box test, and is tested according to How (How) the tested software is finished and What (What) functions are finished. Currently, in the development and test process of a project, developers and testers can use a JUnit to complete unit testing.
While: the method is a basic circulation mode of the computer, and enters the circulation when the condition is met, and jumps out of the circulation when the condition is not met after the circulation is entered.
Multithreading: refers to a technique for implementing concurrent execution of multiple threads from software or hardware. Computers with multithreading capability are capable of executing more than one thread at a time due to hardware support, thereby improving overall processing performance.
Thread pool: in a form of multithreaded processing, tasks are added to a queue during processing, and then automatically initiated after a thread is created. A thread usage pattern, while a thread pool maintains multiple threads awaiting a supervisor to allocate concurrently executable tasks.
Assertion of: is a first order logic in a program (e.g., a logical decision that results in true or false) in order to represent that the corresponding assertion should be true in order to verify the result expected by the software developer when the program is executed to the location of the assertion. If the assertion is not true, the program will abort execution and give an error message.
Timestamp: typically a sequence of characters, uniquely identifies a time at a moment in time, and the time at which the data was generated can be authenticated by a time stamp during production.
Time delay: refers to the time required for a message or packet to travel from one end of a network to another. It includes transmission delay, propagation delay, processing delay and queuing delay. The delay is the time interval from the first bit of the data packet entering the router to the last bit being output from the router.
The following describes in detail the technical solutions provided by the embodiments of the present specification with reference to the accompanying drawings.
Fig. 1 is an application scenario schematic diagram of a method for detecting streaming data in an embodiment of the present disclosure.
In this embodiment of the present disclosure, as shown in fig. 1, the flow data is collected by the SDK method, and the tester may manually define a start tangent point and an end tangent point, and the SDK method may automatically transmit the start tangent point data and the end tangent point data, so as to detect the acquired flow data.
The streaming data to be tested can be sent to the kafka message middleware by the SDK method.
The tester activates the junit launcher, which may include: the system comprises an assertion construction factory and a thread initiator, wherein the assertion construction factory can construct a start assertion and an end assertion in a reflection mode, the thread initiator can start the start assertion and the end assertion to respectively and circularly detect data in the kafka message middleware, the start assertion detects start tangent point data, and the end assertion detects end tangent point data, so that the correctness of streaming data can be continuously detected in real time, and the testing efficiency is improved.
Next, a method for detecting streaming data provided in an embodiment of the present disclosure will be specifically described with reference to the accompanying drawings.
Fig. 2 is a flow chart of a method for detecting streaming data according to an embodiment of the present disclosure. From the program perspective, the execution subject of the flow may be a program or an application client that is installed on an application server. On the other hand, from a hardware point of view, the execution subject of the flow may be a terminal device or the like.
As shown in fig. 2, the process may include the steps of:
step 210: the tested program sends the to-be-tested streaming data generated in the first time period of the tested program to the kafka message middleware server; the stream data to be tested comprises start tangent point data and end tangent point data.
In this embodiment of the present disclosure, a tester may manually define a start tangent point and an end tangent point of data to be tested, where the start tangent point may be used to inform a program where to start testing, check whether the output data is correct, and the end tangent point may be used to inform a program where to end, and check whether the output data is correct.
The method has the advantages that the SDK method can be used for collecting streaming data, a tester can manually define a start tangential point and an end tangential point, the SDK method can automatically transmit the start tangential point data and the end tangential point data, a method for transmitting data in the SDK is called, the streaming data can be transmitted to the kafka message middleware server in a thread pool asynchronous operation mode, and asynchronous realization is achieved: the data transmission belongs to IO (input and output) intensive operation, mainly takes time on the data transmission of a network, the original time delay is relatively small, the subsequent operation is directly carried out after a task is handed to the thread pool in an asynchronous mode by using the thread pool, the time delay is smaller than that of the direct IO operation, the streaming calculation belongs to CPU intensive operation, the execution time is long, compared with the execution time, the transmission time delay from the asynchronous data transmission of a tool to the kafka operation is negligible, the overall invasiveness is small, an exception handling mechanism is arranged in the transmission method, and the test code exception cannot influence the original program code logic.
In practical applications, the start tangent point and the end tangent point may be set at any position in the program under test, so as to detect the program in the middle of any start tangent point and end tangent point.
Step 220: the kafka message middleware server distributes the streaming data to be tested to a test server.
In the embodiment of the specification, the test server can be started manually, and can consume the streaming data to be tested in the kafka message middleware, and in practical application, the test server can be a JUnit starter, and the JUnit starter can comprise an assertion construction factory and a thread starter.
Step 230: and the test server detects whether the starting tangent point data meets the pre-assertion or not to obtain a first detection result.
In the embodiment of the specification, a tester can manually write an assertion method, an assertion construction factory can create a pre-assertion and a post-assertion in a reflection mode, and a test server can respectively start the pre-assertion and the post-assertion through a thread starter, so that streaming data can be consumed and checked.
The test server can detect the start tangent point data by adopting the pre-assertion, detect whether the streaming data to be tested is correct, and in practical application, start the thread starter, and the thread continuously pulls the streaming data to be tested in the kafka message middleware in a while circulation mode to put the streaming data to be tested into the pre-assertion for verification, so that the correctness of the streaming data can be continuously checked, and the detection efficiency of the streaming data is improved.
Step 240: and the test server detects whether the ending tangent point data meets the post-assertion or not to obtain a second detection result.
In the embodiment of the present disclosure, the test server may detect whether the streaming data to be tested is correct by using post-assertion to detect end tangent point data. In practical application, a thread starter is started, and a thread continuously pulls the streaming data to be tested in the kafka message middleware in a while circulation mode, and the streaming data to be tested is put into the post-assertion for verification, so that the correctness of the streaming data can be continuously checked, and the detection efficiency of the streaming data is improved.
Step 250: and if the first detection result indicates that the streaming data meets the pre-assertion, and if the second detection result indicates that the streaming data meets the post-assertion, determining that the streaming data to be tested passes detection.
In the embodiment of the specification, when the start tangent point data is detected to meet the pre-assertion and the end tangent point data is detected to meet the post-assertion, the streaming data to be tested is determined to be correct. Specifically, the correctness of the streaming data can be detected by judging whether the starting tangent point data is empty, whether the attribute field of the data object is empty, whether the field type is correct, whether the field value is correct, whether the field accords with the expected range, and the like.
In order to facilitate understanding, for example, in an application scenario of automatic driving, after an abnormal parking condition occurs in an automatic driving vehicle, all internet-connected vehicle information in a range of 200 meters behind the vehicle is acquired, a safety early warning is issued to the acquired internet-connected vehicle, and a tester can detect automatic driving data after the abnormal parking of the vehicle occurs.
The tester can set the start tangent point in the code before the network connection around the self-vehicle is started, and set the end tangent point in the code after the self-vehicle Zhou Weiwang connection result is obtained, so that the network connection operation in the range of 200 meters behind the self-vehicle can be verified. The front-end assertion mainly verifies whether the abnormal vehicle is empty or not, whether the vehicle state is normal or not, and continuously verifies whether the specific range of the polygon 200 m behind the abnormal vehicle is correct or not, and the rear-end assertion mainly verifies whether the vehicle which is supposed to be acquired is already acquired or not, whether the vehicle which is not supposed to be acquired is not acquired or not, and whether the acquired vehicle state is normal or not. By detecting the correctness of the automatic driving data, the automatic driving vehicle can be better controlled.
It should be understood that the method according to one or more embodiments of the present disclosure may include the steps in which some of the steps are interchanged as needed, or some of the steps may be omitted or deleted.
In the method in fig. 2, the tested program sends the generated stream data to be tested including the start tangent point data and the end tangent point data to the kafka message middleware server, and the test server consumes the stream data in the kafka message middleware server to be tested, so that the correctness of the stream data can be continuously tested in real time, and the testing efficiency is improved.
Based on the method in fig. 2, the examples of the present specification also provide some specific embodiments of the method, as described below.
Optionally, the method in the embodiment of the present specification may further include:
and if the first detection result indicates that the start tangent point data does not meet the pre-assertion, recording the first detection result, determining that the streaming data to be tested fails to pass the detection, and stopping the test.
In practical application, when the start tangent point data is detected not to meet the pre-assertion, that is, when the streaming data to be tested is detected to be wrong, the test server can record error information and stop the test.
Optionally, the method in the embodiment of the present specification may further include:
and if the second detection result indicates that the ending tangent point data does not meet the post-assertion, recording the second detection result, determining that the streaming data to be tested does not pass the detection, and stopping the test.
In practical application, when the end tangent point data is detected not to meet the post-assertion, that is, when the stream data to be tested is detected to be wrong, the test server can record error information and stop the test.
In order to detect the sequentiality of the streaming data in the embodiment of the present disclosure, optionally, the method in the embodiment of the present disclosure may further include:
judging whether the first time stamp of the current streaming data is larger than the second time stamp of the last streaming data or not, and obtaining a first judging result;
if the first judgment result indicates that the first time stamp is larger than the second time stamp, determining that the streaming data to be tested passes detection;
and if the first judgment result indicates that the first time stamp is smaller than the second time stamp, recording the first judgment result, determining that the streaming data to be tested fails to pass the detection, and stopping the test.
In particular, streaming data is sequenced from the kafka message middleware server, and it is necessary to detect the sequence of the streaming data.
The method comprises the steps of obtaining a time stamp of current streaming data and a time stamp of last streaming data, comparing the time stamp of the current streaming data with the time stamp of the last streaming data according to time sequence, if the time stamp of the current streaming data is larger than the time stamp of the last streaming data, the method can indicate that the sequence of the streaming data is correct, then continuously obtaining a time stamp of next streaming data, and comparing the time stamp of the next streaming data with the time stamp of the current streaming data. If the time stamp of the current streaming data is smaller than the time stamp of the last streaming data, it may indicate that the order of the streaming data is wrong, and the test server may record error information and stop the test.
In practical application, the sequence of the streaming data can be continuously detected by comparing the time stamps of the streaming data, so that the detection efficiency of the streaming data is improved.
For example: in the technical field of automatic driving, if the sequence of the automatic driving data is problematic, an abnormal situation can occur in the automatic driving vehicle, for example, the situation that the vehicle is backward at a certain moment can occur in the forward running process of the vehicle, and the automatic driving vehicle can be better controlled by detecting the sequence of the streaming data.
In order to detect the relative delay of the streaming data in the embodiment of the present disclosure, optionally, the method in the embodiment of the present disclosure may further include:
acquiring a first time stamp of current streaming data and a second time stamp of last streaming data;
judging whether the difference value between the first time stamp and the second time stamp is larger than a preset time or not to obtain a second judging result;
and if the second judgment result indicates that the difference value is larger than the preset time, recording the second judgment result, and storing the difference value.
Specifically, there is a certain delay in streaming data flowing out from the kafka message middleware server, and the delay of the streaming data needs to be detected.
The difference between the time stamps of the current streaming data and the time stamp of the last streaming data may be obtained, and the difference may be compared with a preset time, where the preset time may be the interval time of data transmission. For example: if the data is transmitted at intervals of 100 ms and the preset time is 100 ms, the difference between the time stamp of the current streaming data and the time stamp of the last streaming data should be 100 ms, and because the data transmission has a certain time delay, the difference between the time stamp of the current streaming data and the time stamp of the last streaming data may be more than 100 ms, if the difference is 150 ms, the relative time delay of the data transmission is 50 ms, then the time stamp of the next streaming data may be continuously acquired, the time stamp of the next streaming data and the time stamp of the current streaming data are compared, the relative time delay time is recorded, and the test result is displayed.
In practical application, the difference value of the time stamps of the streaming data is compared with the preset time, so that the relative time delay of the streaming data can be continuously detected, and the detection efficiency of the streaming data is improved.
For example: in the technical field of automatic driving, if the time delay of automatic driving data is longer, abnormal conditions can appear in the automatic driving vehicle, for example, the condition that the vehicle continues to run forward after stopping possibly appears in the process of running forward, and the automatic driving vehicle can be better controlled by detecting the relative time delay of streaming data and recording the relative time delay time.
Optionally, the method in the embodiment of the present specification may further include:
the tested program sends the stream data to be tested generated in the second time period of the tested program to the kafka message middleware server; the second time period is a next time period to the first time period.
In particular, the SDK method may continuously transmit streaming data including start and end cut point data, and continuously send streaming data to be tested to the kafka message middleware for subsequent continuous detection of the streaming data.
Optionally, the method in the embodiment of the present specification may further include:
the test server detects the streaming data in a multithreading mode.
Specifically, the thread initiator in the test server can start the assertion in a multithreading manner, so that the assertion can be ensured to be continuously started, and the data processing capacity of the test server is improved.
Optionally, the method in the embodiment of the present specification may further include:
the pre-predicate and the post-predicate call a predicate method using a reflection.
Specifically, a tester can manually write an assertion method, an assertion construction factory can create a pre-assertion and a post-assertion in a reflection mode, and reflection refers to a reflection mechanism in Java, and assertion operation can be performed when stream data is acquired.
In the embodiment of the specification, the tested program sends the generated streaming data to be tested, which includes the start tangential point data and the end tangential point data, to the kafka message middleware server, and the testing server consumes the streaming data in the kafka message middleware server to detect through assertion, so that the correctness of the streaming data can be continuously detected in real time.
Fig. 3 is a schematic structural diagram of a streaming data detection device according to an embodiment of the present disclosure.
The detection apparatus for stream data described in the embodiment of the present specification may include:
the sending module 302, the tested program sends the to-be-tested streaming data generated in the first time period of the tested program to the kafka message middleware server; the streaming data to be tested comprises start tangential point data and end tangential point data;
the kafka message middleware server distributes the streaming data to be tested to a test server;
a judging module 304, wherein the test server detects whether the start tangent point data meets the pre-assertion or not to obtain a first detection result;
the test server detects whether the ending tangent point data meets post-assertion or not to obtain a second detection result;
a determining module 306, configured to determine that the streaming data to be tested passes detection if the first detection result indicates that the start tangent point data meets the pre-assertion and if the second detection result indicates that the end tangent point data meets the post-assertion.
Based on the same thought, the embodiment of the specification also provides equipment corresponding to the method.
Fig. 4 is a schematic structural diagram of a streaming data detection device according to an embodiment of the present disclosure.
As shown in fig. 4, the apparatus 400 may include:
at least one processor 410; the method comprises the steps of,
a memory 430 communicatively coupled to the at least one processor; wherein,,
the memory 430 stores instructions 420 executable by the at least one processor 410, the instructions being executable by the at least one processor 410 to enable the at least one processor 410 to:
the tested program sends the to-be-tested streaming data generated in the first time period of the tested program to the kafka message middleware server; the streaming data to be tested comprises start tangential point data and end tangential point data;
the kafka message middleware server distributes the streaming data to be tested to a test server;
the test server detects whether the starting tangent point data meets the pre-assertion or not to obtain a first detection result;
the test server detects whether the ending tangent point data meets post-assertion or not to obtain a second detection result;
and if the first detection result indicates that the start tangent point data meets the pre-assertion, and if the second detection result indicates that the end tangent point data meets the post-assertion, determining that the streaming data to be tested passes detection.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the apparatus shown in fig. 4, the description is relatively simple, as it is substantially similar to the method embodiment, with reference to the partial description of the method embodiment.
In the 90 s of the 20 th century, improvements to one technology could clearly be distinguished as improvements in hardware (e.g., improvements to circuit structures such as diodes, transistors, switches, etc.) or software (improvements to the process flow). However, with the development of technology, many improvements of the current method flows can be regarded as direct improvements of hardware circuit structures. Designers almost always obtain corresponding hardware circuit structures by programming improved method flows into hardware circuits. Therefore, an improvement of a method flow cannot be said to be realized by a hardware entity module. For example, a programmable logic device (ProgrammableLogicDevice, PLD), such as a Field programmable gate array (Field ProgrammableGateArray, FPGA), is an integrated circuit whose logic function is determined by the programming of the device by a user. A designer programs to "integrate" a digital system onto a PLD without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Moreover, nowadays, instead of manually manufacturing integrated circuit chips, such programming is mostly implemented with "logic compiler" software, which is similar to the software compiler used in program development and writing, and the original code before being compiled is also written in a specific programming language, which is called hardware description language (HardwareDescriptionLanguage, HDL), while HDL is not only one but a plurality of kinds, such as ABEL (AdvancedBooleanExpressionLanguage), AHDL (AlteraHardwareDescriptionLanguage), confluence, CUPL (Cornell UniversityProgrammingLanguage), HDCal, JHDL (JavaHardwareDescription Language), lava, lola, myHDL, PALASM, RHDL (RubyHardwareDescription Language), etc., VHDL (Very-High-SpeedIntegratedCircuit HardwareDescriptionLanguage) and Verilog are most commonly used at present. It will also be apparent to those skilled in the art that a hardware circuit implementing the logic method flow can be readily obtained by merely slightly programming the method flow into an integrated circuit using several of the hardware description languages described above.
The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, application specific integrated circuits (Application SpecificIntegratedCircuit, ASIC), programmable logic controllers, and embedded microcontrollers, examples of which include, but are not limited to, the following microcontrollers: ARC625D, atmelAT91SAM, microchipPIC F26K20 and silicane labsc8051F320, the memory controller may also be implemented as part of the control logic of the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller in a pure computer readable program code, it is well possible to implement the same functionality by logically programming the method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Such a controller may thus be regarded as a kind of hardware component, and means for performing various functions included therein may also be regarded as structures within the hardware component. Or even means for achieving the various functions may be regarded as either software modules implementing the methods or structures within hardware components.
The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functions of each element may be implemented in one or more software and/or hardware elements when implemented in the present application.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, etc., such as Read Only Memory (ROM) or flash memory (flashRAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and changes may be made to the present application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. which are within the spirit and principles of the present application are intended to be included within the scope of the claims of the present application.

Claims (10)

1. A method for detecting streaming data, comprising:
the tested program sends the to-be-tested streaming data generated in the first time period of the tested program to the kafka message middleware server; the streaming data to be tested comprises start tangential point data and end tangential point data;
the kafka message middleware server distributes the streaming data to be tested to a test server;
the test server detects whether the starting tangent point data meets the pre-assertion or not to obtain a first detection result;
the test server detects whether the ending tangent point data meets post-assertion or not to obtain a second detection result;
and if the first detection result indicates that the start tangent point data meets the pre-assertion, and if the second detection result indicates that the end tangent point data meets the post-assertion, determining that the streaming data to be tested passes detection.
2. The method of claim 1, wherein the method further comprises:
and if the first detection result indicates that the start tangent point data does not meet the pre-assertion, recording the first detection result, determining that the streaming data to be tested fails to pass the detection, and stopping the test.
3. The method of claim 1, wherein the method further comprises:
and if the second detection result indicates that the ending tangent point data does not meet the post-assertion, recording the second detection result, determining that the streaming data to be tested does not pass the detection, and stopping the test.
4. The method of claim 1, wherein the method further comprises:
judging whether the first time stamp of the current streaming data is larger than the second time stamp of the last streaming data or not, and obtaining a first judging result;
if the first judgment result indicates that the first time stamp is larger than the second time stamp, determining that the streaming data to be tested passes detection;
and if the first judgment result indicates that the first time stamp is smaller than the second time stamp, recording the first judgment result, determining that the streaming data to be tested fails to pass the detection, and stopping the test.
5. The method of claim 1, wherein the method further comprises:
acquiring a first time stamp of current streaming data and a second time stamp of last streaming data;
judging whether the difference value between the first time stamp and the second time stamp is larger than a preset time or not to obtain a second judging result;
and if the second judgment result indicates that the difference value is larger than the preset time, recording the second judgment result, and storing the difference value.
6. The method of claim 1, wherein the method further comprises:
the tested program sends the stream data to be tested generated in the second time period of the tested program to the kafka message middleware server; the second time period is a next time period to the first time period.
7. The method of claim 1, wherein the method further comprises:
the test server detects the streaming data in a multithreading mode.
8. The method of claim 1, wherein the method further comprises:
the pre-predicate and the post-predicate call a predicate method using a reflection.
9. A streaming data detection apparatus, comprising:
and a sending module: the tested program sends the to-be-tested streaming data generated in the first time period of the tested program to the kafka message middleware server; the streaming data to be tested comprises start tangential point data and end tangential point data;
the kafka message middleware server distributes the streaming data to be tested to a test server;
and a judging module: the test server detects whether the starting tangent point data meets the pre-assertion or not to obtain a first detection result;
the test server detects whether the ending tangent point data meets post-assertion or not to obtain a second detection result;
and a determination module: and if the first detection result indicates that the start tangent point data meets the pre-assertion, and if the second detection result indicates that the end tangent point data meets the post-assertion, determining that the streaming data to be tested passes detection.
10. A streaming data detection device, comprising:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores instructions executable by the at least one processor to enable the at least one processor to:
the tested program sends the to-be-tested streaming data generated in the first time period of the tested program to the kafka message middleware server; the streaming data to be tested comprises start tangential point data and end tangential point data;
the kafka message middleware server distributes the streaming data to be tested to a test server;
the test server detects whether the starting tangent point data meets the pre-assertion or not to obtain a first detection result;
the test server detects whether the ending tangent point data meets post-assertion or not to obtain a second detection result;
and if the first detection result indicates that the start tangent point data meets the pre-assertion, and if the second detection result indicates that the end tangent point data meets the post-assertion, determining that the streaming data to be tested passes detection.
CN202211725891.8A 2022-12-29 2022-12-29 Method, device and equipment for detecting stream data Pending CN116055349A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211725891.8A CN116055349A (en) 2022-12-29 2022-12-29 Method, device and equipment for detecting stream data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211725891.8A CN116055349A (en) 2022-12-29 2022-12-29 Method, device and equipment for detecting stream data

Publications (1)

Publication Number Publication Date
CN116055349A true CN116055349A (en) 2023-05-02

Family

ID=86128885

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211725891.8A Pending CN116055349A (en) 2022-12-29 2022-12-29 Method, device and equipment for detecting stream data

Country Status (1)

Country Link
CN (1) CN116055349A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117724986A (en) * 2024-02-08 2024-03-19 云账户技术(天津)有限公司 Method and device for automatically operating kafka in use case execution

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117724986A (en) * 2024-02-08 2024-03-19 云账户技术(天津)有限公司 Method and device for automatically operating kafka in use case execution
CN117724986B (en) * 2024-02-08 2024-05-07 云账户技术(天津)有限公司 Method and device for automatically operating kafka in use case execution

Similar Documents

Publication Publication Date Title
US9235497B2 (en) Method and system for detecting concurrency programming errors in kernel modules and device drivers
US8141053B2 (en) Call stack sampling using a virtual machine
US20070266376A1 (en) Kernel-aware debugging system, medium, and method
KR102537875B1 (en) Method and apparatus for dinamically injecting fault for vehicle ecu software test
KR101459867B1 (en) System test apparatus
CN110018914B (en) Shared memory based message acquisition method and device
KR20110114432A (en) Method and system for detecting deadlock in multithread program
CN104809058A (en) Method and device for dynamically dispatching concurrence quantity in test execution
CN116055349A (en) Method, device and equipment for detecting stream data
CN117032903A (en) Simulation debugging method and device, storage medium and electronic equipment
CN101706752B (en) Method and device for in-situ software error positioning
CN108021495B (en) Code debugging method based on playback
CN113360389A (en) Performance test method, device, equipment and storage medium
CN107153601B (en) Unit performance test method and device
CN102521132A (en) Automated testing method and automated testing system for real-time output logs
KR20200077297A (en) Method and Apparatus for detecting atomicity violation for shared memory used in multi-process/multi-thread
He et al. Software reliability analysis on embedded system based on SFMEA and SFTA model
US8090991B2 (en) Information processing apparatus, method, and computer program product for self-diagnosis for utilizing multiple diagnostic devices, each having exclusive access to a resource
CN109857665A (en) A kind of test execution method and device of test case
CN115327953B (en) Simulation test method and device for intelligent driving algorithm, electronic equipment and medium
CN116594862B (en) DBMS test method and device, electronic equipment and readable storage medium
CN117076336B (en) Testing method and device of cloud edge cooperative system, storage medium and equipment
CN112859526B (en) Synchronous diagnosis method and device for scanning photoetching machine, medium and electronic equipment
CN106250260A (en) Processor overflows monitoring and adjustment method and device
CN117191060A (en) Map data abnormality diagnosis method, device and equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination