CN116579784A - Processing method and device for assembly data, searching method and system and electronic equipment - Google Patents

Processing method and device for assembly data, searching method and system and electronic equipment Download PDF

Info

Publication number
CN116579784A
CN116579784A CN202310415945.9A CN202310415945A CN116579784A CN 116579784 A CN116579784 A CN 116579784A CN 202310415945 A CN202310415945 A CN 202310415945A CN 116579784 A CN116579784 A CN 116579784A
Authority
CN
China
Prior art keywords
image data
assembly
data
target image
determining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310415945.9A
Other languages
Chinese (zh)
Inventor
陈罡
徐欣
钱广璞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Electric Group Corp
Original Assignee
Shanghai Electric Group Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Electric Group Corp filed Critical Shanghai Electric Group Corp
Priority to CN202310415945.9A priority Critical patent/CN116579784A/en
Publication of CN116579784A publication Critical patent/CN116579784A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/018—Certifying business or products
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/903—Querying
    • G06F16/90335—Query processing
    • G06F16/90344—Query processing by using string matching techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/04—Manufacturing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Strategic Management (AREA)
  • General Business, Economics & Management (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Manufacturing & Machinery (AREA)
  • Finance (AREA)
  • Human Resources & Organizations (AREA)
  • Primary Health Care (AREA)
  • Tourism & Hospitality (AREA)
  • Computational Linguistics (AREA)
  • Development Economics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Accounting & Taxation (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Human Computer Interaction (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • General Factory Administration (AREA)

Abstract

The invention discloses a processing method and a device, a searching method and a system for assembly data and electronic equipment, wherein the processing method comprises the following steps: receiving an opening instruction, and controlling a camera to acquire video data of an assembly site; collecting target image data in video data; inputting the target image data into a target detection model, and determining an assembly action corresponding to the target image data according to the target detection model; standard image data matched with the assembly action is obtained; determining a time node of target image data in video data, and comparing the target image data with standard image data to obtain a comparison result; associating the storage time node with the assembly data; wherein the assembly data includes at least one of: the comparison result, the target image data, and the video clip containing the target image data. The invention realizes real-time detection and real-time feedback of the assembly process, and simultaneously can trace the assembly data, thereby improving the reliability of assembly.

Description

Processing method and device for assembly data, searching method and system and electronic equipment
Technical Field
The invention relates to the field of assembly monitoring, in particular to a processing method and device, a searching method and system and electronic equipment of assembly data.
Background
Currently, the manual assembly process of the discrete production and manufacturing links is mainly detected by manually performing speculative inspection (such as air tightness, external flaw detection, operation, voltage/resistance and the like) on an assembly body from outside after the assembly is completed. Not traceable to the actual assembly process. It is also impossible to determine whether the same lot of products has the same problem after a problem has occurred. Part of the manual assembly work is recorded by a camera. But because the amount of data is too large, manual re-tracing is almost impossible. The relevant assembly process is not trace enough and it is not possible to quickly check if there are other problems in the same lot after one problem in the assembly process is found. If the relevant sensor is installed, not only the cost of raw materials and suppliers will be increased, but also the assembly process will be greatly changed.
Disclosure of Invention
The invention aims to overcome the defect that products cannot be detected by tracing in the assembly process in the prior art, and provides a processing method and device, a searching method and system and electronic equipment for assembly data.
The invention solves the technical problems by the following technical scheme:
in a first aspect, there is provided a processing method of assembly data, the processing method including:
receiving an opening instruction, and controlling a camera to acquire video data of an assembly site;
collecting target image data in the video data;
inputting the target image data into a target detection model, and determining an assembly action corresponding to the target image data according to the target detection model;
acquiring standard image data matched with the assembly action;
determining a time node of the target image data in the video data, and comparing the target image data with the standard image data to obtain a comparison result;
storing the time node and the assembly data in an associated mode; wherein the assembly data includes at least one of: the comparison result, the target image data, a video clip containing the target image data.
Optionally, the step of acquiring target image data in the video data comprises:
performing frame extraction processing on the video data to obtain image data;
and respectively calculating the similarity of the two adjacent frames of image data, and determining the two frames of image data with the similarity smaller than a similarity threshold value as target image data.
Optionally, the step of acquiring target image data in the video data further comprises:
performing frame extraction processing on the video data to obtain image data;
monitoring a first area and a second area of the image data respectively, and determining the image data as target image data when the pixels of the first area change and the pixels of the second area do not change; wherein the second region is other regions than the first region.
Optionally, the method further comprises:
and when an image acquisition instruction is received, acquiring target image data in the video data, and determining a time node of the target image data in the video data.
Optionally, the target detection model is obtained through training an image data sample marked with a target detection result;
wherein the image data sample is image data of an assembly site.
In a second aspect, there is provided a search method of assembly data, the assembly data being obtained according to the processing method described in the first aspect, the search method comprising:
obtaining keywords;
and determining assembly data matched with the keywords.
Optionally, the step of determining the assembly data matching the keyword includes:
converting the keywords into character strings;
searching according to the character string;
and determining assembly data matched with the keywords.
In a third aspect, there is provided a processing apparatus for assembling data, to which the processing method described in the first aspect is applied, the processing apparatus comprising:
the controller is used for controlling the assembly steps and man-machine interaction in the assembly process;
the camera is electrically connected with the controller and is used for collecting video data in the assembly process and sending image data of the time node in the video data to the processor;
and the processor is electrically connected with the camera and is used for receiving the image data, carrying out image detection and comparing the image data with standard image data.
In a fourth aspect, there is provided a search system for assembly data, the assembly data being obtained according to the processing method of the first aspect, the search system comprising:
the acquisition module is used for acquiring the keywords;
and the determining module is used for determining the assembly data matched with the keywords.
In a fifth aspect, there is provided an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of the first or second aspect when executing the computer program.
On the basis of conforming to the common knowledge in the field, the above preferred conditions can be arbitrarily combined to obtain the preferred examples of the invention.
The invention has the positive progress effects that: the image data of the assembly site is acquired, the image data is compared, the assembly process is monitored in real time, the comparison result is fed back in real time, the problem of products in the assembly process is avoided, the quality of the products is guaranteed, the assembly process is traceable by storing the assembly data, and the quality of the products in the same batch can be conveniently detected in a tracing manner when the products go out of the problem.
Drawings
FIG. 1 is a flowchart of a method for processing assembly data according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of an assembly data processing device according to an embodiment of the present invention;
FIG. 3 is a connection diagram of an assembly data processing device according to an embodiment of the present invention;
FIG. 4 is a flowchart of another method for processing assembly data according to an embodiment of the present invention;
FIG. 5 is a schematic diagram illustrating an operation procedure of a method for processing assembly data according to an embodiment of the present invention;
FIG. 6 is a flowchart of another method for processing assembly data according to an embodiment of the present invention;
FIG. 7 is a flowchart of a method for searching assembly data according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a system for searching assembly data according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The invention is further illustrated by means of the following examples, which are not intended to limit the scope of the invention.
Fig. 1 is a flowchart of a processing method of assembly data according to an embodiment of the present invention, where the method includes the following steps:
s11, receiving an opening instruction, and controlling the camera to acquire video data of the assembly site.
When the assembly operation is started, the camera is opened, and the camera is controlled to acquire video data of an assembly site, wherein the assembly site is a scene during the assembly operation.
S12, collecting target image data in the video data.
In one embodiment, the step of acquiring target image data in the video data comprises:
performing frame extraction processing on the video data to obtain image data; and respectively calculating the similarity of the two adjacent frames of image data, and determining the two frames of image data with the similarity smaller than a similarity threshold value as target image data. For example, 20 seconds of video data is collected, the first 7 seconds is a screw for installing a product, and 7 seconds later is a nut has been installed on the product, and then the image data at 7 seconds is determined as target image data.
The similarity threshold is set according to actual conditions.
Judging the assembly step of the product according to the similarity of two adjacent image data, wherein when the similarity of the two adjacent image data is larger than or equal to a similarity threshold value, the similarity of the two image data is higher, and the two image data are similar, so that the product in the assembly process is unchanged; when the similarity of two frames of adjacent image data is smaller than a similarity threshold value, the similarity of the two frames of image data is lower, which means that the product in the assembly process is changed, for example, a screw is installed and a shell is welded; wherein the assembling step is a preset assembling step.
In one embodiment, the step of acquiring target image data in the video data further comprises:
and performing frame extraction processing on the video data to obtain image data, respectively monitoring a first area and a second area of the image data, and determining the image data as target image data when the pixels of the first area change and the pixels of the second area do not change, wherein the second area is other areas except the first area.
The first area is set according to actual conditions. For example, a center region of the image data may be determined as the first region.
And judging the assembly step of the product according to the pixel change of different areas of the image data, wherein when the pixel of the first area is changed and the pixel of the second area is not changed, the assembly step indicates that the product in the assembly process is changed. For example, the housing is welded to the first region, the pixels of the first region are changed, and the pixels of the second region are unchanged.
S13, inputting the target image data into a target detection model, and determining the assembly action corresponding to the target image data according to the target detection model.
And determining an assembling action corresponding to the target image data according to the target image data so as to detect the target image data later and obtain the assembling state of the product.
Wherein, the assembly action is set up by oneself according to actual conditions.
In one embodiment, the object detection model is trained from image data samples labeled with the object detection results, wherein the image data samples are image data of an assembly site.
The training steps comprise:
acquiring an image data sample of an assembly site, wherein the image data is marked with marking information, and the marking information comprises a target detection result;
inputting the image data sample into the initial model to output a recognition result of the image data from the initial model;
calculating a loss error according to the labeling information and the identification result, and adjusting parameters of the initial model according to the loss error;
and determining the initial model meeting the iteration stop condition as a target detection model.
The iteration stop condition includes, but is not limited to, that the accuracy of the initial model meets the requirement, or that the number of iterations reaches a target iteration number, which is determined according to the actual situation.
S14, standard image data matched with the assembly action is acquired.
The standard image data is generally obtained by fitting image data of a product acquired after each assembly action is completed under different conditions, and a storage position of the standard image data is set according to actual conditions, for example, a database.
S15, determining a time node of the target image data in the video data, and comparing the target image data with the standard image data to obtain a comparison result.
The time node of the target image data in the video data is determined, so that the follow-up tracing is facilitated according to the time node, the image data obtained according to each time node is compared with the standard image, the possible existing assembly problem can be rapidly checked, the method can also be used for post-tracing, meanwhile, the comparison process of the target image data and the standard image data is automated, and the image comparison and inspection efficiency is improved.
If errors occur, timely reminding can be performed, so that the incorrect assembly parts are brought into the next process, packaging and shipping are performed, higher inspection/rework cost and potential safety hazards are caused, quality assurance of assembly links is guaranteed, errors can be reminded, the process can be monitored, responsibility can be traced, and additional modification of the assembly process is not needed. When a fault occurs after a product is delivered, the corresponding error step can be quickly found by re-tracing each step of the faulty product, and then the same step on the same batch of products, factories and assembly lines is quickly positioned and re-traced through a time node to determine whether systematic errors exist in the assembly process or not, and the process optimization is performed by the method.
S16, associating the storage time node with the assembly data.
Wherein the assembly data includes at least one of: the comparison result, the target image data, and the video clip containing the target image data.
Wherein, in addition to the above parameters, the assembly data includes, but is not limited to, assembly step, assembly status, camera number, assembly time, assembly site.
The assembly data and the time nodes are stored, tracing to the product according to the time nodes or the assembly data is facilitated, real-time detection and real-time feedback to the assembly process are realized, meanwhile, the assembly data can be traced back, each assembly step is thinned, the reliability of assembly is improved, the problem of the product in the assembly process is avoided, the quality of the product is guaranteed, and the tracing to the quality of the product in the same batch is convenient when the problem of the product occurs.
In one embodiment, further comprising:
and when an image acquisition instruction is received, acquiring target image data in the video data, and determining a time node of the target image data in the video data.
When the assembly steps are changed, such as the assembly steps are finished, the assembly steps are suspended or the assembly steps are replaced, an image acquisition instruction is sent, target image data are acquired, the time node of the target image data in the video data is determined, the image data of the time node can be stored when each assembly step is changed, and the method is beneficial to tracing according to the assembly steps and further finding out problems.
Corresponding to the foregoing processing method of the assembly data, the embodiment of the present invention further provides a processing device of the assembly data, where the processing device, as shown in fig. 2, includes:
the controller 21 is used for controlling the assembly step and man-machine interaction in the assembly process, and also used for controlling voice to inform a user of the current assembly step, switching the assembly step, determining a time node and controlling video recording and suspension, and the user is a worker in the embodiment of the invention; the camera 22 is electrically connected with the controller and is used for collecting video data in the assembly process and sending image data of time nodes in the video data to the processor; the processor 23 is electrically connected with the camera and is used for receiving the image data, carrying out image detection and comparing the image data with standard image data; the image detection comprises image recognition based on artificial intelligence, and the processor system can be deployed on the edge side of the site for real-time monitoring and can be combined with the background of the database for detecting the image data.
In one embodiment, the processing device further comprises a headset electrically connected to the controller for receiving the electrical signal from the controller, converting the electrical signal into an audio signal, and outputting the audio signal.
The headset comprises a noise reducing headset with a microphone for information notification and for reception of voice commands in case of a partly inaccessible controller, which informs the user via the headset of a corresponding operation reminder when the assembly step is started, while the camera starts to acquire video data.
In one embodiment, the connection relationship between the controller and other components in the processing device is shown in fig. 3, where the controller is connected to the earphone by wire or wirelessly, and the controller is connected to the camera by wire or wirelessly, and is connected to the gateway by wire or wirelessly with the data center or the edge device.
In one embodiment, the controller is internally provided with an alarm, and the alarm is used for generating an alarm prompt when the comparison result of the image data and the standard image data is negative, so that a product with quality problems is prevented from entering the next assembly step, the subsequent problems caused by the problem of the product in the assembly process are avoided, and the product quality is ensured.
The method of processing the assembly data is further described below in connection with the processing means:
as shown in fig. 4, the controller is provided with complete assembly steps, and in the embodiment of the present invention, the assembly steps include four assembly steps a, B, C and D, wherein a is the controller, B is the camera, C is the earphone, and D is the processor. The method comprises the steps that an assembly object, namely a product, is located at a fixed position in an assembly site, a camera is fixed at a position where video data of the assembly object can be completely collected, if the assembly object is large and needs to be shot by a plurality of cameras, the corresponding cameras are associated when each assembly step is set, a user wears a headset, clicks a controller to enter a program, selects an assembly step to be performed, and enters a control interface of the assembly step. Clicking the button selects start step a, the camera starts to collect video data, and the controller stores time node a1 at which step starts, indicating that assembly step a at that station has started.
As shown in fig. 5, the start button is clicked again to enter a pause state, and the controller stores a time node a2 for the pause of the step, which means that the assembly step a on the station is paused at the time node a2, mainly for the video camera to continue to collect video data or stop recording according to specific requirements. And clicking the pause button again to reenter the working state, and continuously acquiring video data by the camera, wherein the controller stores a time node a3 at which the step is restarted, and the assembly step a on the station continuously works at the time node a 3.
As shown in fig. 6, clicking the stop or next button indicates that the assembly step has been completed or the next step is entered, the controller stores a time node a0 at which the step stopped, indicates that the assembly step a at the station stopped at the time node a0, and intercepts the video data for separate storage for image detection by a subsequent processor and for comparing the image data with standard image data.
As shown in table 1, after all the assembly steps are completed, generating a table associated storage time node and assembly data; the assembly data comprises, but is not limited to, comparison results, target image data, video clips containing the target image data, assembly steps, assembly states, camera numbers, assembly time and assembly places, and retrieval and traceability are facilitated.
TABLE 1
The embodiment of the invention also provides a searching method of the assembly data, as shown in fig. 7, wherein the assembly data is obtained according to the processing method, and the searching method comprises the following steps:
s71, acquiring keywords.
The manner of obtaining the keywords includes a user empirically presuming a problem of the product or detecting a problem with the image, determining keywords of the assembly process, such as assembly time, assembly step, etc.
S72, determining assembly data matched with the keywords.
And searching the assembly data according to the keywords, determining the assembly data matched with the keywords, realizing traceability for the real assembly process, judging whether the same problem exists according to the assembly data of the products in the same batch, and improving the efficiency and reliability of the inspection when the problem occurs to the products.
In one embodiment, the step of determining the fitting data that matches the keyword comprises:
and converting the keywords into character strings, searching according to the character strings, and determining assembly data matched with the keywords.
Searching the position of the character string in the assembly data and returning the corresponding index to determine the assembly data matched with the keywords, thereby being beneficial to a user to find the matched complete assembly data according to the keywords in the assembly step. By re-tracing each step of the faulty product, the corresponding faulty step can be quickly found, and then the same step on the same batch of products, factories and assembly lines can be quickly located and re-traced through the time node at the same time, so as to determine whether systematic errors exist in the assembly process.
The method of searching for assembly data is further described in conjunction with table 1 below:
the user presumes the problem of the product or detects the finding problem according to experience, confirm the keyword of the assembly process, for example when the assembly step is a, the image detects and finds the abnormality, confirm the time node of the abnormal image in the video data, for the assembly step a resumes after suspending, namely 00:14:56, obtain the keyword as the time node, look for according to the keyword, obtain the assembly data that matches with this time node, for example the camera number is cam01, the assembly personnel is Zhang three, the assembly step is a, obtain the image data according to the time node of the image in the video data at the same time, ensure that each assembly step has stored the assembly data, help to trace to the source and find the problem further according to the keyword.
Corresponding to the above searching method, the embodiment of the present invention further provides a searching system for the assembly data, as shown in fig. 8, where the searching system includes:
an obtaining module 81, configured to obtain a keyword;
a determining module 82 for determining assembly data matching the keywords.
In one embodiment, the determining module includes:
a conversion unit for converting the keyword into a character string;
the searching unit is used for searching according to the character string;
and the determining unit is used for determining the assembly data matched with the keywords.
The embodiment of the invention also provides an electronic device, as shown in fig. 9, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor implements the method for processing and searching the assembly data according to any of the above embodiments when executing the computer program. The electronic device 90 shown in fig. 9 is merely an example, and should not be construed as limiting the functionality and scope of use of embodiments of the present invention. As shown in fig. 9, the electronic device 90 may be embodied in the form of a general purpose computing device, which may be a server device, for example. Components of the electronic device 90 may include, but are not limited to: the at least one processor 91, the at least one memory 92, a bus 93 connecting the different system components, including the memory 92 and the processor 91.
The bus 93 includes a data bus, an address bus, and a control bus.
The memory 92 may include volatile memory such as Random Access Memory (RAM) 921 and/or cache memory 922, and may further include Read Only Memory (ROM) 923.
Memory 92 may also include a program tool 925 (or utility) having a set (at least one) of program modules 924, such program modules 924 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.
The processor 91 executes various functional applications and data processing such as the processing method and the searching method of the assembly data described in any of the above embodiments by running a computer program stored in the memory 92.
The electronic device 90 may also communicate with one or more external devices 94. Such communication may occur through an input/output (I/O) interface 95. Also, model-generated electronic device 90 may also communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the internet via network adapter 96. As shown in fig. 9, the network adapter 96 communicates with other modules of the electronic device 90 via the bus 93. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with electronic device 90, including, but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, data backup storage systems, and the like.
The embodiment of the invention also provides a computer storage medium, on which a computer program is stored, which when executed by a processor, implements the processing method and the searching method of the assembly data provided by any of the above embodiments.
While specific embodiments of the invention have been described above, it will be appreciated by those skilled in the art that this is by way of example only, and the scope of the invention is defined by the appended claims. Various changes and modifications to these embodiments may be made by those skilled in the art without departing from the principles and spirit of the invention, but such changes and modifications fall within the scope of the invention.

Claims (10)

1. A method of processing assembly data, the method comprising:
receiving an opening instruction, and controlling a camera to acquire video data of an assembly site;
collecting target image data in the video data;
inputting the target image data into a target detection model, and determining an assembly action corresponding to the target image data according to the target detection model;
acquiring standard image data matched with the assembly action;
determining a time node of the target image data in the video data, and comparing the target image data with the standard image data to obtain a comparison result;
storing the time node and the assembly data in an associated mode; wherein the assembly data includes at least one of: the comparison result, the target image data, a video clip containing the target image data.
2. The processing method according to claim 1, wherein the step of acquiring target image data in the video data includes:
performing frame extraction processing on the video data to obtain image data;
and respectively calculating the similarity of the two adjacent frames of image data, and determining the two frames of image data with the similarity smaller than a similarity threshold value as target image data.
3. The method of processing of claim 1, wherein the step of acquiring target image data in the video data further comprises:
performing frame extraction processing on the video data to obtain image data;
monitoring a first area and a second area of the image data respectively, and determining the image data as target image data when the pixels of the first area change and the pixels of the second area do not change; wherein the second region is other regions than the first region.
4. The method of processing according to claim 1, further comprising:
and when an image acquisition instruction is received, acquiring target image data in the video data, and determining a time node of the target image data in the video data.
5. The processing method according to claim 1, wherein the target detection model is obtained by training an image data sample labeled with a target detection result;
wherein the image data sample is image data of an assembly site.
6. A method of searching for fitting data, wherein the fitting data is obtained according to the processing method of any one of claims 1 to 5, the method comprising:
obtaining keywords;
and determining assembly data matched with the keywords.
7. The lookup method as claimed in claim 6 wherein said step of determining fitting data that matches said keyword comprises:
converting the keywords into character strings;
searching according to the character string;
and determining assembly data matched with the keywords.
8. A processing device for assembly data, characterized in that the processing method according to any one of claims 1 to 5 is applied, said processing device comprising:
the controller is used for controlling the assembly steps and man-machine interaction in the assembly process;
the camera is electrically connected with the controller and is used for collecting video data in the assembly process and sending image data of the time node in the video data to the processor;
and the processor is electrically connected with the camera and is used for receiving the image data, carrying out image detection and comparing the image data with standard image data.
9. A lookup system for fitting data, wherein the fitting data is obtained according to the processing method of any one of claims 1 to 5, the lookup system comprising:
the acquisition module is used for acquiring the keywords;
and the determining module is used for determining the assembly data matched with the keywords.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any of claims 1-7 when executing the computer program.
CN202310415945.9A 2023-04-19 2023-04-19 Processing method and device for assembly data, searching method and system and electronic equipment Pending CN116579784A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310415945.9A CN116579784A (en) 2023-04-19 2023-04-19 Processing method and device for assembly data, searching method and system and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310415945.9A CN116579784A (en) 2023-04-19 2023-04-19 Processing method and device for assembly data, searching method and system and electronic equipment

Publications (1)

Publication Number Publication Date
CN116579784A true CN116579784A (en) 2023-08-11

Family

ID=87533126

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310415945.9A Pending CN116579784A (en) 2023-04-19 2023-04-19 Processing method and device for assembly data, searching method and system and electronic equipment

Country Status (1)

Country Link
CN (1) CN116579784A (en)

Similar Documents

Publication Publication Date Title
WO2017084186A1 (en) System and method for automatic monitoring and intelligent analysis of flexible circuit board manufacturing process
KR102116064B1 (en) Plant diagnosis method using the same system
US11263072B2 (en) Recovery of application from error
CN117474357B (en) Power distribution room operation and maintenance management method and system based on deep learning
CN111858352B (en) Method, device, equipment and storage medium for automatic test monitoring
WO2018017973A1 (en) Computational analysis of observations for determination of feedback
CN111844029A (en) Robot early warning monitoring method and device
CN113888024A (en) Operation monitoring method and device, electronic equipment and storage medium
CN117041029A (en) Network equipment fault processing method and device, electronic equipment and storage medium
CN112465812A (en) Dust detection device and method for photovoltaic module
JP2020123230A (en) Trace data acquisition system, trace data acquisition method, and program
CN110727669A (en) Device and method for cleaning sensor data of power system
CN112183555B (en) Method and system for detecting welding quality, electronic device and storage medium
CN116579784A (en) Processing method and device for assembly data, searching method and system and electronic equipment
CN112400194B (en) System and method for triggering training events
US20200082281A1 (en) Verification device
CN111708654A (en) Method and equipment for repairing virtual machine fault
CN110597198A (en) Quality control device, quality control system and quality control method for TFT substrate glass
CN115827318A (en) Data center exception handling method and device, electronic equipment and medium
Cheng et al. Sensor selection and stage & result classifications for automated miniature screwdriving
CN114372500A (en) Intelligent factory control system based on big data
CN112785556A (en) Reinspection method, reinspection device, electronic equipment and computer-readable storage medium
CN112631272A (en) Method and equipment for remotely recovering power of robot
CN114001939B (en) Line connection device monitoring method, device, equipment and storage medium
KR20210009921A (en) Method and apparatus for reporting errors of robot

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination