CN113609335A - Target object searching method, system, electronic equipment and storage medium - Google Patents
Target object searching method, system, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113609335A CN113609335A CN202110924993.1A CN202110924993A CN113609335A CN 113609335 A CN113609335 A CN 113609335A CN 202110924993 A CN202110924993 A CN 202110924993A CN 113609335 A CN113609335 A CN 113609335A
- Authority
- CN
- China
- Prior art keywords
- information data
- main body
- similarity
- data
- query
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/7837—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using objects detected or recognised in the video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/7837—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using objects detected or recognised in the video content
- G06F16/784—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using objects detected or recognised in the video content the detected or recognised objects being people
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Abstract
The invention relates to an electronic information technology, and provides a target object searching method, a target object searching system, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring image data; inputting image data into two pre-trained preset type recognition models to obtain structured information data and descriptive information data; obtaining main information data according to the descriptive information data; establishing a relation between all main body information data corresponding to the same descriptive information data as a first relation; establishing a relationship between the structured information data and the corresponding main body information data as a second relationship; responding to query data input by a user, calculating similarity between the query data and the structured information, and when the similarity is greater than a first preset value, corresponding to the structured information data and a second relation according to the similarity to obtain query main body information data; and obtaining other main body information data corresponding to the query main body information data according to the query main body information data and the first relation. The workload of monitoring personnel is small.
Description
Technical Field
The present invention relates to electronic information technologies, and in particular, to a method and a system for searching a target object, an electronic device, and a storage medium.
Background
The security monitoring system is a system for displaying and recording field images in a monitoring and fortifying area by utilizing a video detection technology, can reflect a monitored object in real time, vividly and really, and mainly comprises four parts, namely front-end equipment, transmission equipment, processing/control equipment and recording/displaying equipment, so that the long-time monitoring is realized by replacing manpower, and people can see all the actual conditions of the monitored area.
In the current security field, a large amount of information is generated from cameras every day. When an emergency occurs, a monitoring person needs to manually find clues of target objects related to the emergency from a large amount of videos, the workload of the monitoring person is large, the target objects are difficult to track in time, further time delay is caused, and the optimal time for finding the target objects is easy to miss.
Disclosure of Invention
In view of the above, the present invention provides a target object searching method, a target object searching system, an electronic device, and a storage medium, and aims to solve the technical problems that the workload of a monitoring person is large, it is difficult to track a target object in time, time delay is further caused, and the best time for finding the target object is easily missed in the conventional target object searching method.
In order to achieve the above object, the present invention provides a target object searching method, including:
acquiring image data;
inputting the image data into a first pre-set type recognition model trained in advance to obtain structured information data; inputting the image data into a pre-trained second preset type recognition model to obtain descriptive information data; obtaining main information data according to the descriptive information data;
establishing a relationship between all main body information data corresponding to the same descriptive information data; establishing a relation between the structured information data and the corresponding main body information data;
responding to query data input by a user, calculating the similarity between the query data and the structured information to be used as first similarity, judging whether the first similarity is greater than a first preset value, and when the first similarity is greater than the first preset value, obtaining main information data corresponding to the structured information data corresponding to the first similarity according to the first similarity and the relation between the structured information data and the corresponding main information data to be used as query main information data; and obtaining other main body information data which belong to the same descriptive information data as the query main body information data according to the query main body information data and the relationship between the query main body information data and the other main body information data corresponding to the same descriptive information data.
In one embodiment, the pre-trained second predetermined type recognition model comprises an encoder and a decoder; the inputting the image data into a pre-trained second preset type recognition model to obtain descriptive information data comprises:
inputting the image data into the encoder to obtain an intermediate vector; and inputting the intermediate vector into the decoder to obtain descriptive information data.
In one embodiment, the obtaining the main information data according to the descriptive information data includes:
and matching the descriptive information data with a preset rule, and when the descriptive information data is successfully matched with the preset rule, taking the information successfully matched with the preset rule in the descriptive information data as main information data.
In one embodiment, the obtaining the main information data according to the descriptive information data includes:
and matching the descriptive information data with a preset rule, when the descriptive information data is not matched with the preset rule, calculating the similarity between the descriptive information data and the entity information data in the preset database as a second similarity, and when the second similarity is greater than a second preset value, using the entity information data corresponding to the second similarity as main information data.
In one embodiment, the method further includes determining whether the main body information data exists in a preset database, and when the main body information data does not exist in the preset database, newly establishing the main body information data in the preset database as entity information data.
In one embodiment, the establishing a relationship between the structured information data and the corresponding main body information data includes:
and establishing a relation between the structured information data and the corresponding main information data through a preset artificial dictionary.
In one embodiment, the second predetermined type recognition model is a long-term and short-term memory model.
In order to achieve the above object, the present invention further provides a target object searching system, including:
the acquisition module is used for acquiring image data;
the input module is used for inputting the image data into a first pre-set type recognition model which is trained in advance to obtain structured information data; inputting the image data into a pre-trained second preset type recognition model to obtain descriptive information data; obtaining main information data according to the descriptive information data;
the establishing module is used for establishing a relationship between all main body information data corresponding to the same descriptive information data; establishing a relation between the structured information data and the corresponding main body information data;
the computing module is used for responding to query data input by a user, computing the similarity between the query data and the structured information to serve as first similarity, judging whether the first similarity is larger than a first preset value, and when the first similarity is larger than the first preset value, obtaining main information data corresponding to the structured information data corresponding to the first similarity to serve as query main information data according to the first similarity corresponding to the structured information data and the relation between the structured information data and the corresponding main information data; and obtaining other main body information data which belong to the same descriptive information data as the query main body information data according to the query main body information data and the relationship between the query main body information data and the other main body information data corresponding to the same descriptive information data.
In order to achieve the above object, the present invention also provides an electronic device, including:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a target object finding method as described above.
To achieve the above object, the present invention further provides a computer readable storage medium, in which a target object searching program is stored, and when the target object searching program is executed by a processor, the steps of the target object searching method are implemented.
The invention provides a target object searching method, a target object searching system, electronic equipment and a storage medium, and image data are obtained; inputting the image data into a first pre-set type recognition model trained in advance to obtain structured information data; inputting the image data into a pre-trained second preset type recognition model to obtain descriptive information data; obtaining main information data according to the descriptive information data; establishing a relationship between all main body information data corresponding to the same descriptive information data; establishing a relation between the structured information data and the corresponding main body information data; responding to query data input by a user, calculating the similarity between the query data and the structured information to be used as first similarity, judging whether the first similarity is greater than a first preset value, and when the first similarity is greater than the first preset value, obtaining main information data corresponding to the structured information data corresponding to the first similarity according to the first similarity and the relation between the structured information data and the corresponding main information data to be used as query main information data; and obtaining other main body information data which belong to the same descriptive information data as the query main body information data according to the query main body information data and the relationship between the query main body information data and the other main body information data corresponding to the same descriptive information data. Therefore, the invention can inquire the relevant main body information data more timely and conveniently according to the inquiry data input by the user, has less workload of monitoring personnel, more conveniently and timely tracks the target object, effectively reduces time delay and more easily grasps the best opportunity for finding the target object.
Drawings
FIG. 1 is a diagram of an electronic device according to a preferred embodiment of the present invention;
FIG. 2 is a block diagram of a preferred embodiment of a target object search system according to the present invention;
FIG. 3 is a flowchart illustrating a preferred embodiment of a target object searching method according to the present invention;
the objects, features and advantages of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a schematic diagram of an electronic device 1 according to a preferred embodiment of the invention.
The electronic device 1 includes but is not limited to: memory 11, processor 12, display 13, and network interface 14. The electronic device 1 is connected to a network through a network interface 14 to obtain raw data. The network may be a wireless or wired network such as an Intranet (Intranet), the Internet (Internet), a Global System for Mobile communications (GSM), Wideband Code Division Multiple Access (WCDMA), a 4G network, a 5G network, Bluetooth (Bluetooth), Wi-Fi, or a communication network.
The memory 11 includes at least one type of readable storage medium including a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the storage 11 may be an internal storage unit of the electronic device 1, such as a hard disk or a memory of the electronic device 1. In other embodiments, the memory 11 may also be an external storage device of the electronic device 1, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like equipped with the electronic device 1. Of course, the memory 11 may also comprise both an internal memory unit and an external memory device of the electronic device 1. In this embodiment, the memory 11 is generally used for storing an operating system installed in the electronic device 1 and various application software, such as a program code of the target object searching program 10. Further, the memory 11 may also be used to temporarily store various types of data that have been output or are to be output.
Processor 12 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 12 is typically used for controlling the overall operation of the electronic device 1, such as performing data interaction or communication related control and processing. In this embodiment, the processor 12 is configured to run the program code stored in the memory 11 or process data, for example, run the program code of the target object search program 10.
The display 13 may be referred to as a display screen or display unit. In some embodiments, the display 13 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an Organic Light-Emitting Diode (OLED) touch screen, or the like. The display 13 is used for displaying information processed in the electronic device 1 and for displaying a visual work interface, e.g. displaying the results of data statistics.
The network interface 14 may optionally comprise a standard wired interface, a wireless interface (e.g. WI-FI interface), the network interface 14 typically being used for establishing a communication connection between the electronic device 1 and other electronic devices.
Fig. 1 shows only the electronic device 1 and the cloud database 2 with components 11-14 and the target object finding program 10, but it is to be understood that not all of the shown components are required to be implemented, and that more or fewer components may be implemented instead.
Optionally, the electronic device 1 may further comprise a user interface, the user interface may comprise a Display (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface may further comprise a standard wired interface and a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an Organic Light-Emitting Diode (OLED) touch screen, or the like. The display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the electronic device 1 and for displaying a visualized user interface, among other things.
The electronic device 1 may further include a Radio Frequency (RF) circuit, a sensor, an audio circuit, and the like, which are not described in detail herein.
In the above embodiment, the processor 12 may implement the following steps when executing the target object searching program 10 stored in the memory 11:
acquiring image data;
inputting the image data into a first pre-set type recognition model trained in advance to obtain structured information data; inputting the image data into a pre-trained second preset type recognition model to obtain descriptive information data; obtaining main information data according to the descriptive information data;
establishing a relationship between all main body information data corresponding to the same descriptive information data; establishing a relation between the structured information data and the corresponding main body information data;
responding to query data input by a user, calculating the similarity between the query data and the structured information to be used as first similarity, judging whether the first similarity is greater than a first preset value, and when the first similarity is greater than the first preset value, obtaining main information data corresponding to the structured information data corresponding to the first similarity according to the first similarity and the relation between the structured information data and the corresponding main information data to be used as query main information data; and obtaining other main body information data which belong to the same descriptive information data as the query main body information data according to the query main body information data and the relationship between the query main body information data and the other main body information data corresponding to the same descriptive information data.
For a detailed description of the above steps, please refer to the following description of fig. 2 regarding a functional block diagram of an embodiment of the target object searching system 100 and fig. 3 regarding a flowchart of an embodiment of the target object searching method.
Referring to fig. 2, a functional block diagram of the target object searching system 100 according to the present invention is shown.
The target object finding system 100 of the present invention may be installed in an electronic device. The target object lookup system 100 may employ a Browser/Server (B/S) schema to manage the platform. Depending on the implemented functionality, the target object finding system 100 may comprise an obtaining module 110, an input module 120, a building module 130 and a calculating module 140. The module in the present invention may also be referred to as a unit, and refers to a series of computer program segments that can be executed by a processor of an electronic device and can perform a fixed function, and are stored in a memory of the electronic device.
In the present embodiment, the functions regarding the respective modules/units are as follows:
the obtaining module 110 is configured to obtain image data.
In the present embodiment, for example, the camera is controlled to capture image data, thereby obtaining image data. Of course, the image data input by the user can also be received. The image data includes video and/or pictures.
An input module 120, configured to input the image data into a pre-trained first preset type recognition model to obtain structured information data; inputting the image data into a pre-trained second preset type recognition model to obtain descriptive information data; and obtaining main information data according to the descriptive information data.
In this embodiment, one or more copies of the acquired image data may be obtained, where one copy of the image data is input into the first predetermined type identification model, and the other copy of the image data is input into the second predetermined type identification model. The pre-trained first preset type recognition model is a pre-trained deep learning model and can convert image data into structured information data. The pre-trained second preset type recognition model is a pre-trained long-short term memory model. When the image data is a video, the first preset type identification model can respectively identify the structural information data of each frame of image, and the second preset type identification model can respectively identify the descriptive information data of each frame of image. In some embodiments, the descriptive information data includes subject information data, which may be extracted directly from the descriptive information data.
For example, a picture containing pedestrians and vehicles is input into a first preset type recognition model, and structured information data output by the first preset type recognition model are divided into two groups. One set of structured information data is "age: young, whether wearing safety helmet: the color of the upper part of the body is: yellow, sex: in the male. The other set of structured information data is "vehicle type: truck, color: red, number plate: XX'. The same picture is input into a second preset type recognition model, and descriptive information data output by the second preset type recognition model is 'a youth wearing a yellow jacket passes by near a truck'.
Optionally, the pre-trained second preset type recognition model comprises an encoder and a decoder; the inputting the image data into a pre-trained second preset type recognition model to obtain descriptive information data comprises: inputting the image data into the encoder to obtain an intermediate vector; and inputting the intermediate vector into the decoder to obtain descriptive information data.
It will be appreciated that the encoder employs a deep convolutional neural network for converting the image into intermediate vectors of fixed length. The decoder is a recurrent neural network and is used to generate the target statement.
Optionally, the obtaining the main information data according to the descriptive information data includes: and matching the descriptive information data with a preset rule, and when the descriptive information data is successfully matched with the preset rule, taking the information successfully matched with the preset rule in the descriptive information data as main information data.
It is understood that the preset rules are preset and may include trucks, adolescents, teenagers, bicycles, etc. Taking the descriptive information data obtained from the picture as an example, the descriptive information data can be subjected to word segmentation to obtain target words, matching the corresponding target words of ' a youth wearing a yellow jacket passes near a truck ' with the preset rules, and taking the youth ' and the truck ' as two main information data if the matching of the corresponding target words of ' the youth ' and the truck ' in the descriptive information data target words with the preset rules is successful.
Optionally, the obtaining the main information data according to the descriptive information data includes: and matching the descriptive information data with a preset rule, when the descriptive information data is not matched with the preset rule, calculating the similarity between the descriptive information data and the entity information data in the preset database as a second similarity, and when the second similarity is greater than a second preset value, using the entity information data corresponding to the second similarity as main information data.
It will be appreciated that the information obtained from the original information is relatively scattered and has no chapter, and sometimes even the descriptive information and the structured information are very different. The preset database stores entity information data, and the entity information data are manually defined relatively standard information data. The main information data corresponding to the descriptive information data can be obtained through similarity calculation. And when the second similarity is smaller than or equal to a second preset value, returning a result without main body information data to prompt the user so as to facilitate the user to carry out manual marking in time.
Optionally, the system further includes a determining module, configured to determine whether the main body information data exists in a preset database, and when the main body information data does not exist in the preset database, newly establish the main body information data in the preset database as entity information data.
It can be understood that when the main body information data does not exist in the preset database, the main body information data is newly established in the preset database as the entity information data, and the entity information data in the database can be updated.
An establishing module 130, configured to establish a relationship between all main information data corresponding to the same descriptive information data; and establishing a relation between the structured information data and the corresponding main body information data.
In this embodiment, the establishing a relationship between the structured information data and the corresponding main information data includes: and establishing a relation between the structured information data and the corresponding main information data through a preset artificial dictionary. Specifically, the dictionary matching method is adopted to map by constructing an artificial dictionary, and here, some common mapping relations between the structured information and the main body information are defined in an artificial way.
Taking the aforementioned main body information data and structured information data as an example, a relationship is established between two main body information data "youth" and "truck" belonging to the same descriptive information data. The structured information data "age: young, whether wearing safety helmet: the color of the upper part of the body is: yellow, sex: the relation between the male and the main body information data is established. And taking the structured information data as a vehicle type: truck, color: red, number plate: XX "establishes a relationship with the subject information data" truck ".
A calculating module 140, configured to respond to query data input by a user, calculate a similarity between the query data and the structured information as a first similarity, determine whether the first similarity is greater than a first preset value, and when the first similarity is greater than the first preset value, obtain, according to the first similarity corresponding to the structured information data and a relationship between the structured information data and corresponding main information data, main information data corresponding to the first similarity corresponding to the structured information data as query main information data; and obtaining other main body information data which belong to the same descriptive information data as the query main body information data according to the query main body information data and the relationship between the query main body information data and the other main body information data corresponding to the same descriptive information data.
In this embodiment, for example, the user inputs query data "yellow jacket man", and calculates the color of the upper body of the query data and the structured information "upper body color: yellow, sex: male's similarity, the similarity is greater than a first preset value, and the upper body color of the structured information data is determined according to the similarity: yellow, sex: male "and the structured information data" upper body color: yellow, sex: the relationship between "male" and "young" corresponding to the main body information data can be used to obtain "young" corresponding to the structured information data. Then, the subject information data "truck" associated with the subject information data "youth" is obtained from the subject information data "youth" and the relationship between the subject information data "youth" and another subject information data "truck" belonging to the same descriptive information data. Related clues of the target object are found more accurately. According to the main body information data ' truck ' and the corresponding structured information data ' vehicle type: truck, color: red, number plate: XX ", obtaining a vehicle type of structured information data corresponding to the subject information data" truck ": truck, color: red, number plate: XX ", to easily find the truck and tie the truck driver.
Optionally, the system further includes a calculation sub-module, configured to calculate the number of image data, determine whether there are multiple (two or more) image data, and when there are multiple image data, input the image data into a pre-trained first preset type recognition model to obtain structured information data; inputting the image data into a pre-trained second preset type recognition model to obtain descriptive information data; obtaining main body information data according to the descriptive information data, and establishing a relationship between all main body information data corresponding to the same descriptive information data; establishing a relationship between the structured information data and the corresponding subject information data includes:
respectively inputting a plurality of image data into a pre-trained first preset type recognition model to obtain structural information data corresponding to each image data; respectively inputting the plurality of image data into a pre-trained second preset type recognition model to obtain descriptive information data corresponding to each image data; obtaining main body information data corresponding to each image data according to the descriptive information data corresponding to each image data;
establishing a relationship between all main body information data corresponding to the same descriptive information data; establishing a relation between the structured information data and the corresponding main body information data;
the system also comprises a calculation association module used for judging whether the same main body information data exists in all main body information data corresponding to all the descriptive information data, when the same main body information data exists, the same main body information data is used as the associated main body information data, a plurality of descriptive information data corresponding to the associated main body information data are all used as the associated information data, and all the main body information data corresponding to all the associated information data are established into a relationship. In this way, a knowledge graph can be formed.
Responding to search data input by a user, calculating the similarity between the search data and the structured information to serve as a third similarity, judging whether the third similarity is larger than a third preset value, and when the third similarity is larger than the third preset value, obtaining main information data corresponding to the structured information data corresponding to the third similarity to serve as search main information data according to the structured information data corresponding to the third similarity and the relation between the structured information data and the corresponding main information data; and obtaining other main body information data having a relationship with the searching main body information data according to the searching main body information data and the correlation between the searching main body information data and the searching main body information data.
It can be understood that the object a and the object B appear in the image one, and the object C and the object B appear in the image two, so that the structured information data one and the descriptive information data one corresponding to the image one, and the structured information data two and the descriptive information data two corresponding to the image two can be obtained through the above steps. Thus, the main body information data "object a" and "object B" corresponding to the descriptive information data one, and the main body information data "object C" and "object B" corresponding to the descriptive information data two are obtained. And judging that the object B is the same main body information data of the two descriptive information data, taking the object B as associated main body information data, taking the descriptive information data I and the descriptive information data II corresponding to the object B as associated information data, and establishing a relationship between all the main body information data of the descriptive information data I and the descriptive information data II, namely the object A, the object B and the object C. Thus, when the object A is queried, the object B and the object C can be obtained according to the object A and the relationship between the object A and the object B and between the object A and the object C.
The target object searching system provided by the invention obtains image data; inputting the image data into a first pre-set type recognition model trained in advance to obtain structured information data; inputting the image data into a pre-trained second preset type recognition model to obtain descriptive information data; obtaining main information data according to the descriptive information data; establishing a relationship between all main body information data corresponding to the same descriptive information data; establishing a relation between the structured information data and the corresponding main body information data; responding to query data input by a user, calculating the similarity between the query data and the structured information to be used as first similarity, judging whether the first similarity is greater than a first preset value, and when the first similarity is greater than the first preset value, obtaining main information data corresponding to the structured information data corresponding to the first similarity according to the first similarity and the relation between the structured information data and the corresponding main information data to be used as query main information data; and obtaining other main body information data which belong to the same descriptive information data as the query main body information data according to the query main body information data and the relationship between the query main body information data and the other main body information data corresponding to the same descriptive information data. Therefore, the invention can inquire the relevant main body information data more timely and conveniently according to the inquiry data input by the user, has less workload of monitoring personnel, more conveniently and timely tracks the target object, effectively reduces time delay and more easily grasps the best opportunity for finding the target object.
In addition, the invention also provides a target object searching method, which is applied to the electronic equipment. Fig. 3 is a schematic method flow diagram illustrating an embodiment of the target object searching method according to the present invention. The processor 12 of the electronic device 1, when executing the target object finding program 10 stored in the memory 11, implements the following steps of the target object finding method:
step S10: image data is acquired.
In the present embodiment, for example, the camera is controlled to capture image data, thereby obtaining image data. Of course, the image data input by the user can also be received. The image data includes video and/or pictures.
Step S20: inputting the image data into a first pre-set type recognition model trained in advance to obtain structured information data; inputting the image data into a pre-trained second preset type recognition model to obtain descriptive information data; and obtaining main information data according to the descriptive information data.
In this embodiment, the acquired image data may be copied into two copies, wherein one copy of the image data is input into the first predetermined type identification model, and the other copy of the image data is input into the second predetermined type identification model. The pre-trained first preset type recognition model is a pre-trained deep learning model and can convert image data into structured information data. The pre-trained second preset type recognition model is a pre-trained long-short term memory model. When the image data is a video, the first preset type identification model can respectively identify the structural information data of each frame of image, and the second preset type identification model can respectively identify the descriptive information data of each frame of image. In some embodiments, the descriptive information data includes subject information data, which may be extracted directly from the descriptive information data.
For example, a picture containing pedestrians and vehicles is input into a first preset type recognition model, and structured information data output by the first preset type recognition model are divided into two groups. One set of structured information data is "age: young, whether wearing safety helmet: the color of the upper part of the body is: yellow, sex: in the male. The other set of structured information data is "vehicle type: truck, color: red, number plate: XX'. The same picture is input into a second preset type recognition model, and descriptive information data output by the second preset type recognition model is 'a youth wearing a yellow jacket passes by near a truck'.
Optionally, the pre-trained second preset type recognition model comprises an encoder and a decoder; the inputting the image data into a pre-trained second preset type recognition model to obtain descriptive information data comprises: inputting the image data into the encoder to obtain an intermediate vector; and inputting the intermediate vector into the decoder to obtain descriptive information data.
It will be appreciated that the encoder employs a deep convolutional neural network for converting the image into intermediate vectors of fixed length. The decoder is a recurrent neural network and is used to generate the target statement.
Optionally, the obtaining the main information data according to the descriptive information data includes: and matching the descriptive information data with a preset rule, and when the descriptive information data is successfully matched with the preset rule, taking the information successfully matched with the preset rule in the descriptive information data as main information data.
It is understood that the preset rules are preset and may include trucks, adolescents, teenagers, bicycles, etc. Taking the descriptive information data obtained from the picture as an example, the descriptive information data can be subjected to word segmentation to obtain target words, matching the corresponding target words of ' a youth wearing a yellow jacket passes near a truck ' with the preset rules, and taking the youth ' and the truck ' as two main information data if the matching of the corresponding target words of ' the youth ' and the truck ' in the descriptive information data target words with the preset rules is successful.
Optionally, the obtaining the main information data according to the descriptive information data includes: and matching the descriptive information data with a preset rule, when the descriptive information data is not matched with the preset rule, calculating the similarity between the descriptive information data and the entity information data in the preset database as a second similarity, and when the second similarity is greater than a second preset value, using the entity information data corresponding to the second similarity as main information data.
It will be appreciated that the information obtained from the original information is relatively scattered and has no chapter, and sometimes even the descriptive information and the structured information are very different. The preset database stores entity information data, and the entity information data are manually defined relatively standard information data. The main information data corresponding to the descriptive information data can be obtained through similarity calculation. And when the second similarity is smaller than or equal to a second preset value, returning a result without main body information data to prompt the user so as to facilitate the user to carry out manual marking in time.
Optionally, the method further includes determining whether the main body information data exists in a preset database, and when the main body information data does not exist in the preset database, newly building the main body information data in the preset database as entity information data.
It can be understood that when the main body information data does not exist in the preset database, the main body information data is newly established in the preset database as the entity information data, and the entity information data in the database can be updated.
Step S30: establishing a relationship between all main body information data corresponding to the same descriptive information data; and establishing a relation between the structured information data and the corresponding main body information data.
In this embodiment, the establishing a relationship between the structured information data and the corresponding main information data includes: and establishing a relation between the structured information data and the corresponding main information data through a preset artificial dictionary. Specifically, the dictionary matching method is adopted to map by constructing an artificial dictionary, and here, some common mapping relations between the structured information and the main body information are defined in an artificial way.
Taking the aforementioned main body information data and structured information data as an example, a relationship is established between two main body information data "youth" and "truck" belonging to the same descriptive information data. The structured information data "age: young, whether wearing safety helmet: the color of the upper part of the body is: yellow, sex: the relation between the male and the main body information data is established. And taking the structured information data as a vehicle type: truck, color: red, number plate: XX "establishes a relationship with the subject information data" truck ".
Step S40: responding to query data input by a user, calculating the similarity between the query data and the structured information to be used as first similarity, judging whether the first similarity is greater than a first preset value, and when the first similarity is greater than the first preset value, obtaining main information data corresponding to the structured information data corresponding to the first similarity according to the first similarity and the relation between the structured information data and the corresponding main information data to be used as query main information data; and obtaining other main body information data which belong to the same descriptive information data as the query main body information data according to the query main body information data and the relationship between the query main body information data and the other main body information data corresponding to the same descriptive information data.
In this embodiment, for example, the user inputs query data "yellow jacket man", and calculates the color of the upper body of the query data and the structured information "upper body color: yellow, sex: male's similarity, the similarity is greater than a first preset value, and the upper body color of the structured information data is determined according to the similarity: yellow, sex: male "and the structured information data" upper body color: yellow, sex: the relationship between "male" and "young" corresponding to the main body information data can be used to obtain "young" corresponding to the structured information data. Then, the subject information data "truck" associated with the subject information data "youth" is obtained from the subject information data "youth" and the relationship between the subject information data "youth" and another subject information data "truck" belonging to the same descriptive information data. Related clues of the target object are found more accurately. According to the main body information data ' truck ' and the corresponding structured information data ' vehicle type: truck, color: red, number plate: XX ", obtaining a vehicle type of structured information data corresponding to the subject information data" truck ": truck, color: red, number plate: XX ", to easily find the truck and tie the truck driver.
Optionally, the method further includes calculating the number of the image data, determining whether the image data is multiple (two or more), and when the number of the image data is multiple, inputting the image data into a first pre-set type recognition model trained in advance to obtain structured information data; inputting the image data into a pre-trained second preset type recognition model to obtain descriptive information data; obtaining main body information data according to the descriptive information data, and establishing a relationship between all main body information data corresponding to the same descriptive information data; establishing a relationship between the structured information data and the corresponding subject information data includes:
respectively inputting a plurality of image data into a pre-trained first preset type recognition model to obtain structural information data corresponding to each image data; respectively inputting the plurality of image data into a pre-trained second preset type recognition model to obtain descriptive information data corresponding to each image data; obtaining main body information data corresponding to each image data according to the descriptive information data corresponding to each image data;
establishing a relationship between all main body information data corresponding to the same descriptive information data; establishing a relation between the structured information data and the corresponding main body information data;
the method further comprises the step of judging whether the same main body information data exists in all main body information data corresponding to all the descriptive information data, when the same main body information data exists, the same main body information data is used as associated main body information data, a plurality of descriptive information data corresponding to the associated main body information data are used as associated information data, and all main body information data corresponding to all the associated information data are used for establishing a relationship to serve as an association relationship. In this way, a knowledge graph can be formed.
Responding to search data input by a user, calculating the similarity between the search data and the structured information to serve as a third similarity, judging whether the third similarity is larger than a third preset value, and when the third similarity is larger than the third preset value, obtaining main information data corresponding to the structured information data corresponding to the third similarity to serve as search main information data according to the structured information data corresponding to the third similarity and the relation between the structured information data and the corresponding main information data; and obtaining other main body information data having a relationship with the searching main body information data according to the searching main body information data and the correlation between the searching main body information data and the searching main body information data.
It can be understood that the object a and the object B appear in the image one, and the object C and the object B appear in the image two, so that the structured information data one and the descriptive information data one corresponding to the image one, and the structured information data two and the descriptive information data two corresponding to the image two can be obtained through the above steps. Thus, the main body information data "object a" and "object B" corresponding to the descriptive information data one, and the main body information data "object C" and "object B" corresponding to the descriptive information data two are obtained. And judging that the object B is the same main body information data of the two descriptive information data, taking the object B as associated main body information data, taking the descriptive information data I and the descriptive information data II corresponding to the object B as associated information data, and establishing a relationship between all the main body information data of the descriptive information data I and the descriptive information data II, namely the object A, the object B and the object C. Thus, when the object A is queried, the object B and the object C can be obtained according to the object A and the relationship between the object A and the object B and between the object A and the object C.
The target object searching method provided by the invention obtains image data; inputting the image data into a first pre-set type recognition model trained in advance to obtain structured information data; inputting the image data into a pre-trained second preset type recognition model to obtain descriptive information data; obtaining main information data according to the descriptive information data; establishing a relationship between all main body information data corresponding to the same descriptive information data; establishing a relation between the structured information data and the corresponding main body information data; responding to query data input by a user, calculating the similarity between the query data and the structured information to be used as first similarity, judging whether the first similarity is greater than a first preset value, and when the first similarity is greater than the first preset value, obtaining main information data corresponding to the structured information data corresponding to the first similarity according to the first similarity and the relation between the structured information data and the corresponding main information data to be used as query main information data; and obtaining other main body information data which belong to the same descriptive information data as the query main body information data according to the query main body information data and the relationship between the query main body information data and the other main body information data corresponding to the same descriptive information data. Therefore, the invention can inquire the relevant main body information data more timely and conveniently according to the inquiry data input by the user, has less workload of monitoring personnel, more conveniently and timely tracks the target object, effectively reduces time delay and more easily grasps the best opportunity for finding the target object.
Furthermore, the embodiment of the present invention also provides a computer-readable storage medium, which may be any one or any combination of a hard disk, a multimedia card, an SD card, a flash memory card, an SMC, a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM), a portable compact disc read only memory (CD-ROM), a USB memory, and the like. The computer-readable storage medium includes a storage data area and a storage program area, the storage data area stores data created according to the use of the blockchain node, the storage program area stores a target object search program 10, and when executed by a processor, the target object search program 10 implements the following operations:
acquiring image data;
inputting the image data into a first pre-set type recognition model trained in advance to obtain structured information data; inputting the image data into a pre-trained second preset type recognition model to obtain descriptive information data; obtaining main information data according to the descriptive information data;
establishing a relationship between all main body information data corresponding to the same descriptive information data; establishing a relation between the structured information data and the corresponding main body information data;
responding to query data input by a user, calculating the similarity between the query data and the structured information to be used as first similarity, judging whether the first similarity is greater than a first preset value, and when the first similarity is greater than the first preset value, obtaining main information data corresponding to the structured information data corresponding to the first similarity according to the first similarity and the relation between the structured information data and the corresponding main information data to be used as query main information data; and obtaining other main body information data which belong to the same descriptive information data as the query main body information data according to the query main body information data and the relationship between the query main body information data and the other main body information data corresponding to the same descriptive information data.
It should be emphasized that the embodiments of the computer-readable storage medium of the present invention are substantially the same as the embodiments of the target object searching method described above, and are not described herein again.
The specific implementation of the computer-readable storage medium of the present invention is substantially the same as the specific implementation of the target object searching method, and is not described herein again.
It should be noted that the above-mentioned numbers of the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments. And the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, system, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, system, article, or method. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, system, article, or method that includes the element.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (such as a mobile phone, a computer, an electronic system, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A target object searching method is characterized by comprising the following steps:
acquiring image data;
inputting the image data into a first pre-set type recognition model trained in advance to obtain structured information data; inputting the image data into a pre-trained second preset type recognition model to obtain descriptive information data; obtaining main information data according to the descriptive information data;
establishing a relationship between all main body information data corresponding to the same descriptive information data; establishing a relation between the structured information data and the corresponding main body information data;
responding to query data input by a user, calculating the similarity between the query data and the structured information to be used as first similarity, judging whether the first similarity is greater than a first preset value, and when the first similarity is greater than the first preset value, obtaining main information data corresponding to the structured information data corresponding to the first similarity according to the first similarity and the relation between the structured information data and the corresponding main information data to be used as query main information data; and obtaining other main body information data which belong to the same descriptive information data as the query main body information data according to the query main body information data and the relationship between the query main body information data and the other main body information data corresponding to the same descriptive information data.
2. The method of claim 1, wherein the pre-trained second pre-set type recognition model comprises an encoder and a decoder; the inputting the image data into a pre-trained second preset type recognition model to obtain descriptive information data comprises:
inputting the image data into the encoder to obtain an intermediate vector; and inputting the intermediate vector into the decoder to obtain descriptive information data.
3. The method for finding a target object according to claim 1, wherein the obtaining of the main body information data according to the descriptive information data comprises:
and matching the descriptive information data with a preset rule, and when the descriptive information data is successfully matched with the preset rule, taking the information successfully matched with the preset rule in the descriptive information data as main information data.
4. The method for finding a target object according to claim 1, wherein the obtaining of the main body information data according to the descriptive information data comprises:
and matching the descriptive information data with a preset rule, when the descriptive information data is not matched with the preset rule, calculating the similarity between the descriptive information data and the entity information data in the preset database as a second similarity, and when the second similarity is greater than a second preset value, using the entity information data corresponding to the second similarity as main information data.
5. The method according to claim 1, further comprising determining whether the main body information data exists in a preset database, and when the main body information data does not exist in the preset database, newly creating the main body information data in the preset database as entity information data.
6. The method for finding a target object according to claim 1, wherein the correlating the structured information data with the corresponding subject information data comprises:
and establishing a relation between the structured information data and the corresponding main information data through a preset artificial dictionary.
7. The method as claimed in claim 1, wherein the second predetermined type recognition model is a long-short term memory model.
8. A target object searching system applied to electronic equipment is characterized by comprising:
the acquisition module is used for acquiring image data;
the input module is used for inputting the image data into a first pre-set type recognition model which is trained in advance to obtain structured information data; inputting the image data into a pre-trained second preset type recognition model to obtain descriptive information data; obtaining main information data according to the descriptive information data;
the establishing module is used for establishing a relationship between all main body information data corresponding to the same descriptive information data; establishing a relation between the structured information data and the corresponding main body information data;
the computing module is used for responding to query data input by a user, computing the similarity between the query data and the structured information to serve as first similarity, judging whether the first similarity is larger than a first preset value, and when the first similarity is larger than the first preset value, obtaining main information data corresponding to the structured information data corresponding to the first similarity to serve as query main information data according to the first similarity corresponding to the structured information data and the relation between the structured information data and the corresponding main information data; and obtaining other main body information data which belong to the same descriptive information data as the query main body information data according to the query main body information data and the relationship between the query main body information data and the other main body information data corresponding to the same descriptive information data.
9. An electronic device, characterized in that the electronic device comprises:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a target object finding method as claimed in any one of claims 1 to 7.
10. A computer-readable storage medium, in which a target object finding program is stored, which when executed by a processor implements the steps of the target object finding method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110924993.1A CN113609335B (en) | 2021-08-12 | 2021-08-12 | Target object searching method, system, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110924993.1A CN113609335B (en) | 2021-08-12 | 2021-08-12 | Target object searching method, system, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113609335A true CN113609335A (en) | 2021-11-05 |
| CN113609335B CN113609335B (en) | 2023-02-03 |
Family
ID=78308325
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110924993.1A Active CN113609335B (en) | 2021-08-12 | 2021-08-12 | Target object searching method, system, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113609335B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100046842A1 (en) * | 2008-08-19 | 2010-02-25 | Conwell William Y | Methods and Systems for Content Processing |
| CN105900094A (en) * | 2014-01-15 | 2016-08-24 | 微软技术许可有限责任公司 | Automated multimedia content recognition |
| CN106682060A (en) * | 2015-11-11 | 2017-05-17 | 奥多比公司 | Structured Knowledge Modeling, Extraction and Localization from Images |
| US20180060356A1 (en) * | 2015-03-13 | 2018-03-01 | Hitachi, Ltd. | Image Search Device and Method for Searching Image |
| CN110674112A (en) * | 2019-09-23 | 2020-01-10 | 北京百分点信息科技有限公司 | Data query method and device and electronic equipment |
| CN111552792A (en) * | 2020-04-30 | 2020-08-18 | 中国建设银行股份有限公司 | Information query method and device, electronic equipment and storage medium |
| CN111581623A (en) * | 2020-05-09 | 2020-08-25 | 深圳物控智联科技有限公司 | Intelligent data interaction method and device, electronic equipment and storage medium |
-
2021
- 2021-08-12 CN CN202110924993.1A patent/CN113609335B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100046842A1 (en) * | 2008-08-19 | 2010-02-25 | Conwell William Y | Methods and Systems for Content Processing |
| CN105900094A (en) * | 2014-01-15 | 2016-08-24 | 微软技术许可有限责任公司 | Automated multimedia content recognition |
| US20180060356A1 (en) * | 2015-03-13 | 2018-03-01 | Hitachi, Ltd. | Image Search Device and Method for Searching Image |
| CN106682060A (en) * | 2015-11-11 | 2017-05-17 | 奥多比公司 | Structured Knowledge Modeling, Extraction and Localization from Images |
| CN110674112A (en) * | 2019-09-23 | 2020-01-10 | 北京百分点信息科技有限公司 | Data query method and device and electronic equipment |
| CN111552792A (en) * | 2020-04-30 | 2020-08-18 | 中国建设银行股份有限公司 | Information query method and device, electronic equipment and storage medium |
| CN111581623A (en) * | 2020-05-09 | 2020-08-25 | 深圳物控智联科技有限公司 | Intelligent data interaction method and device, electronic equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 倪春乐: ""大数据背景下的侦查创新与现实局限"", 《公安学研究》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113609335B (en) | 2023-02-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10872424B2 (en) | Object tracking using object attributes | |
| EP3471021A1 (en) | Method for determining target intelligently followed by unmanned aerial vehicle, unmanned aerial vehicle and remote controller | |
| US20190057244A1 (en) | Method for determining target through intelligent following of unmanned aerial vehicle, unmanned aerial vehicle and remote control | |
| US20160371305A1 (en) | Method, device and apparatus for generating picture search library, and picture search method, device and apparatus | |
| CN110781768A (en) | Target object detection method and device, electronic device and medium | |
| US20130243249A1 (en) | Electronic device and method for recognizing image and searching for concerning information | |
| WO2018121006A1 (en) | Method and device for license plate positioning | |
| US9633272B2 (en) | Real time object scanning using a mobile phone and cloud-based visual search engine | |
| US20130258198A1 (en) | Video search system and method | |
| US20170352162A1 (en) | Region-of-interest extraction device and region-of-interest extraction method | |
| CN112100431B (en) | Evaluation method, device and equipment of OCR system and readable storage medium | |
| CN109636582B (en) | Credit information management method, apparatus, device and storage medium | |
| CN110723432A (en) | Garbage classification method and augmented reality equipment | |
| CN111209490A (en) | Friend-making recommendation method based on user information, electronic device and storage medium | |
| CN111553302B (en) | Key frame selection method, device, equipment and computer readable storage medium | |
| CN112561973A (en) | Method and device for training image registration model and electronic equipment | |
| CN115331150A (en) | Image recognition method, image recognition device, electronic equipment and storage medium | |
| CN111177568B (en) | Object pushing method based on multi-source data, electronic device and storage medium | |
| CN113971821A (en) | Driver information determination method and device, terminal device and storage medium | |
| CN113704184A (en) | File classification method, device, medium and equipment | |
| CN113689475A (en) | Cross-border head trajectory tracking method, equipment and storage medium | |
| CN113609335B (en) | Target object searching method, system, electronic equipment and storage medium | |
| CN110619304A (en) | Vehicle type recognition method, system, device and computer readable medium | |
| CN111126049A (en) | Object relation prediction method and device, terminal equipment and readable storage medium | |
| CN112836115A (en) | Information recommendation method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |