KR102597045B1 - Method, system and non-transitory computer-readable recording medium for estimating information on objects based on image in lpwan environment - Google Patents

Abstract

According to one aspect of the present invention, as a method for estimating information about an object based on an image in an LPWAN (Low Power Wide Area Network) environment, object detection is learned to estimate upper category information of an object to be detected in a captured image. Obtaining a result of detecting the object in a captured image of the object using a model and a partial image related to the captured image generated based on the detection result, and obtaining a partial image related to the captured image from the partial image related to the captured image. Using an object classification model learned to estimate subcategory information, estimating subcategory information of the object based on the obtained detection result and the partial image, wherein the detection result includes the upper category of the object. A method by which information is included is provided.

Description

Method, system, and non-transitory computer-readable recording medium for estimating information about an object based on an image in an LPWAN (Low Power Wide Area Network) environment {METHOD, SYSTEM AND NON-TRANSITORY COMPUTER-READABLE RECORDING MEDIUM FOR ESTIMATING INFORMATION ON OBJECTS BASED ON IMAGE IN LPWAN ENVIRONMENT}

The present invention relates to a method, system, and non-transitory computer-readable recording medium for estimating information about an object based on an image in a Low Power Wide Area Network (LPWAN) environment.

In recent years, as artificial intelligence-related technology has developed rapidly, various technologies for detecting objects in images using artificial intelligence have been introduced.

As an example of the prior art in this regard, there is a technology for detecting a specific object in the above captured image by analyzing the captured image acquired by a plurality of network cameras using an artificial neural network model. In this way, an artificial neural network Since a significant amount of calculations must be processed to detect objects in an image using , it is common to have a computer system that includes a high-end GPU (Graphic Processor Unit).

In addition, if it is necessary to obtain images in real time from multiple network cameras and respond in real time with the results of detecting a specific object in the images, not only is the cost of building such a computer system higher, but also the multiple As the amount of data exchanged between network cameras and the above computer system increases, communication costs also become very high.

Recently, as an alternative to reduce costs, industrial edge computing solutions that utilize lightweight GPUs have emerged, but due to the nature of GPUs that inherently process large amounts of calculations, there is a problem in that they consume more power than simple embedded systems. .

Additionally, recently, instead of using high-specification GPUs, microprocessors or FPGAs (Field Programmable Gate Arrays) with functions to accelerate only matrix multiplication operations are built into embedded systems to create artificial neural network models at low cost without consuming large amounts of power. There are also attempts to support object detection based on However, in this case, not only must the artificial neural network model, which is software, be modified and lightweight to suit the above-mentioned special hardware, but there is also a problem that the performance of the artificial neural network model deteriorates due to such modification and lightweighting.

Meanwhile, concepts such as edge computing and on-device AI, which have recently been in the spotlight, reduce the load on the central server by distributing its functions and reduce the amount of data exchanged with the central server through the communication network. The goal is to increase the model’s response speed. In particular, reducing the communication load, that is, the amount of data exchanged with the central server, is a factor that not only increases the response speed of the AI model, but also has a direct impact on reducing system construction costs.

However, in building an object detection system using an artificial neural network model, a processor with limited computing power is used instead of a high-end GPU for the purpose of reducing system construction costs, and a broadband communication network such as Ethernet, Wi-Fi, and LTE is used. Instead, if you use a communication network with limited bandwidth such as LPWAN (Low Power Wide Area Network) (i.e., minimize the amount of data exchanged with the server), the artificial neural network model will be better than if you use a high-end GPU and broadband communication network. There is a high possibility that performance will deteriorate. As described above, object detection based on an artificial neural network model requires the processing of many operations.

Accordingly, the present inventor(s) supports the construction of an object detection system using an artificial neural network model at a low cost by using a network camera with limited computing power and an LPWAN with limited bandwidth, while supporting the performance of the artificial neural network model. We propose a technology that supports maintaining high detection speed and accuracy.

Registered Patent Publication No. 10-2029751 (October 8, 2019)

The purpose of the present invention is to solve all the problems of the prior art described above.

In addition, the present invention uses an object detection model that is learned to estimate the upper category information of the object to be detected in the captured image, based on the result of detecting the above object in the captured image of the above object and the above detection result. By acquiring a partial image related to the above captured image and using an object classification model that is learned to estimate subcategory information of the above object from the partial image related to the above captured image, the obtained detection results and the above Another purpose is to estimate the lower category information of the above object based on the partial image of and include the upper category information of the above object in the above detection result.

In addition, the present invention detects the above object in the captured image of the above object using an object detection model that is learned to estimate the upper category information of the object to be detected in the captured image, and detects the above detection result and the above detection. Another purpose is to transmit a partial image related to the above captured image generated based on the result to the server, and to include upper category information of the above object in the above detection result.

Another purpose of the present invention is to support the construction of an object detection system using an artificial neural network model at a low cost while maintaining high performance of the artificial neural network model.

A representative configuration of the present invention to achieve the above object is as follows.

According to one aspect of the present invention, the object is detected in the captured image of the object using an object detection model that is learned to estimate the upper category information of the object to be detected in the captured image, and the detection result is generated based on the detection result. Obtaining a partial image related to the captured image, and using an object classification model learned to estimate subcategory information of the object from the partial image related to the captured image, to the obtained detection result and the partial image A method is provided, including the step of estimating lower category information of the object based on the detection result, and including upper category information of the object.

According to another aspect of the present invention, detecting the object in a captured image of the object using an object detection model learned to estimate upper category information of an object to be detected in the captured image, and the detection result and the detection A method is provided, including the step of transmitting a partial image related to the captured image generated based on a result to a server, and the detection result includes upper category information of the object.

According to another aspect of the present invention, based on the result of detecting the object in the captured image of the object and the detection result using an object detection model that is learned to estimate the upper category information of the object to be detected in the captured image, The detection result obtained using a detection result acquisition unit that acquires a partial image related to the captured image, and an object classification model that is learned to estimate subcategory information of the object from the partial image related to the captured image, and A system is provided, including an object classification unit that estimates lower category information of the object based on the partial image, and wherein the detection result includes higher category information of the object.

According to another aspect of the present invention, an object detection unit that detects the object in a captured image of the object using an object detection model that is learned to estimate upper category information of the object to be detected in the captured image, and the detection result and A system is provided, including a detection result management unit that transmits a partial image related to the captured image generated based on the detection result to a server, and the detection result includes upper category information of the object.

In addition, another method for implementing the present invention, another system, and a non-transitory computer-readable recording medium for recording a computer program for executing the method are further provided.

According to the present invention, it is possible to support building an object detection system using an artificial neural network model at a low cost while maintaining high performance of the artificial neural network model.

Figure 1 is a diagram showing a schematic configuration of an entire system for estimating information about an object based on an image in an LPWAN environment according to an embodiment of the present invention.
Figure 2 is a diagram illustrating in detail the internal configuration of a server according to an embodiment of the present invention.
Figure 3 is a diagram illustrating in detail the internal configuration of a network camera according to an embodiment of the present invention.
Figure 4 is a diagram illustrating a process for estimating information about an object according to an embodiment of the present invention.

The detailed description of the present invention described below refers to the accompanying drawings, which show by way of example specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different from one another but are not necessarily mutually exclusive. For example, specific shapes, structures and characteristics described herein may be implemented with changes from one embodiment to another without departing from the spirit and scope of the invention. Additionally, it should be understood that the location or arrangement of individual components within each embodiment may be changed without departing from the spirit and scope of the present invention. Accordingly, the detailed description described below is not to be taken in a limiting sense, and the scope of the present invention should be taken to encompass the scope claimed by the claims and all equivalents thereof. Like reference numbers in the drawings indicate identical or similar elements throughout various aspects.

Hereinafter, several preferred embodiments of the present invention will be described in detail with reference to the attached drawings in order to enable those skilled in the art to easily practice the present invention.

Configuration of the entire system

Figure 1 is a diagram showing a schematic configuration of an entire system for estimating information about an object based on an image in an LPWAN environment according to an embodiment of the present invention.

As shown in FIG. 1, the entire system according to an embodiment of the present invention may include an LPWAN 100, a server 200, and a network camera 300.

First, LPWAN (Low Power Wide Area Network; 100) according to an embodiment of the present invention refers to a low-power wireless wide area network that has a very wide service range of 10 km or more and provides a communication speed of hundreds of kilobits per second (kbps). do. These LPWANs (100) include LoRaWAN, SIGFOX, LTE-MTC, Narrowband Internet of Things (NB-IoT), and in particular, LoRaWAN Although the communication speed is slower than existing short-range wireless communications such as Wi-Fi, Bluetooth, and Zigbee, it has the feature of enabling long-distance communication of approximately 30 km in open areas and approximately 1 km in urban areas.

Next, the server 200 according to an embodiment of the present invention detects the above object in the captured image of the above object using an object detection model that is learned to estimate the upper category information of the object to be detected in the captured image. Obtain a partial image related to the above captured image generated based on the above captured result and the above detection result, and use an object classification model that is learned to estimate subcategory information of the above object from the partial image related to the above captured image. Accordingly, a function of estimating subcategory information of the above object can be performed based on the obtained detection result and the above partial image. Here, according to an embodiment of the present invention, the above detection result may include higher category information of the above object.

Next, the network camera 300 according to an embodiment of the present invention detects the above object in the captured image of the above object using an object detection model that is learned to estimate the upper category information of the object to be detected in the captured image. It can perform a function of detecting and transmitting the above detection result and a partial image related to the above captured image generated based on the above detection result to the server. Here, according to an embodiment of the present invention, the above detection result may include higher category information of the above object.

The configuration and functions of the server 200 and the network camera 300 according to the present invention will be discussed in detail through the detailed description below.

Meanwhile, the network camera 300 according to an embodiment of the present invention is a digital device that includes a function for communicating with the server 200 and an image capture function in an LPWAN environment, and is equipped with a memory means and a microprocessor. Any digital device with computing power can be adopted as the network camera 300 according to the present invention. Here, the network camera 300 according to an embodiment of the present invention may refer to the network camera itself (e.g., a commercial security camera), but is connected (or combined) wired and/or wirelessly therewith. It may also refer to all possible hardware devices.

Meanwhile, the server 200 and the network camera 300 may include an application (not shown) that supports a function of estimating information about an object based on an image in an LPWAN environment according to the present invention. Such applications may be downloaded from an external application distribution server (not shown). Here, at least part of the application may be replaced with a hardware device or firmware device that can perform substantially the same or equivalent functions as necessary.

Server configuration

Below, we will look at the internal configuration of the server 200 and the function of each component, which performs important functions for implementing the present invention.

FIG. 2 is a diagram illustrating in detail the internal configuration of the server 200 according to an embodiment of the present invention.

As shown in Figure 2, the server 200 according to an embodiment of the present invention includes a detection result acquisition unit 210, an object classification unit 220, a model management unit 230, a communication unit 240, and a control unit ( 250). According to an embodiment of the present invention, at least some of the detection result acquisition unit 210, object classification unit 220, model management unit 230, communication unit 240, and control unit 250 are connected to an external system (USA). It may be a program module that communicates with (shown). These program modules may be included in the server 200 in the form of operating systems, application program modules, or other program modules, and may be physically stored in various known storage devices. Additionally, these program modules may be stored in a remote memory device capable of communicating with the server 200. Meanwhile, such program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform specific tasks or execute specific abstract data types according to the present invention.

Meanwhile, although the server 200 has been described as above, this description is illustrative, and at least some of the components or functions of the server 200 may be realized or included in an external system (not shown) as needed. is obvious to those skilled in the art.

First, the detection result acquisition unit 210 according to an embodiment of the present invention uses an object detection model that is learned to estimate the upper category information of the object to be detected in the captured image. It is possible to perform a function of acquiring a partial image related to the above captured image generated based on the object detection result and the above detection result.

Specifically, the object detection unit 310 according to an embodiment of the present invention can detect the above object in a captured image of the object using the above object detection model, and based on the result of detecting the above object Thus, a partial image related to the above captured image can be created. And, the detection result acquisition unit 210 according to an embodiment of the present invention can acquire the above detection result and the above partial image in an LPWAN environment.

More specifically, according to an embodiment of the present invention, the above detection result may include upper category information of the object to be detected in the captured image and identification information of the network camera 300 that performed the detection. In other words, the detection result obtained by the detection result acquisition unit 210 according to an embodiment of the present invention may mean the result of detecting an object to be detected in an image captured by the network camera 300 at a higher category level. . And, according to an embodiment of the present invention, the above upper category information may be associated with two or more lower category information of the object selected by the user, that is, the object to be detected in the captured image.

For example, according to an embodiment of the present invention, it can be assumed that 'bus' and 'truck' are selected by the user as subcategory information of the object to be detected in the captured image. In this case, the detection result acquisition unit 210 according to an embodiment of the present invention determines the upper category information of the above object as 'vehicle', or determines that the upper category information of the above user is associated with the above lower category information. As information, appropriate information can be provided to the user above to enable selection of 'vehicle', 'large vehicle', etc.

For another example, according to an embodiment of the present invention, it may be assumed that 'person' is selected by the user as upper category information of the object to be detected in the captured image. In this case, the detection result acquisition unit 210 according to an embodiment of the present invention determines the lower category information of the above object as 'Adult' and 'Children', or determines that the above user combines the above upper category information and As related sub-category information, appropriate information can be provided to the above users so that they can select 'adult', 'child', 'man', 'woman', etc.

However, the types of upper category information and lower category information of an object and the method of determining the corresponding information according to an embodiment of the present invention are not limited to those described above, and may vary within the scope of achieving the purpose of the present invention. may be changed.

Continuing, the partial image related to the captured image acquired by the detection result acquisition unit 210 according to an embodiment of the present invention is an image related to the area where the object was detected in the above captured image (for example, the object above may refer to an image cut according to the boundary of a detected bounding box). And, when a plurality of objects are detected in the captured image, the object detection unit 310 according to an embodiment of the present invention may generate a partial image for each of the plurality of detected objects. And, the detection result acquisition unit 210 according to an embodiment of the present invention can acquire the partial images generated above. Meanwhile, according to an embodiment of the present invention, the above partial image may refer only to the partial image itself, but may also include information (e.g., coordinates) regarding the position of the partial image in the captured image. . Since the detection result acquisition unit 210 according to an embodiment of the present invention acquires only a partial image of the captured image rather than the entire captured image of the object captured by the network camera 300 as described above, LPWAN and Likewise, the object detection system can be operated stably even in a communication environment where bandwidth is limited.

Meanwhile, the object detection model used in the object detection unit 310 according to an embodiment of the present invention may be distributed by reducing the weight of the object detection model generated in the server 200.

Specifically, the model management unit 230 according to an embodiment of the present invention may create an object detection model that is learned to estimate upper category information of an object to be detected in a captured image. According to one embodiment of the present invention, the learning data about the object to be detected in the captured image used in generating this object detection model is the captured image about the above object and labeling data about the captured image. The image may include information about the area where the above object is located and information about the upper category of the above object. That is, the model management unit 230 according to an embodiment of the present invention may train the object detection model so that the object detection model estimates upper category information rather than lower category information of the object. Accordingly, the above object detection model reduces the types of objects to be detected, so it can detect objects with higher accuracy while processing a smaller amount of operations compared to the case of estimating subcategory information.

Continuing, the model management unit 230 according to an embodiment of the present invention can distribute the lightweight object detection model created as above to the network camera 300.

More specifically, the model management unit 230 according to an embodiment of the present invention creates an object detection model that is learned to estimate the upper category information of the object to be detected in the captured image, and performs pruning and quantization. ), the created model can be made lightweight by using artificial neural network model lightweighting algorithms such as Knowledge Distillation. In addition, the model management unit 230 according to an embodiment of the present invention uses the lightweight model as described above to enable smooth use even with the network camera 300, which has lower computing power than the server 200. This is an object detection model used by the object detection unit 310 according to an embodiment and can be distributed to the network camera 300. However, the lightweight algorithm according to an embodiment of the present invention is not limited to those listed above, and may be changed in various ways within the scope of achieving the purpose of the present invention.

Meanwhile, details not described above regarding the function of the object detection unit 310 according to an embodiment of the present invention will be described later.

Next, the object classification unit 220 according to an embodiment of the present invention calculates the subcategory information of the object from the partial image related to the captured image acquired by the detection result acquisition unit 210 according to an embodiment of the present invention. Using an object classification model that is learned to estimate, subcategory information of the above object is estimated based on the detection result obtained by the detection result acquisition unit 210 according to an embodiment of the present invention and the above partial image. It can perform the function:

Specifically, when a partial image related to a captured image is acquired, the object classification unit 220 according to an embodiment of the present invention uses an object classification model that is learned to estimate subcategory information of the object from the partial image. , Based on the result of detecting an object in the above captured image by the object detection unit 310 according to an embodiment of the present invention (specifically, upper category information of the above object) and the above partial image, the above object Subcategory information can be estimated.

More specifically, the model management unit 230 according to an embodiment of the present invention may generate an object classification model that is learned to estimate subcategory information of an object from the above partial image. According to an embodiment of the present invention, the learning data about the object used in generating this object classification model includes images about the area where the above object is detected in the captured image about the above object, that is, partial images and Labeling data about that partial image may include subcategory information of the above object. That is, the model management unit 230 according to an embodiment of the present invention can train the above object detection model to estimate information on two or more subcategories of the above object selected by the user. The function of specifically classifying objects to be detected in captured images requires processing a large amount of calculations. According to one embodiment of the present invention, this function has a higher computational capacity compared to the network camera 300, as described above. By allowing the operation to be performed on a high-level server 200, the computational burden on the network camera 300 can be reduced.

Next, the communication unit 240 according to an embodiment of the present invention performs a function that enables data transmission and reception from/to the detection result acquisition unit 210, the object classification unit 220, and the model management unit 230. You can.

Lastly, the control unit 250 according to an embodiment of the present invention has a function of controlling the flow of data between the detection result acquisition unit 210, the object classification unit 220, the model management unit 230, and the communication unit 240. It can be done. That is, the control unit 250 according to the present invention controls the data flow to/from the outside of the server 200 or the data flow between each component of the server 200, thereby controlling the detection result acquisition unit 210 and the object classification unit. (220), the model management unit 230, and the communication unit 240 can each be controlled to perform their own functions.

Configuration of network cameras

Below, we will look at the internal configuration of the network camera 300 and the function of each component, which performs important functions for implementing the present invention.

FIG. 3 is a diagram illustrating in detail the internal configuration of the network camera 300 according to an embodiment of the present invention.

As shown in FIG. 3, the network camera 300 according to an embodiment of the present invention is configured to include an object detection unit 310, a detection result management unit 320, a communication unit 330, and a control unit 340. You can. According to an embodiment of the present invention, the object detection unit 310, the detection result management unit 320, the communication unit 330, and the control unit 340 are program modules, at least some of which communicate with an external system (not shown). It can be. These program modules may be included in the network camera 300 in the form of an operating system, application program module, or other program module, and may be physically stored in various known storage devices. Additionally, these program modules may be stored in a remote memory device capable of communicating with the network camera 300. Meanwhile, such program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform specific tasks or execute specific abstract data types according to the present invention.

Meanwhile, although the network camera 300 has been described as above, this description is illustrative and at least some of the components or functions of the network camera 300 are realized or included within an external system (not shown) as needed. It is obvious to those skilled in the art that this may be possible.

First, the object detector 310 according to an embodiment of the present invention detects the above object in the captured image of the above object using an object detection model that is learned to estimate the upper category information of the object to be detected in the captured image. It can perform a detection function.

Specifically, the object detection unit 310 according to an embodiment of the present invention can detect the above object in a captured image of the object using the above object detection model, and the above detection result includes the above detection Top category information of the detected object and identification information of the network camera 300 that performed the detection may be included.

Continuing, the object detection model used by the object detector 310 according to an embodiment of the present invention to detect objects at a higher category level in the captured image as above is R-CNN (Region-based Convolutional Neural Networks), YOLO It may be created based on an artificial neural network-based object recognition model such as (You Only Look Once) or SSD (Single Shot multibox Detector). However, the artificial neural network-based object recognition model according to an embodiment of the present invention is not limited to those listed above, and may be changed in various ways within the scope of achieving the purpose of the present invention.

And, according to an embodiment of the present invention, the upper category information of the object to be detected in the captured image may be associated with two or more lower category information of the object selected by the user, that is, the object to be detected in the captured image. Meanwhile, since the above upper-category information and lower-category information have been explained in detail above, their description will be omitted here.

Meanwhile, the object detection model used in the object detection unit 310 according to an embodiment of the present invention may be distributed by reducing the weight of the object detection model generated in the server 200.

Specifically, the model management unit 230 according to an embodiment of the present invention may create an object detection model that is learned to estimate upper category information of an object to be detected in a captured image. In addition, the model management unit 230 according to an embodiment of the present invention can distribute the lightweight object detection model created as above to the network camera 300. Meanwhile, since the creation of the above object detection model and the lightweighting of the model have been explained in detail above, the description thereof will be omitted here.

Next, the detection result management unit 320 according to an embodiment of the present invention, as described above, detects the above object in the captured image of the object and the portion related to the above captured image generated based on the detection result. The function of transmitting video to the server 200 can be performed.

Specifically, the object detector 310 according to an embodiment of the present invention generates an image related to an area where an object is detected at a higher category level in a captured image (for example, a bounding box where the above object is detected). (an image cropped according to the boundary of ), that is, a partial image can be created. In addition, when a plurality of objects are detected in the captured image, the object detection unit 310 according to an embodiment of the present invention may generate a partial image for each of the plurality of detected objects. And, the detection result management unit 320 according to an embodiment of the present invention can transmit the partial image generated as above to the server. Meanwhile, according to an embodiment of the present invention, the above partial image may refer only to the partial image itself, but may also include information (e.g., coordinates) regarding the position of the partial image in the captured image. .

Figure 4 is a diagram illustrating a process for estimating information about an object according to an embodiment of the present invention.

Referring to FIG. 4, according to an embodiment of the present invention, the network camera 300 (specifically, the object detection unit 310) provides upper category information (e.g., vehicle) of the object to be detected in the captured image. Using an object detection model that is learned to estimate the object, the corresponding object can be detected in the captured image 410 of the object (411 and 412).

Continuing to refer to FIG. 4 , the object detection unit 310 according to an embodiment of the present invention may generate partial images 420 and 430 related to the captured image 410 based on the result of detecting the corresponding object. . And, the detection result management unit 320 according to an embodiment of the present invention may transmit the above detection result and the above partial images 420 and 430 to the server 200.

Continuing to refer to FIG. 4, when the detection result acquisition unit 210 according to an embodiment of the present invention acquires the above detection result and the above partial images 420 and 430, according to an embodiment of the present invention The object classifier 220 can estimate the subcategory information of the above object using an object classification model that is learned to estimate the subcategory information (for example, sedan or truck) of the object from the above partial image. There are (420 and 450).

The embodiments according to the present invention described above can be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and usable by those skilled in the computer software field. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. medium), and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include not only machine language code such as that created by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. A hardware device can be converted into one or more software modules to perform processing according to the invention and vice versa.

In the above, the present invention has been described in terms of specific details, such as specific components, and limited embodiments and drawings, but this is only provided to facilitate a more general understanding of the present invention, and the present invention is not limited to the above embodiments. Anyone with ordinary knowledge in the technical field to which the invention pertains can make various modifications and changes from this description.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and the scope of the patent claims described below as well as all scopes equivalent to or equivalently changed from the scope of the claims are within the scope of the spirit of the present invention. It will be said to belong to

100: communication network
200: server
210: Detection result acquisition unit
220: Object classification unit
230: Model management department
300: network camera
310: object detection unit
320: Detection result management unit
240, 330: Department of Communications
250, 340: Control unit

Claims (13)

A method for estimating information about an object based on images captured through a network camera in an LPWAN (Low Power Wide Area Network) environment,
In the server, generating a first object detection model that is learned to estimate upper category information of an object to be detected from a captured image and an object classification model that is learned to estimate lower category information of the object from a partial image for the captured image;
In a network camera, the object is detected in a captured image of the object using a second object detection model that estimates upper category information of the object to be detected from the captured image, and the captured image is generated based on the detection result. Obtaining information about a partial image and a position occupied by the partial image in the captured image,
In a network camera, transmitting the detection result, a partial image related to the captured image generated based on the detection result, and information about the position of the partial image in the captured image to a server;
In a server, using the object classification model, estimating subcategory information of the object based on the obtained detection result and the partial image,
The detection result includes upper category information of the object,
The upper category information is associated with two or more lower category information of the object selected by the user,
The first object detection model is a captured image of an object and labeling data about the captured image. Information about the area where the object is located in the captured image and upper category information of the object are converted into learning data about the object to be detected in the captured image. Learned by using,
The second object detection model is a lightweight version of the first object detection model created in the server and distributed to the network camera.
method. delete delete delete delete delete A non-transitory computer-readable recording medium recording a computer program for executing the method according to claim 1. A system for estimating information about an object based on images captured through a network camera in an LPWAN (Low Power Wide Area Network) environment,
Includes servers and network cameras,
The server is,
A model management unit that generates a first object detection model that is learned to estimate upper category information of an object to be detected from a captured image and an object classification model that is learned to estimate lower category information of the object from a partial image of the captured image;
As a result of detecting the object in a captured image of the object using a second object detection model of the network camera that is learned to estimate upper category information of the object to be detected in the captured image, the object is generated based on the detection result. A detection result acquisition unit that acquires information about a partial image related to a captured image and a position of the partial image in the captured image, and
An object classification unit that estimates subcategory information of the object based on the obtained detection result and the partial image using the object classification model,
The detection result includes upper category information of the object,
The upper category information is associated with two or more lower category information of the object selected by the user,
The first object detection model is a captured image of an object and labeling data about the captured image. Information about the area where the object is located in the captured image and upper category information of the object are converted into learning data about the object to be detected in the captured image. Learned by using,
The second object detection model is a lightweight version of the first object detection model and distributed to the network camera.
system. delete delete delete delete delete

Images