WO2023223386A1 - 検出装置、カメラシステム、検出方法、及び検出プログラム - Google Patents

検出装置、カメラシステム、検出方法、及び検出プログラム Download PDF

Info

Publication number
WO2023223386A1
WO2023223386A1 PCT/JP2022/020398 JP2022020398W WO2023223386A1 WO 2023223386 A1 WO2023223386 A1 WO 2023223386A1 JP 2022020398 W JP2022020398 W JP 2022020398W WO 2023223386 A1 WO2023223386 A1 WO 2023223386A1
Authority
WO
WIPO (PCT)
Prior art keywords
model
detection
trained
accuracy
models
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/020398
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
悠介 太田
敬志 上村
正英 小池
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mitsubishi Electric Corp
Original Assignee
Mitsubishi Electric Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mitsubishi Electric Corp filed Critical Mitsubishi Electric Corp
Priority to US18/862,572 priority Critical patent/US20250287098A1/en
Priority to JP2022563098A priority patent/JP7270856B1/ja
Priority to PCT/JP2022/020398 priority patent/WO2023223386A1/ja
Priority to CN202280095730.7A priority patent/CN119156634A/zh
Publication of WO2023223386A1 publication Critical patent/WO2023223386A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/64—Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19602—Image analysis to detect motion of the intruder, e.g. by frame subtraction
    • G08B13/19613—Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B29/00—Checking or monitoring of signalling or alarm systems; Prevention or correction of operating errors, e.g. preventing unauthorised operation
    • G08B29/18—Prevention or correction of operating errors
    • G08B29/185—Signal analysis techniques for reducing or preventing false alarms or for enhancing the reliability of the system
    • G08B29/186—Fuzzy logic; neural networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/617—Upgrading or updating of programs or applications for camera control
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/667—Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes

Definitions

  • the present disclosure relates to a detection device, a camera system, a detection method, and a detection program.
  • an AI model with high detection accuracy may not be selected from among the plurality of AI models.
  • the present disclosure provides a detection device, a camera system, a detection method, and a detection program that make it possible to determine and use a trained model with high detection accuracy from a plurality of trained models for detecting an object from a video.
  • the purpose is to provide.
  • the detection device of the present disclosure uses a trained model selected from a plurality of trained models, inputs a video to the selected trained model, and uses a detection result of detecting an object from the video as an input to the selected trained model.
  • a detection processing unit that executes a detection process as an output of the trained model, causes the detection process to be executed for each of the plurality of trained models, and calculates the accuracy of the detection result for each of the plurality of trained models;
  • a model control unit that performs a determination process to determine a recommended trained model from among the plurality of trained models based on the accuracy, and the detection process after the determination process is performed to select the recommended trained model from among the plurality of trained models based on the accuracy. It is characterized by being executed using.
  • the detection method of the present disclosure is a method implemented by a detection device, which uses a trained model selected from a plurality of trained models, uses a video as an input to the selected trained model, and uses the video as an input. executing a detection process in which a detection result of detecting an object is an output of the selected trained model; and executing the detection process for each of the plurality of trained models, and each of the plurality of trained models a step of calculating the accuracy of the detection result for and determining a recommended trained model from among the plurality of trained models based on the accuracy;
  • the method is characterized by comprising a step of executing the method using the trained model.
  • a trained model with high detection accuracy can be determined and used from a plurality of trained models for detecting objects from videos.
  • FIG. 1 is a block diagram schematically showing the configuration of a detection device and a camera system according to Embodiment 1.
  • FIG. 1 is a diagram illustrating an example of a hardware configuration of a detection device and a camera system according to Embodiment 1.
  • FIG. 3 is an explanatory diagram showing the operation of a person detection AI model used in the detection processing section of the detection device according to the first embodiment.
  • FIG. 3 is an explanatory diagram showing the operation of a face detection AI model used in the detection processing section of the detection device according to the first embodiment.
  • FIG. 3 is an explanatory diagram showing a state in which a plurality of AI models are deployed in the working memory of the detection device according to the first embodiment.
  • FIG. 7 is a flowchart illustrating a process for determining a recommended AI model used in the detection processing unit of the detection device according to the first embodiment.
  • 7 is a flowchart illustrating processing after determining a recommended AI model to be used in the detection processing unit of the detection device according to the first embodiment.
  • FIG. 3 is an explanatory diagram showing a process of switching an AI model used in a detection processing section of the detection device according to the first embodiment.
  • FIG. 2 is a block diagram schematically showing the configuration of a detection device and a camera system according to a second embodiment.
  • FIG. 7 is an explanatory diagram showing an operation when deploying an AI model to the working memory of the detection device according to the second embodiment.
  • FIG. 7 is a flowchart illustrating a process for determining a recommended AI model used in a detection processing unit of a detection device according to a second embodiment.
  • 3 is a block diagram schematically showing the configuration of a detection device and a camera system according to Embodiment 3.
  • FIG. 7 is a diagram illustrating an example of a process for determining a recommended AI model used in a detection processing unit of a detection device according to Embodiment 3.
  • FIG. 7 is a diagram illustrating another example of the process for determining a recommended AI model used in the detection processing section of the detection device according to Embodiment 3.
  • FIG. 1 is a block diagram schematically showing the configuration of a detection device 10 and a camera system 1 according to the first embodiment.
  • the detection device 10 is a device that can implement the detection method according to the first embodiment.
  • the camera system 1 includes a detection device 10 and an imaging unit (ie, camera) 50 as a video input unit that photographs video.
  • the detection device 10 includes a detection processing section 11, a model control section 12, a model expansion section 16, and a working memory 17 as an internal memory.
  • the model control unit 12 includes a model switching control unit 13, a detection accuracy calculation unit 14, and a model determination unit 15.
  • the detection device 10 is connected to a storage device 40 that includes a storage medium as an external memory, for example, via a network.
  • the storage device 40 may be part of the detection device 10.
  • the storage device 40 may include a plurality of storage media located at a plurality of different locations.
  • the storage device 40 stores a plurality of AI models 41, 42, and 43 that are trained models.
  • the number of stored AI models may be two or more.
  • the imaging unit 50 is, for example, a surveillance camera installed indoors or outdoors.
  • This surveillance camera is, for example, a visible light camera or an infrared camera.
  • the imaging unit 50 captures an image suitable for the purpose of the camera system 1.
  • This video includes, for example, a video shot near the entrance of a building, a video shot of a road, a video shot inside a building, and the like.
  • the detection processing unit 11 uses an AI model selected from the plurality of available AI models 41, 42, and 43, inputs the video D1 to the selected AI model, and generates a detection result of detecting an object from the video D1. Execute a detection process using D2 as the output of the selected AI model.
  • the model control unit 12 causes the detection processing unit 11 to execute detection processing for each of the plurality of AI models 41, 42, 43, calculates the accuracy of the detection result for each of the plurality of AI models 41, 42, 43, Based on this accuracy, a determination process is performed to determine a recommended AI model from among the plurality of AI models 41, 42, and 43.
  • the detection processing by the detection processing unit 11 after the determination processing is performed using the recommended AI model.
  • the plurality of AI models that can be used are not limited to those stored in the storage device 40.
  • the plurality of AI models that can be used may be stored in a plurality of different storage devices, or may be stored in a storage device of a communicable network server, for example.
  • the detection processing unit 11 selects an AI model switched by the model switching control unit 13 from one or more AI models (for example, AI models 41, 42, 43) developed in the work memory 17; Alternatively, the recommended AI model that is the AI model determined by the model determination unit 15 is selected, and the selected AI model is used to perform object detection processing from the video D1 and output the detection result D2.
  • the object may be a person or an animal.
  • the detection result D2 includes, for example, the coordinates and detection accuracy of detection frames for the number of detected objects (for example, the detection frame 123b in FIG. 3, which will be described later).
  • the detection processing unit 11 When detecting two or more objects, the detection processing unit 11 sets the average value (that is, an example of a statistical value) of the detection accuracy output for the number of objects detected as the frame detection accuracy.
  • the detection accuracy is output as a numerical value between "0" and "1", for example.
  • the detection processing unit 11 may treat something other than the average value as the detection accuracy, or may use a statistical value that has been subjected to other statistical processing. In the following description, the average value Av will be used as the detection accuracy.
  • the model switching control unit 13 checks the number of frames of the video on which the detection processing was performed by the detection processing unit 11, and performs switching control of the AI model used by the detection processing unit 11. Details of the switching control will be explained with reference to FIGS. 6 and 7, which will be described later.
  • the detection accuracy calculation unit 14 calculates the detection accuracy for a predetermined number of frames of the video from the detection results output from each AI model when the detection processing unit 11 performs object detection processing using each AI model.
  • An average value Av (for example, average values Av1, Av2, Av3 corresponding to the AI models 41, 42, 43) is calculated for each AI model.
  • the model determination unit 15 uses the average value Av of detection accuracy for each AI model obtained by the detection accuracy calculation unit 14 to determine which AI models are selected from among the plurality of AI models 41, 42, and 43 developed in the working memory 17. It is determined which AI model to use as the recommended AI model to be used by the detection processing unit 11.
  • the recommended AI model is, for example, the AI model with the largest average value Av of detection accuracy.
  • FIG. 2 is a diagram showing an example of the hardware configuration of the detection device 10 and the camera system 1 equipped with the same.
  • the detection device 10 is, for example, a computer as an information processing device.
  • the detection device 10 includes a processor 101, a memory 102, a nonvolatile storage device 103, and an interface 104.
  • the processor 101 is a CPU (Central Processing Unit) or the like.
  • the memory 102 is, for example, a volatile semiconductor memory such as RAM (Random Access Memory).
  • the nonvolatile storage device 103 is a hard disk drive (HDD), solid state drive (SSD), or the like.
  • An interface 104 is provided for communicating with other devices.
  • the imaging unit 50 and an external storage device 40 may be connected to the interface 104 .
  • the processing circuit may be dedicated hardware or may be the processor 101 that executes a program stored in the memory 102.
  • the processor 101 may be any one of a processing device, an arithmetic device, a microprocessor, a microcomputer, and a DSP (Digital Signal Processor).
  • the processing circuit is dedicated hardware, the processing circuit is, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) ray ), or a combination of any of these.
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • the detection method or detection program according to the first embodiment is realized by software, firmware, or a combination of software and firmware.
  • Software and firmware are written as programs and stored in memory 102.
  • the detection program is installed in the detection device 10 by downloading via a network or by installing from an information recording medium such as an optical disk.
  • the processor 101 can realize the functions of each part shown in FIG. 1 by reading and executing the detection program stored in the memory 102.
  • the detection device 10 may be partially realized by dedicated hardware and partially realized by software or firmware. In this way, the processing circuit can implement the functions of each functional block shown in FIG. 1 using hardware, software, firmware, or any combination thereof.
  • FIG. 3 is an explanatory diagram showing the operation of the person detection AI model 122, which is an example of the AI model used in the detection processing unit 11 of the detection device 10 according to the first embodiment.
  • FIG. 3 shows an output (that is, a detection result) 123 when a video is input 121 to the human detection AI model 122.
  • the person detection AI model 122 outputs a detection frame 123b of the person 123a and detection accuracies of "0.3", “0.8”, and “1", as shown in the output 123. In this frame, three people are detected, and the detection accuracy for each person is "0.3", “0.8", and “1". Therefore, the detection accuracy of this frame by the human detection AI model 122 is calculated as ⁇ (0.3+0.8+1)/3 ⁇ and is "0.7".
  • FIG. 4 is an explanatory diagram showing the operation of the face detection AI model 125, which is an example of the AI model used in the detection processing unit 11 of the detection device 10 according to the first embodiment.
  • FIG. 4 shows an output (that is, a detection result) 126 when a video is used as an input 124 of the face detection AI model 125.
  • the face detection AI model 125 outputs a detection frame 126b of the face 126a and detection accuracies of "0.8", "0.9”, and “1", as shown in the output 126. In this frame, three faces are detected, and the detection accuracy of each face is "0.8", "0.9", and "1". Therefore, the detection accuracy of this frame by the face detection AI model 125 is calculated as ⁇ (0.8+0.9+1)/3 ⁇ and is "0.9".
  • FIG. 5 is an explanatory diagram showing a state when a plurality of AI models are deployed in the working memory 17 of the detection device 10 according to the first embodiment.
  • the storage device 40 stores a plurality of AI models 41, 42, and 43.
  • the storage medium of the storage device 40 is, for example, a nonvolatile memory.
  • the AI models 41, 42, and 43 are generated using, for example, deep learning.
  • the AI models 41, 42, and 43 are, for example, AI models published on a website or the like.
  • the AI models 41, 42, and 43 may be generated by a learning device that collects learning data (for example, video data) and performs learning using the collected learning data.
  • video data used for learning there are no restrictions on the video data used for learning, and for example, video data published on a website may be used. Further, the video data used for learning may be video data taken with a digital camera or video data taken with a surveillance camera in which the detection device 10 is planned to be installed. Furthermore, there is no limit to the number of AI models stored in the storage device 40 as long as it can be held within the capacity of the storage device 40.
  • the AI models 41, 42, and 43 are AI models capable of object detection, and include a person detection AI model, a face detection AI model, an object detection AI model with high detection accuracy for object detection in daytime video, Examples of the object detection model include an object detection model that has a high detection accuracy of object detection in a video of an evening situation, and an object detection AI model that has a high detection accuracy of object detection in a video of a night situation.
  • the person detection AI model is a model that detects whether there is a person in the captured video.Since the model is generated using video data taken from various angles or heights as training data, it is possible to detect whether a person is facing the camera direction. It is possible to detect with high detection accuracy not only when the user is facing the camera, but also when the user is facing to the side or behind the camera. However, since the whole person is detected by looking at it, detection accuracy decreases if even a part of the person is hidden. In the output 123 of FIG. 3, the person 123a shown at the rear left side is partially hidden by the person in front of him, so the detection accuracy is reduced.
  • the face detection AI model is a model that detects whether there is a face in the shot video, and the training data uses video data shot at various angles and heights within the range where the face can be seen, so it is a model that detects whether there is a face in the shot video. Detection is possible even if the object is hidden, but detection accuracy decreases when the object is facing sideways or backwards with respect to the camera direction.
  • the face 126a shown at the rear left of the output 126 in FIG. 4 is partially hidden by the person in front, but the face part is visible, so the detection accuracy is high.
  • the object detection AI model with high detection accuracy for object detection in videos of daytime situations is a model generated based on video data taken during the day, and the object detection AI model with high detection accuracy of object detection in videos of evening situations.
  • the AI model is a model generated based on video data taken in the evening, and the object detection AI model with high detection accuracy for object detection in videos of night situations is generated based on video data taken at night. This is a model that has
  • the model development unit 16 develops the plurality of AI models 41, 42, and 43 stored in the storage device 40 into the working memory 17, as shown in FIG.
  • FIG. 6 is a flowchart showing a process for determining a recommended AI model used by the detection processing unit 11 of the detection device 10 according to the first embodiment. Using FIG. 6, a recommended AI model with high detection accuracy is determined from the plurality of AI models 41, 42, and 43 in the working memory 17.
  • the model development unit 16 develops the plurality of AI models 41, 42, and 43 stored in the storage device 40 into the working memory 17 (step S101).
  • the imaging unit 50 also captures a video to obtain a video (that is, multiple frames of video data) (step S102).
  • the detection processing unit 11 executes the detection process using the recommended AI model that is the AI model switched by the model switching control unit 13 or the AI model determined by the model determination unit 15.
  • the AI model 41 is used in the first frame after the detection device 10 is activated.
  • the AI model 41 is used for object detection in the first frame, but the AI model used initially may be another AI model (step S103).
  • the model switching control unit 13 determines whether the detection process for a predetermined number of frames (ie, a set number) m1 frames has been completed (step S104).
  • step S104 if the model switching control unit 13 determines that the detection processing for m1 frames has not been completed, the model switching control unit 13 determines that the AI model used by the detection processing unit 11 is , confirms which AI model it is (step S105).
  • the model switching control unit 13 switches the AI model used by the detection processing unit 11 to the AI model used to detect the object in the next frame. If the AI model 41 is used to detect the object in the current frame, the AI model used to detect the object in the next frame is switched to the AI model 42 (step S105a). If the AI model 42 is used to detect the object in the current frame, the AI model used to detect the object in the next frame is switched to the AI model 43 (step S105b). If the AI model 43 is used to detect the object in the current frame, the AI model used to detect the object in the next frame is switched to the AI model 41 (step S105c).
  • step S104 if the model switching control unit 13 determines that the detection processing for m1 frames has been completed, the detection accuracy calculation unit 14 outputs The average value Av of detection accuracy is calculated for each AI model from the detection accuracy of the frames processed and the number of processed frames (step S106).
  • the model determining unit 15 determines an AI model with a high average value Av of detection accuracy (for example, an AI model with the highest average value Av) as a recommended AI model (step S107).
  • FIG. 7 is a flowchart showing the process after determining the recommended AI model to be used by the detection processing unit 11 of the detection device 10 according to the first embodiment.
  • FIG. 7 shows the process after the model determining unit 15 determines an AI model with high detection accuracy as the recommended AI model.
  • the imaging unit 50 photographs a video (step S102a).
  • the detection processing unit 11 executes the detection process using the recommended AI model determined by the model determination unit 15 in step S107 of FIG. 6 (step S103a).
  • the model switching control unit 13 determines whether detection processing for a predetermined number of frames (ie, a set number) m2 frames has been completed (step S104a).
  • step S104a if the model switching control unit 13 determines that the detection processing for m2 frames has not been completed, it repeats the processing in steps S102a and S103a. If the model switching control unit 13 determines in step S104a that the detection processing for m2 frames has been completed, the process returns to step S102 in FIG.
  • FIG. 8 is an explanatory diagram showing a process of switching the AI model used in the detection processing unit 11 of the detection device 10 according to the first embodiment.
  • FIG. 8 shows an example of the switching order of AI models used by the detection processing unit 11.
  • three AI models 41, 42, and 43 are developed in the working memory 17.
  • the detection processing unit 11 uses the AI model 41 to perform object detection processing (step S121).
  • the detection processing unit 11 uses the AI model 42 to perform object detection processing (step S122).
  • the detection processing unit 11 uses the AI model 43 to execute an object detection process (step S123).
  • the detection processing unit 11 uses the AI model 41 to perform object detection processing (step S124).
  • the detection device 10 converts the AI model (that is, the AI models 41, 42, and 43 developed in the working memory 17) into frames m1 in order to obtain the average value of object detection accuracy. Switch a number of times equal to .
  • the model switching control unit 13 determines that the AI model has been switched a number of times equal to the number of frames m1
  • the model determining unit 15 selects the detection processing unit from the next frame onwards based on the average value Av of the accuracy of the detection results. Determine the recommended AI model to be used in step 11.
  • the recommended AI model determined by the model determination unit 15 will be described as an AI model 42.
  • the detection processing unit 11 uses the AI model 42, which is the determined recommended AI model, to perform object detection processing on m2 frames (step S125).
  • the detection device 10 When the model switching control unit 13 determines that the detection process for m2 frames has been completed, the detection device 10 performs the processes in steps S102, S103, S104, S105, and S105a, steps S102, S103, and S104. , S105, S105b, and steps S102, S103, S104, S105, and S105c are repeated (step S126).
  • the determination by the model determination unit 15 is performed after the total number of frames processed by the three AI models 41, 42, and 43 reaches a predetermined number m1 of frames.
  • the determination of the recommended AI model by the model determination unit 15 may be performed after each of the AI models 41, 42, and 43 has completed detection processing for a predetermined number of frames (for example, m3). good.
  • the camera system 1 according to the first embodiment can be used in the following situations.
  • the detection accuracy can be maintained at a high level by switching between the person detection AI model and the face detection AI model according to the accuracy of the detection process.
  • the AI model 41 is assumed to be a person detection AI model
  • the AI model 42 is assumed to be a face detection AI model. If the person is facing sideways or backwards, or if the person is far from the imaging unit of the camera where the area of the face is small, the detection accuracy of the output of the AI model 41 may be higher. is expected. Furthermore, when the number of people overlapping each other increases, it is expected that the detection accuracy of the output of the AI model 42 will become higher. Therefore, under these environments, the AI model 41 and the AI model 42 are used in the detection processing unit 11, respectively. When there are few people and people are facing forward, it is not known which AI model will be used, but the detection processing unit 11 uses the AI model with higher detection accuracy among these two AI models.
  • AI model 41 is an object detection AI model with high object detection accuracy in daytime video
  • AI model 42 is an object detection AI model with high object detection accuracy in evening video
  • AI model 43 is a night detection AI model.
  • Object detection is performed with high detection accuracy in object detection in a video of a situation.
  • the detection accuracy when using AI model 41 is expected to be the highest, in the evening time
  • the detection accuracy when using AI model 42 is expected to be the highest, and in the evening
  • the AI model 43 will be the highest. Therefore, under these environments, the AI model 41, the AI model 42, and the AI model 43 are used in the detection processing unit 11, respectively.
  • the AI model with the highest detection accuracy among these three AI models is the one that has the highest detection accuracy. It is used in the detection processing section 11.
  • an AI model specialized for the time period is prepared, and the detection processing unit 11 uses the AI model with the highest detection accuracy among them, so that objects can be detected with high detection accuracy in any time period. can do.
  • the average value Av of object detection accuracy is calculated for each of the plurality of AI models developed in the working memory 17, and the AI model with the larger average value Av ( Recommended AI model) is used to detect objects in the next frame onward, so that the object detection process is performed with the AI model that has the highest detection accuracy under the environment from among multiple AI models. Can be done.
  • the AI model used may be switched depending on whether the lights in the room are on or off. Furthermore, in the first embodiment, the AI model to be used may be switched depending on whether or not light from the sun is inserted. In the first embodiment, the AI model used may be switched depending on whether the direction of light from the sun is front lighting or back lighting.
  • the AI model determined by the model determining unit 15 is switched after processing an arbitrarily specified number of frames, so that an AI model with high detection accuracy under that environment is used. As a result, high detection accuracy can be maintained even in camera systems that take pictures over long periods of time, such as surveillance cameras.
  • the detection processing unit 11 performs detection processing for each frame of the video and outputs the detection result, but the frames to be subjected to the detection processing do not have to be strictly consecutive; for example, some frames may be skipped or frames may be periodically thinned out.
  • FIG. 9 is a block diagram schematically showing the configuration of the detection device 20 and camera system 2 according to the second embodiment.
  • the detection device 20 is a device that can implement the detection method according to the second embodiment.
  • components that are the same as or correspond to those shown in FIG. 1 are given the same reference numerals as those shown in FIG.
  • the detection device 20 according to the second embodiment differs from the first embodiment in that the configuration of the model expansion unit 16 and the model release unit 16a that releases the AI model expanded in the working memory 17a are provided. This is different from the detection device 10.
  • FIG. 10 is an explanatory diagram showing the operation when deploying the AI model in the working memory 17a of the detection device 20 according to the second embodiment.
  • FIG. 10 shows a process when the capacity of the AI model 41a is large and only a portion of the available AI model can be developed into the working memory 17a.
  • FIG. 11 is a flowchart showing a process for determining a recommended AI model used by the detection processing unit 11 of the detection device 20 according to the second embodiment.
  • the flow of AI model switching when only one of the plurality of AI models 41a and 42a stored in the storage device 40 can be developed into the working memory 17a will be described using FIG. 11.
  • the model development unit 16 develops the AI model 41a into the working memory 17a (step S101a).
  • the imaging unit 50 photographs a video (step S102).
  • the detection processing unit 11 executes object detection processing using the AI model developed in the working memory 17a by the model development unit 16.
  • the detection processing unit 11 uses the AI model 41a to perform object detection processing in the first frame (step S103).
  • the model switching control unit 13 determines whether object detection processing in a predetermined number of frames (ie, a set number) m3 frames has been completed (step S104).
  • the model switching control unit 13 determines which AI model used by the detection processing unit 11 is It is confirmed whether it is an AI model (step S105).
  • the model release unit 16a releases the AI model 41a from the working memory 17a (step S105a).
  • the model development unit 16 develops the AI model 42a from the storage device 40 into the working memory 17a (step S105aa).
  • the model release unit 16a releases the AI model 42a from the working memory 17a (step S105b).
  • the model development unit 16 develops the AI model 41a from the storage device 40 into the working memory (step S105bb).
  • the working memory 17a Even if only some of the available AI models can be deployed in the working memory 17a, that is, even if a large AI model is used, the working memory It is possible to operate an AI model with high detection accuracy by repeatedly deploying as many AI models as can be deployed in 17a and releasing them after processing is completed. Furthermore, in this case, the detection device 10 can be operated while switching the AI memory developed in the work memory 17a to one of the plurality of AI models.
  • FIG. 12 is a block diagram schematically showing the configuration of the detection device 30 and camera system 3 according to the third embodiment.
  • the detection device 30 is a device that can implement the detection method according to the third embodiment.
  • components that are the same as or correspond to those shown in FIG. 1 are given the same reference numerals as those shown in FIG.
  • the detection device 30 according to the third embodiment differs from the detection device 10 according to the first embodiment in the configuration and operation of the model control section 12a.
  • the number m1 of frames is not determined in advance, but the model control unit 12a is You can decide how many frames to process.
  • FIG. 13 is a diagram illustrating an example of a process for determining a recommended AI model used by the detection processing unit 11 of the detection device 30 according to the third embodiment.
  • FIG. 13 shows how the average value Av of the detection accuracy of the three AI models 41, 42, and 43 changes with solid lines, broken lines, and dotted lines, respectively.
  • the model control unit 12a sets a detection accuracy threshold Th.
  • the threshold Th is temporarily set to "0.8".
  • the model control unit 12a switches the three AI models for each frame to improve the detection accuracy.
  • the calculation unit 14a calculates the average value Av of detection accuracy.
  • the model determining unit 15a selects the AI model 41 as This frame is determined as the recommended AI model to be used by the detection processing unit 11 from that frame onward.
  • the criterion for determining the recommended AI model to be used by the detection processing unit 11 is whether the average value Av of detection accuracy exceeds the threshold Th, and does not determine the number of frames in advance. .
  • the detection accuracy calculation unit 14a calculates the average value Av of the detection accuracy, and determines for each frame whether the average value Av is less than the threshold Th.
  • the model control unit 12a outputs the average value Av of the detection accuracy while switching between the three AI models.
  • the criterion for determining up to what point the recommended AI model (solid line, broken line, dotted line in FIG. 13) is used by the detection processing unit 11 is based on whether the average value Av of detection accuracy has fallen below the threshold Th. Yes, the number of frames is not specified in advance.
  • one AI model is determined as the recommended AI model from three AI models, but two or more AI models may be determined as the recommended AI model from two or more AI models. .
  • the model control unit 12 calculates the accuracy of each of the plurality of trained models, and sets a predetermined first threshold Th1 and a predetermined second threshold Th2 lower than the first threshold Th1. Then, calculate accuracy for each of the plurality of trained models, and when a trained model with accuracy exceeding the first threshold Th1 occurs, one or more trained models have accuracy exceeding the second threshold Th2. Determine the trained model of as the recommended trained model.
  • FIG. 14 is a diagram showing another example of the process for determining the recommended AI model used by the detection processing unit 11 of the detection device 30 according to the third embodiment.
  • FIG. 14 shows changes in the average detection accuracy Av of the four AI models 41, 42, 43, and 44 using solid lines, broken lines, dotted lines, and dashed-dotted lines, respectively.
  • FIG. 14 shows an example in which three AI models are determined from four AI models 41 to 44. If it is desired to determine a plurality of AI models, a threshold (hereinafter referred to as "second threshold Th2”) having a lower value than the threshold (hereinafter referred to as "first threshold Th1”) is set.
  • second threshold Th2 having a lower value than the threshold
  • the model control unit 12a determines whether the remaining three AI models exceed the second threshold Th2.
  • AI models 42, 43, and 44 exceed the second threshold Th2 at the 200th frame, all available AI models may be operated. For example, depending on the detection accuracy value of the AI model with the highest detection accuracy (assuming AI model 41), if AI model 41 exceeds the first threshold Th1, up to four AI models can be used; Even if the operating rules of the AI models can be changed based on two thresholds depending on the availability of hardware resources or processing load, such as up to four AI models if the threshold Th1 is not exceeded. good.
  • Embodiment 3 when the detection accuracy decreases, the use of that AI model is immediately stopped and the AI model used is switched to another AI model, so the number of frames is determined in advance. Detection accuracy can be maintained at a higher level than when specifying .
  • the third embodiment by setting a plurality of AI models as recommended AI models used in the detection processing unit, it is possible to operate a plurality of AI models with high detection accuracy.
  • the third embodiment is the same as the first or second embodiment.
  • 1 to 3 camera system 10, 20, 30 detection device, 50 imaging unit, 11 detection processing unit, 12 model control unit, 13, 13a model switching control unit, 14, 14a detection accuracy calculation unit, 15, 15a model determination unit , 16 Model development section, 16a Model release section, 17, 17a Working memory, 40 Storage device, 41, 42, 43, 44, 41a, 42a AI model (trained model), 121, 124 Input, 122 Person detection AI model , 123 Output, 123a Person detection result (object detection result), 123b Detection frame, 125 Face detection AI model, 126 Output, 126a Face detection result (object detection result), 126b Output.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Automation & Control Theory (AREA)
  • Computer Security & Cryptography (AREA)
  • Mathematical Physics (AREA)
  • Fuzzy Systems (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
PCT/JP2022/020398 2022-05-16 2022-05-16 検出装置、カメラシステム、検出方法、及び検出プログラム Ceased WO2023223386A1 (ja)

Priority Applications (4)

Application Number Priority Date Filing Date Title
US18/862,572 US20250287098A1 (en) 2022-05-16 2022-05-16 Detection device, camera system, detection method, and storage medium storing detection program
JP2022563098A JP7270856B1 (ja) 2022-05-16 2022-05-16 検出装置、カメラシステム、検出方法、及び検出プログラム
PCT/JP2022/020398 WO2023223386A1 (ja) 2022-05-16 2022-05-16 検出装置、カメラシステム、検出方法、及び検出プログラム
CN202280095730.7A CN119156634A (zh) 2022-05-16 2022-05-16 检测装置、摄像头系统、检测方法以及检测程序

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/020398 WO2023223386A1 (ja) 2022-05-16 2022-05-16 検出装置、カメラシステム、検出方法、及び検出プログラム

Publications (1)

Publication Number Publication Date
WO2023223386A1 true WO2023223386A1 (ja) 2023-11-23

Family

ID=86316761

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/020398 Ceased WO2023223386A1 (ja) 2022-05-16 2022-05-16 検出装置、カメラシステム、検出方法、及び検出プログラム

Country Status (4)

Country Link
US (1) US20250287098A1 (https=)
JP (1) JP7270856B1 (https=)
CN (1) CN119156634A (https=)
WO (1) WO2023223386A1 (https=)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019087229A (ja) * 2017-11-02 2019-06-06 キヤノン株式会社 情報処理装置、情報処理装置の制御方法及びプログラム
JP2019118097A (ja) * 2017-12-26 2019-07-18 キヤノン株式会社 画像処理方法、画像処理装置、撮像装置、プログラム、記憶媒体
JP2019175140A (ja) * 2018-03-28 2019-10-10 株式会社パスコ 建築物抽出システム

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11144786B2 (en) * 2017-11-02 2021-10-12 Canon Kabushiki Kaisha Information processing apparatus, method for controlling information processing apparatus, and storage medium
JP7064346B2 (ja) * 2018-02-13 2022-05-10 オリンパス株式会社 撮像装置、情報端末、撮像装置の制御方法、および情報端末の制御方法
US11321618B2 (en) * 2018-04-25 2022-05-03 Om Digital Solutions Corporation Learning device, image pickup apparatus, image processing device, learning method, non-transient computer-readable recording medium for recording learning program, display control method and inference model manufacturing method
JP7313942B2 (ja) * 2019-07-17 2023-07-25 株式会社エビデント 評価支援方法、評価支援システム、プログラム
JP7453772B2 (ja) * 2019-11-01 2024-03-21 キヤノン株式会社 情報処理装置及びその制御方法、撮像装置システム、プログラム
EP4207740A4 (en) * 2020-10-28 2024-02-21 Samsung Electronics Co., Ltd. ELECTRONIC DEVICE AND METHOD FOR CONTROLLING SAME

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019087229A (ja) * 2017-11-02 2019-06-06 キヤノン株式会社 情報処理装置、情報処理装置の制御方法及びプログラム
JP2019118097A (ja) * 2017-12-26 2019-07-18 キヤノン株式会社 画像処理方法、画像処理装置、撮像装置、プログラム、記憶媒体
JP2019175140A (ja) * 2018-03-28 2019-10-10 株式会社パスコ 建築物抽出システム

Also Published As

Publication number Publication date
JP7270856B1 (ja) 2023-05-10
CN119156634A (zh) 2024-12-17
US20250287098A1 (en) 2025-09-11
JPWO2023223386A1 (https=) 2023-11-23

Similar Documents

Publication Publication Date Title
WO2020057355A1 (zh) 一种三维建模的方法及其装置
JP6973533B2 (ja) 解析装置、解析方法及びプログラム
JP7040633B2 (ja) 群集レベル推定のための方法、システム及びプログラム
US20150002665A1 (en) Video-surveillance method, corresponding system, and computer program product
JP6246508B2 (ja) 動体画像抽出方法及び動体画像抽出装置
CN110427025B (zh) 一种巡检设备路径调整方法、装置和巡检设备
EP3400405B1 (en) Automatic lighting and security device
JP7040634B2 (ja) 群集推定手法の統合及び自動切り替えのための方法、システム及びプログラム
JP2002222427A (ja) デジタルカメラにおける動き検出方法
US20240414443A1 (en) Noise removal from surveillance camera image by means of ai-based object recognition
JP5192437B2 (ja) 物体領域検出装置、物体領域検出方法および物体領域検出プログラム
US10949713B2 (en) Image analyzing device with object detection using selectable object model and image analyzing method thereof
JP7492595B2 (ja) 監視システム、解析装置及びａｉモデル生成方法
JP2010238094A (ja) 操作入力装置および操作入力方法、並びにプログラム
US10192319B1 (en) Surveillance method and computing device using the same
CN113420615A (zh) 人脸活体检测方法和装置
JP7270856B1 (ja) 検出装置、カメラシステム、検出方法、及び検出プログラム
CN114639172B (zh) 高空抛物预警方法、系统、电子设备及存储介质
JP5950628B2 (ja) 物体検知装置、物体検知方法及びプログラム
Zweng et al. Introducing a statistical behavior model into camera-based fall detection
TWI589158B (zh) 監控資料的原始畫面儲存系統及其儲存方法
Senior et al. Interactive motion analysis for video surveillance and long term scene monitoring
JP2010166212A (ja) 動画像処理装置、動画像処理方法、プログラム及び記憶媒体
CN117891516A (zh) 唤醒设备的方法、系统及芯片
JP7369783B2 (ja) 情報処理方法、プログラム及び情報処理装置

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2022563098

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22942591

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202280095730.7

Country of ref document: CN

Ref document number: 18862572

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22942591

Country of ref document: EP

Kind code of ref document: A1

WWP Wipo information: published in national office

Ref document number: 18862572

Country of ref document: US