WO2023113196A1 - Dispositif et procédé pour fournir un procédé d'analyse intelligente d'image - Google Patents

Dispositif et procédé pour fournir un procédé d'analyse intelligente d'image Download PDF

Info

Publication number
WO2023113196A1
WO2023113196A1 PCT/KR2022/016083 KR2022016083W WO2023113196A1 WO 2023113196 A1 WO2023113196 A1 WO 2023113196A1 KR 2022016083 W KR2022016083 W KR 2022016083W WO 2023113196 A1 WO2023113196 A1 WO 2023113196A1
Authority
WO
WIPO (PCT)
Prior art keywords
detection
electronic device
conditions
sensing
intelligent
Prior art date
Application number
PCT/KR2022/016083
Other languages
English (en)
Korean (ko)
Inventor
최인명
신호환
원동일
이지훈
이진형
Original Assignee
삼성전자 주식회사
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from KR1020210193295A external-priority patent/KR20230091736A/ko
Application filed by 삼성전자 주식회사 filed Critical 삼성전자 주식회사
Publication of WO2023113196A1 publication Critical patent/WO2023113196A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition

Definitions

  • Various embodiments disclosed in this document relate to an apparatus and method for providing an intelligent image analysis method.
  • AI artificial intelligence
  • Bixby artificial intelligence
  • the technology for providing such a high-level service is a technology that self-learns and judges, and the recognition rate improves as it is used. It can be composed of element technologies that mimic the functions of the human brain, such as cognition and judgment.
  • visual understanding is a technology that recognizes and processes like human vision, and may include object recognition, object tracking, image search, person recognition, scene understanding, or spatial understanding.
  • test takers For example, in the case of a non-face-to-face online test, due to the nature of the online test, real-time management of test takers may be required, and fairness in the evaluation of test takers may be required.
  • current artificial intelligence systems can only detect specific motions in input images. Since the method of detecting only a specific motion assumes only a case in which the candidate has to perform a specific action, automated verification may be difficult when the candidate performs various actions or the surrounding environment of the candidate changes.
  • Various embodiments of the present disclosure relate to an apparatus and method for providing an intelligent video analysis method capable of generating a rule, which is a verification criterion, through acquisition, monitoring, and analysis of target data.
  • an apparatus for intelligent video analysis includes at least one processor and a memory, and when the memory is executed, the at least one processor analyzes at least one input image, thereby generating at least one image. Acquire a component, identify a plurality of sensing conditions based on the at least one component, generate a plurality of sensing condition sets based on the plurality of sensing conditions, and among the plurality of sensing condition sets Based on at least one, it may be set to store instructions for detecting the presence or absence of abnormalities in the target image.
  • a method for intelligent video analysis includes obtaining at least one component by analyzing at least one input image, identifying a plurality of sensing conditions based on the at least one component, An operation of generating a plurality of detection condition sets based on the plurality of detection conditions, and an operation of detecting whether there is an abnormality in the target image based on at least one of the plurality of detection condition sets.
  • rules that are verification criteria suitable for situations at a minimum cost through acquisition, monitoring, and analysis of object data, and various motions of the object can be detected based on the generated rules.
  • FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments.
  • FIG. 2 is a configuration diagram of a data processing system for collecting and processing target data according to various embodiments.
  • FIG 3 is an internal block diagram of an apparatus for providing an intelligent video analysis method according to various embodiments.
  • FIG. 4 is a diagram for explaining an operation of an analysis engine according to various embodiments.
  • FIG. 5 is a flowchart illustrating an operation of an apparatus for providing an intelligent video analysis method according to various embodiments.
  • FIG. 6 is a flowchart of an operation for generating a ruleset according to various embodiments.
  • FIG. 7 is an exemplary diagram illustrating an online test method according to various embodiments.
  • FIG. 8 is an exemplary view illustrating a real-time viewer image according to various embodiments.
  • FIG. 9 is an exemplary diagram for explaining ruleset creation and detection list creation operations according to various embodiments.
  • 10A is an exemplary diagram for explaining a method of generating a ruleset based on an input image according to various embodiments.
  • 10B is an exemplary view illustrating a rule set recommendation method according to various embodiments.
  • 11 is a flowchart illustrating an operation in an online test according to various embodiments.
  • FIG. 12 is an exemplary diagram illustrating real-time gazer images according to various embodiments.
  • FIG. 1 is a block diagram of an electronic device 101 within a network environment 100, according to various embodiments.
  • an electronic device 101 communicates with an electronic device 102 through a first network 198 (eg, a short-range wireless communication network) or through a second network 199. It may communicate with at least one of the electronic device 104 or the server 108 through (eg, a long-distance wireless communication network). According to one embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108 .
  • a first network 198 eg, a short-range wireless communication network
  • the server 108 e.g, a long-distance wireless communication network
  • the electronic device 101 includes a processor 120, a memory 130, an input module 150, an audio output module 155, a display module 160, an audio module 170, a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or the antenna module 197 may be included.
  • at least one of these components eg, the connection terminal 178) may be omitted or one or more other components may be added.
  • some of these components eg, sensor module 176, camera module 180, or antenna module 197) are integrated into a single component (eg, display module 160). It can be.
  • the processor 120 for example, executes software (eg, the program 140) to cause at least one other component (eg, hardware or software component) of the electronic device 101 connected to the processor 120. It can control and perform various data processing or calculations. According to one embodiment, as at least part of data processing or operation, the processor 120 transfers instructions or data received from other components (e.g., sensor module 176 or communication module 190) to volatile memory 132. , processing commands or data stored in the volatile memory 132 , and storing resultant data in the non-volatile memory 134 .
  • software eg, the program 140
  • the processor 120 transfers instructions or data received from other components (e.g., sensor module 176 or communication module 190) to volatile memory 132. , processing commands or data stored in the volatile memory 132 , and storing resultant data in the non-volatile memory 134 .
  • the processor 120 may include a main processor 121 (eg, a central processing unit or an application processor) or a secondary processor 123 (eg, a graphic processing unit, a neural network processing unit ( NPU: neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor).
  • a main processor 121 eg, a central processing unit or an application processor
  • a secondary processor 123 eg, a graphic processing unit, a neural network processing unit ( NPU: neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor.
  • NPU neural network processing unit
  • the secondary processor 123 may be implemented separately from or as part of the main processor 121 .
  • the secondary processor 123 may, for example, take the place of the main processor 121 while the main processor 121 is in an inactive (eg, sleep) state, or the main processor 121 is active (eg, running an application). ) state, together with the main processor 121, at least one of the components of the electronic device 101 (eg, the display module 160, the sensor module 176, or the communication module 190) It is possible to control at least some of the related functions or states.
  • the auxiliary processor 123 eg, image signal processor or communication processor
  • the auxiliary processor 123 may include a hardware structure specialized for processing an artificial intelligence model.
  • AI models can be created through machine learning. Such learning may be performed, for example, in the electronic device 101 itself where the artificial intelligence model is performed, or may be performed through a separate server (eg, the server 108).
  • the learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning or reinforcement learning, but in the above example Not limited.
  • the artificial intelligence model may include a plurality of artificial neural network layers.
  • Artificial neural networks include deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), It may be one of deep Q-networks or a combination of two or more of the foregoing, but is not limited to the foregoing examples.
  • the artificial intelligence model may include, in addition or alternatively, software structures in addition to hardware structures.
  • the memory 130 may store various data used by at least one component (eg, the processor 120 or the sensor module 176) of the electronic device 101 .
  • the data may include, for example, input data or output data for software (eg, program 140) and commands related thereto.
  • the memory 130 may include volatile memory 132 or non-volatile memory 134 .
  • the program 140 may be stored as software in the memory 130 and may include, for example, an operating system 142 , middleware 144 , or an application 146 .
  • the input module 150 may receive a command or data to be used by a component (eg, the processor 120) of the electronic device 101 from the outside of the electronic device 101 (eg, a user).
  • the input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (eg, a button), or a digital pen (eg, a stylus pen).
  • the sound output module 155 may output sound signals to the outside of the electronic device 101 .
  • the sound output module 155 may include, for example, a speaker or a receiver.
  • the speaker can be used for general purposes such as multimedia playback or recording playback.
  • a receiver may be used to receive an incoming call. According to one embodiment, the receiver may be implemented separately from the speaker or as part of it.
  • the display module 160 may visually provide information to the outside of the electronic device 101 (eg, a user).
  • the display module 160 may include, for example, a display, a hologram device, or a projector and a control circuit for controlling the device.
  • the display module 160 may include a touch sensor set to detect a touch or a pressure sensor set to measure the intensity of force generated by the touch.
  • the audio module 170 may convert sound into an electrical signal or vice versa. According to one embodiment, the audio module 170 acquires sound through the input module 150, the sound output module 155, or an external electronic device connected directly or wirelessly to the electronic device 101 (eg: Sound may be output through the electronic device 102 (eg, a speaker or a headphone).
  • the audio module 170 acquires sound through the input module 150, the sound output module 155, or an external electronic device connected directly or wirelessly to the electronic device 101 (eg: Sound may be output through the electronic device 102 (eg, a speaker or a headphone).
  • the sensor module 176 detects an operating state (eg, power or temperature) of the electronic device 101 or an external environmental state (eg, a user state), and generates an electrical signal or data value corresponding to the detected state. can do.
  • the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a bio sensor, It may include a temperature sensor, humidity sensor, or light sensor.
  • the interface 177 may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device 101 to an external electronic device (eg, the electronic device 102).
  • the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.
  • HDMI high definition multimedia interface
  • USB universal serial bus
  • SD card interface Secure Digital Card interface
  • audio interface audio interface
  • connection terminal 178 may include a connector through which the electronic device 101 may be physically connected to an external electronic device (eg, the electronic device 102).
  • the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).
  • the haptic module 179 may convert electrical signals into mechanical stimuli (eg, vibration or motion) or electrical stimuli that a user may perceive through tactile or kinesthetic senses.
  • the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.
  • the camera module 180 may capture still images and moving images. According to one embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.
  • the power management module 188 may manage power supplied to the electronic device 101 .
  • the power management module 188 may be implemented as at least part of a power management integrated circuit (PMIC), for example.
  • PMIC power management integrated circuit
  • the battery 189 may supply power to at least one component of the electronic device 101 .
  • the battery 189 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.
  • the communication module 190 is a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (eg, the electronic device 102, the electronic device 104, or the server 108). Establishment and communication through the established communication channel may be supported.
  • the communication module 190 may include one or more communication processors that operate independently of the processor 120 (eg, an application processor) and support direct (eg, wired) communication or wireless communication.
  • the communication module 190 is a wireless communication module 192 (eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (eg, : a local area network (LAN) communication module or a power line communication module).
  • a wireless communication module 192 eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module
  • GNSS global navigation satellite system
  • wired communication module 194 eg, : a local area network (LAN) communication module or a power line communication module.
  • a corresponding communication module is a first network 198 (eg, a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (eg, legacy It may communicate with the external electronic device 104 through a cellular network, a 5G network, a next-generation communication network, the Internet, or a telecommunications network such as a computer network (eg, a LAN or a WAN).
  • a telecommunications network such as a computer network (eg, a LAN or a WAN).
  • These various types of communication modules may be integrated as one component (eg, a single chip) or implemented as a plurality of separate components (eg, multiple chips).
  • the wireless communication module 192 uses subscriber information (eg, International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199.
  • subscriber information eg, International Mobile Subscriber Identifier (IMSI)
  • IMSI International Mobile Subscriber Identifier
  • the electronic device 101 may be identified or authenticated.
  • the wireless communication module 192 may support a 5G network after a 4G network and a next-generation communication technology, for example, NR access technology (new radio access technology).
  • NR access technologies include high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and access of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low latency (URLLC)).
  • eMBB enhanced mobile broadband
  • mMTC massive machine type communications
  • URLLC ultra-reliable and low latency
  • -latency communications can be supported.
  • the wireless communication module 192 may support a high frequency band (eg, mmWave band) to achieve a high data rate, for example.
  • the wireless communication module 192 uses various technologies for securing performance in a high frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), and full-dimensional multiplexing. Technologies such as input/output (FD-MIMO: full dimensional MIMO), array antenna, analog beam-forming, or large scale antenna may be supported.
  • the wireless communication module 192 may support various requirements defined for the electronic device 101, an external electronic device (eg, the electronic device 104), or a network system (eg, the second network 199).
  • the wireless communication module 192 is a peak data rate for eMBB realization (eg, 20 Gbps or more), a loss coverage for mMTC realization (eg, 164 dB or less), or a U-plane latency for URLLC realization (eg, Example: downlink (DL) and uplink (UL) each of 0.5 ms or less, or round trip 1 ms or less) may be supported.
  • eMBB peak data rate for eMBB realization
  • a loss coverage for mMTC realization eg, 164 dB or less
  • U-plane latency for URLLC realization eg, Example: downlink (DL) and uplink (UL) each of 0.5 ms or less, or round trip 1 ms or less
  • the antenna module 197 may transmit or receive signals or power to the outside (eg, an external electronic device).
  • the antenna module 197 may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (eg, PCB).
  • the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is selected from the plurality of antennas by the communication module 190, for example. can be chosen A signal or power may be transmitted or received between the communication module 190 and an external electronic device through the selected at least one antenna.
  • other components eg, a radio frequency integrated circuit (RFIC) may be additionally formed as a part of the antenna module 197 in addition to the radiator.
  • RFIC radio frequency integrated circuit
  • the antenna module 197 may form a mmWave antenna module.
  • the mmWave antenna module includes a printed circuit board, an RFIC disposed on or adjacent to a first surface (eg, a lower surface) of the printed circuit board and capable of supporting a designated high frequency band (eg, mmWave band); and a plurality of antennas (eg, array antennas) disposed on or adjacent to a second surface (eg, a top surface or a side surface) of the printed circuit board and capable of transmitting or receiving signals of the designated high frequency band. can do.
  • peripheral devices eg, a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)
  • signal e.g. commands or data
  • commands or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199 .
  • Each of the external electronic devices 102 or 104 may be the same as or different from the electronic device 101 .
  • all or part of operations executed in the electronic device 101 may be executed in one or more external electronic devices among the external electronic devices 102 , 104 , or 108 .
  • the electronic device 101 when the electronic device 101 needs to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device 101 instead of executing the function or service by itself.
  • one or more external electronic devices may be requested to perform the function or at least part of the service.
  • One or more external electronic devices receiving the request may execute at least a part of the requested function or service or an additional function or service related to the request, and deliver the execution result to the electronic device 101 .
  • the electronic device 101 may provide the result as at least part of a response to the request as it is or additionally processed.
  • cloud computing distributed computing, mobile edge computing (MEC), or client-server computing technology may be used.
  • the electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing.
  • the external electronic device 104 may include an internet of things (IoT) device.
  • Server 108 may be an intelligent server using machine learning and/or neural networks. According to one embodiment, the external electronic device 104 or server 108 may be included in the second network 199 .
  • the electronic device 101 may be applied to intelligent services (eg, smart home, smart city, smart car, or health care) based on 5G communication technology and IoT-related technology.
  • FIG. 2 is a configuration diagram 200 of a data processing system for collecting and processing target data according to various embodiments.
  • the data processing system 200 may include an electronic device 101 , an external electronic device 102 , an intelligent server 250 and one or more service providers 260 .
  • the intelligent server 250 may communicate with the electronic device 101, the external electronic device 102, and/or one or more service providers 260 through the network 240.
  • the intelligent server 250 may collect various types of input data through the network 240, and may accumulate and manage them in the form of a database. According to one embodiment, the intelligent server 250 may receive input data, for example, may receive training data or raw data. Intelligent server 250 may generate a sensing condition based on the input data. This input data can be used to learn the learning model loaded in the intelligent server 250.
  • the term 'rule' of the present disclosure may generally mean a sensing condition for processing various types of input data based on an artificial intelligence model.
  • Various types of input data (or content) to be processed based on the artificial intelligence model may include media data such as images, video, and audio data, and electronic documents, but are not limited thereto, and may be electronically processed by the artificial intelligence model. It may further include electronic data of a kind that can be analyzed (eg, values of software and sensors).
  • an image and/or video may be acquired as input data by a learning model, and the intelligent server 250 may perform object recognition, object tracking, Various motions corresponding to detection conditions or surrounding situations may be detected through image search, person recognition, scene understanding, or spatial understanding.
  • a plurality of sensing conditions are required to detect various motions from input data or to detect a surrounding situation, and a combination of the plurality of sensing conditions may be referred to as a rule set.
  • the storage form of these combinations may be referred to as a ruleset table.
  • the ruleset may be provided or generated by a learning model or an artificial intelligence system, for example, the intelligent server 250 .
  • the rule set may be selected from a set of predefined detection conditions (or a rule set table) or may be generated in real time in response to a user request.
  • the set of predefined sensing conditions may be generated or updated based on a learning model.
  • the intelligent server 250 may select at least one sensing condition set (eg, a rule set) from among a plurality of predefined sensing condition sets, or may dynamically or in real time generate sensing conditions.
  • the learned sensing conditions may be stored in various types of storage formats, such as database tables, spreadsheet files, or text files.
  • the set of sensing conditions may not simply be a list of sensing conditions, but may be grouped by combining or classifying components constituting the sensing conditions according to a specific condition.
  • the artificial intelligence model is a model learned based on a designated type of learning algorithm, and may be artificial intelligence models implemented to output (or obtain) result data by receiving and calculating various types of data. Learning is performed based on these input data, and learning models (eg, machine learning models and deep learning models) are generated and stored in the intelligent server 250 or transferred to the electronic device 101 and stored therein.
  • learning models eg, machine learning models and deep learning models
  • the learning model may be manufactured in the form of at least one hardware chip and loaded into the intelligent server 250 .
  • the learning model may be manufactured in the form of a dedicated hardware chip for artificial intelligence, or may be manufactured as a part of an existing general-purpose processor (eg, CPU, AP) and installed in various electronic devices described above.
  • the learning model loaded in the intelligent server 250 may be loaded in the electronic device 101 or the external electronic device 102, or may be loaded in the service provider 260 as well. In one embodiment, the learning model may be received (or downloaded) from the intelligent server 250 to the electronic device 101, but is not limited thereto.
  • sensing conditions generated based on input data or sensing conditions learned based on training data may be stored in a storage system accessible to the service provider 260 .
  • the storage system accessible to the service provider 260 may be implemented within the intelligent server 250 or implemented independently of the intelligent server 250 and managed by the intelligent server 250 .
  • the intelligent server 250 may provide the sensing conditions generated or learned based on the input data to the service provider 260, such as a development company, so that they may be used for online testing.
  • the service provider 260 such as a development company
  • the service provider 260 may analyze input data in real time based on a rule set provided from the intelligent server 250 and provide various services to users according to the analysis result.
  • the service provider 260 may create a service related to the analyzed data or provide the analyzed data to the electronic device 101 or the external electronic device 102 .
  • the service provider 260 may provide an online test service to the electronic device 101 .
  • the service provider 260 may be implemented in the form of a server, and may provide various services to users according to analysis results, and the types of services may not be limited thereto.
  • the service provider 260 may operate as a test-taking server that provides information necessary for the test taker to take the test, and may also operate as a supervising server that supervises the test taker's test-taking situation in real time. there is.
  • the electronic device 101 may operate as the intelligent server 250. According to an embodiment, the electronic device 101 may generate or learn sensing conditions based on input data. For example, when a camera application of the electronic device 101 is executed and/or driven, a learning model may detect various motions corresponding to detection conditions or detect surrounding situations based on detection conditions from input data. can According to another embodiment, the electronic device 101 may be an electronic device of a test taker. For example, the electronic device 101 may be an electronic device used by a test taker to take an exam. For example, the electronic device 101 may receive information related to taking a test including test questions from the service provider 260, and may transmit information necessary for taking the test (eg, test taker information) from the service provider 260. ) can be transmitted.
  • the electronic device 101 may receive information related to taking a test including test questions from the service provider 260, and may transmit information necessary for taking the test (eg, test taker information) from the service provider 260. ) can be transmitted.
  • the external electronic device 102 is a device for remotely supervising a test taker, and may be a device that captures a gaze image of the test taker in real time.
  • the external electronic device 102 may transmit to the service provider 260 information necessary for test supervision including images of the candidate and the surrounding environment (eg, real-time video of the candidate) to the service provider 260 .
  • the service provider 260 can analyze the received information and operate as a supervision server that supervises the gaze of the candidate in real time using the analyzed information.
  • the server for taking the test and the server for supervising the test may operate as independent devices, but are not limited thereto and may be included in one server (eg, the service provider 260) and operated.
  • the electronic device for taking a test and the electronic device for supervising a test may be included in and operated as one electronic device (eg, the electronic device 101).
  • the service provider 260 may provide a service for supervising test takers in real time based on detection conditions provided from the intelligent server 250 in order to prevent cheating. These detection conditions may be rules for detecting cheating of the test taker from input data.
  • the intelligent server 250 may provide the service provider 260 with sensing conditions necessary for supervision in real time. Accordingly, the service provider 260 may analyze the real-time candidate image data using an artificial intelligence algorithm, and notify an abnormal state if there is an abnormality as a result of the analysis.
  • Each operation of the aforementioned electronic device 101, external electronic device 102, and/or service provider 260 may be replaced with an operation of the intelligent server 250.
  • the intelligent server 250 analyzes real-time candidate image data using a learning model, and if there is an abnormality as a result of the analysis based on a rule set, the supervising subject may be notified of the abnormal state.
  • a set of detection conditions (eg, a rule set) serving as a criterion for verification of a verification target (eg, a candidate) needs to be diverse and high in accuracy.
  • a verification target eg, a candidate
  • FIG. 3 is an internal block diagram of an apparatus for providing an intelligent video analysis method according to various embodiments.
  • FIG. 3 will be described with reference to FIG. 4 .
  • 4 is a diagram for explaining an operation of an analysis engine according to various embodiments.
  • an apparatus for providing an intelligent video analysis method (hereinafter referred to as an intelligent video analysis apparatus) may be implemented in various electronic devices such as a smart phone and a tablet PC.
  • the intelligent video analysis device 300 may include a processor 320 , a memory 330 and/or a communication interface 390 .
  • 3 shows an example of the configuration of the intelligent video analysis device 300, but is not limited to the configurations shown in FIG. 3 and may be implemented to include more configurations than those shown in FIG.
  • the intelligent video analysis device 300 of FIG. 3 may be the electronic device 101 or the intelligent server 250 of FIG. 2 . Accordingly, the operation of the intelligent server 250 may be replaced with the operation of the electronic device 101 .
  • the intelligent server 250 may operate using a pre-stored learning model, but is not limited thereto, and according to various embodiments, the electronic device 101 receives the learning model from the intelligent server 250 ( or download) to generate, edit, or learn a rule set for processing input data.
  • the intelligent video analysis device 300 is the electronic device 101 as an example, but it can be replaced with the intelligent server 250 as a matter of course.
  • the communication interface 390 may include software and/or hardware modules for wired/wireless communication with a network or server (eg, the intelligent server 250 or the service provider 260). Input data or learned sensing conditions may be provided from the outside through the communication interface 390, or a learning model may be provided.
  • a network or server eg, the intelligent server 250 or the service provider 260.
  • Input data or learned sensing conditions may be provided from the outside through the communication interface 390, or a learning model may be provided.
  • the memory 330 may store instructions for an operation of analyzing input data (eg, a real-time candidate image and/or video) using an artificial intelligence algorithm.
  • the memory 330 may include a database 335 that stores sensing conditions (or rule sets) generated based on input data or sensing conditions learned based on training data in a table form.
  • the processor 320 may detect components constituting the sensing conditions by analyzing the input data when obtaining the input data. For example, the processor 320 may obtain elements such as body parts, motions, objects, or duration information from the input data through analysis of the input data. At this time, the processor 320 may generate detection conditions by combining each component according to user settings, and otherwise obtain corresponding components from input data through repetitive learning and automatically based on the acquired components. You can also create detection conditions.
  • the processor 320 may perform an operation of removing and normalizing noise from the learned sensing conditions. Through such repetitive learning, accuracy of detection conditions serving as criteria for detecting cheating of the test taker may be increased.
  • the processor 320 may detect whether the target image is abnormal based on at least one sensing condition among sensing conditions (or rule sets) generated based on input data or a combination of sensing conditions learned based on training data.
  • a detection list for detecting anomalies in the target image may be generated using a combination of detection conditions.
  • the detection list is generated by including a recommended detection condition among learned combinations of detection conditions or by a user, for example, a supervisor. It may be generated by including a detection condition corresponding to the selection. In this way, the detection list may be modified by selecting desired detection conditions according to user settings in addition to the detection list provided by default.
  • the user may select a combination of sensing conditions that the user wants to use from combinations of learned sensing conditions stored in the database 335, and if there is no sensing condition that the user wants to use again among the combinations of learned sensing conditions,
  • the processor 320 may generate new sensing conditions based on input data. In this way, detection conditions generated automatically or according to user settings can be separately managed through the 'my detection list' function.
  • the processor 320 may perform analysis on the input data in order to obtain components for constructing a sensing condition from the input data. Reference will be made to FIG. 4 to describe the analysis of such input data in detail.
  • FIG. 4 is a diagram for explaining an operation of an analysis engine according to various embodiments.
  • the analysis engine 400 includes a body recognition engine 410, a motion analysis engine 420, an object recognition engine 430, a time analysis engine 440, and/or a sound analysis engine 450. can do.
  • the analysis engine 400 may be a module mounted on the processor 320 of FIG. 3 or implemented in an independent form.
  • the body recognition engine 410 may analyze input data (eg, images and/or videos) when acquiring them. For example, the body recognition engine 410 may recognize objects included in the input image (eg, body parts such as head, hand, body, and arm) and then classify the detected objects for each part. In addition, the body recognition engine 410 may recognize face parts such as eyes, nose, mouth, and ears, and may identify a face angle and a gaze direction using a face mesh. Therefore, through the body recognition engine 410, components related to the input data may be classified for each body part.
  • objects included in the input image eg, body parts such as head, hand, body, and arm
  • face parts such as eyes, nose, mouth, and ears
  • the motion analysis engine 420 may analyze a state (or motion) of an object included in input data (eg, an image and/or video). For example, the motion analysis engine 420 uses a posenet algorithm or a body pix algorithm to estimate motion, lying down, running, Behaviors such as raising arms, jumping, bending, and disappearing can be identified. Accordingly, components related to the input data may be classified according to behaviors through the motion analysis engine 420 .
  • the object recognition engine 430 uses object detection technology to determine the type of object (eg, a monitor, a mobile phone, a keyboard, a book) included in input data (eg, an image and/or video). objects) can be identified. Accordingly, components related to the input data may be classified for each object through the object recognition engine 430 .
  • object detection technology uses object detection technology to determine the type of object (eg, a monitor, a mobile phone, a keyboard, a book) included in input data (eg, an image and/or video). objects) can be identified. Accordingly, components related to the input data may be classified for each object through the object recognition engine 430 .
  • the time analysis engine 440 may analyze the duration of a result detected by at least one of the body recognition engine 410, the motion analysis engine 420, and the object recognition engine 430. For example, components related to the input data through the time analysis engine 440 may be classified according to required time, maintenance time, or exposure time according to the action.
  • the sound analysis engine 450 may acquire audio data such as speech and/or ambient sounds as input data through a microphone and analyze them. Externally generated sounds may include voices (or utterances) of speakers (eg, a user and/or other speakers (or others)), life noise, and ambient (or background) noises. For example, the sound analysis engine 450 may classify voice and noise as components related to audio data through signal processing that analyzes the size of a db or recognizes a language.
  • a case of classifying various components for generating a set of detection conditions (eg, a rule set) from input data has been described, and body parts, motions, objects, time, or sounds are examples of the components.
  • a set of detection conditions eg, a rule set
  • body parts, motions, objects, time, or sounds are examples of the components.
  • any component that can be independently extracted such as electronic data (eg, software, sensor values) that can be electronically analyzed by an artificial intelligence model from input data, may be used.
  • the apparatus 300 for intelligent image analysis includes at least one processor 320 and a memory 330, and the memory 330, when executed, the at least one processor ( 320) acquires at least one component by analyzing at least one input image, identifies a plurality of sensing conditions based on the at least one component, and performs a plurality of sensing conditions based on the plurality of sensing conditions. It may be configured to generate sensing condition sets and store instructions for detecting whether or not there is an abnormality in the target image based on at least one of the plurality of sensing condition sets.
  • the at least one processor may be set to generate the plurality of sensing condition sets through repetitive learning based on the plurality of sensing conditions.
  • the at least one component may include first data representing a body part included in the at least one input image, second data representing an operation of an object, third data representing an object, or the object. It may include at least one of the fourth data indicating the time related to the operation of the .
  • the at least one processor combines the first to fourth data included in the at least one component, and generates the plurality of sensing conditions through the combination. can be set to
  • the at least one processor may be set to generate the plurality of sensing conditions in consideration of overlapping sensing conditions through the combination.
  • the at least one processor may be configured to recommend a detection condition set having the highest similarity comparison result score with the generated plurality of detection conditions among the plurality of detection condition sets. there is.
  • the instructions may be set such that the at least one processor generates a new sensing condition associated with the plurality of generated sensing conditions and recommends the new sensing condition.
  • the at least one processor may be configured to recommend a sensing condition set among the plurality of sensing condition sets based on a frequency of use.
  • the at least one processor may be configured to recommend a sensing condition set based on a frequency of use and a number of recommendations among the plurality of sensing condition sets.
  • the at least one processor may be configured to detect whether or not there is an abnormality in the target image based on a recommended sensing condition set among the plurality of sensing condition sets.
  • FIG. 5 is a flowchart 500 of operations in the intelligent video analysis device 300 according to various embodiments.
  • the intelligent video analysis device 300 may acquire at least one component by analyzing at least one input image in operation 505 .
  • the at least one component may include first data representing a body part included in the at least one input image, second data representing an operation of an object, third data representing an object, or the object. It may include at least one of the fourth data indicating the time related to the operation of the .
  • the intelligent video analysis device 300 may identify a plurality of sensing conditions based on the at least one component. According to an embodiment, the intelligent video analysis device 300 may combine the first to fourth data included in the at least one component, and generate the plurality of detection conditions through the combination. .
  • the first sensing condition may be generated by combining the first data and the second data
  • the second sensing condition may be generated by combining the second data and the fourth data
  • the first to third data A third sensing condition may be created by combining all of them.
  • the intelligent video analysis device 300 may generate a plurality of different sensing conditions through various combinations of components.
  • the intelligent video analysis device 300 may generate the plurality of sensing conditions in consideration of overlapping sensing conditions through the combination.
  • the intelligent video analysis apparatus 300 may generate a plurality of detection condition sets based on the plurality of detection conditions.
  • the intelligent video analysis device 300 may generate the plurality of detection condition sets through iterative learning (eg, machine learning) based on the plurality of detection conditions.
  • the intelligent video analysis device 300 may detect whether there is an abnormality in the target image based on at least one of the plurality of detection condition sets.
  • the intelligent video analysis device 300 may detect whether or not there is an abnormality in the target image based on a recommended sensing condition set among the plurality of sensing condition sets.
  • a set of detection conditions may be recommended based on the following method.
  • the intelligent video analysis device 300 may recommend a sensing condition set having the highest similarity comparison result score with the generated plurality of sensing conditions among the plurality of sensing condition sets. For example, the intelligent video analysis apparatus 300 may identify a similarity between a plurality of generated detection conditions (eg, rulesets) and existing detection condition sets (eg, ruleset tables). If the plurality of generated sensing conditions include conditions such as 'there is no person on the screen', 'there are many people on the screen', and 'book', a sensing condition set similar to these conditions may be recommended.
  • a plurality of generated detection conditions eg, rulesets
  • existing detection condition sets eg, ruleset tables
  • the intelligent video analysis device 300 may create a new sensing condition associated with the plurality of generated sensing conditions and recommend the created new sensing condition. For example, the intelligent video analysis device 300 considers connectivity between the plurality of generated detection conditions and, if the generated plurality of detection conditions include a condition of 'book', generally reads a book. Since the hand-raising motion is predictable, the 'hand-raising' condition can be recommended as a new detection condition.
  • the intelligent video analysis device 300 may recommend a detection condition set based on a frequency of use among the plurality of detection condition sets. For example, the intelligent video analysis device 300 may recommend a detection condition set with a high frequency of use based on data accumulated through previous tests. If a specific sensing condition set was frequently used in previous tests, the score for the specific sensing condition set may be in a state of being increased through repetitive learning. Accordingly, the intelligent video analysis device 300 may recommend a detection condition set having the highest score.
  • the intelligent video analysis device 300 may recommend a sensing condition set based on the frequency of use and the number of recommendations among the plurality of sensing condition sets. For example, the intelligent video analysis device 300 may recommend a corresponding detection condition when many of the detection conditions of 'there is no person on the screen' are satisfied in previous tests. That is, the detection condition may be recommended when situations satisfying the detection condition of 'no person on the screen' occur frequently during supervision and management, and when the detection condition of 'no person on the screen' is recommended a lot.
  • FIG. 6 is a flowchart 600 of an operation for generating a ruleset according to various embodiments.
  • the intelligent video analysis device 300 may obtain an input image.
  • the input image denotes an image and/or video of a trainee, and may be a plurality of images captured during the test time.
  • the intelligent video analysis device 300 may detect body parts, motions, objects, or duration information corresponding to components of the rule (or detection condition) by analyzing the input image.
  • the intelligent video analysis apparatus 300 may generate a plurality of rule sets through a combination of body parts, motions, objects, or duration information.
  • the intelligent video analysis device 300 may generate and/or update a rule set table based on the learning algorithm. In operation 625, the intelligent video analysis device 300 may generate a verification list using a rule set table.
  • the intelligent video analysis device 300 may identify whether a verification request is received from a supervisor as the actual exam begins.
  • the intelligent video analysis device 300 may analyze a target image (eg, a real-time candidate's image and/or video) using the verification list.
  • the intelligent video analysis device 300 may output an analysis result. For example, when components such as body parts, objects such as things, and motions of objects are extracted from input data of a test taker in real time, a combination of the components corresponds to a combination of detection conditions included in the verification list. can identify what it does. If a combination of these components corresponds to a combination of detection conditions on the verification list, the candidate's action may be regarded as a cheating action. Specific examples of detection conditions for detecting such fraudulent behavior will be described later with reference to FIG. 12 .
  • FIG. 7 is an exemplary diagram illustrating an online test method according to various embodiments. 8 will be referred to for better understanding of the description of FIG. 7 . 8 is an exemplary view illustrating a real-time viewer image according to various embodiments.
  • an intelligent server 250 may receive an image of a candidate 710 from the electronic device 101 in real time through communication with the electronic device 101, and manage and supervise the online test. It can be transmitted to the service provider 260 that provides the service.
  • the electronic device 101 may be a device that captures an image of the candidate 710 and a surrounding environment in real time.
  • the intelligent server 250 may capture a gaze image of the candidate 710 in real time using another electronic device (eg, the electronic device 101 or the external electronic device 102).
  • the intelligent server 250 provides a real-time gaze image for remotely supervising the candidate 710 to the service provider 260 and at the same time detects various motions of the candidate 710 or detection conditions used to detect surrounding situations. can be created, edited and/or taught.
  • the intelligent server 250 may process and analyze real-time input data (eg, candidate image and/or video, sound) and use it as information necessary for supervision.
  • the intelligent server 250 may analyze input data related to a test taker in real time and recognize a specific object required for supervision from the input data.
  • the intelligent server 250 may include a specific body part 810 and face part 820 (eg, the candidate's face, both hands), surrounding objects (eg, the electronic device 824 possessed by the candidate), Devices 830 and 840 required for gaze may be recognized as specific objects.
  • the intelligent server 250 may obtain data representing the operation of the object by extracting data representing surrounding objects in addition to the specific object in real time, and objects such as required time, retention time, or exposure time according to the operation of the object Data representing the time associated with the operation of may be obtained. Accordingly, the intelligent server 250 may monitor the candidate's behavior and the surrounding environment of the candidate in real time by using a part of the candidate's body and/or an object around the candidate as a recognition target object.
  • the intelligent server 250 may identify whether the monitored behavior of the candidate and the surrounding environment of the candidate fall within a sensing range defined by a set of sensing conditions. For example, the intelligent server 250 may set a detection list using a set of detection conditions, and the detection list may be selectable or changeable according to user settings. Accordingly, the intelligent server 250 determines that an abnormality has occurred for the candidate when the candidate's behavior and the candidate's surrounding environment correspond to the detection list, for example, when the candidate's surrounding environment is out of the range of the gaze condition, and outputs a notification informing of the abnormal state. . For example, the intelligent server 250 may display a notification signal through a warning window or output a notification sound to inform a supervisor of an abnormal condition.
  • FIG. 9 is an exemplary diagram for explaining ruleset creation and detection list creation operations according to various embodiments.
  • a rule set component 905 represents various components for generating a set of detection conditions (eg, a rule set) from input data
  • body parts, motions, or time are examples of the components. are doing For example, data by part (e.g. face, hand, body, arm, ...), data by motion (e.g. disappear, movement,%), data by time (e.g. 0.1 second, 1 second, 4 second, ...)
  • Data can be classified by component as in 9 illustrates an example of components detectable from input data, but the types are not limited thereto.
  • any component that can be independently extracted such as electronic data (eg, software, sensor values) that can be electronically analyzed by an artificial intelligence model from input data, may be used.
  • the intelligent video analysis device 300 may generate a plurality of detection conditions 911 and 912 through a combination (or definition) of each data classified in the rule set component 905 .
  • a plurality of sensing conditions may be generated by combining classified data according to a definition method set as a default in advance. Then, when a lot of training data is accumulated through repetitive learning, data combinations may be changed according to various conditions such as frequency of use, and accordingly, various combinations of detection conditions (or rule sets) 911 and 912 are generated. It can be.
  • the first rule set 911 and the second rule set 912 may be stored in a table form 910 through a combination of each classified data.
  • the intelligent video analysis device 300 may configure detection lists 920 necessary for supervision among a plurality of detection condition sets generated through repetitive learning.
  • each of the detection lists 921 and 922 may be generated according to user selection, or may be generated through a recommendation method.
  • detection list 1 921 may include a first rule set 911 and a second rule set 912
  • detection list 2 922 may include rulesets different from the first rule set 911. can be configured.
  • This detection list represents a combination of detection conditions for detecting abnormality from the input data of the candidate, and may be selected by, for example, an actual test-taking supervisor.
  • the intelligent video analysis device 300 may select at least one sensing condition set (eg, a rule set) or dynamically or in real time generate sensing conditions by a learning model.
  • the intelligent video analysis device 300 may classify (930) items to be detected using a detection list. For example, when classifying content corresponding to detection conditions included in the detection list, there may be some overlapping content, so the intelligent video analysis device 300 performs a classification operation 930 to exclude overlapping detection conditions. can be performed.
  • the intelligent video analysis device 300 may generate sensing conditions based on overlapping or not when generating sensing conditions through a combination of components obtained through analysis of input data. For example, in the case of a plurality of detection conditions, at least some constituting the detection conditions may overlap.
  • Data representing object a in image A through image analysis in the intelligent video analysis device 300 (eg, 'book' is When data representing object a) and data representing object b are obtained, and data representing object a (e.g., data indicating that 'book' has been detected) and data representing object c are acquired in image B, common object a (eg, data indicating that 'book' has been detected) is left as a valid (or highly reliable) detection condition, and then other data can be organized. For example, when sensing conditions overlap, only at least one most efficient sensing condition may be left and the rest may be discarded.
  • the intelligent video analysis device 300 may determine whether there is an abnormality 960 corresponding to the conditions classified 930 through analysis 950 of the target image.
  • the target image may be used as data for automatically generating detection conditions in the learning model.
  • a list of detections selected for candidate supervision i.e. a set of detection conditions, conditions for left hand: disappearing 2 seconds, hand raised 1 second, for object: person 2 seconds, book 2 seconds, face for disappearing 2 seconds.
  • the intelligent video analysis device 300 may identify whether the image falls within the range of the set of detection conditions.
  • the intelligent video analysis device 300 may identify that the condition of left hand disappearing for 2 seconds and face moving for 2 seconds according to the analysis result of the target image corresponds to the detection condition set. Accordingly, the intelligent video analysis device 300 may regard the candidate as a cheater and report the abnormal state to the supervisor.
  • the intelligent video analysis device 300 performs the input data It is possible to automatically generate detection conditions through a combination of each component based on the analysis result by analyzing . Since these detection conditions are learned by the learning model using input data related to various behaviors of the candidate and surrounding situations, it is possible to easily monitor whether or not the candidate cheated.
  • FIG. 10A is an exemplary diagram for explaining a method of generating a ruleset based on an input image according to various embodiments
  • FIG. 10B is a diagram continuing from FIG. 10A and illustrating a method of recommending a ruleset according to various embodiments.
  • the intelligent video analysis device 300 may obtain a plurality of images 1000.
  • the plurality of images 1000 may include training images (or abnormal motion images) 1001 , 1002 , and 1003 serving as standards for detecting cheating.
  • the plurality of images 1000 may further include input images of a viewer captured in real time.
  • the intelligent video analysis device 300 may analyze the plurality of images 1000 and obtain components from each of the plurality of images through analysis.
  • the intelligent video analysis device 300 may generate (1010) a plurality of detection conditions (eg, a rule set) through a combination of these components.
  • a plurality of detection conditions eg, a rule set
  • the first rule set 1012 includes a condition of 'no person on the screen' classified as 'motion'
  • the second rule set 1013 includes the first image ( 1001)
  • the third rule set 1014 includes the condition of 'book' classified as 'object' there is.
  • the intelligent video analysis apparatus 300 sets newly generated detection conditions (eg, the first rule set 1012, the second rule set 1013, and the third rule set 1014) into a plurality of detection condition sets. (eg, the learned ruleset set table 1020). As a result of the comparison, the intelligent video analysis apparatus 300 may recommend a sensing condition set having the highest score as a result of similarity comparison with the generated sensing conditions among the plurality of sensing condition sets.
  • newly generated detection conditions eg, the first rule set 1012, the second rule set 1013, and the third rule set (eg, the learned rule set table 1020) among a plurality of detection condition sets (eg, the learned rule set table 1020) 1014)
  • the most similar detection condition combination eg, rule set #4
  • the intelligent video analysis apparatus 300 determines a new detection condition related to newly created detection conditions (eg, the first rule set 1012, the second rule set 1013, and the third rule set 1014). can create Accordingly, the intelligent video analysis apparatus 300 may add the generated detection conditions together with a new detection condition to a plurality of detection condition sets (eg, the learned rule set table 1020). For example, a detection condition combination (eg, rule set #1, rule set #n) 1030 and 1070 including a new detection condition associated with the first rule set 1012 is created and added to the table 1020 can do. In a similar way, the detection condition combination (eg, rule set #1) 1040 may include a new detection condition related to the second rule set 1013 .
  • a detection condition combination eg, rule set #1, rule set #n
  • the detection condition combination (eg, rule set #1) 1040 may include a new detection condition related to the second rule set 1013 .
  • the intelligent video analysis apparatus 300 may recommend at least one detection condition set based on frequency of use among a plurality of detection condition sets (eg, the learned rule set table 1020).
  • the intelligent video analysis apparatus 300 recommends at least one detection condition set among a plurality of detection condition sets (eg, the learned rule set table 1020) in consideration of the number of recommendations in addition to the frequency of use. You may.
  • combinations (1030, 1040, 1050, 1060, 1070) of detection conditions (or rule sets) generated based on input data or learned detection conditions based on training data can be stored and managed in a table form. there is.
  • components corresponding to Table 1 below are extracted from input data, the components are compared and analyzed with previous sensing conditions, and learned sensing conditions may be generated.
  • new combinations of sensing conditions can be added to these combinations of sensing conditions (1030, 1040, 1050, 1060, and 1070) through repetitive learning when acquiring input data, and can be edited and updated for each test.
  • 11 is a flowchart illustrating an operation in an online test according to various embodiments.
  • the intelligent video analysis device 300 may select a test detection list in operation 1110.
  • the test detection list may be determined by the supervisor's selection from among the basic detection lists.
  • the side camera rule set detection list 1115 may include a combination of components such as body part, camera gaze, arm raised, book, and cup.
  • a basic detection list corresponding to each online test format may be pre-configured.
  • a function of additionally modifying the detection list to suit the current online test format may be provided.
  • the intelligent video analysis device 300 may further modify the selected list according to user input.
  • the basic detection list may be continuously updated using learned detection conditions.
  • the candidate's image may be input in real time in operation 1130, and the intelligent video analysis device 300 detects an abnormal state based on the detection conditions of the detection list modified in operation 1135. can detect If an abnormal state is detected, the supervisor may be notified of the abnormal state in operation 1140 .
  • FIG. 12 is an exemplary diagram illustrating real-time gazer images according to various embodiments.
  • the intelligent video analysis device 300 may analyze input data (eg, a real-time viewer image and/or video).
  • the intelligent video analysis device 300 may acquire components for configuring detection conditions that are standards for verification of a verification target (eg, a candidate) required for test supervision through input data analysis.
  • the intelligent video analysis device 300 may generate a set of various types of sensing conditions through repetitive learning based on the sensing conditions according to the combination of these components.
  • the intelligent video analysis device 300 may generate a detection list for detecting cheating from a set of detection conditions in response to a supervisor's selection.
  • the intelligent video analysis device 300 may detect fraud based on the detection list as shown in Table 2 above. For example, the intelligent video analysis device 300 selects images (1210, 1215, 1220, 1225, 1230, 1235, and 1240) for which analysis results are misconduct among input data (eg, real-time image and/or video). , 1245, 1250), it can be regarded as cheating.
  • input data eg, real-time image and/or video
  • 1245, 1250 it can be regarded as cheating.
  • the set of detection conditions generated in this way may be continuously updated through repetitive learning (eg, machine learning) based on input data. Therefore, since a set of existing sensing conditions can be edited and a new sensing condition related to it can be added, various sensing conditions with high accuracy can be provided in consideration of the candidate's behavior and the surrounding environment.
  • repetitive learning eg, machine learning
  • Electronic devices may be devices of various types.
  • the electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance.
  • a portable communication device eg, a smart phone
  • a computer device e.g., a smart phone
  • a portable multimedia device e.g., a portable medical device
  • a camera e.g., a portable medical device
  • a camera e.g., a portable medical device
  • a camera e.g., a portable medical device
  • a camera e.g., a camera
  • a wearable device e.g., a smart bracelet
  • first, second, or first or secondary may simply be used to distinguish a given component from other corresponding components, and may be used to refer to a given component in another aspect (eg, importance or order) is not limited.
  • a (e.g., first) component is said to be “coupled” or “connected” to another (e.g., second) component, with or without the terms “functionally” or “communicatively.”
  • the certain component may be connected to the other component directly (eg by wire), wirelessly, or through a third component.
  • module used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and is interchangeable with terms such as, for example, logic, logical blocks, parts, or circuits.
  • a module may be an integrally constructed component or a minimal unit of components or a portion thereof that performs one or more functions.
  • the module may be implemented in the form of an application-specific integrated circuit (ASIC).
  • ASIC application-specific integrated circuit
  • a storage medium eg, internal memory 136 or external memory 138
  • a machine eg, electronic device 101
  • a processor eg, the processor 120
  • a device eg, the electronic device 101
  • the one or more instructions may include code generated by a compiler or code executable by an interpreter.
  • the device-readable storage medium may be provided in the form of a non-transitory storage medium.
  • the storage medium is a tangible device and does not contain a signal (e.g. electromagnetic wave), and this term refers to the case where data is stored semi-permanently in the storage medium. It does not discriminate when it is temporarily stored.
  • a signal e.g. electromagnetic wave
  • the method according to various embodiments disclosed in this document may be included and provided in a computer program product.
  • Computer program products may be traded between sellers and buyers as commodities.
  • a computer program product is distributed in the form of a device-readable storage medium (e.g. compact disc read only memory (CD-ROM)), or through an application store (e.g. Play Store TM ) or on two user devices (e.g. It can be distributed (eg downloaded or uploaded) online, directly between smart phones.
  • a device e.g. compact disc read only memory (CD-ROM)
  • an application store e.g. Play Store TM
  • It can be distributed (eg downloaded or uploaded) online, directly between smart phones.
  • at least part of the computer program product may be temporarily stored or temporarily created in a storage medium readable by a device such as a manufacturer's server, an application store server, or a relay server's memory.
  • each component (eg, module or program) of the above-described components may include a single object or a plurality of entities, and some of the plurality of entities may be separately disposed in other components. there is.
  • one or more components or operations among the aforementioned corresponding components may be omitted, or one or more other components or operations may be added.
  • a plurality of components eg modules or programs
  • the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by a corresponding component of the plurality of components prior to the integration. .
  • the actions performed by a module, program, or other component are executed sequentially, in parallel, iteratively, or heuristically, or one or more of the actions are executed in a different order, or omitted. or one or more other actions may be added.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

Un dispositif d'analyse intelligente d'image, selon divers modes de réalisation, comprend au moins un processeur et une mémoire, la mémoire pouvant être configurée pour stocker des instructions qui, lors de leur exécution, permettent audit processeur d'obtenir au moins une composante en analysant au moins une image d'entrée, d'identifier une pluralité de conditions de détection sur la base de ladite composante, de générer une pluralité d'ensembles de conditions de détection sur la base de la pluralité de conditions de détection, et de détecter la présence/l'absence d'anomalies dans une image cible sur la base d'au moins un ensemble de conditions de détection parmi la pluralité d'ensembles de conditions de détection. La présente invention peut concerner divers autres modes de réalisation.
PCT/KR2022/016083 2021-12-16 2022-10-20 Dispositif et procédé pour fournir un procédé d'analyse intelligente d'image WO2023113196A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
KR10-2021-0180462 2021-12-16
KR20210180462 2021-12-16
KR1020210193295A KR20230091736A (ko) 2021-12-16 2021-12-30 지능형 영상 분석 방법을 제공하기 위한 장치 및 방법
KR10-2021-0193295 2021-12-30

Publications (1)

Publication Number Publication Date
WO2023113196A1 true WO2023113196A1 (fr) 2023-06-22

Family

ID=86772909

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2022/016083 WO2023113196A1 (fr) 2021-12-16 2022-10-20 Dispositif et procédé pour fournir un procédé d'analyse intelligente d'image

Country Status (1)

Country Link
WO (1) WO2023113196A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102060301B1 (ko) * 2019-04-19 2020-02-11 한국전력공사 전력데이터 기반 생활 행동 패턴 분석 장치 및 방법
KR20210023273A (ko) * 2019-08-22 2021-03-04 아이브스 주식회사 터널 상황 판단 장치 및 그 방법
KR20210066541A (ko) * 2019-11-28 2021-06-07 서은석 영상 분석 기반 비정상 객체 검출 시스템 및 방법
KR102321917B1 (ko) * 2020-07-13 2021-11-04 주식회사 두두아이티 온라인 시험 화상 원격 관리 및 감독 시스템
KR20210138477A (ko) * 2020-05-12 2021-11-19 삼성전자주식회사 온라인 시험을 제공하는 전자 장치 및 그 방법

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102060301B1 (ko) * 2019-04-19 2020-02-11 한국전력공사 전력데이터 기반 생활 행동 패턴 분석 장치 및 방법
KR20210023273A (ko) * 2019-08-22 2021-03-04 아이브스 주식회사 터널 상황 판단 장치 및 그 방법
KR20210066541A (ko) * 2019-11-28 2021-06-07 서은석 영상 분석 기반 비정상 객체 검출 시스템 및 방법
KR20210138477A (ko) * 2020-05-12 2021-11-19 삼성전자주식회사 온라인 시험을 제공하는 전자 장치 및 그 방법
KR102321917B1 (ko) * 2020-07-13 2021-11-04 주식회사 두두아이티 온라인 시험 화상 원격 관리 및 감독 시스템

Similar Documents

Publication Publication Date Title
WO2020130689A1 (fr) Dispositif électronique pour recommander un contenu de jeu et son procédé de fonctionnement
WO2020159217A1 (fr) Dispositif électronique et procédé de détermination de tâche comprenant plusieurs actions
WO2019177373A1 (fr) Dispositif électronique pour commander une fonction prédéfinie sur la base d'un temps de réponse d'un dispositif électronique externe à une entrée d'utilisateur, et procédé associé
WO2020153785A1 (fr) Dispositif électronique et procédé pour fournir un objet graphique correspondant à des informations d'émotion en utilisant celui-ci
WO2018124633A1 (fr) Dispositif électronique et procédé de délivrance d'un message par celui-ci
WO2021045552A1 (fr) Dispositif électronique de synthèse d'image et son procédé de fonctionnement
WO2022211271A1 (fr) Dispositif électronique pour traiter une saisie manuscrite sur la base d'un apprentissage, son procédé de fonctionnement et support de stockage
WO2022131739A1 (fr) Procédé de traitement d'image et dispositif électronique le prenant en charge
KR20210138477A (ko) 온라인 시험을 제공하는 전자 장치 및 그 방법
WO2023113196A1 (fr) Dispositif et procédé pour fournir un procédé d'analyse intelligente d'image
WO2019225875A1 (fr) Procédé et appareil de suivi d'inventaire
WO2023277380A1 (fr) Procédé de construction d'une interface utilisateur sur la base d'un champ d'entrée, et dispositif électronique
WO2022124829A1 (fr) Dispositif électronique de fourniture d'interaction basée sur voix d'utilisateur et son procédé
WO2022181869A1 (fr) Dispositif et procédé mettant en œuvre un modèle d'apprentissage automatique partagé parmi de multiples applications
WO2020130734A1 (fr) Dispositif électronique permettant la fourniture d'une réaction en fonction d'un état d'utilisateur et procédé de fonctionnement correspondant
KR20230091736A (ko) 지능형 영상 분석 방법을 제공하기 위한 장치 및 방법
WO2023101159A1 (fr) Dispositif et procédé de fourniture de contenu audiovisuel pour personne handicapée
WO2021235798A1 (fr) Dispositif électronique et procédé de réalisation d'une authentification utilisateur à l'aide d'une entrée sur le clavier dans le dispositif électronique
WO2023054913A1 (fr) Dispositif électronique qui identifie une force tactile et son procédé de fonctionnement
WO2022154417A1 (fr) Dispositif électronique pour identifier un état émotionnel, et son procédé de fonctionnement
WO2023090747A1 (fr) Dispositif électronique d'acquisition d'image à un instant prévu par l'utilisateur et son procédé de commande
WO2022191418A1 (fr) Dispositif électronique et procédé de déplacement d'une section de lecture de contenu multimédia
WO2022158932A1 (fr) Procédé de fourniture de contenu multimédia personnalisé et dispositif électronique l'utilisant
WO2024048955A1 (fr) Dispositif électronique pour photographier une image animée, son procédé de fonctionnement, et support de stockage
WO2023059000A1 (fr) Procédé et dispositif d'aide à l'apprentissage

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22907667

Country of ref document: EP

Kind code of ref document: A1