WO2019126908A1 - Image data processing method, device and equipment - Google Patents

Image data processing method, device and equipment Download PDF

Info

Publication number
WO2019126908A1
WO2019126908A1 PCT/CN2017/118174 CN2017118174W WO2019126908A1 WO 2019126908 A1 WO2019126908 A1 WO 2019126908A1 CN 2017118174 W CN2017118174 W CN 2017118174W WO 2019126908 A1 WO2019126908 A1 WO 2019126908A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
target object
training
recognition model
feature
Prior art date
Application number
PCT/CN2017/118174
Other languages
French (fr)
Chinese (zh)
Inventor
张李亮
李思晋
封旭阳
赵丛
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2017/118174 priority Critical patent/WO2019126908A1/en
Priority to CN201780005969.XA priority patent/CN108701214A/en
Publication of WO2019126908A1 publication Critical patent/WO2019126908A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics

Definitions

  • the present invention relates to the field of electronic technologies, and in particular, to an image processing method, apparatus, and device.
  • the preset recognition model is configured to identify a first image of the target object and a designated area of the second image of the target object.
  • an embodiment of the present invention provides an image processing apparatus, where the apparatus includes:
  • a determining module configured to determine state information of the target object according to the action feature description information
  • the first image (the first image includes the RGB image or the gray image) and the depth image collected by the first image sensor and the second image sensor can be used as the signal input of the preset recognition model, which can be realized.
  • the first image data is complementary to the depth image data, and the recognition model is optimized in combination with the depth map based on the RGB image, the gray image, and the like, thereby improving driver fatigue detection in a designated area such as a cab. The accuracy of the security is improved.
  • FIG. 2 is a schematic structural diagram of another image data processing system according to an embodiment of the present invention.
  • the embodiment of the present invention is applied to an image processing apparatus, where the image processing apparatus includes a first image sensor and a second image sensor, the first image sensor may be a monocular vision sensor, and the second image sensor may be a multi-eye vision sensor.
  • the first image sensor and the second image sensor may be disposed in a camera of the image processing apparatus, for example, a monocular vision sensor is disposed in the monocular camera, and the multi-view visual sensor is disposed in the multi-view camera.
  • FIG. 1 is a schematic flowchart of an image data processing method according to an embodiment of the present invention.
  • the method is applicable to an image processing apparatus, where the image processing apparatus includes a first image sensor and a second image sensor.
  • the image data method described in the example includes:
  • the image processing apparatus may receive the first image of the target object collected by the first image sensor and the second image of the target object collected by the second image sensor, so that the first image and the second image may be The image is used as an input to the signal.
  • the image processing apparatus may detect the light in the current scene, and the light of the current scene does not satisfy the preset light intensity, and the image processing apparatus may turn on the fill light to call the monocular vision sensor (ie, An image sensor is configured to acquire image data of the target object, and the image data of the acquired target object is used as an input of a preset recognition model.
  • the monocular vision sensor ie, An image sensor is configured to acquire image data of the target object, and the image data of the acquired target object is used as an input of a preset recognition model.
  • the image processing apparatus can improve the image quality by turning on the fill light. That is to say, the image processing device can detect the light in the current scene, the light of the current scene does not satisfy the preset light intensity, and the image processing device can determine that the light in the current scene is weak, can open the fill light, and call the monocular.
  • the visual sensor ie, the first image sensor
  • S102 Input the first image of the target object and the second image of the target object into a preset recognition model, and obtain description information for describing an action feature of the designated area of the target object.
  • the preset recognition model is used to identify a first image of the target object and a designated area of the second image of the target object, and the preset recognition model may refer to a neural network recognition model.
  • the image processing apparatus may input the first image of the target object and the second image of the target object into a preset recognition model, where the preset recognition model is used to initialize the first image. Identifying, identifying the target object in the first image, the preset recognition model is further configured to perform depth recognition on the second image according to the identified target object, that is, identify the target object in the second image a region, the description information of the action feature for describing the specified region of the target object is obtained, and the first image and the second image are used as signals of the input end of the recognition model, so that the accuracy of identifying the action feature of the specified region can be improved, and at the same time By identifying only the designated area, the efficiency of obtaining the description information of the action characteristics of the specified area of the target object can be improved, and the resource consumption of the image processing apparatus can be saved.
  • the designated area of the target object may refer to an eye area, a mouth area, a nose area, and the like of the target object
  • the description information of the action feature may include description information of the closed eye feature of the eye area of the target object, or Descriptive information of the opening feature of the mouth region of the target object, or description information of the eye region, the distance feature of the mouth region and the nose region, and the like.
  • the image processing device may determine state information of the target object according to the action feature description information, where the state information may be used to indicate whether the target object is in a fatigue state, and may detect whether the target object is in fatigue through image data processing. State can improve the efficiency of fatigue detection.
  • the action feature description information includes: description information of the mouth area of the target object in an open feature
  • the specific manner of determining the state information of the target object by the action feature description information includes: according to the description information of the mouth region of the target object obtained in the preset time interval, the mouth region of the target object is in Zhang The number of times the feature is opened, if the number of times the mouth region of the target object is in the open feature is greater than the first preset value, determining state information indicating that the target object is in the specified state.
  • the preset time interval is 1 minute
  • the second preset threshold is 4 times
  • the image processing apparatus obtains the mouth of the target object according to the multi-frame first image and the second image in the preset time interval.
  • the part area is in the description information of the open feature, and the number of times the mouth area of the target object is in the open feature is counted. If the mouth area of the target object is in the open feature number of 5 times, the image processing apparatus can determine the The number of times the mouth region of the target object is in the open feature is greater than the second pre-threshold, and status information indicating that the target object is in a fatigue state is determined.
  • the image processing device may determine whether the target object is in a fatigue state according to the facial motion feature of the target object.
  • the image processing apparatus may count, according to the description information of the open feature of the mouth region of the target object obtained in the preset time interval, the number of times the mouth region of the target object is in the open feature, if the target object And determining, by the image processing device, the state object of the target object The mouth area is in the manner of opening the feature number, and it is judged whether the target object is in a fatigue state, and the accuracy of detecting the fatigue state can be improved.
  • the target object in order to prevent the target object from being in a speaking state, it is erroneously determined that the target object is in a specified state (ie, the specified state refers to a fatigue state), and thus the mouth region is in an open feature and may refer to the target object.
  • the distance between the upper lip and the lower lip is greater than a preset distance threshold to improve the accuracy of the image processing device in detecting the fatigue state.
  • the specified area of the target object includes: an eye area of the target object;
  • the motion feature description information includes: description information of the eye area of the target object in a closed eye feature;
  • the specific manner of determining the state information of the target object by the action feature description information includes: according to the description information of the eye region of the target object obtained in the preset time interval, the eye region of the target object is in the closed eye The number of times of the feature, if the number of times the eye region of the target object is in the closed eye feature is greater than the second pre-threshold, determining state information indicating that the target object is in the designated state.
  • the preset time interval is 1 minute
  • the second preset threshold is 5 times
  • the image processing apparatus obtains the eye of the target object according to the multi-frame first image and the second image in the preset time interval.
  • the part area is in the description information of the closed-eye feature, and the number of times the eye area of the target object is in the closed-eye feature is counted. If the number of times the eye area of the target object is in the closed-eye feature is 6 times, the image processing apparatus can determine the The number of times the eye region of the target object is in the closed eye feature is greater than the second pre-threshold, and status information indicating that the target object is in a fatigue state is determined.
  • the image processing apparatus may count, according to the description information of the closed eye feature of the eye region of the target object obtained in the preset time interval, the number of times the eye region of the target object is in the closed eye feature, if If the number of times the eye region of the target object is in the closed eye feature is greater than the second pre-threshold value, determining state information indicating that the target object is in the specified state (the designated state may refer to the fatigue state), and the image processing device counts the target object by counting The manner in which the eye region is in the degree of closing the eye feature determines whether the target object is in a fatigue state, and the accuracy of detecting the fatigue state can be improved.
  • the image processing device may receive the first image captured by the first image sensor and the second image of the target object collected by the second image sensor, and the first image and the second image of the target object.
  • the image is input into a preset recognition model, and the description information of the action feature for describing the specified area of the target object is obtained, and the state information of the target object is determined according to the description information of the action feature, and is collected by using multiple image sensors.
  • the recognition model the image data can be complemented by various signals, thereby providing sufficient information amount for the input end of the preset recognition model, and combining the depth map on the basis of the gray image or the RGB image.
  • the recognition model is optimized to improve the accuracy of fatigue detection.
  • the image data processing system includes an image processing device 201, a vehicle 202, and a driving position located in the vehicle 201.
  • the target object 203 ie, the target object is also the driver
  • the image processing apparatus 201 may include a plurality of sensors (the first image sensor 2011 and the second image sensor 2012 are taken as an example), and the image processing apparatus 201 and
  • the image processing device 201 may be disposed on the roof of the vehicle 202 near the driving position, or may be disposed on the console of the vehicle 202 so that the image data of the target object can be clearly collected.
  • the image data processing system can be used to implement an image data processing method. Specifically, refer to FIG. 3, which is an image data processing method according to an embodiment of the present invention.
  • the image data processing method includes:
  • the image processing apparatus 201 may search for an identification model associated with the object identifier of the target object, and use the associated recognition model as a preset recognition model, so as to adopt the recognition model associated with the object identifier to the target object.
  • the image is identified to improve the accuracy of the recognition.
  • the first image and the second image of the training object are collected, and the first image of the training object and the designated area of the second image are trained by using an initial recognition model to obtain the trained recognition model.
  • the first image sensor is called to acquire a training first image of the target object, and the second image sensor is called to collect the target.
  • the second image of the object is trained, and the preset recognition model is trained according to the training first image and the training second image.
  • the image processing device 201 when the image processing device 201 detects that the recognition accuracy of the preset recognition model is low, or receives a training instruction for the preset recognition model, the image processing device 201 may invoke the first image sensor 2011. Collecting a training first image of the target object, and calling the second image sensor 2012 to collect a training second image of the target object, and training the preset recognition model according to the training first image and the training second image, The recognition accuracy of the preset recognition model is improved, thereby improving the accuracy of identifying fatigue driving.
  • the image processing apparatus 201 determines that the target object is in a fatigue driving state according to the state information of the target object, it may be determined that the driving of the vehicle 202 needs to be suspended, and the image processing apparatus 201 may output prompt information to prompt the target object. Suspension of driving the vehicle can improve the safety of driving the vehicle.
  • the image processing apparatus 201 determines that the target object is in a fatigue driving state according to the state information of the target object, it may be determined that the automatic driving mode of the vehicle 202 needs to be activated, and then controlling the vehicle to start the automatic driving mode may prevent Traffic accidents caused by fatigue driving can improve the safety of driving.
  • the second image sensor 402 is configured to collect a second image of the target object.
  • An identification module 404 configured to input a first image of the target object and a second image of the target object into a preset recognition model, to obtain description information for describing an action feature of a specified area of the target object .
  • the first training module 409 is configured to train the preset recognition model according to the training first image and the training second image.
  • the first image sensor 401 is further configured to collect a first image of the training object.
  • the processor 501 calls the program code stored in the memory 502, and may also perform the following operations:

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)

Abstract

Provided are an image data processing method, a device and equipment. The method comprises steps of: receiving a first image of a target object acquired by a first image sensor, and a second image of the target object acquired by a second image sensor; inputting the first image and the second image of the target object to a preset identification model so as to obtain description information used for describing action characteristics within a designated area of the target object; and according to the description information of the action characteristics, determining state information of the target object. Thus, accuracy of fatigue detection can be improved.

Description

图像数据处理方法、装置及设备Image data processing method, device and device 技术领域Technical field
本发明涉及电子技术领域，尤其涉及图像处理方法、装置及设备。The present invention relates to the field of electronic technologies, and in particular, to an image processing method, apparatus, and device.
背景技术Background technique
随着交通技术的发展及人们生活水平的提高，驾车出行方式以其特有的优越性已经成为大多数人们出行的最佳选择，给人们的出行带来了便利性和舒适度。但是，由于疲劳驾驶引起的交通事故，给人们的生命安全及财产造成巨大的影响。With the development of transportation technology and the improvement of people's living standards, driving mode has become the best choice for most people with its unique superiority, which brings convenience and comfort to people's travel. However, traffic accidents caused by fatigue driving have a huge impact on people's lives and property.
实际应用中，通过检测车辆是否压到道路上的交通标志线，来检测驾驶员是否处于疲劳驾驶状态，但是，如果驾驶员的驾驶技术水平不高，也可能压倒道路上的交通标志线，从而导致将该驾驶员误判为疲劳驾驶，可见，上述检测疲劳驾驶的方式的准确度较低。In practical applications, it is detected whether the driver is in a fatigue driving state by detecting whether the vehicle is pressed to the traffic sign line on the road. However, if the driving skill level of the driver is not high, the traffic sign line on the road may be overwhelmed. As a result, the driver is mistakenly judged as fatigue driving, and it can be seen that the above-described method of detecting fatigue driving is low in accuracy.
发明内容Summary of the invention
本发明实施例公开了一种图像数据处理方法、装置及设备，可通过对驾驶员等目标对象的图像数据处理提高检测疲劳驾驶的准确度。The embodiment of the invention discloses an image data processing method, device and device, which can improve the accuracy of detecting fatigue driving by processing image data of a target object such as a driver.
第一方面，本发明实施例提供了一种图像数据处理方法，该方法包括：In a first aspect, an embodiment of the present invention provides an image data processing method, where the method includes:
接收所述第一图像传感器采集到的目标对象的第一图像，及所述第二图像传感器采集到的所述目标对象的第二图像，所述第一图像包括灰度图像或RGB图像中的至少一种，所述第二图像包括深度图像；Receiving a first image of the target object collected by the first image sensor and a second image of the target object collected by the second image sensor, where the first image includes a grayscale image or an RGB image At least one of the second images comprising a depth image;
将所述目标对象的第一图像和所述目标对象的第二图像输入到预设的识别模型中，得到用于描述所述目标对象的指定区域的动作特征的描述信息；Inputting a first image of the target object and a second image of the target object into a preset recognition model to obtain description information for describing an action feature of a specified region of the target object;
根据所述动作特征描述信息确定所述目标对象的状态信息；Determining state information of the target object according to the action feature description information;
所述预设的识别模型用于对所述目标对象的第一图像和所述目标对象的第二图像的指定区域进行识别。The preset recognition model is configured to identify a first image of the target object and a designated area of the second image of the target object.
第二方面，本发明实施例提供了一种图像处理装置，该装置包括：In a second aspect, an embodiment of the present invention provides an image processing apparatus, where the apparatus includes:
接收模块，用于接收所述第一图像传感器采集到的目标对象的第一图像， 及所述第二图像传感器采集到的所述目标对象的第二图像，所述第一图像包括灰度图像或RGB图像中的至少一种，所述第二图像包括深度图像；a receiving module, configured to receive a first image of the target object acquired by the first image sensor, and a second image of the target object collected by the second image sensor, where the first image includes a grayscale image Or at least one of RGB images, the second image comprising a depth image;
识别模块，用于将所述目标对象的第一图像和所述目标对象的第二图像输入到预设的识别模型中，得到用于描述所述目标对象的指定区域的动作特征的描述信息；An identification module, configured to input a first image of the target object and a second image of the target object into a preset recognition model, to obtain description information for describing an action feature of a specified area of the target object;
确定模块，用于根据所述动作特征描述信息确定所述目标对象的状态信息；a determining module, configured to determine state information of the target object according to the action feature description information;
所述预设的识别模型用于对所述目标对象的第一图像和所述目标对象的第二图像的指定区域进行识别。The preset recognition model is configured to identify a first image of the target object and a designated area of the second image of the target object.
第三方面，本发明实施例提供了一种图像处理设备，该设备包括：处理器和存储器，所述处理器和所述存储器通过总线连接，所述存储器存储有可执行程序代码，所述处理器用于调用所述可执行程序代码，执行本发明实施例第一方面所述的图像数据处理方法。In a third aspect, an embodiment of the present invention provides an image processing apparatus, including: a processor and a memory, wherein the processor and the memory are connected by a bus, and the memory stores executable program code, and the processing The apparatus is configured to invoke the executable program code to execute the image data processing method according to the first aspect of the embodiments of the present invention.
第四方面，本发明实施例提供了一种计算机可读存储介质，其上存储有计算机程序，该计算机程序被至少一个处理器执行时，可以实现上述第一方面所述的图像数据处理方法。In a fourth aspect, an embodiment of the present invention provides a computer readable storage medium, where a computer program is stored thereon, and when the computer program is executed by at least one processor, the image data processing method described in the first aspect may be implemented.
第五方面，本发明实施例提供了一种计算机程序产品，该计算机程序产品包括存储了计算机程序的非瞬时性计算机可读存储介质，该计算机程序可操作来使计算机实现上述第一方面所述的图像数据处理方法。In a fifth aspect, an embodiment of the present invention provides a computer program product, comprising: a non-transitory computer readable storage medium storing a computer program, the computer program being operative to cause a computer to implement the first aspect described above Image data processing method.
通过本发明实施例可以根据以第一图像传感器及第二图像传感器采集到的第一图像(第一图像包括RGB图像或灰度图像)及深度图像作为预设的识别模型的信号输入，可以实现第一图像数据与深度图像数据的互补，并且在RGB图像、灰度图像等图像的基础上结合深度图对识别模型进行了优化处理，进而提高对诸如驾驶室内等指定区域内的驾驶员疲劳检测的准确度，提高了安全性。According to the embodiment of the present invention, the first image (the first image includes the RGB image or the gray image) and the depth image collected by the first image sensor and the second image sensor can be used as the signal input of the preset recognition model, which can be realized. The first image data is complementary to the depth image data, and the recognition model is optimized in combination with the depth map based on the RGB image, the gray image, and the like, thereby improving driver fatigue detection in a designated area such as a cab. The accuracy of the security is improved.
附图说明DRAWINGS
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一 些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings to be used in the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without paying for creative labor.
图1是本发明实施例公开的一种图像数据处理方法的流程示意图；1 is a schematic flow chart of an image data processing method according to an embodiment of the present invention;
图2是本发明实施例公开的另一种图像数据处理系统的结构示意图；2 is a schematic structural diagram of another image data processing system according to an embodiment of the present invention;
图3是本发明实施例公开的另一种图像数据处理方法的流程示意图；3 is a schematic flowchart diagram of another image data processing method according to an embodiment of the present invention;
图4是本发明实施例公开的一种图像处理装置的结构示意图；4 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention;
图5是本发明实施例公开的一种图像处理设备的结构示意图。FIG. 5 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention.
具体实施方式Detailed ways
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.
本发明实施例应用于图像处理装置，该图像处理装置包括第一图像传感器及第二图像传感器，该第一图像传感器可以是指单目视觉传感器，第二图像传感器可以是指多目视觉传感器，第一图像传感器及第二图像传感器可以设置于图像处理装置的摄像头中，如，将单目视觉传感器设置于单目摄像头中，将多目视觉传感器设置于多目摄像头中。The embodiment of the present invention is applied to an image processing apparatus, where the image processing apparatus includes a first image sensor and a second image sensor, the first image sensor may be a monocular vision sensor, and the second image sensor may be a multi-eye vision sensor. The first image sensor and the second image sensor may be disposed in a camera of the image processing apparatus, for example, a monocular vision sensor is disposed in the monocular camera, and the multi-view visual sensor is disposed in the multi-view camera.
本发明实施例中的图像处理装置可以与车辆连接，并可以设置在车辆中，该图像处理装置的第一图像传感器及第二传感器可以随着驾驶位上目标对象的姿势变化动态调整采集图像的角度，以便可以清晰地采集到驾驶位上的目标对象的图像。The image processing apparatus in the embodiment of the present invention may be connected to the vehicle and may be disposed in the vehicle, and the first image sensor and the second sensor of the image processing apparatus may dynamically adjust the collected image according to the posture change of the target object on the driving position. Angle so that the image of the target object on the driver's seat can be clearly captured.
本发明实施例可以应用于检测目标对象(该目标对象可以是指用户)是否处于疲劳状态，更具体地，可以应用于检测驾驶员是否是疲劳驾驶。The embodiment of the present invention can be applied to detect whether a target object (which may refer to a user) is in a fatigue state, and more specifically, can be applied to detect whether the driver is fatigue driving.
本发明实施例中的第一图像包括灰度图像或RGB图像中的至少一种，第二图像包括深度图像。The first image in the embodiment of the present invention includes at least one of a grayscale image or an RGB image, and the second image includes a depth image.
基于当前的疲劳驾驶检测方法的准确度较低的问题，本发明提出一种图像数据处理方法、装置及设备，图像处理装置可以接收第一图像传感器采集到的目标对象的第一图像，即RGB(Red Green Blue)图像为具备红、绿、蓝颜色 的彩色图像，及第二图像传感器采集到目标对象的第二图像，将该目标对象的第一图像及第二图像输入到预设的识别模型中，得到用于描述该目标对象的指定区域的动作特征的描述信息，根据该动作特征描述信息确定该目标对象的状态信息，该目标对象的状态信息用于指示该目标对像是否处于疲劳状态。本发明以多种传感器采集的目标对象的图像数据作为信号的输入，可以实现多种信号的互补，从而为预设的识别模型的输入端提供足够多的信息量，以便可以提高疲劳检测的准确度。The present invention provides an image data processing method, apparatus and device. The image processing device can receive a first image of a target object collected by the first image sensor, that is, RGB, based on the problem that the accuracy of the current fatigue driving detection method is low. (Red Green Blue) image is a color image with red, green, and blue colors, and the second image sensor collects a second image of the target object, and inputs the first image and the second image of the target object to a preset recognition. In the model, the description information of the action feature for describing the specified area of the target object is obtained, and the state information of the target object is determined according to the action feature description information, and the state information of the target object is used to indicate whether the target object is in fatigue status. The invention uses the image data of the target object collected by various sensors as the input of the signal, can realize the complementation of various signals, thereby providing sufficient information amount for the input end of the preset recognition model, so as to improve the accuracy of the fatigue detection. degree.
本发明实施例公开了一种图像数据处理方法、装置及设备，用于基于图像数据处理方式检测目标对象是否是疲劳状态，以提高疲劳检测的准确度，以下分别进行详细说明。The embodiment of the invention discloses an image data processing method, device and device for detecting whether a target object is in a fatigue state based on an image data processing manner, so as to improve the accuracy of fatigue detection, which are respectively described in detail below.
请参阅图1，图1为本发明实施例提供的一种图像数据处理方法的流程示意图，该方法可应用于图像处理装置，该图像处理装置包括第一图像传感器及第二图像传感器，本实施例中所描述的图像数据方法，包括：Referring to FIG. 1 , FIG. 1 is a schematic flowchart of an image data processing method according to an embodiment of the present invention. The method is applicable to an image processing apparatus, where the image processing apparatus includes a first image sensor and a second image sensor. The image data method described in the example includes:
S101、接收该第一图像传感器采集到的目标对象的第一图像，及该第二图像传感器采集到的该目标对象的第二图像。S101. Receive a first image of the target object collected by the first image sensor, and a second image of the target object collected by the second image sensor.
其中，该第一图像包括灰度图像或RGB图像中的至少一种，该第二图像包括深度图像。Wherein the first image comprises at least one of a grayscale image or an RGB image, the second image comprising a depth image.
本发明实施例中，如果采用单目视觉传感器采集的图像数据作为信号的输入，在环境光线不足的情况下，单目视觉传感器采集到的图像数据的质量大大降低，从而使得图像处理装置难以从图像数据中获取到需要的信息；如果采用红外传感器采集的图像数据作为信号的输入，由于红外传感器难以精准地捕获到目标对象的五官，从而使得图像处理装置难以从图像数据中获取到需要的信息。也就是说，如果以单个传感器采集到的图像数据作为信号的输入，难以保证为预设的识别模型的输入端提供足够的信息量，因此，图像处理装置可以采用多种图像传感器采集到的图像数据作为信号的输入，可以实现多种信号的互补，从而为预设的识别模型的输入端提供足够多的信息量。In the embodiment of the present invention, if the image data collected by the monocular vision sensor is used as the input of the signal, the quality of the image data collected by the monocular vision sensor is greatly reduced in the case of insufficient ambient light, thereby making the image processing apparatus difficult to Obtaining the required information in the image data; if the image data acquired by the infrared sensor is used as the input of the signal, it is difficult for the infrared sensor to accurately capture the facial features of the target object, thereby making it difficult for the image processing apparatus to obtain the required information from the image data. . That is to say, if the image data collected by a single sensor is used as the input of the signal, it is difficult to ensure that a sufficient amount of information is provided for the input end of the preset recognition model. Therefore, the image processing apparatus can adopt images acquired by various image sensors. As an input to the signal, data can be complemented by multiple signals to provide enough information for the input of the preset recognition model.
具体的，图像处理装置可以接收该第一图像传感器采集到的目标对象的第一图像，及该第二图像传感器采集到的该目标对象的第二图像，以便可以将该 第一图像及第二图像作为信号的输入。Specifically, the image processing apparatus may receive the first image of the target object collected by the first image sensor and the second image of the target object collected by the second image sensor, so that the first image and the second image may be The image is used as an input to the signal.
作为一种可选的实施方式，图像处理装置可以检测当前场景下的光线，当前的场景的光线不满足预设的光线强度，图像处理装置可以打开补光灯，调用单目视觉传感器(即第一图像传感器)，以采集目标对象的图像数据，将采集到的目标对象的图像数据作为预设的识别模型的输入。As an optional implementation manner, the image processing apparatus may detect the light in the current scene, and the light of the current scene does not satisfy the preset light intensity, and the image processing apparatus may turn on the fill light to call the monocular vision sensor (ie, An image sensor is configured to acquire image data of the target object, and the image data of the acquired target object is used as an input of a preset recognition model.
本发明实施例中，为了解决单目视觉传感器在光线较弱的场景下，成像质量较低的问题，图像处理装置可以通过打开补光灯来改善图像的质量。也就是说，图像处理装置可以检测当前场景下的光线，当前的场景的光线不满足预设的光线强度，图像处理装置可以确定当前场景下的光线较弱，可以打开补光灯，调用单目视觉传感器(即第一图像传感器)，以采集目标对象的图像数据，将采集到的目标对象的图像数据作为预设的识别模型的输入，以提高采集图像的质量。In the embodiment of the present invention, in order to solve the problem that the monocular vision sensor has low imaging quality in a scene with weak light, the image processing apparatus can improve the image quality by turning on the fill light. That is to say, the image processing device can detect the light in the current scene, the light of the current scene does not satisfy the preset light intensity, and the image processing device can determine that the light in the current scene is weak, can open the fill light, and call the monocular. The visual sensor (ie, the first image sensor) collects the image data of the target object, and uses the image data of the acquired target object as an input of the preset recognition model to improve the quality of the captured image.
S102、将该目标对象的第一图像和该目标对象的第二图像输入到预设的识别模型中，得到用于描述该目标对象的指定区域的动作特征的描述信息。S102. Input the first image of the target object and the second image of the target object into a preset recognition model, and obtain description information for describing an action feature of the designated area of the target object.
其中，该预设的识别模型用于对该目标对象的第一图像和该目标对象的第二图像的指定区域进行识别，该预设的识别模型可以是指神经网络识别模型。The preset recognition model is used to identify a first image of the target object and a designated area of the second image of the target object, and the preset recognition model may refer to a neural network recognition model.
本发明实施例中，图像处理装置可以将该目标对象的第一图像和该目标对象的第二图像输入到预设的识别模型中，该预设的识别模型用于对该第一图像进行初始识别，识别出该第一图像中的目标对象，该预设的识别模型还用于根据识别出的目标对象对该第二图像进行深度识别，即识别出该第二图像中的目标对象的指定区域，得到用于描述该目标对象的指定区域的动作特征的描述信息，采用第一图像及第二图像作为该识别模型的输入端的信号，可以提高识别出指定区域的动作特征的准确度，同时仅对指定区域进行识别，可以提高获取该目标对象的指定区域的动作特性的描述信息的效率，可以节省图像处理设备的资源消耗。In the embodiment of the present invention, the image processing apparatus may input the first image of the target object and the second image of the target object into a preset recognition model, where the preset recognition model is used to initialize the first image. Identifying, identifying the target object in the first image, the preset recognition model is further configured to perform depth recognition on the second image according to the identified target object, that is, identify the target object in the second image a region, the description information of the action feature for describing the specified region of the target object is obtained, and the first image and the second image are used as signals of the input end of the recognition model, so that the accuracy of identifying the action feature of the specified region can be improved, and at the same time By identifying only the designated area, the efficiency of obtaining the description information of the action characteristics of the specified area of the target object can be improved, and the resource consumption of the image processing apparatus can be saved.
其中，该目标对象的指定区域可以是指该目标对象的眼部区域、嘴部区域、鼻子区域等，动作特征的描述信息可以包括该目标对象的眼部区域的闭眼特征的描述信息，或该目标对象的嘴部区域的张开特征的描述信息，或眼部区域、嘴部区域与鼻子区域的距离特征的描述信息等。The designated area of the target object may refer to an eye area, a mouth area, a nose area, and the like of the target object, and the description information of the action feature may include description information of the closed eye feature of the eye area of the target object, or Descriptive information of the opening feature of the mouth region of the target object, or description information of the eye region, the distance feature of the mouth region and the nose region, and the like.
S103、根据该动作特征描述信息确定该目标对象的状态信息。S103. Determine status information of the target object according to the action feature description information.
本发明实施例中，图像处理装置可以根据该动作特征描述信息确定该目标对象的状态信息，该状态信息可以用于指示该目标对象是否处于疲劳状态，可以通过图像数据处理检测目标对象是否处于疲劳状态，可以提高疲劳检测的效率。In the embodiment of the present invention, the image processing device may determine state information of the target object according to the action feature description information, where the state information may be used to indicate whether the target object is in a fatigue state, and may detect whether the target object is in fatigue through image data processing. State can improve the efficiency of fatigue detection.
作为一种可选的实施方式，若该目标对象的指定区域包括：该目标对象的嘴部区域；该动作特征描述信息包括：该目标对象的嘴部区域处于张开特征的描述信息；上述根据该动作特征描述信息确定该目标对象的状态信息的具体方式包括：根据预设时间间隔内得到的该目标对象的嘴部区域处于张开特征的描述信息，统计该目标对象的嘴部区域处于张开特征的次数，若该目标对象的嘴部区域处于张开特征的次数大于第一预设值，则确定指示所述目标对象处于指定状态的状态信息。As an optional implementation manner, if the designated area of the target object includes: a mouth area of the target object; the action feature description information includes: description information of the mouth area of the target object in an open feature; The specific manner of determining the state information of the target object by the action feature description information includes: according to the description information of the mouth region of the target object obtained in the preset time interval, the mouth region of the target object is in Zhang The number of times the feature is opened, if the number of times the mouth region of the target object is in the open feature is greater than the first preset value, determining state information indicating that the target object is in the specified state.
举例来说，该预设时间间隔为1分钟，该第二预设阈值为4次，图像处理装置根据预设时间间隔内的多帧第一图像及第二图像，得到的该目标对象的嘴部区域处于张开特征的描述信息，统计该目标对象的嘴部区域处于张开特征的次数，若该目标对象的嘴部区域处于张开特征的次数为5次，则图像处理装置可以确定该目标对象的嘴部区域处于张开特征的次数大于第二预阈值，并确定指示该目标对象处于疲劳状态的状态信息。For example, the preset time interval is 1 minute, and the second preset threshold is 4 times, and the image processing apparatus obtains the mouth of the target object according to the multi-frame first image and the second image in the preset time interval. The part area is in the description information of the open feature, and the number of times the mouth area of the target object is in the open feature is counted. If the mouth area of the target object is in the open feature number of 5 times, the image processing apparatus can determine the The number of times the mouth region of the target object is in the open feature is greater than the second pre-threshold, and status information indicating that the target object is in a fatigue state is determined.
本发明实施例中，由于目标对象处于疲劳状态时，该目标对象的面部会表现出不同的动作特征，因此图像处理装置可以根据该目标对象的面部动作特征，判断该目标对象是否处于疲劳状态。也就是说，图像处理装置可以根据预设时间间隔内得到的该目标对象的嘴部区域处于张开特征的描述信息，统计该目标对象的嘴部区域处于张开特征的次数，若该目标对象的嘴部区域处于张开特征的次数大于第一预设阈值，则确定指示所述目标对象处于指定状态(该指定状态可以是指疲劳状态)的状态信息，图像处理装置通过统计该目标对象的嘴部区域处于张开特征的次数的方式，判断该目标对象是否处于疲劳状态，可以提高检测疲劳状态的准确度。In the embodiment of the present invention, when the target object is in a fatigue state, the face of the target object may exhibit different motion features. Therefore, the image processing device may determine whether the target object is in a fatigue state according to the facial motion feature of the target object. In other words, the image processing apparatus may count, according to the description information of the open feature of the mouth region of the target object obtained in the preset time interval, the number of times the mouth region of the target object is in the open feature, if the target object And determining, by the image processing device, the state object of the target object The mouth area is in the manner of opening the feature number, and it is judged whether the target object is in a fatigue state, and the accuracy of detecting the fatigue state can be improved.
需要说明的是，为了防止将该目标对象处于说话状态，误判为该目标对象处于指定状态(即该指定状态是指疲劳状态)，因此上述嘴部区域处于张开特 征可以是指该目标对象的上嘴唇与下嘴唇的距离大于预设的距离阈值，以提高图像处理装置检测疲劳状态的准确度。It should be noted that, in order to prevent the target object from being in a speaking state, it is erroneously determined that the target object is in a specified state (ie, the specified state refers to a fatigue state), and thus the mouth region is in an open feature and may refer to the target object. The distance between the upper lip and the lower lip is greater than a preset distance threshold to improve the accuracy of the image processing device in detecting the fatigue state.
作为一种可选的实施方式，该目标对象的指定区域包括：该目标对象的眼部区域；该动作特征描述信息包括：该目标对象的眼部区域处于闭眼特征的描述信息；上述根据该动作特征描述信息确定该目标对象的状态信息的具体方式包括：根据预设时间间隔内得到的该目标对象的眼部区域处于闭眼特征的描述信息，统计该目标对象的眼部区域处于闭眼特征的次数，若该目标对象的眼部区域处于闭眼特征的次数大于第二预阈值，则确定指示该目标对象处于指定状态的状态信息。In an optional implementation manner, the specified area of the target object includes: an eye area of the target object; the motion feature description information includes: description information of the eye area of the target object in a closed eye feature; The specific manner of determining the state information of the target object by the action feature description information includes: according to the description information of the eye region of the target object obtained in the preset time interval, the eye region of the target object is in the closed eye The number of times of the feature, if the number of times the eye region of the target object is in the closed eye feature is greater than the second pre-threshold, determining state information indicating that the target object is in the designated state.
举例来说，该预设时间间隔为1分钟，该第二预设阈值为5次，图像处理装置根据预设时间间隔内的多帧第一图像及第二图像，得到的该目标对象的眼部区域处于闭眼特征的描述信息，统计该目标对象的眼部区域处于闭眼特征的次数，若该目标对象的眼部区域处于闭眼特征的次数为6次，则图像处理装置可以确定该目标对象的眼部区域处于闭眼特征的次数大于第二预阈值，并确定指示该目标对象处于疲劳状态的状态信息。For example, the preset time interval is 1 minute, and the second preset threshold is 5 times, and the image processing apparatus obtains the eye of the target object according to the multi-frame first image and the second image in the preset time interval. The part area is in the description information of the closed-eye feature, and the number of times the eye area of the target object is in the closed-eye feature is counted. If the number of times the eye area of the target object is in the closed-eye feature is 6 times, the image processing apparatus can determine the The number of times the eye region of the target object is in the closed eye feature is greater than the second pre-threshold, and status information indicating that the target object is in a fatigue state is determined.
本发明实施例中，图像处理装置可以根据预设时间间隔内得到的该目标对象的眼部区域处于闭眼特征的描述信息，统计该目标对象的眼部区域处于闭眼特征的次数，若该目标对象的眼部区域处于闭眼特征的次数大于第二预阈值，则确定指示该目标对象处于指定状态(该指定状态可以是指疲劳状态)的状态信息，图像处理装置通过统计该目标对象的眼部区域处于闭眼特征的次数的方式，判断该目标对象是否处于疲劳状态，可以提高检测疲劳状态的准确度。In the embodiment of the present invention, the image processing apparatus may count, according to the description information of the closed eye feature of the eye region of the target object obtained in the preset time interval, the number of times the eye region of the target object is in the closed eye feature, if If the number of times the eye region of the target object is in the closed eye feature is greater than the second pre-threshold value, determining state information indicating that the target object is in the specified state (the designated state may refer to the fatigue state), and the image processing device counts the target object by counting The manner in which the eye region is in the degree of closing the eye feature determines whether the target object is in a fatigue state, and the accuracy of detecting the fatigue state can be improved.
本发明实施例中，图像处理装置可以接收第一图像传感器采集到目标对象的第一图像，及第二图像传感器采集到的目标对象的第二图像，将该目标对象的第一图像及第二图像输入到预设的识别模型中，得到用于描述该目标对象的指定区域的动作特征的描述信息，根据该动作特征的描述信息确定该目标对象的状态信息，通过以多种图像传感器采集到的图像数据作为该识别模型的信号输入，可以实现多种信号的互补，从而为预设的识别模型的输入端提供足够多的信息量，并且在灰度图像或RGB图像的基础上结合深度图对识别模型进行了优化处理，进而提高疲劳检测的准确度。In the embodiment of the present invention, the image processing device may receive the first image captured by the first image sensor and the second image of the target object collected by the second image sensor, and the first image and the second image of the target object. The image is input into a preset recognition model, and the description information of the action feature for describing the specified area of the target object is obtained, and the state information of the target object is determined according to the description information of the action feature, and is collected by using multiple image sensors. As the signal input of the recognition model, the image data can be complemented by various signals, thereby providing sufficient information amount for the input end of the preset recognition model, and combining the depth map on the basis of the gray image or the RGB image. The recognition model is optimized to improve the accuracy of fatigue detection.
基于上述对图像数据处理方法的描述，本发明实施例提供一种图像数据处理系统，如图2所示，该图像数据处理系统包括图像处理装置201、车辆202、及位于该车辆201的驾驶位上的目标对象203(即该目标对象也就是驾驶员)，图像处理装置201可以包括多种传感器(图中以第一图像传感器2011及第二图像传感器2012为例)，该图像处理装置201与车辆202相连，该图像处理装置201可以设置在该车辆202中的靠近驾驶位的车顶上，也可以设置在该车辆202的控制台上，以便可以清晰地采集到目标对象的图像数据，该图像数据处理系统可以用于实现一种图像数据处理方法，具体的，请参见3，图3为本发明实施例提供的一种图像数据处理方法，该图像数据处理方法包括：Based on the above description of the image data processing method, an embodiment of the present invention provides an image data processing system. As shown in FIG. 2, the image data processing system includes an image processing device 201, a vehicle 202, and a driving position located in the vehicle 201. The target object 203 (ie, the target object is also the driver), the image processing apparatus 201 may include a plurality of sensors (the first image sensor 2011 and the second image sensor 2012 are taken as an example), and the image processing apparatus 201 and The image processing device 201 may be disposed on the roof of the vehicle 202 near the driving position, or may be disposed on the console of the vehicle 202 so that the image data of the target object can be clearly collected. The image data processing system can be used to implement an image data processing method. Specifically, refer to FIG. 3, which is an image data processing method according to an embodiment of the present invention. The image data processing method includes:
S301、若检测到目标对象，则获取该目标对象的对象标识。S301. If the target object is detected, obtain an object identifier of the target object.
本发明实施例中，图像处理装置201可以采用第一图像传感器或第二图像传感器采集车辆202的驾驶位的图像，以判断该车辆的驾驶位是否有目标对象，若存在目标对象，则获取该目标对象的对象标识。In the embodiment of the present invention, the image processing device 201 may use the first image sensor or the second image sensor to collect an image of the driving position of the vehicle 202 to determine whether the driving position of the vehicle has a target object, and if the target object exists, acquire the The object ID of the target object.
其中，该目标对象的对象标识可以是指某个人的标识，如姓名；还可以是指该目标对象所在地的标识，如中国；还可以是指目标对象的性别标识，如男士或女士。The object identifier of the target object may refer to a person's identifier, such as a name; or may be an identifier of the location of the target object, such as China; or may refer to a gender identifier of the target object, such as a man or a woman.
S302、查找与该目标对象的对象标识关联的识别模型，将关联的识别模型作为该预设的识别模型。S302. Search for a recognition model associated with the object identifier of the target object, and use the associated recognition model as the preset recognition model.
本发明实施例中，图像处理装置201可以查找与该目标对象的对象标识关联的识别模型，将该关联的识别模型作为预设的识别模型，以便采用与对象标识关联的识别模型对目标对象的图像进行识别，可以提高识别的准确度。In the embodiment of the present invention, the image processing apparatus 201 may search for an identification model associated with the object identifier of the target object, and use the associated recognition model as a preset recognition model, so as to adopt the recognition model associated with the object identifier to the target object. The image is identified to improve the accuracy of the recognition.
举例来说，若该对象标识为某个人的姓名，则图像处理装置201调用与该姓名关联的识别模型，若该对象标识为目标对象的性别标识(如男士)，则图像处理装置201可以调用与该目标对象的性别关联的识别模型。For example, if the object identifier is a person's name, the image processing apparatus 201 calls the recognition model associated with the name, and if the object identifier is the gender identifier of the target object (such as a man), the image processing apparatus 201 can call A recognition model associated with the gender of the target object.
需要说明的是，图像处理装置201可以存储大量的识别模型，并从存储的识别模型中调用需要的识别模型；图像处理装置201还可以通过网络连接从网络服务器中调用需要的识别模型，以节省该图像处理装置201的内存空间。It should be noted that the image processing apparatus 201 can store a large number of recognition models and call a required recognition model from the stored recognition model; the image processing apparatus 201 can also call a required recognition model from the network server through a network connection to save The memory space of the image processing apparatus 201.
S303、接收该第一图像传感器采集到的目标对象的第一图像，及该第二图 像传感器采集到的该目标对象的第二图像。S303. Receive a first image of the target object collected by the first image sensor, and a second image of the target object collected by the second image sensor.
S304、将该目标对象的第一图像和该目标对象的第二图像输入到预设的识别模型中，得到用于描述该目标对象的指定区域的动作特征的描述信息。S304. Input the first image of the target object and the second image of the target object into a preset recognition model, and obtain description information for describing an action feature of the designated area of the target object.
作为一种可选的实施方式，采集训练对象的第一图像及第二图像，采用初始识别模型对该训练对象的第一图像及第二图像的指定区域进行训练，得到训练后的该识别模型。As an optional implementation manner, the first image and the second image of the training object are collected, and the first image of the training object and the designated area of the second image are trained by using an initial recognition model to obtain the trained recognition model. .
本发明实施例中，图像处理装置201可以采用第一图像及第二图像对识别模型进行优化，可以提高对目标对象的动作特征的识别准确度。也就是说，图像处理装置201可以采集训练对象的第一图像及第二图像，采用初始识别模型对该训练对象的第一图像及第二图像的指定区域进行训练，得到训练后的该识别模型，经过大量训练后，可以到预设的识别模型，以便提高识别图像数据的准确度。In the embodiment of the present invention, the image processing device 201 can optimize the recognition model by using the first image and the second image, and can improve the recognition accuracy of the action features of the target object. That is to say, the image processing device 201 can collect the first image and the second image of the training object, and use the initial recognition model to train the designated regions of the first image and the second image of the training object to obtain the trained recognition model. After a lot of training, you can go to the preset recognition model to improve the accuracy of identifying image data.
作为一种可选的实施方式，上述采用初始识别模型对该训练对象的第一图像及第二图像的指定区域进行训练，得到训练后的该识别模型的具体方式包括：获取该训练对象当前的训练语料，采用该初始识别模型对训练对象的第一图像及第二图像的指定区域进行识别，得到训练描述信息，确定该训练对象当前的训练语料与该训练描述信息的相似度，若该相似度小于预设相似度值，则调整该初始识别模型中的识别参数，得到训练后的该识别模型。As an optional implementation manner, the initial recognition model is used to train the first image of the training object and the designated area of the second image, and the specific manner of obtaining the trained recognition model includes: acquiring the current training object Training corpus, using the initial recognition model to identify the first image of the training object and the designated area of the second image, obtaining training description information, determining the similarity between the current training corpus of the training object and the training description information, if the similarity If the degree is less than the preset similarity value, the identification parameter in the initial recognition model is adjusted to obtain the trained recognition model.
本发明实施例中，图像处理装置201可以接收输入的该训练对象当前的训练语料，采用该初始识别模型对训练对象的第一图像及第二图像的指定区域进行识别，得到训练描述信息，确定该训练对象当前的训练语料与该训练描述信息的相似度，若该相似度小于预设相似度值，则确定该初始模型的识别精度较低，图像处理装置201可以调整该初始识别模型中的识别参数，将下一训练对象的第一图像及第二图像输入到调整后的识别模型中，重复执行上述步骤，经过大量训练后，若多次训练对象的训练语料与该训练描述信息的相似度大于预设相似度值，即将稳定性及识别精度较高的识别模型作为训练后的该识别模型。In the embodiment of the present invention, the image processing apparatus 201 may receive the input training corpus of the training object, and use the initial recognition model to identify the first image of the training object and the designated area of the second image, and obtain training description information, and determine a similarity between the current training corpus of the training object and the training description information. If the similarity is less than the preset similarity value, determining that the initial model has low recognition accuracy, the image processing apparatus 201 may adjust the initial recognition model. Identifying parameters, inputting the first image and the second image of the next training object into the adjusted recognition model, and repeatedly performing the above steps. After a large amount of training, if the training corpus of the training object is similar to the training description information The degree is greater than the preset similarity value, that is, the recognition model with high stability and recognition accuracy is used as the recognition model after training.
图像处理装置201可以采集不同地域、不同环境或不同场景下的用户的第一图像及第二图像作为上述训练对象的第一图像及第二图像，以便可以提高识 别模型的鲁棒性；图像处理装置201也可以仅采集使用该车辆202的频率较高的用户的第一图像及第二图像作为上述训练对象的第一图像及第二图像，可以降低训练的复杂度，并可以提高识别模型的利用率。The image processing device 201 can collect the first image and the second image of the user in different regions, different environments or different scenarios as the first image and the second image of the training object, so that the robustness of the recognition model can be improved; image processing The device 201 may also collect only the first image and the second image of the user with a higher frequency of the vehicle 202 as the first image and the second image of the training object, thereby reducing training complexity and improving the recognition model. Utilization rate.
作为一种可选的实施方式，如果检测到对该预设的识别模型的训练指令，调用该第一图像传感器采集所述目标对象的训练第一图像，及调用该第二图像传感器采集该目标对象的训练第二图像，根据该训练第一图像及训练第二图像对该预设的识别模型进行训练。As an optional implementation manner, if a training instruction for the preset recognition model is detected, the first image sensor is called to acquire a training first image of the target object, and the second image sensor is called to collect the target. The second image of the object is trained, and the preset recognition model is trained according to the training first image and the training second image.
本发明实施例中，图像处理装置201检测到该预设识别模型的识别精度较低，或接收到对该预设的识别模型进行训练指令时，图像处理装置201可以调用该第一图像传感器2011采集该目标对象的训练第一图像，及调用该第二图像传感器2012采集该目标对象的训练第二图像，根据该训练第一图像及训练第二图像对该预设的识别模型进行训练，以提高该预设识别模型的识别精度，进而提高识别疲劳驾驶的准确度。In the embodiment of the present invention, when the image processing device 201 detects that the recognition accuracy of the preset recognition model is low, or receives a training instruction for the preset recognition model, the image processing device 201 may invoke the first image sensor 2011. Collecting a training first image of the target object, and calling the second image sensor 2012 to collect a training second image of the target object, and training the preset recognition model according to the training first image and the training second image, The recognition accuracy of the preset recognition model is improved, thereby improving the accuracy of identifying fatigue driving.
S305、根据该动作特征描述信息确定该目标对象的状态信息。S305. Determine status information of the target object according to the action feature description information.
其中，状态信息用于至少该目标对象是否处于疲劳驾驶状态。The status information is used to determine whether at least the target object is in a fatigue driving state.
作为一种可选的实施方式，若根据该目标对象的状态信息确定需要暂停驾驶该车辆，则输出提示信息，该提示信息用于提示该目标对象暂停驾驶该车辆。As an optional implementation manner, if it is determined according to the state information of the target object that the vehicle needs to be suspended, the prompt information is output, and the prompt information is used to prompt the target object to pause driving the vehicle.
本发明实施例中，如果图像处理装置201根据该目标对象的状态信息确定该目标对象处于疲劳驾驶状态，可以确定需要暂停驾驶该车辆202，图像处理装置201可以输出提示信息，以提示该目标对象暂停驾驶该车辆，可以提高车辆驾驶的安全性。In the embodiment of the present invention, if the image processing apparatus 201 determines that the target object is in a fatigue driving state according to the state information of the target object, it may be determined that the driving of the vehicle 202 needs to be suspended, and the image processing apparatus 201 may output prompt information to prompt the target object. Suspension of driving the vehicle can improve the safety of driving the vehicle.
其中，该提示信息可以采用语音的方式提示，还可以采用在显示屏上显示的方式提示，或者多种结合的方式提示。The prompt information may be prompted by voice, or may be prompted by a manner displayed on a display screen, or a plurality of combined manners.
作为一种可选的实施方式，若根据该目标对象的状态信息确定需要启动该车辆的自动驾驶模式，则控制该车辆启动自动驾驶模式。As an optional implementation manner, if it is determined according to the state information of the target object that the automatic driving mode of the vehicle needs to be activated, the vehicle is controlled to start the automatic driving mode.
本发明实施例中，如果图像处理装置201根据该目标对象的状态信息确定该目标对象处于疲劳驾驶状态，可以确定需要启动该车辆202的自动驾驶模式，则控制该车辆启动自动驾驶模式，可以防止疲劳驾驶引起的交通事故发生，可以提高车辆驾驶的安全性。In the embodiment of the present invention, if the image processing apparatus 201 determines that the target object is in a fatigue driving state according to the state information of the target object, it may be determined that the automatic driving mode of the vehicle 202 needs to be activated, and then controlling the vehicle to start the automatic driving mode may prevent Traffic accidents caused by fatigue driving can improve the safety of driving.
本发明实施例，图像处理装置可以与车辆建立连接，该图像处理装置检测到位于该车辆驾驶位的目标对象，可以获取该目标对象的对象标识，获取与该对象标识关联的识别模型，将关联的识别模型作为预设的识别模型，可以提高识别的准确性。另外，该图像处理装置采用多种图像传感器采集的图像数据作为预设识别模型的信号输入，可以实现多种信号的互补，从而为预设的识别模型的输入端提供足够多的信息量，并且在RGB图像或灰度图像的基础上结合深度图对识别模型进行了优化处理，进而提高疲劳驾驶检测的准确度，并可以提高车辆驾驶的安全性。In the embodiment of the present invention, the image processing apparatus may establish a connection with the vehicle, the image processing apparatus detects the target object located in the driving position of the vehicle, acquires the object identifier of the target object, acquires an identification model associated with the object identifier, and associates The recognition model as a preset recognition model can improve the accuracy of recognition. In addition, the image processing apparatus adopts image data collected by a plurality of image sensors as a signal input of a preset recognition model, and can complement various signals, thereby providing a sufficient amount of information for the input end of the preset recognition model, and The recognition model is optimized based on the RGB image or the gray image combined with the depth map, thereby improving the accuracy of the fatigue driving detection and improving the driving safety of the vehicle.
基于上述对图像数据处理方法及图像数据处理系统的描述，本发明实施例提供一种图像处理装置，请参见图4，如图4所示的图像处理装置可以包括：Based on the above description of the image data processing method and the image data processing system, an embodiment of the present invention provides an image processing apparatus. Referring to FIG. 4, the image processing apparatus shown in FIG. 4 may include:
第一图像传感器401，用于采集目标对象的第一图像。The first image sensor 401 is configured to collect a first image of the target object.
第二图像传感器402，用于采集目标对象的第二图像。The second image sensor 402 is configured to collect a second image of the target object.
其中，所述第一图像包括灰度图像或RGB图像中的至少一种，所述第二图像包括深度图像。Wherein the first image comprises at least one of a grayscale image or an RGB image, the second image comprising a depth image.
接收模块403，用于接收所述第一图像传感器采集到的目标对象的第一图像，及所述第二图像传感器采集到的所述目标对象的第二图像。The receiving module 403 is configured to receive a first image of the target object collected by the first image sensor and a second image of the target object collected by the second image sensor.
识别模块404，用于将所述目标对象的第一图像和所述目标对象的第二图像输入到预设的识别模型中，得到用于描述所述目标对象的指定区域的动作特征的描述信息。An identification module 404, configured to input a first image of the target object and a second image of the target object into a preset recognition model, to obtain description information for describing an action feature of a specified area of the target object .
确定模块405，用于根据所述动作特征描述信息确定所述目标对象的状态信息。The determining module 405 is configured to determine state information of the target object according to the action feature description information.
其中，所述预设的识别模型用于对所述目标对象的第一图像和所述目标对象的第二图像的指定区域进行识别。The preset recognition model is configured to identify a first image of the target object and a designated area of the second image of the target object.
其中，所述目标对象的指定区域包括：所述目标对象的嘴部区域；所述动作特征描述信息包括：所述目标对象的嘴部区域处于张开特征的描述信息。The designated area of the target object includes: a mouth area of the target object; and the action feature description information includes: description information of the mouth area of the target object being in an open feature.
可选的，所述确定模块405，具体用于根据预设时间间隔内得到的所述目标对象的嘴部区域处于张开特征的描述信息，统计所述目标对象的嘴部区域处于张开特征的次数；若所述目标对象的嘴部区域处于张开特征的次数大于第一 预设阈值，则确定指示所述目标对象处于指定状态的状态信息。Optionally, the determining module 405 is configured to: according to the description information of the mouth region of the target object obtained in the preset time interval, the mouth region of the target object is in an open feature. The number of times; if the number of times the mouth region of the target object is in the open feature is greater than the first predetermined threshold, determining state information indicating that the target object is in the specified state.
其中，所述目标对象的指定区域包括：所述目标对象的眼部区域；所述动作特征描述信息包括：所述目标对象的眼部区域处于闭眼特征的描述信息。The specified area of the target object includes: an eye area of the target object; and the action feature description information includes: description information of the eye area of the target object in a closed eye feature.
可选的，所述确定模块405，用于根据预设时间间隔内得到的所述目标对象的眼部区域处于闭眼特征的描述信息，统计所述目标对象的眼部区域处于闭眼特征的次数；若所述目标对象的眼部区域处于闭眼特征的次数大于第二预阈值，则确定指示所述目标对象处于指定状态的状态信息。Optionally, the determining module 405 is configured to: according to the description information of the eye region of the target object obtained in the preset time interval, the eye region of the target object is in the closed eye feature The number of times; if the number of times the eye region of the target object is in the closed eye feature is greater than the second pre-threshold, determining state information indicating that the target object is in the specified state.
可选的，所述图像处理装置与车辆连接，所述图像处理装置用于采集位于驾驶位的对象的图像信息。Optionally, the image processing device is connected to a vehicle, and the image processing device is configured to collect image information of an object located in a driving position.
可选的，输出模块406，用于若根据所述目标对象的状态信息确定需要暂停驾驶所述车辆，则输出提示信息，所述提示信息用于提示所述目标对象暂停驾驶所述车辆。Optionally, the output module 406 is configured to: if it is determined that the vehicle needs to be paused according to the state information of the target object, output prompt information, where the prompt information is used to prompt the target object to pause driving the vehicle.
可选的，控制模块407，用于若根据所述目标对象的状态信息确定需要启动所述车辆的自动驾驶模式，则控制所述车辆启动自动驾驶模式。Optionally, the control module 407 is configured to control the vehicle to start the automatic driving mode if it is determined according to the state information of the target object that the automatic driving mode of the vehicle needs to be started.
可选的，调用模块408，用于如果检测到对所述预设的识别模型的训练指令，调用所述第一图像传感器采集所述目标对象的训练第一图像，及调用所述第二图像传感器采集所述目标对象的训练第二图像。Optionally, the calling module 408 is configured to: if the training instruction for the preset recognition model is detected, invoke the first image sensor to acquire a training first image of the target object, and invoke the second image A sensor acquires a training second image of the target object.
可选的，第一训练模块409，用于根据所述训练第一图像及训练第二图像对所述预设的识别模型进行训练。Optionally, the first training module 409 is configured to train the preset recognition model according to the training first image and the training second image.
可选的，第一图像传感器401，还用于采集训练对象的第一图像。Optionally, the first image sensor 401 is further configured to collect a first image of the training object.
可选的，第二图像传感器402，还用于采集训练对象的第二图像。Optionally, the second image sensor 402 is further configured to collect a second image of the training object.
可选的，第二训练模块410，用于采用初始识别模型对所述训练对象的第一图像及第二图像的指定区域进行训练，得到训练后的所述识别模型。Optionally, the second training module 410 is configured to train the designated areas of the first image and the second image of the training object by using an initial recognition model to obtain the trained recognition model.
可选的，第二训练模块410，具体用于获取所述训练对象当前的训练语料；采用所述初始识别模型对训练对象的第一图像及第二图像的指定区域进行识别，得到训练描述信息；确定所述训练对象当前的训练语料与所述训练描述信息的相似度；若所述相似度小于预设相似度值，则调整所述初始识别模型中的识别参数，得到训练后的所述识别模型。Optionally, the second training module 410 is configured to acquire a current training corpus of the training object, and use the initial recognition model to identify a first image of the training object and a designated area of the second image to obtain training description information. Determining a similarity between the current training corpus of the training object and the training description information; if the similarity is less than a preset similarity value, adjusting the identification parameter in the initial recognition model to obtain the trained Identify the model.
可选的，获取模块411，用于若检测到目标对象，则获取所述目标对象的 对象标识。Optionally, the obtaining module 411 is configured to acquire an object identifier of the target object if the target object is detected.
可选的，查找模块412，用于查找与所述目标对象的对象标识关联的识别模型。Optionally, the searching module 412 is configured to search for a recognition model associated with the object identifier of the target object.
可选的，所述接收模块403，具体用于将关联的识别模型作为所述预设的识别模型，并执行所述接收所述第一图像传感器采集到的目标对象的第一图像，及所述第二图像传感器采集到的所述目标对象的第二图像的步骤。Optionally, the receiving module 403 is specifically configured to use the associated recognition model as the preset recognition model, and execute the first image that receives the target object collected by the first image sensor, and The step of the second image of the target object collected by the second image sensor.
本发明实施例中，图像处理装置可以接收第一图像传感器采集到目标对象的第一图像，及第二图像传感器采集到的目标对象的第二图像，将该目标对象的第一图像及第二图像输入到预设的识别模型中，得到用于描述该目标对象的指定区域的动作特征的描述信息，根据该动作特征的描述信息确定该目标对象的状态信息，通过以多种图像传感器采集到的图像数据作为该识别模型的信号输入，可以实现多种信号的互补，从而为预设的识别模型的输入端提供足够多的信息量，进而提高疲劳检测的准确度。In the embodiment of the present invention, the image processing device may receive the first image captured by the first image sensor and the second image of the target object collected by the second image sensor, and the first image and the second image of the target object. The image is input into a preset recognition model, and the description information of the action feature for describing the specified area of the target object is obtained, and the state information of the target object is determined according to the description information of the action feature, and is collected by using multiple image sensors. As the signal input of the recognition model, the image data can realize the complementation of various signals, thereby providing sufficient information amount for the input end of the preset recognition model, thereby improving the accuracy of the fatigue detection.
请参见图5，图5是本发明实施例提供的一种图像处理设备的示意性框图。如图所示的本实施例中的一种图像处理设备可以包括：至少一个处理器501，例如CPU；至少一个存储器502，通信装置503，传感器504、控制器505，上述处理器501、存储器502、通信装置503，传感器504、控制器505通过总线506连接。Referring to FIG. 5, FIG. 5 is a schematic block diagram of an image processing apparatus according to an embodiment of the present invention. An image processing apparatus in this embodiment as shown may include: at least one processor 501, such as a CPU; at least one memory 502, a communication device 503, a sensor 504, a controller 505, the processor 501, and the memory 502. The communication device 503, the sensor 504, and the controller 505 are connected by a bus 506.
其中，通信装置503，可以用于输出提示信息，还可以用于建立与车辆的通信连接，并向车辆发送指令。The communication device 503 can be used to output prompt information, and can also be used to establish a communication connection with the vehicle and send an instruction to the vehicle.
传感器504，包括第一图像传感器及第二图像传感器，第一图像传感器可以是指单目视觉传感器，第二图像传感器可以是指多目视觉传感器，第一图像传感器，用于采集目标对象的第一图像，第二图像传感器，用于采用目标对象的第二图像。The sensor 504 includes a first image sensor and a second image sensor. The first image sensor may be a monocular vision sensor, the second image sensor may be a multi-eye vision sensor, and the first image sensor is used to collect a target object. An image, a second image sensor, for employing the second image of the target object.
控制器505，用于在需要控制车辆启动自动驾驶时，控制车辆启动自动模式。The controller 505 is configured to control the vehicle to start the automatic mode when it is required to control the vehicle to start the automatic driving.
存储器502用于存储指令，处理器501调用存储器502中存储的程序代码。 Memory 502 is used to store instructions, and processor 501 calls program code stored in memory 502.
具体的，处理器501调用存储器502中存储的程序代码，执行以下操作：Specifically, the processor 501 calls the program code stored in the memory 502 to perform the following operations:
接收所述第一图像传感器采集到的目标对象的第一图像，及所述第二图像传感器采集到的所述目标对象的第二图像；Receiving a first image of the target object collected by the first image sensor, and a second image of the target object collected by the second image sensor;
将所述目标对象的第一图像和所述目标对象的第二图像输入到预设的识别模型中，得到用于描述所述目标对象的指定区域的动作特征的描述信息；Inputting a first image of the target object and a second image of the target object into a preset recognition model to obtain description information for describing an action feature of a specified region of the target object;
根据所述动作特征描述信息确定所述目标对象的状态信息；Determining state information of the target object according to the action feature description information;
所述预设的识别模型用于对所述目标对象的第一图像和所述目标对象的第二图像的指定区域进行识别。The preset recognition model is configured to identify a first image of the target object and a designated area of the second image of the target object.
其中，所述第一图像包括灰度图像或RGB图像中的至少一种，所述第二图像包括深度图像。Wherein the first image comprises at least one of a grayscale image or an RGB image, the second image comprising a depth image.
可选的，所述目标对象的指定区域包括：所述目标对象的嘴部区域；所述动作特征描述信息包括：所述目标对象的嘴部区域处于张开特征的描述信息；处理器501调用存储器502中存储的程序代码，还可以执行以下操作：Optionally, the specified area of the target object includes: a mouth area of the target object; the action feature description information includes: description information of the mouth area of the target object in an open feature; The program code stored in the memory 502 can also perform the following operations:
根据预设时间间隔内得到的所述目标对象的嘴部区域处于张开特征的描述信息，统计所述目标对象的嘴部区域处于张开特征的次数；And counting, according to the description information of the open feature of the mouth region of the target object obtained in the preset time interval, counting the number of times the mouth region of the target object is in the open feature;
若所述目标对象的嘴部区域处于张开特征的次数大于第一预设阈值，则确定指示所述目标对象处于指定状态的状态信息。And determining, if the number of times the mouth region of the target object is in the open feature is greater than the first preset threshold, status information indicating that the target object is in the specified state.
可选的，所述目标对象的指定区域包括：所述目标对象的眼部区域；所述动作特征描述信息包括：所述目标对象的眼部区域处于闭眼特征的描述信息；处理器501调用存储器502中存储的程序代码，还可以执行以下操作：Optionally, the specified area of the target object includes: an eye area of the target object; the action feature description information includes: description information of the eye area of the target object in a closed eye feature; The program code stored in the memory 502 can also perform the following operations:
根据预设时间间隔内得到的所述目标对象的眼部区域处于闭眼特征的描述信息，统计所述目标对象的眼部区域处于闭眼特征的次数；And counting, according to the description information of the closed eye feature of the eye region of the target object obtained in the preset time interval, counting the number of times the eye region of the target object is in the closed eye feature;
若所述目标对象的眼部区域处于闭眼特征的次数大于第二预阈值，则确定指示所述目标对象处于指定状态的状态信息。If the number of times the eye region of the target object is in the closed eye feature is greater than the second pre-threshold, determining state information indicating that the target object is in the designated state.
可选的，所述图像处理装置与车辆连接，所述图像处理装置用于采集位于驾驶位的对象的图像信息。Optionally, the image processing device is connected to a vehicle, and the image processing device is configured to collect image information of an object located in a driving position.
可选的，处理器501调用存储器502中存储的程序代码，还可以执行以下操作：Optionally, the processor 501 calls the program code stored in the memory 502, and may also perform the following operations:
若根据所述目标对象的状态信息确定需要暂停驾驶所述车辆，则输出提示信息，所述提示信息用于提示所述目标对象暂停驾驶所述车辆。If it is determined according to the state information of the target object that the vehicle needs to be suspended, the prompt information is output, and the prompt information is used to prompt the target object to pause driving the vehicle.
可选的，处理器501调用存储器502中存储的程序代码，还可以执行以下操作：Optionally, the processor 501 calls the program code stored in the memory 502, and may also perform the following operations:
若根据所述目标对象的状态信息确定需要启动所述车辆的自动驾驶模式，则控制所述车辆启动自动驾驶模式。If it is determined according to the state information of the target object that the automatic driving mode of the vehicle needs to be activated, the vehicle is controlled to start the automatic driving mode.
可选的，处理器501调用存储器502中存储的程序代码，还可以执行以下操作：Optionally, the processor 501 calls the program code stored in the memory 502, and may also perform the following operations:
如果检测到对所述预设的识别模型的训练指令，调用所述第一图像传感器采集所述目标对象的训练第一图像，及调用所述第二图像传感器采集所述目标对象的训练第二图像；If the training instruction for the preset recognition model is detected, calling the first image sensor to acquire a training first image of the target object, and calling the second image sensor to acquire the training target of the target object image;
根据所述训练第一图像及训练第二图像对所述预设的识别模型进行训练。The preset recognition model is trained according to the training first image and the training second image.
可选的，处理器501调用存储器502中存储的程序代码，还可以执行以下操作：Optionally, the processor 501 calls the program code stored in the memory 502, and may also perform the following operations:
采集训练对象的第一图像及第二图像；Collecting a first image and a second image of the training object;
采用初始识别模型对所述训练对象的第一图像及第二图像的指定区域进行训练，得到训练后的所述识别模型。The first image of the training object and the designated area of the second image are trained by using an initial recognition model to obtain the trained recognition model.
可选的，处理器501调用存储器502中存储的程序代码，还可以执行以下操作：Optionally, the processor 501 calls the program code stored in the memory 502, and may also perform the following operations:
获取所述训练对象当前的训练语料；Obtaining a current training corpus of the training object;
采用所述初始识别模型对训练对象的第一图像及第二图像的指定区域进行识别，得到训练描述信息；Using the initial recognition model to identify the first image of the training object and the designated area of the second image to obtain training description information;
确定所述训练对象当前的训练语料与所述训练描述信息的相似度；Determining a similarity between the current training corpus of the training object and the training description information;
若所述相似度小于预设相似度值，则调整所述初始识别模型中的识别参数，得到训练后的所述识别模型。If the similarity is less than the preset similarity value, the identification parameter in the initial recognition model is adjusted to obtain the trained recognition model.
可选的，处理器501调用存储器502中存储的程序代码，还可以执行以下操作：Optionally, the processor 501 calls the program code stored in the memory 502, and may also perform the following operations:
若检测到目标对象，则获取所述目标对象的对象标识；Obtaining an object identifier of the target object if the target object is detected;
查找与所述目标对象的对象标识关联的识别模型；Finding a recognition model associated with the object identifier of the target object;
将关联的识别模型作为所述预设的识别模型，并执行所述接收所述第一图像传感器采集到的目标对象的第一图像，及所述第二图像传感器采集到的所述 目标对象的第二图像的步骤。Using the associated recognition model as the preset recognition model, and performing the receiving the first image of the target object collected by the first image sensor, and the target object collected by the second image sensor The step of the second image.
本发明实施例中，图像处理装置可以接收第一图像传感器采集到目标对象的第一图像，及第二图像传感器采集到的目标对象的第二图像，将该目标对象的第一图像及第二图像输入到预设的识别模型中，得到用于描述该目标对象的指定区域的动作特征的描述信息，根据该动作特征的描述信息确定该目标对象的状态信息，通过以多种图像传感器采集到的图像数据作为该识别模型的信号输入，可以实现多种信号的互补，从而为预设的识别模型的输入端提供足够多的信息量，并且在灰度图像、RGB图像的基础上结合深度图对识别模型进行了优化处理，进而提高疲劳检测的准确度。In the embodiment of the present invention, the image processing device may receive the first image captured by the first image sensor and the second image of the target object collected by the second image sensor, and the first image and the second image of the target object. The image is input into a preset recognition model, and the description information of the action feature for describing the specified area of the target object is obtained, and the state information of the target object is determined according to the description information of the action feature, and is collected by using multiple image sensors. As the signal input of the recognition model, the image data can realize the complementation of various signals, thereby providing sufficient information amount for the input end of the preset recognition model, and combining the depth map on the basis of the gray image and the RGB image. The recognition model is optimized to improve the accuracy of fatigue detection.
本申请还提供了一种计算机程序产品，该计算机程序产品包括存储了计算机程序的非瞬时性计算机可读存储介质，该计算机程序可操作来使计算机执行上述图1和图3对应实施例中的图像数据方法的步骤，该计算机程序产品解决问题的实施方式以及有益效果可以参见上述图1和图3的图像数据方法的实施方式以及有益效果，重复之处不再赘述。The present application also provides a computer program product comprising a non-transitory computer readable storage medium storing a computer program operative to cause a computer to perform the above-described embodiments of FIGS. 1 and 3 The steps of the image data method, the implementation of the problem and the beneficial effects of the computer program product can be referred to the embodiments and the beneficial effects of the image data method of FIG. 1 and FIG. 3 above, and the repeated description is omitted.
需要说明的是，对于前述的各个方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本发明并不受所描述的动作顺序的限制，因为依据本发明，某一些步骤可以采用其他顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于优选实施例，所涉及的动作和模块并不一定是本发明所必须的。It should be noted that, for the foregoing various method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should understand that the present invention is not limited by the described action sequence. Because certain steps may be performed in other sequences or concurrently in accordance with the present invention. In addition, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.
本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成，该程序可以存储于一计算机可读存储介质中，存储介质可以包括：闪存盘、只读存储器(Read-Only Memory，ROM)、随机存取器(Random Access Memory，RAM)、磁盘或光盘等。A person skilled in the art may understand that all or part of the various steps of the foregoing embodiments may be performed by a program to instruct related hardware. The program may be stored in a computer readable storage medium, and the storage medium may include: Flash disk, Read-Only Memory (ROM), Random Access Memory (RAM), disk or optical disk.
以上所揭露的仅为本发明一种部分实施例而已，当然不能以此来限定本发明之权利范围，本领域普通技术人员可以理解实现上述实施例的全部或部分流程，并依本发明权利要求所作的等同变化，仍属于发明所涵盖的范围。The above disclosure is only a part of the embodiments of the present invention, and of course, the scope of the present invention is not limited thereto, and those skilled in the art can understand all or part of the process of implementing the above embodiments, and according to the claims of the present invention. Equivalent changes made are still within the scope of the invention.

Claims (27)

  1. 一种图像数据处理方法，其特征在于，应用于图像处理装置，所述图像处理装置包括第一图像传感器及第二图像传感器，所述方法包括：An image data processing method is applied to an image processing apparatus, the image processing apparatus comprising a first image sensor and a second image sensor, the method comprising:
    接收所述第一图像传感器采集到的目标对象的第一图像，及所述第二图像传感器采集到的所述目标对象的第二图像，所述第一图像包括灰度图像或RGB图像中的至少一种，所述第二图像包括深度图像；Receiving a first image of the target object collected by the first image sensor and a second image of the target object collected by the second image sensor, where the first image includes a grayscale image or an RGB image At least one of the second images comprising a depth image;
    将所述目标对象的第一图像和所述目标对象的第二图像输入到预设的识别模型中，得到用于描述所述目标对象的指定区域的动作特征的描述信息；Inputting a first image of the target object and a second image of the target object into a preset recognition model to obtain description information for describing an action feature of a specified region of the target object;
    根据所述动作特征描述信息确定所述目标对象的状态信息。Determining state information of the target object according to the action feature description information.
  2. 根据权利要求1所述的方法，其特征在于，所述目标对象的指定区域包括：所述目标对象的嘴部区域；所述动作特征描述信息包括：所述目标对象的嘴部区域处于张开特征的描述信息。The method according to claim 1, wherein the designated area of the target object comprises: a mouth area of the target object; and the action feature description information comprises: the mouth area of the target object is open Description of the feature.
  3. 根据权利要求2所述的方法，其特征在于，所述根据所述动作特征描述信息确定所述目标对象的状态信息，包括：The method according to claim 2, wherein the determining the state information of the target object according to the action feature description information comprises:
    根据预设时间间隔内得到的所述目标对象的嘴部区域处于张开特征的描述信息，统计所述目标对象的嘴部区域处于张开特征的次数；And counting, according to the description information of the open feature of the mouth region of the target object obtained in the preset time interval, counting the number of times the mouth region of the target object is in the open feature;
    若所述目标对象的嘴部区域处于张开特征的次数大于第一预设阈值，则确定指示所述目标对象处于指定状态的状态信息。And determining, if the number of times the mouth region of the target object is in the open feature is greater than the first preset threshold, status information indicating that the target object is in the specified state.
  4. 根据权利要求1所述的方法，其特征在于，所述目标对象的指定区域包括：所述目标对象的眼部区域；所述动作特征描述信息包括：所述目标对象的眼部区域处于闭眼特征的描述信息。The method according to claim 1, wherein the designated area of the target object comprises: an eye area of the target object; and the action feature description information comprises: the eye area of the target object is in a closed eye Description of the feature.
  5. 根据权利要求4所述的方法，其特征在于，所述根据所述动作特征描述信息确定所述目标对象的状态信息，包括：The method according to claim 4, wherein the determining the state information of the target object according to the action feature description information comprises:
    根据预设时间间隔内得到的所述目标对象的眼部区域处于闭眼特征的描述信息，统计所述目标对象的眼部区域处于闭眼特征的次数；And counting, according to the description information of the closed eye feature of the eye region of the target object obtained in the preset time interval, counting the number of times the eye region of the target object is in the closed eye feature;
    若所述目标对象的眼部区域处于闭眼特征的次数大于第二预阈值，则确定指示所述目标对象处于指定状态的状态信息。If the number of times the eye region of the target object is in the closed eye feature is greater than the second pre-threshold, determining state information indicating that the target object is in the designated state.
  6. 根据权利要求1-5任一项所述的方法，其特征在于，所述图像处理装置与车辆连接，所述图像处理装置用于采集位于驾驶位的对象的图像信息。The method according to any one of claims 1 to 5, wherein the image processing device is connected to a vehicle, and the image processing device is configured to collect image information of an object located in a driving position.
  7. 根据权利要求6所述的方法，其特征在于，还包括：The method of claim 6 further comprising:
    若根据所述目标对象的状态信息确定需要暂停驾驶所述车辆，则输出提示信息，所述提示信息用于提示所述目标对象暂停驾驶所述车辆。If it is determined according to the state information of the target object that the vehicle needs to be suspended, the prompt information is output, and the prompt information is used to prompt the target object to pause driving the vehicle.
  8. 根据权利要求6所述的方法，其特征在于，还包括：The method of claim 6 further comprising:
    若根据所述目标对象的状态信息确定需要启动所述车辆的自动驾驶模式，则控制所述车辆启动自动驾驶模式。If it is determined according to the state information of the target object that the automatic driving mode of the vehicle needs to be activated, the vehicle is controlled to start the automatic driving mode.
  9. 根据权利要求7或8所述的方法，其特征在于，还包括：The method according to claim 7 or 8, further comprising:
    如果检测到对所述预设的识别模型的训练指令，调用所述第一图像传感器采集所述目标对象的训练第一图像，及调用所述第二图像传感器采集所述目标对象的训练第二图像；If the training instruction for the preset recognition model is detected, calling the first image sensor to acquire a training first image of the target object, and calling the second image sensor to acquire the training target of the target object image;
    根据所述训练第一图像及训练第二图像对所述预设的识别模型进行训练。The preset recognition model is trained according to the training first image and the training second image.
  10. 根据权利要求7或8所述的方法，其特征在于，还包括：The method according to claim 7 or 8, further comprising:
    采集训练对象的第一图像及第二图像；Collecting a first image and a second image of the training object;
    采用初始识别模型对所述训练对象的第一图像及第二图像的指定区域进行训练，得到训练后的所述识别模型。The first image of the training object and the designated area of the second image are trained by using an initial recognition model to obtain the trained recognition model.
  11. 根据权利要求10所述的方法，其特征在于，所述采用初始识别模型对所述训练对象的第一图像及第二图像的指定区域进行训练，得到训练后的所述识别模型，包括：The method according to claim 10, wherein the initial recognition model is used to train the designated areas of the first image and the second image of the training object to obtain the trained recognition model, including:
    获取所述训练对象当前的训练语料；Obtaining a current training corpus of the training object;
    采用所述初始识别模型对训练对象的第一图像及第二图像的指定区域进行识别，得到训练描述信息；Using the initial recognition model to identify the first image of the training object and the designated area of the second image to obtain training description information;
    确定所述训练对象当前的训练语料与所述训练描述信息的相似度；Determining a similarity between the current training corpus of the training object and the training description information;
    若所述相似度小于预设相似度值，则调整所述初始识别模型中的识别参数，得到训练后的所述识别模型。If the similarity is less than the preset similarity value, the identification parameter in the initial recognition model is adjusted to obtain the trained recognition model.
  12. 根据权利要求1或11所述的方法，其特征在于，还包括：The method according to claim 1 or 11, further comprising:
    若检测到目标对象，则获取所述目标对象的对象标识；Obtaining an object identifier of the target object if the target object is detected;
    查找与所述目标对象的对象标识关联的识别模型；Finding a recognition model associated with the object identifier of the target object;
    将关联的识别模型作为所述预设的识别模型，并执行所述接收所述第一图像传感器采集到的目标对象的第一图像，及所述第二图像传感器采集到的所述目标对象的第二图像的步骤。Using the associated recognition model as the preset recognition model, and performing the receiving the first image of the target object collected by the first image sensor, and the target object collected by the second image sensor The step of the second image.
  13. 一种图像处理装置，其特征在于，所述图像处理装置包括第一图像传感器及第二图像传感器，所述装置包括：An image processing apparatus, comprising: a first image sensor and a second image sensor, the apparatus comprising:
    接收模块，用于接收所述第一图像传感器采集到的目标对象的第一图像，及所述第二图像传感器采集到的所述目标对象的第二图像，所述第一图像包括灰度图像或RGB图像中的至少一种，所述第二图像包括深度图像；a receiving module, configured to receive a first image of the target object collected by the first image sensor, and a second image of the target object collected by the second image sensor, where the first image includes a grayscale image Or at least one of RGB images, the second image comprising a depth image;
    识别模块，用于将所述目标对象的第一图像和所述目标对象的第二图像输入到预设的识别模型中，得到用于描述所述目标对象的指定区域的动作特征的描述信息；An identification module, configured to input a first image of the target object and a second image of the target object into a preset recognition model, to obtain description information for describing an action feature of a specified area of the target object;
    确定模块，用于根据所述动作特征描述信息确定所述目标对象的状态信息；a determining module, configured to determine state information of the target object according to the action feature description information;
    所述预设的识别模型用于对所述目标对象的第一图像和所述目标对象的第二图像的指定区域进行识别。The preset recognition model is configured to identify a first image of the target object and a designated area of the second image of the target object.
  14. 根据权利要求13所述的装置，其特征在于，所述目标对象的指定区域包括：所述目标对象的嘴部区域；所述动作特征描述信息包括：所述目标对象的嘴部区域处于张开特征的描述信息。The apparatus according to claim 13, wherein the designated area of the target object comprises: a mouth area of the target object; and the action feature description information comprises: the mouth area of the target object is open Description of the feature.
  15. 根据权利要求14所述的装置，其特征在于，The device of claim 14 wherein:
    所述确定模块，具体用于根据预设时间间隔内得到的所述目标对象的嘴部区域处于张开特征的描述信息，统计所述目标对象的嘴部区域处于张开特征的次数；若所述目标对象的嘴部区域处于张开特征的次数大于第一预设阈值，则确定指示所述目标对象处于指定状态的状态信息。The determining module is configured to: according to the description information of the mouth region of the target object obtained in the preset time interval, the number of times the mouth region of the target object is in the open feature; When the number of times the mouth region of the target object is in the open feature is greater than the first preset threshold, state information indicating that the target object is in the specified state is determined.
  16. 根据权利要求13所述的装置，其特征在于，所述目标对象的指定区域包括：所述目标对象的眼部区域；所述动作特征描述信息包括：所述目标对象的眼部区域处于闭眼特征的描述信息。The apparatus according to claim 13, wherein the designated area of the target object comprises: an eye area of the target object; and the action feature description information comprises: an eye area of the target object is in a closed eye Description of the feature.
  17. 根据权利要求16所述的装置，其特征在于，The device of claim 16 wherein:
    所述确定模块，用于根据预设时间间隔内得到的所述目标对象的眼部区域处于闭眼特征的描述信息，统计所述目标对象的眼部区域处于闭眼特征的次数；若所述目标对象的眼部区域处于闭眼特征的次数大于第二预阈值，则确定指示所述目标对象处于指定状态的状态信息。The determining module is configured to count, according to the description information of the closed eye feature of the eye region of the target object obtained in the preset time interval, the number of times the eye region of the target object is in the closed eye feature; If the number of times the eye region of the target object is in the closed eye feature is greater than the second pre-threshold, state information indicating that the target object is in the specified state is determined.
  18. 根据权利要求13-17任一项所述的装置，其特征在于，所述图像处理装置与车辆连接，所述图像处理装置用于采集位于驾驶位的对象的图像信息。Apparatus according to any one of claims 13-17, wherein said image processing means is coupled to a vehicle, said image processing means for acquiring image information of an object located in the driver's seat.
  19. 根据权利要求18所述的装置，其特征在于，还包括：The device according to claim 18, further comprising:
    输出模块，用于若根据所述目标对象的状态信息确定需要暂停驾驶所述车辆，则输出提示信息，所述提示信息用于提示所述目标对象暂停驾驶所述车辆。And an output module, configured to output prompt information for prompting the target object to pause driving the vehicle if it is determined that the vehicle needs to be paused according to the state information of the target object.
  20. 根据权利要求18所述的装置，其特征在于，还包括：The device according to claim 18, further comprising:
    控制模块，用于若根据所述目标对象的状态信息确定需要启动所述车辆的自动驾驶模式，则控制所述车辆启动自动驾驶模式。And a control module, configured to control the vehicle to start an automatic driving mode if it is determined according to state information of the target object that an automatic driving mode of the vehicle needs to be activated.
  21. 根据权利要求19或20所述的装置，其特征在于，还包括：The device according to claim 19 or 20, further comprising:
    调用模块，用于如果检测到对所述预设的识别模型的训练指令，调用所述第一图像传感器采集所述目标对象的训练第一图像，及调用所述第二图像传感器采集所述目标对象的训练第二图像；Calling a module, if the training instruction for the preset recognition model is detected, calling the first image sensor to acquire a training first image of the target object, and calling the second image sensor to collect the target Training the second image of the object;
    第一训练模块，用于根据所述训练第一图像及训练第二图像对所述预设的识别模型进行训练。The first training module is configured to train the preset recognition model according to the training the first image and the training the second image.
  22. 根据权利要求19或20所述的装置，其特征在于，还包括：The device according to claim 19 or 20, further comprising:
    所述第一图像传感器，还用于采集训练对象的第一图像；The first image sensor is further configured to collect a first image of the training object;
    所述第二图像传感器，还用于采集所述训练对象的第二图像；The second image sensor is further configured to collect a second image of the training object;
    所述图像处理装置还包括：The image processing apparatus further includes:
    第二训练模块，用于采用初始识别模型对所述训练对象的第一图像及第二图像的指定区域进行训练，得到训练后的所述识别模型。And a second training module, configured to use the initial recognition model to train the first image of the training object and the designated area of the second image to obtain the trained recognition model.
  23. 根据权利要求22所述的装置，其特征在于，The device according to claim 22, wherein
    第二训练模块，具体用于获取所述训练对象当前的训练语料；采用所述初始识别模型对训练对象的第一图像及第二图像的指定区域进行识别，得到训练描述信息；确定所述训练对象当前的训练语料与所述训练描述信息的相似度；若所述相似度小于预设相似度值，则调整所述初始识别模型中的识别参数，得到训练后的所述识别模型。a second training module, configured to acquire a current training corpus of the training object; use the initial recognition model to identify a first image of the training object and a designated area of the second image, to obtain training description information; and determine the training The similarity between the current training corpus of the object and the training description information; if the similarity is less than the preset similarity value, adjusting the identification parameter in the initial recognition model to obtain the trained recognition model.
  24. 根据权利要求13或23所述的装置，其特征在于，还包括：The device according to claim 13 or 23, further comprising:
    获取模块，用于若检测到目标对象，则获取所述目标对象的对象标识；An acquiring module, configured to acquire an object identifier of the target object if the target object is detected;
    查找模块，用于查找与所述目标对象的对象标识关联的识别模型；a finding module, configured to find a recognition model associated with the object identifier of the target object;
    所述接收模块，具体用于将关联的识别模型作为所述预设的识别模型，并执行所述接收所述第一图像传感器采集到的目标对象的第一图像，及所述第二图像传感器采集到的所述目标对象的第二图像的步骤。The receiving module is specifically configured to use the associated recognition model as the preset recognition model, and execute the first image that receives the target object collected by the first image sensor, and the second image sensor The step of acquiring the second image of the target object.
  25. 一种图像处理设备，其特征在于，包括：处理器和存储器，所述处理器和所述存储器通过总线连接，所述存储器存储有可执行程序代码，所述处理 器用于调用所述可执行程序代码，执行如权利要求1至12中任一项所述的图像数据处理方法。An image processing apparatus, comprising: a processor and a memory, wherein the processor and the memory are connected by a bus, the memory stores executable program code, and the processor is configured to invoke the executable program The code performs the image data processing method according to any one of claims 1 to 12.
  26. 一种计算机可读存储介质，其特征在于，所述计算机存储介质存储有计算机程序，所述计算机程序包括程序指令，所述程序指令当被处理器执行时使所述处理器执行如权利要求1至12中任一项所述的图像数据方法的步骤。A computer readable storage medium, characterized in that the computer storage medium stores a computer program, the computer program comprising program instructions, the program instructions, when executed by a processor, causing the processor to execute as claimed in claim 1. The steps of the image data method of any of 12.
  27. 一种计算机程序产品，其特征在于，所述计算机程序产品包括存储了计算机程序的非瞬时性计算机可读存储介质，所述计算机程序可操作来使计算机实现权利要求1至12中任一项所述的图像数据方法的步骤。A computer program product, comprising: a non-transitory computer readable storage medium storing a computer program, the computer program being operative to cause a computer to implement any one of claims 1 to 12. The steps of the image data method described.
PCT/CN2017/118174 2017-12-25 2017-12-25 Image data processing method, device and equipment WO2019126908A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2017/118174 WO2019126908A1 (en) 2017-12-25 2017-12-25 Image data processing method, device and equipment
CN201780005969.XA CN108701214A (en) 2017-12-25 2017-12-25 Image processing method, device and equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/118174 WO2019126908A1 (en) 2017-12-25 2017-12-25 Image data processing method, device and equipment

Publications (1)

Publication Number Publication Date
WO2019126908A1 true WO2019126908A1 (en) 2019-07-04

Family

ID=63843765

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/118174 WO2019126908A1 (en) 2017-12-25 2017-12-25 Image data processing method, device and equipment

Country Status (2)

Country Link
CN (1) CN108701214A (en)
WO (1) WO2019126908A1 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111770317A (en) * 2020-07-22 2020-10-13 平安国际智慧城市科技股份有限公司 Video monitoring method, device, equipment and medium for intelligent community
CN112068218A (en) * 2020-09-14 2020-12-11 北京数衍科技有限公司 Self-adaptive detection method and device for pedestrian steps
CN112378916A (en) * 2020-11-10 2021-02-19 厦门长江电子科技有限公司 Image grading automatic detection system and method based on machine vision
CN112926498A (en) * 2021-03-20 2021-06-08 杭州知存智能科技有限公司 In-vivo detection method based on multi-channel fusion and local dynamic generation of depth information
CN113610004A (en) * 2021-08-09 2021-11-05 上海擎朗智能科技有限公司 Image processing method, robot and medium
CN112926498B (en) * 2021-03-20 2024-05-24 杭州知存智能科技有限公司 Living body detection method and device based on multichannel fusion and depth information local dynamic generation

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110147713A (en) * 2019-03-28 2019-08-20 石化盈科信息技术有限责任公司 A kind of method for detecting fatigue driving and system
CN112204566A (en) * 2019-08-15 2021-01-08 深圳市大疆创新科技有限公司 Image processing method and device based on machine vision
CN110570400B (en) * 2019-08-19 2022-11-11 河北极目楚天微电子科技有限公司 Information processing method and device for chip 3D packaging detection
CN114022871A (en) * 2021-11-10 2022-02-08 中国民用航空飞行学院 Unmanned aerial vehicle driver fatigue detection method and system based on depth perception technology
WO2023108364A1 (en) * 2021-12-13 2023-06-22 华为技术有限公司 Method and apparatus for detecting driver state, and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20130124763A (en) * 2012-05-07 2013-11-15 현대모비스 주식회사 System of determining tiredness driving based on the driver's face image and methed thereof
CN103714321A (en) * 2013-12-26 2014-04-09 苏州清研微视电子科技有限公司 Driver face locating system based on distance image and strength image
CN104504856A (en) * 2014-12-30 2015-04-08 天津大学 Fatigue driving detection method based on Kinect and face recognition
CN105740767A (en) * 2016-01-22 2016-07-06 江苏大学 Driver road rage real-time identification and warning method based on facial features
CN107126224A (en) * 2017-06-20 2017-09-05 中南大学 A kind of real-time monitoring of track train driver status based on Kinect and method for early warning and system

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080091628A1 (en) * 2006-08-16 2008-04-17 Narayan Srinivasa Cognitive architecture for learning, action, and perception
US20090086269A1 (en) * 2007-09-28 2009-04-02 Kyocera Mita Corporation Image Forming Apparatus and Image Forming System
KR20150085035A (en) * 2012-11-15 2015-07-22 가부시키가이샤 한도오따이 에네루기 켄큐쇼 Liquid crystal display device
US20140184484A1 (en) * 2012-12-28 2014-07-03 Semiconductor Energy Laboratory Co., Ltd. Display device
CN103473530B (en) * 2013-08-30 2016-06-15 天津理工大学 Self adaptation action identification method based on multi views and multi-modal feature
CN103714660B (en) * 2013-12-26 2017-02-08 苏州清研微视电子科技有限公司 System for achieving fatigue driving judgment on basis of image processing and fusion between heart rate characteristic and expression characteristic
CN103810491B (en) * 2014-02-19 2017-02-22 北京工业大学 Head posture estimation interest point detection method fusing depth and gray scale image characteristic points
CN104268520A (en) * 2014-09-22 2015-01-07 天津理工大学 Human motion recognition method based on depth movement trail
CN104616437A (en) * 2015-02-27 2015-05-13 浪潮集团有限公司 Vehicle-mounted fatigue identification system and method
CN104809445B (en) * 2015-05-07 2017-12-19 吉林大学 method for detecting fatigue driving based on eye and mouth state
CN106897659B (en) * 2015-12-18 2019-05-24 腾讯科技（深圳）有限公司 The recognition methods of blink movement and device
CN106203394B (en) * 2016-07-26 2019-04-26 浙江捷尚视觉科技股份有限公司 Fatigue driving safety monitoring method based on human eye state detection
CN106446811A (en) * 2016-09-12 2017-02-22 北京智芯原动科技有限公司 Deep-learning-based driver's fatigue detection method and apparatus
CN106485214A (en) * 2016-09-28 2017-03-08 天津工业大学 A kind of eyes based on convolutional neural networks and mouth state identification method
CN106599806A (en) * 2016-12-01 2017-04-26 西安理工大学 Local curved-surface geometric feature-based human body action recognition method
CN107203753B (en) * 2017-05-25 2020-09-08 西安工业大学 Action recognition method based on fuzzy neural network and graph model reasoning
CN107229922A (en) * 2017-06-12 2017-10-03 西南科技大学 A kind of fatigue driving monitoring method and device
CN107358794A (en) * 2017-06-13 2017-11-17 深圳前海慧泊中安运营管理有限公司 Data processing method and device
CN107492074A (en) * 2017-07-21 2017-12-19 触景无限科技（北京）有限公司 Image acquisition and processing method, device and terminal device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20130124763A (en) * 2012-05-07 2013-11-15 현대모비스 주식회사 System of determining tiredness driving based on the driver's face image and methed thereof
CN103714321A (en) * 2013-12-26 2014-04-09 苏州清研微视电子科技有限公司 Driver face locating system based on distance image and strength image
CN104504856A (en) * 2014-12-30 2015-04-08 天津大学 Fatigue driving detection method based on Kinect and face recognition
CN105740767A (en) * 2016-01-22 2016-07-06 江苏大学 Driver road rage real-time identification and warning method based on facial features
CN107126224A (en) * 2017-06-20 2017-09-05 中南大学 A kind of real-time monitoring of track train driver status based on Kinect and method for early warning and system

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111770317A (en) * 2020-07-22 2020-10-13 平安国际智慧城市科技股份有限公司 Video monitoring method, device, equipment and medium for intelligent community
CN111770317B (en) * 2020-07-22 2023-02-03 平安国际智慧城市科技股份有限公司 Video monitoring method, device, equipment and medium for intelligent community
CN112068218A (en) * 2020-09-14 2020-12-11 北京数衍科技有限公司 Self-adaptive detection method and device for pedestrian steps
CN112378916A (en) * 2020-11-10 2021-02-19 厦门长江电子科技有限公司 Image grading automatic detection system and method based on machine vision
CN112378916B (en) * 2020-11-10 2024-03-29 厦门长江电子科技有限公司 Automatic image grading detection system and method based on machine vision
CN112926498A (en) * 2021-03-20 2021-06-08 杭州知存智能科技有限公司 In-vivo detection method based on multi-channel fusion and local dynamic generation of depth information
CN112926498B (en) * 2021-03-20 2024-05-24 杭州知存智能科技有限公司 Living body detection method and device based on multichannel fusion and depth information local dynamic generation
CN113610004A (en) * 2021-08-09 2021-11-05 上海擎朗智能科技有限公司 Image processing method, robot and medium
CN113610004B (en) * 2021-08-09 2024-04-05 上海擎朗智能科技有限公司 Image processing method, robot and medium

Also Published As

Publication number Publication date
CN108701214A (en) 2018-10-23

Similar Documents

Publication Publication Date Title
WO2019126908A1 (en) Image data processing method, device and equipment
WO2019232972A1 (en) Driving management method and system, vehicle-mounted intelligent system, electronic device and medium
US10509961B2 (en) Blindman navigation method and blindman navigation apparatus
WO2020207423A1 (en) Skin type detection method, skin type grade classification method and skin type detection apparatus
WO2018188453A1 (en) Method for determining human face area, storage medium, and computer device
EP3163543B1 (en) Alarming method and device
EP3647129A1 (en) Vehicle, vehicle door unlocking control method and apparatus, and vehicle door unlocking system
CN110032966B (en) Human body proximity detection method for intelligent service, intelligent service method and device
CN106485191B (en) A kind of method for detecting fatigue state of driver and system
JP5737400B2 (en) Red eye detector
JP5482737B2 (en) Visual load amount estimation device, driving support device, and visual load amount estimation program
WO2019196558A1 (en) Screen light method, device, mobile terminal, and storage medium
TW201941104A (en) Control method for smart device, apparatus, device, and storage medium
US11423673B2 (en) Method and device for detecting state of holding steering wheel
US11227156B2 (en) Personalized eye openness estimation
CN111985373A (en) Safety early warning method and device based on traffic intersection identification and electronic equipment
CN112183356A (en) Driving behavior detection method and device and readable storage medium
CN104063041B (en) A kind of information processing method and electronic equipment
WO2024001617A1 (en) Method and apparatus for identifying behavior of playing with mobile phone
KR101236266B1 (en) Manage system of parking space information
CN110660187B (en) Forest fire alarm monitoring system based on edge calculation
TWI550440B (en) Method and system for detecting person to use handheld apparatus
KR101669447B1 (en) System and the method for recognizing drowsiness
US9996740B2 (en) Information processing device, information processing method, program, and information storage medium
JP6742837B2 (en) Image processing apparatus, image processing method and program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17936146

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17936146

Country of ref document: EP

Kind code of ref document: A1