WO2023184783A1 - 图像检测识别装置及其检测识别方法 - Google Patents

图像检测识别装置及其检测识别方法 Download PDF

Info

Publication number
WO2023184783A1
WO2023184783A1 PCT/CN2022/105235 CN2022105235W WO2023184783A1 WO 2023184783 A1 WO2023184783 A1 WO 2023184783A1 CN 2022105235 W CN2022105235 W CN 2022105235W WO 2023184783 A1 WO2023184783 A1 WO 2023184783A1
Authority
WO
WIPO (PCT)
Prior art keywords
vertex
coordinate
midpoint
image data
display area
Prior art date
Application number
PCT/CN2022/105235
Other languages
English (en)
French (fr)
Inventor
胡正东
徐靖熙
侯成龙
陈晶
Original Assignee
杭州涂鸦信息技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 杭州涂鸦信息技术有限公司 filed Critical 杭州涂鸦信息技术有限公司
Publication of WO2023184783A1 publication Critical patent/WO2023184783A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N17/00—Diagnosis, testing or measuring for television systems or their details
    • H04N17/004—Diagnosis, testing or measuring for television systems or their details for digital television systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N17/00—Diagnosis, testing or measuring for television systems or their details
    • H04N17/002—Diagnosis, testing or measuring for television systems or their details for television cameras

Definitions

  • Embodiments of the present application relate to the field of Internet of Things smart homes, and in particular to image detection and recognition devices and detection and recognition methods thereof.
  • the immersive smart TV light strip is a product that integrates smart lighting and smart cameras. It mainly obtains the color of the TV screen in real time through the camera, and then controls the light strip installed on the back of the TV to display the corresponding color, achieving an immersive viewing experience.
  • TV screen area recognition is mainly achieved through manual calibration of currently marketed products, that is, through visual images, users manually complete area marking through interface interaction.
  • Embodiments of the present application provide an image detection and recognition device and a detection and recognition method thereof, which have higher accuracy, thereby optimizing the detection and recognition of the TV screen display area, realizing real-time screen color picking, and being easy to operate and improving user experience. Feel.
  • An image detection and recognition device provided by an embodiment of the present application is used in home intelligent equipment, including: a priori information storage module, a camera module, an image processing module, and a communication module; wherein:
  • a priori information storage module used to save the prior information required when the image detection and recognition device is first activated, and transmit the prior information to the image processing module when the image detection and recognition device is first activated, to causing the image processing module to determine the position information of the rectangular display area according to the a priori information;
  • a camera module configured to acquire image data of a rectangular display area and transmit the image data to the image processing module, so that the image processing module determines the position information of the rectangular display area according to the image data;
  • An image processing module used to obtain the position information of the rectangular display area through an inter-frame relative difference algorithm
  • Communication module used for information interaction between the image detection and recognition device and the host or terminal, connected to the host or terminal in a wired connection or wireless connection, so that various hosts or terminals can connect to the image detection and recognition device
  • the device has image detection and recognition functions.
  • the camera module acquires image data of the rectangular display area through a fisheye camera installed close to the midpoint of either side of the rectangular display area.
  • the a priori information includes the first vertex coordinate, the second vertex coordinate, the first midpoint coordinate and the area where the opposite side is located on the installation edge; wherein the installation edge is the rectangular display area close to the installation location.
  • the opposite side is the side opposite to the installation side in the rectangular display area
  • the first midpoint is the midpoint between the first vertex and the second vertex of the installation side.
  • An image detection and recognition method provided by an embodiment of the present application is applied to any of the above image detection and recognition devices, including:
  • Obtain a priori information which includes the first vertex coordinate, the second vertex coordinate, the first midpoint coordinate and the area of the opposite side located on the installation edge; wherein the installation edge is the rectangular display area close to the installation The side of the fisheye camera, the opposite side is the side opposite to the installation side in the rectangular display area, and the first midpoint is the midpoint between the first vertex and the second vertex of the installation side. ;
  • first image data and second image data of the rectangular display area where the first image data and second image data are image data of the rectangular display area in different brightness states;
  • the third midpoint coordinate is determined by the first vertex coordinate and the third vertex coordinate
  • the fourth midpoint coordinate is determined by the second vertex coordinate and the fourth vertex coordinate
  • the third midpoint coordinate is the coordinate of the midpoint between the first vertex and the third vertex
  • the fourth midpoint coordinate is the coordinate of the midpoint between the second vertex and the fourth vertex
  • the first vertex coordinate, the second vertex coordinate, the third vertex coordinate, the fourth vertex coordinate, the first midpoint coordinate, the second midpoint coordinate, the third midpoint coordinate Point coordinates and the fourth midpoint coordinate are determined as position information of the rectangular display area.
  • the obtaining a priori information includes:
  • the first vertex coordinate, the second vertex coordinate, the first midpoint coordinate and the area where the opposite edge is located are determined based on the annotation results of multiple sample images, and are used as prior information.
  • the obtaining the first image data and the second image data of the rectangular display area includes:
  • the first image data and the second image data of the rectangular display area in different brightness states are obtained through interaction with the user terminal.
  • the inter-frame relative difference algorithm is applied to the first image data and the second image data to confirm the third vertex coordinates, the fourth vertex coordinates and the second center coordinates of the opposite sides of the rectangular display area.
  • Point coordinates include:
  • the area where the pixels become larger is marked as a white pixel area, and the area where the pixels become smaller is marked as a black pixel area, and the difference is obtained image;
  • the first ratio reaches the preset first condition, confirm that the ordinate of the current row is the ordinate of the third vertex and the fourth vertex of the opposite side.
  • the third vertex and The abscissa coordinate of the fourth vertex is the abscissa coordinate of the two sides of the rectangular frame perpendicular to the X-axis, and the third vertex coordinate and the fourth vertex coordinate are obtained;
  • Central processing unit memory and input and output interfaces
  • the memory is a short-term storage memory or a persistent storage memory
  • the central processing unit is configured to communicate with the memory and execute instruction operations in the memory to perform the aforementioned image detection and recognition method.
  • An embodiment of the present application provides a computer-readable storage medium that includes instructions. When the instructions are run on a computer, they cause the computer to execute the aforementioned image detection and recognition method.
  • the image data of the rectangular display area is obtained through the camera module, and the image processing module is used to obtain the position information of the rectangular display area through the inter-frame relative difference algorithm and combined with manual calibration prior information, which has higher accuracy, thereby optimizing the TV screen.
  • the detection and identification of the display area enables real-time screen color selection and is easy to operate, improving the user experience.
  • Figure 1 is a structural diagram of an image detection and recognition device provided by an embodiment of the present application.
  • Figure 2 is a schematic diagram of an implementation of the image detection and recognition method provided by the embodiment of the present application.
  • Figure 3 is a schematic structural diagram of an image detection and recognition device provided by an embodiment of the present application.
  • Figure 4 is an interactive interface of a user terminal provided by an embodiment of the present application.
  • Figure 5 is a screenshot of the monitoring screen of the fisheye camera provided by the embodiment of the present application and a screenshot of the corresponding interactive interface of the user terminal;
  • Figure 6 is an interactive interface of a user terminal when the television set provided by the embodiment of the present application is turned on;
  • Figure 7 is an interactive interface of a user terminal when the television set is turned off according to an embodiment of the present application
  • Figure 8 is a screenshot of the television provided by the embodiment of the present application displayed on the user terminal in the off state;
  • Figure 9 is a screenshot of the television provided by the embodiment of the present application displayed on the user terminal when it is turned on;
  • Figure 10 is a differential image provided by an embodiment of the present application.
  • Embodiments of the present application provide an image detection and recognition device and a detection and recognition method thereof, which have higher accuracy, thereby optimizing the detection and recognition of the TV screen display area, realizing real-time screen color picking, and being easy to operate and improving user experience. Feel.
  • the immersive smart TV light strip is a product that integrates smart lighting and smart cameras. It mainly obtains the color of the TV screen in real time through the camera, and then controls the light strip installed on the back of the TV to display the corresponding color, achieving an immersive viewing experience.
  • TV screen area recognition is mainly achieved through manual calibration of currently marketed products, that is, through visual images, users manually complete area marking through interface interaction.
  • an implementation of the image detection and recognition device provided by the embodiment of the present application includes: a priori information storage module 101 , camera module 102, image processing module 103 and communication module 104. in:
  • the prior information storage module 101 is used to save the prior information required when the image detection and recognition device is first activated, and transmit the prior information to the image processing module when the image detection and recognition device is first activated, so that the image processing module can
  • the prior information determines the position information of the rectangular display area.
  • the display area should be detected, identified and calibrated when it is first used after installation. It is worth noting that in the embodiment of this application
  • the rectangular display area is used to refer to the rectangular display area of television screens and other devices with display functions.
  • the initial activation in the embodiment of the present application is not limited to the first time the device is turned on, and should also include the system reset of the device. placement situation.
  • a priori information is some information or parameters obtained by the technical developers of the device through multiple experiments during the device design and manufacturing phase to enable the device to perform its functions. It is pre-stored in the a priori information storage module 101 during the device production phase. , in order to achieve the technical effect of this device.
  • this device acquires images of TV screens of multiple sizes in advance during the design and development stage, specifically by selecting TVs covering 35-70 inches in size.
  • a designated area such as the top edge or bottom edge of the TV screen close to the midpoint
  • the two vertices of the installation edge of the camera always fall through the camera
  • the specific coordinate positions of these two vertices and the midpoint of the installation edge are obtained in this way and set to fixed coordinate values that are pre-stored in the prior information storage module 101 middle.
  • the developers also found that the bottom edge of the TV screen can still fall within a fixed range after being deformed by the camera imaging, and this range can be represented by a rectangular frame, so this rectangle
  • the position of the frame is also stored in the prior information storage module 101 as prior information.
  • the camera module 102 is used to obtain image data of the rectangular display area and transmit the image data to the image processing module, so that the image processing module determines the position information of the rectangular display area according to the image data.
  • the image processing module 103 is used to obtain the position information of the rectangular display area through the inter-frame relative difference algorithm.
  • the camera module 102 captures multiple frame images in different brightness states, and uses an inter-frame difference algorithm to obtain the relative changes between different frames. Since the multiple frame images acquired in this embodiment contain The pixels include both the TV screen part and the environment part where the TV is placed. Therefore, when adjusting the screen brightness of the TV, there is a relative change trend in the pixel changes at the same position in each frame of image. In this embodiment, through this A relative change trend is used to obtain the position information of the rectangular display area. The method used is improved based on the relative change trend compared to the inter-frame difference algorithm known in the industry, so the "inter-frame relative difference algorithm" is used in this application. The description method reflects the specific implementation process.
  • the communication module 104 is used for information interaction between the image detection and recognition device and the host or terminal: it is provided with one or more interfaces including USB and serial SPI bus, and is connected to the host or terminal in a wired connection or a wireless connection. This allows various hosts or terminals with different interfaces to be connected to the image detection and recognition device and have image detection and recognition functions.
  • the image data of the rectangular display area is obtained through the camera module 102, and the image processing module 103 is used to obtain the position information of the rectangular display area through the inter-frame relative difference algorithm and combined with manually calibrated prior information, which has higher accuracy. , thereby optimizing the detection and identification of the TV screen display area, realizing real-time screen color selection, and easy operation, improving the user experience.
  • Another implementation of the image detection and recognition device provided by the embodiment of the present application includes: a priori information storage module 101, a camera module 102, an image processing module 103, and a communication module 104. in:
  • the prior information storage module 101 is used to save the prior information required when the image detection and recognition device is first activated, and transmit the prior information to the image processing module when the image detection and recognition device is first activated, so that the image processing module can
  • the prior information determines the position information of the rectangular display area.
  • the display area should be detected, identified and calibrated when it is first used after installation. It is worth noting that in the embodiment of this application
  • the rectangular display area is used to refer to the rectangular display area of television screens and other devices with display functions.
  • the initial activation in the embodiment of the present application is not limited to the first time the device is turned on, and should also include the system reset of the device. placement situation.
  • a priori information is some information or parameters obtained by the technical developers of the device through multiple experiments during the device design and manufacturing phase to enable the device to perform its functions. It is pre-stored in the a priori information storage module 101 during the device production phase. , in order to achieve the technical effect of this device.
  • this device acquires images of TV screens of multiple sizes in advance during the design and development stage, specifically by selecting TVs covering 35-70 inches in size.
  • a designated area such as the top edge or bottom edge of the TV screen close to the midpoint
  • the two vertices of the installation edge of the camera always fall through the camera
  • the specific coordinate positions of these two vertices and the midpoint of the installation edge are obtained in this way and set to fixed coordinate values that are pre-stored in the prior information storage module 101 middle.
  • the developers also found that the bottom edge of the TV screen can still fall within a fixed range after being deformed by the camera imaging, and this range can be represented by a rectangular frame, so this rectangle
  • the position of the frame is also stored in the prior information storage module 101 as prior information.
  • the camera module 102 is used to obtain image data of the rectangular display area and transmit the image data to the image processing module, so that the image processing module determines the position information of the rectangular display area according to the image data.
  • the camera module 102 in this device uses a fish-eye camera to obtain a 180° wide-angle monitoring image, so that when the device is installed on the top or bottom edge of the TV screen, the image of the TV screen can still be fully reflected on the in the surveillance screen.
  • the camera module 102 may also have other wide-angle cameras that can achieve the same technical effect, and the details are not limited here.
  • the user can interact through the user terminal, so that the device can obtain specific installation position information, so as to facilitate the subsequent steps of detecting the TV display area.
  • the specific interactive interface of the user terminal can be shown in Figure 4 below.
  • Figure 5 above is a screenshot of the surveillance screen obtained by installing the fisheye camera near the midpoint of the top edge of the 70-inch TV.
  • the difference between the two methods is that the vertex position of the TV set is different.
  • the vertex position can be transformed by rotation.
  • the positive direction of the X-axis in the system is the abscissa.
  • Vertically downward along the upper left corner of the image is the positive direction of the Y-axis in the computer coordinate system, that is, the ordinate.
  • the third step is to obtain the three coordinate points on the left, center and right of the bottom of the TV, as marked in the lower left corner and lower right corner in Figure 5.
  • the black frame in the picture can determine the upper and lower range and length range of the bottom of the TV. After calibration and measurement, it can be confirmed that the bottom of different 35-70-inch TVs can fall within the black frame;
  • the midpoint of the left and right borders of the TV is calculated from the midpoint of the line connecting the upper and lower vertices of the left and right sides, as shown in Figure 5.
  • the actual midpoints of the left and right sides of the TV are marked;
  • the position of the TV in the camera frame can be initially determined, providing support for subsequent algorithms to detect more accurate vertex coordinates.
  • the image processing module 103 is configured to obtain the position information of the rectangular display area through an inter-frame relative difference algorithm.
  • the camera module 102 captures multiple frame images in different brightness states, and uses an inter-frame difference algorithm to obtain the relative changes between different frames. Since the multiple frame images acquired in this embodiment contain The pixels include both the TV screen part and the environment part where the TV is placed. Therefore, when adjusting the screen brightness of the TV, there is a relative change trend in the pixel changes at the same position in each frame of image. In this embodiment, through this A relative change trend is used to obtain the position information of the rectangular display area. The method used is improved based on the relative change trend compared to the inter-frame difference algorithm known in the industry, so the "inter-frame relative difference algorithm" is used in this application. The description method reflects the specific implementation process.
  • two frames of images of the TV in the on and off states can be obtained through user interaction on the user terminal to provide image data for subsequent algorithms.
  • the interactive interface is shown in Figures 6 and 7 below.
  • the area where the pixels in the image become larger is marked as a white area, and vice versa. It is a black area, and the final difference image is shown in Figure 10.
  • the large white area of the image is the TV screen area, and the other black areas are the background area outside the TV.
  • the approximate area of the TV can be quickly located.
  • the left and right vertices and midpoints of the top of the TV are all manually calibrated using manual calibration data, and then combined with the inter-frame relative difference algorithm to locate the bottom vertex of the TV.
  • the communication module 104 is used for information interaction between the image detection and recognition device and the host or terminal, and is connected to the host or terminal through a wired connection or a wireless connection, so that various hosts or terminals can connect to the image detection and recognition device.
  • Image detection and recognition function is used for information interaction between the image detection and recognition device and the host or terminal, and is connected to the host or terminal through a wired connection or a wireless connection, so that various hosts or terminals can connect to the image detection and recognition device.
  • Image detection and recognition function is used for information interaction between the image detection and recognition device and the host or terminal, and is connected to the host or terminal through a wired connection or a wireless connection, so that various hosts or terminals can connect to the image detection and recognition device.
  • the camera module 102 acquires the image data of the rectangular display area through a fisheye camera, so that the TV screen can appear completely in the monitoring screen.
  • the image processing module 103 is used to use the inter-frame relative difference algorithm combined with manual calibration.
  • the position information of the rectangular display area is obtained through the test information with higher accuracy, thereby optimizing the detection and identification of the TV screen display area, realizing real-time screen color picking, and specifically embodying the implementation process of user terminal interaction. It can be seen that It is easy to operate and can improve the user's experience.
  • an implementation of the image detection and recognition method provided by the embodiment of the present application includes steps 201 to 205 .
  • the prior information includes the first vertex coordinate, the second vertex coordinate, the first midpoint coordinate and the area where the opposite side is located on the installation side; where the installation side is the rectangular display area close to the side where the fisheye camera is installed, and the opposite side is the rectangular display area.
  • the first midpoint is the midpoint between the first vertex and the second vertex of the mounting side.
  • sample image data are monitoring images captured by a fisheye camera on rectangular display areas of different sizes.
  • Establish a target coordinate system based on the sample image with the upper left vertex of the sample image as the origin of the coordinates, horizontally to the right along the upper left vertex of the image as the positive direction of the X axis, and vertically downward along the upper left vertex of the image as the positive direction of the Y axis.
  • the midpoint of the top edge of the sample image is first determined as the first midpoint, and the coordinates of the first midpoint are determined based on its specific position in the target coordinate system.
  • multiple sample images are enlarged one by one. The developer observes the enlarged sample images with the naked eye and determines the specific positions of the first vertex and the second vertex of the installation edge in each sample image and annotates them. The first vertex coordinates and the second vertex coordinates in the multiple sample images are determined in combination with the target coordinate system.
  • the first vertex coordinates in the multiple sample images are formed into a first vertex coordinate set
  • the second vertex coordinates in the multiple sample images are formed into a second vertex coordinate set
  • the first vertex coordinate set is averaged to obtain the average value
  • the coordinates are determined as the first vertex coordinates obtained through manual calibration in the prior information. Similarly, this method can also be used to obtain the second vertex coordinates in the prior information.
  • the developers also found that the bottom edge of the TV screen can still fall within a fixed range after being deformed by the camera imaging, and this range can be represented by a rectangular box, so this rectangular box The location is also used as prior information
  • the first image data and the second image data are image data of the rectangular display area in different brightness states.
  • the fisheye camera can be controlled by interacting with the user terminal to obtain the first image data and the second image data of the rectangular display area in different brightness states.
  • the area where the pixels become larger is marked as a white pixel area, and the area where the pixels become smaller is marked as a black pixel area, thereby obtaining a differential image.
  • the abscissa coordinate is the abscissa coordinate of the two sides of the rectangular frame perpendicular to the X-axis, and the third and fourth vertex coordinates are obtained.
  • the first ratio reaches the preset second condition, confirm that the ordinate of the current row is the ordinate of the second midpoint of the opposite side, take the abscissa of the third vertex and the The middle value of the abscissa coordinates of the four vertices is used as the abscissa coordinate of the second midpoint, and the second midpoint coordinate is obtained.
  • the preset first condition may be: during the traversal process, when the proportion of white pixels is less than 90% twice in a row.
  • the preset second condition may be: during the traversal process, when the proportion of white pixels is less than 60% twice in a row.
  • the third midpoint coordinate is the coordinate of the midpoint between the first vertex and the third vertex
  • the fourth midpoint coordinate is the coordinate of the midpoint between the second vertex and the fourth vertex.
  • the image data of the rectangular display area is obtained through a fisheye camera, so that the TV screen can appear completely in the monitoring screen.
  • the position of the rectangular display area is obtained through the inter-frame relative difference algorithm and combined with the prior information of manual calibration. Information has higher accuracy, thereby optimizing the detection and identification of the TV screen display area, realizing real-time screen color picking, and improving the user experience.
  • FIG 3 is a schematic structural diagram of an image detection and recognition device provided by an embodiment of the present application.
  • the image detection and recognition device 300 may include one or more central processing units (CPU) 301 and a memory 305.
  • the memory 305 stores a or more than one application or data.
  • the memory 305 can be volatile storage or persistent storage.
  • the program stored in the memory 305 may include one or more modules, and each module may include a series of instruction operations in the image detection and recognition device.
  • the central processing unit 301 may be configured to communicate with the memory 305 and execute a series of instruction operations in the memory 305 on the image detection and recognition device 300 .
  • the image detection and recognition device 300 may also include one or more power supplies 302, one or more wired or wireless network interfaces 303, one or more input and output interfaces 304, and/or, one or more operating systems, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
  • one or more operating systems such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
  • the central processing unit 301 can perform the operations performed by the image detection and recognition device in the embodiment shown in FIGS. 1 to 2 , and the details will not be described again here.
  • the disclosed systems, devices and methods can be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components may be combined or can be integrated into another system, or some features can be ignored, or not implemented.
  • the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application can be integrated into one processing unit, each unit can exist physically alone, or two or more units can be integrated into one unit.
  • the above integrated units can be implemented in the form of hardware or software functional units.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium.
  • the technical solution of the present application is essentially or contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), magnetic disk or optical disk and other media that can store program code. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

本申请实施例公开了图像检测识别装置及其检测识别方法，用于优化电视机屏幕显示区域的检测和识别，实现实时屏幕取色，从而控制灯带颜色变化，提高用户的使用感受。本申请实施例方法包括：先验信息保存模块、摄像模块、图像处理模块、通讯模块；其中：先验信息保存模块，用于保存所述图像检测识别装置在初次启用时所需的先验信息，且在所述图像检测识别装置初次启用时将所述先验信息传输至图像处理模块；摄像模块，用于获取矩形显示区域的图像数据，并将所述图像数据传输至所述图像处理模块；图像处理模块，用于通过帧间相对差算法获取所述矩形显示区域的位置信息；通讯模块，用于所述图像检测识别装置与主机或终端之间的信息交互。

Description

图像检测识别装置及其检测识别方法 技术领域
本申请实施例涉及物联网智能家居领域，尤其涉及图像检测识别装置及其检测识别方法。
背景技术
随着物联网家用智能设备的普及，涌现出越来越多的智能应用场景。沉浸式智能电视机灯带就是一种融合智能照明、智能摄像机的产品。它主要通过摄像机实时获取电视机屏幕颜色，再控制安装在电视机背面的灯带显示对应颜色，实现一种沉浸式观影体验。
此类产品中主要技术可以概括为三部分，即电视机屏幕区域识别、实时屏幕取色、实时灯带控制。电视屏幕区域识别，目前上市产品主要通过人工标定来实现，即通过可视画面，用户通过界面交互手动完成区域标记。
但是这种标定过程在人为交互过程中会导致产品操作复杂、步骤冗余，用户体验差且还有可能标定结果不准确。
发明内容
本申请实施例提供了图像检测识别装置及其检测识别方法，具有更高的准确度，从而实现优化电视机屏幕显示区域的检测和识别，实现实时屏幕取色，且操作简便，提高用户的使用感受。
本申请实施例提供的一种图像检测识别装置，应用于家居智能设备，包括：先验信息保存模块、摄像模块、图像处理模块、通讯模块；其中：
先验信息保存模块，用于保存所述图像检测识别装置在初次启用时所需的先验信息，且在所述图像检测识别装置初次启用时将所述先验信息传输至图像处理模块，以使得所述图像处理模块根据所述先验信息确定矩形显示区域的位置信息；
摄像模块，用于获取矩形显示区域的图像数据，并将所述图像数据传输至所述图像处理模块，以使得所述图像处理模块根据所述图像数据确定所述矩形显示区域的位置信息；
图像处理模块，用于通过帧间相对差算法获取所述矩形显示区域的位置信息；
通讯模块，用于所述图像检测识别装置与主机或终端之间的信息交互，以有线连接方式或无线连接方式连接于主机或终端，以使得各种主机或终端都能连接所述图像检测识别装置而具备图像检测识别功能。
可选的，所述摄像模块通过安装于靠近所述矩形显示区域的任一边的中点位置的鱼眼摄像头获取矩形显示区域的图像数据。
可选的，通过人工标定的方式获取先验信息，并将所述先验信息保存在所述先验信息保存模块中。
可选的，所述先验信息包括位于安装边的第一顶点坐标、第二顶点坐标、第一中点 坐标以及相对边所在区域；其中，所述安装边为所述矩形显示区域靠近安装所述鱼眼摄像头的边，所述相对边为所述矩形显示区域中与所述安装边相对的边，第一中点为所述安装边的第一顶点与第二顶点之间的中点。
本申请实施例提供的一种图像检测识别方法，应用于上述任一项所述的图像检测识别装置，包括：
获取先验信息，所述先验信息包括位于安装边的第一顶点坐标、第二顶点坐标、第一中点坐标以及相对边所在区域；其中，所述安装边为所述矩形显示区域靠近安装所述鱼眼摄像头的边，所述相对边为所述矩形显示区域中与所述安装边相对的边，第一中点为所述安装边的第一顶点与第二顶点之间的中点；
获取矩形显示区域的第一图像数据和第二图像数据，所述第一图像数据和所述第二图像数据为所述矩形显示区域在不同亮度状态下的图像数据；
对所述第一图像数据和所述第二图像数据运用帧间相对差算法，确认所述矩形显示区域的相对边的第三顶点坐标、第四顶点坐标和第二中点坐标，其中第二中点为矩形显示区域位于相对边的中点；
结合所述先验数据，通过所述第一顶点坐标与所述第三顶点坐标确定第三中点坐标，通过所述第二顶点坐标与所述第四顶点坐标确定第四中点坐标；其中，所述第三中点坐标为第一顶点与第三顶点之间的中点的坐标，所述第四中点坐标为第二顶点与第四顶点之间的中点的坐标；
将所述第一顶点坐标、所述第二顶点坐标、所述第三顶点坐标、所述第四顶点坐标、所述第一中点坐标、所述第二中点坐标、所述第三中点坐标以及所述第四中点坐标确定为所述矩形显示区域的位置信息。
可选的，所述获取先验信息包括：
获取多份样本图像数据，所述样本图像数据由所述鱼眼摄像头对不同尺寸的矩形显示区域进行拍摄得到；
根据样本图像建立目标坐标系，以样本图像的左上角顶点为坐标原点，沿着图像左上角顶点水平向右为X轴正方向，沿着图像左上角顶点垂直向下为Y轴正方向；
通过人工标定的方法在所述目标坐标系上标注每一个样本图像中位于安装边的第一顶点坐标、第二顶点坐标、第一中点坐标以及相对边所在区域；
根据多份样本图像的标注结果确定第一顶点坐标、第二顶点坐标、第一中点坐标以及相对边所在区域，并作为先验信息。
可选的，所述获取矩形显示区域的第一图像数据和第二图像数据包括：
通过与用户终端交互的形式获取所述矩形显示区域在不同亮度状态下的第一图像数据和第二图像数据。
可选的，所述对所述第一图像数据和所述第二图像数据运用帧间相对差算法，确认所述矩形显示区域的相对边的第三顶点坐标、第四顶点坐标和第二中点坐标包括：
根据所述第一图像数据和所述第二图像数据在不同亮度状态下的相对像素变化，将像素变大的区域标记为白色像素区域，将像素变小的区域标记为黑色像素区域，得到差分图像；
根据所述先验信息在所述差分图像上标注相对边所在区域；其中，相对边所在区域以矩形框表示，且矩形框内包括部分白色像素区域和部分黑色像素区域；
以矩形框左上角顶点为起点，一个像素点作为基本单位，每次统计矩形框内水平方向一行像素点的数量，且计算白色像素点数量在一行像素点总数中所占的第一比例；
沿Y轴方向遍历矩形框，当所述第一比例达到预设的第一条件时，确认当前行的纵坐标为相对边的第三顶点和第四顶点的纵坐标，所述第三顶点和所述第四顶点的横坐标为矩形框垂直于X轴的两条边的横坐标，得到所述第三顶点坐标和所述第四顶点坐标；
继续沿Y轴方向遍历矩形框，当所述第一比例达到预设的第二条件时，确认当前行的纵坐标为位于相对边的第二中点的纵坐标，取第三顶点的横坐标与第四顶点的横坐标的中间值作为所述第二中点的横坐标，得到所述第二中点坐标。
本申请实施例提供的一种图像检测识别设备，其特征在于，包括：
中央处理器，存储器以及输入输出接口；
所述存储器为短暂存储存储器或持久存储存储器；
所述中央处理器配置为与所述存储器通信，并执行所述存储器中的指令操作以执行前述的图像检测识别方法。
本申请实施例提供的一种计算机可读存储介质，包括指令，当所述指令在计算机上运行时，使得计算机执行前述的图像检测识别方法。
从以上技术方案可以看出，本申请实施例具有以下优点：
通过摄像模块获取矩形显示区域的图像数据，采用图像处理模块通过帧间相对差算法并结合人工标定的先验信息获取矩形显示区域的位置信息，具有更高的准确度，从而实现优化电视机屏幕显示区域的检测和识别，实现实时屏幕取色，且操作简便，提高用户的使用感受。
附图说明
图1为本申请实施例提供的图像检测识别装置的组成架构图；
图2为本申请实施例提供的图像检测识别方法的一种实施方式的示意图；
图3为本申请实施例提供的图像检测识别装置的结构示意图；
图4为本申请实施例提供的用户终端的交互界面；
图5为本申请实施例提供的鱼眼摄像头的监控画面截图和对应的用户终端的交互界面截图；
图6为本申请实施例提供的电视机在打开状态下用户终端的交互界面；
图7为本申请实施例提供的电视机在关闭状态下用户终端的交互界面；
图8为本申请实施例提供的电视机在关闭状态下在用户终端显示的截图；
图9为本申请实施例提供的电视机在打开状态下在用户终端显示的截图；
图10为本申请实施例提供的差分图像。
具体实施方式
本申请实施例提供了图像检测识别装置及其检测识别方法，具有更高的准确度，从 而实现优化电视机屏幕显示区域的检测和识别，实现实时屏幕取色，且操作简便，提高用户的使用感受。
随着物联网家用智能设备的普及，涌现出越来越多的智能应用场景。沉浸式智能电视机灯带就是一种融合智能照明、智能摄像机的产品。它主要通过摄像机实时获取电视机屏幕颜色，再控制安装在电视机背面的灯带显示对应颜色，实现一种沉浸式观影体验。
此类产品中主要技术可以概括为三部分，即电视机屏幕区域识别、实时屏幕取色、实时灯带控制。电视屏幕区域识别，目前上市产品主要通过人工标定来实现，即通过可视画面，用户通过界面交互手动完成区域标记。
但是这种标定过程在人为交互过程中会导致产品操作复杂、步骤冗余，用户体验差且还有可能标定结果不准确。
基于此，本申请实施例提供了一种图像检测识别装置，应用于家居智能设备，请参阅图1，本申请实施例提供的图像检测识别装置的一种实施方式包括：先验信息保存模块101、摄像模块102、图像处理模块103以及通讯模块104。其中：
先验信息保存模块101，用于保存图像检测识别装置在初次启用时所需的先验信息，且在图像检测识别装置初次启用时将先验信息传输至图像处理模块，以使得图像处理模块根据先验信息确定矩形显示区域的位置信息。
结合本装置的使用场景，当本装置用于检测电视机或者其他具有显示器功能的设备时，应该在安装后初次启用时进行显示区域的检测识别和校准，值得注意的是，本申请实施例中矩形显示区域用于指代电视机屏幕等其他具有显示器功能的设备的矩形显示区域，本申请实施例中的初次启用也不局限于本装置第一次开机使用，还应当包括本装置进行系统重置的情况。
先验信息是由本装置的技术开发人员在装置设计制造阶段通过多次实验得到的用于实现本装置执行其功能的一些信息或参数，在装置生产阶段将其预存于先验信息保存模块101中，以便实现本装置的技术效果。
具体地，为了实现检测识别到矩形显示区域的位置信息的技术效果，本装置通过在设计开发阶段预先对多个尺寸的电视机屏幕进行图像获取，具体到通过选取覆盖35-70寸大小的电视机屏幕作为样本，通过实验验证当本装置的摄像头安装于指定区域时(如电视机屏幕的顶边或者底边接近中点的位置)，摄像头的安装边的两个顶点总是落在通过摄像头看到的监控画面图像中的一定范围内，所以这两个顶点，以及安装边的中点的具体坐标位置就通过这种方式获取并设定为固定的坐标值预存在先验信息保存模块101中。且在开发试验阶段，开发人员还发现电视机屏幕的底边在经过摄像头成像存在变形的情况后，仍然可以落在一个固定的范围内，且这个范围可用一个矩形框来表示，所以将这个矩形框的位置也作为先验信息保存在先验信息保存模块101中。
摄像模块102，用于获取矩形显示区域的图像数据，并将图像数据传输至图像处理模块，以使得图像处理模块根据图像数据确定矩形显示区域的位置信息。
图像处理模块103，用于通过帧间相对差算法获取矩形显示区域的位置信息。
通过摄像模块102拍摄不同亮度状态下的多帧图像，对这多帧图像采用帧间差算法去 获取到不同帧之间的相对变化情况，由于本实施例中所获取的多帧图像中所包含的像素点既包括电视机屏幕部分，也包括放置电视机的环境部分，所以当调节电视机的屏幕亮度时，每一帧图像的相同位置的像素变化存在相对变化趋势，本实施例中通过这种相对变化趋势从而获得矩形显示区域的位置信息，所采用的方法相对于行业内所公知的帧间差算法进行了基于相对变化趋势的改进，所以在本申请中采用“帧间相对差算法”的描述方式来体现具体的实现过程。
通讯模块104，用于图像检测识别装置与主机或终端之间的信息交互：设有包括USB、串口SPI总线中的一个或多个接口，以有线连接方式或无线连接方式连接与主机或终端，以使得具备不同接口的各种主机或终端都能连接图像检测识别装置而具备图像检测识别功能。
本实施例中，通过摄像模块102获取矩形显示区域的图像数据，采用图像处理模块103通过帧间相对差算法并结合人工标定的先验信息获取矩形显示区域的位置信息，具有更高的准确度，从而实现优化电视机屏幕显示区域的检测和识别，实现实时屏幕取色，且操作简便，提高用户的使用感受。
本申请实施例提供的图像检测识别装置的另一种实施方式包括：先验信息保存模块101、摄像模块102、图像处理模块103以及通讯模块104。其中：
先验信息保存模块101，用于保存图像检测识别装置在初次启用时所需的先验信息，且在图像检测识别装置初次启用时将先验信息传输至图像处理模块，以使得图像处理模块根据先验信息确定矩形显示区域的位置信息。
结合本装置的使用场景，当本装置用于检测电视机或者其他具有显示器功能的设备时，应该在安装后初次启用时进行显示区域的检测识别和校准，值得注意的是，本申请实施例中矩形显示区域用于指代电视机屏幕等其他具有显示器功能的设备的矩形显示区域，本申请实施例中的初次启用也不局限于本装置第一次开机使用，还应当包括本装置进行系统重置的情况。
先验信息是由本装置的技术开发人员在装置设计制造阶段通过多次实验得到的用于实现本装置执行其功能的一些信息或参数，在装置生产阶段将其预存于先验信息保存模块101中，以便实现本装置的技术效果。
具体地，为了实现检测识别到矩形显示区域的位置信息的技术效果，本装置通过在设计开发阶段预先对多个尺寸的电视机屏幕进行图像获取，具体到通过选取覆盖35-70寸大小的电视机屏幕作为样本，通过实验验证当本装置的摄像头安装于指定区域时(如电视机屏幕的顶边或者底边接近中点的位置)，摄像头的安装边的两个顶点总是落在通过摄像头看到的监控画面图像中的一定范围内，所以这两个顶点，以及安装边的中点的具体坐标位置就通过这种方式获取并设定为固定的坐标值预存在先验信息保存模块101中。且在开发试验阶段，开发人员还发现电视机屏幕的底边在经过摄像头成像存在变形的情况后，仍然可以落在一个固定的范围内，且这个范围可用一个矩形框来表示，所以将这个矩形框的位置也作为先验信息保存在先验信息保存模块101中。
摄像模块102，用于获取矩形显示区域的图像数据，并将图像数据传输至图像处理模块，以使得图像处理模块根据图像数据确定矩形显示区域的位置信息。
本装置中摄像模块102采用鱼眼摄像头以获取到180°广角的监控画面，使得当本装置安装于电视机屏幕的顶端或底边上时，仍能将电视机屏幕的画面较为完整地体现在监控画面中。摄像模块102也可以有其他能实现同样技术效果的广角摄像头，具体此处不做限定。
在摄像头安装时，用户可以通过用户终端进行交互，以使得本装置获取到具体的安装位置信息，以便于后续检测电视机显示区域的步骤实现。具体的用户终端的交互界面可以如下图4所示。
例如，当鱼眼摄像头安装在电视机屏幕上方时，摄像头所拍摄的监控画面截图和对应的安装方式在用户终端的界面截图如图5所示。
如上图5为将鱼眼摄像头安装于70寸电视机顶边接近中点的位置所获取到的监控画面截图。两种方式的区别在于获取的电视机顶点位置不同，具体计算中顶点位置通过旋转变换即可。在人工标定获取先验信息的过程中，首先通过以上截图画面建立坐标系，其中图像左上角顶点为计算机坐标系中的原点(0，0)，沿着图像左上角顶点水平往右为计算机坐标系中的X轴正方向，即横坐标。沿着图像左上角垂直往下为计算机坐标系中的Y轴正方向即纵坐标。
具体步骤如下：
将摄像头按照产品使用规则，安装在不同尺寸电视机顶部后，获取监控画面截图；
首先确定鱼眼图像顶部中点坐标，如图5中标注的实际电视机顶部中点；
再获取电视机顶部左右两个顶点坐标，如图5中左上角和右上角标注；
第三步获取电视机底部左中右三个坐标点，如图5中左下角和右下角标注。图中黑色框可以确定电视机底边的上下范围和长度范围，经过标定测量可确认不同35-70寸的电视机底边都能落在黑色边框内部；
电视机左右边框的中点，由左右两边的上下顶点连线中点计算得到，如图5中标注的实际电视机左、右边中点；
经过以上标定步骤，可以初步确定电视机在摄像头画面中的位置，为后续算法检测更精确顶点坐标提供支持。
图像处理模块103，用于通过帧间相对差算法获取所述矩形显示区域的位置信息。
通过摄像模块102拍摄不同亮度状态下的多帧图像，对这多帧图像采用帧间差算法去获取到不同帧之间的相对变化情况，由于本实施例中所获取的多帧图像中所包含的像素点既包括电视机屏幕部分，也包括放置电视机的环境部分，所以当调节电视机的屏幕亮度时，每一帧图像的相同位置的像素变化存在相对变化趋势，本实施例中通过这种相对变化趋势从而获得矩形显示区域的位置信息，所采用的方法相对于行业内所公知的帧间差算法进行了基于相对变化趋势的改进，所以在本申请中采用“帧间相对差算法”的描述方式来体现具体的实现过程。
本实施例中，可以通过用户在用户终端交互的方式获取电视机在打开和关闭状态下的两帧图像，为后续算法提供图像数据，交互界面如下图6、图7所示。
获取到的电视机在关、开状态下的两帧图像在用户终端显示的截图如下图8、图9所示，将两幅图片做帧间差分，获取差分图像如下图10。
差分过程中，利用电视机从关到开的状态，电视机屏幕区域亮度必然增大而背景区域亮度必然减小这一相对变化趋势，将图像中像素变大的区域标记为白色区域，反之标记为黑色区域，最终差分图像如图10所示。其中图像的大块白色区域为电视机屏幕区域，其他黑色区域为电视机以外的背景区域。结合先验信息可以快速定位电视机大概区域，本实施例中的电视机顶部的左、右顶点和中点都使用人工标定数据，继而结合帧间相对差算法，定位电视机底部顶点。
通讯模块104，用于图像检测识别装置与主机或终端之间的信息交互，以有线连接方式或无线连接方式连接与主机或终端，以使得各种主机或终端都能连接图像检测识别装置而具备图像检测识别功能。
本实施例中，摄像模块102通过鱼眼摄像头获取矩形显示区域的图像数据，使得电视机屏幕能够完整地出现在监控画面中，采用图像处理模块103通过帧间相对差算法并结合人工标定的先验信息获取矩形显示区域的位置信息，具有更高的准确度，从而实现优化电视机屏幕显示区域的检测和识别，实现实时屏幕取色，并且具体体现了用户终端交互的实现过程，可看出其操作简便，能够实现提高用户的使用感受。
请参阅图2，本申请实施例提供的图像检测识别方法的一种实施方式包括步骤201至步骤205。
201、获取先验信息。
先验信息包括位于安装边的第一顶点坐标、第二顶点坐标、第一中点坐标以及相对边所在区域；其中，安装边为矩形显示区域靠近安装鱼眼摄像头的边，相对边为矩形显示区域中与安装边相对的边，第一中点为所述安装边的第一顶点与第二顶点之间的中点。
具体地，获取多份样本图像数据，样本图像数据为由鱼眼摄像头对不同尺寸的矩形显示区域进行拍摄得到的监控画面。根据样本图像建立目标坐标系，以样本图像的左上角顶点为坐标原点，沿着图像左上角顶点水平向右为X轴正方向，沿着图像左上角顶点垂直向下为Y轴正方向。
在人工标定过程中，首先将样本图像顶边的中点确定为第一中点，根据其在目标坐标系中的具体位置确定第一中点坐标。其次，对多份样本图像逐份进行放大，开发人员通过对放大后的样本图像进行肉眼观测并确定每一份样本图像中的安装边的第一顶点和第二顶点的具体位置并进行标注，结合目标坐标系确定多份样本图像中的第一顶点坐标和第二顶点坐标。将多份样本图像中的第一顶点坐标形成第一顶点坐标集合，将多份样本图像中的第二顶点坐标形成第二顶点坐标集合，对第一顶点坐标集合求平均值，得到的平均值坐标确定为先验信息中通过人工标定方式获取的第一顶点坐标，同理，也可用此方法获取先验信息中的第二顶点坐标。
开发人员在人工标定过程中还发现电视机屏幕的底边在经过摄像头成像存在变形的情况后，仍然可以落在一个固定的范围内，且这个范围可用一个矩形框来表示，所以将这个矩形框的位置也作为先验信息
202、获取矩形显示区域的第一图像数据和第二图像数据。
第一图像数据和第二图像数据为矩形显示区域在不同亮度状态下的图像数据。具体 可通过与用户终端交互的形式控制鱼眼摄像头获取矩形显示区域在不同亮度状态下的第一图像数据和第二图像数据。
203、对第一图像数据和第二图像数据运用帧间相对差算法，确认矩形显示区域的相对边的第三顶点坐标、第四顶点坐标和第二中点坐标。
根据第一图像数据和第二图像数据在不同亮度状态下的相对像素变化，将像素变大的区域标记为白色像素区域，将像素变小的区域标记为黑色像素区域，得到差分图像。根据先验信息在差分图像上标注相对边所在区域；其中，相对边所在区域以矩形框表示，且矩形框内包括部分白色像素区域和部分黑色像素区域。以矩形框左上角顶点为起点，一个像素点作为基本单位，每次统计矩形框内水平方向一行像素点的数量，且计算白色像素点数量在一行像素点总数中所占的第一比例。沿Y轴方向遍历矩形框，当第一比例达到预设的第一条件时，确认当前行的纵坐标为相对边的第三顶点和第四顶点的纵坐标，第三顶点和第四顶点的横坐标为矩形框垂直于X轴的两条边的横坐标，得到第三顶点坐标和第四顶点坐标。继续沿Y轴方向遍历矩形框，当第一比例达到预设的第二条件时，确认当前行的纵坐标为位于相对边的第二中点的纵坐标，取第三顶点的横坐标与第四顶点的横坐标的中间值作为第二中点的横坐标，得到第二中点坐标。
具体地，预设的第一条件可以为：遍历过程中，当白色像素占比连续两次小于90％。预设的第二条件可以为：遍历过程中，当白色像素占比连续两次小于60％。以上数据通过开发人员在开发测试过程中试验得出的效果最优值，在实际应用过程中，可以通过调节阈值来增强本方案的容错性，得到相对较优的检测识别结果，具体此处不做限定。
204、结合先验数据，通过第一顶点坐标与第三顶点坐标确定第三中点坐标，通过第二顶点坐标与第四顶点坐标确定第四中点坐标。
第三中点坐标为第一顶点与第三顶点之间的中点的坐标，第四中点坐标为第二顶点与第四顶点之间的中点的坐标。
205、将第一顶点坐标、第二顶点坐标、第三顶点坐标、第四顶点坐标、第一中点坐标、第二中点坐标、第三中点坐标以及第四中点坐标确定为矩形显示区域的位置信息。
本实施例中，通过鱼眼摄像头获取矩形显示区域的图像数据，使得电视机屏幕能够完整地出现在监控画面中，通过帧间相对差算法并结合人工标定的先验信息获取矩形显示区域的位置信息，具有更高的准确度，从而实现优化电视机屏幕显示区域的检测和识别，实现实时屏幕取色，提高用户的使用感受。
图3是本申请实施例提供的图像检测识别装置结构示意图，该图像检测识别装置300可以包括一个或一个以上中央处理器(central processing units，CPU)301和存储器305，该存储器305中存储有一个或一个以上的应用程序或数据。
其中，存储器305可以是易失性存储或持久存储。存储在存储器305的程序可以包括一个或一个以上模块，每个模块可以包括对图像检测识别装置中的一系列指令操作。更进一步地，中央处理器301可以设置为与存储器305通信，在图像检测识别装置300上执行存储器305中的一系列指令操作。
图像检测识别装置300还可以包括一个或一个以上电源302，一个或一个以上有线或无线网络接口303，一个或一个以上输入输出接口304，和/或，一个或一个以上操作系 统，例如Windows ServerTM，Mac OS XTM，UnixTM,LinuxTM，FreeBSDTM等。
该中央处理器301可以执行前述图1至图2所示实施例中图像检测识别装置所执行的操作，具体此处不再赘述。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统，装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，read-only memory)、随机存取存储器(RAM，random access memory)、磁碟或者光盘等各种可以存储程序代码的介质。

Claims (10)

  1. 一种图像检测识别装置，应用于家居智能设备，其特征在于，包括：先验信息保存模块、摄像模块、图像处理模块、通讯模块；其中：
    所述先验信息保存模块用于保存所述图像检测识别装置在初次启用时所需的先验信息，且在所述图像检测识别装置初次启用时将所述先验信息传输至所述图像处理模块，以使得所述图像处理模块根据所述先验信息确定矩形显示区域的位置信息；
    所述摄像模块用于获取所述矩形显示区域的图像数据，并将所述图像数据传输至所述图像处理模块，以使得所述图像处理模块根据所述图像数据确定所述矩形显示区域的位置信息；
    所述图像处理模块用于通过帧间相对差算法获取所述矩形显示区域的位置信息；
    所述通讯模块用于所述图像检测识别装置与主机或终端之间的信息交互，以有线连接方式或无线连接方式连接于所述主机或终端，以使得所述主机或终端能连接所述图像检测识别装置而具备图像检测识别功能。
  2. 根据权利要求1所述的图像检测识别装置，其特征在于，所述摄像模块通过安装于靠近所述矩形显示区域的任一边的中点位置的鱼眼摄像头获取所述矩形显示区域的图像数据。
  3. 根据权利要求2所述的图像检测识别装置，其特征在于，通过人工标定的方式获取所述先验信息，并将所述先验信息保存在所述先验信息保存模块中。
  4. 根据权利要求3所述的图像检测识别装置，其特征在于，所述先验信息包括位于安装边的第一顶点坐标、第二顶点坐标、第一中点坐标以及相对边所在区域；其中，所述安装边为所述矩形显示区域靠近安装所述鱼眼摄像头的边，所述相对边为所述矩形显示区域中与所述安装边相对的边，所述第一中点为所述安装边的第一顶点与第二顶点之间的中点。
  5. 一种图像检测识别方法，应用于权利要求2至4中任一项所述的图像检测识别装置，其特征在于，包括：
    获取先验信息，所述先验信息包括位于安装边的第一顶点坐标、第二顶点坐标、第一中点坐标以及相对边所在区域；其中，所述安装边为所述矩形显示区域靠近安装所述鱼眼摄像头的边，所述相对边为所述矩形显示区域中与所述安装边相对的边，所述第一中点为所述安装边的第一顶点与第二顶点之间的中点；
    获取所述矩形显示区域的第一图像数据和第二图像数据，所述第一图像数据和所述第二图像数据为所述矩形显示区域在不同亮度状态下的图像数据；
    对所述第一图像数据和所述第二图像数据运用帧间相对差算法，确认所述矩形显示区域的相对边的第三顶点坐标、第四顶点坐标和第二中点坐标，其中所述第二中点为矩形显示区域位于所述相对边的中点；
    结合所述先验数据，通过所述第一顶点坐标与所述第三顶点坐标确定第三中点坐标，通过所述第二顶点坐标与所述第四顶点坐标确定第四中点坐标；其中，所述第三中点坐标为所述第一顶点与第三顶点之间的中点的坐标，所述第四中点坐标为所述第二顶点与第四顶点之间的中点的坐标；
    将所述第一顶点坐标、所述第二顶点坐标、所述第三顶点坐标、所述第四顶点坐标、 所述第一中点坐标、所述第二中点坐标、所述第三中点坐标以及所述第四中点坐标确定为所述矩形显示区域的位置信息。
  6. 根据权利要求5所述的图像检测识别方法，其特征在于，所述获取先验信息包括：
    获取多份样本图像数据，所述样本图像数据由所述鱼眼摄像头对不同尺寸的矩形显示区域进行拍摄得到；
    根据所述样本图像数据建立目标坐标系，以所述样本图像数据的左上角顶点为坐标原点，沿着所述样本图像数据的左上角顶点水平向右为X轴正方向，沿着所述样本图像数据的左上角顶点垂直向下为Y轴正方向；
    通过人工标定的方式在所述目标坐标系上标注每一个样本图像数据中位于安装边的第一顶点坐标、第二顶点坐标、第一中点坐标以及相对边所在区域；
    根据所述多份样本图像数据的标注结果确定所述第一顶点坐标、第二顶点坐标、第一中点坐标以及相对边所在区域，并作为先验信息。
  7. 根据权利要求6所述的图像检测识别方法，其特征在于，所述获取所述矩形显示区域的第一图像数据和第二图像数据包括：
    通过与用户终端交互的形式获取所述矩形显示区域在不同亮度状态下的第一图像数据和第二图像数据。
  8. 根据权利要求7所述的图像检测识别方法，其特征在于，所述对所述第一图像数据和所述第二图像数据运用帧间相对差算法，确认所述矩形显示区域的相对边的第三顶点坐标、第四顶点坐标和第二中点坐标包括：
    根据所述第一图像数据和所述第二图像数据在不同亮度状态下的相对像素变化，将像素变大的区域标记为白色像素区域，将像素变小的区域标记为黑色像素区域，得到差分图像；
    根据所述先验信息在所述差分图像上标注相对边所在区域；其中，相对边所在区域以矩形框表示，且矩形框内包括部分白色像素区域和部分黑色像素区域；
    以所述矩形框左上角顶点为起点，一个像素点作为基本单位，每次统计所述矩形框内水平方向一行像素点的数量，且计算白色像素点数量在一行像素点总数中所占的第一比例；
    沿Y轴方向遍历所述矩形框，当所述第一比例达到预设的第一条件时，确认当前行的纵坐标为所述相对边的第三顶点和第四顶点的纵坐标，所述第三顶点和所述第四顶点的横坐标为所述矩形框垂直于X轴的两条边的横坐标，得到所述第三顶点坐标和所述第四顶点坐标；
    继续沿Y轴方向遍历所述矩形框，当所述第一比例达到预设的第二条件时，确认当前行的纵坐标为位于所述相对边的第二中点的纵坐标，取所述第三顶点的横坐标与所述第四顶点的横坐标的中间值作为所述第二中点的横坐标，得到所述第二中点坐标。
  9. 一种图像检测识别设备，其特征在于，包括：
    中央处理器，存储器以及输入输出接口；
    所述存储器为短暂存储存储器或持久存储存储器；
    所述中央处理器配置为与所述存储器通信，并执行所述存储器中的指令操作以执行权 利要求5至8中任意一项所述的方法。
  10. 一种计算机可读存储介质，包括指令，当所述指令在计算机上运行时，使得计算机执行如权利要求5至8中任意一项所述的方法。
PCT/CN2022/105235 2022-04-02 2022-07-12 图像检测识别装置及其检测识别方法 WO2023184783A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210351911.3A CN114727097A (zh) 2022-04-02 2022-04-02 图像检测识别装置及其检测识别方法
CN202210351911.3 2022-04-02

Publications (1)

Publication Number Publication Date
WO2023184783A1 true WO2023184783A1 (zh) 2023-10-05

Family

ID=82241647

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/105235 WO2023184783A1 (zh) 2022-04-02 2022-07-12 图像检测识别装置及其检测识别方法

Country Status (2)

Country Link
CN (1) CN114727097A (zh)
WO (1) WO2023184783A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114727097A (zh) * 2022-04-02 2022-07-08 杭州涂鸦信息技术有限公司 图像检测识别装置及其检测识别方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102289328A (zh) * 2011-08-03 2011-12-21 王珂 一种基于手持摄像头的人机交互方法及装置
CN102984563A (zh) * 2011-09-05 2013-03-20 富士通半导体（上海）有限公司 一种智能遥控电视系统及其遥控方法
JP2013161247A (ja) * 2012-02-03 2013-08-19 Sharp Corp 操作装置、表示装置、遠隔操作システム、操作装置の制御方法、制御プログラム、および、記録媒体
CN107103865A (zh) * 2017-04-10 2017-08-29 青岛海信电器股份有限公司 检测显示屏中显示区域的方法和装置
CN108414193A (zh) * 2018-03-28 2018-08-17 京东方科技集团股份有限公司 显示屏的检测区域定位方法和装置
CN114727097A (zh) * 2022-04-02 2022-07-08 杭州涂鸦信息技术有限公司 图像检测识别装置及其检测识别方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102289328A (zh) * 2011-08-03 2011-12-21 王珂 一种基于手持摄像头的人机交互方法及装置
CN102984563A (zh) * 2011-09-05 2013-03-20 富士通半导体（上海）有限公司 一种智能遥控电视系统及其遥控方法
JP2013161247A (ja) * 2012-02-03 2013-08-19 Sharp Corp 操作装置、表示装置、遠隔操作システム、操作装置の制御方法、制御プログラム、および、記録媒体
CN107103865A (zh) * 2017-04-10 2017-08-29 青岛海信电器股份有限公司 检测显示屏中显示区域的方法和装置
CN108414193A (zh) * 2018-03-28 2018-08-17 京东方科技集团股份有限公司 显示屏的检测区域定位方法和装置
CN114727097A (zh) * 2022-04-02 2022-07-08 杭州涂鸦信息技术有限公司 图像检测识别装置及其检测识别方法

Also Published As

Publication number Publication date
CN114727097A (zh) 2022-07-08

Similar Documents

Publication Publication Date Title
US9819870B2 (en) Photographing method and electronic device
WO2022247418A1 (zh) 图像校正方法及拍摄设备
CN112272292B (zh) 投影校正方法、装置和存储介质
WO2021031781A1 (zh) 投影图像校准方法、装置及投影设备
TWI714018B (zh) 顯示牆的排版辨識方法以及使用此方法的電子裝置
US9992486B2 (en) Method of enhanced alignment of two means of projection
US9432645B2 (en) Information processing method and electronic device
US10168984B2 (en) Image receiving apparatus and method thereof for determining an orientation of a screen of an electronic apparatus
WO2023184783A1 (zh) 图像检测识别装置及其检测识别方法
WO2022222510A1 (zh) 交互控制方法、终端设备及存储介质
JP2012238293A (ja) 入力装置
TWI749370B (zh) 臉部辨識方法及其相關電腦系統
WO2022105277A1 (zh) 投影控制方法、装置、投影光机及可读存储介质
WO2022091577A1 (ja) 情報処理装置および情報処理方法
CN109040729B (zh) 图像白平衡校正方法、装置、存储介质及终端
US20240015272A1 (en) Electronic apparatus and control method thereof
WO2024055531A1 (zh) 照度计数值识别方法、电子设备及存储介质
US9552059B2 (en) Information processing method and electronic device
CN111369513A (zh) 一种异常检测方法、装置、终端设备及存储介质
US11750778B2 (en) Method for adjusting pixel values of blending image and projection system thereof
CN115657893A (zh) 一种显示控制方法、显示控制装置及智能设备
CN112791400A (zh) 基于电视的游戏画面显示处理方法、装置、终端及介质
US20240177321A1 (en) Image processing method, image processing device, electronic device, and storage medium
WO2023134215A1 (zh) 一种调试摄像机的方法及相关设备
CN115834849A (zh) 远程在线教育用投影梯形校正触发系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22934605

Country of ref document: EP

Kind code of ref document: A1