CN112001430A - Refrigerator food material detection method and device, storage medium and electronic device - Google Patents

Refrigerator food material detection method and device, storage medium and electronic device Download PDF

Info

Publication number
CN112001430A
CN112001430A CN202010790806.0A CN202010790806A CN112001430A CN 112001430 A CN112001430 A CN 112001430A CN 202010790806 A CN202010790806 A CN 202010790806A CN 112001430 A CN112001430 A CN 112001430A
Authority
CN
China
Prior art keywords
image
blocks
food material
similarity
block
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010790806.0A
Other languages
Chinese (zh)
Inventor
刘彦甲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Haier Uplus Intelligent Technology Beijing Co Ltd
Original Assignee
Haier Uplus Intelligent Technology Beijing Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Haier Uplus Intelligent Technology Beijing Co Ltd filed Critical Haier Uplus Intelligent Technology Beijing Co Ltd
Priority to CN202010790806.0A priority Critical patent/CN112001430A/en
Publication of CN112001430A publication Critical patent/CN112001430A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30128—Food products

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Cold Air Circulating Systems And Constructional Details In Refrigerators (AREA)

Abstract

The invention provides a refrigerator food material detection method, a refrigerator food material detection device, a storage medium and an electronic device, wherein the method comprises the following steps: acquiring a first image and a second image which are acquired from the same food material storage area in a refrigerator at different time; respectively carrying out image blocking and Hash coding on the first image and the second image to obtain the similarity between the blocks of the first image and the corresponding blocks of the second image; and determining the food material change information of the food material storage area of the refrigerator according to the similarity. According to the method and the device, the images of the refrigerator food materials are detected in a statistical image processing mode, so that a large amount of data and labels do not need to be collected, and the effect of rapidly detecting the food materials is achieved. And a special GPU server is not needed, so that resources are saved.

Description

Refrigerator food material detection method and device, storage medium and electronic device
Technical Field
The embodiment of the invention relates to the field of image processing, in particular to a refrigerator food material detection method and device, a storage medium and an electronic device.
Background
Since a user often performs an input and output operation for food materials in a refrigerator, and often forgets what food materials are input or output due to a long operation time or other problems, the user wants to check newly input or output food materials, and the user wants to know the latest changed food materials in the refrigerator.
In the related technology, the food material detection is realized by adopting a deep learning algorithm, and according to the detection results of the two pictures, the reduced or increased food materials in the two pictures are compared, so that the change of which food material is changed is determined. The deep learning algorithm needs to collect a large amount of sample training, labels samples, deploys by adopting a special GPU algorithm server, and uses a cloud computing scheme.
Aiming at the problems that in the related technology, a deep learning algorithm needs to collect a large amount of data, needs to calibrate the data, and is time-consuming and labor-consuming, an effective solution is not provided at present.
Disclosure of Invention
The embodiment of the invention provides a refrigerator food material detection method, a device, a storage medium and an electronic device, and at least solves the problem that the change of food materials cannot be rapidly and flexibly indicated when the refrigerator food materials are put in or taken out in the related art.
According to an embodiment of the invention, a refrigerator food material detection method is provided, which comprises the following steps: acquiring a first image and a second image which are acquired from the same food material storage area in a refrigerator at different time; respectively carrying out image blocking and Hash coding on the first image and the second image to obtain the similarity between the blocks of the first image and the corresponding blocks of the second image; and determining the food material change information of the food material storage area of the refrigerator according to the similarity.
In one exemplary embodiment, the image blocking and hash encoding the first and second images, respectively, may include: dividing the first image and the second image into the same m multiplied by n blocks respectively, wherein each block comprises S pixel points, and m, n and S are positive integers; converting each block into a gray level image, and calculating the gray level average value of pixel points of each block; marking the pixel points of which the gray values are greater than or equal to the average gray value of each block as 1, and marking the pixel points of which the gray values are less than the average gray value of the block as 0; and combining the mark values of all the pixel points in each block to obtain the hash code with s bits of the block.
Further, obtaining the similarity of the block of the first image and the corresponding block of the second image may include: and comparing the Hash codes of the blocks of the first image with the Hash codes of the corresponding blocks of the second image to obtain the similarity of the blocks of the first image and the corresponding blocks of the second image.
In an exemplary embodiment, comparing the hash codes of the blocks of the first image with the hash codes of the corresponding blocks of the second image to obtain the similarity between the blocks of the first image and the corresponding blocks of the second image may include: comparing the value of each bit in the Hash codes of the blocks of the first image with the value of the corresponding bit in the Hash codes of the corresponding blocks of the second image, and if the values are the same, marking the values as 1, and if the values are not the same, marking the values as 0; and accumulating the mark values of all bits of the Hash codes of the blocks of the first image and dividing the accumulated mark values by the length of the Hash codes of the blocks to obtain the similarity between the blocks and the corresponding blocks of the second image.
In an exemplary embodiment, determining the food material change information of the food material storage area of the refrigerator according to the similarity may include: respectively comparing the similarity of each block of the first image and the second image with a preset threshold value, and carrying out binarization marking on each block according to the comparison result; combining the blocks marked by the binaryzation according to the sequence of block division to respectively form a contrast characteristic map of the first image and a contrast characteristic map of the second image; and marking the position information of the change of the food material in the contrast characteristic diagram of the first image and the contrast characteristic diagram of the second image respectively.
In an exemplary embodiment, comparing the similarity of each block of the first image and the second image with a preset threshold, and binarizing and labeling the block according to the comparison result may include: when the similarity of the blocks is larger than the preset threshold, the value of each pixel point of the blocks is marked as a first pixel value, and when the similarity of the blocks is smaller than or equal to the preset threshold, the value of each pixel point of the blocks is marked as a second pixel value.
In an exemplary embodiment, the marking of the position information of the food material change in the comparison feature map of the first image and the comparison feature map of the second image respectively may include: and respectively merging connected domains of the blocks marked as the second pixel values on the contrast characteristic diagram of the first image and the contrast characteristic diagram of the second image, and marking the circumscribed rectangular frame of each connected domain as the position of the change of the food material.
According to another embodiment of the present invention, there is provided a refrigerator food material detecting apparatus including: the acquisition module is used for acquiring a first image and a second image which are acquired from the same food material storage area in the refrigerator at different times; the Hash coding module is used for respectively carrying out image blocking and Hash coding on the first image and the second image to obtain the similarity between the blocks of the first image and the corresponding blocks of the second image; and the determining module is used for determining the food material change information of the food material storage area of the refrigerator according to the similarity.
According to a further embodiment of the present invention, there is also provided a storage medium having a computer program stored therein, wherein the computer program is arranged to perform the steps of any of the above method embodiments when executed.
According to yet another embodiment of the present invention, there is also provided an electronic device, including a memory in which a computer program is stored and a processor configured to execute the computer program to perform the steps in any of the above method embodiments.
In the embodiment of the invention, the images of the refrigerator food materials are detected by adopting a statistical image processing mode, so that a large amount of data and labels do not need to be collected, and the effect of quickly detecting the food materials is achieved. And a special Graphic Processing Unit (GPU) is not needed, so that resources are saved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
fig. 1 is a block diagram of a hardware structure of a computer terminal of a refrigerator food material detection method according to an embodiment of the invention;
FIG. 2 is a flowchart of a refrigerator food material detection method according to an embodiment of the invention;
FIG. 3 is a block diagram of a refrigerator food material detecting apparatus according to an embodiment of the present invention;
fig. 4 is a flowchart of a refrigerator food material taking and placing detection method based on image hashing according to an alternative embodiment of the invention;
fig. 5 is a flowchart of a specific detection method for refrigerator food material taking based on image hashing according to an alternative embodiment of the invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
In order to better understand the technical solutions of the embodiments and the alternative embodiments of the present invention, the following description is made on possible application scenarios in the embodiments and the alternative embodiments of the present invention, but is not limited to the application of the following scenarios.
The method provided by the first embodiment of the present application may be executed in a computer terminal, or a similar computing device. Taking an example of the method running on a computer terminal, fig. 1 is a hardware structure block diagram of the computer terminal of the method for detecting refrigerator food materials according to the embodiment of the invention. As shown in fig. 1, the computer terminal 10 may include one or more (only one shown in fig. 1) processors 102 (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, and in an exemplary embodiment, may also include a transmission device 106 for communication functions and an input-output device 108. It will be understood by those skilled in the art that the structure shown in fig. 1 is only an illustration and is not intended to limit the structure of the computer terminal. For example, the computer terminal 10 may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.
The memory 104 may be used to store a computer program, for example, a software program and a module of an application software, such as a computer program corresponding to the refrigerator food material detection method in the embodiment of the present invention, and the processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, so as to implement the method described above. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 106 is used for receiving or transmitting data via a network. Specific examples of the network described above may include wired or wireless networks provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network adapter (NIC) that can be connected to other Network devices via a broadband Network so as to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.
In this embodiment, a method for detecting a refrigerator food material operated in the computer terminal is provided, and fig. 2 is a flowchart of the method for detecting a refrigerator food material according to the embodiment of the present invention, as shown in fig. 2, the flowchart includes the following steps:
step S201, acquiring a first image and a second image which are acquired from the same food material storage area in a refrigerator at different time;
step S202, respectively carrying out image blocking and Hash coding on the first image and the second image to obtain the similarity between the blocks of the first image and the corresponding blocks of the second image;
step S203, determining the food material change information of the food material storage area of the refrigerator according to the similarity.
In this embodiment, step S202 may include: dividing the first image and the second image into the same m multiplied by n blocks respectively, wherein each block comprises S pixel points, and m, n and S are positive integers; converting each block into a gray level image, and calculating the gray level average value of pixel points of each block; marking the pixel points of which the gray values are greater than or equal to the average gray value of each block as 1, and marking the pixel points of which the gray values are less than the average gray value of the block as 0; and combining the mark values of all the pixel points in each block to obtain the hash code with s bits of the block.
In this embodiment, step S202 may further include: and comparing the Hash codes of the blocks of the first image with the Hash codes of the corresponding blocks of the second image to obtain the similarity of the blocks of the first image and the corresponding blocks of the second image.
In this embodiment, comparing the hash code of the block of the first image with the hash code of the corresponding block of the second image to obtain the similarity between the block of the first image and the corresponding block of the second image may include: comparing the value of each bit in the Hash codes of the blocks of the first image with the value of the corresponding bit in the Hash codes of the corresponding blocks of the second image, and if the values are the same, marking the values as 1, and if the values are not the same, marking the values as 0; and accumulating the mark values of all bits of the Hash codes of the blocks of the first image and dividing the accumulated mark values by the length of the Hash codes of the blocks to obtain the similarity between the blocks and the corresponding blocks of the second image.
In this embodiment, step S203 may include: respectively comparing the similarity of each block of the first image and the second image with a preset threshold value, and carrying out binarization marking on each block according to the comparison result; combining the blocks marked by the binaryzation according to the sequence of block division to respectively form a contrast characteristic map of the first image and a contrast characteristic map of the second image; and marking the position information of the change of the food material in the contrast characteristic diagram of the first image and the contrast characteristic diagram of the second image respectively.
In this embodiment, comparing the similarity of each block of the first image and the second image with a preset threshold, and binarizing and labeling the block according to the comparison result may include: when the similarity of the blocks is larger than the preset threshold, the value of each pixel point of the blocks is marked as a first pixel value, and when the similarity of the blocks is smaller than or equal to the preset threshold, the value of each pixel point of the blocks is marked as a second pixel value.
In this embodiment, the marking the position information of the food material change in the contrast characteristic diagram of the first image and the contrast characteristic diagram of the second image respectively includes: and respectively merging connected domains of the blocks marked as the second pixel values on the contrast characteristic diagram of the first image and the contrast characteristic diagram of the second image, and marking the circumscribed rectangular frame of each connected domain as the position of the change of the food material.
Through the steps, the similarity of the corresponding blocks of the images acquired in the same food material storage area in the refrigerator at different times is obtained, and therefore the food material change information of the food material storage area of the refrigerator is obtained, and the problem that the change of the food material cannot be rapidly and flexibly indicated when the refrigerator food material is put in or taken out in the related art is solved. Because the deep learning algorithm needs to acquire a large amount of data, needs to calibrate the data, and is time-consuming and labor-consuming, the food material detection can be completed by adopting the statistical image processing algorithm without acquiring a large amount of data and labels, and the effects of quick detection, flexible deployment at the front end, no need of using a special GPU server and small network dependence are achieved.
In an exemplary embodiment, the main body of the above steps may be a terminal or the like, but is not limited thereto.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
The embodiment also provides a refrigerator food material detection device, which is used for implementing the above embodiments and preferred embodiments, and the description of the device is omitted. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
Fig. 3 is a block diagram of a refrigerator food material detection apparatus according to an embodiment of the present invention, and as shown in fig. 3, the apparatus includes an obtaining module 10, a hash encoding module 20, and a determining module 30.
The acquiring module 10 is configured to acquire a first image and a second image acquired from a same food material storage area in the refrigerator at different times.
The hash coding module 20 is configured to perform image blocking and hash coding on the first image and the second image, respectively, to obtain similarity between a block of the first image and a corresponding block of the second image.
The determining module 30 is configured to determine the food material change information of the food material storage area of the refrigerator according to the similarity.
It should be noted that, the above modules may be implemented by software or hardware, and for the latter, the following may be implemented, but not limited to: the modules are all positioned in the same processor; alternatively, the modules are respectively located in different processors in any combination.
Embodiments of the present invention also provide a storage medium having a computer program stored therein, wherein the computer program is arranged to perform the steps of any of the above method embodiments when executed.
In an exemplary embodiment, in the present embodiment, the storage medium may be configured to store a computer program for executing the steps of:
s1, acquiring a first image and a second image which are acquired from the same food material storage area in the refrigerator at different times;
s2, performing image blocking and Hash coding on the first image and the second image respectively to obtain the similarity between the blocks of the first image and the corresponding blocks of the second image;
and S3, determining the food material change information of the food material storage area of the refrigerator according to the similarity.
In an exemplary embodiment, the storage medium may include, but is not limited to: various media capable of storing computer programs, such as a usb disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk.
Embodiments of the present invention also provide an electronic device comprising a memory having a computer program stored therein and a processor arranged to run the computer program to perform the steps of any of the above method embodiments.
In an exemplary embodiment, the electronic apparatus may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.
In an exemplary embodiment, in the present embodiment, the processor may be configured to execute the following steps by a computer program:
s1, acquiring a first image and a second image which are acquired from the same food material storage area in the refrigerator at different times;
s2, performing image blocking and Hash coding on the first image and the second image respectively to obtain the similarity between the blocks of the first image and the corresponding blocks of the second image;
and S3, determining the food material change information of the food material storage area of the refrigerator according to the similarity.
In an exemplary embodiment, for specific examples in this embodiment, reference may be made to the examples described in the above embodiments and optional implementation manners, and details of this embodiment are not described herein again.
In order to facilitate understanding of the technical solutions provided by the present invention, the following detailed description will be made with reference to embodiments of specific scenarios.
The embodiment provides a method for detecting the placement and the taking of refrigerator food materials based on images and a detailed solution of the scheme.
Specifically, the embodiment proposes that an object put in (taken out) of an image is obtained by calculating two images before and after putting in (before and after taking out) based on two food material images before and after putting in (before and after taking out) shot by a built-in camera in a refrigerator through an image algorithm, and a region of image change is detected, so as to remind a user of a new put in (taken out) food material at which position.
The embodiment can solve the problem that the user is reminded of changing the food materials when the food materials are put in or taken out of the refrigerator. Due to the fact that the deep learning algorithm needs to acquire a large amount of data, needs to calibrate the data, is time-consuming and labor-consuming, food material detection can be completed without acquiring a large amount of data and labels by adopting the traditional image processing algorithm, the detection speed is high, the front end can be flexibly deployed, a special GPU algorithm server is not needed, and dependence on a network is small.
Fig. 4 is a flowchart of a method for detecting food material pick-and-place for a refrigerator based on image hashing according to an alternative embodiment of the invention, and as shown in fig. 4, the flowchart includes the following steps:
step S401, first performing blocking processing on two pictures before and after the shot food material is put into the refrigerator, where the size of each block is 8 × 8 pixels, and performing hash coding on the blocked pictures respectively.
In this embodiment, step S401 may include step S4011: the RGB image is converted into a gray scale image, and the image is converted into 64-level gray scale, wherein all pixel points have 64 colors in total.
In this embodiment, step S401 may further include step S4012: the gray level average of all 64 pixels is calculated.
In this embodiment, step S401 may further include step S4013: comparing the gray level of each pixel with the average value; greater than or equal to the average value, noted 1; less than the average, noted as 0.
In this embodiment, step S401 may further include step S4014: the comparison results in step S4013 are combined together to form a 64-bit integer, which is the fingerprint of the picture, wherein the order of combination is not important, and the hash coding of the picture can be obtained as long as all pictures are ensured to adopt the same order.
And S402, comparing the Hash codes of the two small images after the two small images are blocked before and after being shot, and calculating the Hamming distance.
In this embodiment, step S402 may include step S4021: if the hash codes are the same, adding 1, and dividing the obtained result by the length of the hash code; and when the obtained Hamming distance is larger than a set threshold value, setting all pixel values of the small block to be 255, when the obtained Hamming distance is smaller than the set threshold value, setting all pixel values of the small block to be 0, and recombining all small blocks into an image according to a blocking sequence to obtain a contrast characteristic diagram.
In this embodiment, step S402 may further include step S4022: and merging connected domains of the characteristic images, and searching a circumscribed rectangular frame to obtain an image contrast region frame.
Fig. 5 is a flowchart of a specific detection method for picking and placing refrigerator food materials based on image hashing according to an alternative embodiment of the present invention, and as shown in fig. 5, the flowchart includes the following steps:
step S501, shooting an image I before food materials are put in a refrigeratormnAnd picture I 'after food materials are put in'mnTwo images are respectively subjected to block processing and divided into m × n blocks, the pixel size of each block is 8 × 8, and each block is marked as Smn,S′mn；
Step S502, calculating each image Smn,S′mnHash encoding of (H)mn，H′mnWherein the hash code HmnThe calculation method comprises the following steps:
Figure BDA0002623684520000111
x is the pixel value, uxIs SmnAverage value of (d); calculate Hmn，H′mnOf (2), i.e. Hmn，H′mnThe same value is marked as 1, and the obtained value is added and divided by the Hash coding length to obtain the similarity F of the two small imagesmn；
Step S503, when FmnIf the value is more than the threshold value T, taking S'mnEach pixel in the block is marked as 0, i.e. the blocks with small similarity in the first image and the second image are marked as black blocks. When F is presentmnWhen the temperature is less than or equal to the threshold value T, taking S'mnEach pixel in the block is marked as 255, i.e. the block with large similarity in the two pictures is marked as a white block. Wherein T is a similarity artificial threshold value which is selected according to experience or actual scenes and is in the range of 0-1.
Step S504, F of all small blocks is calculatedmnAnd after marking, the mark is putThe small block after being recorded is marked as SmnWill be S ″)mnRe-combining into a new image I' in the order of the blocksmn，I″mnNamely, the detection characteristic diagram is obtained;
step S505, adding I ″)mnAnd merging the connected domains, and marking the external rectangular frames of all the connected domains, namely the detected food material change positions. In this embodiment, the white blocks represent different parts of the two pictures, and therefore, the white blocks in the pictures are merged into a connected domain, and the circumscribed rectangle frame of each connected domain is marked as the food material change position identifier.
By the embodiment of the invention, the problem that the user is not reminded of the change of the food materials when the food materials of the refrigerator are put in or taken out is solved. Due to the fact that the deep learning algorithm needs to acquire a large amount of data, needs to calibrate the data, is time-consuming and labor-consuming, food material detection can be completed by the aid of the statistical image processing algorithm without acquiring a large amount of data and labels, detection speed is high, the food material detection can be flexibly deployed at the front end, a special GPU server is not needed, and dependence on a network is small.
It will be apparent to those skilled in the art that the various modules or steps of the invention described above may be implemented using a general purpose computing device, which may be centralized on a single computing device or distributed across a network of computing devices, and in one exemplary embodiment may be implemented using program code executable by a computing device, such that the steps shown and described may be executed by a computing device stored in a memory device and, in some cases, executed in a sequence different from that shown and described herein, or separately fabricated into individual integrated circuit modules, or multiple ones of them fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A refrigerator food material detection method is characterized by comprising the following steps:
acquiring a first image and a second image which are acquired from the same food material storage area in a refrigerator at different time;
respectively carrying out image blocking and Hash coding on the first image and the second image to obtain the similarity between the blocks of the first image and the corresponding blocks of the second image;
and determining the food material change information of the food material storage area of the refrigerator according to the similarity.
2. The method of claim 1, wherein image blocking and hash encoding the first and second images, respectively, comprises:
dividing the first image and the second image into the same m multiplied by n blocks respectively, wherein each block comprises S pixel points, and m, n and S are positive integers;
converting each block into a gray level image, and calculating the gray level average value of pixel points of each block;
marking the pixel points of which the gray values are greater than or equal to the average gray value of each block as 1, and marking the pixel points of which the gray values are less than the average gray value of the block as 0;
and combining the mark values of all the pixel points in each block to obtain the hash code with s bits of the block.
3. The method of claim 2, wherein obtaining the similarity of the patches of the first image to the corresponding patches of the second image comprises:
and comparing the Hash codes of the blocks of the first image with the Hash codes of the corresponding blocks of the second image to obtain the similarity of the blocks of the first image and the corresponding blocks of the second image.
4. The method of claim 3, wherein comparing the hash codes of the blocks of the first image with the hash codes of the corresponding blocks of the second image to obtain the similarity between the blocks of the first image and the corresponding blocks of the second image comprises:
comparing the value of each bit in the Hash codes of the blocks of the first image with the value of the corresponding bit in the Hash codes of the corresponding blocks of the second image, and if the values are the same, marking the values as 1, and if the values are not the same, marking the values as 0;
and accumulating the mark values of all bits of the Hash codes of the blocks of the first image and dividing the accumulated mark values by the length of the Hash codes of the blocks to obtain the similarity between the blocks and the corresponding blocks of the second image.
5. The method of claim 3, wherein determining the food material change information for the food material storage area of the refrigerator according to the similarity comprises:
respectively comparing the similarity of each block of the first image and the second image with a preset threshold value, and carrying out binarization marking on each block according to the comparison result;
combining the blocks marked by the binaryzation according to the sequence of block division to respectively form a contrast characteristic map of the first image and a contrast characteristic map of the second image;
and marking the position information of the change of the food material in the contrast characteristic diagram of the first image and the contrast characteristic diagram of the second image respectively.
6. The method according to claim 5, wherein the similarity of each block of the first image and the second image is compared with a preset threshold, and binarization labeling of the blocks according to the comparison result comprises:
when the similarity of the blocks is larger than the preset threshold, the value of each pixel point of the blocks is marked as a first pixel value, and when the similarity of the blocks is smaller than or equal to the preset threshold, the value of each pixel point of the blocks is marked as a second pixel value.
7. The method of claim 5, wherein marking the position information of the food material change in the comparison feature map of the first image and the comparison feature map of the second image comprises:
and respectively merging connected domains of the blocks marked as the second pixel values on the contrast characteristic diagram of the first image and the contrast characteristic diagram of the second image, and marking the circumscribed rectangular frame of each connected domain as the position of the change of the food material.
8. A refrigerator food material detection device, comprising:
the acquisition module is used for acquiring a first image and a second image which are acquired from the same food material storage area in the refrigerator at different times;
the Hash coding module is used for respectively carrying out image blocking and Hash coding on the first image and the second image to obtain the similarity between the blocks of the first image and the corresponding blocks of the second image;
and the determining module is used for determining the food material change information of the food material storage area of the refrigerator according to the similarity.
9. A computer-readable storage medium, in which a computer program is stored, wherein the computer program is arranged to perform the method of any of claims 1 to 7 when executed, or to perform the method of any of claims 6 to 10.
10. An electronic device comprising a memory and a processor, wherein the memory has stored therein a computer program, and wherein the processor is arranged to execute the computer program to perform the method of any of claims 1 to 7.
CN202010790806.0A 2020-08-07 2020-08-07 Refrigerator food material detection method and device, storage medium and electronic device Pending CN112001430A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010790806.0A CN112001430A (en) 2020-08-07 2020-08-07 Refrigerator food material detection method and device, storage medium and electronic device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010790806.0A CN112001430A (en) 2020-08-07 2020-08-07 Refrigerator food material detection method and device, storage medium and electronic device

Publications (1)

Publication Number Publication Date
CN112001430A true CN112001430A (en) 2020-11-27

Family

ID=73463882

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010790806.0A Pending CN112001430A (en) 2020-08-07 2020-08-07 Refrigerator food material detection method and device, storage medium and electronic device

Country Status (1)

Country Link
CN (1) CN112001430A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112584152A (en) * 2020-12-14 2021-03-30 上海贝锐信息科技股份有限公司 Difference detection and regional dynamic encoding method in remote desktop, electronic device and computer-readable storage medium
CN112712021A (en) * 2020-12-29 2021-04-27 华信咨询设计研究院有限公司 Grain surface abnormal state identification method based on perceptual hash and connected domain analysis algorithm
TWI748788B (en) * 2020-12-08 2021-12-01 台灣松下電器股份有限公司 Judgment method for accessing items and smart refrigerator

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108615253A (en) * 2018-04-12 2018-10-02 广东数相智能科技有限公司 Image generating method, device and computer readable storage medium
CN110149553A (en) * 2019-05-10 2019-08-20 腾讯科技（深圳）有限公司 Treating method and apparatus, storage medium and the electronic device of image
CN110516100A (en) * 2019-08-29 2019-11-29 武汉纺织大学 A kind of calculation method of image similarity, system, storage medium and electronic equipment

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108615253A (en) * 2018-04-12 2018-10-02 广东数相智能科技有限公司 Image generating method, device and computer readable storage medium
CN110149553A (en) * 2019-05-10 2019-08-20 腾讯科技（深圳）有限公司 Treating method and apparatus, storage medium and the electronic device of image
CN110516100A (en) * 2019-08-29 2019-11-29 武汉纺织大学 A kind of calculation method of image similarity, system, storage medium and electronic equipment

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
KALAFINAIAN: "图像相似度中的Hash算法", 《URL:HTTPS://WWW.CNBLOGS.COM/KALAFINAIAN/P/11260808.HTML》, pages 1 - 11 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI748788B (en) * 2020-12-08 2021-12-01 台灣松下電器股份有限公司 Judgment method for accessing items and smart refrigerator
CN112584152A (en) * 2020-12-14 2021-03-30 上海贝锐信息科技股份有限公司 Difference detection and regional dynamic encoding method in remote desktop, electronic device and computer-readable storage medium
CN112712021A (en) * 2020-12-29 2021-04-27 华信咨询设计研究院有限公司 Grain surface abnormal state identification method based on perceptual hash and connected domain analysis algorithm
CN112712021B (en) * 2020-12-29 2022-06-17 华信咨询设计研究院有限公司 Grain surface abnormal state identification method based on perceptual hash and connected domain analysis algorithm

Similar Documents

Publication Publication Date Title
CN112001430A (en) Refrigerator food material detection method and device, storage medium and electronic device
CN106484837B (en) Method and device for detecting similar video files
WO2020098250A1 (en) Character recognition method, server, and computer readable storage medium
US9628805B2 (en) Tunable multi-part perceptual image hashing
CN108647245B (en) Multimedia resource matching method and device, storage medium and electronic device
CN111080526B (en) Method, device, equipment and medium for measuring and calculating farmland area of aerial image
WO2020024744A1 (en) Image feature point detecting method, terminal device, and storage medium
CN109376256B (en) Image searching method and device
CN112434715B (en) Target identification method and device based on artificial intelligence and storage medium
CN112115292A (en) Picture searching method and device, storage medium and electronic device
CN108647264A (en) A kind of image automatic annotation method and device based on support vector machines
CN110472499B (en) Pedestrian re-identification method and device
CN110704657A (en) Recommendation method and device for image tag and electronic equipment
CN111821693B (en) Method, device, equipment and storage medium for detecting perspective plug-in of game
CN110991298A (en) Image processing method and device, storage medium and electronic device
WO2019198634A1 (en) Learning data generation device, variable region detection method, and computer program
CN108648189A (en) Image fuzzy detection method, apparatus, computing device and readable storage medium storing program for executing
CN112084939A (en) Image feature data management method and device, computer equipment and storage medium
CN111598176A (en) Image matching processing method and device
US10026010B2 (en) Image quality estimation using a reference image portion
CN110503600A (en) Feature point detecting method, device, electronic equipment and readable storage medium storing program for executing
CN112001289A (en) Article detection method and apparatus, storage medium, and electronic apparatus
CN114187545B (en) Progressive lens identification method and device, electronic equipment and storage medium
CN109685755B (en) Electronic photo generation method, device, equipment and computer storage medium
CN113490009B (en) Content information implantation method, device, server and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination