WO2021096030A1 - Image object recognition system based on deep learning - Google Patents

Image object recognition system based on deep learning Download PDF

Info

Publication number
WO2021096030A1
WO2021096030A1 PCT/KR2020/010531 KR2020010531W WO2021096030A1 WO 2021096030 A1 WO2021096030 A1 WO 2021096030A1 KR 2020010531 W KR2020010531 W KR 2020010531W WO 2021096030 A1 WO2021096030 A1 WO 2021096030A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
mirrors
area
storage box
recognition system
Prior art date
Application number
PCT/KR2020/010531
Other languages
French (fr)
Korean (ko)
Inventor
오장욱
김지훈
김형준
김형균
Original Assignee
주식회사 베이리스
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 주식회사 베이리스 filed Critical 주식회사 베이리스
Publication of WO2021096030A1 publication Critical patent/WO2021096030A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries

Definitions

  • the present invention relates to an image object recognition system based on deep learning.
  • 1 is an example of a camera image acquired for a narrow interior space.
  • the present invention is an invention aimed at solving the technical problem as described above, and when a product is stored in an internal space such as a refrigerator or a vending machine, an image of the product covered by a reflective image using a mirror is also secured and the corresponding Its purpose is to provide an image object recognition system based on deep learning that can recognize products.
  • the image object recognition system of the present invention includes a plurality of mirrors installed on a plurality of sidewalls inside a storage box capable of storing goods; A camera that acquires images of the inside of the storage box and the plurality of mirrors; A controller for generating a control signal for controlling the operation of the plurality of mirrors; And a computing device for recognizing a product in a storage box by comparing pre-designated areas of the image obtained from the camera.
  • the front glass of the plurality of mirrors is in a transparent or opaque state by the control signal.
  • sections in the transparent state of each of the front glass of the plurality of mirrors are characterized in that they are angular to each other.
  • the computing device compares a first reflective area, which is a partial area of the first mirror among a plurality of mirrors, with a first inner area of an inner space of the storage box adjacent to the sidewall on which the first mirror is installed, and a plurality of It is preferable to recognize the product inside the storage box by comparing the second reflection area, which is a partial area of the area of the second mirror among the mirrors, and the second interior area of the storage box inner space adjacent to the sidewall on which the second mirror is installed.
  • a first reflective area which is a partial area of the first mirror among a plurality of mirrors
  • a first image for comparing the first reflective area and the first inner area and a second image for comparing the second reflective area and the second inner area; It characterized in that it is obtained from.
  • the image object recognition system of the present invention when a product is stored in an internal space such as a refrigerator or a vending machine, when the product is stored in an internal space such as a refrigerator or a vending machine, the product covered by a reflective image using a mirror You can also recognize the product by securing an image for it.
  • 1 is an example of a camera image acquired for a narrow interior space.
  • FIG. 2 is a block diagram of an image object recognition system according to an embodiment of the present invention.
  • FIG 3 is an exemplary view of a plurality of mirrors installed on a side wall of an inner space of a storage box such as a refrigerator or vending machine.
  • FIG. 4 is an exemplary diagram of an image area acquired by a camera.
  • FIG. 5 is an exemplary diagram of a timing diagram of a control signal for controlling the respective transparent state of the front glass of a plurality of mirrors.
  • FIG. 6 is a comparative explanatory diagram of a region by a computing device.
  • FIG. 2 shows a configuration diagram of an image object recognition system 100 based on deep learning according to a preferred embodiment of the present invention.
  • the image object recognition system 100 based on deep learning includes a plurality of mirrors 10, a camera 20, a controller 30, and a computing device. It is comprised of (40).
  • FIG 3 is an exemplary view of a plurality of mirrors 10 installed on side walls of an inner space of a storage box S such as a refrigerator or vending machine.
  • a plurality of mirrors 10 are installed on a plurality of sidewalls inside the storage box S that can store products.
  • the mirrors 10 may not be installed on some side walls.
  • the camera 20 serves to acquire images of the inside of the storage box S and the plurality of mirrors 10.
  • FIG 4 is an exemplary diagram of an image area acquired by the camera 20.
  • the controller 30 serves to generate a control signal for controlling the operation of the plurality of mirrors 10.
  • the controller 30 may be implemented using a processor such as an MCU or the like.
  • the front glass of the plurality of mirrors 10 can be controlled in a transparent or opaque state.
  • the front glass may be transparent or opaque by attaching a PDLC film or the like. For example, when an electric signal is applied to the film of the corresponding windshield, a current flows, and the film becomes transparent from an opaque state. However, depending on the film, when an electrical signal is applied to the film, it may be changed from a transparent state to an opaque state.
  • the sections in the transparent state of each of the front glass of the plurality of mirrors 10 need to be visually different from each other.
  • each mirror 10 is sequentially transparent by the controller 30.
  • mirrors 10 other than one mirror 10 in a transparent state become opaque. Since the image is not reflected, it does not occur that the image of the opaque mirror 10 is reflected by the transparent mirror 10.
  • the computing device 40 plays a role of recognizing a product inside the storage box S by comparing pre-designated areas of the image acquired from the camera 20 with each other. Specifically, it is preferable that the computing device 40 recognizes the product inside the storage box S by deep learning. In addition, since there are images reflected by a plurality of mirrors 10 in the image acquired from the camera 20, products covered by other products in the storage box S are also recognized by the image reflected by the mirror 10. It becomes possible.
  • FIG. 6 is a comparative explanatory diagram of regions by the computing device 40.
  • the computing device 40 compares areas as follows.
  • a first reflection area which is a partial area of the first mirror, and a first inner area of the storage box (S) inner space adjacent to the sidewall on which the first mirror is installed.
  • the second reflective area which is a partial area of the second mirror, and the second inner area of the storage box (S) inner space adjacent to the sidewall on which the second mirror is installed.
  • a third reflective area which is a partial area of the third mirror, and a third inner area of the storage box (S) inner space adjacent to the side wall on which the third mirror is installed.
  • the first to fourth reflective areas are set to be limited not to all areas of the mirror, but to a partial area to which the product is reflected, so that an image area for comparison is optimized.
  • the comparison area was optimized only in the area adjacent to the mirror. Due to the limitation of this area, the computing device 40 can recognize a product through a smaller operation.
  • the first image for comparing the first reflective area and the first inner area; the second image for comparing the second reflective area and the second inner area; and the third reflective area and the third inner area; are compared.
  • the third image for comparison and the fourth image for comparing the fourth reflective area and the fourth inner area are obtained from the camera 20 visually with each other.
  • the computing device 40 may recognize unobstructed objects and hidden objects located in some areas of the storage box S by an image of a partial area of the inner space of the storage box S and its reflection image. do.
  • the present invention when a reflection image is obtained through the mirror 10 located at the edge of the lens field of view of the camera 20, an image of the obscured object may be obtained.
  • an image of the obscured object may be obtained.
  • an object image of a blind spot that may be caused by placing a product in a narrow storage box (S) such as a refrigerator can be recognized as a single camera 20 through the installation of the side mirror 10. , It can be used for detection of inventory of products, etc.
  • S narrow storage box
  • the product image inside the storage box S and the image recognized through the mirror 10 attached to the side can be distinguished.
  • the concealed object is re-reflected through the side mirror 10, in the present invention, it is not determined that it is reflected by the mirror 10, but may be determined as another object. Accordingly, in the present invention, the object recognized in the reflected area by separating the area of the image needs to be separately managed as a reflected object.
  • the reflected image there may be objects that are concealed in reverse.
  • an object of a part of the reflected image area and a part of the image of an actual inner space is compared.
  • the image object recognition system 100 of the present invention when a product is stored in an internal space such as a refrigerator or a vending machine, an image of the product covered by the reflected image using the mirror 10 is also secured. Thus, it can be seen that the product can be recognized.

Abstract

This image object recognition system comprises: a plurality of mirrors mounted to a plurality of side walls inside a container which can store a product; a camera for acquiring images of the inside of the container and the plurality of mirrors; and a controller for generating control signals for controlling the operation of the plurality of mirrors.

Description

딥러닝에 기반한 이미지 객체 인식 시스템Image object recognition system based on deep learning
본 발명은 딥러닝에 기반한 이미지 객체 인식 시스템에 관한 것이다.The present invention relates to an image object recognition system based on deep learning.
냉장고나 자동 판매기와 같은 보관함의 내부 공간의 상부 중앙에 위치한 카메라에 의해 한정된 공간 내부의 이미지를 획득할 때, 카메라 렌즈 화각의 가장자리의 객체들은 카메라 렌즈와 가깝고 큰 객체에 가려지는 경우가 빈번하다. When acquiring an image inside a limited space by a camera located in the upper center of an internal space of a storage box such as a refrigerator or vending machine, objects at the edge of the camera lens' angle of view are often obscured by a large object close to the camera lens.
도 1은 좁은 내부 공간에 대해 획득된 카메라 이미지의 예시이다.1 is an example of a camera image acquired for a narrow interior space.
도 1로부터 알 수 있는 바와 같이 'A'라고 표시된 상품 뒤에 가려진 'B' 상품의 경우, 딥러닝 시에 객체 인식에 어려움이 있다.As can be seen from FIG. 1, in the case of the product “B” hidden behind the product marked “A”, it is difficult to recognize an object during deep learning.
이러한 가려진 객체들이 많은 경우, 카메라로부터 획득된 이미지를 이용하여 해당 공간 내부의 상품인 객체를 인식하기 위한 시스템에서 딥러닝을 실시하게 되면, 가려진 객체들에 대한 올바른 인식이 어렵다.When there are many such hidden objects, if deep learning is performed in a system for recognizing an object that is a product inside a corresponding space using an image acquired from a camera, it is difficult to correctly recognize the obscured objects.
본 발명은 전술한 바와 같은 기술적 과제를 해결하는 데 목적이 있는 발명으로서, 냉장고나 자동 판매기와 같은 내부 공간에 상품이 보관된 경우, 거울을 이용한 반사 이미지에 의해 가려진 상품에 대한 이미지도 확보하여 해당 상품을 인식할 수 있는 딥러닝에 기반한 이미지 객체 인식 시스템을 제공하는 것에 그 목적이 있다.The present invention is an invention aimed at solving the technical problem as described above, and when a product is stored in an internal space such as a refrigerator or a vending machine, an image of the product covered by a reflective image using a mirror is also secured and the corresponding Its purpose is to provide an image object recognition system based on deep learning that can recognize products.
본 발명의 이미지 객체 인식 시스템은, 상품을 보관할 수 있는 보관함 내부의 다수의 측벽에 설치된 다수의 거울; 상기 보관함 내부 및 상기 다수의 거울의 이미지를 획득하는 카메라; 상기 다수의 거울의 동작을 제어하는 제어 신호를 생성하는 제어기; 및 상기 카메라로부터 획득된 이미지의 미리 지정된 영역들끼리 비교하여, 보관함 내부의 상품을 인식하는 컴퓨팅 장치;를 포함하는 것을 특징으로 한다.The image object recognition system of the present invention includes a plurality of mirrors installed on a plurality of sidewalls inside a storage box capable of storing goods; A camera that acquires images of the inside of the storage box and the plurality of mirrors; A controller for generating a control signal for controlling the operation of the plurality of mirrors; And a computing device for recognizing a product in a storage box by comparing pre-designated areas of the image obtained from the camera.
아울러, 상기 제어 신호에 의해, 다수의 거울의 전면 유리는, 투명 또는 불투명 상태로 되는 것이 바람직하다. 구체적으로, 다수의 거울의 전면 유리 각각의 투명한 상태인 구간은, 서로 이시(異時)인 것을 특징으로 한다.In addition, it is preferable that the front glass of the plurality of mirrors is in a transparent or opaque state by the control signal. Specifically, sections in the transparent state of each of the front glass of the plurality of mirrors are characterized in that they are angular to each other.
구체적으로, 상기 컴퓨팅 장치는, 다수의 거울 중 제 1 거울의 영역 중 일부 영역인 제 1 반사 영역;과 상기 제 1 거울이 설치된 측벽과 인접한 보관함 내부 공간의 제 1 내부 영역;을 비교하고, 다수의 거울 중 제 2 거울의 영역 중 일부 영역인 제 2 반사 영역;과 상기 제 2 거울이 설치된 측벽과 인접한 보관함 내부 공간의 제 2 내부 영역;을 비교하여, 상기 보관함 내부의 상품을 인식하는 것이 바람직하다.Specifically, the computing device compares a first reflective area, which is a partial area of the first mirror among a plurality of mirrors, with a first inner area of an inner space of the storage box adjacent to the sidewall on which the first mirror is installed, and a plurality of It is preferable to recognize the product inside the storage box by comparing the second reflection area, which is a partial area of the area of the second mirror among the mirrors, and the second interior area of the storage box inner space adjacent to the sidewall on which the second mirror is installed. Do.
또한, 상기 제 1 반사 영역과 상기 제 1 내부 영역;을 비교하기 위한 제 1 이미지와, 상기 제 2 반사 영역;과 상기 제 2 내부 영역;을 비교하기 위한 제 2 이미지는, 서로 이시적으로 카메라로부터 획득된 것을 특징으로 한다.In addition, a first image for comparing the first reflective area and the first inner area; and a second image for comparing the second reflective area and the second inner area; It characterized in that it is obtained from.
본 발명의 이미지 객체 인식 시스템에 따르면, 냉장고나 자동 판매기와 같은 내부 공간에 상품이 보관된 경우, 냉장고나 자동 판매기와 같은 내부 공간에 상품이 보관된 경우, 거울을 이용한 반사 이미지에 의해 가려진 상품에 대한 이미지도 확보하여 해당 상품을 인식할 수 있다.According to the image object recognition system of the present invention, when a product is stored in an internal space such as a refrigerator or a vending machine, when the product is stored in an internal space such as a refrigerator or a vending machine, the product covered by a reflective image using a mirror You can also recognize the product by securing an image for it.
도 1은 좁은 내부 공간에 대해 획득된 카메라 이미지의 예시.1 is an example of a camera image acquired for a narrow interior space.
도 2는 본 발명의 바람직한 일실시예에 따른 이미지 객체 인식 시스템의 구성도.2 is a block diagram of an image object recognition system according to an embodiment of the present invention.
도 3은 냉장고나 자동 판매기와 같은 보관함의 내부 공간 측벽에 설치된 다수의 거울의 예시도.3 is an exemplary view of a plurality of mirrors installed on a side wall of an inner space of a storage box such as a refrigerator or vending machine.
도 4는 카메라에 의해 획득되는 이미지 영역의 예시도.4 is an exemplary diagram of an image area acquired by a camera.
도 5는 다수의 거울의 전면 유리의 각각의 투명 상태를 제어하기 위한 제어 신호의 타이밍도의 예시도.5 is an exemplary diagram of a timing diagram of a control signal for controlling the respective transparent state of the front glass of a plurality of mirrors.
도 6은 컴퓨팅 장치에 의한 영역의 비교 설명도.6 is a comparative explanatory diagram of a region by a computing device.
이하, 첨부된 도면을 참조하면서 본 발명의 실시예에 따른 딥러닝에 기반한 이미지 객체 인식 시스템에 대해 상세히 설명하기로 한다. 본 발명의 하기의 실시예는 본 발명을 구체화하기 위한 것일 뿐 본 발명의 권리 범위를 제한하거나 한정하는 것이 아님은 물론이다. 본 발명의 상세한 설명 및 실시예로부터 본 발명이 속하는 기술 분야의 전문가가 용이하게 유추할 수 있는 것은 본 발명의 권리 범위에 속하는 것으로 해석된다.Hereinafter, an image object recognition system based on deep learning according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings. It goes without saying that the following examples of the present invention are for embodiing the present invention and do not limit or limit the scope of the present invention. What can be easily inferred by experts in the technical field to which the present invention pertains from the detailed description and examples of the present invention is interpreted as belonging to the scope of the present invention.
먼저, 도 2는 본 발명의 바람직한 일실시예에 따른 딥러닝에 기반한 이미지 객체 인식 시스템(100)의 구성도를 나타낸다.First, FIG. 2 shows a configuration diagram of an image object recognition system 100 based on deep learning according to a preferred embodiment of the present invention.
도 2로부터 알 수 있는 바와 같이, 본 발명의 바람직한 일실시예에 따른 딥러닝에 기반한 이미지 객체 인식 시스템(100)은, 다수의 거울(10), 카메라(20), 제어기(30) 및 컴퓨팅 장치(40)를 포함하여 구성된다.As can be seen from FIG. 2, the image object recognition system 100 based on deep learning according to a preferred embodiment of the present invention includes a plurality of mirrors 10, a camera 20, a controller 30, and a computing device. It is comprised of (40).
도 3은 냉장고나 자동 판매기와 같은 보관함(S)의 내부 공간 측벽에 설치된 다수의 거울(10)의 예시도이다.3 is an exemplary view of a plurality of mirrors 10 installed on side walls of an inner space of a storage box S such as a refrigerator or vending machine.
즉, 다수의 거울(10)은, 상품을 보관할 수 있는 보관함(S) 내부의 다수의 측벽에 설치된다. 아울러, 모든 측벽에 거울(10)이 모두 설치될 수 없는 경우라면, 일부 측벽에 대해서는, 거울이 설치되지 않을 수 있음은 물론이다.That is, a plurality of mirrors 10 are installed on a plurality of sidewalls inside the storage box S that can store products. In addition, if all the mirrors 10 cannot be installed on all side walls, it goes without saying that the mirrors may not be installed on some side walls.
카메라(20)는, 보관함(S) 내부 및 다수의 거울(10)의 이미지를 획득하는 역할을 한다. The camera 20 serves to acquire images of the inside of the storage box S and the plurality of mirrors 10.
도 4는 카메라(20)에 의해 획득되는 이미지 영역의 예시도이다.4 is an exemplary diagram of an image area acquired by the camera 20.
제어기(30)는, 다수의 거울(10)의 동작을 제어하는 제어 신호를 생성하는 역할을 한다. 제어기(30)는 MCU 등과 같은 프로세서를 이용하여 구현될 수 있다. 제어 신호에 의해, 다수의 거울(10)의 전면 유리는, 투명 또는 불투명 상태로 제어될 수 있다. 전면 유리는 PDLC 필름 등을 부착하여 투명 또는 불투명 상태의 천이를 가능하게 할 수 있다. 예를 들면, 해당 전면 유리의 필름에 전기 신호가 인가되면 전류가 흘러 불투명 상태로부터 투명 상태가 될 수 있다. 다만, 필름에 따라서는, 필름에 전기 신호가 인가될 경우, 투명 상태로부터 불투명 상태로 될 수도 있다. The controller 30 serves to generate a control signal for controlling the operation of the plurality of mirrors 10. The controller 30 may be implemented using a processor such as an MCU or the like. By the control signal, the front glass of the plurality of mirrors 10 can be controlled in a transparent or opaque state. The front glass may be transparent or opaque by attaching a PDLC film or the like. For example, when an electric signal is applied to the film of the corresponding windshield, a current flows, and the film becomes transparent from an opaque state. However, depending on the film, when an electrical signal is applied to the film, it may be changed from a transparent state to an opaque state.
다수의 거울(10)이 동시에 동작할 경우, 보관함(S) 내부의 이미지가 거울(10) 각각에 반사될 뿐만 아니라, 다른 거울(10)에 반사된 이미지도 반사되어, 실제 반사된 보관함(S) 내부의 이미지를 특정하는 것에 어려움이 있다. When a plurality of mirrors 10 operate at the same time, not only the image inside the storage box S is reflected to each of the mirrors 10, but also the image reflected by the other mirrors 10 is reflected, so that the actually reflected storage box (S ) Difficulty in specifying the image inside.
이러한 문제를 해결하고자, 다수의 거울(10)의 전면 유리 각각의 투명한 상태인 구간은, 서로 이시(異時)적일 필요가 있다.In order to solve this problem, the sections in the transparent state of each of the front glass of the plurality of mirrors 10 need to be visually different from each other.
도 5는 다수의 거울(10)의 전면 유리의 각각의 투명 상태를 제어하기 위한 제어 신호의 타이밍도의 예시도이다. 도 5로부터 알 수 있는 바와 같이, 제어기(30)에 의해 각각의 거울(10)은 순차적으로 투명한 상태가 되는 것이 바람직하다.5 is an exemplary diagram of a timing diagram of a control signal for controlling each transparent state of the front glass of the plurality of mirrors 10. As can be seen from FIG. 5, it is preferable that each mirror 10 is sequentially transparent by the controller 30.
도 5와 같이, 이시적으로 다수의 거울(10)의 전면 유리 각각의 투명한 상태인 구간을 설정하는 것에 의해, 투명한 상태의 하나의 거울(10) 이외의 거울(10)은 불투명한 상태가 되어 이미지가 반사되지 않으므로, 불투명한 거울(10)의 이미지가 투명한 거울(10)에 반사되는 일 또한 발생하지 않는다.As shown in FIG. 5, by setting a section that is in a transparent state of each of the front glass of the plurality of mirrors 10, mirrors 10 other than one mirror 10 in a transparent state become opaque. Since the image is not reflected, it does not occur that the image of the opaque mirror 10 is reflected by the transparent mirror 10.
컴퓨팅 장치(40)는, 카메라(20)로부터 획득된 이미지의 미리 지정된 영역들끼리 비교하여, 보관함(S) 내부의 상품을 인식하는 역할을 한다. 구체적으로 컴퓨팅 장치(40)는, 딥러닝에 의해 보관함(S) 내부의 상품을 인식하는 것이 바람직하다. 아울러, 카메라(20)로부터 획득된 이미지에는 다수의 거울(10)에 반사된 이미지도 존재하므로, 보관함(S) 내부에서 다른 상품에 의해 가려진 상품도 거울(10)에 반사된 이미지에 의해 인식이 가능하게 된다.The computing device 40 plays a role of recognizing a product inside the storage box S by comparing pre-designated areas of the image acquired from the camera 20 with each other. Specifically, it is preferable that the computing device 40 recognizes the product inside the storage box S by deep learning. In addition, since there are images reflected by a plurality of mirrors 10 in the image acquired from the camera 20, products covered by other products in the storage box S are also recognized by the image reflected by the mirror 10. It becomes possible.
도 6은 컴퓨팅 장치(40)에 의한 영역의 비교 설명도이다.6 is a comparative explanatory diagram of regions by the computing device 40.
도 6으로부터 알 수 있는 바와 같이, 컴퓨팅 장치(40)는, 다음과 같이 영역을 비교한다.As can be seen from FIG. 6, the computing device 40 compares areas as follows.
(1) 제 1 거울의 영역 중 일부 영역인 제 1 반사 영역과 제 1 거울이 설치된 측벽과 인접한 보관함(S) 내부 공간의 제 1 내부 영역.(1) A first reflection area, which is a partial area of the first mirror, and a first inner area of the storage box (S) inner space adjacent to the sidewall on which the first mirror is installed.
(2) 제 2 거울의 영역 중 일부 영역인 제 2 반사 영역과 제 2 거울이 설치된 측벽과 인접한 보관함(S) 내부 공간의 제 2 내부 영역.(2) The second reflective area, which is a partial area of the second mirror, and the second inner area of the storage box (S) inner space adjacent to the sidewall on which the second mirror is installed.
(3) 제 3 거울의 영역 중 일부 영역인 제 3 반사 영역과 제 3 거울이 설치된 측벽과 인접한 보관함(S) 내부 공간의 제 3 내부 영역.(3) A third reflective area, which is a partial area of the third mirror, and a third inner area of the storage box (S) inner space adjacent to the side wall on which the third mirror is installed.
(4) 제 4 거울의 영역 중 일부 영역인 제 4 반사 영역과 제 4 거울이 설치된 측벽과 인접한 보관함(S) 내부 공간의 제 4 내부 영역.(4) A fourth inner area of the inner space of the storage box (S) adjacent to the side wall on which the fourth mirror is installed and the fourth reflective area, which is a partial area of the fourth mirror.
본 발명에서는 제 1 반사 영역 내지 제 4 반사 영역은 해당 거울의 모든 영역이 아니라, 상품이 반사되는 일부 영역으로 제한되어 설정되어, 비교하기 위한 이미지 영역을 최적화하였다. 마찬가지로, 보관함(S) 내부 공간에 대해서도 해당 거울과 인접한 영역만으로 비교 영역을 최적화하였다. 이러한 영역의 제한에 의해 컴퓨팅 장치(40)는 보다 작은 연산에 의해 상품을 인식할 수 있게 된다.In the present invention, the first to fourth reflective areas are set to be limited not to all areas of the mirror, but to a partial area to which the product is reflected, so that an image area for comparison is optimized. Similarly, for the interior space of the storage box (S), the comparison area was optimized only in the area adjacent to the mirror. Due to the limitation of this area, the computing device 40 can recognize a product through a smaller operation.
아울러, 제 1 반사 영역과 제 1 내부 영역;을 비교하기 위한 제 1 이미지, 제 2 반사 영역과 제 2 내부 영역;을 비교하기 위한 제 2 이미지, 제 3 반사 영역과 제 3 내부 영역;을 비교하기 위한 제 3 이미지 및 제 4 반사 영역과 제 4 내부 영역;을 비교하기 위한 제 4 이미지는, 서로 이시적으로 카메라(20)로부터 획득된 것을 특징으로 한다. In addition, the first image for comparing the first reflective area and the first inner area; the second image for comparing the second reflective area and the second inner area; and the third reflective area and the third inner area; are compared. The third image for comparison and the fourth image for comparing the fourth reflective area and the fourth inner area are obtained from the camera 20 visually with each other.
즉, 해당 거울이 투명한 상태가 되어 보관함(S) 내부 공간의 이미지를 반사하는 경우에만, 해당 반사 영역과 내부 공간의 비교가 의미가 있는 까닭이다.That is, this is because the comparison between the reflective area and the inner space is meaningful only when the mirror becomes transparent and reflects the image of the inner space of the storage box (S).
또한, 컴퓨팅 장치(40)는, 보관함(S) 내부 공간 중 일부 영역의 이미지 및 그 반사 이미지에 의해, 보관함(S) 내부 공간 중 일부 영역에 위치한 가려지지 않은 객체 및 가려진 객체를 인식할 수 있게 된다.In addition, the computing device 40 may recognize unobstructed objects and hidden objects located in some areas of the storage box S by an image of a partial area of the inner space of the storage box S and its reflection image. do.
상술한 바와 본 발명의 특징을 정리해 보기로 한다.As described above, the features of the present invention will be summarized.
본 발명에서는, 카메라(20) 렌즈 화각의 가장자리에 위치한 거울(10)을 통해 반사 이미지를 획득하면 가려진 객체에 대한 이미지를 획득할 수 있다. 아울러, 본 발명에서는, 다수의 거울(10)을 통해 다수의 반사 이미지를 획득하더라도, 가장자리에 해당하는 반사 이미지의 일부만 딥러닝을 위한 입력으로 하여, 부하를 줄이고 판단이 용이한 객체를 기준으로 은폐된 객체가 없는지 판단한다.In the present invention, when a reflection image is obtained through the mirror 10 located at the edge of the lens field of view of the camera 20, an image of the obscured object may be obtained. In addition, in the present invention, even if a plurality of reflection images are acquired through a plurality of mirrors 10, only a part of the reflection image corresponding to the edge is used as an input for deep learning, thereby reducing the load and concealing based on an object that is easy to determine. Determine whether there is no object that has been created.
본 발명은 냉장고와 같은 협소한 보관함(S)의 내부 공간에 상품을 배치하는 것에 의해 발생할 수 있는 사각지대의 객체 이미지까지 측면 거울(10) 설치를 통해 하나의 카메라(20)로 인식할 수 있어, 상품의 재고의 검출 등에 활용될 수 있다. In the present invention, an object image of a blind spot that may be caused by placing a product in a narrow storage box (S) such as a refrigerator can be recognized as a single camera 20 through the installation of the side mirror 10. , It can be used for detection of inventory of products, etc.
카메라(20)를 통해 캡처한 이미지를 컴퓨팅 장치(40)의 딥러닝 모듈 입력으로 보내면, 실제 보관함(S) 내부의 상품 이미지와 측면에 부착된 거울(10)을 통해 인식한 이미지를 구분할 수 있다. 아울러, 은폐된 객체가 측면 거울(10)을 통해 재반사되었다면, 본 발명에서는 거울(10)에 반사되었다고 판단하지 않고 또 다른 객체로 판단할 수 있다. 따라서, 본 발명에서는 이미지의 영역을 분리시켜 반사된 영역에서 인식한 객체는 반사된 객체라고 별도 관리할 필요가 있다.When the image captured through the camera 20 is sent to the deep learning module input of the computing device 40, the product image inside the storage box S and the image recognized through the mirror 10 attached to the side can be distinguished. . In addition, if the concealed object is re-reflected through the side mirror 10, in the present invention, it is not determined that it is reflected by the mirror 10, but may be determined as another object. Accordingly, in the present invention, the object recognized in the reflected area by separating the area of the image needs to be separately managed as a reflected object.
반사 이미지에서도 역으로 은폐가 된 객체가 있을 수 있다. 이를 해결하고자 본 발명에서는, 반사된 이미지 영역의 일부와 가운데 실제 내부 공간의 이미지의 일부 영역의 객체의 비교를 실시한다.In the reflected image, there may be objects that are concealed in reverse. In order to solve this problem, in the present invention, an object of a part of the reflected image area and a part of the image of an actual inner space is compared.
상술한 바와 같이, 본 발명의 이미지 객체 인식 시스템(100)에 따르면, 냉장고나 자동 판매기와 같은 내부 공간에 상품이 보관된 경우, 거울(10)을 이용한 반사 이미지에 의해 가려진 상품에 대한 이미지도 확보하여 해당 상품을 인식할 수 있음을 알 수 있다.As described above, according to the image object recognition system 100 of the present invention, when a product is stored in an internal space such as a refrigerator or a vending machine, an image of the product covered by the reflected image using the mirror 10 is also secured. Thus, it can be seen that the product can be recognized.

Claims (6)

  1. 이미지 객체 인식 시스템에 있어서,In the image object recognition system,
    상품을 보관할 수 있는 보관함 내부의 다수의 측벽에 설치된 다수의 거울; 및A plurality of mirrors installed on a plurality of side walls inside a storage box capable of storing goods; And
    상기 보관함 내부 및 상기 다수의 거울의 이미지를 획득하는 카메라;를 포함하는 것을 특징으로 하는 이미지 객체 인식 시스템.And a camera that acquires images of the inside of the storage box and the plurality of mirrors.
  2. 제1항에 있어서,The method of claim 1,
    상기 이미지 객체 인식 시스템은,The image object recognition system,
    상기 다수의 거울의 동작을 제어하는 제어 신호를 생성하는 제어기;를 더 포함하되,Further comprising a; a controller for generating a control signal for controlling the operation of the plurality of mirrors,
    상기 제어 신호에 의해, 다수의 거울의 전면 유리는,By the control signal, the front glass of the plurality of mirrors,
    투명 또는 불투명 상태로 되는 것을 특징으로 하는 이미지 객체 인식 시스템.Image object recognition system, characterized in that the transparent or opaque state.
  3. 제2항에 있어서,The method of claim 2,
    다수의 거울의 전면 유리 각각에 대해 투명한 상태인 구간은,The section that is transparent to each of the front glass of multiple mirrors,
    서로 이시(異時)인 것을 특징으로 하는 이미지 객체 인식 시스템.An image object recognition system, characterized in that they are at the same time.
  4. 제1항에 있어서,The method of claim 1,
    상기 이미지 객체 인식 시스템은,The image object recognition system,
    상기 카메라로부터 획득된 이미지의 미리 지정된 영역들끼리 비교하여, 보관함 내부의 상품을 인식하는 컴퓨팅 장치;를 더 포함하는 것을 특징으로 하는 이미지 객체 인식 시스템.And a computing device for recognizing a product in a storage box by comparing pre-designated areas of the image obtained from the camera.
  5. 제4항에 있어서,The method of claim 4,
    상기 컴퓨팅 장치는,The computing device,
    다수의 거울 중 제 1 거울의 영역 중 일부 영역인 제 1 반사 영역;과 상기 제 1 거울이 설치된 측벽과 인접한 보관함 내부 공간의 제 1 내부 영역;을 비교하고, A first reflection area, which is a partial area of the first mirror among the plurality of mirrors, is compared with a first inner area of an inner space of the storage box adjacent to the sidewall on which the first mirror is installed,
    다수의 거울 중 제 2 거울의 영역 중 일부 영역인 제 2 반사 영역;과 상기 제 2 거울이 설치된 측벽과 인접한 보관함 내부 공간의 제 2 내부 영역;을 비교하여, Comparing a second reflective area, which is a partial area of the second mirror among the plurality of mirrors, and a second inner area of the inner space of the storage box adjacent to the sidewall on which the second mirror is installed,
    상기 보관함 내부의 상품을 인식하는 것을 특징으로 하는 이미지 객체 인식 시스템.An image object recognition system, characterized in that recognizing a product inside the storage box.
  6. 제5항에 있어서,The method of claim 5,
    상기 제 1 반사 영역과 상기 제 1 내부 영역;을 비교하기 위한 제 1 이미지와, 상기 제 2 반사 영역;과 상기 제 2 내부 영역;을 비교하기 위한 제 2 이미지는,A first image for comparing the first reflective area and the first inner area; and a second image for comparing the second reflective area and the second inner area;
    서로 이시적으로 카메라로부터 획득된 것을 특징으로 하는 이미지 객체 인식 시스템.Image object recognition system, characterized in that obtained from a camera visually from each other.
PCT/KR2020/010531 2019-11-14 2020-08-10 Image object recognition system based on deep learning WO2021096030A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020190145762A KR102294115B1 (en) 2019-11-14 2019-11-14 Object recognition system of image based on deep learning
KR10-2019-0145762 2019-11-14

Publications (1)

Publication Number Publication Date
WO2021096030A1 true WO2021096030A1 (en) 2021-05-20

Family

ID=75912100

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2020/010531 WO2021096030A1 (en) 2019-11-14 2020-08-10 Image object recognition system based on deep learning

Country Status (2)

Country Link
KR (1) KR102294115B1 (en)
WO (1) WO2021096030A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20230166304A (en) 2022-05-30 2023-12-07 경북대학교 산학협력단 Deep learning-based radar sensor failure classification device and method using camera and radar sensor

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016196981A (en) * 2015-04-03 2016-11-24 三菱電機株式会社 Storage shed management system and storage shed management method
JP2019023592A (en) * 2017-07-24 2019-02-14 シブヤ精機株式会社 Article inspection device
CN208781326U (en) * 2018-08-07 2019-04-23 上海韬林机械有限公司 A kind of Vending Machine of visual identity commodity
KR101991424B1 (en) * 2018-02-09 2019-06-20 충북대학교 산학협력단 Refrigerator with visible device for inner side
JP2019168134A (en) * 2018-03-22 2019-10-03 三菱電機株式会社 Refrigerator system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016196981A (en) * 2015-04-03 2016-11-24 三菱電機株式会社 Storage shed management system and storage shed management method
JP2019023592A (en) * 2017-07-24 2019-02-14 シブヤ精機株式会社 Article inspection device
KR101991424B1 (en) * 2018-02-09 2019-06-20 충북대학교 산학협력단 Refrigerator with visible device for inner side
JP2019168134A (en) * 2018-03-22 2019-10-03 三菱電機株式会社 Refrigerator system
CN208781326U (en) * 2018-08-07 2019-04-23 上海韬林机械有限公司 A kind of Vending Machine of visual identity commodity

Also Published As

Publication number Publication date
KR102294115B1 (en) 2021-08-26
KR20210058389A (en) 2021-05-24

Similar Documents

Publication Publication Date Title
EP0432680B1 (en) Monitoring system employing infrared image
CN101295355B (en) Face image capturing apparatus
US7986812B2 (en) On-vehicle camera with two or more angles of view
WO2012023639A1 (en) Method for counting objects and apparatus using a plurality of sensors
IL163681A0 (en) Imaging system for a passenger bridge or the like for docking automatically with an aircraft
WO2021096030A1 (en) Image object recognition system based on deep learning
WO2017195965A1 (en) Apparatus and method for image processing according to vehicle speed
JPH11224160A (en) Coordinate input system and display device
WO2021172833A1 (en) Object recognition device, object recognition method and computer-readable recording medium for performing same
WO2022014851A1 (en) Lidar system capable of setting detection area
CN112550151A (en) Camera sharing system of automatic driving vehicle
WO2023120818A1 (en) Traffic flow control device for controlling traffic flow in which autonomous vehicles are mixed, and method using same
WO2017111201A1 (en) Night image display apparatus and image processing method thereof
WO2019017692A1 (en) Tof module and object recognition device using tof module
CA2171987A1 (en) Multi-direction camera
US5748910A (en) Automatic enabling/disabling of termination impedance for a computer bus
CN104980630A (en) Surveillance Camera
WO2018074707A1 (en) Rain sensor for vehicle and windshield wiper driving device of vehicle having same
CN113364989B (en) Camera strobe control method and device, electronic equipment and storage medium
WO2021075629A1 (en) Signage content display device
WO2023158233A1 (en) Camera device and license plate recognition system having same
WO2024029701A1 (en) Projector and control method thereof
JP2819695B2 (en) Projector device
US20220375064A1 (en) Vehicle imaging station
KR102214022B1 (en) Method for identificating traffic lights, device and program using the same

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20886271

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20886271

Country of ref document: EP

Kind code of ref document: A1