WO2023073869A1 - 扉開角度算出方法及び収納庫 - Google Patents

扉開角度算出方法及び収納庫 Download PDF

Info

Publication number
WO2023073869A1
WO2023073869A1 PCT/JP2021/039840 JP2021039840W WO2023073869A1 WO 2023073869 A1 WO2023073869 A1 WO 2023073869A1 JP 2021039840 W JP2021039840 W JP 2021039840W WO 2023073869 A1 WO2023073869 A1 WO 2023073869A1
Authority
WO
WIPO (PCT)
Prior art keywords
door
edge
opening angle
see
refrigerator compartment
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/039840
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
勝 青山
慎太郎 草野
禎多 関口
孝典 三浦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hitachi Power Solutions Co Ltd
Hitachi Global Life Solutions Inc
Original Assignee
Hitachi Power Solutions Co Ltd
Hitachi Global Life Solutions Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hitachi Power Solutions Co Ltd, Hitachi Global Life Solutions Inc filed Critical Hitachi Power Solutions Co Ltd
Priority to PCT/JP2021/039840 priority Critical patent/WO2023073869A1/ja
Priority to JP2022577371A priority patent/JP7426514B2/ja
Priority to CN202180100589.0A priority patent/CN117651843A/zh
Publication of WO2023073869A1 publication Critical patent/WO2023073869A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
    • F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
    • F25D23/00—General constructional features
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/26—Measuring arrangements characterised by the use of optical techniques for measuring angles or tapers; for testing the alignment of axes

Definitions

  • This disclosure relates to a door opening angle calculation method and the like.
  • Patent Document 1 describes "In a refrigerator equipped with a volume resistor attached to a hinge of a refrigerator compartment, ... Detecting the opening and closing angle" is described. Further, Patent Document 2 describes that "a new image is created using data of a plurality of the images captured by the imaging unit from different angles based on the opening/closing angle of the door.”
  • JP 2005-164196 A Japanese Patent No. 6323279
  • Patent Document 1 requires a volume resistor to be provided in the hinge of the refrigerator door, which leads to an increase in manufacturing costs. Further, the technique described in Patent Document 2 captures images of the interior of the refrigerator from various angles to create a three-dimensional composite image, but there is still room for simplification of the process.
  • a first edge which is an edge of the housing on the captured image
  • the edge of the door on the captured image is calculated based on the number of pixels between the second edge, which is an edge.
  • FIG. 2 is a front view of the refrigerator according to the first embodiment with left and right refrigerating compartment doors opened;
  • FIG. 2 is a front view of the refrigerator according to the first embodiment with doors, shelves, etc. removed.
  • Fig. 2 is a plan view of the refrigerator according to the first embodiment with left and right refrigerator compartment doors opened; It is a perspective view at the time of looking up at the camera unit of the refrigerator which concerns on 1st Embodiment from diagonally below.
  • FIG. 6B is a cross-sectional view taken along line II-II of FIG. 6A in the refrigerator according to the first embodiment.
  • 1 is a system configuration diagram including a camera unit of a refrigerator according to a first embodiment;
  • FIG. It is an example of a state in which image processing is not performed, which is a result of photographing by the camera unit of the refrigerator according to the first embodiment.
  • It is an example of a state in which predetermined image processing has been performed as a result of photographing by the camera unit of the refrigerator according to the first embodiment.
  • 4 is a flowchart of processing executed by a control unit of the refrigerator according to the first embodiment;
  • FIG. 4 is an explanatory diagram relating to extraction of feature points on a photographed image of the refrigerator according to the first embodiment;
  • FIG. 4 is an explanatory diagram showing a series of feature points on a photographed image of the refrigerator according to the first embodiment;
  • FIG. 4 is an explanatory diagram showing a trapezoidal portion included in a region on a photographed image of the refrigerator according to the first embodiment; It is explanatory drawing regarding the trapezoidal part between the 1st edge and 2nd edge in the refrigerator which concerns on 1st Embodiment.
  • FIG. 4 is an explanatory diagram showing the relationship between the area of the trapezoidal portion and the opening angle of the refrigerator compartment door in the refrigerator according to the first embodiment;
  • FIG. 10 is an explanatory diagram of a state in which the opening angle of the refrigerator compartment door is less than a predetermined value in the refrigerator according to the second embodiment;
  • FIG. 10 is an explanatory diagram of a state in which the opening angle of the refrigerating compartment door is equal to or greater than a predetermined value in the refrigerator according to the second embodiment
  • FIG. 11 is an explanatory diagram including a region on a photographed image of a refrigerator according to a modification, a first edge, and a second edge;
  • FIG. 1 is a front view of refrigerator 100 according to the first embodiment.
  • a refrigerator 100 storage box
  • the housing 1 has a plurality of storage chambers inside.
  • a refrigerating compartment 21 As the storage compartments of the refrigerator 100, a refrigerating compartment 21, an ice making compartment 22, an upper freezing compartment 23, a vegetable compartment 24, and a lower freezing compartment 25 are arranged in order from the top.
  • a refrigerating compartment 21 As the storage compartments of the refrigerator 100, a refrigerating compartment 21, an ice making compartment 22, an upper freezing compartment 23, a vegetable compartment 24, and a lower freezing compartment 25 are arranged in order from the top.
  • the refrigerator 100 includes left and right refrigerator compartment doors 211 and 212 (doors) as hinged doors that form a refrigerator compartment 21 (storage compartment) together with the housing 1 .
  • the left refrigerating compartment door 211 is rotatable around the axis of a hinge 211a (see FIG. 5). The same applies to the refrigerator compartment door 212 on the right side.
  • Refrigerator 100 also includes an icemaker door 221 and an upper freezer compartment door 231, as well as a vegetable compartment door 241 and a lower freezer compartment door 251, as drawer doors.
  • the refrigerator compartment 21 is provided with a plurality of shelf plates 213 (see FIG. 3) that partition the refrigerator compartment 21 in a predetermined manner.
  • a plurality of door pockets 211c (see FIG. 3) for storing food and the like are provided inside the refrigerator compartment door 211 on the left side (the same applies to the refrigerator compartment door 212 on the right side).
  • the ice making chamber 22 shown in FIG. 1 is provided with an ice making chamber container (not shown) that is pulled out integrally with the ice making chamber door 221 .
  • the upper freezer compartment 23 is provided with an upper freezer compartment container (not shown), and the vegetable compartment 24 is provided with a vegetable compartment container (not shown).
  • a lower freezer compartment container (not shown) is provided.
  • the refrigerator 100 includes a compressor, a radiator (condenser), a capillary tube (throttle mechanism), and an evaporator.
  • the refrigerant circulates sequentially through the compressor, radiator, capillary tube, and evaporator, and the air in the storage room is cooled by heat exchange with the refrigerant flowing through the evaporator.
  • the camera unit 3 shown in FIG. 1 is for taking an image of the storage room of the refrigerator 100 and the like, and is installed on the upper surface of the housing 1 .
  • FIG. 2 is a side view of refrigerator 100 .
  • the camera unit 3 includes a body portion 31 and a support portion 32.
  • the main unit 31 is for capturing an image of the storage room of the refrigerator 100 and the like.
  • the support portion 32 supports the body portion 31 and is installed on the upper surface of the housing 1 .
  • a lens 31a (see also FIG. 6) is provided near the front end of the main body 31.
  • the lens 31a is an optical element that refracts and converges light.
  • a fish-eye lens for example, is used as such a lens 31a.
  • the lens 31a faces downward so that the camera unit 3 can capture an image of the refrigerator compartment 21 (see FIG. 3). It's becoming
  • the lens 31a is positioned forward of the front end of the housing 1 (that is, the opening 10a). More preferably, the lens 31a is positioned further forward than the front surfaces of the refrigerator compartment doors 211 and 212 (see FIG. 1) in the closed state. As a result, for example, when the refrigerator compartment doors 211 and 212 are opened, the refrigerator compartment 21 can easily enter the field of view of the lens 31a. It is also possible to take an image of the vegetable compartment 24 and the like (see FIG. 1) with the camera unit 3 in a state where the drawer-type door such as the vegetable compartment door 241 is opened. In this manner, the lens 31a is provided at a position where each storage chamber can be viewed from above.
  • FIG. 3 is a front view of refrigerator 100 with left and right refrigerator compartment doors 211 and 212 opened.
  • the inner plate 211b of the left refrigerating compartment door 211 is provided with a plurality of door pockets 211c for storing foodstuffs, etc. (the same applies to the right refrigerating compartment door 212).
  • the left and right refrigerator compartment doors 211 and 212 are opened, the foodstuffs in the refrigerator compartment 21 and the door pockets 211c and 212c enter the visual field of the lens 31a of the camera unit 3 (FIG. 9). see also).
  • a rectangular frame-shaped packing 211d is provided near the edge of the inner plate 211b.
  • the packing 211d is a member made of resin for increasing the airtightness of the refrigerator compartment 21 .
  • the packing 211d contacts (adheres) around the opening 10a of the housing 1 (see FIG. 4).
  • a side piece 211e shown in FIG. 3 is a plate-like member forming a side surface of the refrigerator compartment door 211 and elongated in the vertical direction.
  • the refrigerator compartment door 211 also includes another side piece 211f (see FIG. 5) provided near the axis of the hinge 211a. I have.
  • These side pieces 211e and 211f both extend vertically and are provided parallel to the axis of the hinge 211a (see FIG. 5).
  • the side pieces 211e and 211f may be adjacent to the rectangular frame-shaped packing 211d (see FIG. 3), or may partially overlap the packing 211d. The same applies to the refrigerator compartment door 212 on the right side.
  • the left and right refrigerator compartment doors 211 and 212 are each provided with a shooting button 5 that is pressed by the user.
  • one photographing button 5 is provided below each of the left and right side pieces 211e and 212e.
  • FIG. 4 is a front view of refrigerator 100 with doors, shelves, and the like removed.
  • the front side (front side) of the housing 1 is provided with an opening 10a corresponding to the refrigerator compartment 21 and openings 10b, 10c, 10d, and 10e corresponding to the other compartments.
  • the opening 10a of the refrigerator compartment 21 is surrounded by a left horizontal plate 1a, a right horizontal plate 1b, an upper horizontal plate 1c, and a heat insulating partition wall 1d.
  • the heat insulating partition wall 1 d is a heat insulating wall that partitions the refrigerator compartment 21 and the ice making compartment 22 and also partitions the refrigerator compartment 21 and the upper freezer compartment 23 .
  • the horizontal plates 1a, 1b, and 1c are thin plate-like members forming part of the front surface of the housing 1, and are installed near the opening 10a of the inner box 12.
  • the left horizontal plate 1a is provided on the left side of the opening 10a of the refrigerator compartment 21 and extends parallel (that is, vertically) to the axis of the hinge 211a (see FIG. 5).
  • the right side plate 1b is provided on the right side of the opening 10a of the refrigerator compartment 21 and extends parallel to the axis of the hinge 212a (see FIG. 5).
  • These horizontal plates 1a and 1b are elongated vertically to the left of the opening 10a of the refrigerator compartment 21, the opening 10b of the ice making compartment 22, the opening 10d of the vegetable compartment 24, and the opening 10e of the lower freezer compartment 25. ing.
  • the upper horizontal plate 1c extends slenderly in the horizontal direction so as to connect the vicinity of the upper end of the left horizontal plate 1a and the vicinity of the upper end of the right horizontal plate 1b.
  • the rectangular frame-shaped packing 211d see FIG. 3 of this refrigerating compartment door 211 moves toward the left lateral plate 1a and the upper lateral plate 1c. In addition to (left half), it is in close contact with the front of the heat insulating partition wall 1d.
  • FIG. 5 is a plan view of refrigerator 100 with left and right refrigerator compartment doors 211 and 212 opened.
  • the upper surface of the housing 1 is provided with hinges 211a and 212a.
  • the left hinge 211a rotatably supports the refrigerating compartment door 211 (similarly to the right hinge 212a).
  • the camera unit 3 is arranged slightly to the left of the center of the housing 1 of the refrigerator 100 in the horizontal direction. More specifically, the camera unit 3 is arranged directly above the seam between the closed refrigerator doors 211 and 212 (see also FIG. 1). This makes it easier for food items in the door pockets 211c and 212c to enter the field of view of the camera unit 3 when the refrigerator compartment doors 211 and 212 are opened.
  • FIG. 6 is a perspective view of the camera unit 3 as viewed obliquely from below.
  • a body portion 31 of the camera unit 3 shown in FIG. 6 includes a case 31b and a cover 31c in addition to the lens 31a (see also FIG. 2).
  • a case 31b of the main body portion 31 generally has a rectangular parallelepiped shape elongated in the front-rear direction.
  • a circular hole (not shown) is provided on the lower surface near the front end of the case 31b, through which the lens 31a is exposed.
  • an elongated hole (reference numeral not shown) is provided in the rear side (deep side) of the lens 31a in the left-right direction, and the cover 31c is fitted into this hole.
  • the cover 31c is a translucent resin member that protects the camera LED 31d (illumination unit: see FIG. 7B).
  • the cover 31c is subjected to a predetermined texturing process.
  • the "texture” refers to a fine uneven shape formed on the surface of the cover 31c.
  • FIG. 7A is a front view of the camera unit 3.
  • the body portion 31 of the camera unit 3 is installed above the support portion 32 . Further, the body portion 31 is installed near the center of the support portion 32 in the left-right direction.
  • the camera unit 3 includes a lens 31a (see also FIG. 6), a cover 31c (see also FIG. 6), and a case 31b, as well as a camera LED 31d and a circuit board 31e. I have.
  • the camera LED 31d (illumination unit) is installed in the refrigerator compartment 21 and the like so that the camera unit 3 can photograph the refrigerator compartment 21 (see FIG. 3) and the door pockets 211c and 212c (see FIG. 3) under moderate brightness.
  • This camera LED 31d is provided outside the housing 1 of the refrigerator 100 (see FIG. 3).
  • a translucent cover 31c (see also FIG. 6) is provided below the camera LED 31d.
  • the camera LED 31d be positioned in front of the front end (the opening 10a of the housing 1) of the housing 1 (see FIG. 2) of the refrigerator 100. This makes it easier for the light emitted from the camera LED 31d to enter the refrigerating compartment 21 and the like, which in turn makes it easier to obtain a clear photographing result.
  • the circuit board 31e is a printed board on which an imaging device 31f (see FIG. 8) and a camera/communication control SoC 31g (System-on-a-Chip: see FIG. 8), which will be described later, are mounted.
  • an imaging device 31f see FIG. 8
  • a camera/communication control SoC 31g System-on-a-Chip: see FIG. 8
  • FIG. 8 is a system configuration diagram including the camera unit 3 of the refrigerator 100.
  • refrigerator 100 includes door sensor 4 , photographing button 5 , and control section 6 in addition to camera unit 3 .
  • Door sensor 4 outputs a predetermined signal indicating the open/closed state of refrigerator compartment doors 211 and 212 to control unit 6 .
  • FIG. 8 shows one door sensor 4, it actually corresponds to the door sensor 4 corresponding to the left refrigerator compartment door 211 (see FIG. 1) and the right refrigerator compartment door 212 (see FIG. 1).
  • the shooting button 5 (see also FIG. 3) is a button that is pushed by the user when shooting manually using the camera unit 3, as described above.
  • the control unit 6 is, for example, a microcomputer, and includes electronic circuits such as a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and various interfaces (not shown). . Then, the program stored in the ROM is read out and developed in the RAM, and the CPU executes various processes. Processing performed by the control unit 6 will be described later.
  • the camera unit 3 includes a lens 31a (see also FIG. 7B), a camera LED 31d (see also FIG. 7B), an imaging element 31f, a camera/communication control SoC 31g, and a wireless LAN unit 31h.
  • the imaging device 31f is a device that photoelectrically converts light incident through the lens 31a to generate captured image data.
  • a CCD sensor Charge Coupled Device
  • CMOS sensor Complementary Metal Oxide Semiconductor
  • the camera/communication control SoC 31g is an integrated circuit designed to integrate circuits with microcomputer functions and other applied functions on a single chip and work together.
  • the camera/communication control SoC 31g performs predetermined communication with the control unit 6 and outputs a photographing command to the image sensor 31f.
  • captured image data is input from the image sensor 31f to the camera/communication control SoC 31g.
  • the camera/communication control SoC 31g is an example of an integrated circuit, and other types may be used.
  • the photographing unit 30 shown in FIG. 8 has a function of photographing at least the refrigerator compartment 21 (storage room), and is provided outside the housing 1 (see FIG. 3) of the refrigerator 100 .
  • the photographing unit 30 is arranged so that the gap between the refrigerator compartment doors 211 and 212 and the housing 1 (see FIG. 3) is within the field of view. are placed.
  • the photographing unit 30 includes a lens 31a, an imaging device 31f, and a camera/communication control SoC 31g.
  • the wireless LAN unit 31h transmits the captured image data output from the camera/communication control SoC 31g to the router 51.
  • the router 51 is a communication relay device, and transmits captured image data received from the wireless LAN unit 31 h to the server 53 via the network 52 .
  • the server 53 performs predetermined processing on the captured image data, and stores the processed image data in association with the identification information of the refrigerator 100 .
  • server 53 receives a request signal for an image of the inside of refrigerator 100 from mobile terminal 54 of the user, server 53 transmits captured image data to mobile terminal 54 .
  • a mobile terminal 54 a tablet, a wearable terminal, or the like is used in addition to a mobile phone or a smart phone.
  • FIG. 9 shows an example of the result of photographing by the camera unit, in which image processing has not been performed.
  • the left and right are reversed with respect to FIG. 3 and the like (the same applies to FIG. 10).
  • both the left and right refrigerator compartment doors 211 and 212 are open, and in addition to the refrigerator compartment 21, the door pocket 211c of the left refrigerator compartment door 211 and the door pocket 212c of the right refrigerator compartment door 212 are opened. is also filmed.
  • FIG. 10 shows an example of the result of photographing by the camera unit 3 after predetermined image processing has been performed. For example, by performing predetermined image processing in the server 53 (see FIG. 8) based on the photographing result of FIG. converted to a simple image. By displaying such an image on the mobile terminal 54 (see FIG. 8), the user can grasp at a glance what kind of food is stored in the refrigerator compartment 21 or the like.
  • ⁇ Processing of control section> For example, when at least one of the refrigerator compartment doors 211 and 212 (see FIG. 1) is opened and an open signal is input from the door sensor 4 (see FIG. 8), the controller 6 (see FIG. 8) controls the camera LED 31d. is turned on, and photographing by the photographing unit 30 is repeated in a predetermined manner. The photographed image data is generated each time such photographing is performed. It is desirable that the photograph be taken with the camera open.
  • the control unit 6 detects the opening angles of the refrigerator compartment doors 211 and 212 using captured image data.
  • the "opening angle” is the angle at which the refrigerator compartment doors 211 and 212 are open.
  • the control unit 6 converts the photographed image data (color image) into grayscale, and then performs a series of processes shown in FIG. .
  • FIG. 11 is a flowchart of processing executed by the control unit of the refrigerator (see FIG. 8 as appropriate). Note that FIG. 11 mainly shows processing related to calculation of the opening angle of the refrigerator compartment doors 211 and 212 .
  • the control unit 6 performs binarization processing. For example, in the captured image, the control unit 6 sets the value of pixels whose brightness is equal to or higher than a predetermined value to '1' (white), and sets the value of pixels whose brightness is less than the predetermined value to '0' (black). Perform binarization processing.
  • step S102 the control unit 6 extracts edges on the captured image.
  • the control unit 6 extracts edges on the captured image by using, for example, a Sobel filter.
  • the edge extraction method is not limited to the Sobel filter, and other well-known methods such as the Laplacian filter and the Canny method may be used.
  • step S103 the control unit 6 extracts feature points on the captured image.
  • FIG. 12A is an explanatory diagram regarding extraction of feature points on a captured image.
  • FIG. 12A shows a photographed image of a state in which left and right refrigerator compartment doors 211 and 212 are opened and subjected to binarization processing.
  • the left and right are reversed with respect to FIG. 3 etc. (the same applies to FIGS. 12B and 12C).
  • the packing 212d (see also FIG. 3) of the right refrigerating compartment door 212 is a photographed image. included in In addition to the left and right horizontal plates 1a and 1b (see FIG. 4), the left and right side pieces 211f and 212f (see FIG. 5) are also included in the photographed image, although reference numerals are not shown in FIG. 12A.
  • the reason why these codes are not shown in FIG. 12A is that the pixel value is '0' (black) in the binarization process.
  • a rectangular area 7a shown in FIG. 12A is an area used for calculating the opening angle of the right refrigerator compartment door 212, and its position and size on the captured image are set in advance. Specifically, when the right refrigerator compartment door 212 is open, the boundary between the inner box 12 (see FIG. 4) and the right side plate 1b (see FIG. 4) is the area 7a on the captured image. This area 7a is set in advance so that it can pass through. Also, when the right refrigerator compartment door 212 is open, the boundary between the packing 212d (see FIG. 3) of the refrigerator compartment door 212 and the side piece 212f (see FIG. 5) is the area 7a on the photographed image. This area 7a is set in advance so that it can pass through. The same applies to the other rectangular area 7b.
  • a feature point 8m shown in FIG. 12A is a candidate point included in the boundary between the packing 212d (see FIG. 3) of the right refrigerator compartment door 212 (see FIG. 3) and the side piece 212f (see FIG. 5) in the captured image.
  • the control unit 6 (see FIG. 8) sequentially reads the values of pixels in a predetermined vertical row included in the area 7a of the captured image. In this case, the point where the pixel value changes from '1' (white) to '0' (black) can be included in the boundary between the packing 212d (see FIG. 3) and the side piece 212f (see FIG. 5). It is a feature point 8m with high quality.
  • the packing 212d (see FIG. 3) of the refrigerating compartment door 212 may be painted in a color, such as white, whose value tends to be '1' when binarization is performed.
  • the side piece 212f (see FIG. 5) adjacent to the packing 212d may be painted in a predetermined color that easily becomes '0' when the binarization process is performed.
  • each of the packing 212d and the side piece 212f is painted with a predetermined color so that a luminance difference occurs between the packing 212d (see FIG. 3) and the side piece 212f (see FIG. 5) on the captured image. preferably. This makes it easier to extract the boundary between the packing 212d and the side piece 212f as a second edge 92 (see FIG. 13), which will be described later.
  • a feature point 8n shown in FIG. 12A is a candidate point included in the boundary between the inner box 12 (see FIG. 4) of the housing 1 and the horizontal plate 1b (see FIG. 4) in the captured image.
  • the control unit 6 (see FIG. 8) sequentially reads the values of pixels in a predetermined vertical row included in the area 7a of the captured image. In this case, the point where the pixel value changes from '0' (black) to '1' (white) is included in the boundary between the inner box 12 (see FIG. 4) and the horizontal plate 1b (see FIG. 4). It is a feature point 8n that is highly likely to be
  • the inner box 12 may be painted in a color, such as white, whose value tends to become '1' when binarization processing is performed.
  • the horizontal plate 1b (see FIG. 4) adjacent to the inner box 12 may be painted in a color that tends to become '0' when the binarization process is performed.
  • each of the inner box 12 and the horizontal plate 1b (see FIG. 4) is painted in a predetermined color so that a difference in brightness occurs between the inner box 12 and the horizontal plate 1b (see FIG. 4) on the photographed image. preferable. This makes it easier to extract the boundary between the inner box 12 and the horizontal plate 1b as a first edge 91 (see FIG. 13), which will be described later.
  • the control unit 6 After extracting feature points for a predetermined vertical row in the region 7a, the control unit 6 extracts feature points for an adjacent vertical row. (see right arrow in FIG. 12A). Then, the control unit 6 extracts feature points for all vertical columns included in the region 7a (S103 in FIG. 11). Similar processing is performed in the other region 7b.
  • step S104 of FIG. 11 the control unit 6 calculates an approximate line. That is, the control unit 6 identifies two approximate lines (straight lines) indicating edges on the captured image based on the plurality of feature points extracted in step S103.
  • well-known methods such as the least-squares method, may be used in calculation of these approximate lines.
  • FIG. 12B is an explanatory diagram showing a series of feature points on a captured image.
  • a plurality of feature points 8n are distributed so as to be continuous at the boundary between the inner box 12 and the horizontal plate 1b (see FIG. 4).
  • a first edge 91 (see FIG. 13), which is the edge of the housing 1 (see FIG. 4) on the captured image, is calculated as a predetermined approximation line.
  • This first edge 91 is an edge on the captured image corresponding to the boundary between the inner box 12 and the horizontal plate 1b (see FIG. 4).
  • a plurality of feature points 8m are distributed so as to be continuous at the boundary between the packing 212d of the refrigerator compartment door 212 and the side piece 212f (see FIG. 5).
  • a second edge 92 (see FIG. 13), which is the edge of the refrigerator compartment door 212 (door) on the photographed image, is calculated as a predetermined approximation line.
  • This second edge 92 is an edge on the photographed image corresponding to the boundary between the packing 212d of the refrigerator compartment door 212 and the side piece 212f (see FIG. 5).
  • the first edge 91 and the second edge 92 (see FIG. 13) are positioned relative to the axis of the hinge 212a (see FIG. 5) of the refrigerator compartment door 212 (door) in real space. parallel.
  • the right refrigerator compartment door 212 is opened, most of the light irradiated near the right edge of the inner box 12 is emitted from the interior light (not shown) of the refrigerator 100 or the camera LED 31d (see FIG. 7B). Therefore, light, objects, and the like around refrigerator 100 hardly affect the extraction of first edge 91 and second edge 92 .
  • the control section 6 can extract the first edge 91 and the second edge 92 with high precision.
  • the cover 31c (see FIG. 6) through which the light from the camera LED 31d is transmitted is subjected to a predetermined texturing process.
  • the light from the camera LED 31d is appropriately diffused, and the light illuminating the refrigerator compartment doors 211 and 212 and the housing 1 is made uniform.
  • the boundary between the portion where the light from the camera LED 31d is irradiated and the portion where the light is hardly irradiated and is shadowed is blurred, the first edge 91 (see FIG. 13) and the second edge 91 (see FIG. 13) are blurred. Higher accuracy can be achieved when extracting the edge 92 (see FIG. 13).
  • step S103 extraction of feature points: see FIG. 11
  • feature points are extracted at positions away from the original first edge 91 (see FIG. 13) and second edge 92 (see FIG. 13). Sometimes it is done. Edges based on such feature points (noise) are often interrupted in the middle of the rectangular area 7a. On the other hand, the first edge 91 (see FIG. 13) and the second edge 92 (see FIG. 13) are almost never interrupted within the region 7a.
  • the control unit 6 excludes feature points included in edges that are interrupted in the middle of the region 7 a from candidates for the first edge 91 and the second edge 92 .
  • step S105 of FIG. 11 the control unit 6 calculates the area of the trapezoidal portion.
  • FIG. 12C is an explanatory diagram showing a trapezoidal portion 9a included in the region 7a on the captured image.
  • the trapezoidal portion 9a shown in FIG. 12C includes a line segment L1 (see FIG. 13) included in the region 7a at the first edge 91 (see FIG. 13) and a line segment L1 (see FIG. 13) included in the region 7a at the second edge 92 (see FIG. 13). This is the portion included in the region 7a between another line segment L2 (see FIG. 13) that is drawn.
  • the first edge 91 is a boundary between the inner box 12 and the horizontal plate 1b (see FIG. 4), even if the refrigerator compartment door 212 is opened, the first edge 91 on the photographed image position rarely changes.
  • the second edge 92 is a boundary between the packing 212d of the refrigerator compartment door 212 and the side piece 212f (see FIG. 5)
  • the opening angle of the refrigerator compartment door 212 is large.
  • the controller 6 calculates the opening angle of the refrigerator compartment door 212 (door) based on the area (the number of pixels) of the trapezoidal portion 9a on the photographed image.
  • FIG. 13 is an explanatory diagram of the trapezoidal portion 9a between the first edge 91 and the second edge 92.
  • FIG. 13 the region 7a of FIG. 12C is extracted and illustrated.
  • the linear first edge 91 and the linear second edge 92 intersect a pair of vertical opposite sides (boundary lines) of the rectangular region 7a.
  • the rectangular area 7a is set in advance so that the first edge 91 and the second edge 92 pass through on the captured image.
  • Points 61 and 62 shown in FIG. 13 are points of intersection between a pair of opposite sides of the rectangular region 7a and the first edge 91 .
  • Points 63 and 64 are points of intersection between a pair of opposite sides of the rectangular area 7 a and the second edge 92 .
  • the length (the number of pixels) of the line segment L4 connecting the points 62 and 64 is shorter than the length (the number of pixels) of the line segment L3 connecting the points 61 and 63 . This is because the line segment L4 has a lower height position in the real space than the line segment L3 and has a smaller number of pixels on the captured image.
  • step S105 the control unit 6 calculates the area of the trapezoidal portion 9a included in the region 7a between the first edge 91 and the second edge 92.
  • step S ⁇ b>106 control unit 6 calculates the opening angle of refrigerator compartment door 212 . That is, the control unit 6 calculates the opening angle corresponding to the area of the trapezoidal portion 9a calculated in step S105.
  • FIG. 14 is an explanatory diagram showing the relationship between the area of the trapezoidal portion and the opening angle of the refrigerator compartment door.
  • the horizontal axis of FIG. 14 is the area (the number of pixels) of the trapezoidal portion 9a (see FIGS. 12C and 13) included in the region 7a.
  • the vertical axis in FIG. 14 is the opening angle of the refrigerator compartment door 212 .
  • Curve C1 in FIG. 14 is data for deriving the opening angle of refrigerator compartment door 212 from the area of trapezoidal portion 9a, and is set in advance.
  • a predetermined data table may be used, or a predetermined monotonically increasing function may be used.
  • the control unit 6 calculates the opening angle ⁇ 1 of the refrigerator compartment door 212, for example, based on the area S1 of the trapezoidal portion 9a (see FIG. 13) and the data of the curve C1 shown in FIG. 14 (S106: See Figure 11). In this way, when the hinged refrigerating compartment door 212 (door) provided in the housing 1 (see FIG. 3) is opened, the control unit 6 controls the first edge, which is the edge of the housing 1 on the captured image. 91 (see FIG. 13) and a second edge 92 (see FIG. 13) which is the edge of the refrigerator door 212 on the photographed image, the opening angle of the refrigerator door 212 is calculated based on the number of pixels. . The opening angle of the left refrigerator compartment door 211 (see FIG. 3) is also calculated in the same manner. After performing the process of step S106 in FIG. 11, the control unit 6 ends the series of processes (END). A series of processes shown in FIG. 11 are performed each time new captured image data is input.
  • the control unit 6 based on the number of pixels between the first edge 91 and the second edge 92 (the area of the trapezoidal portion 9a: see FIG. 13), The opening angles of the refrigerator compartment doors 211 and 212 are calculated. As a result, the opening angles of the refrigerator compartment doors 211 and 212 can be calculated easily and with high accuracy.
  • the manufacturing cost of the refrigerator 100 can be reduced.
  • the user can check the food stored in the refrigerator 100 by looking at the portable terminal 54 (see FIG. 8) without opening the refrigerator compartment doors 211, 212 and the like. In this way, since the convenience for the user is enhanced, it is possible to contribute to society.
  • the second embodiment differs from the first embodiment in that when the open angle of the refrigerator compartment doors 211 and 212 reaches a predetermined value, the rectangular area used for calculating the next open angle is expanded. .
  • Others structure of refrigerator 100, etc.: see FIGS. 1 to 8) are the same as in the first embodiment. Therefore, the portions different from the first embodiment will be described, and the description of the overlapping portions will be omitted.
  • FIG. 15A is an explanatory diagram of a state in which the opening angle of the refrigerator compartment door is less than a predetermined value in the refrigerator according to the second embodiment (see also FIGS. 3 and 8 as appropriate). Note that the region 7a shown in FIG. 15A is the same as that described in the first embodiment (see FIG. 13). Also, the definitions of the first edge 91 and the second edge 92 are the same as those described in the first embodiment. Calculation of the opening angle of the right refrigerator compartment door 212 will be mainly described below, but the calculation of the left refrigerator compartment door 211 is the same.
  • the calculation of the opening angle of the refrigerator compartment door 212 is repeated in a predetermined manner.
  • an open signal is input from the door sensor 4 (see FIG. 8)
  • calculation of the opening angle of the refrigerator compartment door 212 is repeated in a predetermined manner.
  • the controller 6 calculates the next opening angle based on the opening angle of the refrigerating compartment door 212 so that the first edge 91 and the second edge 92 are appropriately extracted. I am trying to change the area to be used.
  • a first region 7c shown in FIG. 15A is a photographed image that the controller 6 uses to calculate the next opening angle when the opening angle of the refrigerator compartment door 212 is less than a predetermined value (for example, the opening angle ⁇ 100°). This is the upper area.
  • the first area 7c is set in advance as a rectangular area so that the door pocket 212c inside the refrigerator compartment door 212 does not overlap on the photographed image when the opening angle of the refrigerator compartment door 212 is less than a predetermined value. ing. That is, when the opening angle of the refrigerator compartment door 212 is less than the predetermined value, the door pocket 212c does not overlap the first area 7c on the captured image. Further, the first area 7c is set so that the first edge 91 and the second edge 92 pass through the first area 7c on the photographed image when the refrigerator compartment door 212 is opened.
  • the control unit 6 uses the first area 7c when calculating the opening angle for the first time. That is, the control unit 6 determines the area (the number of pixels) on the photographed image of the trapezoidal portion 9b that is included in the rectangular first region 7c and sandwiched between the first edge 91 and the second edge 92. Based on this, the opening angle of refrigerator compartment door 212 is calculated. Data indicating the correspondence between the area of the trapezoidal portion 9b and the opening angle of the refrigerator compartment door 212 is stored in advance.
  • the opening angle of the refrigerator compartment door 212 can be calculated appropriately by using the first area 7c. Until the calculation result of the opening angle of refrigerator compartment door 212 reaches a predetermined value, control unit 6 uses first region 7c when calculating the next opening angle. Note that as the opening angle of the refrigerator compartment door 212 increases, the door pocket 212c on the photographed image moves as indicated by the white arrow shown in FIG. 15A.
  • FIG. 15B is an explanatory diagram of a state in which the opening angle of the refrigerator compartment door is equal to or greater than a predetermined value in the refrigerator according to the second embodiment (see also FIGS. 3 and 8 as appropriate).
  • the region 7a (second region) shown in FIG. 15B is the same as that described in the first embodiment (see FIG. 13). That is, the region 7a (second region) is when the first edge 91 and the second edge 92 pass through itself and the opening angle of the refrigerator compartment door 212 is a predetermined value or more (for example, the opening angle ⁇ 100°). is set in advance as a rectangular area that does not overlap with the door pocket 212c.
  • the area 7a (second area) includes the first area 7c. That is, the rectangular area 7a has a wider horizontal width in the captured image than the first area 7c so as to include a captured image of a lower height position in the real space. .
  • the opening angle of the refrigerator compartment door 212 can be calculated with high accuracy.
  • control unit 6 expands the area used for calculating the next opening angle of refrigerating compartment door 212 from first area 7c to area 7a (second area). to As a result, the controller 6 can accurately calculate the opening angle of the refrigerator compartment door 212 when the refrigerator compartment door 212 is fully opened.
  • the calculation of the opening angle based on the first region 7c is slightly less accurate than the case of the region 7a (second region), but there is no particular problem. This is because when the opening angle of the refrigerator compartment door 212 is less than the predetermined value, the refrigerator compartment door 212 is not sufficiently opened, and the photographed image data is often discarded without being transmitted to the server. . In short, the calculation of the opening angle using the first area 7c is performed to detect the timing when the opening angle of the refrigerator compartment door 212 reaches a predetermined value (the timing when the rectangular area is widened).
  • the control unit 6 when the opening angle of the refrigerating compartment door 212 reaches a predetermined value, the control unit 6 changes the area used for calculating the next opening angle from the first area 7c to the area 7a (second area ). As a result, the controller 6 can accurately calculate the opening angle of the refrigerator compartment door 212 when the refrigerator compartment door 212 is fully opened.
  • FIG. 16 is an explanatory diagram including a region on a photographed image of a refrigerator according to a modification, a first edge 91, and a second edge 92.
  • FIG. Point 65 shown in FIG. 16 is the midpoint between points 61 and 62 on first edge 91 .
  • a point 66 shown in FIG. 16 is the middle point between the points 63 and 64 on the second edge 92 .
  • a line segment L3 is a vertical line segment connecting the points 63 and 64 .
  • the rest of the symbols are the same as those described with reference to FIG. For example, the greater the opening angle of refrigerator compartment door 212, the longer the lengths (the number of pixels) of line segments L3, L4, and L5.
  • the control unit 6 calculates the sum (the number of pixels) of the lengths of the line segments L3, L2, and L3 based on the captured image, and calculates the opening angle of the refrigerator compartment door corresponding to this sum. Such processing is also included in the fact that the controller 6 calculates the opening angle of the refrigerator compartment door based on the number of pixels between the first edge 91 and the second edge 92 .
  • control unit 6 calculates the opening angle of the refrigerator compartment door 212
  • one or two of the line segments L3, L2, and L3 shown in FIG. 16 may be used.
  • the control unit 6 may calculate the opening angle of the refrigerator compartment door 212 based on the sum of the line segments L3 and L2, or calculate the opening angle of the refrigerator compartment door 212 based on the length of the line segment L5.
  • An opening angle may be calculated.
  • the horizontal position of the line segment L4 can be changed as appropriate.
  • the controller 6 may calculate the opening angle of the refrigerating compartment door 212 based on the length (the number of pixels) of the line segment.
  • the present invention is not limited to this. That is, the camera unit 3 may be provided at another predetermined position outside the housing 1 .
  • the portion corresponding to the region 7a (see FIG. 13) on the photographed image is colored differently from the other portions, and the inner box inside the region 7a is A luminance difference may be generated between 12 (for example, white) and the horizontal plates 1a and 1b. This facilitates extraction of the first edge 91 .
  • the side piece 212f (see FIG. 5) of the refrigerator compartment door 212 the part corresponding to the area 7a (see FIG. 13) on the photographed image is colored differently from the other parts, and the packing 212d is formed in the area 7a. (eg, white) and the side piece 212f may have a luminance difference. This facilitates extraction of the second edge 92 .
  • the present invention is not limited to this.
  • the camera LED may be arranged in front of the lens 31a.
  • a plurality of camera LEDs may be arranged in a predetermined manner.
  • the controller 6 may calculate the opening angles of the refrigerator compartment doors 211 and 212 based on the number of pixels between the first edge 91 and the second edge 92 without using the rectangular area 7a.
  • the control unit 6 uses a portion higher than the door pockets 211c and 212c (see FIG. 3) in real space. may calculate the opening angles of the refrigerator compartment doors 211 and 212 . This can prevent the door pockets 211c and 212c from interfering with the calculation of the opening angle.
  • the edge on the captured image corresponding to the boundary between the inner box 12 (see FIG. 4) and the horizontal plate 1b (see FIG. 4) is used as the first edge 91 .
  • another predetermined boundary line on the housing 1 may be used as the first edge 91 .
  • the second edge 92 an edge on the photographed image corresponding to the boundary between the packing 212d (see FIG. 3) of the refrigerator compartment door 212 and the side piece 212f (see FIG. 5) is used.
  • the present invention is not limited to this. That is, as the second edge 92, another predetermined boundary line on the refrigerating compartment door 212 may be used.
  • each embodiment the boundary lines corresponding to the first edge 91 and the second edge 92 are parallel to the axes of the hinges 211a and 212a (see FIG. 5), but the present invention is not limited to this. In other words, each embodiment can be applied even when the boundary lines are not parallel to the axes of the hinges 211a and 212a.
  • the horizontal plates 1a, 1b, 1c (see FIG. 4) of the housing 1 are provided as independent members
  • the present invention is not limited to this.
  • the horizontal plates 1a, 1b, 1c may be integrated with the outer case 11 (see FIG. 4).
  • the side pieces 211f, 212f may be integrated with the refrigerator compartment doors 211, 212.
  • each embodiment can be applied to a refrigerator with a single-opening refrigerator compartment door (not shown) and a portable refrigerator (not shown).
  • the “storage box” to which each embodiment can be applied is not limited to refrigerators. That is, each embodiment can be applied to various kinds of hinged "storage boxes". In this case, the housing of the “storage box” may have a configuration in which there is no distinction between the outer box and the inner box.
  • each embodiment can be applied to a chest-type freezer or storage.
  • each embodiment can be applied to a storage with an upward opening door (flip-up door), a downward opening door, a folding door, or the like. Further, for example, each embodiment can be applied to a closet or the like having a folding door.
  • all or part of a program for realizing the method of calculating the opening angle of the refrigerator compartment doors 211 and 212 (the method of calculating the door opening angle) described in each embodiment can be executed by one or more computers. good too.
  • the program described above can be provided via a communication line, or can be written in a recording medium such as a CD-ROM and distributed.
  • each embodiment is described in detail for easy understanding of the present disclosure, and is not necessarily limited to those having all the described configurations. Moreover, it is possible to add, delete, or replace a part of the configuration of the embodiment with another configuration. Further, the mechanisms and configurations described above show those considered necessary for explanation, and do not necessarily show all the mechanisms and configurations on the product.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Chemical & Material Sciences (AREA)
  • Combustion & Propulsion (AREA)
  • Mechanical Engineering (AREA)
  • Thermal Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Cold Air Circulating Systems And Constructional Details In Refrigerators (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Devices That Are Associated With Refrigeration Equipment (AREA)
PCT/JP2021/039840 2021-10-28 2021-10-28 扉開角度算出方法及び収納庫 Ceased WO2023073869A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
PCT/JP2021/039840 WO2023073869A1 (ja) 2021-10-28 2021-10-28 扉開角度算出方法及び収納庫
JP2022577371A JP7426514B2 (ja) 2021-10-28 2021-10-28 扉開角度算出方法及び収納庫
CN202180100589.0A CN117651843A (zh) 2021-10-28 2021-10-28 门打开角度计算方法和收纳库

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2021/039840 WO2023073869A1 (ja) 2021-10-28 2021-10-28 扉開角度算出方法及び収納庫

Publications (1)

Publication Number Publication Date
WO2023073869A1 true WO2023073869A1 (ja) 2023-05-04

Family

ID=86157558

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/039840 Ceased WO2023073869A1 (ja) 2021-10-28 2021-10-28 扉開角度算出方法及び収納庫

Country Status (3)

Country Link
JP (1) JP7426514B2 (https=)
CN (1) CN117651843A (https=)
WO (1) WO2023073869A1 (https=)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7578631B2 (ja) 2022-01-25 2024-11-06 日立グローバルライフソリューションズ株式会社 ドア開角度算出方法及び収納庫

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003042626A (ja) * 2001-07-26 2003-02-13 Toshiba Corp 冷蔵庫
JP2014149597A (ja) * 2013-01-31 2014-08-21 Secom Co Ltd 通行物体検出装置
JP2015105765A (ja) * 2013-11-29 2015-06-08 日立アプライアンス株式会社 冷蔵庫

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102152400B1 (ko) * 2014-04-15 2020-09-08 엘지전자 주식회사 냉장고 및 그 제어 방법
DE102013221494A1 (de) * 2013-10-23 2015-04-23 Robert Bosch Gmbh Verfahren und Vorrichtung zum Bestimmen eines Drehwinkels eines Objekts um eine Drehachse
CN107461979A (zh) * 2016-06-03 2017-12-12 博西华电器（江苏）有限公司 冰箱
FI130829B1 (fi) * 2019-01-02 2024-04-12 Procemex Oy Ltd Menetelmä koneluettavan koodin käyttämiseksi kameran ohjaamiseen havaitsemaan ja tarkkailemaan kohteita
KR102234771B1 (ko) * 2019-12-20 2021-04-01 엘지전자 주식회사 인공 지능 냉장고

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003042626A (ja) * 2001-07-26 2003-02-13 Toshiba Corp 冷蔵庫
JP2014149597A (ja) * 2013-01-31 2014-08-21 Secom Co Ltd 通行物体検出装置
JP2015105765A (ja) * 2013-11-29 2015-06-08 日立アプライアンス株式会社 冷蔵庫

Also Published As

Publication number Publication date
JPWO2023073869A1 (https=) 2023-05-04
CN117651843A (zh) 2024-03-05
JP7426514B2 (ja) 2024-02-01

Similar Documents

Publication Publication Date Title
CN105492850B (zh) 具有摄像模块的制冷器具
DE102013211096A1 (de) Kältegerätesystem
DE212014000095U1 (de) Kühlschrank und Kühlschranksystem
DE102013211099A1 (de) Kältegerät mit einem Kameramodul
JP7594569B2 (ja) 冷蔵庫
DE102013211095A1 (de) Kältegerät mit einer Tür
WO2023073869A1 (ja) 扉開角度算出方法及び収納庫
US20230058922A1 (en) Appliance with collocated cameras
JP7313156B2 (ja) 冷蔵庫
DE102013211098A1 (de) Kältegerät mit einem Kameramodul
JP2014238217A (ja) 冷蔵庫
JP7578631B2 (ja) ドア開角度算出方法及び収納庫
CN117751264A (zh) 制冷设备
KR20150045256A (ko) 냉장고
JP7734161B2 (ja) 貯蔵庫及び貯蔵庫システム
TWI846010B (zh) 冰箱
WO2023067966A1 (ja) 貯蔵庫並びに貯蔵庫及び撮影装置が行う撮影方法
JP6641109B2 (ja) 冷蔵庫
CN113824864A (zh) 食材存储设备及图像处理方法
JP7688001B2 (ja) 貯蔵庫及びカメラユニット
JP7353340B2 (ja) 冷蔵庫
KR20220030042A (ko) 냉장고
JP7337889B2 (ja) 冷蔵庫
JP7744488B2 (ja) 貯蔵庫
CN111698395A (zh) 用于箱体内的摄像头组件和冰箱

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 2022577371

Country of ref document: JP

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21962417

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202180100589.0

Country of ref document: CN

WWE Wipo information: entry into national phase

Ref document number: 2401000914

Country of ref document: TH

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21962417

Country of ref document: EP

Kind code of ref document: A1