JP7744488B2 - storage facility - Google Patents

Description

This disclosure relates to a storage facility.

Technology for photographing the storage compartment of a refrigerator, etc., using a camera is known. Patent Document 1 discloses that, in principle, the selection of photographs taken is performed based on the angle of the door (0278), but that if a photo selection command is not issued, a first-in, first-out method is used to temporarily store a small number of photographs (0096). It also discloses that, in consideration of costs, the time interval between photographs taken by a camera that takes continuous photographs can be made longer than 1/30 of a second (0271, 0274).

EP 2909549 (EP, B1)

Patent Document 1 does not provide detailed disclosure regarding the relationship between the shooting time interval setting and the timing of selecting the captured photos.

The reservoir according to the present disclosure comprises:
a housing having a storage chamber;
a door that closes the opening of the storage chamber;
a door sensor that detects whether the door is open or closed;
and an imaging unit that images at least the storage chamber,
a control unit that, after the door opening operation is detected by the door sensor , issues a command to the photographing unit to photograph while the door opening operation is being detected, determines whether to save or discard the most recently photographed image based on the degree of opening of the door, and, in accordance with the determination, completes the saving if the image is to be saved so that it can be selected as an image that can be sent to the server, or completes the discarding if the image is to be discarded because it cannot be selected as an image that can be sent to the server, and then repeats the next photographing, decision on whether to save and discard, and execution of saving or discarding until the open door is closed;
The criteria for the determination are not to unconditionally determine whether to store the latest captured image, but rather there is a possibility of either storing or discarding the image.

FIG. 1 is a front view of a refrigerator according to a first embodiment. FIG. 1 is a side view of a refrigerator according to a first embodiment. FIG. 1 is a front view of a refrigerator according to a first embodiment with left and right refrigerator compartment doors open. FIG. 1 is a plan view of a refrigerator according to a first embodiment with left and right refrigerator compartment doors open. 1 is a perspective view of a camera unit of a refrigerator according to a first embodiment, viewed from diagonally below. FIG. FIG. 2 is a front view of the camera unit of the refrigerator according to the first embodiment. FIG. 6B is a cross-sectional view taken along the line II-II in FIG. 6A in the refrigerator according to the first embodiment. FIG. 1 is a system configuration diagram including a camera unit of a refrigerator according to a first embodiment. 10 is a flowchart of an automatic photography process executed by a control unit of the refrigerator according to the first embodiment. 10 is a flowchart of an automatic photography process executed by a control unit of the refrigerator according to the first embodiment. 5A to 5C are explanatory diagrams showing a time series of processes in which the refrigerator compartment door of the refrigerator according to the first embodiment is opened and closed. 10 is an example of a photograph taken by a camera unit of a refrigerator according to the first embodiment, in which no image processing has been performed. 10 is an example of a photograph taken by a camera unit of a refrigerator according to the first embodiment, in which image processing has been performed. FIG. 10 is an explanatory diagram showing candidates for areas that serve as a reference when adjusting exposure in the refrigerator according to the first embodiment. FIG. 10 is an explanatory diagram showing a reference area for adjusting exposure in a refrigerator according to a comparative example. 10A and 10B are a front view and a bottom view of left and right refrigerator compartment doors provided in a refrigerator according to a second embodiment. FIG. 11 is a cross-sectional perspective view of the lower part of the right-side refrigerator compartment door of the refrigerator according to the second embodiment. FIG. 11 is a cross-sectional view of the lower part of the right-side refrigerator compartment door of the refrigerator according to the second embodiment.

First Embodiment
FIG. 1 is a front view of a refrigerator 100 according to the first embodiment.
Refrigerator 100 is an appliance for cooling food and the like, and includes a housing 1, doors such as refrigerator compartment doors 211 and 212, and a camera unit 3. Housing 1 has multiple storage compartments inside. In the example of Fig. 1, the storage compartments of refrigerator 100 include, from top to bottom, refrigerator compartment 21, ice maker compartment 22 and upper freezer compartment 23 lined up on the left and right, vegetable compartment 24, and lower freezer compartment 25.

The housing 1 is constructed with an outer box 11 made of steel plate and an inner box (not shown) made of resin, between which a heat insulating material (not shown) such as urethane foam is filled. The front side (front face) of the housing 1 has multiple openings 10 (see Figures 2 and 4) corresponding to each compartment. For example, when food or the like is placed through the opening 10 of the refrigerator compartment 21, the refrigerator compartment doors 211 and 212 are opened. Furthermore, when the refrigerator compartment doors 211 and 212 are closed, the opening 10 of the refrigerator compartment 21 is blocked. In this way, the refrigerator compartment doors 211 and 212 (doors) have the function of blocking the opening 10 of the housing 1. The same applies to the other doors.

The refrigerator 100 is equipped with left and right refrigerator compartment doors 211, 212 as French-style doors that, together with the housing 1, form the refrigerator compartment 21. The left refrigerator compartment door 211 (left door) is rotatable around the axis of the hinge 211a (see Figure 4) at the left end. The same is true for the right refrigerator compartment door 212 (right door). The refrigerator 100 also has drawer-type doors, including an ice maker compartment door 221 and an upper freezer compartment door 231, as well as a vegetable compartment door 241 and a lower freezer compartment door 251.

The refrigerator compartment 21 is provided with multiple shelves 213 (see Figure 3) that divide the compartment 21 into sections. The left refrigerator compartment door 211 has multiple door pockets 211c (see Figure 3) on the inside for storing food and other items (the same applies to the right refrigerator compartment door 212). The ice making compartment 22 has an ice making compartment container (not shown) that slides out together with the ice making compartment door 221. Similarly, the upper freezer compartment 23 has an upper freezer container (not shown), the vegetable compartment 24 has a vegetable container (not shown), and the lower freezer compartment 25 has a lower freezer container (not shown).

The refrigerator 100 also includes a compressor, a radiator (condenser), a capillary tube (throttle mechanism), and an evaporator, all of which are not shown. Refrigerant circulates through the compressor, radiator, capillary tube, and evaporator in this order, and the air in the storage compartment is cooled by heat exchange with the refrigerant flowing through the evaporator. The camera unit 3 shown in Figure 1 is installed on the top surface of the housing 1 and is used to capture images of the storage compartment and other areas of the refrigerator 100.

FIG. 2 is a side view of the refrigerator 100.
2, the camera unit 3 includes a main body 31 and a support 32. The main body 31 has a function of capturing images of the storage compartment, etc. The support 32 supports the main body 31 and is installed on the top surface of the housing 1.

A lens 31a (see also Figure 5) is provided near the front end of the main body 31. The lens 31a is an optical element that refracts and focuses light. For example, a fisheye lens is used as this lens 31a. When the refrigerator compartment doors 211, 212 (see Figure 1) are open, the lens 31a faces downward so that the camera unit 3 can capture an image of the refrigerator compartment 21 (see Figure 1).

As shown in Figure 2, lens 31a is located forward of the front end of housing 1 (opening 10 of housing 1). More preferably, lens 31a is located further forward than the front of refrigerator compartment doors 211, 212 (see Figure 1) in the closed state. This makes it easier for the refrigerator compartment 21 to be within the field of view of lens 31a when refrigerator compartment doors 211, 212 are opened, for example. Note that it is also possible to capture images of vegetable compartment 24, etc. (see Figure 1) with camera unit 3 when a drawer-type door such as vegetable compartment door 241 is open. In this way, lens 31a is located in a position that allows a view down on each storage compartment from above.

FIG. 3 is a front view of the refrigerator 100 with the left and right refrigerator compartment doors 211 and 212 open.
As described above, the inner panel 211b of the left refrigerator compartment door 211 is provided with a plurality of door pockets 211c for storing food and the like (the same is true for the right refrigerator compartment door 212). When the left and right refrigerator compartment doors 211, 212 are opened, the food and the like in the refrigerator compartment 21 and the door pockets 211c, 212c come into the field of view of the lens 31a of the camera unit 3 in a bird's-eye view (see also FIG. 10).

The rotating partition 211d shown in Figure 3 is a partition that prevents cold air from leaking through the gap between the refrigerator compartment doors 211 and 212. In the example shown in Figure 3, the rotating partition 211d is provided at the end of the left refrigerator compartment door 211 opposite the axis of the hinge 211a (see Figure 4), and is designed to rotate in a predetermined manner as the refrigerator compartment door 211 is opened or closed.

In addition, when manually photographing the refrigerator compartment 21 using the camera unit 3, one photographing button 7 that the user presses is provided on each of the left and right refrigerator compartment doors 211, 212. In the example of Figure 3, the photographing button 7 is provided on the left refrigerator compartment door 211 (door) at the bottom of the surface opposite the hinge 211a (see Figure 4) of this refrigerator compartment door 211 (the same is true for the right refrigerator compartment door 212). From another perspective, when the left and right refrigerator compartment doors 211, 212 are closed (see Figure 1), the photographing button 7 is provided on the surface of one of the refrigerator compartment doors 211, 212 facing the other (near the rotating partition 211d in Figure 3).

Photographing performed when at least one of these two photographing buttons 7 is pressed by the user is called "manual photographing." Photographing performed by the control unit 8 (see Figure 7) based on the elapsed time from when the refrigerator compartment doors 211, 212 begin to open is called "automatic photographing."

FIG. 4 is a plan view of the refrigerator 100 with the left and right refrigerator compartment doors 211 and 212 open.
In the example of Fig. 4, the main body 31 of the camera unit 3 is disposed slightly left of the center of the housing 1 of the refrigerator 100 in the left-right direction. To explain in more detail, the main body 31 of the camera unit 3 is disposed directly above the joint between the refrigerator compartment doors 211, 212 when they are closed (see also Fig. 1). By arranging the main body 31 in this manner, for example, when the refrigerator compartment doors 211, 212 are opened, food and the like in the door pockets 211c, 212c are more likely to come into the field of view of the camera unit 3.

Furthermore, when using the refrigerator 100, the user is often conscious of the seam (boundary) between the refrigerator compartment doors 211, 212. Therefore, by positioning the main body 31 directly above the seam between the refrigerator compartment doors 211, 212, the user is less likely to feel uncomfortable compared to when the main body 31 is positioned left-right away from the seam.

FIG. 5 is a perspective view of the camera unit 3 when viewed from diagonally below.
The main body 31 of the camera unit 3 shown in Fig. 5 includes the lens 31a (see also Fig. 2) described above, a case 31b, and a camera LED 31c (light emitting diode: illumination unit). In the example of Fig. 5, the case 31b of the main body 31 is generally rectangular parallelepiped that is elongated in the front-to-rear direction. A circular hole (not shown) is provided on the underside near the front end of the case 31b, and the lens 31a is exposed through this hole. The camera LED 31c is provided behind (at the back of) the lens 31a.

The camera LED 31c is a light source that irradiates the refrigerator compartment 21 (see FIG. 3) and door pockets 211c, 212c (see FIG. 3) with light so that the camera unit 3 can photograph the refrigerator compartment 21 (see FIG. 3) and door pockets 211c, 212c under appropriate lighting conditions. The camera LED 31c (illumination unit) is provided on the outside of the housing 1 (see FIG. 2) of the refrigerator 100. Details will be described later, but in the first embodiment, the camera unit 3 photographs the refrigerator compartment 21 (see FIG. 3) with the interior light 4 (see FIG. 7) of the refrigerator 100 turned off and the camera LED 31c turned on.

As shown in Figure 5, by providing lens 31a in front of camera LED 31c, reflected light from refrigerator compartment 21, etc. (see Figure 3) can easily enter lens 31a when refrigerator compartment doors 211, 212 are open. This allows food stored in refrigerator compartment 21, etc. to be photographed appropriately. It is preferable that camera LED 31c be located forward of the front end (opening 10 of housing 1) of refrigerator 100's housing 1 (see Figure 2). This makes it easier for light emitted from camera LED 31c to enter refrigerator compartment 21, etc., and ultimately makes it easier to obtain clear photographic results.

FIG. 6A is a front view of the camera unit 3. FIG.
6A, the main body 31 of the camera unit 3 is installed on the upper side of the support part 32. The main body 31 is also installed near the center of the support part 32 in the left-right direction.

FIG. 6B is a cross-sectional view taken along line II-II of FIG. 6A.
As shown in FIG. 6B , the camera unit 3 includes a lens 31a (see also FIG. 5 ), a camera LED 31c (see also FIG. 5 ), a case 31b, and a circuit board 31d. The circuit board 31d is a printed circuit board on which an image sensor 31e (see FIG. 7 ) and a camera/communication control SoC 31f (System-on-a-Chip: see FIG. 7 ), which will be described later, are mounted. By housing the lens 31a, the camera LED 31c, and the circuit board 31d in a single case 31b in this way, the camera unit 3 can be made more compact and the length of the wiring can be shortened compared to when these components are provided separately.

FIG. 7 is a system configuration diagram including the camera unit 3 of the refrigerator 100.
7 , in addition to the camera unit 3, the refrigerator 100 is equipped with an interior light 4, a buzzer 5, a door sensor 6, a photography button 7, and a control unit 8. The interior light 4 is a light source that irradiates light into the interior of the refrigerator 100 and is provided inside the housing 1. The buzzer 5 is provided in a predetermined location in the housing 1 and is sounded when the interior light 4 is turned off, for example, during photography by the camera unit 3.

The door sensor 6 outputs a predetermined signal indicating the open/closed state of the refrigerator compartment doors 211, 212 to the control unit 8. Note that while one door sensor 6 is shown in Figure 7, in reality, there is a door sensor 6 corresponding to the left refrigerator compartment door 211 (see Figure 1) and another door sensor 6 corresponding to the right refrigerator compartment door 212 (see Figure 1). For example, when the left refrigerator compartment door 211 is opened, an open signal is output from the door sensor 6 corresponding to this refrigerator compartment door 211 to the control unit 8. As mentioned above, the capture button 7 (see also Figure 3) is a button that is pressed by the user when manually capturing an image using the camera unit 3.

The control unit 8 is, for example, a microcomputer, and although not shown, is configured to include electronic circuits such as a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), and various interfaces. The CPU reads programs stored in the ROM and loads them into the RAM, allowing the CPU to execute various processes. In addition to performing the "automatic shooting" described above, the control unit 8 also performs "manual shooting" when the shooting button 7 is pressed. Details of the processing performed by the control unit 8 will be described later.

The camera unit 3 includes a lens 31a (see also Figure 5) and a camera LED 31c (see also Figure 5), as well as an image sensor 31e, a camera/communication control SoC 31f, and a wireless LAN unit 31g. The image sensor 31e photoelectrically converts light incident through the lens 31a to generate captured image data. For example, a CCD sensor (Charge Coupled Device) or a CMOS sensor (Complementary Metal Oxide Semiconductor) is used as the image sensor 31e.

The camera/communication control SoC31f is an integrated circuit that integrates microcontroller functions and other circuits with applied functions on a single chip and is designed to function in conjunction with each other. The camera/communication control SoC31f communicates with the control unit 8 in a predetermined manner and outputs a shooting command to the image sensor 31e. This causes the image sensor 31e to input captured image data to the camera/communication control SoC31f. Note that the camera/communication control SoC31f is an example of an integrated circuit, and other types of integrated circuits may also be used.

The imaging unit 30 shown in Figure 7 has the function of capturing images of at least the refrigerator compartment 21 (storage compartment), and is provided on the outside of the housing 1 (see Figure 1) of the refrigerator 100. As shown in Figure 7, the imaging unit 30 is composed of a lens 31a, an image sensor 31e, and a camera/communication control SoC 31f.

Wireless LAN unit 31g transmits captured image data output from camera/communication control SoC 31f to router 51. Router 51 is a communication relay device that transmits captured image data received from wireless LAN unit 31g to server 53 via network 52. Server 53 performs predetermined processing on the captured image data and stores it in association with the identification information of refrigerator 100. Furthermore, when server 53 receives a request signal for an image of the interior of refrigerator 100 from the user's mobile terminal 54, it transmits the captured image data to mobile terminal 54. Such mobile terminal 54 may be a mobile phone, smartphone, tablet, wearable device, etc.

<Processing of the control unit>
8A and 8B are flowcharts of the automatic photography process executed by the control unit of the refrigerator (see FIG. 7 as appropriate).
The following describes, as an example, the process in which camera unit 3 captures an image of refrigerator compartment 21 when at least one of refrigerator compartment doors 211, 212 (see FIG. 3) is open. It should be noted that at "START" in FIG. 8A, both left and right refrigerator compartment doors 211, 212 (see FIG. 3) are closed. Furthermore, the user can switch the automatic photography setting and the manual photography setting on and off by operating mobile terminal 54 (see FIG. 7). In the examples of FIGS. 8A and 8B, it is assumed that the automatic photography setting is on (automatic photography enabled) and that the manual photography setting is also on (manual photography enabled).

In step S101, the control unit 8 determines whether an open signal has been input from the door sensor 6. That is, the control unit 8 determines whether at least one of the left and right refrigerator compartment doors 211, 212 (see Figure 3) has been opened. In step S101, if an open signal has not been input from the door sensor 6 (S101: No), the control unit 8 repeats the processing of step S101. On the other hand, if an open signal has been input from the door sensor 6 in step S101 (S101: Yes), the processing of the control unit 8 proceeds to step S102.

In step S102, the control unit 8 turns on the interior light 4 of the refrigerator 100 and also turns on the camera LED 31c. In other words, the control unit 8 turns on the interior light 4 and also turns on the camera LED 31c (illumination unit) immediately after at least one of the refrigerator compartment doors 211, 212 (doors) is opened. As described below, the control unit 8 turns off the interior light 4 a predetermined time after an open signal is input from the door sensor 6 (S103: Yes, S104), but until then the interior light 4 remains on. This illuminates the refrigerator compartment 21, making it easier for the user to see the refrigerator compartment 21. Furthermore, this reduces the user's discomfort compared to when the interior light 4 is off when the user opens at least one of the refrigerator compartment doors 211, 212.

Furthermore, the control unit 8 turns on the camera LED 31c along with the interior light 4 (S102), thereby preparing for subsequent photography (S106). Furthermore, the time required to adjust exposure during photography can be shortened compared to when the interior light 4 and camera LED 31c are turned on at different times. The camera LED 31c remains lit until both the left and right refrigerator compartment doors 211, 212 are closed.

Next, in step S103, the control unit 8 determines whether a predetermined time has elapsed since the open signal was input from the door sensor 6. This predetermined time (e.g., several seconds) is the time until the interior light 4 is turned off (S104) and is set in advance. For example, the predetermined time in step S103 is set in advance, taking into account the time required for subsequent exposure adjustment (S105), so that there is a high probability that the refrigerator compartment doors 211, 212 will be open beyond the predetermined opening angle when the image is captured (S106).

In step S103, if the predetermined time has not elapsed since the open signal was input from the door sensor 6 (S103: No), the control unit 8 repeats the processing of step S103. On the other hand, if the predetermined time has elapsed since the open signal was input from the door sensor 6 (S103: Yes), the processing of the control unit 8 proceeds to step S104.

In step S104, the control unit 8 turns off the interior light 4 and sounds the buzzer 5. That is, the control unit 8 sounds the buzzer 5 when turning off the interior light 4 after at least one of the refrigerator compartment doors 211, 212 (doors) has been opened. By sounding the buzzer 5 in this way, it is possible to reduce the sense of discomfort felt by the user when the interior light 4 is turned off, and it is also possible for the user to more easily understand that the light has been turned off to allow photography.

Next, in step S105, the control unit 8 performs exposure adjustment. That is, the control unit 8 adjusts the gain (sensitivity to brightness) when amplifying the electrical signal output from the image sensor 31e, as well as adjusting the shutter speed of the camera unit 3, etc. For example, the control unit 8 adjusts the gain so that the average brightness of the captured image data falls within a predetermined range. By performing exposure adjustment in this manner, it is possible to suppress blown-out highlights (halation) and crushed shadows in the captured image. Note that image input and exposure adjustment may be repeated in a predetermined manner from the time at least one of the refrigerator compartment doors 211, 212 is opened until both refrigerator compartment doors 211, 212 are closed.

Next, in step S106, the control unit 8 outputs a shooting command. That is, the control unit 8 captures the electrical signal output from the image sensor 31e by photoelectric conversion and further amplified by a predetermined gain as captured image data. In this way, after at least one of the refrigerator compartment doors 211, 212 (doors) is opened (S101: Yes), the control unit 8 turns off the interior light 4 and further causes the camera LED 31c (illumination unit) to be lit, causing the shooting unit 30 to take a picture (S102, S104, S106).

Although not specifically shown in Figures 8A and 8B, it is preferable for the control unit 8 to cause the photographing unit 30 to take an image a predetermined time (e.g., several seconds) after the interior light 4 is turned off. By providing a predetermined time between the turning off of the interior light 4 and the taking of an image in this way, exposure adjustment (S105) can be performed appropriately.

Also, although not specifically shown in Figures 8A and 8B, the control unit 8 may take another photograph each time a predetermined time has elapsed since the first photograph. The length of this predetermined time may be the same as the predetermined time in step S103, or it may be different.

In step S107, the control unit 8 recognizes the opening angle of the refrigerator compartment doors 211, 212. For example, the control unit 8 performs image processing such as edge detection to recognize the opening angle of the refrigerator compartment door 211, 212 that is open. Note that the "opening angle" is the angle at which the refrigerator compartment doors 211, 212 are open, and is recognized, for example, based on the front end of the housing 1 (see Figure 4) when the refrigerator 100 is viewed from above.

Next, in step S108 of Figure 8B, the control unit 8 determines whether the opening angle of the refrigerator compartment doors 211, 212 is equal to or greater than a predetermined value. This predetermined value is a preset threshold value for the opening angle that serves as the criterion for determining whether to save the photographed results (S109) or discard them (S110).

In step S108, if the opening angle of at least one of the refrigerator compartment doors 211, 212 is equal to or greater than the predetermined value (S108: Yes), the control unit 8 proceeds to step S109. In step S109, the control unit 8 saves the image capture results and proceeds to step S112. This is because if the opening angle of the refrigerator compartment doors 211, 212 is equal to or greater than the predetermined value, there is a high possibility that the refrigerator compartment 21, etc. has been properly photographed.

In addition, the photographic results when only the left refrigerator compartment door 211 is open, the photographic results when only the right refrigerator compartment door 212 is open, and the photographic results when both the left and right refrigerator compartment doors 211 and 212 are open may be saved in different memory areas.

Furthermore, in step S108, if the opening angles of the refrigerator compartment doors 211, 212 are both less than the predetermined value (S108: No), the control unit 8 proceeds to step S110. In step S110, the control unit 8 discards the image capture results. This is because if the opening angles of the refrigerator compartment doors 211, 212 are less than the predetermined value, it is highly likely that the refrigerator compartment 21, etc. have not been properly imaged.

Then, in step S111, the control unit 8 sounds the buzzer 5. That is, if an image is captured when the opening angle of the refrigerator compartment doors 211, 212 (doors) is less than a predetermined value (S108: No), the control unit 8 sounds the buzzer 5 (S111). This notifies the user that the image capture results have been discarded. Note that the sound of the buzzer 5 in step S111 may be different from the sound of the buzzer 5 in step S104 (when the interior light 4 is turned off) so that the user can distinguish between the two. After processing step S111, the control unit 8 proceeds to step S112. Instead of sounding the buzzer 5, the refrigerator or the mobile terminal 54 used by the user may output a user-perceptible signal such as light or vibration.

In step S112, the control unit 8 turns on the interior light 4. That is, the control unit 8 turns on the interior light 4 while leaving the camera LED 31c on. In this way, after the control unit 8 causes the photographing unit 30 to take an image (S106 in FIG. 8A), it turns on the interior light 4 (S112 in FIG. 8B). This brightly illuminates the refrigerator compartment 21, making it easier for the user to see the refrigerator compartment 21.

Next, in step S113, the control unit 8 determines whether a close signal has been input from the door sensor 6. If a close signal has not been input from the door sensor 6 (S113: No), the control unit 8 returns to step S102. On the other hand, if a close signal has been input from the door sensor 6 (S113: Yes), the control unit 8 proceeds to step S114.

In step S114, the control unit 8 turns off the interior light 4 and the camera LED 31c.
In step S115, the control unit 8 selects a photographing result. For example, the control unit 8 selects the most recent of the multiple photographing results stored as the photographing result to be transmitted to the server 53. In this way, by completing the saving or discarding of the photographing result for each photograph, it is possible to reduce the amount of storage required by the control unit 8, reduce the amount of data that needs to be transmitted to the server 53, or appropriately notify the user of the fact when the photographing result is to be discarded. Of course, it is also possible to notify the user of the fact when the photographing result is to be saved. It is sufficient if the user can recognize whether the photographing result has been saved or discarded.
To make it easier for the user to recognize each photographic result, the time interval between photographs can be, for example, three seconds or more. In this way, when an output is generated that identifies a certain photographic result, the user can easily understand that the output is for that photographic result. For example, if the time interval is less than three seconds, it may be difficult for the user to distinguish whether the output is for a certain photographic result or for a photographic result one or more years before or after that.
On the other hand, if the time interval is too long, the opening time of the refrigerator compartment doors 211, 212 will be long, resulting in a large amount of cold air flowing out of the refrigerator compartment, so the time interval is set to 8 seconds or less, preferably 5 seconds or less.
If step S111 is not executed, step S106 may be executed to continuously capture multiple images, in which case steps S108 to S110 can be executed to process each of the multiple image data.

In step S116, the control unit 8 transmits the photographed results to the server 53. That is, the control unit 8 transmits the photographed image data to the server 53 sequentially via the wireless LAN unit 31g, router 51, and network 52 shown in FIG. 7. After performing the processing of step S116, the control unit 8 ends the series of processes (END).

In addition, in the server 53, for example, the right and left areas of the refrigerator compartment 21 (see Figure 3), as well as the area of the left refrigerator compartment door 211 (see Figure 3) and the area of the right refrigerator compartment door 212 (see Figure 3) are extracted from the photographed results. The server 53 then individually updates the images of each area to the latest version. Furthermore, in response to a request signal from the mobile terminal 54, the captured image data is sent from the server 53 to the mobile terminal 54 and displayed in a specified manner on the display (not shown) of the mobile terminal 54.

FIG. 9 is an explanatory diagram showing the process of opening and closing the refrigerator compartment doors 211 and 212 in time sequence.
9 , the states of refrigerator compartment doors 211, 212, as well as the states of interior light 4 and camera LED 31c are shown in chronological order in the alphabetical order A, B, C, ..., J, K, and L. Also, in FIG. 9 , when interior light 4 of refrigerator 100 is on, it is shown (e.g., state B), and when interior light 4 is off, it is not shown (e.g., state A). Similarly, in FIG. 9 , when camera LED 31c is on, it is shown (e.g., state B), and when camera LED 31c is off, it is not shown (e.g., state A).

In the example shown in FIG. 9, the user opens the left refrigerator door 211 from state A, where both the left and right refrigerator doors 211, 212 are closed (states B, C, and D). After this, the camera LED 31c remains lit until both the left and right refrigerator doors 211, 212 are closed (S102 in FIG. 8A). Furthermore, as shown in states B and C, both the interior light 4 and the camera LED 31c remain lit (S102) until a predetermined time has elapsed since the left refrigerator door 211 began to be opened (S103: No). Then, when a predetermined time has elapsed since the left refrigerator door 211 began to be opened (S103: Yes in FIG. 8A), the interior light 4 is turned off and automatic photography is performed with the camera LED 31c lit, as shown in state D (S104-S106). Then, in state E, the interior light 4 is turned on again (S112 in FIG. 8B).

In states E and F, the left refrigerator door 211 is closed while the right refrigerator door 212 is open. In this case, as shown in state F, the interior light 4 is turned off and automatic photography is performed again with the camera LED 31c turned on (S104 to S106 in Figure 8A). Then, in the example of Figure 9, the user closes the left refrigerator door 211 once and then opens it again, as shown in states G, H, and I.

Even in such a case, as shown in State I, automatic photography is performed again with the interior light 4 turned off and the camera LED 31c turned on (S104 to S106 in Figure 8A). This allows photography to be performed with both the left and right refrigerator compartment doors 211, 212 open. After that, as shown in State J, the interior light 4 is turned on again (S112 in Figure 8B). In this way, every time at least one of the left and right refrigerator compartment doors 211, 212 is opened, the control unit 8 causes the camera unit 3 to photograph the refrigerator compartment 21, etc.

Furthermore, as shown in state K, when the user presses the capture button 7 (see Figure 3), the control unit 8 performs manual capture. In this case, although not specifically shown in the flowcharts of Figures 8A and 8B, the control unit 8 performs the following process. That is, when the capture button 7 is pressed after at least one of the refrigerator compartment doors 211, 212 (doors) is opened, the control unit 8 turns off the interior light 4 and causes the capture unit 30 to capture an image with the camera LED 31c (illumination unit) turned on (state K). Note that when the interior light 4 is turned off, the control unit 8 may sound the buzzer 5 to reduce any sense of discomfort felt by the user. After performing manual capture, the control unit 8 turns the interior light 4 back on, as shown in state L.

When a photograph is taken by operating the capture button 7 (manual capture), it is preferable that the control unit 8 display the results of this capture on the mobile device 54 (see Figure 7) in priority over other capture results (automatic capture results). This allows the user to check the results of the photograph taken by pressing the capture button 7 on their own mobile device 54 (see Figure 7).

Furthermore, after taking a photograph based on the operation of the photographing button 7 (after manual photographing), it is preferable that the control unit 8 not take another photograph (automatic photographing) while at least one of the refrigerator compartment doors 211, 212 (doors) is open. This is because if the results of manual photographing are sent preferentially, there is no particular need to take an automatic photograph afterwards.

Furthermore, if the user does not fully close at least one of the refrigerator doors 211, 212, leaving the door ajar, automatic photography is performed at predetermined intervals, but the photographed images are not sent to the server 53 (see Figure 7). This is because, in the case of ajar doors, the open angle of the refrigerator doors 211, 212 is usually less than the predetermined value (S108: No in Figure 8B), and the photographed images are discarded (S110).

FIG. 10 shows an example of the result of photography by the camera unit, without image processing.
In Fig. 10, the upper side of the page is the front and the lower side is the rear, so the left and right are reversed compared to Fig. 3 (the same applies to Figs. 11, 12A, and 12B). In the example of Fig. 10, both the left and right refrigerator compartment doors 211, 212 are open, and in addition to the refrigerator compartment 21, door pocket 211c of the left refrigerator compartment door 211 and door pocket 212c of the right refrigerator compartment door 212 are also photographed.

In particular, by using a fisheye lens as the lens 31a of the camera unit 3 (see Figure 5), it is possible to capture images with a wide angle of view. However, without image processing, as shown in Figure 10, ridgelines that are actually straight appear curved, and even for ridgelines of the same length, the number of pixels in the captured image decreases as they become farther away from the camera unit 3.

FIG. 11 shows an example of the photographic result of the camera unit 3 after image processing.
For example, predetermined image processing is performed by server 53 (see FIG. 7) based on the photographed result of Fig. 10, and the images are converted into images that appear as if they were taken from the front of refrigerator compartment 21 and left and right refrigerator compartment doors 211, 212. Such images are displayed on mobile terminal 54 (see FIG. 7), allowing the user to see at a glance what foods are stored in refrigerator compartment 21, etc.
Next, the details of the exposure adjustment process (S105 in FIG. 8A) by the control unit 8 will be described in detail with reference to FIG. 12A (first embodiment) and FIG. 12B (comparative example).

FIG. 12A is an explanatory diagram showing candidates for areas that serve as a reference when performing exposure adjustment in the refrigerator according to the first embodiment.
As described above, the exposure adjustment process (S105 in FIG. 8A) adjusts the gain used to amplify the electrical signal output from the image sensor 31e (see FIG. 7). In the first embodiment, at least one of the three rectangular regions 61 to 63 shown in FIG. 12A is used as a reference for indicating the degree of image brightness in exposure adjustment. That is, at least one of the region 61 on the left refrigerator compartment door 211, the region 62 on the right refrigerator compartment door 212, and a predetermined region 63 between these regions 61 and 62 is used. Note that the positions and sizes of the regions 61 to 63 may be changed as appropriate based on the opening angles of the refrigerator compartment doors 211 and 212.

In the example of Figure 12A, the left refrigerator door 211 is open, but the right refrigerator door 212 is closed. In this case, the control unit 8 adjusts the exposure based on the average brightness of the area 61 corresponding to the left refrigerator door 211. In other words, the control unit 8 adjusts the gain when amplifying the electrical signal output from the image sensor 31e (see Figure 7) so that the average brightness of the area 61 falls within a predetermined range. As a result, the area 62 of the right refrigerator door 212, which is closed, and the area 63, part of which is outside the left refrigerator door 211, are excluded from the basis for adjusting the gain. Therefore, even when the area around the refrigerator is dark, the control unit 8 can adjust the exposure appropriately without increasing the gain too much.

In this way, when either the right-side refrigerator door 212 (right door) or the left-side refrigerator door 211 (left door) is opened, the control unit 8 adjusts the exposure gain during photography based on the area corresponding to the door within the field of view of the photography unit 30. Furthermore, signals indicating the open/closed state of the left and right refrigerator doors 211, 212 are output to the control unit 8 from the left and right door sensors 6 (see Figure 7), respectively.

Alternatively, exposure adjustment may be performed using the area with the highest average image brightness among the three areas 61-63 as the reference. In other words, the control unit 8 adjusts the exposure gain during shooting based on the area with the highest average image brightness among the multiple areas 61-63 included in the field of view of the shooting unit 30. This prevents the gain from becoming too high, and ultimately reduces blown-out highlights (halation) in the captured image.

In addition, exposure adjustment may be performed based on the area of the left or right refrigerator compartment door 211, 212 that includes at least a portion of the door that has received an open signal from the door sensor 6 and that has the highest average image brightness. This allows the control unit 8 to appropriately set the gain for exposure adjustment even in situations where, for example, light irradiated from outside the refrigerator 100 is reflected by the surface of the closed refrigerator compartment door 212. This makes it possible to prevent overexposure and underexposure from occurring in the captured image.

FIG. 12B is an explanatory diagram showing an area that serves as a reference for brightness when adjusting exposure in a refrigerator according to a comparative example.
In the comparative example of Fig. 12B, a single rectangular area 60 is set as a reference for adjusting exposure. For example, when the left and right refrigerator compartment doors 211, 212 are open, rectangular area 60 is set to include both refrigerator compartment doors 211, 212. When a single large area 60 like this is used and the surroundings of refrigerator 100 are dark, right refrigerator compartment door 212 appears dark in the image captured, so a higher gain is set to increase sensitivity to light. As a result, overexposure is likely to occur in the left refrigerator compartment door 211 when it is open and in the left half of refrigerator compartment 21 in the captured image.

In contrast, in the first embodiment (see Figure 12A), the control unit 8 selects the reference area for adjusting exposure based on, for example, the open/closed state of the refrigerator compartment doors 211, 212. This reduces blown-out highlights in the captured image, allowing a clear image that is easy for the user to view to be displayed on the mobile terminal 54 (see Figure 7).

The wall surfaces of the refrigerator compartment 21 (storage compartment) and the inner panels 211b, 212b (see Figure 3) of the refrigerator compartment doors 211, 212 (doors) may each be white. Here, "white" refers to a color that reflects light with a reflectance of 50% or more across substantially the entire visible light spectrum. From another perspective, "white" refers to a color in which the brightness of each of the R (Red), G (Green), and B (Blue) elements in the RGB color system is substantially the same and that reflects light with a reflectance of 50% or more.

When the wall surfaces of the refrigerator compartment 21 are white, the captured image tends to be overexposed. However, the control unit 8 can suppress overexposure by appropriately selecting the reference area when adjusting the exposure. In this way, according to the first embodiment, the camera unit 3 can be used to appropriately capture images of the refrigerator compartment 21, etc. This makes it easier for the user to visually confirm the state of the refrigerator compartment 21, etc. by looking at the mobile terminal 54.

Second Embodiment
The second embodiment differs from the first embodiment in that a photo button 7A (see FIG. 13) is provided near handles 211e, 212e (see FIG. 13) provided on the undersides of left and right refrigerator compartment doors 211, 212 (see FIG. 13). The rest of the second embodiment is the same as the first embodiment. Therefore, only the differences from the first embodiment will be described, and a description of the overlapping parts will be omitted.

FIG. 13 is a front view and a bottom view of left and right refrigerator compartment doors 211, 212 provided in a refrigerator 100A according to the second embodiment.
13, a bottom view of the refrigerator compartment doors 211, 212 is shown below the front view of the refrigerator compartment doors 211, 212. As shown in Fig. 13, a handle portion 211e that is recessed upward is provided on the bottom surface of the left refrigerator compartment door 211, extending over a predetermined length in the horizontal direction.

Furthermore, on the underside of the left refrigerator compartment door 211, inside the handle 211e (on the opposite side from the hinge shaft 211f), there is provided a capture button 7A that is pressed by the user when taking a manual photo. In other words, the capture button 7A is provided outside the handle 211e of the refrigerator compartment door 211 (door), near this handle 211e. Similarly, a handle 212e and a capture button 7A are provided on the underside of the right refrigerator compartment door 212.

By positioning the capture button 7A in this manner, it is possible to prevent the user from accidentally pressing the capture button 7A when, for example, placing their hand on the handle 212e to open the right-side refrigerator door 212. When a user places their right hand on the right-side handle 211e, their palm faces upward and their right thumb is positioned on the outside (right side). In the example of Figure 13, the capture button 7A is located on the inside (left side) of the handle 212e. This prevents the user's right thumb from accidentally touching the capture button 7A when they place their right hand on the handle 212e. The same can be said for the capture button 7A on the left-side refrigerator door 211.

Furthermore, since the photo button 7A is provided near the handle 212e, the user can place their hand on the refrigerator compartment door 212 and move their hand to press the photo button 7A by feel of their fingertips. Therefore, the user does not need to release their hand from the refrigerator compartment door 212 when pressing the photo button 7A, improving operability. Furthermore, since the user can take a photo with the refrigerator compartment door 212 opened to a predetermined position and the rotation of the refrigerator compartment door 212 stopped, blurring of the captured image can be reduced. Furthermore, compared to providing the photo button 7A on the inside of the refrigerator compartment door 212 or on the wall of the refrigerator compartment 21, the user's hand can be prevented from appearing in the captured image. Furthermore, providing the photo button 7A near the handle 212e also allows the wiring for the rotating partition 211d (see FIG. 3) and the operation panel (not shown) to be shared with the wiring for the photo button 7A.
The processing of the control unit 8 (see FIG. 7) based on the operation of the shooting button 7A is the same as that in the first embodiment (see FIGS. 8A and 8B), and therefore will not be described here.

FIG. 14A is a cross-sectional perspective view of the lower part of the right refrigerator compartment door 212 of the refrigerator.
The configuration in Fig. 14A corresponds to Fig. 13. As shown in Fig. 14A, the photo button 7A on the right-side refrigerator compartment door 212 includes a resin switch portion 71A that is pressed by the user, a laminate film 72A attached to the outside of the switch portion 71A, and a switch board 73A that generates a predetermined electrical signal when the switch portion 71A is pressed. A plurality of wires 80 are connected to the switch board 73A. These wires 80 include power lines and signal lines. The left-side refrigerator compartment door 211 (see Fig. 13) also has a similar configuration.

FIG. 14B is a cross-sectional view of the lower part of the right refrigerator compartment door 212 of the refrigerator.
The configuration in Fig. 14B corresponds to Fig. 13 and Fig. 14A. As shown in Fig. 14B, a partition wall 92 is provided inside the refrigerator compartment door 212 (door) to separate an area 91a around the photograph button 7A from the rest of the room. The partition wall 92 is provided with an insertion hole 93 for the wiring 80 connected to the photograph button 7A. A seal member 94 is provided in the insertion hole 93 to close the insertion hole 93. For example, urethane foam is used as the seal member 94. The wiring 80 is inserted through the seal member 94. In other words, the insertion hole 93 is closed by the seal member 94 with the wiring 80 inserted through the insertion hole 93.

By providing this type of sealing member 94, when an insulating material such as urethane is injected (filled) into the internal space of the refrigerator compartment door 212 during manufacturing, the insulating material can be prevented from leaking out toward the capture button 7A through the insertion hole 93. It also makes it possible to pull out the wiring 80 connected to the capture button 7A to the area 91b above the partition wall 92.

<<Variations>>
The refrigerator 100 and the like according to the present disclosure have been described above in accordance with the various embodiments, but the refrigerator 100 and the like are not limited to these embodiments and various modifications can be made.
For example, in each embodiment, the camera unit 3 is provided on the top surface of the housing 1 of the refrigerator 100 (see FIG. 2 ), but this is not limiting. That is, the camera unit 3 may be provided at another predetermined position outside the housing 1.

In addition, in each embodiment, a configuration has been described in which the camera LED 31c (see FIG. 5) is arranged behind the lens 31a (see FIG. 5), but this is not limited to this. For example, the camera LED may be arranged in front of the lens 31a. Furthermore, multiple camera LEDs may be arranged in a predetermined manner.

In addition, in each embodiment, as an example, a case has been described in which the camera unit 3 photographs the interior of the refrigerator compartment 21 and the refrigerator compartment doors 211 and 212, but this is not limited to this. For example, the camera unit 3 may photograph at least one of the drawer-type ice-making compartment door 221 (door: see FIG. 1), the upper freezer compartment door 231 (door), the vegetable compartment door 241 (door), and the lower freezer compartment door 251 (door).

In addition, in each embodiment, a configuration in which the photo button 7 (see Figure 3) is provided on the left and right refrigerator compartment doors 211, 212 has been described, but this is not limited to this. For example, a photo button (not shown) may be provided on at least one of the drawer-type ice maker door 221 (see Figure 1), the upper freezer compartment door 231, the vegetable compartment door 241, and the lower freezer compartment door 251.

In addition, in each embodiment, when at least one of the refrigerator compartment doors 211, 212 is opened (S101: Yes in Figure 8A), the control unit 8 turns on the interior light 4 and the camera LED 31c (S102). However, this is not limited to this. In other words, when at least one of the refrigerator compartment doors 211, 212 is opened, the control unit 8 may turn on the interior light 4 and the camera LED 31c at a predetermined lag.

In addition, in each embodiment, the control unit 8 keeps the camera LED 31c lit from the time at least one of the left and right refrigerator compartment doors 211, 212 is opened until both refrigerator compartment doors 211, 212 are closed, but this is not limited to this. For example, when exposure adjustment (S105 in Figure 8A) is not being performed, the control unit 8 may turn off the camera LED 31c at a predetermined timing.

In addition, in the second embodiment, a configuration in which recessed handle portions 211e, 212e are provided on the underside of the refrigerator compartment doors 211, 212 (see Figure 13) has been described, but this is not limited to this. For example, a configuration in which a predetermined door handle (not shown) is provided on the refrigerator compartment doors 211, 212 may also be used. In this case, a photo button (not shown) may be provided on the outside of the door handle, or a photo button (not shown) may be provided in a predetermined location on the door handle itself.

Furthermore, the refrigerator 100 and the like described in each embodiment are merely examples and can be applied to other types of refrigerators or storage cabinets. For example, each embodiment can be applied to a refrigerator with a single-wing refrigerator compartment door (not shown) or a portable refrigerator (not shown). Photographing can be performed by receiving information via wired or wireless communication from a computing device or sensor provided in the storage cabinet, and the photographing device may be separate from the storage cabinet.

Furthermore, each embodiment has been described in detail to clearly explain the present invention, and is not necessarily limited to having all of the described configurations. Furthermore, some of the configurations of the embodiments can be added to, deleted from, or replaced with other configurations.
Furthermore, the mechanisms and configurations described above are those that are considered necessary for the explanation, and do not necessarily represent all mechanisms and configurations of the product.

100, 100A refrigerator (storage)
1 Housing 10 Opening 21 Refrigerating compartment (storage compartment)
211 Refrigerator door (door, first door)
212 Refrigerator door (door, first door)
211a, 212a Hinge 211b, 212b Inner plate 211e, 212e Handle portion 22 Ice making compartment (storage compartment)
221 Ice maker door (door, second door)
23 Upper freezer compartment (storage compartment)
231 Upper freezer door (door, second door)
24 Vegetable compartment (storage compartment)
241 Vegetable compartment door (door, second door)
25 Lower freezer compartment (storage compartment)
251 Lower freezer door (door, second door)
3 Camera unit 4 Interior light 5 Buzzer 6 Door sensor 7, 7A Photo button 8 Control unit 31c Camera LED
30: Imaging unit 54: Portable terminal 61, 62, 63: Area 80: Wiring 91a: Area 92: Partition wall 93: Insertion hole 94: Sealing member

Claims (3)

a housing having a storage chamber;
a door that closes the opening of the storage chamber;
a door sensor that detects whether the door is open or closed;
and an imaging unit that images at least the storage chamber,
a control unit that, after the door opening operation is detected by the door sensor , issues a command to the photographing unit to photograph while the door opening operation is being detected, determines whether to save or discard the most recently photographed image based on the degree of opening of the door, and, in accordance with the determination, completes the saving if the image is to be saved so that it can be selected as an image that can be sent to the server, or completes the discarding if the image is to be discarded because it cannot be selected as an image that can be sent to the server, and then repeats the next photographing, decision on whether to save and discard, and execution of saving or discarding until the open door is closed;
The criteria for the above-mentioned judgment are not to unconditionally judge whether the latest captured image should be saved, but rather there is a possibility of either saving or discarding the image.
A storage facility characterized by : The doors include a hinged first door and a drawer-type second door;
The second door is lower in height than the first door,
The storage facility according to claim 1 , wherein an area in the field of view of the photographing unit in which the second door is visible is set to be outside the scope of exposure adjustment in the photographing unit when the first door is opened. The control unit recognizes the opening angle of the door as the degree of opening of the door,
The criteria for the judgment are based on the result of comparing the opening angle with a preset criterion.
The storage facility according to claim 1, characterized in that