JP7561815B2 - Camera-linked lighting control system - Google Patents

Description

The present invention relates to a camera-linked lighting control system equipped with a camera and a lighting device.

A system has been proposed that links lighting devices and cameras and controls the lighting devices based on image information captured by the camera. Patent Document 1 proposes a system that uses a camera and an image processing unit as a person position detection means and controls lighting so as to turn on lighting devices corresponding to the positions of people.

Patent document 2 describes a lighting device that makes meat look more vivid.

Patent document 3 describes a lighting system in which a light-emitting unit is driven based on light-emitting mode data associated with the type of food being illuminated.

Patent document 4 describes an illumination device that can irradiate optimal light for each food ingredient using a first light source with a warm color system in which the spectral components of blue light, red light, and green light are mixed in a specified ratio, a second light source with a high color temperature in which the spectral components of blue light, red light, and green light are mixed in a ratio different from that of the first light source, and a third light source that has only a blue light spectral component.

JP 2014-007123 A JP 2013-127855 A JP 2013-077387 A Patent No. 5507148

For example, in supermarkets, it is desirable to provide lighting that is most suitable for the products on display, and lighting devices that optimize the color of lighting light according to the type of product are commercially available. However, in the fruit section, for example, it may be necessary to stop selling spring fruits and display summer fruits, or to expand the display area depending on sales. In such cases, there is the issue that the lighting devices must also be replaced.

The present invention aims to provide a camera-linked lighting control system that can adjust the chromaticity of the lighting light to suit the object, without having to change the lighting device depending on the object.

The present invention provides a camera-linked lighting control system including a camera, a device having an image recognition function, a lighting control device, and an object lighting device that irradiates lighting light ,
Taking an image of an object with the camera;
The device having the image recognition function cuts out a target area from the image captured by the camera, and performs object recognition to recognize the target object or the color of the target object in the target area;
The lighting control device controls the chromaticity of the illumination light in accordance with the object or a color of the object obtained by the object recognition.
It is a camera-linked lighting control system.

The present invention provides a camera-linked lighting control system including a camera, a device having an image recognition function, a lighting control device, and an object lighting device that irradiates lighting light ,
Taking an image of an object with the camera;
The device having the image recognition function extracts an object image area from the image captured by the camera using the discrimination ability obtained from the learning data, calculates the chromaticity of the object image area,
The lighting control device performs lighting control to make the illumination light closer to the chromaticity of the object image area.
It is a camera-linked lighting control system.

The present invention provides a camera-linked lighting control system including a camera, a device having an image recognition function, a lighting control device, and an object lighting device that irradiates lighting light ,
Taking an image of an object with the camera;
The device having the image recognition function extracts an object image area from the image captured by the camera using the discrimination ability obtained from the learning data and identifies the type of the object;
The lighting control device performs lighting control to set the illumination light to a chromaticity determined according to the type of the object.
It is a camera-linked lighting control system.

The present invention provides a camera-linked lighting control system including a camera, a device having an image recognition function, a lighting control device, and an object lighting device that irradiates lighting light ,
The device having the image recognition function calculates at least one of an average value of a red component, an average value of a green component, and an average value of a blue component of the pixels in an image of a target area including an object captured by the camera,
the lighting control device controls the chromaticity of the illumination light using at least one of the average value of the red component, the average value of the green component, and the average value of the blue component.
It is a camera-linked lighting control system.

The present invention provides a camera-linked lighting control system including a camera, a device having an image recognition function, a lighting control device, and an object lighting device that irradiates lighting light ,
The device having the image recognition function recognizes an object identification member photographed by the camera, the object identification member indicating a type of object or a lighting condition for the object,
the lighting control device controls the chromaticity of the illumination light in accordance with the type of the object indicated by the object identification member or the lighting conditions for the object.
It is a camera-linked lighting control system.

In the present invention, the device having the image recognition function may be a camera-linked lighting control system that divides the target area or the object image area into areas of two or more representative colors to recognize the object or identify the type of the object .

In the present invention, the object lighting device may be a camera-linked lighting control system that illuminates the object with lighting light having a chromaticity on the blackbody radiation locus when photographing the object.

In the present invention, the lighting control device may be a camera-linked lighting control system that adjusts the illumination light according to the brightness of the object in the image .

In the present invention, there are a plurality of the object illumination devices,
The device having the image recognition function cuts out a plurality of the target areas from the image captured by the camera, associates the target areas with the target lighting devices, and performs the target recognition for each target area.
The lighting control device may be a camera-linked lighting control system that controls the lighting light individually for each of the target areas.

In the present invention, the lighting control device may be a camera-linked lighting control system that changes the chromaticity of the illumination light in the same direction as the chromaticity deviation of the chromaticity coordinates in response to the chromaticity deviation between the chromaticity coordinates of the object and the chromaticity coordinates on the blackbody radiation locus that are close to the chromaticity coordinates.

In the present invention, the object recognition may be performed using at least one of a representative color of the object, a color distribution of the object , a size of the object, and a shape of the object , in a camera-linked lighting control system.

In the present invention, the camera-linked lighting control system further comprises a general lighting device;
The camera-linked lighting control system may be such that, when the chromaticity of the illumination light from the general lighting device is changed, the chromaticity of the illumination light from the object lighting device is also changed in the same direction.

In the present invention, the change in chromaticity of the illumination light of the general illumination device may be a change due to a scheduled operation.

In the present invention, when the device having the image recognition function recognizes that the object does not exist,
The lighting control device may be a camera-linked lighting control system that reduces the total luminous flux of the illumination light to a value smaller than the total luminous flux when the object is present, or that turns off the illumination light.

In the present invention, the lighting control device may be a camera-linked lighting control system that increases a color temperature of the illumination light when the average value of the green component is large.

In the present invention, the lighting control device may be a camera-linked lighting control system that lowers a color temperature of the illumination light when the average value of the red component is large.

In the present invention , the lighting control device may be a camera-linked lighting control system that controls the chromaticity of the illumination light using at least one of the standard deviation of the red component of the pixel and the standard deviation of the green component of the pixel.

In the present invention, the object illumination device includes at least a first light-emitting element, a second light-emitting element, and a third light-emitting element as light-emitting elements of three or more colors,
The present invention may be a camera-linked lighting control system capable of controlling the color temperature and color deviation of the illumination light by mixing the lights emitted by the three color light-emitting elements.

In the present invention, in the object illumination device,
The light emitted by the first light-emitting element, the second light-emitting element, and the third light-emitting element all have a correlated color temperature of 2000 K or more and 6000 K or less,
The camera-linked lighting control system may be such that a difference in color deviation Duv between the light emitted by the first light-emitting element, the second light-emitting element, and the third light-emitting element with the largest color deviation Duv and the light emitted by the light emitted by the third light-emitting element with the smallest color deviation Duv is 10 or more and 50 or less.

In the present invention, the camera-linked lighting control system may be such that a color deviation Duv of light emitted by the first light-emitting element, the second light-emitting element, and the third light-emitting element is not less than −50 and not more than +20.

In the present invention, the spectrum of each of the lights emitted by the first light-emitting element, the second light-emitting element, and the third light-emitting element may have a ratio of the maximum value in the wavelength range of 570 nm or more and 600 nm or less to the maximum value in the wavelength range of 490 nm or more and 570 nm or less of 0.25 or more.

In the present invention, the object lighting device further includes a fourth light-emitting element that emits light having a correlated color temperature of 2000 K or more and 6000 K or less and a color deviation Duv of −50 or more and +20 or less;
The camera-linked lighting control system may be configured so that a light intensity ratio of light emitted by the first light-emitting element, the second light-emitting element, the third light-emitting element, and the fourth light-emitting element is controllable.

In the present invention, the illumination light of the object illumination device can be controlled to a chromaticity suitable for the color tone of the object, thereby making the object appear in the desired color tone.

FIG. 1 is a configuration diagram of a camera-linked lighting control system according to a first embodiment. Flowchart of image recognition and lighting by the camera-linked lighting control system of the first embodiment Image captured for area setting in the camera-linked lighting control system of embodiment 1 An image showing an object and a background in the camera-linked lighting control system of the first embodiment Chromaticity diagram for explaining chromaticity control in the camera-linked lighting control system of the first embodiment. A chromaticity diagram explaining the chromaticities of the three colors red, yellow-white, and blue-white FIG. 1 is a configuration diagram of a camera-linked lighting control system according to a second embodiment. Graph showing an example of scheduled operation of general lighting devices Chromaticity diagram for explaining chromaticity control of general lighting devices and object lighting devices FIG. 1 is an explanatory diagram of a method of using a two-dimensional barcode to indicate the lighting conditions for illuminating an object or an area of interest in the first embodiment. FIG. 1 is an illustration of a method of using an LED signal transmitter to indicate lighting conditions for illuminating an object or area of interest in accordance with a first embodiment. FIG. 11 is an explanatory diagram showing the relationship between food and the color temperature of illumination light that makes the food taste delicious in the third embodiment. FIG. 1 is a configuration diagram of a camera-linked lighting system according to a fourth embodiment. Inside the lamp body of the object lighting device of the fourth embodiment FIG. 11 is a plan view of a light source used in the object illumination device of the fourth embodiment; Chromaticity diagram of three-color LEDs used in the object illumination device of embodiment 4 Spectral diagram of LEDs (D, F) used in the object illumination device of embodiment 4 Spectral diagram of LEDs (D, E) used in the object illumination device of embodiment 4 Chromaticity diagram (full chromaticity range) of three-color LEDs used in the object illumination device of embodiment 4 Schematic diagram to explain color unevenness FIG. 13 is a configuration diagram of an object lighting device used in the camera-linked lighting system of the fifth embodiment. FIG. 13 is an exploded perspective view of a lamp body of an object illumination device according to a fifth embodiment. Chromaticity diagram of four-color LEDs used in the object illumination device of embodiment 5 Synthetic spectrum obtained in the object illumination device of embodiment 5 FIG. 13 is a configuration diagram of a camera-linked lighting control system according to a sixth embodiment. Flowchart of image recognition and lighting by the camera-linked lighting control system of the sixth embodiment FIG. 13 is an explanatory diagram of image segmentation in the camera-linked lighting control system of the sixth embodiment. FIG. 13 is a configuration diagram of a camera-linked lighting control system according to a seventh embodiment. FIG. 13 is an explanatory diagram of image segmentation in the camera-linked lighting control system of the seventh embodiment. Flowchart of image recognition and lighting by the camera-linked lighting control system of embodiment 7 FIG. 13 is an explanatory diagram of a photograph of a refrigerated case with a meat tray and its image segmentation in the camera-linked lighting control system of the seventh embodiment.

<Embodiment 1>
<Applications>
1 shows the configuration of a camera-linked lighting control system 100 according to the first embodiment. This is a camera-linked lighting control system that photographs an object with a camera and illuminates the object with light of a color tone suited to the color of the object.

<Configuration>
The camera-linked lighting control system 100 includes a camera 110 installed on a ceiling 190, a camera and lighting control device 130, object lighting devices 160A and 160B, a gateway 155, a camera control device monitor 132 located at a person's hand, a second lighting control device 150, and the like.

<Object Illumination Device>
The single or multiple object lighting devices 160 (the number of devices may be any number, but in the case of two, for example, the object lighting devices 160A and 160B) are, for example, spotlights, and respectively irradiate objects 180A and 180B (objects 180 when A and B are not distinguished) in object regions 183A and 183B (object regions 183 when A and B are not distinguished) with illumination light 162A and 162B (illumination light 162 when A and B are not distinguished). The object lighting device 160 can change the chromaticity of the illumination light, and is preferably one that can change the color deviation Duv (described later), which is an element other than color temperature. Such a change in the chromaticity of the illumination light is performed using the second illumination control device 150 or the camera and illumination control device 130.

＜Target Object＞
Objects 180A and 180B are placed on backgrounds 182A and 182B, respectively (when A and B are not to be distinguished, they are referred to as background 182).

<Lighting control device>
The second lighting control device 150 is a tablet-type terminal device that includes a touch panel display that serves as both an input unit and a display unit, and also includes a control unit, a storage unit, and a wireless communication unit. By operating lighting control software in the second lighting control device 150, the brightness (total luminous flux) and color of the object lighting device 160 and the like can be controlled. The lighting control software can also perform schedule control that changes the control content depending on the time. The second lighting control device 150 may be a computer, a smartphone, or the like.

<Gateway>
It is possible for the second lighting control device 150 to send a lighting control signal directly (without going through the gateway 155) to the object lighting device 160, but particularly when the second lighting control device 150 is a portable device such as a tablet, it is also possible to transmit, for example, schedule control information via wireless communication 152 to the gateway 155 installed in a fixed location, which the gateway 155 stores, and at a predetermined time transmits a lighting control signal to the object lighting device 160 via wireless communication 154. This is because there are cases in which normal wireless communication cannot be performed from the portable second lighting control device 150.

<Camera and camera/lighting control device>
The camera 110 is, for example, a camera with a fisheye lens, and captures an image 113 within a field of view 112 and transmits it to a camera and lighting control device 130 .

The camera and lighting control device 130 is a device that has camera control functions and lighting control functions, and also has an image recognition function. Its functions are realized by a computer, which is hardware, and software. As an output device, for example, it is equipped with a camera control device monitor 132, which displays the image 113 and the control contents. The camera and lighting control device 130 is connected to the gateway 155 via a communication line 157.

The camera/lighting control device 130 may not have a camera control device monitor 132 and may be operated by communicating with another PC or tablet.

<Lighting control configuration>
The object lighting device 160 performs an operation preset by the second lighting control device 150, which is composed of a tablet or the like, or the camera and lighting control device 130. The operation is stored in, for example, a gateway 155, which is a repeater. When an instruction is received from the camera and lighting control device 130 or the second lighting control device 150, the gateway 155 transmits an operation preset in response to the instruction to the object lighting device 160, and the object lighting device 160 operates according to the instruction of the second lighting control device 150 and the camera and lighting control device 130. Note that communication between these devices is performed through a communication line 117 between the camera 110 and the camera and lighting control device 130, a communication line 157 between the camera and lighting control device 130 and the gateway 155, a wireless communication 152 between the second lighting control device 150 and the gateway 155, and a wireless communication 154 between the gateway 155 and the object lighting device 160, but the wired communication may be wireless communication, and the wireless communication may be wired communication. In addition, communication using infrared rays or visible light may be performed as the communication.

<Image recognition and lighting flow chart>
A flowchart of image recognition and lighting by the camera-linked lighting control system 100 is shown in Figure 2. S1 to S5 are initial settings, and are performed when the target area is changed due to rearrangements in the room. S6 and onwards are performed for each target area (although the camera 110 captures an image of the entire area). The loop from S11 to S6, or S9 to S6, can be performed manually when the target object is changed, or in the case of a store, it can be performed once a day before opening, or at regular intervals, etc., as appropriate.

"S1. Initial illumination" turns on the object illumination devices 160A and 160B.

In "S2. Acquiring initial image", an image 113 is captured by the camera 110, and the image data is input to the camera and lighting control device 130.

In "S3. Area setting", the image 113 captured for area setting, as shown in Figure 3, is displayed on the camera control device monitor 132. Although the image 113 has distortions typical of fisheye lens images, the operator manually specifies the target areas 183A and 183B, which are as small as possible and have shapes such as a rectangle or a circle, that contain the objects 180A and 180B, and stores them in the camera and lighting control device 130. The coordinates of the target areas set here are also applied to cutting out the target areas in images captured later.

In addition to manually setting the target area, it can also be set automatically. Figure 4 shows an image containing an object 180 and a background 182. The image recognition function can recognize the area containing the background 182 and set it as the target area 183. In this case, if a "distinctive background" that is easy for the image recognition function to recognize is used as the background pattern, there is an advantage that even a relatively low-cost device with image recognition function can recognize the background. Here, a grid pattern of white (W), light gray (WD), and gray (D) is used as the "distinctive background." In this case, the image recognition function can recognize corners, etc., indicated by CN, and recognize them as "background" from a regular pattern.

In "S4. Matching areas and lighting devices", the camera control device monitor 132 matches the target area 183A with the object lighting device 160A, and the target area 183B with the object lighting device 160B (similarly if there are other object lighting devices and target areas).

This enables the camera and lighting control device 130 to control the object lighting devices 160A and 160B via the gateway 155.

In "S5. Acquiring background image", an image of the entire area is captured excluding the object. This makes it possible to obtain images of the target areas 183A and 183B without the object.

"S6. Reducing the color deviation of the illumination light in the target area to zero" is a desirable step in "S10. Recognizing the target object" for accurately recognizing the chromaticity of the target object, but is not essential. "Reducing the color deviation to zero" means using illumination light close to the "blackbody radiation locus BBL" described below.

In "S7. Acquiring target image", the camera 110 acquires an image 113 of the target object placed on it.

In "S8. Cutting out the area of the target image", the camera and lighting control device 130 cuts out the image of the target area 183A and 183B whose area was set in S3.

In "S9. Comparison of background image and target image of target area", the camera and lighting control device 130 compares the background image and target image acquired in S5 for the target area. Here, the "target image" is as shown in FIG. 4.

If the object 180 is placed in the target image, it is determined to be "different" from the background image, and the process proceeds to the next step (object recognition).

On the other hand, if the object 180 is not placed in the target image, it will be "same", so the process returns to S6 and the color deviation of the illumination light in the target area is set to zero. Because there is no object, there is no need to emphasize the color, and it is desirable to illuminate with a natural color.

<Object Recognition>
In “S10. Recognizing object”, the image recognition function recognizes the object 180 in the object area 183 .

The "image recognition function" is realized as a function executed by software in the camera and lighting control device 130, and at least a part of the camera and lighting control device 130 becomes the "image recognition function". On the other hand, the "image recognition function" may be realized by being built into the camera 110. Also, the "image recognition function" may be realized in an external server or external PC connected to the camera-linked lighting control system 100.

There are various methods for recognizing objects. One is a method in which the image recognition function does not recognize the shape, but recognizes the color histogram for each pixel, extracts colors other than the color components in the background image, and identifies the color of the object 180. Here, it is also possible to simply detect what color the object is, without going so far as to recognize what the object is. This allows lighting control such as "if the object is red, the color deviation Duv of the chromaticity of the illumination light is made negative," or "if the object is green or yellow, the color deviation Duv of the chromaticity of the illumination light is made positive," as described below.

When the color of the object is similar, the size and shape of the object can be identified, and the object and its color can be recognized based on that. In other words, a small "cherry" (for example, 1 cm in diameter) and a large "apple" (for example, approximately 7 cm in diameter) can be recognized based on size. Also, a "cherry" and a "strawberry" (for example, 3 cm in diameter) can be recognized based on size.

In addition, when the color of the object is similar, the object can be recognized by its shape, such as a "round lemon" or a "long, thin banana."

In other words, it is desirable to recognize an object using at least one of its color distribution, size, and shape.

Furthermore, objects may be recognized using a technique known as deep learning. By having AI (Artificial Intelligence) learn the relationships between many images and objects in advance, the AI as an image recognition function can recognize objects.

On the other hand, as a method of recognizing the object 180, instead of the object 180 itself, a poster displaying the object 180 such as a barcode or two-dimensional barcode may be recognized.

Figure 10 is an explanatory diagram of a method for identifying the lighting conditions for illuminating an object or target area using a two-dimensional barcode. As shown in Figure 10, a QR code (registered trademark) 184, which is a two-dimensional barcode that covers a large area within a target area 183, is prepared, and this QR code 184 is photographed by a camera 110. The area of the QR code 184 is made large because the camera 110 has a wide angle and the target area is captured small, but the QR code 184 need only be large enough to be identified by the camera 110.

When the QR code 184 indicates the name of an object (e.g., "marbled beef"), the camera and lighting control device 130 controls the chromaticity of the object lighting device 160 to the chromaticity corresponding to the object name. At this time, a table showing the correspondence between the object name and the lighting conditions (chromaticity, color temperature, color deviation, total luminous flux) is stored in advance in the camera and lighting control device 130 or in a storage device accessible by the camera and lighting control device 130. When the QR code 184 indicates the lighting conditions themselves, the camera and lighting control device 130 controls the lighting conditions such as the chromaticity of the object lighting device 160 with the lighting conditions indicated by the QR code 184 read by the camera 110.

In addition, instead of the QR code 184, a single or multiple patterns or marks such as circles, squares, triangles, or crosses corresponding to the object or lighting conditions (chromaticity, color temperature, color deviation, total luminous flux) may be provided in the target area 183, and specifically, a covering such as a tablecloth with a pattern or mark may be placed.

Also, an LED signal transmitter 185 may be installed in the target area to indicate the lighting conditions for illuminating the target object or target area. FIG. 11 is an explanatory diagram of a method using an LED signal transmitter. The LED signal transmitter 185 blinks visible light or infrared light corresponding to, for example, the name of the target object or the lighting conditions, and the camera and lighting control device 130 analyzes the blinking through the camera 110 and controls the lighting conditions of the target lighting device 160.

The LED signal transmitter 185 may be equipped with multiple LEDs, and the lighting conditions may be controlled by the number of LEDs that are lit within the target area, for example, to indicate "for vegetables" when one LED is lit or flashing, or "for meat" when two LEDs are lit or flashing.

The above-mentioned items that indicate objects, such as signs, barcodes, two-dimensional barcodes, patterns and marks, and LED signal transmitters, are collectively referred to as object identification components.

The camera-linked lighting control system 100 may operate continuously, for example, during the store's business hours, with the camera 110 automatically photographing the object identification member and the camera-linked lighting control system 100 automatically controlling the lighting conditions. For example, the store manager may train a new sales clerk to replace the object identification member and display a different product when a product in the target area is sold out, and the camera-linked lighting control system 100 may automatically illuminate the new product with lighting conditions suitable for that product without the new sales clerk having to manually give instructions to change the lighting conditions.

<Lighting control according to the color of the object>
In "S11. Illuminate the target area with illumination light according to the recognition result", illumination control is performed according to the color of the target. If there are multiple target areas, illumination control is performed individually for each object illumination device associated with each target area to illuminate the target area.

5 is a chromaticity diagram for explaining chromaticity control in the camera-linked lighting control system 100, showing chromaticity (x, y) in the CIE 1931 chromaticity coordinates. The x-axis indicates the x chromaticity coordinates, the y-axis indicates the y chromaticity coordinates, BBL indicates the blackbody radiation locus, _TCCT indicates the correlated color temperature, and Duv indicates color deviation (1000 times duv).

As described in "S6. Reducing the color deviation of the illumination light in the target area to zero", in order to accurately capture the color of the object 180 when recognizing the object 180, it is preferable that the emitted color of the object illumination device 160 is a color close to the blackbody radiation locus BBL, and one example is light with a color temperature of 4000K and color deviation Duv = 0 as shown at point Z. In other words, if light of a color suited to a different object was irradiated before recognizing the object 180, it is preferable (but not essential) to "reset" it to a color along the blackbody radiation locus BBL before recognizing the color of the object.

As described in "S5. Acquiring a background image", before recognizing the object 180, only the background 182 is illuminated with light from point Z without the object 180 being placed on it, and the image is compared with the image of the object 180. This can improve color recognition of the object 180. In this case, it is preferable that the background 182 be an achromatic color such as gray.

Returning to "S11. Illuminate the target area with illumination light according to the recognition results," if the recognized object is "bread" or "sweets," the chromaticity coordinates of the illumination light are set to point N1 or N2. If the object is "beef," the chromaticity of the illumination light is set to point M2. If the object is pork, the chromaticity of the illumination light is set to point M1. If the object is blue fish (sardines, mackerel, etc.), the chromaticity of the illumination light is set to point F. If the object is leafy vegetables, the chromaticity of the illumination light is set to point G. If the object is a yellow lemon, the chromaticity of the illumination light is set to point L.

These chromaticities of the illumination light are examples, and other chromaticities of the illumination light may be used depending on the type and condition of the food. For example, even for red foods, it is preferable to change the chromaticity of the illumination light finely depending on whether it is apples, strawberries, beef, salmon roe, wine, etc. Even if the object is green, it is preferable to change the chromaticity of the illumination light finely depending on whether it is melon or leafy vegetables. Even if the object is yellow, it is preferable to change the chromaticity of the illumination light finely depending on whether it is lemon, banana, or egg dish. In addition, since the chromaticity of the illumination light is influenced by people's preferences, it is preferable that it can be adjusted by users such as supermarket sales staff. In that case, it is fine to adjust it depending on sales, but it may also be adjusted taking into account the age, sex, eye color (race), and income level of the main customers.

Not only for food, but also for clothing, car bodies, stationery and other products where color is important, this camera-linked lighting control system can be used to illuminate products with a chromaticity that brings out their appeal, leading to increased sales.

If the camera 110 determines in "S9. Comparing the background image of the target area with the target image" that the target object 180 is not present, it is preferable to make the total luminous flux of the illumination light from the target object illumination device 160 smaller than the total luminous flux when the target object 180 is present (for example, to one-tenth of the total luminous flux), or to turn it off. The purpose of this is not only to reduce power consumption, but also, in the case of a product, to visually indicate that the product is sold out.

Whether or not the object 180 exists is determined, for example, as follows.
First, a background image in which no object exists is captured in advance, and the object area is cut out.
Then, in the target region, feature amounts/vectors are calculated and the calculation results are stored as A.
Then, for example, images of the target area are captured at regular time intervals during business hours, and the target area is cut out.
Then, in the target region, feature amounts/vectors are calculated and the calculation results are stored as B.
B is compared with A, and it is determined whether the difference is greater than a preset threshold.
If the difference between B and A is not greater than the threshold, it is determined that B and A are similar, that is, there is no target object.

AKAZE (Accelerated KAZE) feature detection can be used as a method for calculating features/vectors. This is a method of matching by converting the input image to monochrome (an image containing only the V of HSV values (Hue, Saturation, and Value)). Using an image containing only V has the advantage of reducing the amount of processing and absorbing color changes in the background image caused by the color of the lighting.

Because AKAZE feature detection cannot accurately calculate similarity for images without a pattern or with rough edges, the following image correlation method can be used.

Calculate the average values Aave and Bave of the pixel information of A and B. This is for brightness correction of the background image and the target image.
Next, the average value of B*Ave/Bave-A=E is taken as the similarity. In terms of meaning, it is the sum of the differences between each pixel after luminance correction.
The presence or absence of an object is judged as follows: if E<threshold (images are similar), it is judged that there is no object; if E≧threshold (images are different), it is judged that there is an object.

These algorithms have been described as examples of determining whether an object is present or not, but they can also be used to determine what an object is.

In "S10. Recognizing objects", the K-means method can be used to extract representative colors from the image data of the object and recognize what the object is from the representative colors. This is a method in which each pixel of an image plotted in a color space such as RGB or HSV is classified into N color regions (N is 2 or more), and the average value of the region with the largest number of pixels in the classified color space is used as the representative color.

In such cases, the representative color may be the color of the background, tray, plate, label, or other object, but it is desirable to perform processing to remove such representative colors from the candidates for the object's color.

As a result, if the object is beef, for example, it can be determined that its representative color is closer to the representative color of beef than the representative color of an apple that has been acquired in advance, and the object can be identified.

As described above, it is desirable to match the chromaticity of the illumination light to the color of the object, but it is particularly preferable to control the chromaticity of the illumination light so that "if the object is red, the color deviation Duv of the chromaticity of the illumination light is set to the negative side," and "if the object is green or yellow, the color deviation Duv of the chromaticity of the illumination light is set to the positive side." In other words, it is preferable to "change the chromaticity of the illumination light in the same direction as the chromaticity deviation of the object color according to the color deviation, which is the difference between the chromaticity coordinates of the object and the chromaticity coordinates on the blackbody radiation locus that are close to those chromaticity coordinates." Such chromaticity of the illumination light emphasizes the hue of the object, highlighting the deliciousness of food and the vividness of other objects, which can lead to increased product sales.

It is better to use the chromaticity of the illumination light that is not the color of the object itself, but rather a chromaticity on the blackbody radiation locus that is slightly closer to the color of the object. For example, the chromaticity coordinates of the above-mentioned "leafy vegetables" themselves are green, at (0.35, 0.50), but the chromaticity of the illumination light for G is (0.37, 0.40), which is closer to the blackbody radiation locus. In this way, a person looking at the object will perceive the color of the object as vivid when illuminated with normal white light. If illuminated with the color of the object, even a white tray with the object on it will appear to be the color of the object, resulting in very unnatural coloring.

In S11, not only the chromaticity of the object lighting device 160 is adjusted but also the total luminous flux may be controlled (dimmed). The camera 110 can detect whether the object 180 is bright or dark in the image of the object area 183, and can dim the object lighting device 160 that illuminates the object area so that the object 180 has an appropriate brightness.

<Light source of lighting device>
The light source of the object lighting device 160 is one that can vary the color deviation Duv. Normal lighting devices include fixed color lighting devices that use a light source of one color, and color-adjustable lighting devices that use light sources of two colors and have variable color temperature, but neither of these has a function to vary the color deviation Duv. A lighting device with three or more color light sources, for example, one that uses the three primary colors R, G, and B (red, green, and blue), is suitable for the object lighting device 160. Also suitable is a lighting device that uses the three colors R, Yw, and Bw (red, yellow-white, and bluish-white) described below.

Figure 6 shows a chromaticity diagram explaining the chromaticity of the three colors R, Yw, and Bw (red, yellowish white, and bluish white). An R-LED, which is a light-emitting element for the primary color red (R), a Bw-LED, which is a light-emitting element for the non-primary color bluish white (Bw), and a Yw-LED, which is a light-emitting element for the non-primary color yellowish white (Yw), are used. For red (R), the chromaticity of the color in the range enclosed by the chromaticity boundary line E, (0.66, 0.23), (0.423, 0.355), and (0.5, 0.5) in the CIE 1931 chromaticity coordinates shown in Figure 6, and for bluish white (Bw), the CIE 1931 chromaticity of In the coordinate system, the color of the chromaticity in the range surrounded by (0.336, 0.24), (0.352, 0.44), (0.15, 0.2), and (0.2, 0.1) and the yellowish white color (Yw) are preferably colors of the chromaticity in the range surrounded by (0.5, 0.5), (0.423, 0.355), (0.342, 0.312), (0.352, 0.44), (0.37, 0.63) and the chromaticity boundary line E in the CIE1931 chromaticity coordinate system. This makes it possible to widely reproduce colors along the blackbody radiation locus, such as 1800K to 12000K, and colors in the practical range, such as the colors of the blue sky and the sunset, and also to change Duv. In addition, by using the three colors R, Yw, and Bw, the visual sensitivity of Bw (bluish white) light is higher than that of B (blue) light, which is closer to the chromaticity boundary line, so there is an advantage that the power-luminous flux conversion efficiency (lm/W) can be made relatively good compared to when light-emitting elements of the three primary colors R, G, and B are used.

<Embodiment 2>
<Applications>
A camera-linked lighting control system 100B according to a second embodiment is shown in Fig. 7. This is a system that photographs an object with a camera and illuminates the object with light of a color suitable for the object's color. General lighting devices 170A and 170B (collectively referred to as general lighting device 170) are added to the camera-linked lighting control system 100.

<Configuration>
A general lighting device 170, which is a new component of the camera-linked lighting control system 100B, illuminates the entire room, and is installed, for example, on a ceiling 190 above a passageway.

The second lighting control device 150 operates the general lighting device 170 in a scheduled manner, changing the color temperature and dimming rate according to the time. A graph showing an example of scheduled operation is shown in FIG. 8, but for simplicity, the color temperature is changed to only three times, A, B, and C. It is desirable to change the time when the color temperature and dimming rate change depending on the time of sunrise and sunset, but FIG. 8 only shows an example of a specific season without considering such changes. In the morning, time period A, the color temperature is increased to 5000 K (point Z (C) in FIG. 9 described later) and the dimming rate is set to 100%, providing bright illumination; in the daytime, time period B, the color temperature is slightly lowered to 4000 K (point Z) and the dimming rate is set to 90%, and in the evening, time period C, the color temperature is slightly lowered to 3000 K (point Z (W)) and the dimming rate is set to 80%.

In this embodiment, when the color temperature of the general lighting device 170 is changed, the color temperature of the object lighting device 160 for the object is also changed in conjunction with the general lighting device. Figure 9 is a chromaticity diagram for explaining the chromaticity control of the general lighting device and the object lighting device.

When the chromaticity of the illumination light from the general illumination device 170, which is the overall illumination, is Z(C), Z, or Z(W), the chromaticity of the illumination light from the object illumination device 160, which is the object, is N2(C), N2, or N2(W), respectively. This reduces the difference in chromaticity between the illumination light from the general illumination and the illumination light from the object illumination. Note that it is not necessary for the correlated color temperature ( _TCCT ) to change by the same amount as the change in color temperature of the overall illumination, and it is sufficient if it changes in approximately the same direction.

<Embodiment 3>
＜Overview＞
In this embodiment, an experiment is conducted in advance to determine the correlated color temperature that humans perceive as delicious for each type of food, which is the object, and the results are stored in the camera and lighting control device 130. Then, the camera 110 captures the actual object 180, which is food, and performs pixel analysis to control the object lighting device 160 to a correlated color temperature that humans perceive as delicious. Therefore, although the control content differs from that of the first embodiment, the same hardware as in Fig. 1 is used, and therefore the same reference numerals as those of the first embodiment are used for the respective elements.

<Applications>
Using this system, it is possible to light up the food and deli sections of supermarkets according to the type of food or deli, and also to light up restaurants according to the type of food, which can stimulate customers' desire to buy and increase customer satisfaction.

<Experiment on correlated color temperature that makes people feel delicious>
In an experiment on correlated color temperature that people find pleasant, 10 types of dishes (including dishes with unprocessed ingredients such as salads containing carrots, tomatoes, corn, and leafy vegetables) were shown to four subjects, and the correlated color temperature of the lighting on the dishes was changed to measure the subjects' brain waves. The subjects' brain waves were then used to determine whether the dishes looked delicious. The brain waves were measured according to the instructions of an expert, and the evaluation was also entrusted to an expert. In addition, photos were taken of each dish using automatic exposure, and the average and standard deviation of the R, G, and B values of the pixels were analyzed. However, the R, G, and B values of the pixels each take values between 0 and 255. The correlated color temperature that people find delicious was then expressed as a function of the RGB values of the pixels.

Fig. 12 shows the relationship between the correlated color temperature of the lighting that makes food taste delicious, obtained from the results of this experiment. The horizontal axis is the correlated color temperature of the lighting, and the diagram shows the food shown in the photo and the correlated color temperature that makes the food taste delicious.
The relationship between food image data and the correlated color temperature of lighting that makes food taste delicious obtained in this experiment is as follows:

where CCT is the correlated color temperature of the illumination light (unit: K), Rave, Gave, and Bave are the averages of the R, G, and B values of the pixels in the shooting area, respectively, and Rsd, Gsd, and Bsd are the standard deviations of the R, G, and B values of the pixels in the shooting area, respectively.

This method does not identify the type of food and set the correlated color temperature according to the type of food, but rather adjusts the correlated color temperature based on pixel analysis of the target food. This makes it possible to set a correlated color temperature for lighting that makes any food, even food without a name, taste delicious. Note that it may also be used in conjunction with a method that identifies the type of food and sets the lighting to a correlated color temperature that is pre-determined according to the food, which allows for more stable lighting control in cases where the type of food is fixed, such as in a restaurant.

The parameters in this formula are experimental values and will change depending on the dish or subject, but we were able to say the following general trends.

For food with a high pixel R value in the photographed area (when Rave is large), for example food with a large area of red or brown, the Rave coefficient is negative, so a low color temperature tends to be preferred.

In the case of food with many pixel G values in the photographed area (when Gave is large), such as green leafy vegetables, the Gave coefficient is positive, so a high color temperature tends to be preferred. This is a new finding that became clear in this experiment. Previous predictions had been made that the effect of a large amount of "green" in a pixel on color temperature would be relatively neutral, meaning that the green coefficient would be small.

In the case of dishes with a high pixel B value in the shooting area (when Bave is large), the coefficient of Bave is negative, so there was no tendency for high color temperatures to be preferred for dishes with a high blue component. This is also a surprising result, but it may be related to the fact that many dishes do not look blue, and that Bave can be high for dishes with high R and G values other than B (for example, rice, which is white).

The coefficients of Rsd, Gsd, and Bsd, which indicate the standard deviation (variation) of the R, G, and B values of the pixels in the photographed area, were positive for Rsd, negative for Gsd, and slightly positive for Bsd. Therefore, it was found that a positive correlated color temperature is preferred for objects with a large variation in the R values of each pixel in the photographed area (large Rsd), and a negative correlated color temperature is preferred for objects with a large variation in the G values of each pixel in the photographed area (large Gsd). However, with colorful dishes, Rsd, Gsd, and Bsd all tend to be large, which can result in the effect on CCT being canceled out to some extent.

<Lighting control to achieve correlated color temperature that makes people feel delicious>
In the camera-linked lighting control system 100 shown in Fig. 1, the above formula is entered into the camera and lighting control device 130, or into a computer or server accessible by the camera and lighting control device 130, and the food, which is the object 180, is photographed by the camera 110, and the object region 183 is extracted. In this case, it is preferable to extract only the area where the food is present, for example, the plate portion, as the object region 183, but it is also possible to use the entire deli section of a supermarket or the entire table of a restaurant as the object region 183, and exclude background pixels other than the object. In order to exclude background pixels, there are methods such as photographing the background in advance to recognize the characteristics of the background pixels, or manually setting the chromaticity range of the background pixels.

The average value and standard deviation of the RGB values for each pixel in the target area 183, preferably outside the background, are derived and applied to the above formula to control the correlated color temperature of the target lighting device 160. By performing this type of control in restaurants and deli counters, it is possible to make food look more delicious, which is thought to contribute to increased customer satisfaction and increased sales.

The above formula is merely one example, and the key point of this embodiment is to analyze the color components of the pixels in the target area and perform chromaticity control, including correlated color temperature control of the illumination light, based on the analysis results. In this case, it is also relatively important whether each coefficient in the above formula is positive or negative.

In the above explanation, a camera is used in which each pixel has a primary color filter of either R (red), G (green), or B (blue), but a camera in which each pixel has a three-color filter of either C (cyan), M (magenta), or Y (yellow), or a four-color filter of either C, M, Y, or G, may also be used.

<Example of changing Duv as well>
In the above experiment, only the CCT was changed, but preliminary experiments have also been started in which Duv is also changed.

One way to change Duv is to gradually link it to CCT. For example,
CCT<3500 → Duv=-6
3500≦CCT<4000 → Duv=-3
4000≦CCT<5000 → Duv=0
5000≦CCT<5500 → Duv=3
5500≦CCT → Duv=6
and a method of gradually changing Duv to the negative side when the CCT is small, and gradually changing Duv to the positive side when the CCT is large.

There is also a method of changing Duv by continuously linking it to CCT. For example,
CCT<3500 → Duv=-6
3500≦CCT<5500 → Duv=(CCT-3500)*12/2000-6
5500≦CCT → Duv=6
and a method of continuously changing Duv to the negative side when the CCT is small, and continuously changing Duv to the positive side when the CCT is large.

Either way, the difference in appearance caused by the change in lighting due to the different dishes will be clear, making the appeal stronger.

The above values are just examples. For example, if Duv is on the positive side, it may be undesirable for the subject to have a slight greenish cast. Therefore, the Duv value may be shifted from the above values, such as making Duv negative at low CCTs and Duv zero at high CCTs.

<Embodiment 4>
<Basic configuration>
13 shows a configuration diagram of a camera-linked lighting control system 300 according to this embodiment. The camera-linked lighting control system 300 includes a camera 310, a camera and lighting control device 350, an object lighting device 360, and an object 370. The object lighting device 360 is a spotlight, and includes a lamp body 361, a power supply unit 380, and an arm 375 that connects the lamp body 361 to the power supply unit 380.

<Camera>
Camera 310 installed on ceiling 390 is connected to camera and lighting control device 350 via control signal 312. Camera 310 captures an image of object 370, and the image is used to recognize the object within a target area in the image or the chromaticity (color distribution) of the object by an image recognition function in the camera, an external server, or the camera and lighting control device 350.

<Camera and lighting control device>
The camera and lighting control device 350 is composed of a tablet, a smartphone, or a PC, and runs lighting control software 351. The lighting control software 351 adjusts the light emission intensity of the three color LEDs according to the object or the chromaticity of the object photographed by the camera 310 and recognized by the image recognition function. The light emission intensity conditions of the three color LEDs are transmitted as a wireless control signal 352.

<Light unit>
14 is an explanatory diagram showing the inside surrounded by a frame 367 of a lamp body 361. The lamp body 361 comprises a heat sink 364, a light source 320 placed thereon, a lens 368, and a frame 367. As shown in FIG. 13, light emitted from the light source 320 is radiated by the lens 368 as illumination light 362 at a predetermined radiation angle, and illuminates an object 370.

<Power supply section>
13, power supply unit 380 is attached to wiring duct rail 395 installed on ceiling 390 by front attachment part 386 and rear attachment part 388, and is supplied with power (the inside of the wiring duct rail is usually not visible, but in this drawing, the wiring duct rail is shown semi-transparently so that the attached state of power supply unit 380 can be seen). Attachment is performed by turning lever 385.

The power supply unit 380 has a slot for inserting a wireless module 382, and the wireless module 382 transmits a wireless control signal 352 sent from an external camera/lighting control device 350 to the power supply unit 380.

The power supply unit 380 has 3-channel drive outputs for independently driving the three color LEDs. The wireless module 382 transmits a control signal to the power supply unit 380, thereby controlling each of the 3-channel drive outputs.

<Light source>
A plan view of the light source 320 used in this embodiment is shown in Fig. 15. The light source 320 is a light emitting element of three colors arranged in a light emitting area 322 on a substrate 321 having a wiring pattern. As the light emitting elements, 17 pieces each of CSP (Chip Size Package) type LEDs 323-1 (E), CSP type LEDs 323-2 (D), and CSP type LEDs 323-3 (F) are arranged as shown in Fig. 15 and connected in series by wiring inside the substrate. The light source 320 has mounting holes 325, and is fastened to the heat sink 364 by passing screws through the mounting holes 325. The light source 320 has a wiring terminal 326-1 (for CSP type LED 323-1), a wiring terminal 326-2 (for CSP type LED 323-2), a wiring terminal 326-3 (for CSP type LED 323-3), and a wiring terminal 326-0 (common terminal), and is connected to a power supply unit 380 having three outputs, and the light emission of each LED is controlled.

<LED Structure>
The CSP type LED 323-1, CSP type LED 323-2, and CSP type LED 323-3, which are light-emitting elements used in the embodiment, are provided with a blue LED chip made of InGaN or the like, a phosphor-containing resin covering the side and top of the blue LED chip, and a terminal for passing a current on the bottom or side of each LED. As the phosphor, one or several of a green phosphor that absorbs the light of the blue LED and emits green light, a yellow phosphor that emits yellow light, and a red phosphor that emits red light are used.

In each of the above LEDs, for example, (Y1 _{- xGdx} ) _3Al5O12 :Ce2 ⁺ (0≦x≦1) can be used as the yellow phosphor, for example, _Lu3Al5O12 : _Ce2 ⁺ can be used as the green phosphor, and for example, _{SrxCa1 -xAlSiN3 :} Eu3 ⁺ (0≦ _x ≦ ₁ ) phosphor, _Sr [ _LiAl3N4 ]:Eu2 ^+, _{or K2SiF6} : _Mn4 ⁺ phosphor can be used as the red phosphor. Quantum dots can also be used.

<LED chromaticity>
16 is a chromaticity diagram (CIE 1931 chromaticity coordinate diagram) for explaining the chromaticity of these LEDs, in which the line connecting the chromaticities of the blackbody radiation locus is shown as BBL. Also shown is a line representing the correlated color temperature T _CCT , which is the chromaticity that appears to be the closest color to the color temperature T _CT on the BBL, and a line representing the color deviation Duv from the BBL.
The chromaticities of the CSP type LED 323-1, CSP type LED 323-2, and CSP type LED 323-3, which are the light-emitting elements used in this embodiment, are "E" (LED for meat), "D" (LED for bread), and "F" (LED for fresh fish) in Fig. 16. However, "for meat, for bread, and for fresh fish" are merely examples, and "E" is suitable for various objects such as red fruits and "D" for fried foods.

The chromaticity coordinates of the CSP type LED 323-1(D) are (0.458.0.407).
The chromaticity coordinates of the CSP type LED 323-2(E) are (0.412, 0.327).
The chromaticity coordinates of the CSP type LED 323-3(F) are (0.358, 0.342).
The correlated color temperature T _CCT (unit: K (Kelvin)) is approximately 2700K for E and D, and approximately 4500K for F, the difference being approximately 1800K.

The color deviation Duv from BBL is about -1 for D and about -31 for E, and the difference is about 30.
Moreover, the color deviation Duv of each of these three color LEDs is on the negative side of the black body locus (BBL=Black Body Locus).

The chromaticity of each LED can also be displayed in CIE1976 chromaticity coordinates (u', v') instead of CIE1931 chromaticity coordinates (x, y), and the two can be converted to each other using the conversion formulas u' = 4x/(-2x + 12y + 3), v' = 9y/(-2x + 12y + 3). It may also be displayed in other chromaticity coordinate systems.

In particular, the chromaticity of the CSP type LED 323-2 is characterized by a color that is far from the blackbody radiation locus (BBL) and has a large absolute value of color deviation Duv of -31. By combining the CSP type LED 323-2 with the CSP type LED 323-1 and CSP type LED 323-3, which are relatively close to the blackbody radiation locus, the color deviation can be adjusted appropriately.

By appropriately combining the light from the light-emitting elements CSP type LED 323-1 (D), CSP type LED 323-2 (E), and CSP type LED 323-3 (F), it is possible to obtain light of the chromaticity of the triangle and its interior in FIG. 16. For example, the colors of the points indicated as DE, EF, and EF in the figure can be suitably used to match the ingredients.

Conventionally, color-adjustable lighting that changes color temperature has been used to synthesize multiple colors. Color-adjustable lighting changes the "color temperature" along the blackbody radiation locus, adjusting between "cool colors, high color temperature, fluorescent light color" and "warm colors, low color temperature, incandescent light color," achieving lighting that does not seem strange to people. Unlike such "living space" lighting, this embodiment uses lighting that changes the "color deviation from the BBL," which seems strange to people, to bring out the color of the object.

The light-emitting elements CSP type LED 323-1 (D), CSP type LED 323-2 (E), and CSP type LED 323-3 (F) also have characteristic spectra. The spectra of D and F after passing through an optical system such as lens 368 are shown in FIG. 17, and the spectra of D and E are shown in FIG. 18. Each graph is normalized by the maximum value of G (green) in the wavelength range of 490 to 570 nm. Unlike the synthesis of RGB monochromatic light described later, the spectrum of each light-emitting element has spectral components in the entire wavelength range, but the maximum value of R (red) in the wavelength range of 600 to 660 nm > the maximum value of G (green) in the wavelength range of 490 to 570 nm > the maximum value of Y (yellow) in the wavelength range of 570 to 600 nm, and the shape of Y is suppressed (the maximum value of Y is smaller than the maximum values of R and G), emphasizing red. Therefore, the spectrum obtained by synthesizing D, E, and F also has a relationship of "R>G>Y".

Figure 17 compares the spectra of D and F by normalizing the light intensity by the maximum value of G, and shows that F, which has a high color temperature, weakens the R component and strengthens the B component compared to D, which has a low color temperature.

Figure 18 compares the spectra of D and E after normalizing the light intensity by the maximum value of G, and shows that E, which has a large absolute value on the negative side of Duv, strengthens the B and R components compared to D, which has a small absolute value of Duv.

Note that the vertical axis in Figures 17 and 18 represents light intensity, and does not take luminosity into account. The luminosity coefficient in the CIE 1931 relative luminosity curve is 0.06472 at a wavelength of 450 nm (B), 0.95447 at a wavelength of 540 nm (G), 0.89636 at a wavelength of 580 nm (Y), and 0.29803 at a wavelength of 630 nm (R). The coefficients for G and Y are close to 1, while the coefficients for B and R are small. Therefore, the influence of the G and Y components on the overall appearance of the color is large.

Taking this into consideration, in the spectrum of each color, the maximum value of Y (a value that does not take into account visual sensitivity) should be 0.25 or more, with the maximum value of G being 1, and it is more preferable that it is 0.5 or more. On the other hand, it is preferable that the maximum value of Y is below the line connecting the maximum values of G and R, and the Y component is suppressed, and it is more preferable that the maximum value of Y is below the maximum value of G.

Figure 19 will be used to explain the difference between a method that uses light-emitting elements of three colors, D, E, and F, each with a distinctive spectrum, and the RGB (red, green, blue) method, which can in principle produce all colors. In Figure 19, R is red, G is green, and B is blue. Since all colors can be obtained using RGB, one might wonder if there is no difference from the RGB method even if the three colors D, E, and F are used, but as Figure 19 shows, the area enclosed by the three colors D, E, and F is extremely small compared to the area enclosed by the three colors RGB.

A schematic diagram for explaining color unevenness is shown in FIG. 20. The light source 320' is composed of three color LEDs (E, F, G), and illuminates a white object 370' through a lens 368'. At that time, the light from the three LEDs does not completely overlap, so parts shown as D-rich, E-rich, and F-rich are generated. If a light source of the primary colors "RGB" was used, these parts would be R-rich, G-rich, and B-rich, but since R, G, and B are different primary colors, color unevenness would be noticeable. In the present invention, a primary color light source is not used, and three color LEDs in a limited chromaticity range are used, so that the difference in chromaticity in the parts where the light does not overlap sufficiently is small to begin with, essentially reducing color unevenness.

By using D, E, and F lighting, which contain not only R, G, and B components but also a moderate Y component and have a spectrum that is originally suitable for object lighting, or lighting with a mixture of these colors, a variety of foods can be identified by the camera, such as red and yellow fruits, green vegetables, red meat, sashimi, and salmon, fish with blue backs, and brown bread, baked goods, and fried foods, and lighting suitable for each food can be realized. Although this object lighting device was developed with food ingredients in mind, it has been shown to look beautiful when used to light a variety of objects, such as clothing, car bodies, furniture, small items, and flowers. However, there are cases where a color looks beautiful in the store but different at home, so it is preferable to adjust and add light such as E, which has a large absolute value of color deviation from the BBL. This object lighting device is convenient because it allows fine adjustment of the amount of color deviation, that is, how the color appears, by "mixing colors."

In addition, in the light source 320 shown in FIG. 15, the chromaticity difference between the three color LEDs arranged in the light-emitting area 322 is small, which has the advantage that color unevenness when irradiated onto an object is small and almost unnoticeable.

The correlated color temperature T _CCT of each light-emitting element used in the present invention is 2000K or more, preferably 2500K or more, and is 6000K or less, preferably 5000K or less.

The color deviation Duv (1000 times the color deviation duv) from the BBL of each light-emitting element used in the present invention is -50 or more (absolute value 50 or less), preferably -40 or more (absolute value 40 or less), +20 or less, preferably +10 or less, and the difference in color deviation between the two light-emitting elements with the largest difference in color deviation is preferably 50 or less, and preferably 40 or less. By using light-emitting elements with such limited color deviation, color unevenness can be reduced. On the other hand, in order to accommodate objects of various colors, the color deviation between the two light-emitting elements with the largest difference in color deviation is preferably 10 or more, preferably 20 or more, and more preferably 30 or more.

In order to accommodate objects of various colors, the difference in correlated color temperature T _CCT between the light-emitting element with the lowest color temperature (D in embodiment 4) and the light-emitting element with the highest color temperature (F in embodiment 4) used in the present invention is 400 K or more, preferably 800 K or more, and more preferably 1200 K or more. On the other hand, from the viewpoint of reducing color unevenness, the difference is 3000 K or less, preferably 2000 K or less.

<Embodiment 5>
<Basic configuration>
21 shows an object lighting device 560 used in the camera-linked lighting control system according to this embodiment. The object lighting device 560 is a spotlight, and is made up of a light body 561, a power supply unit 580, and an arm 575 that connects the light body 561 to the power supply unit 580, and can be connected to a wiring duct rail. In order to wirelessly control the emitted light color, the power supply unit 580 is provided with a wireless module 582, which is a detachable wireless transceiver, and the emitted light color is changed using a camera and lighting control device 350 connected to the camera 310.

<Light unit>
22 is an exploded perspective view of the lamp body 561. The lamp body 561 includes a heat sink 564, a printed circuit board 521 placed thereon, a multi-lens 566, and twelve lenses 568 integrally molded within the multi-lens. Twelve surface-mounted LEDs 523 are mounted on the printed circuit board 521, and each of the surface-mounted LEDs 523 corresponds to a lens 568.

<Light source>
Fig. 23 is a chromaticity diagram for explaining the chromaticity of LEDs. The surface mount LED 523 is composed of three LEDs of each of the four colors shown in Fig. 23: B27 (2700K), A27 (T _CCT = 2700K, Duv = -10), B46 (4600K), and A46 (T _CCT = 4600K, Duv = -10). This allows the LEDs to emit colors of chromaticity within the square shown in Fig. 23, for example, AW30 (T _CCT = 3000K, Duv = -4), AW35 (T _CCT = 3500K, Duv = -5), and AW42 (T _CCT = 4200K, Duv = -5).

As an example of the spectrum obtained by combining the colors in FIG. 23, the combined spectrum of AW35, that is, T _CCT =3500K, is shown in FIG. 24. The blue component around 450 nm is indicated as B, the green component around 550 nm is indicated as G, the yellow component around 590 nm is indicated as Y, and the red component around 630 nm is indicated as R. In the clothing field, in order to show the object in a natural color appearance, it is preferable that there is light in the entire spectrum range and the color rendering Ra is 90 or more, more preferably 93 or more, and even more preferably 95 or more. Under such conditions, considering that whitish colors that look less yellow are preferred in the clothing field, the Y component is slightly reduced and the R component is increased compared to a typical high color rendering LED shown by the dotted line in FIG. 23. As a result, the color temperature does not change much, and the color deviation Duv is about -5. The half-width indicated by the dotted line is about 180 nm, but is preferably 150 nm or more. In particular, when the peak of R is taken as 100%, Y (590 nm) is 50% or more, preferably 60% or more, and more preferably 70% or more.

The difference in color deviation Duv between the first light-emitting element, designated B here, and the second light-emitting element, designated A, is preferably 5 or more, more preferably 10 or more. On the other hand, for objects that are often seen under normal lighting, such as clothing, car bodies, furniture, small items, and flowers, it is preferable that the difference in Duv be 15 or less, so that little sense of incongruity is created when illuminated with this object lighting device and when illuminated with a normal object lighting device. For food products, it is preferable to set the difference in Duv to 15 or more, with an emphasis on how the colors look in the store, to make meat and red fruit look particularly attractive.

As an example of controlling chromaticity in clothing, first, the clothing category or chromaticity distribution is analyzed based on the color trends used in clothing photographed by a camera. It is generally considered best to control the lighting to a high color temperature for summer clothing and casual clothing, and a low color temperature for winter clothing and luxury clothing. Also, to emphasize red, it is best to set Duv to the negative side. For this type of camera-linked lighting control, it is desirable to fine-tune the control program according to orders based on the brand image of the clothing store.

<Embodiment 6>
<Applications>
The configuration of a camera-linked lighting control system 600 of this embodiment is shown in Fig. 25. This is a camera-linked lighting control system that photographs clothing as an object with a camera and illuminates the object with light of a color suitable for the color of the object.

<Configuration>
The camera-linked lighting control system 600 includes a camera 610 installed on a ceiling 690, a camera and lighting control device 630, an object lighting device 660, a gateway 655, and a camera control device monitor 632 located at a person's hand.

<Camera and camera/lighting control device>
The camera 610 captures an image 613 of an object 680 within its field of view 612 and transmits it to a camera and lighting control device 630 via communication line 617 .

The camera and lighting control device 630 is a device that has camera control functions and lighting control functions, and also has an image recognition function. Its functions are realized by a computer (hardware) and software. As an output device, for example, it is equipped with a camera control device monitor 632, which displays the image 613 and the control contents. The camera and lighting control device 630 is connected to the gateway 655 via a communication line 657.

<Gateway>
The gateway 655 transmits the lighting control signal to the object lighting device 660 via wireless communication 654 .

<Object Illumination Device>
The object lighting device 660 is, for example, a spotlight, and illuminates an object 680 (an object 680F, which is clothing in this case) in an object area 683 with illumination light 662. The object lighting device 660 can change the chromaticity of the illumination light, and in particular can change the color deviation Duv, which is an element other than color temperature. Such control of the chromaticity of the illumination light is performed using the camera and illumination control device 630.

<Operation Overview>
The operation flowchart of the camera-linked lighting control system 600 is shown in Figure 26. In the first step, the lighting device is turned on to illuminate the object. In the second step, the object is photographed by the camera and an image is acquired. In the third step, image segmentation, which will be described below, is performed to extract the object image region. In the fourth step, the colors in the object image region are divided by 4-way clustering (divided into four groups of similar colors). In the fifth step, the average image of the object image region is calculated by taking a weighted average of the chromaticity and area of each cluster. In the sixth step, the object is illuminated with illumination light according to the average chromaticity of the object image region (this is referred to as the "chromaticity of the object image region" or "object color").

<Identifying object image regions using image segmentation>
Image segmentation, which is an AI technology, is performed to identify the area of the clothing, which is the object. Fig. 27 is an explanatory diagram of image segmentation. Fig. 27(a) shows a part of an image captured by the camera 610. From this captured image, the area of the object reflected in the image is extracted as the object image area F in Fig. 27(b), and the object is identified for the part specified by the object image area F.
To achieve high prediction accuracy in image segmentation, learning data is necessary. Here, learning to recognize the shape of clothing is performed in advance, and the image processing device (a software function of the camera and lighting control device 630, or a software function of an external server accessed by the camera and lighting control device 630, or an independent dedicated image processing device) refers to the learning data to identify which parts of the image are clothing and sets them as the object image area F.

As described in detail in the seventh embodiment, by referring to the learning data, it is possible to not only simply identify that the object is clothing, but also to identify the type and color of the clothing, such as a white shirt, a navy blue jacket, a red cardigan, or red and blue mixed pants, and the lighting system may irradiate lighting light according to the identified type and color of the clothing. The learning data used in this case may be stored in the camera and lighting control device 630 equipped with an image recognition function, or may be stored in an external server.

<Identifying the color of an object>
An object image region F determined to be "clothing" may include, for example, the entirety of a blue jacket and a white shirt. Therefore, multi-division clustering is performed to classify the regions into groups of similar color tones, such as F1 and F2 in FIG. 27(b), and the average chromaticity is calculated. Note that the number of divisions in multi-division clustering is preferably between 2 and 6, and in this study, it was set to 4. With current technology, a division number of 6 or less tends to result in fast processing speed and fewer classification errors, since the amount of information is not excessively large.

For simplicity, a two-part clustering will be used. The chromaticity coordinates ( _x1 , _y1 ) of the first color and the number of pixels _a1 that is its area are obtained, and the chromaticity coordinates ( _x2 , _y2 ) of the second color and the number of pixels _a2 that is its area are obtained. From these, the average chromaticity is ( _xf , _yf ) = ( _a1 ( _x1 , _y1 ) + _a2 ( _x2 , _y2 ))/( _a1 + _a2 ).
It is calculated as:

This "multi-partition clustering" is the same technology as that described above in "S10. Recognizing objects" as the K-means method, which classifies each pixel in the image data of the object into N color regions (N is an integer equal to or greater than 2), extracts a representative color whose representative color is the average value of the region with the largest number of pixels in the classified color space, and recognizes what the object is from the representative color. Here, AI is used to narrow down the object image region in advance, and a representative color is extracted for that, allowing for a more appropriate determination of the object.

<Lighting with chromaticity according to the object>
The camera and lighting control device 630 sets the reference chromaticity coordinates to ( _xw , _yw ) and sets the color of the illumination light to a color that is slightly closer to the above-mentioned clothing color ( _xf , _yf ) than white, for example, chromaticity ( _xw +k( _xf - _xw ), _yw +k( _yf - _yw )), where 0<k≦1, and controls the object lighting device 660. The object lighting device 660 provides illumination of that color. However, if the color of the illumination light determined here is not very different from the color of the current illumination light, it is possible not to perform lighting control in order to avoid small changes in the color of the illumination light.

The reference chromaticity coordinates may be, for example, white (0.33, 0.33) or the chromaticity coordinates of the initial illumination light. Also, if you want to illuminate summer clothing with a bluish light to emphasize freshness, the reference chromaticity coordinates may be, for example, (0.38, 0.38) or (0.43, 0.40), which is blackbody radiation at approximately 4000K.

In the above example, chromaticity coordinates according to CIE 1931 were used, but since CIE 1931 is a coordinate system in which the difference in coordinates and the color difference perceived by humans are different, other chromaticity coordinate systems, for example the uv coordinate system according to CIE 1960 or the u'v' chromaticity coordinate system according to CIE 1976 mentioned above, may also be used. In other words, the reference chromaticity coordinates may be ( _u'w , _v'w ), and the color of the illumination light may be a color that is slightly closer to the color of the clothing than white, for example, chromaticity ( _u'w +k( _u'f - _u'w ), _v'w +k( _v'f - _v'w )), where 0<k≦1.

<Embodiment 7>
<Applications>
The camera-linked lighting control system 600 of this embodiment is the same as that of the sixth embodiment, and therefore a description thereof will be omitted. However, the image processing is targeted at "meat 680M in a tray 681T" as shown in Fig. 28. If wrapping is applied to the meat 680M, a price tag label 681L1 or a label 681L2 indicating a sale or the like may be attached, and these need to be removed by image processing.

<Identifying object image regions using image segmentation>
Image segmentation, which is an AI technology, is performed to identify the meat region, which is the object. FIG. 29 is an explanatory diagram of image segmentation. FIG. 29(a) shows a part of an image captured by the camera 610. From this captured image, the region of the object reflected in the image is extracted as the object image region M in FIG. 29(b). The image processing device identifies what object the part specified by the object image region M is.
In order to achieve high prediction accuracy in image segmentation, a large amount of training data is required. Here, training to recognize meat is performed in advance, and the image processing device identifies which parts of the image are meat based on the discrimination ability obtained from the training data, and sets them as the object image region M. At that time, the image processing device determines that the labels 681L1 and 681L2 are not objects.

<Identifying the color of an object>
For the object image region M determined to be "meat" by image segmentation, the image processing device performs multi-division clustering as necessary (although this is not required) and determines the average color of the pixels in that region. The chromaticity coordinates of this average color are ( _xm , _ym ).

<Lighting with chromaticity according to the object>
In the first control method, the camera and lighting control device 630 sets the reference chromaticity coordinates to ( _xw , _yw ), sets the color of the illumination light to a color that is slightly closer to the color of flesh than to white, for example, chromaticity ( _xw +k( _xm - _xw ), _yw +k( _ym - _yw )), where 0<k≦1, and controls the object lighting device 660. The object lighting device 660 provides illumination of that color.

The second control method uses image segmentation and, if necessary, multi-partition clustering to identify the type of meat, specifically lean beef, marbled beef, pork offcuts, pork for cutlets, chicken thighs, chicken wings, and the like. In this case, multi-partition clustering can be used to use information on the distribution of red meat and white fat. Depending on the type of meat recognized as the object, the object illumination device 660 illuminates the object 680 with a chromaticity that corresponds to the type of meat that has been preset. The flow in this case is shown in Figure 30.

Lighting control that uses a method for identifying the image region of an object, such as image segmentation, has the advantage that it is not affected by the fact that the color of the lighting does not change even if the proportion of the background in the image increases as the object decreases, for example when meat is sold and there is little left. For example, Figure 31 (a) is a photo of a refrigerated case containing trays of meat, and Figure 31 (b) is a photo in which the meat region has been extracted using image segmentation (image segmentation was performed by dividing the case into the upper and lower halves). With this method, the same lighting control can be performed regardless of the number of trays of meat. Note that when the number of trays reaches zero, control may be performed such as lowering the dimming rate of the lighting, turning off the lighting, or changing the lighting color to the standard color (the color before it was used for meat).

<Modifications and other details>
The above describes an embodiment of the lighting device according to the present invention. However, the exemplified lighting device can also be configured as follows, for example, and it goes without saying that the present invention is not limited to the lighting device as shown in the above embodiment.

(1) The image of the object lighting device 160 shown in the figure is based on the image of a spotlight, and the image of the general lighting device 170 is based on the image of a square-shaped recessed lighting device, but the form is not limited to other lighting devices, such as downlights, universal downlights with adjustable lighting direction, linear base lights, floodlights, flexible tape lights, etc.

(2) General lighting devices may be used for indoor lighting as well as outdoor lighting, and may be installed anywhere.

(3) When the three primary colors R (red), G (green), and B (blue) are used as the colors of 3CH, blue may be a color that satisfies y<0.4 and x<0.2 and is on the chromaticity boundary line E side of blue-white, and green may be a color that satisfies y>0.4 and is not yellow-white or blue-white.

(4) The color of the illumination light before object recognition is not limited to the color of point Z (color temperature 4000K). For example, if the object is meat, it may be set to a color on the blackbody radiation locus of 3500K in advance, and if the object is fish, it may be set to 5000K. Note that, although it is a color on the blackbody radiation locus, if the absolute value of Duv is 1 or less, it may be considered to be a "color on the blackbody radiation locus."

(5) A lighting control signal may be sent directly from the lighting control device to the lighting device without using a gateway, which is a wireless communication repeater.

(6) The wireless communication method may be, for example, Bluetooth (registered trademark) or Wi-Fi (registered trademark).

(7) The camera/lighting control device is a device that combines camera control functions such as capturing images with a camera, image recognition functions such as segmenting target areas and recognizing objects, and lighting control functions such as lighting control based on object recognition. Although an example is described in which these three functions are realized in software on one computer, these three functions may be handled by a different device. For example, the function of the above-mentioned "lighting control device" may be provided in a "second lighting control device," or the "image recognition function" may be built into the camera or may be realized by a server on a network (not shown). In other words, the "device" or "function" in this specification is not limited to simple hardware, but also refers to a combination of hardware and software, and the place where the function is realized is not limited to the places exemplified in this application.

(8) The light-emitting element is not limited to an LED, but may be any element that emits light, such as an organic electroluminescent (organic LED), an element in which the phosphor and excitation light source are separated, an element that uses discharge, or an element that is excited by an electron beam.

(9) As for the LEDs of each color, in the fourth embodiment, CSP type LEDs are mounted on one board, and in the fifth embodiment, SMD (Surface Mount Device) type LEDs are mounted on a printed board. However, COB type LEDs may also be used, in which blue LED chips are arranged on a board and multiple blue LED chips are covered with a resin containing phosphor. These LEDs can also be used in the first to third embodiments.

(10) Although three-color LEDs and four-color LEDs have been given as examples of light sources using LEDs of multiple light colors, it is also possible to use only two-color LEDs with different color deviations (for example, D and E, F and E in embodiment 4, and A46 and B27 in embodiment 5).

(11) The chromaticity of each LED in the embodiment is an example, and other chromaticities may also be used.

(12) Although an example of wireless control has been shown as a method for controlling the light color of the object lighting device, wired control may also be used.

(13) In this application, the term "object" is used as a concept that includes not only "merchandise" but also "non-sale items" such as art and antiques in a museum and children's paintings at an exhibition.

The above disclosed embodiments are illustrative in all respects and are not intended to be a basis for restrictive interpretation. Therefore, the technical scope of the present invention is not interpreted solely by the above described embodiments, but is defined based on the claims. Furthermore, all modifications within the scope and meaning equivalent to the claims are included.

100, 100B, 300, 600 Camera-linked lighting control system 110, 310, 610 Camera 112, 612 Field of view 113, 613 Image 117, 617 Communication line 130, 350, 630 Camera and lighting control device 132, 632 Camera control device monitor 150 Second lighting control device 152, 154, 654 Wireless communication 155, 655 Gateway 157, 657 Communication line 160, 160A, 160B, 360, 560, 660 Object lighting device 162, 162A, 162B, 362, 662 Illumination light 170, 170A, 170B General lighting device 180, 180A, 180B, 370, 680, 680F, 680M Objects 182, 182A, 182B Background 183, 183A, 183B , 683 Object area 184 QR Code (registered trademark)
185 LED signal transmitter 190, 390, 690 Ceiling 312, 352 Control signal 320 Light source 321 Substrate 322 Light-emitting area 323 CSP type LED
325 Mounting hole 326 Wiring terminal 351 Lighting control software
3 61, 561 Light body 364, 564 Heat sink 367 Frame 368, 568 Lens
3 75, 575 Arm 380, 580 Power supply unit 382, 582 Wireless module 385 Lever 386 Front mounting part 388 Rear mounting part 395 Wiring duct rail 521 Printed circuit board 523 Surface mount type LED
566 Multi-lens F, M Object image area

Claims (13)

A camera-linked lighting control system including a camera, a device having an image recognition function, a lighting control device , and an object lighting device,
Illuminating an object area where an object may exist with illumination light from the object illumination device;
The camera photographs the target area to obtain a target area image;
The device having the image recognition function classifies pixels of the object area image into a plurality of color areas, excludes color areas other than the object from among the color areas, and recognizes a representative color, which is an average color of the color area with the largest number of pixels present in the classified color space among the remaining color areas, as the color of the object;
the lighting control device controls the chromaticity of the illumination light in accordance with a color of the object;
The object lighting device irradiates the illumination light controlled by the lighting control device onto the object area.
Camera-linked lighting control system. A camera-linked lighting control system including a camera, a device having an image recognition function, a lighting control device, and an object lighting device,
Illuminating an object area where an object may exist with illumination light from the object illumination device;
The camera photographs the target area to obtain a target area image;
The device having the image recognition function recognizes the type or color of the object from the object area image,
the lighting control device controls the chromaticity of the illumination light in accordance with a type of the object or a color of the object;
When the device having the image recognition function recognizes that the object does not exist in the object area image , the lighting control device controls the total luminous flux of the illumination light to be smaller than the total luminous flux when the object exists, or controls the illumination light to be turned off .
Camera-linked lighting control system. The object lighting device illuminates the object with the illumination light having a chromaticity on a blackbody radiation locus when photographing the object.
3. The camera-linked lighting control system according to claim 1 or 2 . The lighting control device adjusts the illumination light according to the brightness of the object in the object area image.
3. The camera-linked lighting control system according to claim 1 or 2 . There are a plurality of the object illumination devices;
The device having the image recognition function cuts out a plurality of the object regions from the image captured by the camera, associates the plurality of object regions with the object lighting device, and recognizes the color of the object for each object region;
The lighting control device controls the illumination light individually for each of the target areas.
3. The camera-linked lighting control system according to claim 1 or 2 . the lighting control device changes the chromaticity of the illumination light in a direction similar to the chromaticity deviation between the chromaticity coordinates of the object and the chromaticity coordinates on a blackbody radiation locus that are close to the chromaticity coordinates of the object.
3. The camera-linked lighting control system according to claim 1 or 2 . The camera-linked lighting control system further comprises a general lighting device;
When the chromaticity of the illumination light of the general illumination device is changed, the chromaticity of the illumination light of the object illumination device is also changed in the same direction.
3. The camera-linked lighting control system according to claim 1 or 2 . The change in chromaticity of the illumination light of the general illumination device is due to a schedule operation.
The camera-linked lighting control system according to claim 7 . The object lighting device includes at least a first light-emitting element, a second light-emitting element, and a third light-emitting element as light-emitting elements of three or more colors,
The color temperature and color deviation of the illumination light can be controlled by mixing the colors of the lights emitted by the light-emitting elements.
3. The camera-linked lighting control system according to claim 1 or 2 . In the object illumination device,
The light emitted by the first light-emitting element, the second light-emitting element, and the third light-emitting element all have a correlated color temperature of 2000 K or more and 6000 K or less,
a difference in color deviation Duv between the light having the largest color deviation Duv and the light having the smallest color deviation Duv among the light emitted by the first light-emitting element, the second light-emitting element, and the third light-emitting element is 10 or more and 50 or less;
The camera-linked lighting control system according to claim 9 . a color deviation Duv of light emitted by the first light-emitting element, the second light-emitting element, and the third light-emitting element is greater than or equal to −50 and less than or equal to +20;
The camera-linked lighting control system according to claim 9 . The spectrum of each of the lights emitted by the first light-emitting element, the second light-emitting element, and the third light-emitting element has a ratio of a maximum value in a wavelength region of 570 nm or more and 600 nm or less to a maximum value in a wavelength region of 490 nm or more and 570 nm or less of 0.25 or more.
The camera-linked lighting control system according to claim 9 . The object lighting device further includes a fourth light-emitting element that emits light having a correlated color temperature of 2000 K or more and 6000 K or less and a color deviation Duv of −50 or more and +20 or less;
a light amount ratio between the first light emitting element, the second light emitting element, the third light emitting element, and the fourth light emitting element is controllable;
The camera-linked lighting control system according to claim 9 .