JP2013127535A - Display device, method and program of controlling the same, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To eliminate difficulties in viewing a display screen with sunglasses worn on.SOLUTION: An imaging device (150) takes an image of a user who browses a screen of an image display unit (120). A system control unit (122) causes an image processing unit (112) to detect a face from an image taken by the imaging device (150). The system control unit (122) determines whether or not the user wears sunglasses on the basis of the face detection result. When the user wears sunglasses, the system control unit (122) makes a backlight of the image display unit (120) brighter.

Description

  The present invention relates to a display device, a control method and program thereof, and a recording medium.

  In recent years, many devices such as a videophone system, a mobile phone, and a tablet PC that can photograph a user side with an in-camera have been commercialized.

  For this type of device that may be used outdoors, a display device that automatically adjusts the display brightness according to the ambient brightness is employed.

  Japanese Patent Application Laid-Open No. 2004-151867 describes a technique for brightly illuminating a subject (photographer) by increasing the brightness of a backlight of a liquid crystal display when the subject is dark when photographing with an in-camera.

JP-A-2005-181602

  In a display device that automatically adjusts the display brightness in accordance with the ambient brightness, for example, when viewing the screen with sunglasses, the screen becomes too dark and difficult to see.

  Since the subject is not dark when the subject wears sunglasses, the brightness of the display screen cannot be appropriately controlled even when the technique described in Patent Document 1 is applied.

  An object of the present invention is to present a display device that appropriately controls the brightness of a display screen, a control method and program thereof, and a recording medium when a viewer of the display screen wears sunglasses.

  The display device according to the present invention includes an image display means for displaying an image, an imaging means for photographing a user viewing the display of the image display means, and the user wearing sunglasses based on a photographed image of the imaging means. A determination unit that determines whether or not the user is wearing; and a control unit that controls to increase the luminance of the display unit when the determination unit determines that the user is wearing the sunglasses. It is characterized by that.

  According to the present invention, when the user viewing the display on the image display means wears sunglasses, the brightness of the image display means is automatically increased. it can.

It is a schematic block diagram of one Example of this invention. It is an external view of a present Example. It is an operation | movement flowchart of a present Example. It is an operation | movement flowchart of the sunglasses detection of a present Example. It is a schematic diagram which shows the positional relationship of the user who wears sunglasses, and an image display part. It is operation | movement explanatory drawing of a present Example.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a schematic block diagram of a digital camera incorporating an embodiment of a display device according to the present invention, and FIG. 2 shows an external view thereof.

  The imaging lens 102 of the imaging apparatus 100 illustrated in FIG. 1 includes a zoom lens and a focus lens, and the shutter 104 has a diaphragm function. The image sensor 106 is a CCD image sensor or a CMOS image sensor that converts an optical image into an electrical signal. The A / D converter 108 converts the analog image signal output from the image sensor 106 into a digital signal. The barrier 110 covers the imaging system including the imaging lens 102 to protect the imaging system including the imaging lens 102, the shutter 104, and the imaging element 106 from contamination and damage.

  The output image data of the A / D converter 108 is written into the memory 116 via the image processing unit 112 and the memory control unit 114 or via the memory control unit 114. The memory 116 has a storage capacity sufficient to store a predetermined number of still images or a moving image and sound for a predetermined time.

  The memory 116 also serves as an image display memory (video memory). The D / A converter 118 converts the display image data stored in the memory 116 into an analog signal and supplies the analog signal to the image display unit 120. The image display unit 120 includes a liquid crystal display (LCD), and displays an image corresponding to the analog image signal from the D / A converter 118. By supplying the output image data of the A / D converter 108 to the D / A converter 118 via the memory 116, the image display unit 120 can be used as an electronic viewfinder. This is a so-called through image display.

  The image processing unit 112 performs resize processing such as predetermined pixel interpolation and reduction and color conversion processing on the image data from the A / D converter 108 or the memory control unit 114, and the processed image data is stored in the memory control unit 114. To supply. The image processing unit 112 also performs arithmetic processing for exposure control and focusing control using image data obtained by imaging, and supplies the calculation result to the system control unit 122. The system control unit 122 performs exposure control and distance measurement control according to the calculation result of the image processing unit 112. As a result, TTL (through the lens) AF (autofocus) processing, AE (automatic exposure) processing, and EF (flash pre-emission) processing are performed. The image processing unit 112 further performs predetermined calculation processing using the captured image data, and also performs TTL AWB (auto white balance) processing based on the obtained calculation result.

  The nonvolatile memory 124 is an electrically erasable / recordable memory, and is composed of, for example, an EEPROM. The nonvolatile memory 124 stores constants and programs for operating the system control unit 122. The program here includes a control program for executing various flowcharts described later.

  The system control unit 122 controls the entire imaging apparatus 100. The system control unit 122 implements each process described later by executing a program stored in the nonvolatile memory 124. The system memory 126 includes a RAM, and stores constants and variables for operation of the system control unit 122, a program read from the nonvolatile memory 124, and the like for operation of the system control unit 122. The system control unit 122 controls the memory 116, the D / A converter 118, the image display unit 120, and the like for image display.

  The system timer 128 measures the time used for various controls by a built-in clock.

  The mode switch 130, the shutter button 132 (first shutter switch 132 a and second shutter switch 132 b), and the operation unit 136 are used to input various operation instructions to the system control unit 122. The operation unit 136 includes various switches, buttons, a touch panel, and the like that accept user operations. The mode switch 130 is used for switching various modes.

  The mode switch 130 switches the operation mode of the system control unit 122 to any one of a still image recording mode, a moving image recording mode, a reproduction mode, and the like. The still image recording mode includes an auto shooting mode, an auto scene discrimination mode, a manual mode, various scene modes for setting shooting for each shooting scene, a program AE mode, a custom mode, and the like. The mode switch 130 can directly switch to any of these modes included in the still image shooting mode. Of course, after switching to the still image shooting mode once with the mode switch 130, it may be switched to any of these modes included in the still image shooting mode using another operation member. Similarly, the moving image shooting mode includes a plurality of modes.

  The first shutter switch 132a is turned on when the shutter button 132 is being operated halfway (so-called shooting preparation instruction), and generates the first shutter switch signal SW1. The system control unit 122 starts operations such as AF (autofocus) processing, AE (automatic exposure) processing, AWB (auto white balance) processing, and EF (flash pre-emission) processing in response to the first shutter switch signal SW1.

  The second shutter switch 132b is turned on when the operation of the shutter button 132 is completed, so-called full press (shooting instruction), and generates a second shutter switch signal SW2. In response to the second shutter switch signal SW2, the system control unit 122 starts a series of photographing processing (recording processing) operations from reading a signal from the image sensor 106 to writing photographing image data to the recording medium 146.

  A part of the operation unit 136 is displayed on the image display unit 120 and includes various function icons that can be touch-operated, and appropriate functions are assigned to each scene. Typical function buttons include, for example, an end button, a return button, an image advance button, a jump button, a narrowing button, and an attribute change button. For example, when a menu button is pressed, various settable menu screens are displayed on the image display unit 120. The user can make various settings using the menu screen displayed on the image display unit 120, and the four-way button and the SET button.

  The controller wheel 137 included in the operation unit 136 can be rotated, and is used when a selection item is designated together with a direction button. When the controller wheel 137 is rotated, an electrical pulse signal is generated according to the operation amount, and the system control unit 122 controls each unit of the imaging apparatus 100 based on the pulse signal. Based on the pulse signal from the controller wheel 137, the system control unit 122 can identify the angle and the rotational speed at which the controller wheel 137 is rotated. The controller wheel 137 is an example of an operation member that can detect a rotation operation. For example, it may be a dial type operation member that rotates in response to a user's rotation operation and generates a pulse signal. Further, the operation member may be a so-called touch wheel that detects an operation of rotating a user's finger or the like.

  The power switch 138 is used to switch on / off the power of the imaging apparatus 100.

  The power control unit 140 includes a battery detection circuit, a DC-DC converter, a switch circuit that switches a block to be energized, and the like, and detects whether or not a battery is installed, the type of battery, and the remaining battery level. Further, the power control unit 140 controls the DC-DC converter based on the detection result and the instruction from the system control unit 122, and supplies a necessary voltage to each unit including the recording medium 146 for a necessary period. The power supply unit 142 includes a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, or an AC adapter.

  The recording medium I / F 144 is an interface that connects the recording medium 146 to the imaging apparatus 100. The recording medium 146 is formed of a memory card-shaped semiconductor memory for recording a captured image, a magnetic disk, or the like.

  The imaging lens 148, the imaging device 150, and the A / D converter 152 are imaging units for the in-camera, and are controlled by the system control unit 122, and the captured image by the imaging device 150 is processed by the image processing unit 112.

  An external device can be connected to the connector 154 via a connection cable 156. The recording medium slot 158 is a slot for storing the recording medium 146, and the recording medium I / F 144 is disposed therein. The recording medium slot 158 can be closed by the lid 160 in a state where the recording medium 146 is accommodated.

  The imaging apparatus 100 can perform imaging using selectively one center point AF (autofocus) and face AF. The center single point AF is a function for performing AF with respect to one point at the center position in the photographing screen. The face AF is a function for performing AF on the face in the shooting screen detected by the face detection function.

  The face detection function will be described. The system control unit 122 sends the image data to be detected to the image processing unit 112. The image processing unit 112 applies a horizontal bandpass filter to the image data under the control of the system control unit 122, and then applies a vertical bandpass filter. An edge component is detected from the image data by these horizontal and vertical spatial bandpass filter processes.

  The system control unit 122 performs pattern matching on the detected edge component, and extracts a candidate group of eyes, nose, mouth, and ears. The system control unit 122 determines that an eye pair that satisfies a preset condition (for example, the distance and inclination of two eyes) from the extracted eye candidate group is an eye pair, and the subject having the eye pair N as a candidate group of eyes. Then, the system control unit 122 associates the narrowed-down eye candidate group with other parts (nose, mouth, ears) that form the corresponding face, and passes a preset non-face condition filter. , Detect the face. The system control unit 122 outputs face information obtained from the face detection result, and ends the process. At this time, the system control unit 122 stores feature quantities such as the number of faces in the system memory 126.

  As described above, image analysis is performed on the image data displayed or reproduced and displayed in the live view, the feature amount of the image data is extracted, and the subject information is detected. In this embodiment, face information is taken as an example of subject information, but there are various other types of subject information such as red-eye determination, eye detection, eye-brow detection, and smile detection.

  When autofocusing a specific subject portion on the screen, exposure control, flash light control, and white balance control can be applied to the subject portion. For example, the face AE, the face FE, and the face WB can be simultaneously executed for the face AF. The face AE is a process for optimizing the exposure of the entire screen according to the brightness of the detected face. The face FE is a process that performs dimming control of the flash around the detected face. The face WB is a process for optimizing the white balance (WB) of the entire screen according to the detected face color.

  FIG. 3A is a control flowchart of this embodiment. The image display unit 120 includes a liquid crystal display (LCD) and includes a backlight whose brightness can be controlled. Further, the user can switch various settings of the imaging apparatus 100 from a menu displayed on the image display unit 120. For example, the brightness of the backlight of the image display unit 120 and the on / off state of the in-camera can be set and changed.

  In step S301, the imaging apparatus 100 is activated in response to pressing of the power switch 138. In step S302, the system control unit 122 determines whether the in-camera is set to on or off. When the in-camera is set to OFF (S302), the system control unit 122 executes the backlight brightness setting of the image display unit 120 according to the camera setting in step S308, and starts image display of the image display unit 120 in step S309. To do.

  When the in-camera is set to ON (S302), the system control unit 122 performs face detection from the captured image of the in-camera in step S303, and determines whether a face is detected. If no face is detected (S303), the system control unit 122 turns off the backlight of the image display unit 120 in step S304, and starts image display of the image display unit 120 in step S309. Of course, in this case, the image display itself may not be performed.

  If a face is detected in step S303, the system control unit 122 performs sunglasses detection in step S305. Detailed processing for detecting sunglasses will be described later. In step S306, the system control unit 122 determines whether sunglasses are detected. When sunglasses are detected (S306), the system control unit 122 sets the brightness of the backlight of the image display unit 120 to the maximum in step S307, and starts image display of the image display unit 120 in step S309. When sunglasses are not detected (S306), the system control unit 122 sets the brightness of the backlight of the image display unit 120 according to the camera setting in step S308, and starts image display of the image display unit 120 in step S309.

  FIG. 3B shows a detailed flowchart of the sunglasses detection process (S305). When the sunglasses detection is started (S310), the system control unit 122 specifies the position of the detected face eye in step S311. Next, in step S312, the system control unit 122 extracts a sunglasses area when it is assumed that the user is wearing from the size and orientation of the face.

  In step S <b> 313, the system control unit 122 integrates the signal value of the pixel determined as the sunglasses area, and compares the integrated average value with the threshold value B. When the integrated average value is larger than the threshold value B (S313), the system control unit 122 determines that sunglasses are not worn (S315). When the integrated average value is equal to or less than the threshold value B (S313), the system control unit 122 determines that there is sunglasses (S314).

  FIG. 4 is a side view showing the positional relationship between the imaging device 100 and the user. FIG. 5 is a schematic diagram of an image obtained by photographing the user with the in-camera. With reference to FIG.4 and FIG.5, the sunglasses detection operation | movement of a present Example is demonstrated.

  When the in-camera is set to ON, the in-camera is always photographing the user. Fig.5 (a) shows the example of an image in case a user is wearing sunglasses. FIG. 5B shows an example image when the user is not wearing sunglasses. FIG. 5C shows the sunglasses determination result for the image example shown in FIG. 5A, and FIG. 5D shows the sunglasses determination result for the image example shown in FIG. 5B.

  As described above, on the captured images shown in FIGS. 5A and 5B, the eye position is specified from the size and orientation of the face, and the sunglasses area in the case of wearing sunglasses is extracted. That is, the sunglasses area is a portion around the eyes that is covered with sunglasses when wearing sunglasses. The shaded areas in FIGS. 5C and 5D indicate the extracted sunglasses area. In this embodiment, the sunglasses area is extracted in units of areas obtained by dividing the screen into 13 horizontal blocks and 14 vertical blocks. It goes without saying that the detection accuracy increases as the number of horizontal or vertical divisions increases.

  When the integrated average value of the luminance of the sunglasses area in the example shown in FIG. 5C is Bs and the integrated average value of the luminance of the sunglasses area in the example shown in FIG. 5D is Bd, Bs <B <Bd Become. Accordingly, the system control unit 122 determines that the example illustrated in FIG. 5C is wearing sunglasses, and determines that the example illustrated in FIG. 5D is not wearing sunglasses.

  In this embodiment, when it is determined that the user wears sunglasses, the backlight luminance of the image display unit 120 is increased. However, a dialog that warns the user of wearing sunglasses may be displayed or voiced.

  If it is determined that the sunglasses are worn, it may be determined whether the sunglasses are cold-colored or warm-colored, and the color of the display image of the image display unit 120 may be changed so as to look close to a natural color according to the determination result. .

  The ambient light amount of the imaging apparatus 100 can be estimated by mode setting (such as night view mode or beach mode). Therefore, the sunglasses wearing determination threshold value B may be changed according to the set camera mode.

  The control by the system control unit 122 may be performed by one piece of hardware, or the entire apparatus may be controlled by a plurality of pieces of hardware sharing the processing.

  Although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are also included in the present invention. . Furthermore, the above-described example is merely an illustrative embodiment of the present invention.

  Further, although the embodiment in which the present invention is applied to an imaging apparatus such as a digital camera has been described, this is an example. The present invention is generally an apparatus including an imaging unit that captures a user and a display unit that is viewed by the user, and it is useful to control the display mode of the display unit depending on whether the user wears sunglasses. Applicable to the device. For example, the present invention can be applied to a personal computer, a PDA, a mobile phone terminal, a portable image viewer, a printer device having a display, a digital photo frame, a music player, a game machine, an electronic book reader, a video phone device, and the like.

  (Other Embodiments) The present invention can also be realized as a software program for realizing the functions of the above-described embodiments. The software (program) is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads and executes the program code. Such a program and a storage medium storing the program can also be protected by the present invention.

Claims (7)

Image display means for displaying an image;
Imaging means for photographing a user browsing the display of the image display means;
Determining means for determining whether the user is wearing sunglasses based on a captured image of the imaging means;
And a control unit configured to control the display unit to increase brightness when the determination unit determines that the user is wearing the sunglasses. The determination means includes
Face detection means for detecting the face of the user from the captured image of the imaging means;
A sunglasses wearing judging means for judging a sunglasses area from a position of the face of the face detected by the face detecting means and a face size, and judging whether or not sunglasses are worn by comparing a luminance of the sunglasses area with a threshold value; The display device according to claim 1, further comprising:   The display device according to claim 2, wherein the sunglasses wearing determination unit changes the threshold value according to ambient brightness.   4. The apparatus according to claim 1, further comprising means for notifying the user of wearing of sunglasses when the judging means judges that the user is wearing the sunglasses. The display device described in 1. In a display device comprising an image display means for displaying an image and an imaging means for photographing a user browsing the display of the image display means, a method for controlling the brightness of the image display means,
A determination step of determining whether the user is wearing sunglasses based on a captured image of the imaging means;
And a control step of controlling to increase the luminance of the display means when it is determined in the determination step that the user is wearing the sunglasses. In a display device comprising an image display means for displaying an image, an imaging means for photographing a user who views the display of the image display means, and a control means for controlling the image display means, the brightness of the image display means A control program for controlling the above-mentioned control means,
Determining means for determining whether the user is wearing sunglasses based on a captured image of the imaging means;
A control program for a display device, characterized in that when the determination means determines that the user is wearing the sunglasses, the control means functions as a control means for controlling the brightness of the display means to increase. In a display device comprising an image display means for displaying an image, an imaging means for photographing a user who views the display of the image display means, and a control means for controlling the image display means, the brightness of the image display means Recording medium storing a control program for controlling the control, the control program, the control means,
Determining means for determining whether the user is wearing sunglasses based on a captured image of the imaging means;
A recording medium characterized in that the determination means functions as a control means for controlling to increase the luminance of the display means when it is determined that the user wears the sunglasses.

Images