JP5225014B2 - Image processing apparatus and display control method - Google Patents

Description

The present invention relates to an image processing apparatus and a display control method for selecting a character in the balloon that is displayed superimposed on the image data.

  With the widespread use of digital cameras, compact photo printers as image processing apparatuses have become widespread in order to meet the demand for easily printing photographic images taken with digital cameras.

  In recent years, compact photo printers are mainly equipped with a liquid crystal display screen, and print a photographic image of a print candidate after previewing it on a display screen such as a liquid crystal display in advance. The display screens to be installed are also increasing in size, resolution, and image quality, and there are some screens that can be enlarged to display details.

  For this reason, a graphic user interface (GUI) is installed, and traditionally only prints photographic images as they are, but it has an additional function to process and print original images by adding frames and stamps. It can be installed.

  Typical examples of image objects include frames, stamps, and speech bubbles. The frame is intended to function as a decorative frame for a photo that is equivalent to that used in a photo stand or album, and the stamp is attached to the photo to enhance the decorative effect. A speech bubble refers to a space that represents lines (character strings) used in comics or the like, and is placed near the face of the subject person.

  “Fukidashi” in the original sense is a space that is conventionally provided in a picture to express the characters' lines in comics. Create a cloud-shaped space instead of an ellipse, and use a continuous small ellipse instead of a triangle (screen 2002 in FIG. 2). Expresses what you are thinking about. Depending on the scene, it may indicate that you are speaking quietly.

  If you make the characters in the speech bubble an eerie typeface (such as an old typeface), it will be expressed that you are scolding something scary, or if you use a fuzzy character, it will be expressed that the throat is in a rattling state. There are various shapes.

  Normally, the operation member of a compact photo printer has a limited number of buttons due to size and cost restrictions, and does not support a full-fledged character input function. Therefore, it is common that the speech bubble space is printed with a solid white color such as white, and the user enters a line (character string) using an oil-based pen or the like.

  When compositing such a speech bubble with an image, it may detect a face in the image data and automatically place a speech bubble near the face area, or control the face area so that it is not covered with a speech bubble. There is also a system that assists the user in setting the balloon stiffness (Patent Document 1).

  In recent years, techniques for detecting facial expressions, particularly human facial expressions, are being developed. In general, when a person is photographed with a camera, it is desired that the facial expression of the person who is the subject is photographed with good facial expression timing such as a smile. Therefore, the application of facial expression detection technology to digital cameras, such as automatically releasing the shutter when a smile occurs, is being studied.

In the existing facial expression detection technique disclosed in Patent Document 2, a change in the facial expression detected can be detected quantitatively.
JP 2007-310487 A JP 2005-266984 A

  Since the speech bubble function of the prior art uses a GUI having a small display screen and a poor configuration of the button center, it is difficult to specify fine parameters such as position, shape, color, etc., and the usability is not good. Also, because the character input function was not supported, it was not possible to input a character string in the speech bubble.

  An object of the present invention is to provide an image processing apparatus capable of improving the operability when the speech bubble function is used, and a control method therefor.

  In order to achieve the above object, an image processing apparatus according to claim 1, an analysis unit for analyzing a facial expression included in image data, and a display for synthesizing a speech bubble to the image data for display on a display unit. A control unit; a selection unit that selects a character string to be displayed in the speech bubble by switching the character string to be displayed in the speech bubble; and a selection unit that selects the character string by the selection unit. And determining means for determining a character string to be displayed first according to the facial expression analyzed by the analyzing means.

  According to the image processing apparatus of the present invention, it is possible to improve the operability when the speech bubble function is used.

  Hereinafter, the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block configuration diagram of an image processing apparatus according to an embodiment of the present invention.

  1A, an image processing apparatus (printing apparatus) 1 includes a PC 2 or a memory card (not shown) inserted in a memory card slot 3, or an imaging apparatus 4 such as a digital camera / digital video camera, and an interface 101. Connected through.

  Hereinafter, the configuration of the image processing apparatus 1 will be described together with the operation.

  Image data (not shown) input from the PC 2 or a memory card (not shown) inserted into the memory card slot 3 or the imaging device 4 is stored in the reception buffer 102 via the interface 101.

  The received JPEG image data is subjected to preprocessing such as resizing and rotation in the first signal processing unit 103 and then stored in the RAM 114 of (b) constituting the control unit 104.

  When the CPU 113 operates based on a program (not shown) stored in the ROM 115, the following is possible.

  That is, the image data stored in the RAM 114 is displayed on the first display unit 106 for previewing, specifying the number of copies using the input unit 105, and N-UP printing (N images are arranged on one sheet). Printing) such as printing) and trimming.

  Note that the face detection processing referred to in the present invention is performed by the first signal processing unit 103, and the facial expression detection processing is performed by the second signal processing unit 109. The detection result can be shared by each signal processing unit by recording it in the EXIF tag of the image file. The first signal processing unit 103 can execute face detection processing, and the second signal processing unit 109 can execute facial expression detection processing.

  When the user uses the GUI to operate the operation buttons of the input unit 105 to instruct image data printing, the second signal processing unit 109 converts the image data into a data format that the printer engine unit 200 can interpret. And stored in the frame memory 107. Thereafter, the printer engine unit 200 prints an image on a sheet based on the frame memory 1072 and the troubled data. When an error occurs in the printer engine unit 200, the second display unit 108 is used to flash the LED, etc., and alert the user.

  FIG. 2 shows an example of a screen when the speech bubble and the character string initially displayed on the speech bubble and character string selection screen are automatically selected using the expression detection function of the second signal processing unit 109 in FIG. FIG.

  The screen 2001 in (a) is when a smile is detected. When a smile is detected, the input character string “Thank you” is selected first when the screen shifts to the speech bubble and character string selection screen. It is displayed in the state that has been.

  The screen 2002 of (b) is displayed when a troubled face is detected, and the input character string is displayed in a state where “I'm sorry” indicating apology is selected.

  FIG. 3 is a diagram illustrating an example of a screen when the speech bubble and the character string are manually selected from the input unit 105 of the image processing apparatus 1 in FIG. 1 on the speech bubble and character string selection screen. As shown in FIG. 2, on the speech bubble and character string selection screen, options corresponding to the facial expression are first displayed. However, other options can be selected by the user's operation thereafter.

  The user displays a preview of the image data on the first display unit 106. Subsequently, the user operates the button (c) of the input unit 105 to perform a GUI-based menu operation.

  A screen 3001 in (a) shows a state where a menu for selecting the shape of the balloon is displayed and an ellipse is selected.

  The screen 3002 in (b) displays a menu for selecting a character string and selects “Thank you”. The shape and color of the speech bubble and the character string arranged in the speech bubble are both stored in the ROM 115.

  FIG. 4 is a diagram showing an example of a screen in the case where the shape of the speech bubble that is initially displayed on the speech bubble selection screen is automatically determined using the facial expression detection processing function of the second signal processing unit 109 in FIG. It is.

  In FIG. 4, a screen 4001 is a case of a natural face. In this case, a speech bubble in which a triangle is combined with a rectangular space is first displayed on the speech bubble selection screen. A screen 4002 is a case where a smile is detected, and a speech bubble in which a triangle is combined with an ellipse is first displayed.

  A screen 4003 is a case where an angry face is detected, and an explosion shape representing anger is displayed first. A screen 4004 is a case where a troubled face is detected, and a balloon of a combination of a small ellipse continuous with a cloud-shaped space showing a haze is first displayed.

  FIG. 5 is an example of a speech bubble selection screen. When the speech bubble selection screen is entered, the speech bubble color displayed first is automatically selected using the facial expression detection processing function of the second signal processing unit 109 in FIG. Selected.

  In FIG. 5, a screen 5001 is a case where the facial expression detection result is a natural face, and the speech bubble is in a state where white is initially selected. A screen 5002 is a case where a smile is detected, and a light color is initially selected as a speech bubble color.

  A screen 5003 is a case where an angry face is detected, and a warm-colored dark color is selected as a speech bubble color. A screen 5004 is a case where a troubled face is detected, and a cold color is selected as a speech bubble color.

  FIG. 6 is a diagram showing an example of an input character string screen selected according to the result of facial expression detection processing by the second signal processing unit 109 in FIG.

  In FIG. 6, a screen 6001 is a case where the result of facial expression detection is a natural face, and “Otsukama-sama” is selected and displayed as the initial default value on the character string selection screen. A screen 6002 is a case of smiling, and a word of thanks “thank you” is displayed / selected first.

  A screen 6003 is an angry face, and is displayed in a state where the anger word “Kara!” Is selected. A screen 6004 is a case of an embarrassed face, and the confusion word “sorry” is selected and displayed.

  FIG. 7 is an example of a character string selection screen, and a character font selected as the initial default value on the character string selection screen is determined according to the result of the facial expression detection process.

  In FIG. 7, a screen 7001 is a case where the result of facial expression detection is a natural face. On the character string selection screen, Mincho is the initial setting value. A screen 7002 is a case of smiling, and a pop body that naturally brings out a smile is selected.

  A screen 7003 is a case of an angry face, and a square gothic body with a corner is selected. A screen 7004 is a case of a troubled face, and expresses confusion and sadness by using an italic body that tilts the head.

  As described above, in the present embodiment, the facial expression included in the image to which the character string or speech bubble is combined is detected on the character string selection screen / speech selection screen, and is displayed first on the character string / speech selection screen. The selected paper is determined according to the result of facial expression detection. Alternatively, the option selected first in the character string / speech selection image is determined according to the result of facial expression detection.

  In other words, the selected paper that is initially displayed or selected on the character string selection screen / speech selection screen changes according to the facial expression detection result.

  In the above-described embodiment, the balloon shape, balloon color, character string, and character string format have been described as examples. However, the present invention may be applied to other selection items. Further, since it is the option that is initially displayed or selected as the default that is determined as a result of facial expression detection, the user can change the option on the selection screen.

  Depending on the result of facial expression detection, the option that is initially displayed / selected is automatically determined, so you can easily set speech bubbles and text strings, and when words are detected by facial expression detection or other options When it is desired to select the item, it can be changed later, so that usability is improved.

  FIG. 8 is a flowchart of the process of the balloon combined printing according to this embodiment. These processes are executed by the CPU 113 reading a program from the ROM 115 and controlling each unit of the image processing apparatus 1 based on the read program.

  When the balloon printing process is selected by the user, these processes are executed. First, an image selection screen is displayed (display control) on the display unit 106. The user selects an image to be printed by operating the input unit 105 from the image data stored in the memory card or the storage medium 111 attached to the memory slot 3 displayed on the image selection screen (S801). ).

  When an image is selected, face detection of the selected image and facial expression detection for the detected image are performed (S802). If face detection / expression detection result information is recorded as additional information in the image data, the face detection / expression detection can be performed by analyzing the additional information without necessarily performing face detection / expression detection in the image processing device. The result of facial expression detection may be acquired.

  In S803, according to the facial expression detection result obtained in S802, among the character string data that can be combined stored in the image processing apparatus, character string candidates to be combined with the balloons that can be selected in S806 and S807 are candidate characters. Determine as a column.

  That is, the selectable character string changes according to the detected facial expression. For example, if a smile is detected, “Thank you”, “Yeah!”, “HAPPY”, and “Lucky” can be selected. “Scared” can be selected.

  If no character or face is detected or facial expression cannot be detected, all character strings stored in the image processing apparatus are set as candidate character strings. In addition, when there are a plurality of faces in one image and a plurality of types of facial expressions are included, character strings corresponding to all the facial expressions may be used as candidate character strings.

  Next, in step S804, the shape and color of the speech bubble to be displayed as the initial setting on the speech bubble selection screen, and the font and character string of the character string are determined. As the character string, one of the candidate character strings determined in S803 is determined as an initial setting. The initial settings for the shape and color of the speech bubble and the typeface of the character string are determined as described above.

  Then, the balloon shape and color according to the initial settings determined in step S804, the typeface of the character string, the balloon and the character string are generated with the character string, and combined with the image to be printed, and the balloon / character string selection screen is displayed. Is displayed.

  In the speech bubble / character string selection screen, the shape, color, typeface of character string, and character string can be changed, and the character string can be selected from the candidate character strings determined in S803. If a character string switching operation is performed (YES in S806), the character string is synthesized in place of another (next) candidate character string and displayed on the balloon / character string selection screen (S807).

  If the operation is for changing the shape or color of the speech bubble instead of the character string selection operation (YES in S808), the speech bubble shape or color is changed according to the operation, and the speech bubble / character string selection screen is displayed. It is displayed (S809).

  Change the typeface of the character string as well as the shape and color of the speech bubble. When the selection of the balloon / character string is completed and the user performs a determination operation (YES in S810), the printing process is started. In the printing process, the balloon and character string selected on the balloon / character string selection screen are combined with the image to be printed selected in step S <b> 801, print data is created, and the print data is sent to the printer engine unit 200. The printer engine unit 200 prints an image according to the print data.

  In this way, it is possible to print an image in which a speech bubble and a character string are combined.

  As described above, the image processing apparatus of the present embodiment stores speech balloons and character string data, and can synthesize speech balloons and character strings into images and print them. Speech balloons and character string data are stored, and speech balloons and character strings can be combined with images and printed.

  Then, when selecting a speech bubble / character string to be combined with the image, the facial expression information included in the image to be printed is acquired, and the user can select a character string option according to the acquired facial expression information. Is automatically narrowed down.

  Therefore, it is possible to easily select a character string that matches the expression. In particular, when there are many selected sheets (types of character strings), selecting from many options is a difficult task, and thus the effect of implementing the present invention is increased.

  In the present embodiment, only candidate character strings can be selected according to the facial expression detection result, but the candidate character strings may be switched preferentially on the character string selection screen. In this case, when the character string changing operation is performed, the candidate character strings are first displayed sequentially, and then other candidate character strings are displayed.

  Further, not only the character string but also the shape and color of the speech bubble may be automatically narrowed down on the image processing apparatus side according to the expression.

  In addition, since it may be possible to select a character string other than the candidate character string, it is possible to cancel the narrowing of the character string, speech bubble shape, and color on the speech bubble / character string selection screen by performing a specific operation. good.

  In addition, by preparing a menu in which character strings are divided into categories for each facial expression in advance, it is possible to move to that menu even after a candidate character string is automatically selected, so that character strings corresponding to other facial expressions May be configured to be selectable.

  Although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are not limited thereto. Included in the invention.

  Also, a software program that realizes the functions of the above-described embodiments is supplied from a recording medium directly to a system or apparatus having a computer that can execute the program using wired / wireless communication, and the program is executed. Cases are also included in the present invention.

  Accordingly, the program code itself supplied and installed in the computer in order to implement the functional processing of the present invention by the computer also realizes the present invention. That is, the computer program itself for realizing the functional processing of the present invention is also included in the present invention.

  In this case, the program may be in any form as long as it has a program function, such as an object code, a program executed by an interpreter, or script data supplied to the OS.

  As a recording medium for supplying the program, for example, a magnetic recording medium such as a flexible disk, a hard disk, a magnetic tape, MO, CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-R, DVD- RW etc. can be considered. Further, an optical / magneto-optical storage medium or a nonvolatile semiconductor memory may be used.

It is a block block diagram of the image processing apparatus which concerns on embodiment of this invention. It is a figure which shows the example of a screen at the time of selecting a speech balloon and a character string automatically using the facial expression detection function which the 2nd signal processing part 109 in FIG. 1 has. It is a figure which shows the example of a screen at the time of selecting manually a speech balloon and a character string from the input part 105 of the image processing apparatus 1 of FIG. It is a figure which shows the example of a screen at the time of selecting the shape of a speech bubble automatically using the facial expression detection process information which the 2nd signal processing part 109 in FIG. 1 has. It is a figure which shows the example of a screen at the time of selecting the speech bubble automatically using the facial expression detection process information which the 2nd signal processing part 109 in FIG. 1 has. It is a figure which shows the example of the screen of the input character string selected according to the result of the facial expression detection process by the 2nd signal processing part 109 in FIG. It is a figure which shows the example of the screen of the character typeface selected according to the result of the facial expression detection process by the 2nd signal processing part 109 in FIG. It is a flow of a balloon printing process of the image processing apparatus.

Explanation of symbols

1 Image processing device 2 PC
3 Memory Slot 4 Imaging Device 101 Interface 103 First Signal Processing Unit 104 Control Unit 105 Input Unit 106 First Display Unit 107 Frame Memory 109 Second Signal Processing Unit 113 CPU
114 RAM
115 ROM
200 Printer engine

Claims (5)

Analysis means for analyzing facial expressions included in the image data;
Display control means for combining the image data with a speech bubble and displaying it on a display means;
Selection means for selecting a character string to be displayed in the speech bubble from among a plurality of character strings by switching a character string to be displayed in the speech bubble;
An image processing apparatus comprising: a determination unit that determines a character string to be displayed first in the speech bubble according to the facial expression analyzed by the analysis unit when a character string is selected by the selection unit.   The image processing apparatus according to claim 1, wherein the selection unit can select a character string corresponding to the facial expression analyzed by the analysis unit. A balloon selection screen display means for displaying a balloon selection screen for selecting a balloon color or shape to be combined with the image data from a plurality of candidates on the display means;
The speech bubble selection screen display means, when displaying the speech bubble selection screen, displays a speech bubble having a shape or color corresponding to the facial expression analyzed by the analysis means in a preselected state. The image processing apparatus according to 1 or 2.   4. The image processing apparatus according to claim 1, further comprising: a printing unit configured to synthesize the character string selected by the speech bubble and the selection unit with the image data and print it. 5. An analysis process for analyzing facial expressions included in the image data;
A display control step for displaying a speech bubble over the image data on a display means;
In accordance with an operation for selecting a character string from a plurality of character strings, a character string selection step of switching a character string to be displayed in the speech bubble,
A display control method comprising: a determination step of determining a character string to be displayed first in the speech bubble in the character string selection step according to the facial expression analyzed in the analysis step .

Images