JP5071404B2 - Image processing method, image processing apparatus, and image processing program - Google Patents

Description

  The present invention relates to an image processing method, an image processing apparatus, and an image processing program for evaluating a smile state based on a measurement result of a change amount of a characteristic part of a face image detected from image data.

  In recent years, for the purpose of smoothly communicating with others, there is an increasing demand for objectively evaluating the richness of facial expressions in the state of emotions.

  Therefore, in the following Patent Document 1, the change in the facial feature of the user extracted by the camera is measured, and the facial expression related to the richness of the facial expression is quantified based on the measurement result, thereby expressing the user's facial expression. What has been made possible to evaluate objectively is disclosed.

JP 2008-264367 A

  By the way, in private contact with friends, acquaintances, etc., or in the customer service business of the customer service industry, it is possible to improve the impression given to others and customers by the smile of themselves (employees). And deepen customer relationships.

  Therefore, by using the technology disclosed in Patent Document 1, the richness of one's smile, that is, the degree of smile (called smile level) is objectively evaluated, and the smile level is further increased based on the evaluation result. It is conceivable to perform training for improvement.

  However, what is disclosed in Patent Document 1 merely captures a change in a user's instantaneous facial expression and evaluates based on the instantaneous change.

  Among human emotions, especially smiles related to “joy” and “easy” are recognized (evaluated) as smiles by other people for the first time when they are not instantaneous but persistent. It is thought. Therefore, the technique disclosed in Patent Document 1 cannot objectively evaluate the persistence of a smile, and is insufficient to obtain a more accurate evaluation of the smile status.

  The present invention provides an image processing method, an image processing apparatus, and an image processing program capable of objectively evaluating the sustainability of a user's smile and, as a result, more accurately evaluating the state of the user's smile. The purpose is to provide.

  The present invention includes a step of detecting a face image from image data by a face image detecting unit, a step of measuring a change amount of a characteristic part of the face image detected by the face image detecting unit by a measuring unit, A step of evaluating a smile state by an evaluation unit based on a measurement result, the step of setting a measurement time by the measurement unit by a measurement time setting unit, and the measurement time set by the measurement time setting unit, A step of measuring the amount of change a plurality of times by a measuring unit, and a step of digitizing a smile recognition level as a smile level by a smile level digitizing unit of the evaluation unit based on the change amount measured by the measuring unit And a step of integrating the digitized smile degree by the integration unit of the evaluation means within the measurement time, and an integration value integrated by the integration unit. And a step of generating smile evaluation information related to the evaluation of the smile state by the smile evaluation information generation unit of the evaluation unit, and outputting the smile evaluation information generated by the smile evaluation information generation unit by the output unit An image processing method including steps, an image processing apparatus that executes the method, and an image processing program.

  The output means may be composed of a liquid crystal display, a CRT display, or the like, a display device for displaying smile evaluation information represented by characters, graphics, etc. on the screen, or a voice output device such as a speaker for outputting the smile evaluation information by voice guidance. it can.

  According to the present invention, it is possible to objectively evaluate not only the instantaneous smile but also the sustainability of the smile within the measurement time, and as a result, the user's smile state can be more accurately evaluated.

As an aspect of the present invention, the smile evaluation information generation unit includes an input unit that inputs a target value of the smile level, and the smile evaluation information generation unit calculates an average value of the smile level based on the integrated value integrated by the integration unit. Meanwhile, as the smile evaluation information, average value information related to the average value is generated, and the output means outputs the target value input by the input means and the average value information generated by the smile evaluation information generation unit. Can be output together.
Thereby, the difference between the degree of smile persistence within the measurement time and the target can be easily recognized. Then, as the user continues training for making a smile in the future, the difference can be used as a guideline for setting a training schedule, a training menu, and the like.

As an aspect of the present invention, there is provided an average value storage means for storing a plurality of the average values calculated over the measurement time for one user, and the smile evaluation information generating unit calculates each average value for the plurality of cases as the average value. While reading from the average value storage means, personal history information in which the respective average values are arranged in time series is generated as the smile evaluation information, and the output means outputs the personal history information.
Thereby, the user can watch the result of training, such as how much the sustainability of his / her smile is improved, in the long term. Therefore, it is possible to more easily set a schedule for a future training, a training menu, and the like.

As an aspect of the present invention, there is provided integrated value storage means for storing the integrated value of the smile degree integrated by the integrating unit for a plurality of users, and the smile evaluation information generating unit stores the user stored in the integrated value storage means The ranking information related to the ranking processing result is generated as the smile evaluation information while performing ranking processing based on the magnitude of the integration value, and the output means generates the ranking It can be set as the structure which outputs the information relevant to.
Thereby, the sustainability of own smile and the sustainability of smiles of other users can be relatively evaluated. For this reason, competition consciousness can be raised among several users, As a result, the sustainability improvement of a smile can be promoted more.

As an aspect of the present invention, the integration unit is configured to simultaneously integrate smile levels for each user in a state in which the face image detection unit detects a plurality of user face images, and the smile evaluation information generation unit includes: Calculating the sum of the integrated values of the plurality of users, calculating the ratio of the integrated value of each user to the sum of the integrated values, and generating ratio information related to the ratio as the smile evaluation information; The output means may be configured to output the ratio information.
Thereby, it is possible to enjoy making a smile as if it were a game, and the user can naturally wear a continuous smile.

As an aspect of the present invention, there is provided display control means for executing control for extracting a region including a face image from the image data and displaying the face image on a display means, and the display control means includes A face image extraction unit that selects and extracts a face image display area including the detected face image from the image data is provided, and based on the image data and the position of the face image display area in all image data A background image extraction unit that searches for the widest area from the areas excluding the face image display area, selects the area as a background image display area based on the search result, and extracts the area; The display control means includes, for the display means, a face image display area including the face image extracted by the face image extraction unit and the background image display area extracted by the background image extraction unit. It can be configured to perform control to display the column.
Thereby, a user other than the user who has operated the device can easily specify the position for detecting his / her face image based on the background image display area displayed in parallel with the face image. Can do.

As an aspect of the present invention, the information processing apparatus includes an advice information storage unit that stores advice information set in advance according to the smile level, and the output unit reads the advice information read from the advice information storage unit according to the smile level, It can be set as the structure output within the said measurement time.
Thereby, accurate advice can be sent to the user under measurement in real time, and as a result, the sustainability of smile can be further improved.

As an aspect of the present invention, search means for searching for the highest smile degree image data when the highest smile degree is exhibited within the measurement time, and the highest smile degree image data is stored based on a search result of the search means. An image data storage means can be provided.
Thereby, when the user who performed the measurement or the other user performs training for making a smile, the face image data showing the maximum smile can be used as a reference material.

  According to the present invention, it is possible to objectively evaluate the sustainability of a user's smile, and as a result, the user's smile status can be more accurately evaluated.

The figure which shows the use condition of the image processing apparatus which concerns on embodiment of this invention. The block diagram which shows an image processing apparatus. The figure for demonstrating the user's face shape specific method by a control apparatus. The figure for demonstrating the change of the feature part of the face in the state of a smile. The figure which shows a smile degree calculation map. The figure which shows an initial screen. (A) The figure which shows a login screen in training mode, (b) The figure which shows a training screen. The figure which shows a training result display screen. The figure which shows the display screen of a user management tab among setting screens. The figure which shows the display screen of a personal setting tab among setting screens. The figure which shows the display screen of a general setting tab among setting screens. The figure for demonstrating the display process of an advice message. The figure which shows the ranking measurement screen in ranking mode. The figure for demonstrating the integrated value calculation process in ranking mode. The figure which shows a ranking result display screen. (A) The figure which shows the picked-up image of a camera, (b) The figure which shows the battle | competition screen in battle | competition mode. The figure which shows the battle | competition screen in battle | competition mode, Comprising: The figure which shows the state in battle | competition. The figure for demonstrating the ratio calculation process in battle | competition mode. The flowchart which shows the operation | movement in the training mode of an image processing apparatus. The flowchart which shows operation | movement of a smile degree calculation subroutine. 6 is a flowchart showing an operation in a ranking mode of the image processing apparatus. The flowchart which shows the operation | movement in battle | competition mode of an image processing apparatus. (A) The figure which shows the appropriate inspection screen in aptitude inspection mode, (b) The figure which shows an evaluation rank table. The figure which shows the appropriate test result display screen in aptitude test mode.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a diagram illustrating a usage state of an image processing apparatus 1 according to an embodiment of the present invention. As shown in FIG. 1, the image processing apparatus 1 mainly includes a computer 1 a that executes various calculations for evaluating a smile state, a display device 10 as a display device, and a keyboard 21 that constitutes an input device 20. And a personal computer 1b (hereinafter abbreviated as a computer 1b) provided with a mouse 22, both computers 1a and 1b being connected by a LAN cable 2.

  In FIG. 1, a camera 3 for photographing the user U is connected to a computer 1 a via a dedicated cable 4. With such connection, the image data of the user U taken by the camera 3 is transmitted to the computer 1b via the computer 1a and displayed on the display device 10. In FIG. 1, a notebook personal computer is used as the computer 1b. However, the present invention is not limited to this as long as it includes a display device and an input device. In the present embodiment, the image processing apparatus 1 is configured by the two computers 1a and 1b. However, the computer 1a may be integrated in the computer 1b.

  FIG. 2 is a block diagram showing the image processing apparatus 1. As shown in FIG. 2, the image processing apparatus 1 is configured by connecting a display device 10, an input device 20, a storage device 40, and a timer device 50 to a control device 30.

  The control device 30 is constituted by a CPU, for example, and executes a control operation based on a program stored in the storage device 40. In FIG. 2, for convenience of explanation, a control device and a storage device mounted on the two computers 1 a and 1 b are shown integrally.

  The display device 10 performs arithmetic processing on the face image of the user U taken by the camera 3 (see FIG. 1) by the control device 30, and displays the image data generated as a result. The input device 20 is, for example, for inputting a command or data related to the operation by an appropriate input operation of the user U.

  The storage device 40 includes, for example, a ROM, a RAM, a hard disk, and the like. The storage device 40 mainly includes a program for the control device 30 to execute a control operation, as well as three-dimensional face model data and a smile described later. A degree calculation map, measurement time, user registration information, smile degree target value, advice message information, number of outputs, and the like are stored.

  The timer device 50 is for setting the measurement time stored in the storage device 40.

  FIG. 3 is a diagram for explaining a method of specifying the face shape of the user U by the control device 30. In this embodiment, in order to specify the face shape of the user U, data of the three-dimensional face model M as shown in FIG. 3A is stored in the storage device 40 shown in FIG. This three-dimensional face model M is created based on samples of face images for thousands of people, and connects the surface of the face and facial features such as eyes, nose and mouth with lines. ing.

  The control device 30 according to the present embodiment first detects the face image of the user U from the image taken by the camera 3 and extracts the image data. Then, as shown in FIG. 3B, the above-described three-dimensional face model M is adapted to the two-dimensional face image data (referred to as fitting). Thereby, the unevenness | corrugation of the user's U face surface and the position of a characteristic part are specified, and the user's U face shape is specified.

  Furthermore, in this embodiment, it is possible to quantify the smile state, that is, the recognition level of smiles by others, as the smile level from the face shape of the user U specified by the above-described method. For example, when the user U is in a smiling state, as shown in FIG. 4, the number of wrinkles (portion P1 surrounded by a circle in the drawing) increases, and the opening width of the mouth (arrow portion P2 in the drawing) increases. Changes such as an increase in the corner of the mouth (part P3 surrounded by a circle in the figure), a decrease in the corner of the eye (part P4 surrounded by a circle in the figure), and a decrease in the opening width of the eye (indicated by an arrow part P2 in the figure) appear. . Therefore, the control device 30 calculates the smile level based on the measurement of the amount of change in each of the characteristic parts P1 to P5.

  Specifically, for each of the characteristic parts P1 to P5, a smile degree calculation map as shown in FIGS. 5A to 5E is created in advance, showing the relationship between the amount of change and the frequency related to the smile degree. is doing. And based on the measurement result of the variation | change_quantity of characteristic site | part P1-P5, it is possible to calculate a smile degree by calculating | requiring the frequency | count corresponding to it.

  Here, the smile degree calculation map shown in FIG. 5 is created on the basis of more than 1,000 face samples. Specifically, this face sample is shown to an evaluator (monitor), and whether or not each expression is in a smiling state is evaluated. Then, it is divided into half of the face samples evaluated by most evaluators as smiling and half of the face samples evaluated by most evaluators as not smiling.

  Further, average values “x1”, “x3”, “x5”, “x7”, and “x9” (see FIG. 5) are calculated for the amount of change in the characteristic portions P1 to P5 in the face sample that is not evaluated as a smile. . And when each measured value of the said parts P1-P5 is less than the said average value "x1", "x3", "x5", "x7", "x9", it defines as frequency "0", respectively.

  On the other hand, the average values “x2”, “x4”, “x6”, “x8”, “x10” (see FIG. 5) of the variation amounts of the respective characteristic portions P1 to P5 in the face sample evaluated as smile are calculated. . And when each measured value of said site | part P1-P5 is more than said average value "x2", "x4", "x6", "x8", "x10", it defines as frequency "100", respectively.

  In the smile degree calculation map, a straight line graph is created based on such definition of the frequency, and the relationship between the amount of change and the frequency is defined for each of the characteristic parts P1 to P5. In this smile level calculation map, for example, when the number of wrinkles of the user U is (x2−x1) / 2 (lines), the frequency is set to 50.

  Here, the control device 30 obtains the frequency for each of the characteristic parts P1 to P5 based on the smile level calculation map, and further calculates the average value of the frequency, whereby the smile level of the user U is quantified. (This method of calculating the smile level by calculating the average value is called integrated reasoning.)

  This quantified smile level is, in other words, the level of recognition recognized by other people as being in a smile state. By expressing this level of recognition in the form of a smile level, the smile status can be objectively determined. It can be used as a guide when showing in

  However, in evaluating (recognizing) that someone else is smiling, the contribution ratio differs for each of the characteristic parts P1 to P5. For this reason, when the average value of the frequencies is actually obtained, the frequencies are weighted for the characteristic portions that are considered to have a high contribution rate.

  Next, a usage example of the image processing apparatus 1 according to the present embodiment will be described with reference to FIGS. First, when the image processing apparatus 1 is activated, the control apparatus 30 causes the display apparatus 10 to display an initial screen 11 as shown in FIG.

  On the initial screen 11, a training mode selection button 11a, a ranking mode selection button 11b, and a battle mode selection button 11c are displayed, and the user U presses any of the buttons 11a to 11c by operating the input device 20. The operation mode is selected.

  First, an example of use in the training mode will be described with reference to FIGS. When the training mode selection button 11a is pressed on the initial screen 11A shown in FIG. 6, the control device 30 starts a so-called training mode and causes the display device 10 to display a login screen 12A as shown in FIG. . On the login screen 12A, a password input unit 12a and a character list display unit 12b for selecting a password input character are displayed. Here, the user U1 (see FIG. 7B) can select a character by pressing the character displayed on the character list display unit 12b using the input device 20.

  At this time, the control device 30 performs verification of the input password and a pre-registered password. When the input password is correct, the control device 30 displays a training screen 12B as shown in FIG. To display. In the training screen 12B, when the face of the user U1 is within the shooting range of the camera 3 (see FIG. 1), a face image is displayed, and the control device 30 detects the position of the face of the user U1 and detects the face of the user U1. A frame portion 12c surrounding the face image is displayed.

  And the control apparatus 30 detects each characteristic site | part P1-P5 (refer FIG. 4) of the user's U1 face displayed on the training screen 12B, and based on the said method demonstrated with reference to FIGS. 3-5. Start calculating smile level. This smile start calculation starts training for making a smile.

  On the training screen 12B, the calculated smile level is indicated by a number on the display portion 12d of the frame portion 12c (“17” in the figure), and is displayed in the form of a figure depending on the size of the triangular display portion 12e. In the display unit 12e, the triangular portion expands as the smile level increases.

  In this embodiment, when calculation of the smile level is started in the training mode, the timer device 50 (see FIG. 2) is activated, and the amount of change of the characteristic parts P1 to P5 is measured during a preset measurement time. And the smile level is calculated. On the training screen 12B, a display unit 12f indicating the elapsed time from the start of measurement is displayed (7 seconds in the figure), and a preset measurement (setting) time (30 seconds in the figure) is displayed on the display unit 12g. Yes.

  In the present embodiment, the control device 30 extracts the face image data of the user U1 every predetermined time (for example, 0.5 seconds) and stores the extracted face image data in the storage device 40, while newly extracting the face image. Smile level is calculated for each data. On the training screen 12B, the smile level is updated and displayed in real time on the display units 12d and 12e.

  On the training screen 12B, an advice message corresponding to the calculated smile level is displayed on the display unit 12h. As a result, the user U1 can be notified of matters necessary for increasing the smile level in the form of a message.

  On the training screen 12B, a smileiness index that is a preset target value of the smile level is displayed on the display unit 12i (“80” in the figure). Thereby, the user U1 can know the achievement level of smile making by comparing the current smile level with the target value.

  Further, in the training mode, the control device 30 accumulates the smile degree updated every moment in this way. An average value is calculated each time based on the integrated value and the number of times image data is extracted, and the average smile level is displayed as average value information on the display unit 12j of the training screen 12B (FIG. Then “25”).

  Then, when the measurement time has elapsed, the training result display screen 12C as shown in FIG. On the training result display screen 12C, the calculation result of the final total average smile degree (“25” in the figure) is displayed on the display unit 12k.

  In addition, the control device 30 stores the maximum value of the smile level during the measurement time, that is, the maximum smile level in the storage device 40. On the training result display screen 12C, the maximum smile level stored in the storage device 40 within the set time is displayed as the MAX smile level on the maximum smile level display unit 12m ("61" in the figure). Further, on the training result display screen 12C, a maximum smile level history graph 12n is displayed at the bottom, and a maximum smile level history (described later) generated based on the past measurement results of the user U1 is displayed in a graph form. Has been.

  By the way, the image processing apparatus 1 can display a setting screen 13 for performing various settings as shown in FIGS. 9 to 11 by an appropriate operation of the user U1 or the like using the input device 20. FIG. 9 shows a state in which the user management tab 13A is displayed and selected on the setting screen 13, and various management information such as personal information required when the user U1 using the image processing apparatus 1 logs in. It is displayed.

  In the user management tab 13A, an account column 13a, a password column 13b, a personal name column 13c, a target smile level column 13d, and an authority column 13e are displayed.

  In the account column 13a, an account assigned to a user who can use the image processing apparatus 1 is shown. In the password column 13b, a password set by each user is shown. In the personal name column 13c, personal names such as real names and nicknames of the respective users are shown.

  In the target smile level column 13d, the target smile level set for each user in the training mode described above is shown. The authority column 13e indicates an authorized person who can change the management information. Here, the administrator is, for example, the owner of the image processing apparatus 1 or the manager.

  Here, when changing various management information already registered as a user of the image processing apparatus 1 or newly registering a user, a predetermined input is made using the input device 20 on the display screen of the user management tab 13A. After the operation, the inputted information is stored in the storage device 40 by pressing the setting / registration button 13f.

  FIG. 10 shows a state where the personal setting tab 13B is selected and displayed on the setting screen 13, and various types of information for each user who has performed measurement in the training mode are displayed.

  The personal setting tab 13B shows personal history information related to the measurement result when the measurement was performed in the training mode in the past. In the “training mode personal history” list, the history date / time column 13g, the maximum smile level column 13h, the average smile A degree column 13i is displayed. The personal history information is stored in the storage device 40, and the control device 30 reads out the data in the storage device 40 and displays it on the display screen of the personal setting tab 13B by an appropriate operation of the user. .

  In the present embodiment, among the measurement results obtained when the measurement was performed in the training mode in the past, the maximum smile degree and the average smile degree are arranged in a time series and graphed. The history information is displayed on the display unit 13j. The corresponding user can set the number of cases desired to be graphed retroactively from the latest measurement result (the number of times of measurement performed over the set time) in the setting unit 13k.

  In FIG. 10, the setting of the setting unit 13k is set to graph 10 cases from the latest. Based on this setting, the maximum smile degree history graph 12n on the training result display screen 12C shown in FIG. 8 displays the transition of the maximum smile degree for the past 10 cases including the latest result.

  In the present embodiment, in addition to the personal history information such as the average smile level and the maximum smile level calculated in the training mode, the face image data of the user U1 indicating the maximum smile level can be stored in the storage device 40. It has become.

  Here, in the setting unit 13m, the user himself / herself saves the face image data (photograph) at the maximum smile degree, or saves the user's personal history information in the training mode, or both. The control device 30 selects data to be stored in the storage device 40 based on the selection result.

  In the personal setting tab 13B, the setting section 13n can set a target smile level in the training mode (“smileiness” on the training screen 12B).

  Here, when a setting value is newly set or changed in the setting sections 13k, 13m, 13n of the personal setting tab 13B, the setting / registration button 13p is pressed to change the setting. Information is stored in the storage device 40.

  FIG. 11 shows a state where the overall setting tab 13C is displayed and selected on the setting screen 13, and various information related to the ranking mode and the battle mode described later are displayed in addition to the training mode described above.

  In the overall setting tab 13C, a message pattern displayed on the display unit 12h (see FIG. 7) is displayed, and a smile level column 13q and an advice message column 13r are displayed in the “smile level and advice message” list. .

  The storage device 40 stores advice message patterns in which different words are set according to the smile level, and the pattern list is shown in the “smile level and advice message” list.

  Here, as shown in FIG. 12, the advice message is updated every plurality of extraction times of face image data (in the drawing, every 6 extraction times, that is, every 3 seconds) under the control of the control device 30. It has come to be. In FIG. 12, an advice message is updated and displayed at the time of the next face image data extraction based on the degree of smile at the extraction time when hatching is performed. For example, in FIG. 12, when the number of extractions is the twelfth, the smile degree is “16”. Therefore, when the number of extractions is the thirteenth, the smile degree “16” corresponding to the smile degree “16” is displayed based on the list of FIG. An advice message is displayed, saying, “Please smile as much as you can!”.

  In addition, you may make it select an advice message from the said list based not only on the present smile degree but on the average smile degree from the measurement start. Further, an advice message may be selected from the list based on the average smile level of 6 extractions (3 seconds).

  In addition, although the update of the advice message in the control device 30 is executed at each extraction time, the number of extraction times is predetermined (for example, 6 extraction times) in consideration of the time required for the user U1 to grasp the message content. Control may be performed to maintain the display state until reaching.

  In the overall setting tab 13C shown in FIG. 11, the setting unit 13s sets the measurement time in the training mode, the setting unit 13t sets the ranking time (measurement time) in the ranking mode, and the setting unit 13u sets the ranking maintaining smile degree. In the setting unit 13v (described later), the battle time (measurement time) in the battle mode can be set.

  In addition, the setting unit 13w designates whether to store face image data (photographs) at the maximum smile level, or save personal history information of users in the training mode, or both for all users. It can be done. And the number of cases to be stored retroactively from the latest measurement result can be set by the setting unit 13x. The control device 30 selects data to be stored in the storage device 40 based on the selection result in the setting unit 13x.

  Here, when a setting value new setting operation or a changing operation is performed in the setting sections 13s to 13x of the overall setting tab 13C, by pressing the setting / registration button 13y, the set and changed information is displayed. It is stored in the storage device 40.

  Next, a usage example in the ranking mode will be described with reference to FIGS. When the ranking mode selection button 11b is pressed on the initial screen 11A shown in FIG. 6, the control device 30 starts a so-called ranking mode, and displays the same screen as the login screen 12A of FIG. Let Then, the user U2 (see FIG. 13) who uses the ranking mode is requested to input a password.

  Here, when the input password is correct, the control device 30 causes the display device 10 to display a ranking measurement screen 14A as shown in FIG. In the ranking measurement screen 14A, as in the training mode, the smile level is calculated while displaying the frame portion 14a surrounding the face image of the user U2, and the calculated value is displayed on the display portions 14b and 14c. With this smile level calculation start, measurement for competing smile levels with other users described later is started.

  On the ranking measurement screen 14A, the calculated smile level is indicated by a number on the display portion 14b of the frame portion 14a (26 in the figure) as in the training screen 12B (see FIG. 6), and the shape of the figure is displayed by the display portion 14c. Is displayed.

  Also, in the ranking mode, the timer device 50 is operated to perform measurement at a ranking time (measurement time) preset in the setting unit 13t of the overall setting tab 13C (see FIG. 11). On the ranking measurement screen 14A, the elapsed time from the start of measurement (5 seconds in the figure) and the ranking time (30 seconds in the figure) are displayed on the display unit 14d and the display unit 14e, respectively.

  In the ranking measurement screen 14A, the measurement results of other users who have already performed the measurement in the ranking mode are displayed on the display unit 14f.

  Here, in the storage device 40, information on the result of measurement in the ranking mode is stored for a plurality of users together with the personal information of the user, and the control device 30 reads the information in the storage device 40. Display on the display unit 14f is possible.

  In the display unit 14f, the personal name displayed on the display screen of the user management tab 13A and the number of accumulated points for each user are displayed, and a user with a large number of accumulated points (here, “A mountain B that has acquired“ 800 ”points). "Man") shows so-called ranking information in which the results of each user are arranged in the downward direction.

  By the way, on the ranking measurement screen 14A, the maintenance smile degree as a target value set in advance on the display screen of the tab 13C (see FIG. 11) is displayed on the display unit 14g (“90” in the figure).

  Also, on the ranking measurement screen 14A, as shown in FIG. 13, an advice message corresponding to the smile level is displayed on the display unit 14h.

  Here, in the ranking mode, the control device 30 compares the calculated smile level with the maintenance smile level each time. When the smile level becomes equal to or higher than the maintenance smile level, the smile level at that time is integrated as an acquisition point. The points are added and stored in the storage device 40.

  For example, in FIG. 14, when the number of extractions is the tenth, the smile level is “92”, so the control device 30 adds “92” points to the accumulated points so far. In the same figure, when the accumulated point number becomes “351”, the number of extraction times at which the smile level becomes equal to or higher than the maintenance smile level is 15th, and the accumulated point number “351” so far becomes “98” again. Points are added and the accumulated point is set to “449”. In this way, the smile level at that time is further added to the accumulated points only for the number of extractions when the smile level of the user U2 is greater than or equal to the maintenance smile level “90”.

  Here, when the measurement time has elapsed, the display device 10 displays a ranking result display screen 14B as shown in FIG. On the ranking result display screen 14B, ranking information including the accumulated point measurement results of other users who have already completed the measurement is updated and displayed on the display unit 14i. In the illustrated example, the accumulated number of points of the user U2 (account “111” “C river D child”) is “674”, and “account B11” (accumulated point “800”) of the account “11” This indicates that the user has been ranked in the second place between “E field F beauty” (accumulation point “332”) of the account “12”.

  Note that the ranking result display screen 14B is not necessarily limited to displaying an account, and may display a personal name, or may display a personal name and an account together.

  Next, a usage example in the battle mode will be described with reference to FIGS. When the battle mode selection button 11c is pressed on the initial screen 11A shown in the figure, the control device 30 starts a so-called battle mode.

  Then, as illustrated in FIG. 16, the control device 30 includes, from the captured image data of the camera 3 (see FIG. 1), an area Ar1 that includes the face image of the user U3 who has operated the initial screen 11, and the area Ar1. A background image area Ar2 is extracted, and a battle screen 15 in which both areas Ar1 and Ar2 are displayed in parallel is displayed as shown in FIG.

  Here, in the battle mode, as shown in FIG. 17, the smile levels of the two users U3 and U4 are simultaneously calculated and integrated, and the integrated values are competed. 16 and 17 show a case where the user U3 first operates the image processing apparatus 1 and then invites the user U4 who has not operated the apparatus 1 as an opponent in the battle mode.

  In this case, if only the image of the area Ar1 of the user U3 is displayed on the display device 10, when the user U4 is invited to the user U3, the user U4 is within the shooting range of the camera 3 where the user U4 is located. There is a problem of not knowing.

  Therefore, in the battle mode, as described above, the background image area Ar2 excluding the area Ar1 including the face image of the user U3 is displayed, and the face of the user U4 himself / herself is displayed on the camera based on the image of the area Ar2. 3, that is, a position for detecting its own face image by the control device 30 can be easily specified.

  Specifically, the control device 30 first selects and extracts a region Ar1 centered on the already detected face image from all the captured image data of the camera 3. Based on the position of the area Ar1, the position (area) that can secure the widest area for the area Ar2 among the remaining areas excluding the area Ar1 in all the captured images of the camera 3 is determined. Select automatically. That is, here, the control device 30 selects and extracts a region that can ensure the widest horizontal direction and vertical direction other than the region Ar1, and extracts the region as a region Ar2 for displaying a background image. Display on the display device 10 and wait for the second user U4.

  In the example illustrated in FIG. 16, the painting α is displayed on the background of the user U3, and the lower left corner F1 of the frame F is displayed in the area Ar2 of the background image. The user U4 invited by the user U3 looks at the image displayed in FIG. 16B and relies on the position of the corner F1 of the frame F so that his / her face enters the shooting range of the camera 3 Can be easily identified.

  When the face images of the fighting users U3 and U4 are displayed on the display device 10, the control device 30 detects the face images of the two people and starts calculating the smile levels of the two at the same time. As in the above-described modes, the smile level is indicated by numbers on the display portions 15c and 15d of the frame portions 15a and 15b (in the figure, the smile level of the user U3 is “88” and the smile level of the user U4 is “50”). ) And displayed in the form of a figure by the display units 15e and 15f.

  Also in the battle mode, the timer device 50 operates to measure the battle time (measurement time) set in advance by the setting unit 13v of the overall setting tab 13C (see FIG. 11). On the screen 15, the elapsed time from the start of the battle is displayed on the display unit 15g (4 seconds in the figure).

  Here, in the battle mode, the control device 30 calculates the smile levels of both, accumulates the smile levels, and stores the results in the storage device 40.

  Also, the battle screen 15 displays a battle bar 15h as a ratio information display unit, and the ratio of the smile values of both users U3 and U4 described above depends on the ratio of the lengths of the bars 15h1 and 15h2. As shown in the form.

  For example, in FIG. 18, when the number of extractions is the 41st, the smile level of the user U3 is “50”, and the integrated value up to the previous time is “200”. The newly calculated smile degree “50” is added to “200” ((1) in the figure). At this time, since the smile level of the user U4 is “80” and the previous integrated value is “240”, the control device 30 sets the newly calculated smile to the previous integrated value “240”. The degree “80” is added ((2) in the figure).

  Then, the control device 30 calculates the sum of the integrated values of both users U3 and U4 ((1) + (2) in the figure), and the ratio of the integrated value of the user U3 to the sum ((1) / in the figure). (1) + (2)) and the ratio of the integrated values of the user U4 ((2) / (1) + (2) in the figure) are calculated.

  In the battle bar 15h, as shown in FIG. 17, bars 15h1 and 15h2 having different colors corresponding to the positions of the users U3 and U4 are displayed. And in this battle | competition bar 15h, based on the calculated ratio of the integrated value of the user U3 and the ratio of the integrated value of the user U4, the ratio is set to the length in the horizontal direction of the bars 15h1 and 15h2 of the battle screen 15. It is expressed by the ratio of.

  Here, when the battle time elapses, the display device 10 finally displays the letters “Win!” 15i indicating that the player is a winner on the side where the ratio is high (here, the user U3). The display is the display of the match result.

  In this battle mode, the smile level to be added may be weighted at a predetermined time just before the end to give a possibility of reverse at the end. Thereby, game nature can be raised and it can make a user recognize the joy of making a smile more strongly.

  In the battle mode, unlike the training mode and the ranking mode described above, the advice message is not displayed on the battle screen 15. However, the advice message may also be displayed in the battle mode.

Next, the operation in the training mode of the image processing apparatus 1 according to the present embodiment will be described with the flowcharts shown in FIGS. 19 and 20.
The control device 30 monitors whether or not a predetermined operation for pressing the training mode selection button 11a (see FIG. 6) has been performed by the user (step S1). When the control device 30 determines that the predetermined operation is present (step S1: YES) and the user U1 performs a login operation (password input or the like) (step S2), the control device 30 detects the face image of the user U1. Processing is executed (step S3). On the other hand, when it is determined that the predetermined operation has not been performed (step S1: NO), the process of step S1 is continued.

  When the face image of the user U1 is detected in step S3, the control device 30 extracts the face image data, displays it on the display device 10, and stores the face image data in the storage device 40 (step S4). Then, a smile degree calculation subroutine is executed on the extracted face image data (step S5).

  In the smile degree calculation subroutine, as shown in FIG. 20, first, fitting is performed to match the three-dimensional face model M (see FIG. 3) with the face image data of the user U1 (step S51), and the face shape of the user U1 is specified. (Step S52).

  Next, the control device 30 measures changes in the facial feature parts P1 to P5 (step S53), and obtains frequencies corresponding to the changes in the feature parts P1 to P5. Then, by executing integrated reasoning by calculating the average value of these frequencies (step S54), the smile level is obtained, and the smile level calculation subroutine is terminated.

  Returning to FIG. 19, the control device 30 causes the display device 10 to display the average smile degree together with the smile degree while calculating the integrated value of the calculated smile degree and the average smile degree. At this time, the smile level is simultaneously stored in the storage device 40 (step S6).

  Further, the control device 30 determines whether or not it is the update timing of the advice message (step S7), and if it is determined that it is the update timing (step S7: YES), it corresponds to the smile degree from the storage device 40. The advice message is read and displayed (step S8), and the process proceeds to step S9. On the other hand, when it is determined that it is not the update timing (step S7: NO), the control device 30 proceeds to step S9 without updating the advice message.

  In step S9, the control device 30 determines whether or not the measurement time has reached a predetermined set time. Here, when the predetermined measurement time has elapsed (step S9: YES), the control device 30 proceeds to step S10, and when the measurement time has not elapsed (step S9: NO), the step Returning to S3, the processes after step S3 are repeated.

  The control device 30 retrieves the maximum smile degree within the measurement time from the smile degree data stored in the storage device 40 in step S10, and uses the maximum smile degree together with the final overall average smile degree as a training result of the display device 10. It is displayed on the display screen 12C (step S11).

  Next, when saving of the face image data indicating the maximum smile level is designated on the setting screen 13 (step S12: YES), the control device 30 stores the face image data indicating the maximum smile level as a storage device. 40 (step S13), the process proceeds to step S14, but when the saving of the face image data is not designated (step S12: NO), the control device 30 executes the storing process of the face image data. Instead, the process proceeds to step S14.

  Here, image data other than the face image data indicating the highest smile level, or image data when the storage of the face image data indicating the highest smile level is not designated is used to secure the capacity of the storage device 40. It is preferable to delete at an appropriate timing.

  Next, when saving of the personal history information of the user U1 is specified on the setting screen 13 (step S14: YES), the control device 30 stores the average value and the maximum smile degree data in the storage device 40. Then, the personal history information is updated by adding the result of the average value and the maximum smile degree measured this time to the past measurement result of the user U1 (step S15). Then, the control device 30 creates a graph related to the personal history information (Step S16), displays the graph of the highest smile level on the training result display screen 12C (Step S17), and ends the series of processes. .

  On the other hand, when the storage of the personal history information of the user U1 is not designated (step S14: NO), the control device 30 performs a series of processes without storing the data of the personal history information of the user U1 in the storage device 40. finish.

Next, the operation in the ranking mode of the image processing apparatus 1 according to the present embodiment will be described with the flowchart shown in FIG.
The control device 30 monitors whether or not a predetermined operation for pressing the training mode selection button 11b (see FIG. 6) has been performed by the user (step S101). When the control device 30 determines that the predetermined operation is present (step S101: YES) and the user U2 performs a login operation (step S102), the processing of steps S103 to S105 corresponding to steps S3 to S5 in FIG. Execute. In step S <b> 106, the smile degree is displayed on the display device 10 and the smile degree is stored in the storage device 40. On the other hand, if it is determined that the predetermined operation has not been performed (step S101: NO), the process of step S101 is continued.

  In step S107, the control device 30 determines whether or not the calculated smile level is equal to or higher than the maintenance smile level (target value), and if it is determined that the smile level is equal to or higher than the maintenance smile level (step S107: YES), Smile level integration is performed (step 108), and the process proceeds to step S109. If the smile level is not equal to or greater than the maintenance smile level (step S107: NO), the process proceeds to step S109 without performing smile level integration. To do.

  In steps S109 and S110, processing similar to that in steps SS7 and S8 in FIG. 19 is performed. When a predetermined measurement time has elapsed in step S110 (step S111: YES), the control device 30 proceeds to step S112. When the measurement time has not elapsed (step S111: NO), the process returns to step S103, and the processes after step S103 are repeated.

  In step S112, registration processing for ranking display of smile value integrated value data is performed (step S112). In step S113, it is determined whether or not the integrated value of another user has already been registered. Here, when there is already registration (step S113: YES), the control device 30 rearranges the ranking based on the integrated value data registered in step S112 (step S114). Then, the latest ranking information is displayed (step S115), and the process proceeds to step S116. On the other hand, if there is no other user's accumulated value registration (step S112: NO), the process proceeds to step S116 without performing ranking rearrangement or the like.

  In step S116, the control device 30 executes a process corresponding to step S12 in FIG. Here, when saving of face image data indicating the maximum smile level is designated (step S116: YES), the control device 30 searches for the maximum smile level (step S117) and indicates the maximum smile level. The face image data is stored in the storage device 40 (step S118), and the series of processing ends. On the other hand, when the saving of the face image data is not designated (step S116: NO), the control device 30 ends the series of processes without executing the search process of the highest smile level and the storage process of the face image data. To do.

Next, the operation in the battle mode of the image processing apparatus 1 according to the present embodiment will be described with the flowchart shown in FIG.
The control device 30 monitors whether or not a predetermined operation for pressing the training mode selection button 11c (see FIG. 6) has been performed by the user (step S201). When the control device 30 determines that the predetermined operation is present (step S201: YES), the processing of steps S202 and S203 corresponding to steps S3 and S4 in FIG. 19 is executed. Here, in step S203, the control device 30 selects and extracts an area Ar1 (see FIGS. 16 and 17) centered on the already detected face image from all the captured image data of the camera 3, The area Ar1 is displayed on the display device 10.

  On the other hand, when it is determined that the predetermined operation has not been performed (step S201: NO), the process of step S201 is continued.

  In step S204, the control device 30 searches for the area Ar2 for displaying the background image. Specifically, the most extensive area Ar2 is searched from the areas excluding the area Ar1 based on the entire captured image data of the camera 3 and the position of the area Ar1 in the entire image data.

  In step S205, based on the search result in step S204, the control device 30 selects and extracts the region Ar2 as the background image display region, and causes the display device 10 to display the region Ar2.

  Next, when detecting the face image of the second user U4 (step S206), the control device 30 extracts the face images of the two users U3 and U4 simultaneously and displays them on the display device 10 while displaying the face image data. Is stored in the storage device 40 (step S207).

  Here, the control device 30 executes Step S208 corresponding to Step S5 in FIG. In step S209, the control device 30 causes the display device 10 to display the smile levels of the users U3 and U4, and stores the smile levels in the storage device 40. At the same time, the control device 30 stores the smile levels of the users U3 and U4. Each integrated value and the ratio of the integrated values are calculated. And based on the calculation result in step S209, the control apparatus 30 updates the display of the battle bar on the battle screen 15 (step S210).

  Next, in step S211, processing corresponding to step S9 in FIG. 19 is performed. When a predetermined measurement time has elapsed in step S210 (step S211: YES), the control device 30 proceeds to step S212. If the measurement time has not elapsed (step S211: NO), the process returns to step S206, and the processes after step S206 are repeated.

  In step S212, the control device 30 determines the winning / losing result based on the ratio of the final integrated value, and in step S213, the battle result based on the determination in step S212 is displayed.

  In Steps S214 to S216, the control device 30 executes processing corresponding to Steps S116 to S118 in FIG. 21, and then ends a series of processing.

  As described above, in the image processing apparatus 1 according to the present embodiment, in any mode, the control device 30 integrates the smile level within the measurement time, and based on the integrated value, the average value information in the training mode ( Total average smile degree), ranking information in ranking mode, and ratio information in battle mode are generated, and various information is displayed on the display device 10.

  By integrating the smile level within the measurement time, the users U1 to U4 can objectively evaluate not only the instantaneous smile but also the persistence of their smile within the measurement time. Smile status can be evaluated more accurately.

  In particular, on the training screen 12B in the training mode, the average smile degree based on the integrated value and the target value (smileiness) of the smile degree are displayed together, and the degree of smile persistence within the measurement time And the difference from the target can be easily recognized. Then, when the user U1 continues training for making a smile in the future, the difference can be used as a guideline when setting a training schedule, a training menu, and the like.

  Further, as shown in the graph 13j of FIG. 10, by displaying the transition of the average smile degree for a plurality of cases as the personal history information for one user U1, the user U1 has the ability to maintain his / her smile. It is possible to watch the training results such as the degree of improvement in the long term. Therefore, it is possible to more easily set a schedule for a future training, a training menu, and the like.

  This training mode is suitable, for example, in the hospitality business when an employee before starting work warms up to make a smile, or when a new employee tries to improve the sustainability of a smile. Moreover, even if it is an individual, it is suitable when it is desired to improve the degree of impression to friends and acquaintances.

  In particular, in the ranking mode, by displaying the result of the integrated value of other users as ranking information, it is possible to relatively evaluate the persistence of own smile and the persistence of smiles of other users. For this reason, a plurality of users who use the image processing apparatus 1 can enhance competition consciousness, and as a result, it is possible to further promote the improvement of the smile sustainability.

  This ranking mode is suitable for improving the sustainability of all employees' smiles, for example, in the hospitality industry, and it is possible to improve the degree of impression given to customers in the store by raising the competitive awareness of each employee. it can.

  In addition, in particular, in the battle mode, the integrated value of smile levels of a plurality of users (two people in FIGS. 17 and 22) is calculated at the same time, and the ratio information of the integrated values is displayed, thereby making a smile. The game can be enjoyed as if it were a game, and the user can naturally wear a continuous smile.

  Further, by displaying an advice message corresponding to the smile level within the measurement time on the training screen 12B and the ranking measurement screen 14A, it is possible to send accurate advice in real time to the user U1 who is measuring, and as a result. , Can further improve the sustainability of smiles.

  Further, by searching for the maximum smile level within the measurement time and storing the face image data at that time in the storage device 40, the user who performed the measurement or another user performs training for making a smile. In this case, face image data showing the maximum smile level can be used as a reference material.

  By the way, in the training mode described above, the total average smile level is displayed as average value information based on the calculated average value. However, a new appropriate inspection mode is provided, and the total average smile level is absolutely set. Evaluation criteria to be evaluated may be set in advance, and evaluation rank information ranking the results of the overall average smile degree may be displayed.

  FIG. 23A shows a proper inspection screen 112 in the proper inspection mode. FIG. 23A shows a case where the degree of smile of the examinee U5 is measured in order to check the suitability as an employee who is in charge of reception work such as accommodation facilities.

  On the appropriate inspection screen 112A, as in the frame portion 12c (see FIG. 6), the smile level is displayed in the form of a graphic (“70” in the figure) by the display portion 112b and the display portion 112c of the frame portion 112a.

  Further, even in the proper test mode, measurement is performed at a preset time (measurement time) by the operation of the timer device 50. On the aptitude test screen 112A, the elapsed time from the start of measurement is displayed on the display unit 112d (1 minute in the figure). In the appropriate inspection screen 112A, the average smile degree (“72” in the figure) is displayed on the display unit 112e and the measurement time is displayed on the display unit 112f.

  By the way, in executing the aptitude test mode, an evaluation rank table T for distributing the total average smile degree within the measurement time into four evaluation ranks “A” to “D” as shown in FIG. It is stored in the storage device 40.

  Then, when the predetermined measurement time has elapsed and the total average smile degree is calculated, an inspection result display screen 112 </ b> B as shown in FIG. 24 is displayed on the display device 10. On the test result display screen 112B, the average smile level (“73” in the figure) indicating the total average smile level of the examinee U5 is displayed on the display unit 112g as average value information. Then, on the display unit 112h, the evaluation rank (“B” rank in the figure) based on the evaluation rank table T in FIG. 23B is also displayed.

  In this way, by displaying the evaluation rank information that ranks the overall average smile degree, it is possible to objectively evaluate the suitability of the examinee U5 in the aptitude test in the hospitality industry, and obtain a fair test result Can do. And the versatility of this apparatus 1 can be improved by using the smile evaluation apparatus 1 for such an aptitude test.

  The smile evaluation device 1 calculates the smile level based on a sample of more than 1,000 faces, but the present invention is not necessarily limited to this. For example, face image data may be acquired for each user, and a smile degree calculation map for each user may be created based on the comparison between the expressionless state and the smile state. Thereby, evaluation based on the characteristics and individuality of each user's face shape can be performed, and more accurate calculation of the smile degree can be realized.

  In the image processing apparatus 1, various information related to the smile level integrated value is mainly output by the screen display of the display apparatus 10. However, the present invention is not necessarily limited thereto. It may be output.

In correspondence between the configuration of the present invention and the above-described embodiment,
The face image detection means of the present invention corresponds to the control device 30 that executes steps S3, S103, and S202,
Similarly,
The measuring means corresponds to the control device 30 that executes step S53,
The evaluation means corresponds to the control device 30 that executes steps S5, S6, S16, S105, S108, S114, S207, S208, and S212.
The measurement time setting means corresponds to the timer device 50,
The output means corresponds to the display device 10,
The smile degree digitizing unit corresponds to the control device 30 that executes steps S5, S105, and S208.
The integrating unit corresponds to the control device 30 that executes steps S6, S108, and S209,
The smile evaluation information generation unit corresponds to the control device 30 that executes steps S6, S16, S108, S114, and S212.
Mean value storage means, integrated value storage means, advice information storage means, image data storage means correspond to the storage device 40,
The display control means corresponds to the control device 30 that executes steps S4, S104, S203, S205, and S207,
The face image display area corresponds to the area Ar1,
The face image extraction unit corresponds to the control device 30 that executes steps S4, S104, S203, and S207,
The background image display area corresponds to the area Ar2,
The background image extraction unit corresponds to the control device 30 that executes steps S204 and S205,
The search means corresponds to the control device 30 that executes steps S10, S117, and S215.
The present invention is not limited only to the configuration of the above-described embodiment, and many embodiments can be obtained.

DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus 10 ... Display apparatus 20 ... Input apparatus 30 ... Control apparatus 40 ... Memory | storage means 50 ... Timer apparatus

Claims (10)

A face image detecting means for detecting a face image from image data;
Measuring means for measuring the amount of change in the characteristic part of the face image detected by the face image detecting means;
An image processing apparatus comprising: an evaluation unit that evaluates a smile state based on a measurement result of the measurement unit;
Measurement time setting means for setting a measurement time by the measurement means;
Output means,
The measurement means measures the change amount a plurality of times within the measurement time set by the measurement time setting means,
The evaluation means includes a smile degree quantification unit that quantifies a smile recognition degree as a smile degree based on the amount of change measured by the measurement means;
A summing unit that sums up the smile level digitized by the smile level digitizing unit within the measurement time;
A smile evaluation information generating unit that generates smile evaluation information related to the evaluation of the smile status based on the integrated value integrated by the integrating unit;
The output means is an image processing apparatus that outputs the smile evaluation information generated by the smile evaluation information generation unit. Input means for inputting a target value of the smile degree;
The smile evaluation information generating unit calculates an average value of the smile degree based on the integrated value integrated by the integrating unit,
As the smile evaluation information, generate average value information related to the average value,
The image processing apparatus according to claim 1, wherein the output unit outputs the target value input by the input unit and the average value information generated by the smile evaluation information generation unit. The average value calculated over the measurement time is provided with an average value storage means for storing a plurality of cases in one user,
The smile evaluation information generation unit generates personal history information in which the average values are arranged in time series as the smile evaluation information while reading the average values for the plurality of cases from the average value storage unit,
The image processing apparatus according to claim 2, wherein the output unit outputs the personal history information. The integrated value storage means for storing the integrated value of the smile degree integrated by the integrating unit for a plurality of users,
The smile evaluation information generation unit reads the user's integrated value stored in the integrated value storage unit, and executes a ranking process based on the magnitude of the integrated value,
As the smile evaluation information, generate ranking information related to the ranking processing result,
The image processing apparatus according to claim 1, wherein the output unit outputs information related to the ranking. The integration unit is configured to simultaneously integrate smile levels for each user in a state in which a plurality of user face images are detected by the face image detection unit,
The smile evaluation information generation unit calculates a sum of integrated values of the plurality of users,
Calculate the ratio of each user's integrated value to the total integrated value,
As the smile evaluation information, generate ratio information related to the ratio,
The image processing apparatus according to claim 1, wherein the output unit outputs the ratio information. A display control means for executing control for extracting a region including a face image from the image data and displaying the face image on a display means;
The display control means includes a face image extraction unit that selects and extracts a face image display area including the face image detected by the face image detection means from the image data,
Based on the image data and the position in the entire image data of the face image display area, the most extensive area is searched from the areas excluding the face image display area, and based on the search result, A background image extraction unit for selecting and extracting the region as a background image display region;
The display control means controls the display means to display in parallel the face image display area including the face image extracted by the face image extraction section and the background image display area extracted by the background image extraction section. The image processing apparatus according to claim 5, wherein Provided with advice information storage means for storing advice information set in advance according to the degree of smile;
The image processing apparatus according to claim 1, wherein the output unit outputs the advice information read from the advice information storage unit according to the smile level within the measurement time. Search means for searching for the highest smile degree image data when the highest smile degree is indicated within the measurement time;
The image processing apparatus according to claim 1, further comprising an image data storage unit that stores the highest smile degree image data based on a search result of the search unit. Detecting a face image from the image data by a face image detecting means;
Measuring the amount of change of the characteristic part of the face image detected by the face image detecting means by the measuring means;
An image processing method comprising a step of evaluating a smile state by an evaluation unit based on a measurement result of the measurement unit,
Setting a measurement time by the measurement means by a measurement time setting means;
Measuring the amount of change a plurality of times by the measuring means within the measuring time set by the measuring time setting means;
Based on the amount of change measured by the measuring unit, the smile level digitizing unit of the evaluation unit quantifies the smile recognition level as a smile level;
Integrating the digitized smile level by the integration unit of the evaluation means within the measurement time;
A step of generating smile evaluation information related to the evaluation of a smile state by the smile evaluation information generation unit of the evaluation means based on the integrated value integrated by the integration unit;
An image processing method comprising: outputting the smile evaluation information generated by the smile evaluation information generation unit by an output unit. On the computer,
Detecting a face image from the image data by a face image detecting means;
Measuring the amount of change of the characteristic part of the face image detected by the face image detecting means by the measuring means;
An image processing program for executing a step of evaluating a smile state by an evaluation unit based on a measurement result of the measurement unit,
Setting a measurement time by the measurement means by a measurement time setting means;
Measuring the amount of change a plurality of times by the measuring means within the measuring time set by the measuring time setting means;
Based on the amount of change measured by the measuring unit, the smile level digitizing unit of the evaluation unit quantifies the smile recognition level as a smile level;
Integrating the digitized smile level by the integration unit of the evaluation means within the measurement time;
A step of generating smile evaluation information related to the evaluation of a smile state by the smile evaluation information generation unit of the evaluation means based on the integrated value integrated by the integration unit;
An image processing program for executing the step of outputting the smile evaluation information generated by the smile evaluation information generation unit by an output means.

Images