JP7773243B2 - Provision availability determination system, provision availability determination method, and provision availability determination program - Google Patents

Description

The present invention relates to a provision eligibility determination system, a provision eligibility determination method, and a provision eligibility determination program.

Technologies have been known to prevent meals from being served to the wrong person (so-called mis-serving).

For example, the misidentification prevention system described in Patent Document 1 is a system that uses images to prevent misidentification. The system includes a patient information recording unit that records patient information, an image acquisition unit that acquires image information, and a comparison unit that compares the patient information with the image information. The comparison unit in this system compares the treatment to be performed or the item to be provided, obtained from the patient information, with the image information and displays matches and/or mismatches.

JP 2023-47459 A

When providing meals to a patient, it is sometimes important to consider not only allergenic ingredients but also the form in which the menu is presented (e.g., bite-sized, chopped, extra chopped, blended, mousse, etc.). However, the misidentification prevention system described in Patent Document 1 is only able to prevent incorrect serving of food by taking into account the patient's allergies, and does not take into account the form in which the menu is presented.

The present invention therefore aims to determine whether or not a meal can be served, taking into account the menu presentation format.

A serving readiness determination system according to one aspect of the present invention includes an image acquisition unit that acquires a meal image of a meal to be served to a subject and a meal card image of a meal card that is provided to the subject along with the meal and that bears identification information identifying the subject and at least one of the serving conditions of the meal; a meal condition identification unit that identifies the meal condition information corresponding to the meal card based on a pre-set condition correspondence relationship between the identification information and meal condition information related to the serving conditions; a menu identification unit that identifies menu information related to the menu included in the meal shown by the meal image based on a menu correspondence relationship learned based on multiple combinations of learning images of learning menus and learning menu information related to the learning menus; a content identification unit that identifies form information related to the serving form of each identified menu based on form correspondence relationships learned based on multiple combinations of each of multiple serving forms and learning images of the learning menu prepared in each of the multiple serving forms; and a serving readiness determination unit that determines whether the meal shown by the meal image can be served based on the identified meal condition information and form information.

According to the present invention, it is possible to determine whether or not a meal can be served, taking into account the menu presentation format.

FIG. 1 is a diagram showing an overview of processing in a provision availability determination system 100 according to an embodiment of the present invention. 1 is a diagram showing the configuration of a provision availability determination system 100 according to an embodiment of the present invention. FIG. 10 is a diagram showing an example of a food image. FIG. 10 is a diagram showing an example of a meal card. FIG. 4 is a diagram showing an example of subject information stored in a storage unit 110. FIG. 10 is a diagram showing an outline of menu correspondence relationships. FIG. 10 is a diagram showing an outline of a morphological correspondence relationship. FIG. 1 is a diagram showing an outline of ingredient correspondence relationships. 10 is a flowchart illustrating an example of processing in the provision availability determination system 100. 10 is a flowchart illustrating an example of processing in the provision availability determination system 100.

A preferred embodiment of the present invention will be described with reference to the accompanying drawings. Figure 1 is a diagram showing an overview of the processing in a provision eligibility determination system 100, which is one embodiment of the present invention.

The provision eligibility determination system 100 is an information processing system implemented by a provision eligibility determination program. The provision eligibility determination system 100 determines whether or not to provide a provided meal based on a meal image of the provided meal to be provided to the subject, and a meal card image of a meal card that is provided to the subject along with the provided meal and contains identification information that identifies the subject and/or the conditions for the provision format of the provided meal.

First, the provision eligibility determination system 100 acquires a meal image and a meal ticket image of the meal to be provided to the subject. The provision eligibility determination system 100 identifies the subject's meal requirements based on the meal ticket image (S101).

Next, the serving eligibility determination system 100 identifies the menu items included in the served meal based on the meal image (S102). The serving eligibility determination system 100 identifies the serving format of the identified menu items (e.g., bite-sized, chopped, extra chopped, blender, mousse, etc.) based on the meal image (S103).

Then, the provision eligibility determination system 100 determines whether or not to provide the meal to the subject based on the identified meal conditions and the type of meal to be provided (S104).

The provision eligibility determination system 100 can execute provision eligibility determination processing, for example, when providing a meal to a subject at a facility used by the subject. Here, the facility used by the subject may be, for example, a medical facility, a nursing home, an accommodation facility, etc.

Figure 2 is a diagram showing the configuration of a provision eligibility determination system 100, one embodiment of the present invention. The provision eligibility determination system 100 is communicably connected to a menu management device 200, a subject information management device 300, a first camera 400, and a second camera 500 via a network such as the Internet. Details of the provision eligibility determination system 100 will be described later.

The menu management device 200 is an information processing device that manages menu information related to the meal menu provided to the subject.

The menu management device 200 can, for example, manage scheduled menu information for scheduled menus that may be provided to the subject (e.g., menus scheduled to be provided on the current day or in the coming week).

The menu management device 200 can also manage, for example, menu information for meals provided in the past.

The menu management device 200 can also manage learning menu information used in the learning process by the learning unit 125, which will be described later. The learning menu information may be menu information for meals that have been served in the past, or it may be menu information that has been generated for learning purposes and has not been served before.

In this embodiment, the term "menu" is used to refer to, for example, at least one dish (especially, for example, multiple dishes) served in a single meal (a menu, so to speak). That is, a "menu" includes, for example, "nikujaga" (meat and potatoes) and "hachitōsai" (eight treasure dishes). Note that a "menu" may also be a "menu" that includes multiple dishes, in which case "menu" may be read as "menu" as appropriate. In this case, the provision eligibility determination system 100 can perform the processing described in this embodiment for each dish included in the menu (for example, "nikujaga" (meat and potatoes) and "hachitōsai" (eight treasure dishes)).

The subject information management device 300 is an information processing device that manages subject information related to subjects. Details of the subject information will be described later.

The subject information management device 300 can accept registration of subject information, for example, in response to an operation by the administrator of the facility used by the subject. At this time, the administrator of the facility may register the subject information based on information recorded on a medical questionnaire, medical record, etc., for example.

The first camera 400 is a photographing device that captures meal images and meal ticket images.

The first camera 400 is installed, for example, in a predetermined space (e.g., a cafeteria) where multiple subjects receive meals. The first camera 400 may be installed, for example, at a counter in the predetermined space where the meals are provided to a server who delivers the meals to the subjects, and may capture images of the meals and meal ticket images as the meals are provided to the server.

Furthermore, the first camera 400 may be, for example, a camera installed in a room used by the subject (e.g., a private room or a shared room). In this case, the first camera 400 may, for example, capture a meal image and a meal ticket image when the meal is served to the subject in the room.

The second camera 500 is a photographing device that captures an image of the meal card and an image of the subject's face.

The second camera 500 is installed, for example, in a predetermined space (e.g., a cafeteria) where multiple subjects receive provided meals, and can detect the provision of meals to each of the multiple subjects and capture a meal card image and an image of the subject's face. The second camera 500 may detect the provision of meals by, for example, detecting that a tray with the provided meal has been handed to the subject, or that the subject or a person assisting the subject (e.g., a nurse, caregiver, etc.) has raised their hand. At this time, in response to detecting the provision of the provided meal, the second camera 500 may zoom in on the provision and capture a meal card image and an image of the subject's face.

The first camera 400 and the second camera 500 may be the same camera or different cameras. Furthermore, the first camera 400 and the second camera 500 may be cameras dedicated to processing in the provision eligibility determination system 100, or may be general-purpose cameras (for example, cameras that can also be used for surveillance or security purposes).

The first camera 400 and the second camera 500 may be, for example, cameras provided on an information processing device (e.g., a portable information processing device) used by staff at a specified facility.

In Figure 2, one menu management device 200, one subject information management device 300, one first camera 400, and one second camera 500 are shown, but there may be multiple menu management devices 200, multiple subject information management devices 300, multiple first cameras 400, and multiple second cameras 500.

Next, the details of the provision eligibility determination system 100 will be described. The provision eligibility determination system 100 comprises a memory unit 110, an image acquisition unit 115, a subject information acquisition unit 120, a learning unit 125, a dietary condition identification unit 130, a menu identification unit 135, a content identification unit 140, an authentication unit 145, a provision eligibility determination unit 150, and a notification unit 155. Each unit shown in Figure 2 can be realized, for example, by using a memory area or by a processor executing a program stored in the memory area.

The memory unit 110 stores information processed by the provision eligibility determination system 100. The memory unit 110 can store, for example, meal images, meal card images, subject information, identified meal condition information, identified menu information, identified morphology information, identified ingredient information, authentication result information, and determination result information, as described below.

The image acquisition unit 115 acquires a meal image of the meal provided to the subject, as well as a meal card image of a meal card provided to the subject along with the meal and bearing identification information identifying the subject and/or the conditions for the form in which the meal is provided, and stores the acquired meal image and meal card image in the memory unit 110. The image acquisition unit 115 may also store the acquired meal image and meal card image in the memory unit 110 in association with each other.

Here, a meal card is, for example, a card placed on a tray when serving food. The material and shape of the meal card are not important.

Conventionally, meal cards may contain, for example, information that identifies the subject (e.g., the subject's name) and the subject's dietary requirements. In this case, the provider providing the meal to the subject would, for example, identify the name written on the meal card and the subject's face, and confirm that the meal was prepared specifically for the subject before providing it to the subject. At this time, the provider would refer to the dietary requirements written on the meal card, check the meal at the time of provision, and confirm that it meets the dietary requirements (i.e., for example, whether it contains ingredients that should not be given to the subject, or whether it is provided in a form appropriate for the subject), before providing the meal.

In this embodiment, the meal ticket simply contains identification information that identifies the recipient and/or the conditions for the meal's presentation. The identification information may be text, a predetermined code (e.g., a barcode or two-dimensional code), or an image.

In addition, the image acquisition unit 115 can acquire learning images of the learning menu to be used in the learning process of the learning unit 125, which will be described later.

Figure 3 shows an example of a meal image. The meal image includes multiple menus (e.g., menu 301).

Figure 4 is a diagram showing an example of a meal card. The meal card includes, for example, area 401 in which identification information identifying the subject is written, and area 402 in which identification information identifying the conditions for the serving format of the meal is written. In the example shown in Figure 4, the respective information is written using text in areas 401 and 402, but instead of text, the meal card may also contain information readable by a specified device (for example, a two-dimensional code).

The subject information acquisition unit 120 acquires subject information about the subject and stores it in the memory unit 110.

The subject information acquisition unit 120 can acquire subject information, for example, from the subject information management device 300.

The subject information acquisition unit 120 can acquire subject information for the subject corresponding to the meal card, for example, based on the identification information written on the meal card image. Specifically, if the identification information is information that identifies the subject, the subject information acquisition unit 120 accesses, for example, the subject information management device 300 and acquires subject information corresponding to the subject identified by the identification information. Furthermore, if the identification information is information that identifies dietary conditions, the subject information acquisition unit 120 acquires the dietary condition information written on the identification information as subject information.

The subject information includes, for example, identification information for identifying the subject (e.g., ID, name, etc.) and dietary condition information regarding the subject's dietary conditions.

Figure 5 is a diagram showing an example of subject information stored in the storage unit 110. The subject information stored in the storage unit 110 includes, for example, a subject ID and dietary condition information. The dietary condition information includes, for example, allergy information regarding the subject's allergies and format information regarding the conditions for the format of food served to the subject.

The form information may be, for example, "bite-sized," "chopped," "extra-chopped," "mixer," or "mousse." However, the number and names of form categories are not limited to these.

The learning unit 125 learns the correspondence relationships used in the processing of the provision eligibility determination system 100 and stores the learned correspondence relationships in the memory unit 110. The learning unit 125 can learn, for example, menu correspondence relationships, form correspondence relationships, and ingredient correspondence relationships.

The menu correspondence is a correspondence that is learned based on, for example, multiple combinations of learning images of learning menus and learning menu information related to the learning menus. In other words, the menu identification unit 135, described below, can input an image of a specific menu into the menu correspondence, thereby identifying the menu information (e.g., menu name) contained in the image.

Figure 6A shows an overview of the menu correspondence relationship. For example, the menu correspondence relationship is calculated by accepting input of a food image (S601) and identifying the menu information contained in the image (S602).

The form correspondence is a correspondence that is learned based on, for example, multiple combinations of multiple serving forms and training images of training menu items cooked in each of the multiple serving forms. In other words, by inputting a menu image into the form correspondence, the content identification unit 140, described below, can identify the serving form of the menu item included in the image (e.g., bite-sized, chopped, extra chopped, blender, mousse, etc.).

The shape correspondence is, for example, a correspondence relationship that learns the shapes of multiple serving styles for each menu item (e.g., dish). The shape correspondence may also be learned based on menu information (e.g., menu name).

Figure 6B shows an overview of the form correspondence relationship. For example, the form correspondence relationship is calculated by accepting input of a meal image containing an image of a specific menu item (S603) and identifying the serving form of the menu item included in the image (S604).

The ingredient correspondence is a correspondence that is learned based on, for example, multiple combinations of learning images, learning menu information, and learning ingredient information about learning ingredients included in the learning menu. In other words, the content identification unit 140, described below, can identify the ingredients included in a specific menu by inputting an image and menu information (e.g., the menu name) of that menu into the ingredient correspondence.

Figure 6C shows an overview of the ingredient correspondence relationship. The ingredient correspondence relationship is calculated by, for example, accepting input of a meal image including an image of a specific menu (S605) and identifying the ingredients included in that menu (S606).

The learning unit 125 can learn menu correspondence relationships, form correspondence relationships, and ingredient correspondence relationships, for example, through machine learning. Note that the learning algorithm used by the learning unit 125 is not limited to this.

The meal condition identification unit 130 identifies the meal condition information corresponding to the meal card based on the pre-set condition correspondence relationship between the identification information and the meal condition information related to the conditions for the serving format, and stores the identified meal condition information in the memory unit 110.

Here, the condition correspondence relationship may be, for example, subject information including a subject ID and meal condition information. In this case, the meal condition identification unit 130 can identify the meal condition information corresponding to the meal card based on the subject information.

In addition, if the meal card contains identification information that identifies the meal conditions, the meal condition identification unit 130 may, based on the identification information, identify the meal condition information indicated by the identification information as the meal condition information corresponding to the meal card.

The menu identification unit 135 identifies menu information related to the menu included in the provided meal indicated by the meal image based on the menu correspondence relationship, and stores the identified menu information in the memory unit 110.

The menu specification unit 135 can also specify menu information based on pre-set scheduled menu information regarding the scheduled menu that may be provided to the subject. The scheduled menu information may be information managed by the menu management device 200, for example. This limits the menu candidates that may be included in the meal to the scope of the scheduled menu, allowing the provision eligibility determination system 100 to perform the menu specification process easily and with high accuracy.

The content identification unit 140 identifies form information related to the presentation form of each identified menu based on the form correspondence relationship, and stores the identified form information in the storage unit 110.

In addition, the content identification unit 140 can further identify ingredient information regarding ingredients included in the provided meal shown in the meal image based on the menu information, and store the identified ingredient information in the memory unit 110.

In addition, the content identification unit 140 can identify ingredient information further based on the ingredient correspondence relationship.

The ingredient information to be identified may be, for example, "carrot," "onion," "peanut," etc. The ingredient information to be identified may be ingredient information related to ingredients used in the menu, or may be only ingredient information related to allergenic ingredients among the ingredients used in the menu.

The content identification unit 140 can also identify ingredient information based on the pre-set planned ingredient correspondence between the planned menu information and the planned ingredient information for planned ingredients that may be included in the planned menu. This limits the candidate ingredients that may be included in the menu to the range of planned ingredients, allowing the provision eligibility determination system 100 to simplify and increase the accuracy of the ingredient identification process.

The authentication unit 145 authenticates the subject to whom the meal is to be provided based on the information identifying the subject written on the meal ticket and information about the subject obtained when the meal is provided, and stores authentication result information regarding the authentication result in the memory unit 110. The authentication unit 145 can perform the authentication process, for example, when the identification information written on the meal ticket includes information identifying the subject.

The authentication unit 145 can perform authentication based on, for example, information identifying the subject written on the meal ticket and an image of the subject's face acquired when the meal is served, and store authentication result information regarding the authentication results in the memory unit 110.

The authentication unit 145 may also authenticate the person to whom the meal is to be provided, for example, based on information identifying the person contained in the meal card image captured by the second camera 500 and an image of the person's face captured by the second camera 500.

Here, the second camera 500 is installed, for example, in a predetermined space (e.g., a cafeteria) where multiple subjects receive provided meals, detects the provision of provided meals to each of the multiple subjects, and captures a meal card image and an image of the subject's face. In other words, the authentication unit 145 can authenticate the subject receiving the provided meal based on, for example, the meal card image and the image of the subject's face captured upon detecting the provision of the provided meal. This allows the provision eligibility determination system 100 to ensure that the correct provided meal is provided to the correct subject, even in a predetermined space where multiple subjects receive provided meals.

The provision availability determination unit 150 determines whether the food indicated by the food image can be provided based on the identified meal condition information and morphology information, and stores determination result information regarding the determination result in the storage unit 110.

The provision eligibility determination unit 150 can determine that the provided meal can be provided, for example, if the form indicated by the specified form information (i.e., the provision form of the menu included in the provided meal) satisfies the dietary conditions indicated by the specified dietary condition information (i.e., the dietary conditions of the subject). This allows the provision eligibility determination system 100 to determine whether or not to provide the meal, taking into account the provision form of the menu included in the provided meal.

The provision eligibility determination unit 150 can also determine whether or not to provide the meal based on the ingredient information. That is, the provision eligibility determination unit 150 can determine that the provided meal can be provided if, for example, the ingredients indicated in the ingredient information (i.e., the ingredients included in the provided meal) satisfy the dietary conditions indicated in the identified dietary condition information (i.e., the dietary conditions of the subject). This allows the provision eligibility determination system 100 to determine whether or not to provide the meal by further taking into consideration the ingredients included in the provided meal.

The provision eligibility determination unit 150 can also determine whether or not to provide the meal based on the authentication result (e.g., authentication result information). That is, the provision eligibility determination unit 150 can determine that the provided meal can be provided if, for example, it is determined that the subject indicated on the meal ticket and the person to whom the provided meal is to be provided match through authentication processing by the authentication unit 145, based on the authentication result. This allows the provision eligibility determination system 100 to determine whether or not to provide the meal taking into account the ingredients contained in the provided meal.

The provision eligibility determination unit 150 may perform a first provision eligibility determination process and a second provision eligibility determination process. In the first provision eligibility determination process, the provision eligibility determination unit 150 may determine whether to provide the provided meal shown in the meal image based on, for example, the identified meal condition information and form information, or further based on ingredient information. In the second provision eligibility determination process, the provision eligibility determination unit 150 may determine whether to provide the meal based on the authentication results. This allows the provision eligibility determination unit 150 to perform the provision eligibility determination process in two stages. This allows the subject to be provided with an appropriate provided meal, even if, for example, the counter where the provided meal is served and the place where the subject receives the provided meal are separate (for example, in the case of a cafeteria).

Note that the result of the determination by the provision feasibility determination unit 150 is not limited to "yes" or "no", and may be, for example, "confirmation required" to alert the provider of the provided meal.

The notification unit 155 notifies an external information processing device of warning information based on the judgment result (e.g., judgment result information). The notification unit 155 can, for example, notify a device (e.g., a portable information processing device) used by staff at a facility used by the subject.

For example, if it is determined based on the results of the determination that the provided meal can be provided, the notification unit 155 can notify the determination result that it can be provided. Furthermore, for example, if it is determined based on the results of the determination that the provided meal cannot be provided, the notification unit 155 can notify the determination result that it cannot be provided.

The notification unit 155 may, for example, only notify in the case of either "OK" or "NOT OK," and not notify in the case of the other. Specifically, the notification unit 155 may notify the result of the "NOT OK" determination only when it is determined that the provision of the meal is not OK. This allows for effective attention to be drawn to the person receiving the notification (for example, staff).

Figure 7 is a flowchart showing an example of processing in the provision eligibility determination system 100. The flowchart shown in Figure 7 is a flowchart related to the correspondence relationship learning process in the provision eligibility determination system 100.

First, the provision eligibility determination system 100 (e.g., the learning information acquisition unit) acquires learning information (e.g., learning images, learning menu information, etc.) (S701). At this time, the image acquisition unit 115 may acquire learning images of learning menus. Next, the learning unit 125 learns menu correspondence relationships, shape correspondence relationships, and ingredient correspondence relationships based on the learning information (S702).

Figure 8 is a flowchart showing an example of processing in the provision eligibility determination system 100. The flowchart shown in Figure 8 is a flowchart related to the provision eligibility determination processing in the provision eligibility determination system 100.

First, the image acquisition unit 115 acquires a meal image and a meal ticket image (S801). The subject information acquisition unit 120 acquires subject information (S802).

The meal condition identification unit 130 identifies the meal conditions corresponding to the meal card based on the condition correspondence relationship (S803). The menu identification unit 135 identifies menu information related to the menu items included in the meal shown in the meal image based on the menu correspondence relationship, and the content identification unit 140 identifies form information related to the serving form of each identified menu item (S804). At this time, the content identification unit 140 may also identify ingredient information corresponding to the menu information based on the ingredient correspondence relationship.

The authentication unit 145 authenticates the person to whom the meal will be provided (S805). The provision eligibility determination unit 150 determines whether the meal can be provided (S806). The notification unit 155 notifies the result of the determination (S807).

The order of processing in the flowchart shown in Figure 8 can be changed as appropriate. For example, the processing by the meal condition identification unit 130 only needs to be performed before the processing by the provision eligibility determination unit 150, and may be performed after the processing by the menu identification unit 135 and the content identification unit 140.

The provision eligibility determination system 100 can be used, for example, by facility staff in the subject's room (e.g., the room where the subject's bed is located) at a facility used by the subject when providing a meal to the subject. Note that the subject's room may be, for example, a private room or a shared room.

In this case, for example, a staff member delivers the provided meal to the subject's room. The staff member uses a device (e.g., a portable information processing device) to photograph the provided meal and meal ticket and provide the photograph to the provision eligibility determination system 100, and the image acquisition unit 115 acquires the meal image and meal ticket image. Next, a determination is made as to whether the meal can be provided using a process similar to that shown in Figure 8, and the notification unit 155 notifies the staff member (e.g., a device used by the staff member) of the determination result. At this time, the staff member may also photograph the subject's face using the device used by the staff member, and the authentication unit 145 may perform authentication processing based on the photographed image.

The provision eligibility determination system 100 can also be used, for example, in a designated space (e.g., a cafeteria) in a facility used by the subject where multiple subjects receive provided meals, when facility staff provide the provided meal to the subject.

We will use a cafeteria as an example. A cafeteria may have, for example, a counter where a server who delivers the food to the subject receives the food, and a dining space where multiple subjects eat. For example, when a counter staff member provides a meal to the server at the counter, a first camera 400 installed at the counter takes an image of the food and a meal ticket image and provides them to the provision eligibility determination system 100, and the image acquisition unit 115 acquires the meal image and meal ticket image. Next, a determination is made as to whether the food can be provided using a process similar to that shown in Figure 8, and the notification unit 155 notifies the counter staff or the server (for example, a device used by the counter staff or server) of the determination result. The server receives the provided meal in accordance with the result of the determination and delivers it to the subject.

The server serves the meal to the target person and performs a predetermined action (for example, raising their hand). The second camera 500 detects this predetermined action as the serving of the meal, acquires an image of the meal ticket and an image of the target person's face, and the authentication unit 145 authenticates them. The serving permission determination unit 150 then further determines whether or not to serve the meal based on the results of the authentication process, and the notification unit 155 notifies them. Depending on the result of the determination, the server allows the target person to eat the served meal.

In this way, the provision eligibility determination system 100 can learn the provision format of the meal and determine whether or not to provide it based on the provision format. This allows the provision eligibility determination system 100 to provide a meal that is appropriate for the target person.

Furthermore, the provision eligibility determination system 100 can perform a provision eligibility determination in two stages: first, determining whether or not to provide a meal based on the form of provision of the meal and the meal conditions written on the meal card, and second, determining whether or not to provide a meal based on the results of the authentication process by the authentication unit 145 (i.e., information about the person to whom the meal will be provided (e.g., a facial image) and the identification information written on the meal card). This makes it possible to provide an appropriate meal to the person, even if, for example, the counter where the meal is provided and the place where the person receives the meal are separate (e.g., in the case of a cafeteria).

The above describes one embodiment of the present invention. The provision eligibility determination system 100 acquires a meal image and a meal card image, identifies meal condition information corresponding to the meal card based on the condition correspondence relationship, identifies menu information related to the menu items included in the provided meal based on the menu correspondence relationship, identifies form information related to the serving form of each identified menu item based on the form correspondence relationship, and determines whether or not the provided meal shown in the meal image can be provided based on the identified meal condition information and form information. This allows the provision eligibility determination system 100 to determine whether or not to provide the meal, taking into account the serving form of the menu.

In addition, the provision eligibility determination system 100 can notify an external information processing device of warning information based on the determination result. This allows the provision eligibility determination system 100 to notify the determination result, for example, to staff at the facility used by the subject.

The serving eligibility determination system 100 can further identify ingredient information regarding ingredients contained in the provided meal based on the menu information, and determine whether or not to serve the meal based on the ingredient information. The serving eligibility determination system 100 can also identify ingredient information based on ingredient correspondence relationships. This allows the serving eligibility determination system 100 to determine whether or not to serve the meal, taking into account the ingredients contained in the provided meal, for example, the presence or absence of allergenic ingredients.

The provision eligibility determination system 100 can also identify menu information based on the scheduled menu information. This allows the provision eligibility determination system 100 to easily and accurately identify menus within the scheduled menu.

The serving availability determination system 100 can also identify ingredient information based on the planned ingredient correspondence relationship. This allows the serving availability determination system 100 to easily and accurately identify ingredients within the range of planned ingredients.

The provision eligibility determination system 100 also authenticates the person to whom the meal is to be provided based on the information identifying the person listed on the meal ticket and information about the person obtained when the meal is provided, and can determine whether or not to provide the meal based on the authentication results. This allows the provision eligibility determination system 100 to authenticate the person to whom the meal is to be provided and provide the meal, preventing incorrect serving of the meal.

In addition, the provision eligibility determination system 100 can authenticate the subject based on an image of their face.

The provision eligibility determination system 100 also detects the provision of food to each of the multiple subjects using a camera installed in a specified space where multiple subjects receive provided meals, and can determine whether or not to provide the food based on the captured meal card image and image of the subject's face. This allows the provision eligibility determination system 100 to prevent incorrect serving of food, even when multiple subjects each receive provided meals in a cafeteria, for example.

Note that the present embodiment is intended to facilitate understanding of the present invention and is not intended to limit the scope of the present invention. The present invention may be modified or improved without departing from its spirit, and equivalents thereof are also included within the scope of the present invention.

In addition, in this invention, the term "unit" does not simply mean a physical means, but also includes cases where the functions of that "unit" are realized by software. Furthermore, the functions of one "unit" or device may be realized by two or more physical means, devices, or software, and the functions of two or more "units" or devices may be realized by one physical means, device, or software.

100 Provision possibility determination system, 110 Memory unit, 115 Image acquisition unit, 120 Subject information acquisition unit, 125 Learning unit, 130 Dietary condition identification unit, 135 Menu identification unit, 140 Content identification unit, 145 Authentication unit, 150 Provision possibility determination unit, 155 Notification unit, 200 Menu management device, 300 Subject information management device, 400 First camera, 500 Second camera

Claims (12)

an image acquisition unit that acquires a meal image of the meal provided to the subject, and a meal card image of a meal card that is provided to the subject together with the meal and that contains identification information that identifies the subject and/or the conditions of the meal's provision format;
a meal condition specification unit that specifies the meal condition information corresponding to the meal ticket based on a condition correspondence relationship between the identification information and meal condition information relating to the conditions of the serving format, which is set in advance;
a menu identification unit that identifies menu information related to a menu included in the meal shown by the meal image based on a menu correspondence relationship learned based on a plurality of combinations of learning images of learning menus and learning menu information related to the learning menus, and based on scheduled menu information related to a scheduled menu that may be provided to the subject that is set in advance;
a content identification unit that identifies form information regarding the serving form of each of the identified menus based on form correspondence relationships learned based on multiple combinations of each of a plurality of serving forms and training images of training menus cooked in each of the plurality of serving forms;
a provision availability determination unit that determines whether the provided meal shown in the meal image can be provided based on the specified meal condition information and the morphology information;
A provision availability determination system comprising: The provision availability determination system of claim 1 further comprises a notification unit that notifies an external information processing device of warning information based on the result of the determination. The content identification unit further identifies ingredient information related to ingredients included in the provided meal shown in the meal image based on the menu information,
The provision availability determination unit determines whether or not the food is available based further on the ingredient information.
The provision possibility determination system according to claim 1 or 2. The provision eligibility determination system of claim 3, wherein the content identification unit further identifies the ingredient information corresponding to the menu information based on ingredient correspondence relationships learned based on multiple combinations of the learning image, the learning menu information, and learning ingredient information related to learning ingredients included in the learning menu. the content specifying unit specifies the ingredient information based on a predetermined correspondence relationship between the scheduled menu information and scheduled ingredient information of scheduled ingredients that may be included in the scheduled menu.
The provision possibility determination system according to claim 4 . the identification information includes information that identifies the subject;
The provision determination system further includes an authentication unit that authenticates the target person to whom the provided meal is provided based on information identifying the target person written on the meal ticket and information about the target person acquired when the provided meal is provided,
the provision permission determination unit determines whether or not to provide the information further based on a result of the authentication.
The provision possibility determination system according to claim 1 or 2. The provision possibility determination system according to claim 6 , wherein the information about the subject is an image of the subject's face. Further provided is a first camera that captures the meal image and the meal ticket image,
The image acquisition unit acquires the meal image and the meal ticket image photographed by the first camera.
The provision possibility determination system according to claim 1 or 2. the identification information includes information that identifies the subject;
The provision availability determination system includes:
a second camera for capturing the meal ticket image and an image of the subject's face;
an authentication unit that authenticates the target person who will be provided with the meal based on information identifying the target person included in the meal ticket image captured by the second camera and an image of the target person's face captured by the second camera;
Furthermore,
the provision permission determination unit determines whether or not to provide the information further based on a result of the authentication.
The provision possibility determination system according to claim 8 . The provision possibility determination system described in claim 9, wherein the second camera is installed in a specified space where multiple subjects receive the provided meals, detects the provision of the provided meals to each of the multiple subjects, and captures the meal ticket image and an image of the subject's face. The computer
Acquire a meal image of the meal provided to the subject, and a meal card image of a meal card provided to the subject together with the provided meal, which contains identification information identifying the subject and/or the conditions of the form of provision of the provided meal;
Identifying the meal condition information corresponding to the meal ticket based on a condition correspondence relationship between the identification information and meal condition information relating to the conditions of the serving format, which is set in advance;
Identifying menu information related to the menu included in the meal shown in the meal image based on a menu correspondence relationship learned based on a plurality of combinations of learning images of learning menus and learning menu information related to the learning menus, and based on scheduled menu information related to a scheduled menu that may be provided to the subject that is set in advance ;
Identifying form information relating to the serving form of each of the identified menus based on form correspondence relationships learned based on a plurality of combinations of each of a plurality of serving forms and learning images of the learning menus cooked in each of the plurality of serving forms;
determining whether or not the meal shown in the meal image can be provided based on the specified meal condition information and the morphology information;
Method of determining availability. On the computer,
Acquiring a meal image of the meal provided to the subject, and a meal card image of a meal card provided to the subject together with the meal, the meal card having identification information identifying the subject and/or the conditions of the form of provision of the meal;
Identifying the meal condition information corresponding to the meal ticket based on a condition correspondence relationship between the identification information and meal condition information relating to the conditions of the serving format, which is set in advance;
Identifying menu information related to a menu included in the meal shown by the meal image based on a menu correspondence relationship learned based on a plurality of combinations of learning images of learning menus and learning menu information related to the learning menus, and based on scheduled menu information related to a scheduled menu that may be provided to the subject that is set in advance;
Identifying form information relating to the serving form of each of the identified menus based on form correspondence relationships learned based on a plurality of combinations of each of a plurality of serving forms and learning images of the learning menus cooked in each of the plurality of serving forms;
determining whether or not the meal shown in the meal image can be provided based on the specified meal condition information and the morphological information;
A program that determines whether or not to provide the service.