JP7779336B2 - Reporting support device, reporting support method, and program - Google Patents

Description

This disclosure relates to technology that supports responses during emergency calls.

When an accident occurs, people at the scene of the accident will make an emergency call to the police station, fire department, etc. depending on the situation. The operator in the control room who receives the emergency call will usually understand the situation based on the voice of the caller.

Patent Document 1 discloses technology for understanding the situation at the scene when an emergency call is made. Specifically, Patent Document 1 discloses that when an emergency call is received, a video transmission request is sent to the caller's mobile device, and the video captured by the mobile device is received. The technology disclosed in Patent Document 1 helps understand the situation at the scene by having the operator review the video.

Regarding emergency calls, Patent Document 2 discloses a method for understanding the situation when transporting a patient via ambulance following an emergency call, in which identification information of the patient is acquired at the time of ambulance transport and medical information of the patient is acquired based on the identification information. Furthermore, in relation to technology for acquiring images from a mobile device, Patent Document 3 discloses a technology for performing image processing on a captured image depending on whether or not consent has been given to the use of the patient's portrait, and then transmitting the image.

Japanese Patent Application Laid-Open No. 2021-072642 International Publication No. 2019/198358 Japanese Patent Application Laid-Open No. 2005-110004

The operator in the control room who receives the call will issue various instructions not only to the caller but also to the police, fire department, or ambulance team that rushes to the scene. Therefore, the operator needs to accurately understand the situation.

When the control room is able to receive video footage captured by the caller, it becomes easier for the operator to grasp the situation at the scene compared to when they only hear the caller's audio. However, there is a limit to the information that an operator can obtain by viewing the video footage.

This disclosure has been made in consideration of the above-mentioned issues, and one of its objectives is to provide a reporting support device, etc. that can assist operators in understanding the situation when a report is received.

A reporting support device according to one aspect of the present disclosure includes an acquisition means for acquiring a captured image from a reporter terminal, which is a terminal owned by the reporter; an extraction means for extracting feature information relating to an object contained in the captured image; an identification means for identifying the object based on the extracted feature information; and an output means for outputting information relating to the identified object.

A reporting support method according to one aspect of the present disclosure acquires a captured image from a reporter terminal, which is a terminal owned by the reporter, extracts feature information about an object contained in the captured image, identifies the object based on the extracted feature information, and outputs information about the identified object.

A computer-readable storage medium according to one aspect of the present disclosure stores a program that causes a computer to execute the following processes: acquiring a captured image from a reporter terminal, which is a terminal owned by the reporter; extracting feature information about an object contained in the captured image; identifying the object based on the extracted feature information; and outputting information about the identified object.

This disclosure can assist operators in understanding the situation when a report is received.

1 is a diagram schematically illustrating an example of a configuration of a reporting support system including a reporting support device according to a first embodiment of the present disclosure. 1 is a block diagram illustrating an example of a reporting support device according to a first embodiment of the present disclosure. 4 is a flowchart illustrating an example of an operation of the reporting support device according to the first embodiment of the present disclosure. FIG. 10 is a block diagram illustrating an example of a configuration of a reporting support system according to a second embodiment of the present disclosure. FIG. 10 is a sequence diagram illustrating an example of an operation of the reporting support system according to the second embodiment of the present disclosure. FIG. 10 is a diagram schematically illustrating an example of a configuration of a reporting support system according to a third embodiment of the present disclosure. FIG. 11 is a diagram illustrating an example of a database according to a third embodiment of the present disclosure. FIG. 10 is a block diagram illustrating an example of a configuration of a reporting support system according to a third embodiment of the present disclosure. FIG. 11 is a diagram illustrating an example of an output image according to a third embodiment of the present disclosure. FIG. 11 is a sequence diagram illustrating an example of an operation of the reporting support system according to the third embodiment of the present disclosure. FIG. 10 is a diagram schematically illustrating an example of a configuration of a reporting support system according to a fourth embodiment of the present disclosure. FIG. 10 is a block diagram illustrating an example of a configuration of a reporting support system according to a fourth embodiment of the present disclosure. FIG. 13 is a diagram illustrating an example of an output image according to a fourth embodiment of the present disclosure. FIG. 10 is a sequence diagram illustrating an example of an operation of the reporting support system according to the fourth embodiment of the present disclosure. FIG. 13 is a block diagram illustrating an example of a configuration of a reporting support system according to a fifth embodiment of the present disclosure. FIG. 13 is a diagram illustrating an example of a captured image according to a fifth embodiment of the present disclosure. FIG. 13 is a sequence diagram illustrating an example of an operation of the reporting support system according to the fifth embodiment of the present disclosure. FIG. 13 is a diagram showing a shooting screen according to a seventh modification of the present disclosure. FIG. 2 is a block diagram illustrating an example of a hardware configuration of a computer device that realizes a reporting support system according to the first, second, third, fourth, and fifth embodiments of the present disclosure.

Embodiments of the present disclosure are described below with reference to the drawings.

First Embodiment
First, the mechanism of reporting will be explained. A caller makes an emergency call by, for example, entering a predetermined phone number on a mobile phone. This call is connected to a command system in a command room. The command room refers to an organization that issues commands to dispatch those who should rush to the scene (for example, police, fire department, and ambulance team) depending on the content of the call. An operator in the command room communicates with the caller via the command system. Then, based on the information obtained from the caller, the operator gives instructions to the caller or uses the command system to give instructions to those who should rush to the scene. An example of the reporting support device of the present disclosure is incorporated into a command system used in this way.

Next, we will provide an overview of the reporting support device disclosed herein.

Figure 1 is a diagram schematically illustrating an example of the configuration of a reporting support system 1000 including a reporting support device 100. As shown in Figure 1, the reporting support system 1000 comprises a reporting support device 100 and a reporter terminal 200. The reporting support device 100 is communicatively connected to the reporter terminal 200 and an external device 300 via a wireless or wired network. The external device 300 may be provided in the reporting support device 100.

The reporter terminal 200 is a terminal owned by the reporter. The reporter terminal 200 is, for example, a mobile phone, smartphone, or tablet terminal, but is not limited to these examples. The reporter terminal 200 may be any terminal that has at least the functions of making a report, taking a photograph, and transmitting the photographed image. For example, when the reporter makes a report using the reporter terminal 200, the report is accepted by the reporting support device 100. Then, the operator in the control room who accepts the report communicates with the reporter via the reporting support device 100 and the reporter terminal 200.

Figure 2 is a block diagram showing an example of the functional configuration of the reporting support device 100 of the first embodiment. As shown in Figure 2, the reporting support device 100 includes an acquisition unit 110, an extraction unit 120, an identification unit 130, and an output unit 140.

The acquisition unit 110 acquires the captured image from the reporter terminal 200. For example, when a report is made by the reporter terminal 200, the reporter terminal 200 takes a photograph. At this time, the reporting support device 100 may send a signal to the reporter terminal 200 requesting that a photograph be taken, thereby activating the camera mounted on the reporter terminal 200. Alternatively, the reporter terminal 200 may start the camera when it accepts input from the reporter. The reporter terminal 200 transmits the captured image generated by capturing the image to the reporting support device 100. The acquisition unit 110 acquires the captured image transmitted from the reporter terminal 200. In this way, the acquisition unit 110 acquires the captured image from the reporter terminal 200 owned by the reporter. The acquisition unit 110 is an example of an acquisition means.

The extraction unit 120 extracts feature information from the captured image. Feature information is information that indicates the features of an object included in the captured image. Feature information is used to identify the object. The use of feature information will be described later. The object may be, for example, a person, or a non-human object such as a sign, billboard, or building. In this disclosure, such non-human objects are referred to as features. When the object is a person, the feature information may be, for example, feature quantities related to the person's face or feature quantities related to the person's appearance. A person's appearance may include, for example, the person's face, hairstyle, and body shape, or the clothing worn and luggage carried by the person. When the object is a feature, the feature information may be, for example, feature quantities related to the appearance of the feature, text information written on the feature, or a partial image extracted from the captured image that includes the feature. In this way, the extraction unit 120 extracts feature information about objects included in the captured image.

The identification unit 130 identifies the object based on the feature information. For example, a database containing information in which feature information and information related to objects are previously associated is stored in the external device 300. At this time, the identification unit 130 compares the feature information extracted by the extraction unit 120 with the feature information contained in the database. For example, if the comparison results in a match, the identification unit 130 identifies the object by identifying information associated with the matching feature information. For example, assume that the database contains information in which feature amounts related to a person's face are associated with identification information that identifies the person or personal information such as the person's name. In this case, the identification unit 130 compares the feature amounts related to the person's face extracted by the extraction unit 120 with the feature amounts contained in the database. If the comparison results in a match, the identification unit 130 identifies the identification information or personal information associated with the matching feature amounts.

Also, for example, suppose the database contains information that associates features related to a person's appearance with attributes indicating gender, age, etc. In this case, the identification unit 130 compares the features related to the person's appearance extracted by the extraction unit 120 with the features contained in the database. If the comparison results in a match, the identification unit 130 identifies the attribute associated with the matching feature.

Note that the method for identifying an object is not limited to this example. For example, the identification unit 130 may identify attributes based on a learning model that has previously learned the relationship between features related to a person's appearance and attributes. In this case, the learning model may be stored in the external device 300. Also, for example, if the feature information is text information or a partial image, the identification unit 130 may search for the feature information using a search engine. The identification unit 130 may then treat the search results as the result of identifying the object.

In this way, the identification unit 130 identifies the object based on the extracted feature information. The identification unit 130 is an example of an identification means.

The output unit 140 outputs information about the object. For example, the output unit 140 outputs information about the object identified by the identification unit 130 to an output device (not shown). Here, the output device is, for example, a display and a speaker. The output device may be a device included in the reporting support device 100, or may be another device communicatively connected to the reporting support device 100. The output unit 140 may output, for example, identification information, personal information, or attributes of the person identified by the identification unit 130. The output unit 140 may also output, for example, search results by the identification unit 130. Without being limited to this, the output unit 140 may output information further obtained from information about the object identified by the identification unit 130. For example, a person's identification information and the person's medical information are associated and stored in a device capable of communicating with the reporting support device 100. Here, medical information includes the person's health insurance card information, information about medical treatment, health checkup results, information about medications taken, etc. The output unit 140 may acquire medical information associated with the identification information identified by the identification unit 130, and output the acquired medical information. Furthermore, for example, the output unit 140 may output information about a feature acquired from a search result by the identification unit 130. For example, if the search result includes position information about the feature, the output unit 140 may output the position information about the feature. In this way, the output unit 140 outputs information about the identified object. The output unit 140 is an example of an output means.

Next, an example of the operation of the reporting support device 100 will be described using Figure 3. Note that in this disclosure, each step in the flowchart will be represented by a number assigned to each step, such as "S1."

Figure 3 is a flowchart illustrating an example of the operation of the reporting support device 100. The acquisition unit 110 acquires a captured image from a reporter terminal, which is a terminal owned by the reporter (S1). The extraction unit 120 extracts feature information related to an object included in the captured image (S2). The identification unit 130 identifies the object based on the extracted feature information (S3). The output unit 140 outputs information related to the identified object (S4).

In this way, the reporting support device 100 of the first embodiment acquires a captured image from the reporter terminal 200, which is a terminal owned by the reporter, and extracts feature information about objects included in the captured image. The reporting support device 100 then identifies the object based on the extracted feature information and outputs information about the identified object. For example, assume that an operator in a control room receives a report from a reporter about an accident in which an injured person has occurred. In such a case, if the captured image includes the injured person, the reporting support device 100 can output information about the injured person. The operator can, for example, use the output information about the injured person to provide further instructions to the reporter or to send commands to emergency responders or other personnel dispatched to the scene. Furthermore, if the captured image includes features such as signs, billboards, and buildings near the reporter, the reporting support device 100 can output information about the features. The operator can, for example, use the output information about the features to identify the reporter's location. In this way, the reporting support device 100 of the first embodiment can assist the operator in understanding the situation when receiving a report.

Second Embodiment
Next, a reporting support device according to a second embodiment will be described. In the second embodiment, the reporting support device 100 described in the first embodiment will be described in more detail. Note that some of the content overlapping with the content described in the first embodiment will not be described again.

Figure 4 is a block diagram showing an example of the configuration of the reporting support system 1000. As shown in Figure 4, the reporting support system 1000 comprises a reporting support device 100 and a reporter terminal 200. The reporting support device 100 and the reporter terminal 200 are connected so that they can communicate with each other.

[Details of the reporting support device 100]
As shown in FIG. 4 , the reporting support device 100 includes an acquiring unit 110 , an extracting unit 120 , an identifying unit 130 , an output unit 140 , a receiving unit 150 , and an imaging requesting unit 160 .

The reception unit 150 receives a report from the reporter terminal 200. When the reception unit 150 receives a report, a call is initiated between the reporting support device 100 and the reporter terminal 200, for example, using a telephone line.

The photography requesting unit 160 sends a photography request to the reporter terminal 200. For example, the photography requesting unit 160 sends a message using the telephone number of the reporter terminal 200. At this time, the photography requesting unit 160 may send the message using SMS (Short Message Service). The message may include, for example, a URL (Uniform Resource Locator). For example, when the URL of the message is opened on the reporter terminal 200, data communication between the reporter terminal 200 and the reporting support device 100 begins. The photography requesting unit 160 may send, for example, such a message as a photography request. The photography requesting unit 160 may automatically send a photography request when a report is accepted by the accepting unit 150. Furthermore, the photography requesting unit 160 may send a photography request when an operation to send a photography request is accepted from an operator after the report is accepted.

The acquisition unit 110 acquires the captured image from the reporter terminal 200 when data communication is initiated in response to a capture request. The captured image may be one or more still images or may be video. If the captured image is video, the acquisition unit 110 may continue to acquire the video being captured by the reporter terminal 200 in real time.

The extraction unit 120 extracts feature information from the captured image. In this case, the extraction unit 120 may extract feature information from any of the captured images acquired by the acquisition unit 110.

The identification unit 130 identifies an object included in the captured image based on the extracted feature information. For example, a database containing information associating feature information with information related to objects is stored in advance in the external device 300. At this time, the identification unit 130 compares the feature information extracted by the extraction unit 120 with the feature information included in the database. If the comparison results in a match, the identification unit 130 identifies the object by identifying information associated with the matching feature information. Furthermore, for example, if the feature information is text information or a partial image, the identification unit 130 may search for the feature information using a search engine. The identification unit 130 may treat the search results as the result of identifying the object.

The identification unit 130 may also identify an object based on a pre-generated learning model. The learning model is a model that learns the relationship between feature information and information about the object. For example, the learning model may be a model that takes feature quantities related to a person's appearance as input and outputs the person's attributes. The learning model may be generated using various machine learning algorithms. Examples of machine learning algorithms include, but are not limited to, SVM (Support Vector Machine), neural networks, and random forests.

The output unit 140 outputs information about the object identified by the identification unit 130. For example, the output unit 140 outputs to an output device that is communicatively connected to the reporting support device 100. The output device may be, for example, a device that can be seen by an operator, or may be a device owned by the person who should rush to the scene.

[Details of the reporter terminal 200]
The reporter terminal 200 includes a reporting unit 210 , a request receiving unit 220 , an image capturing unit 230 , and a data transmitting unit 240 .

The reporting unit 210 makes a report. More specifically, the reporting unit 210 accepts input of an emergency call telephone number from the caller. The reporting unit 210 makes the report to the call destination corresponding to the input telephone number, for example, using a telephone line. When the report is accepted by the report support device 100 at the call destination, the reporting unit 210 starts two-way communication with the call destination. In other words, the reporting unit 210 realizes a call between the caller and an operator at the call destination via the caller terminal 200 and the report support device 100.

The request receiving unit 220 receives a shooting request from the reporting support device 100. For example, the request receiving unit 220 receives a message using SMS as described above as a shooting request. Then, the reporter opens the URL included in the message, thereby starting data communication between the reporting support device 100 and the reporter terminal 200. This data communication may be realized, for example, by WebRTC (Web Real Time Communication). In other words, the reporter opens the URL included in the message, thereby allowing data communication via a browser between the reporting support device 100 and the reporter terminal 200.

The photographing unit 230 takes a photograph and generates a photographed image. That is, the photographing unit 230 has the functionality of a camera installed in the reporter terminal 200. For example, when the request receiving unit 220 receives a photographing request, the photographing unit 230 activates the camera function. The photographing unit 230 then starts photographing. For example, the photographing unit 230 may activate the camera function and start photographing in response to opening a URL included in a message such as the one described above. Note that the timing at which photographing starts is not limited to this example. For example, the photographing unit 230 may start photographing by accepting an operation from the reporter after the camera function has been activated.

The data transmission unit 240 transmits the captured image to the reporting support device 100. For example, the data transmission unit 240 may transmit the video captured by the imaging unit 230 to the reporting support device 100 in real time. Also, if a still image is captured by the imaging unit 230, the data transmission unit 240 may transmit the still image to the reporting support device 100 each time it is captured.

[Example of operation of the reporting support system 1000]
Next, an example of the operation of the reporting support system 1000 will be described with reference to FIG.

Figure 5 is a sequence diagram illustrating an example of the operation of the reporting support system 1000. The reporting unit 210 of the reporter terminal 200 makes a report (S101). The reception unit 150 of the reporting support device 100 receives a report from the reporter terminal 200 (S102). Next, the image capture request unit 160 sends an image capture request to the reporter terminal 200 (S103). The request receiving unit 220 of the reporter terminal 200 receives the image capture request (S104).

In response to the request for capturing an image being received by the request receiving unit 220, the capturing unit 230 captures an image (S105). The data transmitting unit 240 transmits the captured image to the reporting support device 100 (S106).

The acquisition unit 110 of the reporting support device 100 acquires a captured image from the reporter terminal 200 (S107). The extraction unit 120 extracts feature information from the acquired captured image (S108). The identification unit 130 identifies an object based on the extracted feature information (S109). The output unit 140 outputs information about the identified object (S110).

In this way, the reporting support device 100 of the second embodiment acquires a captured image from the reporter terminal 200, which is a terminal owned by the reporter, and extracts feature information about an object contained in the captured image. The reporting support device 100 then identifies the object based on the extracted feature information and outputs information about the identified object. This allows the reporting support device 100 of the second embodiment to achieve the same effect as the first embodiment. In other words, when the reporting support device 100 receives a report, it can assist the operator in understanding the situation.

[Modification 1]
In the above embodiment, an example has been described in which the reporter terminal 200 makes a report using a telephone line, and the reporting support device 100 sends a photographing request using SMS to the reporter terminal 200. The operations of the reporter terminal 200 and the reporting support device 100 are not limited to this form.

For example, the reporter terminal 200 may make a report using data communication rather than a telephone line. The reporting unit 210 makes the report using data communication. The reception unit 150 of the reporting support device 100 receives the report. When the reception unit 150 receives the report, the image capture request unit 160 sends an image capture request. At this time, the image capture request may be a signal that controls the image capture unit 230 of the reporter terminal 200. In this case, when the request receiving unit 220 receives the image capture request, the image capture unit 230 may automatically start capturing images.

In addition to this example, the shooting request may be a notification urging the reporter to start shooting. In this case, when the request receiving unit 220 receives the shooting request, a message indicating that shooting should start may be displayed on the reporter terminal 200. Then, the shooting unit 230 may start shooting in response to an operation by the reporter.

In this way, the reporting support device 100 and reporter terminal 200 of variant example 1 do not use telephone lines, so there is no need to use separate lines for reporting and sending and receiving captured images.

[Modification 2]
As an example of how the identification unit 130 identifies an object, the identification unit 130 compares the feature information extracted by the extraction unit 120 with feature information included in a database. The database stores information in advance that associates feature information with information about objects. Such a database may be generated by an external organization or may be generated by the reporting support system 1000.

When the reporting support system 1000 generates a database in which facial features of a person are associated with the person's identification information, information including the features may be acquired in advance in the reporter terminal 200. For example, the photographing unit 230 photographs the reporter's face when no incident, accident, etc. has occurred. The reporter terminal 200 also accepts input of the reporter's personal information. The data transmitting unit 240 transmits the reporter's facial image and the input personal information to the reporting support device 100. Then, in the reporting support device 100, for example, a generating unit (not shown) generates a database by associating the facial image or features extracted from the facial image with the personal information.

At this time, the reporter terminal 200 may obtain consent in advance from the reporter regarding the use of the characteristic information. For example, the image capturing unit 230 may output a notification to the reporter terminal 200 confirming whether or not the reporter has consented to the use of the characteristic information. Then, when the reporter has input consent, the image capturing unit 230 may start capturing images. The output notification may be, for example, a sentence such as "Do you agree to the transmission of personal information and facial information?" displayed on the display of the reporter terminal 200. In other words, the notification may be a notification confirming whether or not the reporting support system 1000 may perform facial recognition of people, including the reporter, from images captured by the reporter terminal 200.

[Modification 3]
If consent to the use of feature information has not been obtained in advance, when the request receiving unit 220 receives an image capture request, the reporter terminal 200 may confirm whether or not the reporter agrees to sending the captured image. For example, when the request receiving unit 220 receives an image capture request, the image capturing unit 230 outputs a notification to the reporter terminal 200 confirming whether or not the reporter agrees to sending the captured image. Then, when the reporter inputs consent, the image capturing unit 230 may start capturing. The output notification may be, for example, a sentence such as "Do you agree to send the captured image?" displayed on the display of the reporter terminal 200.

The image capture request may also include a notification confirming whether consent has been granted. In other words, the image capture request unit 160 of the reporting support device 100 may be controlled to output the notification to the reporter terminal 200.

[Modification 4]
The reporting assistance system 1000 may perform different processing depending on whether or not consent has been given to the use of the characteristic information, as described in the second and third modifications.

For example, suppose that consent to perform facial recognition of people, including the reporter, from an image captured by the reporter terminal 200 has not been obtained. In this case, when the request receiving unit 220 receives a capture request, the capture unit 230 may apply processing such as mosaic to the area of the person's face in the captured image. In other words, the capture unit 230 may generate a captured image that hides the person's face. The data transmission unit 240 then transmits the processed captured image to the reporting support device 100. In this case, the reporting support device 100 may output location information based on features contained in the captured image.

In this way, the reporter terminal 200 may process areas of the captured image for which the reporter has not given consent to the use of the characteristic information. In this case, the reporting support device 100 may perform processing by the extraction unit 120, identification unit 130, and output unit 140 using areas of the captured image excluding areas for which consent to the use of the characteristic information has not been given.

Third Embodiment
Next, a report support system 1001 according to a third embodiment will be described. In the third embodiment, an example in which information about the identified person is further acquired will be described. Note that some of the content overlapping with the content described in the first and second embodiments will not be described.

Figure 6 is a diagram showing a schematic example of the configuration of the reporting support system 1001. As shown in Figure 6, the reporting support system 1001 comprises a reporting support device 101 and a reporter terminal 200. The reporting support device 101 is communicatively connected to the reporter terminal 200, an external device 300, a personal information server, and a field terminal 500.

In this embodiment, an example is described in which, when an injured person is present, a caller other than the injured person makes a call for an emergency, but the content described in this embodiment is not limited to this example. In other words, the content described in this embodiment can be applied to various situations in which a caller makes a call.

The external device 300 is, for example, a device that includes a database for identifying a person. In this case, the database contains information in which at least features related to a person's face are associated with the person's identification information. Figure 7 is a diagram showing an example of a database. In the example of Figure 7, identification information, features, and personal information are associated. For example, the record in the first row of Figure 7 indicates that a person with identification information "001" has a feature value of "X", a name of "Taro Tanaka," and an age of "25." Note that in this example, name and age are listed as personal information, but personal information is not limited to these. Personal information may also include a person's address, telephone number, occupation, etc.

The identification information may be any information that can uniquely identify a person. For example, the identification information may be information related to a person's official certificate. Official certificates include, for example, a driver's license and a My Number card. For example, the identification information may be the license number of a driver's license or the personal number (also referred to as "My Number") of a My Number card. Furthermore, without being limited to this example, the identification information may be certification information related to an official certificate. The certification information may be, for example, information indicating the validity of a My Number card, and may be an electronic certificate issued by an external certification authority, or a code generated using the electronic certificate.

The information server 400 includes information in which identification information is associated with information about a person. For example, the information server 400 may include information in which identification information is associated with medical information. As described above, the medical information may include a person's health insurance card information, medical information such as medical records, health checkup results, and information about medications taken. In addition to medical information, the information server 400 may also include information in which identification information is associated with family composition information indicating the person's family composition. The family composition information may be information indicating personal information about the family of the person corresponding to the identification information.

The on-site terminal 500 is, for example, a terminal owned by the person who is to rush to the scene. The on-site terminal 500 may be, for example, a personal computer. The on-site terminal 500 may also be a portable terminal such as a mobile phone, smartphone, or tablet terminal. For example, the on-site terminal 500 may be a terminal owned by an emergency team member, or may be a device installed in an ambulance. The on-site terminal 500 may also be a device installed in the medical institution to which the person is to be transported by ambulance.

[Details of the Report Support Device 101]
8 is a block diagram showing an example of the configuration of the reporting support system 1001. As shown in FIG. 8, the reporting support system 1001 includes a reporting support device 101 and a reporter terminal 200.

The reporting support device 101 comprises an acquisition unit 110, an extraction unit 120, an identification unit 130, an output unit 141, a reception unit 150, and a photographing request unit 160.

The extraction unit 120 extracts features related to the face of a person included in the captured image. The features related to the face of a person are referred to as first features.

The identification unit 130 identifies a person included in the captured image based on the first feature. Specifically, the identification unit 130 compares the first feature with the feature included in the database of the external device 300. If a match is found, the identification unit 130 identifies identification information associated with the matching feature. The method for identifying a person based on the feature related to the person's face may use an algorithm used for existing face recognition.

In the example database of Figure 7, the identification unit 130 may also identify the person's name and age. In other words, "identifying a person" may mean identifying the person's identification information or the person's personal information.

The output unit 141 includes a related information acquisition unit 1401 and an information output unit 1402. The related information acquisition unit 1401 acquires information associated with the identification information. Specifically, the related information acquisition unit 1401 acquires, from an information server, information associated with identification information similar to the identification information identified by the identification unit 130. For example, if the identified identification information is "001", the related information acquisition unit 1401 acquires medical information associated with the identification information "001". In this way, the related information acquisition unit 1401 can use the identification information identified by the identification unit 130 to acquire information associated with the identification information from another device other than the external device 300.

The information output unit 1402 outputs information. Specifically, it may output information identified by the identification unit 130, or it may output information acquired by the related information acquisition unit 1401. For example, the information output unit 1402 displays the medical information acquired by the related information acquisition unit 1401 on a display connected to the notification support device 101. At this time, the information output unit 1402 may also output the information identified by the identification unit 130 or the information acquired by the related information acquisition unit 1401 to the on-site terminal 500. In that case, for example, the personal information and medical information of the identified person is displayed on the display of the on-site terminal.

The information output unit 1402 may also output the acquired information by superimposing it on the captured image. Figure 9 is a diagram showing an example of an output image. In the example of Figure 9, the personal information and medical information of the injured person is superimposed on the captured image showing the injured person. In this way, the information output unit 1402 may output the output image.

[Example of operation of the reporting support system 1001]
Next, an example of the operation of the reporting support system 1001 will be described with reference to FIG.

Figure 10 is a sequence diagram illustrating an example of the operation of the reporting support system 1001. Note that the processing of S201 to S207 is similar to the processing of S101 to S107 in Figure 5, so the explanation will be omitted.

The extraction unit 120 extracts feature information (S208). Specifically, the extraction unit 120 extracts a first feature, which is a feature related to the person's face, from the captured image. The identification unit 130 identifies the person based on the feature information (S209). Specifically, the identification unit 130 identifies the person by comparing the first feature with a feature included in the database of the external device 300. The related information acquisition unit 1401 then acquires information associated with the identification information of the identified person (S210). For example, the related information acquisition unit 1401 acquires medical information about the identified person. The information output unit 1402 then outputs the information acquired by the related information acquisition unit 1401 (S211). For example, the information output unit 1402 outputs the medical information to a display or on-site terminal 500 connected to the reporting support device 101.

In this way, the reporting support device 101 of the third embodiment extracts a first feature amount related to the face of a person included in a captured image as feature information, identifies the person included in the captured image based on the first feature amount, and outputs information about the identified person. This allows the reporting support device 101 to provide information about the identified person to, for example, an operator. Therefore, the reporting support device 101 can support the operator in understanding the situation.

Furthermore, the reporting support device 101 of the third embodiment may specify identification information that identifies the specified person, acquire information associated with the identification information, and output the acquired information. Furthermore, the information associated with the identification information may be, for example, medical information about the specified person. For example, suppose there is a subject who needs to be transported by ambulance, such as an injured person. In this case, if the subject is photographed by the reporter terminal 200, the reporting support device 101 can output the subject's medical information. Therefore, the operator or emergency team can respond according to the subject's medical information, etc.

[Modification 5]
In the above embodiment, the external device 300 is described as having a database containing information in which at least feature amounts and identification information are associated with each other. In this case, the database may contain, instead of feature amounts, images containing a person's face associated with the identification information. In this case, the identification unit 130 may identify the person by comparing the first feature amount extracted by the extraction unit 120 with feature amounts extracted from images included in the database.

<Fourth embodiment>
Next, a fourth embodiment of the reporting support system 1002 will be described. In the fourth embodiment, a specific example of identifying attributes of a person appearing in a captured image will be described. Note that some of the content overlapping with the content described in the first to third embodiments will not be described.

Figure 11 is a diagram showing a schematic example of the configuration of a reporting support system 1002. As shown in Figure 11, the reporting support system 1002 comprises a reporting support device 102 and a reporter terminal 201. The reporting support device 102 is communicatively connected to the reporter terminal 200, an external device 300, a personal information server, and an on-site terminal 500. The reporter terminal 201 is also communicatively connected to a wearable terminal 600.

In this embodiment, an example is described in which, when an injured person is present, a caller other than the injured person makes an emergency call, but the content described in this embodiment is not limited to this example. In other words, the content described in this embodiment can be applied to various situations in which a caller makes a call.

The wearable terminal 600 is a device worn by a person. For example, the wearable terminal 600 may be a wristwatch-type device. A wristwatch-type wearable terminal 600 is also called, for example, a smart watch. The wearable terminal 600 measures the biometric information of the wearer. Examples of biometric information include, but are not limited to, blood pressure, heart rate, body temperature, and blood oxygen concentration. The wearable terminal 600 transmits the measured biometric information to the reporter terminal 201.

[Details of the reporter terminal 201]
12 is a block diagram showing an example of the configuration of the reporting support system 1002. As shown in FIG. 12 , the reporter terminal 201 includes a reporting unit 210, a request receiving unit 220, an imaging unit 231, a data transmitting unit 240, and a biometric information acquiring unit 250.

The photographing unit 231 photographs in the same manner as the photographing unit 230. In this case, the photographing unit 231 may have the function of a thermal camera. That is, the photographing unit 231 uses a sensor capable of detecting infrared rays to acquire temperature information based on the strength of the infrared rays. In this case, if the photographing unit 231 has a function to detect people from the photographed image, the photographing unit 231 may acquire temperature information corresponding to the detected people. The photographing unit 231 may generate a photographed image using the infrared rays detected by the sensor. For example, the photographing unit 231 may generate an image colored according to the strength of the infrared rays.

The biometric information acquisition unit 250 acquires biometric information from the wearable terminal 600. For example, the wearable terminal 600 is worn by an injured person. For example, the biometric information acquisition unit 250 transmits a request to measure biometric information to the wearable terminal 600. Upon receiving the measurement request, the wearable terminal 600 measures the biometric information. Then, the wearable terminal 600 transmits the measured biometric information to the reporter terminal 201. The biometric information acquisition unit 250 acquires the biometric information transmitted from the wearable terminal 600.

The data transmission unit 240 transmits the captured image generated by the imaging unit 231 to the reporting support device 102. At this time, the data transmission unit 240 may also transmit temperature information acquired by the imaging unit 231, or may also transmit an image captured using infrared rays. The data transmission unit 240 may also transmit biometric information acquired by the biometric information acquisition unit 250. Here, if temperature information corresponding to a person has been acquired by the imaging unit 231, the data transmission unit 240 may transmit the temperature information as biometric information.

[Details of the report support device 102]
The reporting support device 102 includes an acquisition unit 111 , an extraction unit 120 , an identification unit 130 , an output unit 141 , a reception unit 150 , and an imaging request unit 160 .

The acquisition unit 111 acquires captured images in the same manner as the acquisition unit 110. The acquisition unit 111 also acquires biometric information from the reporter terminal 201. If an image captured using infrared light is also transmitted from the reporter terminal 201, the acquisition unit 111 may also acquire the captured image.

The extraction unit 120 extracts features related to the appearance of a person included in a captured image. The features related to the person's appearance are referred to as second features.

The identification unit 130 identifies the attributes of the person included in the captured image based on the second feature. Here, attributes include, but are not limited to, the person's age, gender, and height. For example, the attributes may include the person's injury status. The injury status includes the location of the person's external injury, whether or not the person is bleeding, and whether or not the person is conscious. The identification unit 130 may identify the person's attributes using a learning model. In this case, a learning model that has learned the relationship between the feature values related to the person's appearance and the person's attributes is stored in the external device 300. That is, the learning model has learned, for example, the relationship between the feature values related to the person's appearance and the person's injury status. The learning model is, for example, a model that receives the feature values related to the person's appearance as input and outputs the person's attributes. The identification unit 130 then inputs the second feature values extracted by the extraction unit 120 into the learning model. The identification unit 130 identifies the attributes output for the second feature values as the attributes of the person included in the captured image.

The information output unit 1402 outputs the attributes and biometric information of the identified person. For example, the information output unit 1402 displays information indicating the attributes of the identified person and the biometric information on a display connected to the reporting support device 102. At this time, the information output unit 1402 may also output the information indicating the attributes of the identified person and the biometric information to the on-site terminal 500. In that case, for example, the information indicating the attributes of the identified person and the biometric information are displayed on the display of the on-site terminal.

The information output unit 1402 may also output information indicating the attributes of the identified person and biometric information superimposed on the captured image. Figure 13 is a diagram showing an example of an output image. In the example of Figure 13, information indicating the attributes of the injured person and biometric information are superimposed on a captured image showing the injured person.

[Example of operation of the reporting support system 1002]
Next, an example of the operation of the reporting support system 1002 will be described with reference to FIG.

Figure 14 is a sequence diagram illustrating an example of the operation of the reporting support system 1002. Note that the processing of S301 to S304 is similar to the processing of S101 to S104 in Figure 5, so the explanation will be omitted.

The photographing unit 231 photographs the subject in response to a photographing request received by the request receiving unit 220 (S305). At this time, the photographing unit 231 may use a sensor capable of detecting infrared rays to obtain temperature information based on the strength of the infrared rays, or may generate a photographed image using infrared rays.

The biometric information acquisition unit 250 sends a measurement request to the wearable terminal 600 (S306). Then, the biometric information acquisition unit 250 acquires biometric information from the wearable terminal 600 (S307). The data transmission unit 240 transmits the captured image and the biometric information to the reporting support device 102 (S308).

The acquisition unit 111 of the reporting support device 102 acquires the captured image and biometric information (S309). The extraction unit 120 extracts feature information from the captured image (S310). Specifically, the extraction unit 120 extracts a second feature amount, which is a feature amount related to the person's appearance, from the captured image. The identification unit 130 identifies the person's attributes based on the feature information (S311). The information output unit 1402 outputs various information (S312). For example, the information output unit 1402 outputs information indicating the identified attributes and biometric information.

In this way, the reporting support device 102 of the fourth embodiment extracts the second feature amount related to the appearance of the person included in the captured image as feature information, identifies the attributes of the person included in the captured image based on the second feature amount, and outputs information indicating the identified attributes. This allows the reporting support device 102 to provide, for example, information indicating the attributes of the identified person to an operator. Therefore, the reporting support device 101 can assist the operator in understanding the situation.

In addition, the reporting support device 102 of the fourth embodiment acquires biometric information acquired at the reporter terminal and outputs the biometric information as information about the person included in the captured image. This allows the reporting support device 102 to provide the operator with biometric information about the person appearing in the captured image.

[Modification 6]
The wearable terminal 600 may transmit the biometric information to the reporting support device 102. That is, the reporting support device 102 may acquire the biometric information from the wearable terminal 600. In this case, the reporter terminal 201 may include, in advance, information on the data transmission destination (i.e., information for transmitting data to the reporting support device 102) in the measurement request.

Fifth Embodiment
Next, a reporting support system 1003 according to a fifth embodiment will be described. In the fifth embodiment, a specific example of identifying a feature object appearing in a captured image will be described. Note that some of the content overlapping with the content described in the first to fourth embodiments will not be described.

The reporting support system 1003 of the fifth embodiment includes a reporting support device 103 instead of the reporting support device 102 shown in FIG. 11. That is, the reporting support system 1003 includes the reporting support device 103 and a reporter terminal 201.

[Details of the report support device 103]
15 is a block diagram showing an example of the configuration of the reporting support system 1003. As shown in FIG. 15 , the reporting support device 103 includes an acquiring unit 111, an extracting unit 120, an identifying unit 130, a receiving unit 150, and an imaging requesting unit 160.

The extraction unit 120 extracts feature quantities related to the appearance of characteristic objects included in the captured image. The feature quantities related to the appearance of characteristic objects are referred to as third feature quantities. Here, an example of identifying an object based on the third feature quantities will be described. In the following identification example, the captured image shown in Figure 13 will be described as having been acquired by the acquisition unit 111 from the reporter terminal 201. Figure 16 is a diagram showing an example of a captured image. In the example of Figure 16, a building called Supermarket ABC is included in the captured image.

[Specific Example 1]
In specific example 1, the extraction unit 120 extracts feature quantities related to edges, corners, brightness gradients, etc. of the captured image as third feature quantities. In this case, the external device 300 includes an image database. The image database includes information in which images including feature objects are associated with information about the feature objects. The information about the feature objects is, for example, the name and location information of the feature objects. For example, the image database includes an image of Supermarket ABC and information in which information about the name "Supermarket ABC" is associated with location information "Kawasaki Ward, Kawasaki City, Kanagawa Prefecture."

The identification unit 130 uses the extracted third feature to perform an image search in the image database. Specifically, the identification unit 130 compares the third feature with the feature of the image contained in the image database. If the comparison is a match (i.e., if the search is a hit), the identification unit 130 identifies information about the feature associated with the matching feature. For example, the identification unit 130 identifies information about the name "Supermarket ABC" and location information of "Kawasaki Ward, Kawasaki City, Kanagawa Prefecture."

The related information acquisition unit 1403 acquires information about the identified feature. The information output unit 1402 then outputs information about the identified feature. For example, the information output unit 1402 displays the name "Supermarket ABC" and the location information "Kawasaki Ward, Kawasaki City, Kanagawa Prefecture" on a display connected to the reporting support device 103.

[Specific Example 2]
In specific example 2, the extraction unit 120 extracts text information from the captured image as the third feature. In the example of Fig. 16, the captured image includes the text "Supermarket ABC Kawasaki Ekimae Store." The extraction unit 120 extracts, for example, the text information "Supermarket ABC Kawasaki Ekimae Store" as the third feature.

The identification unit 130 performs a search using the extracted third feature. For example, the identification unit 130 searches a search engine for the text information "Supermarket ABC Kawasaki Ekimae Store." The identification unit 130 then identifies the search results.

The related information acquisition unit 1403 acquires information contained in the search results. For example, if the search results include location information, the related information acquisition unit 1403 acquires the location information.

The information output unit 1402 may output the search results or may output the information acquired by the related information acquisition unit 1403.

[Example of operation of the reporting support system 1003]
Next, an example of the operation of the reporting support system 1003 will be described with reference to FIG.

Figure 17 is a sequence diagram illustrating an example of the operation of the reporting support system 1003. Note that the processing of S401 to S407 is similar to the processing of S101 to S107 in Figure 5, so description thereof will be omitted.

The extraction unit 120 extracts feature information (S408). Specifically, the extraction unit 120 extracts a third feature amount, which is a feature amount related to the appearance of the feature, from the captured image. The identification unit 130 identifies the feature based on the feature information (S409). For example, the identification unit 130 uses the third feature amount to perform an image search in the image database of the external device 300. If the search returns a hit, the identification unit 130 identifies information related to the feature.

The related information acquisition unit 1403 acquires information about the identified feature (S410). Then, the information output unit 1402 outputs the acquired information about the feature (S411).

In this way, the reporting support device 103 of the fifth embodiment extracts a third feature amount related to the appearance of a characteristic object contained in a captured image as characteristic information, identifies the characteristic object contained in the captured image based on the third feature amount, and outputs information related to the identified characteristic object. This allows the reporting support device 103 to provide information about the identified characteristic object to, for example, an operator. When a report is made, the operator is required to identify the location of the caller. However, there is a possibility that the caller may not know their own location or may panic and be unable to determine their location. In response to this, the reporting support device 103 can output information related to characteristic objects contained in captured images around the caller, allowing the operator to identify their location using the output information. In other words, the reporting support device 101 can assist the operator in understanding the situation.

In addition, the information about the identified feature may be location information, which allows the operator to more quickly determine the location of the caller.

[Modification 7]
In each of the above embodiments, the reporting support device may control the display of shooting support information on the shooting screen of the reporter terminal. The shooting support information is information indicating the object to be photographed. FIG. 18 is a diagram illustrating an example of a shooting screen. The shooting request unit 160 superimposes the shooting support information on the shooting screen. In the example of FIG. 18 , when an injured person is present, the message "Please take a photo so that the entire injured person is visible" is displayed on the screen of the reporter terminal as shooting support information. Furthermore, for example, the shooting request unit 160 may control the display of the message "Please take a photo so that surrounding signs are visible" on the screen of the reporter terminal. This allows the reporter to properly photograph the object to be photographed.

<Example of hardware configuration of reporting support device>
The hardware constituting the reporting support device of the first, second, third, fourth, and fifth embodiments will be described below. Fig. 19 is a block diagram showing an example of the hardware configuration of a computer device that realizes the reporting support system in each embodiment. The reporting support device and reporting support method described in each embodiment and each modified example are realized in a computer device 90.

As shown in Figure 19, the computer device 90 includes a processor 91, a RAM (Random Access Memory) 92, a ROM (Read Only Memory) 93, a storage device 94, an input/output interface 95, a bus 96, and a drive device 97. The reporting support device may be realized by multiple electrical circuits.

The storage device 94 stores a program (computer program) 98. The processor 91 executes the program 98 of the reporting support device using the RAM 92. Specifically, for example, the program 98 includes a program that causes a computer to execute the processes shown in Figures 3, 5, 10, 14, and 17. The functions of each component of the reporting support device are realized in response to the processor 91 executing the program 98. The program 98 may be stored in the ROM 93. The program 98 may also be recorded on the storage medium 80 and read out using the drive device 97, or may be transmitted to the computer device 90 from an external device (not shown) via a network (not shown).

The input/output interface 95 exchanges data with peripheral devices (keyboard, mouse, display device, etc.) 99. The input/output interface 95 functions as a means for acquiring or outputting data. The bus 96 connects each component.

There are various variations in how the reporting support device can be realized. For example, the reporting support device can be realized as a dedicated device. Also, the reporting support device can be realized based on a combination of multiple devices.

The scope of each embodiment also includes a processing method in which a program for realizing each component of the function of each embodiment is recorded on a storage medium, the program recorded on the storage medium is read as code, and the program is executed on a computer. In other words, a computer-readable storage medium is also included in the scope of each embodiment. Furthermore, the storage medium on which the above-mentioned program is recorded and the program itself are also included in each embodiment.

The storage medium may be, for example, a floppy disk, hard disk, optical disk, magneto-optical disk, CD (Compact Disc)-ROM, magnetic tape, non-volatile memory card, or ROM, but is not limited to these examples. Furthermore, the programs recorded on the storage medium are not limited to standalone programs that execute processes; programs that run on an OS (Operating System) in conjunction with other software or expansion board functions and execute processes are also included within the scope of each embodiment.

The present invention has been described above with reference to embodiments, but the present invention is not limited to the above embodiments. Various modifications that would be understood by a person skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

The above embodiments and variations can be combined as appropriate.

Some or all of the above embodiments may also be described as, but are not limited to, the following notes:

<Additional Notes>
[Appendix 1]
an acquisition means for acquiring a captured image from a reporter terminal, which is a terminal owned by the reporter;
extraction means for extracting feature information relating to an object included in the photographed image;
an identification means for identifying the object based on the extracted feature information;
and an output means for outputting information about the identified object.
Report support device.

[Appendix 2]
the extraction means extracts a first feature amount related to a face of a person included in the photographed image as the feature information;
the identifying means identifies a person included in the captured image based on the first feature amount;
the output means outputs information about the identified person.
2. A reporting support device as described in appendix 1.

[Appendix 3]
The specifying means specifies identification information that identifies the specified person; and the output means acquires information associated with the identification information and outputs the acquired information.
3. A reporting support device as described in appendix 2.

[Appendix 4]
the information associated with the identification information is medical information of the identified person;
4. A reporting support device as described in appendix 3.

[Appendix 5]
the extraction means extracts a second feature amount relating to the appearance of a person included in the photographed image as the feature information;
the identifying means identifies an attribute of the person included in the captured image based on the second feature amount;
the output means outputs information indicating the identified attribute.
5. A notification support device according to any one of appendices 1 to 4.

[Appendix 6]
the acquiring means acquires the biometric information acquired at the reporter terminal,
the output means outputs the biometric information as information about the person included in the photographed image.
6. A reporting support device according to any one of appendixes 1 to 5.

[Appendix 7]
the biological information is information measured by a wearable device capable of communicating with the reporter device, and is at least one of the person's blood pressure, heart rate, body temperature, and blood oxygen concentration;
7. A reporting support device as described in appendix 6.

[Appendix 8]
the extraction means extracts a third feature amount relating to an appearance of a feature object included in the photographed image as the feature information;
the specifying means specifies position information of the feature based on the third feature amount;
The output means outputs the position information.
8. A reporting support device according to any one of appendices 1 to 7.

[Appendix 9]
The captured image is acquired from the reporter's terminal, which is a terminal owned by the reporter,
extracting feature information about an object included in the captured image;
Identifying the object based on the extracted feature information;
outputting information about the identified object;
How to support reporting.

[Supplementary Note 10]
A process of acquiring a captured image from a reporter terminal, which is a terminal owned by the reporter;
A process of extracting feature information about an object included in the captured image;
A process of identifying the object based on the extracted feature information;
and a process of outputting information about the identified object.
A computer-readable storage medium.

100, 101, 102, 103 Reporting support device 110, 111 Acquisition unit 120 Extraction unit 130 Identification unit 140, 141, 142 Output unit 150 Reception unit 160 Photography request unit 200, 201 Reporter terminal 210 Reporting unit 220 Request receiving unit 230, 231 Photography unit 240 Data transmission unit 250 Biometric information acquisition unit 1000, 1001, 1002, 1003 Reporting support system

Claims (10)

an acquisition means for acquiring a captured image from a reporter terminal, which is a terminal owned by the reporter;
consent information acquisition means for acquiring information indicating whether or not consent is given to the use of feature information relating to a predetermined object from the reporter terminal;
an extraction means for extracting the feature information relating to an object included in the photographed image;
an identification means for identifying the object based on the extracted feature information;
an output means for outputting information about the identified object ,
If consent is not obtained for the use of the characteristic information,
the extraction means extracts the feature information from an area of the photographed image excluding an area corresponding to the predetermined object for which consent was not obtained;
the identifying means identifies the object included in an area excluding an area corresponding to the predetermined object based on the extracted feature information.
Report support device. the consent information acquisition means acquires information indicating whether or not consent has been given to the use of the characteristic information relating to the person;
If consent is obtained for the use of the characteristic information,
the extraction means extracts a first feature amount related to a face of a person included in the photographed image as the feature information;
the identifying means identifies a person included in the captured image based on the first feature amount;
the output means outputs information about the identified person.
The notification support device according to claim 1 . The identification means identifies identification information that identifies the identified person,
the output means acquires information associated with the identification information and outputs the acquired information.
The notification support device according to claim 2 . the information associated with the identification information is medical information of the identified person;
The notification support device according to claim 3 . the consent information acquisition means acquires information indicating whether or not consent has been given to the use of the characteristic information relating to the person;
If consent is obtained for the use of the characteristic information,
the extraction means extracts a second feature amount relating to the appearance of a person included in the photographed image as the feature information;
the identifying means identifies an attribute of the person included in the captured image based on the second feature amount;
the output means outputs information indicating the identified attribute.
The notification support device according to any one of claims 1 to 4. the acquiring means acquires the biometric information acquired at the reporter terminal,
the output means outputs the biometric information as information about the person included in the photographed image.
The notification support device according to any one of claims 1 to 5. the biological information is information measured by a wearable device capable of communicating with the reporter device, and is at least one of the person's blood pressure, heart rate, body temperature, and blood oxygen concentration;
The notification support device according to claim 6. the extraction means extracts a third feature amount relating to an appearance of a feature object included in the photographed image as the feature information;
the specifying means specifies position information of the feature based on the third feature amount;
The output means outputs the position information.
The notification support device according to any one of claims 1 to 7. The captured image is acquired from the reporter's terminal, which is a terminal owned by the reporter,
acquiring information indicating whether or not there is consent to the use of feature information relating to a predetermined object from the reporter terminal;
extracting the feature information relating to an object included in the captured image;
Identifying the object based on the extracted feature information;
outputting information about the identified object;
If consent is not obtained for the use of the characteristic information,
In the extracting step, the feature information is extracted from an area of the photographed image excluding an area corresponding to the predetermined object for which consent was not obtained;
In the step of identifying, the object included in an area excluding an area corresponding to the predetermined object is identified based on the extracted feature information.
How to support reporting. A process of acquiring a captured image from a reporter terminal, which is a terminal owned by the reporter;
A process of acquiring information indicating whether or not consent has been given to the use of feature information relating to a predetermined object from the reporter terminal;
A process of extracting the feature information regarding an object included in the captured image;
A process of identifying the object based on the extracted feature information;
outputting information about the identified object;
on the computer ,
If consent is not obtained for the use of the characteristic information,
In the extraction process, the feature information is extracted from an area of the photographed image excluding an area corresponding to the predetermined object for which consent was not obtained;
In the identifying process, the object included in an area excluding an area corresponding to the predetermined object is identified based on the extracted feature information.
program.