JP7652194B2 - Information processing device, information processing method, and program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and a program, and in particular to an information processing device, an information processing method, and a program for monitoring the occurrence of events such as crimes.

Patent Document 1 describes a system that uses images captured by an imaging device mounted on a vehicle to detect incidents such as aggressive driving and driving too close to the side of a vehicle, determines the level of the detected incident, and transmits information according to the determined level to a communication device. It also describes that events that occur in an area can be saved as mapping information, which can be used to track perpetrators and predict escape routes.

International Publication No. 2020/031924

The system described in Patent Document 1 stores events that occur in an area as mapping information, and when a similar event occurs, tracks the perpetrator and predicts his or her escape route. However, unless the perpetrator causes a similar event in the same area, it is not possible to track the perpetrator or predict his or her escape route.

The present invention was made in consideration of the above circumstances, and its purpose is to be able to accurately estimate a criminal's escape route.

In each aspect of the present invention, the following configurations are adopted to solve the above-mentioned problems.

The first aspect relates to an information processing device.
An information processing device according to a first aspect includes:
an event information generating means for detecting an occurrence of an event by processing an image and generating event information relating to the event;
The vehicle has a route generating means for generating an estimated travel route using a difference between the generated event information and predefined expected event information for the event.

A second aspect relates to an information processing method implemented by at least one computer.
An information processing method according to a second aspect of the present invention comprises:
An information processing device,
Processing the images to detect the occurrence of an event and generate event information regarding the event;
The method includes generating an estimated travel route using a difference between the generated event information and expected event information that is predetermined for the event.

Another aspect of the present invention may be a program for causing at least one computer to execute the method of the second aspect, or a computer-readable recording medium having such a program recorded thereon. This recording medium includes a non-transitory tangible medium.
The computer program includes computer program code which, when executed by a computer, causes the computer to carry out the information processing method on an information processing device.

In addition, any combination of the above components, and any conversion of the expression of the present invention between a method, device, system, recording medium, computer program, etc., are also valid aspects of the present invention.

Furthermore, the various components of the present invention do not necessarily have to be independent entities, but may be formed as a single member by multiple components, one component may be formed from multiple components, one component may be part of another component, or part of one component may overlap with part of another component, etc.

In addition, although the method and computer program of the present invention describe a number of steps in a sequential order, the order does not limit the order in which the steps are executed. Therefore, when implementing the method and computer program of the present invention, the order of the steps can be changed as long as it does not cause any problems in terms of the content.

Furthermore, the steps of the method and computer program of the present invention are not limited to being executed at different times. Thus, a step may occur while another step is being executed, or the execution timing of a step may overlap in part or in whole with the execution timing of another step, etc.

According to each of the above aspects, it is possible to provide technology that can accurately estimate a criminal's escape route.

1 is a diagram illustrating an overview of a monitoring system according to an embodiment. 2 is a block diagram illustrating an example of a hardware configuration of a computer that realizes an information processing device and an image processing device in the monitoring system shown in FIG. 1 . 1 is a functional block diagram illustrating an example of a logical configuration of an information processing apparatus according to an embodiment. FIG. 2 is a diagram for explaining data stored in a storage device. 1 is a flowchart illustrating an example of the operation of the information processing apparatus according to the embodiment. 1 is a flowchart illustrating an example of the operation of the information processing apparatus according to the embodiment. 1 is a functional block diagram illustrating an example of a logical configuration of an information processing apparatus according to an embodiment.

The following describes an embodiment of the present invention with reference to the drawings. In all drawings, similar components are given similar reference symbols and descriptions are omitted where appropriate. In addition, in each of the following drawings, configurations of parts that are not related to the essence of the present invention are omitted and are not shown.

In the embodiments, "acquisition" includes at least one of the following: the device goes to retrieve data or information stored in another device or storage medium (active acquisition), and the device inputs data or information output from another device (passive acquisition). Examples of active acquisition include making a request or inquiry to another device and receiving a reply, and accessing and reading information from another device or storage medium. Examples of passive acquisition include receiving information that is distributed (or transmitted, push notification, etc.). Furthermore, "acquisition" may mean selecting and acquiring data or information from received data or information, or selecting and receiving distributed data or information.

First Embodiment
<System Overview>
1 is a diagram showing a schematic overview of a surveillance system 1 according to an embodiment. The surveillance system 1 includes an information processing device 100, an image processing device 200, and at least one camera 5. In the surveillance system 1, for example, the image processing device 200 processes images captured by the camera 5 installed in a city to recognize people and the like, and the information processing device 100 detects the occurrence of an event (for example, a criminal act, etc.) using the processing result of the image processing device 200 and estimates the escape route of a criminal (person 10).

The camera 5 may be specialized for the monitoring system 1, or may utilize a camera that has been installed previously. Alternatively, the camera 5 may be an in-vehicle camera. Furthermore, the monitoring system 1 may further include a sensor 7 such as an odor intensity sensor that detects odors, a temperature sensor, a smoke detector, a gas leak detector, or a sensor that detects the pressing of a report button. The sensor 7 is not limited to these, and any sensor 7 that acquires information that can be used to identify the event to be detected may be used. Furthermore, the camera 5 may include a microphone (not shown), and the monitoring system 1 may include a microphone (not shown) installed separately from the camera 5. Furthermore, the monitoring system 1 includes a storage device 300.

The storage device 300 may be a device included within at least one of the information processing device 100 and the image processing device 200, or may be a device separate from the information processing device 100 and the image processing device 200, or may be a combination of these.

The camera 5 generates a captured image. The camera 5 includes an imaging element such as a lens and a CCD (Charge Coupled Device) image sensor. The camera 5 may include a mechanism for controlling the orientation of the camera body and lens, zoom control, focusing, etc., by tracking the movement of a person entering the field of view.

It is preferable that the camera 5 captures an image of an area including a location where the event that the information processing device 100 is detecting may occur. The camera 5 may also capture an image of a date, time, or period during which the event that the information processing device 100 is detecting is likely to occur.

For example, the camera 5 may capture images of areas including streets, roads, passageways, vacant lots, back alleys, buildings and premises such as stores and facilities, specific floors within buildings, and areas around store cash registers in times and areas where crimes are common (e.g., places with little foot traffic, areas with poor security, tourist destinations, etc.). Alternatively, at venues such as crowded events and festivals, the number of cameras 5, the locations where they are captured, and the capture range may be changed to suit the date and time of the event.

The image generated by the camera 5 is transmitted to the information processing device 100 or the image processing device 200 via the communication network 3. The image may be transmitted from the camera 5 to the information processing device 100 and then transmitted from the information processing device 100 to the image processing device 200 for image processing, or may be transmitted from the camera 5 to the image processing device 200 and the result of image processing transmitted from the image processing device 200 to the information processing device 100, or a combination of these.

The connection method between the camera 5 or sensor 7 and the information processing device 100 or image processing device 200 may be wireless or wired. In the case of a wireless connection, the camera 5 or sensor 7, the information processing device 100, and the image processing device 200 each have a wireless communication function. The camera 5 may be a network camera such as an IP (Internet Protocol) camera.

The images generated by the camera 5 can be used in the following ways: (a1) learning the normal state of the shooting location, (a2) detecting the occurrence of an event, and (a3) learning about the event detected in (a2).

For example, when an image is used to detect the occurrence of the event (a2), the image is preferably transmitted to the information processing device 100 or image processing device 200 in real time. However, the image transmitted to the information processing device 100 or image processing device 200 does not have to be transmitted immediately from the camera 5, and may be an image delayed by a predetermined time. The image generated by the camera 5 may be temporarily stored in a storage device (which may be the storage device 300 or another storage device (including a recording medium)), and the information processing device 100 or image processing device 200 may read the image from the storage device sequentially or at predetermined intervals. Furthermore, the image transmitted to the information processing device 100 or image processing device 200 is preferably a moving image, but may be a frame image at a predetermined interval or a still image.

On the other hand, when an image is used to learn the occurrence of an event (a3) or to learn the normal state of the area (a1), the image does not have to be transmitted to the information processing device 100 or the image processing device 200 in real time, and after it is once stored in a storage device (including a recording medium), the information processing device 100 or the image processing device 200 may read and process the image from the storage device. When an image is used to learn the normal state of the area (a1), it is preferable to also take into account environmental information such as seasonal factors. The environmental information may include, for example, forecast information on bad weather in the area (especially rain, snow, fog, typhoons, squalls, etc.), information indicating the damage caused by natural disasters, sunset and sunrise times, information indicating the mass outbreak of insects, and information indicating the occurrence of fires (including forest fires, etc.).

For example, it may be possible to learn images corresponding to times when crimes are likely to occur (such as at night), images taken at different times of the day, distinguishing between weekdays and holidays (or by day of the week), and images corresponding to locations and dates, times, or periods of time that correspond to specific occasions such as events, festivals, rallies, and other such occasions.

Furthermore, after an event is detected, the multiple cameras 5 may be switched to capture images to track the movement of a person related to the event (i.e., a criminal). Alternatively, the movement of a person related to the event may be tracked using images captured by the multiple cameras 5. The movement of a person related to the event may also be tracked using images captured by the multiple cameras 5 that exist along an estimated movement path generated by the path generation unit 104, which will be described later. The person related to the event may include the victim, and since an accomplice may be pretending to be the victim, for example, all people related to the event who were at that location may be targeted for tracking.

<Hardware configuration example>
FIG. 2 is a block diagram illustrating an example of a hardware configuration of a computer 1000 that realizes the information processing device 100 and the image processing device 200 of the monitoring system 1 shown in FIG.

The computer 1000 has a bus 1010, a processor 1020, a memory 1030, a storage device 1040, an input/output interface 1050, and a network interface 1060.

The bus 1010 is a data transmission path for the processor 1020, memory 1030, storage device 1040, input/output interface 1050, and network interface 1060 to transmit and receive data to and from each other. However, the method of connecting the processor 1020 and the like to each other is not limited to a bus connection.

Processor 1020 is a processor realized by a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) etc.

Memory 1030 is a main storage device realized by RAM (Random Access Memory) or the like.

The storage device 1040 is an auxiliary storage device realized by a hard disk drive (HDD), a solid state drive (SSD), a memory card, or a read only memory (ROM). The storage device 1040 stores program modules that realize each function of the information processing device 100 of the monitoring system 1 (for example, the event information generating unit 102 and the path generating unit 104 in FIG. 3, which will be described later, or the output processing unit 106 in FIG. 7, etc.). The processor 1020 reads each of these program modules into the memory 1030 and executes them, thereby realizing each function corresponding to the program module. The storage device 1040 also functions as a storage unit (not shown) that stores various information used by the information processing device 100 or the image processing device 200. The storage device 300 may also be realized by the storage device 1040.

The program module may be recorded on a recording medium. The recording medium on which the program module is recorded may include a non-transitory, tangible medium usable by the computer 1000, and the program code readable by the computer 1000 (processor 1020) may be embedded in the medium.

The input/output interface 1050 is an interface for connecting the computer 1000 to various input/output devices.

The network interface 1060 is an interface for connecting the computer 1000 to the communication network 3. This communication network 3 is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). The method for connecting the network interface 1060 to the communication network 3 may be a wireless connection or a wired connection. However, the network interface 1060 may not be used.

The computer 1000 then connects to necessary equipment (e.g., a camera 5, a sensor 7, a microphone (not shown), a display (not shown), a speaker (not shown), etc.) via the input/output interface 1050 or the network interface 1060.

The monitoring system 1 is realized by a combination of an information processing device 100 and an image processing device 200, and is therefore realized by multiple computers 1000 that constitute each of them. The information processing device 100 is, for example, a server computer. The image processing device 200 may be a device separate from the information processing device 100, may be a device included inside the information processing device 100, or may be a combination of these.

Each component of the information processing device 100 of this embodiment shown in FIG. 3, which will be described later, is realized by any combination of hardware and software of the computer 1000 shown in FIG. 2. Those skilled in the art will understand that there are various variations in the realization method and device. The functional block diagram showing the information processing device 100 of each embodiment shows blocks of logical functional units, rather than a hardware-based configuration.

<Functional configuration example>
3, the information processing device 100 in FIG. 1 includes an event information generating unit 102 and a path generating unit 104.
The event information generating unit 102 detects the occurrence of an event by processing the image, and generates event information related to the event. The path generating unit 104 generates an estimated movement path using the difference between the generated event information and expected event information that is predetermined for the event.

Image processing is performed by the image processing device 200. The image processing device 200 analyzes the image using the image recognition model 340 described below, recognizes people, vehicles, etc., recognizes the actions of people (the actions of the criminal) and the surrounding situation (the damage situation), and identifies the events occurring in the image.

Examples of specified events include crimes and incidents. Examples of specified events include theft, robbery, pickpocketing, shoplifting, blackmail, fights, violence, assault, injury or injury resulting in death, kidnapping, bombing, murder, random street attacks, terrorism, hostage situations, arson, traffic accidents (hit-and-run, hit-and-run, reckless driving, etc.), destruction of structures (vandalism), etc.

1, 3, and FIG. 7 described later, the information processing device 100 and the image processing device 200 are each configured with one piece of hardware (e.g., a server). However, the information processing device 100 and the image processing device 200 may each be configured with multiple pieces of hardware (e.g., multiple servers). For example, the image processing device 200 and the event information generation unit 102 of the information processing device 100 may be realized by a first server, the path generation unit 104 may be realized by a second server, and the output processing unit 106 may be realized by a third server. Also, the image processing device 200, the event information generation unit 102 of the information processing device 100, and the path generation unit 104 may be realized by a first server, and the output processing unit 106 may be realized by a second server.

Figure 4 is a diagram for explaining the data stored in storage device 300. Storage device 300 includes a crime database 310, an area database 320, a map database 330, an image recognition model 340, and a crime model 350. Here, the events to be detected are described as being limited to crimes, but may include events other than crimes.

The crime database 310 stores information about crimes, classified by the type of specified event (crime) mentioned above, including information such as the location of the crime scene, the number of perpetrators, clothing, ages, time taken to commit the crime, the murder weapon, the perpetrator's possessions, whether there was a victim, information about the victim, the perpetrator's actions at the time of the crime, the perpetrator's interactions with the victim, sounds (e.g. explosions) and smells generated during the crime, etc.

The crime database 310 may be constructed by machine learning using past crime footage, etc. Furthermore, if a criminal act is captured in an image generated by the camera 5, the crime database 310 may register information relating to the crime obtained by machine learning using the image. Furthermore, the crime database 310 may be provided for each region. By providing a crime database for each region, it becomes possible to accurately identify crimes specific to the region.

For example, in a robbery crime targeting tourists using a firearm, a criminal on a motorcycle or vehicle threatens the victim (tourist) with the firearm, forces them to remove their bag and valuables, steals them, and then escapes on the motorcycle or vehicle. The crime takes about 30 seconds. Information such as the actions of the criminals when they get off the vehicle, the number of criminals, the respective positions of the criminals and the victim, the time they speak, how they draw their weapon, and the time they draw their weapon may be analyzed from the image and registered in the crime database 310.

The area database 320 stores information indicating normal conditions by area or building. Information indicating normal conditions by area may include, for example, the gender, age, and physical build (height and body type (slim, tall, small, large, obese, etc.)) of people in the area (people living their daily lives, people who are here to work, people visiting for a short period of time for tourism or business, etc.), information on vehicles traveling through the area or parked vehicles (such as information indicating the area of the vehicle model, color, and license plate number), or noise information and odor level information for the area.

The image processing device 200 collects images captured by the camera 5 in advance for each predetermined area and building at a predetermined timing, recognizes and learns information indicating normal conditions for each predetermined area and building, and stores the information in the area database 320. The predetermined timing for collecting images is, for example, periodically, and the shooting time period, shooting cycle, shooting day, etc. may be changed depending on the shooting location.

The image processing device 200 recognizes objects such as people, vehicles, and garbage contained in the captured image, extracts features, and learns about the recognized people and vehicles. For example, for people, the image processing device 200 learns the age (generation), gender, clothing (appearance (light clothing, formal clothing, heavy equipment, dirty, barefoot, etc.)), accessories, hairstyle, hair color, physique (bones), etc., and registers them in the area database 320 in association with the area or building. For vehicles, the image processing device 200 learns the model, regional information written on the license plate, vehicle color, window color (smoke), number of passengers, etc., and registers them in the area database 320 in association with the area or building.

Furthermore, audio may be collected by area or building from the microphone (not shown) of camera 5, and audio analysis may be performed using an audio processing device (not shown) to learn language, noise level, emergency vehicle alert sounds, emergency bell sounds, in-house announcements, chimes, etc. Odor intensity information may be acquired by area or building from the odor intensity sensor and learned. The learned audio information and odor intensity information are registered in the area database 320 in association with the area or building.

The map database 330 is used to generate an estimated route of movement, and is also used when displaying the generated estimated route of movement on a map, as described in the embodiment described below. The map information includes information on roads, passageways, means of transportation, and the like. The map information may also include, for example, information on areas where neither the police nor the military have jurisdiction if a criminal escapes. Since it is preferable to arrest a criminal before he or she escapes into such an area, when displaying the estimated route of movement, it is preferable to also show the area in question as a warning.

The image recognition model 340 is used by the image processing device 200 to recognize people and vehicles from images. The image recognition model 340 stores various person and vehicle characteristic information. The image recognition model 340 may be provided for each area. By providing the image recognition model for each area, image recognition can be performed with high accuracy.

Similar to the area database 320, the image processing device 200 may collect images captured by the camera 5 in advance for each predetermined area or building at a predetermined timing, recognize people, vehicles, etc. from the images, and register their features in the image recognition model 340. The predetermined timing for collecting images may be, for example, periodically or at any timing, and the time period, shooting cycle, shooting day, etc. may be changed depending on the shooting location.

The crime model 350 stores anticipated event information. The anticipated event information includes, for each event, information such as the time it takes to commit the crime, the type of money and valuables stolen and the scale of the damage, whether or not a weapon is present, the clothing (equipment) of the criminal, the age and sex of the criminal, the number of criminals involved in the crime, and how the criminals interacted with the victim.

Returning to FIG. 3, the image processing device 200 recognizes people, vehicles, objects, etc. contained in the captured image based on the normal conditions learned for each specified area and each specified building, and the event information generation unit 102 uses the image recognition results to compare them with the crime database 310 and detect the occurrence of a specified event. In addition, the type of the detected event can be identified using the crime database 310.

A method for identifying a specific event based on the image recognition results obtained by image analysis using the image processing device 200 may include, for example, determining what type of event (such as a type of crime) the event is based on specific conditions. Examples of event determination conditions are shown below.

For example, to determine that an event is a "robbery", the following determination conditions may be used:
(b1) A murder weapon (a gun, knife, etc. have been learned) is detected from an image, and one person, Person A, is pointing the murder weapon at another person, Person B.
In the image (b2), person B has one or both hands raised.
(b3) In the image, person B or another person C who is thought to be an accomplice is stealing money or goods, or person B or another person D is handing them over to person A or person C.

For example, in order to determine that an event is "theft among tourists (robbery of tourists)", the following determination conditions may be used:
(c1) In the image, person A approaches person B and comes within a radius of 1 meter of person B, and person A is talking to person B.
(c2) In the image, person A takes out something and makes the motion of pouring it on person B (for example, pouring something smelly on him).
(c3) In the image, person A or another person C, who is thought to be an accomplice, is snatching something from person B.
(c4) The crime took less than 30 seconds.
(c5) Nothing was destroyed as a result of the crime.

The event information generating unit 102 may determine that a corresponding crime has occurred when the result of the image analysis satisfies all of a plurality of conditions, or may determine that a crime may have occurred even if all of the conditions are not satisfied. The event information generating unit 102 may calculate the probability (percentage, etc.) of the possibility of the occurrence of an event.

When an event is detected, the event information generating unit 102 generates event information related to the event from the recognition result of the image in which the event is detected. The event information generated by the event information generating unit 102 is information indicating the detailed contents of the determined crime, accident, or incident. The event information includes, for example, at least one of the following: type of crime or incident, information about the people involved (number of people, distinction between perpetrator and victim, information indicating the characteristics of each person), presence or absence of a murder weapon, type of murder weapon, time of crime (time it took to commit the crime), presence or absence and scale of destroyed objects, presence or absence and condition of injured people (state indicating state of collapse, state of sitting, presence or absence of movement, whether eyes are open, whether or not bleeding, etc.) and number of injured people, whether a vehicle was used, and information indicating the characteristics of the vehicle used.

The event information includes information indicating the scale of the event, with at least a portion of the scale of the event being determined using the difference between images taken before and after the event occurred. Furthermore, the scale of the event includes the duration of the event (i.e., the time it took to commit the crime). For example, the greater the difference between images taken before and after the event occurred, the greater the estimated scale of damage caused by the event. Furthermore, the time it took to commit the crime can be calculated by measuring the time from when the criminal appeared in the image to when the criminal fled and disappeared from the image.

The estimated movement route generated by the route generation unit 104 includes information indicating the expected destination of a person or vehicle related to an event detected from an image, and the route from the location of the event to the destination. For example, if the event is a crime, the estimated movement route indicates the destination to which the perpetrator will flee and the route from the location of the crime to the destination. In other words, when a crime occurs, the route generation unit 104 estimates the destination of the perpetrator and the route from the location of the event to the destination, and generates the estimated movement route.

The people who are the subject of route estimation are not limited to criminals. People related to the event may include the victim. For example, since an accomplice may pretend to be a victim or a bystander, all people related to the event who were at that location (all people included in the image) may be the subjects of route estimation. Alternatively, since it is possible that the victim accidentally commits an injury resulting in death against the criminal in self-defense, it is preferable to subject all people related to the event who were at that location (all people included in the image) to route estimation.

When a criminal commits a crime, he or she may find themselves in a situation where, for example, an unexpected event occurs and they panic, they become impatient because the crime takes time to commit, they become frustrated because they are unable to steal as much money or valuables as they had hoped, or they become anxious because an alarm goes off. In such a state, it is highly likely that the criminal's ability to think decreases and they will attempt to act in a way that is closer to their everyday lifestyle. In other words, it can be assumed that criminals will flee to places where they are likely to spend their daily lives.

In this embodiment, for each type of crime, expected event information indicating the expected content of the crime is created and registered in the crime model 350. The path generation unit 104 then compares this crime model 350 with the event information generated by processing the image acquired from the camera 5, and determines the difference. In other words, the path generation unit 104 determines whether there is a discrepancy between the expected content of the crime and the actual content of the crime as the difference, and uses this as the excitement level indicating the criminal's psychological state. The excitement level may be indicated, for example, in five levels from level 1 to level 5. For example, a high level of excitement may be level 5, and a low level of excitement may be level 1. Alternatively, the excitement level may be a number indicated, for example, from 1 to 100, and is not particularly limited.

When calculating the level of excitement, multiple items of crime information can be used as criteria. For example, the criteria can be the time taken to commit the crime, the scale of destruction at the crime scene, physical information indicating the physical strength of the criminal, etc., which will be described later. The excitement level can be calculated by finding the difference between the estimated crime event information and the actual crime event information using at least one of these items. When multiple items are used, the excitement level can be calculated by adding up the differences and dividing by the number of items. Alternatively, each item can be multiplied by a predetermined weighting coefficient (for example, the time taken to commit the crime is multiplied by coefficient k1 = 0.4, the scale of destruction at the crime scene is multiplied by coefficient k2 = 0.4, and the physical information is multiplied by coefficient k3 = 0.2) and then summed up.

On the other hand, in the case of planned or organized crime, the criminal often acts calmly and rarely fails, so the crime is completed in a short time. In other words, since the crime is committed in the same way as the expected event information, there is no discrepancy between the expected event information and the event information indicating the actual crime. In other words, the difference between the expected event information and the event information calculated by the route generation unit 104 becomes smaller, and the level of excitement also becomes lower.

From the above, the route generation unit 104 generates an estimated movement route according to the excitement level as follows. For example, when the excitement level of the criminal is high, it is considered that the criminal's thinking ability is reduced and the criminal is likely to behave in a manner similar to his/her daily lifestyle. In other words, the route generation unit 104 generates an estimated movement route by estimating that the criminal will flee to a place where he/she is likely to live on a daily basis, and generating a route with the place where the crime was committed as the starting point and this place as the destination. On the other hand, when the excitement level of the criminal is low, the criminal can act based on calm judgment, so the escape route is likely to be in line with the plan, and there is a possibility that the criminal will flee to a place farther from the crime scene or overseas. Therefore, when the excitement level is low, the route generation unit 104 generates an estimated movement route by estimating a bus stop or station of public transportation such as an airport or a bus or train bound for the airport as the destination, and generating a route with the place where the crime was committed as the starting point and this destination as the destination.

<Example of operation>
5 is a flowchart showing an example of the operation of the information processing device 100 of this embodiment. When the event information generating unit 102 detects the occurrence of an event by processing an image (YES in step S101), it generates event information related to the detected event (step S103). Then, the path generating unit 104 generates an estimated movement path using the difference between the generated event information and expected event information that is predetermined for the event (step S105).

More specifically, in step S101, the event information generation unit 102 causes the image processing device 200 to process the image acquired from the camera 5. The image processing device 200 uses the image recognition model 340 to recognize people, vehicles, weapons, surrounding conditions, and the like from the image. The event information generation unit 102 uses the crime database 310 to compare the image recognition results from the image processing device 200, detect the occurrence of a crime, and identify the type of crime. Furthermore, in step S103, the event information generation unit 102 generates event information using the image recognition results.

Then, in step S105, the route generation unit 104 obtains expected event information corresponding to the type of crime identified by the event information generation unit 102 from the crime model 350, compares the expected event information with the event information generated by the event information generation unit 102, and calculates the difference. If the degree of excitement of the criminal indicated by the calculated difference is higher than a threshold value, for example, if the threshold value is level 4 and the degree of excitement is level 5, the route generation unit 104 estimates that the criminal's escape destination is an area where the criminal lives daily. In this embodiment, for example, if the crime location is a store in a downtown area, the criminal's destination may be a residential area on the outskirts of the crime location. In addition, if the degree of excitement is level 1, it is highly likely that the crime is organized and planned, so the criminal's destination may be, for example, a nearby airport for a high-speed run, or a limousine bus stop for going to the airport. The route generation unit 104 generates a route to the airport or limousine bus stop.

According to this embodiment, the event information generating unit 102 detects the occurrence of an event by processing the image and generates event information related to the detected event, and the route generating unit 104 generates an estimated movement route using the difference between the generated event information and the expected event information that is predetermined for the event. In this way, when an event such as a crime is detected from the image, an estimated escape route of the criminal involved in the crime can be generated.

Second Embodiment
The information processing device 100 of this embodiment is the same as the above embodiment except that it has a configuration for generating an estimated movement path using attribute information of a person related to an event included in a captured image. The information processing device 100 of this embodiment has the same configuration as the first embodiment, and will be described using Fig. 3. However, the information processing device 100 of this embodiment may be combined with at least one of the configurations of the other embodiments to the extent that no contradiction occurs.

<Functional configuration example>
The event information generating unit 102 processes the images to generate related people associated with the event and their attribute information. The route generating unit 104 generates an estimated movement route using the generated related people and their attribute information.

The relevant person related to the event is at least the criminal. Furthermore, the victim or all people who were present at the scene, in other words, all or at least some of the people included in the image, may be regarded as the relevant person. The attribute information of the relevant person includes, for example, at least one of the following: the number of relevant people, age (generation), sex, clothing (appearance), accessories (including hats, glasses, masks, etc.), hairstyle, hair color, and physique (height and body type (slim, tall, small, large, obese, etc.)). The event information generating unit 102 may also include information indicating the characteristics of the vehicle used by the criminal in the attribute information of the relevant person. The information indicating the characteristics of the vehicle includes, for example, at least one of the following: the vehicle model, the area information written on the license plate, the vehicle color, the window color (smoked), the number of passengers, etc.

The route generating unit 104 uses the area database 320 to compare the attribute information of the related person generated by the event information generating unit 102 to identify the area where the related person lives daily or the building where the related person works. As in the first embodiment, the route generating unit 104 compares the expected event information with the event information generated from the image, finds the difference, and sets it as the excitement level. If the excitement level is high (e.g., level 5), the route generating unit 104 identifies the area where the identified related person lives daily as the destination, and generates an estimated travel route from the crime scene to the destination. Furthermore, as other destinations, for example, hangouts, entertainment facilities, bars, etc. where people of the same age gather based on the age group of the criminal may be the destination.

<Example of operation>
Fig. 6 is a flowchart showing an example of the operation of the information processing device 100 of this embodiment. Step S101 in Fig. 1 corresponds to steps S201 and S205 in Fig. 6, step S103 in Fig. 1 corresponds to step S203 in Fig. 6, and step S105 in Fig. 1 corresponds to steps S209 to S215 in Fig. 6.

First, the event information generation unit 102 acquires a captured image from the camera 5 and has the image processed by the image processing device 200 (step S201). Then, the event information generation unit 102 compares the image acquired from the camera 5 with the crime database 310, identifies the type of crime, and further generates event information (step S203). Then, when the event information generation unit 102 detects the occurrence of a crime (YES in step S205), the route generation unit 104 uses the person's features extracted from the image by the image processing device 200 to compare the image with the area database 320 and identify the place where the person is thought to live on a daily basis (referred to as the person's living area) (step S207).

For example, in a multi-ethnic country, culture, language, clothing, etc. vary from region to region. Also, in a given area such as a metropolitan area, people from different regions often live together in the same area. For this reason, the route generation unit 104 may be able to identify a person's living area by identifying the language used by the person through voice analysis or identifying clothing, etc. through image processing.

Then, the route generation unit 104 obtains crime information corresponding to the crime identified in step S203 from the crime model 350, calculates the difference with the event information generated in step S203, and calculates the excitement level (step S209). Then, if the excitement level calculated in step S209 is equal to or greater than the threshold, that is, if the excitement level is high (YES in step S211), the route generation unit 104 selects the living area of the related person as the destination and generates an estimated movement route from the crime scene to the destination (step S213). Also, if the excitement level calculated in step S209 is less than the threshold, that is, if the excitement level is low (NO in step S211), the route generation unit 104 selects an airport or the like as the destination and generates an estimated movement route from the crime scene to the destination (step S215).

For example, the event information generation unit 102 may determine, based on the perpetrator's clothing, whether the perpetrator is someone who uses public transportation such as trains or buses, and whether the person regularly lives in an office district or an upscale residential area.

According to this embodiment, the event information generating unit 102 generates event information using people related to the event and their attribute information, and the route generating unit 104 generates an estimated movement route using this event information. With this configuration, the area of the escape destination can be specified, and the escape route can be estimated with high accuracy.

Third Embodiment
The information processing device 100 of this embodiment differs from the second embodiment in that it has a configuration in which the physical strength of a person related to an event is estimated using attribute information of the person, and an estimated movement route is generated taking into account an escape possible range. The information processing device 100 of this embodiment has the same configuration as the first embodiment, and will be described using Fig. 3. However, the information processing device 100 of this embodiment may be combined with at least one of the configurations of the other embodiments to the extent that no contradiction occurs.

The storage device 300 stores standard physical information. The standard physical information includes standard values for a person's walking speed (e.g., stride length) and escape range (physical strength) in association with attribute information such as the person's face, physical information (height, weight), clothing, and gender.

For example, standard physical information for a male individual with a standard height of 180 cm, weight of 80 kg, age in his/her 20s, stride length of 1.2 meters, and escape range of 30 minutes are stored in the storage device 300 in association with the individual's attribute information.

This standard physical information may be included in the crime model 350. Then, the path generation unit 104 determines the difference between this standard physical information and the physical information included in the event information generated by the event information generation unit 102, and calculates the excitement level using the difference in this physical information and the difference in other event information.

However, it is preferable that the standard physical information takes into account the weather, country, geographical conditions, etc. That is, the walking speed of the standard physical information may be slowed down when it is raining, and the walking range (physical strength) of the standard physical information may be shortened if the travel route is uphill. Weather information may be weather forecast information for the area where the crime occurred or the area including the estimated travel route, obtained from a specified website. Furthermore, it is preferable that the geographical information is registered in advance in the map database 330.

In this embodiment, since the event information includes physical information, the event information generating unit 102 calculates the degree of excitement of the criminal by taking into account the criminal's escape ability based on his/her physique, etc. This makes it possible to estimate an appropriate movement route with more accuracy.

Fourth Embodiment
7 is a functional block diagram showing an example of a logical configuration of the information processing device 100 of this embodiment. The information processing device 100 of this embodiment differs from the other embodiments in that it has a configuration for outputting information on a detected event and an estimated movement route. The information processing device 100 further includes an output processing unit 106 in addition to the configuration of the information processing device 100 of FIG. 3. However, the information processing device 100 of this embodiment may be combined with at least one of the configurations of the other embodiments to the extent that no contradiction occurs.

The output processing unit 106 outputs the estimated travel route generated by the route generation unit 104 to a specified output destination. There are various possible output destinations, but for example, they may be a monitor screen for monitoring on the display device 110 of the monitoring center, a terminal (not shown) carried by a police officer or security guard, or a monitor screen for monitoring on a display device (not shown) at a police station. The output method includes at least one of displaying on a monitor screen, sending an email, and outputting a voice or alarm sound from a speaker (not shown). At least one of the destinations of the email address, the IP address of the mobile terminal, and the mobile phone number to which the email is to be sent may be registered in advance, or input may be accepted at the time of sending.

Terminals include mobile phones, smartphones, tablet terminals, PDAs (Personal Digital Assistants), notebook PCs (Personal Computers), and other personal computers.

The output may include information about the detected event, such as the time of occurrence of the event, the location of the event, the details of the crime, the damage situation, the number of perpetrators, information about the perpetrators (a facial photo or attribute information analyzed from the image (e.g., gender, age, appearance, clothing, etc.)), and the estimated route of movement. Multiple candidates for the estimated route of movement may be output. Multiple candidates for the estimated destination may also be output.

The output processing unit 106 may also multi-display a map showing the estimated travel route and an image from a camera 5 installed on the estimated travel route on a display at the monitoring center.

Furthermore, the output processing unit 106 may transmit crime-related event information and estimated route information to a terminal of a police officer near the estimated destination and route, based on the estimated destination and route. By quickly reporting the crime occurrence situation and escape route to the police in this way, assistance can be provided to capture the criminal before he or she escapes into an armed area where even the police or military cannot enter.

The information processing device 100 may further include an operation reception unit (not shown). The operation reception unit receives operations by the administrator of the monitoring system 1 on various setting screens displayed on the display device 110. For example, the operation reception unit receives operations for registering crime information to be registered in the crime database 310. For example, the types and methods of crime that occur vary depending on the country or region. For this reason, it is preferable to construct the crime database 310 to suit the region.

For example, an administrator can use the operation reception unit to input video footage of past crimes in the area, and have the footage undergo machine learning to be registered in the crime database 310. Similarly, the other area databases 320, map database 330, image recognition model 340, and crime model 350 can also register information specific to the area. It is preferable that this information be updated as the situation changes.

Furthermore, in the above-mentioned configuration for transmitting information on the estimated movement route to the terminal of a police official, when multiple crimes have occurred, the operation reception unit may receive input of information on the scene where the crime occurred and the time when the crime occurred in order to specify a crime for which police officials are requested to provide support. For the specified crime, the output processing unit 106 may identify a terminal of a police official present in the vicinity of the estimated movement route and transmit information on the estimated movement route to the identified terminal of the police official.

The current location of the police officer may be determined by obtaining location information from the GPS (Global Positioning System) receiving function of the police officer's device.

According to this embodiment, the output processing unit 106 can output the estimated movement route generated by the route generation unit 104 to a specified output destination. This makes it possible to transmit the criminal's escape route to a police officer's terminal after detecting the occurrence of an event (crime), thereby supporting the rapid arrest of the criminal.

Although the embodiments of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various configurations other than those described above can also be adopted.
For example, the camera 5 may acquire images of a road that is likely to be an escape route for the criminal from the camera 5 installed on the road, and generate information indicating the congestion state of the road. When the criminal is escaping using a vehicle, the route generation unit 104 may change the estimated movement route taking into account the congestion state of the road (traffic jam information).

In the above embodiment, the route generation unit 104 estimated one destination and generated an estimated travel route. In other forms, the route generation unit 104 may estimate multiple candidates as destinations. The output processing unit 106 may plot the multiple candidates on a map in order of proximity to the location where the crime occurred and display them on the screen of the display device 110. Of the multiple destination candidates, a destination closer to the crime scene may be selected when the excitement level is higher, and a destination farther from the crime scene may be selected when the excitement level is lower.

In addition, multiple candidates for the route from the crime location to the destination may be estimated. When multiple estimated travel routes are generated, a route with a shorter distance may be selected from the multiple route candidates when the excitement level is high, and a route with a longer distance may be selected when the excitement level is low.

When multiple destinations and/or multiple estimated travel routes are generated, the output processing unit 106 may display all of the multiple destinations and/or multiple estimated travel routes on the screen of the display device 110, or may display a selected portion of the destinations and/or estimated travel routes.

In addition, the route generation unit 104 may further use event information when generating an estimated movement route. Specifically, for example, an estimated movement route may be generated according to the type of crime. For example, if the type of crime is a robbery that is considered to be an organized crime, the route generation unit 104 may generate an estimated movement route with a hideout or hideout of an organized crime group as the destination.

In addition, since a crime involving multiple associated individuals is also considered to be an organized crime, the route generation unit 104 may generate an estimated movement route using a hideout or hideout of an organized crime group as the destination.

Although the present invention has been described above with reference to the embodiments and examples, the present invention is not limited to the above-mentioned embodiments and examples. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.
In the present invention, when information about a user is acquired and/or used, this shall be done lawfully.

Below, examples of reference forms are given.
1. An event information generating means for detecting the occurrence of an event by processing an image and generating event information relating to the event;
a route generating means for generating an estimated travel route using a difference between the generated event information and predefined expected event information for the event;
An information processing device comprising:
2. In the information processing device according to 1.
the event information generating means processes the image to generate related persons and their attribute information related to the event;
The route generation means generates the estimated movement route using the generated related person and the attribute information thereof.
3. In the information processing device according to 1 or 2,
The information processing device, wherein the event information includes a scale of the event.
4. In the information processing device according to 3.,
An information processing device wherein at least a portion of the scale of the event is determined using a difference between images taken at a given time before and after the occurrence of the event.
5. In the information processing device according to 3 or 4,
The information processing device, wherein the magnitude of the event includes a duration of the event.
6. In the information processing device according to any one of 1 to 5,
The information processing device, wherein the route generation means further uses the event information when generating the estimated travel route.
7. In the information processing device according to 6.,
The event information includes a type of the event.
8. In the information processing device according to 6. or 7.,
An information processing device, wherein the scale of the event includes the number of people involved.
9. An information processing device,
Processing the images to detect the occurrence of an event and generate event information regarding the event;
generating an estimated travel route using a difference between the generated event information and predefined expected event information for the event;
Information processing methods.
10. In the information processing method according to 9.,
The information processing device,
Processing the images to generate relevant people and their attribute information related to the event;
The information processing method further comprises generating the estimated movement route using the generated related person and the attribute information thereof.
11. In the information processing method according to 9. or 10.,
An information processing method, wherein the event information includes a scale of the event.
12. In the information processing method according to 11.,
A method of information processing, wherein the magnitude of the event is determined at least in part using a difference between images taken at a given time before and after the occurrence of the event.
13. In the information processing method according to 11 or 12,
The magnitude of the event includes a duration of the event.
14. In the information processing method according to any one of 9. to 13.,
The information processing device,
The information processing method further comprises using the event information when generating the estimated travel path.
15. In the information processing method according to 14.,
An information processing method, wherein the event information includes a type of the event.
16. In the information processing method according to 14. or 15.,
An information processing method, wherein the scale of the event includes the number of people involved.
17. On the computer:
processing the images to detect occurrence of an event and generate event information relating to said event;
A program for executing a procedure for generating an estimated movement route using a difference between the generated event information and predefined expected event information for the event.
18. In the program according to 17.,
processing the images to generate relevant people and their attribute information related to the event;
A program for causing a computer to execute a procedure for generating the estimated movement route using the generated related person and the attribute information thereof.
19. In the program according to 17. or 18.,
The event information includes a scale of the event.
20. In the program according to 19.,
The program, wherein the magnitude of the event is determined at least in part using a difference between images taken at a given time before and after the occurrence of the event.
21. In the program according to 19. or 20.,
The magnitude of the event includes a duration of the event.
22. In the program according to any one of 17. to 21.,
A program for causing a computer to execute a procedure for further using the event information when generating the estimated movement path.
23. In the program according to 22.,
The event information includes a type of the event.
24. In the program according to 22. or 23.,
The scale of the event includes the number of people involved.

1 Surveillance system 3 Communication network 5 Camera 7 Sensor 10 Person 100 Information processing device 102 Event information generation unit 104 Route generation unit 106 Output processing unit 110 Display device 200 Image processing device 300 Storage device 310 Crime database 320 Area database 330 Map database 340 Image recognition model 350 Crime model 1000 Computer 1010 Bus 1020 Processor 1030 Memory 1040 Storage device 1050 Input/output interface 1060 Network interface

Claims (12)

an event information generating means for detecting an occurrence of an event by processing an image and generating event information relating to the event;
a route generating means for generating an estimated travel route using a difference between the generated event information and predefined expected event information for the event;
Equipped with
The anticipated event information is information indicating the content of an anticipated event for each of the events including crimes and incidents,
The route generation means compares the content of the detected event indicated by the event information with the content of the event indicated by the expected event information corresponding to the detected event, and generates the estimated travel route using the difference. 2. The information processing device according to claim 1,
the event information generating means processes the image to generate related persons and their attribute information related to the event;
The route generation means generates the estimated movement route using the generated related person and the attribute information thereof. 3. The information processing device according to claim 1,
The route generation means uses the difference between the event information and the expected event information to identify an excitement level indicating a psychological state of a person related to the event, and generates the estimated movement route according to the excitement level. 4. The information processing device according to claim 1,
The information processing device, wherein the event information includes a scale of the event. In the information processing device according to claim 4, which cites claim 2 or 3 ,
The relevant persons include criminals and victims related to the crimes and incidents;
The route generation means is an information processing device that compares the scale of the event contained in the event information generated by the event information generation means with the scale of the event contained in the expected event information, and uses the difference to identify an excitement level indicating the psychological state of the relevant person related to the event, and generates the estimated movement route of at least the criminal in accordance with the excitement level. 6. The information processing device according to claim 4,
An information processing device in which the scale of the event is determined at least in part using a difference between images taken before and after the event occurs. 7. The information processing device according to claim 4,
The information processing device, wherein the magnitude of the event includes a duration of the event. 8. The information processing device according to claim 1,
The information processing device, wherein the route generation means further uses the event information when generating the estimated travel route. 9. The information processing device according to claim 8,
The event information includes a type of the event. In the information processing device according to claim 8 or 9, which refers to any one of claims 4 to 7 which refers to claim 2 or 3 ,
An information processing device, wherein the scale of the event includes the number of related persons. An information processing device,
Processing the images to detect the occurrence of an event and generate event information regarding the event;
generating an estimated movement path using a difference between the generated event information and expected event information that is predetermined for the event;
The anticipated event information is information indicating the content of an anticipated event for each of the events including crimes and incidents,
An information processing method, in which, when generating the estimated movement route, content of the detected event indicated by the event information is compared with content of the event indicated by the expected event information corresponding to the detected event, and the estimated movement route is generated using the difference. On the computer,
processing the images to detect occurrence of an event and generate event information relating to said event;
generating an estimated movement path using a difference between the generated event information and predefined expected event information for the event;
The anticipated event information is information indicating the content of an anticipated event for each of the events including crimes and incidents,
A program for generating the estimated movement route by comparing content of the detected event indicated by the event information with content of the event indicated by the expected event information corresponding to the detected event, and generating the estimated movement route using the difference.

Images