JP7775034B2 - Information processing device, information processing method, and program - Google Patents

Description

Embodiments of the present invention relate to an information processing device, an information processing method, and a program.

A system for detecting abnormal sounds, such as screams, is provided. Such a system collects sounds in a predetermined area. The system uses a model for detecting abnormal sounds to detect abnormal sounds from the collected sounds.

Also, due to automated driving, there may be times when there is no operator such as a conductor on board the vehicle.

Therefore, there is a demand for technology that can effectively detect abnormal sounds inside vehicles.

Japanese Patent Application Laid-Open No. 2020-182140

To solve the above problem, we provide an information processing device, information processing method, and program that can effectively detect abnormal sounds inside a vehicle.

According to an embodiment, an information processing device includes a microphone interface, a memory, and a processor. The microphone interface is connected to a microphone that collects sounds inside a vehicle. The memory stores peripheral information indicating conditions that affect the sound and an acoustic model for detecting abnormal sounds, in association with each other. The processor generates peripheral information related to the vehicle, retrieves the acoustic model corresponding to the generated peripheral information from the memory, and uses the retrieved acoustic model to detect abnormal sounds from sound data indicating sounds collected by the microphone. The acoustic model includes a noise reduction model for removing noise , including background sounds, in the conditions indicated by the peripheral information . When the processor determines that the sound data includes an abnormal sound, it removes noise from the sound data using the noise reduction model included in the retrieved acoustic model to extract the abnormal sound .

FIG. 1 is a schematic diagram of a vehicle system according to a first embodiment. FIG. 2 is a block diagram showing a control system of the vehicle according to the first embodiment. FIG. 3 is a diagram illustrating an example of an acoustic model database according to the first embodiment. FIG. 4 is a diagram illustrating an example of the operation of the vehicle according to the first embodiment. FIG. 5 is a diagram illustrating an example of the operation of the vehicle according to the second embodiment.

Hereinafter, embodiments will be described with reference to the drawings.
(First embodiment)
The vehicle provided in the vehicle system according to the embodiment constitutes a train that travels on a railroad track. The vehicle collects internal sounds. The vehicle detects abnormal sounds from the collected sounds. When an abnormal sound is detected, the vehicle notifies an operator such as a driver that an abnormal sound has been detected. Furthermore, in the case of autonomous driving, the vehicle stops or notifies a higher-level device that an abnormal sound has been detected. For example, abnormal sounds include screams, shouts, gunfire, collision sounds, explosions, popping sounds, and the sound of glass breaking. The content of the abnormal sound is not limited to a specific configuration.

Figure 1 is a schematic diagram of a vehicle system 100 according to an embodiment. As shown in Figure 1, the vehicle system 100 is composed of a vehicle 10, a host device 40, and other components. The vehicle 10 and the host device 40 are connected to each other so that they can communicate with each other.

Vehicle 10 constitutes a train that runs on track R. Vehicle 10 runs on track R either by being operated by a driver or automatically.

The vehicle 10 includes a housing 1, a camera 2, an antenna 3, a microphone 4, etc.

The housing 1 forms the exterior of the vehicle 10. For example, the housing 1 is formed so that it can accommodate people, luggage, etc. inside.

Camera 2 is installed in housing 1. For example, camera 2 is installed facing downward on the top of housing 1. Camera 2 captures images of the inside of housing 1 (i.e., the inside of vehicle 10). For example, camera 2 captures images of people staying in housing 1.

For example, the camera 2 is a CCD (Charge Coupled Device) camera. The camera 2 may also be equipped with a light that illuminates the inside of the housing 1.
The camera 2 may also be made up of multiple cameras.

Antenna 3 is also installed in housing 1. Antenna 3 is an antenna for receiving signals for positioning vehicle 10. For example, antenna 3 is an antenna for receiving GNSS (Global Navigation Satellite System) signals.

A microphone 4 is also installed in the housing 1. For example, the microphone 4 is installed inside the housing 1. The microphone 4 collects sounds inside the housing 1 (i.e., the inside of the vehicle 10). The microphone 4 may also be made up of multiple microphones.

The host device 40 manages the operation of the vehicle 10. The host device 40 provides the vehicle 10 with various information related to the operation of the vehicle 10. The host device 40 also acquires various information related to operation from the vehicle 10. For example, the host device 40 is a server managed by a railway company.

Next, the vehicle 10 will be described.
Fig. 2 shows an example of the configuration of the vehicle 10 according to the embodiment. Fig. 2 is a block diagram showing the example of the configuration of the vehicle 10. As shown in Fig. 2, the vehicle 10 includes a camera 2, an antenna 3, a microphone 4, an information processing device 5, and the like.

The information processing device 5 detects abnormal sounds from sounds inside the vehicle 10.
The information processing device 5 includes a processor 11, a ROM 12, a RAM 13, an NVM 14, a communication unit 15, an operation unit 16, a display unit 17, a camera interface 18, an antenna interface 19, and a microphone interface 20.

The processor 11, ROM 12, RAM 13, NVM 14, communication unit 15, operation unit 16, display unit 17, camera interface 18, antenna interface 19, and microphone interface 20 are connected to one another via a data bus or the like. The camera interface 18 is connected to the camera 2. The antenna interface 19 is connected to the antenna 3. The microphone interface 20 is connected to the microphone 4.

Note that the vehicle 10 and information processing device 5 may be equipped with additional components as needed in addition to the components shown in FIG. 2, or certain components may be excluded from the vehicle 10.

Processor 11 has the function of controlling the overall operation of vehicle 10. Processor 11 may also include an internal cache and various interfaces. Processor 11 performs various processes by executing programs pre-stored in internal memory, ROM 12, or NVM 14.

Note that some of the various functions realized by processor 11 executing programs may be realized by hardware circuits. In this case, processor 11 controls the functions executed by the hardware circuits.

ROM 12 is a non-volatile memory in which control programs, control data, etc. are pre-stored. The control programs and control data stored in ROM 12 are pre-installed according to the specifications of vehicle 10.

RAM 13 is a volatile memory. RAM 13 temporarily stores data being processed by processor 11. RAM 13 stores various application programs based on instructions from processor 11. RAM 13 may also store data required to execute application programs and the execution results of application programs.

NVM 14 is a non-volatile memory to which data can be written and rewritten. NVM 14 is composed of, for example, a hard disk drive (HDD), a solid state drive (SSD), or flash memory. NVM 14 stores control programs, applications, various data, and the like, depending on the operational use of vehicle 10.

NVM14 also stores an acoustic model database containing multiple acoustic models. The acoustic model database will be described later.

The communication unit 15 is an interface that transmits and receives data to and from a higher-level device 40, etc. For example, the communication unit 15 connects to a higher-level device 40, etc. via a network. For example, the communication unit 15 is an interface that supports wired or wireless LAN (Local Area Network) connections.

The operation unit 16 accepts various operation inputs from an operator (e.g., a driver). The operation unit 16 transmits a signal indicating the input operation to the processor 11. The operation unit 16 may be configured as a touch panel.

The display unit 17 displays image data from the processor 11. For example, the display unit 17 is composed of an LCD monitor. If the operation unit 16 is composed of a touch panel, the display unit 17 may be formed integrally with the operation unit 16.

The camera interface 18 is an interface that connects to the camera 2. The camera interface 18 transmits signals from the processor 11 to the camera 2. The camera interface 18 also acquires signals (such as captured images) from the camera 2 and transmits them to the processor 11.

The antenna interface 19 is an interface that connects to the antenna 3. The antenna interface 19 measures the current position of the vehicle 10 based on signals from the antenna 3, etc.

For example, the antenna interface 19 measures the position based on a signal from the antenna 3. After measuring the position, the antenna interface 19 fits the measured position onto a nearby railroad track R based on map information including the position of the railroad track R. The antenna interface 19 acquires the fitted position as the current position of the vehicle 10.
The antenna interface 19 transmits the acquired current location to the processor 11 .

The microphone interface 20 is an interface that connects to the microphone 4. For example, the microphone interface 20 amplifies a signal representing a sound measured by the microphone 4 and converts it into a digital signal. The microphone interface 20 transmits the converted digital signal (sound data) to the processor 11.

The function of the antenna interface 19 may be realized by the processor 11 .
Furthermore, when the vehicle 10 travels by autonomous driving, the information processing device 5 does not need to include the operation unit 16 and the display unit 17 .

Next, the acoustic model database will be described.
The acoustic model database is stored in advance in the NVM 14 .

Figure 3 shows an example of the configuration of an acoustic model database. As shown in Figure 3, the acoustic model database stores multiple acoustic models and multiple pieces of peripheral information, each associated with the other.

The surrounding information is information that the vehicle 10 uses to select an acoustic model. In other words, the surrounding information is information that indicates conditions (e.g., factors) that affect the sounds (mainly background sounds) inside the vehicle 10.

Here, the surrounding area information consists of the current location of the vehicle 10, the current time, and the congestion level. The congestion level is an index related to the number of people accommodated in the housing 1 (for example, the occupancy rate).

The surrounding information may include the contents of announcements made on the train, whether the train is accelerating, decelerating, stopping, or running at a constant speed, or the weather.
The configuration of the peripheral information is not limited to a specific configuration.

Each acoustic model corresponds to a different piece of peripheral information. In other words, an acoustic model is a model for detecting abnormal sounds in the state indicated by the corresponding peripheral information.

As shown in Figure 3, the acoustic model database includes multiple acoustic models 30 (30a, 30b, etc.).

Here, we will explain acoustic model 30a. The other acoustic models 30 have the same configuration as acoustic model 30a, so their explanations will be omitted.

The acoustic model 30a is composed of an anomaly detection model 31a, a noise removal model 32a, and the like.

The abnormality detection model 31a is composed of an abnormal sound model 311a, a background sound model 312a, etc.

The abnormal sound model 311a is a model of an abnormal sound in the state indicated by the peripheral information. For example, the abnormal sound model 311a is a model of sound data in which an abnormal sound is superimposed on background sound in the state indicated by the peripheral information.

The abnormal sound model 311a indicates the feature quantities of the abnormal sound in the state indicated by the peripheral information. For example, the abnormal sound model 311a indicates the distribution of the feature quantities of the abnormal sound (for example, a Gaussian Mixture Model (GMM)).

The background sound model 312a is a model of the background sound in the state indicated by the peripheral information.
The background sound model 312a indicates the feature amount of the background sound in the state indicated by the peripheral information. For example, the background sound model 312a indicates the distribution (for example, GMM) of the feature amount of the background sound.

The noise removal model 32a is a model for removing background sounds from sound data that includes abnormal sounds. For example, the noise removal model 32a is generated based on the difference between the background sounds in the state indicated by the peripheral information and the clean sounds. For example, the noise removal model 32a is related to the frequencies of the sounds that make up the background sounds.

The acoustic model database may be updated as appropriate. For example, new acoustic models may be added to the acoustic model database, and existing acoustic models may be deleted. The contents of the acoustic models may also be updated as appropriate.

Next, we will explain the functions realized by the vehicle 10. The functions realized by the vehicle 10 are realized by the processor 11 executing programs stored in the internal memory, ROM 12, NVM 14, etc.

First, the processor 11 has a function of acquiring sound data using the microphone 4 .
For example, the processor 11 starts the microphone 4 in accordance with an operation input to the operation unit 16. The processor 11 may also start the microphone 4 in accordance with a control signal received from the higher-level device 40 via the communication unit 15.

When the microphone 4 is started, the processor 11 acquires, via the microphone interface 20, sound data indicating the sound inside the housing 1 over a predetermined period of time.
The processor 11 continues to acquire sound data using the microphone 4 .

Note that vehicle 10 may or may not be in operation.

The processor 11 also has the function of generating surrounding information (surrounding information related to the vehicle 10) that indicates the current state of the vehicle 10.

The processor 11 acquires the current position of the vehicle 10 using the antenna 3 and the antenna interface 19. Once the current position is acquired, the processor 11 acquires the current time.

After obtaining the current time, the processor 11 obtains the degree of congestion inside the vehicle 10.
For example, the processor 11 uses the camera 2 to acquire a captured image of the interior of the housing 1 of the vehicle 10. After acquiring the captured image, the processor 11 identifies people from the captured image according to a predetermined image processing algorithm. After identifying people, the processor 11 calculates the congestion level based on the number of identified people, etc.

Once the congestion level is obtained, the processor 11 generates surrounding information indicating the current location of the vehicle 10, the current time, and the congestion level.

The processor 11 also has the function of selecting candidate acoustic models from the acoustic model database based on the generated peripheral information.

Once the peripheral information is generated, processor 11 calculates the degree of similarity between each piece of peripheral information in the acoustic model database and the generated peripheral information according to a predetermined algorithm. For example, processor 11 compares each element of the peripheral information to calculate the degree of similarity. Here, the higher the similarity between the pieces of peripheral information, the higher the value of the degree of similarity.

After calculating the similarity, processor 11 selects candidates from the acoustic model database based on the similarity. For example, processor 11 selects as candidates acoustic models that correspond to a predetermined number of pieces of peripheral information with a high degree of similarity. Processor 11 may also select as candidates acoustic models that correspond to peripheral information with a similarity greater than a predetermined threshold.

The processor 11 also has the function of selecting one acoustic model from the candidates based on the acquired sound data.

Once candidates are selected, processor 11 matches the sound data with the background sound model of each candidate. For example, processor 11 extracts features from the sound data according to a predetermined algorithm. After extracting features from the sound data, processor 11 calculates the similarity between the extracted features and each background sound model. Here, the higher the similarity, the larger the value of the similarity.

After calculating the similarity between the sound data and each background sound model, the processor 11 identifies the background sound model with the highest similarity. Once the background sound model with the highest similarity is identified, the processor 11 selects an acoustic model that includes that background sound model. Here, the highest similarity is referred to as the first similarity.

The processor 11 also has the function of detecting abnormal sounds from sound data using the selected acoustic model.

When one acoustic model is selected from the candidates, the processor 11 matches the sound data with the abnormal sound model of the selected acoustic model. The processor 11 calculates the similarity (second similarity) by matching the sound data with the abnormal sound model. After calculating the second similarity, the processor 11 compares the first similarity with the second similarity.

If the second similarity is greater than the first similarity (i.e., if the sound data is more similar to the abnormal sound model than to the background sound model), the processor 11 determines that the sound data contains an abnormal sound.

Furthermore, if the second similarity is less than or equal to the first similarity, the processor 11 determines that the sound data does not contain any abnormal sound.

The processor 11 may also detect abnormal sounds based on the captured image from the camera 2. For example, the processor 11 detects an abnormality (such as a gun) from the captured image according to a predetermined image processing algorithm. The processor 11 may also detect abnormal sounds based on the abnormality detected from the captured image.

The processor 11 also has a function of removing noise from the sound data.
When it is determined that the sound data contains an abnormal sound, the processor 11 acquires a noise reduction model from the selected acoustic model. After acquiring the noise reduction model, the processor 11 uses the acquired noise reduction model to remove noise (such as background sound) from the sound data. In other words, the processor 11 uses the acquired noise reduction model to extract the abnormal sound from the sound data.

When an abnormal sound is extracted from the sound data, the processor 11 outputs the extracted abnormal sound. For example, the processor 11 transmits the sound data of the extracted abnormal sound to the higher-level device 40 via the communication unit 15 or the like.

The processor 11 also has a function of outputting the abnormal sound detection result.
For example, the processor 11 displays the abnormal sound detection result on the display unit 17. The processor 11 may also transmit the abnormal sound detection result to the higher-level device 40 via the communication unit 15.

Next, an example of the operation of the vehicle 10 will be described.
FIG. 4 is a flowchart illustrating an example of the operation of the vehicle 10.

The processor 11 of the vehicle 10 acquires sound data using the microphone 4 (S11). Once the sound data has been acquired, the processor 11 acquires the current position of the vehicle 10 using the antenna 3 and antenna interface 19 (S12).

After acquiring the current location of the vehicle 10, the processor 11 acquires the current time (S13). After acquiring the current time, the processor 11 acquires the congestion level (S14). After acquiring the congestion level, the processor 11 generates surrounding information indicating the current location of the vehicle 10, the current time, and the congestion level (S15).

Once the peripheral information has been generated, the processor 11 selects acoustic model candidates from the acoustic model database based on the generated peripheral information (S16). Once the candidates have been selected, the processor 11 selects one acoustic model from the candidates based on the acquired sound data (S17).

After selecting one acoustic model, the processor 11 matches the abnormal sound model of the selected acoustic model with the sound data (S18). After matching the abnormal sound model with the sound data, the processor 11 determines whether the sound data contains an abnormal sound (S19).

If it is determined that the sound data contains an abnormal sound (S19, YES), the processor 11 extracts the abnormal sound from the sound data using the noise reduction model of the selected acoustic model (S20).

If it is determined that the sound data does not contain any abnormal sound (S19, NO), or if an abnormal sound is extracted from the sound data (S20), the processor 11 outputs the abnormal sound detection result (S21).
After outputting the abnormal sound detection result, the processor 11 ends its operation.

The processor 11 may repeatedly execute steps S11 to S21.

The anomaly detection model does not have to include an abnormal sound model. In this case, the processor 11 determines that the sound data contains an abnormal sound if the similarity between the sound data and the background sound model is equal to or less than a predetermined threshold.

The abnormality detection model may also include multiple abnormal sound models. For example, the abnormality detection model may include an abnormal sound model for each type of abnormal sound (scream, gunshot, etc.). In this case, processor 11 calculates the similarity between the sound data and each abnormal sound model. Processor 11 determines that the sound data contains an abnormal sound corresponding to the abnormal sound model with the highest similarity.

Furthermore, if the first similarity is equal to or less than a predetermined threshold, the processor 11 may display this fact on the display unit 17. Furthermore, the processor 11 may notify the higher-level device 40 of this fact via the communication unit 15.

Furthermore, if the first similarity is equal to or less than a predetermined threshold, the processor 11 may select an acoustic model based on an acoustic model selected for detecting abnormal sounds during a predetermined period of time in the past. For example, the processor 11 may select the acoustic model that was used most frequently during a predetermined period of time in the past. The processor 11 may also select an acoustic model by weighting the acoustic model used during a predetermined period of time in the past (for example, by weighting the most recent acoustic model).

A vehicle configured as described above selects candidate acoustic models based on ambient information. The vehicle selects one acoustic model from the candidates based on sound data. The vehicle detects abnormal sounds from the sound data based on the selected acoustic model. As a result, the vehicle can detect abnormal sounds based on an acoustic model that is appropriate for the vehicle's state. Therefore, the vehicle can effectively detect abnormal sounds from sounds inside the vehicle.

Second Embodiment
Next, a second embodiment will be described.
The vehicle system according to the second embodiment differs from that according to the first embodiment in that a background sound model is generated, so the other components are denoted by the same reference numerals and detailed description thereof will be omitted.

The configuration of the vehicle system 100 according to the second embodiment is similar to that according to the first embodiment, and therefore will not be described further.

Next, a description will be given of functions realized by the vehicle 10. The functions realized by the vehicle 10 are realized by the processor 11 executing a program stored in the internal memory, the ROM 12, the NVM 14, or the like.
The vehicle 10 according to the second embodiment realizes the following functions in addition to the functions according to the first embodiment.

First, the processor 11 has the function of acquiring sound data indicating background sound using the microphone 4.

For example, the processor 11 starts the microphone 4 in accordance with an operation input to the operation unit 16. The processor 11 may also start the microphone 4 in accordance with a control signal received from the higher-level device 40 via the communication unit 15.

Here, we assume that no abnormal sounds are occurring inside the vehicle 10. In other words, we assume that background sounds are occurring inside the vehicle 10.

When the microphone 4 is started, the processor 11 acquires, via the microphone interface 20, sound data indicative of background sounds over a predetermined period of time.
The processor 11 has a function of generating a background sound model from sound data.

Here, it is assumed that the processor 11 generates peripheral information indicating the state in which the sound data was acquired.

For example, processor 11 extracts features from sound data according to a predetermined algorithm. Once the features are extracted from the sound data, processor 11 generates a background sound model based on the distribution of the extracted features, etc.

Once the background sound model is generated, the processor 11 associates the generated background sound model with the surrounding information and stores them in the acoustic model database.

The processor 11 may also generate a noise reduction model based on sound data. In this case, the processor 11 further associates the generated noise reduction model with surrounding information and stores it in the acoustic model database.

Next, an example of the operation of the vehicle 10 to generate a background sound model will be described.
FIG. 5 is a flowchart illustrating an example of an operation in which the vehicle 10 generates a background sound model.

The processor 11 of the vehicle 10 acquires sound data using the microphone 4 (S31). Once the sound data has been acquired, the processor 11 acquires the current position of the vehicle 10 using the antenna 3 and antenna interface 19 (S32).

After acquiring the current location of the vehicle 10, the processor 11 acquires the current time (S33). After acquiring the current time, the processor 11 acquires the congestion level (S34). After acquiring the congestion level, the processor 11 generates surrounding information indicating the current location of the vehicle 10, the current time, and the congestion level (S35).

Once the ambient information has been generated, the processor 11 generates a background sound model based on the acquired sound data (S36). Once the background sound model has been generated, the processor 11 associates the generated background sound model with the generated ambient information and stores them in the acoustic model database (S37).

Once the generated background sound model and the generated ambient information are associated and stored in the acoustic model database, the processor 11 terminates its operation.

Note that the processor 11 may generate a background sound model in the same way for multiple states. That is, the processor 11 may execute steps S31 to S37 each time the ambient information changes.

The processor 11 may also acquire sound data including abnormal sounds generated by an operator or the like and generate an abnormal sound model.

A vehicle configured as described above generates ambient information. The vehicle also generates a background sound model from sound data. The vehicle stores the generated ambient information and the generated background sound model in association with each other. As a result, the vehicle can generate a background sound model that corresponds to the ambient information.

Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims.
(Appendix 1)
a microphone interface for connecting to a microphone for collecting sounds inside the vehicle;
a memory that stores peripheral information indicating conditions that affect sound and an acoustic model for detecting abnormal sounds in association with each other;
generating surrounding information relating to the vehicle;
retrieving an acoustic model corresponding to the generated peripheral information from the memory;
using the acquired acoustic model to detect abnormal sounds from sound data representing sounds collected by the microphone;
a processor;
An information processing device comprising:
(Appendix 2)
The processor:
Selecting candidate acoustic models based on the generated peripheral information;
selecting an acoustic model from the candidates based on the sound data;
Detecting abnormal sounds from the sound data using the selected acoustic model.
2. The information processing device according to claim 1.
(Appendix 3)
the acoustic model includes a background sound model related to characteristics of background sound;
the processor matches the sound data with a background sound model included in each of the candidate acoustic models to select the acoustic model from the candidates.
3. The information processing device according to claim 2.
(Appendix 4)
the acoustic model includes an abnormal sound model related to a feature of the abnormal sound;
the processor detects an abnormal sound from the sound data using the abnormal sound model.
4. An information processing device according to any one of claims 1 to 3.
(Appendix 5)
the acoustic model includes a noise reduction model for reducing noise;
When the processor determines that the sound data includes an abnormal sound, it removes noise from the sound data using the noise removal model.
5. An information processing device according to any one of appendices 1 to 4.
(Appendix 6)
an antenna interface connected to an antenna for receiving a signal for positioning the vehicle;
The processor generates the surrounding information including a current location of the vehicle.
6. An information processing device according to any one of appendices 1 to 5.
(Appendix 7)
a camera interface connected to a camera that captures images of the interior of the vehicle;
The processor:
Calculating the degree of congestion of the vehicle from the image captured by the camera;
generating the surrounding information including the congestion degree;
7. An information processing device according to any one of appendices 1 to 6.
(Appendix 8)
The processor generates the surrounding information including a current time.
8. An information processing device according to any one of appendices 1 to 7.
(Appendix 9)
The processor:
Acquire sound data indicating background sound using the microphone;
generating a background sound model based on the sound data;
9. An information processing device according to any one of appendices 1 to 8.
(Appendix 10)
1. An information processing method executed by a processor, comprising:
Uses microphones to collect sounds from inside the vehicle,
generating surrounding information relating to the vehicle;
Acquire an acoustic model corresponding to the generated peripheral information from a memory that stores peripheral information indicating conditions that affect the sound and an acoustic model for detecting abnormal sounds in association with each other;
using the acquired acoustic model to detect abnormal sounds from sound data representing sounds collected by the microphone;
Information processing methods.
(Appendix 11)
A program executed by a processor,
the processor,
A function to collect sounds inside the vehicle using a microphone;
generating surrounding information related to the vehicle;
a function of acquiring an acoustic model corresponding to the generated peripheral information from a memory that stores peripheral information indicating conditions that affect sound and an acoustic model for detecting abnormal sounds in association with each other;
a function of detecting abnormal sounds from sound data indicating sounds collected by the microphone using the acquired acoustic model;
A program to make this happen.

1...Housing, 2...Camera, 3...Antenna, 4...Microphone, 5...Information processing device, 10...Vehicle, 11...Processor, 12...ROM, 13...RAM, 14...NVM, 15...Communication unit, 16...Operation unit, 17...Display unit, 18...Camera interface, 19...Antenna interface, 20...Microphone interface, 30...Acoustic model, 30a...Acoustic model, 31a...Anomaly detection model, 32a...Noise reduction model, 40...High-level device, 100...Vehicle system, 311a...Abnormal sound model, 312a...Background sound model.

Claims (10)

a microphone interface for connecting to a microphone for collecting sounds inside the vehicle;
a memory that stores peripheral information indicating conditions that affect sound and an acoustic model for detecting abnormal sounds in association with each other;
generating surrounding information relating to the vehicle;
retrieving the acoustic model corresponding to the generated peripheral information from the memory;
using the acquired acoustic model to detect abnormal sounds from sound data representing sounds collected by the microphone;
a processor,
the acoustic model includes a noise reduction model for removing noise including background sound in the state indicated by the peripheral information ,
When the processor determines that the sound data includes an abnormal sound, the processor removes noise from the sound data using the noise removal model included in the acquired acoustic model to extract the abnormal sound. The processor:
Selecting candidate acoustic models based on the generated peripheral information;
selecting the acoustic model from the candidates based on the sound data;
Detecting abnormal sounds from the sound data using the selected acoustic model.
The information processing device according to claim 1 . the acoustic model includes a background sound model related to characteristics of background sound;
the processor matches the sound data with a background sound model included in each of the candidate acoustic models to select the acoustic model from the candidates.
The information processing device according to claim 2 . the acoustic model includes an abnormal sound model related to a feature of the abnormal sound;
the processor detects an abnormal sound from the sound data using the abnormal sound model.
The information processing device according to claim 1 . an antenna interface connected to an antenna for receiving a signal for positioning the vehicle;
The processor generates the surrounding information including a current location of the vehicle.
The information processing device according to claim 1 . a camera interface connected to a camera that captures images of the interior of the vehicle;
The processor:
Calculating the degree of congestion of the vehicle from the image captured by the camera;
generating the surrounding information including the congestion degree;
The information processing device according to claim 1 . The processor generates the surrounding information including a current time.
The information processing device according to claim 1 . The processor:
Acquire sound data indicating background sound using the microphone;
generating a background sound model based on the sound data;
The information processing device according to claim 1 . 1. An information processing method executed by a processor, comprising:
Uses microphones to collect sounds from inside the vehicle,
generating surrounding information relating to the vehicle;
acquiring, from a memory that stores peripheral information indicating conditions that affect the sound and an acoustic model for detecting abnormal sounds in association with each other, the acoustic model corresponding to the generated peripheral information;
Using the acquired acoustic model, an abnormal sound is detected from sound data representing the sound collected by the microphone;
removing noise from the sound data and extracting abnormal sounds using a noise removal model for removing noise including background sounds in the state indicated by the surrounding information, the noise removal model being included in the acquired acoustic model;
Information processing methods. A program executed by a processor,
the processor,
A function to collect sounds inside the vehicle using a microphone;
generating surrounding information related to the vehicle;
a function of acquiring, from a memory that stores peripheral information indicating conditions that affect sound and an acoustic model for detecting abnormal sounds in association with each other, the acoustic model corresponding to the generated peripheral information;
a function of detecting abnormal sounds from sound data indicating sounds collected by the microphone using the acquired acoustic model;
a function of removing noise from the sound data and extracting abnormal sounds using a noise removal model for removing noise , including background sounds, in the state indicated by the peripheral information, which is included in the acquired acoustic model; and
A program to make this happen.