JP7357030B2 - Communication terminal, program, and display method - Google Patents

Description

The present invention relates to a communication terminal, a program, and a display method.

Patent Document 1 discloses that a specialized technical staff can obtain the inference accuracy of a neural network without going to the location where the camera is installed and adjusting the camera installation position and installation angle, so that the user can An image analysis program is described that allows easy camera installation.
[Prior art documents]
[Patent document]
[Patent Document 1] Japanese Patent Application Publication No. 2020-188449

According to one embodiment of the present invention, a communication terminal is provided. The communication terminal may include a threshold value storage unit that stores a threshold value set by a user of the communication terminal. The communication terminal may include a recognition result receiving unit that receives from the recognition device a recognition result including character information obtained by the recognition device performing recognition processing on the content and recognition accuracy of the character information. The communication terminal may include a display mode determining section that determines a display mode of the character information based on the recognition accuracy and the threshold value. The communication terminal may include a display control section that controls the display section to display the text information based on the display mode determined by the display mode determination section.

The display mode determining unit displays the character information when the recognition accuracy is higher than the threshold, and replaces the character information with a predetermined symbol when the recognition accuracy is lower than the threshold. , the above display mode may be determined. The threshold storage unit may further store another threshold lower than the threshold set by the user. The display mode determining unit replaces and displays the character information with the predetermined symbol when the recognition accuracy is lower than the threshold and higher than the other threshold, The display mode may be determined such that when the character information is lower than the predetermined symbol, the character information is replaced with another predetermined symbol different from the predetermined symbol.

The display mode determining unit may determine the display mode such that the lower the recognition accuracy, the more emphasized the predetermined symbol is displayed. The display mode determining section may determine the display mode so that the character information whose recognition accuracy is higher than the threshold is displayed in an emphasized manner according to the recognition accuracy. The display mode determining unit may determine the display mode so that the higher the recognition accuracy, the more emphasized the character information is displayed.

The threshold storage unit may further store another threshold higher than the threshold set by the user. When the recognition accuracy is higher than the other threshold, the display mode determining unit displays the character information with greater emphasis than when the recognition accuracy is higher than the threshold and lower than the other threshold. , the above display mode may be determined. The display mode determining section may determine the display mode so as to display the character information while changing the color according to the recognition accuracy. The display mode determining unit may determine the display mode so that the character information is displayed while changing the size of the character information according to the recognition accuracy.

The recognition result receiving unit may receive recognition results including the recognition accuracy of each of the plurality of character information and the plurality of character information from the recognition device. The display mode determining unit may determine the display mode of the plurality of character information based on the recognition accuracy and the threshold value of each of the plurality of character information. The display control section may control the display section to display the plurality of character information based on the display mode determined by the display mode determining section. The communication terminal may further include a threshold value setting unit that changes the threshold value according to a ratio of character information whose recognition accuracy is higher than the threshold value for the plurality of character information items during a predetermined period.

According to one embodiment of the present invention, a program for causing a computer to function as the communication terminal is provided.

According to one embodiment of the invention, a computer-implemented method is provided. The method may include storing a threshold value set by a user of the computer. The method may include a recognition result receiving step of receiving, from the recognition device, a recognition result including character information obtained by the recognition device performing recognition processing on the content and recognition accuracy of the character information. The method may include a display mode determining step of determining a display mode of the character information based on the recognition accuracy and the threshold value. The method may include a display control step of controlling the display unit to display the text information based on the display mode determined by the display mode determining step.

Note that the above summary of the invention does not list all the necessary features of the invention. Furthermore, subcombinations of these features may also constitute inventions.

1 schematically depicts an example of a system 10; An example of character information displayed by display unit 150 is schematically shown. Another example of character information displayed by the display unit 150 is schematically shown. Another example of character information displayed by the display unit 150 is schematically shown. An example of the functional configuration of the communication terminal 100 is schematically shown. An example of the flow of processing by the communication terminal 100 is schematically shown. Another example of the flow of processing by the communication terminal 100 is schematically shown. An example of the hardware configuration of a computer 1200 functioning as the communication terminal 100 is schematically shown.

It is desirable that a user of a communication terminal be able to set a display mode for displaying recognition results of recognition processing, such as voice recognition or image recognition, by an external recognition device, depending on the recognition accuracy of the recognition results. The communication terminal 100 according to the present embodiment determines the display mode for displaying the recognition result based on the recognition accuracy of the recognition result and the threshold set by the user of the communication terminal 100. For example, the communication terminal 100 determines the display mode so as to display recognition results in different colors depending on recognition accuracy, or to display only recognition results higher than a predetermined recognition accuracy. Thereby, the communication terminal 100 according to the present embodiment can display the recognition result in an appropriate display mode depending on the purpose of using the recognition result of the user.

Hereinafter, the present invention will be explained through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Furthermore, not all combinations of features described in the embodiments are essential to the solution of the invention.

FIG. 1 schematically depicts an example of a system 10. The system 10 includes a communication terminal 100. System 10 includes a recognition device 200.

The communication terminal 100 receives the recognition result obtained by the recognition device 200 performing recognition processing on the content. The communication terminal 100 receives the recognition result from the recognition device 200 via the network 20, for example.

The content includes, for example, audio content. The content includes, for example, image content. Image content includes, for example, video content. Image content may include still image content.

The recognition result includes, for example, character information obtained by the recognition device 200 performing recognition processing on the content and recognition accuracy of the character information. The recognition result may include a plurality of pieces of character information obtained by the recognition device 200 performing recognition processing on the content and recognition accuracy of each of the plurality of character information.

Network 20 includes, for example, the Internet. Network 20 may include a core network provided by a carrier. The core network may be based on any mobile communication system, such as a 3G (3rd Generation) communication system, an LTE (Long Term Evolution) communication system, a 5G (5th Generation) communication system, and a 6G (6th Generation) communication system. It may be compliant with mobile communication systems after the communication system.

The communication terminal 100 determines the display mode for displaying the character information based on the recognition accuracy of the character information included in the recognition result received from the recognition device 200 and the threshold set by the user of the communication terminal 100. The communication terminal 100 determines the display mode, for example, to display character information whose recognition accuracy is higher than a threshold value, and to display character information whose recognition accuracy is lower than the threshold value by replacing it with a predetermined symbol. The communication terminal 100 may determine the display mode so that the text information is displayed in different colors depending on the recognition accuracy of the text information.

Display unit 150 of communication terminal 100 displays text information according to the determined display mode. The display unit 150 displays text information superimposed on content, for example. Display unit 150 may display only text information.

Communication terminal 100 is, for example, a PC (Personal Computer). Communication terminal 100 may be a smartphone, a tablet terminal, or the like.

For example, the recognition device 200 performs speech recognition processing on content. For example, the recognition device 200 recognizes speech included in content and outputs character information as a recognition result. For example, the recognition device 200 performs speech recognition processing on audio content. The recognition device 200 may perform voice recognition processing on the voice included in the video content.

For example, the recognition device 200 performs speech recognition processing on the content in units of phrases. For example, the recognition device 200 executes speech recognition processing on a word-by-word basis. The recognition device 200 may perform voice recognition processing on a character-by-character basis.

For example, the recognition device 200 performs image recognition processing on image content. The recognition device 200 executes image recognition processing, for example, in order to recognize character information included in image content.

The recognition device 200 may perform image recognition processing to recognize a predetermined object included in the image content. The predetermined object is, for example, a person. When the recognition device 200 recognizes a predetermined object, the recognition result includes a recognition region that is an area including the object recognized by the recognition device 200 and recognition accuracy of the object recognized by the recognition device 200.

The recognition device 200 may perform voice recognition processing and image recognition processing on video content. For example, the recognition device 200 recognizes audio included in video content and outputs character information as a recognition result. The recognition device 200 executes image recognition processing, for example, in order to recognize a predetermined object included in video content. The recognition device 200 may perform image recognition processing to recognize character information included in video content.

The recognition device 200 may be any device that can perform recognition processing on content and send the recognition result to the communication terminal 100. The recognition device 200 is, for example, a server of GCP (registered trademark) (Google Cloud Platform) of Google (registered trademark). The recognition device 200 may be an AWS (registered trademark) (Amazon Web Services) server of Amazon (registered trademark), or the like.

In recent years, services have appeared that provide users with recognition results obtained by an external recognition device performing recognition processing on content. A user may desire to set a display mode for displaying character information on a display unit of a communication terminal based on the recognition accuracy of character information included in recognition results provided by an external recognition device. For example, when the content is a recorded television program, the user may desire a display mode that displays more text information, including text information with low recognition accuracy. On the other hand, when the content is a real-time broadcast television program, the user may desire a display mode that displays only character information with high recognition accuracy. However, in conventional communication terminals, a user cannot set the display mode of character information included in the recognition result provided by an external recognition device based on the recognition accuracy of the character information. As a result, conventional communication terminals have been unable to display character information provided from external recognition devices in a display format that flexibly corresponds to the user's wishes.

On the other hand, according to the communication terminal 100 according to the present embodiment, based on the recognition accuracy of character information included in the recognition result received from the recognition device 200 and the threshold set by the user of the communication terminal 100, Decide how information is displayed. The display unit 150 displays the character information in accordance with the determined display mode. The user of the communication terminal 100 according to this embodiment can set the display mode of text information by setting a threshold value on the communication terminal 100. For example, by setting a low threshold, the user of the communication terminal 100 can set a display mode that displays more text information, including text information with low recognition accuracy. On the other hand, by setting a high threshold value, the user of the communication terminal 100 can set the display mode to display only character information with high recognition accuracy. Thereby, the communication terminal 100 according to the present embodiment can display the character information received from the recognition device 200 in a display mode that flexibly corresponds to the user's wishes.

FIG. 2 schematically shows an example of character information displayed by the display unit 150. In FIG. 2, an example will be described in which character information obtained by performing speech recognition processing on a phrase-by-phrase basis is displayed on the display unit 150.

The recognition device 200 executes a recognition process on the audio content and acquires a plurality of character information in phrase units such as "kono", "problem", "correct answer is", and "A desu". The recognition accuracy of each of the plurality of character information is highest in the order of "A is.", "The correct answer is", "this", and "problem". Communication terminal 100 receives recognition results from recognition device 200 via network 20 .

The communication terminal 100 sets the display mode of the plurality of character information based on the recognition accuracy of each of the plurality of character information included in the recognition result received from the recognition device 200 and the threshold set by the user of the communication terminal 100. do. Here, the recognition accuracy of "It's A.", "The correct answer is", and "this" is higher than the threshold set by the user of the communication terminal 100, and the recognition accuracy of "problem" is higher than the threshold set by the user of the communication terminal 100. The explanation will be continued assuming that the value is lower than the threshold value.

For example, the communication terminal 100 replaces "in question" with a predetermined symbol "-" and displays it in the color of the first opacity, which is the default opacity, and displays "this" in the color of the first opacity, which is the default opacity. ``The correct answer is'' is displayed in a color with a second opacity higher than the first opacity, and ``A is'' is displayed in a color with a third opacity higher than the second opacity. Decide the display mode so that it will be displayed. Display unit 150 displays character information according to the determined display mode.

The communication terminal 100 according to the present embodiment can prevent character information with low recognition accuracy from being displayed on the display unit 150 by replacing the character information with low recognition accuracy with a predetermined symbol. Moreover, the communication terminal 100 according to the present embodiment can make the user easily understand the recognition accuracy of the character information by changing the color of the character information according to the recognition accuracy.

FIG. 3 schematically shows another example of character information displayed by the display unit 150. In FIG. 3, an example will be described in which character information obtained by performing speech recognition processing on a phrase-by-phrase basis is displayed on the display unit 150. In FIG. 3, parts different from those in FIG. 2 will be mainly explained.

For example, the communication terminal 100 replaces "problem" with a predetermined symbol "-" and displays it in the first size which is the default size, displays "this" in the first size, and displays "correct answer". The display mode is determined so that "A" is displayed in a second size larger than the first size, and "A desu" is displayed in a third size larger than the second size. Display unit 150 displays character information according to the determined display mode.

FIG. 4 schematically shows another example of character information displayed by the display unit 150. In FIG. 4, an example will be described in which character information obtained by performing speech recognition processing on a phrase-by-phrase basis is displayed on the display unit 150. In FIG. 4, parts different from those in FIG. 2 will be mainly explained.

For example, the communication terminal 100 may use the first opacity color, which is the default opacity, without replacing "offending" with a predetermined symbol "-" that is lower than the threshold set by the user of the communication terminal 100. "This" is displayed in a first opacity color, "The correct answer is" is displayed in a second opacity color higher than the first opacity, and "A is" is displayed in a second opacity color. The display mode is determined so that the color is displayed in a color with a third opacity higher than the second opacity. Display unit 150 displays character information according to the determined display mode.

FIG. 5 schematically shows an example of the functional configuration of the communication terminal 100. The communication terminal 100 includes a content acquisition section 102, a threshold storage section 104, a threshold setting section 106, a recognition processing request section 108, a recognition result reception section 110, a display mode determination section 112, a display control section 114, and a display section 150. Note that it is not essential that the communication terminal 100 include all of these configurations.

The content acquisition unit 102 acquires content. The content acquisition unit 102 acquires content, for example, by receiving content from an external device. The content acquisition unit 102 may acquire content by generating content.

The threshold value storage unit 104 stores threshold values. The threshold value storage unit 104 stores, for example, one threshold value. The threshold value storage unit 104 may store a plurality of threshold values.

The threshold value storage unit 104 stores, for example, a threshold value for determining the display mode of character information included in the recognition result received from the recognition device 200. The threshold value storage unit 104 may store a threshold value for determining the display mode of the recognition area included in the recognition result received from the recognition device 200.

The threshold value setting unit 106 sets a threshold value. The threshold value setting unit 106 sets, for example, one threshold value. The threshold value setting unit 106 may set a plurality of threshold values.

The threshold value setting unit 106 sets a threshold value, for example, by accepting a user's input using an input device included in the communication terminal 100. The threshold value setting unit 106 may set the threshold value by receiving the threshold value from an external device. The threshold value setting unit 106 stores the set threshold value in the threshold value storage unit 104.

The recognition processing request unit 108 transmits a recognition processing request to the recognition device 200 to request recognition processing for the content acquired by the content acquisition unit 102. The recognition processing request unit 108 transmits a recognition processing request to the recognition device 200 via the network 20, for example.

The recognition processing request includes, for example, content to be recognized. The recognition processing request may include a URL (Uniform Resource Locator) of a database in which content to be recognized is stored. In this case, the recognition device 200 accesses a database storing content to be recognized according to the URL.

The recognition result receiving unit 110 receives recognition results from the recognition device 200. The recognition result receiving unit 110 receives the recognition result from the recognition device 200 via the network 20, for example.

The recognition result receiving unit 110 receives, for example, a recognition result obtained by the recognition device 200 performing recognition processing in response to a recognition processing request. The recognition result receiving unit 110 may receive recognition results other than the recognition results obtained by the recognition device 200 performing the recognition process in response to the recognition process request.

The display mode determining unit 112 determines the display mode of character information included in the recognition result received by the recognition result receiving unit 110. The display mode determining unit 112 determines the display mode of the text information, for example, based on a user's input received using an input device included in the communication terminal 100.

For example, the display mode determining unit 112 determines the display mode of the character information based on the recognition accuracy of the character information included in the recognition result received by the recognition result receiving unit 110 and the threshold value stored in the threshold value setting unit 106. decide. The display mode determining unit 112 determines whether the plurality of pieces of character information are displayed based on the recognition accuracy of each piece of character information included in the recognition result received by the threshold value setting unit 106 and the threshold value stored in the threshold value setting unit 106. The display mode may be determined.

For example, the display mode determining unit 112 displays character information when recognition accuracy is higher than a threshold (hereinafter referred to as "first threshold"), and displays character information in advance when recognition accuracy is lower than the first threshold. The display mode is determined so that the specified symbol is replaced and displayed. For example, the display mode determining unit 112 determines the display mode so that character information whose recognition accuracy is lower than the first threshold value is replaced with a symbol "-" and displayed.

The display mode determining unit 112 determines the display mode, for example, so as to display character information whose recognition accuracy is the same as the first threshold. The display mode determining unit 112 may determine the display mode so that character information whose recognition accuracy is the same as the first threshold value is replaced with a predetermined symbol and displayed.

For example, the display mode determining unit 112 determines the character information based on another threshold (hereinafter referred to as “second threshold”) that is stored in the threshold storage unit 104 and is lower than the first threshold. Decide the display mode. For example, when the recognition accuracy is lower than the first threshold value and higher than the second threshold value, the display mode determining unit 112 replaces the character information with a predetermined symbol and displays it, and when the recognition accuracy is lower than the second threshold value. The display mode is determined so that the character information is replaced with another predetermined symbol different from the predetermined symbol and displayed when the predetermined symbol is lower than the predetermined symbol. For example, the display mode determining unit 112 displays character information whose recognition accuracy is lower than the first threshold value and higher than the second threshold value by replacing it with the symbol "-", and displays character information whose recognition accuracy is lower than the second threshold value. The display mode is determined so that the information is displayed with the symbol "x" replaced. Thereby, the communication terminal 100 allows the user to easily understand character information with particularly low recognition accuracy.

The display mode determining unit 112 determines the display mode, for example, so that character information whose recognition accuracy is the same as the second threshold value is replaced with a predetermined symbol and displayed. The display mode determining unit 112 may determine the display mode so that character information whose recognition accuracy is the same as the second threshold value is replaced with another predetermined symbol and displayed.

The display mode determining unit 112 determines the display mode, for example, so that character information whose recognition accuracy is higher than the first threshold is displayed in an emphasized manner according to the recognition accuracy. The display mode determining unit 112 determines the display mode such that, for example, the higher the recognition accuracy, the more emphasized the character information is displayed.

For example, the display mode determining unit 112 determines the character information based on another threshold (hereinafter referred to as a “third threshold”) that is stored in the threshold storage unit 104 and is higher than the first threshold. Decide the display mode. For example, when the recognition accuracy is higher than the third threshold, the display mode determining unit 112 displays the character information with more emphasis than when the recognition accuracy is higher than the first threshold and lower than the third threshold. Then, determine the display mode.

For example, the display mode determining unit 112 displays character information whose recognition accuracy is the same as the third threshold in a more emphasized manner than character information whose recognition accuracy is higher than the first threshold and lower than the third threshold. to determine the display mode. The display mode determining unit 112 displays character information whose recognition accuracy is the same as the third threshold in the same display mode as character information whose recognition accuracy is higher than the first threshold and lower than the third threshold. The display mode may be determined accordingly.

The display mode determining unit 112 determines the display mode, for example, so that the color of the character information is changed and displayed depending on the recognition accuracy. The display mode determining unit 112 determines the display mode, for example, so that the hue of the color of the character information is changed and displayed according to the recognition accuracy. For example, when changing the hue between red and blue, the display mode determining unit 112 changes the display mode so that the higher the recognition accuracy is, the more blue the hue is displayed, and the lower the recognition accuracy is, the more red the hue is. Determine. The display mode determining unit 112 displays character information whose recognition accuracy is higher than the third threshold in a different hue from character information whose recognition accuracy is higher than the first threshold and lower than the third threshold. The display mode may also be determined.

The display mode determining unit 112 determines the display mode, for example, so that the color saturation of the character information is changed and displayed according to the recognition accuracy. The display mode determining unit 112 determines the display mode such that, for example, the higher the recognition accuracy, the higher the saturation of the color of the character information is displayed. The display mode determining unit 112 displays character information whose recognition accuracy is higher than a third threshold value with higher color saturation than character information whose recognition accuracy is higher than the first threshold value and lower than the third threshold value. The display mode may be determined as follows.

The display mode determining unit 112 determines the display mode, for example, so that the text information is displayed with the brightness of the color changed depending on the recognition accuracy. The display mode determining unit 112 determines the display mode such that, for example, the higher the recognition accuracy, the higher the brightness of the color of the character information is displayed. The display mode determining unit 112 displays character information whose recognition accuracy is higher than the third threshold value with higher color brightness than character information whose recognition accuracy is higher than the first threshold value and lower than the third threshold value. The display mode may be determined as follows.

The display mode determining unit 112 may determine the display mode so that the opacity of the color of the character information is changed depending on the recognition accuracy. The display mode determining unit 112 determines the display mode such that, for example, the higher the recognition accuracy, the higher the opacity of the color of the character information is displayed. The display mode determining unit 112 displays character information whose recognition accuracy is higher than a third threshold value with higher color opacity than character information whose recognition accuracy is higher than the first threshold value and lower than the third threshold value. The display mode may be determined as follows.

The display mode determining unit 112 determines the display mode, for example, so that the size of the character information is changed and displayed depending on the recognition accuracy. The display mode determining unit 112 determines the display mode such that, for example, the higher the recognition accuracy, the larger the size of the character information is displayed. The display mode determining unit 112 displays character information whose recognition accuracy is higher than the third threshold in a larger size than character information whose recognition accuracy is higher than the first threshold and lower than the third threshold. , the display mode may be determined.

The display mode determining unit 112 determines the display mode, for example, so that the thickness of the character information is changed and displayed depending on the recognition accuracy. The display mode determining unit 112 determines the display mode such that, for example, the higher the recognition accuracy, the thicker the character information is displayed. The display mode determining unit 112 sets the display mode so that character information whose recognition accuracy is higher than the third threshold is displayed thicker than character information whose recognition accuracy is higher than the first threshold and lower than the third threshold. You may decide.

The display mode determining unit 112 may determine the display mode so that the font of character information is changed and displayed according to the recognition accuracy. For example, the display mode determining unit 112 displays character information whose recognition accuracy is higher than a third threshold value in a different font from character information whose recognition accuracy is higher than the first threshold value and lower than the third threshold value. The display mode may be determined accordingly.

For example, the display mode determining unit 112 emphasizes a predetermined symbol according to the recognition accuracy, in the same manner as when character information whose recognition accuracy is higher than the first threshold value is emphasized and displayed according to the recognition accuracy. The display mode is determined so that the The display mode determining unit 112 determines the display mode such that, for example, the lower the recognition accuracy, the more emphasized a predetermined symbol is displayed.

The display control unit 114 controls the display on the display unit. The display control unit 114 controls, for example, the display on the display unit 150 included in the communication terminal 100. The display control unit 114 may control the display on a display unit included in an external device. The display section displays character information under the control of the display control section 114.

The display control unit 114 controls the display unit to display the character information included in the recognition result received by the recognition result receiving unit 110, for example, based on the display mode determined by the display mode determining unit 112. The display control section 114 may control the display section to display a plurality of pieces of character information based on the display mode determined by the display mode determining section 112.

When the target content is content that changes over time, such as video content and audio content, the threshold value setting unit 106 sets a plurality of characters that are the recognition results for each predetermined period for each predetermined period. The first threshold value may be changed depending on the recognition accuracy of the information. For example, the threshold value setting unit 106 changes the first threshold value according to the ratio of character information whose recognition accuracy is higher than the first threshold value for a plurality of pieces of character information in a predetermined period. For example, the threshold setting unit 106 sets the first threshold so that the proportion of character information whose recognition accuracy is higher than the first threshold for a plurality of pieces of character information during a predetermined period satisfies a predetermined condition. change.

The predetermined conditions are set by the user of communication terminal 100. The predetermined conditions may be set when the communication terminal 100 is manufactured.

The predetermined condition is, for example, that the proportion of character information whose recognition accuracy is higher than a first threshold value for a plurality of pieces of character information during a predetermined period is a predetermined proportion. As the predetermined ratio, an arbitrary ratio such as 70% or 80% may be set. The predetermined condition may be, for example, that the proportion of character information whose recognition accuracy is higher than a first threshold value for a plurality of character information within a predetermined period is within a predetermined range. . Any range such as 50% to 70%, 60% to 80%, etc. may be set as the predetermined range.

For example, if the proportion of character information whose recognition accuracy is higher than the first threshold value for a plurality of pieces of character information during a predetermined period is higher than the predetermined proportion, the threshold value setting unit 106 increases the first threshold value. change. On the other hand, the threshold value setting unit 106 sets the first threshold value when the proportion of character information whose recognition accuracy is higher than the first threshold value for a plurality of pieces of character information during a predetermined period is lower than the predetermined proportion. Change lower. As a result, for example, the display mode determining unit 112 displays text information when the recognition accuracy is higher than the first threshold, and replaces the text information with a predetermined symbol when the recognition accuracy is lower than the first threshold. In some cases, the ratio of text information to symbols displayed as a result can be adjusted to a desired ratio.

For example, the threshold setting unit 106 sets the first Change the threshold to a higher value. On the other hand, the threshold setting unit 106 sets the first change the threshold value to a lower value. As a result, for example, the display mode determining unit 112 displays text information when the recognition accuracy is higher than the first threshold, and replaces the text information with a predetermined symbol when the recognition accuracy is lower than the first threshold. In some cases, the ratio of text information to symbols displayed as a result can be within a desired range.

The threshold value setting unit 106 changes the first threshold value within a range from a predetermined minimum threshold value to a predetermined maximum threshold value, for example. The predetermined minimum threshold and the predetermined maximum threshold are set by the user of communication terminal 100. The predetermined minimum threshold and the predetermined maximum threshold may be set at the time of manufacturing the communication terminal 100.

For example, the threshold setting unit 106 determines that the first threshold is a predetermined maximum threshold, and that the proportion of character information whose recognition accuracy is higher than the first threshold for a plurality of character information during a predetermined period is , is higher than the predetermined percentage or the upper limit of the predetermined range, the first threshold is not changed. In this case, the display mode determining unit 112 determines the display mode so that character information whose recognition accuracy is higher than the maximum threshold is displayed, and character information whose recognition accuracy is lower than the maximum threshold is displayed by replacing it with a predetermined symbol. do. As a result, for example, if the proportion of character information whose recognition accuracy is higher than the maximum threshold for multiple pieces of character information during a predetermined period is higher than the predetermined proportion or the upper limit of the predetermined range, the display The mode determining unit 112 can prevent character information whose recognition accuracy is higher than the maximum threshold from being displayed in place of a predetermined symbol.

For example, the threshold setting unit 106 determines that the first threshold is a predetermined minimum threshold, and that the proportion of character information whose recognition accuracy is higher than the first threshold for a plurality of character information during a predetermined period is , the first threshold is not changed if it is lower than a predetermined percentage or the lower limit of a predetermined range. In this case, the display mode determining unit 112 determines the display mode so that character information whose recognition accuracy is higher than the minimum threshold is displayed, and character information whose recognition accuracy is lower than the minimum threshold is displayed by replacing it with a predetermined symbol. do. As a result, for example, if the proportion of character information whose recognition accuracy is higher than the minimum threshold for multiple pieces of character information during a predetermined period is lower than the predetermined proportion or the lower limit of the predetermined range, the display The mode determining unit 112 can prevent character information whose recognition accuracy is lower than the minimum threshold from being displayed.

For example, when the recognition device 200 performs recognition processing on image content and outputs a recognition area as a recognition result, the display mode determination unit 112 determines the recognition area included in the recognition result received by the recognition result reception unit 110. Determine the display mode of the area. The display control section 114 may control the display section to display the recognition area based on the display mode determined by the display mode determining section 112.

The display mode determining unit 112 determines the recognition area based on, for example, the recognition accuracy of the object recognized by the recognition device 200 and a threshold stored in the threshold setting unit 106 (hereinafter referred to as the “fourth threshold”). Determine the display mode. For example, the display mode determining unit 112 processes and displays the recognition area as an image when the recognition accuracy of the object is higher than the fourth threshold, and displays the recognition area as an image when the recognition accuracy of the object is lower than the fourth threshold. The display mode of the recognition area is determined so that it is displayed without being processed.

The display mode determining unit 112 determines the display mode of the recognition area such that, for example, when the recognition accuracy of the object is the same as the fourth threshold, the recognition area is subjected to image processing and displayed. The display mode determining unit 112 may determine the display mode of the recognition area so that the recognition area is displayed without image processing when the recognition accuracy of the object is the same as the fourth threshold.

The image processing process may be any type of process as long as it can protect personal information. The image processing process is, for example, mosaic processing. The image processing process may be a filling process. The fourth threshold may be preset low to ensure that personal information is protected.

When the recognition device 200 performs recognition processing on video content and outputs character information and recognition areas as recognition results, the display mode determination unit 112 determines whether the recognition results included in the recognition results received by the recognition result reception unit 110 , the display mode of the character information and the recognition area may be determined respectively. The display control section 114 may control the display section to display the character information and the recognition area, respectively, based on the display mode determined by the display mode determining section 112. For example, if the video content includes a person speaking, the person is subjected to mosaic processing, and an image including text information corresponding to the content of the utterance is displayed on the display unit.

FIG. 6 schematically shows an example of the flow of processing by the communication terminal 100. In FIG. 6, an example will be described in which the content is a recorded television program.

In step (step may be abbreviated as S) 102, the recognition result receiving unit 110 receives the recognition result from the recognition device 200 via the network 20. For example, the recognition result receiving unit 110 receives from the recognition device 200 all the recognition results obtained by the recognition device 200 performing recognition processing on the content.

In S104, the display mode determining unit 112 determines the display mode of the character information included in the recognition result based on the recognition accuracy included in the recognition result and the first threshold stored in the threshold storage unit 104. In S106, the display mode determining unit 112 determines whether the display mode of all character information has been determined. If the display mode determining unit 112 determines that the display mode of all character information has been determined, the process advances to S108. If the display mode determining unit 112 determines that the display mode of all character information has not been determined, the process returns to S102.

In S108, the display control unit 114 controls the display unit 150 to display the text information based on the display mode determined by the display mode determining unit 112 in S104. The display section 150 displays character information under the control of the display control section 114. After that, the process ends.

FIG. 7 schematically shows another example of the flow of processing by the communication terminal 100. In FIG. 7, an example will be described in which the content is a real-time broadcast television program.

In S202, the recognition result receiving unit 110 receives the recognition result from the recognition device 200 via the network 20. The recognition result receiving unit 110 receives from the recognition device 200, for example, a part of the recognition result obtained by the recognition device 200 performing recognition processing on the content.

In S204, the display mode determining unit 112 determines the display mode of the character information included in the recognition result based on the recognition accuracy included in the recognition result and the first threshold stored in the threshold storage unit 104. In S206, the display control unit 114 controls the display unit 150 to display the text information based on the display mode determined by the display mode determining unit 112 in S204. The display section 150 displays character information under the control of the display control section 114.

In S208, the communication terminal 100 determines whether the recognition result receiving unit 110 has received from the recognition device 200 all the recognition results obtained by the recognition device 200 performing recognition processing on the content. When the communication terminal 100 determines that the recognition result receiving unit 110 has received all the recognition results from the recognition device 200, the process ends. If the communication terminal 100 determines that the recognition result receiving unit 110 has not received all recognition results from the recognition device 200, the process returns to S202.

FIG. 8 schematically shows an example of the hardware configuration of a computer 1200 functioning as the communication terminal 100. The program installed on the computer 1200 causes the computer 1200 to function as one or more "parts" of the apparatus according to the above embodiment, or causes the computer 1200 to perform operations associated with the apparatus according to the above embodiment or the one or more "parts" of the apparatus according to the above embodiment. Multiple units may be executed and/or the computer 1200 may execute a process or a step of a process according to the embodiments described above. Such programs may be executed by CPU 1212 to cause computer 1200 to perform certain operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216, which are interconnected by a host controller 1210. The computer 1200 also includes input/output units such as a communication interface 1222, a storage device 1224, a DVD drive 1226, and an IC card drive, which are connected to the host controller 1210 via an input/output controller 1220. DVD drive 1226 may be a DVD-ROM drive, a DVD-RAM drive, or the like. Storage device 1224 may be a hard disk drive, solid state drive, or the like. Computer 1200 also includes legacy input/output units, such as ROM 1230 and keyboard 1242, which are connected to input/output controller 1220 via input/output chip 1240.

The CPU 1212 operates according to programs stored in the ROM 1230 and RAM 1214, thereby controlling each unit. Graphics controller 1216 obtains image data generated by CPU 1212, such as in a frame buffer provided in RAM 1214 or itself, and causes the image data to be displayed on display device 1218.

Communication interface 1222 communicates with other electronic devices via a network. Storage device 1224 stores programs and data used by CPU 1212 within computer 1200. The DVD drive 1226 reads a program or data from a DVD-ROM 1227 or the like and provides it to the storage device 1224. The IC card drive reads programs and data from and/or writes programs and data to the IC card.

ROM 1230 stores therein programs that are dependent on the computer 1200 hardware, such as a boot program that is executed by the computer 1200 upon activation. I/O chip 1240 may also connect various I/O units to I/O controller 1220 via USB ports, parallel ports, serial ports, keyboard ports, mouse ports, etc.

The program is provided by a computer readable storage medium such as a DVD-ROM 1227 or an IC card. The program is read from a computer-readable storage medium, installed in storage device 1224, RAM 1214, or ROM 1230, which are also examples of computer-readable storage media, and executed by CPU 1212. The information processing described in these programs is read by the computer 1200 and provides coordination between the programs and the various types of hardware resources described above. An apparatus or method may be configured to implement the operation or processing of information according to the use of computer 1200.

For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 executes a communication program loaded into the RAM 1214 and sends communication processing to the communication interface 1222 based on the processing written in the communication program. You may give orders. The communication interface 1222 reads transmission data stored in a transmission buffer area provided in a recording medium such as a RAM 1214, a storage device 1224, a DVD-ROM 1227, or an IC card under the control of the CPU 1212, and transmits the read transmission data. Data is transmitted to the network, or received data received from the network is written to a reception buffer area provided on the recording medium.

Further, the CPU 1212 causes the RAM 1214 to read all or a necessary part of a file or database stored in an external recording medium such as a storage device 1224, a DVD drive 1226 (DVD-ROM 1227), or an IC card. Various types of processing may be performed on the data. CPU 1212 may then write the processed data back to an external storage medium.

Various types of information, such as various types of programs, data, tables, and databases, may be stored on a recording medium and subjected to information processing. CPU 1212 performs various types of operations, information processing, conditional determination, conditional branching, unconditional branching, and information retrieval on data read from RAM 1214 as described elsewhere in this disclosure and specified by the program's instruction sequence. Various types of processing may be performed, including /substitutions, etc., and the results are written back to RAM 1214. Further, the CPU 1212 may search for information in a file in a recording medium, a database, or the like. For example, when a plurality of entries are stored in a recording medium, each having an attribute value of a first attribute associated with an attribute value of a second attribute, the CPU 1212 selects the first entry from among the plurality of entries. Search for an entry whose attribute value matches the specified condition, read the attribute value of the second attribute stored in the entry, and then set the attribute value to the first attribute that satisfies the predetermined condition. An attribute value of the associated second attribute may be obtained.

The programs or software modules described above may be stored in a computer-readable storage medium on or near computer 1200. Also, a storage medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby allowing the program to be transferred to the computer 1200 via the network. provide.

Blocks in the flowcharts and block diagrams of the present embodiments may represent stages in a process in which an operation is performed or a "part" of a device responsible for performing the operation. Certain steps and units may be provided with dedicated circuitry, programmable circuitry provided with computer readable instructions stored on a computer readable storage medium, and/or provided with computer readable instructions stored on a computer readable storage medium. May be implemented by a processor. Dedicated circuitry may include digital and/or analog hardware circuits, and may include integrated circuits (ICs) and/or discrete circuits. Programmable circuits can perform AND, OR, EXCLUSIVE OR, NAND, NOR, and other logical operations, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), etc. , flip-flops, registers, and memory elements.

A computer-readable storage medium may include any tangible device capable of storing instructions for execution by a suitable device such that a computer-readable storage medium with instructions stored therein may be illustrated in a flowchart or block diagram. A product will be provided that includes instructions that can be executed to create a means for performing specified operations. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer readable storage media include floppy disks, diskettes, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory). , Electrically Erasable Programmable Read Only Memory (EEPROM), Static Random Access Memory (SRAM), Compact Disk Read Only Memory (CD-ROM), Digital Versatile Disk (DVD), Blu-ray Disc, Memory Stick , integrated circuit cards, and the like.

Computer-readable instructions may include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state configuration data, or instructions such as Smalltalk®, JAVA®, C++, etc. any source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as may include.

The computer-readable instructions are for producing means for a processor of a general purpose computer, special purpose computer, or other programmable data processing device, or programmable circuit to perform the operations specified in the flowchart or block diagrams. A general purpose computer, special purpose computer, or other programmable data processor, locally or over a local area network (LAN), wide area network (WAN), such as the Internet, to execute the computer readable instructions. It may be provided in a processor or programmable circuit of the device. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, and the like.

Although the invention according to the present disclosure has been described above using the embodiments, the technical scope of the invention according to the present disclosure is not limited to the range described in the above embodiments. It will be apparent to those skilled in the art that various changes or improvements can be made to the embodiments described above. It is clear from the claims that such modifications or improvements may be included within the technical scope of the present invention.

The execution order of each process, such as an operation, a procedure, a step, and a stage in the apparatus, system, program, and method shown in the claims, specification, and drawings, specifically refers to "before" or "before". It should be noted that they can be implemented in any order unless the output of the previous process is used in the subsequent process. Even if the claims, specifications, and operational flows in the drawings are explained using "first," "next," etc. for convenience, this does not mean that it is essential to carry out the operations in this order. It's not a thing.

REFERENCE SIGNS LIST 10 system, 20 network, 100 communication terminal, 102 content acquisition unit, 104 threshold storage unit, 106 threshold setting unit, 108 recognition process request unit, 110 recognition result reception unit, 112 display mode determination unit, 114 display control unit, 150 display part, 200 recognition device, 1200 computer, 1210 host controller, 1212 CPU, 1214 RAM, 1216 graphic controller, 1218 display device, 1220 input/output controller, 1222 communication interface, 1224 storage device, 1226 DVD drive, 1227 DVD-ROM, 1230 ROM, 1240 input/output chip

Claims (13)

A communication terminal,
a threshold value storage unit that stores a threshold value set by a user of the communication terminal;
Receiving recognition results from the recognition device, including a plurality of character information obtained by the recognition device performing recognition processing on the content and recognition accuracy of each of the plurality of character information. Department and
a display mode determining unit that determines a display mode of the plurality of character information based on the recognition accuracy and the threshold value of each of the plurality of character information;
a display control unit that controls a display unit to display the plurality of character information based on the display mode determined by the display mode determination unit ;
a threshold value setting unit that changes the threshold value according to a ratio of character information for which the recognition accuracy is higher than the threshold value for the plurality of character information during a predetermined period;
A communication terminal equipped with . The threshold value setting unit changes the threshold value so that a ratio of character information with higher recognition accuracy than the threshold value to the plurality of character information items during a predetermined period falls within a predetermined range. The communication terminal according to item 1. The threshold value setting unit changes the threshold value such that a ratio of character information having higher recognition accuracy than the threshold value to the plurality of character information in a predetermined period becomes a predetermined ratio. 1. The communication terminal according to 1. The display mode determining unit is configured to display the character information when the recognition accuracy is higher than the threshold, and to replace and display the character information with a predetermined symbol when the recognition accuracy is lower than the threshold. The communication terminal according to any one of claims 1 to 3 , wherein the communication terminal determines the display mode. The threshold storage unit further stores another threshold lower than the threshold set by the user,
The display mode determining unit replaces and displays the character information with the predetermined symbol when the recognition accuracy is lower than the threshold and higher than the other threshold, and when the recognition accuracy is lower than the other threshold. determining the display mode so as to display the character information by replacing it with another predetermined symbol different from the predetermined symbol if the
The communication terminal according to claim 4 . The communication terminal according to claim 4 or 5, wherein the display mode determining unit determines the display mode so that the lower the recognition accuracy, the more emphasized the predetermined symbol is displayed. Any one of claims 1 to 5, wherein the display mode determining unit determines the display mode so that the character information whose recognition accuracy is higher than the threshold value is emphasized and displayed according to the recognition accuracy. Communication terminals described in section. The communication terminal according to claim 7 , wherein the display mode determining unit determines the display mode so that the higher the recognition accuracy, the more emphasized the character information is displayed. The threshold storage unit further stores another threshold higher than the threshold set by the user,
The display mode determining unit is configured to display the character information with emphasis when the recognition accuracy is higher than the other threshold value than when the recognition accuracy is higher than the threshold value and lower than the other threshold value. , determining the display mode;
The communication terminal according to claim 7 . The communication terminal according to any one of claims 7 to 9 , wherein the display mode determining unit determines the display mode so that the character information is displayed with a color changed depending on the recognition accuracy. The communication terminal according to any one of claims 7 to 10, wherein the display mode determining unit determines the display mode so that the size of the character information is changed and displayed according to the recognition accuracy. A program for causing a computer to function as the communication terminal according to any one of claims 1 to 11. A display method performed by a computer, the display method comprising:
a threshold storing step of storing a threshold set by a user of the computer;
Receiving recognition results from the recognition device, including a plurality of character information obtained by the recognition device performing recognition processing on the content and recognition accuracy of each of the plurality of character information. stages and
a display mode determining step of determining a display mode of the plurality of character information based on the recognition accuracy and the threshold value of each of the plurality of character information;
a display control step of controlling a display unit to display the plurality of character information based on the display mode determined in the display mode determining step ;
a threshold value setting step of changing the threshold value according to a ratio of character information for which the recognition accuracy is higher than the threshold value for the plurality of character information during a predetermined period;
A display method comprising :

Images