JP6533897B2 - Image data organization system, image data organization device, image data organization server device program, and image data organization device program - Google Patents

Description

  The present invention relates to an image data organization system, an image data organization device, an image data organization server device program, an image data organization terminal device program, and an image data organization device program.

  Currently, smartphones are widely used. A smartphone has various functions as well as a telephone function. The multifunctionality of a smartphone is ensured by various functions such as an operating system executed by a microprocessor (MPU: microprocessor), application software and a camera function built in the smartphone, a GPS (Global Positioning System) function, and the like. The main operating systems currently prevailing are Android (Android: registered trademark) and IOS (iOS: registered trademark).

  With the development of the Internet, cloud computing technology has been provided which can use services provided by servers present on a network without being aware of those servers.

  Patent Document 1 discloses a camera that obtains luminance data by imaging the luminance of skin that changes according to blood flow, a display that displays the image, and RR interval time-series data that detects RR interval time-series data from the luminance data. A detection processing unit for performing the Fourier transform on the RR interval time-series data, and a high frequency component / low frequency component detection processing unit for separating the high frequency component and the low frequency component of the signal subjected to the Fourier transform processing; An autonomic nerve function evaluation device comprising: an analysis result data output processing unit for outputting analysis result data for displaying on a display the activity of the parasympathetic nerve corresponding to the high frequency component and the activity of the sympathetic nerve corresponding to the low frequency component; Etc. are described.

  Fractal dimension is proposed as an index that quantifies visual stimuli. In addition, methods for calculating the fractal order of an image have also been studied. For example, Non-Patent Document 1 describes that the fractal dimension quantifies the visual stimulus from its shape, and the feature represented by the value is called "complexity" of the stimulus. Non-Patent Document 2 describes a method of estimating a fractal dimension that indicates the complexity of the unevenness of the density surface of the image density space. Also, the relationship between image analysis using fractal theory and human psychology has been studied. For example, in Non-Patent Document 3, the eye camera is used to examine the relationship between the concentration position of the viewpoint and the fractal dimension using six pictures for three men and women, and the viewpoint tends to be concentrated at a location where the fractal order is high. It is described that it is seen. In Non-Patent Document 4, a correlation analysis of the fractal order of photographs of smiles and true faces and the positivity of the questionnaire was performed on 23 men and women. As a result, it is described that the fractal dimension and favorability of the whole face are correlated, and the smile order tends to have a higher fractal order. In Non-Patent Document 5, a questionnaire of fractal dimension and subjectivity was conducted on package design of 45 types of canned beers with low recognition for 23 men and women, and canned beer with package design with high fractal dimension It is stated that high purchasing effects can be seen.

Patent publication 2014-140587 gazette

Cutting J., & Garvin, J. J., Fractal curves and complexity, Perception & Psychophysics, 42, 365-370, 1987 Pentland A. P., Fractal-based description of sciences, IEEE Trans. Pattern Analysis and Machine Intelligence, PAMI-6, pp. 661-674, 1984 Nagai M., Oyama-Higa M., & Miao T., Relationship between images image location and fractal dimension, IEEE International Conference on Systems, Man and Cybernetics, Oct. 7-10, 2007, Montreal, pp. 4014-4018, 2007 Oyama-Higa M., Miao T., & Itoh T, Relationship between fractal dimensions and agreement on facial imagery, Computational Models for Life Sciences-CMLS '07: 2007 International Symposium on Computational Models of Life Sciences, AIP Conference Proceedings 952, pp. 104-113, 2007 Oyama-Higa Mayumi, Journal of Business Studies 59 (2), The Kinki University Journal of business and economics Journal pp.77-93, December, 2012

  In recent years, digital techniques have become widespread in image capture, replacing techniques using films that make use of chemical changes. In an image recording apparatus for capturing still images or moving images using digital technology, it is possible to store captured image data in a storage device inexpensively and easily. It is widely used to organize data. For this reason, unnecessary image data tends to be captured in a large amount. As a result, processing for organizing necessary image data from a large amount of image data is required.

  In this case, it takes time and effort for a person to visually check the captured images and to arrange them while judging them one by one. Therefore, if automatic arrangement can be performed based on a predetermined arrangement standard, it is convenient because time and effort can be saved. Furthermore, if it is possible to automatically determine whether or not an object suitable for imaging by a person based on a predetermined organization standard and automatically image, it is possible to save time and effort to organize image data afterward At the same time, it is possible to avoid missing the imaging chance, and to prevent unnecessary consumption of the storage capacity by imaging too much for fear of shooting failure, so reducing the storage capacity of the storage device storing the image. You can also.

  The present invention has been made in view of such problems, and in the case where an image to be captured satisfies a prearrangement standard, a technique for informing the operator of the apparatus whether or not the prearrangement standard is determined in advance. A technique for automatically imaging and a technique for automatically organizing a plurality of captured images are provided.

The image data organization system of the present invention is an image data organization system including an image data organization terminal device and an image data organization server device, and the image data organization terminal device captures an image to obtain image data. Unit, a time-series heartbeat data acquisition unit for acquiring time-series heartbeat data according to a heartbeat when acquiring the image data, and a time-series heartbeat data output for outputting the time-series heartbeat data to the image data organizing server device A processing unit, and an image display unit for displaying the captured images arranged in descending order of healing degree, which are output from the image data organizing server apparatus, the image data organizing server apparatus including: Sotein aligning time and healed detecting unit obtains a healed degree from series heartbeat data, the captured image in the descending order of the healed degree Comprising a processing unit.

  An image data organizing apparatus according to the present invention comprises an imaging unit for capturing and acquiring image data, a time-series heartbeat data acquisition unit for acquiring time-series heartbeat data according to a heartbeat when acquiring the image data, and the time-series heartbeat data. A Fourier transform processing unit for Fourier transforming data, a high frequency component / low frequency component detection processing unit for separating a high frequency component and a low frequency component of the signal subjected to the Fourier transform, and at least a standard obtained by measuring at least the high frequency component An autonomic nerve analysis engine processing unit for obtaining a healing degree by dividing by high frequency components, a sorting processing unit for arranging the image data in descending order of the healing degree, and image data arranged in descending order of the healing degree And an image display unit to be displayed.

  Another image data organizing apparatus according to the present invention includes an imaging unit for capturing and acquiring image data, a time-series heartbeat data acquisition unit for acquiring time-series heartbeat data according to a heartbeat when acquiring the image data, and A heart rate detection unit for determining a heart rate from serial heart rate data; and a healing degree calculation unit for determining a healing degree according to the magnitude of a negative value obtained by subtracting the heart rate in normal times previously calculated from the heart rate The image processing apparatus may further include: a sorting processing unit that arranges the image data in descending order of healing degree; and an image display unit for displaying the image data arranged in descending order of healing degree.

  The image data organizing server device program according to the present invention is an image data organizing server device program to be executed by a computer of the image data organizing server device operating in cooperation with the image data organizing terminal device in the image data organizing system. A Fourier transform processing unit that performs Fourier transform on time-series heartbeat data acquired by the organizing terminal device; a high frequency component / low frequency component detection processing unit that separates high frequency components and low frequency components of the signal subjected to Fourier transform processing; And an autonomic nerve analysis engine processing unit for dividing the high frequency component by a reference high frequency component measured in advance to obtain a healing degree, and a sorting processing unit for arranging the captured images in descending order of the healing degree. To make the computer work.

  Another image data organizing server program according to the present invention is an image data organizing server program to be executed by a computer of an image data organizing server device operating in cooperation with an image data organizing terminal device in an image data organizing system, A healing rate according to the size of a negative value obtained by subtracting the heart rate in normal times, which is obtained beforehand from the heart rate, and a heart rate detection unit for obtaining a heart rate from time-series heart rate data acquired by the image data organizing terminal The computer is operated to include a healing degree computing unit, and a sorting processing unit for arranging the captured images in descending order of the healing degree.

  The image data organizing terminal device program of the present invention is an image data organizing terminal device program to be executed by a computer of the image data organizing terminal device operating in cooperation with the image data organizing server device in the image data organizing system. A time-series heartbeat data acquisition unit that acquires time-series heartbeat data according to heartbeats upon acquisition; a time-series heartbeat data output processing unit that outputs the time-series heartbeat data to the image data organizing server device; And the image display unit for displaying the captured images arranged in descending order of healing degree, which are output from the organizing server device.

  The image data organizing apparatus program according to the present invention is an image data organizing apparatus program to be executed by a computer of the image data organizing apparatus, which comprises: an imaging unit for capturing and acquiring image data; A time-series heartbeat data acquisition unit for acquiring time-series heartbeat data, a Fourier transform processing unit for Fourier transforming the time-series heartbeat data, and a high frequency component for separating high frequency components and low frequency components of the signal subjected to the Fourier transform processing · A low frequency component detection processing unit, an autonomic nerve analysis engine processing unit for obtaining at least a healing degree by dividing at least the high frequency component by a reference high frequency component, and the image data in descending order of the healing degree And an image display unit for displaying the image data arranged in descending order of the degree of healing. The computer to function in.

  Another image data organizing apparatus program according to the present invention is an image data organizing apparatus program to be executed by a computer of the image data organizing apparatus, which comprises: an imaging unit that captures an image to acquire image data; A time-series heartbeat data acquisition unit for acquiring time-series heartbeat data according to the time, a Fourier transform processing unit for Fourier transforming the time-series heartbeat data, and separating high frequency components and low frequency components of the signal subjected to the Fourier transform processing A high-frequency component / low-frequency component detection processing unit, an autonomic nerve analysis engine processing unit for obtaining a healing degree by dividing at least the high-frequency component by a reference high-frequency component measured in advance, the images in descending order of healing degree It has a sorting processing unit that arranges data, and an image display unit that displays the image data arranged in order of high healing degree. Said computer to function as.

According to the technique of the present invention, it is possible to automatically rearrange a plurality of captured images based on predetermined organization criteria. In addition, since the operator of the apparatus can know objectively before imaging, whether the image satisfies the predetermined arrangement standard or not, he or she does not get lost in subjective judgment as to whether or not imaging should be performed. Furthermore, automatic imaging can be performed when the predetermined arrangement standard is satisfied. These effects make it possible to effectively use a large number of images that are big data.

It is a figure showing an image data arrangement system of a 1st embodiment. It is a figure which shows Screen 1 thru | or Screen 4 displayed on the image data rearranging terminal device of 1st Embodiment. It is a figure which shows Screen 5 thru | or Screen 8 displayed on the image data rearranging terminal device of 1st Embodiment. It is a figure which shows Screen 9 displayed on the image data rearranging terminal device of 1st Embodiment, and Screen 10. FIG. It is a figure which shows the process in the image data rearranging terminal device of 1st Embodiment, and an image data rearranging server apparatus. It is a figure which shows the process following FIG. 5 in the image data rearranging terminal device and image data rearranging server device of 1st Embodiment. It is a figure which shows typically the advance logic process of 1st Embodiment. It is a figure which shows the time-sequential luminance data after completion | finish of processing of each step of the autonomic-nerve analysis engine processing in the image data organization server apparatus of 1st Embodiment. It is explanatory drawing about how to set the constant of the low pass filter of 1st Embodiment. It is another explanatory view about how to set a constant of a low pass filter of a 1st embodiment. It is a figure which shows the image data organization system of 4th Embodiment. It is a figure which shows Screen 21 thru | or Screen 24 displayed on the image data rearranging terminal device of 4th Embodiment. It is a figure which shows Screen 25 thru | or Screen 28 displayed on the image data rearranging terminal device of 4th Embodiment. It is a figure which shows the screen 29 displayed on the image data rearranging terminal device of 4th Embodiment. It is a figure which shows the process in the image data rearranging terminal device of 4th Embodiment, and an image data rearranging server apparatus. It is a figure which shows the process which continues in FIG. 15 in the image data rearranging terminal device and image data rearranging server device of 4th Embodiment.

  An image data organizing system, an image data organizing device, an image data organizing server device program, an image data organizing terminal device program, and an image data organizing device program according to embodiments (embodiments) of the invention will be described below. According to the third to third embodiments, the captured image data is automatically sorted afterward automatically based on a predetermined sorting standard, and the operator of the apparatus determines whether the image to be captured satisfies the predetermined sorting standard. The fourth to sixth embodiments described below, in addition to these techniques, automatically take an image when an image to be taken satisfies a predetermined arrangement standard. Related to technology. The seventh embodiment and the eighth embodiment relate to an example of a technique for combining parts of the first to sixth embodiments.

  In the embodiment, the movement of the heart is detected using the heart rate sensor, and the movement of the human heart is further detected from the movement of the heart, and the image data (visually an image) is detected based on the movement of the human heart. Automatically organize. In addition, when automatically organizing image data (visually, images) by another means, the nature of the image data (visually, images) itself is analyzed and automatically organized.

  The movement of the human mind is reflected in the heart rhythm. The heart rate sensor that detects the movement of the heart is a photoplethysmography method that captures changes in reflected light according to the flow of blood flowing through blood vessels that changes due to heart rhythm, and pressure sensors for blood pressure that pulsates according to the heart rhythm. The sphygmomanometer method is used to capture the current, and the electrodes are attached to the skin close to the heart, and the electrocardiographic method is used to capture the current flowing to the muscle that causes the heart rhythm. In the embodiment, sympathetic nerve activity or / and parasympathetic nerve activity obtained by analyzing time-series heart rate data from the heart rate sensor is used as an index for image reduction criteria.

The image data organizing technique from the viewpoint of computer processing of the embodiment (technique of image organizing from a visual point of view) includes an imaging unit that captures an image to acquire image data, and a time-series heartbeat according to a heart rate when acquiring image data. A time-series heartbeat data acquisition unit that acquires data, a healing degree detection unit that obtains healing degree from the time-series heartbeat data, and the images captured in descending order of healing degree detected by the healing degree detection unit And a sorting processing unit.

The healing degree detection unit is a Fourier transform processing unit that performs Fourier transform on time-series heartbeat data, a high frequency component / low frequency component detection processing unit that separates high frequency components and low frequency components of the signal subjected to Fourier transform processing, And an autonomic nerve analysis engine processing unit that detects a reference heart motion. In addition, the healing degree detection unit is a healing rate according to a heart rate detection unit for obtaining a heart rate from time-series heartbeat data, and a negative value obtained by subtracting the heart rate in normal times previously obtained from the heart rate. And a healing degree computing unit. Each component having such technical features may be configured as a client server or may be configured as a single device (stand-alone). Furthermore, each component may be configured as hardware, or may be configured as a computer and a computer program.

  Further, in the embodiment, the image itself such as a landscape visually recognized by a person is analyzed using fractal theory (because the digital signal processing technique is used for analysis, the processing target of the computer is image data), the analysis result and the person The movement of the mind of the person may be statistically related to estimate the movement of the person's mind. That is, the result of fractal analysis of the image may be used as a reference for organizing the image. When fractal analysis is performed, a fractal analysis processing unit that calculates the fractal dimension of the captured image, and a sorting processing unit that arranges the captured images in descending order of the impression that the image is given and healed are provided. You may Here, the relationship between the impression of being healed and the fractal dimension corresponding to the image is specified in advance based on statistical analysis.

  Furthermore, in the embodiment, time-series heartbeat data corresponding to a heartbeat is acquired when acquiring image data. Here, the time of acquisition of the image data and the time of acquisition of the time-series heartbeat data may be substantially simultaneous, and a time-series within a time during which an impression obtained when acquiring the image data is maintained Heart rate data may be acquired. In addition, the timing of acquiring the time-series heartbeat data and detecting the healing degree may determine the timing of detecting the healing degree according to the intention of the device operator, so that the healing degree is automatically detected in real time You may In the embodiment, the computer performs interrupt processing every predetermined time to detect the healing degree in substantially real time every predetermined time.

  Furthermore, in the embodiment, the degree of excitement may be used instead of the degree of healing as the arrangement standard of images.

First Embodiment
The image data organization system of the first embodiment includes an image data organization terminal device and an image data organization server device. The image data organizing system automatically organizes the imaged image data a posteriori, or notifies the operator of the apparatus whether or not the image to be imaged satisfies a predetermined organization standard. Here, the healing degree is used as a predetermined organizing standard for organizing. To be healed refers to a mental state in which the feeling is laid back, relaxed, and released from pain, and the degree of healty means how much healed.

  The image data organizing terminal device is an imaging unit that captures an image of a still image or a moving image, and a time series that captures the brightness of the skin that changes according to the blood flow that is a signal according to the heart rate to obtain time-series brightness data. A luminance data acquisition unit, a time-series heartbeat data output processing unit that outputs time-series luminance data according to the luminance of the skin to the image data organization server device, and a healing degree output from the image data organization server device And an image display unit configured to display captured images arranged in descending order. Here, the time-series luminance data is an aspect of the time-series heartbeat data, and the time-series luminance data acquisition unit is an aspect of the time-series heartbeat data acquisition unit.

  The image data organizing server device includes an RR interval time series data detection processing unit that detects RR interval time series data from time series luminance data, a Fourier transform processing unit that Fourier transforms the RR interval time series data, and Fourier transform processing. A high-frequency component / low-frequency component detection processing unit that separates high-frequency components and low-frequency components of the signal, a healing-degree detection unit that determines the healing degree based on at least the magnitude of the high-frequency component, and a high healing degree And a sorting processing unit that arranges the captured images in order.

  Here, the filter characteristic for separating the low frequency component (the low frequency component of the signal obtained by Fourier transforming the RR interval time series data) and the high frequency component (the low frequency component of the signal obtained by Fourier transforming the RR interval time series data) is It is a standardized predetermined characteristic. For example, the high frequency component is obtained by passing through a filter of a first predetermined frequency band (for example, 0.15 to 0.4 Hz) after Fourier transformation. That is, the high frequency component is an integral value of the power spectrum of the band of the first predetermined frequency. The low frequency component is obtained by passing through a filter of a second predetermined frequency band (for example, 0.04 to 0.15 Hz) whose frequency is lower than the first predetermined frequency. That is, the low frequency component is an integral value of the power spectrum of the band of the second predetermined frequency.

  It is known that the high frequency component is related to the parasympathetic activity and the low frequency component is related to the sympathetic activity. Therefore, the larger the value of the high frequency component or the value obtained by dividing the high frequency component by the low frequency component, indicates that the activity of the parasympathetic nerve is more active. Furthermore, the larger the value of the logarithm of the high frequency component or the value of the logarithm of the low frequency component as the logarithm of the high frequency component, the greater the activity of the parasympathetic nerve.

  Since it is known that the activity of the parasympathetic nerve is activated when healed, if the activity of the parasympathetic nerve is examined, it is possible to know how much healed (the degree of healing). That is, at least the value of high frequency component (value of high frequency component, value of logarithm of high frequency component, value obtained by dividing high frequency component by low frequency component, or value obtained by dividing logarithm of high frequency component by logarithm of low frequency component) is large About, healed.

Specifically, the degree of healing is determined as follows.

Degree of healing = (high frequency component) ........... (1)

Degree of healing = (logarithm of high frequency component) ........... (2)

Degree of healing = (high frequency component / low frequency component) ..... (3)

Degree of healing = {(log of high frequency component / log of low frequency component)} (4)

  However, as described above, the high frequency component and the high frequency component divided by the low frequency component, the logarithm of the high frequency component, and the logarithm of the high frequency component divided by the logarithm of the low frequency component May not be healed. On the other hand, even if their values are small, another person may be healed. That is, the relationship between the value of the high frequency component, the value obtained by dividing the high frequency component by the low frequency component, the value of the logarithm of the high frequency component, and the value obtained by dividing the logarithm of the high frequency component by the logarithm of the low frequency component There are individual differences.

  Therefore, the difference in magnitude between the high frequency component at the measurement time and the reference high frequency component, or the difference in magnitude between the high frequency component / low frequency component at the measurement time and (reference high frequency component / reference low frequency component) Based on the degree of healing (the degree of healing). Here, the reference high frequency component is the value of the high frequency component in the healing state, which is a reference measured in advance for the same person. Further, the reference low frequency component is a value of the low frequency component of the healing state which is a reference measured in advance for the same person. In this way, the healing degree, which is the healing degree, can be standardized without individual differences. Here, the standard level of healing is the degree of healing in a normal daily life, and is neither an excessively healed state nor an excessively excited state, and the ordinary mind in daily life It is the degree of healing in the state of keeping.

Specifically, the healing degree of the standard is determined as follows.

Degree of healing = (high frequency component) / (reference high frequency component) ... (5)

Degree of healing = (log of high frequency component) / (log of reference high frequency component) ··········· (6)

Degree of healing = (high frequency component / low frequency component) / (reference high frequency component / reference low frequency component)
= (High frequency component / reference high frequency component) / (low frequency component / reference low frequency component) · · · (7)

Degree of healing = {(log of high frequency component / log of low frequency component) / (log of reference high frequency component / log of reference low frequency component)}
= {(Log of high frequency component / log of reference high frequency component) / (log of low frequency component / log of reference low frequency component)}. ... (8)

  At least any of the above equations (5), (6), (7) and (8) is based on the difference in magnitude between the high frequency component at the time of measurement and the reference high frequency component measured in advance. Seeking the degree of healing. In the equation (5), the degree of healing is determined based on the difference in magnitude between the high frequency component at the time of measurement and the reference high frequency component measured in advance. In equation (6), the healing degree is determined based only on the difference between the logarithm of the high frequency component at the time of measurement and the logarithm of the reference high frequency component measured in advance. In the equation (7), although the healing degree is determined based on the difference between the high frequency component at the time of measurement and the magnitude of the reference high frequency component measured in advance, the low frequency component at the time of measurement and the standard low frequency component measured in advance The difference in size is also used to determine the degree of healing. In the equation (8), although the healing degree is determined based on the difference between the logarithm of the high frequency component at the time of measurement and the logarithm of the reference high frequency component measured in advance, the logarithm of the low frequency component at the time of measurement The degree of healing is determined using the difference between the measured magnitude of the reference low frequency component and the logarithm. Based on which of the above formulas (5), (6), (7), and (8), the degree of healing can be determined arbitrarily.

  The image data organizing system of the embodiment may include a low pass filtering processing unit for removing noise included in the RR interval time series data (RR interval time series data).

  The image data reduction system of the embodiment may include a Savitzky-Golay filter (Savitky Golay filter) for removing noise contained in the time-series luminance data and performing peak detection.

  The image data reduction system of the embodiment may include a functionalization processing unit for performing continuous interpolation of time-series luminance data.

  The image data organizing system according to the embodiment includes an advance logic processing unit that excludes time-series luminance data of a predetermined number of times or more predetermined in advance as a processing target when the values of the time-series luminance data are continuously the same. It is also good.

  The image data reduction system of the embodiment may display the waveform of the time-series luminance data on the display when performing the detection of the time-series luminance data.

  The image data organizing system according to the embodiment displays an image of a finger capturing an image of the skin brightness which changes according to the blood flow and displays a waveform of the time-series brightness data on the display before detecting the time-series brightness data. You may

  The image data reduction system according to the embodiment obtains an instantaneous heart rate inversely proportional to the value of RR interval time-series data (RR interval time-series data), and calculates the difference between the current instantaneous heart rate and the immediately preceding instantaneous heart rate. May be provided, the abnormal value removal processing unit of the RR interval time-series data may be provided to remove the current RR interval time-series data from the processing target.

  The first embodiment will be described below with reference to the drawings.

  FIG. 1 shows an image data organizing system 10 according to an embodiment.

  The image data organizing system 10 functions as a smart phone 11 functioning as an image data organizing terminal device, a server 12 functioning as an image data organizing server device, and a terminal device-server device connecting means for connecting the smartphone 11 and the server 12 And the Internet 13.

The smartphone 11 has a camera function that functions as part of a pulse wave detection unit and various external data detection units. The smartphone 11 “(1) upload measurement data (brightness data) and captured photos (image data) to the server 12” and “(2) various external data (for example, weather, air temperature, barometric pressure, position data, acceleration Upload data) to the server 12.
Here, the pulse wave detection unit is an embodiment of the imaging unit and is also an embodiment of the time-series heartbeat data acquisition unit. The smartphone 11 acquires time-series heartbeat data from the skin on the blood vessel imaged by the camera function of the embodiment of the imaging unit and the camera.
Here, when uploading various external data to the server 12, the measurement data and the image data are linked with the various external data and uploaded. The contents of these processes will be described later.

The server 12 “configures (3A) (autonomic nervous analysis engine) and analyzes the living body” by the server CPU and a program, and “(3B) (image analysis engine) configures the fractal analysis Perform processing and "(4) Return the analysis result".
The server 12 analyzes the degree of healing of the mind of the smart phone operator (hereinafter referred to as the operator) who is an image pickup person by the biological analysis processing of (3A). The server 12 performs objective analysis processing of the image itself captured by the fractal analysis of (3B). Since (3A) and (3B) are analysis processes from different points of view, both or either of them can be adopted. The server 12 returns the analysis result of the process of (3A) or / and (3B) to the smartphone 11. The content of the analysis result of the process of (3A) is the ranking of the image based on the degree of healing by sorting the set of the degree of healing and the image data stored in the database of the server 12, and (3B) The content of the analysis result of the processing is the ranking of the image based on the fractal dimension by sorting the set of the fractal analysis result (fractal dimension) and the image data stored in the database of the server 12.

  The smartphone 11 functions not only as a mobile phone but also as an operating system (OS) capable of connecting to the Internet and enabling operation of various application software, a liquid crystal display (LCD) and a touch panel as main user interface devices. It is a functional mobile phone. The smartphone 11 functions as an embodiment of the image data organization terminal device. The multifunctionality of the smartphone 11 is ensured by the operating system.

  The smartphone 11 has an image display function exhibited by an image display unit including a liquid crystal screen, a touch panel function exhibited by combining a liquid crystal display screen and a touch panel, an audio transmission function, an audio input function, a camera function, the Internet, in addition to a telephone function. It is common to provide a connection function and a GPS (Global Positioning System) function for detecting position data. The smartphone 11 according to the embodiment further detects an acceleration detection function that detects acceleration data, an action recording function that records the action of the operator (a memo function that the operator inputs and records his own action), and barometric pressure data It has a barometric pressure detection function and a meteorological data detection function that detects meteorological data including weather.

  Then, similar to a general personal computer, a bus line connects peripheral devices for exerting the various functions described above, including a microprocessor (MPU), which is hardware that operates an operating system, and other peripheral devices for performing the various functions described above. There is. The operating system is implemented in the MPU, which controls the peripherals to perform the desired functions described above. As another embodiment of the image data organizing terminal device other than the smartphone 11, even a tablet (multifunctional portable terminal) having the various functions described above and a dedicated image data organizing terminal device specialized for use in the present embodiment Good.

  As shown in FIG. 1, the smartphone 11 is wirelessly connected to the Internet 13 by the Internet connection function. When an image data organizing terminal device specialized for use in the present embodiment is used, the terminal device / server device connecting means may be a dedicated line, or may be connected via the Internet 13 by wire.

  The server 12 is a computer capable of processing high-speed and large-capacity data, and has a high-speed and high-performance CPU (central processing unit: central processing unit), large-capacity ram, large-capacity ROM, and large-capacity rewritable non-volatile memory A memory, etc. can be provided to perform complex processing at high speed. Therefore, even when a large number of clients (smartphones 11 etc.) access the server (server 12) via the Internet 13, the request from the client is received at high speed, and the information is processed instantaneously, and the processing result Are sent back to the client via the high-speed Internet 13.

  The smartphone 11 transmits time-series luminance data, which is an embodiment of time-series heartbeat data to be described later, to the server 12 and transmits analysis result data from the server 12 to the smartphone 11 after arithmetic processing in the server 12 Do. Thus, in the embodiment, the smartphone 11 and the server 12 share processing. The reason is that there is a limit to the allowable size of the application that can be installed on the smartphone 11 and the processing speed, so when processing all the processing inside the smartphone 11, the time required to obtain the analysis result data becomes long It is because it has Adopting a client server system solves such a problem. In addition, a large amount of data obtained from clients at the server can be accumulated and used as an asset of the client or a common asset of many clients using the server 12.

  Thus, the smartphone 11 and the server 12 are connected via the Internet 13 and operate in cooperation with each other. That is, when viewed from the person who operates the smartphone 11, the server 12 connected via the Internet 13 functions as a cloud computer, and the server 12 is not clearly recognized as being a "cloud", as if it were as if it were The process in the server 12 is recognized as if it is the process performed by the smartphone 11.

(Screen displayed on image data organizing terminal)
2 to 4 show images displayed on the liquid crystal screen of the smartphone 11 functioning as an image data organizing terminal device.

  FIG. 2 shows screens 1 to 4 displayed on the image data organizing terminal. FIG. 3 shows screens 5 to 8 displayed on the image data organizing terminal. FIG. 4 shows a screen 9 and a screen 10 displayed on the image data organization terminal device.

  The screen 1 shown in FIG. 2 causes the MPU to first display the liquid crystal screen of the smartphone 11 after the operator, which is one of the applications operating on the OS of the smartphone, activates the “image data organizing terminal device program”. It is a screen. The MPU causes the screen to display, for example, “an application for ranking the image on the travel destination”.

The screen 2 shown in FIG. 2 is a screen for displaying the terms of use of the “image data organization terminal device program” that the MPU causes the liquid crystal screen of the smartphone 11 to display.
When the operator touches "Agree" in the lower part of the screen 2, the MPU recognizes that the operator has agreed to use the "image data organization terminal device program" by the touch panel function, and the screen shown in FIG. 3 3 is displayed on the liquid crystal screen of the smartphone 11.

A screen 3 shown in FIG. 2 is a screen displayed by the MPU on the liquid crystal screen of the smartphone 11 for inputting attribute information.
The screen 3 is a screen for registering information of a user who uses the “image data organization terminal device program”, ie, an operator, as user information, and at the time of registration, as a keyboard (not shown) at an appropriate place on the liquid crystal screen. The function touch key is displayed, and the touch key causes the MPU to recognize the registration information.
As attribute information, "nickname" (for example, tarou), "sex" (for example, male), "birth date" (for example, October 19, 1980), and "good" (for example, Tokyo) The MPU acquires these pieces of information.

A screen 4 shown in FIG. 2 is a screen displayed by the MPU when the operator operates the camera function of the smartphone 11. Once the attribute information is input using the screen 2 and the screen 3, the screen 4 may be displayed immediately after the display of the screen 1. As shown on the screen 4, the MPU causes the liquid crystal screen of the smartphone 11 to display “Let's take an image of the healing scenery!”, And causes the liquid crystal screen of the smartphone 11 to display the subject viewed by the operator. When “Measuring” is touched with a finger, the MPU displays the screen 5 shown in FIG. 3 on the liquid crystal screen of the smartphone 11. The MPU may not display the screen 5 unless the operator touches "Measuring" with a finger, and if the operator does not touch "Measuring" with a finger for a certain period of time, the MPU recovers the healing degree. It may be made to shift to the usual imaging mode which does not measure.

In addition, when the operator touches “Measuring”, the MPU operates the camera function to store the imaged image data in the image storage device. In this way, the image data and the healing degree at that time can be linked.
Here, when the operator touches “Measuring” (see screen 4 ), an image of a landscape etc. is taken, and then the fingertip is brought close to the lens of the camera for detection of healing degree ( screen 5 , See screen 6), so there is a gap between the imaging time and the measurement time of healing degree . Therefore, even during measurement of the degree of healing, continue to look at the captured scenery etc. or make an impression of the moment of imaging (see screen 4) until the measurement of the degree of healing ends (see screen 7) ) It is desirable to keep in mind that the relationship between the taken image and the degree of healing does not collapse.

The screen 5 shown in FIG. 3 is a display screen for notifying the operator that the measurement of the degree of healing is to be started.
The MPU displays “Measurement of degree of healing” on the upper side of the screen 5. The MPU displays in the middle of the screen 5 as “Caution on the current degree of relaxation. Place the tip of the forefinger on the camera and press the measurement start button to start measurement” as a caution of measurement.
The MPU converts time-series luminance data obtained by capturing the tip of the forefinger with a camera in the lower part of the screen 5 into analog values and displays the same on the vertical axis, and displays a pulse wave displaying time on the horizontal axis.
The MPU displays “in standby” in the waveform diagram on the lower side of the screen 5. "On standby" means that the healing degree has not been measured yet, and it is in the preparation stage of the measurement.
The operator aligns the index finger with the lens of the camera so that the pulse wave can be stably captured while the screen 5 is displayed.
When the operator aligns the position of the forefinger with the lens of the camera, the operator can align the forefinger with the image displayed on the right side of the screen middle of the screen 5.
Then, when the operator determines the position of the forefinger with respect to the lens of the camera, the operator touches “measurement start” at the bottom of the screen 5 with the positional relationship fixed. Alternatively, instead of touching “measurement start”, the smartphone 11 may automatically start the measurement.

When the MPU confirms that the fingertip is being imaged or / and recognizes that the luminance signal waveform is a characteristic pulse wave, it automatically images the luminance of the skin, which changes according to the blood flow. The process of acquiring time-series luminance data may be started.
More specifically, the MPU determines that the skin color is included from the ratio of each component separated into R (red), G (green) and B (blue) of the captured image, and the image of the skin-colored portion After detecting that it is a predetermined ratio or more and / or detecting that the time-series luminance data is a repetitive waveform having a substantially constant period and the envelope is substantially constant. Later, acquisition of time-series luminance data is started.

Screen 6 shown in FIG. 3 is a screen displayed while measuring the healing degree.
The MPU displays “in measurement” on the upper side of the screen 6.
The MPU converts the time-series luminance data under measurement into an analog value and displays it on the vertical axis in the middle of the screen 6 and displays time on the horizontal axis.
The MPU causes the lower part of the screen 6 to display "cancel". The "cancel" is for touching the "cancel" when the operator determines that the time-series luminance data being measured is not correct.
When the MPU detects that “cancel” is touched, it shifts the process, displays the screen 5 again, and returns the process to the front. When “cancel” in the lower part of the screen 6 of the screen 6 is long-pressed, the healing degree measurement mode may be exited.
If the MPU does not touch “cancel” on the lower side of the screen 6 while displaying “I'm measuring”, the MPU displays the screen 7 after the measurement is completed.

Screen 7 shown in FIG. 3 is a screen for viewing the measurement result.
The MPU displays “Let's see the result!” In the upper part of the screen 7 and “Next” in the lower part of the screen 7. When the operator touches “next”, the MPU displays the screen 8.
If the operator does not touch “Next”, the screen 7 is maintained, or if the operator does not touch “Next” for a certain period of time or more, the MPU transfers the processing from the healing degree measurement mode to the normal mode. May be

Screen 8 shown in FIG. 3 is a screen for displaying the degree of healing.
The MPU displays the captured image in the middle of the screen 8, displays the healing degree when capturing the image in the upper screen of the screen 8, and captures the image in the lower screen of the screen 8. Make a graph showing the degree of healing.

  When the operator presses the "Share" button, the MPU measures the measurement result (image) to the FaceBook (Facebook: registered trademark), Twitter (Tweeter: registered trademark), LINE (line: registered trademark), etc. via the Internet 13. Post the image and the healed degree).

  When the operator clicks the "link to external data" button, the MPU displays the measurement result (the degree of healing linked to the captured image) and the weather, air temperature, barometric pressure, position data from GPS, acceleration data, etc. The data is linked (linked) and stored in the server 12 via the Internet 13. The weather may be input by the operator, or may be automatically acquired via the Internet 13 based on position data detected by the MPU via the GPS. Even if the temperature is detected by the MPU controlling the temperature detection function provided in the smartphone or automatically acquired from the weather information providing site via the Internet 13 based on the position data detected by the MPU via the GPS Good. Even if the atmospheric pressure is detected by the MPU controlling the atmospheric pressure detection function provided in the smartphone or automatically acquired from the weather information providing site via the Internet 13 based on the position data detected by the MPU via the GPS Good. The position data is detected by the MPU controlling the air pressure detection function provided in the smartphone. The acceleration data is detected by the MPU controlling an acceleration sensor function provided in the smartphone.

  Information such as weather, air temperature, barometric pressure, position data from GPS, acceleration data, etc. is one type of so-called big data. At present, the relationship between the information and healing degree is not clear. These seemingly, information that seems not to be related to the degree of healing, or, although there is an expectation that there is a relationship, information that is not clear as to what is currently the relationship will be collected by the server in the future , To help elucidate the complex relationship between them. For example, the relationship between healing and barometric pressure, the relationship between healing and active movement of the body (detectable with an acceleration sensor), the relationship between healing and altitude, the degree of healing, so-called , To help elucidate the relationship with the power spot.

Operator, when you click the "Share" button and the "attach external data and the string" button, MPU displays a screen 9 of FIG. 4.
If the operator clicks only one of the “share” button or the “link with external data” button within a predetermined time, the MPU displays the screen 9 of FIG. 4 after a predetermined time.
If the operator does not click either the "share" button or the "associate with external data" button within the predetermined time, the MPU displays the screen 9 of FIG. 4 after the predetermined time.

Screen 9 shown in FIG. 4 is a screen for displaying the measurement result.
The screen 9 is a ranked image displayed in descending order of the degree of healing during the same travel as “healing image ranking”.
As the set to be ranked, not only the image of the same travel but also an image captured in the past, an image captured on the same day, and the like can be appropriately selected.

The screen 10 shown in FIG. 4 is another screen for displaying the measurement result.
The screen 10 is the left screen that is ranked based on the degree of healing as “healing image ranking”, and the right screen is ranked based on the fractal analysis.
The range of sets used for ranking can be appropriately selected, such as an image on the same trip, an image captured in the past, an image captured on the same day, and the like. The fractal analysis will be described later.

(Outline of processing in image data organizing terminal device and image data organizing server device)
An outline of processing in the healing degree detection terminal device and processing in the image data organizing server device of the image data organizing system will be described with reference to FIGS. 5 and 6 .

5 and 6 show a series of continuous processes. The upper part of each drawing of FIG. 5 and FIG. 6 shows the processing in the image data rearranging terminal device (smart phone 11), and the lower part of each drawing in FIG. 5 and FIG. 6 shows the processing in the image data rearranging server device (server 12) Show. If there is a corresponding screen in FIG. 2 to FIG. 4 for the process in the upper part of each of FIGS. 5 and 6, the corresponding screen number is attached. The process in the smartphone is the process of steps ST11 to ST18 and the process of steps ST27, ST28, and ST32. The process in the server is the process of steps ST19 to ST26 and the process of steps ST29 to ST31.

The processing order of each process in the series of processes shown in FIGS. 5 and 6 is one embodiment, and this order can be modified and used. In the following, FIG. 5, the description below in the order shown by arrows in FIG. 6.

(Processing in the image data organization terminal device)
The MPU of the image data organization terminal device (smart phone 11) displays the screen 1 when the operator selects the application of “image data organization terminal device program” in step ST11 (start), and displays the screen 1, and a predetermined time after the process start of step ST11 The process moves to step ST12 to display screen 2.
When the operator touches “agree” on the screen 2 in step ST12 (display of the terms of service), the MPU shifts the process to ST13 to display the screen 3.

In step ST13 (image capturing mode), when the operator inputs information of "nickname", "sex", "birth date", and "live" on screen 3 and touches "register" in step ST13 (step ST14) The process is shifted to the image pickup mode to display the screen 4.
In step ST14 (image capturing mode), when the operator touches “Measuring” on the screen 4 in step ST14, the MPU controls the camera function to capture an image currently displayed on the liquid crystal screen and store the storage device. The image data is controlled to be stored in the storage device. Then, the MPU transfers the process to step ST15 to display the screen 5.

When the operator touches “measurement start” on the screen 5 in step ST15 (processing for starting pulse wave measurement mode), the MPU shifts the processing to step ST16.
The MPU starts measurement and displays the screen 6 in step ST16 (measurement start notification process).
When the operator touches “cancel” on the screen 6 for a short time in step ST17 (image processing of an image of a finger close to the camera), the MPU shifts the process to step ST15 to display the screen 5, and the operator When "cancel" on the screen 6 is touched for a long time, the "image data organization terminal device program" is ended.
In step ST16 (measurement start notification processing), if the operator does not touch "cancel" on the screen 6 for a predetermined time, the process proceeds to step ST17 to display the screen 5, and the operator "cancels" the screen 6 Touch for a long time to end the "image data organization terminal program".

In step ST18 (measurement end notification processing / display processing in operation / data upload processing to server), the MPU 7 displays the screen 7 and performs measurement end notification processing / display processing in operation. Furthermore, upload data to the server.
The data to be uploaded from the smartphone 11 to the server 12 is image data (one group of data constituting one screen in the case of still images, and one group of data constituting the moving images in the case of moving images) It is data (a group of time-series luminance data) obtained by imaging the tip of the forefinger immediately after with the camera for a predetermined time.

(Processing in the image data organizing server device)
The CPU of the image data organizing server device (server 12) performs forward logic processing in step ST19 (forward logic).
The forward logic processing is a process for making it easier to detect the peak of the time-series luminance data signal, and when the values of the time-series luminance data are continuously the same, the same value more than a predetermined number of times predetermined Is a process of excluding continuous time-series luminance data from the object of processing.

The CPU performs functionalization processing of time-series luminance data in step ST20 (functionalization processing of luminance data).
The time-series luminance data is discrete time sampling data, and the sampling time is synchronized with the time in the smartphone 11. In the functionalization process, in order to synchronize with the time in the server 12, the time-series luminance data in the time when there is no time-series luminance data detected by the smartphone 11 is functionized and continuously interpolated to enable resampling . By enabling resampling, the Fourier transform processing in step ST25 can be performed with high accuracy.

The CPU performs noise removal and peak detection processing using a Savitzky-Golay filter in step ST21 (noise removal and peak detection processing using a Savitzky-Golay filter).
The Savitzky-Golay filter itself is a known filtering technique. The waveform of the time-series luminance data signal detected by the camera of the smartphone 11 is less accurate than the pulse waveform detected by the dedicated heart rate sensor, and in simple filtering, the pulse wave peak detected by the dedicated heart rate sensor It is difficult to reproduce. Here, when using a simple derivative when reproducing the pulse wave peak, the noise in the high region increases and the purpose can not be achieved. Therefore, the Savitzky-Golay filter is used as a filter that clearly shows peaks without increasing noise.

The CPU performs detection processing of RR interval time-series data in step ST22 (RR interval time-series data detection processing).
The detection process of RR interval time series data (RR interval time series data) is a process of converting a time series luminance data signal into RR interval time series data. The detection process of the RR interval time series data includes a process of detecting a pulse wave peak included in the time series luminance data signal clearly reproduced by the Savitzky-Golay filter in step ST21 by a differential operation, and And a process of detecting the time between two adjacent peaks.

The CPU performs low pass filtering processing in step ST23 (low pass filtering processing).
The low-pass filtering process converts RR interval time-series data to RR interval time-series data that would normally be obtained if the time-series luminance data accurately detected a pulse wave after detecting RR interval time-series data from time-series luminance data It is.

The CPU performs abnormal value removal processing of the RR interval time-series data in step ST24 (abnormal value removal processing of the RR interval time-series data).
The abnormal value removal process of the RR interval time-series data is a process of removing an abnormal value included in the RR interval time-series data. Here, an outlier is a value of RR interval time-series data that would not normally be detected. For detection of an abnormal value, an instantaneous heart rate (HR) inversely proportional to the value of RR interval time-series data obtained by dividing 60 by the value of RR interval time-series data is used. When an abnormal value is detected, it is assumed that noise mixed in the RR interval time-series data (that is, noise mixed in the time-series luminance data signal) can be eliminated in the processing after Fourier transform in step ST25. Do not use RR interval time-series data that indicate. With regard to what is considered as an abnormal value in the abnormal value removal process, the abnormal value may be an abnormal value when any one of the following first to third rules is satisfied, the first It may be an outlier if any two or more combinations of the rule and the third rule are satisfied. The first to third rules will be described in more detail below.

<First rule>
An error occurs if n is a positive integer and t (n) is not the first predetermined value as | 60 / t (n) -60 / t (n-1) | ≦ first predetermined value as RR interval time series data detected at the nth It is a value. That is, when the difference between the current instantaneous heart rate and the immediately preceding instantaneous heart rate is larger than the first predetermined value, the RR interval time-series data at that time is removed.

<Second rule>
When (meanHR.R (N)) is a moving average value, and N is a positive integer (for example, 5, 8), when | 60 / t-meanHR.N (N) | ≧ predetermined second value range Is an outlier. That is, when the difference between the current instantaneous heart rate and the moving average value of the instantaneous heart rates up to immediately before is larger than the second predetermined value, the RR interval time-series data at that time is removed.

<Third rule>
As a result of applying the second rule, when the abnormal value is continuously determined a predetermined number of times (for example, eight times), reset meanHR (8) and recalculate meanHR (8) again . That is, in the third rule, when applying the second rule, when the difference between the current instantaneous heart rate and the moving average value is larger than the second predetermined value, the moving average value is taken as a predetermined number of times. After resetting, a moving average value is newly obtained.

The CPU performs Fourier transform processing in step ST25 (Fourier transform processing).
The RR interval time series data after the outlier removal process obtained in step ST24 is subjected to the Fourier transform process using the known Hamming window and the known FFT (fast Fourier transform). Since functionalization processing of time-series luminance data is performed in step ST20, resampling can be performed at a timing different from the data sampling timing in the smartphone 11 in the fast Fourier transform. Further, since the RR interval time series data after the abnormal value removal process is subjected to the Fourier transform, it is possible to obtain the power spectrum after the Fourier transform which can be originally obtained when the noise is not mixed in the time series luminance data signal.

The CPU performs high frequency component HF / low frequency component LF detection processing in step ST26 (high frequency component (HF), low frequency component (LF) detection processing).
After Fourier transform, the high frequency component HF and the low frequency component LF are detected. The high frequency component HF is an integral value of a high frequency power spectrum of a predetermined frequency range obtained by Fourier transform. The low frequency component LF is an integral value of a low frequency power spectrum of a predetermined frequency range obtained by Fourier transform.

The high-frequency component HF and the low-frequency component LF values obtained in step ST26 relate to the healing degree of the person using the “image data organization terminal device program” at the measurement time point.
In step ST26, analysis result data for displaying the healing degree of the operator is generated based on the high frequency component HF value and the low frequency component LF value which are biological information of the operator.
When the analysis result data is created, the spectrum is brought close to a normal distribution, and if the high frequency component is extremely large or extremely small from the reference high frequency component, the high frequency component HF is used to reduce inter-individual variation. LnHF for logarithmic display and LnLF for low frequency component LF are used. Hereinafter, LnHF is referred to as a high frequency component LnHF, and LnLF is referred to as a low frequency component LnLF. Note that the reference high frequency component LnHF represents the reference high frequency component HF obtained as a result of the measurement in the reference normal heart logarithmically, and the reference low frequency component LF obtained as the result of the measurement in the reference normal heart is displayed logarithmically Is the reference low frequency component LnLF.

  The reference high frequency component HF obtained as a result of the measurement in the reference normal heart and the reference low frequency component LF obtained as the result of the measurement in the reference normal heart are measured in advance by the smartphone 11, and the storage device of the smartphone 11 (for example, , Non-volatile memory). Similarly, the reference high frequency component LnHF and the reference low frequency component LnLF are also stored in the storage device (for example, non-volatile memory) of the smartphone 11.

  The display uses polar coordinates as indicated by a half donut portion in the middle of the screen 8 of FIG. A straight line (a straight line extending radially from the center point) of an "arrow" extending radially from the center point of the half donut (the center point of the outer and inner arcs) and a straight line connecting the left end of the half donut and the left end of the half donut The degree of healing is expressed by the angle between (a straight line passing through the central point).

  When the CPU calculates the degree of healing using equation (5), the degree of healing = (high frequency component HF) / (reference high frequency component HF) is theoretically 0 (zero) to ∞ (infinity) There may be a range up to). When (high frequency component HF) / (reference high frequency component HF) = 0, a straight line passing through a central point radially extending from the central point (a straight line of “arrow” on screen 8), the central point and the left end of the half donut portion The angle with the straight line connecting is 0 °, and the healing degree is 0. On the other hand, when (high frequency component HF) / (reference high frequency component HF) = ∞, the angle between the straight line extending radially from the central point and the straight line passing through the central point and the left end of the half donut portion becomes 180 °. The degree is 100. When (high frequency component HF) / (reference high frequency component HF) = 1, the angle between the straight line extending radially from the central point, the straight line passing through the central point, and the left end of the half donut portion is 90 °. The degree is 50. Here, the numerical value of the healing degree is not limited to that described above, and the healing degree is set to −100 when (high frequency component HF) / (reference high frequency component HF) = 0, and (high frequency component HF) / ( The healing degree may be 0 when the reference high frequency component HF) = 1, and may be 100 when the (high frequency component HF) / (reference high frequency component HF) = ∞.

  When the CPU calculates the degree of healing using equation (6), the degree of healing = (high frequency component LnHF) / (reference high frequency component LnHF) theoretically has a range from −∞ to ∞ obtain. When (high-frequency component LnHF) / (reference high-frequency component LnHF) = − 直線, a straight line (straight line of “arrow” on screen 16) passing radially from the center point, the center point and the left end of the half donut portion The angle between the straight line and the straight line is 0 °, and the degree of healing is 0. On the other hand, when (high frequency component LnHF) / (reference high frequency component LnHF) =. Infin., The angle formed by the straight line radially extending from the central point, the straight line passing through the central point and the left end of the half donut portion becomes 180 °. The degree is 100. When (high frequency component LnHF) / (reference high frequency component LnHF) = 0, the angle between the straight line extending radially from the central point, the straight line passing through the central point, and the left end of the half donut portion becomes 90 °. The degree is 50. Here, the numerical value of the healing degree is not limited to the one described above, and the healing degree at (high-frequency component LnHF) / (reference high-frequency component LnHF) =-と し て is -100, (high-frequency component LnHF) / The healing degree may be 0 when (reference high frequency component LnHF) = 0, and may be 100 when (high frequency component LnHF) / (reference high frequency component LnHF) = ∞.

  When the CPU calculates the degree of healing using the equation (7), the degree of healing = (high frequency component HF / low frequency component LF) / (reference high frequency component HF / reference low frequency component LF) is theoretically Can range from 0 to ∞. When (high frequency component HF / low frequency component LF) / (reference high frequency component HF / reference low frequency component LF) = 0, a straight line passing through the central point radially extending from the central point (straight line of “arrow” of screen 16 The angle between the center point and the straight line connecting the center point and the left end of the half donut portion is 0 °, and the degree of healing is zero. On the other hand, when (high-frequency component HF / low-frequency component LF) / (reference high-frequency component HF / reference low-frequency component LF) = 直線, a straight line passing through the central point radially extending from the central point (“arrow” on screen 16 The angle between the straight line) and the straight line connecting the center point and the left end of the half donut portion is 180 °, and the degree of healing is 100. When (high frequency component HF / low frequency component LF) / (reference high frequency component HF / reference low frequency component LF) = 1, a straight line radially extending from the central point, a straight line passing through the central point, and a half donut portion The angle with the left end is 90 ° and the healing degree is 50. Here, the numerical values of the healing degree are not limited to those described above, and the healing degree when (high frequency component HF / low frequency component LF) / (reference high frequency component HF / reference low frequency component LF) = 0 Assuming that the healing degree is 0 when (high frequency component HF / low frequency component LF) / (reference high frequency component HF / reference low frequency component LF) = 1 as −100 (high frequency component HF / low frequency component LF) The healing degree may be set to 100 when / (reference high frequency component HF / reference low frequency component LF) = ∞.

  When the CPU calculates the degree of healing using the equation (8), the degree of healing = (high frequency component LnHF / low frequency component LnLF) / (reference high frequency component LnHF / reference low frequency component LnLF) theoretically Can range from -∞ to ∞. When (high-frequency component LnHF / low-frequency component LnLF) / (reference high-frequency component LnHF / reference low-frequency component LnLF) = − ∞, a straight line passing through the central point radially extending from the central point (“arrow” of screen 16 The angle between the straight line) and the straight line connecting the center point and the left end of the half donut portion is 0 °, and the degree of healing is zero. On the other hand, when (high-frequency component LnHF / low-frequency component LnLF) / (reference high-frequency component LnHF / reference low-frequency component LnLF) = 、, a straight line extending radially from the central point, a straight line passing through the central point, and a half donut portion The angle with the left end is 180 °, and the healing degree is 100. When (high frequency component LnHF / low frequency component LnLF) / (reference high frequency component LnHF / reference low frequency component LnLF) = 0, a straight line radially extending from the center point, a straight line passing through the center point, and a half donut portion The angle with the left end is 90 ° and the healing degree is 50. Here, the numerical value of the healing degree is not limited to that described above, but the healing degree when (high frequency component LnHF / low frequency component LnLF) / (reference high frequency component LnHF / reference low frequency component LnLF) = − ∞ Where the healing degree is 0 when (high frequency component LnHF / low frequency component LnLF) / (reference high frequency component LnHF / reference low frequency component LnLF) = 0, and (high frequency component LnHF / low frequency component LnLF The healing degree may be set to 100 when / (reference high frequency component LnHF / reference low frequency component LnLF) = ∞.

  The angle θ between the straight line passing through the center point extending radially from the center point of the arc of the half donut portion (the straight line of “arrow” on the screen 16) and the straight line connecting the center point and the left end of the half donut portion is When the equation is used, the CPU obtains it by the following equation (9), and when it uses the equation (7), it obtains it by the following equation (10). Here, K4 and K5 are coefficients, and increasing the values of K4 and K5 emphasizes the degree of healing which is a detected value.

θ (°) = 2 × tan ⁻¹ {K 4 × (high frequency component HF) / (reference high frequency component HF)} ····· (9)

θ (°) = 2 × tan ⁻¹ {K 5 × (high frequency component HF / low frequency component LF) / (reference high frequency component HF / reference low frequency component LF)} (10)

  Further, the angle θ between the straight line passing through the central point radially extending from the center point of the arc of the half donut portion (the straight line of “arrow” on the screen 16) and the straight line connecting the central point and the left end of the half donut portion is When the equation (6) is used, the CPU obtains it by the following equation (11), and when the equation (8) is used, it obtains it by the following equation (12). Here, K6 and K7 are coefficients, and increasing the values of K6 and K7 emphasizes the degree of healing which is a detected value.

θ (°) = [90 + tan ⁻¹ {K 6 × (high frequency component LnHF) / (reference high frequency component LnHF)}]... (11)

θ (°) = [90 + tan ⁻¹ {K 7 × (high frequency component LnHF / low frequency component LnLF) / (reference high frequency component LnHF / reference low frequency component LnLF)}] (12)

  When converting the degree of healing into the range of 0 to 100, the CPU obtains the following equation (13) when using equation (5), and when using equation (7), the following equation (14) Calculated by the equation. Here, K8 and K9 are coefficients, and increasing the values of K8 and K9 emphasizes the degree of healing which is a detected value.

Degree of healing = (100/90) × tan ^-1 {K 8 × (high frequency component HF) / (reference high frequency component HF)} (13)

Degree of healing = (100/90) × tan ⁻¹ {K 9 × (high frequency component HF / low frequency component LF) / (reference high frequency component HF / reference low frequency component LF)} (14)

  When converting the degree of healing into the range of -100 to 100, the CPU obtains the following equation (15) when using equation (5), and when using equation (7), 16) Obtain the equation. Here, K10 and K11 are coefficients, and increasing the values of K10 and K11 emphasizes the healing degree which is a detection value.

Degree of healing = −100 + (200/90) × tan ⁻¹ {K10 × (high frequency component HF) / (reference high frequency component HF)} (15)

Degree of healing = −100 + (200/90) × tan ⁻¹ {K11 × (high frequency component HF / low frequency component LF) / (reference high frequency component HF / reference low frequency component LF)} (16)

  When converting the degree of healing into the range of 0 to 100, the CPU obtains the following equation (17) when using the equation (6), and when using the equation (8), the following (18) Calculated by the equation. Here, K12 and K13 are coefficients, and increasing the values of K12 and K13 emphasizes the healing degree which is a detection value.

Degree of healing = (100/180) × [90 + tan ⁻¹ {K12 × (high frequency component LnHF) / (reference high frequency component LnHF)}] ...... (17 )

Degree of healing = (100/180) × [90 + tan ⁻¹ {K 13 × (high frequency component LnHF / low frequency component LnLF) / (reference high frequency component LnHF / reference low frequency component LnLF)} (14)

  When converting the degree of healing into the range of -100 to 100, the CPU obtains the following equation (19) when using equation (6), and when using equation (8), 20) Obtain the equation. Here, K14 and K15 are coefficients, and increasing the values of K14 and K15 emphasizes the healing degree which is a detected value.

Degree of healing = (100/90) × tan ⁻¹ {K14 × (high frequency component LnHF) / (reference high frequency component LnHF)} (19)

Degree of healing = (100/90) × tan ⁻¹ {K15 × (high frequency component LnHF / low frequency component LnLF) / (reference high frequency component LnHF / reference low frequency component LnLF)} (20)

(Processing in the image data organization terminal device)
In step ST27 (analytical result display processing of healing degree), the MPU displays “healing degree 87/100” on the upper side of the screen 8 as shown in FIG. Display as a graph. The lower graph of the screen 8 classifies the size of the healing degree into three levels (high, middle, low), and it is close to the time when the middle image of the screen 8 is captured (for example, the middle of the screen 8) Immediately after the image is taken, it is displayed where the healing degree is located at three levels. In the example shown in FIG. 8, since the healing degree is 87/100, it is classified as high. It may be displayed as "It seems that I was able to relax considerably."

In step ST28 (the function of associating with external data), when the operator touches “attach with external data” of screen 8, the MPU displays the screen displayed on screen 8 and various external data. Connect
Specifically, the start address of the image data corresponding to the screen displayed on the screen 8 in the storage device (memory) in which the image data is stored (the image data specified by the address of the first group is stored Storage device (memory) that stores external data (weather, air temperature, barometric pressure, position data from GPS, data such as acceleration data, etc.) acquired at the time of touching "head address" and "link with external data" Relate to the address of
Then, in step ST28, the MPU “the analysis result of the degree of healing of the image” and “external data”, and “the association information between the address at which the image is stored and the address at which the external data is stored”. Upload to the image data reduction server device.

  Note that in step ST28, the MPU records its own action at the time of imaging by the operator (for example, a record of its own action such as walking, movement by a vehicle, rest, etc., and further, a numerical value of the subjective healing degree self-judged Records can also be linked. In this case, in addition to the healing degree detected by the CPU of the server 12 based on the equations (1) to (8), the action of the operator at the time of shooting and the subjective surface at the time of shooting Since it can be used as a judgment standard when performing, it is possible to further improve the accuracy of image data organization. For example, in the index detected based on the expressions (1) to (8), “inspiration” may be mixed in addition to the “degree of healing”, and in this case, subjective healing that is self-judged If the degree of healing is high by referring to the degree record, the CPU of the server 12 determines that the degree of healing is detected without fail, and "healing degree" and "inspiring" It can be separated.

(Processing in the image data organizing server device)
The CPU links the image data organizing terminal according to the format of the image data organizing terminal according to the format of the image data organizing terminal in step ST29 (various data (weather, air pressure, air temperature, position, GPS) and measurement data, and stores the image data in the DB). Store in (database).
The CPU performs fractal analysis of the captured image in step ST30 (fractal analysis). The fractal analysis in step ST30 is a process completely independent of the process of associating the healing degree with the image described above. The process of measuring the degree of healing and associating it with the image is to objectively detect the subjective sensitivity of the person and organize the image data, but the fractal analysis is based on the configuration of the image itself. It is a thing to arrange.
The process of step ST30 may be used together with the process of associating the healing degree with the image to organize the image data. Alternatively, the image data may be organized using only one of the process of step ST30 and the process of associating the healing degree with the image.

In step ST31 (totaling up the database), the CPU organizes the image data based on the information stored in the database and a predetermined organizing standard.
For example, sorting of image data performed by the CPU is performed based on a sorting standard (degree of healing in the first embodiment) designated by the operator within a range of image data to be sorted designated by the operator.
In step ST31, the CPU downloads the result of organizing the image data to the image data organizing terminal device.
The result of organizing the image data is such that the order of high healing degree is given to the image data of one screen in the case of a still image or to the moving image of one title in the case of a moving image.

(Processing in the image data organization terminal device)
In step ST32 (display of ranking), the MPU displays the result of arranging the image data obtained in step ST31.
The MPU displays the rankings and thumbnails with high degree of healing as displayed on the screen 9 of FIG. A thumbnail is a reduced image of a still image or a moving image, and is used when displaying a list of many images. In a moving image, an image of a title portion of the moving image is usually used as a thumbnail. By scrolling the screen 9, an image corresponding to the third and subsequent ranks can be viewed.
In this manner, the thumbnail allows the operator to quickly find an image with a high degree of healing at a glance.
In the case where the CPU of the image data organizing server device adopts step ST30 (fractal analysis), the MPU displays the screen 10 of FIG. The thumbnails can be displayed. This makes it possible to compare the degree of healing detected from heart rate information dependent on the human mental state with the degree of healing detected from the image itself.
Although not shown, the screen 9 and the screen 10 may be replaced with a screen showing only the order and thumbnails based on fractal analysis.

(Details of each process of image data organizing server device)
Details of the processing of the autonomic nerve analysis engine having steps ST19 to ST26 described above will be described in order.

(Speed forward logic processing in step ST19)
FIG. 6 is a diagram schematically showing forward logic processing.

  The forward logic processing is logic that shrinks twice in a row if the numerical values of time-series luminance data are the same three or more times. That is, continuous time-series luminance data having the same value of a predetermined number of times (for example, three times) or more predetermined in advance are excluded from the processing targets. Here, the same value may have a width, and the variation of the value within a minute predetermined range may be included in the target of the forward logic processing as the same value. By this process, it is possible to more easily detect a peak in the process of detecting RR interval time-series data in step ST22. FIG. 6A shows time-series luminance data before the forward logic processing. In FIG. 6A, the same data is continuous four times. FIG. 6B shows time-series luminance data after the forward logic processing. In FIG. 6 (b), four consecutive times of the same data are processed so as to be two consecutive times.

(Functionalization processing of time-series luminance data in step ST20)
The functionalization of the time-series luminance data is performed by a B-spline function which passes through the sample points of all the time-series luminance data. By using the B-spline function, it is possible to smoothly interpolate the time-series luminance data at a time when the time-series luminance data detected by the smartphone 11 does not exist as a continuous function. Processing can be performed. Further, it is possible to obtain time-series luminance data for each fixed sample period (for example, 10 msec (milliseconds)) for performing FFT (Fast Fourier Transform) as new time-series data. The B-spline function itself is a known function. Although linear interpolation is also known as a method of data interpolation, the interpolation accuracy of data is improved more than linear interpolation by using a B-spline function.

(Denoisement processing and peak detection processing by Savitzky-Golay filter in step ST21)
The Savitzky-Golay filter is also referred to as a digital smoothing polynomial filter or a least squares smoothing filter. A smoothing filter designed in the time domain and a differential filter. In the Savitzky-Golay filter of the embodiment, for example, the window performs a second derivative by approximating with a 0.9 second, third-order polynomial.

(Detection process of RR interval time series data of step ST22)
After peak detection of the time-series luminance data signal is performed, detection of RR interval time-series data, which is time between adjacent peaks, is performed. The logic for peak detection is a simple sign reversal as is known in the art. The detection of the time between adjacent peaks is performed by a counter operating in synchronization with the clock signal. Here, peak detection can be performed with simple code inversion logic, and detection of RR interval time-series data is possible because of the effect of the Savitzky-Golay filter in step ST21.

(An example of time-series data subjected to the processing from step ST19 to step ST22)
FIG. 7 is a diagram showing time-series luminance data after the processing of each step in the server 12 is completed.

  FIG. 7A shows time-series luminance data uploaded in step ST18. FIG. 7B is a diagram showing time-series luminance data after the functionalization process of the time-series luminance data in step ST20 is completed. FIG. 7C is a diagram showing time-series luminance data after noise removal processing by the Savitzky-Golay filter in step ST21.

(Low-pass filtering process of step ST23)
Ambient light entering the camera is a source of noise as stray light. The camera of the smartphone 11 is designed to exhibit the function of a camera for photographing a normal landscape and a person. Therefore, the level of stray light that becomes noise is relatively large with respect to a slight change in the luminance of the skin captured by the camera, and noise is present in RR interval time-series data (RRI) detected from time-series luminance data in step ST22. Because it is included, the RR interval time series data can not be separated with high accuracy unless noise is properly removed from the RR interval time series data obtained in step ST22.

  Therefore, filter characteristics optimum for removing noise are obtained in advance by experiments, and a low pass filter having such characteristics is used in the process of step ST23. Hereinafter, how to obtain the optimum filter characteristic will be described.

  Using the waveform of RR interval (RRI) time series data obtained from a dedicated, commercially available heart rate sensor as a reference waveform, the characteristics of a low-pass filter that approximates the waveform of RR interval time series data obtained after processing in step ST20 to this reference waveform The Noise is removed from the RR interval time series data detected from the luminance signal obtained from the camera of the smartphone 11 by the low-pass filtering process of step ST23, and the RR interval closer to the RR interval time series data that should be originally detected Time series data were obtained.

  FIG. 8 and FIG. 9 are explanatory diagrams of how to determine the constant of the low pass filter.

  FIG. 8 is a diagram comparing RR interval time-series data obtained from a commercially available dedicated heart rate sensor (MyBeat (registered trademark)) with RR interval time-series data not subjected to the low-pass filtering process of step ST23.

  FIG. 8A shows RR interval time series data obtained from a commercially available heart rate sensor. FIG. 8B is RR interval time-series data obtained from the smartphone 11 before the low-pass filtering process of step ST23. FIGS. 8A and 8B are data obtained simultaneously from the same person.

  The data of the commercially available heart rate sensor in FIG. 8A and the RR interval time-series data from the smartphone 11 in FIG. 8B before the low-pass filtering process in step ST22 are considerably different. Therefore, it was attempted to filter RR interval time-series data from the smartphone 11 by a low pass filter using a smoothing spline.

  FIG. 9 is a diagram showing how RR interval time-series data after low-pass filtering changes when the smoothing parameter (spar) is changed.

  The smoothing parameter (spar) of FIG. 9A is 0.7. The smoothing parameter (spar) of FIG. 9 (b) is 0.6. The smoothing parameter (spar) of FIG. 9C is 0.5. The smoothing parameter (spar) of FIG. 9D is 0.4. The smoothing parameter (spar) of FIG. 9 (e) is 0.3. The smoothing parameter (spar) of FIG. 9F is 0.2.

  The RR interval time series data obtained from the commercially available heart rate sensor shown in FIG. 8A is compared with the RR interval time series data after the low-pass filtering processing shown in FIGS. 9A to 9F. Favorable results which both approximate in the smoothing parameter (spar) in the range of 0.2-0.4 are obtained. The smoothing parameter (spar) in the range of 0.3 to 0.4 gives more preferable results in which the both are more similar. Therefore, in the present embodiment, the range of 0.3 to 0.4 as the value of the smoothing parameter (spar) that brings the RR interval time series data obtained from a commercially available heart rate sensor and the RR interval time series data obtained from the smartphone 11 closest to each other. Is used.

(Abnormal value removal processing of RR interval time series data of step ST24)
Table 1 is a table for explaining abnormal value removal processing of RR interval time-series data. This will be described below with reference to Table 1.

  The first column of Table 1 is a number corresponding to the order in which the RR interval time-series data was detected. The first to fourteenth times at which RR interval time-series data is first detected are described in Table 1.

  The second column of Table 1 shows the values of RR interval time series data. The value of RR interval time series data detected at the first time is the value of RR interval time series data detected at time t1 and time 2. The value of the RR interval time-series data detected at time t2... The 13th time is time t13, and the value of the RR interval time-series data detected at the 14th time is time t14.

  The third column of Table 1 is a value obtained by dividing 60 by the value of the RR interval time-series data to obtain an instantaneous heart rate (HR) inversely proportional to the value of the RR interval time-series data.

  The fourth column in Table 1 shows the results of calculation of the absolute value | 60 / tn−60 / tn−1 | of the increase / decrease amount of each instantaneous heart rate (HR). This is an operation corresponding to rule 1.

  The fifth column of Table 1 moves the instantaneous heart rate of the instant heart rate (HR) and the predetermined number of beats before the instant heart rate (HR) (8 beats in the example of Table 1) It is the absolute value of the difference with the value averaged by averaging. This is an operation corresponding to rule 2. That is, 60 / t9 which is the 9th instantaneous heart rate (HR) and meanHR which is the moving average corresponding to the 9th (8 (t1, t2, t3, t4, t5, t6, t7, t8 )) The absolute value of the difference between (the first to eighth moving averages) is the ninth HR difference. Similarly, 60 / t10 which is the 10th instantaneous heart rate (HR) and meanHR which is the moving average corresponding to the 10th measurement (8 (t2, t3, t4, t5, t6, t7, The absolute value of the difference between t8 and t9)) (moving average from the second time to the ninth time) is the tenth HR difference. Likewise, the calculation is performed for the 11th and subsequent times.

  The fact that the 12th column in Table 1 is described as "error" means that the 12th instantaneous heart rate (HR), 60 / t12, is used as the error value to calculate the HR difference. It does not use, it means that t12 which is the said RR interval time series data is removed and it does not make it the object of data processing.

  When removing RR interval time-series data, the three rules of the first rule, the second rule, and the third rule described above are applied. If any one of the first to third rules is not cleared, the RR interval time-series data may be removed and not subjected to data processing, and the first to third rules may not be used. If it does not clear any two or more combinations of (including all combinations of the first to third rules as a matter of course), the RR interval time-series data is removed and not subjected to data processing It may be

  In the present embodiment, the first rule (rule 1), the second rule (rule 2), and the third rule (rule 3) are used in combination as follows.

  The first rule is used as follows. The first rule is not applied since there is no previous data of RR interval time series data at the first time. When the calculation result of | 60 / t (n) -60 / t (n-1) |, which is the difference between the instantaneous heart rate and the instantaneous heart rate of the previous time, is less than or equal to a first predetermined value for the second and subsequent times The RR interval time series data is adopted only for the calculation, and together with the calculation of meanHR (8), which is the instantaneous heart rate (HR) of 8 beats, | 60 / t (n) −60 / t ( The instantaneous heart rate (60 / t (n)) is used only when the calculation result of n-1) | is less than the first predetermined value.

  The first to eleventh times in Table 1 are processed according to rule 1 when the calculation result of | 60 / t (n) −60 / t (n−1) | is less than or equal to a first predetermined value.

  If the calculation result of | 60 / t (n) -60 / t (n-1) | is larger than the first predetermined value, an instantaneous heart rate (HR) different from the previous one suddenly occurs due to noise or the like (That is, it is determined that RR interval time series data different from the previous one suddenly occurred due to noise or the like), the RR interval time series data is removed and not subjected to data processing. At the same time, in the present embodiment, also when calculating meanHR (8), which is the instantaneous heart rate (HR) of eight beats, the instantaneous heart rate which becomes an error (error) by applying the first rule Do not use (HR) for moving average calculation. Here, the notation of meanHR (8) means a moving average value for an instantaneous heart rate (HR) of 8 beats, and if it is a moving average value of N beats, it is written as meanHR (N) Do. The value of N can be determined appropriately.

  The 13th calculation of meanHR in Table 1 follows this rule, and | 60 / t13-meanHR. (t9, t10, t11)) | (the fourth to eleventh moving averages) are used. The 13th meanHR (| 60 / t13-meanHR) (8 (t5, t6, t7, t8, t9, t10, t11, t12)) that is supposed to be present is not used. In addition, the 14th calculation of meanHR (8) is also based on this rule, and | 60 / t14-meanHR (8 (t5, t6, t7, t8, t9, t10) can be obtained as meanHR. , t11, t13)) | (moving average excluding the 12th time from the 5th time to the 13th time).

  Further, although not shown in Table 1, the second rule is also used in the present embodiment. The second rule is that the abnormal instantaneous heart rate (HR) different from the average value of the previous eight beats when the calculation result of | 60 / t-meanHR (8) | It is determined that the RR interval time-series data is removed and not subjected to data processing. Note that, depending on the magnitude relationship between the first predetermined value and the second predetermined value, | 60 / t (n) −60 / t (n−1) | ≦ first predetermined value (first rule) or | 60 / t-meanHR (N) |) (the second rule) which is the second predetermined value or either of them is determined to be an abnormal value.

  Although not shown in Table 1, the third rule is also used in this embodiment. The third rule is that the calculation result of the moving average value (meanHR, R (8)) for a predetermined number of beats (for example, eight beats) is a range of the second predetermined value continuously for a predetermined number of times (for example, eight times) If not, reset the meanHR (8) and calculate the meanHR (8) again. This operation corresponds to the first to eighth processes of Table 1, but unlike Table 1, since the first value is detected in this case, the first rule is applied to the first time, The moving average value is obtained at the 8th time in Table 1 early once.

(Fourier transform processing of step ST25)
The Fourier transform process tried three types of windows: a square window (without a window), a Hanning window, and a Hamming window as well-known techniques as a window. As a result, the Hamming window is used because the best result is obtained in the separation of the high frequency component HF and the low frequency component LF described later. Since the RR interval time-series data is subjected to functionalization processing, it is possible to perform fast Fourier transform that performs resampling at a constant period.

(Detection processing of high frequency component HF and low frequency component LF in step ST26)
The high frequency component HF represents the activity of the parasympathetic nerve, and the low frequency component LF is defined as an index representing the activity of the sympathetic nerve, and is used in existing articles and literatures. The high frequency component HF is an integral value of a power spectrum of 0.15 to 0.4 Hz obtained by a filter that passes a band of 0.15 to 0.4 Hz after Fourier transform. The low frequency component LF is an integral value of a power spectrum obtained by a filter that passes a band of 0.04 to 0.15 Hz.

  The low frequency component LnLF is a logarithmic representation of the low frequency component LF. The low frequency component LnLF approaches normal distribution when measured multiple times by logarithmic display. The high frequency component LnHF is a logarithmic representation of the high frequency component HF. The high frequency component LnHF approaches a normal distribution when measured a plurality of times by logarithmic display.

  The larger the value of the low frequency component LnLF, the higher the sympathetic activity, and the smaller the value of the low frequency component LnLF, the lower the sympathetic activity. Further, the larger the value of the high frequency component LnHF, the higher the activity of the parasympathetic nerve, and the smaller the value of the high frequency component LnHF, the lower the activity of the parasympathetic nerve.

(Fractal analysis in step ST30)
The fractal analysis in step ST30 performed by the CPU of the server 12 will be described.
Fractal is a term that Mandelbrot named from Latin in the meaning of "Crushed Stone" in 1975, and refers to figures and structures with non-integer dimensions. The feature is self-similarity which is similar to the whole even if the minute part is. There is a fractal dimension as a common measure indicating the degree of complexity of a figure or the like, and the more complex the figure or the like, the larger the value of the fractal dimension.

  As described in Non-Patent Document 4, it is claimed that the fractal dimension of visual information such as a picture or a landscape is related to its preference with respect to the fractal dimension. However, little research has been conducted on the relationship between the degree of healing calculated from the fluctuation of the heart rate and the fractal dimension. Therefore, the inventors (hereinafter simply referred to as inventors) described in the application for the present patent application, as described above, detect healing from heart rate information that depends on human's mental state when viewing an image. It was possible to search whether there was a way to detect similar indices from the image itself, and to extract indices similar to the healness from the image itself. This index, which is extracted from the image itself, is referred to below as the degree of healing obtained from the image itself, or simply the degree of healing.

  That is, the inventors reached the conclusion that when the fractal dimension of the image is low, there is a tendency that the degree of healing obtained from the image itself is high. Briefly described, an image with high fractal dimension generally gives a complicated and messy feeling and tends to give a strong impression, whereas an image with low fractal dimension tends to give a majestic feeling. And if the values of the fractal dimension are similar, the stimuli are similar, and based on the hypothesis that the stimuli when the fractal dimension is a distant value may not be similar, visualizing by the fractal dimension The impact on personal attention will have certain determinism. From such a theory, human psychology obtained from images having similar fractal dimensions is similar, and this psychology affects the heart rate (pulse wave), and images with low fractal dimensions are highly healed. The inventors speculate that it may be detected as

  In the embodiment, in order to calculate the fractal dimension, the original image is converted to grayscale and replaced with a three-dimensional image in which pixel values are represented by shading (cube method). The space thus replaced in three dimensions is called an image density space. As a method of estimating the fractal dimension indicating the complexity of the unevenness of the density surface of the image density space, a method of estimating the density value of pixel (x) according to the fractal brown function was used (see Non-Patent Document 2). Defining the fractal brown function, which is a representative descriptive model of a statistical fractal characterized by self-similarity, with the density value of the image, the following equation (21) is obtained for an arbitrary displacement (x). Here, E [] is an expected value when? X is fixed, l (x) is a concentration value at x, and H and C are constants.

E [| (l (x + Δx) −l (x) |] × | Δx | ^−H = C (21)

  Since H and C in the equation (21) are constants, the equation (21) is logarithmically converted to obtain the equation (22). The slope of the linear portion of the graph in which C and x are plotted logarithmically is represented by H.

log C = log E [| (l (x + Δx) −l (x) |] −H log | Δx | (22)

The fractal dimension D of the three-dimensional surface of the density space of the image by H is given by the following equation (23).

  D = 3-H (23)

As another method of obtaining the fractal dimension D, there is one called a plane method which further simplifies the calculation. In this method, an arbitrary box size whose size is determined by the parameter r is set, and the number of small area units covering the surface is counted. In this method, the equation for H is log N (r) = − H log r + log N (1), and finally the equation corresponding to equation (23) is D = 2−H.

  Thus, the processing of step ST30 (fractal analysis) can be performed to arrange the captured images in descending order of the degree of healing of the images themselves. The inventors believe that if the contrast processing as shown on the screen 10 of FIG. 4 is performed statistically on a large amount of data, the accuracy of the knowledge on the relationship between the fractal dimension and the healing degree is further enhanced. Furthermore, as a result of the accumulation of such statistical processing, the inventors believe that it is possible to automatically organize image data by analyzing the nature of the image itself without measuring the movement of the operator's mind.

  In the method of automatic arrangement of image data using the process of fractal analysis, from the stage of automatic arrangement of image data without measuring the movement of the mind of the operator, fractal analysis in addition to the measurement of the movement of the mind of the operator There are several stages up to the stage of using the results of When using the result of fractal analysis in addition to measurement of the movement of the mind of the operator, it is said that the operator is healed as described above when separating the above-mentioned "degree of healing" and "impression" Automatic separation can be performed without inputting subjectivity. That is, after detecting an image with a high degree of healing detected by the autonomic nerve index according to the equations (1) to (8), only an image judged to have a high degree of healing as a result of fractal analysis It can be detected automatically, and images that have a high degree of healing can be detected by excluding images that are “inspiring”. The same method can be used to separate "excitation" and "stress".

"Modification of the first embodiment"
(First modification)
In the embodiment described above, the connection between the client and the server in the image data organizing system has been described as using the Internet line, but the connection between the two is not limited to this, regardless of the dedicated line or public line. Any line can be used regardless of whether it is a wired communication line or a wireless line. For example, a LAN, a dedicated line using dedicated weak radio waves, a high-speed general-purpose line, or the like can be used.

(2nd modification)
The image data organizing terminal device as a client can use various devices including a smartphone, a tablet, and a dedicated terminal. Further, the image data organizing terminal device may be configured not only with the computer (MPU) and the program executed in the MPU as the center, but also part or all of the processing may be performed by a random logic circuit. In this case, each processing unit is not realized by causing a computer to execute a program, but realized by hardware.

  The image data organizing server device as a server is not only realized by a combination of a computer (CPU) and software, but also part or all of each processing is constituted by hardware such as integrated circuits and random logic circuits. You may do it. In this case, each process performed by hardware such as a random logic circuit is performed not by a computer and a program but by each processing unit configured by hardware.

  In hardware configuration, forward logic processing is performed by the forward logic processing unit, functionalization processing of the time-series luminance data is performed by the functionalization processing unit of the time-series luminance data, and noise removal and peaking are performed by the Savitzky-Golay filter unit. Detection processing is performed, RR interval time series data detection processing unit performs RR interval time series data detection processing, low pass filtering processing unit performs low pass filtering processing, RR interval time series data abnormal value removal processing unit Performs outlier removal processing of RR interval time series data, performs Fourier transform processing in the Fourier transform processing unit, detects high frequency component HF and low frequency component LF in detection processing unit of high frequency component HF and low frequency component LF Execute the process

(Third modification)
Forward logic processing, noise removal processing and peak detection processing by Savitzky-Golay filter, low pass filtering processing, abnormal value removal processing of RR interval time series data are all noise included in time series luminance data or at RR interval It is a process for removing noise included in the series data, and any one of them or a combination of two or more of them (including all combinations of these) can be used appropriately.

  In addition, how to divide the processing in the client's image data organizing terminal device and the processing in the server's image data organizing server device can be determined appropriately. For example, forward correction logic processing (step ST19), functionalization processing of time-series luminance data (step ST20), Savitzky-, in order to correct differences in performance of each camera in the image data organization terminal device in the image data organization terminal device. Noise removal and peak detection processing by Golay filter (step ST21), detection processing of RR interval time series data (step ST22), low pass filtering processing (step ST23), abnormal value removal processing of RR interval time series data (step ST24) Some or all of them may be performed in the image data organizing terminal. By doing this, it is possible to perform appropriate processing in accordance with the characteristics of the camera in the image data organizing terminal device.

(4th modification)
In the first embodiment, since the image data organizing terminal device has one camera, for example, when capturing an image of a landscape, it was not possible to simultaneously capture the fingertip of the hand. Therefore, immediately after capturing an image of a landscape, the position of the camera must be changed to capture the fingertip of the hand, which is cumbersome. Therefore, in order to improve this, we will do the following.

  The camera of the image data organizing terminal device (smart phone 11) is equipped with an auxiliary camera at a position where the fingertip of the hand is imaged when the smartphone 11 is held by hand, in addition to the camera for imaging landscapes and the like. In this way, it is possible to obtain time-series luminance data from the auxiliary camera immediately after capturing a landscape etc. while maintaining the position of the finger of the hand as it is when capturing a landscape etc.

(5th modification)
The results of the above-described fractal analysis can be used as follows. The fractal dimension of the image obtained as a result of the fractal analysis and the music preferred by the person viewing the image of the fractal dimension (music corresponding to the image fractal dimension) are stored in the server 12 in advance. Then, after the server 12 performs fractal analysis on the image sent from the smartphone 11, music corresponding to the image fractal dimension stored in the server 12 is downloaded to the smartphone 11. In this way, the person carrying the smartphone 11 can automatically enjoy the music that most closely matches the scenery currently being viewed, without selecting the music by his / her intention.

  The main points of the first embodiment are summarized as follows.

  The image data organizing system is an image data organizing system including an image data organizing terminal device and an image data organizing server device, and the image data organizing terminal device includes an imaging unit for capturing an image and acquiring image data; A time-series heartbeat data acquisition unit that acquires time-series heartbeat data corresponding to a heartbeat when acquiring data, a time-series heartbeat data output processing unit that outputs time-series heartbeat data to an image data organizing server device, image data organizing An image display unit for displaying the captured images arranged in descending order of degree of healing output from the server device; and the image data organizing server device performs Fourier transform processing to Fourier transform time-series heartbeat data A high frequency component / low frequency component detection processing unit for separating high frequency components and low frequency components of the signal subjected to the Fourier transform processing, and Comprising the autonomic nerve analysis engine processing unit for obtaining the healed degree from the high-frequency component, and Soteingu processing unit that arranges the image captured in high healed order of degree, the. More preferably, the autonomic nerve analysis engine processing unit divides at least the high frequency component by the reference high frequency component measured in advance to obtain the healing degree.

  In addition, the image data organizing server device calculates a fractal dimension of the image captured from the captured image, a fractal analysis processing unit, an impression that the image is given, and a fractal dimension corresponding to the image. The image processing apparatus may further include a sorting processing unit that arranges the captured images in descending order of the impression of healing based on statistical analysis of the relationship.

  The image data organizing terminal device may be a smartphone, the imaging unit may be configured by a camera function of the smartphone, and the time-series heartbeat data acquisition unit may be configured by capturing an image of a finger with the camera function of the smartphone.

  The image data organizing server device program is an image data organizing server device program to be executed by the computer of the image data organizing server device that operates in cooperation with the image data organizing terminal device in the image data organizing system. A Fourier transform processing unit that performs Fourier transform on the acquired time-series heartbeat data, a high frequency component / low frequency component detection processing unit that separates high frequency components and low frequency components of the signal subjected to Fourier transform processing, and degree of healing from high frequency components The computer is functioned to include an autonomic nerve analysis engine processing unit for obtaining the image processing unit, and a sorting processing unit for arranging the images captured in order of high healing degree. More preferably, the autonomic nerve analysis engine processing unit divides at least the high frequency component by the reference high frequency component measured in advance to obtain the healing degree.

  The image data organizing terminal device program is an image data organizing terminal device program to be executed by the computer of the image data organizing terminal device that operates in cooperation with the image data organizing server device in the image data organizing system. A time-series heartbeat data acquisition unit that acquires time-series heartbeat data according to the time-series heartbeat data output processing unit that outputs the time-series heartbeat data to the image data organizing server device; And an image display unit for displaying the captured images arranged in descending order of healing degree.

"2nd Embodiment"
In the first embodiment, the image data reduction system is configured as a client server system. The second embodiment relates to an image data organizing apparatus and an image data organizing apparatus program according to an embodiment in which all processing in the server of the first embodiment is transferred to the client. Such an image data organizing apparatus corresponds to, for example, the case where the smartphone 11 described in FIG. 1 performs all processing independently without receiving the support of the server 12. When the smartphone 11 functioning as an image data organizing device alone performs all of the above-described processing, the server may not be used. Further, the image data organizing apparatus operates only by the image data organizing apparatus program.

  The points of the second embodiment are summarized as follows.

  The image data organizing apparatus Fourier-transforms time-series heartbeat data by an imaging unit that captures an image and acquires image data, a time-series heartbeat data acquisition unit that acquires time-series heartbeat data according to a heartbeat when acquiring image data, and A Fourier transform processing unit; a high frequency component / low frequency component detection processing unit for separating high frequency components and low frequency components of the signal subjected to Fourier transform processing; an autonomic nerve analysis engine processing unit for obtaining healing degree from the high frequency components; The image processing apparatus includes a sorting processing unit that arranges image data in descending order of healing degree, and an image display unit that displays image data arranged in descending order of healing degree. More preferably, the autonomic nerve analysis engine processing unit divides at least the high frequency component by the reference high frequency component measured in advance to obtain the healing degree.

  The image data organizing device program is an image data organizing device program to be executed by a computer of the image data organizing device, and includes an imaging unit that captures and acquires image data, time-series heartbeat data according to heartbeats when acquiring image data. Time-series heart rate data acquisition unit for acquiring the time-series heart rate data, a Fourier transform processing portion for performing Fourier transform on the time-series heart rate data, high frequency component / low frequency component detection process for separating high frequency component and low frequency component of the signal subjected to Fourier transform processing The image display displaying the image data arranged in the order of high healing degree, the sorting processing unit arranging the image data in order of high healing degree, the autonomic nerve analysis engine processing unit for obtaining the healing degree from the high frequency component Functioning the computer to comprise: More preferably, the autonomic nerve analysis engine processing unit divides at least the high frequency component by the reference high frequency component measured in advance to obtain the healing degree.

"3rd Embodiment"
In the first and second embodiments, the image data organizing system, the image data organizing server device, the image data organizing terminal device, the image data organizing server device program, and the image data organizing terminal device program It was to organize image data as an arrangement standard. However, based on the same technical idea as the first embodiment and the second embodiment, the image data can be organized based on other organization criteria different from the “degree of healing”. In the third embodiment, an image data organizing system, an image data organizing server device, an image data organizing terminal device, an image data organizing server device program, an image data organizing terminal device program which organizes image data based on "excitation level" as an organization standard It is.

  As mentioned above, the high frequency component is related to the activity of the parasympathetic nerve. On the other hand, low frequency components are known to be involved in sympathetic nerve activity. The high frequency component divided by the low frequency component also represents the activity of the parasympathetic nerve, the logarithm of the high frequency component also represents the activity of the parasympathetic nerve, and the one dividing the logarithm of the high frequency component by the logarithm of the low frequency component is also the parasympathetic Represents an activity. On the other hand, the low frequency component divided by the high frequency component also represents sympathetic activity, the logarithm of the low frequency component also represents sympathetic activity, and the one dividing the logarithm of the low frequency component by the logarithm of the high frequency component is also sympathetic It is known to represent the activity of Using this, it is possible to organize the image data with the "excitation level" as the organizing reference. In the following description of the third embodiment, although the description will be made without citing the figures, the “healed degree” is replaced with the “excitation degree”, and FIGS. 1 to 5 already described in the third embodiment will be described. It can be incorporated into the description.

  The low frequency component and the low frequency component divided by the high frequency component, the low frequency component logarithm and the low frequency component logarithm divided by the high frequency component logarithm are all sympathetic Activity is high. And, the higher the sympathetic activity, the higher the degree of excitement.

Specifically, the degree of excitement is determined as follows.

Excitedness level = (low frequency component) ........... (24)

Degree of excitement = (logarithm of low frequency component) ........... (25)

Degree of excitement = (low frequency component / high frequency component) ..... (26)

Degree of excitation = {(log of low frequency component / log of high frequency component)} (27)

  However, there are some who are not excited even if the degree of excitement is high, while some are excited even if the degree of excitement is low. That is, the relationship between the low frequency component and the low frequency component divided by the high frequency component, the logarithm of the low frequency component, and the logarithm of the low frequency component divided by the logarithm of the high frequency component There is a difference. Therefore, the degree of excitation is determined based on the difference in magnitude between the low frequency component at the time of measurement and the reference low frequency component. In this way, the degree of excitement, which is the degree of excitement, can be normalized without individual differences.

Specifically, the normalized excitation level is determined as follows.

Excitedness level = (low frequency component LF) / (reference low frequency component LF) ... (28)

Excitation degree = (log LnLF of low frequency component) / (log LnLF of reference low frequency component) ··· (29)

Excitation degree = (low frequency component LF / high frequency component HF) / (reference low frequency component LF / reference high frequency component HF)
= (Low frequency component LF / reference low frequency component LF) / (high frequency component HF / reference high frequency component HF) · · · (30)

Degree of excitation = {(log LnLF of low frequency component / log LnHF of high frequency component) / (log LnLF of reference low frequency component / log LnHF of reference high frequency component)}
= {(Log LnLF of low frequency component / log LnLF of reference low frequency component) / (log LnHF of high frequency component / log LnHF of reference high frequency component)} (31)

  Based on which of the above equations (24) to (31), the degree of excitation can be determined arbitrarily.

  The reference low frequency component is a low frequency component in a reference excitation state, and the reference high frequency component is a high frequency component in a reference excitation state. Here, the reference excited state is an excited state in a normal daily life, and is neither an excessively relaxed state nor an excessively excited state, but in a relaxed calm normal mind. I am excited. Generally, the reference low frequency component and the reference high frequency component in the equations (24) to (31) are the same as the reference low frequency component and the reference high frequency component in the equations (1) to (8).

  The following equations in the third embodiment and the equations in the first embodiment correspond to the following. (32) corresponds to (9), (33) corresponds to (10), (34) corresponds to (11), and (35) corresponds to (12) The equation (36) corresponds to the equation (13), the equation (37) to the equation (14), the equation (38) to the equation (15), and the equation (39) to the equation (16) (40) corresponds to (17), (41) corresponds to (18), (42) corresponds to (19), and (43) corresponds to (20) Corresponds to the) expression. In the following, the logarithm LnLF of the low frequency component is omitted as the low frequency component LnLF, the logarithm LnHF of the high frequency component is abbreviated as the high frequency component LnHF, and the logarithm LnLF of the reference low frequency component is omitted as the reference low frequency component LnLF, and the reference high frequency The logarithm LnHF of the component is abbreviated as the reference high frequency component LnHF.

  The angle θ between the straight line passing through the center point extending radially from the center point of the arc of the half donut portion (the straight line of “arrow” on the screen 16) and the straight line connecting the center point and the left end of the half donut portion is When the equation (29) is used, the CPU obtains the following equation (32). When the equation (29) is used, the CPU obtains the equation (33). Here, K4 and K5 are coefficients, and increasing the values of K4 and K5 emphasizes the degree of excitement.

θ (°) = 2 × tan ⁻¹ {K 4 × (low frequency component LF) / (reference low frequency component LF)} ··· (32)

θ (°) = 2 × tan ⁻¹ {K 5 × (low frequency component LF / high frequency component HF) / (reference low frequency component LF / reference high frequency component HF)}. ..... (33)

  Further, the angle θ between the straight line passing through the central point radially extending from the center point of the arc of the half donut portion (the straight line of “arrow” on the screen 16) and the straight line connecting the central point and the left end of the half donut portion is When the equation (28) is used, it is obtained by the following equation (34), and when the equation (30) is used, it is obtained by the following equation (35). Here, K6 and K7 are coefficients, and increasing the values of K6 and K7 emphasizes the degree of excitement.

θ (°) = [90 + tan ⁻¹ {K6 × (low frequency component LnLF) / (reference low frequency component LnLF)}]. ... (34)

θ (°) = [90 + tan ⁻¹ {K 7 × (low frequency component LnLF / high frequency component LnHF) / (reference low frequency component LnLF / reference high frequency component LnHF)}]. )

  When the excitation level is converted into the range of 0 to 100, it can be obtained by the following equation (36) when the equation (28) is used, and by the following equation (37) when the equation (30) is used Desired. Here, K8 and K9 are coefficients, and increasing the values of K8 and K9 emphasizes the degree of excitement.

Excitedness level = (100/90) × tan ^-1 {K8 × (low frequency component LF) / (reference low frequency component LF)} ·········· ...... (36)

Degree of excitation = (100/90) × tan ⁻¹ {K 9 × (low frequency component LF / high frequency component HF) / (reference low frequency component LF / reference high frequency component HF)}. ... (37)

  When the excitation level is converted into the range of -100 to 100, it can be obtained by the following equation (38) when the equation (28) is used, and when the equation (30) is used, the following equation (39) It is determined by Here, K10 and K11 are coefficients, and increasing the values of K10 and K11 emphasizes the degree of excitement.

Excitation degree = −100 + (200/90) × tan ⁻¹ {K10 × (low frequency component LF) / (reference low frequency component LF)}. .......... (38)

Degree of excitation = −100 + (200/90) × tan ⁻¹ {K11 × (low frequency component LF / high frequency component HF) / (reference low frequency component LF / reference high frequency component HF)}. ... (39)

  When the excitation level is converted into the range of 0 to 100, it can be obtained by the following equation (40) when the equation (28) is used, and by the following equation (41) when the equation (30) is used Desired. Here, K12 and K13 are coefficients, and increasing the values of K12 and K13 emphasizes the degree of excitement.

Excitation degree = (100/180) × [90 + tan ⁻¹ {K12 × (low frequency component LnLF) / (reference low frequency component LnLF)}]. ..... (40)

Degree of excitation = (100/180) × [90 + tan ⁻¹ {K13 × (low frequency component LnLF / high frequency component LnHF) / (reference low frequency component LnLF / reference high frequency component LnHF)} (41)

  When the excitation level is converted into the range of -100 to 100, it can be obtained by the following equation (42) when the equation (28) is used, and when the equation (30) is used, the following equation (43) It is determined by Here, K14 and K15 are coefficients, and increasing the values of K14 and K15 emphasizes the degree of excitement.

Degree of excitement = (100/90) × tan ⁻¹ {K14 × (low frequency component LnLF) / (reference low frequency component LnLF)}. ...... (42)

Excitation degree = (100/90) × tan ⁻¹ {K15 × (low frequency component LnLF / high frequency component LnHF) / (reference low frequency component LnLF / reference high frequency component LnHF)}. )

  As described above, in the third embodiment, the arithmetic expression in the first embodiment is changed from one for obtaining the "healing degree" to one for obtaining the "excitation degree", thereby the same as in the first embodiment. Image data can be organized using the degree of excitement as the organizing standard. The image data suitable for organizing the image data with the degree of excitement as the organizing standard is, for example, image data of a sport match captured while watching a game. While watching a game, a large number of image data captured one after another without thinking too much deeply can be automatically organized without any difficulty, using only the exciting image as a basis for organizing the degree of excitement.

  Note that the MPU records its own action at the time of imaging by the operator (for example, a record of its own action such as walking, moving by a vehicle, rest, etc., and further, a recording of a numerical value of a subjective degree of self judgment). It can also be tied. In this case, the CPU of the server 12 detects the operator based on the expressions (24) to (31). In addition to the excitement level, the action of the operator at the time of shooting and the subjective surface at the time of shooting As the determination criterion in the above case, the accuracy of image data arrangement can be further improved. For example, in the index detected based on the equations (24) to (31), "stress" may be mixed in addition to the "excitation degree". With reference to the record, when the self-judged excitement level is high, the CPU of the server 12 can judge that the excitement level is detected without fail and can separate “excitation level” and “stress” .

"Modification of the third embodiment"
Also in the third embodiment, the same embodiment as the first to fourth modifications of the first embodiment can be implemented.

  The main points of the third embodiment are summarized as follows.

  The image data organizing system is an image data organizing system including an image data organizing terminal device and an image data organizing server device, and the image data organizing terminal device includes an imaging unit for capturing an image and acquiring image data; A time-series heartbeat data acquisition unit that acquires time-series heartbeat data corresponding to a heartbeat when acquiring data, a time-series heartbeat data output processing unit that outputs time-series heartbeat data to an image data organizing server device, image data organizing A Fourier transform processing unit that comprises: an image display unit that displays captured images arranged in descending order of excitement, output from the server device; and the image data organizing server device performs Fourier transform on time-series heartbeat data And a high frequency component / low frequency component detection processing unit for separating high frequency components and low frequency components of the signal subjected to the Fourier transform processing; Comprising the autonomic nerve analysis engine processing unit for determining the level of excitement from frequency components, and a Soteingu processing unit that arranges the image captured in high excitement level order. More preferably, the autonomic nerve analysis engine processing unit divides at least the low frequency component by the reference low frequency component measured in advance to obtain the degree of excitement.

  The image data organizing terminal device may be a smartphone, the imaging unit may be configured by a camera function of the smartphone, and the time-series heartbeat data acquisition unit may be configured by capturing an image of a finger with the camera function of the smartphone.

  The image data organizing server device program is an image data organizing server device program to be executed by the computer of the image data organizing server device that operates in cooperation with the image data organizing terminal device in the image data organizing system. A Fourier transform processing unit that performs Fourier transform on the acquired time-series heartbeat data, a high frequency component / low frequency component detection processing unit that separates high frequency components and low frequency components of the signal subjected to Fourier transform processing, and excitation level from low frequency components The computer is functioned to include an autonomic nerve analysis engine processing unit for obtaining the image processing unit, and a sorting processing unit for arranging the images captured in descending order of the degree of excitement. More preferably, the autonomic nerve analysis engine processing unit divides the low frequency component by the previously measured reference low frequency component to obtain the degree of excitation.

  The image data organizing terminal device program is an image data organizing terminal device program to be executed by the computer of the image data organizing terminal device that operates in cooperation with the image data organizing server device in the image data organizing system. A time-series heartbeat data acquisition unit that acquires time-series heartbeat data according to the time-series heartbeat data output processing unit that outputs the time-series heartbeat data to the image data organizing server device; And an image display unit that displays captured images arranged in descending order of excitement level.

  The image data organizing apparatus Fourier-transforms time-series heartbeat data by an imaging unit that captures an image and acquires image data, a time-series heartbeat data acquisition unit that acquires time-series heartbeat data according to a heartbeat when acquiring image data, and A Fourier transform processing unit; a high frequency component / low frequency component detection processing unit for separating high frequency components and low frequency components of the signal subjected to Fourier transform processing; an autonomic nerve analysis engine processing unit for obtaining an excitation level from the low frequency components; The image processing apparatus includes a sorting processing unit that arranges image data in descending order of excitement level, and an image display unit that displays image data arranged in descending order of excitement level. More preferably, the autonomic nerve analysis engine processing unit divides the low frequency component by the previously measured reference low frequency component to obtain the degree of excitation.

  The image data organizing device program is an image data organizing device program to be executed by a computer of the image data organizing device, and includes an imaging unit that captures and acquires image data, time-series heartbeat data according to heartbeats when acquiring image data. Time-series heart rate data acquisition unit for acquiring the time-series heart rate data, a Fourier transform processing portion for performing Fourier transform on the time-series heart rate data, high frequency component / low frequency component detection process for separating high frequency component and low frequency component of the signal subjected to Fourier transform processing , An autonomic nerve analysis engine processing unit for obtaining an excitation level from low frequency components, a sorting processing section for arranging image data in descending order of excitation level, and an image display unit for displaying image data arranged in descending order of excitation level , To make the computer function. More preferably, the autonomic nerve analysis engine processing unit divides the low frequency component by the previously measured reference low frequency component to obtain the degree of excitation.

"4th Embodiment"
In the image organizing techniques of the first to third embodiments, a technique for informing the operator of the apparatus whether or not an image to be captured satisfies a predetermined organizing criterion, and a plurality of captured images after the fact It provides the technology to organize automatically. In the fourth embodiment, in addition to the above-described technique, since the healing degree can be detected at high speed before imaging, a still image or a moving image having a high healing degree can be immediately selected and imaged. In addition, automatic imaging can be performed when predetermined organization criteria are satisfied.

  In this case, even if a large number of images are captured by automatic imaging, unnecessary images deviated from the organization standard are not captured. Therefore, the storage capacity of the storage device for image data is not wasted. And since only the image with a high degree of healing is taken, sorting can be simplified. Furthermore, when encountering a landscape or the like with a high degree of healing, it is possible to automatically capture the landscape or the like without being conscious of a person, so that the imaging chance is never missed.

  An image data organizing system according to the fourth embodiment will be described with reference to FIG.

  The fourth embodiment has a detection function of the healing degree at high speed independently of imaging, and that the hardware configuration and software processing are different in order to exhibit this function. Since the form is different, the following description will be made focusing on these different points. The same points as the first to third embodiments will not be described.

  The image data organizing system 20 includes an image data organizing terminal device 21, a server 22 functioning as an image data organizing server device, and the Internet 13 connecting the image data organizing terminal device 21 and the server 22.

  The image data organizing terminal device 21 includes a wearable personal computer (personal computer attachable to the body), a heart rate sensor, and various external data detection units. The heart rate sensor has a function of being attached to the skin of the chest and transmitting the heart rate, heart rate waveform (RR interval data), and heart rate period wirelessly to the wearable personal computer in real time. The personal computer controls the heart rate sensor and various external data detection units, and performs uploading and downloading of information with the server 22.

  The image data organizing terminal device 21 has a wearable camera (camera that can be worn on the body) as an image data detection unit in addition to the various external data detection units in the first embodiment. The wearable camera, for example, fixes a band on the head of a person and captures an image in a range equivalent to an image visible to the human eye in a normal state, and the operator of the image data organizing terminal device 21 The imaging range of the image can be enlarged or reduced according to the intention of the operator.

  The image data organizing terminal device 21 is provided with a heartbeat sensor separately from the camera function, so that imaging of a landscape etc. and detection of heartbeat information can be performed independently. It is possible to always detect heart rate information by interrupt processing. Furthermore, since the heart rate sensor is a sensor for detecting a dedicated heart beat placed near the heart, it is possible to detect RR interval time series data with high accuracy (see (a) in FIG. 9). As a result, in autonomic nerve analysis engine processing, it is possible to directly perform Fourier transform processing without preprocessing (see “autonomic nerve analysis engine processing” in FIG. 15). As a result, the interrupt time in the interrupt processing of the MPU can be shortened to such an extent that it can be considered that the healing degree can be detected instantaneously in real time, and the healing degree can be detected substantially simultaneously when image data is acquired. Further, since the wearable camera is included, the scenery etc. that the operator is watching matches the image to be captured, and automatic imaging becomes possible in combination with the detection of the healing degree in real time.

  Instead of the wearable personal computer, a multifunctional portable terminal (tablet) having the same function, a smartphone, or an image data organizing terminal device specialized for use in the present embodiment may be used. Since these devices have Wi-Fi (registered trademark), Bluetooth (registered trademark) or the like as a standard wireless interface, a heart rate sensor (for example, the commercially available MyBeat described above) is connected by wireless can do.

  The image data organizing terminal device 21 “(1) upload measurement data (RR interval data), captured image to the server 22”, “(2) various external data (for example, weather, air temperature, barometric pressure, position data, Upload the acceleration data) to the server 22. When uploading various external data to the server 22, the measurement data and image data are linked with various external data and uploaded. The contents of these processes will be described later.

The server 22 “configures (3A) (autonomic nervous analysis engine) and analyzes the living body” by the server CPU and a program, and “(3B) (image analysis engine) configures the fractal analysis Perform processing and "(4) Return the analysis result".
The server 22 analyzes the healing degree of the operator's mind by the biological analysis process of (3A). The server 22 performs objective analysis processing of the image itself captured by the fractal analysis of (3B). Since (3A) and (3B) are analysis processes from different points of view, both or either of them can be adopted.

  The server 22 returns the analysis result of the processing of (3A) and / or (3B) to the image data organizing terminal device 21. The analysis result of the processing of (3A) is the ranking of the images based on the healing degree by totaling the data accumulated in the database of the server 22, and the analysis result of the processing of (3B) is the database of the server 22 Image ranking based on fractal analysis of accumulated images. In the fourth embodiment, not only the imaging operator determines the imaging timing, but also the imaging timing based on the analysis result of the processing of (3A) and / or the analysis result of the processing of (3B). Informing alert notification can be performed at all times. Furthermore, it is possible to automatically pick up an image as a result of analysis of the process of (3A) and / or as a result of analysis of the process of (3B) on the basis of predetermined arrangement criteria without human intervention.

(Screen displayed on image data organizing terminal)
12 to 14 show images displayed on the liquid crystal screen of the image data organization terminal device 21. FIG.

  FIG. 12 shows a screen 21 to a screen 24 displayed on the image data organizing terminal device 21. FIG. 13 shows the screen 25 to the screen 28 displayed on the image data organization terminal device 21. FIG. 14 shows a screen 29 displayed on the image data organization terminal device 21.

  Since the screens 21 to 23 shown in FIG. 12 are the same as the screens 1 to 3 in the first embodiment, the description will be omitted.

A screen 24 shown in FIG. 12 is a screen prompting the user to wear a heart rate sensor. The MPU of the image data organizing terminal 21 displays “Let's wear a heart rate sensor !!” on the top of the screen 24 and displays the mounting procedure on the screen on the middle of the screen 24.
If the operator wears the heart rate sensor, or touches “Start measurement” at the bottom of the screen 24 immediately after wearing the heart rate sensor, the MPU performs “Autonomic nerve analysis engine processing” in the CPU of the server 22 Ask for Then, the MPU receives the analysis result in “autonomic nerve analysis engine processing” from the server 22, and displays the healing degree on the liquid crystal display surface of the image data organizing terminal device 21 as a screen 25 shown in FIG.

A screen 25 shown in FIG. 13 is an “alert notification screen” for informing the image pickup person of the timing of imaging. In the upper part of FIG. 25, the operator can recognize that "It is just now healed: 70" is displayed and it is 70%. In the middle part of Fig. 25, "You are healed. Now is a photo opportunity !!" is displayed.
Along with the "alert notification screen", "alert notification voice" may be used to notify "It is just healed: 70,""You are healed. Now is a photo opportunity!".
The operator makes a judgment based on the “alert notification screen” or / and the “alert notification voice”, and touches “shoot” in the lower part of FIG. The MPU recognizes that "shoot" is touched and causes the screen 26 to be displayed. When the MPU recognizes that "shoot" is not touched for a predetermined time, it recounts the healing degree at predetermined intervals again. Then, the MPU causes the “alert notification” to be performed again by the “alert notification screen” or / and the “alert notification sound” when the healing degree is a predetermined healing degree or more (for example, 65% or more). Here, the analysis of the degree of healing is automatically performed based on image data captured by the wearable camera with emphasis by the MPU and the CPU.

A screen 26 shown in FIG. 13 is a screen prompting imaging. The upper part of the screen 26 displays “Let's take a photo!”, And the middle part of the screen 26 displays the range of the screen to be imaged.
Although the scenery taken with the wearable camera and the scenery seen by the operator wearing the wearable camera are the same range and angle in advance, they are adjusted in advance. The enlargement and reduction can be operated freely.
When the operator touches the photograph mark on the lower side of the screen 26, the MPU performs an imaging process and controls the image data organizing terminal device 21 and the server 22 to perform a process of detecting the healing degree. Then, the MPU causes the screen 27 to display the analysis result.

The screen 26 shown in FIG. 13 is substantially the same as the screen 4, the screen 27 is the screen 8, the screen 28 is the screen 9, and the screen 29 shown in FIG. 14 is the screen 10. In the fourth embodiment, the absence of the screens 5 to 7 is because imaging of a fingertip is not necessary.
The reason why the healing degree value on the screen 25 and the healing degree value on the screen 27 are different is because the healing time is measured at different points in time, and the healing degree given by the naked eye, etc., through the camera This is because there is a gap between the degree of healing given by the scenery etc seen.

(Outline of processing in image data organizing terminal device and image data organizing server device)
The outline of the processing in the healing degree detection terminal device and the processing in the image data organizing server device of the image data organizing system will be described with reference to FIGS. 15 and 16.

  FIG. 15 and FIG. 16 are diagrams showing a series of continuous processing. The upper part of each drawing of FIG. 15 and FIG. 16 shows the processing in the image data organizing terminal device 21, and the lower part of each drawing of FIG. 15 and FIG. 16 shows the processing in the image data organizing server device (server 22). When there is a corresponding screen in FIG. 12 to FIG. 14 for the process in the upper part of each of FIGS. 15 and 16, the corresponding screen number is added and the description is omitted.

  The process in the image data organizing terminal device 21 is the process of steps ST50 to ST53 and the process of step ST57. The process in the server 22 is the process of steps ST47 to ST49 and the process of steps ST54 to ST56.

  The process order of each process in the series of processes shown in FIGS. 15 and 16 is one embodiment, and this order can be modified and used. The following description will be made in the order indicated by the arrows in FIGS.

(Processing in the image data organization terminal device)
The MPU of the image data organizing terminal 21 performs the processing of steps ST41 (start) to ST43 (input of attribute information). Since this process is the same as the process of steps ST11 (start) to ST13 (input of attribute information), the description is omitted.

The MPU performs the process of step ST44 (attachment screen of heart rate sensor).
This processing is processing for causing the screen 24 to be displayed on the liquid crystal display surface of the image data organization terminal device 21.
The MPU performs the process of step ST45 (measurement start process of the heart rate sensor).
This process starts measurement by the heartbeat sensor to obtain measurement results of RR interval data of a predetermined time length. The predetermined time length is a length sufficient to detect the healing degree with sufficient accuracy. In the fourth embodiment, the RR interval data is accurately detected from the heart rate sensor as shown in FIG. 9 (a).

The MPU performs processing of step ST46 (measurement of RR interval data and upload to server).
This process is a process of uploading RR interval data of a predetermined time length to the server 22.

The MPU performs autonomic nerve analysis engine processing of step ST47 (abnormal value removal processing of RR interval time-series data), step ST48 (Fourier transform processing), and step ST49 (detection processing of high frequency component (HF) and low frequency component (LF)). Do.
This process is simplified as compared with the process of steps ST19 to ST26 in the first embodiment. The reason is that the heart rate sensor is a sensor dedicated to heart rate detection, and noise is less mixed to detect a heart rate signal from a position closer to the heart than a smartphone camera, and good RR interval data can be obtained. . That is, in the fourth embodiment, "forward moving logic", "functionalization processing of luminance data", "noise removal and peak detection processing by Savitzky-Golay filter", and "RR interval time series data detection processing" in the first embodiment. And “low-pass filtering processing”, each processing of “Fourier transform processing” of step ST 48 and “detection processing of high frequency component (HF) and low frequency component (LF)” of step ST 49 with high accuracy. Analysis results can be obtained. The contents of each processing of “Fourier transform processing” of step ST 48 and “detection processing of high frequency components (HF) and low frequency components (LF)” of step ST 49 are the same as the processing of step ST 25 and the processing of step ST 26. I omit explanation.

The MPU performs the process of step ST50 (analysis result display process of healing degree (real time)) and displays the screen 25.
Here, the MPU processes the autonomic nerve analysis engine processing of step ST47, step ST48, and step ST49 by timer interruption for each predetermined time length. Therefore, the MPU of the image data organization terminal device 21 and the CPU of the server 22 analyze the healing degree in real time for each predetermined time length corresponding to the timer interruption time and display it on the liquid crystal of the image data organization terminal device 21. It is possible to perform the processing of step ST50 (analysis result display processing of healing degree (real time)) in real time and update the screen 25 every predetermined time length. However, in the fourth embodiment, the process of step ST50 is performed and the screen 25 is displayed as “alert notification” only when the healing degree exceeds the predetermined value without updating the screen 25 for each predetermined time length. It is like that.

The MPU performs the process of step ST51 (photographing process) and controls the wearable camera to capture an image such as a landscape.
Here, when the operator captures an image such as a landscape, the MPU interrupts the autonomic nerve analysis engine processing for the timer interrupt processing for each predetermined time length, and newly generates an interrupt starting from the imaging time. The autonomic nerve analysis engine processing is performed by this interrupt processing. After that, the autonomic nerve analysis engine processing is continuously performed by the interruption every predetermined time length starting from the imaging time.

The MPU performs the process of step ST52 (analysis result display process of healing degree).
The MPU causes the image to be displayed in step ST51 and displays the healing degree obtained by the interrupt process starting from the time of image pickup as shown on the screen 27 together with the image.

The MPU performs the processing of step ST53 (function of associating with external data) to step ST57 (display of ranking).
The processes of step ST53, step ST54, step ST55, step ST56, and step ST57 of the fourth embodiment are the same as the processes of step ST28, step ST29, step ST30, step ST31, and step ST32 in the first embodiment. Since this is processing, the description is omitted.

"Modification of the fourth embodiment"
(First modification)
Although the connection between the client and the server of the image data organizing system has been described as using the Internet line in the embodiment described above, the connection between the two is not limited to this, and the connection between the both is not limited to this. Any line can be used regardless of whether it is a wired communication line or a wireless line. For example, a LAN, a dedicated line using dedicated weak radio waves, a high-speed general-purpose line, or the like can be used.

(2nd modification)
The image data organizing terminal device as a client can use various things including a wearable personal computer, a tablet, and a dedicated terminal. Further, the image data organizing terminal device may be configured not only with the computer (MPU) and the program executed in the MPU as the center, but also part or all of the processing may be performed by a random logic circuit. In this case, each processing unit is not realized by causing a computer to execute a program, but realized by hardware.

  The image data organizing server device as a server is not only realized by a combination of a computer (CPU) and software, but also part or all of each processing is constituted by hardware such as integrated circuits and random logic circuits. You may do it. In this case, each process performed by hardware such as a random logic circuit is performed not by a computer and a program but by each processing unit configured by hardware.

  In the hardware configuration, the abnormal value removal processing unit for the RR interval time series data performs abnormal value removal processing for the RR interval time series data, and the Fourier transform processing unit performs the Fourier transform processing. The detection processing unit of the component LF executes detection processing of the high frequency component HF and the low frequency component LF.

(Third modification)
How to divide the processing in the client's image data organizing terminal device 21 and the processing in the server 22 can be determined appropriately.

  The points of the fourth embodiment are summarized as follows.

  The image data organizing system is an image data organizing system including an image data organizing terminal device and an image data organizing server device, and the image data organizing terminal device includes an imaging unit for capturing an image and acquiring image data; Time-series heartbeat data acquisition unit that acquires time-series heartbeat data according to each predetermined interrupt time, a time-series heartbeat data output processing unit that outputs time-series heartbeat data to the image data organizing server device, and image data organizing An image display unit for displaying the captured images arranged in descending order of degree of healing output from the server device; and the image data organizing server device performs Fourier transform processing to Fourier transform time-series heartbeat data And a high frequency component / low frequency component detection processing unit for separating high frequency components and low frequency components of the signal subjected to Fourier transform processing; The analysis result of the autonomic nerve analysis engine processing unit is sent to the image data organizing terminal device before the autonomic nerve analysis engine processing unit for obtaining the healing degree from the wave component and the next time-series heartbeat data for each predetermined interruption time is acquired. And a sorting processing unit for arranging the images picked up in descending order of healing degree. More preferably, the autonomic nerve analysis engine processing unit divides the high frequency component by the reference high frequency component measured in advance to obtain the healing degree.

  In addition, the image data organizing server device extracts a boundary of a captured object from the captured image and calculates a fractal dimension by the fractal analysis processing unit, and an image that the image is given, and an image that the image is healed, According to statistical analysis of the relationship with the corresponding fractal dimension, a sorting processing unit may be provided which arranges the images taken in descending order of the impression of being healed.

  The image data organization terminal device may be a wearable personal computer, the imaging unit may be a wearable camera, and the time-series heartbeat data acquisition unit may be a heartbeat sensor disposed in the vicinity of the heart.

  The image data organizing server device program is an image data organizing server device program to be executed by the computer of the image data organizing server device that operates in cooperation with the image data organizing terminal device in the image data organizing system. A Fourier transform processing unit that performs Fourier transform on the acquired time-series heartbeat data, a high frequency component / low frequency component detection processing unit that separates high frequency components and low frequency components of the signal subjected to Fourier transform processing, and degree of healing from high frequency components An autonomic nerve analysis engine processing unit for obtaining an output processing unit that outputs an analysis result of the autonomic nerve analysis engine processing unit to the image data organizing terminal device before acquiring next time-series heartbeat data for each predetermined interrupt time And a sorting processing unit that arranges the images captured in descending order of healing degree. Causing a computer to function to so that. More preferably, the autonomic nerve analysis engine processing unit divides the high frequency component by the reference high frequency component measured in advance to obtain the healing degree.

  The image data organizing terminal device program is an image data organizing terminal device program to be executed by a computer of the image data organizing terminal device operating in cooperation with the image data organizing server device in the image data organizing system, A time-series heartbeat data acquisition unit that acquires heartbeat data at predetermined interrupt times, a time-series heartbeat data output processing unit that outputs time-series heartbeat data to the image data organizing server device, and output from the image data organizing server device And an image display unit for displaying the captured images arranged in descending order of healing degree.

"Fifth embodiment"
In the fourth embodiment, the image data reduction system is configured as a client server system. The fifth embodiment relates to an image data organizing apparatus and an image data organizing apparatus program according to an embodiment in which all processing in the server 22 of the fourth embodiment is transferred to a client. Such an image data organizing apparatus corresponds to, for example, the case where the image data organizing terminal 21 described in FIG. 11 performs all processing independently without the support of the server 22. When the image data organizing apparatus alone carries out all of the above-described processes, the server may not be used.

  The essential points of the fifth embodiment are summarized as follows.

  The image data organizing apparatus Fourier-transforms time-series heartbeat data by an imaging unit that captures an image and acquires image data, a time-series heartbeat data acquisition unit that acquires time-series heartbeat data according to a heartbeat every predetermined interrupt time, and A Fourier transform processing unit; a high frequency component / low frequency component detection processing unit for separating high frequency components and low frequency components of the signal subjected to Fourier transform processing; an autonomic nerve analysis engine processing unit for obtaining healing degree from the high frequency components; An output processing unit that outputs an analysis result of the autonomic nerve analysis engine processing unit to the image data organizing terminal before acquiring next time-series heartbeat data for each predetermined interruption time, and image data in descending order of healing degree And an image display unit for displaying the image data arranged in descending order of the degree of healing. More preferably, the autonomic nerve analysis engine processing unit divides at least the high frequency component by the reference high frequency component measured in advance to obtain the healing degree.

  The image data organizing device program is an image data organizing device program to be executed by a computer of the image data organizing device, and an imaging unit for capturing and acquiring image data, time-series heartbeat data corresponding to a heartbeat every predetermined interrupt time High-frequency component / low-frequency component detection processing for separating high-frequency component and low-frequency component of the signal subjected to Fourier transform processing The analysis result of the autonomic nerve analysis engine processing unit before the next time-series heartbeat data for each predetermined interrupt time is acquired An output processing unit that outputs data to a device, a sorting processing unit that arranges image data in descending order of the degree of healing, and a degree of healing An image display unit for displaying the image data arranged in the order, making a computer function to include. More preferably, the autonomic nerve analysis engine processing unit divides at least the high frequency component by the reference high frequency component measured in advance to obtain the healing degree.

"Sixth embodiment"
In the fourth and fifth embodiments, an image data organization system, an image data organization server device, an image data organization terminal device, an image data organization server device program, an image data organization terminal device program, an image data organization device, and image data The organizing device program was to organize image data using the "healing degree" as the organizing standard. However, on the basis of the same technical idea as the fourth embodiment and the fifth embodiment, the image data can be organized based on other organization criteria different from the “degree of healing”. In the sixth embodiment, an image data organizing system, an image data organizing server device, an image data organizing terminal device, an image data organizing server device program, an image data organizing terminal device program which organizes image data based on “excitability” as organization standard , An image data organizing apparatus and an image data organizing apparatus program.

  The “excitation level” in the sixth embodiment is the same as in the third embodiment, and thus the description is omitted.

  The main points of the sixth embodiment are summarized as follows.

  The image data organizing system is an image data organizing system including an image data organizing terminal device and an image data organizing server device, and the image data organizing terminal device captures an image and acquires image data at predetermined interrupt time intervals. An imaging unit, a time-series heartbeat data acquisition unit for acquiring time-series heartbeat data according to a heartbeat when acquiring image data, and a time-series heartbeat data output processing unit for outputting time-series heartbeat data to an image data organizing server device And an image display unit for displaying captured images arranged in descending order of excitement, output from the image data organizing server device, the image data organizing server device performing Fourier transform on time-series heartbeat data Fourier transform processing unit, high frequency component · low frequency to separate high frequency component and low frequency component of the signal subjected to Fourier transform processing The analysis result of the autonomic nerve analysis engine processing unit before the next time-series heartbeat data for each predetermined interrupt time is acquired, the component detection processing unit, the autonomic nerve analysis engine processing unit for obtaining the degree of excitation from the low frequency component, An output processing unit that outputs data to the data organizing terminal device, and a sorting processing unit that arranges the images captured in descending order of the degree of excitement. More preferably, the autonomic nerve analysis engine processing unit divides at least the low frequency component by the reference low frequency component measured in advance to obtain the degree of excitement.

  The image data organization terminal device may be a wearable personal computer, the imaging unit may be a wearable camera, and the time-series heartbeat data acquisition unit may be a heartbeat sensor disposed in the vicinity of the heart.

  The image data organizing server device program is an image data organizing server device program to be executed by the computer of the image data organizing server device that operates in cooperation with the image data organizing terminal device in the image data organizing system. A Fourier transform processing unit that performs Fourier transform on the acquired time-series heartbeat data, a high frequency component / low frequency component detection processing unit that separates high frequency components and low frequency components of the signal subjected to Fourier transform processing, and excitation level from low frequency components An autonomic nerve analysis engine processing unit for obtaining an output processing unit that outputs an analysis result of the autonomic nerve analysis engine processing unit to the image data organizing terminal device before acquiring next time-series heartbeat data for each predetermined interrupt time And a sorting processing unit that arranges the captured images in descending order of excitement level. The functioning of the sea urchin computer. More preferably, the autonomic nerve analysis engine processing unit divides at least the low frequency component by the reference low frequency component measured in advance to obtain the degree of excitement.

  The image data organizing terminal device program is an image data organizing terminal device program to be executed by a computer of the image data organizing terminal device operating in cooperation with the image data organizing server device in the image data organizing system, A time-series heartbeat data acquisition unit that acquires heartbeat data at predetermined interrupt times, a time-series heartbeat data output processing unit that outputs time-series heartbeat data to the image data organizing server device, and output from the image data organizing server device And an image display unit that displays captured images arranged in descending order of excitement level.

  The image data organizing apparatus Fourier-transforms time-series heartbeat data by an imaging unit that captures an image and acquires image data, a time-series heartbeat data acquisition unit that acquires time-series heartbeat data according to a heartbeat every predetermined interrupt time, and A Fourier transform processing unit; a high frequency component / low frequency component detection processing unit for separating high frequency components and low frequency components of the signal subjected to Fourier transform processing; an autonomic nerve analysis engine processing unit for obtaining an excitation level from the low frequency components; An output processing unit that outputs an analysis result of the autonomic nerve analysis engine processing unit to the image data organizing terminal device before acquiring next time-series heartbeat data for each predetermined interruption time, image data in descending order of excitement degree And an image display unit configured to display image data arranged in descending order of excitement. More preferably, the autonomic nerve analysis engine processing unit divides at least the low frequency component by the reference low frequency component measured in advance to obtain the degree of excitement.

  The image data organizing device program is an image data organizing device program to be executed by a computer of the image data organizing device, and an imaging unit for capturing and acquiring image data, time-series heartbeat data corresponding to a heartbeat every predetermined interrupt time High-frequency component / low-frequency component detection processing for separating high-frequency component and low-frequency component of the signal subjected to Fourier transform processing The analysis result of the autonomic nerve analysis engine processing unit before the next time-series heart rate data for each predetermined interrupt time is acquired An output processing unit that outputs data to a device, a sorting processing unit that arranges image data in descending order of excitement, and an order of high excitement An image display unit for displaying the base is image data, causes a computer to function to include. More preferably, the autonomic nerve analysis engine processing unit divides at least the low frequency component by the reference low frequency component measured in advance to obtain the degree of excitement.

"Seventh embodiment"
The case where the healing degree is detected by the autonomic nerve index detected by the equations (1) to (8) has been described above. In addition to this, the healing degree can also be calculated by using a heart rate (the number of heartbeats per unit time) directly detected from time-series heart rate data from the heart rate sensor. For example, it can be calculated by the following equation (44) or (45). Here, the heart rate in normal times (the reference heart rate) is the heart rate in the normal daily life, not the heart rate in the excessively healed state, too much. It is not the heart rate in the excited state, but it is the heart rate in the state of maintaining normal mind in daily life.

Degree of healing = (Heart rate at the time of measurement-Heart rate in normal times (heart rate as reference)) ... (44)

Degree of healing = (Heart rate at the time of measurement / Heart rate at normal times (Heart rate as reference)) ... (45)

  If it is healed, it is empirically known that the heart rate at the time of measurement tends to be lower than the heart rate in normal times, so in the case of equation (44), if the healing degree is <0 In the case of the equation (45), it is determined that healing is achieved if the healing degree is <1. Furthermore, when the healing degree is more accurately detected, the healing degree <δ 1 detected by the equation (44), the healing degree detected by the equation (45), the healing degree <δ 2 is achieved It may be determined that Here, δ1 is an appropriate constant of negative value, and δ2 is an appropriate constant of 1 or less.

The image data organization system of the seventh embodiment is an image data organization system including an image data organization terminal device and an image data organization server device, and the image data organization terminal device captures an image to acquire image data. An imaging unit, a time-series heartbeat data acquisition unit for acquiring time-series heartbeat data according to a heartbeat when acquiring image data, and a time-series heartbeat data output processing unit for outputting time-series heartbeat data to an image data organizing server device And an image display unit for displaying the captured images arranged in descending order of degree of healing output from the image data organizing server device, the image data organizing server device comprising The degree of healing is determined according to the heart rate detection unit for determining the heart rate, and the negative value obtained by subtracting the heart rate in normal times previously determined from the heart rate Comprises a healed degree calculating unit. Here, the heart rate detection unit and the healing degree calculation unit constitute a healing degree detection unit.

  An image data organizing apparatus according to a seventh embodiment includes an imaging unit that captures an image to acquire image data, a time-series heartbeat data acquisition unit that acquires time-series heartbeat data according to a heartbeat when acquiring image data, and a time-series heartbeat A heart rate detection unit for obtaining a heart rate from the data, a healing degree operation unit for finding a degree of healing according to the magnitude of a negative value obtained by subtracting the heart rate in normal times previously obtained from the heart rate; The image processing apparatus may include a sorting processing unit which arranges the image data in descending order of degree, and an image display unit which displays the image data arranged in descending order of degree of healing.

  The image data organizing server apparatus program according to the seventh embodiment is an image data organizing server apparatus program to be executed by a computer of the image data organizing server apparatus operating in cooperation with the image data organizing terminal apparatus in the image data organizing system. Based on the heart rate detection unit for calculating the heart rate from the time-series heart rate data acquired by the data organizing terminal and the negative value obtained by subtracting the heart rate in normal times previously calculated from the heart rate, the healing degree is determined The computer is functioned to include a healing degree operation unit and a sorting processing unit that arranges the captured images in descending order of healing degree.

  The image data organizing apparatus program according to the seventh embodiment is an image data organizing apparatus program to be executed by a computer of the image data organizing apparatus, which includes an imaging unit for capturing and acquiring image data, and acquisition of image data according to the heart rate. The time-series heart rate data acquisition unit that acquires time-series heart rate data, the heart rate detection unit that determines the heart rate from the time-series heart rate data, and the size of a negative value obtained by subtracting the heart rate in normal times previously obtained from the heart rate In accordance with the above, the healing degree calculating unit for obtaining the healing degree, the sorting processing unit for arranging the image data in descending order of the healing degree, and the image display unit for displaying the image data arranged in descending order of the healing degree And causing the computer to function.

"8th Embodiment"
Embodiments combining the above-described embodiments are also feasible. Thus, subjective feelings may be used to calculate the degree of healing. For example, if the subjectivity is "highly healed" 100 points, if the subjectivity is "very healed" 80 points, if the subjectivity is "somewhat healed" 60 points, the subjectivity "can't say anything" If it was 40 points, if it was "does not heal very much", the value of each item is quantified like 20 points. Furthermore, you may combine the autonomic-nerves index of (1) Formula-(9) Formula mentioned above.

An embodiment combining some or all of the above-described embodiments is also feasible. For example, assuming that a, b, c and d are constants of positive values including 0, the degree of healing may be determined by the following equation (46). Here, in the case where the degree of healing is determined based on the heart rate at least directly or indirectly, any of a or b may not be 0, and the other three may be arbitrarily determined including 0. It is. When the value of each constant of a, b, c and d is 0, it means that the element which is 0 is not considered in calculation of the degree of healing .

Degree of healing = a × (degree of healing detected by autonomic nervous index) + b × (degree of healing detected from heart rate) + c × (degree of healing by subjectivity) + d × (detected by fractal analysis Degree of healing done) ... (46)

  Here, examples of the case of b = 0 are the first embodiment, the second embodiment, and the fourth to sixth embodiments. An example where b ≠ 0 is a seventh embodiment. The example in the case of c = 0 is the case in which the degree of healing by subjectivity is not considered, and the example in the case of c ≠ 0 is the case in which the degree of healing by subjectivity is considered, and is the eighth embodiment. An example in the case of d = 0 is the case in which the fractal analysis is not performed, and an example in the case of d ≠ 0 is the case in which the result of the fractal analysis is used. The example of the embodiment which combined all or part of each embodiment is not limited to this, and various embodiments are possible. Further, it is needless to say that the present invention is not limited to the embodiments, and extends to the same technical idea.

10 Image data rearranging system 11 smartphone 12 server 13 Internet 20 image data rearranging system 21 image data rearranging terminal device 22 server

Claims (10)

An image data organization system comprising an image data organization terminal device and an image data organization server device, comprising:
The image data organizing terminal device
An imaging unit that captures an image and acquires image data;
A time-series heart rate data acquisition unit that acquires time-series heart rate data according to a heart rate when acquiring the image data;
A time-series heartbeat data output processing unit that outputs the time-series heartbeat data to the image data organizing server device;
An image display unit for displaying the captured images arranged in descending order of healing degree, which are output from the image data organizing server device;
The image data organizing server device
A healing degree detection unit for determining the healing degree from the time-series heartbeat data;
Anda Soteingu processor arranging the captured image to the healed descending order of the degree detected by the healed detecting unit,
The healing degree detection unit
A Fourier transform processing unit that performs Fourier transform on the time-series heartbeat data;
A high frequency component / low frequency component detection processing unit for separating the high frequency component and the low frequency component of the signal subjected to the Fourier transform processing;
An autonomic nerve analysis engine processing unit for obtaining the healing degree by dividing at least the high frequency component by a reference high frequency component measured in advance ;
Or
The healing degree detection unit
A heart rate detection unit for determining a heart rate from the time-series heart rate data;
And a healing degree computing unit for finding the healing degree according to the magnitude of a negative value obtained by subtracting the heart rate in normal time, which is obtained in advance from the heart rate .
The operator is notified before imaging that the healing degree detected from the healing degree detection unit is higher than a predetermined arrangement standard.
Image data organization system. An image data organization system comprising an image data organization terminal device and an image data organization server device, comprising:
The image data organizing terminal device
An imaging unit that captures an image and acquires image data;
A time-series heart rate data acquisition unit that acquires time-series heart rate data according to a heart rate when acquiring the image data;
A time-series heartbeat data output processing unit that outputs the time-series heartbeat data to the image data organizing server device;
An image display unit for displaying the captured images arranged in descending order of healing degree, which are output from the image data organizing server device;
The image data organizing server device
A healing degree detection unit for determining the healing degree from the time-series heartbeat data;
Anda Soteingu processor arranging the captured image to the healed descending order of the degree detected by the healed detecting unit,
The healing degree detection unit
A Fourier transform processing unit that performs Fourier transform on the time-series heartbeat data;
A high frequency component / low frequency component detection processing unit for separating the high frequency component and the low frequency component of the signal subjected to the Fourier transform processing;
An autonomic nerve analysis engine processing unit for obtaining the healing degree by dividing at least the high frequency component by a reference high frequency component measured in advance;
Or
The healing degree detection unit
A heart rate detection unit for determining a heart rate from the time-series heart rate data;
And a healing degree computing unit for finding the healing degree according to the magnitude of a negative value obtained by subtracting the heart rate in normal time, which is obtained in advance from the heart rate.
The image is automatically captured when the healing degree detected by the healing degree detection unit is higher than a predetermined arrangement standard.
Image data organization system. The image data organizing server device
The image data organizing system according to claim 1 or 2, further comprising a fractal analysis processing unit that calculates a fractal dimension from the captured image. The image data organizing terminal device is a wearable computer,
The imaging unit is a wearable camera,
The time-series heartbeat data acquisition unit is a heartbeat sensor disposed on the skin near a human heart,
The image data organizing system according to claim 1 or 2, wherein the time-series heartbeat data output processing unit outputs the time-series heartbeat data to the image data organizing server device by interrupt processing every predetermined time. An imaging unit which obtains image data by capturing an image,
A time-series heart rate data acquisition unit that acquires time-series heart rate data according to a heart rate when acquiring the image data;
A healing degree detection unit for obtaining a healing degree from the time-series heartbeat data;
A sorting processing unit that arranges the captured images in descending order of the degree of healing detected by the degree of healing detection unit ;
An image display unit configured to display the captured images arranged in descending order of the degree of healing;
The healing degree detection unit
A Fourier transform processing unit that performs Fourier transform on the time-series heartbeat data;
A high frequency component / low frequency component detection processing unit for separating the high frequency component and the low frequency component of the signal subjected to the Fourier transform processing;
An autonomic nerve analysis engine processing unit for obtaining the healing degree by dividing at least the high frequency component by a reference high frequency component measured in advance ;
Or
The healing degree detection unit
A heart rate detection unit for determining a heart rate from the time-series heart rate data;
Depending on the magnitude of the negative value obtained by subtracting the heart rate at the time previously determined normal from the heart rate, determining the healed degree, possess the healed calculation unit, a
The operator is notified before imaging that the healing degree detected from the healing degree detection unit is higher than a predetermined arrangement standard.
Image data organization device. An imaging unit which obtains image data by capturing an image,
A time-series heart rate data acquisition unit that acquires time-series heart rate data according to a heart rate when acquiring the image data;
A healing degree detection unit for obtaining a healing degree from the time-series heartbeat data;
A sorting processing unit that arranges the captured images in descending order of the degree of healing detected by the degree of healing detection unit ;
An image display unit configured to display the captured images arranged in descending order of the degree of healing;
The healing degree detection unit
A Fourier transform processing unit that performs Fourier transform on the time-series heartbeat data;
A high frequency component / low frequency component detection processing unit for separating the high frequency component and the low frequency component of the signal subjected to the Fourier transform processing;
An autonomic nerve analysis engine processing unit for obtaining the healing degree by dividing at least the high frequency component by a reference high frequency component measured in advance ;
Or
The healing degree detection unit
A heart rate detection unit for determining a heart rate from the time-series heart rate data;
Depending on the magnitude of the negative value obtained by subtracting the heart rate at the time previously determined normal from the heart rate, determining the healed degree, possess the healed calculation unit, a
The image is automatically captured when the healing degree detected by the healing degree detection unit is higher than a predetermined arrangement standard.
Image data organization device. An image data organizing server apparatus program to be executed by a computer of an image data organizing server apparatus operating in cooperation with an image data organizing terminal apparatus that captures an image and acquires image data in the image data organizing system,
A healing degree detecting unit for obtaining a healing degree from time-series heartbeat data acquired by the image data organization terminal device ;
It said computer to function as and a Soteingu processor arranging the captured image to the healed descending order of the degree detected by the healed detecting unit,
The healing degree detection unit
A Fourier transform processing unit that performs Fourier transform on the time-series heartbeat data;
A high frequency component / low frequency component detection processing unit for separating the high frequency component and the low frequency component of the signal subjected to the Fourier transform processing;
Causing the computer to have at least an autonomic nerve analysis engine processing unit for dividing the high frequency component by a reference high frequency component measured in advance to obtain the healing degree ;
Or
The healing degree detection unit
A heart rate detection unit for determining a heart rate from the time-series heart rate data;
Depending on the magnitude of the negative value obtained by subtracting the heart rate in previously determined normal times from the heart rate, determining the healed degree causes the computer to function so as to have a healed calculation unit, a
The operator is notified before imaging that the healing degree detected from the healing degree detection unit is higher than a predetermined arrangement standard.
Image data consolidation server device program. An image data organizing server apparatus program to be executed by a computer of an image data organizing server apparatus operating in cooperation with an image data organizing terminal apparatus that captures an image and acquires image data in the image data organizing system,
A healing degree detecting unit for obtaining a healing degree from time-series heartbeat data acquired by the image data organization terminal device ;
It said computer to function as and a Soteingu processor arranging the captured image to the healed descending order of the degree detected by the healed detecting unit,
The healing degree detection unit
A Fourier transform processing unit that performs Fourier transform on the time-series heartbeat data;
A high frequency component / low frequency component detection processing unit for separating the high frequency component and the low frequency component of the signal subjected to the Fourier transform processing;
Causing the computer to have at least an autonomic nerve analysis engine processing unit for dividing the high frequency component by a reference high frequency component measured in advance to obtain the healing degree ;
Or
The healing degree detection unit
A heart rate detection unit for determining a heart rate from the time-series heart rate data;
Depending on the magnitude of the negative value obtained by subtracting the heart rate in previously determined normal times from the heart rate, determining the healed degree causes the computer to function so as to have a healed calculation unit, a
The image is automatically captured when the healing degree detected by the healing degree detection unit is higher than a predetermined arrangement standard.
Image data consolidation server device program. An image data organizing apparatus program to be executed by a computer of the image data organizing apparatus,
Time series heartbeat data acquisition unit that acquires series heartbeat data when corresponding to the heartbeat on the occasion to obtain the image data by capturing an image,
A healing degree detection unit for obtaining a healing degree from the time-series heartbeat data;
A sorting processing unit that arranges the captured images in descending order of the degree of healing detected by the degree of healing detection unit ;
An image display unit configured to display the captured images arranged in descending order of the degree of healing, and causing the computer to function .
The healing degree detection unit
A Fourier transform processing unit that performs Fourier transform on the time-series heartbeat data;
A high frequency component / low frequency component detection processing unit for separating the high frequency component and the low frequency component of the signal subjected to the Fourier transform processing;
Causing the computer to have at least an autonomic nerve analysis engine processing unit for dividing the high frequency component by a reference high frequency component measured in advance to obtain the healing degree ;
Or
The healing degree detection unit
A heart rate detection unit for determining a heart rate from the time-series heart rate data;
Causing the computer to have a healing degree calculating unit for determining the healing degree according to the magnitude of a negative value obtained by subtracting the heart rate in normal time previously calculated from the heart rate ;
The operator is notified before imaging that the healing degree detected from the healing degree detection unit is higher than a predetermined arrangement standard.
Image data organization program. An image data organizing apparatus program to be executed by a computer of the image data organizing apparatus,
Time series heartbeat data acquisition unit that acquires series heartbeat data when corresponding to the heartbeat on the occasion to obtain the image data by capturing an image,
A healing degree detection unit for obtaining a healing degree from the time-series heartbeat data;
A sorting processing unit that arranges the captured images in descending order of the degree of healing detected by the degree of healing detection unit ;
An image display unit configured to display the captured images arranged in descending order of the degree of healing, and causing the computer to function .
The healing degree detection unit
A Fourier transform processing unit that performs Fourier transform on the time-series heartbeat data;
A high frequency component / low frequency component detection processing unit for separating the high frequency component and the low frequency component of the signal subjected to the Fourier transform processing;
Causing the computer to have at least an autonomic nerve analysis engine processing unit for dividing the high frequency component by a reference high frequency component measured in advance to obtain the healing degree ;
Or
The healing degree detection unit
A heart rate detection unit for determining a heart rate from the time-series heart rate data;
Causing the computer to have a healing degree calculating unit for determining the healing degree according to the magnitude of a negative value obtained by subtracting the heart rate in normal time previously calculated from the heart rate ;
The image is automatically captured when the healing degree detected by the healing degree detection unit is higher than a predetermined arrangement standard.
Image data organization program.