Deprecated: The each() function is deprecated. This message will be suppressed on further calls in /home/zhenxiangba/zhenxiangba.com/public_html/phproxy-improved-master/index.php on line 456
CN112101094A - Suicide risk assessment method based on body language - Google Patents
[go: Go Back, main page]

CN112101094A - Suicide risk assessment method based on body language - Google Patents

Suicide risk assessment method based on body language Download PDF

Info

Publication number
CN112101094A
CN112101094A CN202010764397.7A CN202010764397A CN112101094A CN 112101094 A CN112101094 A CN 112101094A CN 202010764397 A CN202010764397 A CN 202010764397A CN 112101094 A CN112101094 A CN 112101094A
Authority
CN
China
Prior art keywords
layer
input
output
neural network
matrix
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010764397.7A
Other languages
Chinese (zh)
Other versions
CN112101094B (en
Inventor
杜广龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
South China University of Technology SCUT
Original Assignee
South China University of Technology SCUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South China University of Technology SCUT filed Critical South China University of Technology SCUT
Priority to CN202010764397.7A priority Critical patent/CN112101094B/en
Publication of CN112101094A publication Critical patent/CN112101094A/en
Application granted granted Critical
Publication of CN112101094B publication Critical patent/CN112101094B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Human Computer Interaction (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Image Analysis (AREA)

Abstract

本发明公开了一种基于肢体语言的自杀风险评估方法。所述方法包括以下步骤：使用kinect捕获视频采集人员的肢体行为数据，作为输入数据；将输入数据划分为静态的姿势和动态的动作，分别使用卷积神经网络来提取静态姿势的特征和双向长短期记忆神经网络来提取动态动作的特征；使用一个长短期记忆神经网络来有效地融合静态姿势的特征和动态动作的特征，统一到相同的特征空间；通过softmax层输出该人员是否有自杀倾向。本发明同时使用人们的静态姿势特征和动态动作特征，可以有效地提高情感识别的准确度。本发明将人们的动静态特征进行了嵌入，嵌入到统一的特征空间，保证特征之间能够得到更有效、协调的使用。

Figure 202010764397

The invention discloses a suicide risk assessment method based on body language. The method includes the following steps: using kinect to capture the limb behavior data of video collection personnel as input data; dividing the input data into static postures and dynamic actions, and using convolutional neural networks to extract the features of static postures and bidirectional lengths respectively. A short-term memory neural network is used to extract the features of dynamic actions; a long-term and short-term memory neural network is used to effectively fuse the features of static poses and the features of dynamic actions, and unify them into the same feature space; output whether the person has suicidal tendencies through the softmax layer. The present invention simultaneously uses people's static posture features and dynamic action features, which can effectively improve the accuracy of emotion recognition. The invention embeds the dynamic and static features of people into a unified feature space, so as to ensure that the features can be used more effectively and harmoniously.

Figure 202010764397

Description

Suicide risk assessment method based on body language
Technical Field
The invention belongs to the field of emotion recognition, and particularly relates to a suicide risk assessment method based on body language.
Background
The prisoner is easy to hurt himself or others because of the poor social adaptability and unstable psychological state. In order to prevent criminals from self-mutualism or crimes, it is necessary to detect their emotional tendencies. Human emotional tendencies can be predicted in a number of ways, such as electrocardiography, electroencephalography (university of south China school, Nature science, version 2019(5)), speech (computer science, 2015(09):24-28), facial expressions (university of Western Ann university, 2015,35(009): 705-. Among the various emotional signals, physiological signals are widely used for emotion recognition. Recently, limb movements have also become a new feature.
Many models for emotional tendency prediction have been proposed so far, but many problems still need to be solved or deserve further research. Technically, audio data is readily available, but is susceptible to noise. Computational complexity, a single detection of a static gesture or a single detection of a dynamic action to identify an emotion is less complex, but also results in less accuracy of identification. Therefore, there is a need to improve the accuracy of emotion recognition by fusing multiple actions. The emotion of people can be more effectively recognized through the unified static posture and dynamic action, the suicide tendency can be better predicted, and the suicide risk is reduced. For people with high suicide risk, the actions such as cutting veins and lifting are often performed, the action detection is performed more accurately through the posture and action characteristics of dynamic and static combination, the suicide tendency of people can be effectively predicted, and the suicide risk of people is reduced.
Disclosure of Invention
The invention aims to solve the defects of the prior art and provides a new neural network framework, wherein a convolutional neural network and a bidirectional long-short term memory neural network are used for respectively distinguishing static posture characteristics and dynamic action characteristics of personnel, so that the calculation complexity is reduced; and in addition, a unidirectional long-term and short-term memory neural network is embedded into a uniform feature space, so that dynamic and static features are effectively fused. And finally, outputting a final suicide tendency classification result through a softmax function.
The purpose of the invention is realized by at least one of the following technical solutions.
A suicide risk assessment method based on a body language comprises the following steps:
s1, capturing the limb behavior data of the video acquisition personnel by using kinect as input data;
s2, dividing the input data into static postures and dynamic actions, and respectively extracting the characteristics of the static postures and the characteristics of the dynamic actions by using a convolutional neural network and a bidirectional long-short term memory neural network;
s3, a long-short term memory neural network is used for effectively fusing the characteristics of static postures and the characteristics of dynamic actions to be unified to the same characteristic space;
and S4, outputting whether the person has suicide tendency or not through the softmax layer.
Further, in step S2, the static gesture recognition is specifically as follows:
inputting frames in a video into a Convolutional Neural Network (CNN) for training and testing, inputting the frames into video frames, and outputting the video frames into static posture categories corresponding to people in the video frames, wherein the static posture categories comprise right arm uplift, right shoulder uplift, head right inclination, body right inclination, left arm uplift, left shoulder uplift, head left inclination, body straightness and two-hand crossing; CNN uses partial filters to compute convolution, i.e. inner product operation using local submatrices of the entries and local filters.
Further, the Convolutional Neural Network (CNN) includes an input layer, 2 convolutional layers, 2 pooling layers, and a full-link layer, where each pooling layer is a next layer of each convolutional layer;
the input of the input layer is represented as
Figure BDA0002614111560000021
xmnRepresents the mth row and the nth column in the input matrix;
the formula for the convolutional layer is as follows:
Figure BDA0002614111560000022
wherein l represents the l-th convolutionLayer, i represents the ith component of the convolution output matrix; j represents the jth output matrix; j is in the range of 0, N]Where N represents the number of convolution output matrices;
Figure BDA0002614111560000023
a value representing the ith component of the jth output matrix of the ith convolutional layer;
Figure BDA0002614111560000024
representing the value of the ith component of the kth input matrix of the l convolutional layer, bjDenotes the offset of the jth output matrix, m denotes the number of convolution kernels, a denotes the a-th convolution kernel,
Figure BDA0002614111560000025
representing the weight of the kth matrix of the a-th convolution kernel; f is a non-linear sigmoid-type function;
in pooling layers, mean pooling is used, with the input to each mean pooling layer coming from the immediately preceding convolutional layer and the output as the input to the next convolutional layer, and the calculation formula is as follows:
Figure BDA0002614111560000026
wherein ,
Figure BDA0002614111560000027
represents the local output after the pooling process of the pooling layer under the first convolutional layer is finished,
Figure BDA0002614111560000028
is the ith component of the jth output matrix of the convolutional layer;
the calculation formula of the full connection layer is as follows:
Figure BDA0002614111560000029
wherein ,frOutput, ω, representing the r-th component of the fully-connected layerrRepresenting the r-th matrixThe weight value of the weight is calculated,
Figure BDA00026141115600000210
representing the ith component of the 2 nd pooling layer's ith output matrix.
Further, in step S2, the dynamic motion recognition specifically includes the following steps:
executing human body positioning and tracking through an application program interface built in the Kinect; when a human body enters a working space of the robot, the human body is detected through the characteristics of the 3D data, and a human body skeleton is calculated through an API (application programming interface) to obtain skeleton points;
numbering 15 skeleton points from top to bottom and from left to right; coordinates of the 15 skeleton joint points are coordinates of Kinect; detecting human body skeleton point data in real time by using the Kinect, and calculating the behavior characteristics of the human body by using the skeleton point data;
preprocessing the skeleton point data: the acquisition process limits the matrix size to 15 × 3, where 15 denotes the number of skeleton points and 3 denotes the three-dimensional coordinates of each skeleton point, i.e., p ═ p (p ═ p)x,py,pz) Representing the coordinates of the skeleton points in the x, y and z axes, respectively; extracting the motion sequence characteristics of a skeleton point through a Recurrent Neural Network (RNN), and encoding the skeleton point sequence corresponding to continuous actions, namely N set actions into a vector defined as { h }NProcessing { h } by using two-way long-short term memory neural network with conditional random fieldN}。
Further, the recurrent neural network includes an input layer, a hidden layer, and an output layer, specifically as follows:
the input of the input layer is denoted as h0＝{p1,p2,...pT}, wherein ptDenotes the T-th coordinate, and T represents the total number of coordinates.
The calculation formula of the hidden layer is as follows:
st＝f(Wxhh0+Whhst-1)； (11)
wherein ,stW being the state of the hidden layer at time txhAs a weight matrix from the input layer to the hidden layer, WhhFrom a hidden layer to a hidden layerHiding a weight matrix of the layer, wherein f is an activation function;
the calculation formula of the output layer is as follows:
ht＝g(V·st)； (12)
wherein ,htFor output at time t, g is the activation function and V is the weight matrix.
Finally, for the Nth time, the output h is output0To hNSequentially constructing a sequence to obtain
{hN}＝{h0,h1,...,hN}。
Further, given an input sequence hN}＝{h0,h1,...,hNThe output calculation formula of the hidden layer of the bidirectional long and short term memory neural network is as follows:
lt＝σh(Wxhht+Whhlt-1+bh)； (13)
wherein ,ltFor the output of the hidden layer at time t, WxhAs a weight matrix from the input layer to the hidden layer, WhhAs a weight matrix from hidden layer to hidden layer, bhFor concealing the bias of the layer, σhRepresenting an activation function; the bidirectional long-short term memory neural network hidden layer (Bi-LSTM) is used for strengthening bilateral relation, and the first layer is forward LSTM, and the second layer is backward LSTM.
Further, in step S3, the long-short term memory neural network is used to effectively embed into the unified feature space, which is as follows:
using xsm and xdmTo represent the extracted static gesture features and dynamic motion features separately and directly connect them as xc＝(xsm,xdm)，xcIs a persistent feature vector; then x is putcEmbedded as x via an LSTMeThe LSTM uses the forward LSTM of Bi-LSTM extracted dynamic motion features in step S2; after the embedding process is finished, for xc and xePerforming element-by-element multiplication to obtain the cross effect of two vectors; after multiplication, cross effect characteristics are measuredAnd xeConnect again to add the original information back; x needs a normalized self-organizing map to
Figure BDA0002614111560000041
The normalized x size is N × 1.
Further, in step S4, the final result is classified by using the softmax function, and the calculation formula is as follows:
Figure BDA0002614111560000042
wherein ,WeA weight matrix of the e-th emotional tendency is adopted, and b represents the bias of the weight matrix; the emotional tendency categories are suicidal emotional tendency and no suicidal emotional tendency, respectively.
Compared with the prior art, the invention has the following beneficial effects:
(1) the invention uses the kinect with the infrared camera to collect data, and can effectively prevent the image from being influenced by illumination.
(2) The invention simultaneously uses the static posture characteristic and the dynamic action characteristic of people, and can effectively improve the accuracy of emotion recognition.
(3) The invention embeds the dynamic and static characteristics of people into a uniform characteristic space, and ensures that the characteristics can be used more effectively and coordinately.
Drawings
FIG. 1 is a flowchart of a suicide risk assessment method based on body language according to the present invention.
Fig. 2 is a diagram of a human body action feature extraction process in the present invention.
Detailed Description
Specific implementations of the present invention will be further described with reference to the following examples and drawings, but the embodiments of the present invention are not limited thereto.
Example (b):
a suicide risk assessment method based on body language, as shown in fig. 1, includes the following steps:
s1, in the embodiment, a kinect placed in a range of 5 meters is used for capturing the body behavior data of the person staying in the room, and the body behavior data is used as input data;
s2, as shown in fig. 2, dividing the input data into static posture and dynamic action, and extracting the features of the static posture and the features of the dynamic action by using a convolutional neural network and a bidirectional long-short term memory neural network, respectively;
the static gesture recognition is specifically as follows:
inputting frames in a video into a Convolutional Neural Network (CNN) for training and testing, inputting the frames into video frames, and outputting the video frames into static posture categories corresponding to people in the video frames, wherein the static posture categories comprise right arm uplift, right shoulder uplift, head right inclination, body right inclination, left arm uplift, left shoulder uplift, head left inclination, body straightness and two-hand crossing; CNN uses partial filters to compute convolution, i.e. inner product operation using local submatrices of the entries and local filters.
The Convolutional Neural Network (CNN) comprises an input layer, 2 convolutional layers, 2 pooling layers and a full-connection layer, wherein each pooling layer is the next layer of each convolutional layer;
the input of the input layer is represented as
Figure BDA0002614111560000051
xmnRepresents the mth row and the nth column in the input matrix;
the formula for the convolutional layer is as follows:
Figure BDA0002614111560000052
wherein l represents the l convolutional layer, and i represents the i component of the convolutional output matrix; j represents the jth output matrix; j is in the range of 0, N]Where N represents the number of convolution output matrices;
Figure BDA0002614111560000053
to representThe value of the ith component of the jth output matrix of the jth convolutional layer;
Figure BDA0002614111560000054
representing the value of the ith component of the kth input matrix of the l convolutional layer, bjDenotes the offset of the jth output matrix, m denotes the number of convolution kernels, a denotes the a-th convolution kernel,
Figure BDA0002614111560000055
representing the weight of the kth matrix of the a-th convolution kernel; f is a non-linear sigmoid-type function;
in pooling layers, mean pooling is used, with the input to each mean pooling layer coming from the immediately preceding convolutional layer and the output as the input to the next convolutional layer, and the calculation formula is as follows:
Figure BDA0002614111560000056
wherein ,
Figure BDA0002614111560000057
represents the local output after the pooling process of the pooling layer under the first convolutional layer is finished,
Figure BDA0002614111560000058
is the ith component of the jth output matrix of the convolutional layer;
the calculation formula of the full connection layer is as follows:
Figure BDA0002614111560000059
wherein ,frOutput, ω, representing the r-th component of the fully-connected layerrThe weight values of the r-th matrix are represented,
Figure BDA00026141115600000510
representing the ith component of the 2 nd pooling layer's ith output matrix.
The dynamic action recognition is specifically as follows:
executing human body positioning and tracking through an application program interface built in the Kinect; when a human body enters a working space of the robot, the human body is detected through the characteristics of the 3D data, and a human body skeleton is calculated through an API (application programming interface) to obtain skeleton points;
numbering 15 skeleton points from top to bottom and from left to right; coordinates of the 15 skeleton joint points are coordinates of Kinect; detecting human body skeleton point data in real time by using the Kinect, and calculating the behavior characteristics of the human body by using the skeleton point data;
preprocessing the skeleton point data: the acquisition process limits the matrix size to 15 × 3, where 15 denotes the number of skeleton points and 3 denotes the three-dimensional coordinates of each skeleton point, i.e., p ═ p (p ═ p)x,py,pz) Representing the coordinates of the skeleton points in the x, y and z axes, respectively; extracting the motion sequence characteristics of a skeleton point through a Recurrent Neural Network (RNN), and encoding the skeleton point sequence corresponding to continuous actions, namely N set actions into a vector defined as { h }N}; the recurrent neural network comprises an input layer, a hidden layer and an output layer, and specifically comprises the following steps:
the input of the input layer is denoted as h0＝{p1,p2,...pT}, wherein ptDenotes the T-th coordinate, and T represents the total number of coordinates.
The calculation formula of the hidden layer is as follows:
st＝f(Wxhh0+Whhst-1)； (18)
wherein ,stW being the state of the hidden layer at time txhAs a weight matrix from the input layer to the hidden layer, WhhF is a weight matrix from the hidden layer to the hidden layer, and is an activation function;
the calculation formula of the output layer is as follows:
ht＝g(V·st)； (19)
wherein ,htFor output at time t, g is the activation function and V is the weight matrix.
Finally, for the Nth time, the output h is output0To hNBuild into order in orderIs listed to obtain
{hN}＝{h0,h1,...,hN}。
Processing of h Using a two-way Long-short term memory neural network with conditional random fieldsNGiven an input sequence hN}＝{h0,h1,...,hNThe output calculation formula of the hidden layer of the bidirectional long and short term memory neural network is as follows:
lt＝σh(Wxhht+Whhlt-1+bh)； (20)
wherein ,ltFor the output of the hidden layer at time t, WxhAs a weight matrix from the input layer to the hidden layer, WhhAs a weight matrix from hidden layer to hidden layer, bhFor concealing the bias of the layer, σhRepresenting an activation function; the bidirectional long-short term memory neural network hidden layer (Bi-LSTM) is used for strengthening bilateral relation, and the first layer is forward LSTM, and the second layer is backward LSTM.
S3, a long-short term memory neural network is used for effectively fusing the characteristics of static postures and the characteristics of dynamic actions, and the characteristics are unified to the same characteristic space, and the details are as follows:
using xsm and xdmTo represent the extracted static gesture features and dynamic motion features separately and directly connect them as xc＝(xsm,xdm)，xcIs a persistent feature vector; then x is putcEmbedded as x via an LSTMeThe LSTM uses the forward LSTM of Bi-LSTM extracted dynamic motion features in step S2; after the embedding process is finished, for xc and xePerforming element-by-element multiplication to obtain the cross effect of two vectors; after multiplication, the cross-effect characteristic is summed with xeConnect again to add the original information back; x needs a normalized self-organizing map to
Figure BDA0002614111560000071
The normalized x size is N × 1.
S4, outputting whether the person has suicide tendency through a softmax layer, wherein the calculation formula is as follows:
Figure BDA0002614111560000072
wherein ,WeA weight matrix of the e-th emotional tendency is adopted, and b represents the bias of the weight matrix; the emotional tendency categories are suicidal emotional tendency and no suicidal emotional tendency, respectively.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can substitute or change the technical solution of the present invention and the inventive concept within the scope of the present invention disclosed by the present invention.

Claims (8)

1.一种基于肢体语言的自杀风险评估方法，其特征在于，包括以下步骤：1. a suicide risk assessment method based on body language, is characterized in that, comprises the following steps: S1、使用kinect捕获视频采集人员的肢体行为数据，作为输入数据；S1. Use kinect to capture the physical behavior data of the video collection personnel as input data; S2、将输入数据划分为静态的姿势和动态的动作，分别使用卷积神经网络来提取静态姿势的特征和双向长短期记忆神经网络来提取动态动作的特征；S2. Divide the input data into static poses and dynamic actions, and use a convolutional neural network to extract the features of static poses and a bidirectional long short-term memory neural network to extract the features of dynamic actions; S3、使用一个长短期记忆神经网络来有效地融合静态姿势的特征和动态动作的特征，统一到相同的特征空间；S3. Use a long short-term memory neural network to effectively fuse the features of static poses and dynamic actions into the same feature space; S4、通过softmax层输出该人员是否有自杀倾向。S4. Output whether the person has suicidal tendencies through the softmax layer. 2.根据权利要求1所述的一种基于肢体语言的自杀风险评估方法，其特征在于，步骤S2中，静态的姿势识别具体如下：2. a kind of suicide risk assessment method based on body language according to claim 1, is characterized in that, in step S2, static posture recognition is specifically as follows: 将视频中的帧输入到卷积神经网络(CNN)中进行训练和测试，输入为视频帧，输出为该视频帧中人员所对应静态姿势类别，包括右臂高举、右肩高耸、头向右倾斜、身体右倾、左臂高举、左肩高耸、头向左倾斜、身体左倾、身体正直和双手交叉；CNN利用部分滤波器计算卷积，即利用输入项的局部子矩阵和局部滤波器进行内积运算。Input the frames in the video into a convolutional neural network (CNN) for training and testing, the input is a video frame, and the output is the static pose category corresponding to the person in the video frame, including the right arm raised, the right shoulder raised, and the head turned to the right. Leaning, body leaning right, left arm raised, left shoulder high, head tilting to the left, body leaning left, body upright and hands crossed; CNN uses partial filters to calculate convolution, that is, uses the local submatrix of the input item and the local filter for inner product operation. 3.根据权利要求2所述的一种基于肢体语言的自杀风险评估方法，其特征在于，所述卷积神经网络(CNN)包括输入层、2个卷积层、2个池化层、全连接层，其中，每个池化层均为每个卷积层的下一层；3. a kind of suicide risk assessment method based on body language according to claim 2, is characterized in that, described convolutional neural network (CNN) comprises input layer, 2 convolution layers, 2 pooling layers, full connection layers, where each pooling layer is the next layer of each convolutional layer; 输入层的输入表示为
Figure FDA0002614111550000011
xmn表示输入矩阵中的第m行第n列；The input to the input layer is represented as
Figure FDA0002614111550000011
x mn represents the mth row and nth column in the input matrix; 卷积层的计算公式如下：The calculation formula of the convolutional layer is as follows:
Figure FDA0002614111550000012
Figure FDA0002614111550000012
 其中，l表示第l个卷积层，i表示卷积输出矩阵第i个分量；j表示第j个输出矩阵；j∈[0,N]，其中N表示卷积输出矩阵的数量；
Figure FDA0002614111550000013
表示第l个卷积层第j个输出矩阵第i个分量的值；
Figure FDA0002614111550000014
表示第l个卷积层第k个输入矩阵第i个分量的值，bj表示第j个输出矩阵的偏置，m表示卷积核数量，a表示第a个卷积核，
Figure FDA0002614111550000015
表示第a个卷积核第k个矩阵的权值；f是一个非线性sigmoid型函数；Among them, l represents the lth convolutional layer, i represents the ith component of the convolution output matrix; j represents the jth output matrix; j∈[0,N], where N represents the number of convolution output matrices;
Figure FDA0002614111550000013
Represents the value of the i-th component of the j-th output matrix of the l-th convolutional layer;
Figure FDA0002614111550000014
represents the value of the i-th component of the k-th input matrix of the l-th convolutional layer, b j represents the offset of the j-th output matrix, m represents the number of convolution kernels, a represents the a-th convolution kernel,
Figure FDA0002614111550000015
Represents the weight of the k-th matrix of the a-th convolution kernel; f is a nonlinear sigmoid function; 池化层中，使用均值池化，每个均值池化层的输入来自于紧跟的上卷积层，输出作为下一个卷积层的输入，计算公式如下：In the pooling layer, mean pooling is used. The input of each mean pooling layer comes from the next convolutional layer, and the output is used as the input of the next convolutional layer. The calculation formula is as follows:
Figure FDA0002614111550000021
Figure FDA0002614111550000021
 其中，
Figure FDA0002614111550000022
代表第l个卷积层下的池化层池化过程结束后的局部输出，
Figure FDA0002614111550000023
为卷积层第j个输出矩阵的第i个分量；in,
Figure FDA0002614111550000022
represents the local output after the pooling process of the pooling layer under the lth convolutional layer,
Figure FDA0002614111550000023
is the ith component of the jth output matrix of the convolutional layer; 全连接层的计算公式如下：The calculation formula of the fully connected layer is as follows:
Figure FDA0002614111550000024
Figure FDA0002614111550000024
 其中，fr表示全连接层第r个分量的输出，ωr表示第r个矩阵的权值，
Figure FDA0002614111550000025
表示第2个池化层第r个输出矩阵的第i个分量。Among them, f r represents the output of the r-th component of the fully connected layer, ω r represents the weight of the r-th matrix,
Figure FDA0002614111550000025
represents the ith component of the rth output matrix of the 2nd pooling layer. 4.根据权利要求1所述的一种基于肢体语言的自杀风险评估方法，其特征在于，步骤S2中，动态的动作识别具体如下：4. a kind of suicide risk assessment method based on body language according to claim 1, is characterized in that, in step S2, dynamic action recognition is specifically as follows: 通过内置在Kinect中的应用程序接口执行人体定位和跟踪；当人体进入机器人工作空间时，通过3D数据的特征检测出人体，通过API计算出人体骨架，得到骨架点；Perform human body positioning and tracking through the application program interface built in Kinect; when the human body enters the robot workspace, the human body is detected by the features of the 3D data, the human body skeleton is calculated through the API, and the skeleton point is obtained; 将15个骨架点从上到下、从左到右进行编号；15个骨架关节点的坐标为Kinect的坐标；利用Kinect实时检测人体骨架点数据，并利用骨架点数据计算人体的行为特征；Number the 15 skeleton points from top to bottom and from left to right; the coordinates of the 15 skeleton joint points are the coordinates of the Kinect; use the Kinect to detect the human body skeleton point data in real time, and use the skeleton point data to calculate the behavior characteristics of the human body; 对骨架点数据进行预处理：获取过程将矩阵大小限制为15×3，其中15表示骨架点的数量，3表示每个骨架点的三维坐标，即p＝(px,py,pz)，分别代表在x，y，z轴的骨架点坐标；通过循环神经网络(RNN)提取一个骨架点的运动序列特征，将连续动作即N个集合动作对应的骨架点序列编码为向量，定义为{hN},并使用带有条件随机场的双向长短期记忆神经网络处理{hN}。Preprocessing the skeleton point data: The acquisition process limits the matrix size to 15×3, where 15 represents the number of skeleton points and 3 represents the three-dimensional coordinates of each skeleton point, i.e. p=(p x , p y , p z ) , respectively represent the coordinates of the skeleton point on the x, y, and z axes; the motion sequence feature of a skeleton point is extracted through a cyclic neural network (RNN), and the continuous action, that is, the skeleton point sequence corresponding to the N set actions, is encoded as a vector, which is defined as {h N }, and processing {h N } using a bidirectional long short-term memory neural network with conditional random fields. 5.根据权利要求4所述的一种基于肢体语言的自杀风险评估方法，其特征在于，所述的循环神经网络包括输入层、隐藏层和输出层，具体如下：5. a kind of suicide risk assessment method based on body language according to claim 4 is characterized in that, described recurrent neural network comprises input layer, hidden layer and output layer, is specifically as follows: 输入层的输入表示为h0＝{p1,p2,...pT}，其中pt表示第t个坐标，T代表坐标总数；The input of the input layer is expressed as h 0 ={p 1 ,p 2 ,...p T }, where p t represents the t-th coordinate, and T represents the total number of coordinates; 隐藏层的计算公式如下：The calculation formula of the hidden layer is as follows: st＝f(Wxhh0+Whhst-1)； (4)s t =f(W xh h 0 +W hh s t-1 ); (4) 其中，st为隐含层在t时刻的状态，Wxh为输入层到隐藏层的权值矩阵，Whh为从隐藏层到隐藏层的权值矩阵，f为激活函数；Among them, s t is the state of the hidden layer at time t, W xh is the weight matrix from the input layer to the hidden layer, W hh is the weight matrix from the hidden layer to the hidden layer, and f is the activation function; 输出层的计算公式如下：The calculation formula of the output layer is as follows: ht＝g(V·st)； (5)h t =g(V·s t ); (5) 其中，ht为在t时刻的输出，g为激活函数，V为权值矩阵；Among them, h t is the output at time t, g is the activation function, and V is the weight matrix; 最后，对第N时刻，将输出从h0到hN按顺序构建成序列，得到{hN}＝{h0,h1,...,hN}。Finally, for the Nth time, the outputs are sequentially constructed from h 0 to h N into a sequence to obtain {h N }={h 0 , h 1 , . . . , h N }. 6.根据权利要求4所述的一种基于肢体语言的自杀风险评估方法，其特征在于，给定一个输入序列{hN}＝{h0,h1,...,hN}，双向长短期记忆神经网络隐藏层的输出计算公式如下：6 . The method for assessing suicide risk based on body language according to claim 4 , wherein, given an input sequence {h N }={h 0 , h 1 , . . . , h N }, two-way The output calculation formula of the hidden layer of the long short-term memory neural network is as follows: lt＝σh(Wxhht+Whhlt-1+bh)； (6)l t =σ h (W xh h t +W hh l t-1 +b h ); (6) 其中，lt为隐含层在t时刻的输出，Wxh为输入层到隐藏层的权值矩阵，Whh为从隐藏层到隐藏层的权值矩阵，bh为隐藏层的偏置，σh表示激活函数；双向长短期记忆神经网络隐藏层(Bi-LSTM)用来加强双边关系，并使第一层是前向LSTM，第二层是后向LSTM。Among them, l t is the output of the hidden layer at time t, W xh is the weight matrix from the input layer to the hidden layer, W hh is the weight matrix from the hidden layer to the hidden layer, b h is the bias of the hidden layer, σ h denotes the activation function; the bidirectional long short-term memory neural network hidden layer (Bi-LSTM) is used to strengthen the bilateral relationship and make the first layer a forward LSTM and the second layer a backward LSTM. 7.根据权利要求1所述的一种基于肢体语言的自杀风险评估方法，其特征在于，步骤S3中，所述使用一个长短期记忆神经网络来有效地嵌入到统一特征空间，具体如下：7. a kind of suicide risk assessment method based on body language according to claim 1, is characterized in that, in step S3, described using a long short-term memory neural network to be embedded in unified feature space effectively, is specifically as follows: 使用xsm和xdm来分别表示提取的静态姿势特征和动态动作特征，并将其直接连接为xc＝(xsm,xdm)，xc是一个持久特征向量；然后将xc经过一个LSTM来嵌入为xe，该LSTM使用步骤S2中提取动态动作特征的Bi-LSTM的前向LSTM；嵌入过程结束后，对xc和xe进行逐元素乘运算，得到两个向量的交叉效应；相乘之后，将交叉效应特性和xe再次连接起来，以将原始信息添加回去；在输入下一个神经网络层之前，x需要规范化的自组织映射为
Figure FDA0002614111550000031
归一化后的x的大小是N×1。Use x sm and x dm to represent the extracted static pose features and dynamic action features, respectively, and connect them directly as x c = (x sm , x dm ), where x c is a persistent feature vector; then pass x c through a LSTM is embedded as x e , which uses the forward LSTM of Bi-LSTM that extracts dynamic action features in step S2; after the embedding process, element-wise multiplication of x c and x e is performed to obtain the cross effect of the two vectors ; After multiplication, the cross-effects feature and x e are connected again to add the original information back; before feeding into the next neural network layer, x needs to be normalized to a self-organizing map of
Figure FDA0002614111550000031
The size of the normalized x is N×1. 8.根据权利要求1所述的一种基于肢体语言的自杀风险评估方法，其特征在于，步骤S4中，所述使用softmax函数分类最终的结果，计算公式如下：8. a kind of suicide risk assessment method based on body language according to claim 1, is characterized in that, in step S4, described use softmax function classification final result, calculation formula is as follows:
Figure FDA0002614111550000032
Figure FDA0002614111550000032
 其中，We为第e类情感倾向的权重矩阵，b代表权重矩阵的偏置；情感倾向类别分别为具有自杀情感倾向和没有自杀情感倾向。Among them, We is the weight matrix of the e -th type of emotional tendency, and b represents the bias of the weight matrix; the categories of emotional tendency are suicidal emotional tendency and no suicidal emotional tendency, respectively.
CN202010764397.7A 2020-08-02 2020-08-02 A body language-based suicide risk assessment method Expired - Fee Related CN112101094B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010764397.7A CN112101094B (en) 2020-08-02 2020-08-02 A body language-based suicide risk assessment method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010764397.7A CN112101094B (en) 2020-08-02 2020-08-02 A body language-based suicide risk assessment method

Publications (2)

Publication Number Publication Date
CN112101094A true CN112101094A (en) 2020-12-18
CN112101094B CN112101094B (en) 2023-08-22

Family

ID=73750478

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010764397.7A Expired - Fee Related CN112101094B (en) 2020-08-02 2020-08-02 A body language-based suicide risk assessment method

Country Status (1)

Country Link
CN (1) CN112101094B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113283978A (en) * 2021-05-06 2021-08-20 北京思图场景数据科技服务有限公司 Financial risk assessment method based on biological basis, behavior characteristics and business characteristics
CN118068820A (en) * 2024-04-19 2024-05-24 四川航天电液控制有限公司 An intelligent fault diagnosis method for hydraulic support controller
US12106531B2 (en) 2021-07-22 2024-10-01 Microsoft Technology Licensing, Llc Focused computer detection of objects in images

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107145524A (en) * 2017-04-12 2017-09-08 清华大学 Suicide risk checking method and system based on microblogging and Fuzzy Cognitive Map
US10043591B1 (en) * 2015-02-06 2018-08-07 Brain Trust Innovations I, Llc System, server and method for preventing suicide
CN109165685A (en) * 2018-08-21 2019-01-08 南京邮电大学 Prison prisoner potentiality risk monitoring method and system based on expression and movement

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10043591B1 (en) * 2015-02-06 2018-08-07 Brain Trust Innovations I, Llc System, server and method for preventing suicide
CN107145524A (en) * 2017-04-12 2017-09-08 清华大学 Suicide risk checking method and system based on microblogging and Fuzzy Cognitive Map
CN109165685A (en) * 2018-08-21 2019-01-08 南京邮电大学 Prison prisoner potentiality risk monitoring method and system based on expression and movement

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
袁?;喻思红;杨冰香;黄智生;阮娟;: "人工智能技术在抑郁及自杀管理中的应用", 中国全科医学, no. 26, pages 12 - 16 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113283978A (en) * 2021-05-06 2021-08-20 北京思图场景数据科技服务有限公司 Financial risk assessment method based on biological basis, behavior characteristics and business characteristics
CN113283978B (en) * 2021-05-06 2024-05-10 北京思图场景数据科技服务有限公司 Financial risk assessment method based on biological basis, behavioral characteristics and business characteristics
US12106531B2 (en) 2021-07-22 2024-10-01 Microsoft Technology Licensing, Llc Focused computer detection of objects in images
CN118068820A (en) * 2024-04-19 2024-05-24 四川航天电液控制有限公司 An intelligent fault diagnosis method for hydraulic support controller

Also Published As

Publication number Publication date
CN112101094B (en) 2023-08-22

Similar Documents

Publication Publication Date Title
CN110490109B (en) An online human rehabilitation action recognition method based on monocular vision
Yadav et al. Real-time Yoga recognition using deep learning
CN108764050B (en) Method, system and equipment for recognizing skeleton behavior based on angle independence
CN113269013B (en) Object behavior analysis method, information display method and electronic equipment
CN112543936B (en) Motion structure self-attention-drawing convolution network model for motion recognition
WO2021114892A1 (en) Environmental semantic understanding-based body movement recognition method, apparatus, device, and storage medium
CN108875708A (en) Video-based behavior analysis method, device, equipment, system and storage medium
CN108932500A (en) A kind of dynamic gesture identification method and system based on deep neural network
WO2018162929A1 (en) Image analysis using neural networks for pose and action identification
GB2608975A (en) Person identification across multiple captured images
CN110458235B (en) Motion posture similarity comparison method in video
CN112101094B (en) A body language-based suicide risk assessment method
CN112597814A (en) Improved Openpos classroom multi-person abnormal behavior and mask wearing detection method
CN116895098B (en) Video human action recognition system and method based on deep learning and privacy protection
CN112487989A (en) Video expression recognition method based on capsule-long-and-short-term memory neural network
CN112149616A (en) A method of character interaction behavior recognition based on dynamic information
CN107943276A (en) Based on the human body behavioral value of big data platform and early warning
Docto et al. Third eye hand glove object detection for visually impaired using You Only Look Once (YOLO) v4-tiny algorithm
Uddin et al. A robust gait recognition system using spatiotemporal features and deep learning
CN120748045A (en) Behavior recognition method for improving space-time diagram convolutional network
Zaidi et al. Mae Mai Muay Thai Style Classification in Movement Appling Long-Term Recurrent Convolution Networks
Hristov Real-time abnormal human activity detection using 1DCNN-LSTM for 3D skeleton data
Kareem et al. Using skeleton based optimized residual neural network architecture of deep learning for human fall detection
CN115131876B (en) An emotion recognition method and system based on human motion gait and posture
CN112818796A (en) Intelligent posture discrimination method and storage device suitable for online invigilation scene

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20230822