深度学习结合改进DBSCAN聚类的数据异常检测
CSTR:
作者:
作者单位:

作者简介:

通讯作者:

中图分类号:

基金项目:

国家自然科学基金资助项目(52208189),江苏省高等学校基础科学(自然科学)(21KJB580006)


Combining Deep Learning with Improved DBSCAN Clustering for Data Anomaly Detection
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    由于结构健康监测系统采集到的数据不可避免存在异常,导致无法从中获取结构真实健康情况,故异常数据检测对结构分析及其状态评估至关重要.为此,提出一种基于组合预测模型的多通道数据异常检测方法.首先,将结构健康监测数据分为两段,前段只有环境引起的间歇性异常,后段包括间歇性异常以及传感器故障造成的数据异常.其次,通过根据余弦核密度估计各数据点的局部密度自适应地选取参数半径,并对基于密度的空间聚类算法(DBSCAN)改进,进而用该改进模型剔除前段数据中的间歇性异常得到清洗数据(即没有问题的正常数据).接着,基于多传感器间的相关性,结合卷积神经网络(CNN)的空间特征和长短期记忆网络(LSTM)的时间特征,训练清洗数据得到代表正常数据特征的数学模型.然后,在数学模型中输入后段数据得到预测数据,并将预测数据与后段数据对比得到预测误差,采用极值理论(EVT)拟合预测误差分布并设置阈值,进而检测数据的异常状况.最后,分析Dowling Hall人行天桥加速度监测数据表明,该方法能够有效提高结构健康监测异常数据的检测能力.

    Abstract:

    Due to the inevitable presence of anomalies in the data collected by the structural health monitoring system, it is impossible to obtain the true health status of the structure. Therefore, anomaly data detection is crucial for structural analysis and state evaluation. A multi-channel data anomaly detection method based on a combination prediction model is proposed. Firstly, the structural health monitoring data is divided into two sections. The first section only includes intermittent anomalies caused by the environment, while the second section includes intermittent anomalies and data anomalies caused by sensor failures. Secondly, by estimating the local density of each data point based on cosine kernel density and adaptively selecting parameter radii, and improving the density based spatial clustering of applications with noise (DBSCAN) algorithm, the improved model is used to remove intermittent anomalies in the previous data and obtain clean data (i.e., normal data without problems). Next, based on the correlation between multiple sensors, combined with the spatial features of convolutional neural networks (CNN) and the temporal features of long short term memory networks (LSTM), a mathematical model representing normal data features is trained to clean the data. Then, the predicted data is obtained by inputting the later stage data into the mathematical model, and the predicted data is compared with the later stage data to obtain the prediction error. The extreme value theory (EVT) algorithm is used to fit the distribution of the prediction error and set a threshold to detect abnormal conditions in the data. Finally, analyzing the acceleration monitoring data of Dowling Hall pedestrian overpass shows that this method can effectively improve the detection ability of abnormal data in structural health monitoring.

    参考文献
    相似文献
    引证文献
引用本文

王典,常军.深度学习结合改进DBSCAN聚类的数据异常检测[J].动力学与控制学报,2025,23(9):74~84; Wang Dian, Chang Jun. Combining Deep Learning with Improved DBSCAN Clustering for Data Anomaly Detection[J]. Journal of Dynamics and Control,2025,23(9):74-84.

复制
分享
相关视频

文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2025-01-01
  • 最后修改日期:2025-03-01
  • 录用日期:
  • 在线发布日期: 2025-09-30
  • 出版日期:
文章二维码

微信公众号二维码

手机版网站二维码