基于网络爬虫技术的健康医疗大数据采集整理系统

发布时间：2018-08-18 20:39

【摘要】：目的快速、准确地获得公共卫生服务系统的医疗数据,并进行数据整理,为建立人群健康风险评估模型提供数据基础。方法运用聚焦网络爬虫技术,设计算法并编程,在自动记录和修正URL异常、原始数据存档、保持登录方式3个方面进行算法改进。将设计好的爬虫应用于爬取已获得授权网站的医疗数据,通过医学数据库系统,对数据进行解析、整理与导出。结果获得多个公共卫生服务基地数据,为当地政府部门提供数据分析报告,利用整理分析的数据完成多项健康风险评估模型建立。结论基于网络爬虫技术建立的数据采集整理系统,可以解决获取及整理网络许可数据的难题,将此技术应用于医药卫生领域,可使现有丰富的医学数据资源得以充分利用并提高利用效率。
[Abstract]:Objective to obtain and collate the medical data of public health service system quickly and accurately so as to provide the data basis for the establishment of population health risk assessment model. Methods using focused web crawler technology, the algorithm was designed and programmed. The algorithm was improved in three aspects: automatic recording and correction of URL anomalies, archiving of raw data, and keeping logon mode. The designed crawler is applied to crawl the medical data of the authorized website. Through the medical database system, the data are analyzed, sorted and exported. Results the data of several public health service bases were obtained, and the data analysis reports were provided for local government departments, and a number of health risk assessment models were built by collating and analyzing the data. Conclusion the data collection and finishing system based on web crawler technology can solve the problem of obtaining and collating the network license data, and this technology can be applied to the field of medicine and health. It can make full use of the abundant medical data resources and improve the utilization efficiency.
【作者单位】：山东大学公共卫生学院生物统计学系;山东大学齐鲁生物医学大数据研究中心;康评健康医疗大数据科技有限公司;山东大学计算机科学与技术学院;
【基金】：国家自然科学基金(81273177)
【分类号】：R-05;TP311.13;TP391.3

【相似文献】