分布式搜索中节点索引量大小估计算法

发布时间：2018-12-24 10:01

【摘要】：分布式搜索是解决对深层网络搜索的有效方案,各节点的索引量大小是分布式搜索引擎描述选择节点的重要参数。为了解决在非合作环境中估算节点索引量大小的问题,提出并实现了基于高频词汇再采样的高频再采样算法和基于文档捕获概率不同假设的异概捕获算法。高频再采样算法在随机采样后基于样本集中的高频词汇进行再采样;而异概捕获算法则利用Logistic函数和条件似然方法估算节点的索引量大小。通过真实网络数据的实验结果表明,这些算法优于已有的采样-再采样与捕获-再捕获算法。
[Abstract]:Distributed search is an effective solution to the deep network search. The index size of each node is an important parameter to describe the selection node of the distributed search engine. In order to solve the problem of estimating the index size of nodes in a non-cooperative environment, a high-frequency resampling algorithm based on high-frequency lexical resampling and an alternative capture algorithm based on different assumptions of document acquisition probability are proposed and implemented. The high-frequency resampling algorithm is based on the high-frequency vocabulary in the sample set after random sampling, while the hetero-probability acquisition algorithm uses Logistic function and conditional likelihood method to estimate the index size of nodes. The experimental results of real network data show that these algorithms are superior to the existing sample-resampling and capture-recapture algorithms.
【作者单位】：清华大学电子工程系;
【分类号】：TP391.3

【共引文献】