基于内存云的大块数据对象并行存取策略

发布时间：2019-04-04 11:35

【摘要】：由于内存云(RAMCloud)只支持最大1 MB的小块数据对象存储,因此当大于1 MB的对象需要存储在内存云集群中就会受到对象大小的限制,无法在集群中进行存储。为了解决内存云存储限制的问题,提出了基于内存云的大块数据对象并行存取策略。该存储策略首先将大块数据对象分割成若干个1 MB的小块数据对象,然后在客户端生成数据摘要,最后使用并行存储算法将客户端分割成的小块数据对象存储在内存云集群中。读取时首先读取数据摘要,然后根据数据摘要从内存云集群中并行读取小块数据对象,并将小块数据对象合并生成大块数据对象。实验结果表明:大块数据对象的并行存取策略在不破坏内存云集群体系结构的前提下存储时间为16~18μs,读取时间为6~7μs。在Infini Band网络架构下,所提并行算法的加速比呈现类似线性的增长,它使大块数据对象也能够像小块数据对象一样在微秒级别下快速、高效地进行存取。
[Abstract]:Because memory cloud (RAMCloud) only supports small block data object storage with maximum 1 MB, when objects larger than 1 MB need to be stored in memory cluster, they will be limited by object size and cannot be stored in cluster. In order to solve the problem of memory cloud storage limitation, a parallel access strategy for large block data objects based on memory cloud is proposed. The storage strategy firstly divides the large block data object into several 1 MB small block data objects, then generates a data summary on the client side, and finally stores the small block data object divided into a client into a memory cloud cluster by using the parallel storage algorithm. When reading, we first read the data summary, then read the small block data object from the memory cluster in parallel according to the data summary, and merge the small block data object to generate the large block data object. The experimental results show that the storage time and read time of the parallel access strategy are 16 ~ 18 渭 s and 6 ~ 7 渭 s respectively without destroying the memory cluster architecture. In the framework of Infini Band network, the speedup of the proposed parallel algorithm increases linearly, which enables the large data objects to be accessed quickly and efficiently at the micro-second level as well as the small-block data objects.
【作者单位】：新疆大学软件学院;新疆大学信息科学与工程学院;
【基金】：国家自然科学基金资助项目(61462079,61262088,61562086,61363083)~~
【分类号】：TP333

【参考文献】