A Distance-Based Packing Method for High Dimensional Data

Kim, T.-W. and Li, K.-J.

Minkowski-sum cost model indicates that balanced data partitioning is not beneficial for high dimensional data. Thus we study several unbalanced partitioning methods and propose cost models for them based on Minkowski-sum cost model. Our cost models indicate that the distance to one of both ends of data space dominates the expected value under uniform data distribution. We generalize studied methods to adapt to data distribution and propose a new partitioning method, called DD-CSP (Distance-based Distribution-adaptive Cyclic Sliced Partition), for high-dimensional index structures. At each partition, it splits data from lower end or higher end to the center of data space based on distance cost function. Based on this fact, we propose a data structure called DSR(Dimension-independent Single value Representation) which takes constant amount of storage to represent MBHs(Minimum Bounding Hyper-cubes) independent of dimension. In our experimental studies, we compare DD-CSP with R-tree, HP, STR, TGS, and methods analyzed in our paper on real and synthetic data sets with wide ranges of dimensions and of selectivities varying from 10^1 to 10^9. In all experiments, we show that our method, DD-CSP, outperforms all other methods and achieves up to 567% savings in response time. Thus it is a clearly winning strategy in terms of range queries and storage requirements.

Cite as: Kim, T.-W. and Li, K.-J. (2003). A Distance-Based Packing Method for High Dimensional Data. In Proc. Fourteenth Australasian Database Conference (ADC2003), Adelaide, Australia. CRPIT, 17. Schewe, K.-D. and Zhou, X., Eds. ACS. 135-144.

(from crpit.com) (local if available)