Distributed Association Rule Mining with Minimum Communication Overhead

Kaosar, M. G., Xu, Z. and Yi, X.

In distributed association rule mining algorithm, one of the major and challenging hindrances is to reduce the communication overhead. Data sites are required to exchange lot of information in the data mining process which may generates massive communication overhead. In this paper we propose an association rule mining algorithm which minimizes the communication overhead among the participating data sites. Instead of transmitting all itemsets and their counts, we propose to transmit a binary vector and count of only frequently large itemsets. Message Passing Interface (MPI) technique is exploited to avoid broadcasting among data sites. Performance study shows that the proposed algorithm performs better than two other well known algorithms known as Fast Distributed Algorithm for Mining Association Rules (FDM) and Count Distribution (CD) in terms of communication overhead.

Cite as: Kaosar, M. G., Xu, Z. and Yi, X. (2009). Distributed Association Rule Mining with Minimum Communication Overhead. In Proc. Australasian Data Mining Conference (AusDM'09) Melbourne, Australia. CRPIT, 101. Kennedy P. J., Ong K. and Christen P. Eds., ACS. 17-24

(from crpit.com) (local if available)