Hash based parallel algorithms for mining association rules

Conference · Tue Dec 31 04:00:00 EST 1996

OSTI ID:535539

Shintani, Takahiko; Kitsuregawa, Masaru ^[1]

---

1. Univ. of Tokyo, Minato (Japan)

In this paper, we propose four parallel algorithms (NPA, SPA, HPA and RPA-ELD) for mining association rules on shared-nothing parallel machines to improve its performance. In NPA, candidate itemsets are just copied amongst all the processors, which can lead to memory overflow for large transaction databases. The remaining three algorithms partition the candidate itemsets over the processors. If it is partitioned simply (SPA), transaction data has to be broadcast to all processors. HPA partitions the candidate itemsets using a hash function to eliminate broadcasting, which also reduces the comparison workload significantly. HPA-ELD fully utilizes the available memory space by detecting the extremely large itemsets and copying them, which is also very effective at flattering the load over the processors. We implemented these algorithms in a shared-nothing environment. Performance evaluations show that the best algorithm, HPA-ELD, attains good linearity on speedup ratio and is effective for handling skew.

• OSTI ID: 535539
• Report Number(s): CONF-961209--
• Country of Publication: United States
• Language: English

Related Subjects

99 GENERAL AND MISCELLANEOUS
ALGORITHMS
ARRAY PROCESSORS
DATA BASE MANAGEMENT
INFORMATION RETRIEVAL
PARALLEL PROCESSING
PERFORMANCE