Gal A. Kaminka: Publications

Sorted by DateClassified by Publication TypeClassified by TopicGrouped by Student (current)Grouped by Former Students

REEF: Resolving Length Bias in Frequent Sequence Mining Using Sampling

Ariella Richardson, Gal A. Kaminka, and Sarit Kraus. REEF: Resolving Length Bias in Frequent Sequence Mining Using Sampling. International Journal On Advances in Intelligent Systems, 7(1--2):208–222, 2014.

Download

[PDF]1.1MB  

Abstract

Classic support based approaches efficiently address frequent sequence mining. However, support based mining has been shown to suffer from a bias towards short sequences. In this paper, we propose a method to resolve this bias when mining the most frequent sequences. In order to resolve the length bias we define norm-frequency,based on the statistical z-score of support, and use it to replace support based frequency. Our approach mines the subsequences that are frequent relative to other subsequences of the same length. Unfortunately, naive use of norm-frequency hinders mining scalability. Using norm-frequency breaks the anti-monotonic property of support, an important part in being able to prune large sets of candidate sequences. We describe a bound that enables pruning to provide scalability. Calculation of the bound uses a preprocessing stage on a sample of the dataset. Sampling the data creates a distortion in the samples measures. We present a method to correct this distortion. We conducted experiments on 4 data sets, including synthetic data, textual data, remote control zapping data and computer user input data. Experimental results establish that we manage to overcome the short sequence bias successfully, and to illustrate the production of meaningful sequences with our mining algorithm.

BibTeX

@article{immm14,
 author = {Ariella Richardson and Gal A. Kaminka and Sarit Kraus},
 title =  {{REEF}: Resolving Length Bias in Frequent Sequence Mining Using Sampling},
 journal = {International Journal On Advances in Intelligent Systems},
 year = {2014},
 OPTnote = {},
 volume = {7},
 number = {1--2},
 pages = {208--222},
 abstract = {Classic support based approaches efficiently address frequent sequence mining. However, support based mining has been shown to 
   suffer from a bias towards short sequences. In this paper, we propose a method to resolve this bias when mining the most frequent sequences. In 
   order to resolve the length bias we define norm-frequency,based on the statistical z-score of support, and use it to replace support based 
   frequency. Our approach mines the subsequences that are frequent relative to other subsequences of the same length. Unfortunately, naive use of 
    norm-frequency hinders mining scalability. Using norm-frequency breaks the anti-monotonic property of support, an important part in being able 
    to prune large sets of candidate sequences. We describe a bound that enables pruning to provide scalability. Calculation of the bound uses 
    a preprocessing stage on a sample of the dataset. Sampling the data creates a distortion in the samples measures. We present a method to correct 
    this distortion. We conducted experiments on 4 data sets, including synthetic data, textual data, remote control zapping data and computer 
    user input data. Experimental results establish that we manage to overcome the short sequence bias successfully, and to illustrate the 
    production of meaningful sequences with our mining algorithm.},
 OPTkeywords = {Frequent Sequence Mining; Data Mining; Zscore; Sampling; Multivariate Sequences},
}

Generated by bib2html.pl (written by Patrick Riley ) on Fri Aug 30, 2024 17:29:51