Sciweavers

Share
IJCAI
2007

Dynamically Weighted Hidden Markov Model for Spam Deobfuscation

8 years 7 months ago
Dynamically Weighted Hidden Markov Model for Spam Deobfuscation
Spam deobfuscation is a processing to detect obfuscated words appeared in spam emails and to convert them back to the original words for correct recognition. Lexicon tree hidden Markov model (LTHMM) was recently shown to be useful in spam deobfuscation. However, LT-HMM suffers from a huge number of states, which is not desirable for practical applications. In this paper we present a complexity-reduced HMM, referred to as dynamically weighted HMM (DW-HMM) where the states involving the same emission probability are grouped into super-states, while preserving state transition probabilities of the original HMM. DWHMM dramatically reduces the number of states and its state transition probabilities are determined in the decoding phase. We illustrate how we convert a LT-HMM to its associated DW-HMM. We confirm the useful behavior of DW-HMM in the task of spam deobfuscation, showing that it significantly reduces the number of states while maintaining the high accuracy.
Seunghak Lee, Iryoung Jeong, Seungjin Choi
Added 29 Oct 2010
Updated 29 Oct 2010
Type Conference
Year 2007
Where IJCAI
Authors Seunghak Lee, Iryoung Jeong, Seungjin Choi
Comments (0)
books