Python垃圾邮件识别

Question

请根据程序编码，给出适合的测试邮件列表示例或能够使程序正确运行的阈值，使得程序能够正确地运行出检测结果

Answer 1

引自免费微信小程序：皆我百晓生

为了确保上述垃圾邮件识别程序能够正确运行并给出合理的检测结果，我们需要关注几个关键点：文本清洗、词汇分割、词汇频率统计、以及基于已知垃圾词汇的频率计算。下面，我将提供一个适合的测试邮件列表示例，并调整阈值以确保程序能够区分垃圾邮件与正常邮件。

测试邮件列表示例

我们将使用以下测试邮件列表，这些邮件包括了明显的垃圾邮件特征（如免费赠品、中奖通知）和正常的邮件内容（如项目更新、账户安全警告）：

test_emails = [
    "Last chance to get your free gift",
    "Project deadline extension",
    "Your account has been compromised",
    "Free money now, now, now",
    "Reminder: Meeting scheduled for tomorrow",
    "Win a prize! Click here!",
    "Urgent: Security update required",
    "Claim your free trial today",
    "Monthly report attached"
]

阈值调整

在垃圾邮件识别中，threshold参数是关键，它决定了何时将邮件标记为垃圾邮件。基于上述示例数据，初始阈值设置为0.1可能不足以准确地区分垃圾邮件和正常邮件，因为一些正常邮件也可能包含少量垃圾词汇（例如，“free”可能出现在正常促销邮件中）。因此，我们可以通过调整阈值来优化分类效果。

阈值选择建议

考虑到垃圾邮件通常含有多个垃圾词汇，而正常邮件即使包含垃圾词汇也往往较少，我们可以尝试将threshold设置为0.2，这意呈只有当垃圾词汇的频率达到总词汇量的20%以上时，才将邮件标记为垃圾邮件。

完整代码示例

下面是完整的代码示例，包括修改后的test_emails列表和调整后的threshold值：

import re
from collections import Counter

def clean_text(text):
    return re.sub(r'\W+', ' ', text).lower()

def tokenize(text):
    return text.split()

def word_freq(words):
    return Counter(words)

def calculate_rate(freq, spamwords):
    total = sum(freq.values())
    rate = sum(freq[word] for word in spamwords if word in freq) / total
    return rate

# 示例数据
spam_emails = [
    "Get rich quick schemes",
    "Free money now",
    "Congratulations! You've won a prize"
]

normal_emails = [
    "Meeting at 3pm",
    "Reminder: grocery list",
    "Project update"
]

# 已知的垃圾邮件词汇
known_spam_words = ['get', 'rich', 'quick', 'free', 'money', 'now', 'congratulations', 'won', 'prize']

def is_spam(email, spamwords=known_spam_words, threshold=0.2):
    cleaned = clean_text(email)
    words = tokenize(cleaned)
    freq = word_freq(words)
    rate = calculate_rate(freq, spamwords)
    # 如果垃圾邮件词汇频率超过阈值，则认定为垃圾邮件
    return rate > threshold

# 测试邮件列表
test_emails = [
    "Last chance to get your free gift",
    "Project deadline extension",
    "Your account has been compromised",
    "Free money now, now, now",
    "Reminder: Meeting scheduled for tomorrow",
    "Win a prize! Click here!",
    "Urgent: Security update required",
    "Claim your free trial today",
    "Monthly report attached"
]

# 检查并打印检测结果
for email in test_emails:
    if is_spam(email):
        print(f'"{email}" -> 垃圾邮件')
    else:
        print(f'"{email}" -> 正常邮件')

通过上述代码和测试邮件列表，可以有效地检测并分类邮件，同时通过调整threshold参数至0.2，可以进一步优化垃圾邮件的识别准确率。