阅读中文版本

Word Frequency Counter in Python

A Python script that reads a text file, counts word frequencies with a stop-word filter, and prints the most common words with their counts.

python --version

python word_freq.py sample.txt

python word_freq.py sample.txt

Prints the top words in descending frequency with their counts.

What it does

Counts word frequencies with collections.Counter, filtering common stop words so the output shows content words rather than grammar.

import collections
import re
import sys

STOP = {"the", "a", "an", "and", "or", "of", "to", "in", "is", "it", "on", "for"}

def words(path):
    with open(path, encoding="utf-8") as handle:
        text = handle.read().lower()
    return [w for w in re.findall(r"[a-z0-9]+", text) if w not in STOP]

def main():
    path = sys.argv[1]
    top = int(sys.argv[2]) if len(sys.argv) > 2 else 10
    counts = collections.Counter(words(path))
    for word, count in counts.most_common(top):
        print(f"{word:<15} {count}")

if __name__ == "__main__":
    main()

Tokenising is a simple lowercased alphanumeric split, which is enough for a quick analysis and easy to extend.

Run and verify

printf 'the quick brown fox jumps over the quick dog\n' > sample.txt
python word_freq.py sample.txt 3

Pass a text file and an optional top count; stop words such as the are removed before counting.

Expected Output

For the sample sentence with a top count of three the script should print quick first. The output is:

quick           2
brown           1
dog             1

Words sort by descending count, ties break by first-seen order, and stop words never appear in the list.

Limitations

It does not stem words, so run and running are counted separately, and it ignores case-insensitive punctuation only.