[PR #185] feat(types): detect MD5/SHA1/SHA256 file hashes on import #2875

New Issue

GiteaMirror · 2026-06-13T06:28:20-05:00

GiteaMirror commented

2026-06-13 06:28:20 -05:00

📋 Pull Request Information

Original PR: https://github.com/reconurge/flowsint/pull/185
Author: @rachit367
Created: 6/9/2026
Status: 🔄 Open

Base: main ← Head: feat/detect-file-hashes

📝 Commits (1)

2e12e42 feat(types): detect MD5/SHA1/SHA256 file hashes on import

📊 Changes

4 files changed (+120 additions, -7 deletions)

View changed files

📝 flowsint-core/tests/import/test_entity_detection.py (+15 -1)
📝 flowsint-types/src/flowsint_types/file.py (+35 -5)
📝 flowsint-types/src/flowsint_types/username.py (+6 -1)
➕ flowsint-types/tests/test_file.py (+64 -0)

📄 Description

What

Makes TXT import recognize MD5 / SHA1 / SHA256 hashes and map them to the File type, so analysts can pivot from a hash to related domains/URLs/IPs (VirusTotal, MalwareBazaar, sandboxes) — as requested in #45.

Closes #45

The bug

File.detect() always returned False, and Username.detect() matches any 3–80 char alphanumeric string (^[a-zA-Z0-9_-]{3,80}$). A hash is alphanumeric and 32/40/64 chars long, so on import a hash was silently detected as a Username, not a File:

detect_type("d41d8cd98f00b204e9800998ecf8427e")  # -> Username  (before)
detect_type("d41d8cd98f00b204e9800998ecf8427e")  # -> File      (after)

(Verified against the live registry: all three hash types resolved to Username.)

The fix

File.detect() now recognizes 32/40/64-char hex strings (MD5/SHA1/SHA256). File.from_string() stores the value in the matching hash_md5 / hash_sha1 / hash_sha256 field (normalized to lowercase) while keeping it as the filename so the node still has a label. The File type already had these hash fields — no schema change needed.
Username.detect() now explicitly rejects hash-shaped strings. File is already registered before Username (indices 14 vs 29), so File wins detection on its own — but this guard removes the implicit dependency on registration order, so a future import reorder can't regress it.

Non-hash filenames are intentionally still not auto-detected (too ambiguous), so existing TXT-import behavior for ordinary values is unchanged.

Testing

uv run --package flowsint-types pytest flowsint-types/tests/ -q   # 54 passed
uv run --package flowsint-core  pytest flowsint-core/tests/import/ -q  # 47 passed

New tests: detection of each hash type (case-insensitive, whitespace-trimmed), rejection of non-hashes (wrong length, non-hex, filenames, usernames), from_string hash-field population + lowercase normalization, the Username regression, and end-to-end detect_type() → File. No DB/Docker needed.

The issue suggested editing entity_detection.py's old DETECTORS list, but that module was since refactored to delegate to each type's detect() method — so the correct home for this logic is File.detect(), which is what this PR does.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/reconurge/flowsint/pull/185 **Author:** [@rachit367](https://github.com/rachit367) **Created:** 6/9/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `feat/detect-file-hashes` --- ### 📝 Commits (1) - [`2e12e42`](https://github.com/reconurge/flowsint/commit/2e12e42bba9e688a467d264fc49baf180d81dbc4) feat(types): detect MD5/SHA1/SHA256 file hashes on import ### 📊 Changes **4 files changed** (+120 additions, -7 deletions) <details> <summary>View changed files</summary> 📝 `flowsint-core/tests/import/test_entity_detection.py` (+15 -1) 📝 `flowsint-types/src/flowsint_types/file.py` (+35 -5) 📝 `flowsint-types/src/flowsint_types/username.py` (+6 -1) ➕ `flowsint-types/tests/test_file.py` (+64 -0) </details> ### 📄 Description ## What Makes TXT import recognize **MD5 / SHA1 / SHA256 hashes** and map them to the `File` type, so analysts can pivot from a hash to related domains/URLs/IPs (VirusTotal, MalwareBazaar, sandboxes) — as requested in #45. Closes #45 ## The bug `File.detect()` always returned `False`, and `Username.detect()` matches any 3–80 char alphanumeric string (`^[a-zA-Z0-9_-]{3,80}$`). A hash is alphanumeric and 32/40/64 chars long, so on import a hash was silently detected as a **Username**, not a File: ```python detect_type("d41d8cd98f00b204e9800998ecf8427e") # -> Username (before) detect_type("d41d8cd98f00b204e9800998ecf8427e") # -> File (after) ``` (Verified against the live registry: all three hash types resolved to `Username`.) ## The fix - **`File.detect()`** now recognizes 32/40/64-char hex strings (MD5/SHA1/SHA256). `File.from_string()` stores the value in the matching `hash_md5` / `hash_sha1` / `hash_sha256` field (normalized to lowercase) while keeping it as the `filename` so the node still has a label. The `File` type already had these hash fields — no schema change needed. - **`Username.detect()`** now explicitly rejects hash-shaped strings. `File` is already registered before `Username` (indices 14 vs 29), so `File` wins detection on its own — but this guard removes the implicit dependency on registration order, so a future import reorder can't regress it. Non-hash filenames are intentionally still **not** auto-detected (too ambiguous), so existing TXT-import behavior for ordinary values is unchanged. ## Testing ``` uv run --package flowsint-types pytest flowsint-types/tests/ -q # 54 passed uv run --package flowsint-core pytest flowsint-core/tests/import/ -q # 47 passed ``` New tests: detection of each hash type (case-insensitive, whitespace-trimmed), rejection of non-hashes (wrong length, non-hex, filenames, usernames), `from_string` hash-field population + lowercase normalization, the Username regression, and end-to-end `detect_type()` → `File`. No DB/Docker needed. > The issue suggested editing `entity_detection.py`'s old `DETECTORS` list, but that module was since refactored to delegate to each type's `detect()` method — so the correct home for this logic is `File.detect()`, which is what this PR does. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2026-06-13 06:28:20 -05:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/flowsint#2875