54 Commits
Author SHA1 Message Date
dextmorgn 2a378954ae fix(types): make username not required for SocialAccount 2026-06-21 16:02:21 +02:00
dextmorgnandGitHub 7ea36bb9af Merge pull request #183 from rachit367/feat/technology-type-and-tech-detect
feat(types,enrichers): add Technology type and tech_detect transformer
2026-06-20 17:57:53 +02:00
dextmorgnandGitHub 48d1df13d6 Merge pull request #185 from rachit367/feat/detect-file-hashes
feat(types): detect MD5/SHA1/SHA256 file hashes on import
2026-06-20 17:50:13 +02:00
rachit367 2e12e42bba feat(types): detect MD5/SHA1/SHA256 file hashes on import
TXT imports of file hashes were detected as Username, because File.detect
always returned False and Username.detect matches any 3-80 char
alphanumeric string. Hashes therefore couldn't be used as File entities
to pivot from (VirusTotal, MalwareBazaar, sandboxes), per #45.

- File.detect now recognizes 32/40/64-char hex strings (MD5/SHA1/SHA256),
  and File.from_string stores the value in the matching hash_* field
  (normalized to lowercase) while keeping it as the filename/label.
- Username.detect now explicitly rejects hash-shaped strings, so hash
  detection is correct regardless of type-registration order (File is
  already registered before Username, but this removes the implicit
  dependency).

Tests cover detection of each hash type (case/whitespace), rejection of
non-hashes, from_string field population, the Username regression, and
end-to-end detect_type() resolution to File.

Closes #45
2026-06-09 11:30:37 +05:30
rachit367 63a97a76f7 feat(types,enrichers): add Technology type and tech_detect transformer
Adds application-level technology fingerprinting, extending coverage
beyond ip/port/website/ssl_certificate as requested in #59.

- New `Technology` type (name, version, category, source) registered in
  the type registry, the package exports, and TYPE_TO_MODEL. Its
  from_string accepts both bare names ("nginx") and wappalyzer-style
  "name:version" pairs ("PHP:8.1").
- New `tech_detect` enricher: Website -> Technology, using the existing
  httpx tool with -td (wappalyzer dataset). Each detected technology
  becomes a Technology node linked to the website with a
  USES_TECHNOLOGY relationship; results are de-duplicated per probe.

Tests cover the Technology type (label, from_string, registry), the
tech entry parser, enricher registration/metadata, and scan() with the
httpx Docker tool mocked, so no Docker daemon is required.

Closes #59
2026-06-09 10:45:43 +05:30
dextmorgn 9fcc60ca63 fix(deps): upgrade Python dependencies to patch 27 known vulnerabilities
pip-audit reported 27 vulnerabilities across 12 packages, including
cryptography PYSEC-2026-36 (CVSS 9.8) used by the vault, a starlette
Host-header path bypass, and python-multipart arbitrary file write.

- cryptography 45.0.7 -> 46.0.7 (pin was capped at <46)
- fastapi 0.115 -> 0.136, pulling starlette 0.46 -> 1.2
- python-multipart 0.0.20 -> 0.0.32
- pyjwt, urllib3, aiohttp, lxml, idna, mako and transitive bumps
- dev tools: pytest 8 -> 9 (requires pytest-httpx >=0.36), black 25 -> 26

All four module test suites pass (458 tests). pip-audit now reports
no known vulnerabilities.
2026-06-05 08:57:19 +02:00
dextmorgn e8ca7b39bf fix(types): apply extra=allow via Pydantic v2 model_config
FlowsintType declared an inner `class ConfigDict: extra = "allow"`, which
Pydantic v2 treats as an ordinary nested class and ignores entirely. The
base therefore fell back to the default extra="ignore", silently dropping
user-supplied extra properties on every entity type. Replaced with
model_config = ConfigDict(extra="allow"). Extra keys are now retained.

flowsint-types (44), flowsint-enrichers (12) and flowsint-core (390) suites pass.
2026-05-30 19:03:30 +02:00
dextmorgn 00ec9d6f26 feat(types): switch to uv 2026-04-11 13:15:06 +02:00
dextmorgn 1ee06f5c8a feat(types): format phone numbers to avoid doublons 2026-03-15 13:07:01 +01:00
dextmorgn 37cdd93257 feat(types): detect individual from spaces 2026-03-15 13:06:40 +01:00
dextmorgn 9e07429e7b feat(types): unique webtracker label 2026-02-12 22:01:26 +01:00
dextmorgn 2bac67e2a8 feat(types): individual.last_name becomes optional 2026-02-07 17:44:55 +01:00
dextmorgn a178ef1bcd feat(chore): update deploy components 2026-01-25 20:45:22 +01:00
dextmorgn 7d33b813de chore(license): migrate from AGPL-3.0 to Apache License 2.0 2026-01-25 16:19:28 +01:00
dextmorgn 21376168ef feat(types): refactor to use nodeLabel 2026-01-21 17:17:53 +01:00
dextmorgn 6491134b73 feat(types): remove unecessary wallet validation 2026-01-11 12:20:43 +01:00
dextmorgn 1b0fb516af fix(types): fix label test for SocialAccount 2026-01-05 21:58:11 +01:00
dextmorgn 3b4f85f66a feat(types): remove restrictive username format 2026-01-03 23:41:21 +01:00
dextmorgn 14421bfcf3 feat(types): use platform in social account label 2026-01-02 00:42:12 +01:00
dextmorgn 3ad9e72ca1 feat(types): socialAccount creating from string for username + tests 2025-12-10 15:20:11 +01:00
dextmorgn 7a24ff8d33 fix(types): auto build username from string for SocialAccount 2025-12-10 15:16:16 +01:00
dextmorgn 4a74674c4d feat(types): username + individual validation 2025-12-05 15:46:21 +01:00
dextmorgn 1d7e7789e6 feat(types): additional website properties + status_code test 2025-12-03 12:35:33 +01:00
dextmorgn 1124921db2 fix(types): is_root_domain 2025-12-03 12:32:38 +01:00
dextmorgn 95d539f32f fix(types): better check_root condition 2025-12-03 12:12:02 +01:00
dextmorgn c3c328f299 feat(types): auto_discovery of types 2025-11-30 12:48:25 +01:00
dextmorgn 136de4240b feat(types): type registry + type detection 2025-11-28 20:08:37 +01:00
dextmorgn ad7bbcfe50 feat(types): 1.0.0 2025-11-27 17:32:50 +01:00
dextmorgn 2123ea5b8d feat: complete refactor 2025-11-27 17:16:41 +01:00
dextmorgn 1e71db1e2d feat(types): primary_key 2025-11-27 09:14:25 +01:00
dextmorgn 22f630d091 fix(types): DNSRecord deprecated field name 2025-11-26 16:05:57 +01:00
dextmorgn c83472baf2 feat(types): update compute_label for location 2025-11-24 14:51:25 +01:00
dextmorgn 76593cdb86 feat(types): remove domain name from dnsrecord 2025-11-21 15:51:17 +01:00
dextmorgn 3933946777 feat(types): add pydantic parser + tests 2025-11-21 12:55:53 +01:00
dextmorgn 8e1359702c fix(types): add missing deps + update poetry.lock 2025-11-21 12:53:00 +01:00
dextmorgn 456d95093e feat(types): label tests 2025-11-21 00:36:28 +01:00
dextmorgn 9a8fe6b139 fix: social_account transforms 2025-11-20 18:35:18 +01:00
dextmorgn ade74800aa feat: minifiy transform logic 2025-11-20 10:59:18 +01:00
dextmorgn 7b91bb2442 feat: type validation + simplified transform system 2025-11-19 22:13:38 +01:00
dextmorgn fa743f3055 feat(types): types validation using pydantic directly 2025-11-19 12:14:58 +01:00
dextmorgn 7965a473fb feat(types): new Port type 2025-11-18 12:54:04 +01:00
dextmorgn a18ac52efc feat(types): support SocialAccount and Username 2025-11-17 16:56:22 +01:00
dextmorgn f78285a1ae feat: update contact email 2025-11-17 16:53:07 +01:00
dextmorgn aecec70408 feat: update licence 2025-11-08 20:26:33 +01:00
dextmorgn 5ea0156147 feat: python = >=3.12,<4.0 2025-10-27 08:56:21 +01:00
dextmorgn a13c6bf73f feat: poetry 2025-10-24 14:11:43 +02:00
dextmorgn 0a85e3fea7 fix: python versions 2025-10-24 13:48:31 +02:00
dextmorgn d3f0150b79 fix: poetry python versions 2025-10-24 13:40:20 +02:00
dextmorgn 0619550107 feat: update readme 2025-09-16 12:31:06 +02:00
dextmorgn 3c74b3d544 feat: GraphNode and GraphEdge revisited + lasso selection 2025-08-28 15:41:16 +02:00