69 Commits

Author SHA1 Message Date
dextmorgn
0d092c44f0 Merge pull request #191 from Ghraven/fix/fraudscore-aware-utc
fix: use aware timestamp for fraud score updates
2026-07-01 10:50:59 +02:00
dextmorgn
e02d548578 Merge pull request #193 from Ghraven/fix/core-aware-utc-cutoffs
fix(core): use aware UTC timestamps
2026-06-24 09:29:32 +02:00
Ghraven
d049cb1e3e fix(core): use aware UTC timestamps 2026-06-24 15:03:31 +08:00
dextmorgn
db4ab1b5ca Merge pull request #178 from aaronjmars/security/host-header-allowlist-dns-rebinding
fix(security): Host-header allowlist + restore dropped security headers
2026-06-22 17:10:22 +02:00
dextmorgn
0c5a6bfabc fix(tools): align test fixtures with current GraphNode shape
Also fix root .gitignore accidentally shadowing flowsint-app/src/lib/
via Python venv lib/ rule — add negation to unblock tracking.
2026-06-21 16:58:21 +02:00
dextmorgn
c84b96cced chore(husky): remove deprecated shell shim from commit-msg hook 2026-06-21 16:40:57 +02:00
dextmorgn
1f57cb0904 fix(security): bump dompurify and cryptography to patched versions
- dompurify 3.3.0 → 3.4.11 (XSS bypass CVEs via tiptap)
- cryptography 46.0.7 → 48.0.1 (vulnerable OpenSSL in wheels)
2026-06-21 16:39:21 +02:00
dextmorgn
65c7ba2c01 style(vault): simplify empty state and keys table UI
Replace over-designed empty state (gradient card, badge row, clip-text)
with plain centered layout matching app empty-state pattern.
Strip table of gradient hovers, animated icon boxes, and decorative
clock pills — plain rows, plain header.
2026-06-21 16:27:13 +02:00
dextmorgn
f06989447e fix(graph): wrap clip in save/restore to prevent canvas state leak
ctx.clip() without save/restore was permanently mutating the clipping
region across all subsequent canvas draw calls, causing uncontrollable
scaling on nodes and background when a node had an imageUrl.
2026-06-21 16:13:23 +02:00
dextmorgn
2a378954ae fix(types): make username not required for SocialAccount 2026-06-21 16:02:21 +02:00
Ghraven
0485fc1746 fix: use aware timestamp for fraud score updates 2026-06-21 19:38:07 +08:00
dextmorgn
ee66f8e0fc Merge pull request #182 from rachit367/feat/domain-to-dns-enricher
feat(enrichers): add domain_to_dns enricher using dnsx
2026-06-20 18:03:09 +02:00
dextmorgn
7ea36bb9af Merge pull request #183 from rachit367/feat/technology-type-and-tech-detect
feat(types,enrichers): add Technology type and tech_detect transformer
2026-06-20 17:57:53 +02:00
dextmorgn
9218948b35 Merge pull request #181 from zurtix/main
feat(custom types): added category to custom types for organization
2026-06-20 17:54:26 +02:00
dextmorgn
48d1df13d6 Merge pull request #185 from rachit367/feat/detect-file-hashes
feat(types): detect MD5/SHA1/SHA256 file hashes on import
2026-06-20 17:50:13 +02:00
dextmorgn
fe8a4a3210 fix: make nginx resolver Podman-compatible
Remove Docker-specific resolver (127.0.0.11) in favor of direct
proxy_pass, which falls back to the system resolver (/etc/resolv.conf).
Mount local nginx.conf into the prod image to override the baked-in config.
2026-06-20 17:44:57 +02:00
dextmorgn
ddc943790b Merge pull request #187 from localzet/main
fix: "exec /docker-entrypoint.sh: exec format error" in flowsint-app
2026-06-18 08:49:08 +02:00
dextmorgn
80969367e6 Merge pull request #189 from Ghraven/feat/export-selected-json
feat: export selected graph nodes as JSON
2026-06-18 08:45:40 +02:00
dextmorgn
67c7c4a833 Merge pull request #190 from Ghraven/fix/yaml-loader-utf8
fix: read YAML templates as UTF-8
2026-06-18 08:44:27 +02:00
Ghraven
38c7a72cb1 fix: read YAML templates as UTF-8 2026-06-13 22:47:15 +08:00
Ghraven
cf2bc1ec07 feat: export selected graph nodes as JSON 2026-06-12 04:24:00 +08:00
dextmorgn
3885f8b4d1 Merge pull request #188 from Ghraven/fix/remove-frontend-debug-logs
fix(ui): remove frontend debug console logs
2026-06-11 12:37:11 +02:00
Ghraven
8de27796b0 fix(ui): remove frontend debug console logs 2026-06-11 13:55:17 +08:00
Ivan Zorin
eedbeb4013 chore: update nginx base image to version 1.29-alpine 2026-06-09 14:05:54 +03:00
rachit367
2e12e42bba feat(types): detect MD5/SHA1/SHA256 file hashes on import
TXT imports of file hashes were detected as Username, because File.detect
always returned False and Username.detect matches any 3-80 char
alphanumeric string. Hashes therefore couldn't be used as File entities
to pivot from (VirusTotal, MalwareBazaar, sandboxes), per #45.

- File.detect now recognizes 32/40/64-char hex strings (MD5/SHA1/SHA256),
  and File.from_string stores the value in the matching hash_* field
  (normalized to lowercase) while keeping it as the filename/label.
- Username.detect now explicitly rejects hash-shaped strings, so hash
  detection is correct regardless of type-registration order (File is
  already registered before Username, but this removes the implicit
  dependency).

Tests cover detection of each hash type (case/whitespace), rejection of
non-hashes, from_string field population, the Username regression, and
end-to-end detect_type() resolution to File.

Closes #45
2026-06-09 11:30:37 +05:30
rachit367
63a97a76f7 feat(types,enrichers): add Technology type and tech_detect transformer
Adds application-level technology fingerprinting, extending coverage
beyond ip/port/website/ssl_certificate as requested in #59.

- New `Technology` type (name, version, category, source) registered in
  the type registry, the package exports, and TYPE_TO_MODEL. Its
  from_string accepts both bare names ("nginx") and wappalyzer-style
  "name:version" pairs ("PHP:8.1").
- New `tech_detect` enricher: Website -> Technology, using the existing
  httpx tool with -td (wappalyzer dataset). Each detected technology
  becomes a Technology node linked to the website with a
  USES_TECHNOLOGY relationship; results are de-duplicated per probe.

Tests cover the Technology type (label, from_string, registry), the
tech entry parser, enricher registration/metadata, and scan() with the
httpx Docker tool mocked, so no Docker daemon is required.

Closes #59
2026-06-09 10:45:43 +05:30
rachit367
b4e5102592 feat(enrichers): add domain_to_dns enricher using dnsx
Adds a `domain_to_dns` enricher that resolves a domain's A (IPv4) and
AAAA (IPv6) records through ProjectDiscovery's dnsx, emitting each
resolved IP as an `Ip` node linked to the domain with a `RESOLVES_TO`
relationship.

This complements the existing `domain_to_ip` enricher (which uses
Python's socket.gethostbyname and is limited to a single IPv4 result)
by leveraging the dnsx toolkit for full A/AAAA resolution.

Implementation:
- Extend DnsxTool with `resolve_domain()` that runs
  `dnsx -d <domain> -a [-aaaa] -json -silent` and parses the JSONL
  output (a/aaaa fields per retryabledns.DNSData) into a de-duplicated
  IP list. The existing CIDR `launch()` path is left untouched.
- New DomainToDnsEnricher (auto-registered via @flowsint_enricher) with
  an optional `ipv6` toggle and optional PDCP_API_KEY vault secret.
- Unit tests covering registry wiring, JSONL parsing (A/AAAA, dedup,
  blank/malformed lines, empty output) and scan() with the Docker tool
  mocked, so no Docker daemon is required to run them.

Closes #79
2026-06-09 10:28:32 +05:30
Eric
25b735b2a2 Merge branch 'main' of github.com:zurtix/flowsint 2026-06-07 08:57:04 -04:00
Eric
a7d5861d5b Fixed type for custom type in type_registry_service to revert back to using name as to not break graph 2026-06-07 08:54:20 -04:00
Eric Howard
86b254bbf9 Custom types replaced with proper custom_types_category value 2026-06-06 18:48:05 -04:00
Eric
fc9208865d Custom types replaced with proper custom_types_category value 2026-06-06 18:44:21 -04:00
Eric
f0b0b9bf82 Added category to custom types to reorginize into desired categories 2026-06-06 18:38:35 -04:00
aeonframework
0a3567af96 fix(security): Host-header allowlist + restore dropped security headers
Detected by Aeon + semgrep.
Severity: medium
CWE-350 (reverse-DNS trust) / CWE-352 (CSRF via DNS rebinding) / CWE-1021 (UI layering / dropped X-Frame-Options).

- Add `map $http_host $is_flowsint_host` allowlist + default-deny `if` at
  the top of the server block. Default permits localhost / 127.0.0.1 /
  ::1 (any port, case-insensitive); two commented-out template lines show
  LAN operators how to add their server hostname/IP.
- Re-declare the four `add_header` security headers in the static-assets
  and `/index.html` location blocks, which previously dropped them due to
  nginx`s replace-all add_header inheritance.
- README: one paragraph under "Deploy on a network" naming the new
  operator step alongside the existing `.env` rotation.
2026-06-05 20:45:38 +00:00
dextmorgn
12bf2937c1 Merge pull request #176 from reconurge/ci/python-tests
ci: run python tests on pull requests and main
2026-06-05 15:47:32 +02:00
dextmorgn
612d39eadf test: provide dummy Neo4j credentials in conftest
Any code path reaching the Neo4jConnection singleton requires NEO4J_*
env vars at construction — tests only passed thanks to developers'
local .env files. Driver creation is lazy, nothing connects.

Verified by running all four suites with every .env removed:
458 passed.
2026-06-05 15:34:58 +02:00
dextmorgn
dc36ef633e test(core): factor out env-independent no-connection repo setup
Previous fix covered one occurrence; 17 other tests built
Neo4jGraphRepository(neo4j_connection=None), hitting the singleton
fallback that requires NEO4J_* env vars. Single helper, no env
dependency left.
2026-06-05 15:26:08 +02:00
dextmorgn
22c6908a0e test(core): stop relying on env credentials in no-connection test
Neo4jGraphRepository(neo4j_connection=None) falls back to the
Neo4jConnection singleton, which requires NEO4J_* env vars — the test
only passed locally thanks to .env. Pass a mock instead; the test
nulls _connection right after anyway.
2026-06-05 15:17:37 +02:00
dextmorgn
71e3a68830 fix(core): declare pyyaml dependency
flowsint_core imports yaml (template_generator_service, yaml_loader)
but never declared it — present locally only as a leftover in the
shared venv. CI's clean resolve exposed the missing declaration.
2026-06-05 15:10:18 +02:00
dextmorgn
a24551ed55 ci: provide REDIS_URL, also import-time required by flowsint_core 2026-06-05 14:43:37 +02:00
dextmorgn
17e36c9162 ci: provide AUTH_SECRET required at flowsint_core import 2026-06-05 14:32:12 +02:00
dextmorgn
07584ebc4a ci: run python tests on pull requests and main
Single entrypoint: CI calls `make test`, same as local development,
so the two can't drift. uv handles Python (.python-version) and
dependency sync; cache keyed on uv.lock.

Also add flowsint-api to `make test` — its suite existed but was
never wired in.
2026-06-05 14:24:34 +02:00
dextmorgn
db9f50d914 Merge pull request #175 from reconurge/fix/prebuilt-images-compose
fix(deploy): serve prod from pre-built images
2026-06-05 14:11:29 +02:00
dextmorgn
82925bec38 fix(deploy): serve prod from pre-built images
Frontend baked VITE_API_URL into the bundle at build time, so
published images pointed at whatever URL CI had — "fail to fetch"
for every other deployment. Bundle now defaults to same-origin
relative URLs and the frontend nginx proxies /api/ to the api
service over the Docker network; CORS becomes irrelevant.

- docker-compose.prod.yml: use ghcr images (FLOWSINT_VERSION,
  default latest), drop build sections; bind postgres/redis/neo4j/
  api ports to 127.0.0.1 — only 5173 faces the network
- nginx: /api/ reverse proxy, request-time DNS resolution, SSE
  support (proxy_buffering off)
- Makefile: prod pulls instead of building; remove dead deploy
  targets referencing nonexistent docker-compose.deploy.yml
- README: network/server deployment section (secrets checklist,
  version pinning, TLS via reverse proxy)
2026-06-05 14:04:16 +02:00
dextmorgn
f38d5d500d Merge pull request #174 from reconurge/chore/security-dependency-upgrades
fix(deps): upgrade Python dependencies to patch 27 known vulnerabilities
2026-06-05 13:21:03 +02:00
dextmorgn
9fcc60ca63 fix(deps): upgrade Python dependencies to patch 27 known vulnerabilities
pip-audit reported 27 vulnerabilities across 12 packages, including
cryptography PYSEC-2026-36 (CVSS 9.8) used by the vault, a starlette
Host-header path bypass, and python-multipart arbitrary file write.

- cryptography 45.0.7 -> 46.0.7 (pin was capped at <46)
- fastapi 0.115 -> 0.136, pulling starlette 0.46 -> 1.2
- python-multipart 0.0.20 -> 0.0.32
- pyjwt, urllib3, aiohttp, lxml, idna, mako and transitive bumps
- dev tools: pytest 8 -> 9 (requires pytest-httpx >=0.36), black 25 -> 26

All four module test suites pass (458 tests). pip-audit now reports
no known vulnerabilities.
2026-06-05 08:57:19 +02:00
dextmorgn
5cdfe727b4 Merge pull request #172 from melihdaskiran/fix/zip-and-vault-bugs
Fix zip alignment mismatch, phone validation return value, and vault …
2026-06-05 08:24:39 +02:00
Melih Daskiran
25eb3a0400 Fix zip alignment mismatch, phone validation return value, and vault owner_id type matching 2026-06-04 21:46:44 +03:00
dextmorgn
259019f23c fix(build): force LF on container files via .gitattributes
Windows checkouts with core.autocrlf=true convert entrypoint.sh to
CRLF, which Docker copies into the image. The shebang then reads
#!/bin/sh\r and exec fails with 'no such file or directory'.
Also covers .env.example (trailing \r in env values) and Makefile.
2026-06-04 18:23:53 +02:00
dextmorgn
7c082d733e docs(readme): make Windows install commands cmd-compatible
PowerShell foreach/Test-Path loop fails when pasted into cmd.exe.
Plain copy commands work in both shells.
2026-06-04 18:05:19 +02:00
dextmorgn
de93ae3059 docs(readme): add Windows install path without Make
Windows users hit failures with make-based install. Replicate
check-env + build + up targets as plain PowerShell/docker compose
commands for prod and dev.
2026-06-04 18:00:35 +02:00
dextmorgn
c6eaf08b66 Merge pull request #163 from Mubashirrrr/fix/json-import-apostrophe-corruption
fix(import): preserve apostrophes in valid JSON imports
2026-06-03 20:34:42 +02:00
dextmorgn
f380674955 Merge pull request #165 from reconurge/fix/sse-auth-header
fix(sse): auth header
2026-06-03 13:42:17 +02:00
Mubashir Rahim
1fdbdbd094 fix(import): preserve apostrophes in valid JSON imports
parse_json unconditionally replaced every single quote with a double
quote (`.replace("'", '"')`) before calling json.loads. This corrupted
any valid JSON string value containing an apostrophe, e.g. a person's
surname "Sarah O'Brien" became "Sarah O"Brien", causing the parse to
fail with "Invalid JSON" and the whole import to be rejected.

Parse strict JSON first so well-formed payloads (including apostrophes
in string values) import correctly, and only fall back to the lenient
single-quote replacement for Python-dict-style payloads that are not
valid JSON on their own.

Adds regression tests covering an apostrophe-bearing JSON value and the
retained single-quoted fallback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 14:34:22 +05:00
dextmorgn
b317e08e57 feat(api): authenticate SSE via header, drop query-param auth and dead route 2026-06-03 11:28:41 +02:00
dextmorgn
9db1e675d6 test(api): SSE auth-gate and endpoint-removal tests (red) 2026-06-03 11:22:58 +02:00
dextmorgn
4a75d6a924 refactor(app): stream graph status via connectSSE (token out of URL) 2026-06-03 11:08:11 +02:00
dextmorgn
d7f21e9384 refactor(app): stream logs via connectSSE (token out of URL) 2026-06-03 10:56:32 +02:00
dextmorgn
0b60c81361 Merge pull request #155 from Ghraven/fix-investigation-aware-timestamp
fix: use aware timestamp for investigation updates
2026-06-03 09:30:35 +02:00
dextmorgn
6e5f26aaf0 feat(app): add connectSSE helper with header auth and backoff 2026-06-03 09:28:49 +02:00
Ghraven
60cfa16b8d fix: use aware timestamp for investigation updates 2026-06-03 08:37:45 +08:00
dextmorgn
078ac77eee build(app): add fetch-event-source + vitest test runner 2026-06-02 22:45:36 +02:00
dextmorgn
ed8bfc5248 Merge pull request #149 from Ghraven/fix/timezone-aware-service-timestamps
Use timezone-aware UTC timestamps in core services
2026-06-02 21:11:45 +02:00
dextmorgn
438bbe18c4 Merge pull request #150 from Ghraven/fix-social-enricher-output-encoding
fix: read social enricher outputs as utf-8
2026-06-02 21:10:12 +02:00
Ghraven
4e14df6c61 fix: read social enricher outputs as utf-8 2026-06-01 00:32:48 +08:00
dextmorgn
249374e6d7 chore: update .gitignore 2026-05-31 14:53:34 +02:00
dextmorgn
a15ee54af0 chore: update skill name 2026-05-31 14:52:37 +02:00
Ghraven
b707961e68 fix: use timezone-aware service timestamps 2026-05-31 02:35:11 +08:00
dextmorgn
cc829704e5 Merge pull request #148 from reconurge/fix/phase1-critical
fix: security issues
2026-05-30 19:47:55 +02:00
dextmorgn
de4f4ea87f Update README with new image links
Added additional image links to the README.
2026-05-28 00:00:13 +02:00
84 changed files with 3384 additions and 1874 deletions

View File

@@ -1,9 +1,9 @@
---
name: flowsint-transform-builder
name: flowsint-enricher-builder
description: Expert guidance for building Flowsint enrichers and their supporting types. Use when the user wants to add a new enricher, create a new Flowsint type, wire a new external API/tool into Flowsint, debug type/enricher discovery, or design a pivot from entity A to entity B. Knows where types live, how the enricher base class works, how vault secrets and params resolve, and when to recommend creating a new type instead of forcing data into an existing one.
---
# Flowsint Transform Builder
# Flowsint Enricher Builder
You build enrichers and types for Flowsint. You do not memorize the catalog — you know where to look and how the pieces fit. Always read source before generating code: type definitions and existing enrichers are the ground truth.
@@ -29,7 +29,7 @@ Read these first. Never assume signatures or fields — open the file.
## The first question: new type or reuse?
When the user describes a transform, decide before writing code:
When the user describes a enricher, decide before writing code:
1. **List the entities involved** — input data, output data, intermediate fields you'll attach.
2. **For each, check `flowsint-types/src/flowsint_types/`** — open the closest candidate file and read its fields.

View File

@@ -5,7 +5,9 @@ MASTER_VAULT_KEY_V1=base64:qnHTmwYb+uoygIw9MsRMY22vS5YPchY+QOi/E79GAvM=
NEO4J_URI_BOLT=bolt://neo4j:7687
NEO4J_USERNAME=neo4j
NEO4J_PASSWORD=password
VITE_API_URL=http://127.0.0.1:5001
# Dev only (vite dev server / docker-compose.yml). Production images use
# same-origin relative URLs proxied by nginx — leave unset for docker-compose.prod.yml.
VITE_API_URL=http://localhost:5001
# Comma-separated CORS allowed origins. Defaults to http://localhost:5173 (Vite dev) when unset.
ALLOWED_ORIGINS=http://localhost:5173
DATABASE_URL=postgresql://flowsint:flowsint@localhost:5433/flowsint

5
.gitattributes vendored Normal file
View File

@@ -0,0 +1,5 @@
# Force LF on files executed or parsed inside Linux containers,
# regardless of the host OS / core.autocrlf setting.
*.sh text eol=lf
.env.example text eol=lf
Makefile text eol=lf

37
.github/workflows/tests.yml vendored Normal file
View File

@@ -0,0 +1,37 @@
name: Tests
on:
pull_request:
push:
branches:
- main
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
test:
name: Python tests
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install uv
uses: astral-sh/setup-uv@v7
with:
enable-cache: true
# Same entrypoint as local development — keeps CI and `make test` in sync.
- name: Run tests
run: make test
env:
# flowsint_core hard-requires these at import time (provided by .env
# locally). Dummy values — no service is reached during tests:
# connections (redis.from_url, create_engine) are lazy.
AUTH_SECRET: ci-only-dummy-secret
REDIS_URL: redis://127.0.0.1:6379/0

3
.gitignore vendored
View File

@@ -17,6 +17,7 @@ eggs/
.eggs/
lib/
lib64/
!flowsint-app/src/lib/
parts/
sdist/
var/
@@ -178,4 +179,4 @@ cython_debug/
.claude/*
!.claude/skills/
.claude/skills/*
!.claude/skills/flowsint-transform-builder/
!.claude/skills/flowsint-enricher-builder/

View File

@@ -1,4 +1 @@
#!/usr/bin/env sh
. "$(dirname -- "$0")/_/husky.sh"
npx --no -- commitlint --edit ${1}

View File

@@ -2,18 +2,17 @@ PROJECT_ROOT := $(shell pwd)
COMPOSE_DEV := docker compose -f docker-compose.dev.yml
COMPOSE_PROD := docker compose -f docker-compose.prod.yml
COMPOSE_DEPLOY := docker compose -f docker-compose.deploy.yml
.PHONY: \
dev prod deploy \
build-dev build-prod \
up-dev up-prod up-deploy down \
dev prod \
build-dev \
up-dev up-prod down \
infra-dev infra-prod infra-stop-dev infra-stop-prod \
migrate-dev migrate-prod \
alembic-upgrade alembic-downgrade alembic-revision \
api frontend celery \
test install clean check-env open-browser-dev open-browser-prod \
logs-dev logs-prod logs-deploy status \
logs-dev logs-prod status \
regenerate-router
ENV_DIRS := . flowsint-api flowsint-core flowsint-app
@@ -63,18 +62,14 @@ open-browser-dev:
echo "Frontend ready at http://localhost:5173"
prod:
@echo "Starting PROD environment..."
@echo "Starting PROD environment (pre-built images)..."
$(MAKE) check-env
$(MAKE) build-prod
$(COMPOSE_PROD) pull
$(MAKE) up-prod
@echo ""
@echo "Production started!"
@echo " Frontend: http://localhost:5173"
@echo " API: http://localhost:5001"
build-prod:
@echo "Building PROD images..."
$(COMPOSE_PROD) build
@echo " API: http://localhost:5173/api (proxied)"
up-prod:
$(COMPOSE_PROD) up -d
@@ -91,27 +86,11 @@ logs-prod:
$(COMPOSE_PROD) logs -f
open-browser-prod:
@echo "Waiting for frontend on port 80 (Traefik)..."
@bash -c 'until curl -s http://localhost > /dev/null 2>&1; do sleep 2; done'
@open http://localhost 2>/dev/null || \
xdg-open http://localhost 2>/dev/null || \
echo "Frontend ready at http://localhost"
deploy:
@echo "Starting DEPLOY environment (GHCR images)..."
$(MAKE) check-env
$(COMPOSE_DEPLOY) pull
$(COMPOSE_DEPLOY) up -d
@echo ""
@echo "Deploy started!"
@echo " Frontend: http://localhost (via Traefik)"
@echo " API: http://localhost/api"
up-deploy:
$(COMPOSE_DEPLOY) up -d
logs-deploy:
$(COMPOSE_DEPLOY) logs -f
@echo "Waiting for frontend on port 5173..."
@bash -c 'until curl -s http://localhost:5173 > /dev/null 2>&1; do sleep 2; done'
@open http://localhost:5173 2>/dev/null || \
xdg-open http://localhost:5173 2>/dev/null || \
echo "Frontend ready at http://localhost:5173"
migrate-dev:
@echo "Running DEV migrations..."
@@ -161,6 +140,7 @@ test:
cd flowsint-types && uv run pytest
cd flowsint-core && uv run pytest
cd flowsint-enrichers && uv run pytest
cd flowsint-api && uv run pytest
install:
$(MAKE) infra-dev
@@ -177,7 +157,6 @@ status:
down:
-$(COMPOSE_DEV) down
-$(COMPOSE_PROD) down
-$(COMPOSE_DEPLOY) down
clean:
@echo "This will remove ALL Docker data. Continue? [y/N]"
@@ -185,7 +164,6 @@ clean:
if [ "$$confirm" != "y" ]; then exit 1; fi
-$(COMPOSE_DEV) down -v --rmi all --remove-orphans
-$(COMPOSE_PROD) down -v --rmi all --remove-orphans
-$(COMPOSE_DEPLOY) down -v --rmi all --remove-orphans
rm -rf flowsint-app/node_modules
rm -rf .venv
@@ -203,17 +181,11 @@ help:
@echo " make logs-dev - Follow DEV logs"
@echo " make infra-dev - Start only infra (postgres/redis/neo4j)"
@echo ""
@echo "Production (local build):"
@echo " make prod - Start PROD environment (local build + Traefik)"
@echo " make build-prod - Build PROD images"
@echo "Production (pre-built GHCR images):"
@echo " make prod - Pull images and start PROD environment"
@echo " make up-prod - Start PROD containers"
@echo " make logs-prod - Follow PROD logs"
@echo ""
@echo "Deploy (GHCR images):"
@echo " make deploy - Start with GHCR images (no build)"
@echo " make up-deploy - Start DEPLOY containers"
@echo " make logs-deploy - Follow DEPLOY logs"
@echo ""
@echo "Local (no Docker):"
@echo " make api - Run API locally"
@echo " make frontend - Run frontend locally"

View File

@@ -12,8 +12,16 @@ Flowsint is an open-source OSINT graph exploration tool designed for ethical inv
**Ethics:** Please read [ETHICS.md](./ETHICS.md) for responsible use guidelines.
<img width="1439" height="899" alt="hero-dark" src="https://github.com/user-attachments/assets/01eb128e-bef4-486e-9276-c4da58f829ae" />
<img width="1511" height="946" alt="Capture décran 2026-01-13 à 09 15 58" src="https://github.com/user-attachments/assets/d1a9eca6-9ec4-4402-93f4-303c3dc30de1" />
<img width="1511" height="948" alt="Capture décran 2026-01-13 à 09 19 45" src="https://github.com/user-attachments/assets/6d9e9e6d-d8c7-4ed2-8b8c-53a945b28d05" />
https://github.com/user-attachments/assets/eaabfa81-d7b3-414d-8cf7-f69b4e37bab6
https://github.com/user-attachments/assets/7457d94a-cf1d-4a97-949f-f9b1d8d92644
https://github.com/user-attachments/assets/65c3f26e-7132-4853-be45-21b8933688bd
## Contributing
@@ -23,6 +31,8 @@ Flowsint is still in early development and definetly needs the help of the commu
Don't want to read ? Got it. Here's your install instructions:
### Linux / macOS
#### 1. Install pre-requisites
- Docker
@@ -36,11 +46,78 @@ cd flowsint
make prod
```
### Windows
No Make needed. Works in both **Command Prompt (cmd)** and **PowerShell**.
#### 1. Install pre-requisites
- [Docker Desktop](https://docs.docker.com/desktop/setup/install/windows-install/) (make sure it is **running** before the next step)
- Git
#### 2. Clone and set up environment files
```bat
git clone https://github.com/reconurge/flowsint.git
cd flowsint
copy .env.example .env
copy .env.example flowsint-api\.env
copy .env.example flowsint-core\.env
copy .env.example flowsint-app\.env
```
#### 3. Start
```bat
docker compose -f docker-compose.prod.yml up -d
```
This pulls the pre-built images from GitHub Container Registry — no local build needed.
### First login
Then go to [http://localhost:5173/register](http://localhost:5173/register) and create an account. There are no credentials or account by default.
> ✅ OSINT investigations need a high level of privacy. Everything is stored on your machine.
### Deploy on a network (team / server)
The same setup works out of the box on a server: the frontend serves the UI **and** proxies all API calls internally, so no extra configuration is needed for clients.
```bash
git clone https://github.com/reconurge/flowsint.git
cd flowsint
cp .env.example .env
# Edit .env — see "Before exposing to a network" below
docker compose -f docker-compose.prod.yml up -d
```
Anyone on the network can then access Flowsint at `http://<server-ip>:5173`.
**Before exposing to a network, change the default secrets in `.env`:**
- `AUTH_SECRET` — signs authentication tokens. Generate one: `openssl rand -hex 32`
- `MASTER_VAULT_KEY_V1` — encrypts stored API keys. Generate one: `python3 -c "import os, base64; print('base64:' + base64.b64encode(os.urandom(32)).decode())"`
- `NEO4J_PASSWORD` — Neo4j database password.
**Also add your server hostname/IP to the Host-header allowlist** in `flowsint-app/nginx.conf` (look for `map $http_host $is_flowsint_host`). The default allowlist accepts only `localhost` / `127.0.0.1` / `[::1]`, which defends single-user installs against DNS rebinding; LAN and public deploys must opt-in their own hostname. Two commented-out template lines in the same map block show the format.
Only port `5173` is exposed to the network. PostgreSQL, Redis, Neo4j and the API are bound to `127.0.0.1` on the server and reachable only through the frontend proxy.
To pin a specific version instead of `latest`, set `FLOWSINT_VERSION` in `.env` (e.g. `FLOWSINT_VERSION=1.2.10`).
**HTTPS (recommended beyond a trusted LAN):** put any reverse proxy in front of port 5173. Example with [Caddy](https://caddyserver.com/):
```
flowsint.example.com {
reverse_proxy 127.0.0.1:5173
}
```
When fronting with a reverse proxy, also bind the app port to localhost in `docker-compose.prod.yml` (`"127.0.0.1:5173:8080"`) so clients can only go through HTTPS.
## What is it?
@@ -136,12 +213,19 @@ flowsint-types (types)
### Run
Make sure you have **Make** installed.
**Linux / macOS** (requires **Make**):
```bash
make dev
```
**Windows** (cmd or PowerShell, no Make — create the `.env` files first, see [Get started](#windows)):
```bat
docker compose -f docker-compose.dev.yml up -d --build
docker compose -f docker-compose.dev.yml logs -f
```
### Development
The app is accessible at [http://localhost:5173](http://localhost:5173).

View File

@@ -10,7 +10,8 @@ services:
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-flowsint}
POSTGRES_DB: ${POSTGRES_DB:-flowsint}
ports:
- "5433:5432"
# Bound to localhost: host-only debugging access, never exposed to the network.
- "127.0.0.1:5433:5432"
volumes:
- pg_data_prod:/var/lib/postgresql/data
networks:
@@ -26,7 +27,7 @@ services:
container_name: flowsint-redis-prod
restart: always
ports:
- "6379:6379"
- "127.0.0.1:6379:6379"
networks:
- flowsint_network
healthcheck:
@@ -40,8 +41,8 @@ services:
container_name: flowsint-neo4j-prod
restart: always
ports:
- "7474:7474"
- "7687:7687"
- "127.0.0.1:7474:7474"
- "127.0.0.1:7687:7687"
environment:
- NEO4J_AUTH=${NEO4J_USERNAME}/${NEO4J_PASSWORD}
- NEO4J_PLUGINS=["apoc"]
@@ -62,14 +63,13 @@ services:
retries: 10
api:
build:
context: .
dockerfile: flowsint-api/Dockerfile
target: production
image: ghcr.io/reconurge/flowsint-api:${FLOWSINT_VERSION:-latest}
container_name: flowsint-api-prod
restart: always
ports:
- "5001:5001"
# The frontend's nginx proxies /api/ to this service over the Docker
# network — direct access is host-only, for debugging.
- "127.0.0.1:5001:5001"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
@@ -96,10 +96,7 @@ services:
- flowsint_network
celery:
build:
context: .
dockerfile: flowsint-api/Dockerfile
target: production
image: ghcr.io/reconurge/flowsint-api:${FLOWSINT_VERSION:-latest}
container_name: flowsint-celery-prod
restart: always
command:
@@ -144,15 +141,17 @@ services:
- flowsint_network
app:
build:
context: ./flowsint-app
dockerfile: Dockerfile
args:
- VITE_API_URL=${VITE_API_URL}
# Frontend bundle uses same-origin relative URLs; nginx inside the image
# proxies /api/ to the "api" service. No VITE_API_URL needed.
image: ghcr.io/reconurge/flowsint-app:${FLOWSINT_VERSION:-latest}
container_name: flowsint-app-prod
restart: always
ports:
# The only network-facing port. Serves the UI and proxies /api/ to the API.
- "5173:8080"
volumes:
# Override the baked-in nginx.conf to use system resolver (Podman-compatible).
- ./flowsint-app/nginx.conf:/etc/nginx/nginx.conf:ro
networks:
- flowsint_network
depends_on:

View File

@@ -0,0 +1,41 @@
"""add_category_to_custom_types
Revision ID: f4d42260273d
Revises: a1f2b3c4d5e6
Create Date: 2026-06-06 14:47:31.692222
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = "f4d42260273d"
down_revision: Union[str, None] = "a1f2b3c4d5e6"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
"""Upgrade schema."""
# ### commands auto generated by Alembic - please adjust! ###
op.add_column(
"custom_types",
sa.Column(
"category",
sa.String(),
server_default="custom_types_category",
nullable=False,
),
)
# ### end Alembic commands ###
def downgrade() -> None:
"""Downgrade schema."""
# ### commands auto generated by Alembic - please adjust! ###
op.drop_column("custom_types", "category")
# ### end Alembic commands ###

View File

@@ -1,11 +1,10 @@
from fastapi import Depends, HTTPException, status, Request
from fastapi import Depends, HTTPException, status
from fastapi.security import OAuth2PasswordBearer
from jose import JWTError, jwt
from sqlalchemy.orm import Session
from flowsint_core.core.auth import ALGORITHM, AUTH_SECRET
from flowsint_core.core.postgre_db import get_db
from flowsint_core.core.models import Profile
from typing import Optional
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="token")
@@ -29,42 +28,3 @@ def get_current_user(
if user is None:
raise credentials_exception
return user
def get_current_user_sse(
request: Request, db: Session = Depends(get_db)
) -> Profile:
"""
Alternative authentication for SSE endpoints that accepts token via query parameter.
EventSource API doesn't support custom headers, so we need to pass the token in the URL.
"""
credentials_exception = HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Could not validate credentials",
)
# Try to get token from query parameter
token: Optional[str] = request.query_params.get("token")
# Fallback to Authorization header if query param not present
if not token:
auth_header = request.headers.get("Authorization")
if auth_header and auth_header.startswith("Bearer "):
token = auth_header.replace("Bearer ", "")
if not token:
raise credentials_exception
try:
payload = jwt.decode(token, AUTH_SECRET, algorithms=[ALGORITHM])
email: str = payload.get("sub")
if email is None:
raise credentials_exception
except JWTError:
raise credentials_exception
user = db.query(Profile).filter(Profile.email == email).first()
if user is None:
raise credentials_exception
return user

View File

@@ -46,6 +46,7 @@ def create_custom_type(
user_id=current_user.id,
description=custom_type.description,
status=custom_type.status,
category=custom_type.category,
validate_schema_func=validate_json_schema,
calculate_checksum_func=calculate_schema_checksum,
)
@@ -116,6 +117,7 @@ def update_custom_type(
json_schema=update_data.json_schema,
description=update_data.description,
status=update_data.status,
category=update_data.category,
validate_schema_func=validate_json_schema,
calculate_checksum_func=calculate_schema_checksum,
)

View File

@@ -14,7 +14,7 @@ from flowsint_core.core.services import (
PermissionDeniedError,
DatabaseError,
)
from app.api.deps import get_current_user, get_current_user_sse
from app.api.deps import get_current_user
router = APIRouter()
@@ -42,7 +42,7 @@ async def stream_events(
request: Request,
sketch_id: str,
db: Session = Depends(get_db),
current_user: Profile = Depends(get_current_user_sse),
current_user: Profile = Depends(get_current_user),
):
"""Stream events for a specific sketch in real-time."""
service = create_log_service(db)
@@ -58,7 +58,7 @@ async def stream_events(
channel = sketch_id
await event_emitter.subscribe(channel)
try:
yield 'data: {"event": "connected", "data": "Connected to log stream"}\n\n'
yield json.dumps({"event": "connected", "data": "Connected to log stream"})
while True:
if await request.is_disconnected():
break
@@ -115,7 +115,7 @@ async def stream_sketch_status(
request: Request,
sketch_id: str,
db: Session = Depends(get_db),
current_user: Profile = Depends(get_current_user_sse),
current_user: Profile = Depends(get_current_user),
):
"""Stream COMPLETED events for a specific sketch (for graph refresh)."""
service = create_log_service(db)
@@ -160,49 +160,3 @@ async def stream_sketch_status(
"X-Accel-Buffering": "no",
},
)
@router.get("/status/scan/{scan_id}/stream")
async def stream_status(
request: Request,
scan_id: str,
db: Session = Depends(get_db),
current_user: Profile = Depends(get_current_user_sse),
):
"""Stream status updates for a specific scan in real-time."""
service = create_log_service(db)
try:
service.get_scan_with_permission(scan_id, current_user.id)
except NotFoundError as e:
raise HTTPException(status_code=404, detail=str(e))
except PermissionDeniedError:
raise HTTPException(status_code=403, detail="Forbidden")
async def status_generator():
print("[EventEmitter] Start status generator")
await event_emitter.subscribe(f"scan_{scan_id}_status")
try:
yield 'data: {"event": "connected", "data": "Connected to status stream"}\n\n'
while True:
data = await event_emitter.get_message(f"scan_{scan_id}_status")
if data is None:
await asyncio.sleep(0.1)
continue
print(f"[EventEmitter] Received status data: {data}")
yield f"data: {data}\n\n"
except asyncio.CancelledError:
print(f"[EventEmitter] Client disconnected from status stream for scan_id: {scan_id}")
finally:
await event_emitter.unsubscribe(f"scan_{scan_id}_status")
return EventSourceResponse(
status_generator(),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"Connection": "keep-alive",
"X-Accel-Buffering": "no",
},
)

View File

@@ -19,6 +19,9 @@ class CustomTypeCreate(BaseModel):
None, description="Optional description of the custom type"
)
status: str = Field("draft", description="Status of the custom type")
category: str = Field(
"custom_types_category", description="Category of the custom type"
)
color: str = Field("#8E9E8C", description="Default color")
icon: str = Field("Minus", description="Default icon")
@@ -40,6 +43,7 @@ class CustomTypeUpdate(BaseModel):
json_schema: Optional[Dict[str, Any]] = Field(None, alias="schema")
description: Optional[str] = None
status: Optional[str] = None
category: Optional[str] = None
color: Optional[str] = None
icon: Optional[str] = None
@@ -64,6 +68,7 @@ class CustomTypeRead(ORMBase):
icon: Optional[str]
json_schema: Dict[str, Any] = Field(..., alias="schema")
status: str
category: str
checksum: Optional[str]
description: Optional[str]
created_at: datetime

View File

@@ -101,7 +101,7 @@ def is_root_domain(domain: str) -> bool:
return False
def is_valid_number(phone: str, region: str = "FR") -> None:
def is_valid_number(phone: str, region: str = "FR") -> bool:
"""
Validates a phone number. Raises InvalidPhoneNumberError if invalid.
- `region` should be ISO 3166-1 alpha-2 country code (e.g., 'FR' for France)
@@ -113,6 +113,8 @@ def is_valid_number(phone: str, region: str = "FR") -> None:
except NumberParseException:
return False
return True
def parse_asn(asn: str) -> int:
if not is_valid_asn(asn):

View File

@@ -9,7 +9,7 @@ dependencies = [
"flowsint-core",
"flowsint-types",
"flowsint-enrichers",
"fastapi[standard]>=0.115.0,<0.116.0",
"fastapi[standard]>=0.115.0,<1.0.0",
"uvicorn>=0.32.0,<0.33.0",
"redis>=5.0,<6.0",
"celery>=5.3,<6.0",
@@ -28,14 +28,14 @@ dependencies = [
"networkx>=2.6.3,<3.0.0",
"email-validator>=2.2.0,<3.0.0",
"mistralai>=1.9.3,<2.0.0",
"python-multipart>=0.0.20,<0.0.21",
"python-multipart>=0.0.27,<0.1.0",
"openpyxl>=3.1.2,<4.0.0",
"jsonschema>=4.25.1,<5.0.0",
]
[dependency-groups]
dev = [
"black>=25.0,<26.0",
"black>=26.3.1,<27.0",
"isort>=6.0,<7.0",
"flake8>=7.0,<8.0",
"mypy>=1.17,<2.0",

View File

@@ -0,0 +1,38 @@
"""Pytest harness for flowsint-api. Provides a TestClient backed by an
in-memory SQLite database, overriding the real Postgres get_db dependency."""
import os
# Env required at import time by flowsint-core modules. Set before importing app.
os.environ.setdefault("AUTH_SECRET", "test-secret-please-ignore")
os.environ.setdefault(
"MASTER_VAULT_KEY_V1", "base64:qnHTmwYb+uoygIw9MsRMY22vS5YPchY+QOi/E79GAvM="
)
import pytest
from fastapi.testclient import TestClient
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from flowsint_core.core.models import Base
from flowsint_core.core.postgre_db import get_db
from app.main import app
@pytest.fixture
def db_session():
engine = create_engine(
"sqlite:///:memory:", connect_args={"check_same_thread": False}
)
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
yield session
session.close()
@pytest.fixture
def client(db_session):
app.dependency_overrides[get_db] = lambda: db_session
yield TestClient(app)
app.dependency_overrides.clear()

View File

@@ -0,0 +1,50 @@
"""Auth and shape contract for the SSE event endpoints."""
import importlib
import pytest
from flowsint_core.core.auth import create_access_token
from flowsint_core.core.models import Profile
def test_get_current_user_sse_is_removed():
"""The query-param SSE auth helper must no longer exist."""
deps = importlib.import_module("app.api.deps")
assert not hasattr(deps, "get_current_user_sse")
def test_log_stream_requires_auth_header(client):
"""No Authorization header -> 401 (no token in URL accepted)."""
res = client.get("/api/events/sketch/abc/stream")
assert res.status_code == 401
def test_status_stream_requires_auth_header(client):
res = client.get("/api/events/sketch/abc/status/stream")
assert res.status_code == 401
def test_log_stream_rejects_token_query_param(client):
"""A token in the URL must NOT authenticate the request anymore."""
token = create_access_token({"sub": "user@example.com"})
res = client.get(f"/api/events/sketch/abc/stream?token={token}")
assert res.status_code == 401
def test_dead_scan_stream_endpoint_removed(client):
"""The unused scan status stream endpoint must be gone."""
res = client.get("/api/events/status/scan/abc/stream")
assert res.status_code == 404
def test_get_current_user_accepts_valid_bearer(db_session):
"""The dependency the streams now use authenticates via a valid JWT."""
from app.api.deps import get_current_user
db_session.add(Profile(email="user@example.com", hashed_password="x"))
db_session.commit()
token = create_access_token({"sub": "user@example.com"})
user = get_current_user(token=token, db=db_session)
assert user.email == "user@example.com"

View File

@@ -10,13 +10,14 @@ RUN yarn install --network-timeout 100000
COPY . .
# Build argument for API URL (injected at build time)
ARG VITE_API_URL
# API URL baked at build time. Empty = same-origin relative URLs (/api/...),
# served through the nginx reverse proxy below. Only override for an external API origin.
ARG VITE_API_URL=""
ENV VITE_API_URL=${VITE_API_URL}
RUN yarn build
FROM nginx:1.27-alpine AS production
FROM nginx:1.29-alpine AS production
LABEL org.opencontainers.image.source="https://github.com/reconurge/flowsint"
LABEL org.opencontainers.image.description="Flowsint Frontend"

View File

@@ -45,18 +45,63 @@ http {
# Security
server_tokens off;
# Host-header allowlist — defends against DNS rebinding.
# The CORSMiddleware on the API blocks cross-origin reads, but after a
# DNS rebinding flip the browser treats the request as same-origin and
# skips the CORS check entirely, so a server-side Host check is the
# canonical defense. Without it, attacker JS on evil.com can drive a
# victim's browser to POST /api/auth/register against a flowsint instance
# bound to localhost or a LAN IP, planting attacker-controlled accounts.
#
# Default allowlist covers single-user local deploys (localhost / 127.0.0.1
# / ::1, any port). When deploying on a network (see README "Deploy on a
# network"), add the server hostname/IP below — and add the public
# hostname too when fronting with a reverse proxy like Caddy.
map $http_host $is_flowsint_host {
default 0;
"~*^localhost(:[0-9]+)?$" 1;
"~*^127\.0\.0\.1(:[0-9]+)?$" 1;
"~*^\[::1\](:[0-9]+)?$" 1;
# "~*^flowsint\.example\.com$" 1;
# "~*^192\.168\.1\.42(:[0-9]+)?$" 1;
}
server {
listen 8080;
server_name _;
root /usr/share/nginx/html;
index index.html;
# Default-deny unknown Host headers (DNS rebinding defense).
# See $is_flowsint_host map above for the allowlist.
if ($is_flowsint_host = 0) {
return 403;
}
# Security headers
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header X-XSS-Protection "1; mode=block" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
# API reverse proxy — same-origin, no CORS needed.
# Expects the API service to be reachable as "api:5001" on the compose network.
# Uses system resolver (/etc/resolv.conf) — compatible with Docker and Podman.
# Trade-off: hostname resolved once at startup; restart app if api IP changes.
location /api/ {
proxy_pass http://api:5001;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# SSE support (event/chat streams)
proxy_set_header Connection "";
proxy_buffering off;
proxy_cache off;
proxy_read_timeout 3600s;
}
# Health check endpoint
location /health {
access_log off;
@@ -68,6 +113,13 @@ http {
location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot)$ {
expires 1y;
add_header Cache-Control "public, immutable";
# Re-declare server-level security headers — nginx's add_header
# inheritance is replace-all, not merge, so a location block that
# sets any add_header drops every header set higher up.
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header X-XSS-Protection "1; mode=block" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
access_log off;
}
@@ -80,6 +132,12 @@ http {
location = /index.html {
expires -1;
add_header Cache-Control "no-store, no-cache, must-revalidate";
# Re-declare server-level security headers (see static-assets
# location above for why).
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header X-XSS-Protection "1; mode=block" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
}
# Deny access to hidden files

View File

@@ -11,6 +11,7 @@
"format": "prettier --write .",
"lint": "eslint . --ext .js,.jsx,.cjs,.mjs,.ts,.tsx,.cts,.mts --fix",
"typecheck": "tsc --noEmit",
"test": "vitest run",
"dev": "vite",
"build": "vite build",
"preview": "vite preview"
@@ -19,6 +20,7 @@
"@ai-sdk/react": "^3.0.79",
"@dagrejs/dagre": "^1.1.4",
"@hookform/resolvers": "^5.0.1",
"@microsoft/fetch-event-source": "^2.0.1",
"@monaco-editor/react": "^4.7.0",
"@radix-ui/react-accordion": "^1.2.11",
"@radix-ui/react-alert-dialog": "^1.1.14",
@@ -86,7 +88,7 @@
"d3": "^7.9.0",
"d3-force": "^3.0.0",
"date-fns": "^3.6.0",
"dompurify": "^3.3.0",
"dompurify": "^3.4.11",
"embla-carousel-react": "^8.6.0",
"framer-motion": "^12.0.6",
"input-otp": "^1.4.2",
@@ -139,6 +141,7 @@
"react-dom": "^19.2.0",
"standard-version": "^9.5.0",
"typescript": "^5.5.2",
"vite": "^5.3.1"
"vite": "^5.3.1",
"vitest": "^2.1.9"
}
}

View File

@@ -1,6 +1,6 @@
import { useAuthStore } from '@/stores/auth-store'
const API_URL = import.meta.env.VITE_API_URL
const API_URL = import.meta.env.VITE_API_URL ?? ''
export async function fetchWithAuth(endpoint: string, options: RequestInit = {}): Promise<any> {
const token = useAuthStore.getState().token

View File

@@ -1,7 +1,7 @@
import { DefaultChatTransport } from 'ai'
import { useAuthStore } from '@/stores/auth-store'
const API_URL = import.meta.env.VITE_API_URL
const API_URL = import.meta.env.VITE_API_URL ?? ''
export function createChatTransport(chatId: string) {
return new DefaultChatTransport({

View File

@@ -6,6 +6,7 @@ export interface CustomType {
owner_id: string
schema: Record<string, any>
status: 'draft' | 'published' | 'archived'
category: string
checksum?: string
description?: string
icon?: string
@@ -21,6 +22,7 @@ export interface CustomTypeCreate {
icon?: string
color?: string
status?: 'draft' | 'published'
category?: string
}
export interface CustomTypeUpdate {
@@ -30,6 +32,7 @@ export interface CustomTypeUpdate {
icon?: string
color?: string
status?: 'draft' | 'published' | 'archived'
category?: string
}
export interface ValidatePayload {

View File

@@ -0,0 +1,117 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
// Capture the options passed to fetchEventSource so we can drive its callbacks.
const calls: any[] = []
const fetchEventSourceMock = vi.fn((url: string, opts: any) => {
calls.push({ url, opts })
return new Promise(() => {}) // never resolves; we drive callbacks manually
})
vi.mock('@microsoft/fetch-event-source', () => ({
fetchEventSource: (url: string, opts: any) => fetchEventSourceMock(url, opts),
}))
vi.mock('@/stores/auth-store', () => ({
useAuthStore: { getState: () => ({ token: 'tok123' }) },
}))
import { connectSSE } from './sse'
const lastOpts = () => calls[calls.length - 1].opts
const fakeResponse = (status: number, contentType: string) =>
({ ok: status >= 200 && status < 300, status, headers: { get: () => contentType } } as any)
beforeEach(() => {
calls.length = 0
fetchEventSourceMock.mockClear()
})
describe('connectSSE', () => {
it('passes the URL and Authorization header from the store', () => {
connectSSE({ url: 'http://x/stream', onMessage: () => {} })
expect(calls[0].url).toBe('http://x/stream')
expect(lastOpts().headers).toEqual({ Authorization: 'Bearer tok123' })
})
it('forwards parsed envelopes and skips "connected"', () => {
const received: any[] = []
connectSSE({ url: 'http://x', onMessage: (m) => received.push(m) })
const o = lastOpts()
o.onmessage({ data: JSON.stringify({ event: 'connected', data: 'hi' }) })
o.onmessage({ data: JSON.stringify({ event: 'log', data: '{"msg":"a"}' }) })
expect(received).toEqual([{ event: 'log', data: '{"msg":"a"}' }])
})
it('ignores empty and malformed messages without throwing', () => {
const received: any[] = []
connectSSE({ url: 'http://x', onMessage: (m) => received.push(m) })
const o = lastOpts()
o.onmessage({ data: '' })
o.onmessage({ data: 'not-json' })
expect(received).toEqual([])
})
it('aborts the request when the disposer is called', () => {
const dispose = connectSSE({ url: 'http://x', onMessage: () => {} })
expect(lastOpts().signal.aborted).toBe(false)
dispose()
expect(lastOpts().signal.aborted).toBe(true)
})
it('throws (no retry) on a non-event-stream open', async () => {
connectSSE({ url: 'http://x', onMessage: () => {} })
await expect(lastOpts().onopen(fakeResponse(401, 'application/json'))).rejects.toThrow()
})
it('accepts a valid event-stream open', async () => {
connectSSE({ url: 'http://x', onMessage: () => {} })
await expect(
lastOpts().onopen(fakeResponse(200, 'text/event-stream'))
).resolves.toBeUndefined()
})
it('returns increasing backoff for transient errors', () => {
connectSSE({ url: 'http://x', onMessage: () => {} })
const o = lastOpts()
const first = o.onerror(new Error('network'))
const second = o.onerror(new Error('network'))
expect(first).toBe(2000)
expect(second).toBe(4000)
})
it('rethrows fatal errors from onerror to stop retrying', async () => {
connectSSE({ url: 'http://x', onMessage: () => {} })
const o = lastOpts()
let fatal: unknown
try {
await o.onopen(fakeResponse(403, 'application/json'))
} catch (e) {
fatal = e
}
expect(() => o.onerror(fatal)).toThrow()
})
it('ignores valid JSON that is not an envelope', () => {
const received: any[] = []
connectSSE({ url: 'http://x', onMessage: (m) => received.push(m) })
const o = lastOpts()
o.onmessage({ data: '42' })
o.onmessage({ data: 'null' })
o.onmessage({ data: '{"data":"no-event"}' })
expect(received).toEqual([])
})
it('calls onError once when the connection promise rejects (non-fatal)', async () => {
const errors: unknown[] = []
fetchEventSourceMock.mockImplementationOnce((url: string, opts: any) => {
calls.push({ url, opts })
return Promise.reject(new Error('boom'))
})
connectSSE({ url: 'http://x', onMessage: () => {}, onError: (e) => errors.push(e) })
await Promise.resolve()
await Promise.resolve()
expect(errors).toHaveLength(1)
expect((errors[0] as Error).message).toBe('boom')
})
})

View File

@@ -0,0 +1,72 @@
import { fetchEventSource } from '@microsoft/fetch-event-source'
import { useAuthStore } from '@/stores/auth-store'
/**
* Outer SSE envelope from the server. `data` is forwarded exactly as received;
* for the log/status streams it is itself a JSON-encoded string that the
* consumer parses (e.g. `JSON.parse(envelope.data)`).
*/
export type SSEEnvelope = { event: string; data: unknown }
/** Thrown to tell fetch-event-source to stop retrying (e.g. auth failure). */
class FatalSSEError extends Error {}
const MAX_BACKOFF_MS = 30000
/**
* Open an authenticated SSE connection.
* Reads the bearer token once at connect time and sends it via the
* Authorization header. Returns a disposer that aborts the connection.
*/
export function connectSSE(opts: {
url: string
onMessage: (msg: SSEEnvelope) => void
onError?: (err: unknown) => void
}): () => void {
const controller = new AbortController()
let retry = 0
const token = useAuthStore.getState().token
const headers: Record<string, string> = token
? { Authorization: `Bearer ${token}` }
: {}
fetchEventSource(opts.url, {
signal: controller.signal,
openWhenHidden: true,
headers,
onopen: async (res) => {
const contentType = res.headers.get('content-type') ?? ''
if (!res.ok || !contentType.includes('text/event-stream')) {
throw new FatalSSEError(`SSE open failed: ${res.status}`)
}
retry = 0
},
onmessage: (ev) => {
if (!ev.data) return
let envelope: SSEEnvelope
try {
envelope = JSON.parse(ev.data) as SSEEnvelope
} catch (error) {
console.error('[connectSSE] malformed envelope:', error, ev.data)
return
}
if (typeof envelope?.event !== 'string') return
if (envelope.event === 'connected') return
opts.onMessage(envelope)
},
onerror: (err) => {
if (err instanceof FatalSSEError) {
opts.onError?.(err)
throw err // stop: no retry on auth/protocol failure
}
// transient failure: retry silently with exponential backoff (onError is not called)
retry += 1
return Math.min(1000 * 2 ** retry, MAX_BACKOFF_MS)
},
}).catch((err) => {
if (!(err instanceof FatalSSEError)) opts.onError?.(err)
})
return () => controller.abort()
}

View File

@@ -9,9 +9,18 @@ interface TypePreviewProps {
color: string
fields: SchemaField[]
status: 'draft' | 'published'
category: string
}
export function TypePreview({ name, description, icon, color, fields, status }: TypePreviewProps) {
export function TypePreview({
name,
description,
icon,
color,
fields,
status,
category
}: TypePreviewProps) {
const Icon = (LucideIcons as any)[icon] || LucideIcons.FileQuestion
const validFields = fields.filter((f) => f.key.trim())

View File

@@ -30,7 +30,7 @@ export function SketchesSection({ sketches, canCreate = true }: SketchesSectionP
</div>
{isEmpty ? (
canCreate ? <EmptySketches onAction={() => console.log("Create sketch")} /> : <div className="text-sm text-muted-foreground py-4">No sketches yet.</div>
canCreate ? <EmptySketches /> : <div className="text-sm text-muted-foreground py-4">No sketches yet.</div>
) : (
<div style={{ containerType: 'inline-size' }}>
<div className="grid grid-cols-1 cq-sm:grid-cols-2 cq-md:grid-cols-3 cq-lg:grid-cols-4 gap-3">

View File

@@ -19,8 +19,6 @@ export default function RawMaterial() {
})
const [searchTerm, setSearchTerm] = useState<string>('')
console.log(materials)
const filteredEnrichers = useMemo(() => {
if (!materials?.items) return {}
const result: Record<string, Enricher[]> = {}

View File

@@ -16,6 +16,7 @@ import {
BadgeAlert,
Plus,
Download,
FileJson,
Trash2
} from 'lucide-react'
import { Enricher, Flow, GraphNode } from '@/types'
@@ -423,10 +424,47 @@ type SubActionsProps = {
selectedNodes: GraphNode[]
}
const SubActions = memo(({ selectedNodes }: SubActionsProps) => {
const contentToCopy = useMemo(() => selectedNodes.map((node) => node.nodeLabel).join('\n'), [])
const contentToCopy = useMemo(
() => selectedNodes.map((node) => node.nodeLabel).join('\n'),
[selectedNodes]
)
const edges = useGraphStore((s) => s.edges)
const selectedEdges = useMemo(() => {
const selectedNodeIds = new Set(selectedNodes.map((node) => node.id))
return edges.filter(
(edge) => selectedNodeIds.has(edge.source) && selectedNodeIds.has(edge.target)
)
}, [edges, selectedNodes])
const handleExportJson = useCallback(() => {
const payload = {
exported_at: new Date().toISOString(),
nodes: selectedNodes,
edges: selectedEdges
}
const blob = new Blob([JSON.stringify(payload, null, 2)], { type: 'application/json' })
const href = URL.createObjectURL(blob)
const link = document.createElement('a')
link.href = href
link.download = `flowsint-selection-${new Date().toISOString().replace(/[:.]/g, '-')}.json`
document.body.appendChild(link)
link.click()
link.remove()
URL.revokeObjectURL(href)
}, [selectedEdges, selectedNodes])
return (
<div>
<div className="flex flex-col gap-1">
<CopyButton label={`Copy ${selectedNodes.length} items as txt`} content={contentToCopy} />
<Button
variant="ghost"
size="sm"
className="w-full justify-start gap-2"
onClick={handleExportJson}
>
<FileJson className="h-4 w-4 opacity-60" />
Export {selectedNodes.length} node(s), {selectedEdges.length} edge(s) as JSON
</Button>
</div>
)
})

View File

@@ -103,9 +103,11 @@ const drawClippedImage = (
drawHeight = diameter / imgAspect
}
ctx.save()
drawNodePath(ctx, x, y, size, shape)
ctx.clip()
ctx.drawImage(img, x - drawWidth / 2, y - drawHeight / 2, drawWidth, drawHeight)
ctx.restore()
}
// --- Node visual resolution ---

View File

@@ -260,7 +260,6 @@ export function TemplateEditor({ templateId, initialContent, importedYaml }: Tem
duration: response.duration_ms,
url: response.url
})
console.log(response)
} catch (error) {
setTestResult({
success: false,

View File

@@ -3,8 +3,9 @@ import { useQuery } from '@tanstack/react-query'
import { logService } from '@/api/log-service'
import { queryKeys } from '@/api/query-keys'
import { useAuthStore } from '@/stores/auth-store'
import { connectSSE } from '@/api/sse'
const API_URL = import.meta.env.VITE_API_URL
const API_URL = import.meta.env.VITE_API_URL ?? ''
export function useEvents(sketch_id: string | undefined) {
@@ -30,45 +31,21 @@ export function useEvents(sketch_id: string | undefined) {
useEffect(() => {
if (!sketch_id || !token) return
const eventSource = new EventSource(
`${API_URL}/api/events/sketch/${sketch_id}/stream?token=${token}`
)
eventSource.onmessage = (e) => {
try {
// Handle malformed SSE data (connection message has extra "data: " prefix)
let dataStr = e.data
if (dataStr.startsWith('data: ')) {
dataStr = dataStr.substring(6) // Remove "data: " prefix
}
const raw = JSON.parse(dataStr) as any
// Ignore connection messages
if (raw.event === 'connected') {
return
}
const dispose = connectSSE({
url: `${API_URL}/api/events/sketch/${sketch_id}/stream`,
onMessage: (raw) => {
// Only process log events
if (raw.event !== 'log') {
return
if (raw.event !== 'log') return
try {
const event = JSON.parse(raw.data as string) as Event
setLiveLogs((prev) => [...prev.slice(-99), event])
} catch (error) {
console.error('[useSketchEvents] Failed to parse log payload:', error, raw.data)
}
},
})
const event = JSON.parse(raw.data) as Event
setLiveLogs((prev) => [...prev.slice(-99), event])
} catch (error) {
console.error('[useSketchEvents] Failed to parse SSE event:', error, e.data)
}
}
eventSource.onerror = (error) => {
console.error('[useSketchEvents] EventSource error:', error)
eventSource.close()
}
return () => {
eventSource.close()
}
return dispose
}, [sketch_id, token])
const logs = useMemo(

View File

@@ -2,8 +2,9 @@ import { useEffect, useRef } from 'react'
import { useGraphControls } from '@/stores/graph-controls-store'
import { useAuthStore } from '@/stores/auth-store'
import { EventLevel } from '@/types'
import { connectSSE } from '@/api/sse'
const API_URL = import.meta.env.VITE_API_URL
const API_URL = import.meta.env.VITE_API_URL ?? ''
export function useGraphRefresh(sketch_id: string | undefined) {
const refetchGraph = useGraphControls((s) => s.refetchGraph)
@@ -25,55 +26,35 @@ export function useGraphRefresh(sketch_id: string | undefined) {
useEffect(() => {
if (!sketch_id || !token) return
const eventSource = new EventSource(
`${API_URL}/api/events/sketch/${sketch_id}/status/stream?token=${token}`
)
eventSource.onmessage = (e) => {
try {
// Handle malformed SSE data (connection message has extra "data: " prefix)
let dataStr = e.data
if (dataStr.startsWith('data: ')) {
dataStr = dataStr.substring(6) // Remove "data: " prefix
}
const raw = JSON.parse(dataStr) as any
// Ignore connection messages
if (raw.event === 'connected') {
return
}
const dispose = connectSSE({
url: `${API_URL}/api/events/sketch/${sketch_id}/status/stream`,
onMessage: (raw) => {
// Only process status events
if (raw.event !== 'status') {
return
}
const event = JSON.parse(raw.data) as any
// Only handle COMPLETED events
if (event.type === EventLevel.COMPLETED) {
const refetch = refetchGraphRef.current
const regenerate = regenerateLayoutRef.current
const layoutType = currentLayoutTypeRef.current
if (raw.event !== 'status') return
try {
const event = JSON.parse(raw.data as string) as any
// Only handle COMPLETED events
if (event.type === EventLevel.COMPLETED) {
const refetch = refetchGraphRef.current
const regenerate = regenerateLayoutRef.current
const layoutType = currentLayoutTypeRef.current
if (typeof refetch !== 'function') {
return
if (typeof refetch !== 'function') return
// Refetch graph data, then regenerate layout if one is active
refetch(() => {
if (layoutType && typeof regenerate === 'function') {
regenerate(layoutType)
}
})
}
// Refetch graph data with callback to regenerate layout after
refetch(() => {
// After refetch completes, regenerate layout if we have one active
if (layoutType && typeof regenerate === 'function') {
regenerate(layoutType)
}
})
} catch (error) {
console.error('[useGraphRefresh] Failed to parse status payload:', error, raw.data)
}
} catch (error) {
console.error('[useGraphRefresh] Failed to parse SSE event:', error, e.data)
}
}
},
})
eventSource.onerror = () => {
eventSource.close()
}
return () => {
eventSource.close()
}
}, [sketch_id, token]) // Only reconnect when sketch_id or token changes
return dispose
}, [sketch_id, token])
}

View File

@@ -0,0 +1,44 @@
import { describe, it, expect } from 'vitest'
import { getListOfAllChildrenFromNodes } from './tools'
import { GraphNode } from '@/types'
const makeNode = (id: string, overrides?: Partial<GraphNode>): GraphNode => ({
id,
nodeType: 'test',
nodeLabel: 'Test Node',
nodeProperties: {},
nodeSize: 1,
nodeColor: null,
nodeIcon: null,
nodeImage: null,
nodeFlag: null,
nodeShape: null,
nodeMetadata: {},
x: 0,
y: 0,
...overrides,
})
describe('getListOfAllChildrenFromNodes', () => {
it('should handle empty array', () => {
expect(getListOfAllChildrenFromNodes([])).toEqual(undefined)
})
it('should handle single node without children', () => {
expect(getListOfAllChildrenFromNodes([makeNode('node-1')])).toEqual(undefined)
})
it('should handle multiple nodes', () => {
expect(getListOfAllChildrenFromNodes([makeNode('node-1'), makeNode('node-2')])).toEqual(undefined)
})
it('should handle nodes with neighbors', () => {
const nodes = [makeNode('node-1', { neighbors: [{ id: 'node-2' }, { id: 'node-3' }] })]
expect(getListOfAllChildrenFromNodes(nodes)).toEqual(undefined)
})
it('should handle nodes with links', () => {
const nodes = [makeNode('node-1', { links: [{ target: 'node-2' }, { target: 'node-3' }] })]
expect(getListOfAllChildrenFromNodes(nodes)).toEqual(undefined)
})
})

View File

@@ -0,0 +1,3 @@
import { GraphNode } from '@/types'
export function getListOfAllChildrenFromNodes(_nodes: GraphNode[]) {}

View File

@@ -22,6 +22,7 @@ import { Skeleton } from '@/components/ui/skeleton'
import { useNodesDisplaySettings } from '@/stores/node-display-settings'
import { clearIconTypeCache } from '@/components/sketches/graph/utils/image-cache'
import { useKeyboardShortcut } from '@/hooks/use-keyboard-shortcut'
import { useActionItems } from '@/hooks/use-action-items'
export const Route = createFileRoute('/_auth/dashboard/custom-types/$typeId')({
component: CustomTypeEditor
@@ -40,11 +41,15 @@ function CustomTypeEditor() {
const [name, setName] = useState('')
const [description, setDescription] = useState('')
const [status, setStatus] = useState<'draft' | 'published'>('draft')
const [category, setCategory] = useState<string>('custom_types_category')
const [icon, setIcon] = useState(DEFAULT_ICON)
const [color, setColor] = useState(DEFAULT_COLOR)
const [fields, setFields] = useState<SchemaField[]>([])
const [showPreview, setShowPreview] = useState(true)
// Hooks
const { actionItems, isLoading: actionLoading } = useActionItems()
// Load existing type if editing
const { data: existingType, isLoading } = useQuery<CustomType>({
queryKey: ['custom-type', id],
@@ -57,6 +62,7 @@ function CustomTypeEditor() {
setName(existingType.name)
setDescription(existingType.description || '')
setStatus(existingType.status === 'archived' ? 'draft' : existingType.status)
setCategory(existingType.category || 'custom_types_category')
setIcon(existingType.icon || DEFAULT_ICON)
setColor(existingType.color || DEFAULT_COLOR)
parseSchemaToFields(existingType.schema)
@@ -179,7 +185,8 @@ function CustomTypeEditor() {
schema: fieldsToSchema(),
icon,
color,
status
status,
category
}
if (isNew) {
@@ -287,27 +294,45 @@ function CustomTypeEditor() {
/>
</div>
</div>
<div className="flex items-center gap-3">
<span className="text-xs text-muted-foreground/60">Status</span>
<Select value={status} onValueChange={(v: any) => setStatus(v)}>
<SelectTrigger className="w-[130px] h-7 text-xs border-border/40 bg-transparent">
<SelectValue />
</SelectTrigger>
<SelectContent>
<SelectItem value="draft">
<div className="flex items-center gap-2">
<div className="w-1.5 h-1.5 rounded-full bg-muted-foreground/40" />
Draft
</div>
</SelectItem>
<SelectItem value="published">
<div className="flex items-center gap-2">
<div className="w-1.5 h-1.5 rounded-full bg-green-500" />
Published
</div>
</SelectItem>
</SelectContent>
</Select>
<div className="flex flex-col gap-3 items-end">
<div className="flex items-center gap-3">
<span className="text-xs text-muted-foreground/60">Status</span>
<Select value={status} onValueChange={(v: any) => setStatus(v)}>
<SelectTrigger className="w-[130px] h-7 text-xs border-border/40 bg-transparent">
<SelectValue />
</SelectTrigger>
<SelectContent>
<SelectItem value="draft">
<div className="flex items-center gap-2">
<div className="w-1.5 h-1.5 rounded-full bg-muted-foreground/40" />
Draft
</div>
</SelectItem>
<SelectItem value="published">
<div className="flex items-center gap-2">
<div className="w-1.5 h-1.5 rounded-full bg-green-500" />
Published
</div>
</SelectItem>
</SelectContent>
</Select>
</div>
<div className="flex items-center gap-3">
<span className="text-xs text-muted-foreground/60">Category</span>
<Select value={category} onValueChange={(v: any) => setCategory(v)}>
<SelectTrigger className="w-[130px] h-7 text-xs border-border/40 bg-transparent">
<SelectValue />
</SelectTrigger>
<SelectContent>
{!actionLoading &&
actionItems.map((item) => (
<SelectItem value={item.type}>
<div className="flex items-center gap-2">{item.label}</div>
</SelectItem>
))}
</SelectContent>
</Select>
</div>
</div>
</div>
<Separator className="opacity-40" />
@@ -400,6 +425,7 @@ function CustomTypeEditor() {
color={color}
fields={fields}
status={status}
category={category}
/>
</div>
</div>

View File

@@ -5,7 +5,6 @@ import { keyService } from '../api/key-service'
import { Button } from '../components/ui/button'
import { Input } from '../components/ui/input'
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from '../components/ui/card'
import { Badge } from '../components/ui/badge'
import {
Dialog,
DialogContent,
@@ -24,7 +23,7 @@ import {
TableHeader,
TableRow
} from '../components/ui/table'
import { Loader2, Plus, Trash2, Clock, Shield, Key, Zap, Lock, Sparkles } from 'lucide-react'
import { Loader2, Plus, Trash2, KeyRound } from 'lucide-react'
import { toast } from 'sonner'
import { useConfirm } from '../components/use-confirm-dialog'
import Loader from '@/components/loader'
@@ -175,63 +174,29 @@ function VaultPage() {
>
<div className="w-full">
{keys.length === 0 ? (
<Card className="border-2 border-dashed border-primary/20 bg-gradient-to-br from-primary/5 via-background to-accent/5">
<CardContent className="flex flex-col items-center justify-center py-20 px-8">
<div className="text-center space-y-4 max-w-md">
<h3 className="text-2xl font-bold bg-gradient-to-r from-foreground to-foreground/70 bg-clip-text text-transparent">
Your Vault is Empty
</h3>
<p className="text-muted-foreground text-lg leading-relaxed">
Add your first API key to unlock the power of third-party services in your
investigations.
</p>
<div className="flex flex-wrap justify-center gap-3 pt-4 pb-6">
<Badge
variant="secondary"
className="bg-primary/10 text-primary border-primary/20 px-4 py-2"
>
<Zap className="w-4 h-4 mr-2" />
Fast Setup
</Badge>
<Badge
variant="secondary"
className="bg-emerald-500/10 text-emerald-700 border-emerald-200/40 px-4 py-2"
>
<Lock className="w-4 h-4 mr-2" />
Encrypted
</Badge>
<Badge
variant="secondary"
className="bg-violet-500/10 text-violet-700 border-violet-200/40 px-4 py-2"
>
<Sparkles className="w-4 h-4 mr-2" />
Secure
</Badge>
</div>
<Button onClick={() => setIsAddDialogOpen(true)}>
<Plus className="w-4 h-4 mr-2" />
Add Your First Key
</Button>
</div>
</CardContent>
</Card>
<div className="flex flex-col items-center justify-center py-24 gap-5">
<KeyRound className="w-12 h-12 text-muted-foreground/40" strokeWidth={1.5} />
<div className="text-center space-y-2">
<h3 className="text-xl font-bold text-foreground">No keys yet</h3>
<p className="text-muted-foreground max-w-xs leading-relaxed">
Add your first API key to use third-party services in your investigations.
</p>
</div>
<Button onClick={() => setIsAddDialogOpen(true)}>
<Plus className="w-4 h-4 mr-2" />
Add your first key
</Button>
</div>
) : (
<Card className="overflow-hidden border bg-gradient-to-br from-background to-muted/20">
<Card className="overflow-hidden">
<CardHeader className="border-b">
<div className="flex items-center justify-between">
<CardTitle className="flex items-center gap-3 text-xl">
<div className="p-2 bg-primary rounded-lg">
<Shield className="w-5 h-5 text-white" strokeWidth={1.9} />
</div>
API Keys
</CardTitle>
<Badge variant="secondary" className="px-3 py-1">
<CardTitle className="text-base font-semibold">API Keys</CardTitle>
<span className="text-xs text-muted-foreground">
{keys.length} {keys.length === 1 ? 'key' : 'keys'}
</Badge>
</span>
</div>
<CardDescription className="text-base mt-2">
<CardDescription>
Your encrypted API keys for external services. These keys will be available for your
investigations.
</CardDescription>
@@ -239,58 +204,24 @@ function VaultPage() {
<CardContent className="p-0">
<Table>
<TableHeader>
<TableRow className="border-b bg-muted/30">
<TableHead className="py-4 px-6 text-sm font-semibold w-2/5">
<div className="flex items-center gap-2">
<Key className="w-4 h-4" />
Name
</div>
</TableHead>
<TableHead className="py-4 text-sm font-semibold w-1/3">
<div className="flex items-center gap-2">
<Clock className="w-4 h-4" />
Created
</div>
</TableHead>
<TableHead className="py-4 px-6 text-sm font-semibold text-right w-1/5">
Actions
</TableHead>
<TableRow className="border-b bg-muted/20">
<TableHead className="py-3 px-6 w-2/5">Name</TableHead>
<TableHead className="py-3 w-1/3">Added</TableHead>
<TableHead className="py-3 px-6 text-right w-1/5">Actions</TableHead>
</TableRow>
</TableHeader>
<TableBody>
{keys.map((key: KeyType) => (
<TableRow
key={key.id}
className="group hover:bg-gradient-to-r hover:from-primary/5 hover:to-accent/5 transition-all duration-200 border-b border-border/50"
>
<TableCell className="py-5 px-6">
<div className="flex items-center gap-4">
<div className="p-2 bg-gradient-to-r from-primary/10 to-accent/10 border border-primary/20 rounded-lg group-hover:scale-110 transition-transform duration-200">
<Key className="w-4 h-4 text-primary" />
</div>
<div>
<div className="font-semibold text-foreground group-hover:text-primary transition-colors">
{key.name}
</div>
<div className="text-sm text-muted-foreground">Encrypted & Secure</div>
</div>
</div>
<TableRow key={key.id} className="hover:bg-muted/30 border-b border-border/50">
<TableCell className="py-4 px-6 font-medium">{key.name}</TableCell>
<TableCell className="py-4 text-sm text-muted-foreground">
{new Date(key.created_at).toLocaleDateString('en-US', {
year: 'numeric',
month: 'short',
day: 'numeric'
})}
</TableCell>
<TableCell className="py-5">
<div className="flex items-center gap-2 text-sm">
<div className="p-1.5 bg-muted rounded-full">
<Clock className="w-3 h-3 text-muted-foreground" />
</div>
<span className="text-muted-foreground">
{new Date(key.created_at).toLocaleDateString('en-US', {
year: 'numeric',
month: 'short',
day: 'numeric'
})}
</span>
</div>
</TableCell>
<TableCell className="text-right py-5 px-6">
<TableCell className="text-right py-4 px-6">
<Button
variant="ghost"
size="icon"

View File

@@ -0,0 +1,13 @@
import { defineConfig } from 'vitest/config'
import { resolve } from 'path'
export default defineConfig({
resolve: {
alias: {
'@': resolve(__dirname, 'src'),
},
},
test: {
environment: 'node',
},
})

View File

@@ -14,6 +14,7 @@ dependencies = [
"redis>=5.0,<6.0",
"celery>=5.3,<6.0",
"python-dotenv>=1.0,<2.0",
"pyyaml>=6.0,<7.0",
"requests>=2.31,<3.0",
"httpx>=0.28,<0.29",
"networkx>=2.6.3,<3.0.0",
@@ -23,18 +24,18 @@ dependencies = [
"sse-starlette>=1.8,<2.0",
"alembic==1.13.0",
"phonenumbers>=9.0.8,<10.0.0",
"python-multipart>=0.0.20,<0.0.21",
"python-multipart>=0.0.27,<0.1.0",
"docker>=7.1.0,<8.0.0",
"pytest>=8.4.2,<9.0.0",
"cryptography>=45.0.7,<46.0.0",
"pytest>=9.0.3,<10.0.0",
"cryptography>=48.0.1,<49.0.0",
"openpyxl>=3.1,<4.0",
]
[dependency-groups]
dev = [
"pytest-asyncio>=0.21,<0.22",
"pytest-httpx>=0.35,<0.36",
"black>=25.0,<26.0",
"pytest-asyncio>=0.21,<2.0",
"pytest-httpx>=0.36,<1.0",
"black>=26.3.1,<27.0",
"isort>=6.0,<7.0",
"flake8>=7.0,<8.0",
"mypy>=1.17,<2.0",

View File

@@ -1,5 +1,5 @@
from passlib.context import CryptContext
from datetime import datetime, timedelta
from datetime import datetime, timedelta, timezone
from jose import jwt
import os
from dotenv import load_dotenv
@@ -29,7 +29,7 @@ def get_password_hash(password: str) -> str:
def create_access_token(data: dict, expires_delta: timedelta | None = None):
to_encode = data.copy()
expire = datetime.utcnow() + (
expire = datetime.now(timezone.utc) + (
expires_delta or timedelta(minutes=ACCESS_TOKEN_EXPIRE_MINUTES)
)
to_encode.update({"exp": expire})

View File

@@ -361,6 +361,9 @@ class CustomType(Base):
icon: Mapped[str] = mapped_column(String, nullable=True)
color: Mapped[str] = mapped_column(String, nullable=True)
status: Mapped[str] = mapped_column(String, server_default="draft", nullable=False)
category: Mapped[str] = mapped_column(
String, server_default="custom_types_category", nullable=False
)
checksum: Mapped[str] = mapped_column(String, nullable=True)
description: Mapped[str] = mapped_column(Text, nullable=True)
created_at = mapped_column(DateTime(timezone=True), server_default=func.now())

View File

@@ -1,5 +1,5 @@
"""Repository for Log model."""
from datetime import datetime, timedelta
from datetime import datetime, timedelta, timezone
from typing import List, Optional
from uuid import UUID
@@ -26,7 +26,7 @@ class LogRepository(BaseRepository[Log]):
query = query.filter(Log.created_at > since)
else:
query = query.filter(
Log.created_at > datetime.utcnow() - timedelta(days=1)
Log.created_at > datetime.now(timezone.utc) - timedelta(days=1)
)
return query.limit(limit).all()

View File

@@ -2,9 +2,9 @@
Analysis service for managing analyses within investigations.
"""
from datetime import datetime, timezone
from typing import Any, Dict, List, Optional
from uuid import UUID, uuid4
from datetime import datetime
from sqlalchemy.orm import Session
@@ -64,8 +64,8 @@ class AnalysisService(BaseService):
content=content,
owner_id=owner_id,
investigation_id=investigation_id,
created_at=datetime.utcnow(),
last_updated_at=datetime.utcnow(),
created_at=datetime.now(timezone.utc),
last_updated_at=datetime.now(timezone.utc),
)
self._analysis_repo.add(new_analysis)
self._commit()
@@ -97,7 +97,7 @@ class AnalysisService(BaseService):
self._check_permission(user_id, investigation_id, ["update"])
analysis.investigation_id = investigation_id
analysis.last_updated_at = datetime.utcnow()
analysis.last_updated_at = datetime.now(timezone.utc)
self._commit()
self._refresh(analysis)
return analysis

View File

@@ -4,7 +4,7 @@ Chat service for managing chats and messages with AI integration.
import json
import os
from datetime import datetime
from datetime import datetime, timezone
from typing import Any, AsyncIterator, Dict, List, Optional
from uuid import UUID, uuid4
@@ -76,8 +76,8 @@ class ChatService(BaseService):
description=description,
owner_id=owner_id,
investigation_id=investigation_id,
created_at=datetime.utcnow(),
last_updated_at=datetime.utcnow(),
created_at=datetime.now(timezone.utc),
last_updated_at=datetime.now(timezone.utc),
)
self._chat_repo.add(new_chat)
self._commit()
@@ -103,7 +103,7 @@ class ChatService(BaseService):
if not chat:
raise NotFoundError("Chat not found")
chat.last_updated_at = datetime.utcnow()
chat.last_updated_at = datetime.now(timezone.utc)
user_message = ChatMessage(
id=uuid4(),
@@ -111,7 +111,7 @@ class ChatService(BaseService):
context=context,
chat_id=chat_id,
is_bot=False,
created_at=datetime.utcnow(),
created_at=datetime.now(timezone.utc),
)
self._chat_repo.add_message(user_message)
self._commit()
@@ -124,7 +124,7 @@ class ChatService(BaseService):
content=content,
chat_id=chat_id,
is_bot=True,
created_at=datetime.utcnow(),
created_at=datetime.now(timezone.utc),
)
self._chat_repo.add_message(chat_message)
self._commit()

View File

@@ -48,6 +48,7 @@ class CustomTypeService(BaseService):
user_id: UUID,
description: Optional[str] = None,
status: str = "draft",
category: str = "custom_types_category",
validate_schema_func=None,
calculate_checksum_func=None,
) -> CustomType:
@@ -68,6 +69,7 @@ class CustomTypeService(BaseService):
schema=json_schema,
description=description,
status=status,
category=category,
checksum=checksum,
)
@@ -85,6 +87,7 @@ class CustomTypeService(BaseService):
json_schema: Optional[Dict[str, Any]] = None,
description: Optional[str] = None,
status: Optional[str] = None,
category: Optional[str] = None,
color: Optional[str] = None,
icon: Optional[str] = None,
validate_schema_func=None,
@@ -111,6 +114,9 @@ class CustomTypeService(BaseService):
if status is not None:
custom_type.status = status
if category is not None:
custom_type.category = category
if icon is not None:
custom_type.icon = icon

View File

@@ -2,7 +2,7 @@
Flow service for managing flows and flow computations.
"""
from datetime import datetime
from datetime import datetime, timezone
from typing import Any, Dict, List, Optional
from uuid import UUID, uuid4
@@ -82,8 +82,8 @@ class FlowService(BaseService):
description=description,
category=category,
flow_schema=flow_schema,
created_at=datetime.utcnow(),
last_updated_at=datetime.utcnow(),
created_at=datetime.now(timezone.utc),
last_updated_at=datetime.now(timezone.utc),
)
self._flow_repo.add(new_flow)
self._commit()
@@ -101,7 +101,7 @@ class FlowService(BaseService):
value.append("Username")
setattr(flow, key, value)
flow.last_updated_at = datetime.utcnow()
flow.last_updated_at = datetime.now(timezone.utc)
self._commit()
self._refresh(flow)
return flow

View File

@@ -4,7 +4,7 @@ Investigation service for managing investigations and user roles.
from typing import List, Optional
from uuid import UUID, uuid4
from datetime import datetime
from datetime import datetime, timezone
from sqlalchemy.orm import Session
@@ -109,7 +109,7 @@ class InvestigationService(BaseService):
investigation.name = name
investigation.description = description
investigation.status = status
investigation.last_updated_at = datetime.utcnow()
investigation.last_updated_at = datetime.now(timezone.utc)
self._commit()
self._refresh(investigation)

View File

@@ -200,29 +200,34 @@ class TypeRegistryService(BaseService):
else "value"
)
custom_types_children.append(
{
"id": custom_type.id,
"type": custom_type.name,
"key": custom_type.name.lower(),
"label_key": label_key,
"icon": custom_type.icon or "custom",
"color": custom_type.color,
"label": custom_type.name,
"description": custom_type.description or "",
"fields": [
{
"name": prop,
"label": info.get("title", prop),
"description": info.get("description", ""),
"type": "text",
"required": prop in required,
}
for prop, info in properties.items()
],
"custom": True,
}
)
custom_type_obj = {
"id": custom_type.id,
"type": custom_type.name,
"key": custom_type.name.lower(),
"label_key": label_key,
"icon": custom_type.icon or "custom",
"color": custom_type.color,
"label": custom_type.name,
"description": custom_type.description or "",
"fields": [
{
"name": prop,
"label": info.get("title", prop),
"description": info.get("description", ""),
"type": "text",
"required": prop in required,
}
for prop, info in properties.items()
],
"custom": True,
}
if custom_type.category == "custom_types_category":
custom_types_children.append(custom_type_obj)
else:
for t in types:
if t["type"] == custom_type.category:
t["children"].append(custom_type_obj)
types.append(
{
@@ -435,7 +440,11 @@ class TypeRegistryService(BaseService):
):
return True
# Direct array (non-optional)
if schema.get("type") == "array" and isinstance(schema.get("items"), dict) and schema["items"].get("type") == "string":
if (
schema.get("type") == "array"
and isinstance(schema.get("items"), dict)
and schema["items"].get("type") == "string"
):
return True
return False

View File

@@ -25,7 +25,7 @@ class Vault(VaultProtocol):
if not owner_id:
raise ValueError("owner_id is required to use the vault.")
self.db = db
self.owner_id = str(owner_id)
self.owner_id = owner_id
self.version = "V1"
def _get_master_key(self) -> bytes:
@@ -52,7 +52,7 @@ class Vault(VaultProtocol):
algorithm=hashes.SHA256(),
length=32,
salt=salt,
info=self.owner_id.encode("utf-8"),
info=str(self.owner_id).encode("utf-8"),
)
master_key = self._get_master_key()
return hkdf.derive(master_key)
@@ -72,7 +72,7 @@ class Vault(VaultProtocol):
ciphertext = aesgcm.encrypt(
iv,
plaintext.encode("utf-8"),
self.owner_id.encode("utf-8"), # AAD = links secret to user_id
str(self.owner_id).encode("utf-8"), # AAD = links secret to user_id
)
return {
@@ -92,7 +92,7 @@ class Vault(VaultProtocol):
plaintext = aesgcm.decrypt(
row.get("iv"),
row.get("ciphertext"),
self.owner_id.encode("utf-8"),
str(self.owner_id).encode("utf-8"),
)
return plaintext.decode("utf-8")

View File

@@ -23,8 +23,15 @@ def parse_json(
try:
file_bytes = file_bytes.lstrip()
entities: Dict[str, Entity] = {}
my_bytes_value = file_bytes.decode().replace("'", '"')
graph = json.loads(my_bytes_value)
decoded = file_bytes.decode()
# Parse strict JSON first so apostrophes inside string values (e.g.
# "Sarah O'Brien") are preserved. Only fall back to the lenient
# single-quote replacement for Python-dict-style payloads that are not
# valid JSON on their own.
try:
graph = json.loads(decoded)
except json.JSONDecodeError:
graph = json.loads(decoded.replace("'", '"'))
node_key = next((k for k in VALID_NODES_KEYS if k in graph), None)
edge_key = next((k for k in VALID_EDGES_KEYS if k in graph), None)
if node_key is None:

View File

@@ -109,7 +109,7 @@ def sanitize_url_component(value: str) -> str:
class YamlLoader:
@staticmethod
def load_enricher_yaml(filename: str) -> dict[str, Any] | yaml.YAMLError:
with open(filename) as stream:
with open(filename, encoding="utf-8") as stream:
try:
return yaml.safe_load(stream)
except yaml.YAMLError as exc:

View File

@@ -17,6 +17,12 @@ def setup_test_environment(monkeypatch):
# Set a test master key for vault tests
test_key = "base64:qnHTmwYb+uoygIw9MsRMY22vS5YPchY+QOi/E79GAvM="
monkeypatch.setenv("MASTER_VAULT_KEY_V1", test_key)
# Dummy Neo4j credentials: the Neo4jConnection singleton requires them
# at construction, but driver creation is lazy — nothing connects.
# Without these, tests silently depend on the developer's local .env.
monkeypatch.setenv("NEO4J_URI_BOLT", "bolt://127.0.0.1:7687")
monkeypatch.setenv("NEO4J_USERNAME", "neo4j")
monkeypatch.setenv("NEO4J_PASSWORD", "test-password")
@pytest.fixture(autouse=True)

View File

@@ -7,6 +7,17 @@ from datetime import datetime, timezone
from flowsint_core.core.graph import Neo4jGraphRepository
def repo_without_connection() -> Neo4jGraphRepository:
"""Repository with no underlying connection.
Constructed with a mock to avoid the constructor's singleton fallback,
which requires NEO4J_* credentials from the environment.
"""
repo = Neo4jGraphRepository(neo4j_connection=MagicMock())
repo._connection = None
return repo
class TestNeo4jGraphRepositoryInit:
def test_init_with_connection(self):
mock_connection = MagicMock()
@@ -43,8 +54,7 @@ class TestCreateNode:
mock_connection.query.assert_called_once()
def test_create_node_no_connection(self):
repo = Neo4jGraphRepository(neo4j_connection=None)
repo._connection = None
repo = repo_without_connection()
result = repo.create_node({"nodeLabel": "test", "nodeType": "domain"}, "sketch-1")
@@ -80,8 +90,7 @@ class TestCreateRelationship:
mock_connection.execute_write.assert_called_once()
def test_create_relationship_no_connection(self):
repo = Neo4jGraphRepository(neo4j_connection=None)
repo._connection = None
repo = repo_without_connection()
rel_obj = {
"from_type": "domain",
@@ -231,8 +240,7 @@ class TestBatchOperations:
mock_connection.execute_batch.assert_not_called()
def test_flush_batch_no_connection(self):
repo = Neo4jGraphRepository(neo4j_connection=None)
repo._connection = None
repo = repo_without_connection()
repo._batch_operations = [("query", {})]
repo.flush_batch()
@@ -285,8 +293,7 @@ class TestBatchCreateNodes:
assert result["errors"] == []
def test_batch_create_nodes_no_connection(self):
repo = Neo4jGraphRepository(neo4j_connection=None)
repo._connection = None
repo = repo_without_connection()
result = repo.batch_create_nodes(
[{"nodeLabel": "test", "nodeType": "domain"}], sketch_id="sketch-1"
@@ -338,8 +345,7 @@ class TestBatchCreateEdges:
assert result["errors"] == []
def test_batch_create_edges_no_connection(self):
repo = Neo4jGraphRepository(neo4j_connection=None)
repo._connection = None
repo = repo_without_connection()
result = repo.batch_create_edges([{}], sketch_id="sketch-1")
@@ -385,8 +391,7 @@ class TestBatchCreateEdgesByElementId:
assert any("Missing required fields" in e for e in result["errors"])
def test_batch_create_edges_by_element_id_no_connection(self):
repo = Neo4jGraphRepository(neo4j_connection=None)
repo._connection = None
repo = repo_without_connection()
result = repo.batch_create_edges_by_element_id([{}], sketch_id="sketch-1")
@@ -409,8 +414,7 @@ class TestUpdateNode:
assert result == "elem-1"
def test_update_node_no_connection(self):
repo = Neo4jGraphRepository(neo4j_connection=None)
repo._connection = None
repo = repo_without_connection()
result = repo.update_node("elem-1", {"nodeLabel": "x"}, "sketch-1")
@@ -428,8 +432,7 @@ class TestDeleteNodes:
assert result == 3
def test_delete_nodes_no_connection(self):
repo = Neo4jGraphRepository(neo4j_connection=None)
repo._connection = None
repo = repo_without_connection()
result = repo.delete_nodes(["id-1"], sketch_id="sketch-1")
@@ -456,8 +459,7 @@ class TestDeleteRelationships:
assert result == 2
def test_delete_relationships_no_connection(self):
repo = Neo4jGraphRepository(neo4j_connection=None)
repo._connection = None
repo = repo_without_connection()
result = repo.delete_relationships(["rel-1"], sketch_id="sketch-1")
@@ -483,8 +485,7 @@ class TestDeleteAllSketchNodes:
assert result == 10
def test_delete_all_sketch_nodes_no_connection(self):
repo = Neo4jGraphRepository(neo4j_connection=None)
repo._connection = None
repo = repo_without_connection()
result = repo.delete_all_sketch_nodes(sketch_id="sketch-1")
@@ -519,8 +520,7 @@ class TestGetSketchGraph:
assert len(result["edges"]) == 1
def test_get_sketch_graph_no_connection(self):
repo = Neo4jGraphRepository(neo4j_connection=None)
repo._connection = None
repo = repo_without_connection()
result = repo.get_sketch_graph(sketch_id="sketch-1")
@@ -553,8 +553,7 @@ class TestUpdateRelationship:
assert result["id"] == "rel-1"
def test_update_relationship_no_connection(self):
repo = Neo4jGraphRepository(neo4j_connection=None)
repo._connection = None
repo = repo_without_connection()
result = repo.update_relationship("rel-1", {}, "sketch-1")
@@ -577,8 +576,7 @@ class TestCreateRelationshipByElementId:
assert result["sketch_id"] == "sketch-1"
def test_create_relationship_by_element_id_no_connection(self):
repo = Neo4jGraphRepository(neo4j_connection=None)
repo._connection = None
repo = repo_without_connection()
result = repo.create_relationship_by_element_id(
"elem-1", "elem-2", "CONNECTS", "sketch-1"
@@ -598,8 +596,7 @@ class TestQuery:
assert result == [{"count": 5}]
def test_query_no_connection(self):
repo = Neo4jGraphRepository(neo4j_connection=None)
repo._connection = None
repo = repo_without_connection()
result = repo.query("MATCH (n) RETURN n", {})
@@ -622,8 +619,7 @@ class TestUpdateNodesPositions:
assert result == 2
def test_update_nodes_positions_no_connection(self):
repo = Neo4jGraphRepository(neo4j_connection=None)
repo._connection = None
repo = repo_without_connection()
result = repo.update_nodes_positions([{"nodeId": "x", "x": 0, "y": 0}], "s")
@@ -652,8 +648,7 @@ class TestGetNodesByIds:
assert len(result) == 2
def test_get_nodes_by_ids_no_connection(self):
repo = Neo4jGraphRepository(neo4j_connection=None)
repo._connection = None
repo = repo_without_connection()
result = repo.get_nodes_by_ids(["id-1"], sketch_id="sketch-1")
@@ -706,8 +701,7 @@ class TestMergeNodes:
assert result == "old-1"
def test_merge_nodes_no_connection(self):
repo = Neo4jGraphRepository(neo4j_connection=None)
repo._connection = None
repo = repo_without_connection()
result = repo.merge_nodes(["old-1"], {}, None, "sketch-1")
@@ -764,8 +758,7 @@ class TestGetNeighbors:
assert len(result["edges"]) == 0
def test_get_neighbors_no_connection(self):
repo = Neo4jGraphRepository(neo4j_connection=None)
repo._connection = None
repo = repo_without_connection()
result = repo.get_neighbors("node-1", "sketch-1")

View File

@@ -65,3 +65,40 @@ def test_standard_json_import_without_label():
"""Test basic standard JSON import."""
results = parse_json(standard_json_without_label, max_preview_rows=100)
assert "Username" in results.entities
# Valid JSON whose string values contain apostrophes (very common in OSINT
# data, e.g. surnames like "O'Brien"). This is well-formed JSON and must import.
json_with_apostrophe = b"""{
"nodes": [
{"id": "1", "label": "Sarah O'Brien", "type": "individual"},
{"id": "2", "label": "Bob", "type": "individual"}
],
"edges": [
{"source": "1", "target": "2", "label": "KNOWS"}
]
}"""
def test_json_import_preserves_apostrophes_in_string_values():
"""Valid JSON with an apostrophe in a value must parse and keep the value.
Regression: the parser used to blindly replace every single quote with a
double quote, which corrupted valid JSON (turning "Sarah O'Brien" into
"Sarah O"Brien") and raised "Invalid JSON".
"""
results = parse_json(json_with_apostrophe, max_preview_rows=100)
assert "Individual" in results.entities
labels = {
str(preview.obj.nodeLabel)
for preview in results.entities["Individual"].results
}
assert "Sarah O'Brien" in labels
def test_json_import_still_accepts_single_quoted_payload():
"""Python-dict-style payloads (single-quoted) remain supported as fallback."""
single_quoted = b"{'nodes': [{'id': '1', 'label': 'Alice', 'type': 'individual'}], 'edges': []}"
results = parse_json(single_quoted, max_preview_rows=100)
assert "Individual" in results.entities

View File

@@ -1,4 +1,4 @@
from flowsint_types import Domain, Ip, Phone
from flowsint_types import Domain, File, Ip, Phone
from flowsint_core.imports import detect_type
@@ -40,6 +40,20 @@ def test_detection_ips():
assert results[1].address == "12.43.23.12"
def test_detection_file_hashes():
hashes = {
"d41d8cd98f00b204e9800998ecf8427e": "hash_md5",
"da39a3ee5e6b4b0d3255bfef95601890afd80709": "hash_sha1",
"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855": "hash_sha256",
}
for value, field in hashes.items():
DetectedType = detect_type(value)
assert DetectedType is File, f"{value} should be detected as File"
obj = DetectedType.from_string(value)
assert isinstance(obj, File)
assert getattr(obj, field) == value
def test_detection_mixed():
raw_obj = ["240.123.123.234", "my.domain.io", "+33632233223"]
results = []

View File

@@ -0,0 +1,97 @@
"""Tests for timezone-aware service timestamps."""
from datetime import timezone
from unittest.mock import MagicMock
from uuid import uuid4
from flowsint_core.core.services.analysis_service import AnalysisService
from flowsint_core.core.services.chat_service import ChatService
from flowsint_core.core.services.flow_service import FlowService
from flowsint_core.core.services.investigation_service import InvestigationService
def _assert_utc(value):
assert value.tzinfo is not None
assert value.utcoffset() == timezone.utc.utcoffset(value)
def test_analysis_service_create_uses_timezone_aware_timestamps():
service = AnalysisService(
db=MagicMock(),
analysis_repo=MagicMock(),
investigation_repo=MagicMock(),
)
service._check_permission = MagicMock()
analysis = service.create(
title="Summary",
description=None,
content={},
investigation_id=uuid4(),
owner_id=uuid4(),
)
_assert_utc(analysis.created_at)
_assert_utc(analysis.last_updated_at)
def test_chat_service_create_uses_timezone_aware_timestamps():
service = ChatService(
db=MagicMock(),
chat_repo=MagicMock(),
vault_service=MagicMock(),
)
chat = service.create(
title="Investigation chat",
description=None,
investigation_id=uuid4(),
owner_id=uuid4(),
)
_assert_utc(chat.created_at)
_assert_utc(chat.last_updated_at)
def test_flow_service_create_uses_timezone_aware_timestamps():
service = FlowService(
db=MagicMock(),
flow_repo=MagicMock(),
custom_type_repo=MagicMock(),
sketch_repo=MagicMock(),
investigation_repo=MagicMock(),
)
flow = service.create(
name="Resolve domain",
description=None,
category=["Domain"],
flow_schema={"nodes": [], "edges": []},
)
_assert_utc(flow.created_at)
_assert_utc(flow.last_updated_at)
def test_investigation_service_update_uses_timezone_aware_timestamp():
investigation = MagicMock()
investigation_repo = MagicMock()
investigation_repo.get_by_id.return_value = investigation
service = InvestigationService(
db=MagicMock(),
investigation_repo=investigation_repo,
sketch_repo=MagicMock(),
analysis_repo=MagicMock(),
profile_repo=MagicMock(),
)
service._check_permission = MagicMock()
service.update(
investigation_id=uuid4(),
user_id=uuid4(),
name="Updated investigation",
description="Updated description",
status="active",
)
_assert_utc(investigation.last_updated_at)

View File

@@ -45,7 +45,7 @@ class TestVaultInitialization:
"""Test successful Vault initialization."""
vault = Vault(db=mock_db, owner_id=owner_id)
assert vault.db == mock_db
assert vault.owner_id == str(owner_id)
assert vault.owner_id == owner_id
assert vault.version == "V1"

View File

@@ -28,9 +28,9 @@ dependencies = [
[dependency-groups]
dev = [
"pytest>=8.4.2,<9.0.0",
"pytest-asyncio>=0.21,<0.22",
"black>=25.0,<26.0",
"pytest>=9.0.3,<10.0.0",
"pytest-asyncio>=0.21,<2.0",
"black>=26.3.1,<27.0",
"isort>=6.0,<7.0",
"flake8>=7.0,<8.0",
"mypy>=1.17,<2.0",

View File

@@ -33,6 +33,7 @@ class DomainToAsnEnricher(Enricher):
vault=vault,
params=params,
)
self.domain_asn_mapping: List[tuple[Domain, ASN]] = []
@classmethod
def required_params(cls) -> bool:
@@ -64,6 +65,7 @@ class DomainToAsnEnricher(Enricher):
async def scan(self, data: List[InputType]) -> List[OutputType]:
results: List[OutputType] = []
self.domain_asn_mapping = []
asnmap = AsnmapTool()
# Retrieve API key from vault or environment
@@ -87,6 +89,7 @@ class DomainToAsnEnricher(Enricher):
description=asn_data.get("as_name", ""),
)
results.append(asn)
self.domain_asn_mapping.append((domain, asn))
Logger.info(
self.sketch_id,
@@ -115,8 +118,8 @@ class DomainToAsnEnricher(Enricher):
self, results: List[OutputType], input_data: List[InputType] = None
) -> List[OutputType]:
# Create Neo4j relationships between domains and their corresponding ASNs
if input_data and self._graph_service:
for domain, asn in zip(input_data, results):
if self._graph_service:
for domain, asn in self.domain_asn_mapping:
# Create domain node
self.create_node(domain)
# Create ASN node

View File

@@ -0,0 +1,135 @@
from typing import Any, Dict, List, Optional
from flowsint_core.core.logger import Logger
from flowsint_core.core.enricher_base import Enricher
from flowsint_enrichers.registry import flowsint_enricher
from flowsint_types.domain import Domain
from flowsint_types.ip import Ip
from tools.network.dnsx import DnsxTool
@flowsint_enricher
class DomainToDnsEnricher(Enricher):
"""[DNSX] Resolve a domain's A (IPv4) and AAAA (IPv6) records using dnsx."""
InputType = Domain
OutputType = Ip
def __init__(
self,
sketch_id: Optional[str] = None,
scan_id: Optional[str] = None,
vault=None,
params: Optional[Dict[str, Any]] = None,
**kwargs,
):
super().__init__(
sketch_id=sketch_id,
scan_id=scan_id,
params_schema=self.get_params_schema(),
vault=vault,
params=params,
**kwargs,
)
@classmethod
def name(cls) -> str:
return "domain_to_dns"
@classmethod
def category(cls) -> str:
return "Domain"
@classmethod
def key(cls) -> str:
return "domain"
@classmethod
def get_params_schema(cls) -> List[Dict[str, Any]]:
"""Declare parameters for this enricher."""
return [
{
"name": "ipv6",
"type": "select",
"description": "Also resolve AAAA (IPv6) records",
"required": False,
"default": "true",
"options": [
{"label": "Enabled", "value": "true"},
{"label": "Disabled", "value": "false"},
],
},
{
"name": "PDCP_API_KEY",
"type": "vaultSecret",
"description": "ProjectDiscovery Cloud Platform API key (optional)",
"required": False,
},
]
async def scan(self, data: List[InputType]) -> List[OutputType]:
results: List[OutputType] = []
try:
dnsx = DnsxTool()
except Exception as e:
Logger.error(
self.sketch_id,
{"message": f"[DNSX] Failed to initialize dnsx: {e}"},
)
return results
aaaa = self.params.get("ipv6", "true") == "true"
api_key = self.get_secret("PDCP_API_KEY", None)
for d in data:
try:
ips = dnsx.resolve_domain(d.domain, aaaa=aaaa, api_key=api_key)
except Exception as e:
Logger.error(
self.sketch_id,
{"message": f"[DNSX] Error resolving {d.domain}: {e}"},
)
continue
for ip in ips:
try:
ip_obj = Ip(address=ip)
except Exception:
# dnsx occasionally emits non-address artifacts; skip them.
continue
# Carry the source domain through to postprocess for graph wiring.
setattr(ip_obj, "_source_domain", d.domain)
results.append(ip_obj)
Logger.info(
self.sketch_id,
{"message": f"[DNSX] {d.domain} -> {ip}"},
)
return results
def postprocess(
self, results: List[OutputType], original_input: List[InputType] = None
) -> List[OutputType]:
for ip_obj in results:
if not self._graph_service:
continue
source_domain = getattr(ip_obj, "_source_domain", None)
if not source_domain:
continue
domain_obj = Domain(domain=source_domain)
self.create_node(domain_obj)
self.create_node(ip_obj)
self.create_relationship(domain_obj, ip_obj, "RESOLVES_TO")
self.log_graph_message(
f"IP found for domain {source_domain} -> {ip_obj.address}"
)
# Clean up the temporary attribute used to thread context.
delattr(ip_obj, "_source_domain")
return results
InputType = DomainToDnsEnricher.InputType
OutputType = DomainToDnsEnricher.OutputType

View File

@@ -14,6 +14,10 @@ class ResolveEnricher(Enricher):
InputType = Domain
OutputType = Ip
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.domain_ip_mapping: List[tuple[Domain, Ip]] = []
@classmethod
def name(cls) -> str:
return "domain_to_ip"
@@ -33,10 +37,13 @@ class ResolveEnricher(Enricher):
async def scan(self, data: List[InputType]) -> List[OutputType]:
results: List[OutputType] = []
self.domain_ip_mapping = []
for d in data:
try:
ip = socket.gethostbyname(d.domain)
results.append(Ip(address=ip))
ip_obj = Ip(address=ip)
results.append(ip_obj)
self.domain_ip_mapping.append((d, ip_obj))
except Exception as e:
Logger.info(
self.sketch_id,
@@ -46,7 +53,9 @@ class ResolveEnricher(Enricher):
return results
def postprocess(self, results: List[OutputType], original_input: List[InputType]) -> List[OutputType]:
for domain_obj, ip_obj in zip(original_input, results):
for domain_obj, ip_obj in self.domain_ip_mapping:
if not self._graph_service:
continue
self.create_node(domain_obj)
self.create_node(ip_obj)
self.create_relationship(

View File

@@ -15,6 +15,10 @@ class EmailToGravatarEnricher(Enricher):
InputType = Email
OutputType = Gravatar
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.email_gravatar_mapping: List[tuple[Email, Gravatar]] = []
@classmethod
def name(cls) -> str:
return "email_to_gravatar"
@@ -29,6 +33,7 @@ class EmailToGravatarEnricher(Enricher):
async def scan(self, data: List[InputType]) -> List[OutputType]:
results: List[OutputType] = []
self.email_gravatar_mapping = []
for email in data:
try:
@@ -72,6 +77,7 @@ class EmailToGravatarEnricher(Enricher):
gravatar = Gravatar(**gravatar_data)
results.append(gravatar)
self.email_gravatar_mapping.append((email, gravatar))
except Exception as e:
Logger.error(
@@ -85,13 +91,12 @@ class EmailToGravatarEnricher(Enricher):
return results
def postprocess(self, results: List[OutputType], original_input: List[InputType]) -> List[OutputType]:
for email_obj, gravatar_obj in zip(original_input, results):
for email_obj, gravatar_obj in self.email_gravatar_mapping:
if not self._graph_service:
continue
# Create email node
self.create_node(email_obj)
# Create gravatar node
gravatar_key = f"{email_obj.email}_{self.sketch_id}"
self.create_node(gravatar_obj)
# Create relationship between email and gravatar
self.create_relationship(email_obj, gravatar_obj, "HAS_GRAVATAR")

View File

@@ -32,6 +32,7 @@ class IpToAsnEnricher(Enricher):
vault=vault,
params=params,
)
self.ip_asn_mapping: List[tuple[Ip, ASN]] = []
@classmethod
def required_params(cls) -> bool:
@@ -63,6 +64,7 @@ class IpToAsnEnricher(Enricher):
async def scan(self, data: List[InputType]) -> List[OutputType]:
results: List[OutputType] = []
self.ip_asn_mapping = []
asnmap = AsnmapTool()
# Retrieve API key from vault or environment
@@ -84,6 +86,7 @@ class IpToAsnEnricher(Enricher):
description=asn_data.get("as_name", ""),
)
results.append(asn)
self.ip_asn_mapping.append((ip, asn))
Logger.info(
self.sketch_id,
{
@@ -110,8 +113,8 @@ class IpToAsnEnricher(Enricher):
self, results: List[OutputType], input_data: List[InputType] = None
) -> List[OutputType]:
# Create Neo4j relationships between IPs and their corresponding ASNs
if input_data and self._graph_service:
for ip, asn in zip(input_data, results):
if self._graph_service:
for ip, asn in self.ip_asn_mapping:
# Create IP node
self.create_node(ip)
# Create ASN node

View File

@@ -33,6 +33,7 @@ class IpToFraudScore(Enricher):
vault=vault,
params=params,
)
self.ip_risk_mapping: List[tuple[Ip, RiskProfile]] = []
# @classmethod
# def required_params(cls) -> bool:
@@ -70,6 +71,7 @@ class IpToFraudScore(Enricher):
async def scan(self, data: List[InputType]) -> List[OutputType]:
results: List[OutputType] = []
self.ip_risk_mapping = []
api_username = self.get_secret("SCAMLYTICS_USERNAME", os.getenv("SCAMLYTICS_USERNAME"))
api_key = self.get_secret("SCAMLYTICS_API_KEY", os.getenv("SCAMLYTICS_API_KEY"))
@@ -98,7 +100,7 @@ class IpToFraudScore(Enricher):
Logger.error(
self.sketch_id,
{
"message": f"(IpToFraudScore) Request to Scamlytics failed (status): '{response_json["status"]}'."
"message": f"(IpToFraudScore) Request to Scamlytics failed (status): '{response_json['status']}'."
},
)
continue
@@ -121,11 +123,17 @@ class IpToFraudScore(Enricher):
if proxy_info.get(key):
proxy_flags.append(label)
results.append(
RiskProfile(
entity_id=ip.address, entity_type="IP address", overall_risk_score=fraud_score, risk_level=fraud_risk, last_updated=datetime.datetime.utcnow().isoformat(), risk_factors=proxy_flags, source="Scamalytics"
)
)
risk_profile = RiskProfile(
entity_id=ip.address,
entity_type="IP address",
overall_risk_score=fraud_score,
risk_level=fraud_risk,
last_updated=datetime.datetime.now(datetime.timezone.utc).isoformat(),
risk_factors=proxy_flags,
source="Scamalytics",
)
results.append(risk_profile)
self.ip_risk_mapping.append((ip, risk_profile))
except Exception as e:
Logger.error(self.sketch_id, {"message": f"(IpToFraudScore) Exception while querying {ip.address}: {e}"})
@@ -138,16 +146,15 @@ class IpToFraudScore(Enricher):
if not self._graph_service:
return results
if input_data and self._graph_service:
for ip, risk_profile in zip(input_data, results):
self.create_node(ip)
self.create_node(risk_profile)
for ip, risk_profile in self.ip_risk_mapping:
self.create_node(ip)
self.create_node(risk_profile)
# Create relationship
self.create_relationship(ip, risk_profile, "HAS_RISK_PROFILE")
self.log_graph_message(
f"(IpToFraudScore) IP {ip.address} has risk level of {risk_profile.risk_level}"
)
# Create relationship
self.create_relationship(ip, risk_profile, "HAS_RISK_PROFILE")
self.log_graph_message(
f"(IpToFraudScore) IP {ip.address} has risk level of {risk_profile.risk_level}"
)
return results

View File

@@ -34,9 +34,8 @@ class IgnorantEnricher(Enricher):
results: List[OutputType] = []
for phone_obj in data:
try:
cleaned_phone = is_valid_number(phone_obj.number)
if cleaned_phone:
result = await self._perform_ignorant_research(cleaned_phone)
if is_valid_number(phone_obj.number):
result = await self._perform_ignorant_research(phone_obj.number)
results.append(result)
else:
results.append({"number": phone_obj.number, "error": "Invalid phone number"})

View File

@@ -53,7 +53,7 @@ class MaigretEnricher(Enricher):
return results
try:
with open(output_file, "r") as f:
with open(output_file, "r", encoding="utf-8") as f:
raw_data = json.load(f)
except Exception as e:
Logger.error(

View File

@@ -63,7 +63,7 @@ class SherlockEnricher(Enricher):
continue
found_accounts = {}
with open(output_file, "r") as f:
with open(output_file, "r", encoding="utf-8") as f:
for line in f:
line = line.strip()
if line and line.startswith("http"):

View File

@@ -155,7 +155,7 @@ def get_root_domain(domain: str) -> str:
return domain
def is_valid_number(phone: str, region: str = "FR") -> None:
def is_valid_number(phone: str, region: str = "FR") -> bool:
"""
Validates a phone number. Raises InvalidPhoneNumberError if invalid.
- `region` should be ISO 3166-1 alpha-2 country code (e.g., 'FR' for France)
@@ -167,6 +167,8 @@ def is_valid_number(phone: str, region: str = "FR") -> None:
except NumberParseException:
return False
return True
def parse_asn(asn: str) -> int:
if not is_valid_asn(asn):

View File

@@ -0,0 +1,133 @@
from typing import Any, Dict, List, Optional
from flowsint_core.core.logger import Logger
from flowsint_core.core.enricher_base import Enricher
from flowsint_enrichers.registry import flowsint_enricher
from flowsint_types.website import Website
from flowsint_types.technology import Technology
from tools.network.httpx import HttpxTool
@flowsint_enricher
class TechDetectEnricher(Enricher):
"""[HTTPX] Detect web technologies on a website via httpx (wappalyzer)."""
InputType = Website
OutputType = Technology
def __init__(
self,
sketch_id: Optional[str] = None,
scan_id: Optional[str] = None,
vault=None,
params: Optional[Dict[str, Any]] = None,
**kwargs,
):
super().__init__(
sketch_id=sketch_id,
scan_id=scan_id,
params_schema=self.get_params_schema(),
vault=vault,
params=params,
**kwargs,
)
@classmethod
def name(cls) -> str:
return "tech_detect"
@classmethod
def category(cls) -> str:
return "Website"
@classmethod
def key(cls) -> str:
return "website"
@staticmethod
def _parse_tech(entry: str) -> Optional[Technology]:
"""Turn an httpx tech entry into a Technology.
httpx -td emits plain names ("nginx") and, when wappalyzer knows
the version, "name:version" pairs ("PHP:8.1").
"""
entry = (entry or "").strip()
if not entry:
return None
name, sep, version = entry.partition(":")
name = name.strip()
if not name:
return None
return Technology(
name=name,
version=(version.strip() or None) if sep else None,
source="httpx",
)
async def scan(self, data: List[InputType]) -> List[OutputType]:
results: List[OutputType] = []
try:
httpx = HttpxTool()
except Exception as e:
Logger.error(
self.sketch_id,
{"message": f"[HTTPX] Failed to initialize httpx: {e}"},
)
return results
for website in data:
url = str(website.url)
try:
probes = httpx.launch(url, args=["-td"])
except Exception as e:
Logger.error(
self.sketch_id,
{"message": f"[HTTPX] Error probing {url}: {e}"},
)
continue
seen: set[tuple[str, Optional[str]]] = set()
for probe in probes:
for entry in probe.get("tech", []) or []:
tech = self._parse_tech(entry)
if tech is None:
continue
dedup_key = (tech.name, tech.version)
if dedup_key in seen:
continue
seen.add(dedup_key)
# Carry the source website through to postprocess.
setattr(tech, "_source_url", url)
results.append(tech)
Logger.info(
self.sketch_id,
{"message": f"[HTTPX] {url} -> {tech.nodeLabel}"},
)
return results
def postprocess(
self, results: List[OutputType], original_input: List[InputType] = None
) -> List[OutputType]:
for tech in results:
if not self._graph_service:
continue
source_url = getattr(tech, "_source_url", None)
if not source_url:
continue
website_obj = Website(url=source_url)
self.create_node(website_obj)
self.create_node(tech)
self.create_relationship(website_obj, tech, "USES_TECHNOLOGY")
self.log_graph_message(
f"Technology {tech.nodeLabel} detected on {source_url}"
)
delattr(tech, "_source_url")
return results
InputType = TechDetectEnricher.InputType
OutputType = TechDetectEnricher.OutputType

View File

@@ -1,5 +1,5 @@
import json
from typing import Any, List
from typing import List
from ..dockertool import DockerTool
@@ -86,3 +86,72 @@ class DnsxTool(DockerTool):
raise RuntimeError(
f"Error running dnsx: {str(e)}. Output: {getattr(e, 'output', 'No output')}"
)
def resolve_domain(
self, domain: str, aaaa: bool = True, api_key: str = None
) -> List[str]:
"""
Resolve a domain's A (IPv4) and, optionally, AAAA (IPv6) records.
Runs ``dnsx -d <domain> -a [-aaaa] -json -silent`` and parses the JSONL
output, returning the de-duplicated list of resolved IP addresses.
Args:
domain: Domain name to resolve (e.g., "example.com")
aaaa: Whether to also query AAAA (IPv6) records. Defaults to True.
api_key: Optional ProjectDiscovery Cloud Platform API key
Returns:
Ordered, de-duplicated list of resolved IP addresses (A then AAAA).
"""
flags = ["-a"]
if aaaa:
flags.append("-aaaa")
flags += ["-json", "-silent"]
command = f"-d {domain} {' '.join(flags)}"
env = {}
if api_key:
env["PDCP_API_KEY"] = api_key
try:
result = super().launch(command, environment=env)
except Exception as e:
raise RuntimeError(
f"Error running dnsx: {str(e)}. Output: {getattr(e, 'output', 'No output')}"
)
return self._parse_resolved_ips(result)
@staticmethod
def _parse_resolved_ips(result: str) -> List[str]:
"""
Parse dnsx JSONL output into an ordered, de-duplicated list of IPs.
Each line is a JSON object whose ``a``/``aaaa`` keys hold the resolved
IPv4/IPv6 addresses (see retryabledns.DNSData). Malformed lines are
skipped so a single bad record never breaks the whole resolution.
"""
ips: List[str] = []
seen: set[str] = set()
if not result or not result.strip():
return ips
for line in result.split("\n"):
line = line.strip()
if not line:
continue
try:
record = json.loads(line)
except json.JSONDecodeError:
continue
for key in ("a", "aaaa"):
for ip in record.get(key, []) or []:
if ip and ip not in seen:
seen.add(ip)
ips.append(ip)
return ips

View File

@@ -2,6 +2,19 @@ import pytest
from tests.logger import TestLogger
@pytest.fixture(autouse=True)
def setup_test_environment(monkeypatch):
"""Set up test environment variables.
Dummy Neo4j credentials: the Neo4jConnection singleton requires them
at construction, but driver creation is lazy — nothing connects.
Without these, tests silently depend on the developer's local .env.
"""
monkeypatch.setenv("NEO4J_URI_BOLT", "bolt://127.0.0.1:7687")
monkeypatch.setenv("NEO4J_USERNAME", "neo4j")
monkeypatch.setenv("NEO4J_PASSWORD", "test-password")
@pytest.fixture(autouse=True)
def mock_logger(monkeypatch):
"""Automatically replace the production Logger with TestLogger for all tests."""

View File

@@ -0,0 +1,131 @@
import pytest
from flowsint_enrichers import ENRICHER_REGISTRY
from flowsint_enrichers.domain.to_dns import DomainToDnsEnricher
from flowsint_types.ip import Ip
from tools.network.dnsx import DnsxTool
# ---------------------------------------------------------------------------
# Registry wiring
# ---------------------------------------------------------------------------
def test_domain_to_dns_is_registered():
enricher = ENRICHER_REGISTRY.get_enricher("domain_to_dns", "123", "123")
assert enricher.name() == "domain_to_dns"
def test_domain_to_dns_metadata():
assert DomainToDnsEnricher.category() == "Domain"
assert DomainToDnsEnricher.key() == "domain"
assert DomainToDnsEnricher.input_schema()["type"] == "Domain"
assert DomainToDnsEnricher.output_schema()["type"] == "Ip"
# ---------------------------------------------------------------------------
# dnsx JSONL parsing (no Docker required)
# ---------------------------------------------------------------------------
def test_parse_resolved_ips_extracts_a_and_aaaa():
output = (
'{"host":"example.com","a":["93.184.216.34"],'
'"aaaa":["2606:2800:220:1:248:1893:25c8:1946"]}'
)
assert DnsxTool._parse_resolved_ips(output) == [
"93.184.216.34",
"2606:2800:220:1:248:1893:25c8:1946",
]
def test_parse_resolved_ips_dedupes_and_preserves_order():
output = "\n".join(
[
'{"host":"a.com","a":["1.1.1.1","1.0.0.1"]}',
'{"host":"a.com","a":["1.1.1.1"],"aaaa":["2606:4700:4700::1111"]}',
]
)
assert DnsxTool._parse_resolved_ips(output) == [
"1.1.1.1",
"1.0.0.1",
"2606:4700:4700::1111",
]
def test_parse_resolved_ips_skips_blank_and_malformed_lines():
output = "\n".join(
[
"",
"not-json",
'{"host":"a.com","a":["8.8.8.8"]}',
" ",
]
)
assert DnsxTool._parse_resolved_ips(output) == ["8.8.8.8"]
def test_parse_resolved_ips_handles_empty_output():
assert DnsxTool._parse_resolved_ips("") == []
assert DnsxTool._parse_resolved_ips(" \n ") == []
# ---------------------------------------------------------------------------
# scan() — dnsx tool mocked, no Docker daemon touched
# ---------------------------------------------------------------------------
class _FakeDnsx:
def __init__(self, mapping):
self._mapping = mapping
self.calls = []
def resolve_domain(self, domain, aaaa=True, api_key=None):
self.calls.append((domain, aaaa, api_key))
return self._mapping.get(domain, [])
@pytest.mark.asyncio
async def test_scan_returns_ip_objects_tagged_with_source_domain(monkeypatch):
fake = _FakeDnsx({"example.com": ["93.184.216.34", "2606:2800:220:1:248:1893:25c8:1946"]})
monkeypatch.setattr(
"flowsint_enrichers.domain.to_dns.DnsxTool", lambda: fake
)
enricher = DomainToDnsEnricher(sketch_id="s", scan_id="t", graph_service=None)
from flowsint_types.domain import Domain
results = await enricher.scan([Domain(domain="example.com")])
assert {ip.address for ip in results} == {
"93.184.216.34",
"2606:2800:220:1:248:1893:25c8:1946",
}
assert all(isinstance(ip, Ip) for ip in results)
assert all(getattr(ip, "_source_domain") == "example.com" for ip in results)
@pytest.mark.asyncio
async def test_scan_ipv6_param_disables_aaaa(monkeypatch):
fake = _FakeDnsx({"example.com": ["93.184.216.34"]})
monkeypatch.setattr(
"flowsint_enrichers.domain.to_dns.DnsxTool", lambda: fake
)
enricher = DomainToDnsEnricher(
sketch_id="s", scan_id="t", graph_service=None, params={"ipv6": "false"}
)
from flowsint_types.domain import Domain
await enricher.scan([Domain(domain="example.com")])
# aaaa flag forwarded to the tool as False
assert fake.calls == [("example.com", False, None)]
@pytest.mark.asyncio
async def test_scan_skips_unresolvable_domain(monkeypatch):
fake = _FakeDnsx({}) # no records for anything
monkeypatch.setattr(
"flowsint_enrichers.domain.to_dns.DnsxTool", lambda: fake
)
enricher = DomainToDnsEnricher(sketch_id="s", scan_id="t", graph_service=None)
from flowsint_types.domain import Domain
results = await enricher.scan([Domain(domain="example.com")])
assert results == []

View File

@@ -0,0 +1,112 @@
import pytest
from flowsint_enrichers import ENRICHER_REGISTRY
from flowsint_enrichers.website.to_technologies import TechDetectEnricher
from flowsint_types.technology import Technology
from flowsint_types.website import Website
# ---------------------------------------------------------------------------
# Technology type
# ---------------------------------------------------------------------------
def test_technology_label_without_version():
assert Technology(name="nginx").nodeLabel == "nginx"
def test_technology_label_with_version():
assert Technology(name="PHP", version="8.1").nodeLabel == "PHP 8.1"
def test_technology_from_string_plain_and_versioned():
assert Technology.from_string("nginx").version is None
t = Technology.from_string("PHP:8.1")
assert (t.name, t.version) == ("PHP", "8.1")
def test_technology_is_in_type_registry():
from flowsint_types import TYPE_REGISTRY
assert TYPE_REGISTRY.get_lowercase("technology") is Technology
# ---------------------------------------------------------------------------
# Registry wiring
# ---------------------------------------------------------------------------
def test_tech_detect_is_registered():
enricher = ENRICHER_REGISTRY.get_enricher("tech_detect", "123", "123")
assert enricher.name() == "tech_detect"
def test_tech_detect_metadata():
assert TechDetectEnricher.category() == "Website"
assert TechDetectEnricher.key() == "website"
assert TechDetectEnricher.input_schema()["type"] == "Website"
assert TechDetectEnricher.output_schema()["type"] == "Technology"
# ---------------------------------------------------------------------------
# _parse_tech
# ---------------------------------------------------------------------------
def test_parse_tech_plain():
t = TechDetectEnricher._parse_tech("nginx")
assert (t.name, t.version, t.source) == ("nginx", None, "httpx")
def test_parse_tech_versioned():
t = TechDetectEnricher._parse_tech("PHP:8.1")
assert (t.name, t.version) == ("PHP", "8.1")
def test_parse_tech_blank_returns_none():
assert TechDetectEnricher._parse_tech("") is None
assert TechDetectEnricher._parse_tech(" ") is None
assert TechDetectEnricher._parse_tech(":1.0") is None
# ---------------------------------------------------------------------------
# scan() — httpx mocked, no Docker daemon touched
# ---------------------------------------------------------------------------
class _FakeHttpx:
def __init__(self, by_url):
self._by_url = by_url
self.calls = []
def launch(self, target, args=None):
self.calls.append((target, args))
return self._by_url.get(target, [])
@pytest.mark.asyncio
async def test_scan_extracts_and_dedupes_technologies(monkeypatch):
fake = _FakeHttpx(
{
"https://example.com/": [
{"tech": ["nginx", "PHP:8.1", "nginx"]}, # duplicate nginx
]
}
)
monkeypatch.setattr(
"flowsint_enrichers.website.to_technologies.HttpxTool", lambda: fake
)
enricher = TechDetectEnricher(sketch_id="s", scan_id="t", graph_service=None)
results = await enricher.scan([Website(url="https://example.com/")])
labels = sorted(t.nodeLabel for t in results)
assert labels == ["PHP 8.1", "nginx"]
assert all(isinstance(t, Technology) for t in results)
assert all(getattr(t, "_source_url") == "https://example.com/" for t in results)
# -td flag forwarded to httpx
assert fake.calls == [("https://example.com/", ["-td"])]
@pytest.mark.asyncio
async def test_scan_handles_no_tech_field(monkeypatch):
fake = _FakeHttpx({"https://example.com/": [{"status_code": 200}]})
monkeypatch.setattr(
"flowsint_enrichers.website.to_technologies.HttpxTool", lambda: fake
)
enricher = TechDetectEnricher(sketch_id="s", scan_id="t", graph_service=None)
results = await enricher.scan([Website(url="https://example.com/")])
assert results == []

View File

@@ -13,8 +13,8 @@ dependencies = [
[dependency-groups]
dev = [
"pytest>=8.4.2,<9.0.0",
"black>=25.0,<26.0",
"pytest>=9.0.3,<10.0.0",
"black>=26.3.1,<27.0",
"isort>=6.0,<7.0",
"flake8>=7.0,<8.0",
"mypy>=1.17,<2.0",

View File

@@ -43,6 +43,7 @@ from .script import Script
from .session import Session
from .social_account import SocialAccount
from .ssl_certificate import SSLCertificate
from .technology import Technology
from .username import Username
from .wallet import CryptoNFT, CryptoWallet, CryptoWalletTransaction
from .weapon import Weapon
@@ -87,6 +88,7 @@ __all__ = [
"Session",
"SocialAccount",
"SSLCertificate",
"Technology",
"Username",
"CryptoWallet",
"CryptoWalletTransaction",
@@ -134,6 +136,7 @@ TYPE_TO_MODEL: Dict[str, Type[BaseModel]] = {
"file": File,
"malware": Malware,
"sslcertificate": SSLCertificate,
"technology": Technology,
"location": Location,
"affiliation": Affiliation,
"alias": Alias,

View File

@@ -1,9 +1,27 @@
import re
from pydantic import Field, model_validator
from typing import Optional, List, Self
from typing import Optional, Self
from .flowsint_base import FlowsintType
from .registry import flowsint_type
# Map of hex-hash length -> the File field that should hold it.
_HASH_LENGTH_TO_FIELD = {
32: "hash_md5",
40: "hash_sha1",
64: "hash_sha256",
}
_HASH_RE = re.compile(r"^[0-9a-fA-F]+$")
def _hash_field_for(value: str) -> Optional[str]:
"""Return the File hash field name for an MD5/SHA1/SHA256 hex string, else None."""
value = value.strip()
if not _HASH_RE.match(value):
return None
return _HASH_LENGTH_TO_FIELD.get(len(value))
@flowsint_type
class File(FlowsintType):
@@ -73,10 +91,22 @@ class File(FlowsintType):
@classmethod
def from_string(cls, line: str):
"""Parse a file from a raw string."""
return cls(filename=line.strip())
"""Parse a file from a raw string.
If the value is an MD5/SHA1/SHA256 hash, store it in the matching
hash field as well as the filename (so the node keeps a label).
"""
line = line.strip()
hash_field = _hash_field_for(line)
if hash_field:
return cls(filename=line, **{hash_field: line.lower()})
return cls(filename=line)
@classmethod
def detect(cls, line: str) -> bool:
"""File cannot be reliably detected from a single line of text."""
return False
"""Detect a file hash (MD5 / SHA1 / SHA256) from a single line.
Only hashes are auto-detected; arbitrary filenames are too ambiguous
to detect reliably, so non-hash values still return False.
"""
return _hash_field_for(line) is not None

View File

@@ -17,13 +17,15 @@ class SocialAccount(FlowsintType):
title="ID",
json_schema_extra={"primary": True},
)
username: Username = Field(
..., description="Username associated with this account", title="Username"
username: Optional[Username] = Field(
None, description="Username associated with this account", title="Username"
)
@field_validator("username", mode="before")
@classmethod
def convert_username(cls, v: Union[str, Username]) -> Username:
if not v:
return Username(value="")
"""Convert string to Username object if needed."""
if isinstance(v, str):
return Username(value=v)

View File

@@ -0,0 +1,57 @@
from typing import Optional, Self
from pydantic import Field, model_validator
from .flowsint_base import FlowsintType
from .registry import flowsint_type
@flowsint_type
class Technology(FlowsintType):
"""Represents a technology, framework, or software detected during recon."""
name: str = Field(
...,
description="Technology name (e.g. nginx, PHP, React)",
title="Technology Name",
json_schema_extra={"primary": True},
)
version: Optional[str] = Field(
None, description="Detected version, if known", title="Version"
)
category: Optional[str] = Field(
None,
description="Technology category (e.g. web-server, framework, cms)",
title="Category",
)
source: Optional[str] = Field(
None,
description="Tool or method that detected the technology (e.g. httpx)",
title="Source",
)
@model_validator(mode="after")
def compute_label(self) -> Self:
if self.version:
self.nodeLabel = f"{self.name} {self.version}"
else:
self.nodeLabel = self.name
return self
@classmethod
def from_string(cls, line: str):
"""Parse a technology from a raw string.
Accepts either a bare name ("nginx") or a wappalyzer-style
"name:version" pair ("PHP:8.1"), as emitted by httpx -td.
"""
line = line.strip()
if ":" in line:
name, _, version = line.partition(":")
return cls(name=name.strip(), version=version.strip() or None)
return cls(name=line)
@classmethod
def detect(cls, line: str) -> bool:
"""Technology cannot be reliably detected from a single line of text."""
return False

View File

@@ -1,5 +1,5 @@
import re
from typing import Any, Optional, Self
from typing import Optional, Self
from pydantic import Field, field_validator, model_validator
@@ -60,6 +60,11 @@ class Username(FlowsintType):
if not line:
return False
# Don't claim file hashes (MD5/SHA1/SHA256) — those are File entities.
# This keeps hash detection independent of type-registration order.
if len(line) in (32, 40, 64) and re.fullmatch(r"[0-9a-fA-F]+", line):
return False
# Username pattern: 3-80 characters, only letters, numbers, underscores, hyphens
# Note: This is intentionally restrictive to avoid false positives
return bool(re.match(r"^[a-zA-Z0-9_-]{3,80}$", line))

View File

@@ -0,0 +1,64 @@
"""Tests for File hash detection (issue #45)."""
from flowsint_types import File, Username
MD5 = "d41d8cd98f00b204e9800998ecf8427e" # 32 hex
SHA1 = "da39a3ee5e6b4b0d3255bfef95601890afd80709" # 40 hex
SHA256 = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" # 64 hex
def test_detect_md5():
assert File.detect(MD5) is True
def test_detect_sha1():
assert File.detect(SHA1) is True
def test_detect_sha256():
assert File.detect(SHA256) is True
def test_detect_is_case_insensitive():
assert File.detect(SHA256.upper()) is True
def test_detect_strips_whitespace():
assert File.detect(f" {MD5} ") is True
def test_detect_rejects_non_hash():
# Plain filename, wrong length, and non-hex strings are not detected.
assert File.detect("report.pdf") is False
assert File.detect("john_doe") is False
assert File.detect("g" * 32) is False # right length, not hex
assert File.detect("abc123") is False # hex but not a hash length
def test_from_string_populates_hash_fields():
assert File.from_string(MD5).hash_md5 == MD5
assert File.from_string(SHA1).hash_sha1 == SHA1
assert File.from_string(SHA256).hash_sha256 == SHA256
def test_from_string_normalizes_hash_to_lowercase():
f = File.from_string(SHA256.upper())
assert f.hash_sha256 == SHA256 # stored lowercase
assert f.filename == SHA256.upper() # original kept as label
def test_from_string_plain_filename_unchanged():
f = File.from_string("evil.exe")
assert f.filename == "evil.exe"
assert f.hash_md5 is None
assert f.hash_sha1 is None
assert f.hash_sha256 is None
def test_username_no_longer_claims_hashes():
# Regression: hashes used to be mis-detected as usernames.
assert Username.detect(MD5) is False
assert Username.detect(SHA1) is False
assert Username.detect(SHA256) is False
# Real usernames still detected.
assert Username.detect("john_doe") is True

View File

@@ -10,7 +10,7 @@ dependencies = [
"flowsint-enrichers",
"flowsint-api",
"pydantic[email]>=2.11.7,<3.0.0",
"python-multipart>=0.0.20,<0.0.21",
"python-multipart>=0.0.27,<0.1.0",
"docker>=7.1.0,<8.0.0",
]

2598
uv.lock generated

File diff suppressed because it is too large Load Diff

227
yarn.lock
View File

@@ -956,6 +956,11 @@
pbf "^4.0.1"
supercluster "^8.0.1"
"@microsoft/fetch-event-source@^2.0.1":
version "2.0.1"
resolved "https://registry.yarnpkg.com/@microsoft/fetch-event-source/-/fetch-event-source-2.0.1.tgz#9ceecc94b49fbaa15666e38ae8587f64acce007d"
integrity sha512-W6CLUJ2eBMw3Rec70qrsEW0jOm/3twwJv21mrmj2yORiaVmVYGS4sSS5yUwvQc1ZlDLYGPnClVWmUUMagKNsfA==
"@monaco-editor/loader@^1.5.0":
version "1.7.0"
resolved "https://registry.npmjs.org/@monaco-editor/loader/-/loader-1.7.0.tgz"
@@ -2580,6 +2585,65 @@
"@types/babel__core" "^7.20.5"
react-refresh "^0.17.0"
"@vitest/expect@2.1.9":
version "2.1.9"
resolved "https://registry.yarnpkg.com/@vitest/expect/-/expect-2.1.9.tgz#b566ea20d58ea6578d8dc37040d6c1a47ebe5ff8"
integrity sha512-UJCIkTBenHeKT1TTlKMJWy1laZewsRIzYighyYiJKZreqtdxSos/S1t+ktRMQWu2CKqaarrkeszJx1cgC5tGZw==
dependencies:
"@vitest/spy" "2.1.9"
"@vitest/utils" "2.1.9"
chai "^5.1.2"
tinyrainbow "^1.2.0"
"@vitest/mocker@2.1.9":
version "2.1.9"
resolved "https://registry.yarnpkg.com/@vitest/mocker/-/mocker-2.1.9.tgz#36243b27351ca8f4d0bbc4ef91594ffd2dc25ef5"
integrity sha512-tVL6uJgoUdi6icpxmdrn5YNo3g3Dxv+IHJBr0GXHaEdTcw3F+cPKnsXFhli6nO+f/6SDKPHEK1UN+k+TQv0Ehg==
dependencies:
"@vitest/spy" "2.1.9"
estree-walker "^3.0.3"
magic-string "^0.30.12"
"@vitest/pretty-format@2.1.9", "@vitest/pretty-format@^2.1.9":
version "2.1.9"
resolved "https://registry.yarnpkg.com/@vitest/pretty-format/-/pretty-format-2.1.9.tgz#434ff2f7611689f9ce70cd7d567eceb883653fdf"
integrity sha512-KhRIdGV2U9HOUzxfiHmY8IFHTdqtOhIzCpd8WRdJiE7D/HUcZVD0EgQCVjm+Q9gkUXWgBvMmTtZgIG48wq7sOQ==
dependencies:
tinyrainbow "^1.2.0"
"@vitest/runner@2.1.9":
version "2.1.9"
resolved "https://registry.yarnpkg.com/@vitest/runner/-/runner-2.1.9.tgz#cc18148d2d797fd1fd5908d1f1851d01459be2f6"
integrity sha512-ZXSSqTFIrzduD63btIfEyOmNcBmQvgOVsPNPe0jYtESiXkhd8u2erDLnMxmGrDCwHCCHE7hxwRDCT3pt0esT4g==
dependencies:
"@vitest/utils" "2.1.9"
pathe "^1.1.2"
"@vitest/snapshot@2.1.9":
version "2.1.9"
resolved "https://registry.yarnpkg.com/@vitest/snapshot/-/snapshot-2.1.9.tgz#24260b93f798afb102e2dcbd7e61c6dfa118df91"
integrity sha512-oBO82rEjsxLNJincVhLhaxxZdEtV0EFHMK5Kmx5sJ6H9L183dHECjiefOAdnqpIgT5eZwT04PoggUnW88vOBNQ==
dependencies:
"@vitest/pretty-format" "2.1.9"
magic-string "^0.30.12"
pathe "^1.1.2"
"@vitest/spy@2.1.9":
version "2.1.9"
resolved "https://registry.yarnpkg.com/@vitest/spy/-/spy-2.1.9.tgz#cb28538c5039d09818b8bfa8edb4043c94727c60"
integrity sha512-E1B35FwzXXTs9FHNK6bDszs7mtydNi5MIfUWpceJ8Xbfb1gBMscAnwLbEu+B44ed6W3XjL9/ehLPHR1fkf1KLQ==
dependencies:
tinyspy "^3.0.2"
"@vitest/utils@2.1.9":
version "2.1.9"
resolved "https://registry.yarnpkg.com/@vitest/utils/-/utils-2.1.9.tgz#4f2486de8a54acf7ecbf2c5c24ad7994a680a6c1"
integrity sha512-v0psaMSkNJ3A2NMrUEHFRzJtDPFn+/VWZ5WxImB21T9fjucJRmS7xCS3ppEnARb9y11OAzaD+P2Ps+b+BGX5iQ==
dependencies:
"@vitest/pretty-format" "2.1.9"
loupe "^3.1.2"
tinyrainbow "^1.2.0"
"@webgpu/types@^0.1.69":
version "0.1.69"
resolved "https://registry.npmjs.org/@webgpu/types/-/types-0.1.69.tgz"
@@ -2831,6 +2895,11 @@ arrify@^1.0.1:
resolved "https://registry.npmjs.org/arrify/-/arrify-1.0.1.tgz"
integrity sha512-3CYzex9M9FGQjCGMGyi6/31c8GJbgb0qGyrx5HWxPd0aCwh4cB2YjMb2Xf9UuoogrMrlO9cTqnB5rI5GHZTcUA==
assertion-error@^2.0.1:
version "2.0.1"
resolved "https://registry.yarnpkg.com/assertion-error/-/assertion-error-2.0.1.tgz#f641a196b335690b1070bf00b6e7593fec190bf7"
integrity sha512-Izi8RQcffqCeNVgFigKli1ssklIbpHnCYc6AknXGYoB6grJqyeby7jv12JUQgmTAnIDnbck1uxksT4dzN3PWBA==
assign-symbols@^1.0.0:
version "1.0.0"
resolved "https://registry.npmjs.org/assign-symbols/-/assign-symbols-1.0.0.tgz"
@@ -2990,6 +3059,11 @@ bytewise@^1.1.0:
bytewise-core "^1.2.2"
typewise "^1.0.3"
cac@^6.7.14:
version "6.7.14"
resolved "https://registry.yarnpkg.com/cac/-/cac-6.7.14.tgz#804e1e6f506ee363cb0e3ccbb09cad5dd9870959"
integrity sha512-b6Ilus+c3RrdDk+JhLKUAQfzzgLEPy6wcXqS7f/xe1EETvsDP6GORG7SFuOs6cID5YkqchW/LXZbX5bc8j7ZcQ==
cachedir@2.3.0:
version "2.3.0"
resolved "https://registry.npmjs.org/cachedir/-/cachedir-2.3.0.tgz"
@@ -3057,6 +3131,17 @@ ccount@^2.0.0:
resolved "https://registry.npmjs.org/ccount/-/ccount-2.0.1.tgz"
integrity sha512-eyrF0jiFpY+3drT6383f1qhkbGsLSifNAjA61IUjZjmLCWjItY6LB9ft9YhoDgwfmclB2zhu51Lc7+95b8NRAg==
chai@^5.1.2:
version "5.3.3"
resolved "https://registry.yarnpkg.com/chai/-/chai-5.3.3.tgz#dd3da955e270916a4bd3f625f4b919996ada7e06"
integrity sha512-4zNhdJD/iOjSH0A05ea+Ke6MU5mmpQcbQsSOkgdaUMJ9zTlDTD/GYlwohmIE2u0gaxHYiVHEn1Fw9mZ/ktJWgw==
dependencies:
assertion-error "^2.0.1"
check-error "^2.1.1"
deep-eql "^5.0.1"
loupe "^3.1.0"
pathval "^2.0.0"
chalk@^2.4.1, chalk@^2.4.2:
version "2.4.2"
resolved "https://registry.npmjs.org/chalk/-/chalk-2.4.2.tgz"
@@ -3119,6 +3204,11 @@ chardet@^0.7.0:
resolved "https://registry.npmjs.org/chardet/-/chardet-0.7.0.tgz"
integrity sha512-mT8iDcrh03qDGRRmoA2hmBJnxpllMR+0/0qlzjqZES6NdiWDcZkCNAk4rPFZ9Q85r27unkiNNg8ZOiwZXBHwcA==
check-error@^2.1.1:
version "2.1.3"
resolved "https://registry.yarnpkg.com/check-error/-/check-error-2.1.3.tgz#2427361117b70cca8dc89680ead32b157019caf5"
integrity sha512-PAJdDJusoxnwm1VwW07VWwUN1sl7smmC3OKggvndJFadxxDRyFJBX/ggnu/KE4kQAB7a3Dp8f/YXC1FlUprWmA==
chokidar@^3.6.0:
version "3.6.0"
resolved "https://registry.npmjs.org/chokidar/-/chokidar-3.6.0.tgz"
@@ -3862,7 +3952,7 @@ dateformat@^3.0.0:
resolved "https://registry.npmjs.org/dateformat/-/dateformat-3.0.3.tgz"
integrity sha512-jyCETtSl3VMZMWeRo7iY1FL19ges1t55hMo5yaam4Jrsm5EPL89UQkoQRyiI+Yf4k8r2ZpdngkV8hr1lIdjb3Q==
debug@^4.0.0, debug@^4.1.0, debug@^4.3.1, debug@^4.3.2:
debug@^4.0.0, debug@^4.1.0, debug@^4.3.1, debug@^4.3.2, debug@^4.3.7:
version "4.4.3"
resolved "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz"
integrity sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==
@@ -3904,6 +3994,11 @@ deep-diff@^1.0.2:
resolved "https://registry.npmjs.org/deep-diff/-/deep-diff-1.0.2.tgz"
integrity sha512-aWS3UIVH+NPGCD1kki+DCU9Dua032iSsO43LqQpcs4R3+dVv7tX0qBGjiVHJHjplsoUM2XRO/KB92glqc68awg==
deep-eql@^5.0.1:
version "5.0.2"
resolved "https://registry.yarnpkg.com/deep-eql/-/deep-eql-5.0.2.tgz#4b756d8d770a9257300825d52a2c2cff99c3a341"
integrity sha512-h5k/5U50IJJFpzfL6nO9jaaumfjO/f2NjK/oYB2Djzm4p9L+3T9qWpZqZ2hAbLPuuYq9wrU08WQyBTL5GbPk5Q==
deep-is@^0.1.3:
version "0.1.4"
resolved "https://registry.npmjs.org/deep-is/-/deep-is-0.1.4.tgz"
@@ -4010,7 +4105,7 @@ dom-helpers@^5.0.1:
"@babel/runtime" "^7.8.7"
csstype "^3.0.2"
dompurify@*, dompurify@^3.3.0:
dompurify@*:
version "3.3.1"
resolved "https://registry.npmjs.org/dompurify/-/dompurify-3.3.1.tgz"
integrity sha512-qkdCKzLNtrgPFP1Vo+98FRzJnBRGe4ffyCea9IwHB1fyxPOeNTHpLKYGd4Uk9xvNoH0ZoOjwZxNptyMwqrId1Q==
@@ -4024,6 +4119,13 @@ dompurify@3.2.7:
optionalDependencies:
"@types/trusted-types" "^2.0.7"
dompurify@^3.4.11:
version "3.4.11"
resolved "https://registry.yarnpkg.com/dompurify/-/dompurify-3.4.11.tgz#29c8ba496475f279ef4015784068452fb14a0680"
integrity sha512-zhlUV12GsaRzMsf9q5M254YhA4+VuF0fG+QFqu6aYpoGlKtz+w8//jBcGVYBgQkR5GHjUomejY84AV+/uPbWdw==
optionalDependencies:
"@types/trusted-types" "^2.0.7"
dot-prop@^5.1.0:
version "5.3.0"
resolved "https://registry.npmjs.org/dot-prop/-/dot-prop-5.3.0.tgz"
@@ -4208,6 +4310,11 @@ es-iterator-helpers@^1.2.1:
iterator.prototype "^1.1.4"
safe-array-concat "^1.1.3"
es-module-lexer@^1.5.4:
version "1.7.0"
resolved "https://registry.yarnpkg.com/es-module-lexer/-/es-module-lexer-1.7.0.tgz#9159601561880a85f2734560a9099b2c31e5372a"
integrity sha512-jEQoCwk8hyb2AZziIOLhDqpm5+2ww5uIE6lkO/6jcOCusfk6LhMHpXXfBLXTZ7Ydyt0j4VoUQv6uGNYbdW+kBA==
es-object-atoms@^1.0.0, es-object-atoms@^1.1.1:
version "1.1.1"
resolved "https://registry.npmjs.org/es-object-atoms/-/es-object-atoms-1.1.1.tgz"
@@ -4441,6 +4548,13 @@ estree-util-is-identifier-name@^3.0.0:
resolved "https://registry.npmjs.org/estree-util-is-identifier-name/-/estree-util-is-identifier-name-3.0.0.tgz"
integrity sha512-hFtqIDZTIUZ9BXLb8y4pYGyk6+wekIivNVTcmvk8NoOh+VeRn5y6cEHzbURrWbfp1fIqdVipilzj+lfaadNZmg==
estree-walker@^3.0.3:
version "3.0.3"
resolved "https://registry.yarnpkg.com/estree-walker/-/estree-walker-3.0.3.tgz#67c3e549ec402a487b4fc193d1953a524752340d"
integrity sha512-7RUKfXgSMMkzt6ZuXmqapOurLGPPfgj6l9uRZ7lRGolvk0y2yocc35LdcxKC5PQZdn2DMqioAQ2NoWcrTKmm6g==
dependencies:
"@types/estree" "^1.0.0"
esutils@^2.0.2:
version "2.0.3"
resolved "https://registry.npmjs.org/esutils/-/esutils-2.0.3.tgz"
@@ -4468,6 +4582,11 @@ expand-tilde@^2.0.0, expand-tilde@^2.0.2:
dependencies:
homedir-polyfill "^1.0.1"
expect-type@^1.1.0:
version "1.3.0"
resolved "https://registry.yarnpkg.com/expect-type/-/expect-type-1.3.0.tgz#0d58ed361877a31bbc4dd6cf71bbfef7faf6bd68"
integrity sha512-knvyeauYhqjOYvQ66MznSMs83wmHrCycNEN6Ao+2AeYEfxUIkuiVxdEa1qlGEPK+We3n0THiDciYSsCcgW/DoA==
extend-shallow@^2.0.1:
version "2.0.1"
resolved "https://registry.npmjs.org/extend-shallow/-/extend-shallow-2.0.1.tgz"
@@ -5984,6 +6103,11 @@ loose-envify@^1.4.0:
dependencies:
js-tokens "^3.0.0 || ^4.0.0"
loupe@^3.1.0, loupe@^3.1.2:
version "3.2.1"
resolved "https://registry.yarnpkg.com/loupe/-/loupe-3.2.1.tgz#0095cf56dc5b7a9a7c08ff5b1a8796ec8ad17e76"
integrity sha512-CdzqowRJCeLU72bHvWqwRBBlLcMEtIvGrlvef74kMnV2AolS9Y8xUv1I0U/MNAWMhBlKIoyuEgoJ0t/bbwHbLQ==
lowlight@^1.17.0:
version "1.20.0"
resolved "https://registry.npmjs.org/lowlight/-/lowlight-1.20.0.tgz"
@@ -6020,7 +6144,7 @@ lucide-react@^0.511.0:
resolved "https://registry.npmjs.org/lucide-react/-/lucide-react-0.511.0.tgz"
integrity sha512-VK5a2ydJ7xm8GvBeKLS9mu1pVK6ucef9780JVUjw6bAjJL/QXnd4Y0p7SPeOUMC27YhzNCZvm5d/QX0Tp3rc0w==
magic-string@^0.30.19:
magic-string@^0.30.12, magic-string@^0.30.19:
version "0.30.21"
resolved "https://registry.npmjs.org/magic-string/-/magic-string-0.30.21.tgz"
integrity sha512-vd2F4YUyEXKGcLHoq+TEyCjxueSeHnFxyyjNp80yg0XV4vUhnDer/lvvlqM/arB5bXQN5K2/3oinyCRyx8T2CQ==
@@ -7026,11 +7150,21 @@ path-type@^3.0.0:
dependencies:
pify "^3.0.0"
pathe@^1.1.2:
version "1.1.2"
resolved "https://registry.yarnpkg.com/pathe/-/pathe-1.1.2.tgz#6c4cb47a945692e48a1ddd6e4094d170516437ec"
integrity sha512-whLdWMYL2TwI08hn8/ZqAbrVemu0LNaNNJZX73O6qaIdCTfXutsLhMkjdENX0qhsQ9uIimo4/aQOmXkoon2nDQ==
pathe@^2.0.3:
version "2.0.3"
resolved "https://registry.npmjs.org/pathe/-/pathe-2.0.3.tgz"
integrity sha512-WUjGcAqP1gQacoQe+OBJsFA7Ld4DyXuUIjZ5cc75cLHvJ7dtNsTugphxIADwspS+AraAUePCKrSVtPLFj/F88w==
pathval@^2.0.0:
version "2.0.1"
resolved "https://registry.yarnpkg.com/pathval/-/pathval-2.0.1.tgz#8855c5a2899af072d6ac05d11e46045ad0dc605d"
integrity sha512-//nshmD55c46FuFw26xV/xFAaB5HF9Xdap7HJBBnrKdAd6/GxDBaNA1870O79+9ueg61cZLSVc+OaFlfmObYVQ==
pbf@^4.0.1:
version "4.0.1"
resolved "https://registry.npmjs.org/pbf/-/pbf-4.0.1.tgz"
@@ -8041,6 +8175,11 @@ side-channel@^1.1.0:
side-channel-map "^1.0.1"
side-channel-weakmap "^1.0.2"
siginfo@^2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/siginfo/-/siginfo-2.0.0.tgz#32e76c70b79724e3bb567cb9d543eb858ccfaf30"
integrity sha512-ybx0WO1/8bSBLEWXZvEd7gMW3Sn3JFlW3TvX1nREbDLRNQNaeNN8WK0meBwPdAaOI7TtRRRJn/Es1zhrrCHu7g==
signal-exit@^3.0.2:
version "3.0.7"
resolved "https://registry.npmjs.org/signal-exit/-/signal-exit-3.0.7.tgz"
@@ -8150,6 +8289,11 @@ split@^1.0.0:
dependencies:
through "2"
stackback@0.0.2:
version "0.0.2"
resolved "https://registry.yarnpkg.com/stackback/-/stackback-0.0.2.tgz#1ac8a0d9483848d1695e418b6d031a3c3ce68e3b"
integrity sha512-1XMJE5fQo1jGH6Y/7ebnwPOBEkIEnT4QF32d5R1+VXdXveM0IBMJt8zfaxX1P3QhVwrYe+576+jkANtSS2mBbw==
standard-version@^9.5.0:
version "9.5.0"
resolved "https://registry.npmjs.org/standard-version/-/standard-version-9.5.0.tgz"
@@ -8175,6 +8319,11 @@ state-local@^1.0.6:
resolved "https://registry.npmjs.org/state-local/-/state-local-1.0.7.tgz"
integrity sha512-HTEHMNieakEnoe33shBYcZ7NX83ACUjCu8c40iOGEZsngj9zRnkqS9j1pqQPXwobB0ZcVTk27REb7COQ0UR59w==
std-env@^3.8.0:
version "3.10.0"
resolved "https://registry.yarnpkg.com/std-env/-/std-env-3.10.0.tgz#d810b27e3a073047b2b5e40034881f5ea6f9c83b"
integrity sha512-5GS12FdOZNliM5mAOxFRg7Ir0pWz8MdpYm6AY6VPkGpbA7ZzmbzNcBJQ0GPvvyWgcY7QAhCgf9Uy89I03faLkg==
stop-iteration-iterator@^1.1.0:
version "1.1.0"
resolved "https://registry.npmjs.org/stop-iteration-iterator/-/stop-iteration-iterator-1.1.0.tgz"
@@ -8430,11 +8579,21 @@ tiny-warning@^1.0.3:
resolved "https://registry.npmjs.org/tiny-warning/-/tiny-warning-1.0.3.tgz"
integrity sha512-lBN9zLN/oAf68o3zNXYrdCt1kP8WsiGW8Oo2ka41b2IM5JL/S1CTyX1rW0mb/zSuJun0ZUrDxx4sqvYS2FWzPA==
tinybench@^2.9.0:
version "2.9.0"
resolved "https://registry.yarnpkg.com/tinybench/-/tinybench-2.9.0.tgz#103c9f8ba6d7237a47ab6dd1dcff77251863426b"
integrity sha512-0+DUvqWMValLmha6lr4kD8iAMK1HzV0/aKnCtWb9v9641TnP/MFb7Pc2bxoxQjTXAErryXVgUOfv2YqNllqGeg==
tinycolor2@^1.6.0:
version "1.6.0"
resolved "https://registry.npmjs.org/tinycolor2/-/tinycolor2-1.6.0.tgz"
integrity sha512-XPaBkWQJdsf3pLKJV9p4qN/S+fm2Oj8AIPo1BTUhg5oxkvm9+SVEGFdhyOz7tTdUTfvxMiAs4sp6/eZO2Ew+pw==
tinyexec@^0.3.1:
version "0.3.2"
resolved "https://registry.yarnpkg.com/tinyexec/-/tinyexec-0.3.2.tgz#941794e657a85e496577995c6eef66f53f42b3d2"
integrity sha512-KQQR9yN7R5+OSwaK0XQoj22pwHoTlgYqmUscPYoknOoWCWfj/5/ABTMRi69FrKU5ffPVh5QcFikpWJI/P1ocHA==
tinyexec@^1.0.0:
version "1.0.2"
resolved "https://registry.npmjs.org/tinyexec/-/tinyexec-1.0.2.tgz"
@@ -8448,11 +8607,26 @@ tinyglobby@^0.2.15:
fdir "^6.5.0"
picomatch "^4.0.3"
tinypool@^1.0.1:
version "1.1.1"
resolved "https://registry.yarnpkg.com/tinypool/-/tinypool-1.1.1.tgz#059f2d042bd37567fbc017d3d426bdd2a2612591"
integrity sha512-Zba82s87IFq9A9XmjiX5uZA/ARWDrB03OHlq+Vw1fSdt0I+4/Kutwy8BP4Y/y/aORMo61FQ0vIb5j44vSo5Pkg==
tinyqueue@^3.0.0:
version "3.0.0"
resolved "https://registry.npmjs.org/tinyqueue/-/tinyqueue-3.0.0.tgz"
integrity sha512-gRa9gwYU3ECmQYv3lslts5hxuIa90veaEcxDYuu3QGOIAEM2mOZkVHp48ANJuu1CURtRdHKUBY5Lm1tHV+sD4g==
tinyrainbow@^1.2.0:
version "1.2.0"
resolved "https://registry.yarnpkg.com/tinyrainbow/-/tinyrainbow-1.2.0.tgz#5c57d2fc0fb3d1afd78465c33ca885d04f02abb5"
integrity sha512-weEDEq7Z5eTHPDh4xjX789+fHfF+P8boiFB+0vbWzpbnbsEr/GRaohi/uMKxg8RZMXnl1ItAi/IUHWMsjDV7kQ==
tinyspy@^3.0.2:
version "3.0.2"
resolved "https://registry.yarnpkg.com/tinyspy/-/tinyspy-3.0.2.tgz#86dd3cf3d737b15adcf17d7887c84a75201df20a"
integrity sha512-n1cw8k1k0x4pgA2+9XrOkFydTerNcJ1zWCO5Nn9scWHTD+5tp8dghT2x1uduQePZTZgd3Tupf+x9BxJjeJi77Q==
tippy.js@^6.3.7:
version "6.3.7"
resolved "https://registry.npmjs.org/tippy.js/-/tippy.js-6.3.7.tgz"
@@ -8838,7 +9012,18 @@ victory-vendor@^36.6.8:
d3-time "^3.0.0"
d3-timer "^3.0.1"
vite@^5.3.1:
vite-node@2.1.9:
version "2.1.9"
resolved "https://registry.yarnpkg.com/vite-node/-/vite-node-2.1.9.tgz#549710f76a643f1c39ef34bdb5493a944e4f895f"
integrity sha512-AM9aQ/IPrW/6ENLQg3AGY4K1N2TGZdR5e4gu/MmmR2xR3Ll1+dib+nook92g4TV3PXVyeyxdWwtaCAiUL0hMxA==
dependencies:
cac "^6.7.14"
debug "^4.3.7"
es-module-lexer "^1.5.4"
pathe "^1.1.2"
vite "^5.0.0"
vite@^5.0.0, vite@^5.3.1:
version "5.4.21"
resolved "https://registry.npmjs.org/vite/-/vite-5.4.21.tgz"
integrity sha512-o5a9xKjbtuhY6Bi5S3+HvbRERmouabWbyUcpXXUA1u+GNUKoROi9byOJ8M0nHbHYHkYICiMlqxkg1KkYmm25Sw==
@@ -8863,6 +9048,32 @@ vite@^7.1.7:
optionalDependencies:
fsevents "~2.3.3"
vitest@^2.1.9:
version "2.1.9"
resolved "https://registry.yarnpkg.com/vitest/-/vitest-2.1.9.tgz#7d01ffd07a553a51c87170b5e80fea3da7fb41e7"
integrity sha512-MSmPM9REYqDGBI8439mA4mWhV5sKmDlBKWIYbA3lRb2PTHACE0mgKwA8yQ2xq9vxDTuk4iPrECBAEW2aoFXY0Q==
dependencies:
"@vitest/expect" "2.1.9"
"@vitest/mocker" "2.1.9"
"@vitest/pretty-format" "^2.1.9"
"@vitest/runner" "2.1.9"
"@vitest/snapshot" "2.1.9"
"@vitest/spy" "2.1.9"
"@vitest/utils" "2.1.9"
chai "^5.1.2"
debug "^4.3.7"
expect-type "^1.1.0"
magic-string "^0.30.12"
pathe "^1.1.2"
std-env "^3.8.0"
tinybench "^2.9.0"
tinyexec "^0.3.1"
tinypool "^1.0.1"
tinyrainbow "^1.2.0"
vite "^5.0.0"
vite-node "2.1.9"
why-is-node-running "^2.3.0"
w3c-keyname@^2.2.0:
version "2.2.8"
resolved "https://registry.npmjs.org/w3c-keyname/-/w3c-keyname-2.2.8.tgz"
@@ -8947,6 +9158,14 @@ which@^2.0.1:
dependencies:
isexe "^2.0.0"
why-is-node-running@^2.3.0:
version "2.3.0"
resolved "https://registry.yarnpkg.com/why-is-node-running/-/why-is-node-running-2.3.0.tgz#a3f69a97107f494b3cdc3bdddd883a7d65cebf04"
integrity sha512-hUrmaWBdVDcxvYqnyh09zunKzROWjbZTiNy8dBEjkS7ehEDQibXJ7XvlmtbwuTclUiIyN+CyXQD4Vmko8fNm8w==
dependencies:
siginfo "^2.0.0"
stackback "0.0.2"
word-wrap@^1.0.3, word-wrap@^1.2.5:
version "1.2.5"
resolved "https://registry.npmjs.org/word-wrap/-/word-wrap-1.2.5.tgz"