116 Commits

Author SHA1 Message Date
dextmorgn
9fcc60ca63 fix(deps): upgrade Python dependencies to patch 27 known vulnerabilities
pip-audit reported 27 vulnerabilities across 12 packages, including
cryptography PYSEC-2026-36 (CVSS 9.8) used by the vault, a starlette
Host-header path bypass, and python-multipart arbitrary file write.

- cryptography 45.0.7 -> 46.0.7 (pin was capped at <46)
- fastapi 0.115 -> 0.136, pulling starlette 0.46 -> 1.2
- python-multipart 0.0.20 -> 0.0.32
- pyjwt, urllib3, aiohttp, lxml, idna, mako and transitive bumps
- dev tools: pytest 8 -> 9 (requires pytest-httpx >=0.36), black 25 -> 26

All four module test suites pass (458 tests). pip-audit now reports
no known vulnerabilities.
2026-06-05 08:57:19 +02:00
dextmorgn
259019f23c fix(build): force LF on container files via .gitattributes
Windows checkouts with core.autocrlf=true convert entrypoint.sh to
CRLF, which Docker copies into the image. The shebang then reads
#!/bin/sh\r and exec fails with 'no such file or directory'.
Also covers .env.example (trailing \r in env values) and Makefile.
2026-06-04 18:23:53 +02:00
dextmorgn
7c082d733e docs(readme): make Windows install commands cmd-compatible
PowerShell foreach/Test-Path loop fails when pasted into cmd.exe.
Plain copy commands work in both shells.
2026-06-04 18:05:19 +02:00
dextmorgn
de93ae3059 docs(readme): add Windows install path without Make
Windows users hit failures with make-based install. Replicate
check-env + build + up targets as plain PowerShell/docker compose
commands for prod and dev.
2026-06-04 18:00:35 +02:00
dextmorgn
c6eaf08b66 Merge pull request #163 from Mubashirrrr/fix/json-import-apostrophe-corruption
fix(import): preserve apostrophes in valid JSON imports
2026-06-03 20:34:42 +02:00
dextmorgn
f380674955 Merge pull request #165 from reconurge/fix/sse-auth-header
fix(sse): auth header
2026-06-03 13:42:17 +02:00
Mubashir Rahim
1fdbdbd094 fix(import): preserve apostrophes in valid JSON imports
parse_json unconditionally replaced every single quote with a double
quote (`.replace("'", '"')`) before calling json.loads. This corrupted
any valid JSON string value containing an apostrophe, e.g. a person's
surname "Sarah O'Brien" became "Sarah O"Brien", causing the parse to
fail with "Invalid JSON" and the whole import to be rejected.

Parse strict JSON first so well-formed payloads (including apostrophes
in string values) import correctly, and only fall back to the lenient
single-quote replacement for Python-dict-style payloads that are not
valid JSON on their own.

Adds regression tests covering an apostrophe-bearing JSON value and the
retained single-quoted fallback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 14:34:22 +05:00
dextmorgn
b317e08e57 feat(api): authenticate SSE via header, drop query-param auth and dead route 2026-06-03 11:28:41 +02:00
dextmorgn
9db1e675d6 test(api): SSE auth-gate and endpoint-removal tests (red) 2026-06-03 11:22:58 +02:00
dextmorgn
4a75d6a924 refactor(app): stream graph status via connectSSE (token out of URL) 2026-06-03 11:08:11 +02:00
dextmorgn
d7f21e9384 refactor(app): stream logs via connectSSE (token out of URL) 2026-06-03 10:56:32 +02:00
dextmorgn
0b60c81361 Merge pull request #155 from Ghraven/fix-investigation-aware-timestamp
fix: use aware timestamp for investigation updates
2026-06-03 09:30:35 +02:00
dextmorgn
6e5f26aaf0 feat(app): add connectSSE helper with header auth and backoff 2026-06-03 09:28:49 +02:00
Ghraven
60cfa16b8d fix: use aware timestamp for investigation updates 2026-06-03 08:37:45 +08:00
dextmorgn
078ac77eee build(app): add fetch-event-source + vitest test runner 2026-06-02 22:45:36 +02:00
dextmorgn
ed8bfc5248 Merge pull request #149 from Ghraven/fix/timezone-aware-service-timestamps
Use timezone-aware UTC timestamps in core services
2026-06-02 21:11:45 +02:00
dextmorgn
438bbe18c4 Merge pull request #150 from Ghraven/fix-social-enricher-output-encoding
fix: read social enricher outputs as utf-8
2026-06-02 21:10:12 +02:00
Ghraven
4e14df6c61 fix: read social enricher outputs as utf-8 2026-06-01 00:32:48 +08:00
dextmorgn
249374e6d7 chore: update .gitignore 2026-05-31 14:53:34 +02:00
dextmorgn
a15ee54af0 chore: update skill name 2026-05-31 14:52:37 +02:00
Ghraven
b707961e68 fix: use timezone-aware service timestamps 2026-05-31 02:35:11 +08:00
dextmorgn
cc829704e5 Merge pull request #148 from reconurge/fix/phase1-critical
fix: security issues
2026-05-30 19:47:55 +02:00
dextmorgn
e8ca7b39bf fix(types): apply extra=allow via Pydantic v2 model_config
FlowsintType declared an inner `class ConfigDict: extra = "allow"`, which
Pydantic v2 treats as an ordinary nested class and ignores entirely. The
base therefore fell back to the default extra="ignore", silently dropping
user-supplied extra properties on every entity type. Replaced with
model_config = ConfigDict(extra="allow"). Extra keys are now retained.

flowsint-types (44), flowsint-enrichers (12) and flowsint-core (390) suites pass.
2026-05-30 19:03:30 +02:00
dextmorgn
1d409d7929 fix(scan): persist results to existing details column, errors to error column
The Scan model exposes details (JSON) and error (Text) columns, but the
enricher/flow tasks assigned to scan.results — a non-mapped attribute.
SQLAlchemy accepted the assignment silently, so commit() never persisted
it and the data was lost. Success paths now write scan.details; failure
paths write scan.error. ScanCreate.results renamed to details to match.

Scan repository tests pass.
2026-05-30 19:02:14 +02:00
dextmorgn
ee06d6cf3e fix(api): restrict CORS origins via ALLOWED_ORIGINS env
allow_origins was ["*"] together with allow_credentials=True, which is
both insecure and rejected by browsers for credentialed requests. Read a
comma-separated ALLOWED_ORIGINS env var instead, defaulting to the Vite
dev origin http://localhost:5173. Documented in .env.example.
2026-05-30 18:49:44 +02:00
dextmorgn
dd00bae9ce fix(api): run production container as non-root flowsint user
The production stage created the flowsint user and chowned all app
files to it, but the USER directive was commented out, so the API
ran as root. Uncommented it. Verified: image builds, container reports
whoami=flowsint and /health returns 200.
2026-05-30 18:49:43 +02:00
dextmorgn
c9d594a8e7 fix(api): load AUTH_SECRET from validated core.auth source
deps.py read AUTH_SECRET via os.getenv() before load_dotenv() ran,
yielding None when the secret was only present in .env. Import the
already-validated AUTH_SECRET from core.auth (loads dotenv first and
raises if unset) instead of re-reading it. Removes the dup + ordering bug.
2026-05-30 18:44:07 +02:00
dextmorgn
de4f4ea87f Update README with new image links
Added additional image links to the README.
2026-05-28 00:00:13 +02:00
dextmorgn
67ba8cfcc6 feat(chore): update SKILL.md 2026-05-21 11:09:28 +02:00
dextmorgn
8fbcf0c025 chore(skills): add flowsint-transform-builder claude skill
Project-scoped Claude Code skill guiding enricher and type creation.
Whitelists .claude/skills/flowsint-transform-builder/ in .gitignore so
the skill ships with the repo while keeping the rest of .claude/ ignored.
2026-05-15 18:55:56 +02:00
dextmorgn
d431ef4fae docs: sync with flowsint-types and flowsint-enrichers (v1.2.8)
- available-enrichers.mdx: add 17 missing enrichers, drop phantom entries
- managing-enrichers.mdx: fix stale Troubleshooting referencing EnricherRegistry.register()
- managing-types.mdx: note 8 registered-but-uncategorized types
- bump all frontmatter to version 1.2.8 / 2026-05-15
2026-05-15 18:55:25 +02:00
dextmorgn
5be480d4f5 Merge pull request #138 from z3rodaycve/feature/collection-enrichers
Release of my collection of Flowsint enrichers
2026-05-10 17:58:55 +02:00
dextmorgn
431d3997f4 Merge pull request #142 from harmsolo13/fix/celery-healthcheck
fix: replace celery container's inherited curl-based healthcheck
2026-04-29 11:35:17 +02:00
harmsolo13
d4dbe8d0fe fix: replace celery container's inherited curl-based healthcheck
The celery container builds from flowsint-api/Dockerfile which carries
a HEALTHCHECK directive that does `curl -f http://localhost:5001/health`.
The API container has an HTTP server on 5001 — celery doesn't, it's a
worker. So the inherited healthcheck always fails and `docker ps` shows
celery as (unhealthy) even when the worker is actively processing jobs.

This is cosmetic noise today but bites in two real ways: (1) restart
policies that key off health won't re-up celery on a real failure
because Docker can't tell good unhealthy from bad unhealthy, (2) any
service that adds `depends_on: celery: condition: service_healthy`
will refuse to start.

Fix: add a service-level healthcheck on celery in both compose files
(prod and dev) that uses celery's own `inspect ping` primitive
against the worker's broker. Compose-level healthcheck overrides the
Dockerfile-level one, so no Dockerfile change needed.

Smoke-tested locally: container goes from (unhealthy) to (healthy)
within ~30s of restart with no other changes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 18:13:26 +09:30
z3rodaycve
f79403d361 Fixed to_ssl enricher to use httpX Docker Tool 2026-04-24 19:49:20 +02:00
z3rodaycve
0d04423b59 Release of DeHashed, httpX, veriphone, scamalytics enrichers 2026-04-21 16:49:12 +02:00
dextmorgn
62e34c6e9b chore: update README.md with discord link 2026-04-19 13:18:55 +02:00
dextmorgn
2a4878c8fc chore: update gh workflow 2026-04-11 15:20:07 +02:00
dextmorgn
6a14484ccd Merge pull request #136 from reconurge/feat/rbac
Feat/rbac
2026-04-11 14:51:34 +02:00
dextmorgn
47b0c05115 feat(app): RBAC permission gating, UserAvatar component, share dialog redesign and responsive grids
- Add UserAvatar and AvatarGroup components to /ui/avatar, migrate all consumers
- Display collaborator avatars in top navbar with AvatarGroup
- Gate edit actions behind canEdit: toolbar (connect, merge, layouts), nodes-view,
  nodes-panel, relationships-view (checkboxes + delete), edge-details-panel,
  graph-navigation (Add tab), import sheet, analysis creation
- Add role badge (owner/admin/editor/viewer) with colored icons in case-header
- Redesign share-dialog: role badges as dropdown triggers, skeleton loading,
  owner/member separation, hover-reveal delete button
- Switch all card grids from viewport breakpoints to CSS container queries
  (dashboard-stats, investigations-list, sketches-section, metrics-grid, custom-types)
- Extend container query classes in styles.css for all needed column counts
2026-04-11 14:44:29 +02:00
dextmorgn
8cd6b237bb fix(core/tests): update failing test 2026-04-11 14:44:29 +02:00
dextmorgn
ce1f50bb72 chore: update yarn.lock 2026-04-11 14:44:28 +02:00
dextmorgn
f5e6fb79ae feat(app): RBAC permission gating, share dialog, profile page and user display
- Add useCan hook reading permissions from $investigationId layout route loader
- Add $investigationId layout route to fetch investigation and share data with children
- Add share dialog with user search autocomplete (debounced) for collaborator management
- Add /dashboard/profile page for editing first name, last name, avatar URL
- Gate all edit/create/delete actions based on user role (deny by default)
  - Graph: node drag, quick add, link creation, context menus, detail panel editing
  - Analysis: editor readonly, title readonly, actions hidden for viewers
  - Investigation: delete (owner only), share (admin+), new sketch/analysis (editor+)
  - Settings: sketch info readonly, delete sketch and import hidden for viewers
  - Selected items panel: enrich, delete, ask AI hidden for viewers
- Fix nav-user to show real user data instead of hardcoded placeholder
- Add user-display utils (getDisplayName, getInitials) shared across components
- Add avatar display in nav-user, case-header collaborators and share dialog
2026-04-11 14:44:28 +02:00
dextmorgn
bb502379bd feat(api): add collaborator endpoints, profile update, user search and backfill migration
- Add GET/POST/PUT/DELETE /investigations/{id}/collaborators endpoints
- Inject current_user_role in all investigation responses
- Add GET/PUT /auth/me for profile management
- Add GET /auth/users/search for share dialog autocomplete
- Add ProfileUpdate schema with extra=forbid
- Add email to ProfileRead
- Add backfill migration for existing investigations missing owner role entries
2026-04-11 14:44:28 +02:00
dextmorgn
1a5d164805 feat(core): add RBAC with ADMIN role, collaborator management and auth user data
- Add ADMIN role to Role enum (OWNER > ADMIN > EDITOR > VIEWER)
- Update permission matrix: ADMIN can read/create/update/manage
- Fix get_with_relations bug filtering by owner_id (blocked non-owner collaborators)
- Add collaborator management methods to InvestigationRepository and InvestigationService
- Return user profile data in auth response
2026-04-11 14:44:28 +02:00
dextmorgn
400b21a7d1 feat(api): install uv Dockerfile 2026-04-11 14:42:39 +02:00
dextmorgn
85a85ed541 chore: switch to uv 2026-04-11 13:17:14 +02:00
dextmorgn
00ec9d6f26 feat(types): switch to uv 2026-04-11 13:15:06 +02:00
dextmorgn
cfaece5af2 feat(enrichers): switch to uv 2026-04-11 13:14:53 +02:00
dextmorgn
6f3af239b2 feat(core): switch to uv 2026-04-11 13:14:41 +02:00
dextmorgn
5569a2f9b7 feat(api): switch to uv 2026-04-11 13:14:16 +02:00
dextmorgn
c5525a8f38 Merge pull request #129 from z3rodaycve/feature/hudsonrock-enrichers
HudsonRock Integration Enrichers for Infostealer related data
2026-04-09 11:50:33 +02:00
dextmorgn
a38a773657 Merge pull request #128 from hillw3318-ui/tags-input
feat(app): Added 'tags-input'. Changed the text input for aliases in details panel
2026-04-08 11:28:40 +02:00
dextmorgn
6852ee3213 fix: missing AUTH_SECRET 2026-04-06 10:02:58 +02:00
dextmorgn
609c957677 feat(enrichers): remove heavy logging on whoxy enrichers 2026-03-16 11:15:16 +01:00
dextmorgn
976bd0ae99 feat(app): vertical link rendering 2026-03-16 11:15:16 +01:00
dextmorgn
4f5c24295c Merge pull request #131 from reconurge/dextmorgn-patch-1
Update README with support badges
2026-03-16 11:14:00 +01:00
dextmorgn
57e72cb337 Update README with support badges
Added support badges for Buy Me A Coffee and Ko-fi.
2026-03-16 11:13:44 +01:00
William Hill
1d376b8ac0 Merge branch 'reconurge:main' into tags-input 2026-03-16 00:32:00 +03:00
bjst0ne
def29144eb fix(app): fixed lists/arrays render in 'details-panel' 2026-03-16 00:37:03 +03:00
dextmorgn
3ef942a2bc feat(app): fix double-click detection and graphRef initialization
Thread MouseEvent through onBackgroundClick for double-click detection.
Use stable callback ref to avoid ForceGraph2D stale closure issues.
Fix graphRef not being set on first load by notifying parent immediately
after ForceGraph2D mounts. Close quick-add input on blur.
2026-03-15 13:34:34 +01:00
dextmorgn
1ee06f5c8a feat(types): format phone numbers to avoid doublons 2026-03-15 13:07:01 +01:00
dextmorgn
37cdd93257 feat(types): detect individual from spaces 2026-03-15 13:06:40 +01:00
dextmorgn
5292a3c541 feat(app): add quick-add node on double-click
Double-click on the graph background opens an inline input at that position.
Typing triggers debounced type detection via the API, showing the detected
type as a badge. On Enter, a node is created at the click position with the
detected type and persisted to the API. Escape or click outside cancels.
2026-03-15 12:37:46 +01:00
dextmorgn
972763343e feat(api): add type detection route and service
Add POST /api/types/detect endpoint that detects the type of a text input
by iterating registered types and calling their detect() classmethod.
Returns detected type with fields and primary field pre-filled.
Falls back to Phrase when no type matches.
2026-03-15 12:35:46 +01:00
dextmorgn
97ef7e4695 feat(app): add alt+drag edge creation between nodes
Allow creating edges by holding Alt and dragging from one node to another.
On release, the edge is added to the graph and the edge context menu opens
with the label input focused and selected, ready for editing. The edge is
only persisted to the API on Enter/submit; Escape removes it from the store.
2026-03-15 11:51:05 +01:00
bjst0ne
3e8e4c8423 Merge branch 'main' into tags-input 2026-03-10 13:37:48 +03:00
bjst0ne
cd67ae369b refactor(app): changed props for 'tags-input' 2026-03-10 12:54:16 +03:00
z3rodaycve
ff9d270ddd Release of HudsonRockToUsername, HudsonRockToPhone, HudsonRockToEmail 2026-03-10 10:24:39 +01:00
dextmorgn
aacbd5ac85 feat(app): add input and output type to enricher template 2026-03-09 19:39:33 +01:00
dextmorgn
6b35cccddb feat(core): add input and output type to template generator 2026-03-09 19:39:08 +01:00
dextmorgn
2ea40c4eae feat(api): add input and output type to enricher template 2026-03-09 19:38:42 +01:00
dextmorgn
611bc1a563 feat(app): improve node and edge rendering 2026-03-08 20:01:55 +01:00
dextmorgn
a2cc352235 feat: update gitignore for DS Store 2026-03-08 19:58:57 +01:00
dextmorgn
16df6b9928 feat(app): update status indicator height 2026-03-04 21:50:15 +01:00
dextmorgn
ffd44968ad feat(app): update toolbars layout 2026-03-04 19:53:50 +01:00
dextmorgn
bb7c955eab feat(app): update details panel 2026-03-04 19:33:02 +01:00
dextmorgn
fd8ddcfd77 Merge pull request #126 from gustavorps/fix/proxy-api
fix(api): Better support to reverse proxy deploy
2026-03-04 09:21:02 +01:00
dextmorgn
ddc36eac4d Merge pull request #125 from gustavorps/feat/soft-delete
feat(core): Soft delete
2026-03-02 20:50:11 +01:00
bjst0ne
5cdd8e70af feat(app): add copy for lists in details-panel 2026-03-02 02:33:57 +03:00
bjst0ne
15878019fa feat(app): add tags-input for list fields in creation form 2026-03-02 02:05:05 +03:00
bjst0ne
523a387661 refactor(app): change tags-input to stacked variant 2026-03-01 19:06:11 +03:00
bjst0ne
171c606e07 fix(app): reduced the height of tags-input 2026-03-01 19:06:11 +03:00
bjst0ne
22400f2e20 refactor(app): change styles to general design 2026-03-01 19:06:11 +03:00
bjst0ne
04e1c2f542 refactor(app): changed condition for aliases input 2026-03-01 19:06:10 +03:00
bjst0ne
ae81c216b0 feat(app): added 'tags-input'. Changed the text input for aliases to tags-input in the details panel 2026-03-01 19:06:10 +03:00
dextmorgn
f49e05eafb fix(core): check if input type is list 2026-02-28 21:29:37 +01:00
dextmorgn
86584ba84e fix(core): prevent sketch break on sick node 2026-02-28 18:35:57 +01:00
dextmorgn
ab13a8e2b0 feat(app/enricher-template): io bar in template enricher layout 2026-02-28 11:52:56 +01:00
dextmorgn
7a2ea266d5 feat(app/enricher-template): io bar to specify input and output type 2026-02-28 11:52:31 +01:00
gustavorps
a574d4ebea feat(core): soft delete 2026-02-20 15:41:46 -03:00
gustavorps
b2cfc67117 fix(app): removing slashes 2026-02-17 04:49:46 -03:00
gustavorps
5ee7bf24e0 fix(api): added redirect_slashes=False to face 307 redirect reverse proxy issue 2026-02-17 04:27:13 -03:00
gustavorps
da0280506e fix(api): added ignore_trailing_slash=True to face 307 redirect reverse proxy issue 2026-02-17 04:21:01 -03:00
gustavorps
d52669740b feat(api): added fastapi standard 2026-02-17 04:04:07 -03:00
dextmorgn
606bcced4b feat(enrichers): add delay to dummy enricher 2026-02-16 09:16:26 +01:00
dextmorgn
6494cee2d0 feat(api): update key service 2026-02-16 09:16:05 +01:00
dextmorgn
c7275594cb refactor(core): use permissions in base service 2026-02-16 09:15:19 +01:00
dextmorgn
4d97932604 feat(app): check chat key exists 2026-02-16 09:14:29 +01:00
dextmorgn
0fa48d3fc1 feat(migrations): backfill_edge_sketch_id 2026-02-15 18:08:59 +01:00
dextmorgn
517b342fc2 feat(enrichers): add delay on whoxy requests 2026-02-15 18:07:58 +01:00
dextmorgn
c05ad758d8 fix(core): missing sketch_id in relationships 2026-02-15 18:07:34 +01:00
dextmorgn
c481a14747 feat(app): path finder 2026-02-15 15:25:35 +01:00
dextmorgn
635f342aba feat(app): remove spaces from custom types 2026-02-15 12:57:29 +01:00
dextmorgn
d6324208a3 feat(app): remove feature flag 2026-02-14 18:00:53 +01:00
dextmorgn
490878ab06 feat(core): return no enricher for custom types 2026-02-14 18:00:27 +01:00
dextmorgn
9ddb708744 Merge pull request #123 from reconurge/fix/custom-types
Fix/custom types
2026-02-14 15:35:59 +01:00
dextmorgn
9d1d5c1416 feat(app): update custom types 2026-02-14 15:35:20 +01:00
dextmorgn
9d1991f45f feat(core): use type service 2026-02-14 15:34:58 +01:00
dextmorgn
eef214bdfa feat(api): custom types updates 2026-02-14 15:34:28 +01:00
dextmorgn
1db16ec13a feat: update yarn.lock 2026-02-14 15:34:04 +01:00
dextmorgn
2032aeee0b feat: update makefile with new commands 2026-02-14 15:32:29 +01:00
dextmorgn
92e5a7097a fix(app): radio group 2026-02-13 13:18:00 +01:00
dextmorgn
b9fa1bbde9 fix(enrichers): individual to domains 2026-02-13 13:17:44 +01:00
dextmorgn
dcce008b8d feat(app): minimal ui updates 2026-02-13 11:23:52 +01:00
dextmorgn
b6f2f8be68 Merge pull request #122 from reconurge/feat/enricher-templates
Feat/enricher templates
2026-02-12 23:53:22 +01:00
261 changed files with 20567 additions and 39221 deletions

View File

@@ -0,0 +1,178 @@
---
name: flowsint-enricher-builder
description: Expert guidance for building Flowsint enrichers and their supporting types. Use when the user wants to add a new enricher, create a new Flowsint type, wire a new external API/tool into Flowsint, debug type/enricher discovery, or design a pivot from entity A to entity B. Knows where types live, how the enricher base class works, how vault secrets and params resolve, and when to recommend creating a new type instead of forcing data into an existing one.
---
# Flowsint Enricher Builder
You build enrichers and types for Flowsint. You do not memorize the catalog — you know where to look and how the pieces fit. Always read source before generating code: type definitions and existing enrichers are the ground truth.
## Authoritative source paths
Read these first. Never assume signatures or fields — open the file.
| What | Path |
|---|---|
| Type definitions | `flowsint-types/src/flowsint_types/<name>.py` |
| Type registry + decorator | `flowsint-types/src/flowsint_types/registry.py` |
| Type package exports | `flowsint-types/src/flowsint_types/__init__.py` |
| Enricher base class | `flowsint-core/src/flowsint_core/core/enricher_base.py` |
| Enricher registry + decorator | `flowsint-enrichers/src/flowsint_enrichers/registry.py` |
| Existing enrichers (templates) | `flowsint-enrichers/src/flowsint_enrichers/<input_type>/to_<output>.py` |
| UI category mapping | `flowsint-core/src/flowsint_core/core/services/type_registry_service.py` (`_get_category_definitions`) |
| Vault interface | `flowsint-core/src/flowsint_core/core/vault.py` |
| Logger interface | `flowsint-core/src/flowsint_core/core/logger.py` |
| Tools (external CLI/API wrappers) | `tools/` (top-level), e.g. `tools.network.subfinder.SubfinderTool` |
| Doc — types tutorial | `docs/developers/managing-types.mdx` |
| Doc — enrichers tutorial | `docs/developers/managing-enrichers.mdx` |
| Doc — enricher catalog | `docs/sources/available-enrichers.mdx` |
## The first question: new type or reuse?
When the user describes a enricher, decide before writing code:
1. **List the entities involved** — input data, output data, intermediate fields you'll attach.
2. **For each, check `flowsint-types/src/flowsint_types/`** — open the closest candidate file and read its fields.
3. **Decide:**
- **Reuse** if existing type covers all required fields (extras allowed — `ConfigDict.extra = "allow"`).
- **Extend an existing type** if 12 fields are missing — propose adding optional fields to the existing model.
- **Create new type** if the entity is conceptually distinct (different primary key, different label semantics, different graph role).
4. **Never cram data into a wrong type.** If a "Domain" enricher returns risk scores, a `RiskProfile` exists — don't stuff scores into `Domain` metadata. If nothing fits, propose a new type and tell the user why.
Surface the decision to the user before generating code: list candidate types you found, what's missing, and your recommendation.
## Anatomy of an enricher
Minimum surface (read `enricher_base.py` for the full contract):
```python
from typing import List
from flowsint_core.core.enricher_base import Enricher
from flowsint_core.core.logger import Logger
from flowsint_enrichers.registry import flowsint_enricher
from flowsint_types import Domain, Ip # or whatever types
@flowsint_enricher
class MyEnricher(Enricher):
"""[Source name] One-line purpose."""
InputType = Domain # base type, not List[Domain]
OutputType = Ip
@classmethod
def name(cls) -> str: return "domain_to_ip" # snake_case, unique
@classmethod
def category(cls) -> str: return "Domain" # see note on casing below
@classmethod
def key(cls) -> str: return "domain" # primary field of InputType
@classmethod
def get_params_schema(cls): # optional, only if params needed
return [...]
async def scan(self, data: List[InputType]) -> List[OutputType]:
...
def postprocess(self, results, input_data):
for src, dst in zip(input_data, results):
self.create_node(src)
self.create_node(dst)
self.create_relationship(src, dst, "RESOLVES_TO")
return results
InputType = MyEnricher.InputType
OutputType = MyEnricher.OutputType
```
**File location:** `flowsint-enrichers/src/flowsint_enrichers/<input_type>/to_<target>.py`. The directory matches the input type's lowercase name. If no directory exists for your input type, create it (no `__init__.py` needed — auto-discovery walks the tree).
**Registration:** the `@flowsint_enricher` decorator does it. Do not edit any `registry.py`. API restart picks up new files via `load_all_enrichers()`.
## Params and secrets
Defined via `get_params_schema()` classmethod. Each entry is a dict:
| Field | Required | Notes |
|---|---|---|
| `name` | yes | Param key; for `vaultSecret`, also the default vault key name |
| `type` | yes | One of `string`, `number`, `select`, `url`, `vaultSecret` |
| `description` | yes | Shown in UI |
| `required` | no | Defaults to `false` |
| `default` | no | Default value |
| `options` | for `select` | List of `{"label": ..., "value": ...}` |
Read params inside `scan()`:
```python
mode = self.params.get("mode", "passive")
api_key = self.get_secret("MY_API_KEY") # vault-resolved during async_init
```
**Vault resolution flow** (see `Enricher.resolve_params` in `enricher_base.py`):
1. If user passed a vault ID in params → vault looked up by that ID.
2. Else → vault looked up by the param name (e.g. `MY_API_KEY`).
3. If `required: true` and nothing found → `Exception("Required vault secret 'MY_API_KEY' is missing...")`.
**Never hardcode keys.** Always declare a `vaultSecret` param. Document the expected vault key name in the docstring.
## Graph operations (postprocess)
`create_node(obj)` and `create_relationship(from_obj, to_obj, rel_label="IS_RELATED_TO")` take Pydantic objects directly. Don't manually construct node dicts — pass the typed instance.
Relationship label convention: `UPPER_SNAKE_CASE` verb phrase (`HAS_DOMAIN`, `RESOLVES_TO`, `FOUND_IN_BREACH`). Be consistent with existing enrichers — grep before inventing a new label.
`self.log_graph_message("...")` for graph-related progress logs. `Logger.info / error / warn(self.sketch_id, {"message": "..."})` for general logs.
## Creating a new type — checklist
When you decide a new type is warranted:
1. **File**: `flowsint-types/src/flowsint_types/<snake_case>.py`.
2. **Class**: `PascalCase`, inherit from `FlowsintType`, decorate with `@flowsint_type`.
3. **Exactly one primary field**: `Field(..., json_schema_extra={"primary": True})`. Must uniquely identify the entity (used as Neo4j MERGE key).
4. **`compute_label`**: `@model_validator(mode='after')`, sets `self.nodeLabel`, returns `self`. Handle `None` for optional fields.
5. **Export in `__init__.py`**: add import + entry in `__all__`.
6. **Category** (optional but recommended): add a `("MyType", "primary_field_name", icon)` tuple in `_get_category_definitions()` in `type_registry_service.py`. Without this, the type works as an enricher I/O but doesn't show in the UI type picker.
7. **Reinstall**: `cd flowsint-types && poetry install` (or `make prod` from repo root).
8. **Test**: write a `tests/test_<name>.py` covering creation, primary uniqueness, `compute_label` with full/partial fields.
Full template + patterns: `docs/developers/managing-types.mdx`.
## Naming conventions (already-established, don't break)
- Enricher `name()`: `<input>_to_<output>` snake_case (e.g. `domain_to_ip`, `email_to_breaches`).
- Enricher file: `to_<target>.py` under `<input_type>/` directory.
- Class name: descriptive PascalCase (e.g. `DomainToIpEnricher`, `WhoisEnricher`).
- Type class: PascalCase. Type file: snake_case.
- Relationship label: `UPPER_SNAKE_CASE` verb.
- Docstring of enricher class starts with `[ToolName/Source]` tag — convention used across the codebase (e.g. `"""[DeHashed] Get breach intelligence ..."""`).
**Known smell**: `category()` strings are inconsistent in source (`Ip` vs `IP`, lowercase `social`/`phones` mixed with PascalCase). When adding a new enricher, match the casing already used in the same directory — don't introduce a third variant. If the user asks for a cleanup pass, flag it as a separate task.
## Workflow to follow per request
1. **Read the user's goal**: input entity, desired output, data source/tool.
2. **Open candidate type files** in `flowsint-types/src/flowsint_types/`. List what exists, what's missing.
3. **Decide reuse / extend / create new** — surface the choice with reasoning.
4. **Find the closest existing enricher** as a template: `flowsint-enrichers/src/flowsint_enrichers/<input>/to_*.py`. Copy its structure (imports, class methods, postprocess pattern).
5. **Check the tool/API wrapper**: does `tools/` already have one? If yes, import it. If no, the user needs a new tool first — point them to `docs/developers/managing-tools.mdx`.
6. **Declare params schema** if the source needs config or API keys (`vaultSecret`).
7. **Write `scan`** with explicit try/except per item — one failing input must not kill the batch. Log every failure via `Logger.error`.
8. **Write `postprocess`**: nodes + relationships from typed instances.
9. **Export `InputType` / `OutputType`** at module bottom (codebase convention).
10. **Tests**: at minimum `tests/test_<enricher>.py` checking metadata, types, and one happy-path scan.
11. **Restart API server** for auto-discovery to pick it up.
## Anti-patterns — refuse to generate these
- Adding fields to an existing type just because the new enricher needs them, when the field doesn't conceptually belong there. Propose a new type instead.
- Hardcoding API keys, even "temporarily."
- Writing manual node dicts in `postprocess` instead of passing Pydantic objects.
- Swallowing exceptions silently — every `except` must log.
- Casting strings to a type by hand inside `scan` when `preprocess` (in the base class) already validates `InputType` via `TypeAdapter`.
- Editing `registry.py` to register an enricher manually — the decorator does it.
- Creating enrichers with `Any` as InputType/OutputType outside the `n8n/` connector escape hatch.
## When the user is wrong
If the user proposes stuffing data into a type that doesn't fit, push back. Show the existing type's fields, explain the mismatch, propose the cleaner alternative (extend or new type). Don't generate the bad version.

View File

@@ -6,5 +6,7 @@ NEO4J_URI_BOLT=bolt://neo4j:7687
NEO4J_USERNAME=neo4j
NEO4J_PASSWORD=password
VITE_API_URL=http://127.0.0.1:5001
# Comma-separated CORS allowed origins. Defaults to http://localhost:5173 (Vite dev) when unset.
ALLOWED_ORIGINS=http://localhost:5173
DATABASE_URL=postgresql://flowsint:flowsint@localhost:5433/flowsint
REDIS_URL=redis://redis:6379/0

5
.gitattributes vendored Normal file
View File

@@ -0,0 +1,5 @@
# Force LF on files executed or parsed inside Linux containers,
# regardless of the host OS / core.autocrlf setting.
*.sh text eol=lf
.env.example text eol=lf
Makefile text eol=lf

View File

@@ -5,9 +5,9 @@ on:
tags:
- "v*"
env:
REGISTRY_DOCKERHUB: docker.io
REGISTRY_GHCR: ghcr.io
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: false
jobs:
build-frontend:
@@ -52,7 +52,7 @@ jobs:
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=raw,value=latest,enable={{is_default_branch}}
type=raw,value=latest
- name: Build and push
uses: docker/build-push-action@v6
@@ -70,9 +70,9 @@ jobs:
- name: Run Trivy vulnerability scanner
id: trivy
uses: aquasecurity/trivy-action@master
uses: aquasecurity/trivy-action@v0.35.0
with:
image-ref: ghcr.io/${{ github.repository_owner }}/flowsint-app:${{ github.ref_name }}
image-ref: ghcr.io/${{ github.repository_owner }}/flowsint-app:${{ steps.meta.outputs.version }}
format: "sarif"
output: "trivy-frontend.sarif"
severity: "CRITICAL,HIGH"
@@ -125,7 +125,7 @@ jobs:
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=raw,value=latest,enable={{is_default_branch}}
type=raw,value=latest
- name: Build and push
uses: docker/build-push-action@v6
@@ -144,9 +144,9 @@ jobs:
- name: Run Trivy vulnerability scanner
id: trivy
uses: aquasecurity/trivy-action@master
uses: aquasecurity/trivy-action@v0.35.0
with:
image-ref: ghcr.io/${{ github.repository_owner }}/flowsint-api:${{ github.ref_name }}
image-ref: ghcr.io/${{ github.repository_owner }}/flowsint-api:${{ steps.meta.outputs.version }}
format: "sarif"
output: "trivy-backend.sarif"
severity: "CRITICAL,HIGH"

7
.gitignore vendored
View File

@@ -2,7 +2,7 @@
__pycache__/
*.py[cod]
*$py.class
.DS_Store
/node_modules/
# C extensions
*.so
@@ -175,4 +175,7 @@ cython_debug/
.pypirc
/certs
.claude/
.claude/*
!.claude/skills/
.claude/skills/*
!.claude/skills/flowsint-enricher-builder/

1
.python-version Normal file
View File

@@ -0,0 +1 @@
3.12

View File

@@ -10,16 +10,14 @@ COMPOSE_DEPLOY := docker compose -f docker-compose.deploy.yml
up-dev up-prod up-deploy down \
infra-dev infra-prod infra-stop-dev infra-stop-prod \
migrate-dev migrate-prod \
alembic-upgrade alembic-downgrade alembic-revision \
api frontend celery \
test install clean check-env open-browser-dev open-browser-prod \
logs-dev logs-prod logs-deploy status
logs-dev logs-prod logs-deploy status \
regenerate-router
ENV_DIRS := . flowsint-api flowsint-core flowsint-app
# =============================================================================
# Environment Setup
# =============================================================================
check-env:
@echo "Checking .env files..."
@for dir in $(ENV_DIRS); do \
@@ -31,10 +29,6 @@ check-env:
fi; \
done
# =============================================================================
# Development
# =============================================================================
dev:
@echo "Starting DEV environment..."
$(MAKE) check-env
@@ -68,10 +62,6 @@ open-browser-dev:
xdg-open http://localhost:5173 2>/dev/null || \
echo "Frontend ready at http://localhost:5173"
# =============================================================================
# Production
# =============================================================================
prod:
@echo "Starting PROD environment..."
$(MAKE) check-env
@@ -107,10 +97,6 @@ open-browser-prod:
xdg-open http://localhost 2>/dev/null || \
echo "Frontend ready at http://localhost"
# =============================================================================
# Deploy (GHCR images)
# =============================================================================
deploy:
@echo "Starting DEPLOY environment (GHCR images)..."
$(MAKE) check-env
@@ -127,10 +113,6 @@ up-deploy:
logs-deploy:
$(COMPOSE_DEPLOY) logs -f
# =============================================================================
# Migrations
# =============================================================================
migrate-dev:
@echo "Running DEV migrations..."
@if ! $(COMPOSE_DEV) ps -q neo4j | grep -q .; then \
@@ -148,42 +130,42 @@ migrate-prod:
fi
yarn migrate
# =============================================================================
# Local Development (without Docker)
# =============================================================================
alembic-upgrade:
@echo "Running Alembic migrations (upgrade head)..."
cd $(PROJECT_ROOT)/flowsint-api && uv run alembic upgrade head
alembic-downgrade:
@echo "Rolling back last Alembic migration..."
cd $(PROJECT_ROOT)/flowsint-api && uv run alembic downgrade -1
alembic-revision:
@if [ -z "$(m)" ]; then \
echo "Usage: make alembic-revision m=\"your migration message\""; exit 1; \
fi
@echo "Creating new Alembic migration: $(m)"
cd $(PROJECT_ROOT)/flowsint-api && uv run alembic revision --autogenerate -m "$(m)"
api:
cd $(PROJECT_ROOT)/flowsint-api && \
poetry run uvicorn app.main:app --host 0.0.0.0 --port 5001 --reload
uv run uvicorn app.main:app --host 0.0.0.0 --port 5001 --reload
frontend:
cd $(PROJECT_ROOT)/flowsint-app && yarn dev
celery:
cd $(PROJECT_ROOT)/flowsint-api && \
poetry run celery -A flowsint_core.core.celery \
uv run celery -A flowsint_core.core.celery \
worker --loglevel=info --pool=threads --concurrency=10
# =============================================================================
# Testing & Installation
# =============================================================================
test:
cd flowsint-types && poetry run pytest
cd flowsint-core && poetry run pytest
cd flowsint-enrichers && poetry run pytest
cd flowsint-types && uv run pytest
cd flowsint-core && uv run pytest
cd flowsint-enrichers && uv run pytest
install:
poetry config virtualenvs.in-project true --local
$(MAKE) infra-dev
poetry install
cd flowsint-core && poetry install
cd flowsint-enrichers && poetry install
cd flowsint-api && poetry install && poetry run alembic upgrade head
# =============================================================================
# Utilities
# =============================================================================
uv sync
cd flowsint-api && uv run alembic upgrade head
status:
@echo "=== DEV Containers ==="
@@ -205,13 +187,11 @@ clean:
-$(COMPOSE_PROD) down -v --rmi all --remove-orphans
-$(COMPOSE_DEPLOY) down -v --rmi all --remove-orphans
rm -rf flowsint-app/node_modules
rm -rf flowsint-core/.venv
rm -rf flowsint-enrichers/.venv
rm -rf flowsint-api/.venv
rm -rf .venv
# =============================================================================
# Help
# =============================================================================
regenerate-router:
@echo "Regenerating flowsint-app/src/routeTree.gen.ts"
cd $(PROJECT_ROOT)/flowsint-app && npx tsr generate
help:
@echo "Flowsint Makefile"
@@ -239,6 +219,13 @@ help:
@echo " make frontend - Run frontend locally"
@echo " make celery - Run Celery worker locally"
@echo ""
@echo "Migrations:"
@echo " make migrate-dev - Run Neo4j DEV migrations"
@echo " make migrate-prod - Run Neo4j PROD migrations"
@echo " make alembic-upgrade - Run Alembic migrations (upgrade head)"
@echo " make alembic-downgrade - Rollback last Alembic migration"
@echo " make alembic-revision m=.. - Create new Alembic migration"
@echo ""
@echo "Utilities:"
@echo " make status - Show container status"
@echo " make down - Stop all containers"

View File

@@ -1,15 +1,27 @@
# Flowsint
[![License: Apache-2.0](https://img.shields.io/badge/License-Apache--2.0-blue.svg)](./LICENSE)
[![License](https://img.shields.io/badge/License-Apache--2.0-blue.svg)](./LICENSE)
[![Ethical Software](https://img.shields.io/badge/ethical-use-blue.svg)](./ETHICS.md)
[![Buy Me A Coffee](https://img.shields.io/badge/Buy%20Me%20a%20coffee-support-FFDD00?logo=buy-me-a-coffee&logoColor=black)](https://www.buymeacoffee.com/dextmorgn)
[![Ko-fi](https://img.shields.io/badge/Ko--fi-support-F16061?logo=ko-fi&logoColor=white)](https://ko-fi.com/P5P01W3GPJ)
[![Discord](https://img.shields.io/badge/Discord-Join%20Server-5865F2?logo=discord&logoColor=white)](https://discord.gg/aST9HMQr)
Flowsint is an open-source OSINT graph exploration tool designed for ethical investigation, transparency, and verification.
**Ethics:** Please read [ETHICS.md](./ETHICS.md) for responsible use guidelines.
<img width="1439" height="899" alt="hero-dark" src="https://github.com/user-attachments/assets/01eb128e-bef4-486e-9276-c4da58f829ae" />
<img width="1511" height="946" alt="Capture décran 2026-01-13 à 09 15 58" src="https://github.com/user-attachments/assets/d1a9eca6-9ec4-4402-93f4-303c3dc30de1" />
<img width="1511" height="948" alt="Capture décran 2026-01-13 à 09 19 45" src="https://github.com/user-attachments/assets/6d9e9e6d-d8c7-4ed2-8b8c-53a945b28d05" />
https://github.com/user-attachments/assets/eaabfa81-d7b3-414d-8cf7-f69b4e37bab6
https://github.com/user-attachments/assets/7457d94a-cf1d-4a97-949f-f9b1d8d92644
https://github.com/user-attachments/assets/65c3f26e-7132-4853-be45-21b8933688bd
## Contributing
@@ -19,6 +31,8 @@ Flowsint is still in early development and definetly needs the help of the commu
Don't want to read ? Got it. Here's your install instructions:
### Linux / macOS
#### 1. Install pre-requisites
- Docker
@@ -32,6 +46,35 @@ cd flowsint
make prod
```
### Windows
No Make needed. Works in both **Command Prompt (cmd)** and **PowerShell**.
#### 1. Install pre-requisites
- [Docker Desktop](https://docs.docker.com/desktop/setup/install/windows-install/) (make sure it is **running** before the next step)
- Git
#### 2. Clone and set up environment files
```bat
git clone https://github.com/reconurge/flowsint.git
cd flowsint
copy .env.example .env
copy .env.example flowsint-api\.env
copy .env.example flowsint-core\.env
copy .env.example flowsint-app\.env
```
#### 3. Build and start
```bat
docker compose -f docker-compose.prod.yml up -d --build
```
### First login
Then go to [http://localhost:5173/register](http://localhost:5173/register) and create an account. There are no credentials or account by default.
@@ -132,12 +175,19 @@ flowsint-types (types)
### Run
Make sure you have **Make** installed.
**Linux / macOS** (requires **Make**):
```bash
make dev
```
**Windows** (cmd or PowerShell, no Make — create the `.env` files first, see [Get started](#windows)):
```bat
docker compose -f docker-compose.dev.yml up -d --build
docker compose -f docker-compose.dev.yml logs -f
```
### Development
The app is accessible at [http://localhost:5173](http://localhost:5173).
@@ -206,19 +256,19 @@ Each module has its own (incomplete) test suite:
```bash
# Test core module
cd flowsint-core
poetry run pytest
uv run pytest
# Test types module
cd ../flowsint-types
poetry run pytest
uv run pytest
# Test enrichers module
cd ../flowsint-enrichers
poetry run pytest
uv run pytest
# Test API module
cd ../flowsint-api
poetry run pytest
uv run pytest
```
## Contributing
@@ -250,3 +300,7 @@ It was created to assist:
Any misuse of this software is strictly prohibited and goes against the ethical principles defined in [ETHICS.md](./ETHICS.md).
## ❤️ Support
[![Buy Me A Coffee](https://img.shields.io/badge/Buy%20Me%20a%20coffee-support-FFDD00?logo=buy-me-a-coffee&logoColor=black)](https://www.buymeacoffee.com/dextmorgn)
[![Ko-fi](https://img.shields.io/badge/Ko--fi-support-F16061?logo=ko-fi&logoColor=white)](https://ko-fi.com/P5P01W3GPJ)

View File

@@ -122,6 +122,15 @@ services:
- MASTER_VAULT_KEY_V1=${MASTER_VAULT_KEY_V1}
- REDIS_URL=redis://redis:6379/0
- SKIP_MIGRATIONS=true
- AUTH_SECRET=${AUTH_SECRET}
healthcheck:
# Celery has no HTTP server — Dockerfile's curl-based healthcheck always fails.
# Use celery's own ping primitive instead.
test: ["CMD-SHELL", "celery -A flowsint_core.core.celery inspect ping -d celery@$$HOSTNAME || exit 1"]
interval: 30s
timeout: 10s
retries: 3
start_period: 30s
depends_on:
postgres:
condition: service_healthy

View File

@@ -122,6 +122,15 @@ services:
- MASTER_VAULT_KEY_V1=${MASTER_VAULT_KEY_V1}
- REDIS_URL=redis://redis:6379/0
- SKIP_MIGRATIONS=true
- AUTH_SECRET=${AUTH_SECRET}
healthcheck:
# Celery has no HTTP server — Dockerfile's curl-based healthcheck always fails.
# Use celery's own ping primitive instead.
test: ["CMD-SHELL", "celery -A flowsint_core.core.celery inspect ping -d celery@$$HOSTNAME || exit 1"]
interval: 30s
timeout: 10s
retries: 3
start_period: 30s
depends_on:
postgres:
condition: service_healthy

View File

@@ -0,0 +1,18 @@
---
title: "Getting started"
description: "This guide explains how you can contribute to Flowsint."
category: "Developers"
order: 7
author: "Flowsint Team"
tags: ["tutorial", "developers", "getting-started"]
version: "1.2.8"
last_updated_at: "2026-05-15"
---
## Contributing
Your contribution to Flowsint is highly encouraged !
You can create your own [Types](/docs/developers/managing-types), [Tools](/docs/developers/managing-tools), [Enrichers](/docs/developers/managing-enrichers) and Flows, and sharing them by proposing [pull requests](https://github.com/reconurge/flowsint/pulls).
Before diving in, you may also want to understand the [Graph format](/docs/developers/graph-format) to learn how nodes and edges are structured in the frontend visualization.

View File

@@ -0,0 +1,244 @@
---
title: "Graph format"
description: "Reference for the GraphNode and GraphEdge types used to represent entities and relationships in the Flowsint graph. Understanding these structures is essential for working with the frontend visualization and the graph database layer."
category: "Developers"
order: 11
author: "Flowsint Team"
tags: ["tutorial", "developers", "graph", "nodes", "edges"]
version: "1.2.8"
last_updated_at: "2026-05-15"
---
## Overview
Flowsint's graph visualization is built on two core data structures: **GraphNode** (representing entities) and **GraphEdge** (representing relationships between entities). These types are defined in `flowsint-app/src/types/graph.ts` and are used throughout the frontend for rendering, interaction, and data management.
Understanding these structures is important when:
- Building enrichers that create nodes and relationships
- Working on the frontend visualization
- Debugging graph rendering issues
- Extending the graph with new visual features
## GraphNode
A `GraphNode` represents a single entity in the graph (e.g., a domain, an IP address, a person). Every node created by an enricher's `create_node()` method ends up as a `GraphNode` in the frontend.
```typescript
type GraphNode = {
id: string
nodeType: string
nodeLabel: string
nodeProperties: NodeProperties
nodeSize: number
nodeColor: string | null
nodeIcon: keyof typeof LucideIcons | null
nodeImage: string | null
nodeFlag: flagColor | null
nodeShape: NodeShape | null
nodeMetadata: NodeMetadata
x: number
y: number
val?: number
neighbors?: any[]
links?: any[]
}
```
### Field reference
| Field | Type | Description |
|-------|------|-------------|
| `id` | `string` | Unique identifier for the node. Derived from the type's primary field value. |
| `nodeType` | `string` | The entity type (e.g., `"Domain"`, `"Ip"`, `"Email"`). Maps to the Pydantic type class name. |
| `nodeLabel` | `string` | Human-readable label displayed on the node. Set by the type's `compute_label()` method. |
| `nodeProperties` | `NodeProperties` | All properties of the entity as key-value pairs. Contains the serialized fields from the Pydantic type. |
| `nodeSize` | `number` | Visual size of the node in the graph. |
| `nodeColor` | `string \| null` | Custom color override for the node (CSS color string), or `null` for default. |
| `nodeIcon` | `keyof LucideIcons \| null` | Icon to display on the node, from the [Lucide icon set](https://lucide.dev/icons/), or `null` for default. |
| `nodeImage` | `string \| null` | URL to an image to display on the node (e.g., a profile picture), or `null`. |
| `nodeFlag` | `flagColor \| null` | Color flag for visual tagging (see Flag colors below), or `null`. |
| `nodeShape` | `NodeShape \| null` | Shape of the node (see Node shapes below), or `null` for default circle. |
| `nodeMetadata` | `NodeMetadata` | Additional metadata as key-value pairs. Used for internal tracking and extended features. |
| `x` | `number` | X coordinate position in the graph canvas. |
| `y` | `number` | Y coordinate position in the graph canvas. |
| `val` | `number` (optional) | Value used by the force-graph engine for sizing calculations. |
| `neighbors` | `any[]` (optional) | Adjacent nodes, populated by the graph engine at runtime. |
| `links` | `any[]` (optional) | Connected edges, populated by the graph engine at runtime. |
### NodeProperties
```typescript
type NodeProperties = {
[key: string]: any
}
```
A flexible key-value object containing all the entity's properties. When a Pydantic type is serialized into a node, its fields become entries in `nodeProperties`. For example, a `Domain` node would have `nodeProperties.domain`, `nodeProperties.root`, etc.
### NodeMetadata
```typescript
type NodeMetadata = {
[key: string]: any
}
```
Similar to `nodeProperties` but used for system-level metadata rather than entity data. This can include information like creation timestamps, source enricher, or other tracking data.
### Node shapes
Nodes can be rendered in four shapes:
```typescript
type NodeShape = 'circle' | 'square' | 'hexagon' | 'triangle'
```
| Shape | Use case |
|-------|----------|
| `circle` | Default shape for most entity types |
| `square` | Often used for infrastructure entities |
| `hexagon` | Used for grouped or aggregate entities |
| `triangle` | Used for alert or warning-related entities |
### Flag colors
Flags provide a visual tagging system for nodes. Users can flag nodes to highlight them during an investigation.
```typescript
type flagColor = 'red' | 'orange' | 'blue' | 'green' | 'yellow'
```
Each flag color maps to specific Tailwind CSS classes for consistent styling:
| Flag | Style |
|------|-------|
| `red` | `text-red-400 fill-red-200` |
| `orange` | `text-orange-400 fill-orange-200` |
| `blue` | `text-blue-400 fill-blue-200` |
| `green` | `text-green-400 fill-green-200` |
| `yellow` | `text-yellow-400 fill-yellow-200` |
## GraphEdge
A `GraphEdge` represents a relationship between two nodes (e.g., "RESOLVES_TO", "HAS_SUBDOMAIN", "FOUND_IN_BREACH"). Every relationship created by an enricher's `create_relationship()` method ends up as a `GraphEdge` in the frontend.
```typescript
type GraphEdge = {
source: GraphNode['id']
target: GraphNode['id']
date?: string
id: string
label: string
caption?: string
type?: string
weight?: number
confidence_level?: number | string
}
```
### Field reference
| Field | Type | Description |
|-------|------|-------------|
| `source` | `string` | ID of the source node (the "from" node in the relationship). |
| `target` | `string` | ID of the target node (the "to" node in the relationship). |
| `id` | `string` | Unique identifier for this edge. |
| `label` | `string` | Display label for the edge (e.g., `"RESOLVES_TO"`, `"HAS_EMAIL"`). This is the relationship type string passed to `create_relationship()`. |
| `date` | `string` (optional) | Timestamp associated with the relationship (ISO 8601 format). |
| `caption` | `string` (optional) | Additional descriptive text displayed on or near the edge. |
| `type` | `string` (optional) | Relationship classification type for filtering or styling. |
| `weight` | `number` (optional) | Numerical weight of the relationship, can be used for visual thickness or importance ranking. |
| `confidence_level` | `number \| string` (optional) | Confidence score for the relationship, indicating how reliable the connection is. |
## How enrichers map to the graph
When you write an enricher and call graph methods in `postprocess()`, here's how the data flows:
### Creating nodes
```python
# In your enricher's postprocess method
self.create_node(domain) # domain is a Pydantic Domain object
```
The graph service automatically:
1. Extracts the **type name** from the Pydantic class (e.g., `"Domain"`) -> `nodeType`
2. Reads the **primary field** value (e.g., `domain.domain`) -> used for `id`
3. Reads the **nodeLabel** field (set by `compute_label()`) -> `nodeLabel`
4. Serializes all fields into `nodeProperties`
5. Sets defaults for visual properties (`nodeSize`, `nodeColor`, etc.)
### Creating relationships
```python
# In your enricher's postprocess method
self.create_relationship(domain, ip, "RESOLVES_TO")
```
The graph service automatically:
1. Identifies the **source node** from `domain`'s type and primary field -> `source`
2. Identifies the **target node** from `ip`'s type and primary field -> `target`
3. Uses the relationship label `"RESOLVES_TO"` -> `label`
4. Generates a unique `id` for the edge
### Relationship naming conventions
Relationship labels follow these conventions:
- Use **UPPER_SNAKE_CASE** (e.g., `"RESOLVES_TO"`, `"HAS_SUBDOMAIN"`)
- Use **verb phrases** that describe the relationship direction (source -> target)
- The default relationship label is `"IS_RELATED_TO"` if none is specified
- Common patterns:
- `HAS_*` for ownership/containment (e.g., `"HAS_EMAIL"`, `"HAS_SUBDOMAIN"`)
- `RESOLVES_TO` for DNS-type lookups
- `FOUND_IN_*` for breach/leak associations
- `REGISTERED_BY` for WHOIS data
## Graph settings
The graph visualization is configurable through settings types:
### ForceGraphSetting
Controls the physics simulation of the force-directed graph layout:
```typescript
type ForceGraphSetting = {
value: any
min?: number
max?: number
step?: number
type?: string
description?: string
}
```
### ExtendedSetting
General-purpose settings with richer type information:
```typescript
type ExtendedSetting = {
value: any
type: string
min?: number
max?: number
step?: number
options?: { value: string; label: string }[]
description?: string
}
```
These settings allow users to customize graph behavior like node repulsion, link distance, simulation speed, and visual preferences.
## Reserved properties
When creating nodes, certain property names are reserved by the system and should not be used as field names in your custom types:
- `id` - Node identifier
- `x`, `y` - Position coordinates
- `nodeLabel` - Display label (set by `compute_label()`)
- `label` - Legacy label field
- `nodeType`, `type` - Entity type identifiers
- `nodeImage`, `nodeIcon`, `nodeColor`, `nodeSize` - Visual properties
- `created_at` - Creation timestamp
- `sketch_id` - Investigation sketch association

View File

@@ -0,0 +1,715 @@
---
title: "Managing enrichers"
description: "Quick start guide to creating Enricher for your OSINT investigations."
category: "Developers"
order: 10
author: "Flowsint Team"
tags: ["tutorial", "developers", "creating-a-new-enricher"]
version: "1.2.8"
last_updated_at: "2026-05-15"
---
## Understanding Enrichers
Enrichers are the high-level business logic layer in Flowsint. While types define data structures and tools wrap external utilities, enrichers orchestrate the entire intelligence gathering workflow. An enricher takes input data of one type, processes it through various tools or APIs, validates and enriches the results, creates graph database nodes and relationships, and returns structured output.
Every enricher in Flowsint follows a two-phase execution model. The scan phase contains the core enriching logic where tools are executed, APIs are called, and data is gathered. Then, the postprocessing phase creates Neo4j graph nodes and relationships while returning the processed results. Input validation and normalization happens automatically through Pydantic type validation.
Enrichers differ from tools in several fundamental ways. Tools are low-level wrappers that return raw data without knowledge of the Flowsint ecosystem. Enrichers are high-level workflows that understand types, create graph nodes, handle parameters, and orchestrate multiple tools. When you want to add a new data source, you create a tool. When you want to add a new intelligence workflow, you create an enricher.
## Enricher architecture
The enricher system is built around an abstract base class that defines the interface and execution flow. Every enricher you create inherits from this base class and implements specific methods.
### The Enricher base class
The base class lives at `flowsint-core/src/flowsint_core/core/enricher_base.py` and provides the framework for all enrichers. Here's what a minimal enricher looks like:
```python
from typing import List
from flowsint_core.core.enricher_base import Enricher
from flowsint_enrichers.registry import flowsint_enricher
from flowsint_types import Domain, Ip
@flowsint_enricher
class MyEnricher(Enricher):
"""Description of what this enricher does."""
# Define input and output types as base types (not lists)
InputType = Domain
OutputType = Ip
@classmethod
def name(cls) -> str:
"""Unique identifier for this enricher."""
return "domain_to_ip"
@classmethod
def category(cls) -> str:
"""Category this enricher belongs to."""
return "Domain"
@classmethod
def key(cls) -> str:
"""Primary key field name for this enricher."""
return "domain"
async def scan(self, data: List[InputType]) -> List[OutputType]:
"""Core enriching logic."""
pass
def postprocess(self, results: List[OutputType], input_data: List[InputType]) -> List[OutputType]:
"""Create graph nodes and relationships."""
pass
# Export types for easy access
InputType = MyEnricher.InputType
OutputType = MyEnricher.OutputType
```
The `InputType` and `OutputType` class attributes define what data the enricher accepts and returns. These should be Pydantic types from the `flowsint-types` package defined as base types (e.g., `Domain`, not `List[Domain]`). The base class uses these type definitions to automatically generate JSON schemas for the API and handle validation automatically.
At the end of the file, you should export the types for easy access by other modules.
### The two phases
Understanding the two execution phases is crucial for writing effective enrichers.
**Scanning** is where the real work happens. This async method receives validated input data as a list of `InputType` instances and executes your intelligence gathering logic. You might instantiate tools, call external APIs, process results, and build up your output data. This phase should focus purely on gathering and processing data without worrying about the graph database. The base class automatically handles input validation through Pydantic before the scan phase begins.
**Postprocessing** creates the graph database structure. After scanning completes, this method receives both the results and the original input. It creates Neo4j nodes for each entity, establishes relationships between them, and returns the final results. This separation keeps graph logic separate from business logic.
## Creating a simple enricher
Let's walk through creating a complete enricher from scratch. We'll build an enricher that converts domains to IP addresses using DNS resolution.
### Setting up the file structure
Enrichers are organized by their input type. Create a new file in the appropriate directory under `flowsint-enrichers/src/flowsint_enrichers/`:
```bash
cd flowsint-enrichers/src/flowsint_enrichers/domain/
touch to_ip.py
```
If you're creating an enricher for a new input type, you may need to create a new directory first.
### Implementing the basic structure
Start with the imports and class definition:
```python
import socket
from typing import List
from flowsint_enrichers.registry import flowsint_enricher
from flowsint_core.core.enricher_base import Enricher
from flowsint_core.core.logger import Logger
from flowsint_types import Domain, Ip
@flowsint_enricher
class DomainToIpEnricher(Enricher):
"""Resolves domain names to their IP addresses using DNS."""
# Define types as base types (not lists)
InputType = Domain
OutputType = Ip
@classmethod
def name(cls) -> str:
return "domain_to_ip"
@classmethod
def category(cls) -> str:
return "Domain"
@classmethod
def key(cls) -> str:
return "domain"
@classmethod
def documentation(cls) -> str:
return """
This enricher resolves domain names to their IP addresses using
standard DNS queries. It accepts a list of domains and returns
the corresponding IP addresses.
"""
# Export types at the end of the file
InputType = DomainToIpEnricher.InputType
OutputType = DomainToIpEnricher.OutputType
```
The `name()` method returns a unique identifier for this enricher. Use lowercase with underscores, following the pattern `inputtype_to_outputtype`. The `category()` groups related enrichers together in the UI. The `key()` specifies which field serves as the primary identifier, typically matching the input type.
### Implementing the scan logic
The scan method contains your core intelligence gathering logic. It receives a list of validated `InputType` instances and returns a list of `OutputType` instances.
```python
async def scan(self, data: List[InputType]) -> List[OutputType]:
"""
Resolve each domain to its IP address.
Args:
data: List of Domain objects to resolve
Returns:
List of Ip objects
"""
results: List[OutputType] = []
for domain in data:
try:
# Perform DNS resolution
ip_address = socket.gethostbyname(domain.domain)
# Create IP object
ip = Ip(address=ip_address)
results.append(ip)
# Log successful resolution
Logger.info(
self.sketch_id,
{"message": f"Resolved {domain.domain} to {ip_address}"}
)
except socket.gaierror as e:
# DNS resolution failed
Logger.info(
self.sketch_id,
{"message": f"Failed to resolve {domain.domain}: {e}"}
)
continue
except Exception as e:
# Unexpected error
Logger.error(
self.sketch_id,
{"message": f"Error resolving {domain.domain}: {e}"}
)
continue
return results
```
This implementation iterates through each domain, performs DNS resolution, creates an IP object for successful resolutions, and logs both successes and failures. The error handling ensures that failures don't crash the entire enricher, which is important when processing many domains.
The input data has already been validated by Pydantic before reaching the scan method, so you can trust that all items are proper `Domain` objects.
### Implementing postprocessing
The postprocess method creates graph database nodes and relationships using the new simplified API:
```python
def postprocess(self, results: List[OutputType], input_data: List[InputType] = None) -> List[OutputType]:
"""
Create graph nodes and relationships.
Args:
results: IP objects from scan phase
input_data: Original Domain objects (preprocessed input)
Returns:
IP objects (unchanged)
"""
# Create nodes and relationships
for domain, ip in zip(input_data, results):
# Create nodes by passing Pydantic objects directly
self.create_node(domain)
self.create_node(ip)
# Create relationship by passing Pydantic objects directly
self.create_relationship(domain, ip, "RESOLVES_TO")
# Log the operation
self.log_graph_message(
f"IP found for domain {domain.domain} -> {ip.address}"
)
return results
```
You can pass Pydantic objects directly to `create_node()` and `create_relationship()`. The methods automatically infer the node types, primary keys, and property values from the Pydantic models.
The `create_node()` method accepts a Pydantic object and automatically creates a Neo4j node with the correct label and properties. The `create_relationship()` method takes two Pydantic objects and a relationship type string, inferring all necessary information from the objects.
## Creating an enricher with tools
Most enrichers use external tools for data gathering. Let's create an enricher that uses the Subfinder tool for subdomain enumeration.
### Importing the tool
Start by importing the tool along with your other dependencies:
```python
from typing import List
from flowsint_core.core.enricher_base import Enricher
from flowsint_enrichers.registry import flowsint_enricher
from flowsint_core.core.logger import Logger
from flowsint_types import Domain
from tools.network.subfinder import SubfinderTool
@flowsint_enricher
class SubdomainEnricher(Enricher):
"""Enumerates subdomains for given domains using Subfinder."""
# Define types as base types
InputType = Domain
OutputType = Domain
@classmethod
def name(cls) -> str:
return "domain_to_subdomains"
@classmethod
def category(cls) -> str:
return "Domain"
@classmethod
def key(cls) -> str:
return "domain"
# Export types
InputType = SubdomainEnricher.InputType
OutputType = SubdomainEnricher.OutputType
```
### Using the tool in scan
The scan method instantiates and uses the tool:
```python
async def scan(self, data: List[InputType]) -> List[OutputType]:
"""
Find subdomains using Subfinder tool.
Args:
data: List of Domain objects
Returns:
List of discovered subdomain Domain objects
"""
results: List[OutputType] = []
# Instantiate the tool
subfinder = SubfinderTool()
for domain in data:
Logger.info(
self.sketch_id,
{"message": f"Enumerating subdomains for {domain.domain}"}
)
try:
# Launch the tool
subdomains = subfinder.launch(domain.domain)
# Convert strings to Domain objects
for subdomain in subdomains:
results.append(Domain(domain=subdomain, root=False))
Logger.info(
self.sketch_id,
{"message": f"Found {len(subdomains)} subdomains for {domain.domain}"}
)
except Exception as e:
Logger.error(
self.sketch_id,
{"message": f"Error enumerating subdomains for {domain.domain}: {e}"}
)
continue
return results
```
Notice how the tool returns raw strings, and the enricher converts them into proper Domain objects. This separation of concerns keeps tools simple while enrichers handle type conversion.
### Creating graph nodes and relationships
The postprocess phase creates parent-child relationships between domains and subdomains:
```python
def postprocess(self, results: List[OutputType], input_data: List[InputType]) -> List[OutputType]:
"""
Create graph nodes and relationships for domains and subdomains.
Args:
results: Discovered subdomain Domain objects
input_data: Original parent Domain objects
Returns:
Subdomain Domain objects
"""
# Create nodes for parent domains
for domain in input_data:
self.create_node(domain)
# Create nodes for subdomains and relationships
for subdomain in results:
self.create_node(subdomain)
# Extract parent domain name and create relationship
parent_domain_name = self._extract_parent_domain(subdomain.domain)
parent_domain = Domain(domain=parent_domain_name)
# Create relationship using Pydantic objects
self.create_relationship(parent_domain, subdomain, "HAS_SUBDOMAIN")
# Log the operation
self.log_graph_message(
f"Subdomain found: {parent_domain_name} -> {subdomain.domain}"
)
return results
def _extract_parent_domain(self, subdomain: str) -> str:
"""Extract parent domain from subdomain."""
parts = subdomain.split('.')
if len(parts) >= 2:
return '.'.join(parts[-2:])
return subdomain
```
## Adding parameters to enrichers
Many enrichers need user-configurable parameters. Let's create an enricher that scans ports with configurable options.
### Defining the parameter schema
The `get_params_schema()` class method defines what parameters your enricher accepts:
```python
from typing import List, Dict, Any, Optional
from flowsint_enrichers.registry import flowsint_enricher
from flowsint_core.core.enricher_base import Enricher
from flowsint_types import Ip, Port
from tools.network.naabu import NaabuTool
@flowsint_enricher
class IpToPortsEnricher(Enricher):
"""Scans IP addresses for open ports."""
# Define types as base types
InputType = Ip
OutputType = Port
@classmethod
def name(cls) -> str:
return "ip_to_ports"
@classmethod
def category(cls) -> str:
return "IP"
@classmethod
def key(cls) -> str:
return "address"
@classmethod
def get_params_schema(cls) -> List[Dict[str, Any]]:
"""Define configurable parameters for this enricher."""
return [
{
"name": "mode",
"type": "select",
"description": "Scan mode: active scanning or passive enumeration",
"required": True,
"default": "passive",
"options": [
{"label": "Passive", "value": "passive"},
{"label": "Active", "value": "active"},
],
},
{
"name": "port_range",
"type": "string",
"description": "Port range to scan (e.g., '1-1000' or '80,443,8080')",
"required": False,
},
{
"name": "top_ports",
"type": "select",
"description": "Scan only the most common ports",
"required": False,
"options": [
{"label": "Top 100", "value": "100"},
{"label": "Top 1000", "value": "1000"},
],
},
{
"name": "PDCP_API_KEY",
"type": "vaultSecret",
"description": "ProjectDiscovery Cloud Platform API key for passive mode",
"required": False,
},
]
# Export types
InputType = IpToPortsEnricher.InputType
OutputType = IpToPortsEnricher.OutputType
```
The parameter schema defines the type, description, whether it's required, default values, and for select parameters, the available options. The `vaultSecret` type integrates with Flowsint's encrypted credential storage.
### Using parameters in your enricher
Parameters are accessed through `self.params` in your scan method:
```python
async def scan(self, data: List[InputType]) -> List[OutputType]:
"""
Scan IPs for open ports using configured parameters.
Args:
data: List of Ip objects to scan
Returns:
List of Port objects
"""
results: List[OutputType] = []
# Extract parameters
mode = self.params.get("mode", "passive")
port_range = self.params.get("port_range")
top_ports = self.params.get("top_ports")
api_key = self.get_secret("PDCP_API_KEY")
# Instantiate tool
naabu = NaabuTool()
for ip in data:
Logger.info(
self.sketch_id,
{"message": f"Scanning {ip.address} in {mode} mode"}
)
try:
# Launch tool with parameters
scan_results = naabu.launch(
target=ip.address,
mode=mode,
port_range=port_range,
top_ports=top_ports,
api_key=api_key
)
# Convert tool results to Port objects
for result in scan_results:
port = Port(
number=result.get("port"),
protocol=result.get("protocol", "tcp").upper(),
state="open",
service=result.get("service"),
banner=result.get("version")
)
results.append(port)
except Exception as e:
Logger.error(
self.sketch_id,
{"message": f"Error scanning {ip.address}: {e}"}
)
continue
return results
```
## Handling multiple output types
Some enrichers produce multiple types of results. You can define a custom return type using Pydantic:
```python
from pydantic import BaseModel
from flowsint_enrichers.registry import flowsint_enricher
from flowsint_core.core.enricher_base import Enricher
from typing import List
from flowsint_types import Website, Email, Phone
class CrawlerResults(BaseModel):
"""Results from web crawler including multiple entity types."""
website: Website
emails: List[Email] = []
phones: List[Phone] = []
@flowsint_enricher
class WebsiteToCrawlerEnricher(Enricher):
"""Crawls websites to extract emails and phone numbers."""
# Define types as base types
InputType = Website
OutputType = CrawlerResults
async def scan(self, data: List[InputType]) -> List[OutputType]:
"""Crawl websites and extract contact information."""
from tools.network.reconcrawl import ReconCrawlTool
results: List[OutputType] = []
crawler_tool = ReconCrawlTool()
for website in data:
try:
# Launch crawler
crawl_data = crawler_tool.launch(website.url)
# Extract entities
emails = [Email(email=e) for e in crawl_data.get("emails", [])]
phones = [Phone(number=p) for p in crawl_data.get("phones", [])]
# Create result object
result = CrawlerResults(
website=website,
emails=emails,
phones=phones
)
results.append(result)
except Exception as e:
Logger.error(self.sketch_id, {"message": f"Crawl error: {e}"})
return results
def postprocess(self, results: List[OutputType], input_data: List[InputType]) -> List[OutputType]:
"""Create nodes for all discovered entities."""
for result in results:
# Create website node using Pydantic object
self.create_node(result.website)
# Create email nodes and relationships
for email in result.emails:
self.create_node(email)
self.create_relationship(result.website, email, "HAS_EMAIL")
# Create phone nodes and relationships
for phone in result.phones:
self.create_node(phone)
self.create_relationship(result.website, phone, "HAS_PHONE")
self.log_graph_message(
f"Processed {len(result.emails)} emails and {len(result.phones)} phones from {result.website.url}"
)
return results
# Export types
InputType = WebsiteToCrawlerEnricher.InputType
OutputType = WebsiteToCrawlerEnricher.OutputType
```
## Registering your enricher
You don't need to register your enricher anywhere, adding the decorator `@flowsint_enricher` to your enricher class triggers the auto discovery.
```python
from flowsint_enrichers.registry import flowsint_enricher
from flowsint_core.core.enricher_base import Enricher
@flowsint_enricher
class MyEnricher(Enricher):
...
```
## Testing your enricher
Creating tests helps ensure your enricher works correctly and makes debugging easier. Create a test file in `flowsint-enrichers/tests/`:
```python
# tests/test_domain_to_ip.py
import pytest
from flowsint_enrichers.domain.to_ip import DomainToIpEnricher
from flowsint_types import Domain, Ip
@pytest.mark.asyncio
async def test_enricher_metadata():
"""Test enricher metadata is correctly defined."""
assert DomainToIpEnricher.name() == "domain_to_ip"
assert DomainToIpEnricher.category() == "Domain"
assert DomainToIpEnricher.key() == "domain"
@pytest.mark.asyncio
async def test_type_definitions():
"""Test InputType and OutputType are correctly defined."""
assert DomainToIpEnricher.InputType == Domain
assert DomainToIpEnricher.OutputType == Ip
@pytest.mark.asyncio
async def test_scan():
"""Test DNS resolution works."""
enricher = DomainToIpEnricher(sketch_id="test", scan_id="test")
input_data = [Domain(domain="example.com")]
results = await enricher.scan(input_data)
assert len(results) > 0
assert isinstance(results[0], Ip)
assert results[0].address # Should have an IP address
```
These tests verify that your enricher's metadata is correct, type definitions are properly set, and the scan logic produces expected results. Input validation is handled automatically by Pydantic, so you don't need to test preprocessing separately.
## Best practices
When creating enrichers, think carefully about error handling. Intelligence gathering involves many external systems that can fail in unpredictable ways. Your enricher should handle errors gracefully, log failures clearly, and continue processing remaining items rather than crashing entirely.
Always use the Logger utility for tracking progress and errors. The logger integrates with Flowsint's monitoring system and helps users understand what's happening during long-running enrichers. Log successful operations at the info level and errors at the error level.
Define InputType and OutputType as base types (e.g., `Domain`, not `List[Domain]`). The base class automatically handles the list wrapping and validation. This makes the type definitions cleaner and more intuitive.
Always export your types at the end of the file using:
```python
InputType = YourEnricher.InputType
OutputType = YourEnricher.OutputType
```
Use the simplified graph API by passing Pydantic objects directly to `create_node()` and `create_relationship()`. This eliminates boilerplate code and reduces errors. The methods automatically infer node types, primary keys, and properties from your Pydantic models.
Separate concerns between the two phases. Scanning should focus on gathering and processing data. Postprocessing should create graph structures. Input validation happens automatically through Pydantic, so you don't need to handle it manually.
Use type hints everywhere. They provide automatic validation, better IDE support, and serve as inline documentation. The InputType and OutputType class attributes should always be Pydantic types from the flowsint-types package.
When working with tools, remember that they return raw data structures. Your enricher is responsible for converting tool output into proper Flowsint types. This type conversion is typically done in the scan phase.
Document your enricher thoroughly. The class docstring, documentation method, and parameter descriptions all appear in the UI. Clear documentation helps users understand what your enricher does and how to configure it.
### Handling API Rate Limits
When working with rate-limited APIs, add delays between requests:
```python
async def scan(self, data: InputType) -> OutputType:
"""Scan with rate limiting to respect API limits."""
import asyncio
results = []
delay_seconds = 1 # Delay between requests
for item in data:
result = await self._query_api(item)
if result:
results.append(result)
# Respect rate limits
await asyncio.sleep(delay_seconds)
return results
```
### Fallback data sources
Implement fallback logic when primary sources fail:
```python
async def scan(self, data: InputType) -> OutputType:
"""Try multiple data sources with fallback logic."""
results = []
for domain in data:
# Try primary source
result = self._query_primary_source(domain)
if not result:
# Fall back to secondary source
Logger.info(
self.sketch_id,
{"message": f"Primary source failed for {domain}, trying fallback"}
)
result = self._query_fallback_source(domain)
if result:
results.append(result)
return results
```
## Troubleshooting
If your enricher doesn't appear in the API, verify that you've applied the `@flowsint_enricher` decorator to your class, that the file lives under `flowsint-enrichers/src/flowsint_enrichers/`, and that you've restarted the API server. Auto-discovery via `load_all_enrichers()` runs at startup, so file additions require a restart.
For import errors, make sure all your dependencies are installed and the enricher file has no syntax errors. Check that you're importing from the correct packages.
If the scan method isn't finding any results, add logging statements to debug what's happening. Verify that tools are installed and accessible, API keys are valid if required, and input data is in the expected format.
When graph relationships aren't appearing, check that you're creating both nodes and relationships in the postprocess method. Verify that the relationship type name is correct and that you're passing the right node objects to `create_relationship()`.
## Next steps
Once you've created and registered your enricher, it becomes available through the API for users to run. Enrichers can be chained together into Flows where the output of one enricher feeds into the input of another. This enables complex intelligence gathering sequences.
Remember that enrichers are the heart of Flowsint's intelligence gathering capabilities. Well-designed enrichers that handle errors gracefully, log progress clearly, and create meaningful graph relationships make the entire platform more powerful and user-friendly.
If you create new enrichers that you think can help the community, it's highly appreciated that you open-source them. Help the community as much as it helps you !

View File

@@ -0,0 +1,512 @@
---
title: "Managing tools"
description: "This guide explains how to create a new tool in the Flowsint ecosystem. Tools are low-level wrappers around external utilities, Docker containers, and APIs that enrichers use to gather intelligence. Understanding the tool architecture will help you extend Flowsint's capabilities with new data sources and reconnaissance utilities."
category: "Developers"
order: 9
author: "Flowsint Team"
tags: ["tutorial", "developers", "creating-a-new-tool"]
version: "1.2.8"
last_updated_at: "2026-05-15"
---
## Understanding tools
Tools in Flowsint serve as abstraction layers between enrichers and external systems. They provide a consistent interface for executing Docker containers, calling APIs, or running python libraries. While enrichers handle high-level orchestration and graph database operations, tools focus exclusively on executing external commands and returning raw results.
Every tool implements a basic interface with methods for naming, categorization, and execution. Tools don't know anything about Pydantic types, Neo4j graphs, or the broader Flowsint architecture. They just wrap external functionality and return data.
Flowsint currently includes tools for subdomain enumeration, port scanning, DNS queries, WHOIS lookups, web crawling, and business intelligence.
## Tool architecture
The tool system has a two-tier inheritance structure. At the base level, you have the abstract `Tool` class that defines the interface every tool must implement. For tools that run in Docker containers, there's an intermediate `DockerTool` class that handles all the container lifecycle management.
### The Tool base class
Every tool inherits from the abstract `Tool` class, which lives at `flowsint-enrichers/src/tools/base.py`. Here's what it looks like:
```python
from abc import ABC, abstractmethod
from typing import Any
class Tool(ABC):
"""Abstract base class for all tools."""
@classmethod
@abstractmethod
def name(cls) -> str:
"""Return the tool name."""
pass
@classmethod
@abstractmethod
def category(cls) -> str:
"""Return the tool category."""
pass
@classmethod
@abstractmethod
def description(cls) -> str:
"""Return a description of what the tool does."""
pass
@classmethod
@abstractmethod
def version(cls) -> str:
"""Return the tool version."""
pass
@abstractmethod
def launch(self, value: str, *args, **kwargs) -> Any:
"""Execute the tool and return results."""
pass
```
Any tool you create must implement these five methods. The first four are class methods that provide metadata about the tool. The `launch` method is where the actual work happens.
### The DockerTool class
Most security and reconnaissance tools run in Docker containers for isolation and portability. The `DockerTool` class at `flowsint-enrichers/src/tools/dockertool.py` provides all the infrastructure for running containerized tools.
When you inherit from `DockerTool`, you get automatic image management, container execution, volume mounting, environment variable handling, and cleanup. You just specify the Docker image name and implement how to construct the command.
Here's a simplified view of what `DockerTool` provides:
```python
class DockerTool(Tool):
"""Base class for tools that run in Docker containers."""
def __init__(self, image: str, default_tag: str = "latest"):
"""Initialize with Docker image information."""
self.image = image
self.default_tag = default_tag
self.docker_client = docker.from_env()
def install(self) -> None:
"""Pull the Docker image if not already present."""
# Pulls image from Docker Hub
pass
def is_installed(self) -> bool:
"""Check if the Docker image exists locally."""
# Checks local images
pass
def launch(self, command: str, volumes: dict = None,
timeout: int = 30, environment: dict = None) -> Any:
"""Run a command in the Docker container."""
# Executes container and returns output
pass
```
The `launch` method in `DockerTool` handles container execution. It sets up the environment, mounts volumes if needed, runs the container, captures output, and cleans up afterward.
## Creating a simple API-based tool
Let's start with the simpler case of creating a tool that calls an external API. We'll create a hypothetical tool for querying a threat intelligence service.
### File structure
Create a new python file in the appropriate category directory under `flowsint-enrichers/src/tools/`. For a security-related tool, you might use `tools/security/`:
```bash
cd flowsint-enrichers/src/tools/security/
touch threat_intel.py
```
If the category directory doesn't exist, create it first and add an `__init__.py` file to make it a python package.
### Basic implementation
Here's a complete example of an API-based tool:
```python
from tools.base import Tool
from typing import Any, Dict, List, Optional
import requests
class ThreatIntelTool(Tool):
"""Query threat intelligence data from an external API."""
api_endpoint = "https://api.threatintel.example.com/v1"
@classmethod
def name(cls) -> str:
"""Return the tool name."""
return "threatintel"
@classmethod
def category(cls) -> str:
"""Return the category this tool belongs to."""
return "Threat Intelligence"
@classmethod
def description(cls) -> str:
"""Return a description of what this tool does."""
return "Queries threat intelligence data for IPs, domains, and hashes"
@classmethod
def version(cls) -> str:
"""Return the tool version."""
return "1.0.0"
def launch(
self,
indicator: str,
indicator_type: str = "ip",
api_key: Optional[str] = None
) -> List[Dict[str, Any]]:
"""
Query the threat intelligence API.
Args:
indicator: The indicator to query (IP, domain, hash, etc.)
indicator_type: Type of indicator (ip, domain, hash)
api_key: API key for authentication
Returns:
List of threat intelligence records
"""
if not api_key:
raise ValueError("API key is required")
headers = {
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json"
}
params = {
"indicator": indicator,
"type": indicator_type
}
try:
response = requests.get(
f"{self.api_endpoint}/query",
headers=headers,
params=params,
timeout=30
)
response.raise_for_status()
return response.json().get("results", [])
except requests.exceptions.RequestException as e:
print(f"Error querying threat intel API: {e}")
return []
```
This tool follows a straightforward pattern. The class methods provide metadata that enrichers and the registry use. The `launch` method implements the actual API interaction, handling authentication, making the request, and returning structured data.
Notice how the tool returns simple python data structures like lists and dictionaries. Tools don't know about Pydantic types or Flowsint models. That's the enricher's job.
## Creating a docker-based tool
Docker-based tools are more common in Flowsint because most reconnaissance utilities need specific dependencies and isolated environments. Let's walk through creating a tool that wraps a hypothetical Docker-based subdomain scanner.
### Setting up the class
Start by inheriting from `DockerTool` and providing the Docker image information:
```python
from tools.dockertool import DockerTool
from typing import List, Optional, Any
class MySubdomainTool(DockerTool):
"""Wrapper for a Docker-based subdomain enumeration tool."""
image = "org/subdomain-scanner"
default_tag = "latest"
def __init__(self):
"""Initialize the tool with Docker image information."""
super().__init__(self.image, self.default_tag)
```
The `image` and `default_tag` class attributes tell `DockerTool` which Docker image to use. When you instantiate the tool, it will automatically connect to the Docker daemon.
### Implementing the launch method
The `launch` method needs to construct the command that runs inside the container and handle the results:
```python
def launch(
self,
domain: str,
timeout: int = 300,
wordlist: Optional[str] = None
) -> List[str]:
"""
Enumerate subdomains for a given domain.
Args:
domain: Target domain to enumerate
timeout: Maximum execution time in seconds
wordlist: Optional path to custom wordlist file
Returns:
List of discovered subdomain strings
"""
# Ensure the Docker image is available
if not self.is_installed():
self.install()
# Build the command that runs inside the container
command = f"-d {domain}"
if wordlist:
command += f" -w {wordlist}"
# Add JSON output flag for easier parsing
command += " -json"
# Execute the container
try:
result = super().launch(
command=command,
timeout=timeout
)
# Parse the output
subdomains = self._parse_output(result)
return subdomains
except Exception as e:
print(f"Error running subdomain scanner: {e}")
return []
def _parse_output(self, output: str) -> List[str]:
"""Parse the tool output and extract subdomains."""
import json
subdomains = []
for line in output.strip().split('\n'):
if not line:
continue
try:
data = json.loads(line)
if 'subdomain' in data:
subdomains.append(data['subdomain'])
except json.JSONDecodeError:
continue
return list(set(subdomains)) # Remove duplicates
```
This implementation shows several important patterns. First, it checks if the Docker image is installed and pulls it if necessary. Second, it constructs the command string that will run inside the container. Third, it calls the parent class's `launch` method to handle the actual container execution. Finally, it parses the output into a clean python data structure.
### Handling volumes
Some tools need access to files on the host system. You can mount volumes when calling the parent's `launch` method:
```python
def launch(self, domain: str, wordlist_path: str = None) -> List[str]:
"""Run the tool with optional wordlist file."""
command = f"-d {domain}"
volumes = None
if wordlist_path:
# Mount the wordlist file into the container
volumes = {
wordlist_path: {
'bind': '/wordlist.txt',
'mode': 'ro' # read-only
}
}
command += " -w /wordlist.txt"
result = super().launch(
command=command,
volumes=volumes
)
return self._parse_output(result)
```
The volumes dictionary maps host paths to container paths. You can specify the mount mode as 'ro' for read-only or 'rw' for read-write.
### Using environment variables
For tools that need API keys or configuration through environment variables:
```python
def launch(self, domain: str, api_key: Optional[str] = None) -> List[str]:
"""Run the tool with optional API key for enhanced scanning."""
command = f"-d {domain}"
environment = {}
if api_key:
environment['API_KEY'] = api_key
result = super().launch(
command=command,
environment=environment
)
return self._parse_output(result)
```
## Testing your tool
Creating tests for your tool helps ensure it works correctly and makes it easier to catch regressions. Create a test file in `flowsint-enrichers/tests/tools/` that mirrors your tool's location:
```python
# tests/tools/security/test_threat_intel.py
from tools.security.threat_intel import ThreatIntelTool
import pytest
def test_tool_metadata():
"""Test that tool metadata is correctly defined."""
assert ThreatIntelTool.name() == "threatintel"
assert ThreatIntelTool.category() == "Threat Intelligence"
assert "threat" in ThreatIntelTool.description().lower()
def test_tool_launch_requires_api_key():
"""Test that launch method requires an API key."""
tool = ThreatIntelTool()
with pytest.raises(ValueError):
tool.launch("192.0.2.1")
def test_tool_launch_with_api_key(monkeypatch):
"""Test successful API query with mocked response."""
tool = ThreatIntelTool()
# Mock the requests.get call
def mock_get(*args, **kwargs):
class MockResponse:
def raise_for_status(self):
pass
def json(self):
return {"results": [{"indicator": "192.0.2.1", "threat_level": "high"}]}
return MockResponse()
monkeypatch.setattr("requests.get", mock_get)
results = tool.launch("192.0.2.1", api_key="test_key")
assert len(results) == 1
assert results[0]["indicator"] == "192.0.2.1"
```
For docker-based tools, your tests need Docker to be running:
```python
# tests/tools/network/test_my_subdomain_tool.py
from tools.network.my_subdomain_tool import MySubdomainTool
import pytest
@pytest.mark.docker
def test_tool_install():
"""Test that the Docker image can be pulled."""
tool = MySubdomainTool()
tool.install()
assert tool.is_installed()
@pytest.mark.docker
def test_tool_launch():
"""Test running the tool against a domain."""
tool = MySubdomainTool()
results = tool.launch("example.com")
assert isinstance(results, list)
```
The `@pytest.mark.docker` decorator helps you separate tests that require Docker from those that don't.
## Best practices
When creating tools, focus on simplicity and single responsibility. Each tool should wrap exactly one external utility or API. Don't try to combine multiple data sources in a single tool. That's what enrichers are for.
Always handle errors gracefully. Network requests fail, Docker containers crash, and APIs return unexpected data. Your tool should catch these errors, log them appropriately, and return empty results or raise clear exceptions rather than crashing.
Return simple data structures from the `launch` method. Use lists, dictionaries, strings, and numbers. Don't return Pydantic models or other complex objects. Remember that tools are low-level utilities that enrichers build upon.
For Docker tools, always check if the image is installed before running it. The pattern of checking `is_installed()` and calling `install()` if necessary ensures the tool works even on fresh installations.
When parsing tool output, be defensive. External tools can return unexpected formats, partial results, or garbage data. Validate and clean the output before returning it. Use try-except blocks around parsing logic.
Document your tool thoroughly. The docstrings and parameter descriptions help other developers understand how to use your tool. Future enrichers will rely on this documentation.
## Integrating your tool
Unlike types, tools don't need to be explicitly registered in a central registry. Enrichers import and use them directly. When you create an enricher that uses your new tool, you simply import it:
```python
# In an enricher file
from tools.security.threat_intel import ThreatIntelTool
class IpToThreatIntelEnricher(Enricher):
async def scan(self, data: List[Ip]) -> List[ThreatReport]:
tool = ThreatIntelTool()
results = []
for ip in data:
intel = tool.launch(
indicator=ip.address,
indicator_type="ip",
api_key=api_key
)
# Process results...
return results
```
The enricher instantiates your tool, calls its `launch` method with appropriate parameters, and processes the results into Flowsint types.
## Common patterns
Several patterns appear frequently in Flowsint tools. Understanding these will help you write tools that fit naturally into the ecosystem.
### The install-check pattern
Most Docker tools follow this pattern at the start of `launch`:
```python
def launch(self, ...):
if not self.is_installed():
self.install()
# Continue with execution
```
This ensures the docker image is available before trying to run it.
### The command builder pattern
Complex tools often build commands incrementally based on parameters:
```python
def launch(self, target: str, mode: str = "fast", verbose: bool = False):
command = f"-target {target}"
if mode == "thorough":
command += " --thorough"
if verbose:
command += " -v"
result = super().launch(command)
```
### The output parser pattern
Many tools separate execution from parsing:
```python
def launch(self, ...):
raw_output = super().launch(command)
return self._parse_output(raw_output)
def _parse_output(self, output: str) -> List[Dict]:
"""Parse raw tool output into structured data."""
# Parsing logic here
```
This separation makes the code easier to test and maintain.
## Next steps
Once you've created your tool and tested it, you can build enrichers that use it. Enrichers orchestrate one or more tools to gather intelligence, validate the results, convert them to Flowsint types, and create graph database nodes and relationships.
If your tool requires API keys or other secrets, enrichers can access them through the vault system. When you implement an enricher that uses your tool, you can define parameters of type `vaultSecret` that pull credentials from the user's encrypted vault.
Remember that tools are just one layer in the Flowsint architecture. They provide the raw capabilities, but enrichers provide the intelligence and graph-building logic that makes the platform powerful.

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,53 @@
---
title: "Enrichers"
description: "Quick start guide to using Enrichers for your OSINT investigations."
category: "Getting started"
order: 4
author: "Flowsint Team"
tags: ["tutorial", "getting-started", "enrichers"]
version: "1.2.8"
last_updated_at: "2026-05-15"
---
### What is an Enricher?
An enricher is an operation that, starting from an input element A (source entity), produces one or more elements B (target entities) by applying a search or correlation method called a pivot.
Example:
```bash
A = my.domain.com (domain name)
p = “DNS resolution” (pivot)
B = 12.23.34.45 (IP address)
```
That said, a pivot is the method or technical process used to derive B from A. The pivot defines how the transformation obtains its result (e.g., DNS resolution, WHOIS lookup, API query, etc.).
Example:
```bash
DNS Resolution → domain → IP
WHOIS Lookup → IP → owner
Reverse Image Search → image → web pages containing that image
```
Flowsint comes with a bunch of prebuilt enrichers, divided into multiple categories. Those enrichers can use standard pivots that your machine can support by default (DNS resolution, WHOIS request, etc.) and some other that depend on external tools.
Those can be :
- Native : DNS resolutions, Whois, etc.
- Docker tools: [subfinder](https://github.com/projectdiscovery/subfinder), [asnmap](https://github.com/projectdiscovery/asnmap), etc.
- Python tools: [sherlock](https://github.com/sherlock-project/sherlock), [maigret](https://github.com/soxoj/maigret), [reconurge/recontrack](https://github.com/reconurge/recontrack), [reconurge/reconcrawl](https://github.com/reconurge/reconcrawl), [reconurge/reconspread](https://github.com/reconurge/reconspread), etc.
- External services (paid or free): [shodan](https://www.shodan.io/), [whoxy](https://www.whoxy.com/), [whoisxmlapi](https://www.whoisxmlapi.com), etc.
### Making your own enrichers
Creating your own enrichers invloves multiple steps, but is not that trivial.
If you plan on writting your own enrichers and think they could help the community, please contribute by making a pull request !
Please refer to [this section](/docs/developers/managing-enrichers) to start building your own enrichers.

View File

@@ -0,0 +1,159 @@
---
title: "Flows"
description: "Quick start guide to using Flows for your OSINT investigations."
category: "Getting started"
order: 5
author: "Flowsint Team"
tags: ["tutorial", "getting-started", "flows"]
version: "1.2.8"
last_updated_at: "2026-05-15"
---
### What are Flows?
Flows are the chaining of multiple enrichers, where the output of one becomes the input of the next, allowing an investigation to be broadened or deepened.
Some enrichers can be chained together, to that they can provide a reproductible and scalable research flow that can be re-applied to other entities of the same type.
### Getting started with Flows
The best way to get started with flows is to go to [http://localhost:5173/dashboard/flows](http://localhost:5173/dashboard/flows) and create a new flow.
From that, you can start building your first flow; a flow consists of **one** input type, and multiple chained enrichers.
Start by drag & dropping your input type to the canva and start using the "**+**" button to add more enrichers to your flow.
Once you're satisfied with the flow, you can start viewing the execution order by pressing the "**compute**" button.
Make sure you give it a descriptive name and save ! (`crtl + s` or presse the save button).
Once it's done, go back to one of your sketches, right click on an item of the same type of the flow, and you should be able to launch it from the "flows" section.
### Flow schema
Every time a flow is launched, a log file is created at `flowsint_core/enricher_logs`.
```json
{
"sketch_id": "6aa808e4-1360-4c4a-b94f-4bed6c914836",
"scan_id": "52ba38a2-3f2c-4c8b-a7a5-42656f8fb845",
"created_at": "2025-10-26T15:57:52.243549",
"updated_at": "2025-10-26T15:57:52.424052",
"status": "completed",
"enricher_branches": [
{
"id": "branch-0",
"name": "Main Flow",
"steps": [
{
"nodeId": "Domain-1761469976169",
"params": {},
"type": "type",
"inputs": {},
"outputs": {
"domain": [
"example.com"
]
},
"status": "pending",
"branchId": "branch-0",
"depth": 0
},
{
"nodeId": "domain_to_ip-1761469978018",
"params": {},
"type": "enricher",
"inputs": {
"Domain": null
},
"outputs": {
"address": "example.com",
"latitude": "example.com",
"longitude": "example.com",
"country": "example.com",
"city": "example.com",
"isp": "example.com"
},
"status": "pending",
"branchId": "branch-0",
"depth": 1
}
]
}
],
"execution_log": [
{
"step_id": "branch-0_domain_to_ip-1761469978018",
"branch_id": "branch-0",
"branch_name": "Main Flow",
"node_id": "domain_to_ip-1761469978018",
"enricher_name": "domain_to_ip",
"inputs": [
"example.com"
],
"outputs": [
{
"address": "12.34.56.78",
"latitude": null,
"longitude": null,
"country": null,
"city": null,
"isp": null
}
],
"status": "completed",
"error": null,
"timestamp": "2025-10-26T15:57:52.274147",
"execution_time_ms": 144,
"cache_hit": false
}
],
"summary": {
"total_steps": 1,
"completed_steps": 1,
"failed_steps": 0,
"total_execution_time_ms": 144
},
"final_results": {
"initial_values": [
"example.com"
],
"branches": [
{
"id": "branch-0",
"name": "Main Flow",
"steps": [
{
"nodeId": "domain_to_ip-1761469978018",
"enricher": "domain_to_ip",
"status": "completed",
"outputs": [
{
"address": "12.34.56.78",
"latitude": null,
"longitude": null,
"country": null,
"city": null,
"isp": null
}
]
}
]
}
],
"results": {
"domain_to_ip-1761469978018": [
{
"address": "12.34.56.78",
"latitude": null,
"longitude": null,
"country": null,
"city": null,
"isp": null
}
]
},
"reference_mapping": {}
}
}
```

View File

@@ -0,0 +1,42 @@
---
title: "Installation"
description: "Quick start guide to using Flowsint for your OSINT investigations."
category: "Getting started"
order: 3
author: "Flowsint Team"
tags: ["tutorial", "quickstart", "installation"]
version: "1.2.8"
last_updated_at: "2026-05-15"
---
### Prerequisites
Before installing Flowsint, ensure you have the following installed on your system:
- **Docker** and **Docker Compose**
- **Make** (for build automation)
- **Git**
### Installation
Clone the repo and run the start command.
```bash
git clone https://github.com/reconurge/flowsint.git
cd flowsint
make prod
```
Some enrichers require API keys. Check out [this section](/docs/getting-started/enrichers#api-keys) to learn more.
The application should automatically open at http://localhost:5173.
### Create your first investigation
Start by logging in or registering from the home page. From your dashboard, create a new investigation by clicking "New investigation." Add your first entity—such as a domain name—to the canvas. Right-click the entity to open the context menu and run an enricher. As results appear, explore the newly discovered entities and relationships directly in the graph.
### Running your first enricher
You could start by discovering subdomains for a domain for example.
Add a domain entity like `example.com` to your investigation, then right-click the domain node and select the "domain_to_subdomains" enricher. Wait for the enricher to complete; newly discovered subdomains will be added to your graph for you to review.

View File

@@ -0,0 +1,58 @@
---
title: "Vault"
description: "Quick start guide to using the Vault to secure your services API keys and secrets."
category: "Getting started"
order: 6
author: "Flowsint Team"
tags: ["tutorial", "getting-started", "vault"]
version: "1.2.8"
last_updated_at: "2026-05-15"
---
## What is the Vault
A good amount of the tools you'll be using in Flowsint require third party API keys.
The **Vault** (*"Coffre fort"* in french) is the place to centralize and securely store those API keys.
Weither you have a local instance of Flowsint or one fully deployed on a distributed system, you need to have your keys securely stored.
## Adding a key
In the Flowsint enricher ecosystem, the API keys follow a specific format, being in uppercase letters, and with a declarative name that follows `<service>_API_KEY`.
## Current limitations
For now, we cannot match a particular key from the Vault to an enricher **directly from the UI**. The enricher declares the API key variable name it requires, like the following in the core of the Enricher:
```python
@classmethod
def get_params_schema(cls) -> List[Dict[str, Any]]:
"""Declare required parameters for this enricher"""
return [
{
"name": "PDCP_API_KEY",
"type": "vaultSecret",
"description": "The ProjectDiscovery Cloud Platform API key for asnmap.",
"required": True,
},
]
```
This is a known limitation and we are working on improving this.
In the meanwhile, here is a list of the needed keys to run Flowsint at it's full potential:
```bash
# for enrichers
WHOXY_API_KEY # Whoxy domain search engine [WHOXY]
PDCP_API_KEY # ProjectDiscovery Cloud Platform [ASNMAP], [NAABU] etc
HIBP_API_KEY # HaveIBeenPwned API key [HIBP]
ETHERSCAN_API_KEY # Etherscan crypto API key [ETHERSCAN]
# for Flo, AI assistant
MISTRAL_API_KEY
# but other providers will be supported soon (ChatGPT, etc.)
```
There are also some other tools that could need a bunch of other API keys like [Subfinder](https://github.com/projectdiscovery/subfinder). Configuring them is not possible for now, but will be soon.
Stay tuned for updates as those mechanisms may vary in the future, as the goal is to keep the user experience as smooth as possible.

59
docs/overview.mdx Normal file
View File

@@ -0,0 +1,59 @@
---
title: "Welcome to Flowsint"
description: "Complete documentation for Flowsint - a modular OSINT investigation platform."
category: "Overview"
order: 1
author: "Flowsint Team"
tags: ["documentation", "overview", "getting-started"]
version: "1.2.8"
last_updated_at: "2026-05-15"
---
### What is Flowsint?
Flowsint is a modular investigation and reconnaissance platform focused on OSINT (Open Source Intelligence). It provides:
- Graph-based visualization of entity relationships
- 30+ automated enrichers for intelligence gathering
- Modular architecture with clean separation of concerns
- Privacy-first design with local data storage
- Extensible platform for custom enrichers
- Automated search flows (more on that [here](/docs/getting-started/flows))
<Alert variant="warning">
<AlertTitle>Disclaimer !</AlertTitle>
<AlertDescription>
The author(s) and contributor(s) of this tool assume no responsibility or liability for any damages, losses, or consequences that may result from the use or misuse of this software.
By using this tool, you acknowledge and agree that:
- You are solely responsible for your use of this software
- You will use this tool in compliance with all applicable laws and regulations
- You will obtain proper authorization before conducting any security testing or reconnaissance activities
- You understand the potential risks and legal implications of using security tools
</AlertDescription>
</Alert>
### Why Flowsint?
If you've already practiced some OSINT, you know that analysts often rely on a multitude of research tools: scripts, third-party services, specialized applications. But these tools often work **in silos** and quickly become obsolete if they're not maintained: a service disappears, an API changes, an access point closes. The analyst juggles with unstable tools and sometimes has to **manually adapt their data** to continue their investigation.
OSINT tools, for most, are **consumables**: they evolve, appear, and disappear. Research methods change, security mechanisms evolve. However, the fundamental need always remains the same: **to see, exploit, and analyze data in a clear and understandable way**. Analysts need to be able to list, centralize, and visualize connections to have a clear understanding of their investigation.
This is exactly what Flowsint was built for: a solid and durable foundation on which your investigations rest. Tools become simple extensions that plug in and unplug easily. A new tool comes out, and with a few manipulations it can be integrated into your investigation workflow.
**Flowsint is the stable infrastructure that allows you to stay agile in the face of constant evolution of methods and sources.**
Flowsint is a local tool, running on your machine only. This ensures a great level of confidentiality, but comes with responsabilities.
<Alert variant="warning">
<AlertTitle>At your own risk</AlertTitle>
<AlertDescription>
All enrichers run locally on your machine. This means you can get banned from some services or get flagged from infrastructures if you start making thousands of requests to the same service.
You need to always know what you are doing, and understand that gathering can go very fast in scale.
For example, you can very easily happen to be making DNS resolution requests for 10 000 IPs.
Know you infrastructure and your limits. Know how the tools used in the enrichers actually work. You can have a list of the available enrichers and tools [here](/docs/sources/available-enrichers).
</AlertDescription>
</Alert>
If all those points are clear for you, let's start investigating !

View File

@@ -0,0 +1,163 @@
---
title: "Enrichers catalog"
description: "Quick start guide to using Enrichers for your OSINT investigations."
category: "Sources"
order: 11
author: "Flowsint Team"
tags: ["tutorial", "getting-started", "enrichers"]
version: "1.2.8"
last_updated_at: "2026-05-15"
---
### ASN
**asn_to_cidrs**: Given an ASN, enumerate its announced CIDR ranges.
Tools/Pivots: [asnmap](https://github.com/projectdiscovery/asnmap) (CLI), [jq](https://jqlang.github.io/jq/) (CLI)
### CIDR
**cidr_to_ips**: Expand a CIDR to IPs by PTR enumeration heuristics.
Tools/Pivots: [dnsx](https://github.com/projectdiscovery/dnsx) (CLI)
### Crypto
**cryptowallet_to_transactions**: Fetch ETH wallet transactions and map wallet-to-wallet relationships.
Tools/APIs: [Etherscan API](https://docs.etherscan.io/)
**cryptowallet_to_nfts**: Fetch ERC-721/1155 NFT transfers for a wallet.
Tools/APIs: [Etherscan API](https://docs.etherscan.io/)
### Domain
**domain_to_ip**: Resolve domains to IPv4 addresses.
Tools/Pivots: DNS resolution (socket)
**domain_to_subdomains**: Discover subdomains for a domain.
Tools/APIs: [subfinder](https://github.com/projectdiscovery/subfinder) (CLI), fallback to [crt.sh JSON API](https://crt.sh/?output=json)
**domain_to_whois**: Retrieve WHOIS registration data for a domain.
Tools/APIs: [python-whois](https://pypi.org/project/python-whois/)
**domain_to_asn**: Map a domain to its ASN by resolving and querying ASN data.
Tools/Pivots: system DNS, [asnmap](https://github.com/projectdiscovery/asnmap) (CLI)
**domain_to_root_domain**: Convert a subdomain to its registrable root.
Tools/Pivots: internal domain utils
**domain_to_history**: Retrieve historical WHOIS records and extract related entities (individuals, organizations, emails, phones, locations).
Tools/APIs: [Whoxy API](https://www.whoxy.com/api/)
**domain_to_website**: Convert a domain to a reachable website URL (HTTP/HTTPS), following redirects.
Tools/Pivots: HTTP HEAD requests
**domain_to_tls**: Retrieve TLS/SSL certificate information for a domain.
Tools/Pivots: [httpx](https://github.com/projectdiscovery/httpx) (CLI)
**domain_to_whois_history**: Retrieve historical WHOIS records for a domain and extract related entities (individuals, organizations, emails, locations).
Tools/APIs: [WhoisXML API](https://whois.whoisxmlapi.com/)
**domain_to_dehashed**: Get breach intelligence (credentials, related individuals) associated with a domain.
Tools/APIs: [DeHashed API](https://www.dehashed.com/docs)
### Email
**email_to_breaches**: Check whether an email appears in known breaches.
Tools/APIs: [Have I Been Pwned API](https://haveibeenpwned.com/API/v3)
**email_to_gravatar**: Check Gravatar existence and profile for an email (via MD5 hash).
Tools/APIs: [Gravatar endpoints](https://en.gravatar.com/site/implement/images/)
**email_to_domain**: Extract the domain part of an email address.
Tools/Pivots: internal email parser
**email_to_domains**: Find domains registered by a given email address; extract related contacts and entities.
Tools/APIs: [Whoxy API](https://www.whoxy.com/api/)
**email_to_username**: Extract the local-part of an email as a Username entity.
Tools/Pivots: internal email parser
**email_to_intelligence**: Get breach intelligence (credentials, related individuals) associated with an email.
Tools/APIs: [DeHashed API](https://www.dehashed.com/docs)
**email_to_device_hudsonrock**: Look up devices compromised by infostealers and associated with an email.
Tools/APIs: [HudsonRock API](https://www.hudsonrock.com/)
### Individual
**individual_to_domains**: Find domains registered by a specific person; extract related contacts and attributes.
Tools/APIs: [Whoxy API](https://www.whoxy.com/api/)
**individual_to_organization**: Find organizations related to a person in French registries.
Tools/APIs: SIRENE (via internal SireneTool) — see [INSEE Sirene API](https://api.insee.fr/catalogue/#/datasets/sirene)
### IP
**ip_to_domain**: Reverse-resolve IPs to domains via PTR and Certificate Transparency pivots.
Tools/APIs: DNS PTR (socket), [crt.sh JSON API](https://crt.sh/?output=json)
**ip_to_infos**: Enrich IPs with geolocation and ISP data.
Tools/APIs: [ip-api.com](https://ip-api.com/)
**ip_to_asn**: Map IPs to their ASN.
Tools/Pivots: AsnmapTool ([asnmap](https://github.com/projectdiscovery/asnmap))
**ip_to_ports**: Scan an IP for open ports and services.
Tools/Pivots: [naabu](https://github.com/projectdiscovery/naabu) (CLI)
**ip_to_fraudscore**: Compute a fraud risk score for an IP address.
Tools/APIs: [Scamalytics API](https://scamalytics.com/ip-api)
**ip_to_intelligence**: Get breach intelligence (credentials, related individuals) associated with an IP.
Tools/APIs: [DeHashed API](https://www.dehashed.com/docs)
### Organization
**org_to_domains**: Find domains registered by an organization; extract contacts and related entities.
Tools/APIs: [Whoxy API](https://www.whoxy.com/api/)
**org_to_infos**: Enrich organizations with French registry data and leaders.
Tools/APIs: SIRENE (SireneTool) — see [INSEE Sirene API](https://api.insee.fr/catalogue/#/datasets/sirene)
**org_to_asn**: Find ASNs associated with an organization name.
Tools/Pivots: [asnmap](https://github.com/projectdiscovery/asnmap) (CLI), [jq](https://jqlang.github.io/jq/) (CLI)
### Phone
**phone_to_infos**: Probe phone footprint across services (demo modules) and normalize number.
Tools/APIs: ignorant modules (Amazon, Snapchat, Instagram), [httpx](https://github.com/projectdiscovery/httpx)
**phone_to_carrier**: Look up carrier, country, and validity metadata for a phone number.
Tools/APIs: [Veriphone API](https://veriphone.io/)
**phone_to_device_hudsonrock**: Look up devices compromised by infostealers and associated with a phone number.
Tools/APIs: [HudsonRock API](https://www.hudsonrock.com/)
### Social
**username_to_socials_sherlock**: Enumerate social accounts for a username using Sherlock.
Tools/Pivots: [sherlock](https://github.com/sherlock-project/sherlock) (CLI)
**username_to_socials_maigret**: Enumerate social accounts for a username using Maigret and parse rich metadata.
Tools/Pivots: [maigret](https://github.com/soxoj/maigret) (CLI)
**username_to_dehashed**: Get breach intelligence (credentials, related individuals) associated with a username.
Tools/APIs: [DeHashed API](https://www.dehashed.com/docs)
**username_to_device_hudsonrock**: Look up devices compromised by infostealers and associated with a username.
Tools/APIs: [HudsonRock API](https://www.hudsonrock.com/)
### Website
**website_to_crawler**: Crawl a website to extract emails and phone numbers.
Tools/APIs: ReconCrawlTool (`reconcrawl`)
**website_to_domain**: Extract the domain name from a website URL.
Tools/Pivots: internal URL parser
**website_to_subdomains**: Find subdomains of a website's domain via external scan.
Tools/APIs: [c99.nl API](https://api.c99.nl/)
**website_to_links**: Crawl a website and collect internal/external links and domains.
Tools/APIs: reconspread Crawler
**website_to_text**: Fetch and extract visible text from a webpage.
Tools/APIs: HTTP GET, [BeautifulSoup](https://www.crummy.com/software/BeautifulSoup/bs4/doc/)
**website_to_webtrackers**: Extract analytics/ads tracking codes from a website.
Tools/APIs: recontrack TrackingCodeExtractor
---
Notes
- Some enrichers optionally depend on docker binaries: `subfinder`, `asnmap`, `dnsx`, `naabu`, `httpx`, and `jq` which are installed in the docker container.
- API-keyed enrichers read keys from params or environment (e.g., `HIBP_API_KEY`, `ETHERSCAN_API_KEY`, `WHOXY_API_KEY`, `WHOISXML_API_KEY`, `DEHASHED_API_KEY`, `SCAMALYTICS_API_KEY`, `VERIPHONE_API_KEY`, `C99_API_KEY`).
- Internal/test enrichers (`domain_to_dummy`, `ip_to_dummy_domains`, `n8n_connector`) are not listed here — they exist in the codebase but are not part of the public catalog.

73
docs/syllabus.mdx Normal file
View File

@@ -0,0 +1,73 @@
---
title: "Syllabus"
description: "Syllabus, just to make sure we speak the same language. Those definitions apply in the context of Flowsint platform."
category: "Overview"
order: 2
author: "Flowsint Team"
tags: ["documentation", "overview", "syllabus"]
version: "1.2.8"
last_updated_at: "2026-05-15"
---
### OSINT
Open Source Intelligence consists of collecting, analyzing, and exploiting **freely** and **openly** available information from search engines, images, social networks, public archives, etc.
### Investigation
A structured process aimed at collecting, correlating, and analyzing information from different sources and enrichers, in order to answer a question or solve a problem. An investigation can be **exploratory** (discovering unknown elements) or **targeted** (validating a hypothesis). An investigation can contain multiple **sketches** (each representing a different view or stage of the analysis) and one or more **analyses**.
### Sketch
Visual result produced by executing one or more enrichers on one or more entities. A sketch represents the current state of the graph derived from collected data at a given moment in the investigation. Multiple sketches can exist for the same investigation to capture different perspectives or stages.
### Analysis
Set of processing, interpretations, and verifications performed on data collected during the investigation. Analyses aim to identify trends, confirm or refute hypotheses, and produce actionable conclusions. They can be **quantitative** (measurements, statistics) or **qualitative** (contextual assessments, behavioral patterns).
### Enricher
An **enricher** is an operation that, from an input element **A** (*source entity*), allows obtaining one or more elements **B** (*target entities*) by applying a search or correlation method called a **pivot**.
> Example:
>
>
> A = `my.domain.com` (*domain name*)
>
> p = "DNS resolution" (*pivot*)
>
> B = `12.23.34.45` (*IP address*).
>
### Pivot
A **pivot** is the method or technical process used to derive **B** from **A**. The pivot defines **how** the enricher obtains its result (e.g., DNS resolution, WHOIS lookup, API query, etc.).
> Examples of pivots:
>
> DNS Resolution → domain → IP
> WHOIS Lookup → IP → owner
> Reverse Image Search → image → web pages containing this image
### Tool
A tool generally refers to a script, program, or service providing a **pivot**, i.e., a means to retrieve or enricher information from an input element.
### Entity
An identifiable object or element manipulated by enrichers (e.g., IP address, domain, email address, user identifier, file hash, etc.). An entity is always associated with a **Sketch**. In the graph, entities are represented as **nodes** (see [Graph format](/docs/developers/graph-format) for technical details).
### Relationship
Defines a link between two entities. This link is generally named (in uppercase) and can be unidirectional or bidirectional.
> Examples of relationships:
>
>
> A = `my.domain.com` → `RESOLVES_TO` → `12.23.34.45`
>
A relationship is always associated between a **source** node (*from*) and a **target** node (*to*). In the graph, relationships are represented as **edges** (see [Graph format](/docs/developers/graph-format) for technical details).
### Flow
The chaining of multiple enrichers, where the output of one becomes the input of the next, allowing to expand or deepen an investigation.

View File

@@ -2,17 +2,12 @@ FROM python:3.12-slim AS builder
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
PIP_NO_CACHE_DIR=1 \
PIP_DISABLE_PIP_VERSION_CHECK=1 \
POETRY_VIRTUALENVS_IN_PROJECT=true \
POETRY_NO_INTERACTION=1 \
POETRY_HOME="/opt/poetry"
ENV PATH="$POETRY_HOME/bin:$PATH"
UV_COMPILE_BYTECODE=1 \
UV_LINK_MODE=copy
WORKDIR /app
# build deps
# build deps + uv
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential \
curl \
@@ -20,23 +15,21 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
libpq-dev \
pkg-config \
libcairo2-dev \
&& rm -rf /var/lib/apt/lists/*
&& rm -rf /var/lib/apt/lists/* \
&& curl -LsSf https://astral.sh/uv/install.sh | sh
# poetry
RUN curl -sSL https://install.python-poetry.org | python3 -
ENV PATH="/root/.local/bin:$PATH"
COPY flowsint-core/pyproject.toml flowsint-core/poetry.lock* ./flowsint-core/
COPY flowsint-types/pyproject.toml flowsint-types/poetry.lock* ./flowsint-types/
COPY flowsint-enrichers/pyproject.toml flowsint-enrichers/poetry.lock* ./flowsint-enrichers/
COPY flowsint-api/pyproject.toml flowsint-api/poetry.lock* ./flowsint-api/
# Copy workspace config
COPY pyproject.toml uv.lock ./
COPY flowsint-core ./flowsint-core
# Copy all workspace members
COPY flowsint-types ./flowsint-types
COPY flowsint-core ./flowsint-core
COPY flowsint-enrichers ./flowsint-enrichers
COPY flowsint-api ./flowsint-api
WORKDIR /app/flowsint-api
RUN poetry install --no-root
RUN uv sync --frozen --no-dev
# DEV
FROM python:3.12-slim AS dev
@@ -44,7 +37,7 @@ FROM python:3.12-slim AS dev
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
APP_ENV=development \
PATH="/app/flowsint-api/.venv/bin:$PATH"
PATH="/app/.venv/bin:$PATH"
# Install runtime dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
@@ -56,7 +49,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
WORKDIR /app
# Copy virtual environment from builder
COPY --from=builder /app/flowsint-api/.venv ./flowsint-api/.venv
COPY --from=builder /app/.venv ./.venv
# Copy application code
COPY flowsint-core ./flowsint-core
@@ -86,7 +79,7 @@ LABEL org.opencontainers.image.licenses="Apache-2.0"
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
APP_ENV=production \
PATH="/app/flowsint-api/.venv/bin:$PATH"
PATH="/app/.venv/bin:$PATH"
# Install runtime dependencies only
RUN apt-get update && apt-get install -y --no-install-recommends \
@@ -102,8 +95,8 @@ RUN groupadd -g 1001 flowsint && \
WORKDIR /app
# Copy virtual environment from builder (production deps only would require separate install)
COPY --from=builder --chown=flowsint:flowsint /app/flowsint-api/.venv ./flowsint-api/.venv
# Copy virtual environment from builder
COPY --from=builder --chown=flowsint:flowsint /app/.venv ./.venv
# Copy application code
COPY --chown=flowsint:flowsint flowsint-core ./flowsint-core
@@ -117,7 +110,7 @@ WORKDIR /app/flowsint-api
RUN chmod +x entrypoint.sh
# Switch to non-root user
# USER flowsint
USER flowsint
EXPOSE 5001
@@ -127,4 +120,4 @@ HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
ENTRYPOINT ["./entrypoint.sh"]
# Production command (no reload)
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "5001"]
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "5001"]

View File

@@ -5,14 +5,14 @@
1. Install Python dependencies:
2.
```bash
poetry install
uv sync
```
## Run
```bash
# dev
poetry run uvicorn app.main:app --host 0.0.0.0 --port 5001 --reload
uv run uvicorn app.main:app --host 0.0.0.0 --port 5001 --reload
# prod
poetry run uvicorn app.main:app --host 0.0.0.0 --port 5001
uv run uvicorn app.main:app --host 0.0.0.0 --port 5001
```

View File

@@ -0,0 +1,59 @@
"""backfill owner roles for existing investigations
Revision ID: a1f2b3c4d5e6
Revises: bac5764d4496
Create Date: 2026-04-11 00:00:00.000000
"""
from typing import Sequence, Union
from uuid import uuid4
from alembic import op
import sqlalchemy as sa
import json
# revision identifiers, used by Alembic.
revision: str = "a1f2b3c4d5e6"
down_revision: Union[str, None] = "bac5764d4496"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
"""Insert OWNER role entry for every investigation that lacks one."""
conn = op.get_bind()
# Find investigations with no entry in investigation_user_roles
rows = conn.execute(
sa.text(
"""
SELECT i.id, i.owner_id
FROM investigations i
LEFT JOIN investigation_user_roles r
ON r.investigation_id = i.id AND r.user_id = i.owner_id
WHERE r.id IS NULL
AND i.owner_id IS NOT NULL
"""
)
).fetchall()
for inv_id, owner_id in rows:
conn.execute(
sa.text(
"""
INSERT INTO investigation_user_roles (id, user_id, investigation_id, roles)
VALUES (:id, :user_id, :investigation_id, :roles)
"""
),
{
"id": str(uuid4()),
"user_id": str(owner_id),
"investigation_id": str(inv_id),
"roles": json.dumps(["owner"]),
},
)
def downgrade() -> None:
"""No-op: we don't remove the backfilled rows."""
pass

View File

@@ -0,0 +1,42 @@
"""add icon and color to custom types
Revision ID: bac5764d4496
Revises: f5fae279ec04
Create Date: 2026-02-14 12:10:26.415916
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects import postgresql
# revision identifiers, used by Alembic.
revision: str = 'bac5764d4496'
down_revision: Union[str, None] = 'f5fae279ec04'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
"""Upgrade schema."""
# ### commands auto generated by Alembic - please adjust! ###
op.add_column('custom_types', sa.Column('icon', sa.String(), nullable=True))
op.add_column('custom_types', sa.Column('color', sa.String(), nullable=True))
op.alter_column('enricher_templates', 'content',
existing_type=postgresql.JSONB(astext_type=sa.Text()),
type_=sa.JSON(),
existing_nullable=False)
# ### end Alembic commands ###
def downgrade() -> None:
"""Downgrade schema."""
# ### commands auto generated by Alembic - please adjust! ###
op.alter_column('enricher_templates', 'content',
existing_type=sa.JSON(),
type_=postgresql.JSONB(astext_type=sa.Text()),
existing_nullable=False)
op.drop_column('custom_types', 'color')
op.drop_column('custom_types', 'icon')
# ### end Alembic commands ###

View File

@@ -1,19 +1,10 @@
from fastapi import Depends, HTTPException, status, Request
from fastapi import Depends, HTTPException, status
from fastapi.security import OAuth2PasswordBearer
from jose import JWTError, jwt
from sqlalchemy.orm import Session
from flowsint_core.core.auth import ALGORITHM
from flowsint_core.core.auth import ALGORITHM, AUTH_SECRET
from flowsint_core.core.postgre_db import get_db
from flowsint_core.core.models import Profile
from typing import Optional
import os
from dotenv import load_dotenv
# Remplace avec ton URL de BDD
AUTH_SECRET = os.getenv("AUTH_SECRET")
load_dotenv()
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="token")
@@ -37,42 +28,3 @@ def get_current_user(
if user is None:
raise credentials_exception
return user
def get_current_user_sse(
request: Request, db: Session = Depends(get_db)
) -> Profile:
"""
Alternative authentication for SSE endpoints that accepts token via query parameter.
EventSource API doesn't support custom headers, so we need to pass the token in the URL.
"""
credentials_exception = HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Could not validate credentials",
)
# Try to get token from query parameter
token: Optional[str] = request.query_params.get("token")
# Fallback to Authorization header if query param not present
if not token:
auth_header = request.headers.get("Authorization")
if auth_header and auth_header.startswith("Bearer "):
token = auth_header.replace("Bearer ", "")
if not token:
raise credentials_exception
try:
payload = jwt.decode(token, AUTH_SECRET, algorithms=[ALGORITHM])
email: str = payload.get("sub")
if email is None:
raise credentials_exception
except JWTError:
raise credentials_exception
user = db.query(Profile).filter(Profile.email == email).first()
if user is None:
raise credentials_exception
return user

View File

@@ -1,7 +1,8 @@
from fastapi import APIRouter, Depends, HTTPException
from fastapi import APIRouter, Depends, HTTPException, Query
from fastapi.security import OAuth2PasswordRequestForm
from sqlalchemy.orm import Session
from sqlalchemy.exc import SQLAlchemyError
from typing import List
from flowsint_core.core.services import (
create_auth_service,
@@ -9,8 +10,10 @@ from flowsint_core.core.services import (
ConflictError,
DatabaseError,
)
from app.api.schemas.profile import ProfileCreate
from flowsint_core.core.models import Profile
from flowsint_core.core.postgre_db import get_db
from app.api.schemas.profile import ProfileCreate, ProfileRead, ProfileUpdate
from app.api.deps import get_current_user
router = APIRouter()
@@ -39,3 +42,40 @@ def register(user: ProfileCreate, db: Session = Depends(get_db)):
except (DatabaseError, SQLAlchemyError) as e:
print(f"[ERROR] DB error during registration: {e}")
raise HTTPException(status_code=500, detail="Internal server error")
@router.get("/me", response_model=ProfileRead)
def get_me(current_user: Profile = Depends(get_current_user)):
return current_user
@router.put("/me", response_model=ProfileRead)
def update_me(
payload: ProfileUpdate,
db: Session = Depends(get_db),
current_user: Profile = Depends(get_current_user),
):
for key, value in payload.model_dump(exclude_unset=True).items():
setattr(current_user, key, value)
db.commit()
db.refresh(current_user)
return current_user
@router.get("/users/search", response_model=List[ProfileRead])
def search_users(
q: str = Query(..., min_length=1),
db: Session = Depends(get_db),
current_user: Profile = Depends(get_current_user),
):
"""Search users by email prefix for the share dialog autocomplete."""
results = (
db.query(Profile)
.filter(
Profile.email.ilike(f"{q}%"),
Profile.id != current_user.id,
)
.limit(5)
.all()
)
return results

View File

@@ -23,7 +23,7 @@ class ChatRequest(BaseModel):
context: Optional[List[str]] = None
@router.get("/", response_model=List[ChatRead])
@router.get("", response_model=List[ChatRead])
def get_chats(
db: Session = Depends(get_db), current_user: Profile = Depends(get_current_user)
):

View File

@@ -1,32 +1,33 @@
"""API routes for custom types management."""
from uuid import UUID
from typing import List
from fastapi import APIRouter, HTTPException, Depends, status
from sqlalchemy.orm import Session
from flowsint_core.core.postgre_db import get_db
from typing import List
from uuid import UUID
from fastapi import APIRouter, Depends, HTTPException, status
from flowsint_core.core.models import Profile
from flowsint_core.core.postgre_db import get_db
from flowsint_core.core.services import (
create_custom_type_service,
ConflictError,
NotFoundError,
ValidationError,
ConflictError,
create_custom_type_service,
)
from sqlalchemy.orm import Session
from app.api.deps import get_current_user
from app.api.schemas.custom_type import (
CustomTypeCreate,
CustomTypeUpdate,
CustomTypeRead,
CustomTypeUpdate,
CustomTypeValidatePayload,
CustomTypeValidateResponse,
)
from app.utils.custom_types import (
calculate_schema_checksum,
validate_json_schema,
validate_payload_against_schema,
calculate_schema_checksum,
)
router = APIRouter()
@@ -110,6 +111,8 @@ def update_custom_type(
custom_type_id=id,
user_id=current_user.id,
name=update_data.name,
icon=update_data.icon,
color=update_data.color,
json_schema=update_data.json_schema,
description=update_data.description,
status=update_data.status,

View File

@@ -18,6 +18,8 @@ from flowsint_core.core.vault import Vault
from flowsint_core.templates.types import Template
from sqlalchemy.orm import Session
from flowsint_types.registry import get_type as get_type_from_registry, load_all_types
from app.api.deps import get_current_user
from app.api.schemas.enricher_template import (
EnricherTemplateCreate,
@@ -84,9 +86,39 @@ async def generate_template(
current_user: Profile = Depends(get_current_user),
):
"""Generate an enricher template from a free-text description using AI."""
load_all_types()
input_schema = None
output_schema = None
if request.input_type:
input_cls = get_type_from_registry(request.input_type, case_sensitive=True)
if input_cls is None:
raise HTTPException(
status_code=422,
detail=f"Unknown input type: '{request.input_type}'",
)
input_schema = input_cls.model_json_schema()
if request.output_type:
output_cls = get_type_from_registry(request.output_type, case_sensitive=True)
if output_cls is None:
raise HTTPException(
status_code=422,
detail=f"Unknown output type: '{request.output_type}'",
)
output_schema = output_cls.model_json_schema()
service = create_template_generator_service(db)
try:
yaml_content = await service.generate(request.prompt, current_user.id)
yaml_content = await service.generate(
prompt=request.prompt,
owner_id=current_user.id,
input_type=request.input_type,
input_schema=input_schema,
output_type=request.output_type,
output_schema=output_schema,
)
except ValidationError as e:
raise HTTPException(status_code=422, detail=str(e))
return EnricherTemplateGenerateResponse(yaml_content=yaml_content)

View File

@@ -9,6 +9,7 @@ from flowsint_core.core.services import (
create_enricher_service,
create_enricher_template_service,
)
from flowsint_core.core.services.type_registry_service import create_type_registry_service
from flowsint_enrichers import ENRICHER_REGISTRY, load_all_enrichers
from pydantic import BaseModel
from sqlalchemy.orm import Session
@@ -26,7 +27,7 @@ class launchEnricherPayload(BaseModel):
router = APIRouter()
@router.get("/")
@router.get("")
def get_enrichers(
category: Optional[str] = Query(None),
db: Session = Depends(get_db),
@@ -48,7 +49,9 @@ async def launch_enricher(
):
try:
# Retrieve nodes from Neo4J by their element IDs
graph_service = create_graph_service(sketch_id=payload.sketch_id)
type_registry = create_type_registry_service(db)
resolver = type_registry.build_type_resolver(current_user.id)
graph_service = create_graph_service(sketch_id=payload.sketch_id, type_resolver=resolver)
entities = graph_service.get_nodes_by_ids_for_task(payload.node_ids)
# Send deserialized nodes

View File

@@ -14,7 +14,7 @@ from flowsint_core.core.services import (
PermissionDeniedError,
DatabaseError,
)
from app.api.deps import get_current_user, get_current_user_sse
from app.api.deps import get_current_user
router = APIRouter()
@@ -42,7 +42,7 @@ async def stream_events(
request: Request,
sketch_id: str,
db: Session = Depends(get_db),
current_user: Profile = Depends(get_current_user_sse),
current_user: Profile = Depends(get_current_user),
):
"""Stream events for a specific sketch in real-time."""
service = create_log_service(db)
@@ -58,7 +58,7 @@ async def stream_events(
channel = sketch_id
await event_emitter.subscribe(channel)
try:
yield 'data: {"event": "connected", "data": "Connected to log stream"}\n\n'
yield json.dumps({"event": "connected", "data": "Connected to log stream"})
while True:
if await request.is_disconnected():
break
@@ -115,7 +115,7 @@ async def stream_sketch_status(
request: Request,
sketch_id: str,
db: Session = Depends(get_db),
current_user: Profile = Depends(get_current_user_sse),
current_user: Profile = Depends(get_current_user),
):
"""Stream COMPLETED events for a specific sketch (for graph refresh)."""
service = create_log_service(db)
@@ -160,49 +160,3 @@ async def stream_sketch_status(
"X-Accel-Buffering": "no",
},
)
@router.get("/status/scan/{scan_id}/stream")
async def stream_status(
request: Request,
scan_id: str,
db: Session = Depends(get_db),
current_user: Profile = Depends(get_current_user_sse),
):
"""Stream status updates for a specific scan in real-time."""
service = create_log_service(db)
try:
service.get_scan_with_permission(scan_id, current_user.id)
except NotFoundError as e:
raise HTTPException(status_code=404, detail=str(e))
except PermissionDeniedError:
raise HTTPException(status_code=403, detail="Forbidden")
async def status_generator():
print("[EventEmitter] Start status generator")
await event_emitter.subscribe(f"scan_{scan_id}_status")
try:
yield 'data: {"event": "connected", "data": "Connected to status stream"}\n\n'
while True:
data = await event_emitter.get_message(f"scan_{scan_id}_status")
if data is None:
await asyncio.sleep(0.1)
continue
print(f"[EventEmitter] Received status data: {data}")
yield f"data: {data}\n\n"
except asyncio.CancelledError:
print(f"[EventEmitter] Client disconnected from status stream for scan_id: {scan_id}")
finally:
await event_emitter.unsubscribe(f"scan_{scan_id}_status")
return EventSourceResponse(
status_generator(),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"Connection": "keep-alive",
"X-Accel-Buffering": "no",
},
)

View File

@@ -11,6 +11,9 @@ from flowsint_core.core.services import (
PermissionDeniedError,
create_flow_service,
)
from flowsint_core.core.services.type_registry_service import (
create_type_registry_service,
)
from flowsint_core.core.types import FlowBranch, FlowEdge, FlowNode, FlowStep
from flowsint_core.utils import extract_input_schema_flow
from flowsint_enrichers import ENRICHER_REGISTRY, load_all_enrichers
@@ -66,7 +69,7 @@ class launchFlowPayload(BaseModel):
router = APIRouter()
@router.get("/", response_model=List[FlowRead])
@router.get("", response_model=List[FlowRead])
def get_flows(
category: Optional[str] = Query(None),
db: Session = Depends(get_db),
@@ -202,7 +205,11 @@ async def launch_flow(
service.get_sketch_for_launch(payload.sketch_id, current_user.id)
# Retrieve entities from Neo4J by their element IDs
graph_service = create_graph_service(sketch_id=payload.sketch_id)
type_registry = create_type_registry_service(db)
resolver = type_registry.build_type_resolver(current_user.id)
graph_service = create_graph_service(
sketch_id=payload.sketch_id, type_resolver=resolver
)
entities = graph_service.get_nodes_by_ids_for_task(payload.node_ids)
# Compute flow branches
@@ -289,6 +296,7 @@ def compute_flow_branches(
FlowStep(
nodeId="error",
inputs={},
params={},
type="error",
outputs={},
status="error",

View File

@@ -10,6 +10,7 @@ from flowsint_core.core.services import (
create_investigation_service,
NotFoundError,
PermissionDeniedError,
ConflictError,
DatabaseError,
)
from app.api.deps import get_current_user
@@ -17,12 +18,24 @@ from app.api.schemas.investigation import (
InvestigationRead,
InvestigationCreate,
InvestigationUpdate,
CollaboratorAdd,
CollaboratorUpdate,
CollaboratorRead,
)
from app.api.schemas.sketch import SketchRead
router = APIRouter()
def _inject_current_user_role(service, investigation, user_id) -> InvestigationRead:
"""Build InvestigationRead with the current user's role attached."""
result = InvestigationRead.model_validate(investigation)
role_entry = service.get_user_role_for_investigation(user_id, investigation.id)
if role_entry and role_entry.roles:
result.current_user_role = role_entry.roles[0].value
return result
@router.get("", response_model=List[InvestigationRead])
def get_investigations(
db: Session = Depends(get_db),
@@ -30,10 +43,14 @@ def get_investigations(
):
"""Get all investigations accessible to the user based on their roles."""
service = create_investigation_service(db)
allowed_roles = [Role.OWNER, Role.EDITOR, Role.VIEWER]
return service.get_accessible_investigations(
allowed_roles = [Role.OWNER, Role.ADMIN, Role.EDITOR, Role.VIEWER]
investigations = service.get_accessible_investigations(
user_id=current_user.id, allowed_roles=allowed_roles
)
return [
_inject_current_user_role(service, inv, current_user.id)
for inv in investigations
]
@router.post(
@@ -45,11 +62,12 @@ def create_investigation(
current_user: Profile = Depends(get_current_user),
):
service = create_investigation_service(db)
return service.create(
investigation = service.create(
name=payload.name,
description=payload.description,
owner_id=current_user.id,
)
return _inject_current_user_role(service, investigation, current_user.id)
@router.get("/{investigation_id}", response_model=InvestigationRead)
@@ -60,7 +78,8 @@ def get_investigation_by_id(
):
service = create_investigation_service(db)
try:
return service.get_by_id(investigation_id, current_user.id)
investigation = service.get_by_id(investigation_id, current_user.id)
return _inject_current_user_role(service, investigation, current_user.id)
except NotFoundError:
raise HTTPException(status_code=404, detail="Investigation not found")
except PermissionDeniedError:
@@ -93,13 +112,14 @@ def update_investigation(
):
service = create_investigation_service(db)
try:
return service.update(
investigation = service.update(
investigation_id=investigation_id,
user_id=current_user.id,
name=payload.name,
description=payload.description,
status=payload.status,
)
return _inject_current_user_role(service, investigation, current_user.id)
except NotFoundError:
raise HTTPException(status_code=404, detail="Investigation not found")
except PermissionDeniedError:
@@ -122,3 +142,126 @@ def delete_investigation(
raise HTTPException(status_code=403, detail="Forbidden")
except DatabaseError:
raise HTTPException(status_code=500, detail="Failed to clean up graph data")
# ── Collaborator endpoints ───────────────────────────────────────────
@router.get(
"/{investigation_id}/collaborators", response_model=List[CollaboratorRead]
)
def get_collaborators(
investigation_id: UUID,
db: Session = Depends(get_db),
current_user: Profile = Depends(get_current_user),
):
service = create_investigation_service(db)
try:
entries = service.get_collaborators(investigation_id, current_user.id)
return [
CollaboratorRead(
id=e.id,
user_id=e.user_id,
roles=[r.value for r in e.roles],
user=e.user,
)
for e in entries
]
except PermissionDeniedError:
raise HTTPException(status_code=403, detail="Forbidden")
@router.post(
"/{investigation_id}/collaborators",
response_model=CollaboratorRead,
status_code=status.HTTP_201_CREATED,
)
def add_collaborator(
investigation_id: UUID,
payload: CollaboratorAdd,
db: Session = Depends(get_db),
current_user: Profile = Depends(get_current_user),
):
service = create_investigation_service(db)
try:
role = Role(payload.role.lower())
except ValueError:
raise HTTPException(status_code=400, detail="Invalid role")
try:
entry = service.add_collaborator(
investigation_id=investigation_id,
user_id=current_user.id,
target_email=payload.email,
role=role,
)
return CollaboratorRead(
id=entry.id,
user_id=entry.user_id,
roles=[r.value for r in entry.roles],
user=entry.user,
)
except NotFoundError as e:
raise HTTPException(status_code=404, detail=str(e.message))
except PermissionDeniedError:
raise HTTPException(status_code=403, detail="Forbidden")
except ConflictError:
raise HTTPException(status_code=409, detail="User is already a collaborator")
@router.put(
"/{investigation_id}/collaborators/{user_id}",
response_model=CollaboratorRead,
)
def update_collaborator_role(
investigation_id: UUID,
user_id: UUID,
payload: CollaboratorUpdate,
db: Session = Depends(get_db),
current_user: Profile = Depends(get_current_user),
):
service = create_investigation_service(db)
try:
role = Role(payload.role.lower())
except ValueError:
raise HTTPException(status_code=400, detail="Invalid role")
try:
entry = service.update_collaborator_role(
investigation_id=investigation_id,
user_id=current_user.id,
target_user_id=user_id,
role=role,
)
return CollaboratorRead(
id=entry.id,
user_id=entry.user_id,
roles=[r.value for r in entry.roles],
user=entry.user,
)
except NotFoundError:
raise HTTPException(status_code=404, detail="Collaborator not found")
except PermissionDeniedError:
raise HTTPException(status_code=403, detail="Forbidden")
@router.delete(
"/{investigation_id}/collaborators/{user_id}",
status_code=status.HTTP_204_NO_CONTENT,
)
def remove_collaborator(
investigation_id: UUID,
user_id: UUID,
db: Session = Depends(get_db),
current_user: Profile = Depends(get_current_user),
):
service = create_investigation_service(db)
try:
service.remove_collaborator(
investigation_id=investigation_id,
user_id=current_user.id,
target_user_id=user_id,
)
return None
except NotFoundError:
raise HTTPException(status_code=404, detail="Collaborator not found")
except PermissionDeniedError:
raise HTTPException(status_code=403, detail="Forbidden")

View File

@@ -1,17 +1,18 @@
from uuid import UUID
from fastapi import APIRouter, HTTPException, Depends, status
from typing import List
from uuid import UUID
from fastapi import APIRouter, Depends, HTTPException, status
from flowsint_core.core.models import Profile
from flowsint_core.core.postgre_db import get_db
from flowsint_core.core.services import (
DatabaseError,
NotFoundError,
create_key_service,
)
from sqlalchemy.orm import Session
from flowsint_core.core.services import (
create_key_service,
NotFoundError,
DatabaseError,
)
from flowsint_core.core.postgre_db import get_db
from flowsint_core.core.models import Profile
from app.api.deps import get_current_user
from app.api.schemas.key import KeyRead, KeyCreate
from app.api.schemas.key import KeyCreate, KeyExists, KeyRead
router = APIRouter()
@@ -33,6 +34,23 @@ def get_keys(
]
@router.get("/chat-key-exists", response_model=KeyExists)
def chat_key_exists(
db: Session = Depends(get_db),
current_user: Profile = Depends(get_current_user),
):
"""
A simple util route to know if any ai chat key exists in the vault for this user
"""
service = create_key_service(db)
try:
key_exists = service.chat_key_exist(current_user.id)
return KeyExists(exists=key_exists)
except NotFoundError as e:
print(e)
return KeyExists(exists=False)
@router.get("/{id}", response_model=KeyRead)
def get_key_by_id(
id: UUID,

View File

@@ -1,28 +1,31 @@
from uuid import UUID
from fastapi import APIRouter, HTTPException, Depends, status
from typing import List
from sqlalchemy.orm import Session
from uuid import UUID
from flowsint_core.core.postgre_db import get_db
from fastapi import APIRouter, Depends, HTTPException, status
from flowsint_core.core.models import Profile
from flowsint_core.core.postgre_db import get_db
from flowsint_core.core.services import (
create_scan_service,
NotFoundError,
PermissionDeniedError,
create_scan_service,
)
from sqlalchemy.orm import Session
from app.api.deps import get_current_user
from app.api.schemas.scan import ScanRead
router = APIRouter()
@router.get("", response_model=List[ScanRead])
@router.get("/sketch/{id}", response_model=List[ScanRead])
def get_scans(
db: Session = Depends(get_db), current_user: Profile = Depends(get_current_user)
id: UUID,
db: Session = Depends(get_db),
current_user: Profile = Depends(get_current_user),
):
"""Get all scans accessible to the current user."""
"""Get all scans accessible to the current user, linked to a sketch."""
service = create_scan_service(db)
return service.get_accessible_scans(current_user.id)
return service.get_accessible_scans_by_sketch_id(current_user.id, id)
@router.get("/{id}", response_model=ScanRead)

View File

@@ -21,6 +21,7 @@ from flowsint_core.core.services import (
ValidationError,
DatabaseError,
)
from flowsint_core.core.services.type_registry_service import create_type_registry_service
from flowsint_core.imports import (
EntityMapping,
ImportService,
@@ -431,7 +432,11 @@ async def analyze_import_file(
raise HTTPException(status_code=400, detail=f"Failed to read file: {str(e)}")
try:
result = ImportService.analyze_file(
type_registry = create_type_registry_service(db)
resolver = type_registry.build_type_resolver(current_user.id)
graph_service = create_graph_service(sketch_id=sketch_id, enable_batching=False, type_resolver=resolver)
import_service = create_import_service(graph_service)
result = import_service.analyze_file(
file_content=content,
filename=file.filename or "unknown.txt",
)
@@ -487,7 +492,9 @@ async def execute_import(
for m in entity_mapping_inputs
]
graph_service = create_graph_service(sketch_id=sketch_id, enable_batching=False)
type_registry = create_type_registry_service(db)
resolver = type_registry.build_type_resolver(current_user.id)
graph_service = create_graph_service(sketch_id=sketch_id, enable_batching=False, type_resolver=resolver)
import_service = create_import_service(graph_service)
try:

View File

@@ -1,4 +1,5 @@
from fastapi import APIRouter, Depends
from pydantic import BaseModel
from sqlalchemy.orm import Session
from flowsint_core.core.models import Profile
@@ -9,10 +10,29 @@ from app.api.deps import get_current_user
router = APIRouter()
@router.get("/")
@router.get("")
async def get_types_list(
db: Session = Depends(get_db), current_user: Profile = Depends(get_current_user)
):
"""Get the complete types list for sketches."""
service = create_type_registry_service(db)
return service.get_types_list(current_user.id)
class DetectRequest(BaseModel):
text: str
@router.post("/detect")
async def detect_type(
body: DetectRequest,
db: Session = Depends(get_db),
current_user: Profile = Depends(get_current_user),
):
"""Detect the type of a given text input.
Returns the detected type and its fields with the primary field pre-filled.
Falls back to Phrase if no type matches.
"""
service = create_type_registry_service(db)
return service.detect_type(body.text)

View File

@@ -1,20 +1,31 @@
from typing import Optional, Dict, Any
from pydantic import BaseModel, UUID4, Field, field_validator
from datetime import datetime
from typing import Any, Dict, Optional
from pydantic import UUID4, BaseModel, Field, field_validator
from .base import ORMBase
class CustomTypeCreate(BaseModel):
"""Schema for creating a new custom type."""
name: str = Field(..., min_length=1, max_length=255, description="Name of the custom type")
json_schema: Dict[str, Any] = Field(..., description="JSON Schema definition", alias="schema")
description: Optional[str] = Field(None, description="Optional description of the custom type")
status: Optional[str] = Field("draft", description="Status: draft or published")
name: str = Field(
..., min_length=1, max_length=255, description="Name of the custom type"
)
json_schema: Dict[str, Any] = Field(
..., description="JSON Schema definition", alias="schema"
)
description: Optional[str] = Field(
None, description="Optional description of the custom type"
)
status: str = Field("draft", description="Status of the custom type")
color: str = Field("#8E9E8C", description="Default color")
icon: str = Field("Minus", description="Default icon")
class Config:
populate_by_name = True
@field_validator('status')
@field_validator("status")
@classmethod
def validate_status(cls, v: str) -> str:
if v not in ["draft", "published", "archived"]:
@@ -24,15 +35,18 @@ class CustomTypeCreate(BaseModel):
class CustomTypeUpdate(BaseModel):
"""Schema for updating an existing custom type."""
name: Optional[str] = Field(None, min_length=1, max_length=255)
json_schema: Optional[Dict[str, Any]] = Field(None, alias="schema")
description: Optional[str] = None
status: Optional[str] = None
color: Optional[str] = None
icon: Optional[str] = None
class Config:
populate_by_name = True
@field_validator('status')
@field_validator("status")
@classmethod
def validate_status(cls, v: Optional[str]) -> Optional[str]:
if v is not None and v not in ["draft", "published", "archived"]:
@@ -42,9 +56,12 @@ class CustomTypeUpdate(BaseModel):
class CustomTypeRead(ORMBase):
"""Schema for reading a custom type."""
id: UUID4
name: str
owner_id: UUID4
color: Optional[str]
icon: Optional[str]
json_schema: Dict[str, Any] = Field(..., alias="schema")
status: str
checksum: Optional[str]
@@ -58,10 +75,14 @@ class CustomTypeRead(ORMBase):
class CustomTypeValidatePayload(BaseModel):
"""Schema for validating a payload against a custom type schema."""
payload: Dict[str, Any] = Field(..., description="Data to validate against the schema")
payload: Dict[str, Any] = Field(
..., description="Data to validate against the schema"
)
class CustomTypeValidateResponse(BaseModel):
"""Response schema for validation."""
valid: bool
errors: Optional[list[str]] = None

View File

@@ -174,6 +174,12 @@ class EnricherTemplateGenerateRequest(BaseModel):
max_length=16000,
description="Free-text description of the desired enricher template",
)
input_type: Optional[str] = Field(
None, description="Flowsint input type name (e.g. 'Ip', 'Domain')"
)
output_type: Optional[str] = Field(
None, description="Flowsint output type name (e.g. 'Ip', 'SocialAccount')"
)
class EnricherTemplateGenerateResponse(BaseModel):

View File

@@ -25,20 +25,7 @@ class InvestigationRead(ORMBase):
owner: Optional[ProfileRead] = None
sketches: list[SketchRead] = []
analyses: list[AnalysisRead] = []
class InvestigationProfileCreate(BaseModel):
investigation_id: UUID4
profile_id: UUID4
role: Optional[str] = "member"
class InvestigationProfileRead(ORMBase):
id: int
created_at: datetime
investigation_id: UUID4
profile_id: UUID4
role: str
current_user_role: Optional[str] = None
class InvestigationUpdate(BaseModel):
@@ -46,3 +33,22 @@ class InvestigationUpdate(BaseModel):
description: Optional[str] = None
last_updated_at: datetime
status: str
# ── Collaborator schemas ─────────────────────────────────────────────
class CollaboratorAdd(BaseModel):
email: str
role: str # "admin", "editor", "viewer"
class CollaboratorUpdate(BaseModel):
role: str # "admin", "editor", "viewer"
class CollaboratorRead(ORMBase):
id: UUID4
user_id: UUID4
roles: list[str] = []
user: Optional[ProfileRead] = None

View File

@@ -1,7 +1,9 @@
from .base import ORMBase
from pydantic import UUID4, BaseModel
from datetime import datetime
from pydantic import UUID4, BaseModel
from .base import ORMBase
class KeyCreate(BaseModel):
key: str
@@ -12,4 +14,8 @@ class KeyRead(ORMBase):
id: UUID4
owner_id: UUID4
name: str
created_at: datetime
created_at: datetime | str
class KeyExists(BaseModel):
exists: bool

View File

@@ -1,5 +1,5 @@
from .base import ORMBase
from pydantic import UUID4, BaseModel, EmailStr
from pydantic import UUID4, BaseModel, ConfigDict, EmailStr
from typing import Optional
@@ -13,3 +13,12 @@ class ProfileRead(ORMBase):
first_name: Optional[str]
last_name: Optional[str]
avatar_url: Optional[str]
email: Optional[str] = None
class ProfileUpdate(BaseModel):
model_config = ConfigDict(extra="forbid")
first_name: Optional[str] = None
last_name: Optional[str] = None
avatar_url: Optional[str] = None

View File

@@ -8,7 +8,7 @@ class ScanCreate(BaseModel):
values: Optional[List[str]] = None
sketch_id: Optional[UUID4] = None
status: Optional[str] = None
results: Optional[Any] = None
details: Optional[Any] = None
class ScanRead(ORMBase):

View File

@@ -1,3 +1,5 @@
import os
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
@@ -16,12 +18,16 @@ from app.api.routes import types
from app.api.routes import custom_types
from app.api.routes import enricher_templates
# Comma-separated list of allowed origins, e.g. "https://app.example.com,https://staging.example.com"
# Falls back to localhost dev origin when unset. Never use "*" with allow_credentials=True.
origins = [
"*",
o.strip()
for o in os.getenv("ALLOWED_ORIGINS", "http://localhost:5173").split(",")
if o.strip()
]
app = FastAPI()
app = FastAPI(ignore_trailing_slash=True, redirect_slashes=False)
app.add_middleware(
CORSMiddleware,

5700
flowsint-api/poetry.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,48 +1,57 @@
[tool.poetry]
[project]
name = "flowsint-api"
version = "1.0.0"
license = "Apache-2.0"
version = "1.2.8"
description = "API server for flowsint"
authors = ["dextmorgn <contact@flowsint.io>"]
packages = [{ include = "app" }]
license = "Apache-2.0"
authors = [{ name = "dextmorgn", email = "contact@flowsint.io" }]
requires-python = ">=3.12,<4.0"
dependencies = [
"flowsint-core",
"flowsint-types",
"flowsint-enrichers",
"fastapi[standard]>=0.115.0,<1.0.0",
"uvicorn>=0.32.0,<0.33.0",
"redis>=5.0,<6.0",
"celery>=5.3,<6.0",
"python-dotenv>=1.0,<2.0",
"python-jose[cryptography]>=3.4,<4.0",
"requests>=2.31,<3.0",
"pydantic>=2.0,<3.0",
"neo4j>=5.0,<6.0",
"sqlalchemy>=2.0,<3.0",
"psycopg2-binary>=2.9,<3.0",
"asyncpg>=0.30,<0.31",
"alembic==1.13.0",
"passlib[bcrypt]>=1.7,<2.0",
"bcrypt>=4.0.0,<5.0.0",
"sse-starlette>=1.8,<2.0",
"networkx>=2.6.3,<3.0.0",
"email-validator>=2.2.0,<3.0.0",
"mistralai>=1.9.3,<2.0.0",
"python-multipart>=0.0.27,<0.1.0",
"openpyxl>=3.1.2,<4.0.0",
"jsonschema>=4.25.1,<5.0.0",
]
[tool.poetry.dependencies]
python = ">=3.12,<4.0"
flowsint-core = { path = "../flowsint-core", develop = true }
flowsint-types = { path = "../flowsint-types", develop = true }
flowsint-enrichers = { path = "../flowsint-enrichers", develop = true }
fastapi = "^0.115.0"
uvicorn = "^0.32.0"
redis = "^5.0"
celery = "^5.3"
python-dotenv = "^1.0"
python-jose = {extras = ["cryptography"], version = "^3.4"}
requests = "^2.31"
pydantic = "^2.0"
neo4j = "^5.0"
sqlalchemy = "^2.0"
psycopg2-binary = "^2.9"
asyncpg = "^0.30"
alembic = "1.13.0"
passlib = {extras = ["bcrypt"], version = "^1.7"}
bcrypt = ">=4.0.0,<5.0.0"
sse-starlette = "^1.8"
networkx = "^2.6.3"
email-validator = "^2.2.0"
mistralai = "^1.9.3"
python-multipart = "^0.0.20"
openpyxl = "^3.1.2"
jsonschema = "^4.25.1"
[tool.poetry.group.dev.dependencies]
black = "^23.0"
isort = "^5.12"
flake8 = "^6.0"
mypy = "^1.5"
[dependency-groups]
dev = [
"black>=26.3.1,<27.0",
"isort>=6.0,<7.0",
"flake8>=7.0,<8.0",
"mypy>=1.17,<2.0",
]
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel]
packages = ["app"]
[tool.uv.sources]
flowsint-core = { workspace = true }
flowsint-types = { workspace = true }
flowsint-enrichers = { workspace = true }
[tool.black]
line-length = 88

View File

@@ -0,0 +1,38 @@
"""Pytest harness for flowsint-api. Provides a TestClient backed by an
in-memory SQLite database, overriding the real Postgres get_db dependency."""
import os
# Env required at import time by flowsint-core modules. Set before importing app.
os.environ.setdefault("AUTH_SECRET", "test-secret-please-ignore")
os.environ.setdefault(
"MASTER_VAULT_KEY_V1", "base64:qnHTmwYb+uoygIw9MsRMY22vS5YPchY+QOi/E79GAvM="
)
import pytest
from fastapi.testclient import TestClient
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from flowsint_core.core.models import Base
from flowsint_core.core.postgre_db import get_db
from app.main import app
@pytest.fixture
def db_session():
engine = create_engine(
"sqlite:///:memory:", connect_args={"check_same_thread": False}
)
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
yield session
session.close()
@pytest.fixture
def client(db_session):
app.dependency_overrides[get_db] = lambda: db_session
yield TestClient(app)
app.dependency_overrides.clear()

View File

@@ -0,0 +1,50 @@
"""Auth and shape contract for the SSE event endpoints."""
import importlib
import pytest
from flowsint_core.core.auth import create_access_token
from flowsint_core.core.models import Profile
def test_get_current_user_sse_is_removed():
"""The query-param SSE auth helper must no longer exist."""
deps = importlib.import_module("app.api.deps")
assert not hasattr(deps, "get_current_user_sse")
def test_log_stream_requires_auth_header(client):
"""No Authorization header -> 401 (no token in URL accepted)."""
res = client.get("/api/events/sketch/abc/stream")
assert res.status_code == 401
def test_status_stream_requires_auth_header(client):
res = client.get("/api/events/sketch/abc/status/stream")
assert res.status_code == 401
def test_log_stream_rejects_token_query_param(client):
"""A token in the URL must NOT authenticate the request anymore."""
token = create_access_token({"sub": "user@example.com"})
res = client.get(f"/api/events/sketch/abc/stream?token={token}")
assert res.status_code == 401
def test_dead_scan_stream_endpoint_removed(client):
"""The unused scan status stream endpoint must be gone."""
res = client.get("/api/events/status/scan/abc/stream")
assert res.status_code == 404
def test_get_current_user_accepts_valid_bearer(db_session):
"""The dependency the streams now use authenticates via a valid JWT."""
from app.api.deps import get_current_user
db_session.add(Profile(email="user@example.com", hashed_password="x"))
db_session.commit()
token = create_access_token({"sub": "user@example.com"})
user = get_current_user(token=token, db=db_session)
assert user.email == "user@example.com"

BIN
flowsint-app/.DS_Store vendored

Binary file not shown.

View File

@@ -11,6 +11,7 @@
"format": "prettier --write .",
"lint": "eslint . --ext .js,.jsx,.cjs,.mjs,.ts,.tsx,.cts,.mts --fix",
"typecheck": "tsc --noEmit",
"test": "vitest run",
"dev": "vite",
"build": "vite build",
"preview": "vite preview"
@@ -19,6 +20,7 @@
"@ai-sdk/react": "^3.0.79",
"@dagrejs/dagre": "^1.1.4",
"@hookform/resolvers": "^5.0.1",
"@microsoft/fetch-event-source": "^2.0.1",
"@monaco-editor/react": "^4.7.0",
"@radix-ui/react-accordion": "^1.2.11",
"@radix-ui/react-alert-dialog": "^1.1.14",
@@ -139,6 +141,7 @@
"react-dom": "^19.2.0",
"standard-version": "^9.5.0",
"typescript": "^5.5.2",
"vite": "^5.3.1"
"vite": "^5.3.1",
"vitest": "^2.1.9"
}
}

View File

@@ -49,6 +49,17 @@ export const authService = {
},
getCurrentUser: async () => {
return fetchWithAuth('/api/users/me')
return fetchWithAuth('/api/auth/me')
},
updateProfile: async (data: { first_name?: string; last_name?: string; avatar_url?: string }) => {
return fetchWithAuth('/api/auth/me', {
method: 'PUT',
body: JSON.stringify(data)
})
},
searchUsers: async (query: string) => {
return fetchWithAuth(`/api/auth/users/search?q=${encodeURIComponent(query)}`)
}
}

View File

@@ -8,6 +8,8 @@ export interface CustomType {
status: 'draft' | 'published' | 'archived'
checksum?: string
description?: string
icon?: string
color?: string
created_at: string
updated_at: string
}
@@ -16,6 +18,8 @@ export interface CustomTypeCreate {
name: string
schema: Record<string, any>
description?: string
icon?: string
color?: string
status?: 'draft' | 'published'
}
@@ -23,6 +27,8 @@ export interface CustomTypeUpdate {
name?: string
schema?: Record<string, any>
description?: string
icon?: string
color?: string
status?: 'draft' | 'published' | 'archived'
}

View File

@@ -21,5 +21,29 @@ export const investigationService = {
return fetchWithAuth(`/api/investigations/${investigationId}`, {
method: 'DELETE'
})
},
// Collaborator management
getCollaborators: async (investigationId: string): Promise<any> => {
return fetchWithAuth(`/api/investigations/${investigationId}/collaborators`, {
method: 'GET'
})
},
addCollaborator: async (investigationId: string, body: { email: string; role: string }): Promise<any> => {
return fetchWithAuth(`/api/investigations/${investigationId}/collaborators`, {
method: 'POST',
body: JSON.stringify(body)
})
},
updateCollaboratorRole: async (investigationId: string, userId: string, body: { role: string }): Promise<any> => {
return fetchWithAuth(`/api/investigations/${investigationId}/collaborators/${userId}`, {
method: 'PUT',
body: JSON.stringify(body)
})
},
removeCollaborator: async (investigationId: string, userId: string): Promise<any> => {
return fetchWithAuth(`/api/investigations/${investigationId}/collaborators/${userId}`, {
method: 'DELETE'
})
}
}

View File

@@ -1,6 +1,6 @@
import { fetchWithAuth } from './api'
export const KeyService = {
export const keyService = {
get: () => fetchWithAuth(`/api/keys`, { method: 'GET' }),
getById: (key_id: string) => fetchWithAuth(`/api/keys/${key_id}`, { method: 'GET' }),
create: (data: { name: string; key: string }) =>
@@ -8,5 +8,6 @@ export const KeyService = {
method: 'POST',
body: JSON.stringify(data)
}),
chatKeyExists: () => fetchWithAuth(`/api/keys/chat-key-exists`, { method: 'GET' }),
deleteById: (key_id: string) => fetchWithAuth(`/api/keys/${key_id}`, { method: 'DELETE' })
}

View File

@@ -7,7 +7,7 @@
// import { investigationService } from './investigation-service'
// import { sketchService } from './sketch-service'
// import { chatCRUDService } from './chat-service'
// import { KeyService } from './key-service'
// import { keyService } from './key-service'
// import { logService } from './log-service'
// import { scanService } from './scan-service'
// import { enricherService } from './enricher-service'
@@ -112,14 +112,14 @@
// export const useKeysList = () => {
// return useQuery({
// queryKey: queryKeys.keys.list,
// queryFn: KeyService.get
// queryFn: keyService.get
// })
// }
// export const useKeyDetail = (keyId: string) => {
// return useQuery({
// queryKey: queryKeys.keys.detail(keyId),
// queryFn: () => KeyService.getById(keyId),
// queryFn: () => keyService.getById(keyId),
// enabled: !!keyId
// })
// }

View File

@@ -13,6 +13,7 @@ export const queryKeys = {
sketches: (investigationId: string) => [investigationId, 'sketches'],
analyses: (investigationId: string) => [investigationId, 'analyses'],
flows: (investigationId: string) => [investigationId, 'flows'],
collaborators: (investigationId: string) => ['investigations', investigationId, 'collaborators'],
dashboard: ['investigations', 'dashboard'],
selector: (investigationId: string) => ['dashboard', 'selector', investigationId]
},

View File

@@ -1,7 +1,7 @@
import { fetchWithAuth } from './api'
export const scanService = {
get: (scan_id: string) => fetchWithAuth(`/api/scans/${scan_id}`, { method: 'GET' }),
getById: (scan_id: string) => fetchWithAuth(`/api/scans/${scan_id}`, { method: 'GET' }),
delete: (scan_id: string) => fetchWithAuth(`/api/scans/${scan_id}`, { method: 'DELETE' }),
getSketchScans: (sketch_id: string) =>
fetchWithAuth(`/api/scans/sketch/${sketch_id}`, { method: 'GET' }),

View File

@@ -79,6 +79,12 @@ export const sketchService = {
method: 'GET'
})
},
detectType: async (text: string): Promise<any> => {
return fetchWithAuth(`/api/types/detect`, {
method: 'POST',
body: JSON.stringify({ text })
})
},
update: async (sketchId: string, body: BodyInit): Promise<any> => {
return fetchWithAuth(`/api/sketches/${sketchId}`, {
method: 'PUT',

View File

@@ -0,0 +1,117 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
// Capture the options passed to fetchEventSource so we can drive its callbacks.
const calls: any[] = []
const fetchEventSourceMock = vi.fn((url: string, opts: any) => {
calls.push({ url, opts })
return new Promise(() => {}) // never resolves; we drive callbacks manually
})
vi.mock('@microsoft/fetch-event-source', () => ({
fetchEventSource: (url: string, opts: any) => fetchEventSourceMock(url, opts),
}))
vi.mock('@/stores/auth-store', () => ({
useAuthStore: { getState: () => ({ token: 'tok123' }) },
}))
import { connectSSE } from './sse'
const lastOpts = () => calls[calls.length - 1].opts
const fakeResponse = (status: number, contentType: string) =>
({ ok: status >= 200 && status < 300, status, headers: { get: () => contentType } } as any)
beforeEach(() => {
calls.length = 0
fetchEventSourceMock.mockClear()
})
describe('connectSSE', () => {
it('passes the URL and Authorization header from the store', () => {
connectSSE({ url: 'http://x/stream', onMessage: () => {} })
expect(calls[0].url).toBe('http://x/stream')
expect(lastOpts().headers).toEqual({ Authorization: 'Bearer tok123' })
})
it('forwards parsed envelopes and skips "connected"', () => {
const received: any[] = []
connectSSE({ url: 'http://x', onMessage: (m) => received.push(m) })
const o = lastOpts()
o.onmessage({ data: JSON.stringify({ event: 'connected', data: 'hi' }) })
o.onmessage({ data: JSON.stringify({ event: 'log', data: '{"msg":"a"}' }) })
expect(received).toEqual([{ event: 'log', data: '{"msg":"a"}' }])
})
it('ignores empty and malformed messages without throwing', () => {
const received: any[] = []
connectSSE({ url: 'http://x', onMessage: (m) => received.push(m) })
const o = lastOpts()
o.onmessage({ data: '' })
o.onmessage({ data: 'not-json' })
expect(received).toEqual([])
})
it('aborts the request when the disposer is called', () => {
const dispose = connectSSE({ url: 'http://x', onMessage: () => {} })
expect(lastOpts().signal.aborted).toBe(false)
dispose()
expect(lastOpts().signal.aborted).toBe(true)
})
it('throws (no retry) on a non-event-stream open', async () => {
connectSSE({ url: 'http://x', onMessage: () => {} })
await expect(lastOpts().onopen(fakeResponse(401, 'application/json'))).rejects.toThrow()
})
it('accepts a valid event-stream open', async () => {
connectSSE({ url: 'http://x', onMessage: () => {} })
await expect(
lastOpts().onopen(fakeResponse(200, 'text/event-stream'))
).resolves.toBeUndefined()
})
it('returns increasing backoff for transient errors', () => {
connectSSE({ url: 'http://x', onMessage: () => {} })
const o = lastOpts()
const first = o.onerror(new Error('network'))
const second = o.onerror(new Error('network'))
expect(first).toBe(2000)
expect(second).toBe(4000)
})
it('rethrows fatal errors from onerror to stop retrying', async () => {
connectSSE({ url: 'http://x', onMessage: () => {} })
const o = lastOpts()
let fatal: unknown
try {
await o.onopen(fakeResponse(403, 'application/json'))
} catch (e) {
fatal = e
}
expect(() => o.onerror(fatal)).toThrow()
})
it('ignores valid JSON that is not an envelope', () => {
const received: any[] = []
connectSSE({ url: 'http://x', onMessage: (m) => received.push(m) })
const o = lastOpts()
o.onmessage({ data: '42' })
o.onmessage({ data: 'null' })
o.onmessage({ data: '{"data":"no-event"}' })
expect(received).toEqual([])
})
it('calls onError once when the connection promise rejects (non-fatal)', async () => {
const errors: unknown[] = []
fetchEventSourceMock.mockImplementationOnce((url: string, opts: any) => {
calls.push({ url, opts })
return Promise.reject(new Error('boom'))
})
connectSSE({ url: 'http://x', onMessage: () => {}, onError: (e) => errors.push(e) })
await Promise.resolve()
await Promise.resolve()
expect(errors).toHaveLength(1)
expect((errors[0] as Error).message).toBe('boom')
})
})

View File

@@ -0,0 +1,72 @@
import { fetchEventSource } from '@microsoft/fetch-event-source'
import { useAuthStore } from '@/stores/auth-store'
/**
* Outer SSE envelope from the server. `data` is forwarded exactly as received;
* for the log/status streams it is itself a JSON-encoded string that the
* consumer parses (e.g. `JSON.parse(envelope.data)`).
*/
export type SSEEnvelope = { event: string; data: unknown }
/** Thrown to tell fetch-event-source to stop retrying (e.g. auth failure). */
class FatalSSEError extends Error {}
const MAX_BACKOFF_MS = 30000
/**
* Open an authenticated SSE connection.
* Reads the bearer token once at connect time and sends it via the
* Authorization header. Returns a disposer that aborts the connection.
*/
export function connectSSE(opts: {
url: string
onMessage: (msg: SSEEnvelope) => void
onError?: (err: unknown) => void
}): () => void {
const controller = new AbortController()
let retry = 0
const token = useAuthStore.getState().token
const headers: Record<string, string> = token
? { Authorization: `Bearer ${token}` }
: {}
fetchEventSource(opts.url, {
signal: controller.signal,
openWhenHidden: true,
headers,
onopen: async (res) => {
const contentType = res.headers.get('content-type') ?? ''
if (!res.ok || !contentType.includes('text/event-stream')) {
throw new FatalSSEError(`SSE open failed: ${res.status}`)
}
retry = 0
},
onmessage: (ev) => {
if (!ev.data) return
let envelope: SSEEnvelope
try {
envelope = JSON.parse(ev.data) as SSEEnvelope
} catch (error) {
console.error('[connectSSE] malformed envelope:', error, ev.data)
return
}
if (typeof envelope?.event !== 'string') return
if (envelope.event === 'connected') return
opts.onMessage(envelope)
},
onerror: (err) => {
if (err instanceof FatalSSEError) {
opts.onError?.(err)
throw err // stop: no retry on auth/protocol failure
}
// transient failure: retry silently with exponential backoff (onError is not called)
retry += 1
return Math.min(1000 * 2 ** retry, MAX_BACKOFF_MS)
},
}).catch((err) => {
if (!(err instanceof FatalSSEError)) opts.onError?.(err)
})
return () => controller.abort()
}

View File

@@ -91,10 +91,18 @@ export const templateService = {
})
},
generate: async (prompt: string): Promise<GenerateTemplateResponse> => {
generate: async (
prompt: string,
inputType?: string,
outputType?: string
): Promise<GenerateTemplateResponse> => {
return fetchWithAuth('/api/enrichers/templates/generate', {
method: 'POST',
body: JSON.stringify({ prompt })
body: JSON.stringify({
prompt,
input_type: inputType || null,
output_type: outputType || null
})
})
}
}

View File

@@ -13,6 +13,7 @@ import { toast } from 'sonner'
import { formatDistanceToNow } from 'date-fns'
import { queryKeys } from '@/api/query-keys'
import ErrorState from '../shared/error-state'
import { usePermissions } from '@/hooks/use-can'
const AnalysisItem = ({ analysis, active }: { analysis: Analysis; active: boolean }) => {
return (
@@ -46,6 +47,7 @@ const AnalysisItem = ({ analysis, active }: { analysis: Analysis; active: boolea
}
const AnalysisList = () => {
const { canEdit } = usePermissions()
const { investigationId, id, type } = useParams({ strict: false })
const queryClient = useQueryClient()
const navigate = useNavigate()
@@ -112,16 +114,18 @@ const AnalysisList = () => {
return (
<div className="w-full h-full bg-card flex flex-col overflow-hidden">
<div className="p-2 flex items-center h-11 gap-2 border-b shrink-0">
<Button
variant="ghost"
size="icon"
className="h-7 w-7"
onClick={() => createMutation.mutate()}
disabled={createMutation.isPending}
title="Create New analysis"
>
<PlusIcon className="h-4 w-4" />
</Button>
{canEdit && (
<Button
variant="ghost"
size="icon"
className="h-7 w-7"
onClick={() => createMutation.mutate()}
disabled={createMutation.isPending}
title="Create New analysis"
>
<PlusIcon className="h-4 w-4" />
</Button>
)}
<Input
type="search"
className="!border border-border h-7"

View File

@@ -19,6 +19,7 @@ import { useConfirm } from '../use-confirm-dialog'
import { Editor } from '@tiptap/core'
import { Link, useParams } from '@tanstack/react-router'
import { useLayoutStore } from '@/stores/layout-store'
import { usePermissions } from '@/hooks/use-can'
import { Popover, PopoverContent, PopoverTrigger } from '@/components/ui/popover'
import {
DropdownMenu,
@@ -78,6 +79,7 @@ export const AnalysisEditor = ({
showToolbar = false
}: AnalysisEditorProps) => {
const { confirm } = useConfirm()
const { canEdit } = usePermissions()
const toggleAnalysis = useLayoutStore((s) => s.toggleAnalysis)
const { investigationId: routeInvestigationId, type } = useParams({ strict: false }) as {
investigationId: string
@@ -318,7 +320,7 @@ export const AnalysisEditor = ({
{/* Title section */}
<div className="flex items-center gap-2 flex-1 min-w-0">
{isEditingTitle ? (
{canEdit && isEditingTitle ? (
<input
className="text-md font-medium bg-transparent outline-none border-none p-0 m-0 w-full"
value={titleValue}
@@ -340,8 +342,8 @@ export const AnalysisEditor = ({
/>
) : (
<span
className={`text-md font-medium truncate min-w-0 flex-1 ${'cursor-pointer hover:text-primary'}`}
onClick={() => setIsEditingTitle(true)}
className={`text-md font-medium truncate min-w-0 flex-1 ${canEdit ? 'cursor-pointer hover:text-primary' : ''}`}
onClick={canEdit ? () => setIsEditingTitle(true) : undefined}
>
{titleValue || 'Untitled Analysis'}
</span>
@@ -350,7 +352,7 @@ export const AnalysisEditor = ({
</div>
{/* Action buttons */}
{showActions && (
{showActions && canEdit && (
<div className="flex items-center gap-1">
<SaveStatusBadge status={saveStatus} />
<DropdownMenu>
@@ -426,29 +428,32 @@ export const AnalysisEditor = ({
}
return content || ''
})()}
onChange={handleEditorChange}
onChange={canEdit ? handleEditorChange : undefined}
className="w-full h-full"
editorContentClassName="p-5 min-h-[300px]"
output="json"
placeholder={'Enter your analysis...'}
autofocus={true}
showToolbar={showToolbar}
placeholder={canEdit ? 'Enter your analysis...' : ''}
autofocus={canEdit}
showToolbar={canEdit && showToolbar}
editorClassName="focus:outline-hidden"
onEditorReady={setEditor}
editable={canEdit}
/>
</div>
) : (
<div className="flex flex-col items-center justify-center h-full text-muted-foreground space-y-3">
<div>No analysis selected.</div>
<Button
className="shadow-none"
variant="outline"
onClick={() => createMutation.mutate()}
disabled={createMutation.isPending}
>
<PlusIcon className="w-4 h-4 mr-2" strokeWidth={1.5} />
Create your first analysis
</Button>
{canEdit && (
<Button
className="shadow-none"
variant="outline"
onClick={() => createMutation.mutate()}
disabled={createMutation.isPending}
>
<PlusIcon className="w-4 h-4 mr-2" strokeWidth={1.5} />
Create your first analysis
</Button>
)}
</div>
)}
</div>

View File

@@ -0,0 +1,246 @@
import * as React from 'react'
import type { Editor } from '@tiptap/react'
import { BubbleMenu } from '@tiptap/react/menus'
import {
Bold,
Italic,
Underline,
Strikethrough,
Code,
Heading1,
Heading2,
Heading3,
Pilcrow,
Link as LinkIcon,
Highlighter,
ChevronDown
} from 'lucide-react'
import { Separator } from '@/components/ui/separator'
import { Popover, PopoverContent, PopoverTrigger } from '@/components/ui/popover'
import { LinkEditBlock } from '../link/link-edit-block'
import { cn } from '@/lib/utils'
interface TextBubbleMenuProps {
editor: Editor
}
const prevent = (e: React.MouseEvent) => e.preventDefault()
const BubbleButton = ({
active,
onClick,
disabled,
children,
}: {
active?: boolean
onClick: () => void
disabled?: boolean
children: React.ReactNode
}) => (
<button
type="button"
className={cn(
'flex h-7 w-7 items-center justify-center rounded-sm text-sm transition-colors',
'hover:bg-accent hover:text-accent-foreground',
active && 'bg-accent text-accent-foreground',
disabled && 'pointer-events-none opacity-50'
)}
onMouseDown={prevent}
onClick={onClick}
disabled={disabled}
>
{children}
</button>
)
export const TextBubbleMenu: React.FC<TextBubbleMenuProps> = ({ editor }) => {
const [showLinkEditor, setShowLinkEditor] = React.useState(false)
const shouldShow = React.useCallback(
({ editor: ed, from, to }: { editor: Editor; from: number; to: number }) => {
if (from === to) return false
if (!ed.isEditable) return false
if (ed.isActive('codeBlock')) return false
if (ed.isActive('link')) return false
return true
},
[]
)
const handleSetLink = React.useCallback(
(url: string, text?: string, openInNewTab?: boolean) => {
if (text) {
editor
.chain()
.focus()
.insertContent({
type: 'text',
text,
marks: [
{
type: 'link',
attrs: { href: url, target: openInNewTab ? '_blank' : '' }
}
]
})
.run()
} else {
editor
.chain()
.focus()
.setLink({ href: url, target: openInNewTab ? '_blank' : '' })
.run()
}
setShowLinkEditor(false)
},
[editor]
)
const activeHeading = editor.isActive('heading', { level: 1 })
? 'H1'
: editor.isActive('heading', { level: 2 })
? 'H2'
: editor.isActive('heading', { level: 3 })
? 'H3'
: 'P'
return (
<BubbleMenu
editor={editor}
pluginKey="textBubbleMenu"
shouldShow={shouldShow}
options={{
placement: 'top',
offset: 8
}}
>
<div
className="flex items-center gap-0.5 rounded-lg border bg-popover p-1 shadow-md"
onMouseDown={prevent}
>
<BubbleButton
active={editor.isActive('bold')}
onClick={() => editor.chain().focus().toggleBold().run()}
disabled={!editor.can().chain().focus().toggleBold().run()}
>
<Bold className="size-4" />
</BubbleButton>
<BubbleButton
active={editor.isActive('italic')}
onClick={() => editor.chain().focus().toggleItalic().run()}
disabled={!editor.can().chain().focus().toggleItalic().run()}
>
<Italic className="size-4" />
</BubbleButton>
<BubbleButton
active={editor.isActive('underline')}
onClick={() => editor.chain().focus().toggleUnderline().run()}
disabled={!editor.can().chain().focus().toggleUnderline().run()}
>
<Underline className="size-4" />
</BubbleButton>
<BubbleButton
active={editor.isActive('strike')}
onClick={() => editor.chain().focus().toggleStrike().run()}
disabled={!editor.can().chain().focus().toggleStrike().run()}
>
<Strikethrough className="size-4" />
</BubbleButton>
<BubbleButton
active={editor.isActive('code')}
onClick={() => editor.chain().focus().toggleCode().run()}
disabled={!editor.can().chain().focus().toggleCode().run()}
>
<Code className="size-4" />
</BubbleButton>
<Separator orientation="vertical" className="mx-0.5 h-6" />
{/* Heading dropdown */}
<Popover>
<PopoverTrigger asChild>
<button
type="button"
className="flex h-7 items-center gap-0.5 rounded-sm px-2 text-xs font-semibold hover:bg-accent hover:text-accent-foreground"
onMouseDown={prevent}
>
{activeHeading}
<ChevronDown className="size-3" />
</button>
</PopoverTrigger>
<PopoverContent className="w-auto p-1" align="start">
<div className="flex flex-col gap-0.5">
<button
type="button"
className="flex items-center gap-2 rounded-sm px-2 py-1 text-sm hover:bg-accent"
onMouseDown={prevent}
onClick={() => editor.chain().focus().setParagraph().run()}
>
<Pilcrow className="size-4" />
Paragraph
</button>
<button
type="button"
className="flex items-center gap-2 rounded-sm px-2 py-1 text-sm hover:bg-accent"
onMouseDown={prevent}
onClick={() => editor.chain().focus().setHeading({ level: 1 }).run()}
>
<Heading1 className="size-4" />
Heading 1
</button>
<button
type="button"
className="flex items-center gap-2 rounded-sm px-2 py-1 text-sm hover:bg-accent"
onMouseDown={prevent}
onClick={() => editor.chain().focus().setHeading({ level: 2 }).run()}
>
<Heading2 className="size-4" />
Heading 2
</button>
<button
type="button"
className="flex items-center gap-2 rounded-sm px-2 py-1 text-sm hover:bg-accent"
onMouseDown={prevent}
onClick={() => editor.chain().focus().setHeading({ level: 3 }).run()}
>
<Heading3 className="size-4" />
Heading 3
</button>
</div>
</PopoverContent>
</Popover>
<Separator orientation="vertical" className="mx-0.5 h-6" />
{/* Link */}
<Popover open={showLinkEditor} onOpenChange={setShowLinkEditor}>
<PopoverTrigger asChild>
<button
type="button"
className={cn(
'flex h-7 w-7 items-center justify-center rounded-sm hover:bg-accent hover:text-accent-foreground',
editor.isActive('link') && 'bg-accent text-accent-foreground'
)}
onMouseDown={prevent}
>
<LinkIcon className="size-4" />
</button>
</PopoverTrigger>
<PopoverContent className="w-full min-w-80 p-0" align="start">
<LinkEditBlock onSave={handleSetLink} className="p-4" />
</PopoverContent>
</Popover>
<Separator orientation="vertical" className="mx-0.5 h-6" />
{/* Highlight */}
<BubbleButton
active={editor.isActive('highlight')}
onClick={() => editor.chain().focus().toggleHighlight().run()}
>
<Highlighter className="size-4" />
</BubbleButton>
</div>
</BubbleMenu>
)
}

View File

@@ -0,0 +1,242 @@
import { Extension } from "@tiptap/core";
import { Plugin, PluginKey } from "@tiptap/pm/state";
const dragHandleKey = new PluginKey("dragHandle");
function findBlockAt(
view: any,
y: number,
): { pos: number; dom: HTMLElement } | null {
const editorRect = view.dom.getBoundingClientRect();
const paddingLeft = parseFloat(getComputedStyle(view.dom).paddingLeft) || 0;
const coords = { left: editorRect.left + paddingLeft + 1, top: y };
const posInfo = view.posAtCoords(coords);
if (!posInfo) return null;
const $pos = view.state.doc.resolve(posInfo.pos);
const wrapperTypes = new Set([
"bulletList", "orderedList", "taskList",
]);
for (let d = $pos.depth; d >= 1; d--) {
const node = $pos.node(d);
if (!node.isBlock) continue;
if (d === 1 && $pos.before(1) === 0) continue;
if (wrapperTypes.has(node.type.name)) continue;
if (node.isTextblock && d > 1) {
const parent = $pos.node(d - 1);
if (parent.type.name === "listItem" || parent.type.name === "taskItem") {
continue;
}
}
const pos = $pos.before(d);
const dom = view.nodeDOM(pos);
if (dom instanceof HTMLElement) {
return { pos, dom };
}
}
return null;
}
export const DragHandle = Extension.create({
name: "dragHandle",
addProseMirrorPlugins() {
const handle = document.createElement("div");
handle.className = "flowsint-drag-handle";
handle.innerHTML = `<svg width="14" height="14" viewBox="0 0 14 14" fill="currentColor"><circle cx="4" cy="2" r="1.5"/><circle cx="10" cy="2" r="1.5"/><circle cx="4" cy="7" r="1.5"/><circle cx="10" cy="7" r="1.5"/><circle cx="4" cy="12" r="1.5"/><circle cx="10" cy="12" r="1.5"/></svg>`;
const indicator = document.createElement("div");
indicator.className = "flowsint-drop-indicator";
let currentBlockPos: number | null = null;
let dragging: { pos: number; ghost: HTMLElement } | null = null;
let hideTimeout: ReturnType<typeof setTimeout> | null = null;
return [
new Plugin({
key: dragHandleKey,
view(view) {
const wrapper = view.dom.parentElement!;
wrapper.style.position = "relative";
wrapper.appendChild(handle);
wrapper.appendChild(indicator);
const positionHandle = (blockDom: HTMLElement, pos: number) => {
if (hideTimeout) clearTimeout(hideTimeout);
currentBlockPos = pos;
const wr = wrapper.getBoundingClientRect();
const br = blockDom.getBoundingClientRect();
handle.style.top = `${br.top - wr.top}px`;
handle.style.left = `${br.left - wr.left - 24}px`;
handle.style.height = `${br.height}px`;
handle.style.display = "flex";
};
const hideAll = () => {
handle.style.display = "none";
indicator.style.display = "none";
currentBlockPos = null;
};
const scheduleHide = () => {
if (hideTimeout) clearTimeout(hideTimeout);
hideTimeout = setTimeout(hideAll, 150);
};
const cancelHide = () => {
if (hideTimeout) {
clearTimeout(hideTimeout);
hideTimeout = null;
}
};
const onEditorMouseMove = (e: MouseEvent) => {
if (dragging || e.buttons > 0) return;
const block = findBlockAt(view, e.clientY);
if (block) {
if (block.pos !== currentBlockPos) {
positionHandle(block.dom, block.pos);
} else {
cancelHide();
}
} else {
scheduleHide();
}
};
const onEditorLeave = () => {
if (!dragging) scheduleHide();
};
const onHandleEnter = () => cancelHide();
const onHandleLeave = () => {
if (!dragging) scheduleHide();
};
const onHandleMouseDown = (e: MouseEvent) => {
e.preventDefault();
if (currentBlockPos === null) return;
const pos = currentBlockPos;
const blockDom = view.nodeDOM(pos) as HTMLElement;
if (!blockDom) return;
const ghost = blockDom.cloneNode(true) as HTMLElement;
ghost.className = "flowsint-drag-ghost";
ghost.style.width = `${blockDom.offsetWidth}px`;
ghost.style.left = `${e.clientX}px`;
ghost.style.top = `${e.clientY}px`;
document.body.appendChild(ghost);
blockDom.classList.add("flowsint-dragging-source");
dragging = { pos, ghost };
hideAll();
const onMouseMove = (ev: MouseEvent) => {
if (!dragging) return;
dragging.ghost.style.left = `${ev.clientX}px`;
dragging.ghost.style.top = `${ev.clientY}px`;
const block = findBlockAt(view, ev.clientY);
if (block) {
const wr = wrapper.getBoundingClientRect();
const br = block.dom.getBoundingClientRect();
const above = ev.clientY < br.top + br.height / 2;
indicator.style.top = `${(above ? br.top : br.bottom) - wr.top - 1}px`;
indicator.style.left = `${br.left - wr.left}px`;
indicator.style.width = `${br.width}px`;
indicator.style.display = "block";
} else {
indicator.style.display = "none";
}
};
const onMouseUp = (ev: MouseEvent) => {
document.removeEventListener("mousemove", onMouseMove);
document.removeEventListener("mouseup", onMouseUp);
if (!dragging) return;
dragging.ghost.remove();
blockDom.classList.remove("flowsint-dragging-source");
indicator.style.display = "none";
const originPos = dragging.pos;
dragging = null;
const originNode = view.state.doc.nodeAt(originPos);
if (!originNode) return;
const target = findBlockAt(view, ev.clientY);
if (!target) return;
if (
originPos < target.pos &&
originPos + originNode.nodeSize > target.pos
)
return;
const targetNode = view.state.doc.nodeAt(target.pos);
if (!targetNode) return;
const targetRect = target.dom.getBoundingClientRect();
const above = ev.clientY < targetRect.top + targetRect.height / 2;
const insertPos = above
? target.pos
: target.pos + targetNode.nodeSize;
if (
insertPos === originPos ||
insertPos === originPos + originNode.nodeSize
)
return;
const tr = view.state.tr;
if (originPos < insertPos) {
const adjusted = insertPos - originNode.nodeSize;
tr.delete(originPos, originPos + originNode.nodeSize);
tr.insert(adjusted, originNode);
} else {
tr.insert(insertPos, originNode);
tr.delete(
originPos + originNode.nodeSize,
originPos + originNode.nodeSize * 2,
);
}
view.dispatch(tr);
};
document.addEventListener("mousemove", onMouseMove);
document.addEventListener("mouseup", onMouseUp);
};
view.dom.addEventListener("mousemove", onEditorMouseMove);
view.dom.addEventListener("mouseleave", onEditorLeave);
handle.addEventListener("mouseenter", onHandleEnter);
handle.addEventListener("mouseleave", onHandleLeave);
handle.addEventListener("mousedown", onHandleMouseDown);
return {
destroy() {
if (hideTimeout) clearTimeout(hideTimeout);
view.dom.removeEventListener("mousemove", onEditorMouseMove);
view.dom.removeEventListener("mouseleave", onEditorLeave);
handle.removeEventListener("mouseenter", onHandleEnter);
handle.removeEventListener("mouseleave", onHandleLeave);
handle.removeEventListener("mousedown", onHandleMouseDown);
handle.remove();
indicator.remove();
if (dragging) {
dragging.ghost.remove();
dragging = null;
}
},
};
},
}),
];
},
});

View File

@@ -0,0 +1 @@
export { DragHandle } from './drag-handle'

View File

@@ -0,0 +1 @@
export { PasteMarkdown } from './markdown-paste'

View File

@@ -0,0 +1,62 @@
import { Extension } from '@tiptap/core'
import { Plugin } from '@tiptap/pm/state'
import { MarkdownManager } from '@tiptap/markdown'
function looksLikeMarkdown(text: string): boolean {
return (
/^#{1,6}\s/m.test(text) ||
/\*\*[^*]+\*\*/.test(text) ||
/\[.+\]\(.+\)/.test(text) ||
/^[-*+]\s/m.test(text) ||
/^\d+\.\s/m.test(text) ||
/^>\s/m.test(text) ||
/^```/m.test(text) ||
/^---$/m.test(text) ||
/!\[.*\]\(.*\)/.test(text) ||
/^- \[[ x]\]/m.test(text)
)
}
export const PasteMarkdown = Extension.create({
name: 'pasteMarkdown',
addStorage() {
return {
markdownManager: null as MarkdownManager | null
}
},
onCreate() {
this.storage.markdownManager = new MarkdownManager({
extensions: this.editor.extensionManager.baseExtensions
})
},
addProseMirrorPlugins() {
const { editor } = this
const storage = this.storage
return [
new Plugin({
props: {
handlePaste(_view, event) {
const text = event.clipboardData?.getData('text/plain')
if (!text) return false
if (storage.markdownManager && looksLikeMarkdown(text)) {
try {
const json = storage.markdownManager.parse(text)
editor.chain().focus().insertContent(json).run()
return true
} catch (e) {
console.error('[PasteMarkdown]', e)
return false
}
}
return false
}
}
})
]
}
})

View File

@@ -0,0 +1,2 @@
export { SlashCommand } from './slash-command'
export type { SlashCommandItem } from './slash-command'

View File

@@ -0,0 +1,131 @@
import { forwardRef, useEffect, useImperativeHandle, useState, useCallback } from 'react'
import type { SlashCommandItem } from './slash-command'
import {
Heading1,
Heading2,
Heading3,
List,
ListOrdered,
ListTodo,
Quote,
Code,
Minus,
ImageIcon
} from 'lucide-react'
import type { LucideIcon } from 'lucide-react'
const iconMap: Record<string, LucideIcon> = {
Heading1,
Heading2,
Heading3,
List,
ListOrdered,
ListTodo,
Quote,
Code,
Minus,
ImageIcon
}
export interface SlashCommandListProps {
items: SlashCommandItem[]
command: (item: SlashCommandItem) => void
}
export interface SlashCommandListRef {
onKeyDown: (props: { event: KeyboardEvent }) => boolean
}
const SlashCommandList = forwardRef<SlashCommandListRef, SlashCommandListProps>((props, ref) => {
const [selectedIndex, setSelectedIndex] = useState(0)
const selectItem = useCallback(
(index: number) => {
const item = props.items[index]
if (item) {
props.command(item)
}
},
[props]
)
useEffect(() => setSelectedIndex(0), [props.items])
useImperativeHandle(ref, () => ({
onKeyDown: ({ event }: { event: KeyboardEvent }) => {
if (event.key === 'ArrowUp') {
setSelectedIndex((prev) => (prev + props.items.length - 1) % props.items.length)
return true
}
if (event.key === 'ArrowDown') {
setSelectedIndex((prev) => (prev + 1) % props.items.length)
return true
}
if (event.key === 'Enter') {
selectItem(selectedIndex)
return true
}
return false
}
}))
// Group items by category
const grouped = props.items.reduce<Record<string, SlashCommandItem[]>>((acc, item) => {
if (!acc[item.category]) acc[item.category] = []
acc[item.category].push(item)
return acc
}, {})
// Flat index tracking
let flatIndex = -1
if (!props.items.length) {
return (
<div className="z-50 w-[280px] overflow-hidden rounded-md border bg-popover p-1 text-popover-foreground shadow-md">
<div className="px-2 py-1.5 text-sm text-muted-foreground">No results</div>
</div>
)
}
return (
<div className="z-50 w-[280px] overflow-hidden rounded-md border bg-popover text-popover-foreground shadow-md">
<div className="overflow-y-auto max-h-[300px] p-1">
{Object.entries(grouped).map(([category, items]) => (
<div key={category}>
<div className="px-2 py-1.5 text-xs font-medium text-muted-foreground">{category}</div>
{items.map((item) => {
flatIndex++
const currentIndex = flatIndex
const Icon = iconMap[item.icon]
return (
<button
key={item.title}
className={`relative flex w-full cursor-pointer select-none items-center gap-3 rounded-sm px-2 py-1.5 text-sm outline-none transition-colors hover:bg-accent hover:text-accent-foreground ${
currentIndex === selectedIndex ? 'bg-accent text-accent-foreground' : ''
}`}
onClick={() => selectItem(currentIndex)}
type="button"
>
{Icon && (
<div className="flex h-8 w-8 shrink-0 items-center justify-center rounded-md border bg-background">
<Icon size={16} />
</div>
)}
<div className="flex flex-col text-left">
<span className="font-medium">{item.title}</span>
<span className="text-xs text-muted-foreground">{item.description}</span>
</div>
</button>
)
})}
</div>
))}
</div>
</div>
)
})
SlashCommandList.displayName = 'SlashCommandList'
export { SlashCommandList }
export default SlashCommandList

View File

@@ -0,0 +1,211 @@
import { Extension } from '@tiptap/core'
import { computePosition, flip, shift } from '@floating-ui/dom'
import { posToDOMRect, ReactRenderer } from '@tiptap/react'
import type { Editor } from '@tiptap/react'
import Suggestion from '@tiptap/suggestion'
import type { SuggestionOptions, SuggestionProps } from '@tiptap/suggestion'
import { SlashCommandList } from './slash-command-list'
import type { SlashCommandListRef } from './slash-command-list'
export interface SlashCommandItem {
title: string
description: string
category: string
icon: string
command: (editor: Editor) => void
}
const updatePosition = (editor: Editor, element: HTMLElement) => {
const virtualElement = {
getBoundingClientRect: () =>
posToDOMRect(editor.view, editor.state.selection.from, editor.state.selection.to)
}
computePosition(virtualElement, element, {
placement: 'bottom-start',
strategy: 'absolute',
middleware: [shift(), flip()]
}).then(({ x, y, strategy }) => {
element.style.width = 'max-content'
element.style.position = strategy
element.style.left = `${x}px`
element.style.top = `${y}px`
})
}
const slashCommandItems: SlashCommandItem[] = [
{
title: 'Heading 1',
description: 'Large section heading',
category: 'Hierarchy',
icon: 'Heading1',
command: (editor) => editor.chain().focus().setHeading({ level: 1 }).run()
},
{
title: 'Heading 2',
description: 'Medium section heading',
category: 'Hierarchy',
icon: 'Heading2',
command: (editor) => editor.chain().focus().setHeading({ level: 2 }).run()
},
{
title: 'Heading 3',
description: 'Small section heading',
category: 'Hierarchy',
icon: 'Heading3',
command: (editor) => editor.chain().focus().setHeading({ level: 3 }).run()
},
{
title: 'Bullet List',
description: 'Create a simple bullet list',
category: 'Lists',
icon: 'List',
command: (editor) => editor.chain().focus().toggleBulletList().run()
},
{
title: 'Numbered List',
description: 'Create a numbered list',
category: 'Lists',
icon: 'ListOrdered',
command: (editor) => editor.chain().focus().toggleOrderedList().run()
},
{
title: 'Task List',
description: 'Create a task checklist',
category: 'Lists',
icon: 'ListTodo',
command: (editor) => editor.chain().focus().toggleTaskList().run()
},
{
title: 'Blockquote',
description: 'Add a quote block',
category: 'Blocks',
icon: 'Quote',
command: (editor) => editor.chain().focus().toggleBlockquote().run()
},
{
title: 'Code Block',
description: 'Add a code snippet',
category: 'Blocks',
icon: 'Code',
command: (editor) => editor.chain().focus().toggleCodeBlock().run()
},
{
title: 'Divider',
description: 'Add a horizontal divider',
category: 'Blocks',
icon: 'Minus',
command: (editor) => editor.chain().focus().setHorizontalRule().run()
},
{
title: 'Image',
description: 'Upload or embed an image',
category: 'Media',
icon: 'ImageIcon',
command: (editor) => {
const input = document.createElement('input')
input.type = 'file'
input.accept = 'image/*'
input.onchange = () => {
const file = input.files?.[0]
if (file) {
const blobUrl = URL.createObjectURL(file)
editor.commands.insertContent({
type: 'image',
attrs: {
src: blobUrl,
alt: file.name,
title: file.name
}
})
}
}
input.click()
}
}
]
export const SlashCommand = Extension.create({
name: 'slashCommand',
addOptions() {
return {
suggestion: {
char: '/',
startOfLine: false,
items: ({ query }: { query: string }): SlashCommandItem[] => {
return slashCommandItems.filter(
(item) =>
item.title.toLowerCase().includes(query.toLowerCase()) ||
item.category.toLowerCase().includes(query.toLowerCase())
)
},
render: () => {
let component: ReactRenderer<SlashCommandListRef> | undefined
return {
onStart: (props: SuggestionProps) => {
component = new ReactRenderer(SlashCommandList, {
props,
editor: props.editor
})
if (!props.clientRect) return
const element = component.element as HTMLElement
element.style.position = 'absolute'
element.style.zIndex = '9999'
document.body.appendChild(element)
updatePosition(props.editor, element)
},
onUpdate(props: SuggestionProps) {
if (!component) return
component.updateProps(props)
if (!props.clientRect) return
const element = component.element as HTMLElement
updatePosition(props.editor, element)
},
onKeyDown(props: { event: KeyboardEvent }) {
if (props.event.key === 'Escape') {
component?.destroy()
return true
}
return component?.ref?.onKeyDown(props) ?? false
},
onExit() {
if (!component) return
component.element.remove()
component.destroy()
}
}
},
command: ({
editor,
range,
props
}: {
editor: Editor
range: { from: number; to: number }
props: SlashCommandItem
}) => {
editor.chain().focus().deleteRange(range).run()
props.command(editor)
}
} as Partial<SuggestionOptions>
}
},
addProseMirrorPlugins() {
return [
Suggestion({
editor: this.editor,
...this.options.suggestion
})
]
}
})
export default SlashCommand

View File

@@ -21,7 +21,7 @@ const ChatHistory = ({
const { confirm } = useConfirm()
const setCurrentChatId = useChatState((s) => s.setCurrentChatId)
const { data: chats, isLoading } = useQuery({
queryKey: ['chats'],
queryKey: ['chats', 'list'],
queryFn: () => chatCRUDService.get()
})

View File

@@ -0,0 +1,31 @@
import { Link } from '@tanstack/react-router'
import { KeyRound, X } from 'lucide-react'
import { Button } from '@/components/ui/button'
export default function ChatInactive({ onClose }: { onClose: any }) {
return (
<div className="flex flex-col h-full items-center justify-center relative py-16 px-4">
<Button
variant="ghost"
size="icon"
className="h-6 w-6 absolute top-3 right-3"
onClick={onClose}
>
<X className="h-4 w-4" />
</Button>
<div className="w-14 h-14 mb-5 text-muted-foreground/30">
<KeyRound className="w-full h-full" strokeWidth={1.5} />
</div>
<h3 className="text-sm font-medium text-foreground mb-1">API key required</h3>
<p className="text-sm text-muted-foreground text-center max-w-[260px] mb-5">
Add a <code className="text-xs bg-muted px-1.5! py-0.5! rounded">MISTRAL_API_KEY</code> in
your vault to start using the chat assistant
</p>
<Link to="/dashboard/vault">
<Button size="sm" className="h-8 text-xs gap-1.5">
Go to Vault
</Button>
</Link>
</div>
)
}

View File

@@ -14,6 +14,8 @@ import ChatHistory from './chat-history'
import { Tooltip, TooltipContent, TooltipProvider, TooltipTrigger } from '../ui/tooltip'
import { CopyButton } from '../copy'
import type { UIMessage } from 'ai'
import { useIsChatActive } from '@/hooks/use-is-chat-active'
import ChatInactive from './chat-inactive'
function FloatingChat() {
const bottomRef = useRef<HTMLDivElement>(null)
@@ -33,6 +35,8 @@ function FloatingChat() {
currentChatId
} = useChat()
const { exists: isChatActive } = useIsChatActive()
const isStreaming = status === 'streaming' || status === 'submitted'
const { isMac } = useKeyboardShortcut({
@@ -94,136 +98,147 @@ function FloatingChat() {
{isOpenChat && (
<div className="fixed bottom-12 overflow-hidden rounded-2xl right-4 shadow-xl z-50">
<ResizableChat minWidth={400} minHeight={400} maxWidth={800} maxHeight={800}>
<Card className="overflow-hidden backdrop-blur! bg-background/90 rounded-2xl gap-0 py-0 h-full">
{view === 'history' ? (
<ChatHistory
setView={setView}
deleteChatMutation={deleteChatMutation}
handleCreateNewChat={handleCreateNewChat}
/>
) : (
<>
<div className="flex items-center justify-between h-10 p-3 border-b">
<div className="flex w-full items-center justify-between gap-1">
<div className="flex items-center gap-2 truncate text-ellipsis">
<Button
variant="ghost"
size="icon"
className="h-6 w-6"
//@ts-ignore
onClick={handleCreateNewChat}
title="Create new chat"
>
<Plus className="h-3 w-3" />
</Button>
<span className="text-sm opacity-60 truncate text-ellipsis">
{currentChat?.title}
</span>
</div>
<div className="flex items-center gap-2">
<Button
variant="ghost"
size="icon"
className="h-6 w-6"
//@ts-ignore
onClick={() => setView('history')}
title="Create new chat"
>
<History className="h-3 w-3 opacity-60" />
</Button>
<Button variant="ghost" size="icon" className="h-6 w-6" onClick={closeChat}>
<X className="h-4 w-4" />
</Button>
{isChatActive ? (
<Card className="overflow-hidden backdrop-blur! bg-background/90 rounded-2xl gap-0 py-0 h-full">
{view === 'history' ? (
<ChatHistory
setView={setView}
deleteChatMutation={deleteChatMutation}
handleCreateNewChat={handleCreateNewChat}
/>
) : (
<>
<div className="flex items-center justify-between h-10 p-3 border-b">
<div className="flex w-full items-center justify-between gap-1">
<div className="flex items-center gap-2 truncate text-ellipsis">
<Button
variant="ghost"
size="icon"
className="h-6 w-6"
//@ts-ignore
onClick={handleCreateNewChat}
title="Create new chat"
>
<Plus className="h-3 w-3" />
</Button>
<span className="text-sm opacity-60 truncate text-ellipsis">
{currentChat?.title}
</span>
</div>
<div className="flex items-center gap-2">
<Button
variant="ghost"
size="icon"
className="h-6 w-6"
//@ts-ignore
onClick={() => setView('history')}
title="Create new chat"
>
<History className="h-3 w-3 opacity-60" />
</Button>
<Button
variant="ghost"
size="icon"
className="h-6 w-6"
onClick={closeChat}
>
<X className="h-4 w-4" />
</Button>
</div>
</div>
</div>
</div>
{/* Content */}
<div className="flex flex-col flex-1 overflow-auto">
{isLoadingChat ? (
<ChatSkeleton />
) : hasMessages ? (
<div className="grow p-4 flex flex-col gap-2">
{messages.map((message: UIMessage) => (
<ChatMessageComponent key={message.id} message={message} />
))}
<div ref={bottomRef} />
</div>
) : null}
{!hasMessages && !isLoadingChat && (
<div className="flex-1 flex flex-col items-center justify-center p-8 text-center text-muted-foreground">
{!currentChatId ? (
<>
<div className="relative mb-6">
<div className="absolute inset-0 bg-linear-to-r from-primary/20 to-primary/10 rounded-full blur-xl animate-pulse"></div>
<div className="relative bg-linear-to-br from-primary/10 to-primary/5 rounded-full p-4 border border-primary/20">
<img
src="/icon.png"
alt="Flowsint"
className="h-12 w-12 object-cover"
/>
{/* Content */}
<div className="flex flex-col flex-1 overflow-auto">
{isLoadingChat ? (
<ChatSkeleton />
) : hasMessages ? (
<div className="grow p-4 flex flex-col gap-2">
{messages.map((message: UIMessage) => (
<ChatMessageComponent key={message.id} message={message} />
))}
<div ref={bottomRef} />
</div>
) : null}
{!hasMessages && !isLoadingChat && (
<div className="flex-1 flex flex-col items-center justify-center p-8 text-center text-muted-foreground">
{!currentChatId ? (
<>
<div className="relative mb-6">
<div className="absolute inset-0 bg-linear-to-r from-primary/20 to-primary/10 rounded-full blur-xl animate-pulse"></div>
<div className="relative bg-linear-to-br from-primary/10 to-primary/5 rounded-full p-4 border border-primary/20">
<img
src="/icon.png"
alt="Flowsint"
className="h-12 w-12 object-cover"
/>
</div>
</div>
</div>
<div className="space-y-2 max-w-sm">
<h3 className="text-lg font-semibold text-foreground">
No conversations yet
</h3>
<p className="text-sm opacity-70">
Start chatting with AI to analyze your investigation data and get
insights. Your conversation history will appear here.
</p>
</div>
<div className="mt-6">
<Button
variant="outline"
size="sm"
onClick={handleCreateNewChat}
className="bg-linear-to-r from-primary/10 to-primary/5 border-primary/20 hover:from-primary/20 hover:to-primary/10"
>
Start your first chat
</Button>
</div>
</>
) : (
<>
<div className="space-y-2 max-w-sm">
<h3 className="text-lg font-semibold text-foreground">
Start your conversation with{' '}
<span className="text-primary">Flo</span>
</h3>
<p className="text-sm opacity-70">
Ask me anything about your investigation. Here are some examples:
</p>
</div>
<div className="mt-6 space-y-3 max-w-md">
<div className="text-xs space-y-1">
<p className="font-medium text-left">Analysis & Insights</p>
<ul className="space-y-1 text-left opacity-60">
<li> "Analyze the connections between these entities"</li>
<li> "What patterns do you see in this data?"</li>
<li> "Summarize the key findings from this investigation"</li>
</ul>
<div className="space-y-2 max-w-sm">
<h3 className="text-lg font-semibold text-foreground">
No conversations yet
</h3>
<p className="text-sm opacity-70">
Start chatting with AI to analyze your investigation data and get
insights. Your conversation history will appear here.
</p>
</div>
<div className="text-xs space-y-1">
<p className="font-medium text-left">Investigation Help</p>
<ul className="space-y-1 text-left opacity-60">
<li> "Suggest next steps for this investigation"</li>
<li> "What should I look for next?"</li>
<li> "Help me organize this investigation"</li>
</ul>
<div className="mt-6">
<Button
variant="outline"
size="sm"
onClick={handleCreateNewChat}
className="bg-linear-to-r from-primary/10 to-primary/5 border-primary/20 hover:from-primary/20 hover:to-primary/10"
>
Start your first chat
</Button>
</div>
</div>
</>
)}
</div>
)}
</div>
<div className="border-t">
<ChatPanel onSend={sendMessage} isLoading={isStreaming} />
</div>
</>
)}
</Card>
</>
) : (
<>
<div className="space-y-2 max-w-sm">
<h3 className="text-lg font-semibold text-foreground">
Start your conversation with{' '}
<span className="text-primary">Flo</span>
</h3>
<p className="text-sm opacity-70">
Ask me anything about your investigation. Here are some examples:
</p>
</div>
<div className="mt-6 space-y-3 max-w-md">
<div className="text-xs space-y-1">
<p className="font-medium text-left">Analysis & Insights</p>
<ul className="space-y-1 text-left opacity-60">
<li> "Analyze the connections between these entities"</li>
<li> "What patterns do you see in this data?"</li>
<li> "Summarize the key findings from this investigation"</li>
</ul>
</div>
<div className="text-xs space-y-1">
<p className="font-medium text-left">Investigation Help</p>
<ul className="space-y-1 text-left opacity-60">
<li> "Suggest next steps for this investigation"</li>
<li> "What should I look for next?"</li>
<li> "Help me organize this investigation"</li>
</ul>
</div>
</div>
</>
)}
</div>
)}
</div>
<div className="border-t">
<ChatPanel onSend={sendMessage} isLoading={isStreaming} />
</div>
</>
)}
</Card>
) : (
<Card className="overflow-hidden backdrop-blur! bg-background/90 rounded-2xl gap-0 py-0 h-full">
<ChatInactive onClose={closeChat} />
</Card>
)}
</ResizableChat>
</div>
)}

View File

@@ -1,6 +1,6 @@
import React, { useState, useCallback, useRef, useEffect } from 'react'
import { cn } from '@/lib/utils'
import React, { useCallback } from 'react'
import { useLayoutStore } from '@/stores/layout-store'
import { ResizableContainer } from '@/components/ui/resizable-container'
interface ResizableChatProps {
children: React.ReactNode
@@ -18,149 +18,27 @@ export const ResizableChat: React.FC<ResizableChatProps> = ({
maxHeight = 800
}) => {
const { chatWidth, chatHeight, setChatDimensions } = useLayoutStore()
const [isResizing, setIsResizing] = useState(false)
const [resizeDirection, setResizeDirection] = useState<'se' | 'sw' | 'ne' | 'nw' | null>(null)
const [startPos, setStartPos] = useState({ x: 0, y: 0 })
const [startDimensions, setStartDimensions] = useState({ width: 0, height: 0 })
const containerRef = useRef<HTMLDivElement>(null)
const handleMouseDown = useCallback(
(direction: 'se' | 'sw' | 'ne' | 'nw') => (e: React.MouseEvent) => {
e.preventDefault()
e.stopPropagation()
setIsResizing(true)
setResizeDirection(direction)
setStartPos({ x: e.clientX, y: e.clientY })
setStartDimensions({ width: chatWidth, height: chatHeight })
document.body.style.cursor = direction.includes('e') ? 'e-resize' : 'w-resize'
document.body.style.userSelect = 'none'
const handleResize = useCallback(
(width: number, height: number) => {
setChatDimensions(width, height)
},
[chatWidth, chatHeight]
[setChatDimensions]
)
const handleMouseMove = useCallback(
(e: MouseEvent): void => {
if (!isResizing || !resizeDirection) return
const deltaX = e.clientX - startPos.x
const deltaY = e.clientY - startPos.y
let newWidth = startDimensions.width
let newHeight = startDimensions.height
// Calculate new dimensions based on resize direction
if (resizeDirection.includes('e')) {
newWidth = Math.max(minWidth, Math.min(maxWidth, startDimensions.width + deltaX))
} else if (resizeDirection.includes('w')) {
newWidth = Math.max(minWidth, Math.min(maxWidth, startDimensions.width - deltaX))
}
if (resizeDirection.includes('s')) {
newHeight = Math.max(minHeight, Math.min(maxHeight, startDimensions.height + deltaY))
} else if (resizeDirection.includes('n')) {
newHeight = Math.max(minHeight, Math.min(maxHeight, startDimensions.height - deltaY))
}
setChatDimensions(newWidth, newHeight)
},
[
isResizing,
resizeDirection,
startPos,
startDimensions,
minWidth,
minHeight,
maxWidth,
maxHeight,
setChatDimensions
]
)
const handleMouseUp = useCallback(() => {
setIsResizing(false)
setResizeDirection(null)
document.body.style.cursor = ''
document.body.style.userSelect = ''
}, [])
useEffect(() => {
if (isResizing) {
document.addEventListener('mousemove', handleMouseMove)
document.addEventListener('mouseup', handleMouseUp)
return () => {
document.removeEventListener('mousemove', handleMouseMove)
document.removeEventListener('mouseup', handleMouseUp)
}
}
return undefined
}, [isResizing, handleMouseMove, handleMouseUp])
return (
<div
ref={containerRef}
className={cn(
'relative group/resize overflow-hidden max-h-[calc(100vh-60px)]',
isResizing && 'pointer-events-none'
)}
style={{
width: chatWidth,
height: chatHeight
}}
<ResizableContainer
defaultWidth={chatWidth}
defaultHeight={chatHeight}
onResize={handleResize}
minWidth={minWidth}
minHeight={minHeight}
maxWidth={maxWidth}
maxHeight={maxHeight}
storageKey="resizable-chat"
className="max-h-[calc(100vh-60px)]"
>
{children}
{/* Resize overlay when active */}
{isResizing && (
<div className="absolute inset-0 bg-primary/5 border-2 border-primary/30 rounded-2xl pointer-events-none" />
)}
{/* Resize handles */}
<div
className={cn(
'absolute bottom-0 right-0 w-4 h-4 cursor-se-resize',
'bg-transparent hover:bg-primary/20 transition-colors',
'rounded-tl-md border-primary/20',
'group-hover/resize:border-primary/40',
isResizing && 'bg-primary/30 border-primary/50'
)}
onMouseDown={handleMouseDown('se')}
/>
<div
className={cn(
'absolute bottom-0 left-0 w-4 h-4 cursor-sw-resize',
'bg-transparent hover:bg-primary/20 transition-colors',
'rounded-tr-md border-primary/20',
'group-hover/resize:border-primary/40',
isResizing && 'bg-primary/30 border-primary/50'
)}
onMouseDown={handleMouseDown('sw')}
/>
<div
className={cn(
'absolute top-0 right-0 w-4 h-4 cursor-ne-resize',
'bg-transparent hover:bg-primary/20 transition-colors',
'rounded-bl-md border-primary/20',
'group-hover/resize:border-primary/40',
isResizing && 'bg-primary/30 border-primary/50'
)}
onMouseDown={handleMouseDown('ne')}
/>
<div
className={cn(
'absolute top-0 left-0 w-4 h-4 cursor-nw-resize',
'bg-transparent hover:bg-primary/20 transition-colors',
'rounded-br-md border-primary/20',
'group-hover/resize:border-primary/40',
isResizing && 'bg-primary/30 border-primary/50'
)}
onMouseDown={handleMouseDown('nw')}
/>
</div>
</ResizableContainer>
)
}

View File

@@ -0,0 +1,186 @@
import { Input } from '@/components/ui/input'
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue
} from '@/components/ui/select'
import { Button } from '@/components/ui/button'
import { GripVertical, Trash2, ChevronRight, Asterisk } from 'lucide-react'
import { cn } from '@/lib/utils'
import { Collapsible, CollapsibleContent, CollapsibleTrigger } from '@/components/ui/collapsible'
import { Textarea } from '@/components/ui/textarea'
import { Switch } from '@/components/ui/switch'
import { useState } from 'react'
import { Reorder, useDragControls } from 'framer-motion'
export interface SchemaField {
id: string
key: string
title: string
type: string
format?: string
description?: string
required: boolean
}
const FIELD_TYPES = [
{ value: 'string', label: 'Text' },
{ value: 'number', label: 'Number' },
{ value: 'integer', label: 'Integer' },
{ value: 'boolean', label: 'Boolean' },
{ value: 'array', label: 'List' },
{ value: 'object', label: 'Object' }
] as const
const FIELD_FORMATS = [
{ value: 'none', label: 'None' },
{ value: 'email', label: 'Email' },
{ value: 'uri', label: 'URL' },
{ value: 'date', label: 'Date' },
{ value: 'date-time', label: 'Date & Time' },
{ value: 'ipv4', label: 'IPv4' },
{ value: 'ipv6', label: 'IPv6' }
] as const
interface FieldRowProps {
field: SchemaField
onUpdate: (updates: Partial<SchemaField>) => void
onDelete: () => void
}
export function FieldRow({ field, onUpdate, onDelete }: FieldRowProps) {
const [expanded, setExpanded] = useState(false)
const dragControls = useDragControls()
return (
<Reorder.Item
value={field}
dragListener={false}
dragControls={dragControls}
className="list-none"
// whileDrag={{ scsale: 1.01, boxShadow: '0 4px 16px rgba(0,0,0,0.08)', zIndex: 50 }}
transition={{ duration: 0.15 }}
>
<Collapsible open={expanded} onOpenChange={setExpanded}>
<div
className={cn(
'group rounded-lg border bg-background relative',
field.required && 'border-primary/20 bg-primary/3'
)}
>
{field.required && (
<div className="absolute -top-2 -left-2 rounded-full border border-primary/50 bg-card">
<Asterisk className="h-4 w-4 text-primary" />
</div>
)}
<div className="flex items-center gap-2 px-2 py-1.5">
<div className="flex items-center gap-1 shrink-0">
<button
className="touch-none p-0.5 rounded hover:bg-muted transition-colors cursor-grab active:cursor-grabbing"
onPointerDown={(e) => dragControls.start(e)}
>
<GripVertical className="h-3.5 w-3.5 text-muted-foreground/40 group-hover:text-muted-foreground/70 transition-colors" />
</button>
<CollapsibleTrigger asChild>
<button className="p-1 rounded hover:bg-muted transition-colors">
<ChevronRight
className={cn(
'h-3.5 w-3.5 text-muted-foreground transition-transform',
expanded && 'rotate-90'
)}
/>
</button>
</CollapsibleTrigger>
</div>
<div className="flex-1 grid grid-cols-[1fr_1fr_120px_120px] gap-2 items-center">
<Input
value={field.key}
onChange={(e) =>
onUpdate({ key: e.target.value.toLowerCase().replace(/\s+/g, '_') })
}
placeholder="field_key"
className="h-8 text-sm border-transparent bg-transparent hover:border-border focus:border-border focus:bg-background transition-colors"
/>
<Input
value={field.title}
onChange={(e) => onUpdate({ title: e.target.value })}
placeholder="Display name"
className="h-8 text-sm border-transparent bg-transparent hover:border-border focus:border-border focus:bg-background transition-colors"
/>
<Select value={field.type} onValueChange={(v: string) => onUpdate({ type: v })}>
<SelectTrigger className="h-8 text-sm border-transparent bg-transparent hover:border-border transition-colors">
<SelectValue />
</SelectTrigger>
<SelectContent>
{FIELD_TYPES.map((t) => (
<SelectItem key={t.value} value={t.value}>
{t.label}
</SelectItem>
))}
</SelectContent>
</Select>
<Select
value={field.format || 'none'}
onValueChange={(v: string) => onUpdate({ format: v === 'none' ? undefined : v })}
>
<SelectTrigger className="h-8 text-sm border-transparent bg-transparent hover:border-border transition-colors">
<SelectValue />
</SelectTrigger>
<SelectContent>
{FIELD_FORMATS.map((f) => (
<SelectItem key={f.value} value={f.value}>
{f.label}
</SelectItem>
))}
</SelectContent>
</Select>
</div>
<div className="flex items-center gap-1 shrink-0">
<Button
variant="ghost"
size="icon"
className="h-7 w-7 opacity-0 group-hover:opacity-100 transition-opacity text-muted-foreground hover:text-destructive"
onClick={onDelete}
>
<Trash2 className="h-3.5 w-3.5" />
</Button>
</div>
</div>
{/* Expanded details */}
<CollapsibleContent>
<div className="px-10 pb-3 space-y-3">
<div className="grid grid-cols-2 gap-4">
<div className="space-y-1.5">
<label className="text-xs font-medium text-muted-foreground">Description</label>
<Textarea
value={field.description || ''}
onChange={(e) => onUpdate({ description: e.target.value })}
placeholder="Describe what this field is for..."
rows={2}
className="text-sm resize-none"
/>
</div>
<div className="space-y-3">
<div className="flex items-center justify-between">
<label className="text-xs font-medium text-muted-foreground">
Required field
</label>
<Switch
checked={field.required}
onCheckedChange={(checked: boolean) => onUpdate({ required: checked })}
/>
</div>
</div>
</div>
</div>
</CollapsibleContent>
</div>
</Collapsible>
</Reorder.Item>
)
}

View File

@@ -0,0 +1,50 @@
import { useState } from 'react'
import * as LucideIcons from 'lucide-react'
import { cn } from '@/lib/utils'
import IconPicker from '@/components/shared/icon-picker/popup'
interface IconPickerTriggerProps {
icon: string
color: string
onIconChange: (icon: string) => void
}
export function IconPickerTrigger({ icon, color, onIconChange }: IconPickerTriggerProps) {
const [open, setOpen] = useState(false)
const Icon = (LucideIcons as any)[icon] || LucideIcons.FileQuestion
return (
<>
<button
onClick={() => setOpen(true)}
className={cn(
'group relative flex items-center justify-center rounded-full transition-all',
'focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring'
)}
style={{
backgroundColor: color,
width: 52,
height: 52
}}
>
<Icon className="text-white" size={26} />
<div
className={cn(
'absolute inset-0 rounded-xl bg-black/0 transition-colors',
'group-hover:bg-black/10'
)}
/>
</button>
<IconPicker
iconType={null}
open={open}
setOpen={setOpen}
onIconChange={(_type, iconName) => {
onIconChange(iconName as string)
}}
/>
</>
)
}

View File

@@ -0,0 +1,111 @@
import * as LucideIcons from 'lucide-react'
import { Badge } from '@/components/ui/badge'
import type { SchemaField } from './field-row'
interface TypePreviewProps {
name: string
description: string
icon: string
color: string
fields: SchemaField[]
status: 'draft' | 'published'
}
export function TypePreview({ name, description, icon, color, fields, status }: TypePreviewProps) {
const Icon = (LucideIcons as any)[icon] || LucideIcons.FileQuestion
const validFields = fields.filter((f) => f.key.trim())
return (
<div className="space-y-6">
{/* Entity card preview */}
<div>
<p className="text-xs font-medium text-muted-foreground mb-3">Entity card</p>
<div className="border border-border/60 rounded-xl p-5 bg-card max-w-sm">
<div className="flex items-start gap-3.5">
<div
className="rounded-full flex items-center justify-center shrink-0"
style={{ backgroundColor: color, width: 44, height: 44 }}
>
<Icon className="text-white" size={22} />
</div>
<div className="min-w-0 flex-1">
<div className="flex items-center gap-2">
<h4 className="font-semibold text-sm truncate">{name || 'Untitled type'}</h4>
</div>
<p className="text-xs text-muted-foreground mt-0.5 line-clamp-2">
{description || 'No description'}
</p>
</div>
</div>
{validFields.length > 0 && (
<div className="mt-4 pt-3 border-t border-border/40 space-y-2">
{validFields.slice(0, 4).map((field) => (
<div key={field.id} className="flex items-center justify-between">
<span className="text-xs text-muted-foreground">{field.title || field.key}</span>
<span className="text-xs text-muted-foreground/60 font-mono">
{field.type}
{field.format && field.format !== 'none' ? `:${field.format}` : ''}
</span>
</div>
))}
{validFields.length > 4 && (
<p className="text-[10px] text-muted-foreground/50">
+{validFields.length - 4} more fields
</p>
)}
</div>
)}
</div>
</div>
{/* Graph node preview */}
<div>
<p className="text-xs font-medium text-muted-foreground mb-3">Graph node</p>
<div className="flex items-center gap-3 p-4 border border-border/60 rounded-xl bg-card max-w-[200px]">
<div
className="rounded-full flex items-center justify-center shrink-0"
style={{ backgroundColor: color, width: 36, height: 36 }}
>
<Icon className="text-white" size={18} />
</div>
<div className="min-w-0">
<p className="text-xs font-medium truncate">{name || 'Untitled'}</p>
<p className="text-[10px] text-muted-foreground">
{validFields.length} field{validFields.length !== 1 ? 's' : ''}
</p>
</div>
</div>
</div>
{/* Schema preview */}
<div>
<p className="text-xs font-medium text-muted-foreground mb-3">JSON Schema</p>
<pre className="p-4 bg-muted/50 rounded-lg text-xs font-mono overflow-x-auto text-muted-foreground leading-relaxed">
{JSON.stringify(buildSchema(name, fields), null, 2)}
</pre>
</div>
</div>
)
}
function buildSchema(name: string, fields: SchemaField[]) {
const properties: Record<string, any> = {}
const required: string[] = []
fields.forEach((field) => {
if (!field.key.trim()) return
const prop: any = { type: field.type, title: field.title || field.key }
if (field.description) prop.description = field.description
if (field.format && field.format !== 'none') prop.format = field.format
properties[field.key] = prop
if (field.required) required.push(field.key)
})
return {
title: name || 'MyCustomType',
type: 'object',
properties,
required
}
}

View File

@@ -5,6 +5,7 @@ import NewAnalysis from "../analyses/new-analysis"
interface EmptyStateProps {
onAction?: () => void
canCreate?: boolean
}
export function EmptyInvestigations({ onAction }: EmptyStateProps) {
@@ -70,7 +71,7 @@ export function EmptySketches({ onAction }: EmptyStateProps) {
)
}
export function EmptyAnalyses({ onAction }: EmptyStateProps) {
export function EmptyAnalyses({ onAction, canCreate = true }: EmptyStateProps) {
return (
<div className="flex flex-col items-center justify-center py-10 px-4">
{/* Minimal document illustration */}
@@ -93,12 +94,14 @@ export function EmptyAnalyses({ onAction }: EmptyStateProps) {
<p className="text-xs text-muted-foreground text-center max-w-[200px] mb-4">
Document findings, patterns, and conclusions from your investigation
</p>
<NewAnalysis>
<Button onClick={onAction} variant="ghost" size="sm" className="h-7 text-xs gap-1.5">
<Plus className="w-3.5 h-3.5" />
Write analysis
</Button>
</NewAnalysis>
{canCreate && (
<NewAnalysis>
<Button onClick={onAction} variant="ghost" size="sm" className="h-7 text-xs gap-1.5">
<Plus className="w-3.5 h-3.5" />
Write analysis
</Button>
</NewAnalysis>
)}
</div>
)
}

View File

@@ -8,24 +8,27 @@ import { Link } from "@tanstack/react-router"
interface AnalysesSectionProps {
analyses: Analysis[]
canCreate?: boolean
}
export function AnalysesSection({ analyses }: AnalysesSectionProps) {
export function AnalysesSection({ analyses, canCreate = true }: AnalysesSectionProps) {
return (
<section>
<div className="flex items-center justify-between mb-4">
<h2 className="text-sm font-medium text-foreground">Analyses</h2>
<NewAnalysis>
<Button variant="ghost" size="sm" className="h-7 text-xs text-muted-foreground hover:text-foreground gap-1">
<Plus className="w-3.5 h-3.5" />
New
</Button>
</NewAnalysis>
{canCreate && (
<NewAnalysis>
<Button variant="ghost" size="sm" className="h-7 text-xs text-muted-foreground hover:text-foreground gap-1">
<Plus className="w-3.5 h-3.5" />
New
</Button>
</NewAnalysis>
)}
</div>
{analyses.length === 0 ?
<div className="border border-dashed rounded-md">
<EmptyAnalyses />
<EmptyAnalyses canCreate={canCreate} />
</div> :
<div className="space-y-1">
{analyses.map((analysis) => (

View File

@@ -1,23 +1,34 @@
import type React from "react"
import { Button } from "@/components/ui/button"
import type React from 'react'
import { Badge } from '@/components/ui/badge'
import { Button } from '@/components/ui/button'
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuTrigger,
} from "@/components/ui/dropdown-menu"
import { MoreHorizontal, Star, Share, Trash2, Clock } from "lucide-react"
import { Investigation } from "@/types"
import { formatDistanceToNow } from "date-fns"
import { useState } from "react"
import { cn } from "@/lib/utils"
import { Avatar, AvatarFallback, AvatarImage } from "@/components/ui/avatar"
import { useQueryClient } from "@tanstack/react-query"
import { investigationService } from "@/api/investigation-service"
import { useConfirm } from "@/components/use-confirm-dialog"
import { queryKeys } from "@/api/query-keys"
import { toast } from "sonner"
import { Link, useRouter } from "@tanstack/react-router"
DropdownMenuTrigger
} from '@/components/ui/dropdown-menu'
import { MoreHorizontal, Star, Share, Trash2, Clock, Crown, Shield, Pencil, Eye } from 'lucide-react'
import { Investigation, Collaborator } from '@/types'
import { formatDistanceToNow } from 'date-fns'
import { useState } from 'react'
import { cn } from '@/lib/utils'
import { AvatarGroup } from '@/components/ui/avatar'
import { useQuery, useQueryClient } from '@tanstack/react-query'
import { investigationService } from '@/api/investigation-service'
import { useConfirm } from '@/components/use-confirm-dialog'
import { queryKeys } from '@/api/query-keys'
import { toast } from 'sonner'
import { Link, useRouter } from '@tanstack/react-router'
import { usePermissions } from '@/hooks/use-can'
import type { InvestigationRole } from '@/types'
import { ShareDialog } from './share-dialog'
const ROLE_CONFIG: Record<InvestigationRole, { label: string; icon: typeof Eye; className: string }> = {
owner: { label: 'Owner', icon: Crown, className: 'bg-amber-500/15 text-amber-700 border-amber-500/30 dark:text-amber-400' },
admin: { label: 'Admin', icon: Shield, className: 'bg-purple-500/15 text-purple-700 border-purple-500/30 dark:text-purple-400' },
editor: { label: 'Editor', icon: Pencil, className: 'bg-blue-500/15 text-blue-700 border-blue-500/30 dark:text-blue-400' },
viewer: { label: 'Viewer', icon: Eye, className: 'bg-zinc-500/15 text-zinc-700 border-zinc-500/30 dark:text-zinc-400' },
}
type CaseOverviewPageProps = {
investigation: Investigation
@@ -28,10 +39,16 @@ export function CaseHeader({ investigation }: CaseOverviewPageProps) {
addSuffix: true
})
const router = useRouter()
const { canDelete, canManage, role } = usePermissions()
const { confirm } = useConfirm()
const queryClient = useQueryClient()
const { data: collaborators = [] } = useQuery<Collaborator[]>({
queryKey: queryKeys.investigations.collaborators(investigation.id),
queryFn: () => investigationService.getCollaborators(investigation.id)
})
const handleDeleteInvestigation = async () => {
const confirmed = await confirm({
title: 'Delete Investigation',
@@ -41,13 +58,9 @@ export function CaseHeader({ investigation }: CaseOverviewPageProps) {
if (confirmed) {
const deletePromise = () =>
investigationService.delete(investigation.id).then(() => {
''
// Invalidate the investigations list
queryClient.invalidateQueries({
queryKey: queryKeys.investigations.list
})
// Also remove related data from cache
queryClient.removeQueries({
queryKey: queryKeys.investigations.detail(investigation.id)
})
@@ -60,7 +73,7 @@ export function CaseHeader({ investigation }: CaseOverviewPageProps) {
queryClient.removeQueries({
queryKey: queryKeys.investigations.flows(investigation.id)
})
router.navigate({ to: "/dashboard" })
router.navigate({ to: '/dashboard' })
})
toast.promise(deletePromise, {
@@ -73,9 +86,11 @@ export function CaseHeader({ investigation }: CaseOverviewPageProps) {
return (
<div className="space-y-6 pb-6 border-b border-border">
{/* Breadcrumb - subtle */}
{/* Breadcrumb */}
<div className="flex items-center gap-1.5 text-sm text-muted-foreground">
<Link to={"/dashboard"} className="hover:text-foreground cursor-pointer transition-colors">Cases</Link>
<Link to={'/dashboard'} className="hover:text-foreground cursor-pointer transition-colors">
Cases
</Link>
<span>/</span>
<span className="text-foreground">{investigation.name}</span>
</div>
@@ -84,63 +99,76 @@ export function CaseHeader({ investigation }: CaseOverviewPageProps) {
<div className="flex items-start justify-between gap-4">
<div className="space-y-2 flex-1">
<div className="flex items-center gap-3">
<h1 className="text-2xl font-semibold text-foreground tracking-tight">{investigation.name}</h1>
<h1 className="text-2xl font-semibold text-foreground tracking-tight">
{investigation.name}
</h1>
<FavoriteButton />
</div>
{/* Inline properties - Notion style */}
{/* Role badge */}
{role && (() => {
const config = ROLE_CONFIG[role]
const Icon = config.icon
return (
<Badge className={cn('gap-1 text-xs font-medium shadow-none', config.className)}>
<Icon className="w-3 h-3" />
{config.label}
</Badge>
)
})()}
{/* Inline properties */}
<div className="flex items-center gap-4 text-sm">
<PropertyPill label="Status" value={investigation.status} valueClass="text-success" />
<PropertyPill label="Priority" value="Medium" valueClass="text-primary" />
<PropertyPill label="Updated" value={lastUpdated} icon={<Clock className="w-3 h-3" />} />
<PropertyPill
label="Updated"
value={lastUpdated}
icon={<Clock className="w-3 h-3" />}
/>
</div>
</div>
{/* Actions - minimal */}
{/* Actions */}
<div className="flex items-center gap-1">
{/* <Button variant="ghost" size="sm" className="text-muted-foreground h-8 px-2">
<Share className="w-4 h-4" />
</Button> */}
<DropdownMenu>
<DropdownMenuTrigger asChild>
{canManage && (
<ShareDialog investigationId={investigation.id}>
<Button variant="ghost" size="sm" className="text-muted-foreground h-8 px-2">
<MoreHorizontal className="w-4 h-4" />
<Share className="w-4 h-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end" className="w-48">
{/* <DropdownMenuItem>Export as PDF</DropdownMenuItem>
<DropdownMenuItem>Duplicate</DropdownMenuItem> */}
{/* <DropdownMenuSeparator /> */}
<DropdownMenuItem onClick={handleDeleteInvestigation} className="text-destructive">
<Trash2 className="w-4 h-4 mr-2" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</ShareDialog>
)}
{canDelete && (
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button variant="ghost" size="sm" className="text-muted-foreground h-8 px-2">
<MoreHorizontal className="w-4 h-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end" className="w-48">
<DropdownMenuItem onClick={handleDeleteInvestigation} className="text-destructive">
<Trash2 className="w-4 h-4 mr-2" />
Delete
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
)}
</div>
</div>
{/* Tags row */}
{/* <div className="flex items-center gap-2 flex-wrap">
<Tag>APT</Tag>
<Tag>Nation-State</Tag>
<Tag>Supply Chain</Tag>
<Tag>SolarWinds</Tag>
<button className="text-xs text-muted-foreground hover:text-foreground transition-colors">+ Add tag</button>
</div> */}
{/* Team */}
<div className="flex items-center gap-2">
<div className="flex -space-x-1.5">
<Avatar className="h-5 w-5">
<AvatarImage src="https://cherry.img.pmdstatic.net/fit/https.3A.2F.2Fimg.2Egamesider.2Ecom.2Fs3.2Ffrgsg.2F1280.2Fthe-last-of-us.2Fdefault_2023-11-27_291826c8-5b2b-4928-a167-259dd0b18a7c.2Ejpeg/1200x675/quality/80/the-last-of-us-saison-2-mauvaise-nouvelle-pedro-pascal.jpg" />
<AvatarFallback>U</AvatarFallback>
</Avatar>
{/* <Avatar initials="SK" />
<Avatar initials="MR" /> */}
</div>
<span className="text-sm text-muted-foreground">1 investigator</span>
{/* <button disabled className="text-xs text-muted-foreground hover:text-foreground transition-colors ml-1">+ Invite</button> */}
<AvatarGroup users={collaborators.map((c) => c.user)} size="md" />
<span className="text-sm text-muted-foreground">
{collaborators.length} investigator{collaborators.length !== 1 ? 's' : ''}
</span>
{canManage && (
<ShareDialog investigationId={investigation.id}>
<button className="text-xs text-muted-foreground hover:text-foreground transition-colors ml-1">
+ Invite
</button>
</ShareDialog>
)}
</div>
</div>
)
@@ -149,8 +177,8 @@ export function CaseHeader({ investigation }: CaseOverviewPageProps) {
function PropertyPill({
label,
value,
valueClass = "text-foreground",
icon,
valueClass = 'text-foreground',
icon
}: {
label: string
value: string
@@ -172,12 +200,14 @@ function PropertyPill({
// return <span className="px-2 py-0.5 rounded bg-secondary text-xs text-secondary-foreground">{children}</span>
// }
const FavoriteButton = () => {
const [fav, setFav] = useState<boolean>(false)
return (
<button onClick={() => setFav(!fav)} className={cn("text-muted-foreground hover:text-warning transition-colors")}>
<Star className={cn("w-4 h-4", fav && "text-warning fill-warning")} />
<button
onClick={() => setFav(!fav)}
className={cn('text-muted-foreground hover:text-warning transition-colors')}
>
<Star className={cn('w-4 h-4', fav && 'text-warning fill-warning')} />
</button>
)
}
}

View File

@@ -1,36 +1,24 @@
import { CaseHeader } from "./case-header"
// import { MetricsGrid } from "./metrics-grid"
// import { ActivityTimeline } from "./activity-timeline"
import { SketchesSection } from "./sketches-section"
import { AnalysesSection } from "./analyses-section"
// import { EvidenceSection } from "./evidence-section"
// import { TasksSection } from "./tasks-section"
import { Investigation } from "@/types"
import { usePermissions } from "@/hooks/use-can"
type CaseOverviewPageProps = {
investigation: Investigation
}
export function CaseOverviewPage({ investigation }: CaseOverviewPageProps) {
const { canCreate } = usePermissions()
return (
<main className="flex-1 h-full overflow-auto">
<div className="max-w-7xl mx-auto px-8 py-8">
<CaseHeader investigation={investigation} />
{/* <MetricsGrid investigation={investigation} /> */}
<div className="space-y-8">
<SketchesSection sketches={investigation.sketches} />
<AnalysesSection analyses={investigation.analyses} />
{/* Two column for smaller sections */}
{/* <div className="grid grid-cols-1 lg:grid-cols-2 gap-8">
<EvidenceSection />
<div className="space-y-8">
<TasksSection />
<ActivityTimeline />
</div>
</div> */}
<SketchesSection sketches={investigation.sketches} canCreate={canCreate} />
<AnalysesSection analyses={investigation.analyses} canCreate={canCreate} />
</div>
</div>
</main>

View File

@@ -126,7 +126,8 @@ export function InvestigationsList({
</table>
</div>
) : (
<div className="grid grid-cols-3 gap-3">
<div style={{ containerType: 'inline-size' }}>
<div className="grid grid-cols-1 cq-sm:grid-cols-2 cq-md:grid-cols-3 gap-3">
{filteredInvestigations.length === 0 && <div>No investigation found.</div>}
{filteredInvestigations.map((inv) => (
<Link
@@ -168,6 +169,7 @@ export function InvestigationsList({
</Link>
))}
</div>
</div>
)}
</div>
)

View File

@@ -1,7 +1,6 @@
import { Network, FileText, Users, Shield, Paperclip, CheckCircle } from "lucide-react"
import { Network, FileText, Users, Shield } from "lucide-react"
import { Investigation } from "@/types"
import { useMemo } from "react"
import { cn } from "@/lib/utils"
type CaseOverviewPageProps = {
investigation: Investigation
@@ -19,7 +18,8 @@ export function MetricsGrid({ investigation }: CaseOverviewPageProps) {
], [sketchCount, analysisCount])
return (
<div className={cn("grid border grid-cols-3 gap-px bg-border rounded-lg overflow-hidden my-6", `md:grid-cols-${metrics.length}`)} >
<div className="my-6" style={{ containerType: 'inline-size' }}>
<div className="grid border grid-cols-2 cq-sm:grid-cols-4 gap-px bg-border rounded-lg overflow-hidden">
{
metrics.map((metric) => (
<div
@@ -32,6 +32,7 @@ export function MetricsGrid({ investigation }: CaseOverviewPageProps) {
</div>
))
}
</ div>
</div>
</div>
)
}

View File

@@ -0,0 +1,405 @@
import { useState, useCallback, useRef, useEffect } from 'react'
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query'
import { investigationService } from '@/api/investigation-service'
import { authService } from '@/api/auth-service'
import { queryKeys } from '@/api/query-keys'
import { toast } from 'sonner'
import type { Collaborator, InvestigationRole, Profile } from '@/types'
import {
Dialog,
DialogContent,
DialogDescription,
DialogHeader,
DialogTitle,
DialogTrigger
} from '@/components/ui/dialog'
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue
} from '@/components/ui/select'
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuTrigger
} from '@/components/ui/dropdown-menu'
import { Button } from '@/components/ui/button'
import { Input } from '@/components/ui/input'
import { UserAvatar } from '@/components/ui/avatar'
import { Badge } from '@/components/ui/badge'
import { Separator } from '@/components/ui/separator'
import { Skeleton } from '@/components/ui/skeleton'
import { X, UserPlus, Crown, Shield, Pencil, Eye, Users, Check } from 'lucide-react'
import { getDisplayName } from '@/lib/user-display'
import { cn } from '@/lib/utils'
const ROLE_OPTIONS: {
value: InvestigationRole
label: string
icon: typeof Eye
className: string
}[] = [
{
value: 'admin',
label: 'Admin',
icon: Shield,
className: 'bg-purple-500/15 text-purple-700 border-purple-500/30 dark:text-purple-400'
},
{
value: 'editor',
label: 'Editor',
icon: Pencil,
className: 'bg-blue-500/15 text-blue-700 border-blue-500/30 dark:text-blue-400'
},
{
value: 'viewer',
label: 'Viewer',
icon: Eye,
className: 'bg-zinc-500/15 text-zinc-700 border-zinc-500/30 dark:text-zinc-400'
}
]
const ROLE_BADGE: Record<
InvestigationRole,
{ label: string; icon: typeof Eye; className: string }
> = {
owner: {
label: 'Owner',
icon: Crown,
className: 'bg-amber-500/15 text-amber-700 border-amber-500/30 dark:text-amber-400'
},
admin: {
label: 'Admin',
icon: Shield,
className: 'bg-purple-500/15 text-purple-700 border-purple-500/30 dark:text-purple-400'
},
editor: {
label: 'Editor',
icon: Pencil,
className: 'bg-blue-500/15 text-blue-700 border-blue-500/30 dark:text-blue-400'
},
viewer: {
label: 'Viewer',
icon: Eye,
className: 'bg-zinc-500/15 text-zinc-700 border-zinc-500/30 dark:text-zinc-400'
}
}
function getRoleFromCollaborator(collab: Collaborator): InvestigationRole {
return (collab.roles[0] ?? 'viewer') as InvestigationRole
}
const RoleBadge = ({
role,
className,
...props
}: { role: InvestigationRole; className?: string } & React.ComponentProps<'div'>) => {
const config = ROLE_BADGE[role]
const Icon = config.icon
return (
<Badge
variant="outline"
className={cn(
'gap-1 text-[11px] font-medium shadow-none px-1.5 py-0 cursor-default',
config.className,
className
)}
{...props}
>
<Icon className="w-3 h-3" />
{config.label}
</Badge>
)
}
interface ShareDialogProps {
investigationId: string
children: React.ReactNode
}
export function ShareDialog({ investigationId, children }: ShareDialogProps) {
const [open, setOpen] = useState(false)
const [query, setQuery] = useState('')
const [selectedEmail, setSelectedEmail] = useState('')
const [role, setRole] = useState<string>('editor')
const [suggestions, setSuggestions] = useState<Profile[]>([])
const [showSuggestions, setShowSuggestions] = useState(false)
const debounceRef = useRef<NodeJS.Timeout | null>(null)
const queryClient = useQueryClient()
const { data: collaborators = [], isLoading } = useQuery<Collaborator[]>({
queryKey: queryKeys.investigations.collaborators(investigationId),
queryFn: () => investigationService.getCollaborators(investigationId),
enabled: open
})
const handleQueryChange = useCallback((value: string) => {
setQuery(value)
setSelectedEmail(value)
if (debounceRef.current) clearTimeout(debounceRef.current)
if (value.length < 2) {
setSuggestions([])
setShowSuggestions(false)
return
}
debounceRef.current = setTimeout(async () => {
try {
const results = await authService.searchUsers(value)
setSuggestions(results)
setShowSuggestions(results.length > 0)
} catch {
setSuggestions([])
setShowSuggestions(false)
}
}, 300)
}, [])
useEffect(() => {
return () => {
if (debounceRef.current) clearTimeout(debounceRef.current)
}
}, [])
const handleSelectUser = (user: Profile) => {
setSelectedEmail(user.email ?? '')
setQuery(getDisplayName(user))
setShowSuggestions(false)
setSuggestions([])
}
const addMutation = useMutation({
mutationFn: (body: { email: string; role: string }) =>
investigationService.addCollaborator(investigationId, body),
onSuccess: () => {
queryClient.invalidateQueries({
queryKey: queryKeys.investigations.collaborators(investigationId)
})
setQuery('')
setSelectedEmail('')
toast.success('Collaborator added')
},
onError: (error: any) => {
const message = error?.message || 'Failed to add collaborator'
toast.error(message)
}
})
const updateMutation = useMutation({
mutationFn: ({ userId, role }: { userId: string; role: string }) =>
investigationService.updateCollaboratorRole(investigationId, userId, { role }),
onSuccess: () => {
queryClient.invalidateQueries({
queryKey: queryKeys.investigations.collaborators(investigationId)
})
toast.success('Role updated')
},
onError: () => toast.error('Failed to update role')
})
const removeMutation = useMutation({
mutationFn: (userId: string) =>
investigationService.removeCollaborator(investigationId, userId),
onSuccess: () => {
queryClient.invalidateQueries({
queryKey: queryKeys.investigations.collaborators(investigationId)
})
toast.success('Collaborator removed')
},
onError: () => toast.error('Failed to remove collaborator')
})
const handleInvite = () => {
if (!selectedEmail.trim()) return
addMutation.mutate({ email: selectedEmail.trim(), role })
}
return (
<Dialog open={open} onOpenChange={setOpen}>
<DialogTrigger asChild>{children}</DialogTrigger>
<DialogContent className="sm:max-w-xl gap-0 p-0 overflow-hidden">
<DialogHeader className="px-5 pt-5 pb-4">
<DialogTitle className="text-base">Share investigation</DialogTitle>
<DialogDescription className="text-sm text-muted-foreground">
Invite collaborators and manage access.
</DialogDescription>
</DialogHeader>
{/* Invite form */}
<div className="px-5 pb-4">
<div className="flex items-center gap-2">
<div className="relative flex-1">
<Input
placeholder="Search by name or email..."
value={query}
onChange={(e) => handleQueryChange(e.target.value)}
onKeyDown={(e) => {
if (e.key === 'Enter') {
handleInvite()
setShowSuggestions(false)
}
if (e.key === 'Escape') setShowSuggestions(false)
}}
onFocus={() => suggestions.length > 0 && setShowSuggestions(true)}
className="h-9"
/>
{showSuggestions && suggestions.length > 0 && (
<div className="absolute z-50 top-full mt-1 w-full rounded-md border bg-popover shadow-lg overflow-hidden">
{suggestions.map((user) => (
<button
key={user.id}
className="w-full flex items-center gap-2.5 px-3 py-2 text-sm hover:bg-accent text-left transition-colors"
onMouseDown={(e) => {
e.preventDefault()
handleSelectUser(user)
}}
>
<UserAvatar user={user} size="sm" />
<div className="flex-1 min-w-0">
<p className="truncate text-sm font-medium">{getDisplayName(user)}</p>
{user.email && (
<p className="truncate text-xs text-muted-foreground">{user.email}</p>
)}
</div>
</button>
))}
</div>
)}
</div>
<Select value={role} onValueChange={setRole}>
<SelectTrigger className="w-[100px] h-9">
<SelectValue />
</SelectTrigger>
<SelectContent>
{ROLE_OPTIONS.map((opt) => {
const Icon = opt.icon
return (
<SelectItem key={opt.value} value={opt.value}>
<span className="flex items-center gap-1.5">
<Icon className="w-3.5 h-3.5 opacity-60" />
{opt.label}
</span>
</SelectItem>
)
})}
</SelectContent>
</Select>
<Button
size="sm"
className="h-9 px-3"
onClick={handleInvite}
disabled={!selectedEmail.trim() || addMutation.isPending}
>
<UserPlus className="w-4 h-4 mr-1.5" />
Invite
</Button>
</div>
</div>
<Separator />
{/* Collaborators list */}
<div className="max-h-72 overflow-y-auto">
{isLoading ? (
<div className="p-5 space-y-3">
{Array.from({ length: 3 }).map((_, i) => (
<div key={i} className="flex items-center gap-3">
<Skeleton className="h-8 w-8 rounded-full" />
<div className="flex-1 space-y-1.5">
<Skeleton className="h-3.5 w-28" />
<Skeleton className="h-3 w-36" />
</div>
<Skeleton className="h-5 w-14 rounded-full" />
</div>
))}
</div>
) : collaborators.length === 0 ? (
<div className="py-10 flex flex-col items-center gap-2 text-center">
<Users className="w-8 h-8 text-muted-foreground/30" />
<p className="text-sm text-muted-foreground">No collaborators yet</p>
</div>
) : (
<div className="p-2">
{collaborators.map((collab) => {
const collabRole = getRoleFromCollaborator(collab)
const isOwner = collabRole === 'owner'
return (
<div
key={collab.id}
className="flex items-center gap-3 px-3 py-2.5 rounded-md hover:bg-muted/50 transition-colors group"
>
<UserAvatar user={collab.user} size="md" />
<div className="flex-1 min-w-0">
<p className="text-sm font-medium truncate">{getDisplayName(collab.user)}</p>
{collab.user?.email && (
<p className="text-xs text-muted-foreground truncate">
{collab.user.email}
</p>
)}
</div>
{isOwner ? (
<RoleBadge role="owner" />
) : (
<DropdownMenu>
<DropdownMenuTrigger asChild>
<button>
<RoleBadge role={collabRole} className="cursor-pointer hover:opacity-80 transition-opacity" />
</button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end" className="min-w-[140px]">
{ROLE_OPTIONS.map((opt) => {
const Icon = opt.icon
const isActive = opt.value === collabRole
return (
<DropdownMenuItem
key={opt.value}
onClick={() => updateMutation.mutate({ userId: collab.user_id, role: opt.value })}
className="flex items-center gap-2"
>
<Icon className="w-3.5 h-3.5 opacity-60" />
{opt.label}
{isActive && <Check className="w-3.5 h-3.5 ml-auto" />}
</DropdownMenuItem>
)
})}
</DropdownMenuContent>
</DropdownMenu>
)}
<Button
variant="ghost"
size="icon"
className={cn(
'h-7 w-7 shrink-0',
isOwner
? 'invisible'
: 'text-muted-foreground/50 hover:text-destructive opacity-0 group-hover:opacity-100 transition-opacity'
)}
onClick={() => !isOwner && removeMutation.mutate(collab.user_id)}
disabled={isOwner}
>
<X className="w-3.5 h-3.5" />
</Button>
</div>
)
})}
</div>
)}
</div>
{/* Footer */}
{collaborators.length > 0 && (
<>
<Separator />
<div className="px-5 py-3">
<p className="text-xs text-muted-foreground">
{collaborators.length} member{collaborators.length !== 1 ? 's' : ''} have access
</p>
</div>
</>
)}
</DialogContent>
</Dialog>
)
}

View File

@@ -9,16 +9,17 @@ import NewSketch from "@/components/sketches/new-sketch"
interface SketchesSectionProps {
sketches: Sketch[]
canCreate?: boolean
}
export function SketchesSection({ sketches }: SketchesSectionProps) {
export function SketchesSection({ sketches, canCreate = true }: SketchesSectionProps) {
const isEmpty = sketches.length === 0
return (
<section className="my-4">
<div className="flex items-center justify-between mb-4">
<h2 className="text-sm font-medium text-foreground">Sketches</h2>
{!isEmpty && (
{!isEmpty && canCreate && (
<NewSketch>
<Button variant="ghost" size="sm" className="h-7 text-xs text-muted-foreground hover:text-foreground gap-1">
<Plus className="w-3.5 h-3.5" />
@@ -29,9 +30,10 @@ export function SketchesSection({ sketches }: SketchesSectionProps) {
</div>
{isEmpty ? (
<EmptySketches onAction={() => console.log("Create sketch")} />
canCreate ? <EmptySketches onAction={() => console.log("Create sketch")} /> : <div className="text-sm text-muted-foreground py-4">No sketches yet.</div>
) : (
<div className="grid grid-cols-2 md:grid-cols-4 gap-3">
<div style={{ containerType: 'inline-size' }}>
<div className="grid grid-cols-1 cq-sm:grid-cols-2 cq-md:grid-cols-3 cq-lg:grid-cols-4 gap-3">
{sketches.map((sketch) => (
<Link
to="/dashboard/investigations/$investigationId/$type/$id"
@@ -74,6 +76,7 @@ export function SketchesSection({ sketches }: SketchesSectionProps) {
</Link>
))}
</div>
</div>
)
}
</section >

View File

@@ -16,7 +16,8 @@ export function DashboardStats({ casesCount, activeCasesCount }: DashboardStatsP
], [casesCount, activeCasesCount])
return (
<div className="grid grid-cols-4 gap-4 mt-8">
<div className="mt-8" style={{ containerType: 'inline-size' }}>
<div className="grid grid-cols-1 cq-sm:grid-cols-2 cq-lg:grid-cols-4 gap-4">
{stats.map((stat) => (
<div key={stat.label} className="px-4 py-3 border border-border rounded-lg bg-card/30">
<div className="flex items-center gap-3">
@@ -31,5 +32,6 @@ export function DashboardStats({ casesCount, activeCasesCount }: DashboardStatsP
</div>
))}
</div>
</div>
)
}

View File

@@ -70,7 +70,13 @@ export function FlowControls({
<Tooltip>
<TooltipTrigger asChild>
<Button variant="outline" size="icon" className="bg-card" onClick={onLayout} data-tour-id="layout-button">
<Button
variant="outline"
size="icon"
className="bg-card"
onClick={onLayout}
data-tour-id="layout-button"
>
<LayoutGrid className="h-4 w-4" />
</Button>
</TooltipTrigger>

View File

@@ -95,7 +95,6 @@ const FlowEditor = memo(({ initialEdges, initialNodes, theme, flow }: FlowEditor
const setLoading = useFlowStore((state) => state.setLoading)
const colors = useNodesDisplaySettings((s) => s.colors)
// #### TanStack Query Mutations ####
// Create flow mutation
const createFlowMutation = useMutation({
mutationFn: flowService.create,
@@ -254,7 +253,7 @@ const FlowEditor = memo(({ initialEdges, initialNodes, theme, flow }: FlowEditor
})
const newNode: FlowNode = {
id: `${enricherData.name}-${Date.now()}`,
type: enricherData.type === 'type' ? 'type' : 'request',
type: enricherData.type === 'type' ? 'type' : 'enricher',
position,
data: {
id: enricherData.id,

View File

@@ -61,7 +61,7 @@ const FlowSheet = ({ onLayout }: { onLayout: () => void }) => {
const position = { x: selectedNode.position.x + 350, y: selectedNode.position.y }
const newNode: FlowNode = {
id: `${enricher.name}-${Date.now()}`,
type: enricher.type === 'type' ? 'type' : 'request',
type: enricher.type === 'type' ? 'type' : 'enricher',
position,
data: {
id: enricher.id,

View File

@@ -8,7 +8,7 @@ import { Input } from '../ui/input'
import KeySelector from '../keys/key-select'
import { type Key } from '@/types/key'
import { useQuery } from '@tanstack/react-query'
import { KeyService } from '@/api/key-service'
import { keyService } from '@/api/key-service'
import { Tabs, TabsList, TabsTrigger, TabsContent } from '../ui/tabs'
import { MemoizedMarkdown } from '../chat/memoized-markdown'
import { cn } from '@/lib/utils'
@@ -39,7 +39,7 @@ const ParamsDialog = () => {
// Fetch keys to convert between IDs and Key objects
const { data: keys = [] } = useQuery<Key[]>({
queryKey: ['keys'],
queryFn: () => KeyService.get()
queryFn: () => keyService.get()
})
const handleSave = useCallback(async () => {

View File

@@ -19,6 +19,8 @@ export default function RawMaterial() {
})
const [searchTerm, setSearchTerm] = useState<string>('')
console.log(materials)
const filteredEnrichers = useMemo(() => {
if (!materials?.items) return {}
const result: Record<string, Enricher[]> = {}
@@ -58,7 +60,7 @@ export default function RawMaterial() {
<Input
type="text"
placeholder="Search enrichers..."
className="pl-8 !border border-border"
className="pl-8 border! border-border"
value={searchTerm}
onChange={(e) => setSearchTerm(e.target.value)}
/>

View File

@@ -6,7 +6,7 @@ import {
SelectTrigger,
SelectValue
} from '@/components/ui/select'
import { KeyService } from '@/api/key-service'
import { keyService } from '@/api/key-service'
import { type Key as KeyType } from '@/types/key'
export default function KeySelector({
@@ -19,7 +19,7 @@ export default function KeySelector({
// Fetch keys
const { data: keys = [], isLoading } = useQuery<KeyType[]>({
queryKey: ['keys'],
queryFn: () => KeyService.get()
queryFn: () => keyService.get()
})
const handleValueChange = (keyId: string) => {

View File

@@ -43,12 +43,8 @@ export const LogPanel = memo(() => {
}
return (
<div className="h-full overflow-hidden border-t p-2">
<TerminalLogViewer
logs={logs}
onRefresh={refetch}
onClear={handleDeleteLogs}
/>
<div className="h-full overflow-hidden border-t">
<TerminalLogViewer logs={logs} onRefresh={refetch} onClear={handleDeleteLogs} />
</div>
)
})

View File

@@ -6,6 +6,7 @@ import { Tooltip, TooltipContent, TooltipProvider, TooltipTrigger } from '../ui/
import { memo } from 'react'
import { isMac } from '@/lib/utils'
import { NavUser } from '../nav-user'
import { CONFIG } from '@/config'
interface NavItem {
icon: LucideIcon
@@ -18,12 +19,14 @@ export const Sidebar = memo(() => {
const navItems: NavItem[] = [
{ icon: Home, label: 'Dashboard', href: '/dashboard/' },
// { icon: Puzzle, label: 'Enrichers', href: '/dashboard/enrichers' },
{ icon: Workflow, label: 'Flows', href: '/dashboard/flows' },
{ icon: Shapes, label: 'Custom types', href: '/dashboard/custom-types' },
{ icon: Lock, label: 'Vault', href: '/dashboard/vault' }
]
if (CONFIG.ENRICHER_TEMPLATES_FEATURE_FLAG)
navItems.push({ icon: Puzzle, label: 'Enrichers', href: '/dashboard/enrichers' })
const commonClasses = 'flex items-center justify-center h-12 w-full rounded-sm hover:bg-muted'
return (

View File

@@ -1,34 +1,74 @@
import { Button } from '@/components/ui/button'
import { Terminal, Unlock } from 'lucide-react'
import { Terminal, Unlock, Zap, ZapOff } from 'lucide-react'
import { ModeToggle } from '../mode-toggle'
import { useLayoutStore } from '@/stores/layout-store'
import Legend from '../sketches/graph/components/legend'
import { Link, useParams } from '@tanstack/react-router'
import InfoDialog from './info'
import { memo } from 'react'
import { isMac } from '@/lib/utils'
import { useQuery } from '@tanstack/react-query'
import { scanService } from '@/api/scan-service'
import { cn } from '@/utils/cn'
import { CONFIG } from '@/config'
export const StatusBar = memo(() => {
const { id: sketch_id } = useParams({ strict: false })
const isOpenConsole = useLayoutStore((s) => s.isOpenConsole)
const toggleConsole = useLayoutStore((s) => s.toggleConsole)
const { data: scans, isLoading } = useQuery({
queryKey: ['scans', 'list'],
queryFn: () => scanService.getSketchScans(sketch_id as string),
enabled: !!sketch_id,
refetchInterval: 2500
// refetchInterval: (data) => {
// // @ts-ignore
// const hasPending = Array.isArray(data) && data?.some((scan) => scan.status === 'PENDING')
// return hasPending ? 2000 : false
// }
})
const pending =
(Array.isArray(scans) && scans?.some((scan) => scan.status === 'PENDING')) || false
return (
<div className="flex items-center bg-card h-8 px-2 text-xs text-muted-foreground">
<div className="flex items-center gap-2">
<div className="flex items-center gap-1">
{sketch_id && (
<Button
variant="ghost"
size="sm"
className="h-6 gap-1 text-xs hover:bg-accent"
onClick={toggleConsole}
>
<Terminal strokeWidth={1.4} className="h-3 w-3" />
<span>
Console {isOpenConsole ? '(Open)' : ''}{' '}
<span className="text-[.7rem] opacity-60">({isMac ? '' : 'ctrl'}D)</span>
</span>
</Button>
<>
<Button
variant="ghost"
size="sm"
className="h-6 gap-1 text-xs hover:bg-accent"
onClick={toggleConsole}
>
<Terminal strokeWidth={1.4} className="h-3 w-3" />
<span>
Console {isOpenConsole ? '(Open)' : ''}{' '}
<span className="text-[.7rem] opacity-60">({isMac ? '⌘' : 'ctrl'}D)</span>
</span>
</Button>
{CONFIG.SCANS_LIST_FEATURE_FLAG && (
<Button
variant="ghost"
size="sm"
className={cn('h-5 gap-1 text-xs hover:bg-accent')}
onClick={toggleConsole}
>
{pending ? (
<Zap strokeWidth={1.4} className="h-3 w-3 text-primary" />
) : (
<ZapOff strokeWidth={1.4} className="h-3 w-3" />
)}
<span>
Scans{' '}
<span className="text-xs opacity-80">
{isLoading ? '(-)' : `(${scans?.length ?? 0})`}
</span>
</span>
{pending && <span className="h-2.5 w-2.5 rounded-full bg-primary animate-pulse" />}
</Button>
)}
</>
)}
</div>
<div className="flex-1"></div>

View File

@@ -3,10 +3,15 @@ import { Link, useNavigate, useParams } from '@tanstack/react-router'
import InvestigationSelector from './investigation-selector'
import SketchSelector from './sketch-selector'
import { memo, useCallback } from 'react'
import { useQuery } from '@tanstack/react-query'
import { Switch } from '../ui/switch'
import { Label } from '../ui/label'
import { useLayoutStore } from '@/stores/layout-store'
import { Button } from '@/components/ui/button'
import { AvatarGroup } from '@/components/ui/avatar'
import { investigationService } from '@/api/investigation-service'
import { queryKeys } from '@/api/query-keys'
import type { Collaborator } from '@/types'
import { ImportSheet } from '../sketches/import-sheet'
import {
DropdownMenu,
@@ -18,20 +23,28 @@ import {
DropdownMenuShortcut,
DropdownMenuTrigger
} from '@/components/ui/dropdown-menu'
import { Ellipsis, Upload } from 'lucide-react'
import { Settings2, Upload } from 'lucide-react'
import { isMac } from '@/lib/utils'
import { useGraphSettingsStore } from '@/stores/graph-settings-store'
import { useMutation } from '@tanstack/react-query'
import { Separator } from '../ui/separator'
import { useConfirm } from '../use-confirm-dialog'
import { sketchService } from '@/api/sketch-service'
import { toast } from 'sonner'
import { useKeyboardShortcut } from '@/hooks/use-keyboard-shortcut'
import { usePermissions } from '@/hooks/use-can'
export const TopNavbar = memo(() => {
const { investigationId, id, type } = useParams({ strict: false })
const toggleAnalysis = useLayoutStore((s) => s.toggleAnalysis)
const isOpenAnalysis = useLayoutStore((s) => s.isOpenAnalysis)
const { data: collaborators = [] } = useQuery<Collaborator[]>({
queryKey: queryKeys.investigations.collaborators(investigationId!),
queryFn: () => investigationService.getCollaborators(investigationId!),
enabled: !!investigationId
})
const handleToggleAnalysis = useCallback(() => toggleAnalysis(), [toggleAnalysis])
return (
@@ -59,7 +72,13 @@ export const TopNavbar = memo(() => {
<Command />
</div>
</div>
<div className="flex items-center gap-4">
<div className="flex items-center gap-3">
{investigationId && collaborators.length > 0 && (
<>
<AvatarGroup users={collaborators.map((c) => c.user)} size="sm" max={5} />
<Separator orientation="vertical" className="h-5" />
</>
)}
<div className="flex items-center space-x-2">
{type === 'graph' && (
<>
@@ -78,7 +97,14 @@ export const TopNavbar = memo(() => {
)
})
export function InvestigationMenu({ investigationId, sketchId }: { investigationId?: string, sketchId: string }) {
export function InvestigationMenu({
investigationId,
sketchId
}: {
investigationId?: string
sketchId: string
}) {
const { canEdit } = usePermissions()
const toggleSettingsModal = useGraphSettingsStore((s) => s.toggleSettingsModal)
const toggleKeyboardShortcutsModal = useGraphSettingsStore((s) => s.toggleKeyboardShortcutsModal)
const setImportModalOpen = useGraphSettingsStore((s) => s.setImportModalOpen)
@@ -137,7 +163,7 @@ export function InvestigationMenu({ investigationId, sketchId }: { investigation
<DropdownMenuTrigger asChild>
<div>
<Button size="icon" variant="ghost">
<Ellipsis />
<Settings2 />
</Button>
</div>
</DropdownMenuTrigger>
@@ -155,23 +181,34 @@ export function InvestigationMenu({ investigationId, sketchId }: { investigation
</DropdownMenuGroup>
<DropdownMenuSeparator />
<DropdownMenuItem>
<a className='h-full w-full' target='_blank' href="https://github.com/reconurge/flowsint">GitHub</a>
<a className="h-full w-full" target="_blank" href="https://github.com/reconurge/flowsint">
GitHub
</a>
</DropdownMenuItem>
<DropdownMenuItem>
<a className='h-full w-full' target='_blank' href="https://github.com/reconurge/flowsint/issues">Support</a>
<a
className="h-full w-full"
target="_blank"
href="https://github.com/reconurge/flowsint/issues"
>
Support
</a>
</DropdownMenuItem>
<DropdownMenuItem disabled>API</DropdownMenuItem>
<DropdownMenuSeparator />
<DropdownMenuItem onClick={() => setImportModalOpen(true)}>
<Upload /> Import entities
</DropdownMenuItem>
<DropdownMenuSeparator />
<DropdownMenuItem onClick={handleDelete} variant="destructive">
Delete sketch
{/* <DropdownMenuShortcut>⇧⌘Q</DropdownMenuShortcut> */}
</DropdownMenuItem>
{canEdit && (
<>
<DropdownMenuSeparator />
<DropdownMenuItem onClick={() => setImportModalOpen(true)}>
<Upload /> Import entities
</DropdownMenuItem>
<DropdownMenuSeparator />
<DropdownMenuItem onClick={handleDelete} variant="destructive">
Delete sketch
</DropdownMenuItem>
</>
)}
</DropdownMenuContent>
<ImportSheet sketchId={sketchId} />
{canEdit && <ImportSheet sketchId={sketchId} />}
</DropdownMenu>
)
}

View File

@@ -12,6 +12,7 @@ const MapPanel = () => {
(node.nodeProperties.latitude && node.nodeProperties.longitude)
)
.map((node) => ({
nodeId: node.id,
lat: node.nodeProperties.latitude != null ? Number(node.nodeProperties.latitude) : undefined,
lon:
node.nodeProperties.longitude != null ? Number(node.nodeProperties.longitude) : undefined,

Some files were not shown because too many files have changed in this diff Show More