Commit Graph

829 Commits

Author SHA1 Message Date
dextmorgn
82925bec38 fix(deploy): serve prod from pre-built images
Frontend baked VITE_API_URL into the bundle at build time, so
published images pointed at whatever URL CI had — "fail to fetch"
for every other deployment. Bundle now defaults to same-origin
relative URLs and the frontend nginx proxies /api/ to the api
service over the Docker network; CORS becomes irrelevant.

- docker-compose.prod.yml: use ghcr images (FLOWSINT_VERSION,
  default latest), drop build sections; bind postgres/redis/neo4j/
  api ports to 127.0.0.1 — only 5173 faces the network
- nginx: /api/ reverse proxy, request-time DNS resolution, SSE
  support (proxy_buffering off)
- Makefile: prod pulls instead of building; remove dead deploy
  targets referencing nonexistent docker-compose.deploy.yml
- README: network/server deployment section (secrets checklist,
  version pinning, TLS via reverse proxy)
2026-06-05 14:04:16 +02:00
dextmorgn
f38d5d500d Merge pull request #174 from reconurge/chore/security-dependency-upgrades
fix(deps): upgrade Python dependencies to patch 27 known vulnerabilities
2026-06-05 13:21:03 +02:00
dextmorgn
9fcc60ca63 fix(deps): upgrade Python dependencies to patch 27 known vulnerabilities
pip-audit reported 27 vulnerabilities across 12 packages, including
cryptography PYSEC-2026-36 (CVSS 9.8) used by the vault, a starlette
Host-header path bypass, and python-multipart arbitrary file write.

- cryptography 45.0.7 -> 46.0.7 (pin was capped at <46)
- fastapi 0.115 -> 0.136, pulling starlette 0.46 -> 1.2
- python-multipart 0.0.20 -> 0.0.32
- pyjwt, urllib3, aiohttp, lxml, idna, mako and transitive bumps
- dev tools: pytest 8 -> 9 (requires pytest-httpx >=0.36), black 25 -> 26

All four module test suites pass (458 tests). pip-audit now reports
no known vulnerabilities.
2026-06-05 08:57:19 +02:00
dextmorgn
5cdfe727b4 Merge pull request #172 from melihdaskiran/fix/zip-and-vault-bugs
Fix zip alignment mismatch, phone validation return value, and vault …
2026-06-05 08:24:39 +02:00
Melih Daskiran
25eb3a0400 Fix zip alignment mismatch, phone validation return value, and vault owner_id type matching 2026-06-04 21:46:44 +03:00
dextmorgn
259019f23c fix(build): force LF on container files via .gitattributes
Windows checkouts with core.autocrlf=true convert entrypoint.sh to
CRLF, which Docker copies into the image. The shebang then reads
#!/bin/sh\r and exec fails with 'no such file or directory'.
Also covers .env.example (trailing \r in env values) and Makefile.
2026-06-04 18:23:53 +02:00
dextmorgn
7c082d733e docs(readme): make Windows install commands cmd-compatible
PowerShell foreach/Test-Path loop fails when pasted into cmd.exe.
Plain copy commands work in both shells.
2026-06-04 18:05:19 +02:00
dextmorgn
de93ae3059 docs(readme): add Windows install path without Make
Windows users hit failures with make-based install. Replicate
check-env + build + up targets as plain PowerShell/docker compose
commands for prod and dev.
2026-06-04 18:00:35 +02:00
dextmorgn
c6eaf08b66 Merge pull request #163 from Mubashirrrr/fix/json-import-apostrophe-corruption
fix(import): preserve apostrophes in valid JSON imports
2026-06-03 20:34:42 +02:00
dextmorgn
f380674955 Merge pull request #165 from reconurge/fix/sse-auth-header
fix(sse): auth header
2026-06-03 13:42:17 +02:00
Mubashir Rahim
1fdbdbd094 fix(import): preserve apostrophes in valid JSON imports
parse_json unconditionally replaced every single quote with a double
quote (`.replace("'", '"')`) before calling json.loads. This corrupted
any valid JSON string value containing an apostrophe, e.g. a person's
surname "Sarah O'Brien" became "Sarah O"Brien", causing the parse to
fail with "Invalid JSON" and the whole import to be rejected.

Parse strict JSON first so well-formed payloads (including apostrophes
in string values) import correctly, and only fall back to the lenient
single-quote replacement for Python-dict-style payloads that are not
valid JSON on their own.

Adds regression tests covering an apostrophe-bearing JSON value and the
retained single-quoted fallback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 14:34:22 +05:00
dextmorgn
b317e08e57 feat(api): authenticate SSE via header, drop query-param auth and dead route 2026-06-03 11:28:41 +02:00
dextmorgn
9db1e675d6 test(api): SSE auth-gate and endpoint-removal tests (red) 2026-06-03 11:22:58 +02:00
dextmorgn
4a75d6a924 refactor(app): stream graph status via connectSSE (token out of URL) 2026-06-03 11:08:11 +02:00
dextmorgn
d7f21e9384 refactor(app): stream logs via connectSSE (token out of URL) 2026-06-03 10:56:32 +02:00
dextmorgn
0b60c81361 Merge pull request #155 from Ghraven/fix-investigation-aware-timestamp
fix: use aware timestamp for investigation updates
2026-06-03 09:30:35 +02:00
dextmorgn
6e5f26aaf0 feat(app): add connectSSE helper with header auth and backoff 2026-06-03 09:28:49 +02:00
Ghraven
60cfa16b8d fix: use aware timestamp for investigation updates 2026-06-03 08:37:45 +08:00
dextmorgn
078ac77eee build(app): add fetch-event-source + vitest test runner 2026-06-02 22:45:36 +02:00
dextmorgn
ed8bfc5248 Merge pull request #149 from Ghraven/fix/timezone-aware-service-timestamps
Use timezone-aware UTC timestamps in core services
2026-06-02 21:11:45 +02:00
dextmorgn
438bbe18c4 Merge pull request #150 from Ghraven/fix-social-enricher-output-encoding
fix: read social enricher outputs as utf-8
2026-06-02 21:10:12 +02:00
Ghraven
4e14df6c61 fix: read social enricher outputs as utf-8 2026-06-01 00:32:48 +08:00
dextmorgn
249374e6d7 chore: update .gitignore v1.2.9 2026-05-31 14:53:34 +02:00
dextmorgn
a15ee54af0 chore: update skill name 2026-05-31 14:52:37 +02:00
Ghraven
b707961e68 fix: use timezone-aware service timestamps 2026-05-31 02:35:11 +08:00
dextmorgn
cc829704e5 Merge pull request #148 from reconurge/fix/phase1-critical
fix: security issues
2026-05-30 19:47:55 +02:00
dextmorgn
e8ca7b39bf fix(types): apply extra=allow via Pydantic v2 model_config
FlowsintType declared an inner `class ConfigDict: extra = "allow"`, which
Pydantic v2 treats as an ordinary nested class and ignores entirely. The
base therefore fell back to the default extra="ignore", silently dropping
user-supplied extra properties on every entity type. Replaced with
model_config = ConfigDict(extra="allow"). Extra keys are now retained.

flowsint-types (44), flowsint-enrichers (12) and flowsint-core (390) suites pass.
2026-05-30 19:03:30 +02:00
dextmorgn
1d409d7929 fix(scan): persist results to existing details column, errors to error column
The Scan model exposes details (JSON) and error (Text) columns, but the
enricher/flow tasks assigned to scan.results — a non-mapped attribute.
SQLAlchemy accepted the assignment silently, so commit() never persisted
it and the data was lost. Success paths now write scan.details; failure
paths write scan.error. ScanCreate.results renamed to details to match.

Scan repository tests pass.
2026-05-30 19:02:14 +02:00
dextmorgn
ee06d6cf3e fix(api): restrict CORS origins via ALLOWED_ORIGINS env
allow_origins was ["*"] together with allow_credentials=True, which is
both insecure and rejected by browsers for credentialed requests. Read a
comma-separated ALLOWED_ORIGINS env var instead, defaulting to the Vite
dev origin http://localhost:5173. Documented in .env.example.
2026-05-30 18:49:44 +02:00
dextmorgn
dd00bae9ce fix(api): run production container as non-root flowsint user
The production stage created the flowsint user and chowned all app
files to it, but the USER directive was commented out, so the API
ran as root. Uncommented it. Verified: image builds, container reports
whoami=flowsint and /health returns 200.
2026-05-30 18:49:43 +02:00
dextmorgn
c9d594a8e7 fix(api): load AUTH_SECRET from validated core.auth source
deps.py read AUTH_SECRET via os.getenv() before load_dotenv() ran,
yielding None when the secret was only present in .env. Import the
already-validated AUTH_SECRET from core.auth (loads dotenv first and
raises if unset) instead of re-reading it. Removes the dup + ordering bug.
2026-05-30 18:44:07 +02:00
dextmorgn
de4f4ea87f Update README with new image links
Added additional image links to the README.
2026-05-28 00:00:13 +02:00
dextmorgn
67ba8cfcc6 feat(chore): update SKILL.md 2026-05-21 11:09:28 +02:00
dextmorgn
8fbcf0c025 chore(skills): add flowsint-transform-builder claude skill
Project-scoped Claude Code skill guiding enricher and type creation.
Whitelists .claude/skills/flowsint-transform-builder/ in .gitignore so
the skill ships with the repo while keeping the rest of .claude/ ignored.
2026-05-15 18:55:56 +02:00
dextmorgn
d431ef4fae docs: sync with flowsint-types and flowsint-enrichers (v1.2.8)
- available-enrichers.mdx: add 17 missing enrichers, drop phantom entries
- managing-enrichers.mdx: fix stale Troubleshooting referencing EnricherRegistry.register()
- managing-types.mdx: note 8 registered-but-uncategorized types
- bump all frontmatter to version 1.2.8 / 2026-05-15
2026-05-15 18:55:25 +02:00
dextmorgn
5be480d4f5 Merge pull request #138 from z3rodaycve/feature/collection-enrichers
Release of my collection of Flowsint enrichers
2026-05-10 17:58:55 +02:00
dextmorgn
431d3997f4 Merge pull request #142 from harmsolo13/fix/celery-healthcheck
fix: replace celery container's inherited curl-based healthcheck
2026-04-29 11:35:17 +02:00
harmsolo13
d4dbe8d0fe fix: replace celery container's inherited curl-based healthcheck
The celery container builds from flowsint-api/Dockerfile which carries
a HEALTHCHECK directive that does `curl -f http://localhost:5001/health`.
The API container has an HTTP server on 5001 — celery doesn't, it's a
worker. So the inherited healthcheck always fails and `docker ps` shows
celery as (unhealthy) even when the worker is actively processing jobs.

This is cosmetic noise today but bites in two real ways: (1) restart
policies that key off health won't re-up celery on a real failure
because Docker can't tell good unhealthy from bad unhealthy, (2) any
service that adds `depends_on: celery: condition: service_healthy`
will refuse to start.

Fix: add a service-level healthcheck on celery in both compose files
(prod and dev) that uses celery's own `inspect ping` primitive
against the worker's broker. Compose-level healthcheck overrides the
Dockerfile-level one, so no Dockerfile change needed.

Smoke-tested locally: container goes from (unhealthy) to (healthy)
within ~30s of restart with no other changes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 18:13:26 +09:30
z3rodaycve
f79403d361 Fixed to_ssl enricher to use httpX Docker Tool 2026-04-24 19:49:20 +02:00
z3rodaycve
0d04423b59 Release of DeHashed, httpX, veriphone, scamalytics enrichers 2026-04-21 16:49:12 +02:00
dextmorgn
62e34c6e9b chore: update README.md with discord link 2026-04-19 13:18:55 +02:00
dextmorgn
2a4878c8fc chore: update gh workflow v1.2.8 2026-04-11 15:20:07 +02:00
dextmorgn
6a14484ccd Merge pull request #136 from reconurge/feat/rbac
Feat/rbac
2026-04-11 14:51:34 +02:00
dextmorgn
47b0c05115 feat(app): RBAC permission gating, UserAvatar component, share dialog redesign and responsive grids
- Add UserAvatar and AvatarGroup components to /ui/avatar, migrate all consumers
- Display collaborator avatars in top navbar with AvatarGroup
- Gate edit actions behind canEdit: toolbar (connect, merge, layouts), nodes-view,
  nodes-panel, relationships-view (checkboxes + delete), edge-details-panel,
  graph-navigation (Add tab), import sheet, analysis creation
- Add role badge (owner/admin/editor/viewer) with colored icons in case-header
- Redesign share-dialog: role badges as dropdown triggers, skeleton loading,
  owner/member separation, hover-reveal delete button
- Switch all card grids from viewport breakpoints to CSS container queries
  (dashboard-stats, investigations-list, sketches-section, metrics-grid, custom-types)
- Extend container query classes in styles.css for all needed column counts
2026-04-11 14:44:29 +02:00
dextmorgn
8cd6b237bb fix(core/tests): update failing test 2026-04-11 14:44:29 +02:00
dextmorgn
ce1f50bb72 chore: update yarn.lock 2026-04-11 14:44:28 +02:00
dextmorgn
f5e6fb79ae feat(app): RBAC permission gating, share dialog, profile page and user display
- Add useCan hook reading permissions from $investigationId layout route loader
- Add $investigationId layout route to fetch investigation and share data with children
- Add share dialog with user search autocomplete (debounced) for collaborator management
- Add /dashboard/profile page for editing first name, last name, avatar URL
- Gate all edit/create/delete actions based on user role (deny by default)
  - Graph: node drag, quick add, link creation, context menus, detail panel editing
  - Analysis: editor readonly, title readonly, actions hidden for viewers
  - Investigation: delete (owner only), share (admin+), new sketch/analysis (editor+)
  - Settings: sketch info readonly, delete sketch and import hidden for viewers
  - Selected items panel: enrich, delete, ask AI hidden for viewers
- Fix nav-user to show real user data instead of hardcoded placeholder
- Add user-display utils (getDisplayName, getInitials) shared across components
- Add avatar display in nav-user, case-header collaborators and share dialog
2026-04-11 14:44:28 +02:00
dextmorgn
bb502379bd feat(api): add collaborator endpoints, profile update, user search and backfill migration
- Add GET/POST/PUT/DELETE /investigations/{id}/collaborators endpoints
- Inject current_user_role in all investigation responses
- Add GET/PUT /auth/me for profile management
- Add GET /auth/users/search for share dialog autocomplete
- Add ProfileUpdate schema with extra=forbid
- Add email to ProfileRead
- Add backfill migration for existing investigations missing owner role entries
2026-04-11 14:44:28 +02:00
dextmorgn
1a5d164805 feat(core): add RBAC with ADMIN role, collaborator management and auth user data
- Add ADMIN role to Role enum (OWNER > ADMIN > EDITOR > VIEWER)
- Update permission matrix: ADMIN can read/create/update/manage
- Fix get_with_relations bug filtering by owner_id (blocked non-owner collaborators)
- Add collaborator management methods to InvestigationRepository and InvestigationService
- Return user profile data in auth response
2026-04-11 14:44:28 +02:00
dextmorgn
400b21a7d1 feat(api): install uv Dockerfile 2026-04-11 14:42:39 +02:00