feat(api): authenticate SSE via header, drop query-param auth and dead route

Use timezone-aware UTC timestamps in core services

.claude/skills/flowsint-enricher-builder/SKILL.md

@@ -0,0 +1,178 @@
+---
+name: flowsint-enricher-builder
+description: Expert guidance for building Flowsint enrichers and their supporting types. Use when the user wants to add a new enricher, create a new Flowsint type, wire a new external API/tool into Flowsint, debug type/enricher discovery, or design a pivot from entity A to entity B. Knows where types live, how the enricher base class works, how vault secrets and params resolve, and when to recommend creating a new type instead of forcing data into an existing one.
+---
+
+# Flowsint Enricher Builder
+
+You build enrichers and types for Flowsint. You do not memorize the catalog — you know where to look and how the pieces fit. Always read source before generating code: type definitions and existing enrichers are the ground truth.
+
+## Authoritative source paths
+
+Read these first. Never assume signatures or fields — open the file.
+
+| What | Path |
+|---|---|
+| Type definitions | `flowsint-types/src/flowsint_types/<name>.py` |
+| Type registry + decorator | `flowsint-types/src/flowsint_types/registry.py` |
+| Type package exports | `flowsint-types/src/flowsint_types/__init__.py` |
+| Enricher base class | `flowsint-core/src/flowsint_core/core/enricher_base.py` |
+| Enricher registry + decorator | `flowsint-enrichers/src/flowsint_enrichers/registry.py` |
+| Existing enrichers (templates) | `flowsint-enrichers/src/flowsint_enrichers/<input_type>/to_<output>.py` |
+| UI category mapping | `flowsint-core/src/flowsint_core/core/services/type_registry_service.py` (`_get_category_definitions`) |
+| Vault interface | `flowsint-core/src/flowsint_core/core/vault.py` |
+| Logger interface | `flowsint-core/src/flowsint_core/core/logger.py` |
+| Tools (external CLI/API wrappers) | `tools/` (top-level), e.g. `tools.network.subfinder.SubfinderTool` |
+| Doc — types tutorial | `docs/developers/managing-types.mdx` |
+| Doc — enrichers tutorial | `docs/developers/managing-enrichers.mdx` |
+| Doc — enricher catalog | `docs/sources/available-enrichers.mdx` |
+
+## The first question: new type or reuse?
+
+When the user describes a enricher, decide before writing code:
+
+1. **List the entities involved** — input data, output data, intermediate fields you'll attach.
+2. **For each, check `flowsint-types/src/flowsint_types/`** — open the closest candidate file and read its fields.
+3. **Decide:**
+   - **Reuse** if existing type covers all required fields (extras allowed — `ConfigDict.extra = "allow"`).
+   - **Extend an existing type** if 1–2 fields are missing — propose adding optional fields to the existing model.
+   - **Create new type** if the entity is conceptually distinct (different primary key, different label semantics, different graph role).
+4. **Never cram data into a wrong type.** If a "Domain" enricher returns risk scores, a `RiskProfile` exists — don't stuff scores into `Domain` metadata. If nothing fits, propose a new type and tell the user why.
+
+Surface the decision to the user before generating code: list candidate types you found, what's missing, and your recommendation.
+
+## Anatomy of an enricher
+
+Minimum surface (read `enricher_base.py` for the full contract):
+
+```python
+from typing import List
+from flowsint_core.core.enricher_base import Enricher
+from flowsint_core.core.logger import Logger
+from flowsint_enrichers.registry import flowsint_enricher
+from flowsint_types import Domain, Ip  # or whatever types
+
+@flowsint_enricher
+class MyEnricher(Enricher):
+    """[Source name] One-line purpose."""
+
+    InputType = Domain      # base type, not List[Domain]
+    OutputType = Ip
+
+    @classmethod
+    def name(cls) -> str: return "domain_to_ip"     # snake_case, unique
+    @classmethod
+    def category(cls) -> str: return "Domain"        # see note on casing below
+    @classmethod
+    def key(cls) -> str: return "domain"             # primary field of InputType
+
+    @classmethod
+    def get_params_schema(cls):                       # optional, only if params needed
+        return [...]
+
+    async def scan(self, data: List[InputType]) -> List[OutputType]:
+        ...
+
+    def postprocess(self, results, input_data):
+        for src, dst in zip(input_data, results):
+            self.create_node(src)
+            self.create_node(dst)
+            self.create_relationship(src, dst, "RESOLVES_TO")
+        return results
+
+InputType = MyEnricher.InputType
+OutputType = MyEnricher.OutputType
+```
+
+**File location:** `flowsint-enrichers/src/flowsint_enrichers/<input_type>/to_<target>.py`. The directory matches the input type's lowercase name. If no directory exists for your input type, create it (no `__init__.py` needed — auto-discovery walks the tree).
+
+**Registration:** the `@flowsint_enricher` decorator does it. Do not edit any `registry.py`. API restart picks up new files via `load_all_enrichers()`.
+
+## Params and secrets
+
+Defined via `get_params_schema()` classmethod. Each entry is a dict:
+
+| Field | Required | Notes |
+|---|---|---|
+| `name` | yes | Param key; for `vaultSecret`, also the default vault key name |
+| `type` | yes | One of `string`, `number`, `select`, `url`, `vaultSecret` |
+| `description` | yes | Shown in UI |
+| `required` | no | Defaults to `false` |
+| `default` | no | Default value |
+| `options` | for `select` | List of `{"label": ..., "value": ...}` |
+
+Read params inside `scan()`:
+
+```python
+mode = self.params.get("mode", "passive")
+api_key = self.get_secret("MY_API_KEY")   # vault-resolved during async_init
+```
+
+**Vault resolution flow** (see `Enricher.resolve_params` in `enricher_base.py`):
+1. If user passed a vault ID in params → vault looked up by that ID.
+2. Else → vault looked up by the param name (e.g. `MY_API_KEY`).
+3. If `required: true` and nothing found → `Exception("Required vault secret 'MY_API_KEY' is missing...")`.
+
+**Never hardcode keys.** Always declare a `vaultSecret` param. Document the expected vault key name in the docstring.
+
+## Graph operations (postprocess)
+
+`create_node(obj)` and `create_relationship(from_obj, to_obj, rel_label="IS_RELATED_TO")` take Pydantic objects directly. Don't manually construct node dicts — pass the typed instance.
+
+Relationship label convention: `UPPER_SNAKE_CASE` verb phrase (`HAS_DOMAIN`, `RESOLVES_TO`, `FOUND_IN_BREACH`). Be consistent with existing enrichers — grep before inventing a new label.
+
+`self.log_graph_message("...")` for graph-related progress logs. `Logger.info / error / warn(self.sketch_id, {"message": "..."})` for general logs.
+
+## Creating a new type — checklist
+
+When you decide a new type is warranted:
+
+1. **File**: `flowsint-types/src/flowsint_types/<snake_case>.py`.
+2. **Class**: `PascalCase`, inherit from `FlowsintType`, decorate with `@flowsint_type`.
+3. **Exactly one primary field**: `Field(..., json_schema_extra={"primary": True})`. Must uniquely identify the entity (used as Neo4j MERGE key).
+4. **`compute_label`**: `@model_validator(mode='after')`, sets `self.nodeLabel`, returns `self`. Handle `None` for optional fields.
+5. **Export in `__init__.py`**: add import + entry in `__all__`.
+6. **Category** (optional but recommended): add a `("MyType", "primary_field_name", icon)` tuple in `_get_category_definitions()` in `type_registry_service.py`. Without this, the type works as an enricher I/O but doesn't show in the UI type picker.
+7. **Reinstall**: `cd flowsint-types && poetry install` (or `make prod` from repo root).
+8. **Test**: write a `tests/test_<name>.py` covering creation, primary uniqueness, `compute_label` with full/partial fields.
+
+Full template + patterns: `docs/developers/managing-types.mdx`.
+
+## Naming conventions (already-established, don't break)
+
+- Enricher `name()`: `<input>_to_<output>` snake_case (e.g. `domain_to_ip`, `email_to_breaches`).
+- Enricher file: `to_<target>.py` under `<input_type>/` directory.
+- Class name: descriptive PascalCase (e.g. `DomainToIpEnricher`, `WhoisEnricher`).
+- Type class: PascalCase. Type file: snake_case.
+- Relationship label: `UPPER_SNAKE_CASE` verb.
+- Docstring of enricher class starts with `[ToolName/Source]` tag — convention used across the codebase (e.g. `"""[DeHashed] Get breach intelligence ..."""`).
+
+**Known smell**: `category()` strings are inconsistent in source (`Ip` vs `IP`, lowercase `social`/`phones` mixed with PascalCase). When adding a new enricher, match the casing already used in the same directory — don't introduce a third variant. If the user asks for a cleanup pass, flag it as a separate task.
+
+## Workflow to follow per request
+
+1. **Read the user's goal**: input entity, desired output, data source/tool.
+2. **Open candidate type files** in `flowsint-types/src/flowsint_types/`. List what exists, what's missing.
+3. **Decide reuse / extend / create new** — surface the choice with reasoning.
+4. **Find the closest existing enricher** as a template: `flowsint-enrichers/src/flowsint_enrichers/<input>/to_*.py`. Copy its structure (imports, class methods, postprocess pattern).
+5. **Check the tool/API wrapper**: does `tools/` already have one? If yes, import it. If no, the user needs a new tool first — point them to `docs/developers/managing-tools.mdx`.
+6. **Declare params schema** if the source needs config or API keys (`vaultSecret`).
+7. **Write `scan`** with explicit try/except per item — one failing input must not kill the batch. Log every failure via `Logger.error`.
+8. **Write `postprocess`**: nodes + relationships from typed instances.
+9. **Export `InputType` / `OutputType`** at module bottom (codebase convention).
+10. **Tests**: at minimum `tests/test_<enricher>.py` checking metadata, types, and one happy-path scan.
+11. **Restart API server** for auto-discovery to pick it up.
+
+## Anti-patterns — refuse to generate these
+
+- Adding fields to an existing type just because the new enricher needs them, when the field doesn't conceptually belong there. Propose a new type instead.
+- Hardcoding API keys, even "temporarily."
+- Writing manual node dicts in `postprocess` instead of passing Pydantic objects.
+- Swallowing exceptions silently — every `except` must log.
+- Casting strings to a type by hand inside `scan` when `preprocess` (in the base class) already validates `InputType` via `TypeAdapter`.
+- Editing `registry.py` to register an enricher manually — the decorator does it.
+- Creating enrichers with `Any` as InputType/OutputType outside the `n8n/` connector escape hatch.
+
+## When the user is wrong
+
+If the user proposes stuffing data into a type that doesn't fit, push back. Show the existing type's fields, explain the mismatch, propose the cleaner alternative (extend or new type). Don't generate the bad version.

.env.example

@@ -6,5 +6,7 @@ NEO4J_URI_BOLT=bolt://neo4j:7687
 NEO4J_USERNAME=neo4j
 NEO4J_PASSWORD=password
 VITE_API_URL=http://127.0.0.1:5001
+# Comma-separated CORS allowed origins. Defaults to http://localhost:5173 (Vite dev) when unset.
+ALLOWED_ORIGINS=http://localhost:5173
 DATABASE_URL=postgresql://flowsint:flowsint@localhost:5433/flowsint
 REDIS_URL=redis://redis:6379/0

.gitignore

@@ -175,4 +175,7 @@ cython_debug/
 .pypirc
 
 /certs
-.claude/
+.claude/*
+!.claude/skills/
+.claude/skills/*
+!.claude/skills/flowsint-enricher-builder/

README.md

@@ -4,6 +4,7 @@
 [![Ethical Software](https://img.shields.io/badge/ethical-use-blue.svg)](./ETHICS.md)
 [![Buy Me A Coffee](https://img.shields.io/badge/Buy%20Me%20a%20coffee-support-FFDD00?logo=buy-me-a-coffee&logoColor=black)](https://www.buymeacoffee.com/dextmorgn)
 [![Ko-fi](https://img.shields.io/badge/Ko--fi-support-F16061?logo=ko-fi&logoColor=white)](https://ko-fi.com/P5P01W3GPJ)
+[![Discord](https://img.shields.io/badge/Discord-Join%20Server-5865F2?logo=discord&logoColor=white)](https://discord.gg/aST9HMQr)
 
 
 Flowsint is an open-source OSINT graph exploration tool designed for ethical investigation, transparency, and verification.
@@ -11,8 +12,16 @@ Flowsint is an open-source OSINT graph exploration tool designed for ethical inv
 **Ethics:** Please read [ETHICS.md](./ETHICS.md) for responsible use guidelines.
 
 <img width="1439" height="899" alt="hero-dark" src="https://github.com/user-attachments/assets/01eb128e-bef4-486e-9276-c4da58f829ae" />
-<img width="1511" height="946" alt="Capture d’écran 2026-01-13 à 09 15 58" src="https://github.com/user-attachments/assets/d1a9eca6-9ec4-4402-93f4-303c3dc30de1" />
-<img width="1511" height="948" alt="Capture d’écran 2026-01-13 à 09 19 45" src="https://github.com/user-attachments/assets/6d9e9e6d-d8c7-4ed2-8b8c-53a945b28d05" />
+
+
+https://github.com/user-attachments/assets/eaabfa81-d7b3-414d-8cf7-f69b4e37bab6
+
+
+https://github.com/user-attachments/assets/7457d94a-cf1d-4a97-949f-f9b1d8d92644
+
+
+https://github.com/user-attachments/assets/65c3f26e-7132-4853-be45-21b8933688bd
+
 
 ## Contributing
 

docker-compose.dev.yml

@@ -123,6 +123,14 @@ services:
       - REDIS_URL=redis://redis:6379/0
       - SKIP_MIGRATIONS=true
       - AUTH_SECRET=${AUTH_SECRET}
+    healthcheck:
+      # Celery has no HTTP server — Dockerfile's curl-based healthcheck always fails.
+      # Use celery's own ping primitive instead.
+      test: ["CMD-SHELL", "celery -A flowsint_core.core.celery inspect ping -d celery@$$HOSTNAME || exit 1"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 30s
     depends_on:
       postgres:
         condition: service_healthy

docker-compose.prod.yml

@@ -123,6 +123,14 @@ services:
       - REDIS_URL=redis://redis:6379/0
       - SKIP_MIGRATIONS=true
       - AUTH_SECRET=${AUTH_SECRET}
+    healthcheck:
+      # Celery has no HTTP server — Dockerfile's curl-based healthcheck always fails.
+      # Use celery's own ping primitive instead.
+      test: ["CMD-SHELL", "celery -A flowsint_core.core.celery inspect ping -d celery@$$HOSTNAME || exit 1"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 30s
     depends_on:
       postgres:
         condition: service_healthy

docs/developers/getting-started.mdx

@@ -0,0 +1,18 @@
+---
+title: "Getting started"
+description: "This guide explains how you can contribute to Flowsint."
+category: "Developers"
+order: 7
+author: "Flowsint Team"
+tags: ["tutorial", "developers", "getting-started"]
+version: "1.2.8"
+last_updated_at: "2026-05-15"
+---
+
+## Contributing
+
+Your contribution to Flowsint is highly encouraged ! 
+
+You can create your own [Types](/docs/developers/managing-types), [Tools](/docs/developers/managing-tools), [Enrichers](/docs/developers/managing-enrichers) and Flows, and sharing them by proposing [pull requests](https://github.com/reconurge/flowsint/pulls).
+
+Before diving in, you may also want to understand the [Graph format](/docs/developers/graph-format) to learn how nodes and edges are structured in the frontend visualization.

docs/developers/graph-format.mdx

@@ -0,0 +1,244 @@
+---
+title: "Graph format"
+description: "Reference for the GraphNode and GraphEdge types used to represent entities and relationships in the Flowsint graph. Understanding these structures is essential for working with the frontend visualization and the graph database layer."
+category: "Developers"
+order: 11
+author: "Flowsint Team"
+tags: ["tutorial", "developers", "graph", "nodes", "edges"]
+version: "1.2.8"
+last_updated_at: "2026-05-15"
+---
+
+## Overview
+
+Flowsint's graph visualization is built on two core data structures: **GraphNode** (representing entities) and **GraphEdge** (representing relationships between entities). These types are defined in `flowsint-app/src/types/graph.ts` and are used throughout the frontend for rendering, interaction, and data management.
+
+Understanding these structures is important when:
+- Building enrichers that create nodes and relationships
+- Working on the frontend visualization
+- Debugging graph rendering issues
+- Extending the graph with new visual features
+
+## GraphNode
+
+A `GraphNode` represents a single entity in the graph (e.g., a domain, an IP address, a person). Every node created by an enricher's `create_node()` method ends up as a `GraphNode` in the frontend.
+
+```typescript
+type GraphNode = {
+  id: string
+  nodeType: string
+  nodeLabel: string
+  nodeProperties: NodeProperties
+  nodeSize: number
+  nodeColor: string | null
+  nodeIcon: keyof typeof LucideIcons | null
+  nodeImage: string | null
+  nodeFlag: flagColor | null
+  nodeShape: NodeShape | null
+  nodeMetadata: NodeMetadata
+  x: number
+  y: number
+  val?: number
+  neighbors?: any[]
+  links?: any[]
+}
+```
+
+### Field reference
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `id` | `string` | Unique identifier for the node. Derived from the type's primary field value. |
+| `nodeType` | `string` | The entity type (e.g., `"Domain"`, `"Ip"`, `"Email"`). Maps to the Pydantic type class name. |
+| `nodeLabel` | `string` | Human-readable label displayed on the node. Set by the type's `compute_label()` method. |
+| `nodeProperties` | `NodeProperties` | All properties of the entity as key-value pairs. Contains the serialized fields from the Pydantic type. |
+| `nodeSize` | `number` | Visual size of the node in the graph. |
+| `nodeColor` | `string \| null` | Custom color override for the node (CSS color string), or `null` for default. |
+| `nodeIcon` | `keyof LucideIcons \| null` | Icon to display on the node, from the [Lucide icon set](https://lucide.dev/icons/), or `null` for default. |
+| `nodeImage` | `string \| null` | URL to an image to display on the node (e.g., a profile picture), or `null`. |
+| `nodeFlag` | `flagColor \| null` | Color flag for visual tagging (see Flag colors below), or `null`. |
+| `nodeShape` | `NodeShape \| null` | Shape of the node (see Node shapes below), or `null` for default circle. |
+| `nodeMetadata` | `NodeMetadata` | Additional metadata as key-value pairs. Used for internal tracking and extended features. |
+| `x` | `number` | X coordinate position in the graph canvas. |
+| `y` | `number` | Y coordinate position in the graph canvas. |
+| `val` | `number` (optional) | Value used by the force-graph engine for sizing calculations. |
+| `neighbors` | `any[]` (optional) | Adjacent nodes, populated by the graph engine at runtime. |
+| `links` | `any[]` (optional) | Connected edges, populated by the graph engine at runtime. |
+
+### NodeProperties
+
+```typescript
+type NodeProperties = {
+  [key: string]: any
+}
+```
+
+A flexible key-value object containing all the entity's properties. When a Pydantic type is serialized into a node, its fields become entries in `nodeProperties`. For example, a `Domain` node would have `nodeProperties.domain`, `nodeProperties.root`, etc.
+
+### NodeMetadata
+
+```typescript
+type NodeMetadata = {
+  [key: string]: any
+}
+```
+
+Similar to `nodeProperties` but used for system-level metadata rather than entity data. This can include information like creation timestamps, source enricher, or other tracking data.
+
+### Node shapes
+
+Nodes can be rendered in four shapes:
+
+```typescript
+type NodeShape = 'circle' | 'square' | 'hexagon' | 'triangle'
+```
+
+| Shape | Use case |
+|-------|----------|
+| `circle` | Default shape for most entity types |
+| `square` | Often used for infrastructure entities |
+| `hexagon` | Used for grouped or aggregate entities |
+| `triangle` | Used for alert or warning-related entities |
+
+### Flag colors
+
+Flags provide a visual tagging system for nodes. Users can flag nodes to highlight them during an investigation.
+
+```typescript
+type flagColor = 'red' | 'orange' | 'blue' | 'green' | 'yellow'
+```
+
+Each flag color maps to specific Tailwind CSS classes for consistent styling:
+
+| Flag | Style |
+|------|-------|
+| `red` | `text-red-400 fill-red-200` |
+| `orange` | `text-orange-400 fill-orange-200` |
+| `blue` | `text-blue-400 fill-blue-200` |
+| `green` | `text-green-400 fill-green-200` |
+| `yellow` | `text-yellow-400 fill-yellow-200` |
+
+## GraphEdge
+
+A `GraphEdge` represents a relationship between two nodes (e.g., "RESOLVES_TO", "HAS_SUBDOMAIN", "FOUND_IN_BREACH"). Every relationship created by an enricher's `create_relationship()` method ends up as a `GraphEdge` in the frontend.
+
+```typescript
+type GraphEdge = {
+  source: GraphNode['id']
+  target: GraphNode['id']
+  date?: string
+  id: string
+  label: string
+  caption?: string
+  type?: string
+  weight?: number
+  confidence_level?: number | string
+}
+```
+
+### Field reference
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `source` | `string` | ID of the source node (the "from" node in the relationship). |
+| `target` | `string` | ID of the target node (the "to" node in the relationship). |
+| `id` | `string` | Unique identifier for this edge. |
+| `label` | `string` | Display label for the edge (e.g., `"RESOLVES_TO"`, `"HAS_EMAIL"`). This is the relationship type string passed to `create_relationship()`. |
+| `date` | `string` (optional) | Timestamp associated with the relationship (ISO 8601 format). |
+| `caption` | `string` (optional) | Additional descriptive text displayed on or near the edge. |
+| `type` | `string` (optional) | Relationship classification type for filtering or styling. |
+| `weight` | `number` (optional) | Numerical weight of the relationship, can be used for visual thickness or importance ranking. |
+| `confidence_level` | `number \| string` (optional) | Confidence score for the relationship, indicating how reliable the connection is. |
+
+## How enrichers map to the graph
+
+When you write an enricher and call graph methods in `postprocess()`, here's how the data flows:
+
+### Creating nodes
+
+```python
+# In your enricher's postprocess method
+self.create_node(domain)  # domain is a Pydantic Domain object
+```
+
+The graph service automatically:
+1. Extracts the **type name** from the Pydantic class (e.g., `"Domain"`) -> `nodeType`
+2. Reads the **primary field** value (e.g., `domain.domain`) -> used for `id`
+3. Reads the **nodeLabel** field (set by `compute_label()`) -> `nodeLabel`
+4. Serializes all fields into `nodeProperties`
+5. Sets defaults for visual properties (`nodeSize`, `nodeColor`, etc.)
+
+### Creating relationships
+
+```python
+# In your enricher's postprocess method
+self.create_relationship(domain, ip, "RESOLVES_TO")
+```
+
+The graph service automatically:
+1. Identifies the **source node** from `domain`'s type and primary field -> `source`
+2. Identifies the **target node** from `ip`'s type and primary field -> `target`
+3. Uses the relationship label `"RESOLVES_TO"` -> `label`
+4. Generates a unique `id` for the edge
+
+### Relationship naming conventions
+
+Relationship labels follow these conventions:
+- Use **UPPER_SNAKE_CASE** (e.g., `"RESOLVES_TO"`, `"HAS_SUBDOMAIN"`)
+- Use **verb phrases** that describe the relationship direction (source -> target)
+- The default relationship label is `"IS_RELATED_TO"` if none is specified
+- Common patterns:
+  - `HAS_*` for ownership/containment (e.g., `"HAS_EMAIL"`, `"HAS_SUBDOMAIN"`)
+  - `RESOLVES_TO` for DNS-type lookups
+  - `FOUND_IN_*` for breach/leak associations
+  - `REGISTERED_BY` for WHOIS data
+
+## Graph settings
+
+The graph visualization is configurable through settings types:
+
+### ForceGraphSetting
+
+Controls the physics simulation of the force-directed graph layout:
+
+```typescript
+type ForceGraphSetting = {
+  value: any
+  min?: number
+  max?: number
+  step?: number
+  type?: string
+  description?: string
+}
+```
+
+### ExtendedSetting
+
+General-purpose settings with richer type information:
+
+```typescript
+type ExtendedSetting = {
+  value: any
+  type: string
+  min?: number
+  max?: number
+  step?: number
+  options?: { value: string; label: string }[]
+  description?: string
+}
+```
+
+These settings allow users to customize graph behavior like node repulsion, link distance, simulation speed, and visual preferences.
+
+## Reserved properties
+
+When creating nodes, certain property names are reserved by the system and should not be used as field names in your custom types:
+
+- `id` - Node identifier
+- `x`, `y` - Position coordinates
+- `nodeLabel` - Display label (set by `compute_label()`)
+- `label` - Legacy label field
+- `nodeType`, `type` - Entity type identifiers
+- `nodeImage`, `nodeIcon`, `nodeColor`, `nodeSize` - Visual properties
+- `created_at` - Creation timestamp
+- `sketch_id` - Investigation sketch association

docs/developers/managing-enrichers.mdx

@@ -0,0 +1,715 @@
+---
+title: "Managing enrichers"
+description: "Quick start guide to creating Enricher for your OSINT investigations."
+category: "Developers"
+order: 10
+author: "Flowsint Team"
+tags: ["tutorial", "developers", "creating-a-new-enricher"]
+version: "1.2.8"
+last_updated_at: "2026-05-15"
+---
+
+## Understanding Enrichers
+
+Enrichers are the high-level business logic layer in Flowsint. While types define data structures and tools wrap external utilities, enrichers orchestrate the entire intelligence gathering workflow. An enricher takes input data of one type, processes it through various tools or APIs, validates and enriches the results, creates graph database nodes and relationships, and returns structured output.
+
+Every enricher in Flowsint follows a two-phase execution model. The scan phase contains the core enriching logic where tools are executed, APIs are called, and data is gathered. Then, the postprocessing phase creates Neo4j graph nodes and relationships while returning the processed results. Input validation and normalization happens automatically through Pydantic type validation.
+
+Enrichers differ from tools in several fundamental ways. Tools are low-level wrappers that return raw data without knowledge of the Flowsint ecosystem. Enrichers are high-level workflows that understand types, create graph nodes, handle parameters, and orchestrate multiple tools. When you want to add a new data source, you create a tool. When you want to add a new intelligence workflow, you create an enricher.
+
+## Enricher architecture
+
+The enricher system is built around an abstract base class that defines the interface and execution flow. Every enricher you create inherits from this base class and implements specific methods.
+
+### The Enricher base class
+
+The base class lives at `flowsint-core/src/flowsint_core/core/enricher_base.py` and provides the framework for all enrichers. Here's what a minimal enricher looks like:
+
+```python
+from typing import List
+from flowsint_core.core.enricher_base import Enricher
+from flowsint_enrichers.registry import flowsint_enricher
+from flowsint_types import Domain, Ip
+
+@flowsint_enricher
+class MyEnricher(Enricher):
+    """Description of what this enricher does."""
+
+    # Define input and output types as base types (not lists)
+    InputType = Domain
+    OutputType = Ip
+
+    @classmethod
+    def name(cls) -> str:
+        """Unique identifier for this enricher."""
+        return "domain_to_ip"
+
+    @classmethod
+    def category(cls) -> str:
+        """Category this enricher belongs to."""
+        return "Domain"
+
+    @classmethod
+    def key(cls) -> str:
+        """Primary key field name for this enricher."""
+        return "domain"
+
+    async def scan(self, data: List[InputType]) -> List[OutputType]:
+        """Core enriching logic."""
+        pass
+
+    def postprocess(self, results: List[OutputType], input_data: List[InputType]) -> List[OutputType]:
+        """Create graph nodes and relationships."""
+        pass
+
+# Export types for easy access
+InputType = MyEnricher.InputType
+OutputType = MyEnricher.OutputType
+```
+
+The `InputType` and `OutputType` class attributes define what data the enricher accepts and returns. These should be Pydantic types from the `flowsint-types` package defined as base types (e.g., `Domain`, not `List[Domain]`). The base class uses these type definitions to automatically generate JSON schemas for the API and handle validation automatically.
+
+At the end of the file, you should export the types for easy access by other modules.
+
+### The two phases
+
+Understanding the two execution phases is crucial for writing effective enrichers.
+
+**Scanning** is where the real work happens. This async method receives validated input data as a list of `InputType` instances and executes your intelligence gathering logic. You might instantiate tools, call external APIs, process results, and build up your output data. This phase should focus purely on gathering and processing data without worrying about the graph database. The base class automatically handles input validation through Pydantic before the scan phase begins.
+
+**Postprocessing** creates the graph database structure. After scanning completes, this method receives both the results and the original input. It creates Neo4j nodes for each entity, establishes relationships between them, and returns the final results. This separation keeps graph logic separate from business logic.
+
+## Creating a simple enricher
+
+Let's walk through creating a complete enricher from scratch. We'll build an enricher that converts domains to IP addresses using DNS resolution.
+
+### Setting up the file structure
+
+Enrichers are organized by their input type. Create a new file in the appropriate directory under `flowsint-enrichers/src/flowsint_enrichers/`:
+
+```bash
+cd flowsint-enrichers/src/flowsint_enrichers/domain/
+touch to_ip.py
+```
+
+If you're creating an enricher for a new input type, you may need to create a new directory first.
+
+### Implementing the basic structure
+
+Start with the imports and class definition:
+
+```python
+import socket
+from typing import List
+from flowsint_enrichers.registry import flowsint_enricher
+from flowsint_core.core.enricher_base import Enricher
+from flowsint_core.core.logger import Logger
+from flowsint_types import Domain, Ip
+
+@flowsint_enricher
+class DomainToIpEnricher(Enricher):
+    """Resolves domain names to their IP addresses using DNS."""
+
+    # Define types as base types (not lists)
+    InputType = Domain
+    OutputType = Ip
+
+    @classmethod
+    def name(cls) -> str:
+        return "domain_to_ip"
+
+    @classmethod
+    def category(cls) -> str:
+        return "Domain"
+
+    @classmethod
+    def key(cls) -> str:
+        return "domain"
+
+    @classmethod
+    def documentation(cls) -> str:
+        return """
+        This enricher resolves domain names to their IP addresses using
+        standard DNS queries. It accepts a list of domains and returns
+        the corresponding IP addresses.
+        """
+
+# Export types at the end of the file
+InputType = DomainToIpEnricher.InputType
+OutputType = DomainToIpEnricher.OutputType
+```
+
+The `name()` method returns a unique identifier for this enricher. Use lowercase with underscores, following the pattern `inputtype_to_outputtype`. The `category()` groups related enrichers together in the UI. The `key()` specifies which field serves as the primary identifier, typically matching the input type.
+
+### Implementing the scan logic
+
+The scan method contains your core intelligence gathering logic. It receives a list of validated `InputType` instances and returns a list of `OutputType` instances.
+
+```python
+    async def scan(self, data: List[InputType]) -> List[OutputType]:
+        """
+        Resolve each domain to its IP address.
+
+        Args:
+            data: List of Domain objects to resolve
+
+        Returns:
+            List of Ip objects
+        """
+        results: List[OutputType] = []
+
+        for domain in data:
+            try:
+                # Perform DNS resolution
+                ip_address = socket.gethostbyname(domain.domain)
+                # Create IP object
+                ip = Ip(address=ip_address)
+                results.append(ip)
+                # Log successful resolution
+                Logger.info(
+                    self.sketch_id,
+                    {"message": f"Resolved {domain.domain} to {ip_address}"}
+                )
+            except socket.gaierror as e:
+                # DNS resolution failed
+                Logger.info(
+                    self.sketch_id,
+                    {"message": f"Failed to resolve {domain.domain}: {e}"}
+                )
+                continue
+            except Exception as e:
+                # Unexpected error
+                Logger.error(
+                    self.sketch_id,
+                    {"message": f"Error resolving {domain.domain}: {e}"}
+                )
+                continue
+        return results
+```
+
+This implementation iterates through each domain, performs DNS resolution, creates an IP object for successful resolutions, and logs both successes and failures. The error handling ensures that failures don't crash the entire enricher, which is important when processing many domains.
+
+The input data has already been validated by Pydantic before reaching the scan method, so you can trust that all items are proper `Domain` objects.
+
+### Implementing postprocessing
+
+The postprocess method creates graph database nodes and relationships using the new simplified API:
+
+```python
+    def postprocess(self, results: List[OutputType], input_data: List[InputType] = None) -> List[OutputType]:
+        """
+        Create graph nodes and relationships.
+
+        Args:
+            results: IP objects from scan phase
+            input_data: Original Domain objects (preprocessed input)
+
+        Returns:
+            IP objects (unchanged)
+        """
+        # Create nodes and relationships
+        for domain, ip in zip(input_data, results):
+            # Create nodes by passing Pydantic objects directly
+            self.create_node(domain)
+            self.create_node(ip)
+
+            # Create relationship by passing Pydantic objects directly
+            self.create_relationship(domain, ip, "RESOLVES_TO")
+
+            # Log the operation
+            self.log_graph_message(
+                f"IP found for domain {domain.domain} -> {ip.address}"
+            )
+
+        return results
+```
+
+You can pass Pydantic objects directly to `create_node()` and `create_relationship()`. The methods automatically infer the node types, primary keys, and property values from the Pydantic models.
+
+The `create_node()` method accepts a Pydantic object and automatically creates a Neo4j node with the correct label and properties. The `create_relationship()` method takes two Pydantic objects and a relationship type string, inferring all necessary information from the objects.
+
+## Creating an enricher with tools
+
+Most enrichers use external tools for data gathering. Let's create an enricher that uses the Subfinder tool for subdomain enumeration.
+
+### Importing the tool
+
+Start by importing the tool along with your other dependencies:
+
+```python
+from typing import List
+from flowsint_core.core.enricher_base import Enricher
+from flowsint_enrichers.registry import flowsint_enricher
+from flowsint_core.core.logger import Logger
+from flowsint_types import Domain
+from tools.network.subfinder import SubfinderTool
+
+@flowsint_enricher
+class SubdomainEnricher(Enricher):
+    """Enumerates subdomains for given domains using Subfinder."""
+
+    # Define types as base types
+    InputType = Domain
+    OutputType = Domain
+
+    @classmethod
+    def name(cls) -> str:
+        return "domain_to_subdomains"
+
+    @classmethod
+    def category(cls) -> str:
+        return "Domain"
+
+    @classmethod
+    def key(cls) -> str:
+        return "domain"
+
+# Export types
+InputType = SubdomainEnricher.InputType
+OutputType = SubdomainEnricher.OutputType
+```
+
+### Using the tool in scan
+
+The scan method instantiates and uses the tool:
+
+```python
+    async def scan(self, data: List[InputType]) -> List[OutputType]:
+        """
+        Find subdomains using Subfinder tool.
+        Args:
+            data: List of Domain objects
+        Returns:
+            List of discovered subdomain Domain objects
+        """
+        results: List[OutputType] = []
+        # Instantiate the tool
+        subfinder = SubfinderTool()
+
+        for domain in data:
+            Logger.info(
+                self.sketch_id,
+                {"message": f"Enumerating subdomains for {domain.domain}"}
+            )
+            try:
+                # Launch the tool
+                subdomains = subfinder.launch(domain.domain)
+                # Convert strings to Domain objects
+                for subdomain in subdomains:
+                    results.append(Domain(domain=subdomain, root=False))
+                Logger.info(
+                    self.sketch_id,
+                    {"message": f"Found {len(subdomains)} subdomains for {domain.domain}"}
+                )
+            except Exception as e:
+                Logger.error(
+                    self.sketch_id,
+                    {"message": f"Error enumerating subdomains for {domain.domain}: {e}"}
+                )
+                continue
+        return results
+```
+
+Notice how the tool returns raw strings, and the enricher converts them into proper Domain objects. This separation of concerns keeps tools simple while enrichers handle type conversion.
+
+### Creating graph nodes and relationships
+
+The postprocess phase creates parent-child relationships between domains and subdomains:
+
+```python
+    def postprocess(self, results: List[OutputType], input_data: List[InputType]) -> List[OutputType]:
+        """
+        Create graph nodes and relationships for domains and subdomains.
+
+        Args:
+            results: Discovered subdomain Domain objects
+            input_data: Original parent Domain objects
+
+        Returns:
+            Subdomain Domain objects
+        """
+        # Create nodes for parent domains
+        for domain in input_data:
+            self.create_node(domain)
+
+        # Create nodes for subdomains and relationships
+        for subdomain in results:
+            self.create_node(subdomain)
+
+            # Extract parent domain name and create relationship
+            parent_domain_name = self._extract_parent_domain(subdomain.domain)
+            parent_domain = Domain(domain=parent_domain_name)
+
+            # Create relationship using Pydantic objects
+            self.create_relationship(parent_domain, subdomain, "HAS_SUBDOMAIN")
+
+            # Log the operation
+            self.log_graph_message(
+                f"Subdomain found: {parent_domain_name} -> {subdomain.domain}"
+            )
+
+        return results
+
+    def _extract_parent_domain(self, subdomain: str) -> str:
+        """Extract parent domain from subdomain."""
+        parts = subdomain.split('.')
+        if len(parts) >= 2:
+            return '.'.join(parts[-2:])
+        return subdomain
+```
+
+## Adding parameters to enrichers
+
+Many enrichers need user-configurable parameters. Let's create an enricher that scans ports with configurable options.
+
+### Defining the parameter schema
+
+The `get_params_schema()` class method defines what parameters your enricher accepts:
+
+```python
+from typing import List, Dict, Any, Optional
+from flowsint_enrichers.registry import flowsint_enricher
+from flowsint_core.core.enricher_base import Enricher
+from flowsint_types import Ip, Port
+from tools.network.naabu import NaabuTool
+
+@flowsint_enricher
+class IpToPortsEnricher(Enricher):
+    """Scans IP addresses for open ports."""
+
+    # Define types as base types
+    InputType = Ip
+    OutputType = Port
+
+    @classmethod
+    def name(cls) -> str:
+        return "ip_to_ports"
+
+    @classmethod
+    def category(cls) -> str:
+        return "IP"
+
+    @classmethod
+    def key(cls) -> str:
+        return "address"
+
+    @classmethod
+    def get_params_schema(cls) -> List[Dict[str, Any]]:
+        """Define configurable parameters for this enricher."""
+        return [
+            {
+                "name": "mode",
+                "type": "select",
+                "description": "Scan mode: active scanning or passive enumeration",
+                "required": True,
+                "default": "passive",
+                "options": [
+                    {"label": "Passive", "value": "passive"},
+                    {"label": "Active", "value": "active"},
+                ],
+            },
+            {
+                "name": "port_range",
+                "type": "string",
+                "description": "Port range to scan (e.g., '1-1000' or '80,443,8080')",
+                "required": False,
+            },
+            {
+                "name": "top_ports",
+                "type": "select",
+                "description": "Scan only the most common ports",
+                "required": False,
+                "options": [
+                    {"label": "Top 100", "value": "100"},
+                    {"label": "Top 1000", "value": "1000"},
+                ],
+            },
+            {
+                "name": "PDCP_API_KEY",
+                "type": "vaultSecret",
+                "description": "ProjectDiscovery Cloud Platform API key for passive mode",
+                "required": False,
+            },
+        ]
+
+# Export types
+InputType = IpToPortsEnricher.InputType
+OutputType = IpToPortsEnricher.OutputType
+```
+
+The parameter schema defines the type, description, whether it's required, default values, and for select parameters, the available options. The `vaultSecret` type integrates with Flowsint's encrypted credential storage.
+
+### Using parameters in your enricher
+
+Parameters are accessed through `self.params` in your scan method:
+
+```python
+    async def scan(self, data: List[InputType]) -> List[OutputType]:
+        """
+        Scan IPs for open ports using configured parameters.
+        Args:
+            data: List of Ip objects to scan
+        Returns:
+            List of Port objects
+        """
+        results: List[OutputType] = []
+        # Extract parameters
+        mode = self.params.get("mode", "passive")
+        port_range = self.params.get("port_range")
+        top_ports = self.params.get("top_ports")
+        api_key = self.get_secret("PDCP_API_KEY")
+
+        # Instantiate tool
+        naabu = NaabuTool()
+
+        for ip in data:
+            Logger.info(
+                self.sketch_id,
+                {"message": f"Scanning {ip.address} in {mode} mode"}
+            )
+            try:
+                # Launch tool with parameters
+                scan_results = naabu.launch(
+                    target=ip.address,
+                    mode=mode,
+                    port_range=port_range,
+                    top_ports=top_ports,
+                    api_key=api_key
+                )
+                # Convert tool results to Port objects
+                for result in scan_results:
+                    port = Port(
+                        number=result.get("port"),
+                        protocol=result.get("protocol", "tcp").upper(),
+                        state="open",
+                        service=result.get("service"),
+                        banner=result.get("version")
+                    )
+                    results.append(port)
+            except Exception as e:
+                Logger.error(
+                    self.sketch_id,
+                    {"message": f"Error scanning {ip.address}: {e}"}
+                )
+                continue
+        return results
+```
+
+## Handling multiple output types
+
+Some enrichers produce multiple types of results. You can define a custom return type using Pydantic:
+
+```python
+from pydantic import BaseModel
+from flowsint_enrichers.registry import flowsint_enricher
+from flowsint_core.core.enricher_base import Enricher
+from typing import List
+from flowsint_types import Website, Email, Phone
+
+class CrawlerResults(BaseModel):
+    """Results from web crawler including multiple entity types."""
+    website: Website
+    emails: List[Email] = []
+    phones: List[Phone] = []
+
+@flowsint_enricher
+class WebsiteToCrawlerEnricher(Enricher):
+    """Crawls websites to extract emails and phone numbers."""
+
+    # Define types as base types
+    InputType = Website
+    OutputType = CrawlerResults
+
+    async def scan(self, data: List[InputType]) -> List[OutputType]:
+        """Crawl websites and extract contact information."""
+        from tools.network.reconcrawl import ReconCrawlTool
+
+        results: List[OutputType] = []
+        crawler_tool = ReconCrawlTool()
+
+        for website in data:
+            try:
+                # Launch crawler
+                crawl_data = crawler_tool.launch(website.url)
+
+                # Extract entities
+                emails = [Email(email=e) for e in crawl_data.get("emails", [])]
+                phones = [Phone(number=p) for p in crawl_data.get("phones", [])]
+
+                # Create result object
+                result = CrawlerResults(
+                    website=website,
+                    emails=emails,
+                    phones=phones
+                )
+                results.append(result)
+
+            except Exception as e:
+                Logger.error(self.sketch_id, {"message": f"Crawl error: {e}"})
+
+        return results
+
+    def postprocess(self, results: List[OutputType], input_data: List[InputType]) -> List[OutputType]:
+        """Create nodes for all discovered entities."""
+        for result in results:
+            # Create website node using Pydantic object
+            self.create_node(result.website)
+
+            # Create email nodes and relationships
+            for email in result.emails:
+                self.create_node(email)
+                self.create_relationship(result.website, email, "HAS_EMAIL")
+
+            # Create phone nodes and relationships
+            for phone in result.phones:
+                self.create_node(phone)
+                self.create_relationship(result.website, phone, "HAS_PHONE")
+
+            self.log_graph_message(
+                f"Processed {len(result.emails)} emails and {len(result.phones)} phones from {result.website.url}"
+            )
+
+        return results
+
+# Export types
+InputType = WebsiteToCrawlerEnricher.InputType
+OutputType = WebsiteToCrawlerEnricher.OutputType
+```
+
+## Registering your enricher
+
+You don't need to register your enricher anywhere, adding the decorator `@flowsint_enricher` to your enricher class triggers the auto discovery.
+
+```python
+from flowsint_enrichers.registry import flowsint_enricher
+from flowsint_core.core.enricher_base import Enricher
+
+@flowsint_enricher
+class MyEnricher(Enricher):
+...
+```
+
+## Testing your enricher
+
+Creating tests helps ensure your enricher works correctly and makes debugging easier. Create a test file in `flowsint-enrichers/tests/`:
+
+```python
+# tests/test_domain_to_ip.py
+import pytest
+from flowsint_enrichers.domain.to_ip import DomainToIpEnricher
+from flowsint_types import Domain, Ip
+
+@pytest.mark.asyncio
+async def test_enricher_metadata():
+    """Test enricher metadata is correctly defined."""
+    assert DomainToIpEnricher.name() == "domain_to_ip"
+    assert DomainToIpEnricher.category() == "Domain"
+    assert DomainToIpEnricher.key() == "domain"
+
+@pytest.mark.asyncio
+async def test_type_definitions():
+    """Test InputType and OutputType are correctly defined."""
+    assert DomainToIpEnricher.InputType == Domain
+    assert DomainToIpEnricher.OutputType == Ip
+
+@pytest.mark.asyncio
+async def test_scan():
+    """Test DNS resolution works."""
+    enricher = DomainToIpEnricher(sketch_id="test", scan_id="test")
+    input_data = [Domain(domain="example.com")]
+    results = await enricher.scan(input_data)
+
+    assert len(results) > 0
+    assert isinstance(results[0], Ip)
+    assert results[0].address  # Should have an IP address
+```
+
+These tests verify that your enricher's metadata is correct, type definitions are properly set, and the scan logic produces expected results. Input validation is handled automatically by Pydantic, so you don't need to test preprocessing separately.
+
+## Best practices
+
+When creating enrichers, think carefully about error handling. Intelligence gathering involves many external systems that can fail in unpredictable ways. Your enricher should handle errors gracefully, log failures clearly, and continue processing remaining items rather than crashing entirely.
+
+Always use the Logger utility for tracking progress and errors. The logger integrates with Flowsint's monitoring system and helps users understand what's happening during long-running enrichers. Log successful operations at the info level and errors at the error level.
+
+Define InputType and OutputType as base types (e.g., `Domain`, not `List[Domain]`). The base class automatically handles the list wrapping and validation. This makes the type definitions cleaner and more intuitive.
+
+Always export your types at the end of the file using:
+```python
+InputType = YourEnricher.InputType
+OutputType = YourEnricher.OutputType
+```
+
+Use the simplified graph API by passing Pydantic objects directly to `create_node()` and `create_relationship()`. This eliminates boilerplate code and reduces errors. The methods automatically infer node types, primary keys, and properties from your Pydantic models.
+
+Separate concerns between the two phases. Scanning should focus on gathering and processing data. Postprocessing should create graph structures. Input validation happens automatically through Pydantic, so you don't need to handle it manually.
+
+Use type hints everywhere. They provide automatic validation, better IDE support, and serve as inline documentation. The InputType and OutputType class attributes should always be Pydantic types from the flowsint-types package.
+
+When working with tools, remember that they return raw data structures. Your enricher is responsible for converting tool output into proper Flowsint types. This type conversion is typically done in the scan phase.
+
+Document your enricher thoroughly. The class docstring, documentation method, and parameter descriptions all appear in the UI. Clear documentation helps users understand what your enricher does and how to configure it.
+
+### Handling API Rate Limits
+
+When working with rate-limited APIs, add delays between requests:
+
+```python
+    async def scan(self, data: InputType) -> OutputType:
+        """Scan with rate limiting to respect API limits."""
+        import asyncio
+        results = []
+        delay_seconds = 1  # Delay between requests
+        for item in data:
+            result = await self._query_api(item)
+            if result:
+                results.append(result)
+            # Respect rate limits
+            await asyncio.sleep(delay_seconds)
+        return results
+```
+
+### Fallback data sources
+
+Implement fallback logic when primary sources fail:
+
+```python
+    async def scan(self, data: InputType) -> OutputType:
+        """Try multiple data sources with fallback logic."""
+        results = []
+
+        for domain in data:
+            # Try primary source
+            result = self._query_primary_source(domain)
+
+            if not result:
+                # Fall back to secondary source
+                Logger.info(
+                    self.sketch_id,
+                    {"message": f"Primary source failed for {domain}, trying fallback"}
+                )
+                result = self._query_fallback_source(domain)
+
+            if result:
+                results.append(result)
+
+        return results
+```
+
+## Troubleshooting
+
+If your enricher doesn't appear in the API, verify that you've applied the `@flowsint_enricher` decorator to your class, that the file lives under `flowsint-enrichers/src/flowsint_enrichers/`, and that you've restarted the API server. Auto-discovery via `load_all_enrichers()` runs at startup, so file additions require a restart.
+
+For import errors, make sure all your dependencies are installed and the enricher file has no syntax errors. Check that you're importing from the correct packages.
+
+If the scan method isn't finding any results, add logging statements to debug what's happening. Verify that tools are installed and accessible, API keys are valid if required, and input data is in the expected format.
+
+When graph relationships aren't appearing, check that you're creating both nodes and relationships in the postprocess method. Verify that the relationship type name is correct and that you're passing the right node objects to `create_relationship()`.
+
+## Next steps
+
+Once you've created and registered your enricher, it becomes available through the API for users to run. Enrichers can be chained together into Flows where the output of one enricher feeds into the input of another. This enables complex intelligence gathering sequences.
+
+Remember that enrichers are the heart of Flowsint's intelligence gathering capabilities. Well-designed enrichers that handle errors gracefully, log progress clearly, and create meaningful graph relationships make the entire platform more powerful and user-friendly.
+
+If you create new enrichers that you think can help the community, it's highly appreciated that you open-source them. Help the community as much as it helps you !

docs/developers/managing-tools.mdx

@@ -0,0 +1,512 @@
+---
+title: "Managing tools"
+description: "This guide explains how to create a new tool in the Flowsint ecosystem. Tools are low-level wrappers around external utilities, Docker containers, and APIs that enrichers use to gather intelligence. Understanding the tool architecture will help you extend Flowsint's capabilities with new data sources and reconnaissance utilities."
+category: "Developers"
+order: 9
+author: "Flowsint Team"
+tags: ["tutorial", "developers", "creating-a-new-tool"]
+version: "1.2.8"
+last_updated_at: "2026-05-15"
+---
+
+## Understanding tools
+
+Tools in Flowsint serve as abstraction layers between enrichers and external systems. They provide a consistent interface for executing Docker containers, calling APIs, or running python libraries. While enrichers handle high-level orchestration and graph database operations, tools focus exclusively on executing external commands and returning raw results.
+
+Every tool implements a basic interface with methods for naming, categorization, and execution. Tools don't know anything about Pydantic types, Neo4j graphs, or the broader Flowsint architecture. They just wrap external functionality and return data.
+
+Flowsint currently includes tools for subdomain enumeration, port scanning, DNS queries, WHOIS lookups, web crawling, and business intelligence.
+
+## Tool architecture
+
+The tool system has a two-tier inheritance structure. At the base level, you have the abstract `Tool` class that defines the interface every tool must implement. For tools that run in Docker containers, there's an intermediate `DockerTool` class that handles all the container lifecycle management.
+
+### The Tool base class
+
+Every tool inherits from the abstract `Tool` class, which lives at `flowsint-enrichers/src/tools/base.py`. Here's what it looks like:
+
+```python
+from abc import ABC, abstractmethod
+from typing import Any
+
+class Tool(ABC):
+    """Abstract base class for all tools."""
+
+    @classmethod
+    @abstractmethod
+    def name(cls) -> str:
+        """Return the tool name."""
+        pass
+
+    @classmethod
+    @abstractmethod
+    def category(cls) -> str:
+        """Return the tool category."""
+        pass
+
+    @classmethod
+    @abstractmethod
+    def description(cls) -> str:
+        """Return a description of what the tool does."""
+        pass
+
+    @classmethod
+    @abstractmethod
+    def version(cls) -> str:
+        """Return the tool version."""
+        pass
+
+    @abstractmethod
+    def launch(self, value: str, *args, **kwargs) -> Any:
+        """Execute the tool and return results."""
+        pass
+```
+
+Any tool you create must implement these five methods. The first four are class methods that provide metadata about the tool. The `launch` method is where the actual work happens.
+
+### The DockerTool class
+
+Most security and reconnaissance tools run in Docker containers for isolation and portability. The `DockerTool` class at `flowsint-enrichers/src/tools/dockertool.py` provides all the infrastructure for running containerized tools.
+
+When you inherit from `DockerTool`, you get automatic image management, container execution, volume mounting, environment variable handling, and cleanup. You just specify the Docker image name and implement how to construct the command.
+
+Here's a simplified view of what `DockerTool` provides:
+
+```python
+class DockerTool(Tool):
+    """Base class for tools that run in Docker containers."""
+
+    def __init__(self, image: str, default_tag: str = "latest"):
+        """Initialize with Docker image information."""
+        self.image = image
+        self.default_tag = default_tag
+        self.docker_client = docker.from_env()
+
+    def install(self) -> None:
+        """Pull the Docker image if not already present."""
+        # Pulls image from Docker Hub
+        pass
+
+    def is_installed(self) -> bool:
+        """Check if the Docker image exists locally."""
+        # Checks local images
+        pass
+
+    def launch(self, command: str, volumes: dict = None,
+               timeout: int = 30, environment: dict = None) -> Any:
+        """Run a command in the Docker container."""
+        # Executes container and returns output
+        pass
+```
+
+The `launch` method in `DockerTool` handles container execution. It sets up the environment, mounts volumes if needed, runs the container, captures output, and cleans up afterward.
+
+## Creating a simple API-based tool
+
+Let's start with the simpler case of creating a tool that calls an external API. We'll create a hypothetical tool for querying a threat intelligence service.
+
+### File structure
+
+Create a new python file in the appropriate category directory under `flowsint-enrichers/src/tools/`. For a security-related tool, you might use `tools/security/`:
+
+```bash
+cd flowsint-enrichers/src/tools/security/
+touch threat_intel.py
+```
+
+If the category directory doesn't exist, create it first and add an `__init__.py` file to make it a python package.
+
+### Basic implementation
+
+Here's a complete example of an API-based tool:
+
+```python
+from tools.base import Tool
+from typing import Any, Dict, List, Optional
+import requests
+
+class ThreatIntelTool(Tool):
+    """Query threat intelligence data from an external API."""
+
+    api_endpoint = "https://api.threatintel.example.com/v1"
+
+    @classmethod
+    def name(cls) -> str:
+        """Return the tool name."""
+        return "threatintel"
+
+    @classmethod
+    def category(cls) -> str:
+        """Return the category this tool belongs to."""
+        return "Threat Intelligence"
+
+    @classmethod
+    def description(cls) -> str:
+        """Return a description of what this tool does."""
+        return "Queries threat intelligence data for IPs, domains, and hashes"
+
+    @classmethod
+    def version(cls) -> str:
+        """Return the tool version."""
+        return "1.0.0"
+
+    def launch(
+        self,
+        indicator: str,
+        indicator_type: str = "ip",
+        api_key: Optional[str] = None
+    ) -> List[Dict[str, Any]]:
+        """
+        Query the threat intelligence API.
+
+        Args:
+            indicator: The indicator to query (IP, domain, hash, etc.)
+            indicator_type: Type of indicator (ip, domain, hash)
+            api_key: API key for authentication
+
+        Returns:
+            List of threat intelligence records
+        """
+        if not api_key:
+            raise ValueError("API key is required")
+
+        headers = {
+            "Authorization": f"Bearer {api_key}",
+            "Content-Type": "application/json"
+        }
+
+        params = {
+            "indicator": indicator,
+            "type": indicator_type
+        }
+
+        try:
+            response = requests.get(
+                f"{self.api_endpoint}/query",
+                headers=headers,
+                params=params,
+                timeout=30
+            )
+            response.raise_for_status()
+            return response.json().get("results", [])
+
+        except requests.exceptions.RequestException as e:
+            print(f"Error querying threat intel API: {e}")
+            return []
+```
+
+This tool follows a straightforward pattern. The class methods provide metadata that enrichers and the registry use. The `launch` method implements the actual API interaction, handling authentication, making the request, and returning structured data.
+
+Notice how the tool returns simple python data structures like lists and dictionaries. Tools don't know about Pydantic types or Flowsint models. That's the enricher's job.
+
+## Creating a docker-based tool
+
+Docker-based tools are more common in Flowsint because most reconnaissance utilities need specific dependencies and isolated environments. Let's walk through creating a tool that wraps a hypothetical Docker-based subdomain scanner.
+
+### Setting up the class
+
+Start by inheriting from `DockerTool` and providing the Docker image information:
+
+```python
+from tools.dockertool import DockerTool
+from typing import List, Optional, Any
+
+class MySubdomainTool(DockerTool):
+    """Wrapper for a Docker-based subdomain enumeration tool."""
+
+    image = "org/subdomain-scanner"
+    default_tag = "latest"
+
+    def __init__(self):
+        """Initialize the tool with Docker image information."""
+        super().__init__(self.image, self.default_tag)
+```
+
+The `image` and `default_tag` class attributes tell `DockerTool` which Docker image to use. When you instantiate the tool, it will automatically connect to the Docker daemon.
+
+### Implementing the launch method
+
+The `launch` method needs to construct the command that runs inside the container and handle the results:
+
+```python
+    def launch(
+        self,
+        domain: str,
+        timeout: int = 300,
+        wordlist: Optional[str] = None
+    ) -> List[str]:
+        """
+        Enumerate subdomains for a given domain.
+
+        Args:
+            domain: Target domain to enumerate
+            timeout: Maximum execution time in seconds
+            wordlist: Optional path to custom wordlist file
+
+        Returns:
+            List of discovered subdomain strings
+        """
+        # Ensure the Docker image is available
+        if not self.is_installed():
+            self.install()
+
+        # Build the command that runs inside the container
+        command = f"-d {domain}"
+
+        if wordlist:
+            command += f" -w {wordlist}"
+
+        # Add JSON output flag for easier parsing
+        command += " -json"
+
+        # Execute the container
+        try:
+            result = super().launch(
+                command=command,
+                timeout=timeout
+            )
+
+            # Parse the output
+            subdomains = self._parse_output(result)
+            return subdomains
+
+        except Exception as e:
+            print(f"Error running subdomain scanner: {e}")
+            return []
+
+    def _parse_output(self, output: str) -> List[str]:
+        """Parse the tool output and extract subdomains."""
+        import json
+
+        subdomains = []
+        for line in output.strip().split('\n'):
+            if not line:
+                continue
+            try:
+                data = json.loads(line)
+                if 'subdomain' in data:
+                    subdomains.append(data['subdomain'])
+            except json.JSONDecodeError:
+                continue
+
+        return list(set(subdomains))  # Remove duplicates
+```
+
+This implementation shows several important patterns. First, it checks if the Docker image is installed and pulls it if necessary. Second, it constructs the command string that will run inside the container. Third, it calls the parent class's `launch` method to handle the actual container execution. Finally, it parses the output into a clean python data structure.
+
+### Handling volumes
+
+Some tools need access to files on the host system. You can mount volumes when calling the parent's `launch` method:
+
+```python
+    def launch(self, domain: str, wordlist_path: str = None) -> List[str]:
+        """Run the tool with optional wordlist file."""
+
+        command = f"-d {domain}"
+        volumes = None
+
+        if wordlist_path:
+            # Mount the wordlist file into the container
+            volumes = {
+                wordlist_path: {
+                    'bind': '/wordlist.txt',
+                    'mode': 'ro'  # read-only
+                }
+            }
+            command += " -w /wordlist.txt"
+
+        result = super().launch(
+            command=command,
+            volumes=volumes
+        )
+
+        return self._parse_output(result)
+```
+
+The volumes dictionary maps host paths to container paths. You can specify the mount mode as 'ro' for read-only or 'rw' for read-write.
+
+### Using environment variables
+
+For tools that need API keys or configuration through environment variables:
+
+```python
+    def launch(self, domain: str, api_key: Optional[str] = None) -> List[str]:
+        """Run the tool with optional API key for enhanced scanning."""
+
+        command = f"-d {domain}"
+        environment = {}
+
+        if api_key:
+            environment['API_KEY'] = api_key
+
+        result = super().launch(
+            command=command,
+            environment=environment
+        )
+
+        return self._parse_output(result)
+```
+
+## Testing your tool
+
+Creating tests for your tool helps ensure it works correctly and makes it easier to catch regressions. Create a test file in `flowsint-enrichers/tests/tools/` that mirrors your tool's location:
+
+```python
+# tests/tools/security/test_threat_intel.py
+from tools.security.threat_intel import ThreatIntelTool
+import pytest
+
+def test_tool_metadata():
+    """Test that tool metadata is correctly defined."""
+    assert ThreatIntelTool.name() == "threatintel"
+    assert ThreatIntelTool.category() == "Threat Intelligence"
+    assert "threat" in ThreatIntelTool.description().lower()
+
+def test_tool_launch_requires_api_key():
+    """Test that launch method requires an API key."""
+    tool = ThreatIntelTool()
+    with pytest.raises(ValueError):
+        tool.launch("192.0.2.1")
+
+def test_tool_launch_with_api_key(monkeypatch):
+    """Test successful API query with mocked response."""
+    tool = ThreatIntelTool()
+
+    # Mock the requests.get call
+    def mock_get(*args, **kwargs):
+        class MockResponse:
+            def raise_for_status(self):
+                pass
+            def json(self):
+                return {"results": [{"indicator": "192.0.2.1", "threat_level": "high"}]}
+        return MockResponse()
+
+    monkeypatch.setattr("requests.get", mock_get)
+
+    results = tool.launch("192.0.2.1", api_key="test_key")
+    assert len(results) == 1
+    assert results[0]["indicator"] == "192.0.2.1"
+```
+
+For docker-based tools, your tests need Docker to be running:
+
+```python
+# tests/tools/network/test_my_subdomain_tool.py
+from tools.network.my_subdomain_tool import MySubdomainTool
+import pytest
+
+@pytest.mark.docker
+def test_tool_install():
+    """Test that the Docker image can be pulled."""
+    tool = MySubdomainTool()
+    tool.install()
+    assert tool.is_installed()
+
+@pytest.mark.docker
+def test_tool_launch():
+    """Test running the tool against a domain."""
+    tool = MySubdomainTool()
+    results = tool.launch("example.com")
+    assert isinstance(results, list)
+```
+
+The `@pytest.mark.docker` decorator helps you separate tests that require Docker from those that don't.
+
+## Best practices
+
+When creating tools, focus on simplicity and single responsibility. Each tool should wrap exactly one external utility or API. Don't try to combine multiple data sources in a single tool. That's what enrichers are for.
+
+Always handle errors gracefully. Network requests fail, Docker containers crash, and APIs return unexpected data. Your tool should catch these errors, log them appropriately, and return empty results or raise clear exceptions rather than crashing.
+
+Return simple data structures from the `launch` method. Use lists, dictionaries, strings, and numbers. Don't return Pydantic models or other complex objects. Remember that tools are low-level utilities that enrichers build upon.
+
+For Docker tools, always check if the image is installed before running it. The pattern of checking `is_installed()` and calling `install()` if necessary ensures the tool works even on fresh installations.
+
+When parsing tool output, be defensive. External tools can return unexpected formats, partial results, or garbage data. Validate and clean the output before returning it. Use try-except blocks around parsing logic.
+
+Document your tool thoroughly. The docstrings and parameter descriptions help other developers understand how to use your tool. Future enrichers will rely on this documentation.
+
+## Integrating your tool
+
+Unlike types, tools don't need to be explicitly registered in a central registry. Enrichers import and use them directly. When you create an enricher that uses your new tool, you simply import it:
+
+```python
+# In an enricher file
+from tools.security.threat_intel import ThreatIntelTool
+
+class IpToThreatIntelEnricher(Enricher):
+    async def scan(self, data: List[Ip]) -> List[ThreatReport]:
+        tool = ThreatIntelTool()
+        results = []
+
+        for ip in data:
+            intel = tool.launch(
+                indicator=ip.address,
+                indicator_type="ip",
+                api_key=api_key
+            )
+            # Process results...
+
+        return results
+```
+
+The enricher instantiates your tool, calls its `launch` method with appropriate parameters, and processes the results into Flowsint types.
+
+## Common patterns
+
+Several patterns appear frequently in Flowsint tools. Understanding these will help you write tools that fit naturally into the ecosystem.
+
+### The install-check pattern
+
+Most Docker tools follow this pattern at the start of `launch`:
+
+```python
+def launch(self, ...):
+    if not self.is_installed():
+        self.install()
+
+    # Continue with execution
+```
+
+This ensures the docker image is available before trying to run it.
+
+### The command builder pattern
+
+Complex tools often build commands incrementally based on parameters:
+
+```python
+def launch(self, target: str, mode: str = "fast", verbose: bool = False):
+    command = f"-target {target}"
+
+    if mode == "thorough":
+        command += " --thorough"
+
+    if verbose:
+        command += " -v"
+
+    result = super().launch(command)
+```
+
+### The output parser pattern
+
+Many tools separate execution from parsing:
+
+```python
+def launch(self, ...):
+    raw_output = super().launch(command)
+    return self._parse_output(raw_output)
+
+def _parse_output(self, output: str) -> List[Dict]:
+    """Parse raw tool output into structured data."""
+    # Parsing logic here
+```
+
+This separation makes the code easier to test and maintain.
+
+## Next steps
+
+Once you've created your tool and tested it, you can build enrichers that use it. Enrichers orchestrate one or more tools to gather intelligence, validate the results, convert them to Flowsint types, and create graph database nodes and relationships.
+
+If your tool requires API keys or other secrets, enrichers can access them through the vault system. When you implement an enricher that uses your tool, you can define parameters of type `vaultSecret` that pull credentials from the user's encrypted vault.
+
+Remember that tools are just one layer in the Flowsint architecture. They provide the raw capabilities, but enrichers provide the intelligence and graph-building logic that makes the platform powerful.

docs/getting-started/enrichers.mdx

@@ -0,0 +1,53 @@
+---
+title: "Enrichers"
+description: "Quick start guide to using Enrichers for your OSINT investigations."
+category: "Getting started"
+order: 4
+author: "Flowsint Team"
+tags: ["tutorial", "getting-started", "enrichers"]
+version: "1.2.8"
+last_updated_at: "2026-05-15"
+---
+
+### What is an Enricher?
+
+An enricher is an operation that, starting from an input element A (source entity), produces one or more elements B (target entities) by applying a search or correlation method called a pivot.
+
+Example:
+
+```bash
+A = my.domain.com (domain name)
+    ↓
+p = “DNS resolution” (pivot)
+    ↓
+B = 12.23.34.45 (IP address)
+```
+
+That said, a pivot is the method or technical process used to derive B from A. The pivot defines how the transformation obtains its result (e.g., DNS resolution, WHOIS lookup, API query, etc.).
+
+Example:
+
+```bash
+DNS Resolution → domain → IP
+    ↓
+WHOIS Lookup → IP → owner
+    ↓
+Reverse Image Search → image → web pages containing that image
+```
+
+Flowsint comes with a bunch of prebuilt enrichers, divided into multiple categories. Those enrichers can use standard pivots that your machine can support by default (DNS resolution, WHOIS request, etc.) and some other that depend on external tools. 
+
+Those can be : 
+
+- Native : DNS resolutions, Whois, etc.
+- Docker tools: [subfinder](https://github.com/projectdiscovery/subfinder), [asnmap](https://github.com/projectdiscovery/asnmap), etc.
+- Python tools: [sherlock](https://github.com/sherlock-project/sherlock), [maigret](https://github.com/soxoj/maigret), [reconurge/recontrack](https://github.com/reconurge/recontrack), [reconurge/reconcrawl](https://github.com/reconurge/reconcrawl), [reconurge/reconspread](https://github.com/reconurge/reconspread), etc. 
+- External services (paid or free): [shodan](https://www.shodan.io/), [whoxy](https://www.whoxy.com/), [whoisxmlapi](https://www.whoisxmlapi.com), etc.
+
+### Making your own enrichers
+
+Creating your own enrichers invloves multiple steps, but is not that trivial.
+
+If you plan on writting your own enrichers and think they could help the community, please contribute by making a pull request !
+
+Please refer to [this section](/docs/developers/managing-enrichers) to start building your own enrichers.

docs/getting-started/flows.mdx

@@ -0,0 +1,159 @@
+---
+title: "Flows"
+description: "Quick start guide to using Flows for your OSINT investigations."
+category: "Getting started"
+order: 5
+author: "Flowsint Team"
+tags: ["tutorial", "getting-started", "flows"]
+version: "1.2.8"
+last_updated_at: "2026-05-15"
+---
+
+### What are Flows?
+
+Flows are the chaining of multiple enrichers, where the output of one becomes the input of the next, allowing an investigation to be broadened or deepened.
+
+Some enrichers can be chained together, to that they can provide a reproductible and scalable research flow that can be re-applied to other entities of the same type.
+
+### Getting started with Flows
+
+The best way to get started with flows is to go to [http://localhost:5173/dashboard/flows](http://localhost:5173/dashboard/flows) and create a new flow.
+
+From that, you can start building your first flow; a flow consists of **one** input type, and multiple chained enrichers.
+
+Start by drag & dropping your input type to the canva and start using the "**+**" button to add more enrichers to your flow.
+
+Once you're satisfied with the flow, you can start viewing the execution order by pressing the "**compute**" button.
+
+Make sure you give it a descriptive name and save ! (`crtl + s` or presse the save button).
+
+Once it's done, go back to one of your sketches, right click on an item of the same type of the flow, and you should be able to launch it from the "flows" section.
+
+### Flow schema
+
+Every time a flow is launched, a log file is created at `flowsint_core/enricher_logs`.
+
+```json
+{
+  "sketch_id": "6aa808e4-1360-4c4a-b94f-4bed6c914836",
+  "scan_id": "52ba38a2-3f2c-4c8b-a7a5-42656f8fb845",
+  "created_at": "2025-10-26T15:57:52.243549",
+  "updated_at": "2025-10-26T15:57:52.424052",
+  "status": "completed",
+  "enricher_branches": [
+    {
+      "id": "branch-0",
+      "name": "Main Flow",
+      "steps": [
+        {
+          "nodeId": "Domain-1761469976169",
+          "params": {},
+          "type": "type",
+          "inputs": {},
+          "outputs": {
+            "domain": [
+              "example.com"
+            ]
+          },
+          "status": "pending",
+          "branchId": "branch-0",
+          "depth": 0
+        },
+        {
+          "nodeId": "domain_to_ip-1761469978018",
+          "params": {},
+          "type": "enricher",
+          "inputs": {
+            "Domain": null
+          },
+          "outputs": {
+            "address": "example.com",
+            "latitude": "example.com",
+            "longitude": "example.com",
+            "country": "example.com",
+            "city": "example.com",
+            "isp": "example.com"
+          },
+          "status": "pending",
+          "branchId": "branch-0",
+          "depth": 1
+        }
+      ]
+    }
+  ],
+  "execution_log": [
+    {
+      "step_id": "branch-0_domain_to_ip-1761469978018",
+      "branch_id": "branch-0",
+      "branch_name": "Main Flow",
+      "node_id": "domain_to_ip-1761469978018",
+      "enricher_name": "domain_to_ip",
+      "inputs": [
+        "example.com"
+      ],
+      "outputs": [
+        {
+          "address": "12.34.56.78",
+          "latitude": null,
+          "longitude": null,
+          "country": null,
+          "city": null,
+          "isp": null
+        }
+      ],
+      "status": "completed",
+      "error": null,
+      "timestamp": "2025-10-26T15:57:52.274147",
+      "execution_time_ms": 144,
+      "cache_hit": false
+    }
+  ],
+  "summary": {
+    "total_steps": 1,
+    "completed_steps": 1,
+    "failed_steps": 0,
+    "total_execution_time_ms": 144
+  },
+  "final_results": {
+    "initial_values": [
+      "example.com"
+    ],
+    "branches": [
+      {
+        "id": "branch-0",
+        "name": "Main Flow",
+        "steps": [
+          {
+            "nodeId": "domain_to_ip-1761469978018",
+            "enricher": "domain_to_ip",
+            "status": "completed",
+            "outputs": [
+              {
+                "address": "12.34.56.78",
+                "latitude": null,
+                "longitude": null,
+                "country": null,
+                "city": null,
+                "isp": null
+              }
+            ]
+          }
+        ]
+      }
+    ],
+    "results": {
+      "domain_to_ip-1761469978018": [
+        {
+          "address": "12.34.56.78",
+          "latitude": null,
+          "longitude": null,
+          "country": null,
+          "city": null,
+          "isp": null
+        }
+      ]
+    },
+    "reference_mapping": {}
+  }
+}
+```

docs/getting-started/quickstart.mdx

@@ -0,0 +1,42 @@
+---
+title: "Installation"
+description: "Quick start guide to using Flowsint for your OSINT investigations."
+category: "Getting started"
+order: 3
+author: "Flowsint Team"
+tags: ["tutorial", "quickstart", "installation"]
+version: "1.2.8"
+last_updated_at: "2026-05-15"
+---
+
+### Prerequisites
+
+Before installing Flowsint, ensure you have the following installed on your system:
+
+- **Docker** and **Docker Compose**
+- **Make** (for build automation)
+- **Git**
+
+### Installation
+
+Clone the repo and run the start command.
+
+```bash
+git clone https://github.com/reconurge/flowsint.git
+cd flowsint
+make prod
+```
+
+Some enrichers require API keys. Check out [this section](/docs/getting-started/enrichers#api-keys) to learn more.
+
+The application should automatically open at http://localhost:5173.
+
+### Create your first investigation
+
+Start by logging in or registering from the home page. From your dashboard, create a new investigation by clicking "New investigation." Add your first entity—such as a domain name—to the canvas. Right-click the entity to open the context menu and run an enricher. As results appear, explore the newly discovered entities and relationships directly in the graph.
+
+### Running your first enricher
+
+You could start by discovering subdomains for a domain for example.
+
+Add a domain entity like `example.com` to your investigation, then right-click the domain node and select the "domain_to_subdomains" enricher. Wait for the enricher to complete; newly discovered subdomains will be added to your graph for you to review.

docs/getting-started/vault.mdx

@@ -0,0 +1,58 @@
+---
+title: "Vault"
+description: "Quick start guide to using the Vault to secure your services API keys and secrets."
+category: "Getting started"
+order: 6
+author: "Flowsint Team"
+tags: ["tutorial", "getting-started", "vault"]
+version: "1.2.8"
+last_updated_at: "2026-05-15"
+---
+
+## What is the Vault
+
+A good amount of the tools you'll be using in Flowsint require third party API keys.
+
+The **Vault** (*"Coffre fort"* in french) is the place to centralize and securely store those API keys. 
+Weither you have a local instance of Flowsint or one fully deployed on a distributed system, you need to have your keys securely stored.
+
+## Adding a key
+
+In the Flowsint enricher ecosystem, the API keys follow a specific format, being in uppercase letters, and with a declarative name that follows `<service>_API_KEY`.
+
+## Current limitations
+
+For now, we cannot match a particular key from the Vault to an enricher **directly from the UI**. The enricher declares the API key variable name it requires, like the following in the core of the Enricher:
+
+```python
+@classmethod
+    def get_params_schema(cls) -> List[Dict[str, Any]]:
+        """Declare required parameters for this enricher"""
+        return [
+            {
+                "name": "PDCP_API_KEY",
+                "type": "vaultSecret",
+                "description": "The ProjectDiscovery Cloud Platform API key for asnmap.",
+                "required": True,
+            },
+        ]
+```
+
+This is a known limitation and we are working on improving this. 
+
+In the meanwhile, here is a list of the needed keys to run Flowsint at it's full potential:
+
+```bash
+# for enrichers
+WHOXY_API_KEY # Whoxy domain search engine [WHOXY]
+PDCP_API_KEY # ProjectDiscovery Cloud Platform [ASNMAP], [NAABU] etc
+HIBP_API_KEY # HaveIBeenPwned API key [HIBP]
+ETHERSCAN_API_KEY # Etherscan crypto API key [ETHERSCAN]
+# for Flo, AI assistant
+MISTRAL_API_KEY
+# but other providers will be supported soon (ChatGPT, etc.)
+```
+
+There are also some other tools that could need a bunch of other API keys like [Subfinder](https://github.com/projectdiscovery/subfinder). Configuring them is not possible for now, but will be soon.
+
+Stay tuned for updates as those mechanisms may vary in the future, as the goal is to keep the user experience as smooth as possible.

docs/overview.mdx

@@ -0,0 +1,59 @@
+---
+title: "Welcome to Flowsint"
+description: "Complete documentation for Flowsint - a modular OSINT investigation platform."
+category: "Overview"
+order: 1
+author: "Flowsint Team"
+tags: ["documentation", "overview", "getting-started"]
+version: "1.2.8"
+last_updated_at: "2026-05-15"
+---
+
+### What is Flowsint?
+
+Flowsint is a modular investigation and reconnaissance platform focused on OSINT (Open Source Intelligence). It provides:
+
+- Graph-based visualization of entity relationships
+- 30+ automated enrichers for intelligence gathering
+- Modular architecture with clean separation of concerns
+- Privacy-first design with local data storage
+- Extensible platform for custom enrichers
+- Automated search flows (more on that [here](/docs/getting-started/flows))
+
+<Alert variant="warning">
+    <AlertTitle>Disclaimer !</AlertTitle>
+    <AlertDescription>
+    The author(s) and contributor(s) of this tool assume no responsibility or liability for any damages, losses, or consequences that may result from the use or misuse of this software.
+    By using this tool, you acknowledge and agree that:
+        - You are solely responsible for your use of this software
+        - You will use this tool in compliance with all applicable laws and regulations
+        - You will obtain proper authorization before conducting any security testing or reconnaissance activities
+        - You understand the potential risks and legal implications of using security tools
+</AlertDescription>
+  </Alert>
+
+### Why Flowsint?
+
+If you've already practiced some OSINT, you know that analysts often rely on a multitude of research tools: scripts, third-party services, specialized applications. But these tools often work **in silos** and quickly become obsolete if they're not maintained: a service disappears, an API changes, an access point closes. The analyst juggles with unstable tools and sometimes has to **manually adapt their data** to continue their investigation.
+
+OSINT tools, for most, are **consumables**: they evolve, appear, and disappear. Research methods change, security mechanisms evolve. However, the fundamental need always remains the same: **to see, exploit, and analyze data in a clear and understandable way**. Analysts need to be able to list, centralize, and visualize connections to have a clear understanding of their investigation.
+
+This is exactly what Flowsint was built for: a solid and durable foundation on which your investigations rest. Tools become simple extensions that plug in and unplug easily. A new tool comes out, and with a few manipulations it can be integrated into your investigation workflow.
+
+**Flowsint is the stable infrastructure that allows you to stay agile in the face of constant evolution of methods and sources.**
+
+Flowsint is a local tool, running on your machine only. This ensures a great level of confidentiality, but comes with responsabilities.
+
+<Alert variant="warning">
+    <AlertTitle>At your own risk</AlertTitle>
+    <AlertDescription>
+    All enrichers run locally on your machine. This means you can get banned from some services or get flagged from infrastructures if you start making thousands of requests to the same service.
+    You need to always know what you are doing, and understand that gathering can go very fast in scale.
+
+    For example, you can very easily happen to be making DNS resolution requests for 10 000 IPs.
+
+    Know you infrastructure and your limits. Know how the tools used in the enrichers actually work. You can have a list of the available enrichers and tools [here](/docs/sources/available-enrichers).
+</AlertDescription>
+  </Alert>
+
+If all those points are clear for you, let's start investigating !

docs/sources/available-enrichers.mdx

@@ -0,0 +1,163 @@
+---
+title: "Enrichers catalog"
+description: "Quick start guide to using Enrichers for your OSINT investigations."
+category: "Sources"
+order: 11
+author: "Flowsint Team"
+tags: ["tutorial", "getting-started", "enrichers"]
+version: "1.2.8"
+last_updated_at: "2026-05-15"
+---
+
+### ASN
+**asn_to_cidrs**: Given an ASN, enumerate its announced CIDR ranges.
+Tools/Pivots: [asnmap](https://github.com/projectdiscovery/asnmap) (CLI), [jq](https://jqlang.github.io/jq/) (CLI)
+
+### CIDR
+**cidr_to_ips**: Expand a CIDR to IPs by PTR enumeration heuristics.
+Tools/Pivots: [dnsx](https://github.com/projectdiscovery/dnsx) (CLI)
+
+### Crypto
+**cryptowallet_to_transactions**: Fetch ETH wallet transactions and map wallet-to-wallet relationships.
+Tools/APIs: [Etherscan API](https://docs.etherscan.io/)
+
+**cryptowallet_to_nfts**: Fetch ERC-721/1155 NFT transfers for a wallet.
+Tools/APIs: [Etherscan API](https://docs.etherscan.io/)
+
+### Domain
+**domain_to_ip**: Resolve domains to IPv4 addresses.
+Tools/Pivots: DNS resolution (socket)
+
+**domain_to_subdomains**: Discover subdomains for a domain.
+Tools/APIs: [subfinder](https://github.com/projectdiscovery/subfinder) (CLI), fallback to [crt.sh JSON API](https://crt.sh/?output=json)
+
+**domain_to_whois**: Retrieve WHOIS registration data for a domain.
+Tools/APIs: [python-whois](https://pypi.org/project/python-whois/)
+
+**domain_to_asn**: Map a domain to its ASN by resolving and querying ASN data.
+Tools/Pivots: system DNS, [asnmap](https://github.com/projectdiscovery/asnmap) (CLI)
+
+**domain_to_root_domain**: Convert a subdomain to its registrable root.
+Tools/Pivots: internal domain utils
+
+**domain_to_history**: Retrieve historical WHOIS records and extract related entities (individuals, organizations, emails, phones, locations).
+Tools/APIs: [Whoxy API](https://www.whoxy.com/api/)
+
+**domain_to_website**: Convert a domain to a reachable website URL (HTTP/HTTPS), following redirects.
+Tools/Pivots: HTTP HEAD requests
+
+**domain_to_tls**: Retrieve TLS/SSL certificate information for a domain.
+Tools/Pivots: [httpx](https://github.com/projectdiscovery/httpx) (CLI)
+
+**domain_to_whois_history**: Retrieve historical WHOIS records for a domain and extract related entities (individuals, organizations, emails, locations).
+Tools/APIs: [WhoisXML API](https://whois.whoisxmlapi.com/)
+
+**domain_to_dehashed**: Get breach intelligence (credentials, related individuals) associated with a domain.
+Tools/APIs: [DeHashed API](https://www.dehashed.com/docs)
+
+### Email
+**email_to_breaches**: Check whether an email appears in known breaches.
+Tools/APIs: [Have I Been Pwned API](https://haveibeenpwned.com/API/v3)
+
+**email_to_gravatar**: Check Gravatar existence and profile for an email (via MD5 hash).
+Tools/APIs: [Gravatar endpoints](https://en.gravatar.com/site/implement/images/)
+
+**email_to_domain**: Extract the domain part of an email address.
+Tools/Pivots: internal email parser
+
+**email_to_domains**: Find domains registered by a given email address; extract related contacts and entities.
+Tools/APIs: [Whoxy API](https://www.whoxy.com/api/)
+
+**email_to_username**: Extract the local-part of an email as a Username entity.
+Tools/Pivots: internal email parser
+
+**email_to_intelligence**: Get breach intelligence (credentials, related individuals) associated with an email.
+Tools/APIs: [DeHashed API](https://www.dehashed.com/docs)
+
+**email_to_device_hudsonrock**: Look up devices compromised by infostealers and associated with an email.
+Tools/APIs: [HudsonRock API](https://www.hudsonrock.com/)
+
+### Individual
+**individual_to_domains**: Find domains registered by a specific person; extract related contacts and attributes.
+Tools/APIs: [Whoxy API](https://www.whoxy.com/api/)
+
+**individual_to_organization**: Find organizations related to a person in French registries.
+Tools/APIs: SIRENE (via internal SireneTool) — see [INSEE Sirene API](https://api.insee.fr/catalogue/#/datasets/sirene)
+
+### IP
+**ip_to_domain**: Reverse-resolve IPs to domains via PTR and Certificate Transparency pivots.
+Tools/APIs: DNS PTR (socket), [crt.sh JSON API](https://crt.sh/?output=json)
+
+**ip_to_infos**: Enrich IPs with geolocation and ISP data.
+Tools/APIs: [ip-api.com](https://ip-api.com/)
+
+**ip_to_asn**: Map IPs to their ASN.
+Tools/Pivots: AsnmapTool ([asnmap](https://github.com/projectdiscovery/asnmap))
+
+**ip_to_ports**: Scan an IP for open ports and services.
+Tools/Pivots: [naabu](https://github.com/projectdiscovery/naabu) (CLI)
+
+**ip_to_fraudscore**: Compute a fraud risk score for an IP address.
+Tools/APIs: [Scamalytics API](https://scamalytics.com/ip-api)
+
+**ip_to_intelligence**: Get breach intelligence (credentials, related individuals) associated with an IP.
+Tools/APIs: [DeHashed API](https://www.dehashed.com/docs)
+
+### Organization
+**org_to_domains**: Find domains registered by an organization; extract contacts and related entities.
+Tools/APIs: [Whoxy API](https://www.whoxy.com/api/)
+
+**org_to_infos**: Enrich organizations with French registry data and leaders.
+Tools/APIs: SIRENE (SireneTool) — see [INSEE Sirene API](https://api.insee.fr/catalogue/#/datasets/sirene)
+
+**org_to_asn**: Find ASNs associated with an organization name.
+Tools/Pivots: [asnmap](https://github.com/projectdiscovery/asnmap) (CLI), [jq](https://jqlang.github.io/jq/) (CLI)
+
+### Phone
+**phone_to_infos**: Probe phone footprint across services (demo modules) and normalize number.
+Tools/APIs: ignorant modules (Amazon, Snapchat, Instagram), [httpx](https://github.com/projectdiscovery/httpx)
+
+**phone_to_carrier**: Look up carrier, country, and validity metadata for a phone number.
+Tools/APIs: [Veriphone API](https://veriphone.io/)
+
+**phone_to_device_hudsonrock**: Look up devices compromised by infostealers and associated with a phone number.
+Tools/APIs: [HudsonRock API](https://www.hudsonrock.com/)
+
+### Social
+**username_to_socials_sherlock**: Enumerate social accounts for a username using Sherlock.
+Tools/Pivots: [sherlock](https://github.com/sherlock-project/sherlock) (CLI)
+
+**username_to_socials_maigret**: Enumerate social accounts for a username using Maigret and parse rich metadata.
+Tools/Pivots: [maigret](https://github.com/soxoj/maigret) (CLI)
+
+**username_to_dehashed**: Get breach intelligence (credentials, related individuals) associated with a username.
+Tools/APIs: [DeHashed API](https://www.dehashed.com/docs)
+
+**username_to_device_hudsonrock**: Look up devices compromised by infostealers and associated with a username.
+Tools/APIs: [HudsonRock API](https://www.hudsonrock.com/)
+
+### Website
+**website_to_crawler**: Crawl a website to extract emails and phone numbers.
+Tools/APIs: ReconCrawlTool (`reconcrawl`)
+
+**website_to_domain**: Extract the domain name from a website URL.
+Tools/Pivots: internal URL parser
+
+**website_to_subdomains**: Find subdomains of a website's domain via external scan.
+Tools/APIs: [c99.nl API](https://api.c99.nl/)
+
+**website_to_links**: Crawl a website and collect internal/external links and domains.
+Tools/APIs: reconspread Crawler
+
+**website_to_text**: Fetch and extract visible text from a webpage.
+Tools/APIs: HTTP GET, [BeautifulSoup](https://www.crummy.com/software/BeautifulSoup/bs4/doc/)
+
+**website_to_webtrackers**: Extract analytics/ads tracking codes from a website.
+Tools/APIs: recontrack TrackingCodeExtractor
+
+---
+
+Notes
+- Some enrichers optionally depend on docker binaries: `subfinder`, `asnmap`, `dnsx`, `naabu`, `httpx`, and `jq` which are installed in the docker container.
+- API-keyed enrichers read keys from params or environment (e.g., `HIBP_API_KEY`, `ETHERSCAN_API_KEY`, `WHOXY_API_KEY`, `WHOISXML_API_KEY`, `DEHASHED_API_KEY`, `SCAMALYTICS_API_KEY`, `VERIPHONE_API_KEY`, `C99_API_KEY`).
+- Internal/test enrichers (`domain_to_dummy`, `ip_to_dummy_domains`, `n8n_connector`) are not listed here — they exist in the codebase but are not part of the public catalog.

docs/syllabus.mdx

@@ -0,0 +1,73 @@
+---
+title: "Syllabus"
+description: "Syllabus, just to make sure we speak the same language. Those definitions apply in the context of Flowsint platform."
+category: "Overview"
+order: 2
+author: "Flowsint Team"
+tags: ["documentation", "overview",  "syllabus"]
+version: "1.2.8"
+last_updated_at: "2026-05-15"
+---
+
+### OSINT
+
+      Open Source Intelligence consists of collecting, analyzing, and exploiting **freely** and **openly** available information from search engines, images, social networks, public archives, etc.
+
+### Investigation
+
+    A structured process aimed at collecting, correlating, and analyzing information from different sources and enrichers, in order to answer a question or solve a problem. An investigation can be **exploratory** (discovering unknown elements) or **targeted** (validating a hypothesis). An investigation can contain multiple **sketches** (each representing a different view or stage of the analysis) and one or more **analyses**.
+
+### Sketch
+
+    Visual result produced by executing one or more enrichers on one or more entities. A sketch represents the current state of the graph derived from collected data at a given moment in the investigation. Multiple sketches can exist for the same investigation to capture different perspectives or stages.
+
+### Analysis
+
+    Set of processing, interpretations, and verifications performed on data collected during the investigation. Analyses aim to identify trends, confirm or refute hypotheses, and produce actionable conclusions. They can be **quantitative** (measurements, statistics) or **qualitative** (contextual assessments, behavioral patterns).
+
+### Enricher
+
+    An **enricher** is an operation that, from an input element **A** (*source entity*), allows obtaining one or more elements **B** (*target entities*) by applying a search or correlation method called a **pivot**.
+
+    > Example:
+    >
+    >
+    > A = `my.domain.com` (*domain name*)
+    >
+    > p = "DNS resolution" (*pivot*)
+    >
+    > B = `12.23.34.45` (*IP address*).
+    >
+### Pivot
+
+    A **pivot** is the method or technical process used to derive **B** from **A**. The pivot defines **how** the enricher obtains its result (e.g., DNS resolution, WHOIS lookup, API query, etc.).
+
+    > Examples of pivots:
+    >
+    > DNS Resolution → domain → IP
+    > WHOIS Lookup → IP → owner
+    > Reverse Image Search → image → web pages containing this image
+
+### Tool
+
+    A tool generally refers to a script, program, or service providing a **pivot**, i.e., a means to retrieve or enricher information from an input element.
+
+### Entity
+
+    An identifiable object or element manipulated by enrichers (e.g., IP address, domain, email address, user identifier, file hash, etc.). An entity is always associated with a **Sketch**. In the graph, entities are represented as **nodes** (see [Graph format](/docs/developers/graph-format) for technical details).
+
+### Relationship
+
+    Defines a link between two entities. This link is generally named (in uppercase) and can be unidirectional or bidirectional.
+
+    > Examples of relationships:
+    >
+    >
+    > A = `my.domain.com` → `RESOLVES_TO` → `12.23.34.45`
+    >
+
+    A relationship is always associated between a **source** node (*from*) and a **target** node (*to*). In the graph, relationships are represented as **edges** (see [Graph format](/docs/developers/graph-format) for technical details).
+
+### Flow
+
+    The chaining of multiple enrichers, where the output of one becomes the input of the next, allowing to expand or deepen an investigation.

flowsint-api/Dockerfile

@@ -110,7 +110,7 @@ WORKDIR /app/flowsint-api
 RUN chmod +x entrypoint.sh
 
 # Switch to non-root user
-# USER flowsint
+USER flowsint
 
 EXPOSE 5001
 
@@ -120,4 +120,4 @@ HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
 ENTRYPOINT ["./entrypoint.sh"]
 
 # Production command (no reload)
-CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "5001"]
+CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "5001"]

flowsint-api/app/api/deps.py

@@ -1,19 +1,10 @@
-from fastapi import Depends, HTTPException, status, Request
+from fastapi import Depends, HTTPException, status
 from fastapi.security import OAuth2PasswordBearer
 from jose import JWTError, jwt
 from sqlalchemy.orm import Session
-from flowsint_core.core.auth import ALGORITHM
+from flowsint_core.core.auth import ALGORITHM, AUTH_SECRET
 from flowsint_core.core.postgre_db import get_db
 from flowsint_core.core.models import Profile
-from typing import Optional
-
-import os
-from dotenv import load_dotenv
-
-# Remplace avec ton URL de BDD
-AUTH_SECRET = os.getenv("AUTH_SECRET")
-
-load_dotenv()
 
 oauth2_scheme = OAuth2PasswordBearer(tokenUrl="token")
 
@@ -37,42 +28,3 @@ def get_current_user(
     if user is None:
         raise credentials_exception
     return user
-
-
-def get_current_user_sse(
-    request: Request, db: Session = Depends(get_db)
-) -> Profile:
-    """
-    Alternative authentication for SSE endpoints that accepts token via query parameter.
-    EventSource API doesn't support custom headers, so we need to pass the token in the URL.
-    """
-    credentials_exception = HTTPException(
-        status_code=status.HTTP_401_UNAUTHORIZED,
-        detail="Could not validate credentials",
-    )
-
-    # Try to get token from query parameter
-    token: Optional[str] = request.query_params.get("token")
-
-    # Fallback to Authorization header if query param not present
-    if not token:
-        auth_header = request.headers.get("Authorization")
-        if auth_header and auth_header.startswith("Bearer "):
-            token = auth_header.replace("Bearer ", "")
-
-    if not token:
-        raise credentials_exception
-
-    try:
-        payload = jwt.decode(token, AUTH_SECRET, algorithms=[ALGORITHM])
-        email: str = payload.get("sub")
-        if email is None:
-            raise credentials_exception
-    except JWTError:
-        raise credentials_exception
-
-    user = db.query(Profile).filter(Profile.email == email).first()
-    if user is None:
-        raise credentials_exception
-
-    return user

flowsint-api/app/api/routes/events.py

@@ -14,7 +14,7 @@ from flowsint_core.core.services import (
     PermissionDeniedError,
     DatabaseError,
 )
-from app.api.deps import get_current_user, get_current_user_sse
+from app.api.deps import get_current_user
 
 router = APIRouter()
 
@@ -42,7 +42,7 @@ async def stream_events(
     request: Request,
     sketch_id: str,
     db: Session = Depends(get_db),
-    current_user: Profile = Depends(get_current_user_sse),
+    current_user: Profile = Depends(get_current_user),
 ):
     """Stream events for a specific sketch in real-time."""
     service = create_log_service(db)
@@ -58,7 +58,7 @@ async def stream_events(
         channel = sketch_id
         await event_emitter.subscribe(channel)
         try:
-            yield 'data: {"event": "connected", "data": "Connected to log stream"}\n\n'
+            yield json.dumps({"event": "connected", "data": "Connected to log stream"})
             while True:
                 if await request.is_disconnected():
                     break
@@ -115,7 +115,7 @@ async def stream_sketch_status(
     request: Request,
     sketch_id: str,
     db: Session = Depends(get_db),
-    current_user: Profile = Depends(get_current_user_sse),
+    current_user: Profile = Depends(get_current_user),
 ):
     """Stream COMPLETED events for a specific sketch (for graph refresh)."""
     service = create_log_service(db)
@@ -160,49 +160,3 @@ async def stream_sketch_status(
             "X-Accel-Buffering": "no",
         },
     )
-
-
-@router.get("/status/scan/{scan_id}/stream")
-async def stream_status(
-    request: Request,
-    scan_id: str,
-    db: Session = Depends(get_db),
-    current_user: Profile = Depends(get_current_user_sse),
-):
-    """Stream status updates for a specific scan in real-time."""
-    service = create_log_service(db)
-    try:
-        service.get_scan_with_permission(scan_id, current_user.id)
-    except NotFoundError as e:
-        raise HTTPException(status_code=404, detail=str(e))
-    except PermissionDeniedError:
-        raise HTTPException(status_code=403, detail="Forbidden")
-
-    async def status_generator():
-        print("[EventEmitter] Start status generator")
-        await event_emitter.subscribe(f"scan_{scan_id}_status")
-        try:
-            yield 'data: {"event": "connected", "data": "Connected to status stream"}\n\n'
-
-            while True:
-                data = await event_emitter.get_message(f"scan_{scan_id}_status")
-                if data is None:
-                    await asyncio.sleep(0.1)
-                    continue
-                print(f"[EventEmitter] Received status data: {data}")
-                yield f"data: {data}\n\n"
-
-        except asyncio.CancelledError:
-            print(f"[EventEmitter] Client disconnected from status stream for scan_id: {scan_id}")
-        finally:
-            await event_emitter.unsubscribe(f"scan_{scan_id}_status")
-
-    return EventSourceResponse(
-        status_generator(),
-        media_type="text/event-stream",
-        headers={
-            "Cache-Control": "no-cache",
-            "Connection": "keep-alive",
-            "X-Accel-Buffering": "no",
-        },
-    )

flowsint-api/app/api/schemas/scan.py

@@ -8,7 +8,7 @@ class ScanCreate(BaseModel):
     values: Optional[List[str]] = None
     sketch_id: Optional[UUID4] = None
     status: Optional[str] = None
-    results: Optional[Any] = None
+    details: Optional[Any] = None
 
 
 class ScanRead(ORMBase):

flowsint-api/app/main.py

@@ -1,3 +1,5 @@
+import os
+
 from fastapi import FastAPI
 from fastapi.middleware.cors import CORSMiddleware
 
@@ -16,8 +18,12 @@ from app.api.routes import types
 from app.api.routes import custom_types
 from app.api.routes import enricher_templates
 
+# Comma-separated list of allowed origins, e.g. "https://app.example.com,https://staging.example.com"
+# Falls back to localhost dev origin when unset. Never use "*" with allow_credentials=True.
 origins = [
-    "*",
+    o.strip()
+    for o in os.getenv("ALLOWED_ORIGINS", "http://localhost:5173").split(",")
+    if o.strip()
 ]
 
 

flowsint-api/tests/conftest.py

@@ -0,0 +1,38 @@
+"""Pytest harness for flowsint-api. Provides a TestClient backed by an
+in-memory SQLite database, overriding the real Postgres get_db dependency."""
+
+import os
+
+# Env required at import time by flowsint-core modules. Set before importing app.
+os.environ.setdefault("AUTH_SECRET", "test-secret-please-ignore")
+os.environ.setdefault(
+    "MASTER_VAULT_KEY_V1", "base64:qnHTmwYb+uoygIw9MsRMY22vS5YPchY+QOi/E79GAvM="
+)
+
+import pytest
+from fastapi.testclient import TestClient
+from sqlalchemy import create_engine
+from sqlalchemy.orm import sessionmaker
+
+from flowsint_core.core.models import Base
+from flowsint_core.core.postgre_db import get_db
+from app.main import app
+
+
+@pytest.fixture
+def db_session():
+    engine = create_engine(
+        "sqlite:///:memory:", connect_args={"check_same_thread": False}
+    )
+    Base.metadata.create_all(engine)
+    Session = sessionmaker(bind=engine)
+    session = Session()
+    yield session
+    session.close()
+
+
+@pytest.fixture
+def client(db_session):
+    app.dependency_overrides[get_db] = lambda: db_session
+    yield TestClient(app)
+    app.dependency_overrides.clear()

flowsint-api/tests/test_events_auth.py

@@ -0,0 +1,50 @@
+"""Auth and shape contract for the SSE event endpoints."""
+
+import importlib
+
+import pytest
+
+from flowsint_core.core.auth import create_access_token
+from flowsint_core.core.models import Profile
+
+
+def test_get_current_user_sse_is_removed():
+    """The query-param SSE auth helper must no longer exist."""
+    deps = importlib.import_module("app.api.deps")
+    assert not hasattr(deps, "get_current_user_sse")
+
+
+def test_log_stream_requires_auth_header(client):
+    """No Authorization header -> 401 (no token in URL accepted)."""
+    res = client.get("/api/events/sketch/abc/stream")
+    assert res.status_code == 401
+
+
+def test_status_stream_requires_auth_header(client):
+    res = client.get("/api/events/sketch/abc/status/stream")
+    assert res.status_code == 401
+
+
+def test_log_stream_rejects_token_query_param(client):
+    """A token in the URL must NOT authenticate the request anymore."""
+    token = create_access_token({"sub": "user@example.com"})
+    res = client.get(f"/api/events/sketch/abc/stream?token={token}")
+    assert res.status_code == 401
+
+
+def test_dead_scan_stream_endpoint_removed(client):
+    """The unused scan status stream endpoint must be gone."""
+    res = client.get("/api/events/status/scan/abc/stream")
+    assert res.status_code == 404
+
+
+def test_get_current_user_accepts_valid_bearer(db_session):
+    """The dependency the streams now use authenticates via a valid JWT."""
+    from app.api.deps import get_current_user
+
+    db_session.add(Profile(email="user@example.com", hashed_password="x"))
+    db_session.commit()
+
+    token = create_access_token({"sub": "user@example.com"})
+    user = get_current_user(token=token, db=db_session)
+    assert user.email == "user@example.com"

flowsint-app/package.json

@@ -11,6 +11,7 @@
     "format": "prettier --write .",
     "lint": "eslint . --ext .js,.jsx,.cjs,.mjs,.ts,.tsx,.cts,.mts --fix",
     "typecheck": "tsc --noEmit",
+    "test": "vitest run",
     "dev": "vite",
     "build": "vite build",
     "preview": "vite preview"
@@ -19,6 +20,7 @@
     "@ai-sdk/react": "^3.0.79",
     "@dagrejs/dagre": "^1.1.4",
     "@hookform/resolvers": "^5.0.1",
+    "@microsoft/fetch-event-source": "^2.0.1",
     "@monaco-editor/react": "^4.7.0",
     "@radix-ui/react-accordion": "^1.2.11",
     "@radix-ui/react-alert-dialog": "^1.1.14",
@@ -139,6 +141,7 @@
     "react-dom": "^19.2.0",
     "standard-version": "^9.5.0",
     "typescript": "^5.5.2",
-    "vite": "^5.3.1"
+    "vite": "^5.3.1",
+    "vitest": "^2.1.9"
   }
 }

flowsint-app/src/api/sse.test.ts

@@ -0,0 +1,117 @@
+import { describe, it, expect, vi, beforeEach } from 'vitest'
+
+// Capture the options passed to fetchEventSource so we can drive its callbacks.
+const calls: any[] = []
+const fetchEventSourceMock = vi.fn((url: string, opts: any) => {
+  calls.push({ url, opts })
+  return new Promise(() => {}) // never resolves; we drive callbacks manually
+})
+
+vi.mock('@microsoft/fetch-event-source', () => ({
+  fetchEventSource: (url: string, opts: any) => fetchEventSourceMock(url, opts),
+}))
+
+vi.mock('@/stores/auth-store', () => ({
+  useAuthStore: { getState: () => ({ token: 'tok123' }) },
+}))
+
+import { connectSSE } from './sse'
+
+const lastOpts = () => calls[calls.length - 1].opts
+
+const fakeResponse = (status: number, contentType: string) =>
+  ({ ok: status >= 200 && status < 300, status, headers: { get: () => contentType } } as any)
+
+beforeEach(() => {
+  calls.length = 0
+  fetchEventSourceMock.mockClear()
+})
+
+describe('connectSSE', () => {
+  it('passes the URL and Authorization header from the store', () => {
+    connectSSE({ url: 'http://x/stream', onMessage: () => {} })
+    expect(calls[0].url).toBe('http://x/stream')
+    expect(lastOpts().headers).toEqual({ Authorization: 'Bearer tok123' })
+  })
+
+  it('forwards parsed envelopes and skips "connected"', () => {
+    const received: any[] = []
+    connectSSE({ url: 'http://x', onMessage: (m) => received.push(m) })
+    const o = lastOpts()
+    o.onmessage({ data: JSON.stringify({ event: 'connected', data: 'hi' }) })
+    o.onmessage({ data: JSON.stringify({ event: 'log', data: '{"msg":"a"}' }) })
+    expect(received).toEqual([{ event: 'log', data: '{"msg":"a"}' }])
+  })
+
+  it('ignores empty and malformed messages without throwing', () => {
+    const received: any[] = []
+    connectSSE({ url: 'http://x', onMessage: (m) => received.push(m) })
+    const o = lastOpts()
+    o.onmessage({ data: '' })
+    o.onmessage({ data: 'not-json' })
+    expect(received).toEqual([])
+  })
+
+  it('aborts the request when the disposer is called', () => {
+    const dispose = connectSSE({ url: 'http://x', onMessage: () => {} })
+    expect(lastOpts().signal.aborted).toBe(false)
+    dispose()
+    expect(lastOpts().signal.aborted).toBe(true)
+  })
+
+  it('throws (no retry) on a non-event-stream open', async () => {
+    connectSSE({ url: 'http://x', onMessage: () => {} })
+    await expect(lastOpts().onopen(fakeResponse(401, 'application/json'))).rejects.toThrow()
+  })
+
+  it('accepts a valid event-stream open', async () => {
+    connectSSE({ url: 'http://x', onMessage: () => {} })
+    await expect(
+      lastOpts().onopen(fakeResponse(200, 'text/event-stream'))
+    ).resolves.toBeUndefined()
+  })
+
+  it('returns increasing backoff for transient errors', () => {
+    connectSSE({ url: 'http://x', onMessage: () => {} })
+    const o = lastOpts()
+    const first = o.onerror(new Error('network'))
+    const second = o.onerror(new Error('network'))
+    expect(first).toBe(2000)
+    expect(second).toBe(4000)
+  })
+
+  it('rethrows fatal errors from onerror to stop retrying', async () => {
+    connectSSE({ url: 'http://x', onMessage: () => {} })
+    const o = lastOpts()
+    let fatal: unknown
+    try {
+      await o.onopen(fakeResponse(403, 'application/json'))
+    } catch (e) {
+      fatal = e
+    }
+    expect(() => o.onerror(fatal)).toThrow()
+  })
+
+  it('ignores valid JSON that is not an envelope', () => {
+    const received: any[] = []
+    connectSSE({ url: 'http://x', onMessage: (m) => received.push(m) })
+    const o = lastOpts()
+    o.onmessage({ data: '42' })
+    o.onmessage({ data: 'null' })
+    o.onmessage({ data: '{"data":"no-event"}' })
+    expect(received).toEqual([])
+  })
+
+  it('calls onError once when the connection promise rejects (non-fatal)', async () => {
+    const errors: unknown[] = []
+    fetchEventSourceMock.mockImplementationOnce((url: string, opts: any) => {
+      calls.push({ url, opts })
+      return Promise.reject(new Error('boom'))
+    })
+    connectSSE({ url: 'http://x', onMessage: () => {}, onError: (e) => errors.push(e) })
+    await Promise.resolve()
+    await Promise.resolve()
+    expect(errors).toHaveLength(1)
+    expect((errors[0] as Error).message).toBe('boom')
+  })
+})

flowsint-app/src/api/sse.ts

@@ -0,0 +1,72 @@
+import { fetchEventSource } from '@microsoft/fetch-event-source'
+import { useAuthStore } from '@/stores/auth-store'
+
+/**
+ * Outer SSE envelope from the server. `data` is forwarded exactly as received;
+ * for the log/status streams it is itself a JSON-encoded string that the
+ * consumer parses (e.g. `JSON.parse(envelope.data)`).
+ */
+export type SSEEnvelope = { event: string; data: unknown }
+
+/** Thrown to tell fetch-event-source to stop retrying (e.g. auth failure). */
+class FatalSSEError extends Error {}
+
+const MAX_BACKOFF_MS = 30000
+
+/**
+ * Open an authenticated SSE connection.
+ * Reads the bearer token once at connect time and sends it via the
+ * Authorization header. Returns a disposer that aborts the connection.
+ */
+export function connectSSE(opts: {
+  url: string
+  onMessage: (msg: SSEEnvelope) => void
+  onError?: (err: unknown) => void
+}): () => void {
+  const controller = new AbortController()
+  let retry = 0
+
+  const token = useAuthStore.getState().token
+  const headers: Record<string, string> = token
+    ? { Authorization: `Bearer ${token}` }
+    : {}
+
+  fetchEventSource(opts.url, {
+    signal: controller.signal,
+    openWhenHidden: true,
+    headers,
+    onopen: async (res) => {
+      const contentType = res.headers.get('content-type') ?? ''
+      if (!res.ok || !contentType.includes('text/event-stream')) {
+        throw new FatalSSEError(`SSE open failed: ${res.status}`)
+      }
+      retry = 0
+    },
+    onmessage: (ev) => {
+      if (!ev.data) return
+      let envelope: SSEEnvelope
+      try {
+        envelope = JSON.parse(ev.data) as SSEEnvelope
+      } catch (error) {
+        console.error('[connectSSE] malformed envelope:', error, ev.data)
+        return
+      }
+      if (typeof envelope?.event !== 'string') return
+      if (envelope.event === 'connected') return
+      opts.onMessage(envelope)
+    },
+    onerror: (err) => {
+      if (err instanceof FatalSSEError) {
+        opts.onError?.(err)
+        throw err // stop: no retry on auth/protocol failure
+      }
+      // transient failure: retry silently with exponential backoff (onError is not called)
+      retry += 1
+      return Math.min(1000 * 2 ** retry, MAX_BACKOFF_MS)
+    },
+  }).catch((err) => {
+    if (!(err instanceof FatalSSEError)) opts.onError?.(err)
+  })
+
+  return () => controller.abort()
+}

flowsint-app/src/hooks/use-events.ts

@@ -3,6 +3,7 @@ import { useQuery } from '@tanstack/react-query'
 import { logService } from '@/api/log-service'
 import { queryKeys } from '@/api/query-keys'
 import { useAuthStore } from '@/stores/auth-store'
+import { connectSSE } from '@/api/sse'
 
 const API_URL = import.meta.env.VITE_API_URL
 
@@ -30,45 +31,21 @@ export function useEvents(sketch_id: string | undefined) {
   useEffect(() => {
     if (!sketch_id || !token) return
 
-    const eventSource = new EventSource(
-      `${API_URL}/api/events/sketch/${sketch_id}/stream?token=${token}`
-    )
-
-    eventSource.onmessage = (e) => {
-      try {
-        // Handle malformed SSE data (connection message has extra "data: " prefix)
-        let dataStr = e.data
-        if (dataStr.startsWith('data: ')) {
-          dataStr = dataStr.substring(6) // Remove "data: " prefix
-        }
-
-        const raw = JSON.parse(dataStr) as any
-
-        // Ignore connection messages
-        if (raw.event === 'connected') {
-          return
-        }
-
+    const dispose = connectSSE({
+      url: `${API_URL}/api/events/sketch/${sketch_id}/stream`,
+      onMessage: (raw) => {
         // Only process log events
-        if (raw.event !== 'log') {
-          return
+        if (raw.event !== 'log') return
+        try {
+          const event = JSON.parse(raw.data as string) as Event
+          setLiveLogs((prev) => [...prev.slice(-99), event])
+        } catch (error) {
+          console.error('[useSketchEvents] Failed to parse log payload:', error, raw.data)
         }
+      },
+    })
 
-        const event = JSON.parse(raw.data) as Event
-        setLiveLogs((prev) => [...prev.slice(-99), event])
-      } catch (error) {
-        console.error('[useSketchEvents] Failed to parse SSE event:', error, e.data)
-      }
-    }
-
-    eventSource.onerror = (error) => {
-      console.error('[useSketchEvents] EventSource error:', error)
-      eventSource.close()
-    }
-
-    return () => {
-      eventSource.close()
-    }
+    return dispose
   }, [sketch_id, token])
 
   const logs = useMemo(

flowsint-app/src/hooks/use-graph-refresh.ts

@@ -2,6 +2,7 @@ import { useEffect, useRef } from 'react'
 import { useGraphControls } from '@/stores/graph-controls-store'
 import { useAuthStore } from '@/stores/auth-store'
 import { EventLevel } from '@/types'
+import { connectSSE } from '@/api/sse'
 
 const API_URL = import.meta.env.VITE_API_URL
 
@@ -25,55 +26,35 @@ export function useGraphRefresh(sketch_id: string | undefined) {
 
   useEffect(() => {
     if (!sketch_id || !token) return
-    const eventSource = new EventSource(
-      `${API_URL}/api/events/sketch/${sketch_id}/status/stream?token=${token}`
-    )
-    eventSource.onmessage = (e) => {
-      try {
-        // Handle malformed SSE data (connection message has extra "data: " prefix)
-        let dataStr = e.data
-        if (dataStr.startsWith('data: ')) {
-          dataStr = dataStr.substring(6) // Remove "data: " prefix
-        }
-        const raw = JSON.parse(dataStr) as any
-        // Ignore connection messages
-        if (raw.event === 'connected') {
-          return
-        }
+
+    const dispose = connectSSE({
+      url: `${API_URL}/api/events/sketch/${sketch_id}/status/stream`,
+      onMessage: (raw) => {
         // Only process status events
-        if (raw.event !== 'status') {
-          return
-        }
-        const event = JSON.parse(raw.data) as any
-        // Only handle COMPLETED events
-        if (event.type === EventLevel.COMPLETED) {
-          const refetch = refetchGraphRef.current
-          const regenerate = regenerateLayoutRef.current
-          const layoutType = currentLayoutTypeRef.current
+        if (raw.event !== 'status') return
+        try {
+          const event = JSON.parse(raw.data as string) as any
+          // Only handle COMPLETED events
+          if (event.type === EventLevel.COMPLETED) {
+            const refetch = refetchGraphRef.current
+            const regenerate = regenerateLayoutRef.current
+            const layoutType = currentLayoutTypeRef.current
 
-          if (typeof refetch !== 'function') {
-            return
+            if (typeof refetch !== 'function') return
+
+            // Refetch graph data, then regenerate layout if one is active
+            refetch(() => {
+              if (layoutType && typeof regenerate === 'function') {
+                regenerate(layoutType)
+              }
+            })
           }
-
-          // Refetch graph data with callback to regenerate layout after
-          refetch(() => {
-            // After refetch completes, regenerate layout if we have one active
-            if (layoutType && typeof regenerate === 'function') {
-              regenerate(layoutType)
-            }
-          })
+        } catch (error) {
+          console.error('[useGraphRefresh] Failed to parse status payload:', error, raw.data)
         }
-      } catch (error) {
-        console.error('[useGraphRefresh] Failed to parse SSE event:', error, e.data)
-      }
-    }
+      },
+    })
 
-    eventSource.onerror = () => {
-      eventSource.close()
-    }
-
-    return () => {
-      eventSource.close()
-    }
-  }, [sketch_id, token]) // Only reconnect when sketch_id or token changes
+    return dispose
+  }, [sketch_id, token])
 }

flowsint-app/vitest.config.ts

@@ -0,0 +1,13 @@
+import { defineConfig } from 'vitest/config'
+import { resolve } from 'path'
+
+export default defineConfig({
+  resolve: {
+    alias: {
+      '@': resolve(__dirname, 'src'),
+    },
+  },
+  test: {
+    environment: 'node',
+  },
+})

flowsint-core/src/flowsint_core/core/services/analysis_service.py

@@ -2,9 +2,9 @@
 Analysis service for managing analyses within investigations.
 """
 
+from datetime import datetime, timezone
 from typing import Any, Dict, List, Optional
 from uuid import UUID, uuid4
-from datetime import datetime
 
 from sqlalchemy.orm import Session
 
@@ -64,8 +64,8 @@ class AnalysisService(BaseService):
             content=content,
             owner_id=owner_id,
             investigation_id=investigation_id,
-            created_at=datetime.utcnow(),
-            last_updated_at=datetime.utcnow(),
+            created_at=datetime.now(timezone.utc),
+            last_updated_at=datetime.now(timezone.utc),
         )
         self._analysis_repo.add(new_analysis)
         self._commit()
@@ -97,7 +97,7 @@ class AnalysisService(BaseService):
             self._check_permission(user_id, investigation_id, ["update"])
             analysis.investigation_id = investigation_id
 
-        analysis.last_updated_at = datetime.utcnow()
+        analysis.last_updated_at = datetime.now(timezone.utc)
         self._commit()
         self._refresh(analysis)
         return analysis

flowsint-core/src/flowsint_core/core/services/chat_service.py

@@ -4,7 +4,7 @@ Chat service for managing chats and messages with AI integration.
 
 import json
 import os
-from datetime import datetime
+from datetime import datetime, timezone
 from typing import Any, AsyncIterator, Dict, List, Optional
 from uuid import UUID, uuid4
 
@@ -76,8 +76,8 @@ class ChatService(BaseService):
             description=description,
             owner_id=owner_id,
             investigation_id=investigation_id,
-            created_at=datetime.utcnow(),
-            last_updated_at=datetime.utcnow(),
+            created_at=datetime.now(timezone.utc),
+            last_updated_at=datetime.now(timezone.utc),
         )
         self._chat_repo.add(new_chat)
         self._commit()
@@ -103,7 +103,7 @@ class ChatService(BaseService):
         if not chat:
             raise NotFoundError("Chat not found")
 
-        chat.last_updated_at = datetime.utcnow()
+        chat.last_updated_at = datetime.now(timezone.utc)
 
         user_message = ChatMessage(
             id=uuid4(),
@@ -111,7 +111,7 @@ class ChatService(BaseService):
             context=context,
             chat_id=chat_id,
             is_bot=False,
-            created_at=datetime.utcnow(),
+            created_at=datetime.now(timezone.utc),
         )
         self._chat_repo.add_message(user_message)
         self._commit()
@@ -124,7 +124,7 @@ class ChatService(BaseService):
             content=content,
             chat_id=chat_id,
             is_bot=True,
-            created_at=datetime.utcnow(),
+            created_at=datetime.now(timezone.utc),
         )
         self._chat_repo.add_message(chat_message)
         self._commit()

flowsint-core/src/flowsint_core/core/services/flow_service.py

@@ -2,7 +2,7 @@
 Flow service for managing flows and flow computations.
 """
 
-from datetime import datetime
+from datetime import datetime, timezone
 from typing import Any, Dict, List, Optional
 from uuid import UUID, uuid4
 
@@ -82,8 +82,8 @@ class FlowService(BaseService):
             description=description,
             category=category,
             flow_schema=flow_schema,
-            created_at=datetime.utcnow(),
-            last_updated_at=datetime.utcnow(),
+            created_at=datetime.now(timezone.utc),
+            last_updated_at=datetime.now(timezone.utc),
         )
         self._flow_repo.add(new_flow)
         self._commit()
@@ -101,7 +101,7 @@ class FlowService(BaseService):
                     value.append("Username")
             setattr(flow, key, value)
 
-        flow.last_updated_at = datetime.utcnow()
+        flow.last_updated_at = datetime.now(timezone.utc)
         self._commit()
         self._refresh(flow)
         return flow

flowsint-core/src/flowsint_core/tasks/enricher.py

@@ -69,10 +69,10 @@ def run_enricher(
         results = asyncio.run(enricher.execute(values=serialized_objects))
 
         scan.status = EventLevel.COMPLETED
-        scan.results = to_json_serializable(results)
+        scan.details = to_json_serializable(results)
         session.commit()
 
-        return {"result": scan.results}
+        return {"result": scan.details}
 
     except Exception as ex:
         session.rollback()
@@ -82,7 +82,7 @@ def run_enricher(
         scan = session.query(Scan).filter(Scan.id == uuid.UUID(self.request.id)).first()
         if scan:
             scan.status = EventLevel.FAILED
-            scan.results = {"error": error_logs}
+            scan.error = error_logs
             session.commit()
 
         self.update_state(state=states.FAILURE)
@@ -145,10 +145,10 @@ def run_template_enricher(
         results = asyncio.run(enricher.execute(values=serialized_objects))
 
         scan.status = EventLevel.COMPLETED
-        scan.results = to_json_serializable(results)
+        scan.details = to_json_serializable(results)
         session.commit()
 
-        return {"result": scan.results}
+        return {"result": scan.details}
 
     except Exception as ex:
         session.rollback()
@@ -162,7 +162,7 @@ def run_template_enricher(
         )
         if scan:
             scan.status = EventLevel.FAILED
-            scan.results = {"error": error_logs}
+            scan.error = error_logs
             session.commit()
 
         self.update_state(state=states.FAILURE)

flowsint-core/src/flowsint_core/tasks/flow.py

@@ -65,10 +65,10 @@ def run_flow(
         results = enricher.scan(values=serialized_objects)
 
         scan.status = EventLevel.COMPLETED
-        scan.results = to_json_serializable(results)
+        scan.details = to_json_serializable(results)
         session.commit()
 
-        return {"result": scan.results}
+        return {"result": scan.details}
 
     except Exception as ex:
         session.rollback()
@@ -78,7 +78,7 @@ def run_flow(
         scan = session.query(Scan).filter(Scan.id == uuid.UUID(self.request.id)).first()
         if scan:
             scan.status = EventLevel.FAILED
-            scan.results = {"error": error_logs}
+            scan.error = error_logs
             session.commit()
 
         self.update_state(state=states.FAILURE)

flowsint-core/tests/services/test_timezone_timestamps.py

@@ -0,0 +1,72 @@
+"""Tests for timezone-aware service timestamps."""
+
+from datetime import timezone
+from unittest.mock import MagicMock
+from uuid import uuid4
+
+from flowsint_core.core.services.analysis_service import AnalysisService
+from flowsint_core.core.services.chat_service import ChatService
+from flowsint_core.core.services.flow_service import FlowService
+
+
+def _assert_utc(value):
+    assert value.tzinfo is not None
+    assert value.utcoffset() == timezone.utc.utcoffset(value)
+
+
+def test_analysis_service_create_uses_timezone_aware_timestamps():
+    service = AnalysisService(
+        db=MagicMock(),
+        analysis_repo=MagicMock(),
+        investigation_repo=MagicMock(),
+    )
+    service._check_permission = MagicMock()
+
+    analysis = service.create(
+        title="Summary",
+        description=None,
+        content={},
+        investigation_id=uuid4(),
+        owner_id=uuid4(),
+    )
+
+    _assert_utc(analysis.created_at)
+    _assert_utc(analysis.last_updated_at)
+
+
+def test_chat_service_create_uses_timezone_aware_timestamps():
+    service = ChatService(
+        db=MagicMock(),
+        chat_repo=MagicMock(),
+        vault_service=MagicMock(),
+    )
+
+    chat = service.create(
+        title="Investigation chat",
+        description=None,
+        investigation_id=uuid4(),
+        owner_id=uuid4(),
+    )
+
+    _assert_utc(chat.created_at)
+    _assert_utc(chat.last_updated_at)
+
+
+def test_flow_service_create_uses_timezone_aware_timestamps():
+    service = FlowService(
+        db=MagicMock(),
+        flow_repo=MagicMock(),
+        custom_type_repo=MagicMock(),
+        sketch_repo=MagicMock(),
+        investigation_repo=MagicMock(),
+    )
+
+    flow = service.create(
+        name="Resolve domain",
+        description=None,
+        category=["Domain"],
+        flow_schema={"nodes": [], "edges": []},
+    )
+
+    _assert_utc(flow.created_at)
+    _assert_utc(flow.last_updated_at)

flowsint-enrichers/src/flowsint_enrichers/domain/to_dehashed.py

@@ -0,0 +1,161 @@
+import json
+import os
+import requests
+
+from typing import Any, Dict, List, Optional
+from flowsint_core.core.enricher_base import Enricher
+from flowsint_enrichers.registry import flowsint_enricher
+from flowsint_types.domain import Domain
+from flowsint_types.individual import Individual
+from flowsint_core.core.logger import Logger
+
+@flowsint_enricher
+class DomainToDehashed(Enricher):
+    """[DeHashed] Get breach intelligence from a domain."""
+
+    # Define types as class attributes - base class handles schema generation automatically
+    InputType = Domain
+    OutputType = Individual
+
+    def __init__(
+        self,
+        sketch_id: Optional[str] = None,
+        scan_id: Optional[str] = None,
+        vault=None,
+        params: Optional[Dict[str, Any]] = None,
+    ):
+        super().__init__(
+            sketch_id=sketch_id,
+            scan_id=scan_id,
+            params_schema=self.get_params_schema(),
+            vault=vault,
+            params=params,
+        )
+
+    # @classmethod
+    # def required_params(cls) -> bool:
+    #     return True
+
+    @classmethod
+    def get_params_schema(cls) -> List[Dict[str, Any]]:
+        """Declare required parameters for this enricher"""
+        return [
+            {
+                "name": "DEHASHED_API_KEY", # Get your API key from dehashed.com/api
+                "type": "vaultSecret",
+                "description": "Your Dehashed API key.",
+                "required": True,
+            }
+        ]
+
+    @classmethod
+    def name(cls) -> str:
+        return "domain_to_dehashed"
+
+    @classmethod
+    def category(cls) -> str:
+        return "Domain"
+
+    @classmethod
+    def key(cls) -> str:
+        return "domain"
+
+    async def scan(self, data: List[InputType]) -> List[OutputType]:
+        results: List[OutputType] = []
+
+        api_key = self.get_secret("DEHASHED_API_KEY", os.getenv("DEHASHED_API_KEY")) 
+
+        for domain in data:
+            try:
+                headers = {'Dehashed-Api-Key': api_key, 'Content-Type': 'application/json'}
+                raw_data = json.dumps({"query": f"domain:{domain.domain}"})
+
+                api_request = requests.post(f'https://api.dehashed.com/v2/search', data=raw_data, headers=headers, timeout=30)
+
+                if api_request.status_code != 200:
+                    if api_request.status_code == 401:
+                        Logger.error(
+                            self.sketch_id,
+                            {
+                                "message": f"(DomainToDehashed) Enricher failed for the domain because API key does not have a valid subscription: '{domain.domain}': {api_request.text}"
+                            },
+                        )
+
+                    elif api_request.status_code == 403:
+                        Logger.error(
+                            self.sketch_id,
+                            {
+                                "message": f"(DomainToDehashed) Enricher failed for the domain because request has malfunctioned (missing API_KEY): '{domain.domain}': {api_request.text}"
+                            },
+                        )
+
+                try:
+                    response_json = api_request.json()
+                except Exception as e:
+                    Logger.error(None, {"message": f"(DomainToDehashed) Failed to parse JSON for {domain.domain}: {e}"})
+                    continue
+
+                dehashed_entries = response_json.get("entries", [])
+                if not dehashed_entries:
+                    Logger.error(
+                            self.sketch_id,
+                            {
+                                "message": f"(DomainToDehashed) Enricher failed for the domain: '{domain.domain}': {api_request.text}"
+                            },
+                        )
+                    continue
+
+
+                api_balance = response_json.get("balance")
+                if api_balance:
+                    Logger.info(self.sketch_id, f'(DomainToDehashed) Your remaining API balance is {api_balance}.')
+
+                for entry in dehashed_entries:
+                    entry_email = entry.get("email")
+                    entry_phone = entry.get("phone")
+                    entry_name = entry.get("name")
+                    entry_socialmedia = entry.get("social")
+                    entry_ip = entry.get("ip_address")
+                    entry_username = entry.get("username")
+                    entry_dob = entry.get("dob")
+
+                    results.append(
+                        Individual(
+                            full_name=entry_name[0] if entry_name else None, 
+                            birth_date=entry_dob[0] if entry_dob else None, 
+                            email_addresses=entry_email if entry_email else None, 
+                            phone_numbers=entry_phone if entry_phone else None, 
+                            social_media_profiles=entry_socialmedia if entry_socialmedia else None, 
+                            ip_addresses=entry_ip if entry_ip else None, 
+                            usernames=entry_username if entry_username else None
+                            )
+                        )
+            except Exception as e:
+                Logger.error(self.sketch_id, {"message": f"(DomainToDehashed) Exception while querying {domain.domain}: {e}"})
+        
+        return results
+
+    def postprocess(
+        self, results: List[OutputType], input_data: List[InputType] = None
+    ) -> List[OutputType]:
+        if not self._graph_service:
+            return results
+
+        if input_data and self._graph_service:
+            for domain in input_data:
+                for individual in results:
+                    self.create_node(domain)
+                    self.create_node(individual)
+
+                    # Create relationship
+                    self.create_relationship(domain, individual, "CONNECTION_WITH")
+                    self.log_graph_message(
+                        f"(DomainToDehashed) Successfully found individual connections with {domain.domain}. "
+                    )
+
+        return results
+
+
+# Make types available at module level for easy access
+InputType = DomainToDehashed.InputType
+OutputType = DomainToDehashed.OutputType

flowsint-enrichers/src/flowsint_enrichers/domain/to_ssl.py

@@ -0,0 +1,100 @@
+import json
+import os
+import requests
+
+from typing import Any, Dict, List, Optional
+from flowsint_core.core.enricher_base import Enricher
+from flowsint_enrichers.registry import flowsint_enricher
+from flowsint_types.domain import Domain
+from flowsint_types.website import Website
+from flowsint_core.core.logger import Logger
+from flowsint_types.address import Location
+
+from tools.network.httpx import HttpxTool
+
+
+@flowsint_enricher
+class DomainToTLS(Enricher):
+    """[httpX] Get TLS information from a domain."""
+
+    # Define types as class attributes - base class handles schema generation automatically
+    InputType = Domain
+    OutputType = Website
+
+    # @classmethod
+    # def required_params(cls) -> bool:
+    #     return True
+
+    @classmethod
+    def name(cls) -> str:
+        return "domain_to_tls"
+
+    @classmethod
+    def category(cls) -> str:
+        return "Domain"
+
+    @classmethod
+    def key(cls) -> str:
+        return "domain"
+
+    async def scan(self, data: List[InputType]) -> List[OutputType]:
+        results: List[OutputType] = []
+
+        Logger.info(self.sketch_id, f"(DomainToTLS) Received {len(data)} inputs")
+        for domain in data:
+            try:
+                tool = HttpxTool()
+                httpx_results  = tool.launch(
+                    target=domain.domain,
+                    args=["-tls-probe"]
+                )
+
+                if not httpx_results :
+                    Logger.error(None, {"message": f"(DomainToTLS) No results for the domain '{domain.domain}'"})
+                    continue
+                else:
+                    for r_line in httpx_results :
+                            web_url = r_line.get("url")
+                            web_title = r_line.get("title")
+                            web_content_type = r_line.get("content_type")
+                            web_statuscode = r_line.get("status_code") # status_code is int
+
+                            results.append(
+                                Website(
+                                    url=web_url if web_url else None,
+                                    domain=domain,
+                                    active=True,
+                                    title=web_title if web_title else None,
+                                    content_type=web_content_type if web_content_type else None,
+                                    status_code=web_statuscode if web_statuscode else None
+                                )
+                            )                
+            except Exception as e:
+                Logger.error(self.sketch_id, {"message": f"(DomainToTLS) Exception while querying the domain '{domain.domain}': {e}"})
+        
+        return results
+
+    def postprocess(
+        self, results: List[OutputType], input_data: List[InputType] = None
+    ) -> List[OutputType]:
+        if not self._graph_service:
+            return results
+
+        if input_data and self._graph_service:
+            for domain in input_data:
+                for website in results:
+                    self.create_node(domain)
+                    self.create_node(website)
+
+                    # Create relationship
+                    self.create_relationship(domain, website, "TLS_INFO")
+                    self.log_graph_message(
+                        f"(DomainToTLS) Successfully fetched TLS information for the domain '{domain.domain}'. "
+                    )
+
+        return results
+
+
+# Make types available at module level for easy access
+InputType = DomainToTLS.InputType
+OutputType = DomainToTLS.OutputType

flowsint-enrichers/src/flowsint_enrichers/email/to_dehashed.py

@@ -0,0 +1,161 @@
+import json
+import os
+import requests
+
+from typing import Any, Dict, List, Optional
+from flowsint_core.core.enricher_base import Enricher
+from flowsint_enrichers.registry import flowsint_enricher
+from flowsint_types.email import Email
+from flowsint_types.individual import Individual
+from flowsint_core.core.logger import Logger
+
+@flowsint_enricher
+class EmailToDehashed(Enricher):
+    """[DeHashed] Get breach intelligence from an email address."""
+
+    # Define types as class attributes - base class handles schema generation automatically
+    InputType = Email
+    OutputType = Individual
+
+    def __init__(
+        self,
+        sketch_id: Optional[str] = None,
+        scan_id: Optional[str] = None,
+        vault=None,
+        params: Optional[Dict[str, Any]] = None,
+    ):
+        super().__init__(
+            sketch_id=sketch_id,
+            scan_id=scan_id,
+            params_schema=self.get_params_schema(),
+            vault=vault,
+            params=params,
+        )
+
+    # @classmethod
+    # def required_params(cls) -> bool:
+    #     return True
+
+    @classmethod
+    def get_params_schema(cls) -> List[Dict[str, Any]]:
+        """Declare required parameters for this enricher"""
+        return [
+            {
+                "name": "DEHASHED_API_KEY", # Get your API key from dehashed.com/api
+                "type": "vaultSecret",
+                "description": "Your Dehashed API key.",
+                "required": True,
+            }
+        ]
+
+    @classmethod
+    def name(cls) -> str:
+        return "email_to_intelligence"
+
+    @classmethod
+    def category(cls) -> str:
+        return "Email"
+
+    @classmethod
+    def key(cls) -> str:
+        return "email"
+
+    async def scan(self, data: List[InputType]) -> List[OutputType]:
+        results: List[OutputType] = []
+
+        api_key = self.get_secret("DEHASHED_API_KEY", os.getenv("DEHASHED_API_KEY")) 
+
+        for email in data:
+            try:
+                headers = {'Dehashed-Api-Key': api_key, 'Content-Type': 'application/json'}
+                raw_data = json.dumps({"query": f"email:{email.email}"})
+
+                api_request = requests.post(f'https://api.dehashed.com/v2/search', data=raw_data, headers=headers, timeout=30)
+
+                if api_request.status_code != 200:
+                    if api_request.status_code == 401:
+                        Logger.error(
+                            self.sketch_id,
+                            {
+                                "message": f"(EmailToDehashed) Enricher failed for the email because API key does not have a valid subscription: '{email.email}': {api_request.text}"
+                            },
+                        )
+
+                    elif api_request.status_code == 403:
+                        Logger.error(
+                            self.sketch_id,
+                            {
+                                "message": f"(EmailToDehashed) Enricher failed for the email because request has malfunctioned (missing API_KEY): '{email.email}': {api_request.text}"
+                            },
+                        )
+
+                try:
+                    response_json = api_request.json()
+                except Exception as e:
+                    Logger.error(None, {"message": f"(EmailToDehashed) Failed to parse JSON for {email.email}: {e}"})
+                    continue
+
+                dehashed_entries = response_json.get("entries", [])
+                if not dehashed_entries:
+                    Logger.error(
+                            self.sketch_id,
+                            {
+                                "message": f"(EmailToDehashed) Enricher failed for the email: '{email.email}': {api_request.text}"
+                            },
+                        )
+                    continue
+
+
+                api_balance = response_json.get("balance")
+                if api_balance:
+                    Logger.info(self.sketch_id, f'(EmailToDehashed) Your remaining API balance is {api_balance}.')
+
+                for entry in dehashed_entries:
+                    entry_email = entry.get("email")
+                    entry_phone = entry.get("phone")
+                    entry_name = entry.get("name")
+                    entry_socialmedia = entry.get("social")
+                    entry_ip = entry.get("ip_address")
+                    entry_username = entry.get("username")
+                    entry_dob = entry.get("dob")
+
+                    results.append(
+                        Individual(
+                            full_name=entry_name[0] if entry_name else None, 
+                            birth_date=entry_dob[0] if entry_dob else None, 
+                            email_addresses=entry_email if entry_email else None, 
+                            phone_numbers=entry_phone if entry_phone else None, 
+                            social_media_profiles=entry_socialmedia if entry_socialmedia else None, 
+                            ip_addresses=entry_ip if entry_ip else None, 
+                            usernames=entry_username if entry_username else None
+                            )
+                        )
+            except Exception as e:
+                Logger.error(self.sketch_id, {"message": f"(EmailToDehashed) Exception while querying email {email.email}: {e}"})
+        
+        return results
+
+    def postprocess(
+        self, results: List[OutputType], input_data: List[InputType] = None
+    ) -> List[OutputType]:
+        if not self._graph_service:
+            return results
+
+        if input_data and self._graph_service:
+            for email in input_data:
+                for individual in results:
+                    self.create_node(email)
+                    self.create_node(individual)
+
+                    # Create relationship
+                    self.create_relationship(email, individual, "CONNECTION_WITH")
+                    self.log_graph_message(
+                        f"(EmailToDehashed) Successfully found individual connections with {email.email}. "
+                    )
+
+        return results
+
+
+# Make types available at module level for easy access
+InputType = EmailToDehashed.InputType
+OutputType = EmailToDehashed.OutputType

flowsint-enrichers/src/flowsint_enrichers/ip/to_dehashed.py

@@ -0,0 +1,163 @@
+import json
+import os
+import requests
+
+from typing import Any, Dict, List, Optional
+from flowsint_core.core.enricher_base import Enricher
+from flowsint_enrichers.registry import flowsint_enricher
+from flowsint_types.ip import Ip
+from flowsint_types.individual import Individual
+from flowsint_core.core.logger import Logger
+
+@flowsint_enricher
+class IpToIntelligence(Enricher):
+    """[DeHashed] Get breach intelligence from an IP address."""
+
+    # Define types as class attributes - base class handles schema generation automatically
+    InputType = Ip
+    OutputType = Individual
+
+    def __init__(
+        self,
+        sketch_id: Optional[str] = None,
+        scan_id: Optional[str] = None,
+        vault=None,
+        params: Optional[Dict[str, Any]] = None,
+    ):
+        super().__init__(
+            sketch_id=sketch_id,
+            scan_id=scan_id,
+            params_schema=self.get_params_schema(),
+            vault=vault,
+            params=params,
+        )
+
+    # @classmethod
+    # def required_params(cls) -> bool:
+    #     return True
+
+    @classmethod
+    def get_params_schema(cls) -> List[Dict[str, Any]]:
+        """Declare required parameters for this enricher"""
+        return [
+            {
+                "name": "DEHASHED_API_KEY", # Get your API key from dehashed.com/api
+                "type": "vaultSecret",
+                "description": "Your Dehashed API key.",
+                "required": True,
+            }
+        ]
+
+    @classmethod
+    def name(cls) -> str:
+        return "ip_to_intelligence"
+
+    @classmethod
+    def category(cls) -> str:
+        return "Ip"
+
+    @classmethod
+    def key(cls) -> str:
+        return "address"
+
+    async def scan(self, data: List[InputType]) -> List[OutputType]:
+        results: List[OutputType] = []
+
+        api_key = self.get_secret("DEHASHED_API_KEY", os.getenv("DEHASHED_API_KEY")) 
+
+        for ip in data:
+            try:
+                headers = {'Dehashed-Api-Key': api_key, 'Content-Type': 'application/json'}
+                raw_data = json.dumps({"query": f"ip_address:{ip.address}"})
+
+                api_request = requests.post(f'https://api.dehashed.com/v2/search', data=raw_data, headers=headers, timeout=30)
+
+                if api_request.status_code != 200:
+                    if api_request.status_code == 401:
+                        Logger.error(
+                            self.sketch_id,
+                            {
+                                "message": f"(IpToIntelligence) Enricher failed for the IP address because API key does not have a valid subscription: '{ip.address}': {api_request.text}"
+                            },
+                        )
+
+                    elif api_request.status_code == 403:
+                        Logger.error(
+                            self.sketch_id,
+                            {
+                                "message": f"(IpToIntelligence) Enricher failed for the IP address because request has malfunctioned (missing API_KEY): '{ip.address}': {api_request.text}"
+                            },
+                        )
+
+                try:
+                    response_json = api_request.json()
+                except Exception as e:
+                    Logger.error(None, {"message": f"(IpToIntelligence) Failed to parse JSON for {ip.address}: {e}"})
+                    continue
+
+                dehashed_entries = response_json.get("entries", [])
+                if not dehashed_entries:
+                    Logger.error(
+                            self.sketch_id,
+                            {
+                                "message": f"(IpToIntelligence) Enricher failed for the IP address: '{ip.address}': {api_request.text}"
+                            },
+                        )
+                    continue
+
+
+                api_balance = response_json.get("balance")
+                if api_balance:
+                    Logger.info(self.sketch_id, f'(EmailToDehashed) Your remaining API balance is {api_balance}.')
+                    
+                time_took = response_json.get("took")
+
+                for entry in dehashed_entries:
+                    entry_email = entry.get("email")
+                    entry_phone = entry.get("phone")
+                    entry_name = entry.get("name")
+                    entry_socialmedia = entry.get("social")
+                    entry_ip = entry.get("ip_address")
+                    entry_username = entry.get("username")
+                    entry_dob = entry.get("dob")
+
+                    results.append(
+                        Individual(
+                            full_name=entry_name[0] if entry_name else None, 
+                            birth_date=entry_dob[0] if entry_dob else None, 
+                            email_addresses=entry_email if entry_email else None, 
+                            phone_numbers=entry_phone if entry_phone else None, 
+                            social_media_profiles=entry_socialmedia if entry_socialmedia else None, 
+                            ip_addresses=entry_ip if entry_ip else None, 
+                            usernames=entry_username if entry_username else None
+                            )
+                        )
+            except Exception as e:
+                Logger.error(self.sketch_id, {"message": f"(IpToIntelligence) Exception while querying {ip.address}: {e}"})
+        
+        return results
+
+    def postprocess(
+        self, results: List[OutputType], input_data: List[InputType] = None
+    ) -> List[OutputType]:
+        if not self._graph_service:
+            return results
+
+        if input_data and self._graph_service:
+            for ip in input_data:
+                for individual in results:
+                    self.create_node(ip)
+                    self.create_node(individual)
+
+                    # Create relationship
+                    self.create_relationship(ip, individual, "CONNECTION_WITH")
+                    self.log_graph_message(
+                        f"(IpToIntelligence) Successfully found individual connections with {ip.address}."
+                    )
+
+        return results
+
+
+# Make types available at module level for easy access
+InputType = IpToIntelligence.InputType
+OutputType = IpToIntelligence.OutputType

flowsint-enrichers/src/flowsint_enrichers/ip/to_fraudscore.py

@@ -0,0 +1,157 @@
+import json
+import os
+import requests
+import datetime
+
+from typing import Any, Dict, List, Optional, Union
+from flowsint_core.core.enricher_base import Enricher
+from flowsint_enrichers.registry import flowsint_enricher
+from flowsint_types.ip import Ip
+from flowsint_types.risk_profile import RiskProfile
+from flowsint_core.utils import is_valid_ip
+from flowsint_core.core.logger import Logger
+
+@flowsint_enricher
+class IpToFraudScore(Enricher):
+    """[Scamalytics] Get fraud score for an IP address."""
+
+    # Define types as class attributes - base class handles schema generation automatically
+    InputType = Ip
+    OutputType = RiskProfile
+
+    def __init__(
+        self,
+        sketch_id: Optional[str] = None,
+        scan_id: Optional[str] = None,
+        vault=None,
+        params: Optional[Dict[str, Any]] = None,
+    ):
+        super().__init__(
+            sketch_id=sketch_id,
+            scan_id=scan_id,
+            params_schema=self.get_params_schema(),
+            vault=vault,
+            params=params,
+        )
+
+    # @classmethod
+    # def required_params(cls) -> bool:
+    #     return True
+
+    @classmethod
+    def get_params_schema(cls) -> List[Dict[str, Any]]:
+        """Declare required parameters for this enricher"""
+        return [
+            {
+                "name": "SCAMLYTICS_USERNAME",
+                "type": "vaultSecret",
+                "description": "The Scamalytics Username.",
+                "required": True,
+            },
+            {
+                "name": "SCAMLYTICS_API_KEY",
+                "type": "vaultSecret",
+                "description": "The Scamalytics API key for IP-based lookups.",
+                "required": True,
+            },
+        ]
+
+    @classmethod
+    def name(cls) -> str:
+        return "ip_to_fraudscore"
+
+    @classmethod
+    def category(cls) -> str:
+        return "Ip"
+
+    @classmethod
+    def key(cls) -> str:
+        return "address"
+
+    async def scan(self, data: List[InputType]) -> List[OutputType]:
+        results: List[OutputType] = []
+
+        api_username = self.get_secret("SCAMLYTICS_USERNAME", os.getenv("SCAMLYTICS_USERNAME")) 
+        api_key = self.get_secret("SCAMLYTICS_API_KEY", os.getenv("SCAMLYTICS_API_KEY")) 
+
+        for ip in data:
+            try:
+                api_request = requests.get(f'https://api12.scamalytics.com/v3/{api_username}/?key={api_key}&ip={ip.address}', timeout=30)
+                
+                if api_request.status_code != 200:
+                    Logger.error(
+                        self.sketch_id,
+                        {
+                            "message": f"(IpToFraudScore) Enricher failed for the IP address: '{ip.address}': {api_request.text}"
+                        },
+                    )
+                    continue
+
+                try:
+                    response_json = api_request.json()
+                except Exception as e:
+                    Logger.error(None, {"message": f"(IpToFraudScore) Failed to parse JSON for {ip.address}: {api_request.text}"})
+                    continue
+
+                scamalytics_info = response_json.get("scamalytics", {})
+                if scamalytics_info.get("status") != "ok":
+                    Logger.error(
+                        self.sketch_id,
+                        {
+                            "message": f"(IpToFraudScore) Request to Scamlytics failed (status): '{response_json["status"]}'."
+                        },
+                    )
+                    continue
+
+                fraud_score = scamalytics_info.get("scamalytics_score")
+                fraud_risk = scamalytics_info.get("scamalytics_risk")
+                proxy_flags = []
+
+                proxy_detectors = {
+                    "is_datacenter": "Datacenter",
+                    "is_vpn": "VPN",
+                    "is_apple_icloud_private_relay": "iCloud Private Relay",
+                    "is_amazon_aws": "Amazon AWS",
+                    "is_google": "Google"
+                }
+
+                proxy_info = scamalytics_info.get("scamalytics_proxy", {})
+
+                for key, label in proxy_detectors.items():
+                    if proxy_info.get(key):
+                        proxy_flags.append(label)
+
+                results.append(
+                    RiskProfile(
+                        entity_id=ip.address, entity_type="IP address", overall_risk_score=fraud_score, risk_level=fraud_risk, last_updated=datetime.datetime.utcnow().isoformat(), risk_factors=proxy_flags, source="Scamalytics"
+                        )
+                    )
+
+            except Exception as e:
+                Logger.error(self.sketch_id, {"message": f"(IpToFraudScore) Exception while querying {ip.address}: {e}"})
+        
+        return results
+
+    def postprocess(
+        self, results: List[OutputType], input_data: List[InputType] = None
+    ) -> List[OutputType]:
+        if not self._graph_service:
+            return results
+
+        if input_data and self._graph_service:
+            for ip, risk_profile in zip(input_data, results):
+                self.create_node(ip)
+                self.create_node(risk_profile)
+
+                # Create relationship
+                self.create_relationship(ip, risk_profile, "HAS_RISK_PROFILE")
+                self.log_graph_message(
+                    f"(IpToFraudScore) IP {ip.address} has risk level of {risk_profile.risk_level}"
+                )
+
+        return results
+
+
+# Make types available at module level for easy access
+InputType = IpToFraudScore.InputType
+OutputType = IpToFraudScore.OutputType

flowsint-enrichers/src/flowsint_enrichers/phone/to_carrier.py

@@ -0,0 +1,120 @@
+import requests
+import os
+
+from typing import Dict, Any, List, Optional
+from flowsint_core.core.enricher_base import Enricher
+from flowsint_enrichers.registry import flowsint_enricher
+from flowsint_core.core.logger import Logger
+from flowsint_types.phone import Phone
+
+
+@flowsint_enricher
+class PhoneToCarrier(Enricher):
+    """[veriphone] Looks up phone number/s carrier using veriphone API."""
+
+    # Define types as class attributes - base class handles schema generation automatically
+    InputType = Phone  
+    OutputType = Phone  
+
+    def __init__(
+        self,
+        sketch_id: Optional[str] = None,
+        scan_id: Optional[str] = None,
+        vault=None,
+        params: Optional[Dict[str, Any]] = None,
+    ):
+        super().__init__(
+            sketch_id=sketch_id,
+            scan_id=scan_id,
+            params_schema=self.get_params_schema(),
+            vault=vault,
+            params=params,
+        )
+
+    @classmethod
+    def get_params_schema(cls) -> List[Dict[str, Any]]:
+        """Declare required parameters for this enricher"""
+        return [
+            {
+                "name": "VERIPHONE_API_KEY",
+                "type": "vaultSecret",
+                "description": "The veriphone API key for phone number carrier lookups.",
+                "required": True,
+            },
+        ]
+
+    @classmethod
+    def name(cls) -> str:
+        return "phone_to_carrier"
+
+    @classmethod
+    def category(cls) -> str:
+        return "phones"
+
+    @classmethod
+    def key(cls) -> str:
+        return "number"
+ 
+    async def scan(self, data: List[InputType]) -> List[OutputType]:
+        results: List[OutputType] = []
+
+        api_key = self.get_secret("VERIPHONE_API_KEY", os.getenv("VERIPHONE_API_KEY"))
+        Logger.debug(self.sketch_id, {"message": f"API key present: {bool(api_key)}"})
+        
+        for phone_obj in data:
+            phonenum_value = phone_obj.number
+            try:
+                api_request = requests.get(f'https://api.veriphone.io/v2/verify?key={api_key}&phone={phonenum_value}', timeout=30)
+                
+                if api_request.status_code != 200:
+                    Logger.error(
+                        self.sketch_id,
+                        {
+                            "message": f"(PhoneToCarrier) Enricher failed for the phone number: '{phonenum_value}': {api_request.text}"
+                        },
+                    )
+                    continue
+
+                try:
+                    response_json = api_request.json()
+                except Exception as e:
+                    Logger.error(None, {"message": f"(PhoneToCarrier) Failed to parse JSON for phone number '{phonenum_value}'': {api_request.text}"})
+                    continue
+
+                carrier = response_json.get("carrier")
+                if carrier:
+                    results.append({
+                        "number": phonenum_value,
+                        "carrier": carrier
+                    })
+
+            except Exception as e:
+                Logger.error(self.sketch_id, {"message": f"(PhoneToCarrier) Exception while querying phone number '{phonenum_value}'': {e}"})
+        
+        return results
+
+    def postprocess(self, results: List[OutputType], original_input: List[InputType]) -> List[OutputType]:
+        if not self._graph_service:
+            return results
+
+        for result in results:
+            number = result["number"]
+            carrier = result["carrier"]
+
+            for phone_obj in original_input:
+                if phone_obj.number == number:
+                    phone_obj.carrier = carrier
+
+                    self.create_node(phone_obj)
+                    
+                    self.log_graph_message(
+                        f"(PhoneToCarrier) Found carrier ({carrier}) for the phone number '{number}'."
+                    )
+                    break
+
+        return results
+
+
+# Make types available at module level for easy access
+InputType = PhoneToCarrier.InputType
+OutputType = PhoneToCarrier.OutputType

flowsint-enrichers/src/flowsint_enrichers/social/to_dehashed.py

@@ -0,0 +1,161 @@
+import json
+import os
+import requests
+
+from typing import Any, Dict, List, Optional
+from flowsint_core.core.enricher_base import Enricher
+from flowsint_enrichers.registry import flowsint_enricher
+from flowsint_types.username import Username
+from flowsint_types.individual import Individual
+from flowsint_core.core.logger import Logger
+
+@flowsint_enricher
+class UsernameToDehashed(Enricher):
+    """[DeHashed] Get breach intelligence from a username."""
+
+    # Define types as class attributes - base class handles schema generation automatically
+    InputType = Username
+    OutputType = Individual
+
+    def __init__(
+        self,
+        sketch_id: Optional[str] = None,
+        scan_id: Optional[str] = None,
+        vault=None,
+        params: Optional[Dict[str, Any]] = None,
+    ):
+        super().__init__(
+            sketch_id=sketch_id,
+            scan_id=scan_id,
+            params_schema=self.get_params_schema(),
+            vault=vault,
+            params=params,
+        )
+
+    # @classmethod
+    # def required_params(cls) -> bool:
+    #     return True
+
+    @classmethod
+    def get_params_schema(cls) -> List[Dict[str, Any]]:
+        """Declare required parameters for this enricher"""
+        return [
+            {
+                "name": "DEHASHED_API_KEY", # Get your API key from dehashed.com/api
+                "type": "vaultSecret",
+                "description": "Your Dehashed API key.",
+                "required": True,
+            }
+        ]
+
+    @classmethod
+    def name(cls) -> str:
+        return "username_to_dehashed"
+
+    @classmethod
+    def category(cls) -> str:
+        return "social"
+
+    @classmethod
+    def key(cls) -> str:
+        return "username"
+
+    async def scan(self, data: List[InputType]) -> List[OutputType]:
+        results: List[OutputType] = []
+
+        api_key = self.get_secret("DEHASHED_API_KEY", os.getenv("DEHASHED_API_KEY")) 
+
+        for username in data:
+            try:
+                headers = {'Dehashed-Api-Key': api_key, 'Content-Type': 'application/json'}
+                raw_data = json.dumps({"query": f"username:{username.value}"})
+
+                api_request = requests.post(f'https://api.dehashed.com/v2/search', data=raw_data, headers=headers, timeout=30)
+
+                if api_request.status_code != 200:
+                    if api_request.status_code == 401:
+                        Logger.error(
+                            self.sketch_id,
+                            {
+                                "message": f"(UsernameToDehashed) Enricher failed for the username because API key does not have a valid subscription: '{username.value}': {api_request.text}"
+                            },
+                        )
+
+                    elif api_request.status_code == 403:
+                        Logger.error(
+                            self.sketch_id,
+                            {
+                                "message": f"(UsernameToDehashed) Enricher failed for the username because request has malfunctioned (missing API_KEY): '{username.value}': {api_request.text}"
+                            },
+                        )
+
+                try:
+                    response_json = api_request.json()
+                except Exception as e:
+                    Logger.error(None, {"message": f"(UsernameToDehashed) Failed to parse JSON for {username.value}: {e}"})
+                    continue
+
+                dehashed_entries = response_json.get("entries", [])
+                if not dehashed_entries:
+                    Logger.error(
+                            self.sketch_id,
+                            {
+                                "message": f"(UsernameToDehashed) failed for the username: '{username.value}': {api_request.text}"
+                            },
+                        )
+                    continue
+
+
+                api_balance = response_json.get("balance")
+                if api_balance:
+                    Logger.info(self.sketch_id, f'(UsernameToDehashed) Your remaining API balance is {api_balance}.')
+
+                for entry in dehashed_entries:
+                    entry_email = entry.get("email")
+                    entry_phone = entry.get("phone")
+                    entry_name = entry.get("name")
+                    entry_socialmedia = entry.get("social")
+                    entry_ip = entry.get("ip_address")
+                    entry_username = entry.get("username")
+                    entry_dob = entry.get("dob")
+
+                    results.append(
+                        Individual(
+                            full_name=entry_name[0] if entry_name else None, 
+                            birth_date=entry_dob[0] if entry_dob else None, 
+                            email_addresses=entry_email if entry_email else None, 
+                            phone_numbers=entry_phone if entry_phone else None, 
+                            social_media_profiles=entry_socialmedia if entry_socialmedia else None, 
+                            ip_addresses=entry_ip if entry_ip else None, 
+                            usernames=entry_username if entry_username else None
+                            )
+                        )
+            except Exception as e:
+                Logger.error(self.sketch_id, {"message": f"(UsernameToDehashed) Exception while querying username {username.value}: {e}"})
+        
+        return results
+
+    def postprocess(
+        self, results: List[OutputType], input_data: List[InputType] = None
+    ) -> List[OutputType]:
+        if not self._graph_service:
+            return results
+
+        if input_data and self._graph_service:
+            for username in input_data:
+                for individual in results:
+                    self.create_node(username)
+                    self.create_node(individual)
+
+                    # Create relationship
+                    self.create_relationship(username, individual, "CONNECTION_WITH")
+                    self.log_graph_message(
+                        f"(UsernameToDehashed) Successfully found individual connections with {username.value}. "
+                    )
+
+        return results
+
+
+# Make types available at module level for easy access
+InputType = UsernameToDehashed.InputType
+OutputType = UsernameToDehashed.OutputType

flowsint-enrichers/src/flowsint_enrichers/social/to_maigret.py

@@ -53,7 +53,7 @@ class MaigretEnricher(Enricher):
             return results
 
         try:
-            with open(output_file, "r") as f:
+            with open(output_file, "r", encoding="utf-8") as f:
                 raw_data = json.load(f)
         except Exception as e:
             Logger.error(

flowsint-enrichers/src/flowsint_enrichers/social/to_sherlock.py

@@ -63,7 +63,7 @@ class SherlockEnricher(Enricher):
                     continue
 
                 found_accounts = {}
-                with open(output_file, "r") as f:
+                with open(output_file, "r", encoding="utf-8") as f:
                     for line in f:
                         line = line.strip()
                         if line and line.startswith("http"):

flowsint-enrichers/src/flowsint_enrichers/website/to_subdomain.py

@@ -0,0 +1,127 @@
+from typing import List, Union, Optional, Dict, Any
+from flowsint_core.core.enricher_base import Enricher
+from flowsint_enrichers.registry import flowsint_enricher
+from flowsint_core.core.logger import Logger
+from flowsint_types.website import Website
+import requests
+import os
+
+@flowsint_enricher
+class WebsiteToSubdomains(Enricher):
+    """[c99.nl] Performs an automated scan to find subdomains of the given domain."""
+
+    InputType = Website
+    OutputType = Website
+
+    def __init__(
+        self,
+        sketch_id: Optional[str] = None,
+        scan_id: Optional[str] = None,
+        vault=None,
+        params: Optional[Dict[str, Any]] = None,
+    ):
+        super().__init__(
+            sketch_id=sketch_id,
+            scan_id=scan_id,
+            params_schema=self.get_params_schema(),
+            vault=vault,
+            params=params,
+        )
+
+    @classmethod
+    def get_params_schema(cls) -> List[Dict[str, Any]]:
+        return [
+            {
+                "name": "C99_API_KEY",
+                "type": "vaultSecret",
+                "description": "Your C99.nl API key",
+                "required": True,
+            },
+        ]
+
+    @classmethod
+    def name(cls) -> str:
+        return "website_to_subdomains"
+
+    @classmethod
+    def category(cls) -> str:
+        return "Website"
+
+    @classmethod
+    def key(cls) -> str:
+        return "url"
+
+    async def scan(self, data: List[InputType]) -> List[OutputType]:
+        results = []
+
+        api_key = self.get_secret("C99_API_KEY", os.getenv("C99_API_KEY"))
+
+        for website in data:
+            try:
+                api_request = requests.get(f'https://api.c99.nl/subdomainfinder?key={api_key}&domain={website.url}', timeout=30)
+
+                if api_request.status_code != 200:
+                    Logger.error(
+                        self.sketch_id,
+                        {
+                            "message": f"(WebsiteToSubdomains) Enricher failed for '{website.url}': {api_request.text}"
+                        },
+                    )
+                    continue
+
+                response_json = api_request.json()
+
+                for subdomain in response_json.get("subdomains", []):
+                    url = subdomain.get("subdomain")
+                    if website.url not in [f'http://{url}', f'https://{url}', f'http://www.{url}', f'https://www.{url}']:
+                        ip = subdomain.get("ip")
+
+                        cloudflare = []
+                        if subdomain.get("cloudflare") is True:
+                            cloudflare.append("Cloudflare")
+                        else:
+                            cloudflare = None
+
+                        results.append(
+                            Website(
+                                url=f"http://{url}", description=f"IP address: {ip}", technologies=cloudflare
+                            )
+                        )
+
+            except Exception as e:
+                Logger.error(
+                    self.sketch_id,
+                    {
+                        "message": f"(WebsiteToSubdomains) Failed to run enricher for {website.url}: {e}"
+                    },
+                )
+
+        return results
+
+    def postprocess(self, results: List[OutputType], original_input: List[InputType]) -> List[OutputType]:
+        if not self._graph_service:
+            return results
+
+        for subdomain in results:
+            try:
+                self.create_node(subdomain)
+
+                for url in original_input:
+                    self.create_node(url)
+                    self.create_relationship(url, subdomain, "IS_SUBDOMAIN")
+                    self.log_graph_message(
+                        f"(WebsiteToSubdomains) Found {url.url}'s subdomain - '{subdomain.url}'"
+                    )
+
+            except Exception as e:
+                Logger.error(
+                    self.sketch_id,
+                    {
+                        "message": f"(WebsiteToSubdomains) relationship error: {e}"
+                    },
+                )
+
+        return results
+
+InputType = WebsiteToSubdomains.InputType
+OutputType = WebsiteToSubdomains.OutputType

flowsint-types/src/flowsint_types/flowsint_base.py

@@ -1,6 +1,6 @@
 from typing import Optional
 
-from pydantic import BaseModel, Field
+from pydantic import BaseModel, ConfigDict, Field
 
 
 class FlowsintType(BaseModel):
@@ -24,6 +24,5 @@ class FlowsintType(BaseModel):
         title="Label",
     )
 
-    # Allow extra keys to support for additional properties from user
-    class ConfigDict:
-        extra = "allow"
+    # Allow extra keys to support additional properties from the user
+    model_config = ConfigDict(extra="allow")

yarn.lock

@@ -956,6 +956,11 @@
     pbf "^4.0.1"
     supercluster "^8.0.1"
 
+"@microsoft/fetch-event-source@^2.0.1":
+  version "2.0.1"
+  resolved "https://registry.yarnpkg.com/@microsoft/fetch-event-source/-/fetch-event-source-2.0.1.tgz#9ceecc94b49fbaa15666e38ae8587f64acce007d"
+  integrity sha512-W6CLUJ2eBMw3Rec70qrsEW0jOm/3twwJv21mrmj2yORiaVmVYGS4sSS5yUwvQc1ZlDLYGPnClVWmUUMagKNsfA==
+
 "@monaco-editor/loader@^1.5.0":
   version "1.7.0"
   resolved "https://registry.npmjs.org/@monaco-editor/loader/-/loader-1.7.0.tgz"
@@ -2580,6 +2585,65 @@
     "@types/babel__core" "^7.20.5"
     react-refresh "^0.17.0"
 
+"@vitest/expect@2.1.9":
+  version "2.1.9"
+  resolved "https://registry.yarnpkg.com/@vitest/expect/-/expect-2.1.9.tgz#b566ea20d58ea6578d8dc37040d6c1a47ebe5ff8"
+  integrity sha512-UJCIkTBenHeKT1TTlKMJWy1laZewsRIzYighyYiJKZreqtdxSos/S1t+ktRMQWu2CKqaarrkeszJx1cgC5tGZw==
+  dependencies:
+    "@vitest/spy" "2.1.9"
+    "@vitest/utils" "2.1.9"
+    chai "^5.1.2"
+    tinyrainbow "^1.2.0"
+
+"@vitest/mocker@2.1.9":
+  version "2.1.9"
+  resolved "https://registry.yarnpkg.com/@vitest/mocker/-/mocker-2.1.9.tgz#36243b27351ca8f4d0bbc4ef91594ffd2dc25ef5"
+  integrity sha512-tVL6uJgoUdi6icpxmdrn5YNo3g3Dxv+IHJBr0GXHaEdTcw3F+cPKnsXFhli6nO+f/6SDKPHEK1UN+k+TQv0Ehg==
+  dependencies:
+    "@vitest/spy" "2.1.9"
+    estree-walker "^3.0.3"
+    magic-string "^0.30.12"
+
+"@vitest/pretty-format@2.1.9", "@vitest/pretty-format@^2.1.9":
+  version "2.1.9"
+  resolved "https://registry.yarnpkg.com/@vitest/pretty-format/-/pretty-format-2.1.9.tgz#434ff2f7611689f9ce70cd7d567eceb883653fdf"
+  integrity sha512-KhRIdGV2U9HOUzxfiHmY8IFHTdqtOhIzCpd8WRdJiE7D/HUcZVD0EgQCVjm+Q9gkUXWgBvMmTtZgIG48wq7sOQ==
+  dependencies:
+    tinyrainbow "^1.2.0"
+
+"@vitest/runner@2.1.9":
+  version "2.1.9"
+  resolved "https://registry.yarnpkg.com/@vitest/runner/-/runner-2.1.9.tgz#cc18148d2d797fd1fd5908d1f1851d01459be2f6"
+  integrity sha512-ZXSSqTFIrzduD63btIfEyOmNcBmQvgOVsPNPe0jYtESiXkhd8u2erDLnMxmGrDCwHCCHE7hxwRDCT3pt0esT4g==
+  dependencies:
+    "@vitest/utils" "2.1.9"
+    pathe "^1.1.2"
+
+"@vitest/snapshot@2.1.9":
+  version "2.1.9"
+  resolved "https://registry.yarnpkg.com/@vitest/snapshot/-/snapshot-2.1.9.tgz#24260b93f798afb102e2dcbd7e61c6dfa118df91"
+  integrity sha512-oBO82rEjsxLNJincVhLhaxxZdEtV0EFHMK5Kmx5sJ6H9L183dHECjiefOAdnqpIgT5eZwT04PoggUnW88vOBNQ==
+  dependencies:
+    "@vitest/pretty-format" "2.1.9"
+    magic-string "^0.30.12"
+    pathe "^1.1.2"
+
+"@vitest/spy@2.1.9":
+  version "2.1.9"
+  resolved "https://registry.yarnpkg.com/@vitest/spy/-/spy-2.1.9.tgz#cb28538c5039d09818b8bfa8edb4043c94727c60"
+  integrity sha512-E1B35FwzXXTs9FHNK6bDszs7mtydNi5MIfUWpceJ8Xbfb1gBMscAnwLbEu+B44ed6W3XjL9/ehLPHR1fkf1KLQ==
+  dependencies:
+    tinyspy "^3.0.2"
+
+"@vitest/utils@2.1.9":
+  version "2.1.9"
+  resolved "https://registry.yarnpkg.com/@vitest/utils/-/utils-2.1.9.tgz#4f2486de8a54acf7ecbf2c5c24ad7994a680a6c1"
+  integrity sha512-v0psaMSkNJ3A2NMrUEHFRzJtDPFn+/VWZ5WxImB21T9fjucJRmS7xCS3ppEnARb9y11OAzaD+P2Ps+b+BGX5iQ==
+  dependencies:
+    "@vitest/pretty-format" "2.1.9"
+    loupe "^3.1.2"
+    tinyrainbow "^1.2.0"
+
 "@webgpu/types@^0.1.69":
   version "0.1.69"
   resolved "https://registry.npmjs.org/@webgpu/types/-/types-0.1.69.tgz"
@@ -2831,6 +2895,11 @@ arrify@^1.0.1:
   resolved "https://registry.npmjs.org/arrify/-/arrify-1.0.1.tgz"
   integrity sha512-3CYzex9M9FGQjCGMGyi6/31c8GJbgb0qGyrx5HWxPd0aCwh4cB2YjMb2Xf9UuoogrMrlO9cTqnB5rI5GHZTcUA==
 
+assertion-error@^2.0.1:
+  version "2.0.1"
+  resolved "https://registry.yarnpkg.com/assertion-error/-/assertion-error-2.0.1.tgz#f641a196b335690b1070bf00b6e7593fec190bf7"
+  integrity sha512-Izi8RQcffqCeNVgFigKli1ssklIbpHnCYc6AknXGYoB6grJqyeby7jv12JUQgmTAnIDnbck1uxksT4dzN3PWBA==
+
 assign-symbols@^1.0.0:
   version "1.0.0"
   resolved "https://registry.npmjs.org/assign-symbols/-/assign-symbols-1.0.0.tgz"
@@ -2990,6 +3059,11 @@ bytewise@^1.1.0:
     bytewise-core "^1.2.2"
     typewise "^1.0.3"
 
+cac@^6.7.14:
+  version "6.7.14"
+  resolved "https://registry.yarnpkg.com/cac/-/cac-6.7.14.tgz#804e1e6f506ee363cb0e3ccbb09cad5dd9870959"
+  integrity sha512-b6Ilus+c3RrdDk+JhLKUAQfzzgLEPy6wcXqS7f/xe1EETvsDP6GORG7SFuOs6cID5YkqchW/LXZbX5bc8j7ZcQ==
+
 cachedir@2.3.0:
   version "2.3.0"
   resolved "https://registry.npmjs.org/cachedir/-/cachedir-2.3.0.tgz"
@@ -3057,6 +3131,17 @@ ccount@^2.0.0:
   resolved "https://registry.npmjs.org/ccount/-/ccount-2.0.1.tgz"
   integrity sha512-eyrF0jiFpY+3drT6383f1qhkbGsLSifNAjA61IUjZjmLCWjItY6LB9ft9YhoDgwfmclB2zhu51Lc7+95b8NRAg==
 
+chai@^5.1.2:
+  version "5.3.3"
+  resolved "https://registry.yarnpkg.com/chai/-/chai-5.3.3.tgz#dd3da955e270916a4bd3f625f4b919996ada7e06"
+  integrity sha512-4zNhdJD/iOjSH0A05ea+Ke6MU5mmpQcbQsSOkgdaUMJ9zTlDTD/GYlwohmIE2u0gaxHYiVHEn1Fw9mZ/ktJWgw==
+  dependencies:
+    assertion-error "^2.0.1"
+    check-error "^2.1.1"
+    deep-eql "^5.0.1"
+    loupe "^3.1.0"
+    pathval "^2.0.0"
+
 chalk@^2.4.1, chalk@^2.4.2:
   version "2.4.2"
   resolved "https://registry.npmjs.org/chalk/-/chalk-2.4.2.tgz"
@@ -3119,6 +3204,11 @@ chardet@^0.7.0:
   resolved "https://registry.npmjs.org/chardet/-/chardet-0.7.0.tgz"
   integrity sha512-mT8iDcrh03qDGRRmoA2hmBJnxpllMR+0/0qlzjqZES6NdiWDcZkCNAk4rPFZ9Q85r27unkiNNg8ZOiwZXBHwcA==
 
+check-error@^2.1.1:
+  version "2.1.3"
+  resolved "https://registry.yarnpkg.com/check-error/-/check-error-2.1.3.tgz#2427361117b70cca8dc89680ead32b157019caf5"
+  integrity sha512-PAJdDJusoxnwm1VwW07VWwUN1sl7smmC3OKggvndJFadxxDRyFJBX/ggnu/KE4kQAB7a3Dp8f/YXC1FlUprWmA==
+
 chokidar@^3.6.0:
   version "3.6.0"
   resolved "https://registry.npmjs.org/chokidar/-/chokidar-3.6.0.tgz"
@@ -3862,7 +3952,7 @@ dateformat@^3.0.0:
   resolved "https://registry.npmjs.org/dateformat/-/dateformat-3.0.3.tgz"
   integrity sha512-jyCETtSl3VMZMWeRo7iY1FL19ges1t55hMo5yaam4Jrsm5EPL89UQkoQRyiI+Yf4k8r2ZpdngkV8hr1lIdjb3Q==
 
-debug@^4.0.0, debug@^4.1.0, debug@^4.3.1, debug@^4.3.2:
+debug@^4.0.0, debug@^4.1.0, debug@^4.3.1, debug@^4.3.2, debug@^4.3.7:
   version "4.4.3"
   resolved "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz"
   integrity sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==
@@ -3904,6 +3994,11 @@ deep-diff@^1.0.2:
   resolved "https://registry.npmjs.org/deep-diff/-/deep-diff-1.0.2.tgz"
   integrity sha512-aWS3UIVH+NPGCD1kki+DCU9Dua032iSsO43LqQpcs4R3+dVv7tX0qBGjiVHJHjplsoUM2XRO/KB92glqc68awg==
 
+deep-eql@^5.0.1:
+  version "5.0.2"
+  resolved "https://registry.yarnpkg.com/deep-eql/-/deep-eql-5.0.2.tgz#4b756d8d770a9257300825d52a2c2cff99c3a341"
+  integrity sha512-h5k/5U50IJJFpzfL6nO9jaaumfjO/f2NjK/oYB2Djzm4p9L+3T9qWpZqZ2hAbLPuuYq9wrU08WQyBTL5GbPk5Q==
+
 deep-is@^0.1.3:
   version "0.1.4"
   resolved "https://registry.npmjs.org/deep-is/-/deep-is-0.1.4.tgz"
@@ -4208,6 +4303,11 @@ es-iterator-helpers@^1.2.1:
     iterator.prototype "^1.1.4"
     safe-array-concat "^1.1.3"
 
+es-module-lexer@^1.5.4:
+  version "1.7.0"
+  resolved "https://registry.yarnpkg.com/es-module-lexer/-/es-module-lexer-1.7.0.tgz#9159601561880a85f2734560a9099b2c31e5372a"
+  integrity sha512-jEQoCwk8hyb2AZziIOLhDqpm5+2ww5uIE6lkO/6jcOCusfk6LhMHpXXfBLXTZ7Ydyt0j4VoUQv6uGNYbdW+kBA==
+
 es-object-atoms@^1.0.0, es-object-atoms@^1.1.1:
   version "1.1.1"
   resolved "https://registry.npmjs.org/es-object-atoms/-/es-object-atoms-1.1.1.tgz"
@@ -4441,6 +4541,13 @@ estree-util-is-identifier-name@^3.0.0:
   resolved "https://registry.npmjs.org/estree-util-is-identifier-name/-/estree-util-is-identifier-name-3.0.0.tgz"
   integrity sha512-hFtqIDZTIUZ9BXLb8y4pYGyk6+wekIivNVTcmvk8NoOh+VeRn5y6cEHzbURrWbfp1fIqdVipilzj+lfaadNZmg==
 
+estree-walker@^3.0.3:
+  version "3.0.3"
+  resolved "https://registry.yarnpkg.com/estree-walker/-/estree-walker-3.0.3.tgz#67c3e549ec402a487b4fc193d1953a524752340d"
+  integrity sha512-7RUKfXgSMMkzt6ZuXmqapOurLGPPfgj6l9uRZ7lRGolvk0y2yocc35LdcxKC5PQZdn2DMqioAQ2NoWcrTKmm6g==
+  dependencies:
+    "@types/estree" "^1.0.0"
+
 esutils@^2.0.2:
   version "2.0.3"
   resolved "https://registry.npmjs.org/esutils/-/esutils-2.0.3.tgz"
@@ -4468,6 +4575,11 @@ expand-tilde@^2.0.0, expand-tilde@^2.0.2:
   dependencies:
     homedir-polyfill "^1.0.1"
 
+expect-type@^1.1.0:
+  version "1.3.0"
+  resolved "https://registry.yarnpkg.com/expect-type/-/expect-type-1.3.0.tgz#0d58ed361877a31bbc4dd6cf71bbfef7faf6bd68"
+  integrity sha512-knvyeauYhqjOYvQ66MznSMs83wmHrCycNEN6Ao+2AeYEfxUIkuiVxdEa1qlGEPK+We3n0THiDciYSsCcgW/DoA==
+
 extend-shallow@^2.0.1:
   version "2.0.1"
   resolved "https://registry.npmjs.org/extend-shallow/-/extend-shallow-2.0.1.tgz"
@@ -5984,6 +6096,11 @@ loose-envify@^1.4.0:
   dependencies:
     js-tokens "^3.0.0 || ^4.0.0"
 
+loupe@^3.1.0, loupe@^3.1.2:
+  version "3.2.1"
+  resolved "https://registry.yarnpkg.com/loupe/-/loupe-3.2.1.tgz#0095cf56dc5b7a9a7c08ff5b1a8796ec8ad17e76"
+  integrity sha512-CdzqowRJCeLU72bHvWqwRBBlLcMEtIvGrlvef74kMnV2AolS9Y8xUv1I0U/MNAWMhBlKIoyuEgoJ0t/bbwHbLQ==
+
 lowlight@^1.17.0:
   version "1.20.0"
   resolved "https://registry.npmjs.org/lowlight/-/lowlight-1.20.0.tgz"
@@ -6020,7 +6137,7 @@ lucide-react@^0.511.0:
   resolved "https://registry.npmjs.org/lucide-react/-/lucide-react-0.511.0.tgz"
   integrity sha512-VK5a2ydJ7xm8GvBeKLS9mu1pVK6ucef9780JVUjw6bAjJL/QXnd4Y0p7SPeOUMC27YhzNCZvm5d/QX0Tp3rc0w==
 
-magic-string@^0.30.19:
+magic-string@^0.30.12, magic-string@^0.30.19:
   version "0.30.21"
   resolved "https://registry.npmjs.org/magic-string/-/magic-string-0.30.21.tgz"
   integrity sha512-vd2F4YUyEXKGcLHoq+TEyCjxueSeHnFxyyjNp80yg0XV4vUhnDer/lvvlqM/arB5bXQN5K2/3oinyCRyx8T2CQ==
@@ -7026,11 +7143,21 @@ path-type@^3.0.0:
   dependencies:
     pify "^3.0.0"
 
+pathe@^1.1.2:
+  version "1.1.2"
+  resolved "https://registry.yarnpkg.com/pathe/-/pathe-1.1.2.tgz#6c4cb47a945692e48a1ddd6e4094d170516437ec"
+  integrity sha512-whLdWMYL2TwI08hn8/ZqAbrVemu0LNaNNJZX73O6qaIdCTfXutsLhMkjdENX0qhsQ9uIimo4/aQOmXkoon2nDQ==
+
 pathe@^2.0.3:
   version "2.0.3"
   resolved "https://registry.npmjs.org/pathe/-/pathe-2.0.3.tgz"
   integrity sha512-WUjGcAqP1gQacoQe+OBJsFA7Ld4DyXuUIjZ5cc75cLHvJ7dtNsTugphxIADwspS+AraAUePCKrSVtPLFj/F88w==
 
+pathval@^2.0.0:
+  version "2.0.1"
+  resolved "https://registry.yarnpkg.com/pathval/-/pathval-2.0.1.tgz#8855c5a2899af072d6ac05d11e46045ad0dc605d"
+  integrity sha512-//nshmD55c46FuFw26xV/xFAaB5HF9Xdap7HJBBnrKdAd6/GxDBaNA1870O79+9ueg61cZLSVc+OaFlfmObYVQ==
+
 pbf@^4.0.1:
   version "4.0.1"
   resolved "https://registry.npmjs.org/pbf/-/pbf-4.0.1.tgz"
@@ -8041,6 +8168,11 @@ side-channel@^1.1.0:
     side-channel-map "^1.0.1"
     side-channel-weakmap "^1.0.2"
 
+siginfo@^2.0.0:
+  version "2.0.0"
+  resolved "https://registry.yarnpkg.com/siginfo/-/siginfo-2.0.0.tgz#32e76c70b79724e3bb567cb9d543eb858ccfaf30"
+  integrity sha512-ybx0WO1/8bSBLEWXZvEd7gMW3Sn3JFlW3TvX1nREbDLRNQNaeNN8WK0meBwPdAaOI7TtRRRJn/Es1zhrrCHu7g==
+
 signal-exit@^3.0.2:
   version "3.0.7"
   resolved "https://registry.npmjs.org/signal-exit/-/signal-exit-3.0.7.tgz"
@@ -8150,6 +8282,11 @@ split@^1.0.0:
   dependencies:
     through "2"
 
+stackback@0.0.2:
+  version "0.0.2"
+  resolved "https://registry.yarnpkg.com/stackback/-/stackback-0.0.2.tgz#1ac8a0d9483848d1695e418b6d031a3c3ce68e3b"
+  integrity sha512-1XMJE5fQo1jGH6Y/7ebnwPOBEkIEnT4QF32d5R1+VXdXveM0IBMJt8zfaxX1P3QhVwrYe+576+jkANtSS2mBbw==
+
 standard-version@^9.5.0:
   version "9.5.0"
   resolved "https://registry.npmjs.org/standard-version/-/standard-version-9.5.0.tgz"
@@ -8175,6 +8312,11 @@ state-local@^1.0.6:
   resolved "https://registry.npmjs.org/state-local/-/state-local-1.0.7.tgz"
   integrity sha512-HTEHMNieakEnoe33shBYcZ7NX83ACUjCu8c40iOGEZsngj9zRnkqS9j1pqQPXwobB0ZcVTk27REb7COQ0UR59w==
 
+std-env@^3.8.0:
+  version "3.10.0"
+  resolved "https://registry.yarnpkg.com/std-env/-/std-env-3.10.0.tgz#d810b27e3a073047b2b5e40034881f5ea6f9c83b"
+  integrity sha512-5GS12FdOZNliM5mAOxFRg7Ir0pWz8MdpYm6AY6VPkGpbA7ZzmbzNcBJQ0GPvvyWgcY7QAhCgf9Uy89I03faLkg==
+
 stop-iteration-iterator@^1.1.0:
   version "1.1.0"
   resolved "https://registry.npmjs.org/stop-iteration-iterator/-/stop-iteration-iterator-1.1.0.tgz"
@@ -8430,11 +8572,21 @@ tiny-warning@^1.0.3:
   resolved "https://registry.npmjs.org/tiny-warning/-/tiny-warning-1.0.3.tgz"
   integrity sha512-lBN9zLN/oAf68o3zNXYrdCt1kP8WsiGW8Oo2ka41b2IM5JL/S1CTyX1rW0mb/zSuJun0ZUrDxx4sqvYS2FWzPA==
 
+tinybench@^2.9.0:
+  version "2.9.0"
+  resolved "https://registry.yarnpkg.com/tinybench/-/tinybench-2.9.0.tgz#103c9f8ba6d7237a47ab6dd1dcff77251863426b"
+  integrity sha512-0+DUvqWMValLmha6lr4kD8iAMK1HzV0/aKnCtWb9v9641TnP/MFb7Pc2bxoxQjTXAErryXVgUOfv2YqNllqGeg==
+
 tinycolor2@^1.6.0:
   version "1.6.0"
   resolved "https://registry.npmjs.org/tinycolor2/-/tinycolor2-1.6.0.tgz"
   integrity sha512-XPaBkWQJdsf3pLKJV9p4qN/S+fm2Oj8AIPo1BTUhg5oxkvm9+SVEGFdhyOz7tTdUTfvxMiAs4sp6/eZO2Ew+pw==
 
+tinyexec@^0.3.1:
+  version "0.3.2"
+  resolved "https://registry.yarnpkg.com/tinyexec/-/tinyexec-0.3.2.tgz#941794e657a85e496577995c6eef66f53f42b3d2"
+  integrity sha512-KQQR9yN7R5+OSwaK0XQoj22pwHoTlgYqmUscPYoknOoWCWfj/5/ABTMRi69FrKU5ffPVh5QcFikpWJI/P1ocHA==
+
 tinyexec@^1.0.0:
   version "1.0.2"
   resolved "https://registry.npmjs.org/tinyexec/-/tinyexec-1.0.2.tgz"
@@ -8448,11 +8600,26 @@ tinyglobby@^0.2.15:
     fdir "^6.5.0"
     picomatch "^4.0.3"
 
+tinypool@^1.0.1:
+  version "1.1.1"
+  resolved "https://registry.yarnpkg.com/tinypool/-/tinypool-1.1.1.tgz#059f2d042bd37567fbc017d3d426bdd2a2612591"
+  integrity sha512-Zba82s87IFq9A9XmjiX5uZA/ARWDrB03OHlq+Vw1fSdt0I+4/Kutwy8BP4Y/y/aORMo61FQ0vIb5j44vSo5Pkg==
+
 tinyqueue@^3.0.0:
   version "3.0.0"
   resolved "https://registry.npmjs.org/tinyqueue/-/tinyqueue-3.0.0.tgz"
   integrity sha512-gRa9gwYU3ECmQYv3lslts5hxuIa90veaEcxDYuu3QGOIAEM2mOZkVHp48ANJuu1CURtRdHKUBY5Lm1tHV+sD4g==
 
+tinyrainbow@^1.2.0:
+  version "1.2.0"
+  resolved "https://registry.yarnpkg.com/tinyrainbow/-/tinyrainbow-1.2.0.tgz#5c57d2fc0fb3d1afd78465c33ca885d04f02abb5"
+  integrity sha512-weEDEq7Z5eTHPDh4xjX789+fHfF+P8boiFB+0vbWzpbnbsEr/GRaohi/uMKxg8RZMXnl1ItAi/IUHWMsjDV7kQ==
+
+tinyspy@^3.0.2:
+  version "3.0.2"
+  resolved "https://registry.yarnpkg.com/tinyspy/-/tinyspy-3.0.2.tgz#86dd3cf3d737b15adcf17d7887c84a75201df20a"
+  integrity sha512-n1cw8k1k0x4pgA2+9XrOkFydTerNcJ1zWCO5Nn9scWHTD+5tp8dghT2x1uduQePZTZgd3Tupf+x9BxJjeJi77Q==
+
 tippy.js@^6.3.7:
   version "6.3.7"
   resolved "https://registry.npmjs.org/tippy.js/-/tippy.js-6.3.7.tgz"
@@ -8838,7 +9005,18 @@ victory-vendor@^36.6.8:
     d3-time "^3.0.0"
     d3-timer "^3.0.1"
 
-vite@^5.3.1:
+vite-node@2.1.9:
+  version "2.1.9"
+  resolved "https://registry.yarnpkg.com/vite-node/-/vite-node-2.1.9.tgz#549710f76a643f1c39ef34bdb5493a944e4f895f"
+  integrity sha512-AM9aQ/IPrW/6ENLQg3AGY4K1N2TGZdR5e4gu/MmmR2xR3Ll1+dib+nook92g4TV3PXVyeyxdWwtaCAiUL0hMxA==
+  dependencies:
+    cac "^6.7.14"
+    debug "^4.3.7"
+    es-module-lexer "^1.5.4"
+    pathe "^1.1.2"
+    vite "^5.0.0"
+
+vite@^5.0.0, vite@^5.3.1:
   version "5.4.21"
   resolved "https://registry.npmjs.org/vite/-/vite-5.4.21.tgz"
   integrity sha512-o5a9xKjbtuhY6Bi5S3+HvbRERmouabWbyUcpXXUA1u+GNUKoROi9byOJ8M0nHbHYHkYICiMlqxkg1KkYmm25Sw==
@@ -8863,6 +9041,32 @@ vite@^7.1.7:
   optionalDependencies:
     fsevents "~2.3.3"
 
+vitest@^2.1.9:
+  version "2.1.9"
+  resolved "https://registry.yarnpkg.com/vitest/-/vitest-2.1.9.tgz#7d01ffd07a553a51c87170b5e80fea3da7fb41e7"
+  integrity sha512-MSmPM9REYqDGBI8439mA4mWhV5sKmDlBKWIYbA3lRb2PTHACE0mgKwA8yQ2xq9vxDTuk4iPrECBAEW2aoFXY0Q==
+  dependencies:
+    "@vitest/expect" "2.1.9"
+    "@vitest/mocker" "2.1.9"
+    "@vitest/pretty-format" "^2.1.9"
+    "@vitest/runner" "2.1.9"
+    "@vitest/snapshot" "2.1.9"
+    "@vitest/spy" "2.1.9"
+    "@vitest/utils" "2.1.9"
+    chai "^5.1.2"
+    debug "^4.3.7"
+    expect-type "^1.1.0"
+    magic-string "^0.30.12"
+    pathe "^1.1.2"
+    std-env "^3.8.0"
+    tinybench "^2.9.0"
+    tinyexec "^0.3.1"
+    tinypool "^1.0.1"
+    tinyrainbow "^1.2.0"
+    vite "^5.0.0"
+    vite-node "2.1.9"
+    why-is-node-running "^2.3.0"
+
 w3c-keyname@^2.2.0:
   version "2.2.8"
   resolved "https://registry.npmjs.org/w3c-keyname/-/w3c-keyname-2.2.8.tgz"
@@ -8947,6 +9151,14 @@ which@^2.0.1:
   dependencies:
     isexe "^2.0.0"
 
+why-is-node-running@^2.3.0:
+  version "2.3.0"
+  resolved "https://registry.yarnpkg.com/why-is-node-running/-/why-is-node-running-2.3.0.tgz#a3f69a97107f494b3cdc3bdddd883a7d65cebf04"
+  integrity sha512-hUrmaWBdVDcxvYqnyh09zunKzROWjbZTiNy8dBEjkS7ehEDQibXJ7XvlmtbwuTclUiIyN+CyXQD4Vmko8fNm8w==
+  dependencies:
+    siginfo "^2.0.0"
+    stackback "0.0.2"
+
 word-wrap@^1.0.3, word-wrap@^1.2.5:
   version "1.2.5"
   resolved "https://registry.npmjs.org/word-wrap/-/word-wrap-1.2.5.tgz"