Extraction Pipeline

FluidRAG can extract Freshdesk support tickets into your knowledge graph on a schedule, making customer issue history available for RAG queries alongside your other ingested documents.

How It Works

The pipeline runs in three stages:

Ticket Extraction — FluidRAG fetches tickets from Freshdesk (with descriptions and conversations included), paginating 100 per page. Internal notes are split from public conversations, and agent/requester IDs are resolved to display names. Each ticket is saved as a standalone Markdown file at /app/extraction_data/freshdesk/raw/ticket_{id}.md.
LLM Preprocessing — Each raw ticket is analyzed by an LLM to determine whether it contains viable knowledge for a company-wide RAG system. Viable tickets are rewritten into a structured document with # Topic, # Summary, and # Detailed Information sections, validated for completeness, and written to /app/extraction_data/freshdesk/processed/[{date}]ticket_{id}_{title}.md. Non-viable tickets (e.g., no technical content) are skipped.
Knowledge Graph Ingestion — Processed Markdown documents are ingested into FalkorDB with entity extraction, embedding generation, and relationship creation, making them searchable via graph and vector search.

Raw Ticket Structure

Each raw Markdown file contains frontmatter metadata followed by the ticket body:

Frontmatter Field	Description
`source`	`freshdesk`
`source_url`	Link to the ticket in Freshdesk
`source_id` / `ticket_id`	Freshdesk ticket ID
`title`	Ticket subject
`created_at` / `updated_at`	ISO 8601 timestamps (UTC)
`requester` / `responder`	Resolved display names
`tenant_name`	`cf_portal_name_ak` custom field
`priority` / `service_impact` / `resolution_detail`	Relevant custom field values
`status`	Ticket status code

The body includes the ticket Description, a Custom Fields section (with readable labels), Conversations (public), and Internal Notes (private). HTML is cleaned to plain text.

Configuring Extraction via Admin UI

Enable Freshdesk Extraction

Open the Admin UI at http://localhost/config-ui
Navigate to the Connectors page
Find Freshdesk in the connector list
Toggle Enabled to on

Setting	Default	Description
`freshdesk_enabled`	`true`	Enable the Freshdesk extraction source (hot-reload)
`freshdesk_domain`	(empty)	Freshdesk subdomain (e.g., `company.freshdesk.com`)

Manual Actions

Action	How
Trigger extraction now	Connectors → Freshdesk → Trigger Extraction (always runs a forced sync)
Reset sync state	Connectors → Freshdesk → Reset Sync State — clears the last sync timestamp so the next run extracts everything
Configure schedule	Set a cron expression (e.g., `0 /8 * *`) in the Freshdesk connector’s extraction settings

API Endpoints

All endpoints require admin authentication.

Endpoint	Method	Description
`/api/connectors/freshdesk`	`GET`	Get Freshdesk connector status and configuration
`/api/connectors/freshdesk`	`PATCH`	Enable/disable the Freshdesk connector (hot-reload)
`/api/connectors/freshdesk/extraction/trigger`	`POST`	Trigger extraction immediately (forced sync; optional `stages` body parameter)
`/api/connectors/freshdesk/extraction/status`	`GET`	Get Freshdesk extraction status
`/api/connectors/freshdesk/extraction/history`	`GET`	Get Freshdesk extraction history
`/api/connectors/freshdesk/extraction/reset`	`POST`	Reset sync state for full resync
`/api/connectors/freshdesk/source-config`	`GET`	Get Freshdesk source configuration
`/api/connectors/freshdesk/source-config`	`PATCH`	Update Freshdesk source settings
`/api/connectors/freshdesk/schedule`	`PATCH`	Set the extraction schedule (cron expression)
`/api/connectors/freshdesk/schedule`	`DELETE`	Remove the extraction schedule