> For the complete documentation index, see [llms.txt](https://docs.predicthq.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.predicthq.com/integrations/third-party-integrations/sftp.md).

# Receive data via SFTP

PredictHQ can deliver Event and Broadcast data via SFTP as regularly updated files - deploying verified real-world context into your environment when Snowflake or AWS Data Exchange aren't options. The result is the same: an always-current local store for model training, explainability, and [provisioned grounding](/integrations/integration-guides/provisioned-grounding.md), which you govern.

SFTP delivery follows the same full + incremental data model used by our other bulk data integrations.

## Overview

When integrating via SFTP, PredictHQ delivers data as full and incremental file exports (same delivery model as our AWS Data Exchange exports). Here’s what you can expect:

* **Initial Full Dump**: upon setup, you receive a full dataset covering all records you have access to.
* **Incremental Updates**: after the initial dump, we provide incremental updates containing only the new or changed records since the last update. By default, PredictHQ delivers these updates daily.
* **Occasional Full Dumps**: at times (either by request or operational need), we may deliver a full dump without prior notice. You can distinguish these by the presence of full (not incremental) in the filename.

## Processing order and change action

To maintain a complete and accurate dataset, process deliveries in the order PredictHQ delivers them (typically by the datetime folder, oldest to newest). Within a single delivery, you can process the individual files in any order or in parallel.

For incremental updates, make sure to check the `change_action` column to work out what action you should take the with record (`insert`, `update`, or `delete`).

### File naming

PredictHQ names files using this pattern:

```
<delivery_config_id>/<datetime>/<data_type>/<delivery_type>-part-<number>.<ext>
```

<table><thead><tr><th width="282.97265625">Field</th><th>Description</th></tr></thead><tbody><tr><td><code>delivery_config_id</code></td><td>PredictHQ identifier for your delivery configuration.</td></tr><tr><td><code>datetime</code></td><td>UTC export timestamp in <code>YYYYMMDD-HHMM</code> format.</td></tr><tr><td><code>data_type</code></td><td><p>The data being delivered. Can be one of the following:</p><ul><li><code>event</code></li><li><code>broadcast</code></li></ul></td></tr><tr><td><code>delivery_type</code></td><td><p>The delivery is either a full export of all available data or incremental based on the previous export. Possible values:</p><ul><li><code>full</code></li><li><code>incremental</code></li></ul></td></tr><tr><td><code>number</code></td><td><p>PredictHQ splits each delivery into multiple files to keep file sizes manageable. Individual files vary in size but don't exceed approximately 1 GB.</p><p>Files within a single delivery are not ordered and do not need to be processed sequentially. They can be processed in parallel. However, you must process deliveries themselves in chronological order to ensure data consistency.</p></td></tr><tr><td><code>ext</code></td><td><p>The file extension indicates the data structure and compression used.</p><p>If compression is used (configurable) PredictHQ compresses the data using Snappy and the file extension is prefixed with <code>snappy</code>.</p><p>Possible values:</p><ul><li><code>parquet</code></li><li><code>ndjson</code> - Newline-delimited JSON</li><li><code>csv</code> - Comma separated values</li><li><code>psv</code> - Pipe separated values</li></ul><p><br>E.g., <code>snappy.parquet</code></p></td></tr></tbody></table>

Within a single delivery, files can be processed in any order. Process deliveries themselves oldest to newest.

## Data retention

The SFTP server retains files for a limited period and automatically deletes them after that period.

Your ingestion process should fetch and persist data promptly. Do not rely on long-term availability of files on the SFTP server.

## Access and authentication

PredictHQ provides:

* An SFTP URL
* A private SSH key for authentication

You use these credentials to connect to the PredictHQ-managed SFTP server and fetch data on your own schedule.

## Typical ingestion flow

Implement an automated process with these steps:

1. Connect to the SFTP server.
2. List the available delivery folders.
3. Select the next unprocessed delivery.
4. Download all files for that delivery.
5. For incremental deliveries, apply records in order using `change_action`.
6. In your own system, record the delivery as processed.

## Backwards compatible changes <a href="#backwards-compatible-changes" id="backwards-compatible-changes"></a>

From time to time, PredictHQ may make backwards-compatible changes to SFTP exports, including:

* Adding new fields or columns
* Adding new files alongside existing ones
* Introducing new event categories or labels

Your ingestion process should tolerate these changes.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.predicthq.com/integrations/third-party-integrations/sftp.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.