Nightly Data Dumps

Raw data for custom in-house analysis

For Enterprise customers, Rep.ai offers nightly data dumps of all visitor activity we logged for that day. Data will be uploaded to your storage provider of choice with a consistent format and naming scheme so that you can automate integrating the data into your own custom tooling.

Contact our sales team at [email protected] to get your organization set up with data dumps.

Example Data

{
  "version": "1.0.0",
  "start_time": 1632787200000,
  "end_time": 1632873600000,
  "users": {
    "1495955663": {
      "id": "1495955663",
      "email_address": "[email protected]",
      "display_name": "Will O'Beirne",
      "title": "Sales Representative",
      "active": true
    }
  },
  "unique_visitors": {
    "e63fbcf8-6be1-4f5b-bd1c-023db6afc4fb": {
      "id": "e63fbcf8-6be1-4f5b-bd1c-023db6afc4fb",
      "custom_id": "83493423",
      "custom_metadata": {
        "display_name": "Visitor Name",
        "email": "[email protected]",
        "any_other_custom_key": 123
      }
    }
  },
  "unique_visitor_sessions": {
    "00a631ba-ccba-4994-a52f-0d6c2bb69608": {
      "id": "00a631ba-ccba-4994-a52f-0d6c2bb69608",
      "session_group": "00a631ba-ccba-4994-a52f-0d6c2bb69608",
      "unique_visitor_id": "e63fbcf8-6be1-4f5b-bd1c-023db6afc4fb",
      "referrer": "http://google.com/",
      "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Safari/605.1.15",
      "start_time": 1632937435000,
      "end_time": 1632937477000
    }
  },
  "unique_visitor_session_actions": [
    {
      "id": "2147483409",
      "action": "REQUESTED_HELP",
      "session_id": "00a631ba-ccba-4994-a52f-0d6c2bb69608",
      "timestamp": 1632937436000
    },
    {
      "id": "1058988876",
      "action": "ALERT",
      "session_id": "00a631ba-ccba-4994-a52f-0d6c2bb69608",
      "timestamp": 1632937437000
    },
    {
      "id": "2085049932",
      "action": "PAGE_VISIT",
      "session_id": "00a631ba-ccba-4994-a52f-0d6c2bb69608",
      "timestamp": 1632937438000,
      "data": {
        "url": "https://website.com",
        "url_title": "Test Website - Homepage"
      }
    },
    {
      "id": "1256524328",
      "action": "FEEDBACK",
      "session_id": "00a631ba-ccba-4994-a52f-0d6c2bb69608",
      "timestamp": 1632937439000,
      "data": {
        "stars": 5
      }
    }
  ],
  "unique_visitor_session_interactions": [
    {
      "id": "1056417425",
      "kind": "SPYING",
      "session_id": "00a631ba-ccba-4994-a52f-0d6c2bb69608",
      "user_id": "1495955663",
      "start_time": 1632937436000,
      "last_seen": 1632937436000,
      "accepted": true,
      "accept_time": 1632937439000
    },
    {
      "id": "77869321",
      "kind": "VIDEO",
      "session_id": "00a631ba-ccba-4994-a52f-0d6c2bb69608",
      "user_id": "1495955663",
      "start_time": 1632937437000,
      "last_seen": 1632937457000,
      "accepted": true,
      "accept_time": 1632937438000
    },
    {
      "id": "325040803",
      "kind": "TAKEOVER",
      "session_id": "00a631ba-ccba-4994-a52f-0d6c2bb69608",
      "user_id": "1495955663",
      "start_time": 1632937436000,
      "last_seen": 1632937436000,
      "accepted": false
    }
  ]
}

Additional Notes

Semantic versioning

Data dumps provide a version number that you can use to determine if your processor is able to handle the data. We use semantic versioning, with the following justifications:

• Patch - Fixes to issues in our reporting. The schema did not change in any meaningful way, but issues that were present in past exports may have been fixed.

• Minor - New keys or types of entries were added that provide more information, but the data is fully backwards compatible.

• Major - Meaningful and potentially breaking changes have happened in the structure of the schema.

Treat IDs as Strings

Despite some IDs being numeric, they should all be treated as strings as we may update their format in the future, and any numeric operations may break as a result.

Sessions & Session Groups

Sessions are single socket connections to the Rep.ai widget. Any full page load or disruption in the socket connection (e.g. going offline, being load balanced to a different server) will start a new session.

Session groups are collections of sessions that happened close together. Session groups are identified by using the ID of the earliest occurring session in the group, for instance sessions "A" and "B" that happened in close succession would both have session_group: "A".

Midnight Cutoff

Sessions that span the gap of the midnight cutoff may be in two days worth of dumps. A session that was ongoing at midnight will have an unset end_time.

Interaction Acceptance

All unique_visitor_session_interactions have an accept and accept_time field, even if there is no UX flow for "accepting" the interaction (e.g. answering a proactive call, allowing a takeover.) This is to allow for granular permissions and compatibility with all UX flows in case we change what a user can or cannot accept in the future.

Interactions that have no acceptance UX will always be accept: true with an accept_time that matches start_time.

New Actions & Interactions

New types of unique_visitor_session_action and unique_visitor_session_interaction will be added in the future as Rep.ai provides more ways to interact with visitors. These will result in minor version bumps, but should be backwards compatible if you filter or ignore unknown actions and interactions out.