Skip to main content
Version: Latest

Ingestion API

This page should get you up and running in no time. You only need two steps to start coding:

Authentication​

Make an authentication call to get an access (bearer) token OR a refresh token. For more information you can check quick start.

  1. Make an API call to the authentication endpoint:
curl --location --request POST 'https://api.alpha-sense.com/auth' \
    --header 'x-api-key: <YOUR API KEY>' \
    --header 'Content-Type: application/x-www-form-urlencoded' \
    --data-urlencode 'grant_type=password' \
    --data-urlencode 'username=<YOUR AlphaSense LOGIN EMAIL>' \
    --data-urlencode 'password=<YOUR AlphaSense LOGIN PASSWORD>' \
    --data-urlencode 'client_id=<YOUR CLIENT ID>' \
    --data-urlencode 'client_secret=<YOUR CLIENT SECRET>'
  1. Make an API call to the authentication endpoint to refresh access token if it has expired (refresh token can be found in the response of authentication API):
curl --location --request POST 'https://api.alpha-sense.com/auth' \
    --header 'x-api-key: <YOUR API KEY>' \
    --header 'Content-Type: application/x-www-form-urlencoded' \
    --data-urlencode 'grant_type=refresh_token' \
    --data-urlencode 'client_id=<YOUR CLIENT ID>' \
    --data-urlencode 'client_secret=<YOUR CLIENT SECRET>'
    --data-urlencode 'refresh_token=<REFRESH TOKEN>'
tip

For content ingestion we can perform the following operations Swagger. Use the token from step 1 to make API calls.

Upload a document to the user’s account​

Make an API call to the content ingestion upload endpoint for uploading a document to the user’s account:

curl --location --request POST 'https://research.alpha-sense.com/services/i/ingestion-api/v1/upload-document' \
  --header 'Authorization: Bearer <access_token>' \
  --header 'ClientId: <YOUR CLIENT ID>' \
  --form  'file' \
  --form  'attachments' \
  --form  'metadata(type:string)= <IngestionUploadMetadata>'

The following document types are supported:

pdf, html, htm, txt, doc, docx, xls, xlsx, ppt, pptx, msg, eml, csv, xlsb, xlsm, one, tsv, ods
NOTE

File is parent document and attachments are children of it which should be attached with main document.

Example Curl (with DEFAULT mode):

curl --location --request POST 'https://research.alpha-sense.com/services/i/ingestion-api/v1/upload-document' \
  --header 'Authorization: bearer token' \
  --header 'clientId: enterprise-sync' \
  --form 'file=@"/document.txt"' \
  --form 'attachments=@20221020 - EnterpriseSync - TSLA - Doc - 1 page.pdf;type=application/pdf' \
  --form 'attachments=@20230201 - Onenote - Demo Page 2 - 1 page.pdf;type=application/pdf' \
  --form 'metadata="{
            \"title\": \"Document Upload\",
            \"companies\": [{
                \"value\": \"US5949181045\",
                \"operation\": \"ADD\",
                \"identifier\": \"ISIN\",
                \"salience\": \"PRIMARY\"
            }],
            \"customTags\": [{
                \"name\": \"pb_tag\",
                \"visibility\": \"PUBLIC\",
                \"operation\": \"ADD\"
            },
            {
                \"name\": \"pV_tag\",
                \"visibility\": \"PRIVATE\",
                \"operation\": \"ADD\"
            }],
            \"shareInfo\": {
                \"mode\": \"DEFAULT\"
            },
            \"docAuthors\": [{
                \"authorName\": \"Author1\",
                \"operation\": \"ADD\"
            }]
            \"documentOwner\": \"username\",
            \"createdTimestamp\": \"yyyy-mm-ddThh:mm:ssZ\",
            \"sourceType\": \"Internal Research\"
          }"'

Modify an existing document​

Make an API call to the content ingestion modify endpoint to modify an existing document

tip

You can replace document content, document metadata, or both. Either file, attachments or metadata is required for performing this action:

curl --location --request PUT 'https://research.alpha-sense.com/services/i/ingestion-api/v1/modify-document/{docId}' \
    --header 'Authorization: Bearer <access_token>' \
    --header 'ClientId: <YOUR CLIENT ID>' \
    --form  'file' \
    --form  'attachments' \
    --form  'metadata(type: String): <IngestionMetadata>'

The following document types are supported:

pdf, html, htm, txt, doc, docx, xls, xlsx, ppt, pptx, msg, eml, csv, xlsb, xlsm, one, tsv, ods
NOTE

File is parent document and attachments are children of it which should be attached with main document.

Example Curl (with DEFAULT mode):

curl --location --request PUT 'https://research.alpha-sense.com/services/i/ingestion-api/v1/modify-document/{docId}' \
    --header 'Authorization: bearer token' \
    --header 'clientId: enterprise-sync' \
    --form 'file=@"/document.txt"' \
    --form 'attachments=@20221020 - EnterpriseSync - TSLA - Doc - 1 page.pdf;type=application/pdf' \
    --form 'attachments=@20230201 - Onenote - Demo Page 2 - 1 page.pdf;type=application/pdf' \
    --form 'metadata="{
              \"title\": \"Document Upload\",
              \"companies\": [{
                \"value\": \"US5949181045\",
                \"operation\": \"ADD\",
                \"identifier\": \"ISIN\",
                \"salience\": \"PRIMARY\"
              }],
              \"customTags\": [{
                  \"name\": \"pb_tag\",
                  \"visibility\": \"PUBLIC\",
                  \"operation\": \"ADD\"
              },
              {
                  \"name\": \"pV_tag\",
                  \"visibility\": \"PRIVATE\",
                  \"operation\": \"ADD\"
              }],
              \"shareInfo\": {
                  \"mode\": \"DEFAULT\"
              },
              \"documentOwner\": \"username\",
              \"sourceType\": \"Internal Research\"
            }"'

Delete endpoint to delete an existing document​

Make an API call to the content ingestion delete endpoint to delete an existing document:

curl --location --request DELETE 'https://research.alpha-sense.com/services/i/ingestion-api/v1/delete-document/{docId}?documentOwner={username}' \
  --header 'Authorization: Bearer <access_token>' \
  --header 'ClientId: <YOUR CLIENT ID>'

Example Curl:

curl --location --request DELETE 'https://research.alpha-sense.com/services/i/ingestion-api/v1/delete-document/{docId}?documentOwner={username}' \
  --header 'Authorization: bearer token' \
  --header 'clientId: enterprise-sync'

Bulk modify metadata on existing documents​

Make an API call to the content ingestion bulk modify endpoint to bulk modify metadata on existing documents.

tip

You can update customTags, companies, shareInfo for uploaded documents in bulk. Document Ids are required for performing this action:

NOTE

Bulk metadata modify has max limit of 250 docs per request.

curl --location --request PATCH 'https://research.alpha-sense.com/services/i/ingestion-api/v1/bulk/modify/metadata' \
  --header 'Authorization: Bearer <access_token>' \
  --header 'ClientId: <YOUR CLIENT ID>' \
  --header 'Content-Type: application/json' \
  --header 'accept: application/json' \
  --data-raw '<IngestionBulkModifyMetadata>'

Example Curl (with DEFAULT mode):

curl --location --request PATCH 'https://research.alpha-sense.com/services/i/ingestion-api/v1/bulk/modify/metadata' \
  --header 'Authorization: bearer token' \
  --header 'clientId: enterprise-sync' \
  --header 'Content-Type: application/json' \
  --header 'accept: application/json' \
  --data-raw "{
            \"docIds\": [\"docId\"],
            \"companies\": [{
                \"value\": \"US5949181045\",
                \"operation\": \"ADD\",
                \"identifier\": \"ISIN\",
                \"salience\": \"PRIMARY\"
              }],
            \"customTags\": [{
                \"name\": \"pb_tag\",
                \"visibility\": \"PUBLIC\",
                \"operation\": \"ADD\"
            },
            {
                \"name\": \"pV_tag\",
                \"visibility\": \"PRIVATE\",
                \"operation\": \"ADD\"
            }],
            \"shareInfo\": {
                \"mode\": \"DEFAULT\"
            },
            \"documentOwner\": \"username\",
            \"sourceType\": \"Internal Research\"
          }"'

Bulk delete existing documents​

Make an API call to the content ingestion bulk delete endpoint to bulk delete existing documents.

info

NOTE Bulk metadata delete has max limit of 250 docs per request.

curl --location --request DELETE 'https://research.alpha-sense.com/services/i/ingestion-api/v1/bulk/delete' \
    --header 'Authorization: Bearer <access_token>' \
    --header 'ClientId: <YOUR CLIENT ID>' \
    --header 'Content-Type: application/json' \
    --header 'accept: application/json' \
    --data-raw '<IngestionBulkDelete>'

Example Curl:

curl --location --request DELETE 'https://research.alpha-sense.com/services/i/ingestion-api/v1/bulk/delete' \
    --header 'Authorization: bearer token' \
    --header 'clientId: enterprise-sync' \
    --header 'Content-Type: application/json' \
    --header 'accept: application/json' \
    --data-raw "{
      \"docIds\": [\"docId\"],
      \"documentOwner\": \"username\"
    }"

Models​

  1. IngestionUploadMetadata Model:
{
  title: String,
  companies: [Company],
  customTags: [CustomTag],
  shareInfo: IngestionApiShareInfo,
  docAuthors: [DocumentAuthor],
  documentOwner: String,
  createdTimestamp: String,
  sourceType: String
}
  1. IngestionMetadata Model:
{
  title: String,
  companies: [Company],
  customTags: [CustomTag],
  shareInfo: IngestionApiShareInfo,
  docAuthors: [DocumentAuthor],
  documentOwner: String,
  sourceType: String
}
  1. Company Model (Optional):
{
  value: String,
  operation: String (ENUM: [ADD, DELETE])
  identifier: String (ENUM: [ISIN, CUSIP, SEDOL, VALOR, TICKER, NAME, CIK]),
  salience: String (ENUM: [PRIMARY, SECONDARY])
}
  1. CustomTag Model (Optional):

Attribute to add tags to a document.

visibility:

  • - PRIVATE: To create tags only visible to the uploader.
  • - PUBLIC: To create tags visible to other users.
{
  name: String
  operation: String(ENUM: [ADD, DELETE])
  visibility: String(ENUM: [PRIVATE, PUBLIC])
}
  1. IngestionApiShareInfo Model:

mode: Attribute to update the sharing permission for a document. It can take below values in mode:

  • - NONE: To upload a document privately not visible to others
  • - DEFAULT: To upload a document to your default group which can be controlled by the Preferences section in AS UI.
  • - EVERYONE: To share the document with everyone in your organization.

Note: If an empty object is passed as shareInfo, then by default the DEFAULT value is used as the mode field.

{
  mode: String(ENUM: [DEFAULT, NONE, EVERYONE])
}
  1. IngestionBulkModifyMetadata Model
{
  docIds: [string]
  companies: [Company],
  customTags: [CustomTag],
  shareInfo: IngestionApiShareInfo,
  documentOwner: String,
  sourceType: String
}
  1. IngestionBulkDelete Model
{
  docIds: [string],
  documentOwner: String
}

Response​

Ingestion Responses
{
  error: FailureResponse
  success: SuccessResponse
  warning: WarningResponse
}
Failure Response
{
  docId: String
  message: String
}
Success Response
{
  docId: String
  message: String
}
Warning Response
{
  docId: String
  message: String
}

Limits and Quota​

  1. Maximum 10 attachments are allowed to be ingested for a Parent document.
  2. Rate limit of 50 Requests Per Minute (RPM).
  3. Max File Size of 150 MB.
  4. Ingestion API supports the following document extensions only - pdf, html, htm, txt, doc, docx, xls, xlsx, ppt, pptx, msg, eml, csv, xlsb, xlsm, one, tsv, ods
  5. Filename must not contain any character (/ ,\\)
  6. Filename must contain only one dot. Double extensions (e.g. file.php.txt) is not permissible
  7. Maximum character length permissible as a filename is 255 characters