Skip to main content
Version: Beta

Build your first Trending Information Dashboard with the AlphaSense SDK

This how-to guide will lead developers through building a standard Trending Information Dashboard with 3 charts built in.

Quick Start & Starting From Scratch​

Before diving into the quick start guide or starting from scratch, ensure you meet the following requirements:

Requirements​

  • Node.js: Version 14.0.0 or higher. You can check your current version by running node -v in your terminal.
  • Programming Language: Ensure you have the latest version of your preferred programming language installed. For JavaScript/TypeScript projects, the Node.js version requirement applies.

You can create a new project with the help of the project template. To create a new project, run the following command:

npx @alphasense/sdk-create-app my-trends

This command will create a new project with the following structure:

my-trends
├── node_modules
├── alphasense.config.ts
├── index.ts
├── package.json
└── tsconfig.json

After updating the configuration (alphasense.config.ts file), you can generate the typings for the queries by running the following command:

npm run generate

After generating the typings, you can start using the SDK in your project! Update the index.ts file however you want, and you can run the code with the following command:

npm run dev

You can see the example result in the console if you have a valid configuration.

Configuration​

The SDK checks for the configuration for the following inputs to make sure the SDK is propertly authenticated:

// Typescript (alphasense.config.ts)
import {AlphaSenseConfig} from '@alphasense/sdk-client'

const config: AlphaSenseConfig = {
  username: '<your-username>',
  password: '<your-password>',
  apiKey: '<your-api-key>',
  clientId: '<your-client-id>',
  clientSecret: '<your-client-secret>',
}

export default config

Importing Required Libraries​

In order to use the SDK, you will need to import the following libraries:

import {createClient, AlphaSenseConfig} from '@alphasense/sdk-client'
import config from './alphasense.config.ts'
import {
  DocumentCountGroupBy,
  enumDocumentCountGroupBy,
} from '@alphasense/sdk-client/dist/generated/schema.ts'

Building the Search Queries​

To start, you will need to build the search queries that you want to use. We will create a new file called api.ts that will contain everything in relation to extracting information from AlphaSense. Here is an example of how to build the search queries with various different filters specific to the trends API:

const searchQueries = [
  {
    // NEGATIVE RISK
    filter: {
      keyword: {query: 'risk negative'},
      date: {customRange: {from: '2020-01-01', to: '2024-01-01'}},
      types: {ids: ['3']},
    },
    groupBy: enumDocumentCountGroupBy.DOCUMENT_TYPES_MONTHLY as DocumentCountGroupBy,
    limit: 250,
  },
  {
    // GENERATIVE AI
    filter: {
      keyword: {query: 'generative ai'},
      date: {preset: 'LAST_6_MONTHS'},
      types: {ids: ['17000', '18000', '19000']},
    },
    groupBy: enumDocumentCountGroupBy.DOCUMENT_TYPES_DAILY as DocumentCountGroupBy,
    limit: 250,
  },
  {
    // ELECTIONS
    filter: {
      keyword: {query: 'election'},
      date: {customRange: {from: '2020-01-01', to: '2024-01-01'}},
      types: {ids: ['31019']},
    },
    groupBy: enumDocumentCountGroupBy.DOCUMENT_TYPES_WEEKLY as DocumentCountGroupBy,
    limit: 250,
  },
]

Searching the Documents​

After you have your search queries ready, you can start searching the documents by building out the client and fetching each of the results to pull back trending information:

// Function to search documents
const documentResults = async (
  filter: Record<string, any>,
  groupBy?: DocumentCountGroupBy,
  limit?: number,
) => {
  const client = await createClient(config)
  try {
    const resp = await client.query({
      documentCountsInSearchResults: {
        items: {
          groupKeys: {
            on_DocumentTypeAndDateGroup: {
              date: true,
              name: true,
            },
          },
          value: true,
        },
        __args: {
          limit,
          filter,
          groupBy:
            groupBy || (enumDocumentCountGroupBy.DOCUMENT_TYPES_QUARTERLY as DocumentCountGroupBy),
        },
      },
    })
    return resp?.documentCountsInSearchResults || []
  } catch (error) {
    console.error('Error fetching document results:', error)
    return []
  }
}

Lastly, we will create a function to fetch all the results from the API and load them right away to be added into the initialization of the dashboard.

// Function to fetch all results
async function fetchAllResults() {
  const allResults = []
  for (const query of searchQueries) {
    console.log(query)
    const results = await documentResults(query.filter, query.groupBy, query.limit)
    allResults.push(results)
  }
  return allResults
}

export {fetchAllResults}

Building the Dashboard​

In a separate file index.ts, we will start building a trends dashboard to get the latest document trends on the searches we had discussed. We will be using the d3 package to create the charts and the lodash package to group the data by month:

To start you will need to import the necessary libraries and instantiate the server and the DOM

import express from 'express'
import {fetchAllResults} from './api.ts'
import D3Node from 'd3-node'
import * as d3 from 'd3'
import {parseISO, format} from 'date-fns'
import {groupBy} from 'lodash'

const app = express()
const port = 3000

After importing the required package we can put together the functions needed to create the page. We will start with the createCharts function that will pull the information from all the API calls and create multipl charts to display it. createCharts will also call createTimeSeriesChart and createErrorChart to create the charts and handle any errors that may occur:

function createCharts(data: any[]): string[] {
  return data.map((dataset, index) => {
    const d3n = new D3Node()
    const svg = d3n.createSVG(800, 400)

    if (!dataset?.items) {
      createErrorChart(svg, index)
    } else {
      const groupedData = groupBy(
        dataset.items.filter(
          (item: {groupKeys: {name: string}[]}) => item.groupKeys[0].name !== 'ALL',
        ),
        (item: {groupKeys: {name: string}[]}) => item.groupKeys[0].name,
      )
      const categories = Object.keys(groupedData)
      const timeSeriesData = processTimeSeriesData(groupedData)
      timeSeriesData.length > 0
        ? createTimeSeriesChart(svg, timeSeriesData, categories, index)
        : createErrorChart(svg, index, 'No time series data available')
    }

    return d3n.svgString()
  })
}

Next we can create a function that will create the various charts needed for the dashboard to display the information chosen in the first analysis. This will include the createErrorChart function in case there is no data available and the createTimeSeriesChart function that will create a time series chart for the data available:

function createErrorChart(svg: any, index: number, message = 'Error in Dataset') {
  svg.append('text').attr('x', 400).attr('y', 200).attr('text-anchor', 'middle').text(message)
}

function createTimeSeriesChart(
  svg: any,
  timeSeriesData: any[],
  categories: string[],
  index: number,
) {
  const margin = {top: 20, right: 30, bottom: 40, left: 50}
  const width = 750 - margin.left - margin.right
  const height = 350 - margin.top - margin.bottom

  const x = d3
    .scaleTime()
    .domain(d3.extent(timeSeriesData, (d: {date: Date}) => d.date) as [Date, Date])
    .range([0, width])
  const y = d3
    .scaleLinear()
    .domain([0, d3.max(timeSeriesData, (d: {value: number}) => d.value)])
    .range([height, 0])
  const color = d3.scaleOrdinal().domain(categories).range(d3.schemeCategory10)

  const g = svg.append('g').attr('transform', `translate(${margin.left},${margin.top})`)

  addAxes(g, x, y, height)
  drawBars(g, timeSeriesData, categories, x, y, width, height, color)
  addLegend(svg, categories, color, width, margin)
  addTitle(svg, ['Risk Negative', 'Generative AI', 'Election'][index])
}

For each of the 3 charts we are looking to build the createTimeSeriesChart will also need to add and draw the various parts of the chart. This includes the addAxes, drawBars, addLegend, and addTitle functions. Let's take a look at the addAxes function:

function addAxes(g: any, x: any, y: any, height: number) {
  g.append('g').attr('transform', `translate(0,${height})`).call(d3.axisBottom(x))
  g.append('g').call(d3.axisLeft(y))
}

Next, we can create the function to draw the bars of the chart:

function drawBars(
  g: any,
  data: any[],
  categories: string[],
  x: any,
  y: any,
  width: number,
  height: number,
  color: any,
) {
  const barWidth = (width / data.length) * 0.8 // 80% of available space
  const categoryWidth = barWidth / categories.length

  const stack = d3
    .stack()
    .keys(categories)
    .value((d: any, key: string) => d.values[key] || 0)

  const groupedData = d3.group(data, (d: {date: Date}) => d.date)
  const stackedData = stack(
    Array.from(groupedData, ([date, values]) => ({
      date,
      values: Object.fromEntries(
        values.map((v: {category: string; value: number}) => [v.category, v.value]),
      ),
    })),
  )

  g.selectAll('.bar-group')
    .data(stackedData)
    .enter()
    .append('g')
    .attr('fill', d => color(d.key))
    .selectAll('rect')
    .data(d => d)
    .enter()
    .append('rect')
    .attr(
      'x',
      (d: d3.SeriesPoint<{date: Date; values: Record<string, number>}>) =>
        x(d.data.date) + barWidth * 0.1,
    )
    .attr('y', d => y(d[1]))
    .attr('height', d => y(d[0]) - y(d[1]))
    .attr('width', barWidth * 0.8)
}

Then we can add the legend and table to the chart:

function addLegend(svg: any, categories: string[], color: any, width: number, margin: any) {
  const legend = svg
    .append('g')
    .attr('transform', `translate(${width + margin.left + 10},${margin.top})`)

  categories.forEach((category, i) => {
    const legendRow = legend.append('g').attr('transform', `translate(0, ${i * 20})`)
    legendRow.append('rect').attr('width', 10).attr('height', 10).attr('fill', color(category))
    legendRow.append('text').attr('x', 15).attr('y', 10).text(category)
  })
}

function addTitle(svg: any, title: string) {
  svg
    .append('text')
    .attr('x', 400)
    .attr('y', 30)
    .attr('text-anchor', 'middle')
    .style('font-size', '16px')
    .style('font-weight', 'bold')
    .text(title)
}

To make sure we are producing complete sets of data we will need to process the time series data and group it by month:

function processTimeSeriesData(groupedData: Record<string, any[]>) {
  return Object.entries(groupedData)
    .flatMap(([category, items]) =>
      items.map(item => {
        let date: Date
        let value: number

        try {
          // Parse date string to Date object
          date = parseISO(item.groupKeys[0].date)
          value = item.value
        } catch (error) {
          console.log('Error processing item:', error)
          date = new Date() // Fallback to current date
          value = 0 // Fallback value
        }

        return {
          category,
          date,
          value,
        }
      }),
    )
    .sort((a, b) => a.date.getTime() - b.date.getTime())
}

Lastly, we will need to set up the server and the DOM to be able to serve the dashboard:

app.get('/', async (req, res) => {
  try {
    const allResults = await fetchAllResults()
    const chartSvgs = createCharts(allResults)

    const html = `
      <!DOCTYPE html>
      <html>
        <head>
          <title>Document Publication Trends</title>
          <style>
            body { font-family: Arial, sans-serif; }
            h1 { text-align: center; }
            .chart { display: block; margin: 20px auto; max-width: 800px; }
          </style>
        </head>
        <body>
          <h1>Document Publication Trends</h1>
          ${chartSvgs.map(svg => `<div class="chart">${svg}</div>`).join('')}
        </body>
      </html>
    `

    res.send(html)
  } catch (error) {
    console.error('Error:', error)
    res.status(500).send('An error occurred')
  }
})

app.listen(port, () => {
  console.log(`Server running at http://localhost:${port}`)
})

Finally, run the app using the following command:

npm run dev

Then, open your browser and navigate to the following URL to see the result: http://localhost:3000

Once all the steps are completed, you can start viewing data directly from AlphaSense in your dashboard relating to the trends we discussed!