Getting data in the form of a CSV file from Bitquery GraphQL API using NodeJS

Getting data in the form of a CSV file from Bitquery GraphQL API using NodeJS

The data which we often get in return from Bitquery GraphQL API is in the form of a GraphQL query. However, we can even generate the GraphQL returned data in the form of a CSV file and JSON. In this article we will be discussing the basic idea of getting data in JSON format using NodeJS and then later converting the JSON formatted data into a CSV file.

Initializing a NodeJS project
Install the following dependencies in the root folder of your NodeJS project

npm init

In the above code snippet, we have installed node-fetch and fs using the NPM (Node Package Manager). Node-fetch is used in order to use the fetch API functionalities within NodeJS environment and fs is used in order to make changes with filesystem within our NodeJS environment.

Installing Dependencies

npm install node-fetch fs

Getting Started

Inside the NodeJS project folder create an index.js file.

Importing Libraries

The following NodeJS libraries must be imported into index.js

const FileSystem = require('fs')
const { writeFile } = require('fs').promises;
const fetch = require('node-fetch')

Making a POST request to Bitquery GraphQL API

Before, we make the POST request to the GraphQL API, we need to setup the endpoints and declare our GraphQL query and both of them inside an asynchronous function called generateCSV( ).

// endpoint url
const url = "https://graphql.bitquery.io/"; 

// GraphQL query 
const query = `
{
  ethereum(network: ethereum) {
    dexTrades(
      date: {is:"2020-11-01"}
      exchangeName: {is: "Uniswap"}, 
      baseCurrency: {is: "0xdac17f958d2ee523a2206206994597c13d831ec7"}, 
      quoteCurrency: {is: "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2"}) {
      baseCurrency {
        symbol
        address
      }
      baseAmount
      quoteCurrency {
        symbol
        address
      }
      quoteAmount

      trades: count
      quotePrice

      side

    }
  }
}
`;

After wrapping the two inside an asynchronous function, the code would look similar to this

async function generateCSV(){
	const query = `
	{
	  ethereum(network: ethereum) {
	    dexTrades(
	      date: {is:"2020-11-01"}
	      exchangeName: {is: "Uniswap"}, 
	      baseCurrency: {is: "0xdac17f958d2ee523a2206206994597c13d831ec7"}, 
	      quoteCurrency: {is: "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2"}) {
	      baseCurrency {
	        symbol
	        address
	      }
	      baseAmount
	      quoteCurrency {
	        symbol
	        address
	      }
	      quoteAmount
	      
	      trades: count
	      quotePrice
	      
	      side
	  
	    }
	  }
	}
	`;
	const url = "https://graphql.bitquery.io/";
}

Now, we would be using the fetch API (using the node-fetch for NodeJS) to make the POST request to the Bitquery GraphQL API.

const opts = {
    method: "POST",
    headers: {
        "Content-Type": "application/json",
        "X-API-KEY": "YOUR UNIQUE API KEY"
    },
    body: JSON.stringify({
        query
    })
};
const results = await fetch(url, opts).then(res => res.json())

In the above code, the fetch function requires two arguments; the endpoint of the API and the opts object which makes the POST request to the API specifying the unique API key of the user while doing so.

The result of the POST request to the GraphQL API is stored in the results variable.

Defining a function which would convert JSON formatted data into a CSV file format

function convertJSONTOCSV(objArray){
    var array = typeof objArray != 'object' ? JSON.parse(objArray) : objArray;
    var str = '';
    for (var i = 0; i < array.length; i++) {
        var line = '';
        for (var index in array[i]) {
            if (line != '') {line += ','}
            line += array[i][index];
        }
        str += line + '\r\n';
    }
    return str;
}

The above function convertJSONTOCSV, moves through the JSON formatted result and returns a string variable which is of CSV format.

Defining a function which would create a CSV file and store the data passed onto it as an argument/parameter

async function writeCSV (fileName, data) {
  try {
  	await writeFile(fileName, data, 'utf8'); 
  } catch (err) {
    console.log(err);
  }
}

In the above code snippet, writeCSV function has been declared which takes two arguments; name of the CSV file and the data. It then uses the writeFile component of the fs library to create a CSV file of the said name and store the data in it.

Completing the generateCSV( )

We will first create a variable which would store the value of the array representing the JSON formatted data, returned from the Bitquery GraphQL API

const objectArray = results.data.ethereum.dexTrades

And then, JSON.stringify the same using the code shown below

const jsonObject = JSON.stringify(objectArray)

Calling the convertJSONTOCSV function along with the writeCSV function

Right after the above code snippet, use the below code snippet to first, convert the JSON formatted data into CSV format and then inserting the values of the same into a CSV file.

const data = convertJSONTOCSV(jsonObject)
writeCSV('results.csv', data)

Alas, the function call and you are about done

generateCSV()

Run the NodeJS application

node index.js

A CSV file would be created in the root folder of your NodeJS project.

Resources
Link to the Github Repository
Link to all the gists

1 Like