Getting data in the form of a CSV file from Bitquery GraphQL API using NodeJS
The data which we often get in return from Bitquery GraphQL API is in the form of a GraphQL query. However, we can even generate the GraphQL returned data in the form of a CSV file and JSON. In this article we will be discussing the basic idea of getting data in JSON format using NodeJS and then later converting the JSON formatted data into a CSV file.
Initializing a NodeJS project
Install the following dependencies in the root folder of your NodeJS project
npm init
In the above code snippet, we have installed node-fetch and fs using the NPM (Node Package Manager). Node-fetch is used in order to use the fetch API functionalities within NodeJS environment and fs is used in order to make changes with filesystem within our NodeJS environment.
Installing Dependencies
npm install node-fetch fs
Getting Started
Inside the NodeJS project folder create an index.js file.
Importing Libraries
The following NodeJS libraries must be imported into index.js
const FileSystem = require('fs')
const { writeFile } = require('fs').promises;
const fetch = require('node-fetch')
Making a POST request to Bitquery GraphQL API
Before, we make the POST request to the GraphQL API, we need to setup the endpoints and declare our GraphQL query and both of them inside an asynchronous function called generateCSV( ).
// endpoint url
const url = "https://graphql.bitquery.io/";
// GraphQL query
const query = `
{
ethereum(network: ethereum) {
dexTrades(
date: {is:"2020-11-01"}
exchangeName: {is: "Uniswap"},
baseCurrency: {is: "0xdac17f958d2ee523a2206206994597c13d831ec7"},
quoteCurrency: {is: "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2"}) {
baseCurrency {
symbol
address
}
baseAmount
quoteCurrency {
symbol
address
}
quoteAmount
trades: count
quotePrice
side
}
}
}
`;
After wrapping the two inside an asynchronous function, the code would look similar to this
async function generateCSV(){
const query = `
{
ethereum(network: ethereum) {
dexTrades(
date: {is:"2020-11-01"}
exchangeName: {is: "Uniswap"},
baseCurrency: {is: "0xdac17f958d2ee523a2206206994597c13d831ec7"},
quoteCurrency: {is: "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2"}) {
baseCurrency {
symbol
address
}
baseAmount
quoteCurrency {
symbol
address
}
quoteAmount
trades: count
quotePrice
side
}
}
}
`;
const url = "https://graphql.bitquery.io/";
}
Now, we would be using the fetch API (using the node-fetch for NodeJS) to make the POST request to the Bitquery GraphQL API.
const opts = {
method: "POST",
headers: {
"Content-Type": "application/json",
"X-API-KEY": "YOUR UNIQUE API KEY"
},
body: JSON.stringify({
query
})
};
const results = await fetch(url, opts).then(res => res.json())
In the above code, the fetch function requires two arguments; the endpoint of the API and the opts object which makes the POST request to the API specifying the unique API key of the user while doing so.
The result of the POST request to the GraphQL API is stored in the results variable.
Defining a function which would convert JSON formatted data into a CSV file format
function convertJSONTOCSV(objArray){
var array = typeof objArray != 'object' ? JSON.parse(objArray) : objArray;
var str = '';
for (var i = 0; i < array.length; i++) {
var line = '';
for (var index in array[i]) {
if (line != '') {line += ','}
line += array[i][index];
}
str += line + '\r\n';
}
return str;
}
The above function convertJSONTOCSV, moves through the JSON formatted result and returns a string variable which is of CSV format.
Defining a function which would create a CSV file and store the data passed onto it as an argument/parameter
async function writeCSV (fileName, data) {
try {
await writeFile(fileName, data, 'utf8');
} catch (err) {
console.log(err);
}
}
In the above code snippet, writeCSV function has been declared which takes two arguments; name of the CSV file and the data. It then uses the writeFile component of the fs library to create a CSV file of the said name and store the data in it.
Completing the generateCSV( )
We will first create a variable which would store the value of the array representing the JSON formatted data, returned from the Bitquery GraphQL API
const objectArray = results.data.ethereum.dexTrades
And then, JSON.stringify the same using the code shown below
const jsonObject = JSON.stringify(objectArray)
Calling the convertJSONTOCSV function along with the writeCSV function
Right after the above code snippet, use the below code snippet to first, convert the JSON formatted data into CSV format and then inserting the values of the same into a CSV file.
const data = convertJSONTOCSV(jsonObject)
writeCSV('results.csv', data)
Alas, the function call and you are about done
generateCSV()
Run the NodeJS application
node index.js
A CSV file would be created in the root folder of your NodeJS project.
Resources
Link to the Github Repository
Link to all the gists