Collect, store, transform and present all with Google Cloud Platform.
This is my first attempt at Google Cloud Platform (GCP), with a little help on google apps scripts I was able to create this dashboard. I was earlier shy with GCP, thought it needs good coding skills. I am not a coder, but can do some simple scripts with Google search. After dabbling for couple of days in GCP - Apps script, BigQuery and Data studio, Mission Accomplished.
This is not a real time reporting, real time streaming can be done. BigQuery was not required, Google sheet is good enough. Google sheet can store up to 2 Million data cells. If your Google sheet grows, data operations are slower. BigQuery is high performance, primarily for online analytical processing (OLAP) and much more.
My objective was to find ' Where are 'Data Scientist' jobs in US?
This is my approach
- Collect data from web/API with Google script
- Save the data to sheet with Google script
- (Optional) Save data to Bigquery with Google script
- Report with Data studio (includes data transformation)
There are other better visualization tools, but I found overall 'Collect to Report' process very easy with GCP. Only 300 line of Apps script was used in this project.
(Job API was discontinued in Oct,2020)
To zoom, click zoom icon at bottom right corner
