Collect to Report with Google Cloud Platform

Collect, store, transform and present all with Google Cloud Platform.

This is my first attempt at Google Cloud Platform (GCP), with a little help on google apps scripts I was able to create this dashboard. I was earlier shy with GCP, thought it needs good coding skills. I am not a coder, but can do some simple scripts with Google search. After dabbling for couple of days in GCP - Apps script, BigQuery and Data studio, Mission Accomplished.

This is not a real time reporting, real time streaming can be done. BigQuery was not required, Google sheet is good enough. Google sheet can store up to 2 Million data cells. If your Google sheet grows, data operations are slower. BigQuery is high performance, primarily for online analytical processing (OLAP) and much more.

My objective was to find ' Where are 'Data Scientist' jobs in US?

This is my approach

1. Collect data from web/API with Google script
2. Save the data to sheet with Google script
3. (Optional) Save data to Bigquery with Google script
4. Report with Data studio (includes data transformation)

There are other better visualization tools, but I found overall 'Collect to Report' process very easy with GCP. Only 300 line of Apps script was used in this project.

(Job API was discontinued in Oct,2020)

To zoom, click zoom icon at bottom right corner 

FP&A Team,Processes and Tools

FP&A Summit by Innovation Enterprise was held in San Diego, Feb 14- 15th. Finance and FP&A Leaders shared their experiences and presented topics on key matters such as FP&A Processes, Recruitment, Team development and Industry insights.

Below are my key takeaways:

1.    Recruitment
a.    Good quality FP&A resources are hard to find. Start recruitment early, making it your #1 priority. Candidates needs to have a solid background in Accounting and in Finance, most importantly with a growth mindset.
b.    People follow leaders - Your past colleagues/subordinates will likely follow you. Ask them ‘How can you add value?’
c.    Quality – Initially, long hours of work are needed for accomplishment, but the quality of work is what matters most in the long-term. Instead of asking ‘Can you work long hours?’ ask them ‘Describe your day at work?’. This will help you to understand ‘What are they producing at the end of the day?’ Open ended questions will provide qualitative feedback on the candidate rather than closed questions that provide minimal response.

2.    Team Composition
a.    Teams should understand the industry, and also challenge the norms or ‘status-quo’ at the same time. Hire a good mix of people from varied industries to provide the most complete range of responses, opinions and solutions.

3.    Development, Goal Setting and Performance Review
a.    ‘What got you here, won't get you there’, identify the attributes which are required to achieve the agreed goals in the long-term.
b.    If you are managing international team, explain in detail what you need. This helps you to align the expectations, and reduces the cultural gap.

4.    Tools and Solution Implementation
a.    Most of them have a tool in place for FP&A, while others are still using spreadsheets.
b.    Although tools were positioned in Gartner ‘Leaders’ quadrant, some were not happy with them. Common complaints were - solution was inflexible; they are still dependent on the consultant to make changes. Tools do not matter, what matters is ‘solution flexibility, scalability, and self-reliant?’
c.    Some also had negative experiences with consultants, others loved them. Spend more time understanding the experience of the consultants, ‘Are consultants experienced with accounting/finance systems? Do they have similar industry implementation experience?’ to ascertain if they are the right fit together.

Hope this helps you. If you like it or have comments please post here.

Power in BI = Power BI

Many times decisions are based upon existing Features. 'The Only Thing That Is Constant Is Change', we do not want to base a decision on existing product features. When comparing we take into consideration how other competitive products/services are evolving. Are they adding new features? How frequently?

Tableau and Qlik historically had lead over Power BI in Data Visualization. I am now convinced that Power BI has the edge over others, here is the Gartner BI and Analytics magic quadrant report which ranks Power BI as visionary.

This is why I believe Power BI has edge over other

1. Organizations uses on an average '13' apps (data sources) - Okta Research finding

Which applies, a business analyst will have at least 13 tables joins to make a meaningful story. He/she has to slog (to understand the tables/views/relationships) if there is content pack, to get data from source. There are plenty of app connectors available on all three tools- Power BI, Tableau and Qlik. Power BI has more app connectors than others, you can quickly and easily get the required data from apps. Microsoft has understood the  importance of participating in open source, as it has officially joined the Linux foundation today. Not only it has connections to existing Microsoft stack (cloud/on premise), but also other others - Salesforce, Google Analytics, Github and many others. 

2. We need to store the data before consuming it, need to save data in SQL and other formats

ETL will be more relevant than ever before, to make a sense from multiple data sources - a good (join) relationship is required. Tableau and Qlik can read files and other data source, but they lack the ETL layer. Microsoft stack (SQL, Azure SQL and others) provides tools to transform and save the data in SQL and other formats. The complex transformation can Microsoft stack, leaving the light transformation to Power BI. Power BI as a part of Microsoft family has tremendous advantage.

3. Price, Price, Price

Look no further. At $9.99/Month is the Power BI pro version, the least expensive of the all products.

There are features which Power BI lacks currently - as discussed here by Tableau. This features comparison will not be relevant in the future, Power BI will catch up quickly. For some scenarios maybe Power BI is not the best Data Visualization tool. As a generalist which most of us are, we will find Power BI to be flexible, powerful and affordable.

Moore's law is slowing down, organization with focus on cost advantage will have better chance to thrive

Is Moore's law about:

A. Smaller, faster transistors? OR
B. Economics of chip making?

My answer is 'A', until 'B' holds true.

"The culprits (for the slowing of Moore's Law) are the rapidly rising costs associated with technology development and the capital equipment needed to produce next-generation nodes" 

McKinsey identifies this trend in the semiconductor industry in this article, and many Industry experts agree with this trend.

With pressing economic challenges and slower technological breakthroughs, organizations are focusing on cutting costs and consolidating of the value chain (via Partnerships and/or Acquisitions) 

The challenge for decision-makers within an organization is:- 

1. Do I have the right information from all functions within an organization to model and plan for the future?

Traditionally EPM vendors had 'whiteboard' models – i.e. models with some features for financial planning but none for Sales and HR. Implementation of these EPM suites took six months or more, with minimal interaction of the various models.  Host Analytics offers a true, unified Cloud EPM suite, where most of the features are right out of the box. Financial, Workforce, Sales and CAPEX planning are all part of Host Analytics EPM offering.

B. Rising costs - How do we manage or decrease its unfavorable effects
Breaking down costs through Activity-based costing (ABC) (or Segment/channels for Sales/Marketing) can help organizations to identify the activities (expensive production process, or segment with higher costs) which need further analysis. You can view this information in Host Analytics (Actual's loaded from ERP and other systems), identify the issues and plan accordingly
Host Analytics offers integration services to many ERP and other systems, and drill-down to transaction details is also available.

C. Rolling forecasts, reporting and what if scenarios - Volatility, ConsolidationsWith Host Analytics Cloud EPM Suite, organizations can create rolling forecasts and create the most realistic picture possible of the future. There is no complex setup to create a rolling forecast.
You don’t have to purchase a different system for close management or consolidations, its available within the Host Analytics suite. Yes, it can even offer multi-currency consolidations and complex intercompany and equity eliminations.
You can also run 'what if' scenarios for consolidations and view the impact on the Balance Sheet and Income Statements.

D. Modeling of non-Financial information
Host Analytics 'Modeling' module recognizes the need for modeling of non-Financial information and flexibility to model and plan using Microsoft Excel as a front-end. With Host Analytics Modeling, you can model e.g. sales leads/opportunities or inventory SKU planning and then link then to other models .

To find out more, register for a Free demo

The Host Analytics Future of Finance Tour Tours starts in San Francisco on August 12th, 2015 

Cloud CPM suits positioned to be next 'Leaders' in Gartner's 2015 report.

Cloud CPM suite providers -Host Analytics, Adaptive Insights, Tagetik and Anaplan are moving into 'Leader' quadrant. Gartner has published 'Magic Quadrant CPM 2015 Report' and identifies trends in CPM suites.

Current hosted solution providers - SAP , Oracle and IBM are current leaders in CPM solution offering. They are late to understand customers needs, as their focus was on their cash cows 'ERP'. Cloud CPM suite providers realized this opportunity, and now with high customer satisfaction, they will move in 'Leader' quadrant.

Three strengths which cloud CPM has are

1. Pure-play 'Cloud' solutions- which translates to reduced costs per user and easy upgrades
2. Mobile device accessibility- user friendly UI and web analytics
3. Pre-built models for planning and consolidation, and function specific rich features. e.g. HR and Sales models.

Host Analytics, was the first pure-play cloud vendor. Host Analytics started offering its cloud CPM service in 2007. It maintains its 'visionary' vendor position. Host Analytics provides function specific rich features to HR and Sales models.

To make migration to Cloud, vendors offer migration paths and tools to move your models and data from legacy to cloud CPM suite.

If you have a need for cloud CPM solution, please contact us. We will be happy to assist you in moving away from 'Excel chaos' or expensive hosted solution.

World Cup 2014 -Predictive Analytics Showdown

Are you in World Cup 2014 friendly pool? Get help from 'experts' on statistically predicting 'who would advance and win the 2014 World Cup'.

If your betting window is closed, enjoy the game.

The models being off-target and you lose, it's very likely you will forget the loss (Freakonomics, bad predictions are rarely punished). Statistical models always learn from past recent events and refine the variables. Hope you win next time.

Here are the collections of World Cup 2014 Data Analytics and Visualizations

Nate Silver Vs. Goldman Sachs Vs. Bloomberg

Remember Nate Silver, 2008 US General Elections Prediction?, He and his team at FiveThirtyEight have statistical models on various subjects- Politics, Economics, Science, Life and Sports.

I like FiveThirtyEight.com the visualization of the model.

FiveThirtyEight explains the statistical models which calculates the probability of teams advancing to the World Cup 2014 final. This model uses ESPN's SPI (Soccer Power Index).

Bloomberg with BSports have a good visualization, you can add your predictions and visualize.

The third model is by Goldman Sachs (they have been doing this since last four World cups)

SAP vs SAS vs R

SAP Lumira has fun facts and profitability.

There is not much analysis by SAS team, except this blog.

R-Bloggers contributors have posted statistical predictions -one favoring Germany and other Brazil.

Statsinthewild has interesting stats and data visualization.

Data Visualization

My favorite tool to visualize and learn World cup data is Google Trends. It uses real-time search data and uses interesting visualization techniques. I LOVE IT !!!

Tableau uses ESPN SPI statistics

What's is the difference between Prediction and Forecasting?
Prediction is a estimate of a specific event in the future. Forecasting is identifying all possible outcomes with each having its own probability, the total probability of all events will be close to 1.

The world just woke up now to 'Predictive Analysis'. The below Google Trends graphs seems to agree

Sneak Peek of New SAP Smart Accounting Functionality-2013 CO Conference

SAP Smart Financials 1.0

We see the next evolution of SAP Business Suite on HANA for Financials in Smart Financials 1.0

SAP Smart Financials 1.0 has new functional features, thanks to the underlying SAP HANA. The is to have

• More granularity for better reporting, and
• Faster close (by removing the reconciliation effort)

SAP Smart Financials is not just a technical change, to implement a good Change Management strategy and high adoption rate is required.

Financial Planning with SAP Smart Financials 1.0

1. The one area which interests me is Financial Planning, Smart Financial data is not available in ERP (as of release 1.0) . This is not a major issue if users report variance in BI. (The release note says the data is stored in BI). 
2. Any processing of financial planning is not supported. This implies users loose 'Integrated' planning scenarios in ERP. It is not good news for clients which use the Integrated Financial Planning feature in SAP ERP.

This above changes should not be a surprise as SAP has been commited on SAP EPM as the planning solution.

To read more about the release note click here SAP Smart Financial Release Notes. Select SAP solutions > SAP ERP Add-Ons > Smart Financials

Comparison of SAP Smart Financials, New GL and Old GL

	Smart Financials 1.0 (SAP ERP on HANA) (2013)	New General Ledger (2005)	Old General Ledger
	Compatible with New GL only
Data Storage	Document tables and line item tables	Document tables, line item tables, indexes	Document tables, line item tables, indexes, has different FI/CO tables
Totals table	None, no inconsistencies between FI documents and totals tables	Yes along with document/line item tables	Yes along with document/line item tables
Views	Tables (index) are replaced by views. View are available for BSIS, BSAS, GLTO, FAGLFLEXT
Special Purpose Ledgers (SPL)	Not available	Available, Can use ‘Ledgers’ concept	Available
Credit Management	Recommends FSCM-Credit Management (FSCM-AR)	FSCM-AR, FIN-AR-CR	FSCM-AR*, FIN-AR-CR
Asset Accounting	New Asset Accounting replaces Classical Asset Accounting	Classical Asset Accounting	Classical Asset Accounting
Constraints	-          Balance Sheet Planning no longer available -          Reports for comparing planning data and actual data in Profit Center Accounting -          Destruction of data or deletion of FI documents (Information Lifecycle Management)

New Features in SAP Smart Financials 1.0

New Asset Accounting: Ledger Approach and Accounts Approach

·         With Smart Financials both the actual values of the leading valuation and the values of parallel valuation in real time.

Line Item Based Controlling with Period Close Optimizations (Enhanced) 

Variance Calculation for Cost Centers (Changed)

·         Option to process the controlling area currency only or both the controlling area currency and the cost object currency.

·         The execution time for processing the controlling area currency only is around half that required for processing the controlling area currency and object currency

WIP Calculation and Results Analysis (Changed)

·         The production orders are valued at actual cost for the selected plant with the defined legal valuation for a single result analysis version

·         Customer enhancement and WIP calculation for product cost collectors and process orders are not supported

·         Multiple valuations is not supported

Variance Calculation for Production Orders (Changed)

·         Accelerated collective variance calculation on production orders compares the target costs (in the form of standard costs) with the control costs (actual cost less scrap cost) of all production orders in the plant, and assigns the resulting variances to variance categories

Smart Financials Reporting (New)

·         Line items reports supported SAP HANA

Integrated Business Planning (New)

·         In the first release of this new planning solution, the development of functionality has been focused on enabling plan data capturing in a very user-friendly way.

·         If further plan data processing is necessary, releasing them to the classic FI/CO plan data tables may be considered in future releases

SAP EPM 10- Matrix Security

I have posted a video here, to demonstrate the Matrix Security in SAP EPM10. You can use multiple dimensions to build you data access profile. This allows administrators to have a complex scenario, and also reduces the total number of data access profiles.

In the video i have demonstrated, how you can create a DAP using a Region and Product combination.

Data Access Profile	Footwear	Apparel	Accessories
VP_Sales	World	World	World
Apparel_Sales	Africa, AsiaPac, Europe	Africa, AsiaPac, Europe, N America, S America	Africa, AsiaPac, Europe, N America, S America
Footwear_Sales	N America, S America	Read Only	Read Only

Is it possible to create a Matrix Profile with three dimensions- Region, Product and Time?

The above scenario will be useful when e.g. a Footwear Sales Manager's role evolves (FY 2015, FY 2016

) and is responsible for apparel and accessories in three year planning cycle. Or a Sales Manager's role will be shrunk  (in FY 2015, 2016) to only Footwear, as the company grows it adds apparel and accessories to a new sales manager.

Link to the YouTube Video

Profitability Analysis in SAP BPC with HANA

If you have a background in SAP COPA (Controlling-Profitability Analysis) and wondered if similar planning can be done in SAP BPC, you will find this article interesting.

SAP COPA is a great tool to collect the actual data, but does not have a good user interface for planning. Users of COPA usually plan in Microsoft Excel sheet and plugin the values in COPA planning templates. This defeats the whole purpose of interactive planning. This is due to the fact the templates are not flexible and cannot be generated dynamically.

In earlier blog we discussed about 'Key Figure' limitation in BPC. 

The other limitation is COPA can have 50 Characters, whereas a BPC Model ('Application' in 7.5 am older versions) can have only 20. An Environment ('ApplicationSet' in BPC 7.5 an older versions) can have 50 or more dimensions, if required we can create multiple Models each with different sets of COPA characteristic (or Characteristic groups) and join them using a common denominator (e.g. a segment).

We have helped clients to address COPA planning limitations in BPC. Contact us to discuss your case scenario.

With HANA, many limitations of COPA using traditional database can be overcome. We will cover COPA+HANA in over next blog. 

Google BigQuery, SAP HANA- Cloud Solutions

Week of May 13th, 2013 was a big deal for SAP and Google. Two major events were held-SAPHIRE in Orlando and Developer I/O 2013 in San Francisco. 

I was lucky to get a Google I/O 2013 ticket, and be on of the 6000 participants. My objective was to learn about Google BigQuery and Google Apps Script.

Both companies are competitors in Cloud based 'Analysis or massive database' , SAP HANA has two product versions 'Appliance ' (Hardware and Software) and 'Cloud'. Where as Google has Cloud solution BigQuery. 

SWOT	SAP HANA Appliance	SAP HANA Cloud	Google BigQuery
Strengths	- Strong SAP Products/Branding	- No downtime - Good for Ad hoc requirements	- Good Enterprise Cloud offerings - No downtime - Good for Ad hoc requirements - Good Developer community
Weakness	- Maybe costs and downtime	- Dependency on AWA (Amazon web services) for cloud storage	- Provides tools, not business suits - some products are evolving (e.g. Data Visualization)
Opportunities	- Increase production adoption - Conversion from Cloud version - Use as database for other applications	- Use as database for other applications - Convert clients with web security concerns - Encourage Developer community to build solutions	- Use as database for other applications (using Compute Engine) - Convert clients with web security concerns
Threats	- Cloud products, and other Appliances	- Google BigQuery, Maybe Amazon	- Maybe Amazon

There were two client presentations on BigQuery - Gamesys and Shutterfly, these clients also use Google Apps Script for data visualization/reporting.

Here are the two Google BigQuery recordings
Big Data Mashups: Enabling Next Generation Analytics Using BigQuery
Here Be BigQuery: Building Social Gaming Infrastructure on the Google Cloud Platform

Another interesting project was All the Ships in the World: Visualizing Data with Google Cloud and Maps, which integrate many Google technologies to visualize real time Data.

I see SAP ideologically now being more open, proving more resources to developers and planning more Cloud based products.

It will be interesting to see how enterprise Cloud database and analysis tools will evolve and their acceptance.

Real-Time Data in BPC

Real-Time data is required for some organizations for tactical Sales & Operational Planning, a good example is FMCG industry.

I have seen requirements in these scenarios,
- New customers added in source system, need to update daily or weekly sales plan NOW in BPC
- Competitive Analysis, need to update this to my sales force NOW
- Month-end, Quarter-end top-end adjustment entered, I have to finalize forecast NOW

There are two issues
1. How to get the data in BPC, and
2. How to collaborate-Real-Time

How to get the data in BPC

If you use SAP BW, you can achieve it with some effort. It's called 'Real-Time Data Acquisition'. This was available in BW 3.5 with restrictions, in BW 7.0 and later versions it got better. You can get Real-Time data from source system using this technology. This is not out of the box solution, customization's and development effort are required.

I am not aware if this can be achieved with BPC Microsoft version, would like to know if you have experienced with this.

How to collaborate-Real-Time

The limitation of BPC is, one has to refresh the report to see the Real-Time data.  In collaborative planning when a team member is updating plan data, others have to wait for their turn to update. When planning portfolio of products and customers, collaboration is slow.

I see a great potential if everyone can see the same data and collaborate Real-Time, just like this Google Drive Real-Time API

Do you have a requirement for Real-Time Planning in BPC?

SAP BPC NW Consolidations Movements in Flow or Account Dimension?

Whats the advantage of flow dimension over accounts?

In my experience is Flow makes things application complex, Account is relatively easier to report and manage.

Account movements are required for many Balance Sheet accounts- e.g. Cash, Fixed Assets. These movements are reported in financial statements in cash flow and disclosure sections.

I believe capturing movements in accounts is much efficient because

1. Reporting is much simpler, no need to remember which flows are valid for which accounts.e.g. Cash account do not need retirements, additions flows dimensions. 
2. If you have flow dimension, then users have to be careful ion which flow they enter data. Overload on Business User.

What's your take?

Profitability Analysis Using SAP BPC

Can I make my Profitability Analysis (COPA) Planning in SAP as simpler as my Cost center or Revenue Planning in SAP BPC?

Agile Answer: Yes, you can.

SAP BPC's design is based on an Amount (Keyfigure), whereas SAP COPA can have multiple value fields (Keyfigure's). 

How do you load the COPA multi-Keyfigure in BPC? The key word to use in your transformation file is MVAL.

Dimension =*MVAL(keyfiguretype1|dim_mem1||keyfiguretype2|dim_mem2||keyfiguretype3|dim_mem3)

For reporting, you need to expand this Dimension on same axis as your key figure.

Challenges with COPA data in BPC

1. I like COPA, but the only limitation for business users in planning scenarios is 'inflexible' layouts. Unlike BPC reports/input schedules, COPA reports/planning layouts have to be configured. Which means a IT resource is required.
2. Users can plan quantities (or revenue) and COPA  can derive the revenue(or quantity) based on standard price or other measures. To achieve similar results enhancements are required in BPC.
3. If you get past #2, now some users would like to copy BPC data to COPA. How do we do it? It’s a complex development with a BAdI.  

We have helped clients to clear all the above barriers to create COPA like scenario in BPC. Do contact us and we will be glad to help you.