How big data and analytics could change your business before 2020

IDC FutureScape

Analyst firm IDC has released its Futurescape worldwide big data and analytics predictions for 2016. In a recent webinar, Ciao Li, senior market analyst, big data and analytics, IDC, Chris Zhang, senior market analyst, software, IDC and Chew Kan Chua, AVP, big data and analytics, and cognitive computing, IDC, shared their insights into the implications of some of the mega-trends affecting the big data and analytics area for companies in the Asia-Pacific region (excluding Japan) (APEJ).

Driving forces

According to IDC, there are three key drivers affecting the big data and analytics area.

1. Digital transformation. There are many new applications for how customers can engage with organisations, such as driverless cars in the manufacturing field, and real-time interactions in media. Big data has an instrumental role in driving these offers, and as companies go down the digital path, they must be able to deploy scalable infrastructure to gain competitive advantage.

2. The merging of the digital and real-life. Another major driver is consumer preference for increased personalisation of services and offers, leading to an increased amount of data about personal profiles being held in the cloud and utilised to provide such services.

3. An abundance of technology options. The growth of cloud-based solutions, open source software, and value-added IP are together bringing much broader options for the end user. While we are now spoilt for choice, and the wide range of options is expected to drive adoption of big data and analytics solutions, this could also create a great deal of complexity.

Big data and analytics maturity in the Asia-Pacific region

IDC’s analysis has found that over the next 12-18 months in the Asia-Pacific, lines of business will be funding most big data and analytics investments. While the CEO is the most common sponsor for big data and analytics projects at 22 per cent, the IT function and CIO are each sponsors in 16 per cent of cases.

Across the region, there are very diverse levels of skill and approach to big data and analytics, which results in significant differences in investment.

Australia and New Zealand are the most mature countries in the region, as businesses are looking for ways to reinvent themselves, and are looking to big data and analytics to achieve this. The recently announced $1 billion innovation scheme announced by Australian Prime Minister Malcolm Turnbull is expected to stimulate further investment, particularly in relation to the Internet of Things and automation.

South Korea and China are countries with potential for strong growth. While Chinese organisations are still learning the value of big data, organisations will be driven to embrace new technologies, particularly in response to plans for smart cities and internet clouds.

India is still turning around, but with a growth in service innovation, IDC predicts there will be a need for enterprises to increasingly leverage data to improve productivity.

The ASEAN countries are least mature in terms of big data and analytics adoption, and broader digitalisation in these countries needs to occur before further growth will be seen here.

Key predictions and advice for buyers

1. Cloud spending to grow 3x faster than on-premise spending for big data and analytics solutions.
While currently, adoption of cloud-based big data and analytics technologies lags behind other areas of technology such as CRM and conferencing solutions, making up only about 4 per cent of the total market size, this could change in the future.

Only 5 per cent of respondents to IDC’s AP C-Suite Barometer plan to deploy big data and analytics as a service in 2016, but IDC expects increasing use of open source technology, data centre expansion to improve availability, and refined policies and regulation, to drive further adoption of cloud-based solutions from 2018 onwards.

Advice for tech buyers: When considering adoption of cloud-based big data and analytics solutions, having the right skill sets in place is essential. It is also important to develop a detailed profile of security requirements and critical continuity requirements, and to engage with your selected cloud service providers on this basis.

2. Inclusion of cognitive computing functionality.
IDC said that by 2020, 40 per cent of all business analytics will incorporate some form of prescriptive analytics, built on top of cognitive computing functionality. By ingesting cast amounts of data and providing accurate advice, cognitive computing will make certain processes faster.

Though the majority of APEJ organisations are still in the earlier stage of big data and analytics investment in which descriptive analytics usually play a stronger role, there will be a move towards more predictive capabilities in the future.

Though IBM’s Watson is perhaps the most well-known of the cognitive computing engines, Apple’s Siri, Microsoft Cortana and Wipro Holmes are other examples. All are opening up APIs to their systems, so that developers can develop intelligent applications based on these capabilities, and so IDC expects it to be much more mainstream by 2017.

Advice for tech buyers: Evaluate existing big data and analytics investments, considering the vendor product roadmap and your choice of consumption model. IT should consider how cognitive functionality could be included in any application development they might be undertaking. While these new capabilities can uncover new insights in data, they may also expose data privacy and access issues, and so this may require more policy consideration and training for those using data in this way.

3. Skills shortage will persist and extend to architects and data management experts.
According to IDC’s AP Continuum Survey, data-related issues are the biggest obstacles in big data and analytics deployments for 39 per cent of respondents. This shortage will see growth of big data-related professional services to the tune of 29 per cent (compound annual growth rate) in APEJ by 2020.

While there has been an increase in relevant courses offered by educational institutions, more time is required to mature this newly trained workforce.

Adding to the challenge is that the set of skills big data and analytics professionals need to be successful is not straightforward – while “the holy grail” is being able to explain solutions in the simplest possible terms, and what should be done to resolve problems, this requires both data science expertise and domain knowledge, so that the insights can be contextualised accurately.

Advice for tech buyers: Having the right skills in place will held to address end user expectations, the relationship between IT and lines of business, and ensure the technical workforce is equipped to deal with new challenges. Buyers should therefore look carefully at their talent acquisition strategies, provide upgrade or retraining opportunities on big data and analytics technology, and truly integrate analytics as part of the organisation’s culture.

4. Between now and 2020, spending on self-service visual discovery and data preparation tools will grow 2.5x faster than similar traditional IT-controlled tools.
Data quality and availability remains the top IT challenge for big data and analytics, and this is only expected to be exacerbated by the appetite for self-service tools amongst lines of business. This could present a real challenge for some organisations, with adoption growing significantly faster than tools IT controls – perhaps necessitating a reassessment of the current centralised IT practices. Self-service delivery, however, could be instrumental in building a data-driven culture across its business units.

Advice for tech buyers: Even though visual tools can be easy to use, training will still be required to ensure wider adoption and the appropriate usage of data. Organisations will also need to focus on data governance, security and vendor management in this area.

5. Monetisation of data will become a major stimulus for big data and analytics projects.
IDC has found that 51 per cent of Asia-Pacific organisations want to monetise their own data in the next 12 months, with location data seen as particularly valuable, and this is expected to skyrocket by 2020.

For example, Qantas has been leveraging its customer loyalty data to offer personalised services and flights for some time. They have now taken this one step further, establishing a new business line called Red Planet, which commercialises consumer insights for data and marketing services.

Though lines of business will be at the centre of efforts to create data monetisation, having the right analytics talent will be crucial, as well as involving IT in serving, masking, cleansing, packaging and pricing data for sale.

Advice for tech buyers: To pursue dollars for data, organisations will definitely need additional resources and innovative business models. As a sensitive area, enterprises should also consider consumer expectations around data governance and privacy, looking past “the letter of the law”. Involving IT in data governance and privacy committees will be essential.

Overall, IDC said over the next five years, enterprises will have many opportunities to leverage big data and analytics solutions to gain productivity and profit, but they must really strategise to think beyond the technology and fully get the benefit of these expanding capabilities.

For more information on IDC FutureScape 2016, visit