A shift in data usage is creating a huge change in retail structures and operating models, finds Retail Week’s new Data Management Report, in association with eClerx.

Big data is big news. There are few people in retail who haven’t heard of it, but perhaps fewer still are making the best possible use of it. That’s not surprising given the volumes of structured and unstructured data retailers are faced with on a daily basis.

Free to Subscribers

The Data Management Report is free to Retail Week subscribers. It is available to download here.

Key findings

80%

Of respondents say product data is still more advanced than customer data.

87%

Rate their preparedness to use data as five out of 10 (with 10 being fully prepared).

45%

Of interviewees say complexity of the data and the diversity of sources are ‘considerable’ obstacles.

36%

Complain of a lack of technical ability to analyse all data sources.

64%

Of interviewees struggle with capacity in their team with regards to data management.

60%

Say the marketing department leads the responsibility for managing big data; 47% say IT does.

33%

Of respondents want more cross-functional collaboration.

IBM estimates that the world generates 2.5 quintillion (18 zeros) bytes of data each day. And more than 90% of data has been created in the past two years. At the turn of the century only 25% of the world’s stored information was digital. Fast-forward to 2014 and just 2% is non-digital.

Myriad surveys have shown that companies want to embrace such data. Retail Week’s Data Management Report, in association with eClerx, digs deeper, capturing the views of a number of senior-level industry leaders, including chief executives, from a diverse range of retailers. It identifies the challenges they face, the opportunities they see and the changes they expect to make. It offers the most detailed picture of data analysis in retail to date.

For those that have become accustomed to using structured data and relatively small samples of behavioural data, big data has come as a bit of a shock. Information comes from everywhere – from sensors used to gather climate information and posts on social media sites, to purchase transaction records and mobile phone GPS signals – and it is coming all the time.

The complexity of the data and the diversity of sources are therefore considerable obstacles to managing it effectively, according to almost half (45%) of those interviewed. Social media,  for instance, has added a layer of unstructured, behavioural data to the structured data retailers have historically been comfortable with. “Hardcore transactional details are fact, but when you start to overlay them with some of our geodemographic and profiling data it makes it quite confusing,” admits the chief executive of a large etailer.

That explains why 80% of retailers feel their product data management is more advanced than that relating to their customers. Retail businesses  (traditionally reliant on walk-in and/or walk-by traffic) are built around product segments and product categories, and this could have hindered progress.

Until now, that is. Many of those interviewed claim their knowledge is beginning to move towards a more balanced appreciation of data, from which a new model could evolve that leads with customers and channels, rather than products. In future, some retailers believe customer data and predictive technology will be more important, with social media data perhaps even driving the direction of their business.

Lack of preparedness

Of course, few retailers are there yet. Asked how prepared they are to create a central hub of information to “bring the data alive for effective selling in today’s retail environment”, two thirds (66%) rate themselves as a five or six out of 10 (with 10 being fully prepared), while 14% feel they are below five. One in five feel they are better prepared than most, scoring their business readiness at seven or eight out of 10.

One of the most frequent reasons given for retailers seeing themselves as not quite as prepared as they’d like to be is a change in systems, skills and structure. Many have only recently recognised the need to implement systems to ensure data analysis is a core part of their business. Investment is now in the pipeline and is expected to accelerate in the near future.

Organisational structures are also changing – IT clearly has a critical role to play in a data-driven business, but sharing the information is poised to become the new norm. This will cause more functions of every retail business to be guided by data. Indeed, 13% of those interviewed believe that “every department should be responsible for data”.

Many retail bosses are aware that a surge of new data could “inundate and confuse” parts of the business. However, a controlled data push – in combination with what retailers noted as a refreshing ‘pull’ from parts of the organisation demanding data – is where many are headed. A third want to encourage more cross-functional collaboration across departments on data management, the report reveals.

Better data analysis is noted by 47% of those interviewed as the most critical factor that needs to change with regards to data management in their company. For the majority, the challenge is recruiting employees with a head for numbers and business  acumen to turn information into insight. Two thirds (64%) say they struggle with capacity in their team, while more than a third (36%) complain of a lack of technical ability to analyse all data sources.

Get the right people and the pace is expected to quicken, with companies keen to move from data collection to comprehension. The aim: to understand customers better, refocus their businesses and allow data to guide them.

Free to Subscribers

The Data Management Report is free to Retail Week subscribers. It is available to download here.