Treasury News Network

Learn & Share the latest News & Analysis in Corporate Treasury

  1. Home
  2. Cash Flow Management & Forecasting
  3. Cash Flow Forecasting

Dark Data: Search, Collate, Conquer

Often Treasury departments are unaware of how much data there is in the places they are not looking. As most companies still rely on outdated technologies such as spreadsheets, they are confined by their limits. On top of that, according to a recent IBM study, more than 80% of all data is unused or unknown to the organization. The proportion of this so-called ‘Dark Data’ reached 93% by 2020. Luckily, Dark Data can appear against a dark night sky and will be visible to anyone who cares to look. But where do you exactly start your journey? As it’s dark, it’s easy to miss.


Dark matter is a form of matter thought to account for approximately 85% of the matter in the universe. It is called dark because it cannot be seen directly, and we do not know what it consists of. But determining what our universe is made of in the infinite void of space is mind-baffling and I will leave this to those better qualified. In stark contrast, there is a finite amount of a company’s data, albeit it is mostly unstructured. Data systems are capturing, generating and storing millions of bytes of data every second of every day, with the amount growing exponentially at breakneck speed.

Nowadays companies are able to capitalize on the extensive data at their disposal. Yet, many treasury departments get stuck at the starting gate, bogged down by the insurmountable amount. Therefore, navigating yourself through the massive amount of data available might appear like a confounding waste of time, that is if it’s done by manual labour. But if a man can be put on the moon with a minuscule 4 kilobytes of RAM, way less computing power than the cell phone in your pocket, we should be able to navigate ourselves through the wonderous data landscape that treasury and other departments bring forward.

What is Dark Data?

So, let’s start at the lift-off. What is Dark Data? As an idea, it is astoundingly simple: Dark data is defined as unused or hidden data from relevant departments which might have intrinsic value. For a company’s treasury department there is a vast amount of data often overlooked.

When setting up a cash forecast, one typically looks at directly retrievable first line/systemic data as you don’t have to search in the vast amount of space, it can hit close to home as well:

  • ERP & TMS data
  • Bank data, which can be retrieved by means of an (API) connection with your bank(s)
  • P&L data
  • Sales budget & forecast
  • CAPEX investments
  • Costs (restructuring, rental) 
  • Financial flows (loans & payments)
  • Manual input

This information is necessary, however typically lacks accuracy. But if you look past the surface of first-line data, there are shiny diamonds to be found. Small, brighter lights are lost in the bigger spectacle.

So, in the case of (indirectly computed) second line / dark data, what are we speaking of exactly?

  • Correlations between different types of cash flows
  • Average projected incoming & outgoing cash flows
  • Customer payment behaviour
  • Projected dividends
  • Future FX reports (risk/exposure)
  • Trends (Historic, seasonality, competition, …), using machine learning
  • New technologies increase/decrease sales

Pitfalls of Dark Data

What is behind the impressive rock formation of data might contain a lot of clutter, and if you scratch the surface, it may contain useless information after all. That’s why it is fundamental to address the potential pitfalls. Not all Dark Data is useful and it creates a misalignment between perceived and actual value. A point of issue that has been a notoriously difficult problem for every company. When looking for extra-terrestrial life you just might catch sight of a spectacularly bright meteor, but in the end, it will just obstruct your view and rattles the otherwise smooth ride. The same goes for cash flow forecasting. What might look useful beforehand turns out to be redundant and meaningless and will only cloud your forecast. For example, that funny GIF or Smiley you send to your colleague through Slack or Teams is considered company data as well but seems like an awful waste of space when it comes to cash flow forecasting. The assumption that “more is better” when it comes to data is not grounded and should be replaced with the mantra “more is better, but only when structured”.

Source & Copyright©2021 - Cashforce


This is why data-based operations often fail at the level of collating. If there is so much data to gather, you still need the time to process it. According to our own survey still, more than 90% of companies use spreadsheets for their day-to-day cash flow operations. Though spreadsheets have the in-built predilection to slow downtime for you, it is still bound by the space-time continuum that inhabits the earth. So why keep wasting valuable time?

Luckily, given the right technology and algorithms, we are able to drag out Dark Data into the light by means of adding structures and delivering a holistic view. Once you have collected the data, the next step involved is making it tangible by means of adding structure and setting up a data strategy to support it. It’s like trying to detect meaningful patterns in the infinite number of stars on a clear, dark night.

The first step is to understand your cash flow and working capital drivers. Next, you can define your cash forecasting horizons and your sources per time horizon. From there on you can integrate and consolidate the data by allowing the relevant subsidiaries/business units to submit their data. Ultimately, you can set up a well-oiled cash forecasting rocket ship.

Yes, there's a lot to structure at first. But, crucially, a lot worth structuring. Now you have all these independent data structures, all pointing towards the same picture: A single version of the truth. That's one small step for your company, one giant leap for your treasury team.


You have figured out the data quality and structures? Houston, we have to lift off. But where do you start to analyze? Just as you wouldn’t investigate the star closest to you with a pair of binoculars, you shouldn’t analyze your data set with a system that isn’t capable of handling high quantities of data. As well as in collating as in conquering data, technology has a vital role to play, but most companies still rely on older technologies, so they are confined by its data reach and limitations. Once you’ve set up your cash flow forecasting workflow, paired with the right technology solution, you are ready to conquer your data.

The next step involved is to apply logic and filters to the data to make forecasts for the intended time horizon. The results can be poured into flexible analytics & reporting so the correct measurements can be drawn. By making use of multiple scenario building abilities based on these different logics and assumptions, you can easily compare different reports and scenarios, preparing your company better for black swan events such as the current crisis.

The Current Crisis

The current crisis and its ripple effects made the demand around cash visibility and cash flow forecasting grow exponentially, with a large portion of it falling on the shoulders of treasurers. New data sets of structured and unstructured Dark Data are analyzed to reveal new trends which can then apply as a new structure of logic. This way you can create new insights by joining other past data with this year’s data.

Another effect of the ongoing crisis is the need for real-time data processing to get insights as they come in order to make data-driven decisions while being able to drill down to the transactional-level detail. Our NextGen platform will be fully API-enabled to facilitate real-time connectivity, solving the issue at once. This technological expertise in combination with flexible versatile data input and workflow capabilities, encapsulated in a centralized system, will result in an improved cash forecast building a bridge between different departments.

Artificial intelligence

What the moon landing was for the ’60s, is what artificial intelligence might become for our time. Advanced AI algorithms will change both the confinements of our inner world as outer cosmic dimensions and the predictions of our future. More crystal clear: The future of cash flows. Not only will it help with the structuring of data, but it will also create new data altogether by separating itself from the larger structures that enable it. Even better, in the nights that follow, the predictions will gradually increase in illumination, becoming more accurate over time.


In a world that only tends to see itself in hindsight, one of the biggest challenges of many companies is trying to predict what is coming and to adapt to the ever-changing environment. In conclusion, though Dark Data does not have the capability to enlighten a broad horizon of forecasting truths, it does illuminate the contours of a better cash forecast. More important though, is to get started, transport your data into your platform, add the right structures and clean it during the process, than to never get started at all. Because just as the night sky sparkles with wonder, so does the company’s potential mine of data. Now it’s up to your treasury department to search, collate and conquer and set for new frontiers in cash flow forecasting.

Like this item? Get our Weekly Update newsletter. Subscribe today

About the author

Also see

Add a comment

New comment submissions are moderated.