Introducing the End-to-End Data Journey
The data economy thrives on the reuse of data and requires data analysis in many shapes and forms. Whereas data can be searched, accessed and processed in many ways, we have identified a common pattern, a consistent journey that is walked by data centric researchers and data scientists to achieve their goals. We call this journey the End-to-End Data Journey.
The End-to-End (E2E) Data Journey is a structured framework that illustrates the secondary use of data, defining the interactions between Data Users and Data Providers. It represents the flow of accessing, processing, and reusing data while ensuring compliance with ethical and legal standards. Structuring the key steps of secondary use of data, the End-to-End Journey aims to accelerate data access, foster collaboration, and ensure secure and responsible data sharing.
Connecting Data Users and Data Providers to solve a data challenge
The End-to-End Journey starts with a problem, a research challenge or hypothesis, that needs one or more sets of data that carry the solution inside. The primary actor of this journey is the user of the data at stake. The Data User can be a researcher, data analyst, policymaker or other actor who has a data challenge to solve. However, the Data User is not alone in this journey, several other actors play along: data providers, data stewards, process facilitators, data authorities, processing experts, and more.
To solve the data challenge of the Data User, we need to involve the different actors, passing through various steps from data discovery to analysing and reviewing results. To explain the roles and interactions, we divide the End-To-End journey in two interconnected pathways, namely the Data User Journey and the Data Provider Journey.
Side-by-side of the Data User, is the Data Provider, or the holder of the data, who ensures the quality of the data, and provides access and reviews the results. The role data provider can be embodied by a data steward or organisation managing valuable datasets. The journey ends with a solution, a proven result, that creates an academic and/or a societal impact.
Illustrating the End-to-End Data Journey
A practical way to understand this journey is to look at an example. We have a Data User, let’s call him Marc, and a Data Provider, let’s call her Anne. To illustrate this example, we call Marc a researcher who needs data to solve his research challenge. However, he could also be data specialist in a company, or a policy maker in an administration.
After identifying an idea with a data challenge, Marc starts the Data User Journey, consisting of six key steps. First, he discovers the required data sets and requests access from the respective data holders. Once he obtains access, he can proceed to analysing the data – and in case of sensitive data – obtain permission, before using the results for research, policymaking, or innovation. Having found a solution to his data challenge, he finishes the journey with publishing, archiving and/or deleting the data. By following a structured process, Marc can focus on deriving insights rather than navigating complex access procedures.
On the other side, we have the Data Provider, Anne who holds the needed data. Anne’s role involves making data discoverable, deciding access conditions, and enabling secure use while maintaining compliance with legal and ethical standards. Anne gets notified by the data access request, and together with her team decides on providing access to the data in a secure processing environment. Finally, she will also review the results, before Marc can finish his research. By structuring this process, Anne ensures that her data is accessible, reliable, and used responsibly, supporting research and innovation.
Benefits of standardising the data journey
The journeys of Marc and Anne operate side-by-side, ensuring that data is both accessible and responsibly managed. With this standardised approach, Marc can generate impactful insights while Anne ensures responsible data governance—paving the way for a more efficient, secure, and transparent data-sharing ecosystem. This interconnected framework ensures that users and providers collaborate effectively, reducing inefficiencies and fostering trust. The End-To-End Journey provides clarity, structure, and security, supporting secondary data use while maintaining the highest ethical and legal standards.
“By standardising this data journey, we can standardise the roles and responsibilities of all actors, the processes and tools, to achieve its goals faster, more securely and more efficiently.”
Bert Verdonck, CEO at LNDS
For Data Users, standardisation reduces administrative barriers, allowing them to focus on analysis rather than navigating access procedures. For Data Providers, it creates a clear framework for compliance, ensuring data is shared responsibly while maintaining security and legal integrity. With a standardised view on this journey we can optimise the process, accelerate the access to data, while staying compliant with regulations at all times.
Impact of the End-To-End journey
So, why do we need faster access to data? Through faster access to data we can allow better decision making, and policy making and through that, drive innovation and productivity in Luxembourg and in Europe. With the structured framework that the End-to-End Journey provides, the journey clarifies the steps involved in the processes needed to access, manage, and reuse data securely and efficiently. By standardising workflows and aligning expectations, it supports responsible secondary use of data, while ensuring compliance with ethical and legal requirements. It serves as a reference point for researchers, data stewards, and organisations navigating the complexities of data sharing. This will be the impact of a standardised End-To-End Data Journey.
In the next articles we will dive further into the details of the End-to-End Journey, looking into the journey from the perspectives of the Data User and the Data Provider separately. Whether you are a researcher seeking solutions or a provider ensuring access, the End-To-End Journey can support in translating the complex procedures into streamlined processes and enable faster access to data.
Explore our Data Services to find a solution to your data challenge.