Process mining and process discovery are overlapping disciplines used in the pursuit of process excellence.
Process mining is an analytical discipline used to model, analyze and optimize processes. Technology is core to modern process mining methods, which rely on fact-based models generated with event data pulled from business information systems. Process discovery is an older technique that often relied on models created from manually-collected, anecdotal information. Today’s process mining solutions, such as Celonis EMS, modernize process discovery by leveraging technology to deliver data-driven models that represent how an organization actually operates, as opposed to subjective interpretations of how people “think” an organization operates.
Businesses, public agencies and organizations of all types use process mining to find and capture value opportunities hidden within their operations by improving the performance of their core processes.
What is process discovery?
Process discovery, sometimes called business process discovery (BPD), is the act of creating a model (i.e. a graphic representation) that can be used to analyze and optimize a process.
Process discovery emerged from the concepts of business process orientation (BPO) and business process management (BPM), which gained traction in the 1980s and 1990s. BPO is a management philosophy that encourages businesses to organize around the processes that add customer value instead of using a more traditional hierarchical structure. BPM is the process of discovering, modeling, analyzing and optimizing business processes.
In the early days of BPO and BPM, process discovery was often a highly manual process. Teams would dig through existing process documentation to identify all the people, activities, objects and systems involved in the workflow. Companies would hire consultants to conduct user interviews and facilitate workshops with employees, subject matter experts (SMEs) and process stakeholders. These in-person process discovery or process mapping exercises often involved people spending hours in a conference room drawing lines on a whiteboard or paper taped to a wall that’s filled with a rainbow of multicolored sticky notes. The resulting process map often resembled a flow chart.
With the rise of process mining technology in the late 1990s, pioneered by Professor Wil Van der Aalst, referred to as the “Godfather of process mining,” process discovery was dramatically transformed. Process mining software creates models using event data extracted from the information systems on which modern businesses operate, such as enterprise resource planning (ERP), customer relationship management (CRM) and supply chain management (SCM). Instead of paper maps based on subjective and sometimes inaccurate human descriptions, process mining creates objective, data-driven models. Process mining allows us to see how processes actually work instead of how people think they work.
What are the limitations of manual process discovery?
Volumes have been written about how to conduct process discovery workshops without putting people to sleep. But the most serious problems with traditional, manual process discovery aren’t keeping session participants awake. Manual process discovery, even done well, has the several limitations:
- Time consuming and expensive: Companies doing manual process discovery often use consultants to conduct the employee, stakeholder and SME interviews, facilitate workshops and dig through existing documentation. This can be a time-consuming and expensive endeavor.
- Subject to human error and bias: There’s an old anecdote that if you ask three people how a process currently works, you’ll get five different answers. Even when they have the best intentions and are trying to accurately describe a process, humans don’t always get it right. Users may misremember information, documentation can be outdated, and stakeholders may want to protect parts of the process that they believe are important to them or their team. Because the insights gained from manual process discovery are often subjective, discussions about root causes and next steps could be up for debate, further complicating and delaying corrective action.
- Delivers a one-time process snapshot: Manual process discovery delivers an “as-is” picture of a process. This As-Is model can be used as the baseline for creating a To-Be process, but it is an inherently backward-looking tool, given the time and effort required to create it.
- Assumes a standard process is best for all cases: Manual process discovery and the resulting “ideal process map” assumes that a standard process is the best process for each process instance, or case. However, this isn’t always true. The best process for any case is the one most likely to produce the desired outcome for that specific process instance.
The limitations of BPM and manual process discovery were a motivating factor for Prof. van der Aalst’s pioneering work on process mining. With the development of process mining techniques and technology, automated process discovery became a step in the broader process mining journey.
To describe how process mining transformed the research and application of BPM, Prof. van der Aalst, wrote that it was “no longer acceptable to waste time and money on creating BPMN (business process management and notation) models that say little about the actual processes.”
What are process mining and object-centric process mining?
Process mining enables organizations to find and capture the value opportunities hidden in their processes–invoices that were paid twice or goods that were shipped but not billed for.
Process mining consists of the following steps:
- Data ingestion
- Process discovery
- Process analytics
- Process benchmarking
- Conformance checking
Data ingestion
During data ingestion, event data is pulled from the information systems that run modern business operations, such as ERP, CRM and SCM. The data is stored in event logs and analyzed to create an unbiased view of how an actual process runs, not how people think it runs. These event logs contain at least three key pieces of data for each event:
Businesses can add additional information to increase context, such as vendor details for an invoice or priority level for a service ticket. Process mining data can also be combined with task mining data, which captures how work flows across a company’s desktop applications.
Process discovery
Once event data is extracted and stored in event logs, process discovery can begin, but not the manual process discovery described above. Using the event logs, process mining technology creates an end-to-end visualization, or map, of the process.
This process map is a chronological sequence of events that shows the different paths cases took from the beginning to the end of the process. Each unique path is called a variant. Analysts, process owners and stakeholders can explore this interactive process map to see the “happy path” (i.e., the path with the most frequent starting activity and the most frequent ending activity) and all the variants.
This digital representation of a real-world system is sometimes referred to as a digital twin.
Process analytics
Using the interactive process map, organizations can perform process analytics to uncover process inefficiencies, quantify their effect on KPIs and identify opportunities to drive value by addressing them.
Process benchmarking
Process mining technology enables organizations to compare process performance across different dimensions through process benchmarking.For example, an account receivables lead may want to examine the time it takes to process an invoice in two different countries, or the head of procurement may want to know throughput time of a purchase order from one supplier versus another. Process benchmarking, lets teams identify best practices and share them across the organization.
Conformance checking
Prof. van der Aalst defines conformance checking relating and comparing events in the event log to activities in the process model to find commonalities and discrepancies between modeled behavior and observed behavior. It enables organizations to see the percentage of cases that conform to a desired process and the percentage that don’t.
This allows people to know if steps are being skipped, taking longer than expected or executed out of order. As businesses make process improvements, conformance checking allows them to measure the number of cases that are following the optimal path.
Object-centric process mining
Object-centric process mining (OCPM) is an approach to process mining that allows organizations to better visualize and analyze the complexity and interconnectedness of modern business operations. OCPM uses an object-centric event log (OCEL) that allows an event to be related to multiple objects, such as sales orders, sales order items and production orders. Unlike a traditional process mining event log, an OCEL captures the one-to-many and many-to-many relationships between multiple objects.
OCPM addresses the limitations of and offers the following advantages over traditional process mining:
- Data extraction is only done once
- Interactions between objects are captured
- Produces three-dimensional event logs and process models
Using OCPM enables organizations to move from a two-dimensional view of their processes, where they look at a single object type, to a three-dimensional view where they can see multiple object types at the same time and how they are related. Celonis was the first technology vendor to fully embrace OCPM with Process Sphere™, a capability of the Celonis EMS that was introduced at Celosphere 2022.
Process mining modernizes process discovery
The evolution of process mining has been two decades in the making, and businesses across industries are using it to drive cash value, manage costs and improve customer satisfaction.
According to Everest, the process mining market grew by 55-60% between 2019 and 2020 and by 75-80% from 2020 to 2021. In their report, ”The Evolution of Process Mining: Game-changing Innovations and Future Outlook,” Everest predicted that by the end of 2023, the process mining software market would have a value of between $1.65-1.75 billion (US). Polaris Market Research predicted that the process mining market to be worth $11 billion by 2030.
In March 2023, Gartner® published its first Magic Quadrant™ for Process Mining Tools. According to Gartner, “by 2025, 80% of organizations driven by the expectations of cost reduction and automation-derived enhanced process efficiency will embed process mining capabilities in at least 10% of their business operations.
This rapid growth is a result of process mining’s power to accelerate digital transformation and deliver significant business value, including the modernization of process discovery.
Process mining addresses the limitations of traditional, manual process discovery and mapping by:
- Eliminating the need for manual techniques: Process mining is faster and cheaper than manual process mapping because it creates models from mined data not workshops, documents and user interviews.
- Providing objective, data-driven models: Process mining creates models that are more accurate than manual process discovery techniques because it pulls data directly from business information systems and desktops, when combined with task mining.
- Enabling continuous process improvement: Manually-created process maps are often outdated as soon as they are completed. Process mining captures data in near real-time and when it’s combined with artificial intelligence (AI), machine learning and automation, through a system like Celonis EMS, it can enable continuous process optimization.
- Finds the optimal process for an outcome: Process mining isn’t just about facilitating standard processes. It’s about finding the optimal process for an outcome using integrated insights from process flow, machine learning and business logic.
Process mining isn’t a replacement for process discovery. Quite the contrary. Process discovery is an essential step along the process mining path. Process mining takes the Directly-Follows Graph (DFG), Petri nets and BPMN (Business Process Model and Notation) models associated with process discovery and expands upon them.
Indeed, process mining practitioners and academics are constantly developing new discovery and mapping approaches that can better scale and handle the increasingly complex processes and data structures that power modern business operations. And, they’re building new solutions that deliver process intelligence to all parts of the enterprise to enable continuous improvement and process excellence.
Source: Celonis
