Cognitive benchmarking: How mass transit can make improvements that truly matter

After years of dealing with limited investment, mass transit agencies are struggling to meet the demands of their dynamic communities. Their trains, buses, facilities and networks are woefully out of date and inefficient. Their riders have found other ways to travel, draining vital revenue. And the estimated cost of bringing systems up to snuff is rising. Indeed, the nonprofit American Public Transportation Association estimated that for U.S. transit agencies to achieve a good state of repair, they’d need to collectively spend $90 billion.

Fortunately, a new approach can help: cognitive benchmarking. Aided by advanced machine learning methodologies, benchmarking helps transit agencies avoid “boiling the ocean” and taking on too many projects. Instead, they can prioritize the handful of improvements that will truly matter.

Further, cognitive benchmarking helps mass transit agencies identify industry best practices and determine how their organization stacks up against other agencies. Armed with this information, a transit authority can see where it lags relative to others, then spend selectively and carefully to make improvements that will deliver the highest material impact on its operational efficiency and customer satisfaction index.

Cognitive benchmarking can succeed where earlier approaches have fallen short because many past attempts at improving mass transit operations and the rider experience were based on gut instincts instead of facts. Even when these guesses were reasonable, decision makers had no way of determining which corrective actions would lead to the best outcome. As a result, many agencies either took on too many projects, often at great cost, or were so overwhelmed by the possibilities that they did nothing.

The way forward

Step 1

To start, a baseline is established, indicating an organization’s standing in more than 120 key performance measures — including financial, technical and operational — against the comparable proficiency of other transit authorities (see Figure 1).

Mass Transit Industry Key Performance Indicators (KPIs) Figure 1: DXC can perform a comparative analysis against 150 peer mass transit organizations and 120+ KPIs, to identify gaps and potential areas of improvement.

Step 2

Next, a gap analysis is conducted using an approach known as data envelopment analysis (DEA). This measures the relative efficiency of a transit authority against best-in-class agencies and the industry’s median. Having measured the “efficiency” gap, machine learning algorithms rank more than 100 possible key performance indicators (KPIs), then narrow these down to a few that will likely make the biggest impact on cost and performance. The gap analysis helps identify and evaluate risks that are unique to each organization (demographic and road conditions, to name a few) and develop KPIs to monitor and address the identified risks.

This benchmarking approach uses publicly available data from respected industry sources. Factors considered can include asset condition, preventive maintenance (PM) logs, capacity and utilization, customer service, environmental impacts, financials, human resources (HR), paratransit and disabled-rider measures, punctuality, safety, security and service continuity. This data is currently available for more than 950 mass transit agencies worldwide.

Often, the information will be detailed in the form of reports the transit agencies submit to national transit departments, such as the U.S. Federal Transit Administration, an agency within the U.S. Department of Transportation. Because these governmental bodies have stringent reporting requirements, the overall integrity of the data is high.

Step 3

Using the information derived in step 2, organizations can develop a roadmap to bridge gaps in underperforming areas relative to best-in-class agencies. Here — as in step 2 — machine learning approaches (such as Lasso regression and Random Forest) can be used to narrow the KPIs down to a manageable few.

Because enterprise asset management tools touch the entire organization, agencies must consider all KPI measures, including back-office and administrative functions, to achieve cost efficiencies at scale. Important data is often either siloed within a part of the organization or duplicated by departments that operate independently of one another.

For example, one agency could have several HR functions across different modes of transportation — one HR function for bus, another for rail, and yet another for light rail. Cognitive benchmarking can be used to identify these overheads through operational cost and employee headcount. Then the agency could make recommendations to centralize or consolidate the HR functions to reduce inefficiencies. That might involve standardizing both organizational structures and processes to achieve higher service levels at a lower total cost.

Highest-impact areas

Armed with benchmark data, a transit authority can quickly determine how well it performs on various measures relative to other agencies.

For example, a major metropolitan transit authority used benchmarking and discovered its vehicle cost per mile, at $6.10, was roughly 75 percent higher than the average for transit agencies of similar size and scale. Another transport agency used cognitive benchmarking and identified that its annual maintenance time — about 1,670 hours per bus — exceeded the average of its peer group by about 65 percent.

In both instances, insights gained from benchmarking alerted the agencies to opportunities for making major improvements and big impacts.

Part of cognitive benchmarking’s “secret sauce” involves coupling the DEA approach with machine learning algorithms to zero in on high-impact areas. DEA starts by taking different types of data inputs from transit agencies to plot a transit agency’s efficiency on an XY chart. This shows “what good feels like.”

Case in point: A chart on bus maintenance costs might display relative placements for “total vehicle maintenance cost per revenue hour” on the horizontal axis, while showing those for “vehicle maintenance costs as a percentage of operating expense” on the vertical. This lets an agency see how its bus maintenance costs compare with those of others, learning whether they’re among the best in class, the average or the lowest performers. To close any gaps, an agency can then use machine learning algorithms to identify the KPIs that could lower its bus maintenance costs.

Yet another benefit of cognitive benchmarking is the way it can help chief operating officers, chief information officers and other C-level executives overcome boardlevel resistance to the cost and complexity of adopting next-generation technology. Presented with this fact-based intelligence, an organization’s board may be more likely to approve funding for both new IT hardware and software systems.

Later, a transit agency can track its progress over time on a reporting dashboard. This allows the agency’s board members to observe improvements made over time, and encourages them to stay engaged. This kind of dashboard can even be adapted to run on smartphones and tablets for mobile, on-the-go use.

How DXC can help

One of the world’s leading IT services firms, DXC Technology is a recognized leader in complex enterprise asset management transformation. DXC has helped customers:

Modernize paper-based operations and facilitate the creation of a mobile-enabled digital workforce, which resulted in a 40 percent reduction in administrative requirements
Transform their EAM capabilities to plan, track and maintain assets and infrastructure, to better predict equipment failures, prevent equipment outages, and quickly repair and maintain equipment — all to ensure improved on-time performance and a better passenger experience