Transport for New South Wales: Enabling Real-time Visualisation and Forecasting
About the Client
Transport for New South Wales (TfNSW) is making New South Wales a better place to live, work and visit. They are responsible for strategy, planning, policy, regulation, funding allocation and other non-service delivery functions for all modes of transport in New South Wales including metro, rail, bus, ferry, light rail. Transport for New South Wales are continually investing in technology to provide and develop a safe, efficient, integrated transport system that keeps customers moving and connects communities.
The Challenge
The next evolution of public transport is real-time, personalised, data-driven engagement with customers across the network. To bring this to life, TfNSW faces the following challenges:
- Responding rapidly in the face of changing weather conditions, public events, and unpredictable delays to consistently deliver a high-quality customer experience
- Providing executives with actionable, real-time insights into network usage and passenger behaviour in a quick and cost-effective manner
- Enabling transport operators to react at speed to insights by providing in-depth operational data
Technology, especially big data, is the key enabler for the evolution of TfNSW. However, two major technical hurdles stood between them and their vision for transport in New South Wales.
Firstly, their existing on-premises systems lacked the scalability and reliability to form the basis for a reliable data platform. Secondly, TfNSW currently uses multiple data sources to help drive customer experience and operational efficiency. There is a significant challenge accessing, processing and analysing these data sets at speed to deliver insights in near real-time, and predictions ahead of time.
The Solution
Contino has worked alongside TfNSW to build a highly-scalable, fully-automated, cloud-native and serverless data analytics platform on AWS.
The platform was built using cloud-native tooling and DevOps ways of working to maximise business agility. AWS Lambda was used to minimise the operational and management requirements of the platform while reducing cost. AWS S3, Kinesis Firehose and DynamoDB worked in concert to enable the ingestion and transformation of high-velocity data feeds.
Contino have worked with us to deliver a fully automated cloud native Opal Analytics platform that provides a near real time view and predictions using Machine Learning on the patronage across the transport network.
David Spalding, Ticketing Solutions Development, Customer Technology & Services, Transport for NSW
The platform ingests data feeds from Opal (Sydney’s contactless fare collection system) in near real-time while aggregating additional data sources to provide actionable insights on passenger capacity, flow and possible interruptions.
Tableau and web dashboards are used at the front-end to provide customisable and automatable reporting capabilities to enable end-users to segment, analyse and visualise the data across the network to gain fresh insights.
The data allows TfNSW to gain a transparent view on:
- How many people are on the network, and on each mode of transport including the newly completed Metro line
- How many trips are planned
- Taps and Boardings in real-time as well as over time, broken down at location level
- Intermodal transfer percentages broken down by mode and location level
- Predictive patronage numbers up to 48 hours ahead
- Usage spikes in particular locations real-time delays and tap-on reversals
The platform incorporates AWS Sagemaker, a machine learning service, to provide powerful predictive capabilities based on past data. TfSNW can now predict how variations in weather will impact transport usage and patronage across each mode of transport, and the entire transport network.
It also provides visibility of activity across the whole public transport network in near-real-time, enabling TfNSW to meet its challenges on multiple fronts.
Operations staff and the leadership team now have access to the data they need to:
- Predict and react quickly to fluctuations in patronage, weather, delays etc.
- Access and deliver insights quickly on what is happening now across all modes of transport, including the newly finalized Metro line
- Drive new initiatives using the Near Real Time Warehouse by incorporating new data sets
- Allow data to be analysed and shared in real time across the whole enterprise, enabling other business units to improve their process and agility
Senior leadership have a near real-time view across the network and mode of transport that they can access from anywhere, on any device. They can slice and dice the data as they wish to get the information they need to make strategic decisions. Previously these insights were not available in near real time, and some situations could take months to access.
Operations teams have access to near real-time dashboards and reports designed and built specifically to drive data-driven decision making and forecast planning.
Additionally, going cloud-native has generated considerable cost savings and significantly increased the scalability and agility of the platform. The use of serverless technologies has reduced all server maintenance and operational burdens.
Lastly, the speed with which the Contino and TfNSW team were able to build the platform has gained wider attention across TfNSW, paving the way for further cloud, DevOps and data focused initiatives.
