Global freight data company lifts, shifts, and transforms massive data warehouse with Azure and Power BI
Data That Empowers the Transportation and Logistics Industry
The transportation sector, which includes everything from trucks to ships to airplanes and everything in between, is a dynamic and complex network of systems and processes that receives information from many sources. Delivering massive amounts of data relating to freight to international businesses that send goods all over the world is complex.
One of our global transportation clients set out on an ambitious digital transformation in infrastructure to provide its customers with 24/7/365 accessibility to their shipping data. They also wanted to improve their customers’ ability to glean knowledge from the data, rather than just reporting numbers.
By moving from a legacy server farm and manual daily data updates to a Microsoft Azure cloud environment combined with Microsoft Power BI dashboards, it achieved the always-on service with unlimited space it needs to grow. Results include lower operating costs, time savings, greater security, better analytics and most importantly, happier customers.
They are a leader in supply chain cost management consulting and freight payment and audit data to World Top 1,000, Fortune 500, and SMB businesses. As a truly global company, they manage invoicing in 28 currencies and provides service delivery in 24 languages.
The company’s data management services provide accurate audits to help customers drive business decisions and lower costs. With customized reporting tools, they deliver proprietary insights and customer-specific analysis to optimize freight expenditures that are mission-critical to its customers.
The technology and customer support teams could see they would need to make changes to support the company’s growth plans and customers’ escalating service requirements. To make themselves future-ready, the company embarked on a bold digital transformation journey. The decision involved a major re-architecture of the company’s client-facing infrastructure, moving from legacy datacenters to Microsoft Azure. Simultaneously, the two teams decided to upgrade the company’s core data analysis services from SQL-based queries to live dashboards using Microsoft Power BI.
With the COVID-19 health crisis disrupting supply chains around the world, end-to-end visibility into freight data has never been more important to our customers. Their Director of Customer Care recalls the stakes for the company, and for its customers. “We knew we had to set the audacious goal of completing the entire lift-shift-transform process without a single minute of downtime. Our customers need access to their data,” he says.
About Transportation & Logistics
The transportation and logistics industry encompasses the planning, execution, and coordination of the movement of goods and people from one place to another. It includes various modes of transportation, such as air, sea, rail, and road, as well as the associated services and infrastructure required to support these operations.
Microsoft Partner Programs
Azure, Azure Power BI, Azure Synapse
- Lower operating costs
- Time savings
- Greater security
- Better analytics reporting
“By working with Microsoft, we can focus on the right things for our business, and not the things that Microsoft is great at. That’s really our goal.”
Chief Information Officer
Moving to Azure for always-on global service and space for unlimited growth
Acquisitions have been a key growth driver for transportation companies, but with several acquisitions all over the world, it meant that our client’s line of business system was distributed on a global scale, resulting in 22 different databases. Fortunately, the company had already ingested the data of each acquisition into a consistent dataset with a single table of common data elements. However, for a transformation of this magnitude, the volume of data presented new challenges. They process more than 230 million shipments a year, so employees and systems have to work with extremely large data sets in real time, with the largest containing 1.7 billion rows. “Just moving that much data around on a regular basis is a challenge,” notes their Manager of Data Services.
Data had been stored in a legacy environment. That system used Perl scripts on older-version SQL servers in a colocated datacenter and was more than 10 years old. The data updates to refresh customer data required large downloads from multiple internal servers, which then had to be uploaded, synced, and validated every day. During the update process, customers could not access their data. With data stored in five different servers, customers who wanted to see their freight data from end-to-end would have to do several extracts and then spend hours stitching the extracted data together. What’s more, the legacy system provided roughly 3,000 subscribers with personalized data view authorization and functions, all of which had to be maintained in the new environment.
“We chose OmniData because we wanted someone who would work alongside us, be part of the team, not someone who would just do the work. Now, we function as one team, talking to each other on a daily basis.”
Manager of Data Services
Selecting the right partner makes the move to Azure smoother and faster
To make the transformation as flawless as possible, our client needed a data management expert with specific experience moving legacy systems to Azure. In addition, they wanted a partner who could help execute its immediate cloud migration plans, as well as future growth. After a careful selection process, the company selected Omnidata.
“We chose OmniData because we wanted someone who would work alongside us, be part of the team, not someone who would just do the work. Now, we function as one team, talking to each other on a daily basis,” said their Manager of Data Services.
As a certified Microsoft partner, OmniData collaborated closely to develop an incremental transition pathway to move legacy data from local servers to Azure. Given the mission-critical nature of the information the client provides its customers, there was a joint imperative that the migration pathway could allow enough time between stages for team members to collaborate, assess, learn, and make necessary modifications.
Hosting in Azure decreases operating burden for customer care team
Looking back on the successful migration, our client noted, “With the data now flowing into Azure Synapse through an updated extract routine, our data is in a secure environment with fingertip accessible backup and disaster recovery. It’s a huge leap for us, and down the road it’s going to make the management of our data so much easier. Azure also makes it easier for us to stay compliant with the 19 [security, privacy, and regulatory] policies our clients require, like GDRP or SOX,” says their Manager of Data Services.
Our client can now give each user within every customer organization their own specific data privileges to ensure they only have access to the data required to do their job. In addition, all users must now use multifactor authentication for security and, most importantly, they can provide the 24/7/365 service its customers need. “No more Patch Tuesdays,” he adds, happily.
Day-to-day operations are much improved. For one, data is now portable. “We can move data from our Azure environment directly to our customers ERP or AP systems, which are also in Azure. They don’t have to download big files and re-upload them into their internal systems anymore,” he says.
Our client can more easily monitor performance metrics, scaling up and down to meet customer demand and optimize query speed. The company can even provide customers with a dedicated workspace instantaneously, if required. No more waiting for new servers.
On the financial side, the move reduces overall costs. Azure provides a simple pay-per-use structure, so there is no need to add or change servers when the company wants to improve customer service levels. The costs are now an operating expense instead of a capital expense, providing more budgetary predictability and consistency.
Layering Power BI on Azure saves hours of manual labor for customers
The CIO observes, migration was not the only goal in mind. “We didn’t go through this massive undertaking simply to improve our storage capacity. We had much bigger plans,” he says.Instead of just hosting data warehouses that allowed only straight SQL queries, the company also added Power BI as a data analytics interface. “In one case, we realized we could put what the customer was trying to solve right on the screen with Power BI.”
This eliminated hours of manual data manipulation for them. Power BI had been suggested by some of their customers, but adoption would require a change in process for others. However, with some training, even customers who were new to Power BI saw immediate benefits and were able to transition smoothly. They can now see live interactive dashboards that provide visibility throughout the shipping portion of their supply chain, regardless of shipping mode. They can also look at summary invoice-level data, or quickly identify anomalies, then drill down on supporting data to find the cause. “The difference in the speed of access and massive improvement in the visualization of what is happening in their supply chain is so valuable. It allows them to stay ahead of the curve,” they are pleased to report.
Customers comb through up to 1,000 data elements to keep track of their freight expenditures. With the ability to look more closely at the components of their freight spend, customers can prevent overpaying, optimize volume discounts, and minimize disputes. Not only can customers access this data more easily, but they can also understand it more quickly. Using Microsoft Power BI, users can see data that tells a story and is actionable. Customers also report that they can now see new opportunities for cost reduction because the data is visualized and illustrated so intuitively.
Moving forward with predictive and prescriptive analytics
“We’ve provided our customers with the ability to make better decisions faster. We want to see the same transformation within our own business,” says their Chief Information Officer.
Thus, in the wake of the successful Azure migration and Power BI deployment, our client is expanding OmniData’s mandate to move its 150-server internal infrastructure to Microsoft 365. They know that to continue its success, it has to grow its client base and raise the caliber of services it offers—and do both faster than ever. “By working with Microsoft, we can focus on the right things for our business, and not the things that Microsoft is great at. That’s really our goal,” she says. “We want to free ourselves up to work on what differentiates us. We are in the very early stages of the capabilities of Power BI and what we are doing to excite our client base. The next phase is about moving forward to predictive and prescriptive analytics. We’ve got a good start to this journey and there’s a significant amount of value and excitement to come.”