One of the world’s largest document migrations to SharePoint Online?

Introduction

The global importance of digital transformation is evident today, with organizations recognizing it as a necessity rather than a mere trend. In line with this imperative, Proventeq is actively helping modernize content infrastructure for a Middle East oil corporation. The customer sought to optimise their operational framework to be more agile, secure, and collaborative. The vision was clear: to foster a modern workplace that not only keeps pace with the ever-evolving digital era but also sets a benchmark for others to follow.

This project consists of transitioning File Shares and a variety of ECM-based sources to SharePoint Online and AzureFS. Although we've already achieved a significant milestone by migrating 1 Petabyte of data, our work isn't finished. This project has the potential to rank among the most extensive File Share migrations into SharePoint Online.

How much is 1 Petabyte worth, exactly?

There are 1,024 TB (or around one million Gigabytes) in one Petabyte (PB). That means in the real-world, 1 PB could equate to 500 billion pages of standard typed text. 

Epic 1 Petabyte Triumph                

Proventeq has achieved the 1 Petabyte milestone working in partnership with one of the world’s leading SI’s, with 75% of the content migrated in less than a year. After trying out other migration tools that are better suited for simple migrations, the Proventeq Migration Accelerator (PMA) was selected as the tool of choice because of its strengths in handling complex, large-volume migration to SharePoint Online. Proventeq’s migration experts were given the task of migrating the content because of our proven expertise in delivering large-scale, enterprise-grade migrations successfully.

Conquering SharePoint Online Migration Obstacles

From the outset of the project, we were apprehensive about whether the SharePoint Online platform and its migration APIs would be able to cope with such large volumes, especially as it required transforming the folder structure and applying new consistent functional taxonomy. Having worked with 20+ Enterprise Content Management platforms over the last decade, we were expecting significant performance challenges.

However, the Proventeq team was able to successfully scale out our Migration Accelerator (PMA) to run more than 10 parallel instances and scale up each instance to run up to 100 threads in parallel. The hyperthreading strategy resulted in a peak throughput of more than 7 TB per day, with the SharePoint migration APIs outdoing the performance expectations.

Assuming an average document size, the SharePoint Online platform has potentially ingested nearly 1 billion documents without any performance impact on end users. Compared to other non-Microsoft ECM platforms, SharePoint Online seems poised to scale seamlessly - kudos to the Microsoft 365 platform and the SharePoint Migration API team.

Outcomes and benefits 

The journey continues...

There remains content that isn't naturally suited for SharePoint Online, with some file structures that contain linked files are better suited. However, our objective is to maintain a unified ecosystem where users can access files seamlessly. Also, as we have now consolidated the platform with a functional-based taxonomy, we can now concentrate on implementing rules and retention policies. This will allow us to dispose of or archive outdated or unused content into more cost-effective storage solutions. Azure File Storage emerges as an ideal choice both in terms of cost and consistent user experience. We're currently crafting a solution where users will be able to access AzureFS content through their familiar SharePoint experience using link-based access (stubs).

Look forward to sharing the next part of this journey with you.

If you are facing a complex and/or large-volume content migration challenge, book a consultation with one of our experts and they will be happy to help.