Cloud

Insights

Introduction

Overall Datacenter exit strategies and solutions are very relevant in terms of reapplying it for large scale digital transformations and migration to Azure cloud. The dispositions bucketing across the Legacy Applications, AS400, COTS, Non-SAP Applications, SAP Integrated Applications, and Enterprise File Share Migrations are repeatable and scalable.

Especially, the Enterprise File Share Migrations Architecture and strategy covering across geographies, multiple data centers, userbase is a key highlight involving groundbreaking next generation solution most relevant scalable solution across customers, industries positioning both Infosys and Microsoft with value edge over other Hyperscaler and transformation service providers.

The data analysis, classification and ownership identification were key factors contributing to the right sizing of the Waves for data migration to cloud and have sprint plans for cutover activities. Region specific strategies in data migration and identification of cloud regions were considered addressing the data privacy and legal requirements specific to the respective Geo’s. While we touched upon few of the third-party tools from benchmarking perspective, the primary aim is to have a cost-effective umbrella solution focused on the Organization digital goals and objectives addressing their future business needs and go to market requirements.

The data referred in this paper are indicative and mere reflection of business scenario in real time, and it doesn’t represent any of the actual data/statistics of the business/company.

Client Context:

One of the leading top five largest food and beverage companies in North America, initiated a cloud transformation program covering its business enterprise and to exit its on-premises data centers across the Europe, Middle East & Russia, North America, Asia Pacific geographies.

Enterprise File Share Transformation:
The program objective is to transform the various NetApp, Windows, Linux File shares across Client datacenters to Microsoft Azure native solution. This migration aims to enhance scalability, stability, and security while lowering the costs associated with maintaining on-premises storage infrastructure. The file share migration project aims to consolidate and migrate approximately ~300+TB of data from Europe, Asia, Australia and North America locations to Azure File Shares spread across multiple Azure regions.

This comprehensive transformation including data analysis & classification, segregation, and data migration using latest and greatest of Microsoft Azure tools and technologies. Leveraging technology enablement, the program ensures seamless data migration, enhanced data availability, improved scalability, and robust security.

Europe Geo

Client operations across 33+ countries covering Europe, middle east and Russian region and with more than 13K users were using the File Shares using both windows and NetApp arrays in the Germany third party Datacenter.

The business service regions include Middle East Countries, Russia, and European Countries. These File Shares data varied from Application data, Departmental data, and End User's data. Application data ranges from factory to grain elevator operations data stored and accessed through various upstream and downstream operations like Pentaho from Russia, Italy, AcNielsen from European countries, etc. Departmental usage of the share's ranges from country specific finance departments handling highly secured and very sensitive financial data, quality departments. Marketing departments, etc.

These file shares were distributed amongst 25 NetApp Virtual Services hosted on on-premises and a dedicated Windows Server. The volume of files stored in these shares are around ~25+ Million files, consuming around ~60 Terabytes of data in size. Also, the data center was connected to the Azure cloud using meager 400 Mbps dedicated link only. Secondly the time zone of the usage is also round the clock for both Factory operations and business operations.

North American Geo

Client operations across North American region constitutes corporate HQ and with more than 12K users were using the Windows File Shares in their primary Datacenter and DR Datacenter at strategic locations, it also serves entire United States of America, Argentina, Venezuela, Canada, and few other countries in Latin America.

These File Shares data varied from Enterprise Applications, Departmental data, and End User's data. Application data ranges from factory to grain elevator operations data stored and accessed through various upstream and downstream operations JDE, BODS, Snowflakes, Data Lake, EDI, Tableau, etc. Departmental usage of the share's ranges from Corporate Taxations, finance departments handling highly secured and very sensitive financial data, plant operations, HR departments, etc.

These file shares were distributed amongst 50+ Windows Servers in 11 cluster configurations. The volume of files stored in these shares are around ~198+ Million files, consuming around ~142 Terabytes of data in size. Both Primary and DR data centers connected to the Azure cloud using 10 Gbps and 1 Gbps dedicated link.

ANZ Geo

Client operations across Australia New Zealand region and with more than 2.5K users were using the File Shares using both windows in the Major Sydney Datacenter, along with mini DCs each in JP, NZ and Indonesia.

These file shares were distributed amongst 5+ Windows Servers in 2 cluster configurations, 4 standalone systems. The volume of files stored in these shares are around ~11+ Million files, consuming around ~27 Terabytes of data in size. Both Sydney datacenter, JP, NZ, Indonesia mini DCs connected to the Azure cloud using meager 400 Mbps, 50 Mbps, 150 Mbps and 150 Mbps dedicated link.

Transformation strategy and solution:

Key Influencing Factors:

Key Program Objectives:

Transformation Solution Highlights:

Solution Approach:

Disposition Key Considerations:

Azure Storage based native options:

From File Shares management standpoint, Azure Storage Platform natively provides the flexibility to adopt Azure Files, Azure Blobs along with Azure NetApp Files. While the Azure Blobs focuses on the unstructured data storage and Analytics solutions through Data Lake Storage.

The idea is to choose between Azure Files and Azure NetApp Files as the disposition strategy for transformation of traditional SMB, NFS file shares from various Datacenter’s across the Geos.

Disposition – Strategic Decision: The decision was to go with Azure Files based on the business needs, performance requirements, and long-term cost considerations.

Migration Key Considerations

In a comparative scenario on having traditional migration with that of DFS N based migration addressing the key considerations from business perspective.

Migration – Strategic Decision: The decision was to go with DFS-N based migration addressing the business needs, program hard milestones, and seamless user experience.

Execution Approach:

Europe Geo:

Phase I: Secured Data Migration

Phase II: Migrate logically grouped Countries in Waves

Sample migration implementation structure - overview

North America and ANZ Geos’:

Phase I: Secured Data Migration

Phase II: Migrate logically grouped Countries in Waves

Data Analysis:

Understanding the construct of the data at on Premises is a Key Success Factor for the File share transformation. It involves various layers of analysis that provides deep insight on to the distributed file share data.

From the usage standpoint, File Share are widely used by

User Access based Data traffic

File Shares are typically accessed / authenticated based on the Active Directory ACLs. Over a period, access is provisioned to wider groups across the organization. The primary challenge is to associate the AD IDs with the users, applications, etc. Also, it is not necessarily providing the insight about who are active current users, it just provides a laundry list of all of IDs having access to the shares.

Below are few of tools considered for access-based data traffic capture

Example Scenario: When engaging specific factory-based users, we identified set of external contract workers using generic IDs to access specific share data. Adding to the complexity these IDs were created historically with never expire password and is not compatible with Kerberos ticket authentication-based access to Azure files.
Solution: We took an elaborate step working Domain Authentication Management Group to upgrade the ID’s authentication.

IP based data traffic:

Gathering IP based traffic capture to analyze the data usage primarily provides insights into the wider array of Applications. This method requires a workshop-based approach amongst the key stakeholders from the client IT, Network, also CMDB master data to fetch the application mapping the corresponding IP address.

Below are few of tools considered for IP based data traffic capture

Example Scenario: Identified an Application is using Debian O/S community edition. Hence the application is not able to establish connection to the Azure storage account using domain credentials.
Solution: Jobs are updated to use the SAS Tokens to establish the connection with Azure Storage Account.

Example Scenario: While validating ~200+ Linux servers on-prem, identified AFT servers having CIFS file system-based mount points referencing to file shares
Solution: Enabled the Shared AVD User to run the ASN macros through Linux servers mountpoint for AFS file shares

Transformation Overview:

Target state architecture focuses areas

Azure Environment Setup

In an on-premises windows SMB environment, any folder can be mapped or exported as a Share by itself. Hence, it is important to understand the Share structure or to identify the Top/Root level folder in a share. This helps in defining the requirements for creating the Resource Group, Storage Account and subsequently file shares in Azure.

Note: Azure file doesn't have feature to map or export subfolders as a Share by itself.

Parameter required for creating Storage Account and Azure Files:

Additionally,

Approach for Storage Account creation:
Storage Account (SA) tier / size to be provisioned based on the above key parameters and inputs collected from the on-prem file shares. Typical scenario standard storage with large file shares enabled 100 TB sizing is provisioned for creating Azure Files. The pricing is generally based on the choice between Standard with utilized storage and Premium storage with Allocated storage, hence 100 TB is a normal sizing practice for creating the Standard SA.

Note: General practice, resource group and storage account creations are executed through pipelines and not handled manually in an enterprise environment. This practice enables the organization to maintain the standards and auto provision the network, security and private endpoint enablement that are established as part of Azure enterprise architecture.

Azure file share creation: Once the Storage Account is created, “Resource Group level – Contributor access” role is to be provisioned for the migration team to create file shares. However, as a standard Azure File are created by the pipelines as appropriate for the respective client organization.

Best Practices:

Note:

We had an interesting scenario, one of the on-prem file share had 33K Root Level shares. While it had a larger merger and acquisition-based history as business reason for its existence. Working with client stakeholders, we evolved a strategy to consolidate the data under 4 root folders at Azure Files and had them successfully migrated.

This demands an extensive communication plan reaching to all users to have them make the relevant adjustments post migration accessing their respective files and folders.

Data Migration using Data Box:

Enabled initial bulk offline data transfer from Germany data center to Azure - East Europe region.

Microsoft Ordering: Contract Name, Company Name, Datacenter Address, Contact no are required for raising the databox order with Microsoft. Typically, the SLA is 10 business days from the date of order for receiving the Databox at the client datacenter. It is possible Microsoft might split the order into multiple shipments is your order has more than one databox. This shipment also contains the return label from Microsoft for these databoxes. These are to be secured as soon as databoxes are unboxed from the shipment.

Image reference from: Quickstart for Microsoft Azure Data Box | Microsoft Learn

Learnings using Databox:

RoboCopy to Databox
Robocopy is used to script the data copy from vServers to Databox. Always ensure identification of top-level shares. Robocopy script should the copy only the root / top level shares, else the data gets duplicated in the databox. This has further ramifications when the databox ingesting the data to Azure Files.

Azure Files incompatibility check
Before have the data copied over the Databox, ensure that the source file shares are scanned for the incompatible characters in the folders and file names. While the data gets copied successfully to the databox, it will have a huge impact on getting the data ingested to the Azure files.

Databox Timelines:

Data Migration to Azure:

Net Generation migration product from Microsoft. Worked seamlessly with the Microsoft Storage Mover - Product Engineering team to enable this migration in timely manner.

MS - Storage Mover Setup at On Premises:
‘Microsoft Storage Mover’ or ‘Mover’ is new generation product providing fully managed migration service that enables migrating on prem files and folders to Azure Storage seamlessly. It enables managing the migration of globally distributed file shares from a single storage mover resource.

DFS N based seamless user experience

DFS-N based migration to enable zero impact to end users and make it 100% transparent from usage standpoint.

DFS (Distributed File System) Namespaces is a role service in Windows Server that enables to group shared folders located on different servers into one or more logically structured namespaces. This makes it possible to give users a virtual view of shared folders, where a single path leads to files located on multiple servers.

As a prerequisite, have the Active Directory joined with the Private endpoint enabled Azure Storage account with default share level permissions.

While configuring the DFS N especially for the migration from on-premises NetApp environment, do have DFS-N and DFS root consolidation.

During the Cutover activities, runbook should have the following tasks executed for successful implementation of the DFS-N based file share path access.

Autobots for Data Transformation

SecureAFS Incompatbot

RemediAFS Incompatbot

DFSRootbot

ActvUsrTrailbot

AutoPathRemountbot

Value proposition:

The above table depicts the average OPEX cost component in a sample scenario based on the storage accounts and utilized storage size as an indicative value projection for budgeting purposes.

Transformation Outcome

Conclusion

The paper presents the research of implementation strategies for Enterprise files share transformation through a systematic approach. The aim is to provide the guided path to begin from the Data Analysis, logical groupings, consolidation of various SMB / NFS based file shares on to umbrella solution with cost effective adoption of Azure native solution.

We analyzed the real time business problem of overarching dependency on file shares with various critical business functions, departments, plants, operation centers, enterprise applications and third-party vendors. Another facet of the File Share transformation also provides a window of opportunity to identify and update the various legacy naming conventions belongs to historically acquired companies, brands still prevalent in the environment. Additionally, historical data dating back to few years up to decades can be classified and dispositions can be applied to get a fresh start to data compliance and security policy implementation.

File Share Transformation at a Glance:

This case study limits itself to the Microsoft cloud native solutions for the enterprise file share transformation. It is primarily due to the hyperscaler choice of the client, and the cost effectiveness compared to the other third-party solution in the like of NetApp on cloud.

Depending on the nature of the client’s requirements some of the third-party solutions like Komprise can be adopted to take advantage of the data life cycle capabilities they offer over the Azure file shares. However, it comes with the additional operational cost and binding to the respective product vendor.

Also, this paper doesn’t discuss the data life cycle management due to its vastness in nature and the expectation is to focus on the Transformation of enterprise file shares from an on-premises datacenter scenario. The overall purpose of paper is to effectively achieve the digital transformation of legacy file shares successfully.

References

Throughout the preparation of this whitepaper, information and insights were drawn from a range of reputable sources, including research papers, articles, and resources. Some of the key references that informed the content of this whitepaper include:

These references provided the foundation upon which the discussions, insights, and recommendations in this whitepaper were based.