Share via

Data lake name sequence

azure_learner 525 Reputation points
2024-12-09T06:29:24.98+00:00

Hi friends, when creating Azure Data Lake the containers are sorted by alphabetical order, it looks a bit incorrect order :

bronze,

landing zone

gold

silver

I want it in the landing, bronze, silver, and gold orders. How can I do that? Is there a workaround? Please suggest one. Thank you.

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,513 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,053 questions
Accepted answer
  1. Keshavulu Dasari 2,570 Reputation points Microsoft Vendor
    2024-12-09T14:44:13.7666667+00:00

    Hi azure_learner ,
    Azure Data Lake containers are sorted alphabetically by default, and there isn't a built-in feature to customize the order directly.
    you can consider the following workaround:
    Metadata or Tags: Use metadata or tags to indicate the order. While this won't change the display order in the Azure portal, it can help in programmatically managing and accessing the containers in the desired sequence.
    Custom Application: Develop a custom application or script that lists the containers in the desired order. This application can fetch the container names and display them in the order you prefer.
    Documentation: Maintain documentation or a reference guide for users to understand the intended order of the containers.

    User's image Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members. If you have any other questions or are still running into more issues, let me know in the "comments" 

    0 comments

1 additional answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Keshavulu Dasari 2,570 Reputation points Microsoft Vendor
    2024-12-09T07:05:45.55+00:00

    Hi azure_learner ,
    Thanks for posting your query on Microsoft Q&A.
    The order of containers in Azure Data Lake. By default, Azure sorts of containers alphabetically, which can indeed lead to an order that doesn't match your desired sequence.
    One workaround is to prefix the container names with numbers to enforce the order you want. For example:

    1. 01-landing-zone
    2. 02-bronze
    3. 03-silver
    4. 04-gold

    This way, the containers will appear in the correct sequence regardless of alphabetical sorting.
    Data lake synonyms in use Raw layer = staging = bronze = landing zone.
    The Bronze layer is where we land all the data from source systems. The table structures in this layer correspond to the source system table structures “as-is,” aside from optional metadata columns that can be added to capture the load date/time, process ID, etc.

    In most cases, it's a good idea to keep the data in the Bronze layer in Delta format, so that subsequent reads from the Bronze layer for ETL are performant — and so that you can do updates in Bronze to write CDC changes. Sometimes, when data arrives in JSON or XML formats, we do see customers landing it in the original source data format and then stage it by changing it to Delta format. So sometimes, we see customers manifest the logical Bronze layer into a physical landing and staging zone.

    It's important to plan your data structure before you land it into a data lake. Having a plan helps you use security, partitioning, and processing effectively.

    For more details, refer to Data lake zones and containers.

    User's image Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members. If you have any other questions or are still running into more issues, let me know in the "comments" .

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.