Share via

Global init scripts (legacy)

  • Article

Important

This documentation has been retired and might not be updated. The products, services, or technologies mentioned in this content are no longer supported. See What are init scripts?.

Warning

Legacy global init scripts and cluster-named init scripts are end-of-life and can no longer be used.

Legacy global init scripts run on every cluster. They are less secure than the new global init script framework, silently ignore failures, and cannot reference environment variables.

Databricks recommends migrating all existing legacy global init scripts to the new global init script framework. Legacy global init scripts are not visible in the UI, but can be found at dbfs:/databricks/init/. They must be deleted or moved from that folder to complete a migration.

Migrate from legacy to new global init scripts

If your Azure Databricks workspace was launched before August 2020, you might still have legacy global init scripts. Legacy global init scripts have been deprecated and are no longer available on new workspaces. They should not be used.

Note

On September 1st, 2023, Azure Databricks will disable legacy global init scripts for all workspaces. Azure Databricks recommends that you migrate your legacy global init scripts to the current global init script framework as soon as possible.

Automated migration notebook

Databricks Engineering has created a notebook to help automate the migration process from legacy global init scripts. For instructions and a link to the notebook download, see Legacy global init script migration notebook.

Manual migration instructions

To migrate from legacy global init scripts to the new global init scripts:

  1. Copy your existing legacy global init scripts from their reserved DBFS location (/databricks/init) and add them to the new global init script framework using either the UI or the REST API. See Global init scripts.

    Keep them disabled until you have completed the next step.

  2. Disable all legacy global init scripts.

    In the admin settings, go to the Global Init Scripts tab and toggle off the Legacy Global Init Scripts switch.

    Disable legacy global init scripts

    Note

    If you don’t have access to the UI, remove all files from the /databricks/init location to stop the execution of legacy init scripts.

  3. Enable your new global init scripts.

    On the Global Init Scripts tab, toggle on the Enabled switch for each init script you want to enable.

  4. Restart all clusters.

Important

Once legacy global init scripts are disabled, legacy scripts stop running immediately. This has two important impacts:

  • Legacy scripts and new global init scripts are not run on nodes added when a running cluster autoscales. To ensure that new scripts run on all clusters and new nodes on existing clusters, you must restart all clusters.
  • Non-idempotent scripts may need to be modified when you migrate to the new global init script framework and disable legacy scripts.