Azure Storage libraries for Java

The Azure Storage libraries for Java provide classes for working with data in your your Azure storage account, and with the storage account itself. For more information about Azure Storage, see Introduction to Azure Storage.

Client library for data access

The Azure Storage client library for Java supports Blob storage, Queue storage, Azure Files, and Azure Data Lake Storage Gen2 (preview library).

Add the package to your project

Add the following dependencies to your Maven pom.xml file as appropriate:

<dependency>
    <groupId>com.azure</groupId>
    <artifactId>azure-storage-blob</artifactId>
    <version>12.4.0</version>
</dependency>

<dependency>
  <groupId>com.azure</groupId>
  <artifactId>azure-storage-queue</artifactId>
  <version>12.3.0</version>
</dependency>

<dependency>
  <groupId>com.azure</groupId>
  <artifactId>azure-storage-file-share</artifactId>
  <version>12.2.0</version>
</dependency>

<dependency>
  <groupId>com.azure</groupId>
  <artifactId>azure-storage-file-datalake</artifactId>
  <version>12.0.0-preview.6</version>
</dependency>

For more information about adding a dependency in Java, see Add a dependency.

Example usage

The following example creates a storage container and uploads a local file to the storage container.

String yourSasToken = "<insert-your-sas-token>";
/* Create a new BlobServiceClient with a SAS Token */
BlobServiceClient blobServiceClient = new BlobServiceClientBuilder()
    .endpoint("https://your-storage-account-url.storage.windows.net")
    .sasToken(yourSasToken)
    .buildClient();

/* Create a new container client */
try {
    containerClient = blobServiceClient.createBlobContainer("my-container-name");
} catch (BlobStorageException ex) {
    // The container may already exist, so don't throw an error
    if (!ex.getErrorCode().equals(BlobErrorCode.CONTAINER_ALREADY_EXISTS)) {
        throw ex;
    }
}

/* Upload the file to the container */
BlobClient blobClient = containerClient.getBlobClient("my-remote-file.jpg");
blobClient.uploadFromFile("my-local-file.jpg");

For more examples, review the Client Library README.

Available packages

The following table describes the recommended versions of the storage client library for Java.

Refer to the Azure SDK Releases page for details on how to install and use the preview packages.

Client library for resource management

Use the Azure Storage resource provider to manage storage accounts, account keys, access tiers, and more. To use the resource provider library, add a dependency to your Maven pom.xml file. The latest version of the resource provider library is available on Maven.

For more information about the resource provider library, see the Management reference. The source code for the resource provider library is available in the Azure Java SDK repository.

The following example creates a new storage account in your subscription and retrieves its access keys.

StorageAccount storageAccount = azureResourceManager.storageAccounts().define(storageAccountName)
    .withRegion(Region.US_EAST)
    .withNewResourceGroup(rgName)
    .create();

// get a list of storage account keys related to the account
List<StorageAccountKey> storageAccountKeys = storageAccount.getKeys();
for (StorageAccountKey key : storageAccountKeys) {
    System.out.println("Key name: " + key.keyName() + " with value "+ key.value());
}

Known issues

Older versions of the Azure Storage SDK for Java (v12) have one or more known critical issues, which are detailed below. These issues may impact the writing or reading of data from your Azure Storage account. Each known issue includes a description, the impacted versions, the minimum safe version, and the recommended action to take. If you are using an older version of a client library, we recommend that you update to the latest version.

If you have questions or need additional help, please create a support ticket using the following options:

• Issue type: Technical
• Service type: Blob Storage
• Summary: #JavaSDKv12
• Problem type: Development
• Problem subtype: Client library or SDK

List of known issues

1. Buffer overwrite issue with BlobOutputStream
2. Invalid data uploaded during retries
3. Upload incorrectly returning as successful when IOException occurs
4. Incorrect data being downloaded with downloadToFile
5. Overwrite parameter not honored while uploading large file, resulting in incorrect overwrite
6. Overwrite operation reversed for overwrite parameter, resulting in incorrect overwrite
7. Message content incorrectly erased when only visibility timeout set
8. Client-side encryption updated to use AES-GCM due to security vulnerabilities in CBC mode
9. Incorrect data being downloaded with downloadToFile() when underlying REST requests are retried
10. InvalidHeaderValue error message when using beta version of SDK
11. Uploading with BlobClient.upload(InputStream data) overwrites existing blob by default
12. Downloading with ShareFileClient.downloadToFile() can write incorrect data to a file

1. Buffer overwrite issue with BlobOutputStream

Issue description

If a BlobOutputStream object is used to upload blobs, in some scenarios this usage may result in an invalid object being written to Azure Blob Storage. BlobOutputStream object can be obtained via BlockBlobClient.getBlobOutputStream().

Uploading a file larger than the value of MaxSingleUploadSize using the write() method of the BlobOutputStream class results in an invalid object being written to Azure Blob Storage. The default value of MaxSingleUploadSize is 256 MiB. You can change this value by calling the setMaxSingleUploadSizeLong() method of the ParallelTransferOptions class.

After the input data size crosses the MaxSingleUploadSize, the write() method of BlobOutputStream returns before making a deep copy of the input data. If the invoking application overwrites the input data byte array with other data before the deep copy takes place, invalid data may be written to the blob.

Issue details

Recommended steps

1. Update client library versions according to the table above.
2. Check if your application code is calling BlockBlobClient.getBlobOutputStream(). If you find it, your application is impacted.

Additionally, you can identify any potentially affected blobs due to this issue in your Azure Storage account. Follow steps below to identify potentially affected blobs:

1. Check whether your application is using BlobOutputStream to upload blobs (obtained via BlockBlobClient.getBlobOutputStream()). If not, then this issue doesn't affect your application. However, we still recommend that you upgrade your application to use version 12.22.1 or later.
2. Get the MaxSingleUploadSize value for your application (256 MiB by default). Scan your code for setMaxSingleUploadSizeLong() method of the ParallelTransferOptions class and get value you provided for this property.
3. Identify the time window when your application used client library version with this issue (12.0 to 12.10.0)
4. Identify all the blobs uploaded in this time window. You can get a list of blobs by calling the List Blobs operation with PowerShell PowerShell, Azure CLI, or another tool. You can also leverage the blob inventory feature.

Following these steps will indicate blobs that are potentially impacted by the critical issue and may be invalid. Inspect these blobs to determine which ones may be invalid.

Back to list of known issues

2. Invalid data uploaded during retries

Issue description

The client libraries listed below have a bug that can upload incorrect data during retries following a failed service request (for example, a retry caused by an HTTP 500 response).

Issue details

Recommended steps

1. Update client library versions according to the table above.

Note: Azure doesn't have the ability to recover incorrectly written objects. As any potential impact occurs before upload, Azure doesn't have a valid copy of any affected object. If you have the original file, it can be reuploaded to your storage account.

Back to list of known issues

3. Upload incorrectly returning as successful when IOException occurs

Issue description

All overloads of void BlobClient.upload() and void BlobClient.uploadWithResponse() silently catch error responses from the storage service. The method should either return or throw as its success/error indicator. The exception, which should have been logged and propagated would instead be directly written to standard error and then swallowed, despite throwing being the only failure indicator for the API. The method therefore successfully returns, making the caller think the operation completed. This results in the blob not having been written to storage, despite the library indicating success.

Issue details

Recommended steps

Update client library versions according to the table above.

Back to list of known issues

4. Incorrect data being downloaded with downloadToFile

Issue description

Asynchronous buffer writing has a race condition where the buffer between the network stream and the file stream could be reused for incoming data before being flushed to file. This results in the downloaded file being corrupted, where some data immediately repeats, overwriting the valid data in its place. The object in Storage is still correct.

Issue details

Recommended steps

Update client library versions according to the table above.

Back to list of known issues

5. Overwrite parameter not honored while uploading large file, resulting in incorrect overwrite

Issue description

The overwrite flag isn't being honored in cases where there's another parallel upload job in progress. This results in not overwriting an object in Storage when intended.

Issue details

Recommended steps

Update client library versions according to the table above.

Back to list of known issues

6. Overwrite operation reversed for overwrite parameter, resulting in incorrect overwrite

Issue description

The overwrite parameter and overwrite operation are reversed in DataLakeFileClient.flush(long) and DataLakeFileClient.flush(long, bool) functions. No other behaviors of the library call into these methods. This results in overwriting an object in Storage when the user didn't intend to, and failing to overwrite when intended.

Issue details

Recommended steps

Update client library versions according to the table above.

Back to list of known issues

7. Message content incorrectly erased when only visibility timeout set

Issue description

Queue message contents are erased in error when only the visibility timeout was set or updated.

Issue details

Recommended steps

Update client library versions according to the table above.

Back to list of known issues

8. Client-side encryption updated to use AES-GCM due to security vulnerabilities in CBC mode

Issue description

To mitigate a security vulnerability found in CBC mode, the Java v12 SDK has released new version of client-side encryption called v2, which uses AES-GCM for client-side encryption instead of CBC mode. The updated SDKs are backward compatible and provide the ability for you to read and write data encrypted with the v1 version. For complete details, please read Azure Storage updating client-side encryption in SDK to address security vulnerability. Section 2 of the blog post outlines steps to take to see if this issue affects you.

Issue details

Recommended steps

Update client library versions according to the table above. Please read Azure Storage updating client-side encryption in SDK to address security vulnerability for recommended action.

Back to list of known issues

9. Incorrect data being downloaded with downloadToFile() when underlying REST requests are retried

Issue description

The Azure SDK for Java Storage library had a bug where incorrect data could be written to a file with the downloadToFile() method when some of the underlying Storage REST requests experienced network failure midway through. This bug was originally introduced in the summer of 2022 and was patched in May 2023 by returning to the previous behavior. The impacted versions are 12.19.0 through 12.22.0. The patch is in 12.22.1.

Issue details

Recommended steps

Update client library versions according to the table above.

Back to list of known issues

10. InvalidHeaderValue error message when using beta version of SDK

In rare scenarios, applications that have upgraded to the latest beta or generally available version of the SDK can receive an InvalidHeaderValue error message. This issue can occur when using any of the Storage libraries. The error message looks similar to the following sample:

HTTP/1.1 400 The value for one of the HTTP headers is not in the correct format.
Content-Length: 328
Content-Type: application/xml
Server: Microsoft-HTTPAPI/2.0
x-ms-request-id: <REMOVED>
Date: Fri, 19 May 2023 17:10:33 GMT
 
<?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidHeaderValue</Code><Message>The value for one of the HTTP headers is not in the correct format.
RequestId:<REMOVED>
Time:2023-05-19T17:10:34.2972651Z</Message><HeaderName>x-ms-version</HeaderName><HeaderValue>yyyy-mm-dd</HeaderValue></Error> 

If you've upgraded to the latest beta or generally available version of the SDK and you experience this error, it's recommended that you downgrade to the previous generally available version of the SDK to see if the issue resolves. If the issue persists, or if the recommendation is not feasible, open a support ticket to explore further options.

Back to list of known issues

11. Uploading with BlobClient.upload(InputStream data) overwrites existing blob by default

Issue description

The BlobClient.upload(InputStream data) method overwrites an existing blob by default. This behavior contradicts the method description. To prevent overwriting an existing blob, you can use the BlobClient.upload(InputStream data, boolean overwrite) method and set the overwrite parameter to false.

Issue details

Recommended steps

Update client library versions according to the table above.

Back to list of known issues

12. Downloading with ShareFileClient.downloadToFile() can write incorrect data to a file

Issue description

In the azure-storage-file-share package, the ShareFileClient.downloadToFile() method can write incorrect data to a file when more than 5 retries occur due to partial network responses while streaming data. The maximum number of retries is unintentionally higher (up to a maximum of 15), which can result in writing to incorrect positions in the file when the operation is retried more than 5 times. This issue doesn't affect data in the Storage service, only data being written to the local file system.

Issue details

Recommended steps

Update client library versions according to the table above.

Back to list of known issues