![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(83.2, 67%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESK%3C/text%3E%3C/svg%3E)
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,491 questions
The 504 Gateway Timeout and 499 errors often occur with Azure Functions when processing larger files due to the default execution timeout limits, network constraints, or resource throttling.
For the Azure Function, you can increase the functionTimeout value in the host.json file. The maximum allowed timeout depends on your hosting plan:
- Consumption Plan: Up to 5 minutes by default, extendable to 10 minutes.
- Premium Plan or Dedicated App Service Plan: No maximum limit, so you can set a longer duration if needed.
ADF HTTP activity has its own timeout configuration, so adjust the timeout there as well.
Reading large files all at once may cause issues. Break down the processing into smaller chunks, perhaps reading and writing data in batches, to reduce memory load and processing time per batch.
I recommed using Durable Functions which enable long-running workflows by allowing the function to checkpoint and resume processing, which can help process large files by breaking down the task into smaller steps.