Cost-Effective Approach for Custom NER in Azure: Azure Language Studio vs. Azure Machine Learning

@Dinnemidi Ananda Kumar With respect to charges that were already incurred, if you any query please raise a support case from azure portal to get clarification on the same. Raising a support case for billing related issues does not require a support subscription.

For the rest of the questions, here is my assessment:

For a custom NER project like this, would Azure Machine Learning offer a more cost-effective solution compared to Azure Language Studio?

Azure ML service does not incur any charges directly, i.e the creation of ML workspace does not incur charges, You are charged only for the supporting services that support Azure ML. That is, compute used for your jobs, container registry, Networking components, Storage and any other Azure services that are dependent on running an experiment or a job. So, using Azure ML you can minimize costs if you plan to use minimal storage and general compute instances that can be shut down when not required.

Are there any specific pricing tips, configurations, or services within AML that can help reduce costs while maintaining high-quality NER model performance?

While training you can use compute instance and AKS for production deployment of trained model. You can control costs of compute instance by scheduling shutdown of the instance when not in use and deleting them completely when not required. When you deploy an endpoint, you can setup scaling mechanism to scale as per incoming traffic. You can estimate the costs with pricing calculator or deploy a test model to check if it actually incurs a lower billing than the current language service. However, you will have to manage storage, scaling and other dependencies required for running an endpoint.

Is it feasible to use Azure Machine Learning to create a similar custom NER model that is just as efficient as the one I built in Azure Language Studio?

Language studio offers the flexibility of readily deploying a model by training on top of baseline models. With Azure ML you will have to develop your own model or use an LLM model from model catalog. I would suggest to stick with language service if your scenario is generic and try Azure ML only if the NER model is complex or needs customized training.

Has anyone else experienced high costs in Language Studio, especially when working with document-heavy or larger-record datasets?

I have only used the service for smaller datasets, I think other users who have used the service at full scale can comment on the costs that they have incurred for heavy usage.

I hope this helps!! Thanks!!

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.