What Linux OS you are seeing the issue?
If its Suse, it could be due to the DHCLIENT, so in your custom script run this command try
sed -i 's/DHCLIENT_SET_HOSTNAME="yes"/DHCLIENT_SET_HOSTNAME="no"/' /etc/sysconfig/network/dhcp and you may need to add reboot at the end of the script.
Can you see what hostname is being displayed for these commands hostname
cat /etc/hostname
and cat /var/lib/waagent/published_hostname
.
From this log (waagent.log) you can see what operation is overriding the hostname change cat /var/log/waagent.log
, to see if any issue with Azure Linux Agent.
One more suggestion is to grep -r at /etc and see if there are any files with old hostname, try to change them and reboot the VM.