[Python][FS][Azure] Enable Azure in pyarrow wheel builds

[Tom-Newton]

Describe the enhancement requested

Child issue of #39317

Currently Azure is disabled in pyarrow wheel builds.

There are a collection of scripts for pyarrow wheel builds on different OSs e.g. python_wheel_macos_build.sh, python_wheel_manylinux_build.sh, python_wheel_unix_test.sh, python_wheel_windows_build.bat, ...

Probably we should try to enable Azure by default in all the cases where S3 and GCS are also enabled by default.

Component(s)

Python

[Tom-Newton]

take

[Tom-Newton]

I'm quite keen to target https://github.com/Azure/azure-sdk-for-cpp/releases/tag/azure-identity_1.6.0 as minimum version of the Azure SDK. This release has a fix for the default credential and adds workload identity to the default credential. Both of these are valuable to me and I assume others would benefit too.

Even the older version that we currently target requires an update to vcpkg though, unless we resort to doing a bundled build of the Azure SDK.

[Tom-Newton]

This is unblocked now. I'm debating whether to go all in and enable Azure everywhere that GCS is enabled. Alternatively I could just start minimal until we actually have some pybinds.