Powering the Growth of Generative AI

Three Morgan Stanley analysts; Stephen Byrd, Global Sustainability and Clean Energy Analyst, Lauren Barry, Real Estate Research Analyst and Chris Boulus, TMT Research Analyst together with Rob Koh, Head of Sustainability Research Australia came together at the 6^th Australia Summit to discuss how we’re going to power the growth of Generative AI. Globally, Australia has the 50th largest population but we’re in the top 5 in terms of data centre (DC) capacity. It’s no secret that supply cannot keep up with demand, so hyperscalers are willing to pay a premium to increase the capability of large language models (LLMs), as well as increase speed to market. Meanwhile, Real Estate is focused on converting the demand (and space needed for data centres) into Industrial Real Estate assets while different players are looking at converting crypto mines into data centres.

Stephen Byrd kicked off the session, highlighting that there has been incredibly rapid development of Generative AI (GenAI) software with mixed results ranging for the incredibly exciting to disappointing. Stephen Byrd explained that some of the most exciting areas of developments in GenAI are arising in life sciences. An example is the alpha fold 3, which deals with protein folding and simulating complex interactions of molecules, and drugs that determine impacts to human health. Another exciting area where we see GenAI popping up is climate modeling, where AI helps with predictive work to minimise damage from extreme weather events and assist in smart grid management. GenAI is also well used in business fundamentals especially marketing and disciplines that require coding. 

Byrd went on to indicate that hyperscalers are ‘all in’ particularly on the technology front. Byrd suggests that three of the hyperscalers are looking to spend a vast amount of money on super computers and indicated it could cost anywhere between US$50-100 billion per hyperscaler, per computer. The computer power they’re heading towards with these ultra large data centres – otherwise known as super computers, is quite remarkable. To quantify this, Byrd gave the example of Frontier (the U.S. Government’s extremely powerful supercomputer) which costs around USD $600 million to build and has computational power of about one exaflop (an exaflop being floating operations per second). Frontier is already incredibly powerful but the version of computers we see the hyperscalers creating would have computational power that would be north of 10,000 times greater than Frontier. This is anticipated to only go up from there due to the increase in computational power from every new generation of chip. 

Byrd also indicated that the pace of chip development is notable. As an example, NVIDIA was previously operating on a two-year development cycle and now moved to a one-year cycle, which essentially means they have three large teams working on three different generations of product simultaneously, in different phases. The amount of money NVIDIA is spending on R&D and the development cycle is significant and is resulting in ever-faster increases in computational power, in addition to the reduction of compute costs. Regarding the cost of computation over the next four years, it’s expected that the cost of compute will drop by another 90%. As that cost of compute drops it’s believed it will lead to a broad increase in use cases for GenAI. 

In terms of business models on the software side, there is interesting debate surrounding the incredibly capable proprietary large language models (LLMs) versus democratised open-source software via free small language models. It is expected that LLMs will continue to increase in capability at a very rapid rate and therefore there will be a large set of corporate and individual customers who will be willing to pay quite a lot of money to access the capabilities of these LLMs. However, there is also a vocal minority of investors who believe we’re moving in the exact opposite direction and feel that these open-source small language models (SLMs) are becoming a misnomer in the sense that they’re not that ‘small’ and as the cost of compute drops, you can see these small language models get bigger and more capable. 

There is a belief that LLMs will experience potential changes to software architecture that can create much more capable LLMs. For example, there’s a software architecture in the AI world called ‘Tree of Thoughts’ and a related version called ‘Graph of Thoughts’ and today these LLMs are essentially trying to complete a sentence or doing the equivalent of completing a sentence as a fairly linear exercise. 

However, in the future with ‘Tree of Thoughts’ it will become less a linear exercise and more a series of decision nodes quite similar to how the human brain operates. The ‘Tree of Thoughts’ software will likely be able to build out a series of decision node over time and the AI software is essentially going to be able to go backwards and forwards over time across that series of decision nodes to reassess, identify mistakes and ultimately redefine its approach. Byrd indicated that it may sound esoteric, but it’s been tried in practice. The majority view these LLMs as simply remarkable and there may be a revenue case with strong return on invested capital (ROIC). 

What is concerning, especially in the US, is the growth in DC development is not keeping up with demand. The Asia Pacific region however, including Australia, is demonstrating a greater capability to execute and build out GenAI-related infrastructure. Europe is a bit of a mixed bag, with areas of Europe struggling to build out DC infrastructure. There are positive signs of development in Spain, Portugal and few Scandinavian countries. 

Byrd indicates that there is large expenditure forecasted for AI infrastructure. For example, if you take our 2024 Global Data Centre growth number of about 12 gigawatts, on average it costs around USD $30 million per megawatt to build a cutting-edge DC. For the 12 gigawatts, it would cost around USD $360 billion on just the infrastructure before you consider power generation and other related categories which equate to a further tens of billion dollars. In 2025, the spend on DC could approach $500 billion globally. DC developers are looking into any solution that will get the DCs up as quickly as possible. 

Australia is ranked 50^th largest in population but are in the top 5 DC players in terms of capacity. The Australian DC capability stands around 1,000 megawatts today (comparatively, the US stands at around 9,000 megawatts) and we may see that grow at 15% the compound annual growth rate (CAGR) for the next 8 years reaching about 2500 megawatts by 2023. 

What gives us confidence, is how the Australian DC market relates to our skilled workforce.  Australia relies on international connectivity, high adoption of the cloud and high concentration of enterprises in Sydney and Melbourne. We believe growth in the DC market is going to continue to grow and two things are driving that: 

1) Australians are using more and more data i.e. in 2010 the average phone plan was 100 megabytes per month, 2015 is grew to 2 gigabytes and now the average mobile plan is around 80 gigabytes – so around 800 times larger over 15 years. Also, the average amount of connected devices has increased. In 2010-2020 the average Australian had two devices connected to the internet and today we have on average 4.5 connected devices which is underpinning the demand for DCs in Australia.

2) The growth for GenAI – which will use more data at the application level but also at the training level. 

To put some numbers around it on the leasing front, globally the last couple of quarters there’s been 1800-2000 megawatts per quarter of incremental demand for DCs. For context, during the peak of COVID that was about 200 megawatts. We’ve seen a massive uptick in demand and naturally the DCs must go somewhere, so we have seen the impacts flow into the real estate market. 

We see industrial real estate becoming beneficiaries as they have a lot of land in good locations, close to cities, close to power and being held at relatively low valuations. It takes US $30 million per megawatt to build these DCs. If you break this down into the components of land, it’s possibly $3-$4 million, then if you were to build that out to just the shell, that probably includes another $5-$6 million so that’s around an additional $10 million per megawatt just for the building. The required infrastructure such as cooling, the mechanical, engineering and plumbing could cost an additional $15-20 million. Of the hyperscalers 60% of their date centre space is leased rather than owned, so there is opportunity in real estate companies in the listed space to take this capital load off the hyperscalers, which have significant funding to spend on chips and training. The cap rates for DCs at the moment would be some of the tightest, globally at the moment.

Crypto mines are being extensively analysed as they use a vast amount of electricity, so perhaps there is some value in transitioning the mining sites into DCs. What is attractive about converting crypto mines was the ‘time to power’ analysis, which is conceptually how much of a premium a DC developer would be willing to pay if there were a power solution that could energise a power centre more rapidly, compared to connecting to the grid in the standard fashion. 

Recently, a crypto mine in the US was converted into a DC which involved Core Weave, a DC developer, and Core Scientific, a crypto mining company – where Core Weave agreed to pay $290 million a year for 12 years for a 200 megawatt site with Core Weave still required to build the DC. The value when converted out to a perina cost of power assuming 70% utilisation rate will be $235 per megawatt hour. To put that in context, the price of power in Texas is $55 a megawatt hour so this premium of $235 is astounding. 

Another example of a company reducing time to power was seen through Amazon, who paid a premium to a US nuclear power plant owner to site an extremely large DC at an existing nuclear power plant and premium on that was $30 per megawatt hour. In hindsight, it seems Amazon scored a good deal. The investing universe is accounting for this factor, especially when reviewing companies that can help reduce the time to power for DCs. 

One thing to consider is what happens in 10-15 years when these hyperscaler contracts come up for repricing as a lot of the hyperscaler contracts are signed with big DC operators and range in contracting terms anywhere from 5 to 15 years. Currently these contracts are being executed at pretty attractive terms due to the supply and demand imbalance. There is risk that in 10-15 years once these data centres are build and created, the optionality or flexibility will go back to the hyperscalers if at the time when they go to renew these contracts there is exceeding supply vs demand. The LLMs may also get more and more efficient as they get smarter and may not require as much data in their oprations. 

The analysis on power demand that around 5% of Australian power is estimated to be directed to DCs and that may grow to 8% by 2023. In the Irish grid that’s moving to about 20% today and in the Dublin metro area it sists around 35%. In places like Singapore, power has become a constraint and the Governments of Ireland and Singapore do tap the breaks every now and then to keep the pace going. 

92% of hyperscalers prioritise power over location. Planning fees in Australia sit around $6 million, while dealing with State Governments and may take up to 2 years to get through the system so it’s incredibly complex and expensive, with some hyperscalers looking to property experts to take them through the process. Accessing DC specialist property developers can also allow for access to assistance with the infrastructure that goes into the DC itself which takes an additional four years to build once you’ve obtained the approvals. It is projected that over the next 7 years there will be a lack of supply.

There has been a consistent trend with each new chip using more power than the prior chip e.g. the shift in the Hopper to the Blackwell saw a 250% increase in computational power however the power usage also increased by around 40%. 

However, if that proliferation is not rapid in terms of use cases, then this relentless march in terms of power efficiency may result in a reduction of power usage – although that’s not envisioned to occur any time soon. 

From a carbon emissions perspective, we will see an understandable increase from DCs – even though a majority of the hyperscalers have made quite strong net-zero commitment, what we’re seeing in practice is the number one focus and prioritisation of AI infrastructure and development is speed to market or any solution in the power sector that can get a DC powered up more quickly will be favoured. 

In same cases we will see natural gas and coal powering these power plants and DCs but we also seen moves toward nuclear power. There are forecasts that 10s of millions of C02 emissions annually from the DC sector over time and that’s just from the power itself and not taking into account the actual construction of the DC and internal infrastructure. Companies may look for creative ways to offset the emission by being highly active in the voluntary carbon market, offset purchases and purchases of renewable energy certificates. In terms of the grid, the world is already under stress and we will see a rise in some degree of fossil power usage as a result of this rapid use of AI.