Jul

13

I am sure there are people who treat what Goldman Sachs says as Gospel.

GEN AI: Too much spend, too little benefit?

Tech giants and beyond are set to spend over $1tn on AI capex in coming years, with so far little to show for it. So, will this large spend ever pay off? MIT’s Daron Acemoglu and GS’ Jim Covello are skeptical, with Acemoglu seeing only limited US economic upside from AI over the next decade and Covello arguing that the technology isn’t designed to solve the complex problems that would justify the costs, which may not decline as many expect.

Big Al writes:

The main critique seems to be one of cost/benefit: benefits are, as of now, too small, and costs too high. But it's apparent, even from the cheap seats, that lots of very smart people are devoting lots of resources to (1) improving existing models and creating new ones, thus increasing benefit; and (2) improving the efficiency of the hardware (and related software) to improve processing efficiency and reduce power consumption. Given how quickly things have developed over the last few years, it's hard to predict where we will be even 2-3 years from now.

Fwiw, this caught my attention:

GS US semiconductor analysts Toshiya Hari, Anmol Makkar, and David Balaban argue that chips will indeed constrain AI growth over the next few years, with demand for chips outstripping supply owing to shortages in High-Bandwidth Memory technology and Chip-on-Wafer-on-Substrate packaging—two critical chip components.

So I asked Gemini:

What companies produce High-Bandwidth Memory?

There are a few major companies that produce High-Bandwidth Memory
(HBM), but the top three are:

Micron Tech
Samsung
SK Hynix (https://en.wikipedia.org/wiki/SK_Hynix)
These companies have been producing HBM since its introduction in 2014, and they continue to develop new and improved versions of the technology. HBM is used in a variety of high-performance computing applications, including artificial intelligence (AI), graphics processing units (GPUs), and supercomputers.

What companies produce Chip-on-Wafer-on-Substrate packaging?

Chip-on-Wafer-on-Substrate (CoWoS) packaging is a type of advanced packaging technology that is becoming increasingly important for high-performance computing applications. Here are some of the leading companies that produce CoWoS packaging:
TSMC (Taiwan Semiconductor Manufacturing Company)
Intel
Samsung
ASE (Advanced Semiconductor Engineering)
Amkor Technology
JCET Group: JCET Group is a Chinese company that is a leading provider
of OSAT services in China.

K. K. Law comments:

Regarding the bandwidth constraints, the other one is communication bandwidth which still has yet to catch up with the computational speed. There are limits as to how close they can move their chips closer together even using 3-D integration to shorten the latency caused by propagation delay. In the near term, there are companies, including Intel, working to move the optical communication pipelines to the chip level. That in and of itself also soon presents a challenge as there are only so many optical fibers can be accommodated right next to the chip.

The generative AI appears to the biggest elephant for now because that gets the most attention from general public. However, AI/ML is penetrating into all sorts of applications that have not got much attention. The Goldman Sachs people, despite they are highly paid, they are surprisingly ignorant in this subject mater.


Comments

Name

Email

Website

Speak your mind

Archives

Resources & Links

Search