According to the report "AI Data Center Market Size, Share, and Trends 2025-2030" released by market research firm MarketsandMarkets in December last year, the AI data center market size is expected to reach approximately USD 236.44 billion (about KRW 328.17 trillion) by 2025, growing annually by 31.6% to reach a market size of up to USD 933.76 billion (about KRW 1,296.05 trillion) by 2030. Even considering the differences in predictions by other market research firms, there is no disagreement that the AI data center market will grow to about USD 800 billion (about KRW 1,110.4 trillion).

MarketsandMarkets cited the increase in AI computation processing through the digitalization of industries, the expansion of AI investments by governments worldwide, and the need for extensive data analysis and high-performance infrastructure through AI as the background for this explosive growth. However, the issue is that the infrastructure demand for AI data centers is concentrated on NVIDIA's graphic processing units (GPUs). Additionally, due to the price fluctuations of high-bandwidth memory (HBM), a key component, and the demand concentration from big tech companies, a global imbalance in AI infrastructure is inevitable.


Given this situation, big tech companies are focusing early on designing their own semiconductors or securing AI semiconductors dedicated to inference. Google has been developing the Tensor Processing Unit (TPU) since 2017 and is using it in its cloud services. AWS also possesses various types of proprietary semiconductors such as Graviton, Trainium, and Inferentia. Recently, even OpenAI, which is not a cloud service company, has decided to launch AI chips with Broadcom, spreading the enthusiasm for securing AI semiconductors to all AI companies.

Global Companies Focus on Securing Semiconductors··· Significant Potential of Domestic AI Semiconductors

While AI companies worldwide are focusing on securing their own semiconductors, Korean companies are targeting the global market with self-designed semiconductors. Notable AI semiconductors in Korea include FuriosaAI's second-generation neural network processing unit (NPU) RNGD and Rebellions' Rebell Quad, which are highly regarded. RNGD, unveiled at Hot Chips 2024, is evaluated as suitable for high-efficiency inference tasks. Rebellions unveiled the chiplet-based AI semiconductor 'Rebell Quad' at Hot Chips 2025 in August this year. Chiplet is a packaging technology that combines semiconductors made with different processes into one. Additionally, Mobillint and DeepX are also targeting the market with server semiconductors and edge AI devices.


As domestic AI semiconductor companies unveil major new products this year, competition for the global market is intensifying. Moreover, Canadian and US-based AI semiconductor companies such as Tenstorrents, SambaNova, Cerebras, and Groq, based in Silicon Valley, are also hurdles to overcome. Semiconductor companies based in Silicon Valley have the advantage of being on their home ground, making it a situation where Korean companies need to strive even harder.

HyperExcel's Papers and Products First Noticed Overseas

Amidst this, there is a company that is already receiving significant attention even before launching next-generation semiconductors. It is 'HyperExcel,' an AI semiconductor company founded in 2023 by Professor Kim Joo-young of KAIST's Department of Electrical and Electronic Engineering. Shortly after its establishment, HyperExcel launched 'Orion,' an FPGA (Field-Programmable Gate Array) based LPU (Large Language Model Processing Unit) developed in collaboration with AMD, attracting industry attention. Currently, they are developing 'Bertha,' an ASIC (Application-Specific Integrated Circuit) based on a 4-nanometer process, aiming for release in the first half of next year.


The potential of HyperExcel's LPU was first noticed in academia. The paper 'LPU: Latency-Optimized and Scalable Processor for Large-Scale Language Model Inference,' published by HyperExcel last year, was selected as the Best Paper for 2024 by the IEEE Computer Society's Micro journal last month. It is the most prestigious award among the paper awards won by domestic AI semiconductor companies.

The IEEE Computer Society, which awarded the Best Paper, is the largest among the 39 technical societies under IEEE, with 380,000 members active in 150 countries. Along with ACM (Association for Computing Machinery), it is considered one of the two major computer societies worldwide. The parent association, IEEE, was established in the United States in 1962 as an association of electrical and electronics engineers, with about 500,000 engineering and technology professionals affiliated.


On September 1, HyperExcel was honored with an award in the AI core technology and chip category at the 'IC Taiwan Grand Challenge' hosted by Taiwan's National Science and Technology Council. The IC Taiwan Grand Challenge selects outstanding companies in AI core technology and chips, smart mobility, manufacturing, healthcare, and sustainability. As Taiwan emerges as the base for global AI semiconductors, attention is focused on the AI core technology and chip category. HyperExcel, along with DeepMentor and femtoAI, received the award, with the committee citing HyperExcel's focus on semiconductor cost efficiency for LLM data processing as the reason for the award.

The high evaluation of HyperExcel's semiconductors by the Taiwanese government is significant. This is because TSMC is already considered a major player in the global semiconductor industry, and NVIDIA, which holds over 90% of the current AI semiconductor market share, is establishing a second headquarters in Taiwan and using it as a semiconductor base. This achievement comes amid the influx of AI semiconductor companies, such as DeepX, establishing branches in Taiwan.

LPU Listed on AWS Cloud, Accelerating Entry into the US Market


In terms of commercialization, a step has been taken forward. HyperExcel recently became the first domestic AI accelerator company to start LPU commercial services through Amazon Web Services' F2 instance. By renting FPGA-based LPU on the AWS Marketplace, machine learning solutions, natural language processing, and generation-text processing tasks can be performed.

AI models operable in the initial version include ▲ Meta Llama 3.1-8B Instruct, 3.2-1B, and 3B Instruct ▲ Naver HyperCLOVA X-SEED-TEXT-Instruct-0.5B and 1.5B ▲ LG AI Research Institute ExaOne-3.5-2.4B and 7.8B Instruct models. With AWS adding domestic AI accelerators as commercial services, various domestic and international companies can directly build LPU-based services with HyperExcel FPGA.

HyperExcel plans to attend Supercomputing 2025 (SC 25), held in St. Louis, Missouri, USA, from November 16 to 21 this year, to introduce LPU-based technology to the US market.

2026, Global AI Hardware Company Competition Intensifies

The core of the AI accelerator market in 2023 was the training tasks for building AI models. However, as GPU prices are too high and supply is difficult, the perception that inference tasks for operating models should use AI accelerators instead of GPUs has taken hold. Consequently, domestic and international AI semiconductor companies are targeting the inference field market suitable for their purposes, rather than directly competing with NVIDIA, and HyperExcel is among them.


However, even if companies target only their respective fields, the number of players is quite large. From 2025 onwards, competition is expected to intensify further. NVIDIA is set to ship the next-generation GPX based on the Rubin architecture in the second half of next year. US-based Cerebras is starting commercial operation of the wafer-scale engine-3, a supercomputer-grade semiconductor, and Tenstorrent, led by Jim Keller, is likely to unveil new chipsets such as Quasar and Grendel next year. FuriosaAI's RNGD is seeing a significant increase in adoption cases, and Rebellions' Rebell Quad is also seeing an increase in main contracts.

HyperExcel's Bertha must also possess strong competitiveness to meet market expectations. Fortunately, unlike other products, HyperExcel focuses on LLM inference and operates in a relatively low-power environment using LPDDR memory. It is clearly differentiating itself and targeting a niche market that existing high-performance semiconductors cannot reach. This year, HyperExcel has clearly demonstrated its presence both in academia and the AI semiconductor industry. By 2026, its narrative will begin in earnest.

IT Donga Reporter Nam Si-hyun (sh@itdonga.com)

Popular News