Nvdia GCT 2024

[ninjaghost]

For people who lazy to finish the vid, I provided a summary below that done by AI

## Nvidia's Journey and the Emergence of Generative AI
- Nvidia founder and CEO Jensen Huang highlighted the company's journey and the rise of a new industry based on generative AI.
- Accelerated computing, leveraging GPUs for faster processing, has reached a tipping point and is transforming various industries.

## Partnerships for Ecosystem Acceleration
- Nvidia announced partnerships with leading companies in engineering simulation, chip design, and EDA software to accelerate their ecosystems using CUDA and connect them to the Omniverse digital twin platform.
- The goal is to enable the creation of products entirely in the digital realm, driving up the scale of computing and enabling the application of generative AI in these industries.
- Partnerships with Ansys, Synopsys, and Cadence will accelerate computational lithography, chip design, and fluid dynamic simulation, ultimately leading to the integration of AI co-pilots and the connection of digital twin platforms.

## Addressing the Computational Demands of Large Language Models
- The speaker discussed the rapid growth of large language models and the computational requirements they demand.
- The latest OpenAI model has 1.8 trillion parameters and requires several trillion tokens for training, resulting in an enormous computational scale.
- To address this challenge, NVIDIA has been building increasingly powerful supercomputers, with the latest being EOS, one of the largest AI supercomputers in the world.

## Introducing the Blackwell Platform
- The speaker emphasized the need for even larger models and introduced the Blackwell platform, named after mathematician and game theorist David Blackwell.
- Blackwell is not just a chip but a platform that includes the most advanced GPU in the world, Hopper, and a new Transformer engine that can dynamically rescale and recast numerical formats to lower precision.
- NVIDIA is also working on synthetic data generation, reinforcement learning, and AI-on-AI training to further enhance the capabilities of large language models.
- The Blackwell platform comes in two types of systems: one that is form-fit-function compatible with Hopper, allowing for easy upgrades, and another that features a fully functioning board with two Blackwell chips and four Blackwell dies connected to a Grace CPU.
- The Blackwell platform represents a significant leap in computational power and opens up new possibilities for large language models and AI research.
## Blackwell Architecture and Features
- NVIDIA introduced the Blackwell architecture, which includes a smaller ALU and a fifth-generation MV link that is twice as fast as Hopper.
- The new Transformer engine has computation in the network, allowing for faster synchronization and collectives between GPUs.
- A Ras engine performs 100% self-test and system test of every single gate and bit of memory on the Blackwell chip, ensuring reliability.
- Secure AI capabilities include data encryption at rest, in transit, and during computation.
- A high-speed compression engine moves data 20 times faster in and out of the computers.
- Compared to Hopper, Blackwell offers 2.5 times the FP8 performance for training per chip and introduces the FP6 format, which doubles the throughput for inference.
- NVIDIA plans to scale up the GPU even further with the Envy link switch, which has 50 billion transistors and four MV links, each with 1.8 terabytes per second of bandwidth.
- The Envy link switch allows every GPU to talk to every other GPU at full speed simultaneously.

## The New DGX System and Performance
- The new DGX system based on Blackwell architecture delivers 720 petaflops of training performance and is the world's first one-exaflop AI system in a single rack.
- The DGX MV link spine provides 130 terabytes per second of bandwidth, more than the aggregate bandwidth of the internet, using 5,000 MVlink cables without the need for transceivers or retimers.
- The speaker introduced the Blackwell DGX system, a powerful AI supercomputer designed for trillion-parameter generative AI.
- Blackwell is liquid-cooled and has a power consumption of 4 megawatts, significantly lower than previous systems.
- Training a GPT model with 1.8 trillion parameters would take 90 days with 2,000 GPUs on Blackwell, compared to 8,000 GPUs on Hopper and 15 megawatts of power.

## Inference Capabilities and Future Data Centers
- Inference, or generation, is becoming increasingly important for large language models, which require high throughput and interactivity.
- The speaker discussed the challenges of distributing work across multiple GPUs to achieve both throughput and interactivity.
- Blackwell's inference capability for large language models is 30 times faster than Hopper.
- The MVLink switch in Blackwell enables faster communication between GPUs.
- Data centers of the future will be thought of as AI factories, generating intelligence rather than electricity.

## Customer Demand and Product Launch Preparations
- There is significant excitement and demand for Blackwell from customers.
- Blackwell comes in various configurations, including liquid-cooled and extreme versions.
- NVIDIA is preparing for the launch of Blackwell, which is expected to be the most successful product launch in the company's history.
- AWS, Google, Oracle, and Microsoft are all gearing up for Blackwell and working with NVIDIA to accelerate various aspects of their services using NVIDIA GPUs and technologies.

## Omniverse Digital Twins and Generative AI Applications
- NVIDIA's global ecosystem of partners, including Wistron, is using Omniverse digital twins to optimize manufacturing processes and reduce costs.
- The generative AI revolution is here, and NVIDIA is using it to understand and generate various forms of data, including text, images, videos, sounds, proteins, genes, brain waves, and climate.
- NVIDIA is creating Earth-2, a digital twin of the Earth, to predict weather at extremely high resolution using a new generative AI model called CORDI.
- CORDI can super-resolve extreme weather events from 25 km to 2 km resolution with 1,000 times the speed and 3,000 times the energy efficiency of conventional weather models.
- NVIDIA is collaborating with The Weather Company to accelerate their weather simulation and integrate Earth-2 with CORDI to provide businesses with more accurate weather predictions.
- Nvidia is involved in regional high-resolution weather prediction and healthcare, particularly in medical imaging, gene sequencing, and computational chemistry.
- Nvidia announced the development of AI models that understand the "language of life," enabling the reconstruction of proteins and facilitating virtual screening for new medicines.

## Nvidia Inference Microservices (Nims)
- Nvidia introduced the concept of "Nims" (Nvidia Inference Microservices), which are pre-trained, optimized, and packaged AI models that can be easily integrated into workflows and accessed through simple APIs.
- Nims can be used to assemble complex software systems by combining various AI models and services, similar to how chatbots might be assembled in the future.
- Nvidia utilizes Nims internally, including a chatbot co-designer that assists chip designers in creating CTL (Compute Trace Library) programs.
- Nvidia provides tools and services to help customize, fine-tune, and evaluate Nims, making them easier to integrate into existing systems.

## Nvidia AI Foundry Platform
- Nvidia's approach to AI involves inventing AI technology, creating tools for modification, and providing infrastructure for fine-tuning and deployment.
- Nvidia AI Foundry is an AI platform that allows companies to build and deploy AI models on their own data.
- Nvidia AI Foundry offers three main pillars: Nemo Microservice, DGX Cloud, and Nims.
- Nemo Microservice helps create AI models that can understand and respond to natural language queries.
- DGX Cloud is a cloud-based platform that provides the infrastructure and tools needed to train and deploy AI models.
- Nims are pre-trained AI models that can be used for various tasks, such as image recognition, natural language processing, and speech recognition.
- Nvidia AI Foundry is working with several major companies, including SAP, ServiceNow, Cohesity, Snowflake, and NetApp, to help them build AI-powered applications.
- Nvidia AI Foundry is also working with Dell to build AI factories, which are end-to-end systems for building and deploying AI models at scale.

## The Future of AI Robotics
- The next wave of AI robotics will require three computers: an AI computer, an autonomous systems computer, and a simulation engine.
- Nvidia has developed the DGX system for training AI, the AGX system for autonomous systems, and the OVX system for running the Omniverse simulation engine.
- Nvidia is showcasing a robotics building called a warehouse, where autonomous systems, including humans and forklifts, interact with each other.
- NVIDIA Omniverse is a platform that enables the creation of virtual worlds and simulations.
- It can be used to test and refine AI systems before they are deployed in the real world.
- NVIDIA and Siemens are partnering to connect Siemens' industrial metaverse to NVIDIA Omniverse.
- This will allow companies to use Omniverse to design, manufacture, and operate industrial facilities.
- NVIDIA is also announcing a new robotics SDK called Isaac Perceptor, which will enable robots to perceive their surroundings and navigate autonomously.
- NVIDIA is working with BYD, the world's largest EV company, to adopt its next-generation AV computer, Thor.
- NVIDIA Jetson, a robotics computer, is 100% CUDA compatible, making it compatible with everything NVIDIA offers.
- Isaac Perceptor has state-of-the-art vision odometry, 3D reconstruction, and depth perception for better environment awareness.
- Isaac Manipulator is a Cuda-accelerated motion planner that can adapt to obstacles and has excellent perception for pose estimation of 3D objects.
- The next generation of robotics will likely be humanoid robotics due to the availability of necessary technology and imitation training data.
- Project Groot is a general-purpose foundation model