A Deep Dive into ChatGPT’s Tech-Stack

Portrait of a young man smiling, wearing a dark grey shirt with a mandarin collar, against a white background with a dark blue brushstroke behind him.
Asharib Ali
An image featuring the text: "Scaling ChatGPT: How OpenAI's Cloud Native AI Manages Millions of Users?", "CLOUD NATIVE ARTIFICIAL INTELLIGENCE", and "Cloud Computing, AI".
Technology
Scaling ChatGPT: How OpenAI’s Cloud Native AI Manages Millions of Users?

Have you ever thought:

  • How ChatGPT can respond so quickly to your queries?
  • What makes it possible for a single platform to manage the interactions of millions of users simultaneously?
  • The complex technology that operates seamlessly behind the scenes at OpenAI?
  • How does the integration of frontend and backend technologies contribute to the robust performance of ChatGPT?
  • What exactly goes on behind the scenes when you prompt ChatGPT and receive an instant response?

Are you curious about what “Cloud Native Artificial Intelligence” means and how it’s applied?

The answers to all these questions boil down to one key concept: “Cloud Native Artificial Intelligence.” 👀

This technology is pivotal in enabling OpenAI to scale ChatGPT, making it one of the most rapidly growing and widely used platforms on the internet.

In this article, I will dive deep into the frontend, backend, and infrastructure that power ChatGPT.

You’ll gain insights into the technical workings and how these elements come together to create a seamless user experience.

Before I start going to deep dive, it is important to understand:

Who is OpenAI?

OpenAI is an AI research and deployment company. Their mission is to ensure that artificial general intelligence benefits all of humanity.

Five key facts about OpenAI:

  1. Foundation: Established in 2015 as a nonprofit by Elon Musk, Sam Altman, and others​ (Wikipedia)​.
  2. Profit Model: Transitioned to a capped-profit model in 2019 to raise more capital​ (OpenAI)​.
  3. Microsoft Partnership: Received significant funding from Microsoft, totaling over $11 billion​ (Forbes Africa)​.
  4. Innovative Products: Developed AI technologies like ChatGPT, DALL-E, Codex, and Whisper​ (Encyclopedia Britannica)​.
  5. High Valuation: Valued at $80 billion as of 2023, among the highest in tech startups​ (Forbes Africa)​.

These points provide a quick overview of OpenAI’s core characteristics and milestones, but now the question is how do they grow insanely?

The answer is simple it’s (ChatGPT) 🤖👈

ChatGPT is a free-to-use and (Pro) AI system. Use it for engaging in conversations, gaining insights, automating tasks, and witnessing the future of AI all in one place, first launched on November 30 2022, and ChatGPT is a sibling model to InstructGPT by OpenAI.

It is worth mentioning that ChatGPT has become an all-pervasive platform used by people from different backgrounds and professions, including those in the IT industry. Even Kids have also found different ways to utilize it, making it a versatile platform for people of all ages.

Now the question is, if it’s a pervasive platform used by millions of people, what’s actually happening behind the scenes? (Which technologies are used for frontend and backend to build the Ultimate AI platform, and of course, the infrastructure that is handling all the stuff for us in a seamless way), Now let’s talk about it.

The Frontend of ChatGPT

  1. Programming Language: They use TypeScript and Nodejs to handle frontend tasks.
  2. Library and Frameworks: They utilize React library (Component-Based Architecture, which helps manage dynamic UI components efficiently) and React Router (Modern & full stack Routing Solution) with Emotion (CSS in JS) to build a responsive user interface/user experience.
  • Before React and React Router, the ChatGPT web used Remix and before it, Next.js as their framework. However, they shifted their frontend tech to simple React and React Router (likely for better server-side rendering and client-side routing), Read the following article to learn more about Remix vs Next.js.
  1. UI Frameworks: They use TailwindCSS, and HeadlessUI, for styling purposes.
  2. Global CDN Usage: Uses Cloudflare to ensure low-latency, fast access worldwide, and Incorporates Cloudflare Bot Management and HTTP Strict Transport Security (HSTS) to protect against attacks.
  3. Performance Tools: Employs Webpack for optimization and HTTP/3 for faster asset delivery.
  4. Data Handling Libraries: Uses Lodash and core-js for simplified complex data operations with the latest ECMAScript features.
  5. Analytics Integration: Tools like Segment, Datadog, and Google Analytics are used to monitor and improve user interactions.

The Backend of ChatGPT

  1. Programming Languages: Primarily uses Python for backend development.
  2. Machine Learning Libraries: Utilizes TensorFlow and PyTorch, which are essential for AI model training, these libraries help with auto-differentiation and GPU acceleration, crucial for handling large datasets.
  3. Model Portability: Employs ONNX to ensure the model works across different software platforms.
  4. Optimization Library: Uses DeepSpeed by Microsoft to enhance training efficiency on distributed systems, DeepSpeed helps distribute the workload across multiple GPUs, speeding up the processing.
  5. Integration of Technologies: The backend integrates various advanced technologies to maintain and enhance the AI’s capabilities.

The Infrastructure of ChatGPT

Now, regarding the topic I was discussing — Cloud Native Artificial Intelligence — that handles all of these tasks for us, let me answer all the questions I initially asked.

First, what does “infrastructure” mean? Infra- means “below,” so infrastructure refers to the “underlying structure” of a company and its products. It is what enables a company like OpenAI to build and run the applications that underpin its business, including compute, network, workplace, and data platform capabilities.

Now, the question is, what happens behind the scenes when we use ChatGPT, and how does it handle millions of users simultaneously? Let’s discuss this in detail and explore the role of Cloud Native Artificial Intelligence here.

As I’ve already highlighted in the key points about OpenAI, Microsoft has made significant investments in the company. You are probably familiar with Azure, Microsoft’s cloud computing platform.

In simple terms, cloud computing involves delivering various computing services — including servers, storage, databases, networking, software, analytics, and intelligence — over the Internet (“the cloud”) to foster faster innovation, flexible resources, and economies of scale.

Initially, Microsoft invested $1 billion in 2019, and following the massive success of ChatGPT, they injected over $10 billion into OpenAI in 2023 to enhance their “infrastructure.”

According to the first launch blog post by OpenAI about ChatGPT, “ChatGPT and GPT-3.5 were trained on Azure’s AI supercomputing infrastructure” which supports much of the cloud computing work for deploying and maintaining OpenAI products due to their strong partnership and funding.

Now, I believe you understand what Cloud Native means. So, what does AI with Cloud entail? Essentially, building an AI system requires numerous components like data collection, machine learning models, development frameworks and tools, and natural language processing (NLP). Of course, “infrastructure” is also necessary for deployment and maintenance, which means we need both AI to build AI applications and infrastructure to manage these applications smoothly.

I’m damn sure that you have a clear understanding of what 'Cloud Native Artificial Intelligence' means, let’s talk about:

What do they use behind the scenes?

ChatGPT operates on an infrastructure built to handle the vast computing needs of large language models. OpenAI collaborates closely with Microsoft, utilizing Azure’s extensive data center capabilities.

This partnership leverages a supercomputer that integrates more than 285,000 CPU cores and 10,000 NVIDIA V100 GPUs, making it one of the largest in the public cloud domain as of the 20s.

This supercomputer is essential for training ChatGPT, employing data parallelism techniques where multiple instances of the model are trained simultaneously across these GPUs, enhancing learning efficiency and model performance​ (TECHCOMMUNITY.MICROSOFT.COM)​ 🎁

Hey! I’m glad you find this topic interesting. If you want to stay updated with the latest cutting-edge, state-of-the-art technologies and content, make sure to follow me on my social handles.

Twitter: https://twitter.com/0xAsharib
Linkedin: https://www.linkedin.com/in/asharibali/
Github: https://github.com/AsharibAli
Website: https://www.asharib.xyz/

Thanks for your time!