AI Frameworks and Development Basics

What it is

An AI framework (deep learning framework) is a software library that lets you build, train and deploy neural networks without writing the math and hardware code yourself. You describe the model in a few lines, and the framework handles tensor storage, gradients and the GPU or NPU.

Without a framework, every layer, every derivative and every device kernel would have to be written by hand. Frameworks make deep learning fast to develop and easy to reuse.

How it works

What a framework provides

Mainstream frameworks

All of them follow the same basic ideas, so knowing one makes the others easy to learn.

Layers of a framework

A framework is usually organized in layers.

The front end is the Python API you call. The graph layer represents and optimizes the computation. The operator layer holds kernels, the hardware specific code for each operation such as matrix multiplication or convolution.

At the bottom are hardware libraries. NVIDIA GPUs use CUDA, and Huawei Ascend NPUs use CANN (Compute Architecture for Neural Networks), Huawei's heterogeneous computing architecture for Ascend.

Tensors

A tensor is the basic data structure: a multidimensional array with a shape and a data type (dtype). The number of dimensions is its rank, or order.

PyTorch stores image batches as NCHW: batch size, channels, height, width. So 32 RGB images of 224 x 224 pixels form a tensor of shape (32, 3, 224, 224). The default floating point dtype is float32.

Computational graphs

A computational graph is a directed graph whose nodes are operations and whose edges carry tensors. The framework uses it to run the forward pass and to work out gradients in the backward pass.

PyTorch is dynamic by default; torch.compile in PyTorch 2 can compile code into optimized graphs. TensorFlow 2 runs eagerly but turns functions into graphs with tf.function. MindSpore offers both a graph mode and a PyNative (dynamic) mode.

Automatic differentiation

Training needs the gradient of the loss with respect to every weight. Automatic differentiation (autograd) records each operation in the forward pass, then applies the chain rule backward through the graph. This is reverse mode differentiation, which is what backpropagation is.

A worked example

Take y = x squared + 3x at x = 2. By hand, dy/dx = 2x + 3 = 2(2) + 3 = 7. PyTorch gets the same result automatically:

import torch
x = torch.tensor(2.0, requires_grad=True)
y = x**2 + 3*x      # forward pass builds the graph
y.backward()        # reverse mode autograd
print(x.grad)       # tensor(7.)

requires_grad=True tells PyTorch to track operations on x. After backward(), the gradient is stored in x.grad.

Moving work to a GPU is a single call such as x.to("cuda"). The model code does not change.

How the exam asks about this

• "Which of the following is NOT a main function of an AI framework?" → Anything outside tensors, autograd, model building, acceleration, distributed training and deployment, such as collecting the raw business data.
• "Which framework is developed by Huawei and supports device, edge and cloud scenarios?" → MindSpore.
• "PyTorch builds its computational graph while the code runs. True or false?" → True; it uses a dynamic graph.