TF01 : Understanding Neural Networks using TensorFlow

Objective:

• The goal is to develop a machine-learning model capable of converting Celsius to Fahrenheit. This model will be trained using a dataset containing pairs of Celsius and Fahrenheit values. The conversion will be based on the formula  F=C×1.8+32
• . After training, the model’s performance will be evaluated by comparing its predictions with the results obtained from the formula.

Below are the important steps for building a Neural network model

1. Import dependencies  - Tensorflow , numpy , etc
2. Setup Training data - 
1. Features - Input
2. Labels - output
3. Examples - Input/output pairs for model training.
3. Create a neural network model
1. Build a layer
2. Assemble layers into the model
3. Compile the model with loss and optimization functions
4. Train the model - 
1. The training model is about tuning the internal variables of the networks to the best possible values so that they can map the input to the output
5. Use the model to predict the weights

Example Code which you can paste in colab and execute : 

#import Dependencies

import tensorflow as tf

import numpy as np

import logging

logger = tf.get_logger()

logger.setLevel(logging.ERROR)

#Setup Training data

celsius_q    = np.array([-40, -10,  0,  8, 15, 22,  38],  dtype=float)

fahrenheit_a = np.array([-40,  14, 32, 46, 59, 72, 100],  dtype=float)

#Create a model (here we have three models)

#input_shape=[1] — This specifies that the input to this layer is a single value

#units=1 — This specifies the number of neurons in the layer.

#The number of neurons defines how many internal variables the layer

#has to try to learn how to solve the problem

l0 = tf.keras.layers.Dense(units=4, input_shape=[1])

l1 = tf.keras.layers.Dense(units=4)

l2 = tf.keras.layers.Dense(units=1)

#Assembly layers to model

model = tf.keras.Sequential([l0, l1, l2])

#Compile the model, with loss and optimizer functions

#Loss function — A way of measuring how far off predictions

#are from the desired outcome.

#(The measured difference is called the "loss".)

#Optimizer function — A way of adjusting internal values in order to reduce the loss.

model.compile(loss='mean_squared_error', optimizer=tf.keras.optimizers.Adam(0.1))

#Train the model

model.fit(celsius_q, fahrenheit_a, epochs=500, verbose=False)

#Prediction

print("Finished training the model")

print(model.predict([100.0]))

#here we can check the predicted output with output generated from formula

#looking into layered weights

print("Model predicts that 100 degrees Celsius is: {} degrees Fahrenheit".format(model.predict([100.0])))

print("These are the l0 variables: {}".format(l0.get_weights()))

print("These are the l1 variables: {}".format(l1.get_weights()))

print("These are the l2 variables: {}".format(l2.get_weights()))

Understing Dense/Full connected neural networks:

 The model is developed using a dense neural network, which is a type of neural network composed of multiple layers. In a dense neural network, each neuron in a layer is connected to all neurons in the preceding layer, creating a fully interconnected structure.

Please learn following terms:

• Feature: The input(s) to our model
• Examples: An input/output pair used for training
• Labels: The output of the model
• Layer: A collection of nodes connected together within a neural network.
• Model: The representation of your neural network
• Dense and Fully Connected (FC): Each node in one layer is connected to each node in the previous layer.
• Weights and biases: The internal variables of model
• Loss: The discrepancy between the desired output and the actual output
• MSE: Mean squared error, a type of loss function that counts a small number of large discrepancies as worse than a large number of small ones.
• Gradient Descent: An algorithm that changes the internal variables a bit at a time to gradually reduce the loss function.
• Optimizer: A specific implementation of the gradient descent algorithm. (There are many algorithms for this. In this course we will only use the “Adam” Optimizer, which stands for ADAptive with Momentum. It is considered the best-practice optimizer.)
• Learning rate: The “step size” for loss improvement during gradient descent.
• Batch: The set of examples used during training of the neural network
• Epoch: A full pass over the entire training dataset
• Forward pass: The computation of output values from input
• Backward pass (backpropagation): The calculation of internal variable adjustments according to the optimizer algorithm, starting from the output layer and working back through each layer to the input.

References

Tensorflow framework learing plan

Udacity Tensorflow course

understanding more on model training