CS331 Neural Computing
CS33115 Neural Computing
Introductory description
This module provides an introduction to the theory and implementation of neural networks and an understanding of the important computational neural network architecture and methodology. It aims to give students sufficient knowledge to enable employment or postgraduate study involving neural networks.
Module aims
This module provides an introduction to the theory and implementation of neural networks, both biological and artificial. It aims to give students sufficient knowledge to enable employment or postgraduate study involving neural networks.
Outline syllabus
This is an indicative module outline only to give an indication of the sort of topics that may be covered. Actual sessions held may differ.

Introduction: What Is a Neural Network; AI and Neural Computing; Three Types of Learning Paradigms; Transfer Learning; PreStage: Making Your Own Neural Network from Scratch: The Tools You Will Need; Demo: Creating a Neural Network for Live Image Classification

Biological Neural Networks: Biological Neuron Structure and Function; Action Potential; Synaptic Transmission; A Brief History of Neural Nets & Deep Learning; Comparison between Biological and Artificial Neural Networks; Basic Principles of Artificial Neural Networks; Different Types of Neural Networks;

SingleLayer Perceptron: McCullochPitts Neuron; Definition of Perceptron; Weights, Bias, Node, Activation Functions; Perceptron Case Study: Digital Number Recognition; Mimic Logic Operations (AND, OR, NOT); XOR Linear Inseparability Problem; Perceptron Learning Rule; Demo: Implementing MP Neuron using MATLAB

MultiLayer Forward Propagation: Representation of MultiLayer Neural Networks; Predicting MultiLayer Neural Network Output; Common Nonlinear Activation Functions; How to Choose an Activation Function for Your Model; Forward Propagation for Deep Neural Networks; Vectorisation; Demo: Recognise SevenSegment Numerals
Using Perceptron 
Loss Function & Gradient Descent: Different Loss Functions (e.g., MAE, MSE, Cross Entropy Loss); 1D & 2D Forms of Gradient Descent Methods; Variations of Gradient Descent (e.g., Momentum, RMSProp, Adam); Saddle Points; Gradient Descent vs. Newton Method; Stochastic & MiniBatch Gradient Descent;

Backpropagation: Computation Graph; Matrix Calculus Revisited; Backpropagation in Logistic Regression; Backpropagation in MultiLayered NNs; Vanishing Gradient Problems

Overfitting & Regularization: Bias and Variance; What is Regularization; How does Regularization help in Reducing Overfitting; Different Regularization Techniques (e.g. Lasso & Ridge Regularization, Dropout, Data Augmentation, Early Stopping)

Recurrent Neural Network: Sequence Model; Onehot Representation; Forward and Backward Propagations; Different Types of Recurrent Neural Networks; Elman and Jordan Networks; Bidirectional Recurrent Neural Network; Vanishing & Exploding Gradient Problems; Gated Recurrent Units (GRU); Long ShortTerm Memory Networks (LSTM); Time Series Analysis (optional)

Graph Neural Networks (GNNs): Basics of Graph Mining; Google PageRank; Pairwise Similarity Model (e.g., SimRank, PenetratingRank, RoleSim); Convolutional Neural Network; Graph Embedding; Node2vec (optional); GNN Applications in NLP (optional);
Learning outcomes
By the end of the module, students should be able to:
 Students completing the module should be able to demonstrate: an understanding of the principles of Neural Networks and a knowledge of their main areas of application;
 The ability to design, implement and analyse the behaviour of simple neural networks.
Indicative reading list

Michael Taylor. Make Your Own Neural Network: An InDepth Visual Introduction for Beginners. Amazon Digital Services LLC  Kdp Print Us, 2017, ISBN: 1549869132, 9781549869136

Graupe Daniel. Principles of Artificial Neural Networks: Basic Designs to Deep Learning (4th Edition). World Scientific, 2019, ISBN: 9811201242, 9789811201240

James Loy. Neural Network Projects with Python: The ultimate guide to using Python to explore the true power of neural networks through six projects, Packt Publishing Ltd, 2019, ISBN: 1789133319, 9781789133318
Subject specific skills
 Understand the basic concepts of biological and artificial neural networks in computer science.
 Apprehend different types of artificial neural networks (e.g. MP neuron, multilayer perceptron, CNN, RNN, GNN) and their real applications.
 Design and implement simple neural networks and deep learning algorithms.
Transferable skills
 Mathematical analysis of neural computing methods.
 Evaluation of neural network algorithms.
 Programming skills in MATLAB/Python.
Study time
Type  Required 

Lectures  22 sessions of 1 hour (15%) 
Seminars  2 sessions of 1 hour (1%) 
Practical classes  6 sessions of 1 hour (4%) 
Private study  120 hours (80%) 
Total  150 hours 
Private study description
Private study, background reading and revision
Costs
No further costs have been identified for this module.
You do not need to pass all assessment components to pass the module.
Students can register for this module without taking any assessment.
Assessment group D3
Weighting  Study time  

assignment  20%  
Inperson Examination  80%  
CS331 Examination

Assessment group R2
Weighting  Study time  

Oncampus Examination  Resit  100%  
CS331 resit Examination ~Platforms  AEP

Feedback on assessment
Individual written feedback on coursework
Courses
This module is Optional for:
 Year 3 of UCSAG4G1 Undergraduate Discrete Mathematics
 Year 3 of UCSAG4G3 Undergraduate Discrete Mathematics
 Year 4 of UCSAG4G4 Undergraduate Discrete Mathematics (with Intercalated Year)
 Year 4 of UCSAG4G2 Undergraduate Discrete Mathematics with Intercalated Year

USTAG1G3 Undergraduate Mathematics and Statistics (BSc MMathStat)
 Year 3 of G1G3 Mathematics and Statistics (BSc MMathStat)
 Year 4 of G1G3 Mathematics and Statistics (BSc MMathStat)

USTAG1G4 Undergraduate Mathematics and Statistics (BSc MMathStat) (with Intercalated Year)
 Year 4 of G1G4 Mathematics and Statistics (BSc MMathStat) (with Intercalated Year)
 Year 5 of G1G4 Mathematics and Statistics (BSc MMathStat) (with Intercalated Year)
This module is Option list A for:
 Year 4 of UCSAG504 MEng Computer Science (with intercalated year)
 Year 3 of UCSAG500 Undergraduate Computer Science
 Year 4 of UCSAG502 Undergraduate Computer Science (with Intercalated Year)
 Year 3 of UCSAG503 Undergraduate Computer Science MEng
 Year 3 of USTAG302 Undergraduate Data Science
 Year 3 of USTAG304 Undergraduate Data Science (MSci)
 Year 4 of USTAG303 Undergraduate Data Science (with Intercalated Year)
This module is Option list B for:
 Year 3 of UCSAG406 Undergraduate Computer Systems Engineering
 Year 3 of UCSAG408 Undergraduate Computer Systems Engineering
 Year 4 of UCSAG407 Undergraduate Computer Systems Engineering (with Intercalated Year)
 Year 4 of UCSAG409 Undergraduate Computer Systems Engineering (with Intercalated Year)
 Year 3 of UCSAGN51 Undergraduate Computer and Business Studies
 Year 4 of UCSAGN5A Undergraduate Computer and Business Studies (with Intercalated Year)
 Year 3 of USTAGG14 Undergraduate Mathematics and Statistics (BSc)
 Year 4 of USTAGG17 Undergraduate Mathematics and Statistics (with Intercalated Year)