课程概况
在本门课程中，您将学习图像和视频处理的基本概念，以及如何用这些概念解决商业或科学中的实际问题。
数字图像和视频现已无处不在：在科技（例如航天、生物医学等领域）、消费、工业、艺术等领域都有应用；而且已经涵盖了电磁频谱中很大一部分：从可见光、红外线到伽马射线，甚至更远。因此，处理图像和视频信号的能力对于工科和理科学生、程序员、科学家来说已经成为了必须掌握的重要技能。数字图像和视频处理使我们正在经历的多媒体技术革命成为可能。几个重要的图像和视频处理的例子：图像劣化的去除（例如将快速移动的汽车的照片去模糊），为了节约存储和高校传输对图像和视频进行的压缩和变换（如果你观看网络视频，或者用社交网络分享图片，那么你就已经用到了这些技术）。
本课程将涵盖图像和视频处理的基础部分。我们会介绍描述和分析2D/3D图像和视频的数学框架，包括空间域、时空域、频域。在本课程中，您将学到基础图像和视频处理方法背后的理论，包括图像/视频增强、回复、压缩，您也将学到如何使用最新工具将这些方法用于实际的任务。从优化工具到统计方法，我们会介绍并使用很多这样的工具。我们也会介绍在现代图像和视频处理中扮演重要角色的“稀疏性”。在所有案例中，特定应用下的示例图像和视频都会被用到。
In this class you will learn the basic principles and tools used to process images and videos, and how to apply them in solving practical problems of commercial and scientific interests.
Digital images and videos are everywhere these days – in thousands of scientific (e.g., astronomical, bio-medical), consumer, industrial, and artistic applications. Moreover they come in a wide range of the electromagnetic spectrum – from visible light and infrared to gamma rays and beyond. The ability to process image and video signals is therefore an incredibly important skill to master for engineering/science students, software developers, and practicing scientists. Digital image and video processing continues to enable the multimedia technology revolution we are experiencing today. Some important examples of image and video processing include the removal of degradations images suffer during acquisition (e.g., removing blur from a picture of a fast moving car), and the compression and transmission of images and videos (if you watch videos online, or share photos via a social media website, you use this everyday!), for economical storage and efficient transmission.
This course will cover the fundamentals of image and video processing. We will provide a mathematical framework to describe and analyze images and videos as two- and three-dimensional signals in the spatial, spatio-temporal, and frequency domains. In this class not only will you learn the theory behind fundamental processing tasks including image/video enhancement, recovery, and compression – but you will also learn how to perform these key processing tasks in practice using state-of-the-art techniques and tools. We will introduce and use a wide variety of such tools – from optimization toolboxes to statistical techniques. Emphasis on the special role sparsity plays in modern image and video processing will also be given. In all cases, example images and videos pertaining to specific application domains will be utilized.
课程大纲
周1
完成时间为 2 小时
Introduction to Image and Video Processing
In this module we look at images and videos as 2-dimensional (2D) and 3-dimensional (3D) signals, and discuss their analog/digital dichotomy. We will also see how the characteristics of an image changes depending on its placement over the electromagnetic spectrum, and how this knowledge can be leveraged in several applications.
3 个视频 （总计 67 分钟）, 5 个阅读材料, 1 个测验
周2
完成时间为 2 小时
Signals and Systems
In this module we introduce the fundamentals of 2D signals and systems. Topics include complex exponential signals, linear space-invariant
systems, 2D convolution, and filtering in the spatial domain.
5 个视频 （总计 82 分钟）, 4 个阅读材料, 1 个测验
周3
完成时间为 2 小时
Fourier Transform and Sampling
In this module we look at 2D signals in the frequency domain. Topics include: 2D Fourier transform, sampling, discrete Fourier transform, and
filtering in the frequency domain.
5 个视频 （总计 92 分钟）, 2 个阅读材料, 1 个测验
周4
完成时间为 3 小时
Motion Estimation
In this module we cover two important topics, motion estimation and color representation and processing. Topics include: applications of
motion estimation, phase correlation, block matching, spatio-temporal gradient methods, and fundamentals of color image processing
5 个视频 （总计 119 分钟）, 2 个阅读材料, 1 个测验
周5
完成时间为 3 小时
Image Enhancement
In this module we cover the important topic of image and video enhancement, i.e., the problem of improving the appearance or usefulness of an image or video. Topics include: point-wise intensity transformation, histogram processing, linear and non-linear noise smoothing, sharpening, homomorphic filtering, pseudo-coloring, and video enhancement.
9 个视频 （总计 170 分钟）, 2 个阅读材料, 1 个测验
周6
完成时间为 3 小时
Image Recovery: Part 1
In this module we study the problem of image and video recovery. Topics include: introduction to image and video recovery, image restoration, matrix-vector notation for images, inverse filtering, constrained least squares (CLS), set-theoretic restoration approaches, iterative restoration algorithms, and spatially adaptive algorithms.
9 个视频 （总计 168 分钟）, 2 个阅读材料, 1 个测验
周7
完成时间为 2 小时
Image Recovery : Part 2
In this module we look at the problem of image and video recovery from a stochastic perspective. Topics include: Wiener restoration filter,
Wiener noise smoothing filter, maximum likelihood and maximum a posteriori estimation, and Bayesian restoration algorithms.
6 个视频 （总计 107 分钟）, 2 个阅读材料, 1 个测验
周8
完成时间为 3 小时
Lossless Compression
In this module we introduce the problem of image and video compression with a focus on lossless compression. Topics include: elements of
information theory, Huffman coding, run-length coding and fax, arithmetic coding, dictionary techniques, and predictive coding.
8 个视频 （总计 155 分钟）, 2 个阅读材料, 1 个测验
周9
完成时间为 3 小时
Image Compression
In this module we cover fundamental approaches towards lossy image compression. Topics include: scalar and vector quantization, differential
pulse-code modulation, fractal image compression, transform coding, JPEG, and subband image compression.
7 个视频 （总计 146 分钟）, 2 个阅读材料, 1 个测验
周10
完成时间为 3 小时
Video Compression
In this module we discus video compression with an emphasis on motion-compensated hybrid video encoding and video compression
standards including H.261, H.263, H.264, H.265, MPEG-1, MPEG-2, and MPEG-4.
6 个视频 （总计 135 分钟）, 2 个阅读材料, 1 个测验
周11
完成时间为 3 小时
Image and Video Segmentation
In this module we introduce the problem of image and video segmentation, and discuss various approaches for performing segmentation
including methods based on intensity discontinuity and intensity similarity, watersheds and K-means algorithms, and other advanced methods.
4 个视频 （总计 110 分钟）, 2 个阅读材料, 1 个测验
周12
完成时间为 3 小时
Sparsity
In this module we introduce the notion of sparsity and discuss how this concept is being applied in image and video processing. Topics include:
sparsity-promoting norms, matching pursuit algorithm, smooth reformulations, and an overview of the applications.
