• ES Español
• EN English

5.49. Computational Vision (Elective)

5.49.1. Justification ↑ Back to top

Computer Vision is a key area of Artificial Intelligence that enables machines to interpret and understand the visual world. This course covers fundamental techniques for image processing, feature extraction, and object recognition. With the advent of deep learning, computer vision has become essential for autonomous vehicles, medical imaging, and augmented reality. Students will learn the mathematical foundations and algorithmic approaches to solve real-world visual perception problems.

5.49.2. Generales Goals ↑ Back to top

1. Understand the fundamentals of image formation and digital representation.
2. Master image processing techniques such as filtering and edge detection.
3. Apply feature extraction methods to identify key patterns in images.
4. Implement object detection and recognition using deep learning models.
5. Analyze 3D vision and motion estimation techniques.

5.49.3. Contribution to Outcomes ↑ Back to top

AG-C08) Problem Analysis: Identifies, formulates, and analyzes complex computing problems. (Usage)

AG-C09) Design and Development of Solutions: Designs, implements, and evaluates solutions for complex computing problems. (Assessment)

AG-C12) Applies computer science theory and software development fundamentals to produce computer-based solutions. (Usage)

5.49.4. Content ↑ Back to top

5.49.4.1. Introduction to AI: Concepts and Intelligent Behavior (5 hours) [Skills AG-C08] ↑ Back to top

Bibliography: (Szeliski, 2022; Gonzalez and Woods, 2018)

Topics

1. Overview of AI problems, Examples of successful recent AI applications
2. Definitions of agents with examples (e.g., reactive, deliberative)
3. What is intelligent behavior?:
   1. The Turing test and its flaws
   2. Multimodal input and output
   3. Simulation of intelligent behavior
   4. Rational versus non-rational reasoning
4. Overview of AI Applications, growth, and impact (economic, societal, ethics)
5. Philosophical issues
6. History of AI

Learning Outcomes

1. Describe the Turing test and the "Chinese Room" thought experiment [Explain]
2. Differentiate between optimal reasoning/behavior and human-like reasoning/behavior [Evaluate]
3. Differentiate the terms: AI, machine learning, and deep learning [Explain]

5.49.4.2. Problem Characteristics and Agent Nature (5 hours) [Skills AG-C08] ↑ Back to top

Bibliography: (Szeliski, 2022; Gonzalez and Woods, 2018)

Topics

1. Problem characteristics:
   1. Fully versus partially observable
   2. Single versus multi-agent
   3. Deterministic versus stochastic
   4. Static versus dynamic
   5. Discrete versus continuous
2. Nature of agents:
   1. Autonomous, semi-autonomous, mixed-initiative autonomy
   2. Reflexive, goal-based, and utility-based
   3. Decision making under uncertainty and with incomplete information
   4. The importance of perception and environmental interactions
   5. Learning-based agents
   6. Embodied agents:
      1. sensors, dynamics, effectors
3. Practice identifying problem characteristics in example environments
4. Additional depth on nature of agents with examples
5. Additional depth on AI Applications, Growth, and Impact (economic, societal, ethics, security)

Learning Outcomes

1. Enumerate the characteristics of a specific problem [Apply]

5.49.4.3. Perception and Computer Vision (12 hours) [Skills AG-C08] ↑ Back to top

Bibliography: (Szeliski, 2022; Prince, 2012)

Topics

1. Computer vision:
   1. Image acquisition, representation, processing, and properties
   2. Shape representation, object recognition, and segmentation
   3. Motion analysis
   4. Generative models
2. Audio and speech recognition
3. Touch and proprioception
4. Other modalities (e.g., olfaction)
5. Modularity in recognition
6. Approaches to pattern recognition:
   1. Classification algorithms and measures of classification quality
   2. Statistical techniques
   3. Deep learning techniques

Learning Outcomes

1. Summarize the importance of image and object recognition in AI and indicate several significant applications of this technology [Summarize]
2. List at least three image-segmentation approaches, such as thresholding, edge-based and region-based algorithms, along with their defining characteristics, strengths, and weaknesses [List]
3. Implement 2d object recognition based on contour-based and/or region-based shape representations [Apply]
4. Distinguish the goals of sound-recognition, speech-recognition, and speaker-recognition and identify how the raw audio signal will be handled differently in each of these cases [Distinguish]
5. Provide at least two examples of a transformation of a data source from one sensory domain to another, e.g., tactile data interpreted as single-band 2d images [Generate]
6. Implement a feature-extraction algorithm on real data, e.g., an edge or corner detector for images or vectors of Fourier coefficients describing a short slice of audio signal [Apply]
7. Implement an algorithm combining features into higher-level percepts, e.g., a contour or polygon from visual primitives or phoneme hypotheses from an audio signal [Apply]
8. Implement a classification algorithm that segments input percepts into output categories and quantitatively evaluates the resulting classification [Apply]
9. Evaluate the performance of the underlying feature-extraction, relative to at least one alternative possible approach (whether implemented or not) in its contribution to the classification task (8), above [Evaluate]
10. Describe at least three classification approaches, their prerequisites for applicability, their strengths, and their shortcomings [Describe]
11. Implement and evaluate a deep learning solution to problems in computer vision, such as object or scene recognition [Apply]

5.49.4.4. Machine Learning Fundamentals (5 hours) [Skills AG-C08] ↑ Back to top

Bibliography: (Goodfellow et al., 2016a; Russell and Norvig, 2020)

Topics

1. Definition and examples of a broad variety of machine learning tasks:
   1. Supervised learning:
      1. Classification
      2. Regression
   2. Reinforcement learning
   3. Unsupervised learning:
      1. Clustering
2. Fundamental ideas:
   1. No free lunch theorem: no one learner can solve all problems; representational design decisions have consequences.
   2. Sources of error and undecidability in machine learning
3. A simple statistical-based supervised learning such as linear regression or decision trees:
   1. Focus on how they work without going into mathematical or optimization details; enough to understand and use existing implementations correctly
4. The overfitting problem/controlling solution complexity (regularization, pruning - intuition only):
   1. The bias (underfitting) - variance (overfitting) tradeoff

Learning Outcomes

1. Describe the differences among the three main styles of learning (supervised, reinforcement, and unsupervised) and determine which is appropriate to a particular problem domain [Explain]
2. Differentiate the terms of AI, machine learning, and deep learning [Evaluate]
3. Frame an application as a classification problem, including the available input features and output to be predicted (e.g., identifying alphabetic characters from pixel grid input) [Apply]
4. Identify overfitting in the context of a problem and learning curves and describe solutions to overfitting [Evaluate]
5. Explain how machine learning works as an optimization/search process [Explain]

5.49.4.5. Machine Learning Data, Representations and Evaluation (5 hours) [Skills AG-C08] ↑ Back to top

Bibliography: (Goodfellow et al., 2016a; Russell and Norvig, 2020)

Topics

1. Working with Data:
   1. Data preprocessing:
      1. Importance and pitfalls of preprocessing choices
   2. Handling missing values (imputing, flag-as-missing):
      1. Implications of imputing vs flag-as-missing
   3. Encoding categorical variables, encoding real-valued data
   4. Normalization/standardization
   5. Emphasis on real data, not textbook examples
2. Representations:
   1. Hypothesis spaces and complexity
   2. Simple basis feature expansion, such as squaring univariate features
   3. Learned feature representations
3. Machine learning evaluation:
   1. Separation of train, validation, and test sets
   2. Performance metrics for classifiers
   3. Estimation of test performance on held-out data
   4. Tuning the parameters of a machine learning model with a validation set
   5. Importance of understanding what a model is doing, where its pitfalls/shortcomings are, and the implications of its decisions

Learning Outcomes

1. Apply two or more simple statistical learning algorithms to a classification task and measure the classifiers' accuracy [Apply]
2. Implement a statistical learning algorithm and the corresponding optimization process to train the classifier and obtain a prediction on new data [Apply]
3. Explain proper ML evaluation procedures, including the differences between training and testing performance, and what can go wrong with the evaluation process leading to inaccurate reporting of ML performance [Explain]
4. Compare two machine learning algorithms on a dataset, implementing the data preprocessing and evaluation methodology (e.g., metrics and handling of train/test splits) from scratch [Apply]

5.49.4.6. Neural Networks and Ethics (4 hours) [Skills AG-C08] ↑ Back to top

Bibliography: (Goodfellow et al., 2016a; Russell and Norvig, 2020)

Topics

1. Basic neural networks:
   1. Fundamentals of understanding how neural networks work and their training process, without details of the calculations
   2. Basic introduction to generative neural networks (e.g., large language models)
2. Ethics for Machine Learning:
   1. Focus on real data, real scenarios, and case studies
   2. Dataset/algorithmic/evaluation bias and unintended consequences

Learning Outcomes

1. Describe the neural network training process and resulting learned representations [Explain]

5.49.4.7. Machine Learning Formulation and Evaluation (3 hours) [Skills AG-C09] ↑ Back to top

Bibliography: (Goodfellow et al., 2016a)

Topics

1. Formulation of simple machine learning as an optimization problem, such as least squares linear regression or logistic regression:
   1. Objective function
   2. Gradient descent
   3. Regularization to avoid overfitting (mathematical formulation)
2. Ensembles of models:
   1. Simple weighted majority combination
3. Performance evaluation:
   1. Other metrics for classification (e.g., error, precision, recall)
   2. Performance metrics for regressors
   3. Confusion matrix
   4. Cross-validation:
      1. Parameter tuning (grid/random search, via cross-validation)

Learning Outcomes

1. Compare and contrast several learning techniques (e.g., decision trees, logistic regression, naive Bayes, neural networks, and belief networks), providing examples of when each strategy is superior [Compare]
2. Evaluate the performance of a simple learning system on a real-world dataset [Evaluate]
3. Characterize the state of the art in learning theory, including its achievements and shortcomings [Explain]
4. Explain the problem of overfitting, along with techniques for detecting and managing the problem [Explain]
5. Explain the triple tradeoff among the size of a hypothesis space, the size of the training set, and performance accuracy [Explain]

5.49.4.8. Deep Learning, RL and Ethics (3 hours) [Skills AG-C09] ↑ Back to top

Bibliography: (Goodfellow et al., 2016a)

Topics

1. Deep learning:
   1. Deep feed-forward networks (intuition only, no mathematics)
   2. Convolutional neural networks (intuition only, no mathematics)
   3. Visualization of learned feature representations from deep nets
   4. Other architectures (generative NN, recurrent NN, transformers, etc.)
2. Overview of reinforcement learning methods
3. Two or more applications of machine learning algorithms:
   1. E.g., medicine and health, economics, vision, natural language, robotics, game play
4. Ethics for Machine Learning:
   1. Continued focus on real data, real scenarios, and case studies
   2. Privacy
   3. Fairness
   4. Intellectual property
   5. Explainability

Learning Outcomes

1. Visualize the training progress of a neural network through learning curves in a well-established toolkit (e.g., TensorBoard) and visualize the learned features of the network [Apply]
2. Given a real-world application of machine learning, describe ethical issues regarding the choices of data, preprocessing steps, algorithm selection, and visualization/presentation of results [Apply]

5.49.4.9. Perception and Computer Vision (12 hours) [Skills AG-C08] ↑ Back to top

Bibliography: (Szeliski, 2022)

Topics

1. Computer vision:
   1. Image acquisition, representation, processing, and properties
   2. Shape representation, object recognition, and segmentation
   3. Motion analysis
   4. Generative models
2. Audio and speech recognition
3. Touch and proprioception
4. Other modalities (e.g., olfaction)
5. Modularity in recognition
6. Approaches to pattern recognition:
   1. Classification algorithms and measures of classification quality
   2. Statistical techniques
   3. Deep learning techniques

Learning Outcomes

1. Summarize the importance of image and object recognition in AI and indicate several significant applications of this technology [Summarize]
2. List at least three image-segmentation approaches, such as thresholding, edge-based and region-based algorithms, along with their defining characteristics, strengths, and weaknesses [List]
3. Implement 2d object recognition based on contour-based and/or region-based shape representations [Apply]
4. Distinguish the goals of sound-recognition, speech-recognition, and speaker-recognition and identify how the raw audio signal will be handled differently in each of these cases [Distinguish]
5. Provide at least two examples of a transformation of a data source from one sensory domain to another, e.g., tactile data interpreted as single-band 2d images [Generate]
6. Implement a feature-extraction algorithm on real data, e.g., an edge or corner detector for images or vectors of Fourier coefficients describing a short slice of audio signal [Apply]
7. Implement an algorithm combining features into higher-level percepts, e.g., a contour or polygon from visual primitives or phoneme hypotheses from an audio signal [Apply]
8. Implement a classification algorithm that segments input percepts into output categories and quantitatively evaluates the resulting classification [Apply]
9. Evaluate the performance of the underlying feature-extraction, relative to at least one alternative possible approach (whether implemented or not) in its contribution to the classification task (8), above [Evaluate]
10. Describe at least three classification approaches, their prerequisites for applicability, their strengths, and their shortcomings [Describe]
11. Implement and evaluate a deep learning solution to problems in computer vision, such as object or scene recognition [Apply]

5.49.5. Bibliography ↑ Back to top

Szeliski, R. (2022). Computer Vision: Algorithms and Applications. Springer, 2nd edition.

Gonzalez, R. C. and Woods, R. E. (2018). Digital Image Processing. Pearson, 4th edition.

Prince, S. J. D. (2012). Computer Vision: Models, Learning, and Inference. Cambridge University Press.

Goodfellow, I., Bengio, Y., and Courville, A. (2016a). Deep Learning. MIT Press.

Russell, S. and Norvig, P. (2020). Artificial Intelligence: A Modern Approach. Pearson, 4th edition.