• ES Español (Latinoamérica)
• EN English

4.6. Mathematical Foundations of Data Science (FDS)

This area provides the formal mathematical underpinnings for extracting knowledge from engineering data, combining information theory, optimization, and statistical learning relevant to sensor systems, communication, and data-driven engineering.

Knowledge Area (KA)	Core Tier1	Core Tier2
4.6.1 Information Theory and Entropy	Elective

4.6.1. FDS/Information Theory and Entropy ↑ Back to top

Quantifying information, redundancy, and uncertainty in engineering signals and communication systems.
Topics:
Core

• Shannon entropy and mutual information for engineering data sources
• Kullback-Leibler divergence and relative entropy in model comparison
• Channel capacity and the Shannon-Hartley theorem for communication systems
• Lossless compression bounds and Huffman coding for engineering data

Learning Outcomes:
Core:

1. Calculate the entropy of discrete probability distributions representing engineering sensor outputs [Usage]
2. Minimize cross-entropy loss as a design objective in sensor classification tasks [Assessment]
3. Explain the concept of channel capacity and its significance in engineering communication system design [Familiarity]
4. Apply information-theoretic measures to evaluate the quality of engineering data streams [Usage]