Data Science / Machine Learning¶

(Draft Status)

Probability, Entropy and Statistics¶

• Probability

  • Probability in a nutshell
  • in German: Zusammenfassung Wahrscheinlichkeitstheorie
  • Slides in German

• Estimators: Mean Squared Error, Variance, Bias

• Entropy and Information Gain

  • Slides in German

• Statistical hypothesis testing

• Distributions
  • Binomial and Beta Distribution
  • Multinomial, Dirichlet Distribution
  • Poisson Distribution

Data¶

• Level of measurements
• Data Handling
• Data Cleansing

Data Bases¶

• Relational Databases
• NoSql Databases
  • Cap theorem

Data Preprocessing, Data Transformations and Dimensionality Reduction¶

• Principal component analysis, Whitening
• Singular Value Decomposition
• Independent Component Analysis
• Feature selection

Exploratory Data Analysis and Visualization¶

• Correlations
• Intro into Exploratory Data Analysis with Pandas

Machine Learning¶

Basics of Machine Learning¶

• Univariate Linear Regression:

• Maximum Likelihood Principle

• Multivariate Linear Regression
• Logistic Regression
• Overfitting, Underfitting, Bias-Variance Tradeoff
• Regularization
• Validation

Association Rule Learning¶

Bayesian Statistics, Bayesian Inference¶

Variational Methods¶

Anomaly Detection¶

Graphical Models¶

Directed Graphical Models (Bayesian Networks)¶

• Bayesian Networks - Representation
• Sampling from a Bayesnet
• Parameter Learning by Maximum Likelihood
• Examples:
  • Factor Analysis
  • Hidden Markov Model - Example: 1D Kalman Filter

Undirected Graphical Models¶

• Introduction
• Conditional Random Fields
  • linear-chain conditonal random fields
• Boltzmann Machines
  • Introduction into Boltzmann Machines

Algorithms¶

Supervised Leaning¶

• Support Vector Machines
• Decision Trees
• Gaussian processes

Ensemble Learning¶

• Bagging
• Boosting
• Decision Forests

Neural Networks¶

Unsupervised Learning¶

• Expectation Maximization - Example Gaussian Mixture Models
• Generative Adversarial Networks

Probability density estimation, Unsupervised Feature Learning and Manifold Learning¶

• Neural Autoregressive Density Estimation (NADE)
• Variational Autoencoder

Reinforcement Learning¶

Time Series Analysis¶

• AR, MA, AR(I)MA
• Hidden Markov Models
• Maximum Entropy Markov Models
• Linear-Chain Conditional Random Fields
• Recurrent Neural Networks

Data Integration¶

Network Analysis¶

Misc¶

• Fuzzy Logic
  • Slides in German - part 1

Semantic Data¶

• Ontologies
• RDF - RDFS
• OWL

Textual data and natural language processing¶

• Information Retrieval

High Scalability - Data Parallelism¶

• Comunication Patterns

Cloud Computing¶

• Hadoop
  • Introduction: Slides in German
  • Hadoop IO: Slides in German
  • HDFS: Slides in German
  • Java MapReduce: Slides in German
  • Sorting: Slides in German
• Spark
  • Example: [Shingling for plagiarism detection]
  • Example: [Ngrams and Language Models]