Thanks to modern software tools that allow to easily process and analyze data at scale, we are now able to extract invaluable insights from the vast amount of data generated daily. As a result, both the business and scientific world are undergoing a revolution which is fueled by one of the most sought after job profiles: the data scientist.

This course covers the fundamental steps of the data science pipeline:

Data wrangling

• Data acqusition (scraping, crawling, parsing, etc.)
• Data manipulation, array programming, dataframes
• The many sources of data problems (and how to fix them): missing data, incorrect data, inconsistent representations
• Data quality testing with crowdsourcing

Data interpretation

• Statistics in practice (distribution fitting, statistical significance, etc.)
• Working with "found data" (design of observational studies, regression analysis)
• Machine learning in practice (supervised and unsupervised, feature engineering, evaluation, etc.)
• Text mining: preprocessing steps, vector space model, topic models
• Social network analysis (properties of real networks, working graph data, etc.)

Data visualization

• Introduction to different plot types (1, 2, and 3 variables), layout best practices, network and geographical data
• Visualization to diagnose data problems, scaling visualization to large datasets, visualizing uncertain data

Reporting

• Results reporting, infographics
• How to publish reproducible results

The students will learn the techniques during the ex-cathedra lectures and will be introduced, in the lab sessions, to the software tools required to complete the homework assignments and the in-class quizzes.

In parallel, the students will embark on a semester-long project, split in agile teams of 3-4 students. In the project, students propose and execute meaningful analyses of a real-world dataset, which will require creativity and the application of the tools encountered in the course. The outcome of this team effort will be a project portfolio that will be made public (and available as open source).

At the end of the semester, students will take a 3-hour final exam in a classroom with their own computer, where they will be asked to complete a data analysis pipeline (both with code and extensive comments) on a dataset they have never worked with before.

The student must have passed an introduction to databases course, OR a course in probability & statistics, OR two separate courses that include programming projects. Programming skills are required (in class we will use mostly Python).

programming, algorithms, probability and statistics, databases

• Construct a coherent understanding of the techniques and software tools required to perform the fundamental steps of the data science pipeline

Students are expected to:

• Attend the lectures and lab sessions
• Complete 2-3 homework assignments
• Complete weekly in-class quizzes (held during lab sessions)
  
• Conduct the class project
  
• Engage during the class, and present their results in front of the other colleagues

- Homework

- Project

- Final exam

• http://ada.epfl.chhttp://ada.epfl.ch

• https://go.epfl.ch/CS-401https://go.epfl.ch/CS-401