Stochastic processes are probabilistic models that describe how systems evolve randomly over time. They are fundamental tools for understanding stock market fluctuations, gene expression, climate modeling and more. This course develops the theoretical foundations of stochastic processes. We will explore how randomness and time interact in complex systems, building up from finite Markov chains to sophisticated continuous-time processes like Brownian motion.
Staff and Organization
Instructor: Mark Sellke
Teaching Fellow: Anvit Garg
Course Assistants: Alex Fleury, Aseel Rawashdeh, Duy Thuc Nguyen, Eric Tang, Ethan Wang, Matthew Tan, Rushil Mallarapu, Shirley Xiong
Lecture Time: Tuesday/Thursday 10:30am - 11:45am
Lecture Location: Science Center Lecture Hall E
Important Websites: Canvas,
Ed Discussion
Prerequisites
STAT 110 or comparable background in probability, and Math 21a/b.
Section/Office Hours
Sections: (location TBA)
1:30pm-2:30pm Monday/Wednesday
3pm-4pm Friday/Saturday
Office Hours:
Mark: Tuesday 4pm-6pm, Science Center 711
(Other Office Hours TBA)
Course Materials
The textbook for this course is Introduction to Stochastic Processes with R by Robert P. Dobrow. Students can download it for free from the Harvard Library.
Grading
Homework (25%), Midterm (20% x2), Final (35%).
Problem Set Policies
There will be a total of ten problem sets. Solutions will be due on Fridays at 11:59 PM and must be submitted through Gradescope as a single PDF file. Submissions on paper or through email will not be accepted. Solutions may be typeset or handwritten and scanned but must be clear, legible, and correctly oriented. Otherwise, the assignment will receive no credit. If multiple files are submitted, only the final pdf submitted prior to the deadline will be graded. Each problem set will contain a few computational problems in addition to theoretical calculations. Use of R is encouraged for these problems, but it is not required. Collaboration with other students is encouraged, but students must write their own solutions in their own words. In addition, students must acknowledge their collaborators on the submitted assignments. The two lowest homework scores will be dropped. To account for unexpected circumstances, every student will be given two extensions, in which homework may be submitted by Saturday 5:00 PM (EST), the day after the normal Friday due date. You do not need to send an email to use this extension.Submitting Problem Sets
Problem sets should be submitted through Gradescope, as a PDF file. The PDF may be typed (e.g. in Latex), or neatly hand-written and scanned (please check for legibility in this case). Make sure to select which pages correspond to which problems, to ensure all of your solutions are graded. See this helpful short video for clarification.Midterm Exams
There will be two in-class midterms, on March 4 and April 8. Notes are not permitted, and make-up exams will not be given. Students with an excused absence will have their score on the final exam used as replacement. Students with an unexcused absence will receive zero for a missed exam.Scribing Notes
Students may sign-up to scribe lecture notes for 1 homework problem of extra credit. Each day should have at most 2 scribes, and each student can sign-up to scribe at most 1 time. Scribes should email their notes to msellke@fas.harvard.edu by noon the day after the lecture. If you use Latex (which is preferred), please send both the pdf and source files. If you hand-write, we may ask you to rewrite your notes if they are illegible or contain mistakes. Notes will be uploaded to Canvas 1-2 days after lecture, and students enrolled in the course can access them here.Course Calendar (Further Sections To Be Added)
Assignment Schedule
| Assignment | Deadline |
|---|---|
| Problem Set 1 | 11:59pm ET 2/7/2025 |
| Problem Set 2 | 11:59pm ET 2/14/2025 |
| Problem Set 3 | 11:59pm ET 2/21/2025 |
| Problem Set 4 | 11:59pm ET 2/28/2025 |
| Midterm I | In class, 3/4/2025 (Tuesday) |
| Problem Set 5 | 11:59pm ET 3/14/2025 |
| Problem Set 6 | 11:59pm ET 3/28/2025 |
| Problem Set 7 | 11:59pm ET 4/4/2025 |
| Problem Set 8 | 11:59pm ET 4/8/2025 |
| Midterm II | In class, 4/18/2025 (Tuesday) |
| Problem Set 9 | 11:59pm ET 4/25/2025 |
| Problem Set 10 | 11:59pm ET 4/30/2025 |
| Final Exam | TBA |
Course Schedule
| Date | Topic | Relevant Textbook Chapters |
|---|---|---|
| Jan 28 | Motivation, Review of probability, Introduction to random walk | 1 |
| Jan 30 | Markov Chain First steps | 2 |
| Feb 4 | Markov chains long time behavior | 3 |
| Feb 6 | Reversibility | 3 |
| Feb 11 | Absorbing chains | 3 |
| Feb 13 | Branching Processes | 4 |
| Feb 18 | Markov Chain Algorithms, Metropolis-Hastings | 5 |
| Feb 20 | Gibbs Samplers | 5 |
| Feb 25 | Markov Chain Algorithms | 5 |
| Feb 27 | Review | - |
| Mar 4 | Poisson process | 6 |
| Mar 6 | Midterm I | - |
| Mar 11 | Continuous time Markov chains | 7 |
| Mar 13 | Birth-death Processes, Queues, Renewal processes | 7 |
| Mar 17-21 | Spring Break | - |
| Mar 25 | Long term behavior, Queues | 7 |
| Mar 27 | Birth-Death chains, Martingales | 7 |
| Apr 1 | Brownian motion: Introduction | 8 |
| Apr 3 | Brownian motion: Computations | 8 |
| Apr 8 | Brownian Motion & Gaussian processes | 8 |
| Apr 10 | Midterm II | - |
| Apr 15 | Stochastic calculus | 9 |
| Apr 17 | Stochastic calculus | 9 |
| Apr 22 | Stochastic Calculus | 9 |
| Apr 24 | Ito's formula | 9 |
| Apr 29 | Review | - |
| TBA | Final Exam | - |