Our next ML study group meeting will take place on Monday the 8^th of October.   I'll cover the contraction theorem.   See relevant section in https://nl.overleaf.com/read/dsmkpzdbbshq  (section 7.1) for slide and lecture notes.   The contraction map theorem is at the heart of Markov Decision Processes, Reinforcement Learning and the existence of a Nash equilibrium.  

Also, we are reigniting the ML study group.  In the next session Ora will cover Gradient Decent which is fundamental to the understanding of deep neural networks and transformer training.  

 Classical Hopfield update operation convergence meeting recording -  https://www.youtube.com/watch?v=zkV8zjVqFoI

 A youtube on the attention is all you need paper -

https://www.youtube.com/watch?v=XowwKOAWYoQ

A youtube on the Hopfield is all you need paper -

Hopfield Networks is All You Need (Paper Explained) - YouTube

 We will cover the update process of classical Hopfield networks.   https://drive.google.com/file/d/16wVcMXIJrn576Guop8zfPv7hzEL8pT8D/view?usp=share_link

 Tomorow we will discuss how backpropagation is done locally at a gate.  This enables a lot of the DNN training methods.   https://drive.google.com/file/d/1Y598WVObk5MYFKLxL0YXPK6qGtt1oju_/view?usp=share_link

 We will start discussing DNN and transformers.  See link.   Recommended book on the topic link.

 How can ERM be fooled - How can ERM be fooled? - YouTube

 Decision making - Decision making - YouTube

ML testing - random variable empirical distribution 

https://eitanfarchiml.blogspot.com/ 

 Interesting article on the use or lack of use of the scientific method by data science -  https://opendatascience.com/the-crisis-of-bad-data-science-and-what-we-can-do-to-avert-it/

 We will continue our discussion of optimization see - https://www.overleaf.com/read/zppqchcctnwf appendix A.1

Also added two exercises to the appendix. 

Guy presented the solution to exercise 2.3.1.   You can view the exercise here https://www.overleaf.com/read/zppqchcctnwf   Clicking on the exercise will get you to a notebook where you can run the solution.   

 

"Why do we need the Lebesgue integral in probability" meeting recording - https://www.youtube.com/watch?v=WbfBEbg6aXE&list=PLC7m9qp0Q1Ye3Fwnooz2k5xh0p5eD1s4k&index=45

 As part of the optimization thread.    Meeting recording on when a function is bounded - https://www.youtube.com/watch?v=q5r-8x-ST3g&list=PLC7m9qp0Q1Ye3Fwnooz2k5xh0p5eD1s4k&index=44

 A nice resource on optimization with a focus on convex optimization - 

https://m.youtube.com/playlist?list=PLXsmhnDvpjORzPelSDs0LSDrfJcqyLlZc

 Back to optimization.  We will lay down some of the foundations and discuss when a function is bounded https://drive.google.com/file/d/15gz6KizutYMlzTheTE8HymFHtDar7jHE/view?usp=sharing

 

Today Samuel Ackerman spoke on "statistically determining which model to choose by comparing performance over cross-validation runs".  Details are on his blog  - https://www.research.ibm.com/haifa/dept/vst/ML-QA.shtml

 We will cover Baysian parameter estimate see link - https://drive.google.com/file/d/1KJiJvbawe-E5NCl1-1dcJchbPCRDGD9p/view?usp=sharing

 Recording of meeting on the continuous version of the Bayes rule - https://www.youtube.com/watch?v=vH4EP6CYNzc&list=PLC7m9qp0Q1Ye3Fwnooz2k5xh0p5eD1s4k&index=42

We will continue the discussion of Independence  in BN and start discussing causation  https://drive.google.com/file/d/1B5rpb2_cOkXJl-Iuo2R0pre0iWKS5snp/view?usp=sharing