“ Students are permitted to use AI assistants for all homework and programming assignments (especially as a reference for understanding any topics that seem confusing), but we strongly encourage you to complete your final submitted version of your assignment without AI. You cannot use any such assistants, or any external materials, during in-class evaluations (both the homework quizzes and the midterms and final).
The rationale behind this policy is a simple one: AI can be extremely helpful as a learning tool (and to be clear, as an actual implementation tool), but over-reliance on these systems can currently be a detriment to learning in many cases. You absolutely need to learn how to code and do other tasks using AI tools, but turning in AI-generated solutions for the relatively short assignments we give you can (at least in our current experience) ultimately lead to substantially less understanding of the material. The choice is yours on assignments, but we believe that you will ultimately perform much better on the in-class quizzes and exams if you do work through your final submitted homework solutions yourself.”
It feels downstream of CMU's "reasonable person principle". They know that people are going to use AI on their homework, but they trust that they want to learn and improve their skills -- and this is good advice for doing so.
I'm somewhat biased because I was involved in a previous, related course. The important takeaways aren't really about gritty debugging of (possibly) large homework assignments, but the high-level overview you get in the process. AI assistance means you could cover more content and build larger, more realistic systems.
An issue in the first iteration of Deep Learning Systems was that every homework built on the previous one, and errors could accumulate in subtle ways that we didn't anticipate. I spent a lot of time bisecting code to find these errors in office hours. It would have been just as educational to diagnose those errors with an LLM. Then students could spend more time implementing cool stuff in CUDA instead of hunting down a subtle bug in their 2d conv backwards pass under time pressure... But I think the breadth and depth of the course was phenomenal, and if courses can go further with AI assistance then it's great.
This new class looks really cool, and Zico is a great teacher.
I'm old but cumulative assignments are nothing new (the build an OS class, build a compiler class, etc) and my recollection is after you submitted an assignment the instructor would release a correct version you could swap in for yours. So any bugs in previous modules (that the TA/grader didn't catch) couldn't hold up the current assignment.
Old too, and in my experience that was often slightly more work than fixing the bugs in my own implementation. I did swap out a borked module in the build an OS class once but otherwise used my own.
I don't think the final evaluation is to "cement the understanding" so much as _verify_ that students have taken accountability for their own learning process.
This is what a student, who truly wants to learn rather than simply complete a course / certification, would do... Use AI tools to explain + learn, but not outsource the learning process itself to the tools.
An AI is never going to make me feel bad for asking a "stupid" question, nor will it ever behave as though I have to earn the right to ask a sophisticated one. The answer is, in some cases, almost immaterial. Building in students the willingness to ask questions accelerates learning.
Does one still need to read, take notes, write, act, etc? Yes. The feeling of newfound knowledge that comes from reading/watching a virtuoso demonstration is always a mirage--I suspect most of us have the memory of watching someone do algebra on the board and then having the feeling of understanding melt away later when faced with the page. That's not changing, so far as I can tell.
But we should acknowledge that allowing students to ask questions at will, even ill-posed questions, might have some value.
> What’s your hypothesis of how AI can accelerate how your brain understands something?
What are your beliefs / hypothesis of how having a human teacher can help you understand something?
AI explanations are no longer terrible garbage. The LLM might not be doing original research, but it has definitely read the textbook. :/ And 1000 related works.
You shouldn't believe the LLM when it tells you how to micro-optimize your code, but you can take suggestions as a starting point and verify them.
This is a bad case of whataboutism (I hate this word but it describes the answer you gave), what do you mean by accelerating understanding? Maybe they are good as suggestion engines, but it is very early to state what you did.
Theyre alot more than "suggestion engines". They can reason with you, show you examples, tell you how to dig deeper and verify what theyre saying, etc.
I have some success with this method: I try to write an explanation of something, then ask the LLM to find problems with the explanation. Sometimes its response leads me to shore up my understanding. Other times its answer doesn’t make sense to me and we dig into why. Whether or not the LLM is correct, it helps me clarify my own learning. It’s basically rubber duck debugging for my brain.
Way back when I was a student I had a professor who had a policy that if your homework scores differed substantially from your exam scores, the homework portion of your grade would be disregarded and your final grade would be determined solely by the midterm and the final. It was a harsh policy, and at the time I hated it, but in retrospect it was fair. Seems even more relevant today.
I started doing the free version of the course a few days ago - the lessons are excellent but what is even better are the homework tasks which allows me to run my tests locally!
It's sometimes easy to just listen and understand, but be unable to write the code myself - having this coding homework task has really helped me solidify this new knowledge.
Can you share what hardware resources are expected for the assignments? Are the students just doing this on their personal computers or are they given cluster access?
Lisp and Prolog never really "vived" nor were they ever really gone/dead. So they can't be revived. They've always been there, in the background, in their niche. As they always will.
Sadly, not part of this course, though Lisp and Prolog are very useful for other things. C's fine for building neural networks from scratch, and you can glue different subsystems together to make anything more complex than that using Python.
Well it's the dominant and most successful implemented AI, would a comp sci course teach every failed computer architecture or focus on the ones that are in wide use today.
Your analogy to computer architectures doesn't make sense, unless comparing GPT-like LLMs to different LLM architectures like Mamba or RWKV. It indeed wouldn't make sense to not teach about Mamba or RWKV in an introductory AI or LLM course.
AI is much broader than LLMs alone. Computer vision, RL, classical ML, recommender systems, speech recognition, ... are still part of AI, just not very visible to the average consumer.
It really depends on the target audience, because a lot of people have no idea what they are using is called an LLM or that there are various types of generative AI.
I think the problem is the under representation of other branches of AI research: knowledge representation, automated reasoning, planning, etc.
These are important topics with important industrial applications which have the only downsides to not be suitable for implementing friendly chatbots and for raising the stocks of Silicon Valley companies.
I doubt renowned US universities don't offer courses that cover those topics.
As someone who studied in a university system where the courses you had to take were mostly set in stone (just starting to offer some electives now), I really fancy the option of being able to choose what you study as much as possible.
The AI course I took was mostly symbolic methods and some classic ML at the end. Most students were not interested at all and would've probably been more engaged studying ML directly. Too bad that wasn't an option.
> AI Policy for the AI Course
“ Students are permitted to use AI assistants for all homework and programming assignments (especially as a reference for understanding any topics that seem confusing), but we strongly encourage you to complete your final submitted version of your assignment without AI. You cannot use any such assistants, or any external materials, during in-class evaluations (both the homework quizzes and the midterms and final).
The rationale behind this policy is a simple one: AI can be extremely helpful as a learning tool (and to be clear, as an actual implementation tool), but over-reliance on these systems can currently be a detriment to learning in many cases. You absolutely need to learn how to code and do other tasks using AI tools, but turning in AI-generated solutions for the relatively short assignments we give you can (at least in our current experience) ultimately lead to substantially less understanding of the material. The choice is yours on assignments, but we believe that you will ultimately perform much better on the in-class quizzes and exams if you do work through your final submitted homework solutions yourself.”
My money is on extraordinarily poor final exam results and/or cheating.
In my day professors said that you'd never have an AI in your pocket
but can I write 5318008 on my AI?
it will as soon as openai gives us dirty mode
True. Why even bother with school anyway?
There are no correct answers, you just have to predict what one could be while micro-dosing.
It feels downstream of CMU's "reasonable person principle". They know that people are going to use AI on their homework, but they trust that they want to learn and improve their skills -- and this is good advice for doing so.
I'm somewhat biased because I was involved in a previous, related course. The important takeaways aren't really about gritty debugging of (possibly) large homework assignments, but the high-level overview you get in the process. AI assistance means you could cover more content and build larger, more realistic systems.
An issue in the first iteration of Deep Learning Systems was that every homework built on the previous one, and errors could accumulate in subtle ways that we didn't anticipate. I spent a lot of time bisecting code to find these errors in office hours. It would have been just as educational to diagnose those errors with an LLM. Then students could spend more time implementing cool stuff in CUDA instead of hunting down a subtle bug in their 2d conv backwards pass under time pressure... But I think the breadth and depth of the course was phenomenal, and if courses can go further with AI assistance then it's great.
This new class looks really cool, and Zico is a great teacher.
I'm old but cumulative assignments are nothing new (the build an OS class, build a compiler class, etc) and my recollection is after you submitted an assignment the instructor would release a correct version you could swap in for yours. So any bugs in previous modules (that the TA/grader didn't catch) couldn't hold up the current assignment.
Old too, and in my experience that was often slightly more work than fixing the bugs in my own implementation. I did swap out a borked module in the build an OS class once but otherwise used my own.
I loved those courses, great memories.
This is the way it should be. AI to speed up the understanding process, and one final evaluation without any help to cement the understanding.
I don't think the final evaluation is to "cement the understanding" so much as _verify_ that students have taken accountability for their own learning process.
^ This
This is what a student, who truly wants to learn rather than simply complete a course / certification, would do... Use AI tools to explain + learn, but not outsource the learning process itself to the tools.
> AI to speed up the understanding process
What’s your hypothesis of how AI can accelerate how your brain understands something?
An AI is never going to make me feel bad for asking a "stupid" question, nor will it ever behave as though I have to earn the right to ask a sophisticated one. The answer is, in some cases, almost immaterial. Building in students the willingness to ask questions accelerates learning.
Does one still need to read, take notes, write, act, etc? Yes. The feeling of newfound knowledge that comes from reading/watching a virtuoso demonstration is always a mirage--I suspect most of us have the memory of watching someone do algebra on the board and then having the feeling of understanding melt away later when faced with the page. That's not changing, so far as I can tell.
But we should acknowledge that allowing students to ask questions at will, even ill-posed questions, might have some value.
Quick, easy access to explanations and examples on complex topics.
In my case, learning enough trig and linear algebra to be useful in game engine programming / rendering has been made a lot easier / more efficient.
The same way Google or Wikipedia enables learning.
> What’s your hypothesis of how AI can accelerate how your brain understands something?
What are your beliefs / hypothesis of how having a human teacher can help you understand something?
AI explanations are no longer terrible garbage. The LLM might not be doing original research, but it has definitely read the textbook. :/ And 1000 related works.
You shouldn't believe the LLM when it tells you how to micro-optimize your code, but you can take suggestions as a starting point and verify them.
"What are your beliefs / hypothesis of how having a human teacher can help you understand something?"
One precondition is the human teacher can challenge you independently of your own self-control/will.
This is a bad case of whataboutism (I hate this word but it describes the answer you gave), what do you mean by accelerating understanding? Maybe they are good as suggestion engines, but it is very early to state what you did.
I use them every day to learn things.
Theyre alot more than "suggestion engines". They can reason with you, show you examples, tell you how to dig deeper and verify what theyre saying, etc.
I have some success with this method: I try to write an explanation of something, then ask the LLM to find problems with the explanation. Sometimes its response leads me to shore up my understanding. Other times its answer doesn’t make sense to me and we dig into why. Whether or not the LLM is correct, it helps me clarify my own learning. It’s basically rubber duck debugging for my brain.
I disagree. I think we should treat AI tools like calculators for the exam.
Way back when I was a student I had a professor who had a policy that if your homework scores differed substantially from your exam scores, the homework portion of your grade would be disregarded and your final grade would be determined solely by the midterm and the final. It was a harsh policy, and at the time I hated it, but in retrospect it was fair. Seems even more relevant today.
For those that are unaware, the instructor of this is on the board of OpenAI:
https://openai.com/index/zico-kolter-joins-openais-board-of-...
I'm a little annoyed that 'modern AI' refers here only on LLMs, modern AI is way bigger than that.
Having said that, it's probably a good course, CMU courses are often great.
I was just expecting way more sota models in many fields due to the title.
If someone has this kind of ressource I would be extremely interested!
I started doing the free version of the course a few days ago - the lessons are excellent but what is even better are the homework tasks which allows me to run my tests locally!
It's sometimes easy to just listen and understand, but be unable to write the code myself - having this coding homework task has really helped me solidify this new knowledge.
10/10 would recommend
Can you share what hardware resources are expected for the assignments? Are the students just doing this on their personal computers or are they given cluster access?
Do you think this is a good course? Or, what do you suggest as a structured course to learn how LLMs work?
Can't wait for postmodern AI.
How to flip burgers better than an AI robot!
"This is the most used AI chatbot in the world!" "Who uses it?" "Other AI chatbots!"
:) Too true
But tbh, it'll more likely be repairing those burger flippin' robots
Nothing on symbolic reasoning ?
I believe that would be part of whats now "classical ai"
It's called GOFAI, or not AI at all. It's basically all machine learning nowadays.
good old-fashioned artificial intelligence
https://en.wikipedia.org/wiki/GOFAI
that would be the exact opposite of modern
No. That will be covered by the Post-modern AI course in the fall semester.
That's not AI.
Why not? It was called AI at the time.
I hope the instructor will publish a textbook to support and accompany the course, will buy in a heartbeat.
To those who actually are actually attempting to complete the course ie doing all the assignments in addition to watching lectures:
In your estimation : How much time does one realistically need every week to successfully finish the course?
Nice to finally see the revival of Lisp and Prolog.
Lisp and Prolog never really "vived" nor were they ever really gone/dead. So they can't be revived. They've always been there, in the background, in their niche. As they always will.
Sadly, not part of this course, though Lisp and Prolog are very useful for other things. C's fine for building neural networks from scratch, and you can glue different subsystems together to make anything more complex than that using Python.
>Sadly, not part of this course,
Huh?
What do they mean by AI then?
Hahaa
prolog in another skin is called erlang you know.
thanks for sharing, these look great.
[flagged]
Well it's the dominant and most successful implemented AI, would a comp sci course teach every failed computer architecture or focus on the ones that are in wide use today.
I think comp sci courses focuse on fundamentals rather than what's popular. Besides, other kinds of AI are not "failures", they have plenty of uses.
Your analogy to computer architectures doesn't make sense, unless comparing GPT-like LLMs to different LLM architectures like Mamba or RWKV. It indeed wouldn't make sense to not teach about Mamba or RWKV in an introductory AI or LLM course.
AI is much broader than LLMs alone. Computer vision, RL, classical ML, recommender systems, speech recognition, ... are still part of AI, just not very visible to the average consumer.
> most successful implemented AI
According to what? Spent money? Number of users? Outcomes and if so which ones?
probably according to marketing and not limited to hallucination
Don't trip over words. The course offers quite a range of knowledge that is suitable outside LLMs. It's an introduction.
It really depends on the target audience, because a lot of people have no idea what they are using is called an LLM or that there are various types of generative AI.
I think the problem is the under representation of other branches of AI research: knowledge representation, automated reasoning, planning, etc.
These are important topics with important industrial applications which have the only downsides to not be suitable for implementing friendly chatbots and for raising the stocks of Silicon Valley companies.
I doubt renowned US universities don't offer courses that cover those topics.
As someone who studied in a university system where the courses you had to take were mostly set in stone (just starting to offer some electives now), I really fancy the option of being able to choose what you study as much as possible.
The AI course I took was mostly symbolic methods and some classic ML at the end. Most students were not interested at all and would've probably been more engaged studying ML directly. Too bad that wasn't an option.
This is a perfectly reasonable take. It's quite outrageous that this was flagged.
[dead]