Multimodal RAG over Lecture Videos
Multimodal RAG over Lecture Videos
A question-answering assistant for video lectures that runs on a student laptop without a graphics card: Whisper turns the speech into timestamped text, slide keyframes are read with OCR and embedded with SigLIP 2 and CLIP, and every answer cites the lecture and the minute:second where it is explained - one click plays the video there.
Category: Generative AI / Information Retrieval / Speech and Vision for Education
CSE Project 33 from ProTechSC Academy, with graphical abstract, video demonstration and project guidance.