Date & Time:
November 19, 2024 12:30 pm – 1:30 pm
Location:
Crerar 298, 5730 S. Ellis Ave., Chicago, IL,
11/19/2024 12:30 PM 11/19/2024 01:30 PM America/Chicago Lin Tan (Purdue)- LLMs for Code: More Data or More Domain Knowledge? Can They Replace Programmers? Crerar 298, 5730 S. Ellis Ave., Chicago, IL,

Abstract: Recent techniques leverage deep learning techniques, including large language models (LLMs), to improve coding tasks such as code generation, automated program repair, security vulnerability fixing, and binary analysis. An important question is, whether adding more data or more domain knowledge to deep-learning models is a more effective direction to improve LLMs for code. I will discuss existing studies and techniques that answer this question positively or negatively. I will also introduce our code-generation benchmark RepoCod, which answers the question, “Can Language Models Replace Programmers?”, to some extent. RepoCod tasks are real-world, whole-function code generation with repository-level context and contain test cases for validation. Our results show that GPT-4o and other LLMs achieve < 30% pass@1 on RepoCode’s code generation tasks.

https://lt-asset.github.io/REPOCOD/

Speakers

Lin Tan

Mary J. Elmore New Frontiers Professor. Purdue University

Lin Tan is a Mary J. Elmore New Frontiers Professor in the Department of Computer Science at Purdue University. She received her PhD from the University of Illinois, Urbana-Champaign. Prior to joining Purdue, she was a Canada Research Chair and an associate professor at the University of Waterloo. Her research interests include software dependability, software-AI synergy, and software text analytics. Some of her research focuses are leveraging machine learning and natural language processing techniques to improve software dependability, and using software approaches to improve the dependability of machine learning systems. Dr. Tan’s co-authored papers have received ACM Distinguished Paper Awards at CCS 2024, ASE 2020, MSR 2018, and FSE 2016; and IEEE Micro’s Top Picks in 2006. Dr. Tan was a recipient of an Early Career Academic Achievement Alumni Award by the University of Illinois, Urbana-Champaign, Canada Research Chair, an NSERC Discovery Accelerator Supplements Award, an Ontario Early Researcher Award, an Ontario Professional Engineers Award–Engineering Medal for Young Engineer, and multiple industry awards including J.P.Morgan AI Faculty Research Awards, Meta/Facebook Research Awards, Google Faculty Research Awards, and an IBM CAS Research Project of the Year Award. She has served as program co-chair of FSE 2024 (one of the top 2 conferences in software engineering). She was an associate editor of IEEE Transactions on Software Engineering (2017-2022) and Springer Empirical Software Engineering Journal (2015-2021). She was the ACM SIGSOFT Treasurer and an elected Member-at-Large (2021-2024).

Related News & Events

the researchers
UChicago CS News

University of Chicago Team Wins $10-Million Laude Institute Grant to Advance AI Weather Forecasting for Low- and Middle-Income Economies

Nov 05, 2026
chicago buildings
UChicago CS News

What MSCAPP Looks Like Inside the Chicago Mayor’s Office

Nov 03, 2026
UChicago CS News

Keeping Science Alive: A University of Chicago Project is Using AI to Update Old Science

Nov 02, 2026
banner
UChicago CS News

From Touch to Integration: UChicago CS at UIST 2026

Oct 27, 2026
headshot
UChicago CS News

Shan Lu Receives the 2026 SIGOPS Mark Weiser Award

Oct 26, 2026
banner
Video

Distinguished Speaker Series: Jon Froehlich (U of Washington)- Always Another Mountain: Persistence, Pushback, and the Pursuit of Research That Matters

Oct 19, 2026
headshot
UChicago CS News

Pedro Lopes 2026 Receives Academic Communicators Network Excellence Award For Digital Communications

Oct 19, 2026
two people reading a book
UChicago CS News

HAVI Award Supports UChicago Researchers Pairing AI with Bayesian Statistics to Decipher the Babylonian Astronomical Diaries

Oct 14, 2026
headshot
UChicago CS News

Todd Frank Dupont, Inventive Computational and Applied Mathematician, 1942–2026

Oct 14, 2026
headshot
UChicago CS News

What Are Data Centers, And Why Do They Need So Much Power? An Expert Explains

Oct 08, 2026
distinguished speaker header
UChicago CS News

Distinguished Speaker Series 2026-27

Oct 07, 2026
robot and crowd
In the News

Learning is a blast – and a bang – at the South Side Science Festival

Oct 07, 2026
arrow-down-largearrow-left-largearrow-right-large-greyarrow-right-large-yellowarrow-right-largearrow-right-smallbutton-arrowclosedocumentfacebookfacet-arrow-down-whitefacet-arrow-downPage 1CheckedCheckedicon-apple-t5backgroundLayer 1icon-google-t5icon-office365-t5icon-outlook-t5backgroundLayer 1icon-outlookcom-t5backgroundLayer 1icon-yahoo-t5backgroundLayer 1internal-yellowinternalintranetlinkedinlinkoutpauseplaypresentationsearch-bluesearchshareslider-arrow-nextslider-arrow-prevtwittervideoyoutube