Week 6 laboratory · 14 October

Find, cite, and summarize papers, by hand and with a tool

Locate one methods paper, write a complete reference, file it in week06-cite/, and summarize it yourself. Then summarize five papers with an automated tool and check what the tool got right. Tonight's guest, Devon Yanitski, built a tool for exactly this problem.

Guest · 18:20

Devon Yanitski

Devon is a PhD student in Psychology at the University of Alberta, in the Computational Memory Lab. Their research covers psychometrics from language embeddings, the neural correlates of memory, and LLM interpretability methods applied to human minds. devon7y.com

  • AP-RAG: search a library of research PDFs and get answers built only from those papers, each passage with its paper, page and citation.
  • AP-RAG Papers Atlas: a 3D map of the 10,000+ papers in Devon's database, laid out by topic.
  • semanticfa (R, with Chris Westbury): factor analysis of questionnaire items from their wording alone. Compare it with the Week 3 inventory.
Why it matters tonight

Searched, not memorized

A chatbot asked for references writes them from memory. Authors, years and titles can each be real and still not belong together. AP-RAG works the other way round: it can only answer from passages it pulled out of papers on disk, and every passage points to a page you can open.

  1. Ingest. Each PDF's text is extracted with page numbers kept, then split by section and paragraph.
  2. Contextualize. A model notes where each chunk sits in its paper, so "this effect" stays findable.
  3. Index. Chunks are embedded for similarity search and linked in a graph of authors, methods, theories and findings.
  4. Ask. A person or an AI agent queries it and gets passages with page-level citations.
18:00 · introduction

What belongs in a citation

  1. A complete reference: authors, year, title, journal or publisher, volume, issue, pages, and a DOI.
  2. The DOI is the check. Paste it after https://doi.org/. If it does not open this paper, the reference is wrong, however polished it looks.
  3. Library search first. Start from the U of A library or Google Scholar with library links, then open the PDF. Reading the abstract is not reading the methods.
  4. Reference managers are optional. Zotero is free: a browser button saves the record and PDF and formats APA. The requirement here is a complete reference in your README.
  5. Keep the source next to the reference. The paper goes in your repository folder so both travel together.
Suggested starting points

One paper for Part 1

  • Donders, F. C. (1868/1969). On the speed of mental processes. Acta Psychologica, 30, 412–431. (translation often reprinted)
  • A short primer on the Big Five or on writing self-report items, assigned in class.
  • A paper on your own project topic that passes the audit below.
Example from class

Attention Atlas

A literature review on attention networks in the brain, built as web pages.

Warm-up · 19:15 · 10 minutes

Citation audit

  1. Ask Copilot or any chatbot: "Give me three APA references for empirical papers on [your project topic], with DOIs."
  2. For each reference, mark yes or no: the DOI resolves; it resolves to this title; the authors match; the paper is about what the chatbot said.
  3. We tally the class: how many passed all four checks, and how many do not exist?
  4. Use one reference that passed every check, or find one in the library instead, for Part 1.
Part 1 · by hand · tonight

Summarize one paper yourself

  1. File the PDF, or a SOURCE.md with a stable URL, in week06-cite/.
  2. In week06-cite/README.md, copy the headings from lab-notes/ANNOTATION.md: complete reference, question, method, one limit, and the Copilot disclosure.
  3. Write five to eight sentences. Name the dependent measure (a time, a score, a count). If you cannot name it, you have not opened the methods.
  4. State one thing the paper cannot support.
  5. Commit Part 1 before you start Part 2, so your own version is on record first. This is individual work.
Part 2 · with a tool · after class

Summarize five papers with an automated tool, then check it

  1. Collect five papers on your project topic, including your Part 1 paper. Each needs a DOI that resolves.
  2. Use one automated tool (Copilot, a chatbot given the PDF, or a library AI search) to write the same three-heading summary for all five. Record the tool and your exact prompt.
  3. Save them as a table in week06-cite/auto-summaries.md: reference, question, method, one limit.
  4. Compare the tool's summary of your Part 1 paper with yours. Mark each heading as matches, partly, or wrong, checking against the PDF.
  5. Write four to six sentences: what the tool missed or invented, how long each approach took, and which you would trust in a literature review. Does the tool answer from the paper in front of it, or from memory?
Tonight

Agenda

TimeBlock
18:00What belongs in a citation. Library search, not a chatbot bibliography.
18:20Guest: Devon Yanitski on building AP-RAG, then questions.
19:05Break.
19:15Citation audit, then Part 1: find, file, and summarize your paper.
20:00Finish Part 1 on your own. Instructor checks references.
20:25Revise until the dependent measure is named in numbers. Commit Part 1 and start Part 2.
Weekly report · due Tuesday 20 October

What to submit

Push week06-cite/ with:

  • README.md: Part 1, your hand summary with a complete reference, plus the PDF or SOURCE.md.
  • auto-summaries.md: Part 2, the five tool summaries, the tool and prompt, the comparison marks, and your four to six sentences.

Both parts are individual work. On Canvas, paste both paths, your Part 1 reference, the Part 2 tool, and your three match marks.